2024-04-25T01:19:06.400 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T01:19:06.400 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_teuthology@teuthology --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322 -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322/orig.config.yaml 2024-04-25T01:19:06.454 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T01:19:06.543 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322 branch: squid description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672322' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 targets: smithi033.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJC/7aXTZiaFnKvcK7sGicQkZMKeEgNtQzYQd5N1D/83fqflhgyAjTM+1tRbtAhwXp7u0OSXWmGqkQiYafjO3bQ= smithi138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB95wc00NAREHwua6/lxVbUx/4V0VFU3obIlX4w3qw1e5EHmad1vzipBPxT7hx4hOWFTf8dSnJoPWjLm02mB2kk= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-25T01:19:06.543 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-25T01:19:06.543 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-25T01:19:06.544 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T01:19:06.545 INFO:teuthology.task.internal:Checking packages... 2024-04-25T01:19:06.568 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-25T01:19:06.568 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T01:19:06.568 INFO:teuthology.packaging:ref: None 2024-04-25T01:19:06.568 INFO:teuthology.packaging:tag: None 2024-04-25T01:19:06.568 INFO:teuthology.packaging:branch: squid 2024-04-25T01:19:06.568 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:19:06.569 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-25T01:19:06.924 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-25T01:19:06.925 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T01:19:06.932 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T01:19:06.932 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T01:19:06.938 INFO:teuthology.task.internal:Saving configuration 2024-04-25T01:19:06.959 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T01:19:06.965 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T01:19:06.992 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 01:14:07.659914', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJC/7aXTZiaFnKvcK7sGicQkZMKeEgNtQzYQd5N1D/83fqflhgyAjTM+1tRbtAhwXp7u0OSXWmGqkQiYafjO3bQ='} 2024-04-25T01:19:07.015 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 01:14:07.658276', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB95wc00NAREHwua6/lxVbUx/4V0VFU3obIlX4w3qw1e5EHmad1vzipBPxT7hx4hOWFTf8dSnJoPWjLm02mB2kk='} 2024-04-25T01:19:07.015 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T01:19:07.021 INFO:teuthology.task.internal:roles: ubuntu@smithi033.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-25T01:19:07.021 INFO:teuthology.task.internal:roles: ubuntu@smithi138.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-25T01:19:07.021 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T01:19:07.092 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fdf286aa3a0>, signals=[15]) 2024-04-25T01:19:07.092 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T01:19:07.097 INFO:teuthology.task.internal:Opening connections... 2024-04-25T01:19:07.098 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:19:07.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:19:07.182 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T01:19:07.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:19:07.257 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T01:19:07.263 DEBUG:teuthology.orchestra.run.smithi033:> uname -m 2024-04-25T01:19:07.282 INFO:teuthology.orchestra.run.smithi033.stdout:x86_64 2024-04-25T01:19:07.282 DEBUG:teuthology.orchestra.run.smithi033:> cat /etc/os-release 2024-04-25T01:19:07.340 INFO:teuthology.orchestra.run.smithi033.stdout:NAME="CentOS Stream" 2024-04-25T01:19:07.340 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION="9" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:ID="centos" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:ID_LIKE="rhel fedora" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:VERSION_ID="9" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:PLATFORM_ID="platform:el9" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:ANSI_COLOR="0;31" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:LOGO="fedora-logo-icon" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:HOME_URL="https://centos.org/" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T01:19:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T01:19:07.342 INFO:teuthology.lock.ops:Updating smithi033.front.sepia.ceph.com on lock server 2024-04-25T01:19:07.362 DEBUG:teuthology.orchestra.run.smithi138:> uname -m 2024-04-25T01:19:07.379 INFO:teuthology.orchestra.run.smithi138.stdout:x86_64 2024-04-25T01:19:07.379 DEBUG:teuthology.orchestra.run.smithi138:> cat /etc/os-release 2024-04-25T01:19:07.435 INFO:teuthology.orchestra.run.smithi138.stdout:NAME="CentOS Stream" 2024-04-25T01:19:07.435 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION="9" 2024-04-25T01:19:07.435 INFO:teuthology.orchestra.run.smithi138.stdout:ID="centos" 2024-04-25T01:19:07.435 INFO:teuthology.orchestra.run.smithi138.stdout:ID_LIKE="rhel fedora" 2024-04-25T01:19:07.435 INFO:teuthology.orchestra.run.smithi138.stdout:VERSION_ID="9" 2024-04-25T01:19:07.435 INFO:teuthology.orchestra.run.smithi138.stdout:PLATFORM_ID="platform:el9" 2024-04-25T01:19:07.436 INFO:teuthology.orchestra.run.smithi138.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T01:19:07.436 INFO:teuthology.orchestra.run.smithi138.stdout:ANSI_COLOR="0;31" 2024-04-25T01:19:07.436 INFO:teuthology.orchestra.run.smithi138.stdout:LOGO="fedora-logo-icon" 2024-04-25T01:19:07.436 INFO:teuthology.orchestra.run.smithi138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T01:19:07.436 INFO:teuthology.orchestra.run.smithi138.stdout:HOME_URL="https://centos.org/" 2024-04-25T01:19:07.436 INFO:teuthology.orchestra.run.smithi138.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T01:19:07.436 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T01:19:07.436 INFO:teuthology.orchestra.run.smithi138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T01:19:07.436 INFO:teuthology.lock.ops:Updating smithi138.front.sepia.ceph.com on lock server 2024-04-25T01:19:07.462 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T01:19:07.470 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T01:19:07.475 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T01:19:07.475 DEBUG:teuthology.orchestra.run.smithi033:> test '!' -e /home/ubuntu/cephtest 2024-04-25T01:19:07.478 DEBUG:teuthology.orchestra.run.smithi138:> test '!' -e /home/ubuntu/cephtest 2024-04-25T01:19:07.493 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T01:19:07.499 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T01:19:07.499 DEBUG:teuthology.orchestra.run.smithi033:> test -z $(ls -A /var/lib/ceph) 2024-04-25T01:19:07.536 DEBUG:teuthology.orchestra.run.smithi138:> test -z $(ls -A /var/lib/ceph) 2024-04-25T01:19:07.563 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T01:19:07.616 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T01:19:07.631 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-25T01:19:07.631 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-25T01:19:07.631 DEBUG:teuthology.orchestra.run.smithi033:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T01:19:07.632 DEBUG:teuthology.orchestra.run.smithi138:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T01:19:07.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:19:07.647 DEBUG:teuthology.orchestra.run.smithi138:> uname -r 2024-04-25T01:19:07.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:19:07.649 DEBUG:teuthology.orchestra.run.smithi033:> uname -r 2024-04-25T01:19:07.703 INFO:teuthology.orchestra.run.smithi138.stdout:5.14.0-437.el9.x86_64 2024-04-25T01:19:07.703 INFO:teuthology.task.kernel:Running kernel on smithi138: 5.14.0-437.el9.x86_64 2024-04-25T01:19:07.704 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum install -y kernel 2024-04-25T01:19:07.707 INFO:teuthology.orchestra.run.smithi033.stdout:5.14.0-437.el9.x86_64 2024-04-25T01:19:07.707 INFO:teuthology.task.kernel:Running kernel on smithi033: 5.14.0-437.el9.x86_64 2024-04-25T01:19:07.708 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum install -y kernel 2024-04-25T01:19:10.110 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T01:19:11.184 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-25T01:19:13.634 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T01:19:20.693 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.2 MB 00:03 2024-04-25T01:19:22.352 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-25T01:19:23.084 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 1.6 MB/s | 19 MB 00:11 2024-04-25T01:19:24.286 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T01:19:27.618 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:00 2024-04-25T01:19:29.385 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-25T01:19:29.561 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T01:19:31.004 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-25T01:19:31.484 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T01:19:31.484 INFO:teuthology.orchestra.run.smithi033.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T01:19:31.580 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:19:31.581 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:19:31.582 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:19:31.676 DEBUG:teuthology.orchestra.run.smithi033:> echo no | sudo yum reinstall kernel || true 2024-04-25T01:19:32.199 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:19:29 AM UTC. 2024-04-25T01:19:32.323 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout:Reinstalling: 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 255 k 2024-04-25T01:19:32.338 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 0 2024-04-25T01:19:32.339 INFO:teuthology.orchestra.run.smithi033.stderr:Operation aborted. 2024-04-25T01:19:32.392 INFO:teuthology.orchestra.run.smithi033.stdout:Is this ok [y/N]: 2024-04-25T01:19:32.393 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum reinstall -y kernel || true 2024-04-25T01:19:32.910 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:19:29 AM UTC. 2024-04-25T01:19:33.030 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:19:33.031 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:19:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-04-25T01:19:33.032 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:19:33.032 INFO:teuthology.orchestra.run.smithi033.stdout:Reinstalling: 2024-04-25T01:19:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:19:33.032 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:19:33.033 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:19:33.033 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:19:33.033 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:19:33.033 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 255 k 2024-04-25T01:19:33.033 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 0 2024-04-25T01:19:33.034 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:19:33.346 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.7 MB/s | 255 kB 00:00 2024-04-25T01:19:33.347 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:19:33.347 INFO:teuthology.orchestra.run.smithi033.stdout:Total 809 kB/s | 255 kB 00:00 2024-04-25T01:19:33.367 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:19:33.558 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:19:33.558 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:19:33.563 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:19:33.563 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:19:33.702 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:19:33.782 INFO:teuthology.orchestra.run.smithi033.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:19:33.782 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:19:35.165 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:35.166 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:19:35.166 INFO:teuthology.orchestra.run.smithi033.stdout:Reinstalled: 2024-04-25T01:19:35.166 INFO:teuthology.orchestra.run.smithi033.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T01:19:35.166 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:19:35.166 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:19:35.321 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T01:19:35.369 INFO:teuthology.orchestra.run.smithi033.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T01:19:35.370 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T01:19:35.370 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T01:19:35.370 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-25T01:19:35.370 DEBUG:teuthology.orchestra.run.smithi033:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T01:19:35.398 INFO:teuthology.orchestra.run.smithi033.stdout:ttyS1 2024-04-25T01:19:35.418 DEBUG:teuthology.parallel:result is None 2024-04-25T01:19:36.095 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T01:19:37.984 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T01:19:37.985 INFO:teuthology.orchestra.run.smithi138.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T01:19:38.079 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:19:38.080 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:19:38.080 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:19:38.177 DEBUG:teuthology.orchestra.run.smithi138:> echo no | sudo yum reinstall kernel || true 2024-04-25T01:19:38.677 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 01:19:36 AM UTC. 2024-04-25T01:19:38.797 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:19:38.817 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:19:38.817 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-25T01:19:38.817 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:19:38.817 INFO:teuthology.orchestra.run.smithi138.stdout:Reinstalling: 2024-04-25T01:19:38.817 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:19:38.818 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:19:38.818 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:19:38.818 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:19:38.818 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:19:38.818 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 255 k 2024-04-25T01:19:38.818 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 0 2024-04-25T01:19:38.819 INFO:teuthology.orchestra.run.smithi138.stderr:Operation aborted. 2024-04-25T01:19:38.871 INFO:teuthology.orchestra.run.smithi138.stdout:Is this ok [y/N]: 2024-04-25T01:19:38.871 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum reinstall -y kernel || true 2024-04-25T01:19:39.362 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 01:19:36 AM UTC. 2024-04-25T01:19:39.477 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:19:39.478 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:19:39.478 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-25T01:19:39.478 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:19:39.478 INFO:teuthology.orchestra.run.smithi138.stdout:Reinstalling: 2024-04-25T01:19:39.478 INFO:teuthology.orchestra.run.smithi138.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T01:19:39.478 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:19:39.478 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:19:39.479 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:19:39.479 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:19:39.479 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 255 k 2024-04-25T01:19:39.479 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 0 2024-04-25T01:19:39.479 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:19:40.247 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-437.el9.x86_64.rpm 365 kB/s | 255 kB 00:00 2024-04-25T01:19:40.247 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:19:40.247 INFO:teuthology.orchestra.run.smithi138.stdout:Total 331 kB/s | 255 kB 00:00 2024-04-25T01:19:40.270 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:19:40.454 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:19:40.454 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:19:40.459 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:19:40.459 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:19:40.594 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:19:40.673 INFO:teuthology.orchestra.run.smithi138.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:19:40.673 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:41.343 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:41.344 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T01:19:42.025 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T01:19:42.025 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:19:42.025 INFO:teuthology.orchestra.run.smithi138.stdout:Reinstalled: 2024-04-25T01:19:42.025 INFO:teuthology.orchestra.run.smithi138.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T01:19:42.025 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:19:42.025 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:19:42.191 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T01:19:42.238 INFO:teuthology.orchestra.run.smithi138.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T01:19:42.238 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T01:19:42.238 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T01:19:42.239 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-25T01:19:42.239 DEBUG:teuthology.orchestra.run.smithi138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T01:19:42.267 INFO:teuthology.orchestra.run.smithi138.stdout:ttyS1 2024-04-25T01:19:42.292 DEBUG:teuthology.parallel:result is None 2024-04-25T01:19:42.292 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T01:19:42.299 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T01:19:42.299 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T01:19:42.302 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T01:19:42.346 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T01:19:42.353 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T01:19:42.360 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T01:19:42.360 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T01:19:42.363 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T01:19:42.414 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T01:19:42.421 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T01:19:42.421 DEBUG:teuthology.orchestra.run.smithi033:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T01:19:42.432 DEBUG:teuthology.orchestra.run.smithi138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T01:19:42.463 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:19:42.474 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:19:42.515 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:19:42.524 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T01:19:42.526 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T01:19:42.535 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T01:19:42.535 DEBUG:teuthology.orchestra.run.smithi033:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T01:19:42.537 DEBUG:teuthology.orchestra.run.smithi138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T01:19:42.562 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T01:19:42.571 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T01:19:42.571 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T01:19:42.605 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T01:19:42.620 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:19:42.704 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:19:42.754 DEBUG:teuthology.orchestra.run.smithi033:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:19:42.836 DEBUG:teuthology.orchestra.run.smithi033:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:19:42.881 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:19:42.881 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T01:19:42.948 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:19:42.989 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T01:19:43.032 DEBUG:teuthology.orchestra.run.smithi138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:19:43.113 DEBUG:teuthology.orchestra.run.smithi138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T01:19:43.152 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:19:43.152 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T01:19:43.215 DEBUG:teuthology.orchestra.run.smithi033:> sudo service rsyslog restart 2024-04-25T01:19:43.218 DEBUG:teuthology.orchestra.run.smithi138:> sudo service rsyslog restart 2024-04-25T01:19:43.255 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:19:43.301 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:19:43.816 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T01:19:43.823 INFO:teuthology.task.internal:Starting timer... 2024-04-25T01:19:43.823 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T01:19:43.902 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T01:19:43.911 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T01:19:43.912 DEBUG:teuthology.orchestra.run.smithi033:> sudo service auditd rotate 2024-04-25T01:19:43.982 INFO:teuthology.orchestra.run.smithi033.stdout:Rotating logs: 2024-04-25T01:19:43.984 DEBUG:teuthology.orchestra.run.smithi138:> sudo service auditd rotate 2024-04-25T01:19:44.027 INFO:teuthology.orchestra.run.smithi138.stdout:Rotating logs: 2024-04-25T01:19:44.029 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T01:19:44.029 DEBUG:teuthology.orchestra.run.smithi033:> /usr/sbin/getenforce 2024-04-25T01:19:44.054 INFO:teuthology.orchestra.run.smithi033.stdout:Permissive 2024-04-25T01:19:44.054 DEBUG:teuthology.orchestra.run.smithi138:> /usr/sbin/getenforce 2024-04-25T01:19:44.090 INFO:teuthology.orchestra.run.smithi138.stdout:Permissive 2024-04-25T01:19:44.090 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi033.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi138.front.sepia.ceph.com': 'permissive'} 2024-04-25T01:19:44.091 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:19:44.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:19:44.121 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:19:44.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:19:44.155 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T01:19:44.155 DEBUG:teuthology.orchestra.run.smithi033:> sudo /usr/sbin/setenforce permissive 2024-04-25T01:19:44.186 DEBUG:teuthology.orchestra.run.smithi138:> sudo /usr/sbin/setenforce permissive 2024-04-25T01:19:44.218 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T01:19:44.229 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T01:19:44.244 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T01:19:44.246 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi033.front.sepia.ceph.com,smithi138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T01:23:13.952 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi033.front.sepia.ceph.com'), Remote(name='ubuntu@smithi138.front.sepia.ceph.com')] 2024-04-25T01:23:13.953 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-25T01:23:13.954 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi033.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:23:14.029 DEBUG:teuthology.orchestra.run.smithi033:> true 2024-04-25T01:23:14.112 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi033.front.sepia.ceph.com' 2024-04-25T01:23:14.113 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T01:23:14.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T01:23:14.186 DEBUG:teuthology.orchestra.run.smithi138:> true 2024-04-25T01:23:14.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi138.front.sepia.ceph.com' 2024-04-25T01:23:14.265 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T01:23:14.279 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T01:23:14.279 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:23:14.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:23:14.282 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T01:23:14.282 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:23:14.309 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:23:14.323 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:23:14.332 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T01:23:14.347 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T01:23:14.347 INFO:teuthology.orchestra.run.smithi033.stderr:sudo: ntpd: command not found 2024-04-25T01:23:14.358 INFO:teuthology.orchestra.run.smithi033.stdout:506 Cannot talk to daemon 2024-04-25T01:23:14.372 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:23:14.372 INFO:teuthology.orchestra.run.smithi138.stderr:sudo: ntpd: command not found 2024-04-25T01:23:14.383 INFO:teuthology.orchestra.run.smithi138.stdout:506 Cannot talk to daemon 2024-04-25T01:23:14.385 INFO:teuthology.orchestra.run.smithi033.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:23:14.397 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T01:23:14.411 INFO:teuthology.orchestra.run.smithi138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T01:23:14.441 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:23:14.445 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:23:14.445 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-25T01:23:14.445 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:23:14.445 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:23:14.445 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:23:14.445 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:23:14.475 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:23:14.479 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:23:14.479 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-25T01:23:14.479 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:23:14.479 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:23:14.479 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:23:14.479 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:23:14.480 INFO:teuthology.run_tasks:Running task install... 2024-04-25T01:23:14.488 DEBUG:teuthology.task.install:project ceph 2024-04-25T01:23:14.488 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T01:23:14.488 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T01:23:14.489 INFO:teuthology.task.install:Using flavor: default 2024-04-25T01:23:14.494 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T01:23:14.494 INFO:teuthology.task.install:extra packages: [] 2024-04-25T01:23:14.494 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:23:14.495 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T01:23:14.495 INFO:teuthology.packaging:ref: None 2024-04-25T01:23:14.495 INFO:teuthology.packaging:tag: None 2024-04-25T01:23:14.495 INFO:teuthology.packaging:branch: quincy 2024-04-25T01:23:14.495 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:23:14.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-25T01:23:14.497 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T01:23:14.497 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T01:23:14.497 INFO:teuthology.packaging:ref: None 2024-04-25T01:23:14.497 INFO:teuthology.packaging:tag: None 2024-04-25T01:23:14.497 INFO:teuthology.packaging:branch: quincy 2024-04-25T01:23:14.497 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:23:14.497 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-25T01:23:14.746 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/ 2024-04-25T01:23:14.746 INFO:teuthology.task.install.rpm:Package version is 17.2.7-902.gc61472c8 2024-04-25T01:23:14.860 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:23:14.861 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:23:14.861 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:23:14.885 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-25T01:23:14.885 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T01:23:14.885 INFO:teuthology.packaging:ref: None 2024-04-25T01:23:14.885 INFO:teuthology.packaging:tag: None 2024-04-25T01:23:14.885 INFO:teuthology.packaging:branch: quincy 2024-04-25T01:23:14.885 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:23:14.886 DEBUG:teuthology.orchestra.run.smithi138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:23:14.949 DEBUG:teuthology.orchestra.run.smithi138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:23:14.962 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/ 2024-04-25T01:23:14.962 INFO:teuthology.task.install.rpm:Package version is 17.2.7-902.gc61472c8 2024-04-25T01:23:15.026 DEBUG:teuthology.orchestra.run.smithi138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:23:15.076 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T01:23:15.076 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:23:15.076 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T01:23:15.088 INFO:teuthology.orchestra.run.smithi138.stdout:check_obsoletes = 1 2024-04-25T01:23:15.090 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum clean all 2024-04-25T01:23:15.105 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-25T01:23:15.106 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T01:23:15.106 INFO:teuthology.packaging:ref: None 2024-04-25T01:23:15.106 INFO:teuthology.packaging:tag: None 2024-04-25T01:23:15.106 INFO:teuthology.packaging:branch: quincy 2024-04-25T01:23:15.106 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:23:15.106 DEBUG:teuthology.orchestra.run.smithi033:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T01:23:15.173 DEBUG:teuthology.orchestra.run.smithi033:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T01:23:15.249 DEBUG:teuthology.orchestra.run.smithi033:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T01:23:15.315 INFO:teuthology.orchestra.run.smithi033.stdout:check_obsoletes = 1 2024-04-25T01:23:15.315 INFO:teuthology.orchestra.run.smithi138.stdout:48 files removed 2024-04-25T01:23:15.316 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum clean all 2024-04-25T01:23:15.342 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-radosgw 2024-04-25T01:23:15.556 INFO:teuthology.orchestra.run.smithi033.stdout:48 files removed 2024-04-25T01:23:15.583 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-radosgw 2024-04-25T01:23:16.046 INFO:teuthology.orchestra.run.smithi138.stdout:ceph packages for x86_64 174 kB/s | 77 kB 00:00 2024-04-25T01:23:16.281 INFO:teuthology.orchestra.run.smithi033.stdout:ceph packages for x86_64 180 kB/s | 77 kB 00:00 2024-04-25T01:23:16.414 INFO:teuthology.orchestra.run.smithi138.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-04-25T01:23:16.699 INFO:teuthology.orchestra.run.smithi033.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-04-25T01:23:16.906 INFO:teuthology.orchestra.run.smithi138.stdout:ceph source packages 3.9 kB/s | 1.8 kB 00:00 2024-04-25T01:23:17.091 INFO:teuthology.orchestra.run.smithi033.stdout:ceph source packages 5.0 kB/s | 1.8 kB 00:00 2024-04-25T01:23:17.456 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-25T01:23:18.341 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-04-25T01:23:21.625 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 19 MB 00:02 2024-04-25T01:23:26.384 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-04-25T01:23:26.682 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - AppStream 2.3 MB/s | 19 MB 00:08 2024-04-25T01:23:28.219 INFO:teuthology.orchestra.run.smithi033.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-25T01:23:30.028 INFO:teuthology.orchestra.run.smithi033.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-04-25T01:23:30.766 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-25T01:23:32.458 INFO:teuthology.orchestra.run.smithi138.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-25T01:23:33.862 INFO:teuthology.orchestra.run.smithi138.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-25T01:23:35.253 INFO:teuthology.orchestra.run.smithi033.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T01:23:37.471 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 14 M 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrading: 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout: librados2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.1 M 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base x86_64 2:17.2.7-902.gc61472c8.el9 ceph 5.0 M 2024-04-25T01:23:37.473 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 22 M 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 637 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 457 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 4.9 M 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 45 k 2024-04-25T01:23:37.474 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 117 k 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs x86_64 2:17.2.7-902.gc61472c8.el9 ceph 156 k 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados x86_64 2:17.2.7-902.gc61472c8.el9 ceph 320 k 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 293 k 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 99 k 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout:Install 22 Packages 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout:Upgrade 2 Packages 2024-04-25T01:23:37.475 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:23:37.476 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 58 M 2024-04-25T01:23:37.476 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:23:38.187 INFO:teuthology.orchestra.run.smithi033.stdout:(1/24): ceph-base-17.2.7-902.gc61472c8.el9.x86_ 8.8 MB/s | 5.0 MB 00:00 2024-04-25T01:23:38.229 INFO:teuthology.orchestra.run.smithi033.stdout:(2/24): ceph-selinux-17.2.7-902.gc61472c8.el9.x 608 kB/s | 25 kB 00:00 2024-04-25T01:23:38.312 INFO:teuthology.orchestra.run.smithi033.stdout:(3/24): libcephfs2-17.2.7-902.gc61472c8.el9.x86 7.5 MB/s | 637 kB 00:00 2024-04-25T01:23:38.387 INFO:teuthology.orchestra.run.smithi033.stdout:(4/24): libradosstriper1-17.2.7-902.gc61472c8.e 6.0 MB/s | 457 kB 00:00 2024-04-25T01:23:38.766 INFO:teuthology.orchestra.run.smithi033.stdout:(5/24): librgw2-17.2.7-902.gc61472c8.el9.x86_64 13 MB/s | 4.9 MB 00:00 2024-04-25T01:23:38.830 INFO:teuthology.orchestra.run.smithi033.stdout:(6/24): python3-ceph-argparse-17.2.7-902.gc6147 710 kB/s | 45 kB 00:00 2024-04-25T01:23:38.897 INFO:teuthology.orchestra.run.smithi033.stdout:(7/24): python3-ceph-common-17.2.7-902.gc61472c 1.7 MB/s | 117 kB 00:00 2024-04-25T01:23:38.963 INFO:teuthology.orchestra.run.smithi033.stdout:(8/24): python3-cephfs-17.2.7-902.gc61472c8.el9 2.3 MB/s | 156 kB 00:00 2024-04-25T01:23:39.001 INFO:teuthology.orchestra.run.smithi138.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T01:23:39.030 INFO:teuthology.orchestra.run.smithi033.stdout:(9/24): python3-rados-17.2.7-902.gc61472c8.el9. 4.7 MB/s | 320 kB 00:00 2024-04-25T01:23:39.097 INFO:teuthology.orchestra.run.smithi033.stdout:(10/24): python3-rbd-17.2.7-902.gc61472c8.el9.x 4.3 MB/s | 293 kB 00:00 2024-04-25T01:23:39.156 INFO:teuthology.orchestra.run.smithi033.stdout:(11/24): python3-rgw-17.2.7-902.gc61472c8.el9.x 1.7 MB/s | 99 kB 00:00 2024-04-25T01:23:39.406 INFO:teuthology.orchestra.run.smithi033.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 641 kB/s | 160 kB 00:00 2024-04-25T01:23:39.657 INFO:teuthology.orchestra.run.smithi033.stdout:(13/24): ceph-common-17.2.7-902.gc61472c8.el9.x 11 MB/s | 22 MB 00:02 2024-04-25T01:23:39.673 INFO:teuthology.orchestra.run.smithi033.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 169 kB/s | 45 kB 00:00 2024-04-25T01:23:40.107 INFO:teuthology.orchestra.run.smithi033.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 675 kB/s | 292 kB 00:00 2024-04-25T01:23:40.157 INFO:teuthology.orchestra.run.smithi033.stdout:(16/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.3 MB/s | 662 kB 00:00 2024-04-25T01:23:40.357 INFO:teuthology.orchestra.run.smithi033.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 169 kB/s | 42 kB 00:00 2024-04-25T01:23:40.691 INFO:teuthology.orchestra.run.smithi033.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 146 kB/s | 49 kB 00:00 2024-04-25T01:23:40.716 INFO:teuthology.orchestra.run.smithi033.stdout:(19/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 553 kB/s | 309 kB 00:00 2024-04-25T01:23:40.867 INFO:teuthology.orchestra.run.smithi033.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 385 kB/s | 67 kB 00:00 2024-04-25T01:23:41.111 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:23:41.113 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:23:41.113 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 14 M 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrading: 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout: librados2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.1 M 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base x86_64 2:17.2.7-902.gc61472c8.el9 ceph 5.0 M 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 22 M 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T01:23:41.114 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 637 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 457 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 4.9 M 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 45 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 117 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs x86_64 2:17.2.7-902.gc61472c8.el9 ceph 156 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados x86_64 2:17.2.7-902.gc61472c8.el9 ceph 320 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 293 k 2024-04-25T01:23:41.115 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 99 k 2024-04-25T01:23:41.116 INFO:teuthology.orchestra.run.smithi138.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T01:23:41.116 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:23:41.116 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:23:41.116 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:23:41.116 INFO:teuthology.orchestra.run.smithi138.stdout:Install 22 Packages 2024-04-25T01:23:41.116 INFO:teuthology.orchestra.run.smithi138.stdout:Upgrade 2 Packages 2024-04-25T01:23:41.116 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:23:41.116 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 58 M 2024-04-25T01:23:41.116 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:23:41.133 INFO:teuthology.orchestra.run.smithi033.stdout:(21/24): librados2-17.2.7-902.gc61472c8.el9.x86 12 MB/s | 3.1 MB 00:00 2024-04-25T01:23:41.300 INFO:teuthology.orchestra.run.smithi033.stdout:(22/24): librbd1-17.2.7-902.gc61472c8.el9.x86_6 18 MB/s | 2.9 MB 00:00 2024-04-25T01:23:41.801 INFO:teuthology.orchestra.run.smithi033.stdout:(23/24): thrift-0.15.0-2.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-04-25T01:23:43.003 INFO:teuthology.orchestra.run.smithi138.stdout:(1/24): ceph-common-17.2.7-902.gc61472c8.el9.x8 15 MB/s | 22 MB 00:01 2024-04-25T01:23:43.044 INFO:teuthology.orchestra.run.smithi138.stdout:(2/24): ceph-selinux-17.2.7-902.gc61472c8.el9.x 610 kB/s | 25 kB 00:00 2024-04-25T01:23:43.059 INFO:teuthology.orchestra.run.smithi033.stdout:(24/24): ceph-radosgw-17.2.7-902.gc61472c8.el9. 2.6 MB/s | 14 MB 00:05 2024-04-25T01:23:43.062 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:23:43.062 INFO:teuthology.orchestra.run.smithi033.stdout:Total 10 MB/s | 58 MB 00:05 2024-04-25T01:23:43.126 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:23:43.137 INFO:teuthology.orchestra.run.smithi138.stdout:(3/24): libcephfs2-17.2.7-902.gc61472c8.el9.x86 6.8 MB/s | 637 kB 00:00 2024-04-25T01:23:43.164 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:23:43.164 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:23:43.228 INFO:teuthology.orchestra.run.smithi138.stdout:(4/24): ceph-base-17.2.7-902.gc61472c8.el9.x86_ 2.9 MB/s | 5.0 MB 00:01 2024-04-25T01:23:43.254 INFO:teuthology.orchestra.run.smithi138.stdout:(5/24): libradosstriper1-17.2.7-902.gc61472c8.e 3.8 MB/s | 457 kB 00:00 2024-04-25T01:23:43.295 INFO:teuthology.orchestra.run.smithi138.stdout:(6/24): python3-ceph-argparse-17.2.7-902.gc6147 1.1 MB/s | 45 kB 00:00 2024-04-25T01:23:43.354 INFO:teuthology.orchestra.run.smithi138.stdout:(7/24): python3-ceph-common-17.2.7-902.gc61472c 2.0 MB/s | 117 kB 00:00 2024-04-25T01:23:43.413 INFO:teuthology.orchestra.run.smithi138.stdout:(8/24): python3-cephfs-17.2.7-902.gc61472c8.el9 2.6 MB/s | 156 kB 00:00 2024-04-25T01:23:43.471 INFO:teuthology.orchestra.run.smithi138.stdout:(9/24): python3-rados-17.2.7-902.gc61472c8.el9. 5.4 MB/s | 320 kB 00:00 2024-04-25T01:23:43.538 INFO:teuthology.orchestra.run.smithi138.stdout:(10/24): python3-rbd-17.2.7-902.gc61472c8.el9.x 4.3 MB/s | 293 kB 00:00 2024-04-25T01:23:43.589 INFO:teuthology.orchestra.run.smithi138.stdout:(11/24): python3-rgw-17.2.7-902.gc61472c8.el9.x 2.0 MB/s | 99 kB 00:00 2024-04-25T01:23:43.621 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:23:43.622 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:23:43.789 INFO:teuthology.orchestra.run.smithi138.stdout:(12/24): librgw2-17.2.7-902.gc61472c8.el9.x86_6 8.8 MB/s | 4.9 MB 00:00 2024-04-25T01:23:43.814 INFO:teuthology.orchestra.run.smithi138.stdout:(13/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 712 kB/s | 160 kB 00:00 2024-04-25T01:23:43.981 INFO:teuthology.orchestra.run.smithi138.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 236 kB/s | 45 kB 00:00 2024-04-25T01:23:44.048 INFO:teuthology.orchestra.run.smithi138.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-04-25T01:23:44.107 INFO:teuthology.orchestra.run.smithi138.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-25T01:23:44.132 INFO:teuthology.orchestra.run.smithi138.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 505 kB/s | 42 kB 00:00 2024-04-25T01:23:44.416 INFO:teuthology.orchestra.run.smithi138.stdout:(18/24): ceph-radosgw-17.2.7-902.gc61472c8.el9. 4.9 MB/s | 14 MB 00:02 2024-04-25T01:23:44.474 INFO:teuthology.orchestra.run.smithi138.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 142 kB/s | 49 kB 00:00 2024-04-25T01:23:44.733 INFO:teuthology.orchestra.run.smithi138.stdout:(20/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 494 kB/s | 309 kB 00:00 2024-04-25T01:23:44.766 INFO:teuthology.orchestra.run.smithi138.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 192 kB/s | 67 kB 00:00 2024-04-25T01:23:44.873 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:23:45.017 INFO:teuthology.orchestra.run.smithi138.stdout:(22/24): librbd1-17.2.7-902.gc61472c8.el9.x86_6 12 MB/s | 2.9 MB 00:00 2024-04-25T01:23:45.059 INFO:teuthology.orchestra.run.smithi138.stdout:(23/24): thrift-0.15.0-2.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-04-25T01:23:45.217 INFO:teuthology.orchestra.run.smithi138.stdout:(24/24): librados2-17.2.7-902.gc61472c8.el9.x86 6.4 MB/s | 3.1 MB 00:00 2024-04-25T01:23:45.221 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:23:45.221 INFO:teuthology.orchestra.run.smithi138.stdout:Total 14 MB/s | 58 MB 00:04 2024-04-25T01:23:45.282 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:23:45.284 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-25T01:23:45.316 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:23:45.316 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:23:45.362 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T01:23:45.455 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T01:23:45.535 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-25T01:23:45.563 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T01:23:45.651 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T01:23:45.700 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-25T01:23:45.728 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-25T01:23:45.751 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:23:45.751 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:23:46.187 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-25T01:23:46.218 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-25T01:23:46.264 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-25T01:23:46.326 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-25T01:23:46.369 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 10/26 2024-04-25T01:23:46.406 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-25T01:23:46.470 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-25T01:23:46.495 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T01:23:46.560 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T01:23:46.600 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-25T01:23:46.629 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-25T01:23:46.661 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-25T01:23:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-25T01:23:47.001 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:23:47.007 INFO:teuthology.orchestra.run.smithi033.stdout: Upgrading : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-25T01:23:47.098 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-25T01:23:47.173 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 19/26 2024-04-25T01:23:47.202 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 20/26 2024-04-25T01:23:47.395 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-25T01:23:47.454 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T01:23:47.545 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T01:23:47.624 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-25T01:23:47.647 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T01:23:47.768 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T01:23:47.814 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-25T01:23:47.843 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-25T01:23:48.278 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-25T01:23:48.308 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-25T01:23:48.370 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-25T01:23:48.408 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-25T01:23:48.442 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 10/26 2024-04-25T01:23:48.479 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-25T01:23:48.542 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-25T01:23:48.577 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T01:23:48.635 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T01:23:48.682 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-25T01:23:48.711 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-25T01:23:48.743 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-25T01:23:49.045 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-25T01:23:49.090 INFO:teuthology.orchestra.run.smithi138.stdout: Upgrading : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-25T01:23:49.179 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-25T01:23:49.254 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 19/26 2024-04-25T01:23:49.284 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 20/26 2024-04-25T01:23:49.702 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T01:23:49.740 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T01:23:50.277 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T01:23:50.318 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-25T01:23:50.415 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-25T01:23:50.415 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:23:50.416 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:23:50.416 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:23:50.445 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T01:23:51.702 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T01:23:51.732 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T01:23:52.238 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T01:23:52.285 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-25T01:23:52.374 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-25T01:23:52.374 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T01:23:52.374 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T01:23:52.374 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:23:52.403 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T01:24:00.095 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T01:24:00.096 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /sys 2024-04-25T01:24:00.096 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /proc 2024-04-25T01:24:00.096 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /mnt 2024-04-25T01:24:00.096 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /var/tmp 2024-04-25T01:24:00.096 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /home 2024-04-25T01:24:00.096 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /root 2024-04-25T01:24:00.096 INFO:teuthology.orchestra.run.smithi033.stdout:skipping the directory /tmp 2024-04-25T01:24:00.096 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:01.436 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-25T01:24:01.517 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-25T01:24:01.517 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:01.518 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:24:01.518 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:24:01.518 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:24:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:01.518 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-25T01:24:01.605 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-25T01:24:01.605 INFO:teuthology.orchestra.run.smithi033.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T01:24:01.924 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T01:24:01.924 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /sys 2024-04-25T01:24:01.924 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /proc 2024-04-25T01:24:01.924 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /mnt 2024-04-25T01:24:01.924 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /var/tmp 2024-04-25T01:24:01.924 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /home 2024-04-25T01:24:01.924 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /root 2024-04-25T01:24:01.924 INFO:teuthology.orchestra.run.smithi138.stdout:skipping the directory /tmp 2024-04-25T01:24:01.924 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:02.917 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T01:24:02.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 1/26 2024-04-25T01:24:02.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T01:24:02.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-25T01:24:02.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T01:24:02.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 5/26 2024-04-25T01:24:02.917 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 6/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 7/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 8/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 9/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 10/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-25T01:24:02.918 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-25T01:24:02.919 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-25T01:24:02.919 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T01:24:02.919 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-25T01:24:02.919 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 25/26 2024-04-25T01:24:03.222 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-25T01:24:03.299 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-25T01:24:03.299 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:03.299 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T01:24:03.299 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:24:03.299 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T01:24:03.299 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:03.301 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-25T01:24:03.383 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-25T01:24:03.383 INFO:teuthology.orchestra.run.smithi138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T01:24:03.634 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout:Upgraded: 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:24:03.635 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:24:03.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:03.637 INFO:teuthology.orchestra.run.smithi033.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:24:03.637 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:03.637 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:03.958 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-test 2024-04-25T01:24:04.498 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:29 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:04.620 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test x86_64 2:17.2.7-902.gc61472c8.el9 ceph 48 M 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite x86_64 2:17.2.7-902.gc61472c8.el9 ceph 163 k 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:24:04.621 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:04.622 INFO:teuthology.orchestra.run.smithi033.stdout:Install 4 Packages 2024-04-25T01:24:04.622 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:04.622 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 48 M 2024-04-25T01:24:04.622 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 192 M 2024-04-25T01:24:04.622 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:24:04.661 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T01:24:04.661 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 1/26 2024-04-25T01:24:04.661 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T01:24:04.662 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-25T01:24:04.662 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T01:24:04.662 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 5/26 2024-04-25T01:24:04.662 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 6/26 2024-04-25T01:24:04.662 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 7/26 2024-04-25T01:24:04.662 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 8/26 2024-04-25T01:24:04.663 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 9/26 2024-04-25T01:24:04.663 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 10/26 2024-04-25T01:24:04.663 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-25T01:24:04.663 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-25T01:24:04.663 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T01:24:04.663 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-25T01:24:04.664 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-25T01:24:04.664 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-25T01:24:04.664 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-25T01:24:04.664 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-25T01:24:04.664 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-25T01:24:04.664 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-25T01:24:04.665 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-25T01:24:04.665 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-25T01:24:04.665 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T01:24:04.665 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-25T01:24:04.665 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 25/26 2024-04-25T01:24:04.886 INFO:teuthology.orchestra.run.smithi033.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.9 MB/s | 305 kB 00:00 2024-04-25T01:24:04.936 INFO:teuthology.orchestra.run.smithi033.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T01:24:04.995 INFO:teuthology.orchestra.run.smithi033.stdout:(3/4): libcephsqlite-17.2.7-902.gc61472c8.el9.x 618 kB/s | 163 kB 00:00 2024-04-25T01:24:05.349 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T01:24:05.349 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:05.349 INFO:teuthology.orchestra.run.smithi138.stdout:Upgraded: 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.350 INFO:teuthology.orchestra.run.smithi138.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:05.351 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:24:05.624 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-test 2024-04-25T01:24:06.211 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:28 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:24:06.335 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:24:06.336 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:06.336 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T01:24:06.336 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:06.336 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:24:06.336 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test x86_64 2:17.2.7-902.gc61472c8.el9 ceph 48 M 2024-04-25T01:24:06.336 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T01:24:06.336 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite x86_64 2:17.2.7-902.gc61472c8.el9 ceph 163 k 2024-04-25T01:24:06.337 INFO:teuthology.orchestra.run.smithi138.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T01:24:06.337 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T01:24:06.337 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:06.337 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:24:06.337 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:06.337 INFO:teuthology.orchestra.run.smithi138.stdout:Install 4 Packages 2024-04-25T01:24:06.337 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:06.338 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 48 M 2024-04-25T01:24:06.338 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 192 M 2024-04-25T01:24:06.338 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:24:06.709 INFO:teuthology.orchestra.run.smithi138.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 305 kB 00:00 2024-04-25T01:24:06.919 INFO:teuthology.orchestra.run.smithi138.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 303 kB/s | 64 kB 00:00 2024-04-25T01:24:07.002 INFO:teuthology.orchestra.run.smithi138.stdout:(3/4): libcephsqlite-17.2.7-902.gc61472c8.el9.x 273 kB/s | 163 kB 00:00 2024-04-25T01:24:07.341 INFO:teuthology.orchestra.run.smithi033.stdout:(4/4): ceph-test-17.2.7-902.gc61472c8.el9.x86_6 18 MB/s | 48 MB 00:02 2024-04-25T01:24:07.343 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:07.343 INFO:teuthology.orchestra.run.smithi033.stdout:Total 18 MB/s | 48 MB 00:02 2024-04-25T01:24:07.370 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:24:07.392 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:24:07.392 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:24:07.773 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:24:07.773 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:24:08.308 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:24:08.383 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-25T01:24:08.424 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-25T01:24:08.451 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-25T01:24:10.179 INFO:teuthology.orchestra.run.smithi138.stdout:(4/4): ceph-test-17.2.7-902.gc61472c8.el9.x86_6 13 MB/s | 48 MB 00:03 2024-04-25T01:24:10.181 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:10.182 INFO:teuthology.orchestra.run.smithi138.stdout:Total 13 MB/s | 48 MB 00:03 2024-04-25T01:24:10.208 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:24:10.227 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:24:10.227 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:24:10.589 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:24:10.589 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:24:11.109 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:24:11.196 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-25T01:24:11.237 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-25T01:24:11.264 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-25T01:24:13.073 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-25T01:24:13.149 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-25T01:24:15.435 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-25T01:24:15.436 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 1/4 2024-04-25T01:24:15.436 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2/4 2024-04-25T01:24:15.436 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-25T01:24:15.635 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-25T01:24:15.720 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-25T01:24:15.728 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-25T01:24:15.728 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:15.728 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:24:15.728 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:15.728 INFO:teuthology.orchestra.run.smithi033.stdout: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:15.728 INFO:teuthology.orchestra.run.smithi033.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:24:15.728 INFO:teuthology.orchestra.run.smithi033.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:24:15.728 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:15.728 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:15.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph 2024-04-25T01:24:16.505 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:41 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:16.632 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:16.635 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:16.635 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-25T01:24:16.635 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:16.635 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:24:16.635 INFO:teuthology.orchestra.run.smithi033.stdout: ceph x86_64 2:17.2.7-902.gc61472c8.el9 ceph 6.5 k 2024-04-25T01:24:16.635 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-25T01:24:16.635 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.1 M 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr x86_64 2:17.2.7-902.gc61472c8.el9 ceph 1.4 M 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 240 k 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.9 M 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 15 M 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:24:16.636 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:24:16.637 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:24:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:24:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:24:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:16.638 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:24:16.638 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:16.638 INFO:teuthology.orchestra.run.smithi033.stdout:Install 33 Packages 2024-04-25T01:24:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:16.639 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 27 M 2024-04-25T01:24:16.639 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 96 M 2024-04-25T01:24:16.639 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:24:17.474 INFO:teuthology.orchestra.run.smithi033.stdout:(1/33): ceph-17.2.7-902.gc61472c8.el9.x86_64.rp 38 kB/s | 6.5 kB 00:00 2024-04-25T01:24:17.707 INFO:teuthology.orchestra.run.smithi033.stdout:(2/33): ceph-mds-17.2.7-902.gc61472c8.el9.x86_6 5.1 MB/s | 2.1 MB 00:00 2024-04-25T01:24:17.978 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-25T01:24:17.979 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 1/4 2024-04-25T01:24:17.979 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2/4 2024-04-25T01:24:17.979 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-25T01:24:18.007 INFO:teuthology.orchestra.run.smithi033.stdout:(3/33): ceph-mon-17.2.7-902.gc61472c8.el9.x86_6 7.4 MB/s | 3.9 MB 00:00 2024-04-25T01:24:18.099 INFO:teuthology.orchestra.run.smithi033.stdout:(4/33): ceph-mgr-modules-core-17.2.7-902.gc6147 2.6 MB/s | 240 kB 00:00 2024-04-25T01:24:18.249 INFO:teuthology.orchestra.run.smithi033.stdout:(5/33): ceph-mgr-17.2.7-902.gc61472c8.el9.x86_6 1.5 MB/s | 1.4 MB 00:00 2024-04-25T01:24:18.254 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-25T01:24:18.254 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:18.254 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:24:18.254 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:18.254 INFO:teuthology.orchestra.run.smithi138.stdout: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:18.254 INFO:teuthology.orchestra.run.smithi138.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T01:24:18.254 INFO:teuthology.orchestra.run.smithi138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T01:24:18.254 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:18.254 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:24:18.366 INFO:teuthology.orchestra.run.smithi033.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 949 kB/s | 253 kB 00:00 2024-04-25T01:24:18.533 INFO:teuthology.orchestra.run.smithi033.stdout:(7/33): ceph-osd-17.2.7-902.gc61472c8.el9.x86_6 19 MB/s | 15 MB 00:00 2024-04-25T01:24:18.555 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph 2024-04-25T01:24:18.558 INFO:teuthology.orchestra.run.smithi033.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 555 kB/s | 106 kB 00:00 2024-04-25T01:24:18.617 INFO:teuthology.orchestra.run.smithi033.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-04-25T01:24:18.725 INFO:teuthology.orchestra.run.smithi033.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.9 MB/s | 215 kB 00:00 2024-04-25T01:24:18.751 INFO:teuthology.orchestra.run.smithi033.stdout:(11/33): python3-pycparser-2.20-6.el9.noarch.rp 622 kB/s | 135 kB 00:00 2024-04-25T01:24:18.826 INFO:teuthology.orchestra.run.smithi033.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 2.1 MB/s | 1.2 MB 00:00 2024-04-25T01:24:18.893 INFO:teuthology.orchestra.run.smithi033.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 245 kB/s | 35 kB 00:00 2024-04-25T01:24:19.002 INFO:teuthology.orchestra.run.smithi033.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 290 kB/s | 51 kB 00:00 2024-04-25T01:24:19.044 INFO:teuthology.orchestra.run.smithi033.stdout:(15/33): python3-mako-1.1.4-6.el9.noarch.rpm 542 kB/s | 172 kB 00:00 2024-04-25T01:24:19.060 INFO:teuthology.orchestra.run.smithi033.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 249 kB/s | 42 kB 00:00 2024-04-25T01:24:19.061 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:41 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:24:19.186 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout: ceph x86_64 2:17.2.7-902.gc61472c8.el9 ceph 6.5 k 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.1 M 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr x86_64 2:17.2.7-902.gc61472c8.el9 ceph 1.4 M 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 240 k 2024-04-25T01:24:19.189 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.9 M 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 15 M 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T01:24:19.190 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:19.191 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:24:19.192 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:19.192 INFO:teuthology.orchestra.run.smithi138.stdout:Install 33 Packages 2024-04-25T01:24:19.192 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:19.192 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 27 M 2024-04-25T01:24:19.192 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 96 M 2024-04-25T01:24:19.192 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:24:19.344 INFO:teuthology.orchestra.run.smithi033.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 127 kB/s | 43 kB 00:00 2024-04-25T01:24:19.486 INFO:teuthology.orchestra.run.smithi033.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 388 kB/s | 172 kB 00:00 2024-04-25T01:24:19.528 INFO:teuthology.orchestra.run.smithi033.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 58 kB/s | 11 kB 00:00 2024-04-25T01:24:19.595 INFO:teuthology.orchestra.run.smithi033.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 671 kB/s | 358 kB 00:00 2024-04-25T01:24:19.745 INFO:teuthology.orchestra.run.smithi033.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 68 kB/s | 18 kB 00:00 2024-04-25T01:24:19.795 INFO:teuthology.orchestra.run.smithi033.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 87 kB/s | 23 kB 00:00 2024-04-25T01:24:19.854 INFO:teuthology.orchestra.run.smithi033.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 75 kB/s | 19 kB 00:00 2024-04-25T01:24:20.013 INFO:teuthology.orchestra.run.smithi033.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 74 kB/s | 20 kB 00:00 2024-04-25T01:24:20.146 INFO:teuthology.orchestra.run.smithi033.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 132 kB/s | 46 kB 00:00 2024-04-25T01:24:20.172 INFO:teuthology.orchestra.run.smithi138.stdout:(1/33): ceph-17.2.7-902.gc61472c8.el9.x86_64.rp 39 kB/s | 6.5 kB 00:00 2024-04-25T01:24:20.197 INFO:teuthology.orchestra.run.smithi033.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 230 kB/s | 79 kB 00:00 2024-04-25T01:24:20.356 INFO:teuthology.orchestra.run.smithi138.stdout:(2/33): ceph-mgr-17.2.7-902.gc61472c8.el9.x86_6 4.0 MB/s | 1.4 MB 00:00 2024-04-25T01:24:20.405 INFO:teuthology.orchestra.run.smithi033.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 63 kB/s | 16 kB 00:00 2024-04-25T01:24:20.490 INFO:teuthology.orchestra.run.smithi138.stdout:(3/33): ceph-mds-17.2.7-902.gc61472c8.el9.x86_6 4.3 MB/s | 2.1 MB 00:00 2024-04-25T01:24:20.522 INFO:teuthology.orchestra.run.smithi033.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-04-25T01:24:20.581 INFO:teuthology.orchestra.run.smithi138.stdout:(4/33): ceph-mgr-modules-core-17.2.7-902.gc6147 2.6 MB/s | 240 kB 00:00 2024-04-25T01:24:20.622 INFO:teuthology.orchestra.run.smithi033.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 211 kB/s | 90 kB 00:00 2024-04-25T01:24:20.673 INFO:teuthology.orchestra.run.smithi033.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 134 kB/s | 36 kB 00:00 2024-04-25T01:24:20.707 INFO:teuthology.orchestra.run.smithi138.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-25T01:24:20.865 INFO:teuthology.orchestra.run.smithi138.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 7.8 MB/s | 1.2 MB 00:00 2024-04-25T01:24:20.940 INFO:teuthology.orchestra.run.smithi033.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 75 kB/s | 20 kB 00:00 2024-04-25T01:24:20.966 INFO:teuthology.orchestra.run.smithi138.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-04-25T01:24:21.048 INFO:teuthology.orchestra.run.smithi033.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 437 kB/s | 230 kB 00:00 2024-04-25T01:24:21.208 INFO:teuthology.orchestra.run.smithi138.stdout:(8/33): ceph-osd-17.2.7-902.gc61472c8.el9.x86_6 18 MB/s | 15 MB 00:00 2024-04-25T01:24:21.215 INFO:teuthology.orchestra.run.smithi033.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 722 kB/s | 427 kB 00:00 2024-04-25T01:24:21.218 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:21.219 INFO:teuthology.orchestra.run.smithi033.stdout:Total 5.9 MB/s | 27 MB 00:04 2024-04-25T01:24:21.233 INFO:teuthology.orchestra.run.smithi138.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 507 kB/s | 135 kB 00:00 2024-04-25T01:24:21.325 INFO:teuthology.orchestra.run.smithi138.stdout:(10/33): ceph-mon-17.2.7-902.gc61472c8.el9.x86_ 3.4 MB/s | 3.9 MB 00:01 2024-04-25T01:24:21.339 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:24:21.379 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:24:21.379 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:24:21.392 INFO:teuthology.orchestra.run.smithi138.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 678 kB/s | 125 kB 00:00 2024-04-25T01:24:21.443 INFO:teuthology.orchestra.run.smithi138.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 1.1 MB/s | 215 kB 00:00 2024-04-25T01:24:21.659 INFO:teuthology.orchestra.run.smithi138.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 130 kB/s | 35 kB 00:00 2024-04-25T01:24:21.663 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:24:21.665 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:24:21.793 INFO:teuthology.orchestra.run.smithi138.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 145 kB/s | 51 kB 00:00 2024-04-25T01:24:21.843 INFO:teuthology.orchestra.run.smithi138.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 227 kB/s | 42 kB 00:00 2024-04-25T01:24:21.868 INFO:teuthology.orchestra.run.smithi138.stdout:(16/33): python3-mako-1.1.4-6.el9.noarch.rpm 333 kB/s | 172 kB 00:00 2024-04-25T01:24:22.110 INFO:teuthology.orchestra.run.smithi138.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-04-25T01:24:22.261 INFO:teuthology.orchestra.run.smithi138.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 71 kB/s | 11 kB 00:00 2024-04-25T01:24:22.294 INFO:teuthology.orchestra.run.smithi138.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 381 kB/s | 172 kB 00:00 2024-04-25T01:24:22.394 INFO:teuthology.orchestra.run.smithi138.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 682 kB/s | 358 kB 00:00 2024-04-25T01:24:22.420 INFO:teuthology.orchestra.run.smithi138.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 111 kB/s | 18 kB 00:00 2024-04-25T01:24:22.445 INFO:teuthology.orchestra.run.smithi138.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 153 kB/s | 23 kB 00:00 2024-04-25T01:24:22.487 INFO:teuthology.orchestra.run.smithi138.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2024-04-25T01:24:22.512 INFO:teuthology.orchestra.run.smithi138.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 213 kB/s | 20 kB 00:00 2024-04-25T01:24:22.537 INFO:teuthology.orchestra.run.smithi138.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 502 kB/s | 46 kB 00:00 2024-04-25T01:24:22.579 INFO:teuthology.orchestra.run.smithi138.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 853 kB/s | 79 kB 00:00 2024-04-25T01:24:22.630 INFO:teuthology.orchestra.run.smithi138.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 178 kB/s | 16 kB 00:00 2024-04-25T01:24:22.680 INFO:teuthology.orchestra.run.smithi138.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 897 kB/s | 90 kB 00:00 2024-04-25T01:24:22.722 INFO:teuthology.orchestra.run.smithi138.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 389 kB/s | 36 kB 00:00 2024-04-25T01:24:22.781 INFO:teuthology.orchestra.run.smithi138.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2024-04-25T01:24:22.806 INFO:teuthology.orchestra.run.smithi138.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-25T01:24:22.836 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:24:22.873 INFO:teuthology.orchestra.run.smithi138.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 217 kB/s | 20 kB 00:00 2024-04-25T01:24:22.901 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-25T01:24:22.906 INFO:teuthology.orchestra.run.smithi138.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.3 MB/s | 427 kB 00:00 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:22.908 INFO:teuthology.orchestra.run.smithi138.stdout:Total 7.3 MB/s | 27 MB 00:03 2024-04-25T01:24:22.934 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-25T01:24:22.966 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-25T01:24:23.026 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:24:23.062 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:24:23.063 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:24:23.148 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-25T01:24:23.195 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-25T01:24:23.237 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-25T01:24:23.267 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-25T01:24:23.301 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-25T01:24:23.329 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:24:23.330 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:24:23.334 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-25T01:24:23.376 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-25T01:24:23.419 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-25T01:24:23.453 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-25T01:24:23.492 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-25T01:24:23.568 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-25T01:24:23.728 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-25T01:24:23.811 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-25T01:24:23.862 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-25T01:24:23.899 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-25T01:24:23.958 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-25T01:24:24.022 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-25T01:24:24.245 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-25T01:24:24.303 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-25T01:24:24.385 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-25T01:24:24.468 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:24:24.482 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-25T01:24:24.519 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-25T01:24:24.542 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-25T01:24:24.574 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-25T01:24:24.608 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-25T01:24:24.642 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-25T01:24:24.709 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-25T01:24:24.803 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-25T01:24:24.805 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 28/33 2024-04-25T01:24:24.837 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-25T01:24:24.843 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-25T01:24:24.862 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-25T01:24:24.862 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:24.862 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:24:24.863 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:24:24.863 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:24:24.863 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:24.879 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-25T01:24:24.917 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-25T01:24:24.943 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-25T01:24:24.968 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-25T01:24:25.008 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-25T01:24:25.044 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-25T01:24:25.078 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-25T01:24:25.117 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-25T01:24:25.190 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-25T01:24:25.339 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-25T01:24:25.426 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-25T01:24:25.479 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-25T01:24:25.517 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-25T01:24:25.574 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-25T01:24:25.630 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-25T01:24:25.826 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-25T01:24:25.888 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-25T01:24:25.966 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-25T01:24:26.055 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-25T01:24:26.104 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-25T01:24:26.221 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-25T01:24:26.247 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-25T01:24:26.278 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-25T01:24:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:24:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:24:26.279 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:24:26.279 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:26.375 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 28/33 2024-04-25T01:24:26.406 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-25T01:24:26.431 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-25T01:24:26.431 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:26.431 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T01:24:26.431 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:24:26.431 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T01:24:26.431 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:26.673 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-25T01:24:26.704 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-25T01:24:26.705 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:26.705 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:24:26.705 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:24:26.705 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:24:26.705 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:26.924 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-25T01:24:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-25T01:24:26.949 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:26.949 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:24:26.949 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:24:26.949 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:24:26.949 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:27.021 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-25T01:24:27.740 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-25T01:24:27.766 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-25T01:24:27.766 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:27.766 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T01:24:27.767 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:24:27.767 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T01:24:27.767 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:28.133 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-25T01:24:28.155 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-25T01:24:28.156 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:28.156 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T01:24:28.156 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:24:28.156 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T01:24:28.156 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:28.350 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-25T01:24:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-25T01:24:28.373 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:28.373 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T01:24:28.373 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:24:28.373 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T01:24:28.373 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:28.445 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 1/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 3/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 4/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 5/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 6/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-25T01:24:28.565 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-25T01:24:28.566 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-25T01:24:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-25T01:24:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-25T01:24:28.567 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-25T01:24:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-25T01:24:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-25T01:24:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-25T01:24:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-25T01:24:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-25T01:24:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-25T01:24:28.568 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:24:29.415 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:24:29.416 INFO:teuthology.orchestra.run.smithi033.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:29.417 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:29.731 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-base 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 1/33 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2/33 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 3/33 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 4/33 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 5/33 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 6/33 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-25T01:24:30.003 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-25T01:24:30.004 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-25T01:24:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-25T01:24:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-25T01:24:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-25T01:24:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-25T01:24:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-25T01:24:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-25T01:24:30.005 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-25T01:24:30.262 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:55 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:30.343 INFO:teuthology.orchestra.run.smithi033.stdout:Package ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:24:30.386 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:30.387 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:24:30.387 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:30.458 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install cephadm 2024-04-25T01:24:30.931 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-25T01:24:30.931 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:30.931 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:24:30.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:30.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:30.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:30.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:30.931 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T01:24:30.932 INFO:teuthology.orchestra.run.smithi138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:30.933 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:24:30.984 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:56 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:31.108 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:31.108 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:31.108 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-25T01:24:31.108 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:31.108 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:24:31.108 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 82 k 2024-04-25T01:24:31.108 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:31.108 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:24:31.109 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:31.109 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-04-25T01:24:31.109 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:31.109 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 82 k 2024-04-25T01:24:31.109 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 372 k 2024-04-25T01:24:31.109 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:24:31.294 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-base 2024-04-25T01:24:31.312 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm-17.2.7-902.gc61472c8.el9.noarch.rpm 405 kB/s | 82 kB 00:00 2024-04-25T01:24:31.313 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:31.313 INFO:teuthology.orchestra.run.smithi033.stdout:Total 401 kB/s | 82 kB 00:00 2024-04-25T01:24:31.313 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:24:31.319 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:24:31.319 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:24:31.321 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:24:31.321 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:24:31.388 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:24:31.813 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:53 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:24:31.870 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T01:24:31.891 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:24:31.933 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:24:31.933 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:24:31.934 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:24:31.959 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T01:24:31.998 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install cephadm 2024-04-25T01:24:32.252 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T01:24:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T01:24:32.477 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:32.478 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:24:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:32.478 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:32.478 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:32.503 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:54 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:24:32.624 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:24:32.624 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 82 k 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 82 k 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 372 k 2024-04-25T01:24:32.625 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:24:32.690 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-immutable-object-cache 2024-04-25T01:24:32.991 INFO:teuthology.orchestra.run.smithi138.stdout:cephadm-17.2.7-902.gc61472c8.el9.noarch.rpm 225 kB/s | 82 kB 00:00 2024-04-25T01:24:32.991 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:32.992 INFO:teuthology.orchestra.run.smithi138.stdout:Total 223 kB/s | 82 kB 00:00 2024-04-25T01:24:32.992 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:24:32.997 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:24:32.997 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:24:32.999 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:24:32.999 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:24:33.078 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:24:33.223 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:00:58 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:33.346 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:33.347 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:33.347 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repo Size 2024-04-25T01:24:33.347 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-902.gc61472c8.el9 ceph 137 k 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 137 k 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 379 k 2024-04-25T01:24:33.348 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:24:33.498 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T01:24:33.560 INFO:teuthology.orchestra.run.smithi033.stdout:ceph-immutable-object-cache-17.2.7-902.gc61472c 650 kB/s | 137 kB 00:00 2024-04-25T01:24:33.560 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:33.561 INFO:teuthology.orchestra.run.smithi033.stdout:Total 644 kB/s | 137 kB 00:00 2024-04-25T01:24:33.561 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:24:33.569 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:24:33.569 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:24:33.593 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T01:24:33.629 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:24:33.629 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:24:33.772 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:24:33.852 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T01:24:33.878 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T01:24:33.878 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:33.878 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:24:33.879 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:33.920 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T01:24:34.178 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T01:24:34.178 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:34.178 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:24:34.178 INFO:teuthology.orchestra.run.smithi138.stdout: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:34.178 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:34.178 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:24:34.399 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-immutable-object-cache 2024-04-25T01:24:34.726 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T01:24:34.726 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:34.726 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:24:34.726 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:34.726 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:34.726 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:34.917 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:56 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:24:34.929 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr 2024-04-25T01:24:35.040 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:24:35.040 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-902.gc61472c8.el9 ceph 137 k 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 137 k 2024-04-25T01:24:35.041 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 379 k 2024-04-25T01:24:35.042 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:24:35.398 INFO:teuthology.orchestra.run.smithi138.stdout:ceph-immutable-object-cache-17.2.7-902.gc61472c 385 kB/s | 137 kB 00:00 2024-04-25T01:24:35.399 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:35.399 INFO:teuthology.orchestra.run.smithi138.stdout:Total 383 kB/s | 137 kB 00:00 2024-04-25T01:24:35.399 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:24:35.407 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:24:35.407 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:24:35.456 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:00 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:35.463 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:24:35.463 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:24:35.537 INFO:teuthology.orchestra.run.smithi033.stdout:Package ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:24:35.580 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:35.581 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:24:35.581 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:35.608 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:24:35.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-dashboard 2024-04-25T01:24:35.682 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T01:24:35.707 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T01:24:35.707 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:24:35.707 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T01:24:35.707 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:36.203 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:01 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:36.330 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:36.331 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-25T01:24:36.331 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:36.331 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:24:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 3.4 M 2024-04-25T01:24:36.331 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-25T01:24:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 24 k 2024-04-25T01:24:36.331 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 15 k 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout:Installing weak dependencies: 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout:Install 7 Packages 2024-04-25T01:24:36.332 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:36.333 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 3.7 M 2024-04-25T01:24:36.333 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 80 M 2024-04-25T01:24:36.333 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:24:36.555 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T01:24:36.556 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:36.556 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:24:36.556 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:24:36.556 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:36.556 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:24:36.622 INFO:teuthology.orchestra.run.smithi033.stdout:(1/7): ceph-grafana-dashboards-17.2.7-902.gc614 129 kB/s | 24 kB 00:00 2024-04-25T01:24:36.639 INFO:teuthology.orchestra.run.smithi033.stdout:(2/7): ceph-prometheus-alerts-17.2.7-902.gc6147 71 kB/s | 15 kB 00:00 2024-04-25T01:24:36.739 INFO:teuthology.orchestra.run.smithi033.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 77 kB/s | 9.0 kB 00:00 2024-04-25T01:24:36.741 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr 2024-04-25T01:24:36.814 INFO:teuthology.orchestra.run.smithi033.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 233 kB/s | 41 kB 00:00 2024-04-25T01:24:36.831 INFO:teuthology.orchestra.run.smithi033.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 334 kB/s | 31 kB 00:00 2024-04-25T01:24:36.973 INFO:teuthology.orchestra.run.smithi033.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.2 MB/s | 188 kB 00:00 2024-04-25T01:24:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:(7/7): ceph-mgr-dashboard-17.2.7-902.gc61472c8. 5.6 MB/s | 3.4 MB 00:00 2024-04-25T01:24:37.033 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:37.034 INFO:teuthology.orchestra.run.smithi033.stdout:Total 5.2 MB/s | 3.7 MB 00:00 2024-04-25T01:24:37.034 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:24:37.043 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:24:37.043 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:24:37.109 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:24:37.109 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:24:37.253 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:59 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:24:37.336 INFO:teuthology.orchestra.run.smithi138.stdout:Package ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:24:37.378 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:24:37.379 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:24:37.379 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:24:37.418 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:24:37.437 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-dashboard 2024-04-25T01:24:37.491 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-25T01:24:37.544 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-25T01:24:37.590 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-25T01:24:37.629 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-25T01:24:37.658 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 5/7 2024-04-25T01:24:37.944 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:00:59 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:24:38.065 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:24:38.066 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:38.066 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T01:24:38.066 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:38.066 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:24:38.066 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 3.4 M 2024-04-25T01:24:38.066 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T01:24:38.066 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 24 k 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 15 k 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout:Installing weak dependencies: 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout:Install 7 Packages 2024-04-25T01:24:38.067 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:38.068 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 3.7 M 2024-04-25T01:24:38.068 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 80 M 2024-04-25T01:24:38.068 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:24:38.344 INFO:teuthology.orchestra.run.smithi138.stdout:(1/7): ceph-prometheus-alerts-17.2.7-902.gc6147 77 kB/s | 15 kB 00:00 2024-04-25T01:24:38.445 INFO:teuthology.orchestra.run.smithi138.stdout:(2/7): ceph-grafana-dashboards-17.2.7-902.gc614 84 kB/s | 24 kB 00:00 2024-04-25T01:24:38.503 INFO:teuthology.orchestra.run.smithi138.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 57 kB/s | 9.0 kB 00:00 2024-04-25T01:24:38.578 INFO:teuthology.orchestra.run.smithi138.stdout:(4/7): ceph-mgr-dashboard-17.2.7-902.gc61472c8. 7.9 MB/s | 3.4 MB 00:00 2024-04-25T01:24:38.679 INFO:teuthology.orchestra.run.smithi138.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 175 kB/s | 31 kB 00:00 2024-04-25T01:24:38.710 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 6/7 2024-04-25T01:24:38.753 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-25T01:24:38.754 INFO:teuthology.orchestra.run.smithi138.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 132 kB/s | 41 kB 00:00 2024-04-25T01:24:38.987 INFO:teuthology.orchestra.run.smithi138.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 461 kB/s | 188 kB 00:00 2024-04-25T01:24:38.989 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:38.989 INFO:teuthology.orchestra.run.smithi138.stdout:Total 4.0 MB/s | 3.7 MB 00:00 2024-04-25T01:24:38.989 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:24:38.998 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:24:38.998 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:24:39.059 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:24:39.061 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:24:39.359 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:24:39.422 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-25T01:24:39.456 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-25T01:24:39.502 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-25T01:24:39.534 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-25T01:24:39.563 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 5/7 2024-04-25T01:24:39.874 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-25T01:24:39.874 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 1/7 2024-04-25T01:24:39.874 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2/7 2024-04-25T01:24:39.874 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 3/7 2024-04-25T01:24:39.874 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-25T01:24:39.874 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-25T01:24:39.874 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-25T01:24:40.133 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:40.134 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:40.335 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-25T01:24:40.555 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 6/7 2024-04-25T01:24:40.592 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-25T01:24:40.869 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:05 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:40.996 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:40.997 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================== 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================== 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 7.4 M 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:24:40.998 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:24:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:24:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:24:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:24:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:24:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:40.999 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:24:40.999 INFO:teuthology.orchestra.run.smithi033.stdout:====================================================================================== 2024-04-25T01:24:40.999 INFO:teuthology.orchestra.run.smithi033.stdout:Install 12 Packages 2024-04-25T01:24:40.999 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:41.000 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 38 M 2024-04-25T01:24:41.000 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 209 M 2024-04-25T01:24:41.000 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:24:41.556 INFO:teuthology.orchestra.run.smithi033.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-25T01:24:41.632 INFO:teuthology.orchestra.run.smithi033.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-25T01:24:41.657 INFO:teuthology.orchestra.run.smithi033.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-04-25T01:24:41.677 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-25T01:24:41.677 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 1/7 2024-04-25T01:24:41.677 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2/7 2024-04-25T01:24:41.677 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 3/7 2024-04-25T01:24:41.677 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-25T01:24:41.678 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-25T01:24:41.678 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-25T01:24:41.698 INFO:teuthology.orchestra.run.smithi033.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 438 kB/s | 18 kB 00:00 2024-04-25T01:24:41.741 INFO:teuthology.orchestra.run.smithi033.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 842 kB/s | 35 kB 00:00 2024-04-25T01:24:41.874 INFO:teuthology.orchestra.run.smithi033.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-04-25T01:24:41.916 INFO:teuthology.orchestra.run.smithi033.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-04-25T01:24:41.934 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-25T01:24:41.934 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:41.934 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:24:41.934 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:41.934 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:41.935 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:41.935 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-25T01:24:41.935 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-25T01:24:41.935 INFO:teuthology.orchestra.run.smithi138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T01:24:41.935 INFO:teuthology.orchestra.run.smithi138.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T01:24:41.935 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:41.935 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:24:42.008 INFO:teuthology.orchestra.run.smithi033.stdout:(8/12): ceph-mgr-diskprediction-local-17.2.7-90 12 MB/s | 7.4 MB 00:00 2024-04-25T01:24:42.083 INFO:teuthology.orchestra.run.smithi033.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 14 MB/s | 4.7 MB 00:00 2024-04-25T01:24:42.118 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-25T01:24:42.158 INFO:teuthology.orchestra.run.smithi033.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-04-25T01:24:42.234 INFO:teuthology.orchestra.run.smithi033.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.1 MB/s | 250 kB 00:00 2024-04-25T01:24:42.426 INFO:teuthology.orchestra.run.smithi033.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 48 MB/s | 16 MB 00:00 2024-04-25T01:24:42.426 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:42.427 INFO:teuthology.orchestra.run.smithi033.stdout:Total 27 MB/s | 38 MB 00:01 2024-04-25T01:24:42.625 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:04 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:24:42.748 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:24:42.749 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-04-25T01:24:42.749 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T01:24:42.749 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-04-25T01:24:42.749 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:24:42.749 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 7.4 M 2024-04-25T01:24:42.749 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:24:42.750 INFO:teuthology.orchestra.run.smithi138.stdout:====================================================================================== 2024-04-25T01:24:42.751 INFO:teuthology.orchestra.run.smithi138.stdout:Install 12 Packages 2024-04-25T01:24:42.751 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:42.751 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 38 M 2024-04-25T01:24:42.751 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 209 M 2024-04-25T01:24:42.751 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:24:42.787 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:24:42.814 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:24:42.814 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:24:43.247 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:24:43.248 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:24:43.620 INFO:teuthology.orchestra.run.smithi138.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 379 kB/s | 195 kB 00:00 2024-04-25T01:24:43.737 INFO:teuthology.orchestra.run.smithi138.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-04-25T01:24:43.855 INFO:teuthology.orchestra.run.smithi138.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-90 9.9 MB/s | 7.4 MB 00:00 2024-04-25T01:24:43.921 INFO:teuthology.orchestra.run.smithi138.stdout:(4/12): libgfortran-11.4.1-3.el9.x86_64.rpm 983 kB/s | 803 kB 00:00 2024-04-25T01:24:43.938 INFO:teuthology.orchestra.run.smithi138.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-04-25T01:24:44.005 INFO:teuthology.orchestra.run.smithi138.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 422 kB/s | 35 kB 00:00 2024-04-25T01:24:44.039 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:24:44.097 INFO:teuthology.orchestra.run.smithi138.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 2.6 MB/s | 244 kB 00:00 2024-04-25T01:24:44.114 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-25T01:24:44.157 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-25T01:24:44.264 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-25T01:24:44.309 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-25T01:24:44.447 INFO:teuthology.orchestra.run.smithi138.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T01:24:44.493 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-25T01:24:44.514 INFO:teuthology.orchestra.run.smithi138.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 8.2 MB/s | 4.7 MB 00:00 2024-04-25T01:24:44.524 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-25T01:24:44.539 INFO:teuthology.orchestra.run.smithi138.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.7 MB/s | 250 kB 00:00 2024-04-25T01:24:44.603 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-25T01:24:44.698 INFO:teuthology.orchestra.run.smithi138.stdout:(11/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 3.1 MB/s | 3.0 MB 00:00 2024-04-25T01:24:44.974 INFO:teuthology.orchestra.run.smithi138.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-25T01:24:44.974 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:44.975 INFO:teuthology.orchestra.run.smithi138.stdout:Total 17 MB/s | 38 MB 00:02 2024-04-25T01:24:45.285 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-25T01:24:45.325 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:24:45.351 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:24:45.351 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:24:45.408 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-25T01:24:45.754 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:24:45.755 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:24:46.499 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:24:46.561 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-25T01:24:46.605 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-25T01:24:46.706 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-25T01:24:46.748 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-25T01:24:46.753 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T01:24:46.925 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-25T01:24:46.955 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-25T01:24:47.032 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-25T01:24:47.672 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-25T01:24:47.787 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-25T01:24:47.845 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-25T01:24:47.873 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-25T01:24:49.019 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T01:24:50.049 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-25T01:24:50.072 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-25T01:24:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-25T01:24:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 1/12 2024-04-25T01:24:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-25T01:24:50.283 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-25T01:24:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-25T01:24:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-25T01:24:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-25T01:24:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-25T01:24:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-25T01:24:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-25T01:24:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T01:24:50.284 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-25T01:24:50.659 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-25T01:24:50.659 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:50.660 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:24:50.660 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:50.660 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:24:50.660 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:24:50.660 INFO:teuthology.orchestra.run.smithi033.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:24:50.660 INFO:teuthology.orchestra.run.smithi033.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:24:50.660 INFO:teuthology.orchestra.run.smithi033.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:24:50.660 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:24:50.661 INFO:teuthology.orchestra.run.smithi033.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:24:50.661 INFO:teuthology.orchestra.run.smithi033.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:24:50.661 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:24:50.661 INFO:teuthology.orchestra.run.smithi033.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:24:50.661 INFO:teuthology.orchestra.run.smithi033.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:24:50.661 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:50.661 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:51.060 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-rook 2024-04-25T01:24:51.589 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:16 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:51.716 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:51.717 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-25T01:24:51.717 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:51.717 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:24:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 47 k 2024-04-25T01:24:51.717 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-25T01:24:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:24:51.717 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:51.718 INFO:teuthology.orchestra.run.smithi033.stdout:Install 13 Packages 2024-04-25T01:24:51.719 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:51.719 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 2.2 M 2024-04-25T01:24:51.719 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 26 M 2024-04-25T01:24:51.719 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:24:52.030 INFO:teuthology.orchestra.run.smithi033.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 156 kB/s | 26 kB 00:00 2024-04-25T01:24:52.055 INFO:teuthology.orchestra.run.smithi033.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 101 kB/s | 19 kB 00:00 2024-04-25T01:24:52.172 INFO:teuthology.orchestra.run.smithi033.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-25T01:24:52.197 INFO:teuthology.orchestra.run.smithi033.stdout:(4/13): ceph-mgr-rook-17.2.7-902.gc61472c8.el9. 143 kB/s | 47 kB 00:00 2024-04-25T01:24:52.223 INFO:teuthology.orchestra.run.smithi033.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.3 MB/s | 222 kB 00:00 2024-04-25T01:24:52.323 INFO:teuthology.orchestra.run.smithi033.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.8 MB/s | 279 kB 00:00 2024-04-25T01:24:52.348 INFO:teuthology.orchestra.run.smithi033.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 357 kB/s | 54 kB 00:00 2024-04-25T01:24:52.482 INFO:teuthology.orchestra.run.smithi033.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 124 kB/s | 32 kB 00:00 2024-04-25T01:24:52.532 INFO:teuthology.orchestra.run.smithi033.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 72 kB/s | 15 kB 00:00 2024-04-25T01:24:52.616 INFO:teuthology.orchestra.run.smithi033.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 773 kB/s | 206 kB 00:00 2024-04-25T01:24:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-25T01:24:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 1/12 2024-04-25T01:24:52.690 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-25T01:24:52.691 INFO:teuthology.orchestra.run.smithi033.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 373 kB/s | 59 kB 00:00 2024-04-25T01:24:52.750 INFO:teuthology.orchestra.run.smithi033.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 3.8 MB/s | 1.0 MB 00:00 2024-04-25T01:24:52.774 INFO:teuthology.orchestra.run.smithi033.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 566 kB/s | 90 kB 00:00 2024-04-25T01:24:52.776 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:52.776 INFO:teuthology.orchestra.run.smithi033.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-04-25T01:24:52.844 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:24:52.860 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:24:52.860 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:24:52.953 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:24:52.953 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T01:24:53.135 INFO:teuthology.orchestra.run.smithi138.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T01:24:53.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T01:24:53.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T01:24:53.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T01:24:53.136 INFO:teuthology.orchestra.run.smithi138.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T01:24:53.136 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:53.136 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:24:53.539 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:24:53.584 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-rook 2024-04-25T01:24:53.721 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-25T01:24:53.778 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-25T01:24:53.842 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-25T01:24:53.878 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-25T01:24:53.920 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-25T01:24:54.017 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-25T01:24:54.106 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:16 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:24:54.129 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-25T01:24:54.180 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-25T01:24:54.228 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 47 k 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T01:24:54.230 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout:Install 13 Packages 2024-04-25T01:24:54.231 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:24:54.232 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.2 M 2024-04-25T01:24:54.232 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 26 M 2024-04-25T01:24:54.232 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:24:54.949 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-25T01:24:54.993 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-25T01:24:55.024 INFO:teuthology.orchestra.run.smithi138.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 71 kB/s | 19 kB 00:00 2024-04-25T01:24:55.026 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-25T01:24:55.049 INFO:teuthology.orchestra.run.smithi138.stdout:(2/13): ceph-mgr-rook-17.2.7-902.gc61472c8.el9. 161 kB/s | 47 kB 00:00 2024-04-25T01:24:55.073 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-25T01:24:55.075 INFO:teuthology.orchestra.run.smithi138.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 80 kB/s | 26 kB 00:00 2024-04-25T01:24:55.096 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-25T01:24:55.442 INFO:teuthology.orchestra.run.smithi138.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 433 kB/s | 159 kB 00:00 2024-04-25T01:24:55.467 INFO:teuthology.orchestra.run.smithi138.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 502 kB/s | 222 kB 00:00 2024-04-25T01:24:55.568 INFO:teuthology.orchestra.run.smithi138.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 428 kB/s | 54 kB 00:00 2024-04-25T01:24:55.676 INFO:teuthology.orchestra.run.smithi138.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 464 kB/s | 279 kB 00:00 2024-04-25T01:24:55.735 INFO:teuthology.orchestra.run.smithi138.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 120 kB/s | 32 kB 00:00 2024-04-25T01:24:55.801 INFO:teuthology.orchestra.run.smithi138.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 64 kB/s | 15 kB 00:00 2024-04-25T01:24:55.863 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-25T01:24:55.863 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 1/13 2024-04-25T01:24:55.863 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-25T01:24:55.863 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-25T01:24:55.863 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-25T01:24:55.863 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-25T01:24:55.863 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-25T01:24:55.864 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-25T01:24:55.864 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-25T01:24:55.864 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-25T01:24:55.864 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-25T01:24:55.864 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-25T01:24:55.864 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-25T01:24:55.977 INFO:teuthology.orchestra.run.smithi138.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 337 kB/s | 59 kB 00:00 2024-04-25T01:24:56.144 INFO:teuthology.orchestra.run.smithi138.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 538 kB/s | 90 kB 00:00 2024-04-25T01:24:56.169 INFO:teuthology.orchestra.run.smithi138.stdout:(12/13): python3-google-auth-2.29.0-1.el9.noarc 417 kB/s | 206 kB 00:00 2024-04-25T01:24:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-25T01:24:56.173 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:24:56.174 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:56.175 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:24:56.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-mgr-cephadm 2024-04-25T01:24:56.778 INFO:teuthology.orchestra.run.smithi138.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.0 MB/s | 1.0 MB 00:01 2024-04-25T01:24:56.778 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:56.778 INFO:teuthology.orchestra.run.smithi138.stdout:Total 884 kB/s | 2.2 MB 00:02 2024-04-25T01:24:56.847 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:24:56.862 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:24:56.862 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:24:56.939 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:21 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:24:56.947 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:24:56.947 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:24:57.064 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:24:57.064 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 123 k 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout:Install 6 Packages 2024-04-25T01:24:57.065 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:24:57.066 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 7.0 M 2024-04-25T01:24:57.066 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 33 M 2024-04-25T01:24:57.066 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:24:57.514 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:24:57.691 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-25T01:24:57.693 INFO:teuthology.orchestra.run.smithi033.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-04-25T01:24:57.746 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-25T01:24:57.809 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-25T01:24:57.846 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-25T01:24:57.852 INFO:teuthology.orchestra.run.smithi033.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-25T01:24:57.888 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-25T01:24:57.961 INFO:teuthology.orchestra.run.smithi033.stdout:(3/6): ceph-mgr-cephadm-17.2.7-902.gc61472c8.el 269 kB/s | 123 kB 00:00 2024-04-25T01:24:57.982 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-25T01:24:58.084 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-25T01:24:58.132 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-25T01:24:58.403 INFO:teuthology.orchestra.run.smithi033.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 105 kB/s | 58 kB 00:00 2024-04-25T01:24:58.511 INFO:teuthology.orchestra.run.smithi033.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 93 kB/s | 51 kB 00:00 2024-04-25T01:24:58.812 INFO:teuthology.orchestra.run.smithi033.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 491 kB/s | 548 kB 00:01 2024-04-25T01:24:58.814 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:24:58.814 INFO:teuthology.orchestra.run.smithi033.stdout:Total 4.0 MB/s | 7.0 MB 00:01 2024-04-25T01:24:58.851 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-25T01:24:58.890 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:24:58.903 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:24:58.903 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:24:58.904 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-25T01:24:58.938 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-25T01:24:58.985 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-25T01:24:59.006 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:24:59.007 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:24:59.009 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-25T01:24:59.316 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:24:59.493 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-25T01:24:59.548 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-25T01:24:59.809 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-25T01:24:59.809 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 1/13 2024-04-25T01:24:59.809 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-25T01:24:59.809 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-25T01:24:59.809 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-25T01:24:59.809 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-25T01:24:59.809 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-25T01:24:59.809 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-25T01:24:59.809 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-25T01:24:59.810 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-25T01:24:59.810 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-25T01:24:59.810 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-25T01:24:59.810 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-25T01:25:00.102 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T01:25:00.105 INFO:teuthology.orchestra.run.smithi138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T01:25:00.106 INFO:teuthology.orchestra.run.smithi138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T01:25:00.106 INFO:teuthology.orchestra.run.smithi138.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T01:25:00.106 INFO:teuthology.orchestra.run.smithi138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T01:25:00.106 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:00.106 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:00.194 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-25T01:25:00.265 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-25T01:25:00.300 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-25T01:25:00.328 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-mgr-cephadm 2024-04-25T01:25:00.839 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:22 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:00.959 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:00.960 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:00.960 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T01:25:00.960 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:00.960 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:25:00.960 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 123 k 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout:Install 6 Packages 2024-04-25T01:25:00.961 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:00.962 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 7.0 M 2024-04-25T01:25:00.962 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 33 M 2024-04-25T01:25:00.962 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-25T01:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/6 2024-04-25T01:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-25T01:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-25T01:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-25T01:25:00.981 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-25T01:25:01.238 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-25T01:25:01.238 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:01.238 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:25:01.238 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:25:01.238 INFO:teuthology.orchestra.run.smithi033.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:25:01.238 INFO:teuthology.orchestra.run.smithi033.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:25:01.238 INFO:teuthology.orchestra.run.smithi033.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:25:01.239 INFO:teuthology.orchestra.run.smithi033.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:25:01.239 INFO:teuthology.orchestra.run.smithi033.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:25:01.239 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:01.239 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:01.373 INFO:teuthology.orchestra.run.smithi138.stdout:(1/6): ceph-mgr-cephadm-17.2.7-902.gc61472c8.el 612 kB/s | 123 kB 00:00 2024-04-25T01:25:01.492 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install ceph-fuse 2024-04-25T01:25:01.565 INFO:teuthology.orchestra.run.smithi138.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.8 MB/s | 548 kB 00:00 2024-04-25T01:25:01.615 INFO:teuthology.orchestra.run.smithi138.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T01:25:01.674 INFO:teuthology.orchestra.run.smithi138.stdout:(4/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 494 kB/s | 248 kB 00:00 2024-04-25T01:25:01.707 INFO:teuthology.orchestra.run.smithi138.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 555 kB/s | 51 kB 00:00 2024-04-25T01:25:02.019 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:27 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:02.142 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:02.143 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repository Size 2024-04-25T01:25:02.143 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:02.143 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:25:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 814 k 2024-04-25T01:25:02.143 INFO:teuthology.orchestra.run.smithi033.stdout:Installing dependencies: 2024-04-25T01:25:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:25:02.143 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:02.144 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:25:02.144 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:02.144 INFO:teuthology.orchestra.run.smithi033.stdout:Install 2 Packages 2024-04-25T01:25:02.144 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:02.144 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 894 k 2024-04-25T01:25:02.144 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 2.6 M 2024-04-25T01:25:02.145 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:25:02.355 INFO:teuthology.orchestra.run.smithi033.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 555 kB/s | 80 kB 00:00 2024-04-25T01:25:02.641 INFO:teuthology.orchestra.run.smithi138.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-04-25T01:25:02.642 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:02.642 INFO:teuthology.orchestra.run.smithi138.stdout:Total 4.1 MB/s | 7.0 MB 00:01 2024-04-25T01:25:02.671 INFO:teuthology.orchestra.run.smithi033.stdout:(2/2): ceph-fuse-17.2.7-902.gc61472c8.el9.x86_6 1.7 MB/s | 814 kB 00:00 2024-04-25T01:25:02.672 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:02.672 INFO:teuthology.orchestra.run.smithi033.stdout:Total 1.7 MB/s | 894 kB 00:00 2024-04-25T01:25:02.684 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:25:02.707 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:25:02.707 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:25:02.713 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:25:02.724 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:25:02.724 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:25:02.788 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:25:02.788 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:25:02.819 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:25:02.819 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:25:02.979 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:25:03.113 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-25T01:25:03.124 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:25:03.179 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-25T01:25:03.289 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-25T01:25:03.343 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-25T01:25:03.858 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-25T01:25:03.888 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-25T01:25:03.889 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/2 2024-04-25T01:25:03.938 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-25T01:25:04.001 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-25T01:25:04.030 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-25T01:25:04.121 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-25T01:25:04.121 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:04.121 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:25:04.122 INFO:teuthology.orchestra.run.smithi033.stdout: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-25T01:25:04.122 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:04.122 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:04.352 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install librados-devel 2024-04-25T01:25:04.874 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-25T01:25:04.874 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/6 2024-04-25T01:25:04.874 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-25T01:25:04.874 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-25T01:25:04.875 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-25T01:25:04.875 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-25T01:25:04.886 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:29 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:05.010 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:05.011 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:05.011 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repo Size 2024-04-25T01:25:05.011 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:05.011 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:25:05.012 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 126 k 2024-04-25T01:25:05.012 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:05.012 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:25:05.012 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:05.012 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-04-25T01:25:05.012 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:05.012 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 126 k 2024-04-25T01:25:05.012 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 448 k 2024-04-25T01:25:05.012 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:25:05.130 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-25T01:25:05.130 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:05.130 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:25:05.130 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T01:25:05.130 INFO:teuthology.orchestra.run.smithi138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T01:25:05.130 INFO:teuthology.orchestra.run.smithi138.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T01:25:05.130 INFO:teuthology.orchestra.run.smithi138.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T01:25:05.131 INFO:teuthology.orchestra.run.smithi138.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T01:25:05.131 INFO:teuthology.orchestra.run.smithi138.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T01:25:05.131 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:05.131 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:05.237 INFO:teuthology.orchestra.run.smithi033.stdout:librados-devel-17.2.7-902.gc61472c8.el9.x86_64. 562 kB/s | 126 kB 00:00 2024-04-25T01:25:05.238 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:05.238 INFO:teuthology.orchestra.run.smithi033.stdout:Total 558 kB/s | 126 kB 00:00 2024-04-25T01:25:05.238 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:25:05.244 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:25:05.244 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:25:05.310 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:25:05.311 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:25:05.375 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install ceph-fuse 2024-04-25T01:25:05.473 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:25:05.603 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:05.881 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:27 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:05.996 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:06.000 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repository Size 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 814 k 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout:Installing dependencies: 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout:Install 2 Packages 2024-04-25T01:25:06.001 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:06.002 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 894 k 2024-04-25T01:25:06.002 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 2.6 M 2024-04-25T01:25:06.002 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:25:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:06.261 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:25:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:06.261 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:06.261 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:06.276 INFO:teuthology.orchestra.run.smithi138.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 645 kB/s | 80 kB 00:00 2024-04-25T01:25:06.451 INFO:teuthology.orchestra.run.smithi138.stdout:(2/2): ceph-fuse-17.2.7-902.gc61472c8.el9.x86_6 2.7 MB/s | 814 kB 00:00 2024-04-25T01:25:06.452 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:06.452 INFO:teuthology.orchestra.run.smithi138.stdout:Total 1.9 MB/s | 894 kB 00:00 2024-04-25T01:25:06.463 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:25:06.486 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:25:06.486 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:25:06.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install libcephfs2 2024-04-25T01:25:06.561 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:25:06.561 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:25:06.750 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:25:06.896 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-25T01:25:06.957 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-25T01:25:07.032 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:32 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:07.113 INFO:teuthology.orchestra.run.smithi033.stdout:Package libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:07.157 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:07.158 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:25:07.159 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:07.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install libcephfs-devel 2024-04-25T01:25:07.626 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-25T01:25:07.626 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/2 2024-04-25T01:25:07.749 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:32 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:07.867 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-25T01:25:07.868 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:07.868 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:25:07.868 INFO:teuthology.orchestra.run.smithi138.stdout: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-25T01:25:07.868 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:07.868 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:07.876 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:07.877 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:07.877 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repo Size 2024-04-25T01:25:07.877 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:07.877 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:25:07.878 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-25T01:25:07.878 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:07.878 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:25:07.878 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:07.878 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-04-25T01:25:07.878 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:07.878 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 25 k 2024-04-25T01:25:07.878 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 110 k 2024-04-25T01:25:07.878 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:25:08.134 INFO:teuthology.orchestra.run.smithi033.stdout:libcephfs-devel-17.2.7-902.gc61472c8.el9.x86_64 98 kB/s | 25 kB 00:00 2024-04-25T01:25:08.135 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:08.135 INFO:teuthology.orchestra.run.smithi033.stdout:Total 97 kB/s | 25 kB 00:00 2024-04-25T01:25:08.135 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:25:08.137 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados-devel 2024-04-25T01:25:08.138 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:25:08.138 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:25:08.140 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:25:08.140 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:25:08.222 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:25:08.322 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:08.650 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:30 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:08.686 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:08.771 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:08.772 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:08.772 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-25T01:25:08.772 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:08.772 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:25:08.772 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 126 k 2024-04-25T01:25:08.772 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:08.772 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:25:08.772 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:08.772 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-25T01:25:08.773 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:08.773 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 126 k 2024-04-25T01:25:08.773 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 448 k 2024-04-25T01:25:08.773 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:25:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:08.894 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:25:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:08.894 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:08.894 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:09.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install librados2 2024-04-25T01:25:09.139 INFO:teuthology.orchestra.run.smithi138.stdout:librados-devel-17.2.7-902.gc61472c8.el9.x86_64. 345 kB/s | 126 kB 00:00 2024-04-25T01:25:09.140 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:09.140 INFO:teuthology.orchestra.run.smithi138.stdout:Total 343 kB/s | 126 kB 00:00 2024-04-25T01:25:09.140 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:25:09.146 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:25:09.146 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:25:09.208 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:25:09.208 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:25:09.361 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:25:09.511 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:09.600 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:34 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:09.682 INFO:teuthology.orchestra.run.smithi033.stdout:Package librados2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:09.726 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:09.727 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:25:09.727 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:09.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install librbd1 2024-04-25T01:25:09.864 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:10.120 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:10.120 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:10.120 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:25:10.120 INFO:teuthology.orchestra.run.smithi138.stdout: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:10.120 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:10.120 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:10.331 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:35 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:10.367 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs2 2024-04-25T01:25:10.413 INFO:teuthology.orchestra.run.smithi033.stdout:Package librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:10.455 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:10.457 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:25:10.457 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:10.521 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install python3-rados 2024-04-25T01:25:10.886 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:32 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:10.965 INFO:teuthology.orchestra.run.smithi138.stdout:Package libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:11.006 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:11.007 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:25:11.008 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:11.053 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:36 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:11.087 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install libcephfs-devel 2024-04-25T01:25:11.134 INFO:teuthology.orchestra.run.smithi033.stdout:Package python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:11.177 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:11.178 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:25:11.178 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:11.241 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install python3-rgw 2024-04-25T01:25:11.606 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:33 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:11.726 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:11.726 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:11.726 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-25T01:25:11.726 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:11.726 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:25:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-25T01:25:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:11.727 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:25:11.727 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:11.727 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-25T01:25:11.727 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:11.727 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 25 k 2024-04-25T01:25:11.727 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 110 k 2024-04-25T01:25:11.727 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:25:11.767 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:36 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:11.848 INFO:teuthology.orchestra.run.smithi033.stdout:Package python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:11.884 INFO:teuthology.orchestra.run.smithi138.stdout:libcephfs-devel-17.2.7-902.gc61472c8.el9.x86_64 159 kB/s | 25 kB 00:00 2024-04-25T01:25:11.885 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:11.885 INFO:teuthology.orchestra.run.smithi138.stdout:Total 157 kB/s | 25 kB 00:00 2024-04-25T01:25:11.886 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:25:11.889 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:25:11.889 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:25:11.890 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:25:11.890 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:25:11.891 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:11.893 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:25:11.893 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:11.962 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install python3-cephfs 2024-04-25T01:25:11.975 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:25:12.071 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:12.399 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:12.485 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:37 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:12.566 INFO:teuthology.orchestra.run.smithi033.stdout:Package python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:12.610 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:25:12.610 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:12.611 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:12.611 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:12.611 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:25:12.611 INFO:teuthology.orchestra.run.smithi138.stdout: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:12.611 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:12.611 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:12.678 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install python3-rbd 2024-04-25T01:25:12.782 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librados2 2024-04-25T01:25:13.216 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:38 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:13.289 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:35 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:13.299 INFO:teuthology.orchestra.run.smithi033.stdout:Package python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:13.342 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:13.342 INFO:teuthology.orchestra.run.smithi033.stdout:Nothing to do. 2024-04-25T01:25:13.343 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:13.367 INFO:teuthology.orchestra.run.smithi138.stdout:Package librados2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:13.405 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install rbd-fuse 2024-04-25T01:25:13.407 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:13.408 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:25:13.408 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:13.472 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install librbd1 2024-04-25T01:25:13.935 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:38 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:13.984 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:36 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:14.058 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:14.059 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-04-25T01:25:14.059 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:14.059 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:25:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 82 k 2024-04-25T01:25:14.059 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:14.060 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:25:14.060 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:14.060 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-04-25T01:25:14.060 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:14.060 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 82 k 2024-04-25T01:25:14.060 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 222 k 2024-04-25T01:25:14.060 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:25:14.064 INFO:teuthology.orchestra.run.smithi138.stdout:Package librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:14.105 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:14.106 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:25:14.106 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:14.168 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rados 2024-04-25T01:25:14.278 INFO:teuthology.orchestra.run.smithi033.stdout:rbd-fuse-17.2.7-902.gc61472c8.el9.x86_64.rpm 377 kB/s | 82 kB 00:00 2024-04-25T01:25:14.278 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:14.279 INFO:teuthology.orchestra.run.smithi033.stdout:Total 374 kB/s | 82 kB 00:00 2024-04-25T01:25:14.279 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:25:14.286 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:25:14.286 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:25:14.352 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:25:14.352 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:25:14.498 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:25:14.618 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:14.675 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:36 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:14.755 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:14.797 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:14.797 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:25:14.798 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:14.859 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rgw 2024-04-25T01:25:14.946 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:15.210 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:25:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:15.210 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:15.211 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:15.369 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:37 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:15.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install rbd-mirror 2024-04-25T01:25:15.448 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:15.489 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:15.490 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:25:15.490 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:15.559 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-cephfs 2024-04-25T01:25:15.971 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:40 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:16.068 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:38 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:16.096 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:16.097 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:16.097 INFO:teuthology.orchestra.run.smithi033.stdout: Package Arch Version Repo Size 2024-04-25T01:25:16.097 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:16.097 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:25:16.097 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-25T01:25:16.097 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:16.097 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:25:16.097 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:16.098 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-04-25T01:25:16.098 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:16.098 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 2.9 M 2024-04-25T01:25:16.098 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 12 M 2024-04-25T01:25:16.098 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:25:16.147 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:16.188 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:16.189 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:25:16.189 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:16.251 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install python3-rbd 2024-04-25T01:25:16.755 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:38 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:16.833 INFO:teuthology.orchestra.run.smithi138.stdout:Package python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T01:25:16.874 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:16.875 INFO:teuthology.orchestra.run.smithi138.stdout:Nothing to do. 2024-04-25T01:25:16.875 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:16.938 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-fuse 2024-04-25T01:25:17.232 INFO:teuthology.orchestra.run.smithi033.stdout:rbd-mirror-17.2.7-902.gc61472c8.el9.x86_64.rpm 2.6 MB/s | 2.9 MB 00:01 2024-04-25T01:25:17.232 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:17.233 INFO:teuthology.orchestra.run.smithi033.stdout:Total 2.6 MB/s | 2.9 MB 00:01 2024-04-25T01:25:17.233 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:25:17.245 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:25:17.245 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:25:17.332 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:25:17.333 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:25:17.440 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:39 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:17.559 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 82 k 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:17.560 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 82 k 2024-04-25T01:25:17.561 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 222 k 2024-04-25T01:25:17.561 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:25:17.762 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-fuse-17.2.7-902.gc61472c8.el9.x86_64.rpm 407 kB/s | 82 kB 00:00 2024-04-25T01:25:17.763 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:17.763 INFO:teuthology.orchestra.run.smithi138.stdout:Total 404 kB/s | 82 kB 00:00 2024-04-25T01:25:17.763 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:25:17.767 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:25:17.769 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:25:17.770 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:25:17.831 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:25:17.831 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:25:17.832 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:17.859 INFO:teuthology.orchestra.run.smithi033.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:25:17.859 INFO:teuthology.orchestra.run.smithi033.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:25:17.859 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:25:17.859 INFO:teuthology.orchestra.run.smithi033.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:25:17.859 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:17.971 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:25:18.081 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:18.437 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:18.712 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:18.713 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:18.713 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:25:18.713 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:18.713 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:18.713 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:18.952 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:18.952 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:18.952 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:25:18.952 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:18.952 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:18.952 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:18.963 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-mirror 2024-04-25T01:25:19.212 DEBUG:teuthology.orchestra.run.smithi033:> sudo yum -y install rbd-nbd 2024-04-25T01:25:19.481 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:41 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:19.602 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:19.603 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:19.603 INFO:teuthology.orchestra.run.smithi138.stdout: Package Arch Version Repo Size 2024-04-25T01:25:19.603 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:19.603 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:25:19.603 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-25T01:25:19.603 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:19.603 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:25:19.603 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:19.604 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-25T01:25:19.604 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:19.604 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 2.9 M 2024-04-25T01:25:19.604 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 12 M 2024-04-25T01:25:19.604 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:25:19.749 INFO:teuthology.orchestra.run.smithi033.stdout:Last metadata expiration check: 0:01:44 ago on Thu 25 Apr 2024 01:23:35 AM UTC. 2024-04-25T01:25:19.874 INFO:teuthology.orchestra.run.smithi033.stdout:Dependencies resolved. 2024-04-25T01:25:19.875 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:19.875 INFO:teuthology.orchestra.run.smithi033.stdout: Package Architecture Version Repository Size 2024-04-25T01:25:19.875 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:19.875 INFO:teuthology.orchestra.run.smithi033.stdout:Installing: 2024-04-25T01:25:19.875 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 165 k 2024-04-25T01:25:19.875 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:19.875 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction Summary 2024-04-25T01:25:19.876 INFO:teuthology.orchestra.run.smithi033.stdout:================================================================================ 2024-04-25T01:25:19.876 INFO:teuthology.orchestra.run.smithi033.stdout:Install 1 Package 2024-04-25T01:25:19.876 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:19.876 INFO:teuthology.orchestra.run.smithi033.stdout:Total download size: 165 k 2024-04-25T01:25:19.876 INFO:teuthology.orchestra.run.smithi033.stdout:Installed size: 477 k 2024-04-25T01:25:19.876 INFO:teuthology.orchestra.run.smithi033.stdout:Downloading Packages: 2024-04-25T01:25:20.094 INFO:teuthology.orchestra.run.smithi033.stdout:rbd-nbd-17.2.7-902.gc61472c8.el9.x86_64.rpm 758 kB/s | 165 kB 00:00 2024-04-25T01:25:20.094 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:20.094 INFO:teuthology.orchestra.run.smithi033.stdout:Total 752 kB/s | 165 kB 00:00 2024-04-25T01:25:20.095 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction check 2024-04-25T01:25:20.103 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction check succeeded. 2024-04-25T01:25:20.103 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction test 2024-04-25T01:25:20.170 INFO:teuthology.orchestra.run.smithi033.stdout:Transaction test succeeded. 2024-04-25T01:25:20.170 INFO:teuthology.orchestra.run.smithi033.stdout:Running transaction 2024-04-25T01:25:20.324 INFO:teuthology.orchestra.run.smithi033.stdout: Preparing : 1/1 2024-04-25T01:25:20.434 INFO:teuthology.orchestra.run.smithi033.stdout: Installing : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:20.557 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-mirror-17.2.7-902.gc61472c8.el9.x86_64.rpm 3.1 MB/s | 2.9 MB 00:00 2024-04-25T01:25:20.557 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:20.558 INFO:teuthology.orchestra.run.smithi138.stdout:Total 3.1 MB/s | 2.9 MB 00:00 2024-04-25T01:25:20.558 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:25:20.567 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:25:20.567 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:25:20.650 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:25:20.651 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:25:20.822 INFO:teuthology.orchestra.run.smithi033.stdout: Running scriptlet: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:21.074 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:25:21.101 INFO:teuthology.orchestra.run.smithi033.stdout: Verifying : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:21.101 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:21.101 INFO:teuthology.orchestra.run.smithi033.stdout:Installed: 2024-04-25T01:25:21.101 INFO:teuthology.orchestra.run.smithi033.stdout: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:21.102 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:25:21.102 INFO:teuthology.orchestra.run.smithi033.stdout:Complete! 2024-04-25T01:25:21.241 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:21.267 INFO:teuthology.orchestra.run.smithi138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T01:25:21.267 INFO:teuthology.orchestra.run.smithi138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T01:25:21.267 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:25:21.267 INFO:teuthology.orchestra.run.smithi138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T01:25:21.267 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:21.365 DEBUG:teuthology.parallel:result is None 2024-04-25T01:25:22.157 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:22.157 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:22.157 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:25:22.157 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:22.157 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:22.157 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:22.393 DEBUG:teuthology.orchestra.run.smithi138:> sudo yum -y install rbd-nbd 2024-04-25T01:25:22.913 INFO:teuthology.orchestra.run.smithi138.stdout:Last metadata expiration check: 0:01:44 ago on Thu 25 Apr 2024 01:23:38 AM UTC. 2024-04-25T01:25:23.038 INFO:teuthology.orchestra.run.smithi138.stdout:Dependencies resolved. 2024-04-25T01:25:23.039 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: Package Architecture Version Repository Size 2024-04-25T01:25:23.039 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:23.039 INFO:teuthology.orchestra.run.smithi138.stdout:Installing: 2024-04-25T01:25:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 165 k 2024-04-25T01:25:23.039 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:23.039 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction Summary 2024-04-25T01:25:23.040 INFO:teuthology.orchestra.run.smithi138.stdout:================================================================================ 2024-04-25T01:25:23.040 INFO:teuthology.orchestra.run.smithi138.stdout:Install 1 Package 2024-04-25T01:25:23.040 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:23.040 INFO:teuthology.orchestra.run.smithi138.stdout:Total download size: 165 k 2024-04-25T01:25:23.040 INFO:teuthology.orchestra.run.smithi138.stdout:Installed size: 477 k 2024-04-25T01:25:23.040 INFO:teuthology.orchestra.run.smithi138.stdout:Downloading Packages: 2024-04-25T01:25:23.269 INFO:teuthology.orchestra.run.smithi138.stdout:rbd-nbd-17.2.7-902.gc61472c8.el9.x86_64.rpm 721 kB/s | 165 kB 00:00 2024-04-25T01:25:23.270 INFO:teuthology.orchestra.run.smithi138.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:23.270 INFO:teuthology.orchestra.run.smithi138.stdout:Total 716 kB/s | 165 kB 00:00 2024-04-25T01:25:23.270 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction check 2024-04-25T01:25:23.277 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction check succeeded. 2024-04-25T01:25:23.277 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction test 2024-04-25T01:25:23.339 INFO:teuthology.orchestra.run.smithi138.stdout:Transaction test succeeded. 2024-04-25T01:25:23.340 INFO:teuthology.orchestra.run.smithi138.stdout:Running transaction 2024-04-25T01:25:23.488 INFO:teuthology.orchestra.run.smithi138.stdout: Preparing : 1/1 2024-04-25T01:25:23.589 INFO:teuthology.orchestra.run.smithi138.stdout: Installing : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:24.007 INFO:teuthology.orchestra.run.smithi138.stdout: Running scriptlet: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: Verifying : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T01:25:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:24.232 INFO:teuthology.orchestra.run.smithi138.stdout:Installed: 2024-04-25T01:25:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T01:25:24.232 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:25:24.232 INFO:teuthology.orchestra.run.smithi138.stdout:Complete! 2024-04-25T01:25:24.455 DEBUG:teuthology.parallel:result is None 2024-04-25T01:25:24.456 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T01:25:24.456 INFO:teuthology.packaging:ref: None 2024-04-25T01:25:24.456 INFO:teuthology.packaging:tag: None 2024-04-25T01:25:24.456 INFO:teuthology.packaging:branch: quincy 2024-04-25T01:25:24.456 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:25:24.456 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-25T01:25:24.749 DEBUG:teuthology.orchestra.run.smithi033:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:25:24.774 INFO:teuthology.orchestra.run.smithi033.stdout:17.2.7-902.gc61472c8.el9 2024-04-25T01:25:24.775 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-902.gc61472c8.el9 2024-04-25T01:25:24.775 INFO:teuthology.task.install:The correct ceph version 17.2.7-902.gc61472c8 is installed. 2024-04-25T01:25:24.777 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T01:25:24.777 INFO:teuthology.packaging:ref: None 2024-04-25T01:25:24.777 INFO:teuthology.packaging:tag: None 2024-04-25T01:25:24.777 INFO:teuthology.packaging:branch: quincy 2024-04-25T01:25:24.777 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:25:24.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-25T01:25:24.981 DEBUG:teuthology.orchestra.run.smithi138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T01:25:25.004 INFO:teuthology.orchestra.run.smithi138.stdout:17.2.7-902.gc61472c8.el9 2024-04-25T01:25:25.004 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-902.gc61472c8.el9 2024-04-25T01:25:25.004 INFO:teuthology.task.install:The correct ceph version 17.2.7-902.gc61472c8 is installed. 2024-04-25T01:25:25.006 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T01:25:25.006 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:25:25.007 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:25:25.037 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:25:25.037 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T01:25:25.069 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T01:25:25.070 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:25:25.070 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:25:25.106 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:25:25.172 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:25:25.173 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T01:25:25.197 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T01:25:25.261 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T01:25:25.261 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:25:25.261 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:25:25.286 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:25:25.351 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:25:25.351 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T01:25:25.375 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T01:25:25.439 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T01:25:25.439 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:25:25.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:25:25.463 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:25:25.528 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:25:25.528 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T01:25:25.552 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T01:25:25.617 INFO:teuthology.run_tasks:Running task print... 2024-04-25T01:25:25.626 INFO:teuthology.task.print:**** done install task... 2024-04-25T01:25:25.626 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T01:25:25.738 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T01:25:25.738 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-04-25T01:25:25.739 INFO:tasks.cephadm:Cluster fsid is b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:25:25.739 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T01:25:25.739 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-25T01:25:25.739 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi033': '172.21.15.33', 'mon.smithi138': '172.21.15.138'} 2024-04-25T01:25:25.739 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T01:25:25.739 DEBUG:teuthology.orchestra.run.smithi033:> sudo hostname $(hostname -s) 2024-04-25T01:25:25.765 DEBUG:teuthology.orchestra.run.smithi138:> sudo hostname $(hostname -s) 2024-04-25T01:25:25.789 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-04-25T01:25:25.789 DEBUG:teuthology.orchestra.run.smithi033:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T01:25:26.124 INFO:teuthology.orchestra.run.smithi033.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 25 01:25 /home/ubuntu/cephtest/cephadm 2024-04-25T01:25:26.125 DEBUG:teuthology.orchestra.run.smithi138:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T01:25:26.399 INFO:teuthology.orchestra.run.smithi138.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 25 01:25 /home/ubuntu/cephtest/cephadm 2024-04-25T01:25:26.400 DEBUG:teuthology.orchestra.run.smithi033:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T01:25:26.415 DEBUG:teuthology.orchestra.run.smithi138:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T01:25:26.441 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-04-25T01:25:26.441 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-25T01:25:26.458 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-25T01:25:28.015 INFO:teuthology.orchestra.run.smithi033.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-25T01:25:28.114 INFO:teuthology.orchestra.run.smithi138.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-25T01:25:55.388 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T01:25:55.389 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-25T01:25:55.389 INFO:teuthology.orchestra.run.smithi138.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-25T01:25:55.389 INFO:teuthology.orchestra.run.smithi138.stdout: "repo_digests": [ 2024-04-25T01:25:55.389 INFO:teuthology.orchestra.run.smithi138.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-25T01:25:55.389 INFO:teuthology.orchestra.run.smithi138.stdout: ] 2024-04-25T01:25:55.389 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T01:25:56.653 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:25:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-25T01:25:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-25T01:25:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "repo_digests": [ 2024-04-25T01:25:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-25T01:25:56.653 INFO:teuthology.orchestra.run.smithi033.stdout: ] 2024-04-25T01:25:56.653 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:25:56.672 DEBUG:teuthology.orchestra.run.smithi033:> sudo mkdir -p /etc/ceph 2024-04-25T01:25:56.715 DEBUG:teuthology.orchestra.run.smithi138:> sudo mkdir -p /etc/ceph 2024-04-25T01:25:56.746 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 777 /etc/ceph 2024-04-25T01:25:56.779 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 777 /etc/ceph 2024-04-25T01:25:56.814 INFO:tasks.cephadm:Writing seed config... 2024-04-25T01:25:56.815 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-25T01:25:56.815 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-25T01:25:56.815 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T01:25:56.815 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T01:25:56.815 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T01:25:56.815 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T01:25:56.815 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T01:25:56.816 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T01:25:56.817 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T01:25:56.818 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T01:25:56.818 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T01:25:56.818 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:25:56.818 DEBUG:teuthology.orchestra.run.smithi033:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T01:25:56.835 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b17b7f0e-02a2-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T01:25:56.835 DEBUG:teuthology.orchestra.run.smithi033:mon.smithi033> sudo journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033.service 2024-04-25T01:25:56.877 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T01:25:56.877 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.33 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:25:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:-------------------------------------------------------------------------------- 2024-04-25T01:25:57.086 INFO:teuthology.orchestra.run.smithi033.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'b17b7f0e-02a2-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.33', '--skip-admin-label'] 2024-04-25T01:25:57.118 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T01:25:57.119 INFO:teuthology.orchestra.run.smithi033.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T01:25:57.119 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying podman|docker is present... 2024-04-25T01:25:57.143 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T01:25:57.143 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying lvm2 is present... 2024-04-25T01:25:57.143 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying time synchronization is in place... 2024-04-25T01:25:57.151 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T01:25:57.151 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T01:25:57.158 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T01:25:57.158 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-25T01:25:57.165 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-04-25T01:25:57.171 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-04-25T01:25:57.171 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-04-25T01:25:57.171 INFO:teuthology.orchestra.run.smithi033.stdout:Repeating the final host check... 2024-04-25T01:25:57.195 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T01:25:57.195 INFO:teuthology.orchestra.run.smithi033.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T01:25:57.195 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl is present 2024-04-25T01:25:57.195 INFO:teuthology.orchestra.run.smithi033.stdout:lvcreate is present 2024-04-25T01:25:57.202 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T01:25:57.202 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T01:25:57.208 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T01:25:57.208 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-25T01:25:57.215 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout enabled 2024-04-25T01:25:57.222 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout active 2024-04-25T01:25:57.222 INFO:teuthology.orchestra.run.smithi033.stdout:Unit chronyd.service is enabled and running 2024-04-25T01:25:57.222 INFO:teuthology.orchestra.run.smithi033.stdout:Host looks OK 2024-04-25T01:25:57.222 INFO:teuthology.orchestra.run.smithi033.stdout:Cluster fsid: b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:25:57.222 INFO:teuthology.orchestra.run.smithi033.stdout:Acquiring lock 140176169259936 on /run/cephadm/b17b7f0e-02a2-11ef-bc93-c7b262605968.lock 2024-04-25T01:25:57.222 INFO:teuthology.orchestra.run.smithi033.stdout:Lock 140176169259936 acquired on /run/cephadm/b17b7f0e-02a2-11ef-bc93-c7b262605968.lock 2024-04-25T01:25:57.223 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 3300 ... 2024-04-25T01:25:57.223 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying IP 172.21.15.33 port 6789 ... 2024-04-25T01:25:57.223 INFO:teuthology.orchestra.run.smithi033.stdout:Base mon IP(s) is [172.21.15.33:3300, 172.21.15.33:6789], mon addrv is [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-04-25T01:25:57.227 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.33 metric 100 2024-04-25T01:25:57.227 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.33 metric 100 2024-04-25T01:25:57.230 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T01:25:57.230 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T01:25:57.230 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T01:25:57.232 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T01:25:57.232 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T01:25:57.233 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T01:25:57.233 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T01:25:57.233 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-25T01:25:57.233 INFO:teuthology.orchestra.run.smithi033.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T01:25:57.233 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-04-25T01:25:57.233 INFO:teuthology.orchestra.run.smithi033.stdout:Mon IP `172.21.15.33` is in CIDR network `172.21.0.0/20` 2024-04-25T01:25:57.234 INFO:teuthology.orchestra.run.smithi033.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T01:25:57.234 INFO:teuthology.orchestra.run.smithi033.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T01:25:57.235 INFO:teuthology.orchestra.run.smithi033.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-25T01:25:57.753 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stdout 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-25T01:25:57.753 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-25T01:25:57.753 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T01:25:57.753 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:0e2af62137028b073c6ccdb8d4d7e0ccccc475ce5f6e225cb983e90813813045 2024-04-25T01:25:57.753 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T01:25:57.753 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Copying config sha256:0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-25T01:25:57.753 INFO:teuthology.orchestra.run.smithi033.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T01:25:58.617 INFO:teuthology.orchestra.run.smithi033.stdout:ceph: stdout ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-25T01:25:58.618 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph version: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-25T01:25:58.618 INFO:teuthology.orchestra.run.smithi033.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T01:25:59.583 INFO:teuthology.orchestra.run.smithi033.stdout:stat: stdout 167 167 2024-04-25T01:25:59.583 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial keys... 2024-04-25T01:26:00.564 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQAnsSlmwwjNNxAAFCHvXrWeNmT6/O8A1PKpwg== 2024-04-25T01:26:01.545 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQAosSlmL1osNxAAPXluH+R3+WUoPwnOiij41g== 2024-04-25T01:26:02.446 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph-authtool: stdout AQApsSlmNgIEMxAAsqYXWUkmOjfEOVaKGLkwzA== 2024-04-25T01:26:02.447 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial monmap... 2024-04-25T01:26:03.412 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T01:26:03.412 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-04-25T01:26:03.413 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:26:03.413 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T01:26:03.413 INFO:teuthology.orchestra.run.smithi033.stdout:monmaptool for smithi033 [v2:172.21.15.33:3300,v1:172.21.15.33:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T01:26:03.413 INFO:teuthology.orchestra.run.smithi033.stdout:setting min_mon_release = octopus 2024-04-25T01:26:03.413 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: set fsid to b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:26:03.413 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T01:26:03.413 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:26:03.413 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mon... 2024-04-25T01:26:04.313 INFO:teuthology.orchestra.run.smithi033.stdout:create mon.smithi033 on 2024-04-25T01:26:04.528 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T01:26:04.671 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T01:26:04.821 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968.target. 2024-04-25T01:26:04.821 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968.target. 2024-04-25T01:26:04.994 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033 2024-04-25T01:26:04.994 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033.service: Unit ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033.service not loaded. 2024-04-25T01:26:05.169 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968.target.wants/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033.service → /etc/systemd/system/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@.service. 2024-04-25T01:26:05.656 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:05 smithi033 podman[25964]: 2024-04-25 01:26:05.637924444 +0000 UTC m=+0.325307912 container start e197c3d8473cfd51b75a4d5b3353a2a3d3845e5889ca2e4701d82bfb4e525684 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T01:26:05.656 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:05 smithi033 ceph-mon[25983]: mkfs b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:26:05.711 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:26:05.711 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-25T01:26:05.718 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:26:05.718 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-25T01:26:05.718 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-25T01:26:05.718 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T01:26:05.719 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon to start... 2024-04-25T01:26:05.719 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mon... 2024-04-25T01:26:05.935 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:05 smithi033 ceph-mon[25983]: mon.smithi033 is new leader, mons smithi033 in quorum (ranks 0) 2024-04-25T01:26:05.935 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:05 smithi033 bash[25964]: e197c3d8473cfd51b75a4d5b3353a2a3d3845e5889ca2e4701d82bfb4e525684 2024-04-25T01:26:05.935 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:05 smithi033 systemd[1]: Started Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:26:06.704 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T01:26:06.704 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout id: b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:26:06.704 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T01:26:06.704 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:06.704 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout services: 2024-04-25T01:26:06.704 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi033 (age 0.568103s) 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout data: 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.161+0000 7fdca9a64700 1 Processor -- start 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.161+0000 7fdca9a64700 1 -- start start 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.161+0000 7fdca9a64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4107840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.161+0000 7fdca9a64700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca4103230 con 0x7fdca4076c10 2024-04-25T01:26:06.705 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.162+0000 7fdca37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4107840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.162+0000 7fdca37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4107840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50830/0 (socket says 172.21.15.33:50830) 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.162+0000 7fdca37fe700 1 -- 172.21.15.33:0/2746803433 learned_addr learned my addr 172.21.15.33:0/2746803433 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.162+0000 7fdca37fe700 1 -- 172.21.15.33:0/2746803433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca4107d80 con 0x7fdca4076c10 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.162+0000 7fdca37fe700 1 --2- 172.21.15.33:0/2746803433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4107840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdc980098e0 tx=0x7fdc98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab333791323fffb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca27fc700 1 -- 172.21.15.33:0/2746803433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc9800b860 con 0x7fdca4076c10 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca27fc700 1 -- 172.21.15.33:0/2746803433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9800b9c0 con 0x7fdca4076c10 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca27fc700 1 -- 172.21.15.33:0/2746803433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc9800bc90 con 0x7fdca4076c10 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca9a64700 1 -- 172.21.15.33:0/2746803433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 msgr2=0x7fdca4107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca9a64700 1 --2- 172.21.15.33:0/2746803433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4107840 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fdc980098e0 tx=0x7fdc98009250 comp rx=0 tx=0).stop 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca9a64700 1 -- 172.21.15.33:0/2746803433 shutdown_connections 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca9a64700 1 --2- 172.21.15.33:0/2746803433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4107840 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca9a64700 1 -- 172.21.15.33:0/2746803433 >> 172.21.15.33:0/2746803433 conn(0x7fdca40fcef0 msgr2=0x7fdca40ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca9a64700 1 -- 172.21.15.33:0/2746803433 shutdown_connections 2024-04-25T01:26:06.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.163+0000 7fdca9a64700 1 -- 172.21.15.33:0/2746803433 wait complete. 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.164+0000 7fdca9a64700 1 Processor -- start 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.164+0000 7fdca9a64700 1 -- start start 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.164+0000 7fdca9a64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4112f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.164+0000 7fdca9a64700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca41134b0 con 0x7fdca4076c10 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.164+0000 7fdca37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4112f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.164+0000 7fdca37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4112f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50840/0 (socket says 172.21.15.33:50840) 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.164+0000 7fdca37fe700 1 -- 172.21.15.33:0/4147058282 learned_addr learned my addr 172.21.15.33:0/4147058282 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.164+0000 7fdca37fe700 1 -- 172.21.15.33:0/4147058282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980094f0 con 0x7fdca4076c10 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.165+0000 7fdca37fe700 1 --2- 172.21.15.33:0/4147058282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4112f70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009290 tx=0x7fdc98016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.165+0000 7fdc8bfff700 1 -- 172.21.15.33:0/4147058282 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc9801c5c0 con 0x7fdca4076c10 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.165+0000 7fdc8bfff700 1 -- 172.21.15.33:0/4147058282 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9801c740 con 0x7fdca4076c10 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.165+0000 7fdca9a64700 1 -- 172.21.15.33:0/4147058282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca4113710 con 0x7fdca4076c10 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.165+0000 7fdc8bfff700 1 -- 172.21.15.33:0/4147058282 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdc9801ca50 con 0x7fdca4076c10 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.165+0000 7fdca9a64700 1 -- 172.21.15.33:0/4147058282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca4114220 con 0x7fdca4076c10 2024-04-25T01:26:06.707 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.165+0000 7fdc8bfff700 1 -- 172.21.15.33:0/4147058282 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fdc9801cbb0 con 0x7fdca4076c10 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.165+0000 7fdc8bfff700 1 -- 172.21.15.33:0/4147058282 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fdc9801abf0 con 0x7fdca4076c10 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.166+0000 7fdca9a64700 1 -- 172.21.15.33:0/4147058282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca405f5c0 con 0x7fdca4076c10 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.168+0000 7fdc8bfff700 1 -- 172.21.15.33:0/4147058282 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fdc9801adb0 con 0x7fdca4076c10 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.240+0000 7fdca9a64700 1 -- 172.21.15.33:0/4147058282 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fdca4107f70 con 0x7fdca4076c10 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.240+0000 7fdc8bfff700 1 -- 172.21.15.33:0/4147058282 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fdc98025440 con 0x7fdca4076c10 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.241+0000 7fdca9a64700 1 -- 172.21.15.33:0/4147058282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 msgr2=0x7fdca4112f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.241+0000 7fdca9a64700 1 --2- 172.21.15.33:0/4147058282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4112f70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009290 tx=0x7fdc98016000 comp rx=0 tx=0).stop 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.241+0000 7fdca9a64700 1 -- 172.21.15.33:0/4147058282 shutdown_connections 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.241+0000 7fdca9a64700 1 --2- 172.21.15.33:0/4147058282 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdca4076c10 0x7fdca4112f70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.241+0000 7fdca9a64700 1 -- 172.21.15.33:0/4147058282 >> 172.21.15.33:0/4147058282 conn(0x7fdca40fcef0 msgr2=0x7fdca40fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.241+0000 7fdca9a64700 1 -- 172.21.15.33:0/4147058282 shutdown_connections 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:06.241+0000 7fdca9a64700 1 -- 172.21.15.33:0/4147058282 wait complete. 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:mon is available 2024-04-25T01:26:06.708 INFO:teuthology.orchestra.run.smithi033.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T01:26:06.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:06 smithi033 ceph-mon[25983]: mon.smithi033 is new leader, mons smithi033 in quorum (ranks 0) 2024-04-25T01:26:06.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:06 smithi033 ceph-mon[25983]: monmap e1: 1 mons at {smithi033=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-25T01:26:06.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:06 smithi033 ceph-mon[25983]: fsmap 2024-04-25T01:26:06.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:06 smithi033 ceph-mon[25983]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T01:26:06.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:06 smithi033 ceph-mon[25983]: mgrmap e1: no daemons active 2024-04-25T01:26:06.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:06 smithi033 ceph-mon[25983]: from='client.? 172.21.15.33:0/4147058282' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T01:26:10.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:10 smithi033 ceph-mon[25983]: from='client.? 172.21.15.33:0/3830729544' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T01:26:10.112 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:10 smithi033 ceph-mon[25983]: from='client.? 172.21.15.33:0/3830729544' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T01:26:15.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:15.956 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.33:3300,v1:172.21.15.33:6789] 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T01:26:15.957 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.936+0000 7f43c5dbf700 1 Processor -- start 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.936+0000 7f43c5dbf700 1 -- start start 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.937+0000 7f43c5dbf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.937+0000 7f43c5dbf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c01031d0 con 0x7f43c0102880 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.937+0000 7f43bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.937+0000 7f43bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50854/0 (socket says 172.21.15.33:50854) 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.937+0000 7f43bf7fe700 1 -- 172.21.15.33:0/1311181911 learned_addr learned my addr 172.21.15.33:0/1311181911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.937+0000 7f43bf7fe700 1 -- 172.21.15.33:0/1311181911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c0103310 con 0x7f43c0102880 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.937+0000 7f43bf7fe700 1 --2- 172.21.15.33:0/1311181911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0102c90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f43b00098e0 tx=0x7f43b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264e4dffa48cf4f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.958 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.937+0000 7f43be7fc700 1 -- 172.21.15.33:0/1311181911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b000b860 con 0x7f43c0102880 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.938+0000 7f43be7fc700 1 -- 172.21.15.33:0/1311181911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b000b9c0 con 0x7f43c0102880 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.938+0000 7f43be7fc700 1 -- 172.21.15.33:0/1311181911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b000bc90 con 0x7f43c0102880 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.938+0000 7f43c5dbf700 1 -- 172.21.15.33:0/1311181911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 msgr2=0x7f43c0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.938+0000 7f43c5dbf700 1 --2- 172.21.15.33:0/1311181911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0102c90 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f43b00098e0 tx=0x7f43b0009250 comp rx=0 tx=0).stop 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.938+0000 7f43c5dbf700 1 -- 172.21.15.33:0/1311181911 shutdown_connections 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.938+0000 7f43c5dbf700 1 --2- 172.21.15.33:0/1311181911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0102c90 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.938+0000 7f43c5dbf700 1 -- 172.21.15.33:0/1311181911 >> 172.21.15.33:0/1311181911 conn(0x7f43c00fe060 msgr2=0x7f43c0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.938+0000 7f43c5dbf700 1 -- 172.21.15.33:0/1311181911 shutdown_connections 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.938+0000 7f43c5dbf700 1 -- 172.21.15.33:0/1311181911 wait complete. 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.939+0000 7f43c5dbf700 1 Processor -- start 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.939+0000 7f43c5dbf700 1 -- start start 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.939+0000 7f43c5dbf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.939+0000 7f43c5dbf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c0079160 con 0x7f43c0102880 2024-04-25T01:26:15.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.939+0000 7f43bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.939+0000 7f43bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50870/0 (socket says 172.21.15.33:50870) 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.939+0000 7f43bf7fe700 1 -- 172.21.15.33:0/3830729544 learned_addr learned my addr 172.21.15.33:0/3830729544 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.939+0000 7f43bf7fe700 1 -- 172.21.15.33:0/3830729544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00094f0 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.940+0000 7f43bf7fe700 1 --2- 172.21.15.33:0/3830729544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0078c20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f43b0003080 tx=0x7f43b0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.940+0000 7f43a7fff700 1 -- 172.21.15.33:0/3830729544 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b001c5c0 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.940+0000 7f43a7fff700 1 -- 172.21.15.33:0/3830729544 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b001c720 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.940+0000 7f43a7fff700 1 -- 172.21.15.33:0/3830729544 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f43b001ca30 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.940+0000 7f43c5dbf700 1 -- 172.21.15.33:0/3830729544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c00771d0 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.940+0000 7f43c5dbf700 1 -- 172.21.15.33:0/3830729544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c00776c0 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.940+0000 7f43a7fff700 1 -- 172.21.15.33:0/3830729544 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f43b001cc60 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.940+0000 7f43a7fff700 1 -- 172.21.15.33:0/3830729544 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f43b001ac80 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.941+0000 7f43c5dbf700 1 -- 172.21.15.33:0/3830729544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c004bed0 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:09.943+0000 7f43a7fff700 1 -- 172.21.15.33:0/3830729544 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f43b001a450 con 0x7f43c0102880 2024-04-25T01:26:15.960 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.014+0000 7f43c5dbf700 1 -- 172.21.15.33:0/3830729544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f43c0103b50 con 0x7f43c0102880 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.017+0000 7f43a7fff700 1 -- 172.21.15.33:0/3830729544 <== mon.0 v2:172.21.15.33:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f43b002e8d0 con 0x7f43c0102880 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.017+0000 7f43a7fff700 1 -- 172.21.15.33:0/3830729544 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f43b0024520 con 0x7f43c0102880 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.018+0000 7f43c5dbf700 1 -- 172.21.15.33:0/3830729544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 msgr2=0x7f43c0078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.018+0000 7f43c5dbf700 1 --2- 172.21.15.33:0/3830729544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0078c20 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f43b0003080 tx=0x7f43b0016000 comp rx=0 tx=0).stop 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.018+0000 7f43c5dbf700 1 -- 172.21.15.33:0/3830729544 shutdown_connections 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.018+0000 7f43c5dbf700 1 --2- 172.21.15.33:0/3830729544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f43c0102880 0x7f43c0078c20 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.018+0000 7f43c5dbf700 1 -- 172.21.15.33:0/3830729544 >> 172.21.15.33:0/3830729544 conn(0x7f43c00fe060 msgr2=0x7f43c0105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.018+0000 7f43c5dbf700 1 -- 172.21.15.33:0/3830729544 shutdown_connections 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:10.018+0000 7f43c5dbf700 1 -- 172.21.15.33:0/3830729544 wait complete. 2024-04-25T01:26:15.961 INFO:teuthology.orchestra.run.smithi033.stdout:Generating new minimal ceph.conf... 2024-04-25T01:26:18.985 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:18 smithi033 ceph-mon[25983]: from='client.? 172.21.15.33:0/369439330' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:26:19.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.629+0000 7fd5a7795700 1 Processor -- start 2024-04-25T01:26:19.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.629+0000 7fd5a7795700 1 -- start start 2024-04-25T01:26:19.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.629+0000 7fd5a7795700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.629+0000 7fd5a7795700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a0105980 con 0x7fd5a0076c10 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.630+0000 7fd5a5531700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.630+0000 7fd5a5531700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42988/0 (socket says 172.21.15.33:42988) 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.630+0000 7fd5a5531700 1 -- 172.21.15.33:0/2977271891 learned_addr learned my addr 172.21.15.33:0/2977271891 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.630+0000 7fd5a5531700 1 -- 172.21.15.33:0/2977271891 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a0105ac0 con 0x7fd5a0076c10 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.630+0000 7fd5a5531700 1 --2- 172.21.15.33:0/2977271891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0105440 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5900098e0 tx=0x7fd590009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c0aa00b8b1c6b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.630+0000 7fd597fff700 1 -- 172.21.15.33:0/2977271891 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd59000b860 con 0x7fd5a0076c10 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.630+0000 7fd597fff700 1 -- 172.21.15.33:0/2977271891 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd59000be50 con 0x7fd5a0076c10 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.630+0000 7fd597fff700 1 -- 172.21.15.33:0/2977271891 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd590010380 con 0x7fd5a0076c10 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.631+0000 7fd5a7795700 1 -- 172.21.15.33:0/2977271891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 msgr2=0x7fd5a0105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.289 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.631+0000 7fd5a7795700 1 --2- 172.21.15.33:0/2977271891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0105440 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd5900098e0 tx=0x7fd590009250 comp rx=0 tx=0).stop 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.631+0000 7fd5a7795700 1 -- 172.21.15.33:0/2977271891 shutdown_connections 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.631+0000 7fd5a7795700 1 --2- 172.21.15.33:0/2977271891 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0105440 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.631+0000 7fd5a7795700 1 -- 172.21.15.33:0/2977271891 >> 172.21.15.33:0/2977271891 conn(0x7fd5a00fcef0 msgr2=0x7fd5a00ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.631+0000 7fd5a7795700 1 -- 172.21.15.33:0/2977271891 shutdown_connections 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.631+0000 7fd5a7795700 1 -- 172.21.15.33:0/2977271891 wait complete. 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.632+0000 7fd5a7795700 1 Processor -- start 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.632+0000 7fd5a7795700 1 -- start start 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.632+0000 7fd5a7795700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.632+0000 7fd5a7795700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a01032d0 con 0x7fd5a0076c10 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.632+0000 7fd5a5531700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:19.290 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.632+0000 7fd5a5531700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42990/0 (socket says 172.21.15.33:42990) 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.632+0000 7fd5a5531700 1 -- 172.21.15.33:0/369439330 learned_addr learned my addr 172.21.15.33:0/369439330 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.632+0000 7fd5a5531700 1 -- 172.21.15.33:0/369439330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5900094f0 con 0x7fd5a0076c10 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.633+0000 7fd5a5531700 1 --2- 172.21.15.33:0/369439330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0102d90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd590006b00 tx=0x7fd590009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.633+0000 7fd595ffb700 1 -- 172.21.15.33:0/369439330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5900107c0 con 0x7fd5a0076c10 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.633+0000 7fd595ffb700 1 -- 172.21.15.33:0/369439330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd590010db0 con 0x7fd5a0076c10 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.633+0000 7fd5a7795700 1 -- 172.21.15.33:0/369439330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a01013a0 con 0x7fd5a0076c10 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.633+0000 7fd595ffb700 1 -- 172.21.15.33:0/369439330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd59001acf0 con 0x7fd5a0076c10 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.633+0000 7fd5a7795700 1 -- 172.21.15.33:0/369439330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a0101890 con 0x7fd5a0076c10 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.633+0000 7fd595ffb700 1 -- 172.21.15.33:0/369439330 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd590024440 con 0x7fd5a0076c10 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.633+0000 7fd595ffb700 1 -- 172.21.15.33:0/369439330 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd590019710 con 0x7fd5a0076c10 2024-04-25T01:26:19.291 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.634+0000 7fd5a7795700 1 -- 172.21.15.33:0/369439330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a004bed0 con 0x7fd5a0076c10 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.636+0000 7fd595ffb700 1 -- 172.21.15.33:0/369439330 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fd590042790 con 0x7fd5a0076c10 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.707+0000 7fd5a7795700 1 -- 172.21.15.33:0/369439330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd5a0101b30 con 0x7fd5a0076c10 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.708+0000 7fd595ffb700 1 -- 172.21.15.33:0/369439330 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fd590031030 con 0x7fd5a0076c10 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.708+0000 7fd5a7795700 1 -- 172.21.15.33:0/369439330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 msgr2=0x7fd5a0102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.708+0000 7fd5a7795700 1 --2- 172.21.15.33:0/369439330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0102d90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd590006b00 tx=0x7fd590009250 comp rx=0 tx=0).stop 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.708+0000 7fd5a7795700 1 -- 172.21.15.33:0/369439330 shutdown_connections 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.708+0000 7fd5a7795700 1 --2- 172.21.15.33:0/369439330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd5a0076c10 0x7fd5a0102d90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.708+0000 7fd5a7795700 1 -- 172.21.15.33:0/369439330 >> 172.21.15.33:0/369439330 conn(0x7fd5a00fcef0 msgr2=0x7fd5a00fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.709+0000 7fd5a7795700 1 -- 172.21.15.33:0/369439330 shutdown_connections 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:18.709+0000 7fd5a7795700 1 -- 172.21.15.33:0/369439330 wait complete. 2024-04-25T01:26:19.292 INFO:teuthology.orchestra.run.smithi033.stdout:Restarting the monitor... 2024-04-25T01:26:19.549 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:19 smithi033 systemd[1]: Stopping Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:26:19.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:19 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[25979]: 2024-04-25T01:26:19.568+0000 7f2c7748d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi033 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:26:19.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:19 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[25979]: 2024-04-25T01:26:19.568+0000 7f2c7748d700 -1 mon.smithi033@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T01:26:19.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:19 smithi033 podman[26227]: 2024-04-25 01:26:19.63056959 +0000 UTC m=+0.153827798 container died e197c3d8473cfd51b75a4d5b3353a2a3d3845e5889ca2e4701d82bfb4e525684 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=quincy-c61472c) 2024-04-25T01:26:20.122 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:19 smithi033 podman[26227]: 2024-04-25 01:26:19.95130392 +0000 UTC m=+0.474562117 container cleanup e197c3d8473cfd51b75a4d5b3353a2a3d3845e5889ca2e4701d82bfb4e525684 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T01:26:20.122 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:19 smithi033 bash[26227]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033 2024-04-25T01:26:20.373 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 podman[26241]: 2024-04-25 01:26:20.143827259 +0000 UTC m=+0.507790216 container remove e197c3d8473cfd51b75a4d5b3353a2a3d3845e5889ca2e4701d82bfb4e525684 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T01:26:20.373 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033.service: Deactivated successfully. 2024-04-25T01:26:20.373 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 systemd[1]: Stopped Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:26:20.373 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 systemd[1]: Starting Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:26:20.832 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 podman[26334]: 2024-04-25 01:26:20.603376384 +0000 UTC m=+0.111514836 container create dd064971bee1519abf6c530333899fc769b74fb8b61a585bffe3cc3b2858da22 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=quincy-c61472c, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 podman[26334]: 2024-04-25 01:26:20.510394631 +0000 UTC m=+0.018533084 image pull 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 quay.ceph.io/ceph-ci/ceph:quincy 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 podman[26334]: 2024-04-25 01:26:20.822927365 +0000 UTC m=+0.331065818 container init dd064971bee1519abf6c530333899fc769b74fb8b61a585bffe3cc3b2858da22 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 podman[26334]: 2024-04-25 01:26:20.826256103 +0000 UTC m=+0.334394558 container start dd064971bee1519abf6c530333899fc769b74fb8b61a585bffe3cc3b2858da22 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable), process ceph-mon, pid 2 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: pidfile_write: ignore empty --pid-file 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: load: jerasure load: lrc 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: RocksDB version: 6.15.5 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Compile date Apr 24 2024 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: DB SUMMARY 2024-04-25T01:26:20.833 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: DB Session ID: EHUQKQ28DQYWYIMPAI2W 2024-04-25T01:26:20.834 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: CURRENT file: CURRENT 2024-04-25T01:26:20.834 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T01:26:20.834 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-04-25T01:26:20.834 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi033/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T01:26:20.834 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi033/store.db: 000010.log size: 98160 ; 2024-04-25T01:26:20.834 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.error_if_exists: 0 2024-04-25T01:26:20.834 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.create_if_missing: 0 2024-04-25T01:26:20.834 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.paranoid_checks: 1 2024-04-25T01:26:20.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T01:26:20.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.env: 0x55b410465b00 2024-04-25T01:26:20.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.fs: Posix File System 2024-04-25T01:26:20.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.info_log: 0x55b41112d620 2024-04-25T01:26:20.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T01:26:20.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.statistics: (nil) 2024-04-25T01:26:20.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.use_fsync: 0 2024-04-25T01:26:20.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_log_file_size: 0 2024-04-25T01:26:20.835 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.allow_fallocate: 1 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.use_direct_reads: 0 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.db_log_dir: 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi033/store.db 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T01:26:20.836 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.write_buffer_manager: 0x55b411e20a20 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T01:26:20.837 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.unordered_write: 0 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.row_cache: None 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.wal_filter: None 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.preserve_deletes: 0 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.two_write_queues: 0 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.atomic_flush: 0 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T01:26:20.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.log_readahead_size: 0 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_background_jobs: 2 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_background_compactions: -1 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_subcompactions: 1 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T01:26:20.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_open_files: -1 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_background_flushes: -1 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Compression algorithms supported: 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: kZSTD supported: 0 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: kXpressCompression supported: 0 2024-04-25T01:26:20.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: kZlibCompression supported: 1 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: kSnappyCompression supported: 1 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: kLZ4Compression supported: 1 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: kBZip2Compression supported: 0 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi033/store.db/MANIFEST-000009 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.merge_operator: 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_filter: None 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T01:26:20.841 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b411100f58) 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_type: 0 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_block_index_type: 0 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_shortening: 1 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: hash_index_allow_collision: 1 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: checksum: 1 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: no_block_cache: 0 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache: 0x55b411133090 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_options: 2024-04-25T01:26:20.842 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: capacity : 536870912 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: num_shard_bits : 4 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: strict_capacity_limit : 0 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_compressed: (nil) 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: persistent_cache: (nil) 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_size: 4096 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_size_deviation: 10 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_restart_interval: 16 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_block_restart_interval: 1 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: metadata_block_size: 4096 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: partition_filters: 0 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: use_delta_encoding: 1 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-25T01:26:20.843 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: whole_key_filtering: 1 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: verify_compression: 0 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: format_version: 4 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: enable_index_compression: 1 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_align: 0 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compression: NoCompression 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.num_levels: 7 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T01:26:20.844 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T01:26:20.845 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T01:26:20.846 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T01:26:20.847 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T01:26:20.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T01:26:20.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T01:26:20.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T01:26:20.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T01:26:20.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T01:26:20.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.arena_block_size: 4194304 2024-04-25T01:26:20.848 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T01:26:20.849 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.table_properties_collectors: 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.inplace_update_support: 0 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.bloom_locality: 0 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.max_successive_merges: 0 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.ttl: 2592000 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T01:26:20.850 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.enable_blob_files: false 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.min_blob_size: 0 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi033/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714008380852713, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714008380853665, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93366, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91478, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84042, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1714008380, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "5ff92dd9-4b6c-435b-add9-8fb8380852f6", "db_session_id": "EHUQKQ28DQYWYIMPAI2W"}} 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-04-25T01:26:20.851 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714008380853999, "job": 1, "event": "recovery_finished"} 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi033/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55b4110c7180 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: DB pointer 0x55b41119c000 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** DB Stats ** 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:26:20.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.5 0.00 0.00 1 0.001 0 0 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.5 0.00 0.00 1 0.001 0 0 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.5 0.00 0.00 1 0.001 0 0 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-25T01:26:20.854 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 134.5 0.00 0.00 1 0.001 0 0 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative compaction: 0.00 GB write, 27.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval compaction: 0.00 GB write, 27.15 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-25T01:26:20.855 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: L0 2/0 92.98 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.5 0.00 0.00 1 0.001 0 0 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Sum 2/0 92.98 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.5 0.00 0.00 1 0.001 0 0 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 134.5 0.00 0.00 1 0.001 0 0 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T01:26:20.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative compaction: 0.00 GB write, 26.90 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: starting mon.smithi033 rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi033 fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mon.smithi033@-1(???) e1 preinit fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mon.smithi033@-1(???).mds e1 new map 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mon.smithi033@-1(???).mds e1 print_map 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: e1 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: legacy client fscid: -1 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:26:20.857 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: No filesystems configured 2024-04-25T01:26:20.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mon.smithi033@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T01:26:20.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mon.smithi033@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:26:20.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mon.smithi033@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:26:20.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mon.smithi033@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T01:26:20.858 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mon.smithi033@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T01:26:20.900 INFO:teuthology.orchestra.run.smithi033.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T01:26:21.152 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mon.smithi033 is new leader, mons smithi033 in quorum (ranks 0) 2024-04-25T01:26:21.152 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: monmap e1: 1 mons at {smithi033=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-25T01:26:21.152 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: fsmap 2024-04-25T01:26:21.152 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T01:26:21.152 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 ceph-mon[26353]: mgrmap e1: no daemons active 2024-04-25T01:26:21.152 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 bash[26334]: dd064971bee1519abf6c530333899fc769b74fb8b61a585bffe3cc3b2858da22 2024-04-25T01:26:21.152 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:20 smithi033 systemd[1]: Started Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:26:21.935 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.361+0000 7f1b9faa1700 1 Processor -- start 2024-04-25T01:26:21.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.361+0000 7f1b9faa1700 1 -- start start 2024-04-25T01:26:21.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.361+0000 7f1b9faa1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b98102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:21.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.361+0000 7f1b9faa1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981031d0 con 0x7f1b98102880 2024-04-25T01:26:21.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.362+0000 7f1b9d83d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b98102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:21.940 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.362+0000 7f1b9d83d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b98102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42992/0 (socket says 172.21.15.33:42992) 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.362+0000 7f1b9d83d700 1 -- 172.21.15.33:0/1941455352 learned_addr learned my addr 172.21.15.33:0/1941455352 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.362+0000 7f1b9d83d700 1 -- 172.21.15.33:0/1941455352 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b98103310 con 0x7f1b98102880 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.362+0000 7f1b9d83d700 1 --2- 172.21.15.33:0/1941455352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b98102c90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1b940098e0 tx=0x7f1b94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b55d8cc14df0b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.362+0000 7f1b9c83b700 1 -- 172.21.15.33:0/1941455352 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b9400b860 con 0x7f1b98102880 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.362+0000 7f1b9c83b700 1 -- 172.21.15.33:0/1941455352 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1b9400be50 con 0x7f1b98102880 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.362+0000 7f1b9c83b700 1 -- 172.21.15.33:0/1941455352 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b94010380 con 0x7f1b98102880 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.363+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1941455352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 msgr2=0x7f1b98102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.363+0000 7f1b9faa1700 1 --2- 172.21.15.33:0/1941455352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b98102c90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1b940098e0 tx=0x7f1b94009250 comp rx=0 tx=0).stop 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.363+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1941455352 shutdown_connections 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.363+0000 7f1b9faa1700 1 --2- 172.21.15.33:0/1941455352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b98102c90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.363+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1941455352 >> 172.21.15.33:0/1941455352 conn(0x7f1b980fe060 msgr2=0x7f1b98100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.363+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1941455352 shutdown_connections 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.363+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1941455352 wait complete. 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.363+0000 7f1b9faa1700 1 Processor -- start 2024-04-25T01:26:21.941 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9faa1700 1 -- start start 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9faa1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b981115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9faa1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98111b30 con 0x7f1b98102880 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9d83d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b981115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9d83d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b981115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42998/0 (socket says 172.21.15.33:42998) 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9d83d700 1 -- 172.21.15.33:0/1043665022 learned_addr learned my addr 172.21.15.33:0/1043665022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9d83d700 1 -- 172.21.15.33:0/1043665022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b940094f0 con 0x7f1b98102880 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9d83d700 1 --2- 172.21.15.33:0/1043665022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b981115f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000c00 tx=0x7f1b94009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b8e7fc700 1 -- 172.21.15.33:0/1043665022 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b940107c0 con 0x7f1b98102880 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b8e7fc700 1 -- 172.21.15.33:0/1043665022 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1b94010db0 con 0x7f1b98102880 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1043665022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b98111d30 con 0x7f1b98102880 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b8e7fc700 1 -- 172.21.15.33:0/1043665022 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1b9401ad00 con 0x7f1b98102880 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.364+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1043665022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b981128a0 con 0x7f1b98102880 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.365+0000 7f1b8e7fc700 1 -- 172.21.15.33:0/1043665022 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f1b94024440 con 0x7f1b98102880 2024-04-25T01:26:21.942 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.365+0000 7f1b8e7fc700 1 -- 172.21.15.33:0/1043665022 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1b94019710 con 0x7f1b98102880 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.365+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1043665022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9805f5c0 con 0x7f1b98102880 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.368+0000 7f1b8e7fc700 1 -- 172.21.15.33:0/1043665022 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f1b9401a530 con 0x7f1b98102880 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.436+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1043665022 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f1b98104650 con 0x7f1b98102880 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.438+0000 7f1b8e7fc700 1 -- 172.21.15.33:0/1043665022 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f1b94010920 con 0x7f1b98102880 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.439+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1043665022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 msgr2=0x7f1b981115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.439+0000 7f1b9faa1700 1 --2- 172.21.15.33:0/1043665022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b981115f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1b94000c00 tx=0x7f1b94009250 comp rx=0 tx=0).stop 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.439+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1043665022 shutdown_connections 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.439+0000 7f1b9faa1700 1 --2- 172.21.15.33:0/1043665022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1b98102880 0x7f1b981115f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.439+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1043665022 >> 172.21.15.33:0/1043665022 conn(0x7f1b980fe060 msgr2=0x7f1b980feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.439+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1043665022 shutdown_connections 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:21.439+0000 7f1b9faa1700 1 -- 172.21.15.33:0/1043665022 wait complete. 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:26:21.943 INFO:teuthology.orchestra.run.smithi033.stdout:Creating mgr... 2024-04-25T01:26:21.944 INFO:teuthology.orchestra.run.smithi033.stdout:Verifying port 9283 ... 2024-04-25T01:26:22.120 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mgr.smithi033.asemba 2024-04-25T01:26:22.120 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mgr.smithi033.asemba.service: Unit ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mgr.smithi033.asemba.service not loaded. 2024-04-25T01:26:22.276 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968.target.wants/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mgr.smithi033.asemba.service → /etc/systemd/system/ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@.service. 2024-04-25T01:26:22.622 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:22 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1043665022' entity='client.admin' 2024-04-25T01:26:22.842 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:26:22.843 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-25T01:26:22.850 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:26:22.850 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-25T01:26:22.850 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-25T01:26:22.850 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T01:26:22.859 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:26:22.859 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-25T01:26:22.867 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:26:22.867 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-25T01:26:22.867 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-25T01:26:22.867 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-04-25T01:26:22.867 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr to start... 2024-04-25T01:26:22.867 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr... 2024-04-25T01:26:23.596 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:23 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/22841563' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T01:26:25.134 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:25.134 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:26:25.134 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "b17b7f0e-02a2-11ef-bc93-c7b262605968", 2024-04-25T01:26:25.134 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "smithi033" 2024-04-25T01:26:25.135 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T01:26:25.136 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:25.137 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:26:25.138 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T01:26:05.674924+0000", 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.352+0000 7fcf9cee9700 1 Processor -- start 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.353+0000 7fcf9cee9700 1 -- start start 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.353+0000 7fcf9cee9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.353+0000 7fcf9cee9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf981031c0 con 0x7fcf98102870 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.353+0000 7fcf9659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.353+0000 7fcf9659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43018/0 (socket says 172.21.15.33:43018) 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.353+0000 7fcf9659c700 1 -- 172.21.15.33:0/1042254396 learned_addr learned my addr 172.21.15.33:0/1042254396 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.354+0000 7fcf9659c700 1 -- 172.21.15.33:0/1042254396 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf98103300 con 0x7fcf98102870 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.354+0000 7fcf9659c700 1 --2- 172.21.15.33:0/1042254396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98102c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf880098e0 tx=0x7fcf88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138b783a84d7061c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.354+0000 7fcf9559a700 1 -- 172.21.15.33:0/1042254396 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf8800b860 con 0x7fcf98102870 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.354+0000 7fcf9559a700 1 -- 172.21.15.33:0/1042254396 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcf8800be50 con 0x7fcf98102870 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.354+0000 7fcf9559a700 1 -- 172.21.15.33:0/1042254396 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf88010380 con 0x7fcf98102870 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 -- 172.21.15.33:0/1042254396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 msgr2=0x7fcf98102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 --2- 172.21.15.33:0/1042254396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98102c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcf880098e0 tx=0x7fcf88009250 comp rx=0 tx=0).stop 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 -- 172.21.15.33:0/1042254396 shutdown_connections 2024-04-25T01:26:25.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 --2- 172.21.15.33:0/1042254396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98102c80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 -- 172.21.15.33:0/1042254396 >> 172.21.15.33:0/1042254396 conn(0x7fcf980fe030 msgr2=0x7fcf98100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 -- 172.21.15.33:0/1042254396 shutdown_connections 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 -- 172.21.15.33:0/1042254396 wait complete. 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 Processor -- start 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 -- start start 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.355+0000 7fcf9cee9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf9cee9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf98079160 con 0x7fcf98102870 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf9659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf9659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43034/0 (socket says 172.21.15.33:43034) 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf9659c700 1 -- 172.21.15.33:0/22841563 learned_addr learned my addr 172.21.15.33:0/22841563 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf9659c700 1 -- 172.21.15.33:0/22841563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf880094f0 con 0x7fcf98102870 2024-04-25T01:26:25.141 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf9659c700 1 --2- 172.21.15.33:0/22841563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98078c20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800b440 tx=0x7fcf88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf86ffd700 1 -- 172.21.15.33:0/22841563 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf88010760 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf86ffd700 1 -- 172.21.15.33:0/22841563 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fcf88010d50 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf9cee9700 1 -- 172.21.15.33:0/22841563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf980771d0 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf86ffd700 1 -- 172.21.15.33:0/22841563 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcf8801ae20 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.356+0000 7fcf9cee9700 1 -- 172.21.15.33:0/22841563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf980776c0 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.357+0000 7fcf86ffd700 1 -- 172.21.15.33:0/22841563 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fcf8802e440 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.357+0000 7fcf86ffd700 1 -- 172.21.15.33:0/22841563 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fcf88024740 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.357+0000 7fcf9cee9700 1 -- 172.21.15.33:0/22841563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9804bed0 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.360+0000 7fcf86ffd700 1 -- 172.21.15.33:0/22841563 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fcf88029070 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.432+0000 7fcf9cee9700 1 -- 172.21.15.33:0/22841563 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fcf98104640 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.432+0000 7fcf86ffd700 1 -- 172.21.15.33:0/22841563 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fcf8801a6a0 con 0x7fcf98102870 2024-04-25T01:26:25.142 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.434+0000 7fcf9cee9700 1 -- 172.21.15.33:0/22841563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 msgr2=0x7fcf98078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:25.143 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.434+0000 7fcf9cee9700 1 --2- 172.21.15.33:0/22841563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98078c20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800b440 tx=0x7fcf88009250 comp rx=0 tx=0).stop 2024-04-25T01:26:25.143 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.434+0000 7fcf9cee9700 1 -- 172.21.15.33:0/22841563 shutdown_connections 2024-04-25T01:26:25.143 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.434+0000 7fcf9cee9700 1 --2- 172.21.15.33:0/22841563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf98102870 0x7fcf98078c20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:25.143 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.434+0000 7fcf9cee9700 1 -- 172.21.15.33:0/22841563 >> 172.21.15.33:0/22841563 conn(0x7fcf980fe030 msgr2=0x7fcf980feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:25.143 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.434+0000 7fcf9cee9700 1 -- 172.21.15.33:0/22841563 shutdown_connections 2024-04-25T01:26:25.143 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:23.434+0000 7fcf9cee9700 1 -- 172.21.15.33:0/22841563 wait complete. 2024-04-25T01:26:25.143 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (1/15)... 2024-04-25T01:26:27.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:27 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1046799256' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T01:26:28.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:28.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:26:28.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "b17b7f0e-02a2-11ef-bc93-c7b262605968", 2024-04-25T01:26:28.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T01:26:28.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T01:26:28.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T01:26:28.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T01:26:28.295 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "smithi033" 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T01:26:28.296 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:28.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T01:26:28.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:28.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T01:26:28.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T01:26:28.297 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T01:26:28.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T01:26:28.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T01:26:28.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T01:26:28.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:28.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T01:26:28.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T01:26:28.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T01:26:28.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T01:26:28.305 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T01:26:28.306 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T01:26:05.674924+0000", 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.605+0000 7f4625f4c700 1 Processor -- start 2024-04-25T01:26:28.307 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.605+0000 7f4625f4c700 1 -- start start 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.605+0000 7f4625f4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f4620101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.605+0000 7f4625f4c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620102090 con 0x7f4620101740 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.605+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f4620101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.605+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f4620101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44130/0 (socket says 172.21.15.33:44130) 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.605+0000 7f461f7fe700 1 -- 172.21.15.33:0/3550651121 learned_addr learned my addr 172.21.15.33:0/3550651121 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.606+0000 7f461f7fe700 1 -- 172.21.15.33:0/3550651121 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46201021d0 con 0x7f4620101740 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.606+0000 7f461f7fe700 1 --2- 172.21.15.33:0/3550651121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f4620101b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f46100098e0 tx=0x7f4610009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=880c4e589fa8462d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.606+0000 7f461e7fc700 1 -- 172.21.15.33:0/3550651121 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f461000b860 con 0x7f4620101740 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.606+0000 7f461e7fc700 1 -- 172.21.15.33:0/3550651121 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f461000be50 con 0x7f4620101740 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.606+0000 7f461e7fc700 1 -- 172.21.15.33:0/3550651121 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4610010380 con 0x7f4620101740 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.607+0000 7f4625f4c700 1 -- 172.21.15.33:0/3550651121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 msgr2=0x7f4620101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.607+0000 7f4625f4c700 1 --2- 172.21.15.33:0/3550651121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f4620101b50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f46100098e0 tx=0x7f4610009250 comp rx=0 tx=0).stop 2024-04-25T01:26:28.308 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.607+0000 7f4625f4c700 1 -- 172.21.15.33:0/3550651121 shutdown_connections 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.607+0000 7f4625f4c700 1 --2- 172.21.15.33:0/3550651121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f4620101b50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.607+0000 7f4625f4c700 1 -- 172.21.15.33:0/3550651121 >> 172.21.15.33:0/3550651121 conn(0x7f46200fcec0 msgr2=0x7f46200ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.607+0000 7f4625f4c700 1 -- 172.21.15.33:0/3550651121 shutdown_connections 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.607+0000 7f4625f4c700 1 -- 172.21.15.33:0/3550651121 wait complete. 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.607+0000 7f4625f4c700 1 Processor -- start 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.607+0000 7f4625f4c700 1 -- start start 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f4625f4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f46201137b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f4625f4c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4620113cf0 con 0x7f4620101740 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f46201137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f461f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f46201137b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44136/0 (socket says 172.21.15.33:44136) 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f461f7fe700 1 -- 172.21.15.33:0/1046799256 learned_addr learned my addr 172.21.15.33:0/1046799256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f461f7fe700 1 -- 172.21.15.33:0/1046799256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46100094f0 con 0x7f4620101740 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f461f7fe700 1 --2- 172.21.15.33:0/1046799256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f46201137b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f4610009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:28.309 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f4607fff700 1 -- 172.21.15.33:0/1046799256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46100107c0 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f4625f4c700 1 -- 172.21.15.33:0/1046799256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4620113ef0 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f4607fff700 1 -- 172.21.15.33:0/1046799256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f4610010db0 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.608+0000 7f4607fff700 1 -- 172.21.15.33:0/1046799256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f461001ad00 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.609+0000 7f4625f4c700 1 -- 172.21.15.33:0/1046799256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4620114a60 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.609+0000 7f4607fff700 1 -- 172.21.15.33:0/1046799256 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f4610010920 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.609+0000 7f4607fff700 1 -- 172.21.15.33:0/1046799256 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4610024db0 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.610+0000 7f4625f4c700 1 -- 172.21.15.33:0/1046799256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462005f5c0 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.612+0000 7f4607fff700 1 -- 172.21.15.33:0/1046799256 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f4610042790 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.686+0000 7f4625f4c700 1 -- 172.21.15.33:0/1046799256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4620103510 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.686+0000 7f4607fff700 1 -- 172.21.15.33:0/1046799256 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f4610031020 con 0x7f4620101740 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.688+0000 7f4625f4c700 1 -- 172.21.15.33:0/1046799256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 msgr2=0x7f46201137b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:28.310 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.688+0000 7f4625f4c700 1 --2- 172.21.15.33:0/1046799256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f46201137b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f4610009250 comp rx=0 tx=0).stop 2024-04-25T01:26:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.688+0000 7f4625f4c700 1 -- 172.21.15.33:0/1046799256 shutdown_connections 2024-04-25T01:26:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.688+0000 7f4625f4c700 1 --2- 172.21.15.33:0/1046799256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4620101740 0x7f46201137b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.688+0000 7f4625f4c700 1 -- 172.21.15.33:0/1046799256 >> 172.21.15.33:0/1046799256 conn(0x7f46200fcec0 msgr2=0x7f46200fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.688+0000 7f4625f4c700 1 -- 172.21.15.33:0/1046799256 shutdown_connections 2024-04-25T01:26:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:27.688+0000 7f4625f4c700 1 -- 172.21.15.33:0/1046799256 wait complete. 2024-04-25T01:26:28.311 INFO:teuthology.orchestra.run.smithi033.stdout:mgr not available, waiting (2/15)... 2024-04-25T01:26:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: Activating manager daemon smithi033.asemba 2024-04-25T01:26:29.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: mgrmap e2: smithi033.asemba(active, starting, since 0.00301466s) 2024-04-25T01:26:29.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:26:29.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:26:29.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:26:29.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:26:29.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:26:29.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: Manager daemon smithi033.asemba is now available 2024-04-25T01:26:29.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:26:29.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/trash_purge_schedule"}]: dispatch 2024-04-25T01:26:29.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' 2024-04-25T01:26:29.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' 2024-04-25T01:26:29.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:29 smithi033 ceph-mon[26353]: from='mgr.14100 172.21.15.33:0/583648681' entity='mgr.smithi033.asemba' 2024-04-25T01:26:31.670 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:31 smithi033 ceph-mon[26353]: mgrmap e3: smithi033.asemba(active, since 1.00788s) 2024-04-25T01:26:31.670 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:31 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/348214916' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsid": "b17b7f0e-02a2-11ef-bc93-c7b262605968", 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T01:26:31.675 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 0 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "smithi033" 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T01:26:31.676 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T01:26:31.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T01:26:31.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T01:26:31.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T01:26:31.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:31.685 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T01:26:31.686 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ], 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T01:26:05.674924+0000", 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T01:26:31.687 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout }, 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.784+0000 7fab4b9c7700 1 Processor -- start 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.785+0000 7fab4b9c7700 1 -- start start 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.785+0000 7fab4b9c7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44077270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.785+0000 7fab4b9c7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440790d0 con 0x7fab44078cc0 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.785+0000 7fab49763700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44077270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.785+0000 7fab49763700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44077270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44206/0 (socket says 172.21.15.33:44206) 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.785+0000 7fab49763700 1 -- 172.21.15.33:0/70137256 learned_addr learned my addr 172.21.15.33:0/70137256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.785+0000 7fab49763700 1 -- 172.21.15.33:0/70137256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440777b0 con 0x7fab44078cc0 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.786+0000 7fab49763700 1 --2- 172.21.15.33:0/70137256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44077270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fab400098e0 tx=0x7fab40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb18b683baadee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.786+0000 7fab3bfff700 1 -- 172.21.15.33:0/70137256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab4000b860 con 0x7fab44078cc0 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.786+0000 7fab3bfff700 1 -- 172.21.15.33:0/70137256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fab4000be50 con 0x7fab44078cc0 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.786+0000 7fab3bfff700 1 -- 172.21.15.33:0/70137256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab40010380 con 0x7fab44078cc0 2024-04-25T01:26:31.688 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.786+0000 7fab4b9c7700 1 -- 172.21.15.33:0/70137256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 msgr2=0x7fab44077270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.786+0000 7fab4b9c7700 1 --2- 172.21.15.33:0/70137256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44077270 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fab400098e0 tx=0x7fab40009250 comp rx=0 tx=0).stop 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.786+0000 7fab4b9c7700 1 -- 172.21.15.33:0/70137256 shutdown_connections 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.786+0000 7fab4b9c7700 1 --2- 172.21.15.33:0/70137256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44077270 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.786+0000 7fab4b9c7700 1 -- 172.21.15.33:0/70137256 >> 172.21.15.33:0/70137256 conn(0x7fab440fe060 msgr2=0x7fab44100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.787+0000 7fab4b9c7700 1 -- 172.21.15.33:0/70137256 shutdown_connections 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.787+0000 7fab4b9c7700 1 -- 172.21.15.33:0/70137256 wait complete. 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.787+0000 7fab4b9c7700 1 Processor -- start 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.787+0000 7fab4b9c7700 1 -- start start 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.787+0000 7fab4b9c7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44113860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.787+0000 7fab4b9c7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab44113da0 con 0x7fab44078cc0 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.787+0000 7fab49763700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.787+0000 7fab49763700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44113860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44216/0 (socket says 172.21.15.33:44216) 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.787+0000 7fab49763700 1 -- 172.21.15.33:0/348214916 learned_addr learned my addr 172.21.15.33:0/348214916 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.788+0000 7fab49763700 1 -- 172.21.15.33:0/348214916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400094f0 con 0x7fab44078cc0 2024-04-25T01:26:31.689 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.788+0000 7fab49763700 1 --2- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44113860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fab4000b440 tx=0x7fab40009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.788+0000 7fab39ffb700 1 -- 172.21.15.33:0/348214916 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab40010760 con 0x7fab44078cc0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.788+0000 7fab39ffb700 1 -- 172.21.15.33:0/348214916 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fab40010d50 con 0x7fab44078cc0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.788+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab44113fa0 con 0x7fab44078cc0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.788+0000 7fab39ffb700 1 -- 172.21.15.33:0/348214916 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fab4001adc0 con 0x7fab44078cc0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.788+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab44114b10 con 0x7fab44078cc0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.789+0000 7fab39ffb700 1 -- 172.21.15.33:0/348214916 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 3) v1 ==== 43259+0+0 (secure 0 0 0) 0x7fab400193c0 con 0x7fab44078cc0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.789+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4405f5c0 con 0x7fab44078cc0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.790+0000 7fab39ffb700 1 --2- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fab30036720 0x7fab30038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.790+0000 7fab39ffb700 1 -- 172.21.15.33:0/348214916 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fab40049e50 con 0x7fab44078cc0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.790+0000 7fab48f62700 1 --2- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fab30036720 0x7fab30038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.790+0000 7fab48f62700 1 --2- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fab30036720 0x7fab30038bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fab34009990 tx=0x7fab34006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:30.795+0000 7fab39ffb700 1 -- 172.21.15.33:0/348214916 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fab40022af0 con 0x7fab44078cc0 2024-04-25T01:26:31.690 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.070+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fab4404bed0 con 0x7fab44078cc0 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.071+0000 7fab39ffb700 1 -- 172.21.15.33:0/348214916 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7fab400108c0 con 0x7fab44078cc0 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fab30036720 msgr2=0x7fab30038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 --2- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fab30036720 0x7fab30038bd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fab34009990 tx=0x7fab34006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 msgr2=0x7fab44113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 --2- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44113860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fab4000b440 tx=0x7fab40009250 comp rx=0 tx=0).stop 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 shutdown_connections 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 --2- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fab30036720 0x7fab30038bd0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 --2- 172.21.15.33:0/348214916 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab44078cc0 0x7fab44113860 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 >> 172.21.15.33:0/348214916 conn(0x7fab440fe060 msgr2=0x7fab440feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 shutdown_connections 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:31.073+0000 7fab4b9c7700 1 -- 172.21.15.33:0/348214916 wait complete. 2024-04-25T01:26:31.691 INFO:teuthology.orchestra.run.smithi033.stdout:mgr is available 2024-04-25T01:26:32.877 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:32 smithi033 ceph-mon[26353]: mgrmap e4: smithi033.asemba(active, since 2s) 2024-04-25T01:26:32.877 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:32 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/3316824497' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T01:26:32.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:32.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T01:26:32.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout fsid = b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:26:32.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T01:26:32.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T01:26:32.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T01:26:32.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T01:26:32.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:32.887 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.092+0000 7f0c15a13700 1 Processor -- start 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.092+0000 7f0c15a13700 1 -- start start 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.092+0000 7f0c15a13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c10102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.092+0000 7f0c15a13700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c101031d0 con 0x7f0c10102880 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.092+0000 7f0c0effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c10102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:32.888 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.092+0000 7f0c0effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c10102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44224/0 (socket says 172.21.15.33:44224) 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.092+0000 7f0c0effd700 1 -- 172.21.15.33:0/1669560862 learned_addr learned my addr 172.21.15.33:0/1669560862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.092+0000 7f0c0effd700 1 -- 172.21.15.33:0/1669560862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10103310 con 0x7f0c10102880 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c0effd700 1 --2- 172.21.15.33:0/1669560862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c10102c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0c000098e0 tx=0x7f0c00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0805bdab64bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c0dffb700 1 -- 172.21.15.33:0/1669560862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c0000b860 con 0x7f0c10102880 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c0dffb700 1 -- 172.21.15.33:0/1669560862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0c0000be50 con 0x7f0c10102880 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c0dffb700 1 -- 172.21.15.33:0/1669560862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c00010380 con 0x7f0c10102880 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c15a13700 1 -- 172.21.15.33:0/1669560862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 msgr2=0x7f0c10102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c15a13700 1 --2- 172.21.15.33:0/1669560862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c10102c90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0c000098e0 tx=0x7f0c00009250 comp rx=0 tx=0).stop 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c15a13700 1 -- 172.21.15.33:0/1669560862 shutdown_connections 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c15a13700 1 --2- 172.21.15.33:0/1669560862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c10102c90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c15a13700 1 -- 172.21.15.33:0/1669560862 >> 172.21.15.33:0/1669560862 conn(0x7f0c100fe060 msgr2=0x7f0c10100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.093+0000 7f0c15a13700 1 -- 172.21.15.33:0/1669560862 shutdown_connections 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.094+0000 7f0c15a13700 1 -- 172.21.15.33:0/1669560862 wait complete. 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.094+0000 7f0c15a13700 1 Processor -- start 2024-04-25T01:26:32.889 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.094+0000 7f0c15a13700 1 -- start start 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.094+0000 7f0c15a13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c101115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.094+0000 7f0c15a13700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c10111b30 con 0x7f0c10102880 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.094+0000 7f0c0effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c101115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.094+0000 7f0c0effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c101115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44228/0 (socket says 172.21.15.33:44228) 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.094+0000 7f0c0effd700 1 -- 172.21.15.33:0/3316824497 learned_addr learned my addr 172.21.15.33:0/3316824497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.095+0000 7f0c0effd700 1 -- 172.21.15.33:0/3316824497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c000094f0 con 0x7f0c10102880 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.095+0000 7f0c0effd700 1 --2- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c101115f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c00000c00 tx=0x7f0c00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.095+0000 7f0bf77fe700 1 -- 172.21.15.33:0/3316824497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c000107c0 con 0x7f0c10102880 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.095+0000 7f0bf77fe700 1 -- 172.21.15.33:0/3316824497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0c00010db0 con 0x7f0c10102880 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.095+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c10111d30 con 0x7f0c10102880 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.095+0000 7f0bf77fe700 1 -- 172.21.15.33:0/3316824497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c0001ad00 con 0x7f0c10102880 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.095+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c101128a0 con 0x7f0c10102880 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.096+0000 7f0bf77fe700 1 -- 172.21.15.33:0/3316824497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f0c000193c0 con 0x7f0c10102880 2024-04-25T01:26:32.890 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.096+0000 7f0bf77fe700 1 --2- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f0bf8036850 0x7f0bf8038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.096+0000 7f0bf77fe700 1 -- 172.21.15.33:0/3316824497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0c0004a5e0 con 0x7f0c10102880 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.096+0000 7f0c0e7fc700 1 --2- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f0bf8036850 0x7f0bf8038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.096+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1005f5c0 con 0x7f0c10102880 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.097+0000 7f0c0e7fc700 1 --2- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f0bf8036850 0x7f0bf8038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009990 tx=0x7f0c04006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.101+0000 7f0bf77fe700 1 -- 172.21.15.33:0/3316824497 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0c00019670 con 0x7f0c10102880 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.295+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0c1004bed0 con 0x7f0c10102880 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.296+0000 7f0bf77fe700 1 -- 172.21.15.33:0/3316824497 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f0c00022340 con 0x7f0c10102880 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f0bf8036850 msgr2=0x7f0bf8038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 --2- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f0bf8036850 0x7f0bf8038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c04009990 tx=0x7f0c04006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 msgr2=0x7f0c101115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 --2- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c101115f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0c00000c00 tx=0x7f0c00009250 comp rx=0 tx=0).stop 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 shutdown_connections 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 --2- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f0bf8036850 0x7f0bf8038d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 --2- 172.21.15.33:0/3316824497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c10102880 0x7f0c101115f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:32.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 >> 172.21.15.33:0/3316824497 conn(0x7f0c100fe060 msgr2=0x7f0c100feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:32.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 shutdown_connections 2024-04-25T01:26:32.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:32.298+0000 7f0c15a13700 1 -- 172.21.15.33:0/3316824497 wait complete. 2024-04-25T01:26:32.892 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling cephadm module... 2024-04-25T01:26:33.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:33 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2884118783' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T01:26:34.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.330+0000 7fc79af71700 1 Processor -- start 2024-04-25T01:26:34.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.330+0000 7fc79af71700 1 -- start start 2024-04-25T01:26:34.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.331+0000 7fc79af71700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794101b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:34.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.331+0000 7fc79af71700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794102090 con 0x7fc794101740 2024-04-25T01:26:34.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.331+0000 7fc798d0d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794101b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:34.209 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.331+0000 7fc798d0d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794101b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44236/0 (socket says 172.21.15.33:44236) 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.331+0000 7fc798d0d700 1 -- 172.21.15.33:0/3448042277 learned_addr learned my addr 172.21.15.33:0/3448042277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.331+0000 7fc798d0d700 1 -- 172.21.15.33:0/3448042277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7941021d0 con 0x7fc794101740 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.331+0000 7fc798d0d700 1 --2- 172.21.15.33:0/3448042277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794101b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc78800bb80 tx=0x7fc78800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de77eec52ddd8a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.331+0000 7fc7937fe700 1 -- 172.21.15.33:0/3448042277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc78800dd00 con 0x7fc794101740 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.332+0000 7fc7937fe700 1 -- 172.21.15.33:0/3448042277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc78800de60 con 0x7fc794101740 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.332+0000 7fc7937fe700 1 -- 172.21.15.33:0/3448042277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc7880188a0 con 0x7fc794101740 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.332+0000 7fc79af71700 1 -- 172.21.15.33:0/3448042277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 msgr2=0x7fc794101b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.332+0000 7fc79af71700 1 --2- 172.21.15.33:0/3448042277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794101b50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc78800bb80 tx=0x7fc78800b4f0 comp rx=0 tx=0).stop 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.332+0000 7fc79af71700 1 -- 172.21.15.33:0/3448042277 shutdown_connections 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.332+0000 7fc79af71700 1 --2- 172.21.15.33:0/3448042277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794101b50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.332+0000 7fc79af71700 1 -- 172.21.15.33:0/3448042277 >> 172.21.15.33:0/3448042277 conn(0x7fc7940fcec0 msgr2=0x7fc7940ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.332+0000 7fc79af71700 1 -- 172.21.15.33:0/3448042277 shutdown_connections 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.332+0000 7fc79af71700 1 -- 172.21.15.33:0/3448042277 wait complete. 2024-04-25T01:26:34.210 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.333+0000 7fc79af71700 1 Processor -- start 2024-04-25T01:26:34.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.333+0000 7fc79af71700 1 -- start start 2024-04-25T01:26:34.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.333+0000 7fc79af71700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794113740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:34.211 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.333+0000 7fc79af71700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc794113c80 con 0x7fc794101740 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.333+0000 7fc798d0d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794113740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.333+0000 7fc798d0d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794113740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44242/0 (socket says 172.21.15.33:44242) 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.333+0000 7fc798d0d700 1 -- 172.21.15.33:0/2884118783 learned_addr learned my addr 172.21.15.33:0/2884118783 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.333+0000 7fc798d0d700 1 -- 172.21.15.33:0/2884118783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78800b790 con 0x7fc794101740 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.334+0000 7fc798d0d700 1 --2- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794113740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc788010fa0 tx=0x7fc78800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.334+0000 7fc7917fa700 1 -- 172.21.15.33:0/2884118783 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc788018e10 con 0x7fc794101740 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.334+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc794113e80 con 0x7fc794101740 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.334+0000 7fc7917fa700 1 -- 172.21.15.33:0/2884118783 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc788010b80 con 0x7fc794101740 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.334+0000 7fc7917fa700 1 -- 172.21.15.33:0/2884118783 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc788021550 con 0x7fc794101740 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.334+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7941149f0 con 0x7fc794101740 2024-04-25T01:26:34.212 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.335+0000 7fc7917fa700 1 -- 172.21.15.33:0/2884118783 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7fc788021780 con 0x7fc794101740 2024-04-25T01:26:34.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.335+0000 7fc7917fa700 1 --2- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fc784036830 0x7fc784038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:34.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.335+0000 7fc7917fa700 1 -- 172.21.15.33:0/2884118783 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc788014070 con 0x7fc794101740 2024-04-25T01:26:34.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.335+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc79405f5c0 con 0x7fc794101740 2024-04-25T01:26:34.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.335+0000 7fc793fff700 1 --2- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fc784036830 0x7fc784038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:34.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.335+0000 7fc793fff700 1 --2- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fc784036830 0x7fc784038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc780009990 tx=0x7fc780006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:34.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.340+0000 7fc7917fa700 1 -- 172.21.15.33:0/2884118783 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc78800edc0 con 0x7fc794101740 2024-04-25T01:26:34.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.588+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc79404bed0 con 0x7fc794101740 2024-04-25T01:26:34.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.641+0000 7fc7917fa700 1 -- 172.21.15.33:0/2884118783 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fc788022a10 con 0x7fc794101740 2024-04-25T01:26:34.213 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.642+0000 7fc7917fa700 1 -- 172.21.15.33:0/2884118783 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc78800e360 con 0x7fc794101740 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fc784036830 msgr2=0x7fc784038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 --2- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fc784036830 0x7fc784038ce0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc780009990 tx=0x7fc780006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 msgr2=0x7fc794113740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 --2- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794113740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc788010fa0 tx=0x7fc78800b4f0 comp rx=0 tx=0).stop 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 shutdown_connections 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 --2- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7fc784036830 0x7fc784038ce0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 --2- 172.21.15.33:0/2884118783 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc794101740 0x7fc794113740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 >> 172.21.15.33:0/2884118783 conn(0x7fc7940fcec0 msgr2=0x7fc794105b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 shutdown_connections 2024-04-25T01:26:34.214 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:33.644+0000 7fc79af71700 1 -- 172.21.15.33:0/2884118783 wait complete. 2024-04-25T01:26:34.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:34 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2884118783' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T01:26:34.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:34 smithi033 ceph-mon[26353]: mgrmap e5: smithi033.asemba(active, since 4s) 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "smithi033.asemba", 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.682+0000 7f019999c700 1 Processor -- start 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.682+0000 7f019999c700 1 -- start start 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.682+0000 7f019999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f0194102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.682+0000 7f019999c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941031d0 con 0x7f0194102880 2024-04-25T01:26:35.532 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.682+0000 7f0192ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f0194102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.682+0000 7f0192ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f0194102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44280/0 (socket says 172.21.15.33:44280) 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.682+0000 7f0192ffd700 1 -- 172.21.15.33:0/1325325332 learned_addr learned my addr 172.21.15.33:0/1325325332 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.683+0000 7f0192ffd700 1 -- 172.21.15.33:0/1325325332 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0194103310 con 0x7f0194102880 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.683+0000 7f0192ffd700 1 --2- 172.21.15.33:0/1325325332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f0194102c90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f01840098e0 tx=0x7f0184009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d535221fd1e79be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.683+0000 7f0191ffb700 1 -- 172.21.15.33:0/1325325332 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f018400b860 con 0x7f0194102880 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.683+0000 7f0191ffb700 1 -- 172.21.15.33:0/1325325332 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f018400be50 con 0x7f0194102880 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.683+0000 7f0191ffb700 1 -- 172.21.15.33:0/1325325332 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0184010380 con 0x7f0194102880 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.684+0000 7f019999c700 1 -- 172.21.15.33:0/1325325332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 msgr2=0x7f0194102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.684+0000 7f019999c700 1 --2- 172.21.15.33:0/1325325332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f0194102c90 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f01840098e0 tx=0x7f0184009250 comp rx=0 tx=0).stop 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.684+0000 7f019999c700 1 -- 172.21.15.33:0/1325325332 shutdown_connections 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.684+0000 7f019999c700 1 --2- 172.21.15.33:0/1325325332 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f0194102c90 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.684+0000 7f019999c700 1 -- 172.21.15.33:0/1325325332 >> 172.21.15.33:0/1325325332 conn(0x7f01940fe060 msgr2=0x7f0194100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.684+0000 7f019999c700 1 -- 172.21.15.33:0/1325325332 shutdown_connections 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.684+0000 7f019999c700 1 -- 172.21.15.33:0/1325325332 wait complete. 2024-04-25T01:26:35.533 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.684+0000 7f019999c700 1 Processor -- start 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.684+0000 7f019999c700 1 -- start start 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f019999c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f01941115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f019999c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0194111b30 con 0x7f0194102880 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f0192ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f01941115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f0192ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f01941115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44282/0 (socket says 172.21.15.33:44282) 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f0192ffd700 1 -- 172.21.15.33:0/1548491988 learned_addr learned my addr 172.21.15.33:0/1548491988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f0192ffd700 1 -- 172.21.15.33:0/1548491988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01840094f0 con 0x7f0194102880 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f0192ffd700 1 --2- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f01941115f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f0184009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f018b7fe700 1 -- 172.21.15.33:0/1548491988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f01840107c0 con 0x7f0194102880 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f018b7fe700 1 -- 172.21.15.33:0/1548491988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f0184010db0 con 0x7f0194102880 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0194111d30 con 0x7f0194102880 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.685+0000 7f018b7fe700 1 -- 172.21.15.33:0/1548491988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f018401ad00 con 0x7f0194102880 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.686+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01941128a0 con 0x7f0194102880 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.686+0000 7f018b7fe700 1 -- 172.21.15.33:0/1548491988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f01840193c0 con 0x7f0194102880 2024-04-25T01:26:35.534 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.687+0000 7f018b7fe700 1 --2- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f01800368a0 0x7f0180038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.687+0000 7f018b7fe700 1 -- 172.21.15.33:0/1548491988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f018404a780 con 0x7f0194102880 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.687+0000 7f01927fc700 1 -- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f01800368a0 msgr2=0x7f0180038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4148578452 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.687+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019405f5c0 con 0x7f0194102880 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.687+0000 7f01927fc700 1 --2- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f01800368a0 0x7f0180038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.692+0000 7f018b7fe700 1 -- 172.21.15.33:0/1548491988 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f01840196c0 con 0x7f0194102880 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.888+0000 7f01927fc700 1 -- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f01800368a0 msgr2=0x7f0180038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4148578452 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.888+0000 7f01927fc700 1 --2- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f01800368a0 0x7f0180038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.984+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0194112b20 con 0x7f0194102880 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.984+0000 7f018b7fe700 1 -- 172.21.15.33:0/1548491988 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f0184031030 con 0x7f0194102880 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.986+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f01800368a0 msgr2=0x7f0180038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.986+0000 7f019999c700 1 --2- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f01800368a0 0x7f0180038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.986+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 msgr2=0x7f01941115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.986+0000 7f019999c700 1 --2- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f01941115f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0184000c00 tx=0x7f0184009250 comp rx=0 tx=0).stop 2024-04-25T01:26:35.535 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.986+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 shutdown_connections 2024-04-25T01:26:35.536 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.986+0000 7f019999c700 1 --2- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f01800368a0 0x7f0180038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:35.536 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.986+0000 7f019999c700 1 --2- 172.21.15.33:0/1548491988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102880 0x7f01941115f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:35.536 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.986+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 >> 172.21.15.33:0/1548491988 conn(0x7f01940fe060 msgr2=0x7f01940feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:35.536 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.987+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 shutdown_connections 2024-04-25T01:26:35.536 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:34.987+0000 7f019999c700 1 -- 172.21.15.33:0/1548491988 wait complete. 2024-04-25T01:26:35.536 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-04-25T01:26:35.536 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 5... 2024-04-25T01:26:35.892 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:35 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1548491988' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T01:26:40.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: Active manager daemon smithi033.asemba restarted 2024-04-25T01:26:40.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: Activating manager daemon smithi033.asemba 2024-04-25T01:26:40.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T01:26:40.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: mgrmap e6: smithi033.asemba(active, starting, since 0.00331478s) 2024-04-25T01:26:40.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:26:40.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:26:40.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:26:40.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:26:40.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:26:40.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: Manager daemon smithi033.asemba is now available 2024-04-25T01:26:40.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:40.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:40.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:26:40.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:26:40.748 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:26:40.748 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:26:40.748 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:40 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/trash_purge_schedule"}]: dispatch 2024-04-25T01:26:41.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:41 smithi033 ceph-mon[26353]: Found migration_current of "None". Setting to last migration. 2024-04-25T01:26:41.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:41 smithi033 ceph-mon[26353]: mgrmap e7: smithi033.asemba(active, since 1.00521s) 2024-04-25T01:26:42.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:26:42.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T01:26:42.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T01:26:42.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:26:42.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.016+0000 7f44cd3c8700 1 Processor -- start 2024-04-25T01:26:42.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.016+0000 7f44cd3c8700 1 -- start start 2024-04-25T01:26:42.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.016+0000 7f44cd3c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8102c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:42.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.016+0000 7f44cd3c8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c8103150 con 0x7f44c8102800 2024-04-25T01:26:42.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.016+0000 7f44c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8102c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.016+0000 7f44c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8102c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44946/0 (socket says 172.21.15.33:44946) 2024-04-25T01:26:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.016+0000 7f44c6ffd700 1 -- 172.21.15.33:0/3500194137 learned_addr learned my addr 172.21.15.33:0/3500194137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.017+0000 7f44c6ffd700 1 -- 172.21.15.33:0/3500194137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c8103290 con 0x7f44c8102800 2024-04-25T01:26:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.017+0000 7f44c6ffd700 1 --2- 172.21.15.33:0/3500194137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8102c10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f44b80098e0 tx=0x7f44b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31444876aed24b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.017+0000 7f44c5ffb700 1 -- 172.21.15.33:0/3500194137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44b800b860 con 0x7f44c8102800 2024-04-25T01:26:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.017+0000 7f44c5ffb700 1 -- 172.21.15.33:0/3500194137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f44b800be50 con 0x7f44c8102800 2024-04-25T01:26:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.017+0000 7f44c5ffb700 1 -- 172.21.15.33:0/3500194137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44b8010380 con 0x7f44c8102800 2024-04-25T01:26:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.018+0000 7f44cd3c8700 1 -- 172.21.15.33:0/3500194137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 msgr2=0x7f44c8102c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:42.034 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.018+0000 7f44cd3c8700 1 --2- 172.21.15.33:0/3500194137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8102c10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f44b80098e0 tx=0x7f44b8009250 comp rx=0 tx=0).stop 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.018+0000 7f44cd3c8700 1 -- 172.21.15.33:0/3500194137 shutdown_connections 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.018+0000 7f44cd3c8700 1 --2- 172.21.15.33:0/3500194137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8102c10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.018+0000 7f44cd3c8700 1 -- 172.21.15.33:0/3500194137 >> 172.21.15.33:0/3500194137 conn(0x7f44c80fdf80 msgr2=0x7f44c8100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.018+0000 7f44cd3c8700 1 -- 172.21.15.33:0/3500194137 shutdown_connections 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.018+0000 7f44cd3c8700 1 -- 172.21.15.33:0/3500194137 wait complete. 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.018+0000 7f44cd3c8700 1 Processor -- start 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.019+0000 7f44cd3c8700 1 -- start start 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.019+0000 7f44cd3c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8111550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.019+0000 7f44cd3c8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c8111a90 con 0x7f44c8102800 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.019+0000 7f44c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8111550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.019+0000 7f44c6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8111550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44952/0 (socket says 172.21.15.33:44952) 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.019+0000 7f44c6ffd700 1 -- 172.21.15.33:0/2359323502 learned_addr learned my addr 172.21.15.33:0/2359323502 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:42.035 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.019+0000 7f44c6ffd700 1 -- 172.21.15.33:0/2359323502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b80094f0 con 0x7f44c8102800 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.019+0000 7f44c6ffd700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8111550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f44b800b440 tx=0x7f44b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.019+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44b8010760 con 0x7f44c8102800 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.020+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f44b8010d50 con 0x7f44c8102800 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.020+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44b801ae20 con 0x7f44c8102800 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.020+0000 7f44cd3c8700 1 -- 172.21.15.33:0/2359323502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c8111c90 con 0x7f44c8102800 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.020+0000 7f44cd3c8700 1 -- 172.21.15.33:0/2359323502 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c8112800 con 0x7f44c8102800 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.021+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f44b802e440 con 0x7f44c8102800 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.021+0000 7f44bf7fe700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 0x7f44b4038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.021+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 --> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44b40393f0 con 0x7f44b4036830 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.021+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f44b804a5b0 con 0x7f44c8102800 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.021+0000 7f44c67fc700 1 -- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 msgr2=0x7f44b4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4148578452 2024-04-25T01:26:42.036 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.021+0000 7f44c67fc700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 0x7f44b4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.222+0000 7f44c67fc700 1 -- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 msgr2=0x7f44b4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4148578452 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.222+0000 7f44c67fc700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 0x7f44b4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.623+0000 7f44c67fc700 1 -- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 msgr2=0x7f44b4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4148578452 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:36.623+0000 7f44c67fc700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 0x7f44b4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:37.424+0000 7f44c67fc700 1 -- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 msgr2=0x7f44b4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4148578452 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:37.424+0000 7f44c67fc700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 0x7f44b4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:39.025+0000 7f44c67fc700 1 -- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 msgr2=0x7f44b4038ce0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/4148578452 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:39.025+0000 7f44c67fc700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 0x7f44b4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:40.397+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7f44b802e790 con 0x7f44c8102800 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:40.398+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 msgr2=0x7f44b4038ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:40.398+0000 7f44bf7fe700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 0x7f44b4038ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.398+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f44b804bfc0 con 0x7f44c8102800 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.398+0000 7f44bf7fe700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f44b4039fc0 0x7f44b403c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:42.037 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.398+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f44b40393f0 con 0x7f44b4039fc0 2024-04-25T01:26:42.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.399+0000 7f44c67fc700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f44b4039fc0 0x7f44b403c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:42.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.400+0000 7f44c67fc700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f44b4039fc0 0x7f44b403c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f44b00039d0 tx=0x7f44b0009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:42.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.400+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f44b40393f0 con 0x7f44b4039fc0 2024-04-25T01:26:42.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.406+0000 7f44cd3c8700 1 -- 172.21.15.33:0/2359323502 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f44c8112120 con 0x7f44b4039fc0 2024-04-25T01:26:42.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44bf7fe700 1 -- 172.21.15.33:0/2359323502 <== mgr.14118 v2:172.21.15.33:6800/3376891250 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f44c8112120 con 0x7f44b4039fc0 2024-04-25T01:26:42.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 -- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f44b4039fc0 msgr2=0x7f44b403c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:42.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f44b4039fc0 0x7f44b403c3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f44b00039d0 tx=0x7f44b0009350 comp rx=0 tx=0).stop 2024-04-25T01:26:42.038 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 -- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 msgr2=0x7f44c8111550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8111550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f44b800b440 tx=0x7f44b8009250 comp rx=0 tx=0).stop 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 -- 172.21.15.33:0/2359323502 shutdown_connections 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f44b4039fc0 0x7f44b403c3a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44c8102800 0x7f44c8111550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 --2- 172.21.15.33:0/2359323502 >> [v2:172.21.15.33:6800/4148578452,v1:172.21.15.33:6801/4148578452] conn(0x7f44b4036830 0x7f44b4038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 -- 172.21.15.33:0/2359323502 >> 172.21.15.33:0/2359323502 conn(0x7f44c80fdf80 msgr2=0x7f44c80fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 -- 172.21.15.33:0/2359323502 shutdown_connections 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:41.407+0000 7f44cd3c8700 1 -- 172.21.15.33:0/2359323502 wait complete. 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 5 is available 2024-04-25T01:26:42.039 INFO:teuthology.orchestra.run.smithi033.stdout:Setting orchestrator backend to cephadm... 2024-04-25T01:26:42.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:42 smithi033 ceph-mon[26353]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T01:26:42.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:42 smithi033 ceph-mon[26353]: [25/Apr/2024:01:26:41] ENGINE Bus STARTING 2024-04-25T01:26:42.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:42 smithi033 ceph-mon[26353]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T01:26:42.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:42 smithi033 ceph-mon[26353]: [25/Apr/2024:01:26:41] ENGINE Serving on https://172.21.15.33:7150 2024-04-25T01:26:42.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:42 smithi033 ceph-mon[26353]: [25/Apr/2024:01:26:41] ENGINE Bus STARTED 2024-04-25T01:26:42.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:42 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:42.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:42 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:26:43.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.489+0000 7f34d3c8d700 1 Processor -- start 2024-04-25T01:26:43.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.489+0000 7f34d3c8d700 1 -- start start 2024-04-25T01:26:43.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.489+0000 7f34d3c8d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc105420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:43.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.489+0000 7f34d3c8d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc105960 con 0x7f34cc076c30 2024-04-25T01:26:43.219 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.489+0000 7f34d1a29700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc105420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.489+0000 7f34d1a29700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc105420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45010/0 (socket says 172.21.15.33:45010) 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.489+0000 7f34d1a29700 1 -- 172.21.15.33:0/3568453437 learned_addr learned my addr 172.21.15.33:0/3568453437 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.489+0000 7f34d1a29700 1 -- 172.21.15.33:0/3568453437 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc105aa0 con 0x7f34cc076c30 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.490+0000 7f34d1a29700 1 --2- 172.21.15.33:0/3568453437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc105420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f34c80098e0 tx=0x7f34c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45eb782db4c7f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.490+0000 7f34d0a27700 1 -- 172.21.15.33:0/3568453437 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34c800b860 con 0x7f34cc076c30 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.490+0000 7f34d0a27700 1 -- 172.21.15.33:0/3568453437 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f34c800be50 con 0x7f34cc076c30 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.490+0000 7f34d0a27700 1 -- 172.21.15.33:0/3568453437 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34c8010380 con 0x7f34cc076c30 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.490+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3568453437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 msgr2=0x7f34cc105420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.490+0000 7f34d3c8d700 1 --2- 172.21.15.33:0/3568453437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc105420 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f34c80098e0 tx=0x7f34c8009250 comp rx=0 tx=0).stop 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.490+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3568453437 shutdown_connections 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.490+0000 7f34d3c8d700 1 --2- 172.21.15.33:0/3568453437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc105420 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.490+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3568453437 >> 172.21.15.33:0/3568453437 conn(0x7f34cc0fcef0 msgr2=0x7f34cc0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.491+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3568453437 shutdown_connections 2024-04-25T01:26:43.220 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.491+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3568453437 wait complete. 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.491+0000 7f34d3c8d700 1 Processor -- start 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.491+0000 7f34d3c8d700 1 -- start start 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.491+0000 7f34d3c8d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc1115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.491+0000 7f34d3c8d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc111b20 con 0x7f34cc076c30 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.491+0000 7f34d1a29700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc1115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.491+0000 7f34d1a29700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc1115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45026/0 (socket says 172.21.15.33:45026) 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.491+0000 7f34d1a29700 1 -- 172.21.15.33:0/3630750949 learned_addr learned my addr 172.21.15.33:0/3630750949 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.492+0000 7f34d1a29700 1 -- 172.21.15.33:0/3630750949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c80094f0 con 0x7f34cc076c30 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.492+0000 7f34d1a29700 1 --2- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc1115e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f34c8003050 tx=0x7f34c8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.492+0000 7f34c27fc700 1 -- 172.21.15.33:0/3630750949 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34c8010760 con 0x7f34cc076c30 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.492+0000 7f34c27fc700 1 -- 172.21.15.33:0/3630750949 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f34c8010d50 con 0x7f34cc076c30 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.492+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc111d20 con 0x7f34cc076c30 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.492+0000 7f34c27fc700 1 -- 172.21.15.33:0/3630750949 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34c801ae20 con 0x7f34cc076c30 2024-04-25T01:26:43.221 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.492+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34cc112890 con 0x7f34cc076c30 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.493+0000 7f34c27fc700 1 -- 172.21.15.33:0/3630750949 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f34c802e440 con 0x7f34cc076c30 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.493+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34cc05f5c0 con 0x7f34cc076c30 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.493+0000 7f34c27fc700 1 --2- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f34b8036720 0x7f34b8038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.493+0000 7f34c27fc700 1 -- 172.21.15.33:0/3630750949 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f34c804a510 con 0x7f34cc076c30 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.493+0000 7f34d1228700 1 --2- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f34b8036720 0x7f34b8038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.494+0000 7f34d1228700 1 --2- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f34b8036720 0x7f34b8038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009990 tx=0x7f34bc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.498+0000 7f34c27fc700 1 -- 172.21.15.33:0/3630750949 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f34c8006d70 con 0x7f34cc076c30 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.530+0000 7f34c27fc700 1 -- 172.21.15.33:0/3630750949 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f34c802d3a0 con 0x7f34cc076c30 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.708+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f34cc0fe0b0 con 0x7f34b8036720 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.717+0000 7f34c27fc700 1 -- 172.21.15.33:0/3630750949 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f34cc0fe0b0 con 0x7f34b8036720 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f34b8036720 msgr2=0x7f34b8038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 --2- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f34b8036720 0x7f34b8038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009990 tx=0x7f34bc006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:43.222 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 msgr2=0x7f34cc1115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:43.223 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 --2- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc1115e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f34c8003050 tx=0x7f34c8009250 comp rx=0 tx=0).stop 2024-04-25T01:26:43.223 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 shutdown_connections 2024-04-25T01:26:43.223 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 --2- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f34b8036720 0x7f34b8038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:43.223 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 --2- 172.21.15.33:0/3630750949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f34cc076c30 0x7f34cc1115e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:43.223 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 >> 172.21.15.33:0/3630750949 conn(0x7f34cc0fcef0 msgr2=0x7f34cc0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:43.223 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 shutdown_connections 2024-04-25T01:26:43.223 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:42.719+0000 7f34d3c8d700 1 -- 172.21.15.33:0/3630750949 wait complete. 2024-04-25T01:26:43.596 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:43 smithi033 ceph-mon[26353]: mgrmap e8: smithi033.asemba(active, since 2s) 2024-04-25T01:26:43.596 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:43 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:43.596 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:43 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.616+0000 7f7fb1bcb700 1 Processor -- start 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.616+0000 7f7fb1bcb700 1 -- start start 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.616+0000 7f7fb1bcb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac105480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.616+0000 7f7fb1bcb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac1059c0 con 0x7f7fac076bf0 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.617+0000 7f7fab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac105480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.617+0000 7f7fab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac105480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45042/0 (socket says 172.21.15.33:45042) 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.617+0000 7f7fab7fe700 1 -- 172.21.15.33:0/2592637861 learned_addr learned my addr 172.21.15.33:0/2592637861 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.617+0000 7f7fab7fe700 1 -- 172.21.15.33:0/2592637861 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac105b00 con 0x7f7fac076bf0 2024-04-25T01:26:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.618+0000 7f7fab7fe700 1 --2- 172.21.15.33:0/2592637861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac105480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009e50 tx=0x7f7f9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c620d2ce915ff6e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.618+0000 7f7faa7fc700 1 -- 172.21.15.33:0/2592637861 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f9c0092e0 con 0x7f7fac076bf0 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.618+0000 7f7faa7fc700 1 -- 172.21.15.33:0/2592637861 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7f9c012430 con 0x7f7fac076bf0 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.618+0000 7f7faa7fc700 1 -- 172.21.15.33:0/2592637861 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f9c0075c0 con 0x7f7fac076bf0 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.619+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2592637861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 msgr2=0x7f7fac105480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.619+0000 7f7fb1bcb700 1 --2- 172.21.15.33:0/2592637861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac105480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009e50 tx=0x7f7f9c009840 comp rx=0 tx=0).stop 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.619+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2592637861 shutdown_connections 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.619+0000 7f7fb1bcb700 1 --2- 172.21.15.33:0/2592637861 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac105480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.619+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2592637861 >> 172.21.15.33:0/2592637861 conn(0x7f7fac0fcef0 msgr2=0x7f7fac0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.619+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2592637861 shutdown_connections 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.619+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2592637861 wait complete. 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.619+0000 7f7fb1bcb700 1 Processor -- start 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.620+0000 7f7fb1bcb700 1 -- start start 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.620+0000 7f7fb1bcb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac111640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.620+0000 7f7fb1bcb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac111b80 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.620+0000 7f7fab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.620+0000 7f7fab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac111640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45050/0 (socket says 172.21.15.33:45050) 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.620+0000 7f7fab7fe700 1 -- 172.21.15.33:0/2912898930 learned_addr learned my addr 172.21.15.33:0/2912898930 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.620+0000 7f7fab7fe700 1 -- 172.21.15.33:0/2912898930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c009ae0 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.621+0000 7f7fab7fe700 1 --2- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac111640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009fd0 tx=0x7f7f9c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.621+0000 7f7f93fff700 1 -- 172.21.15.33:0/2912898930 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f9c0092e0 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.621+0000 7f7f93fff700 1 -- 172.21.15.33:0/2912898930 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7f9c007cc0 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.621+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fac111d80 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.621+0000 7f7f93fff700 1 -- 172.21.15.33:0/2912898930 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f9c01c880 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.621+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac1128f0 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.622+0000 7f7f93fff700 1 -- 172.21.15.33:0/2912898930 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f7f9c019070 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.622+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fac05f5c0 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.622+0000 7f7f93fff700 1 --2- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f7f940368a0 0x7f7f94038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.622+0000 7f7f93fff700 1 -- 172.21.15.33:0/2912898930 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f7f9c049dd0 con 0x7f7fac076bf0 2024-04-25T01:26:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.622+0000 7f7faaffd700 1 --2- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f7f940368a0 0x7f7f94038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.623+0000 7f7faaffd700 1 --2- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f7f940368a0 0x7f7f94038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009990 tx=0x7f7fa0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.627+0000 7f7f93fff700 1 -- 172.21.15.33:0/2912898930 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7f9c04d030 con 0x7f7fac076bf0 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.824+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fac0fe130 con 0x7f7f940368a0 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.824+0000 7f7f93fff700 1 -- 172.21.15.33:0/2912898930 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f7fac0fe130 con 0x7f7f940368a0 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f7f940368a0 msgr2=0x7f7f94038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 --2- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f7f940368a0 0x7f7f94038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009990 tx=0x7f7fa0006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 msgr2=0x7f7fac111640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 --2- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac111640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009fd0 tx=0x7f7f9c009840 comp rx=0 tx=0).stop 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 shutdown_connections 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 --2- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f7f940368a0 0x7f7f94038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 --2- 172.21.15.33:0/2912898930 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fac076bf0 0x7f7fac111640 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 >> 172.21.15.33:0/2912898930 conn(0x7f7fac0fcef0 msgr2=0x7f7fac0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 shutdown_connections 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:43.826+0000 7f7fb1bcb700 1 -- 172.21.15.33:0/2912898930 wait complete. 2024-04-25T01:26:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:Generating ssh key... 2024-04-25T01:26:44.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:44 smithi033 ceph-mon[26353]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:45.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:45 smithi033 ceph-mon[26353]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:45.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:45 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:45.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:45 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.871+0000 7f2239884700 1 Processor -- start 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.871+0000 7f2239884700 1 -- start start 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.871+0000 7f2239884700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f2234102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.871+0000 7f2239884700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22341031c0 con 0x7f2234102870 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.872+0000 7f2232ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f2234102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.872+0000 7f2232ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f2234102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45052/0 (socket says 172.21.15.33:45052) 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.872+0000 7f2232ffd700 1 -- 172.21.15.33:0/1620227676 learned_addr learned my addr 172.21.15.33:0/1620227676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.872+0000 7f2232ffd700 1 -- 172.21.15.33:0/1620227676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2234103300 con 0x7f2234102870 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.872+0000 7f2232ffd700 1 --2- 172.21.15.33:0/1620227676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f2234102c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f222c009e50 tx=0x7f222c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3c27122d61c10a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.872+0000 7f2231ffb700 1 -- 172.21.15.33:0/1620227676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f222c0092e0 con 0x7f2234102870 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.872+0000 7f2231ffb700 1 -- 172.21.15.33:0/1620227676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f222c012430 con 0x7f2234102870 2024-04-25T01:26:45.698 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.872+0000 7f2231ffb700 1 -- 172.21.15.33:0/1620227676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f222c0075c0 con 0x7f2234102870 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.873+0000 7f2239884700 1 -- 172.21.15.33:0/1620227676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 msgr2=0x7f2234102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.873+0000 7f2239884700 1 --2- 172.21.15.33:0/1620227676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f2234102c80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f222c009e50 tx=0x7f222c009840 comp rx=0 tx=0).stop 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.873+0000 7f2239884700 1 -- 172.21.15.33:0/1620227676 shutdown_connections 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.873+0000 7f2239884700 1 --2- 172.21.15.33:0/1620227676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f2234102c80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.873+0000 7f2239884700 1 -- 172.21.15.33:0/1620227676 >> 172.21.15.33:0/1620227676 conn(0x7f22340fe030 msgr2=0x7f2234100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.873+0000 7f2239884700 1 -- 172.21.15.33:0/1620227676 shutdown_connections 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.873+0000 7f2239884700 1 -- 172.21.15.33:0/1620227676 wait complete. 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.874+0000 7f2239884700 1 Processor -- start 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.874+0000 7f2239884700 1 -- start start 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.874+0000 7f2239884700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f22341115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.874+0000 7f2239884700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2234111b00 con 0x7f2234102870 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.874+0000 7f2232ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f22341115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.874+0000 7f2232ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f22341115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45060/0 (socket says 172.21.15.33:45060) 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.874+0000 7f2232ffd700 1 -- 172.21.15.33:0/1807389598 learned_addr learned my addr 172.21.15.33:0/1807389598 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:45.699 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.874+0000 7f2232ffd700 1 -- 172.21.15.33:0/1807389598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c009ae0 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.874+0000 7f2232ffd700 1 --2- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f22341115c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f222c009ab0 tx=0x7f222c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.875+0000 7f222b7fe700 1 -- 172.21.15.33:0/1807389598 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f222c0092e0 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.875+0000 7f222b7fe700 1 -- 172.21.15.33:0/1807389598 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f222c007d10 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.875+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2234111d00 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.875+0000 7f222b7fe700 1 -- 172.21.15.33:0/1807389598 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f222c01c990 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.875+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2234112870 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.876+0000 7f222b7fe700 1 -- 172.21.15.33:0/1807389598 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f222c019070 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.876+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223405f5c0 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.876+0000 7f222b7fe700 1 --2- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f2220036850 0x7f2220038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.876+0000 7f222b7fe700 1 -- 172.21.15.33:0/1807389598 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f222c049f80 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.876+0000 7f22327fc700 1 --2- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f2220036850 0x7f2220038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.876+0000 7f22327fc700 1 --2- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f2220036850 0x7f2220038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f221c009990 tx=0x7f221c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:44.881+0000 7f222b7fe700 1 -- 172.21.15.33:0/1807389598 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f222c019910 con 0x7f2234102870 2024-04-25T01:26:45.700 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.078+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f22340ff1d0 con 0x7f2220036850 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.164+0000 7f222b7fe700 1 -- 172.21.15.33:0/1807389598 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f22340ff1d0 con 0x7f2220036850 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f2220036850 msgr2=0x7f2220038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 --2- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f2220036850 0x7f2220038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f221c009990 tx=0x7f221c006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 msgr2=0x7f22341115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 --2- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f22341115c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f222c009ab0 tx=0x7f222c009840 comp rx=0 tx=0).stop 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 shutdown_connections 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 --2- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f2220036850 0x7f2220038d00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 --2- 172.21.15.33:0/1807389598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2234102870 0x7f22341115c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 >> 172.21.15.33:0/1807389598 conn(0x7f22340fe030 msgr2=0x7f22340feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 shutdown_connections 2024-04-25T01:26:45.701 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:45.166+0000 7f2239884700 1 -- 172.21.15.33:0/1807389598 wait complete. 2024-04-25T01:26:46.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:46 smithi033 ceph-mon[26353]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:46.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:46 smithi033 ceph-mon[26353]: Generating ssh key... 2024-04-25T01:26:46.854 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5OwZBkWrg2DE5htToC59jXuSLRSt7WxyiLLwpS/l5uAkojIbAw5kYefVeR61Eckc0zciNGbm50ALIGZObQXeh58TBwgdXjeiumtkMzUzDnZOKhMpAhiEHkWsHkF4mscPDzn/oKHm/AAqf6xh7dNkfM2CyhFk5HEydyMp0p2wLyWBD79DDMihjBJ30lzD0HT6f+J6ndRHgSX4gTrcV/iCeHn0ZPLEsOdcEWzp4VJxZL5VZzZvYFL4kmEHNtKLkhxl5yCrLbNURSyBSnK74BDpcRhHxeeOsnsPKWh71A3ZzFGGavccQgnpjv+oQ3d2m7wx+PsibCFY6n4jraXLiufM2SjNB2OZlMrvPMcMihwTVq0hbkHFcHaEUbtKSSS36kXeDjfisXlAwPI2jnSzb9GxZVBCiT5aLFz31qs+rvN81dxog6wgF3mlkz3QCepRTCJp67PblwEmnFlbZ8CdKjgDffElAyqGBCPmzkL6YsOw2/iqmzAupnIfNor5p8lAOs40= ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:26:46.854 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.153+0000 7fd82275e700 1 Processor -- start 2024-04-25T01:26:46.854 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.153+0000 7fd82275e700 1 -- start start 2024-04-25T01:26:46.854 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.154+0000 7fd82275e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.154+0000 7fd82275e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c105970 con 0x7fd81c0712f0 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.154+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.154+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51762/0 (socket says 172.21.15.33:51762) 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.154+0000 7fd81bfff700 1 -- 172.21.15.33:0/1487071030 learned_addr learned my addr 172.21.15.33:0/1487071030 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.154+0000 7fd81bfff700 1 -- 172.21.15.33:0/1487071030 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c105ab0 con 0x7fd81c0712f0 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.154+0000 7fd81bfff700 1 --2- 172.21.15.33:0/1487071030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c105430 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0098e0 tx=0x7fd80c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6efa3cec255251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.154+0000 7fd81affd700 1 -- 172.21.15.33:0/1487071030 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd80c00b8f0 con 0x7fd81c0712f0 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.155+0000 7fd81affd700 1 -- 172.21.15.33:0/1487071030 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd80c011420 con 0x7fd81c0712f0 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.155+0000 7fd81affd700 1 -- 172.21.15.33:0/1487071030 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd80c010540 con 0x7fd81c0712f0 2024-04-25T01:26:46.855 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.155+0000 7fd82275e700 1 -- 172.21.15.33:0/1487071030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 msgr2=0x7fd81c105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.155+0000 7fd82275e700 1 --2- 172.21.15.33:0/1487071030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c105430 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fd80c0098e0 tx=0x7fd80c009250 comp rx=0 tx=0).stop 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.155+0000 7fd82275e700 1 -- 172.21.15.33:0/1487071030 shutdown_connections 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.155+0000 7fd82275e700 1 --2- 172.21.15.33:0/1487071030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c105430 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.155+0000 7fd82275e700 1 -- 172.21.15.33:0/1487071030 >> 172.21.15.33:0/1487071030 conn(0x7fd81c0fcec0 msgr2=0x7fd81c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.155+0000 7fd82275e700 1 -- 172.21.15.33:0/1487071030 shutdown_connections 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.155+0000 7fd82275e700 1 -- 172.21.15.33:0/1487071030 wait complete. 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.156+0000 7fd82275e700 1 Processor -- start 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.156+0000 7fd82275e700 1 -- start start 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.156+0000 7fd82275e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c111650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.156+0000 7fd82275e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c111b90 con 0x7fd81c0712f0 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.156+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.156+0000 7fd81bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c111650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51766/0 (socket says 172.21.15.33:51766) 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.156+0000 7fd81bfff700 1 -- 172.21.15.33:0/3185220668 learned_addr learned my addr 172.21.15.33:0/3185220668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:46.856 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.157+0000 7fd81bfff700 1 -- 172.21.15.33:0/3185220668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c0094f0 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.157+0000 7fd81bfff700 1 --2- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c111650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd80c000c00 tx=0x7fd80c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.157+0000 7fd818ff9700 1 -- 172.21.15.33:0/3185220668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd80c0107b0 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.157+0000 7fd818ff9700 1 -- 172.21.15.33:0/3185220668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd80c010da0 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.157+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c111d90 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.157+0000 7fd818ff9700 1 -- 172.21.15.33:0/3185220668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd80c01add0 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.157+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c112900 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.158+0000 7fd818ff9700 1 -- 172.21.15.33:0/3185220668 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fd80c02e440 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.158+0000 7fd818ff9700 1 --2- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd804036850 0x7fd804038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.158+0000 7fd818ff9700 1 -- 172.21.15.33:0/3185220668 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd80c04ac90 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.158+0000 7fd81b7fe700 1 --2- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd804036850 0x7fd804038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.158+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c05f5c0 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.159+0000 7fd81b7fe700 1 --2- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd804036850 0x7fd804038d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd810009990 tx=0x7fd810006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.163+0000 7fd818ff9700 1 -- 172.21.15.33:0/3185220668 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd80c02e790 con 0x7fd81c0712f0 2024-04-25T01:26:46.857 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.359+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd81c0fe190 con 0x7fd804036850 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.360+0000 7fd818ff9700 1 -- 172.21.15.33:0/3185220668 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7fd81c0fe190 con 0x7fd804036850 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.362+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd804036850 msgr2=0x7fd804038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.362+0000 7fd82275e700 1 --2- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd804036850 0x7fd804038d00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd810009990 tx=0x7fd810006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.362+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 msgr2=0x7fd81c111650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.362+0000 7fd82275e700 1 --2- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c111650 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd80c000c00 tx=0x7fd80c009250 comp rx=0 tx=0).stop 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.362+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 shutdown_connections 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.362+0000 7fd82275e700 1 --2- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd804036850 0x7fd804038d00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.363+0000 7fd82275e700 1 --2- 172.21.15.33:0/3185220668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0712f0 0x7fd81c111650 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.363+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 >> 172.21.15.33:0/3185220668 conn(0x7fd81c0fcec0 msgr2=0x7fd81c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.363+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 shutdown_connections 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:46.363+0000 7fd82275e700 1 -- 172.21.15.33:0/3185220668 wait complete. 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T01:26:46.858 INFO:teuthology.orchestra.run.smithi033.stdout:Adding host smithi033... 2024-04-25T01:26:47.808 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:47 smithi033 ceph-mon[26353]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:48.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:48 smithi033 ceph-mon[26353]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:49.637 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:49 smithi033 ceph-mon[26353]: Deploying cephadm binary to smithi033 2024-04-25T01:26:49.637 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:49 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:49.637 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:49 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:26:49.912 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Added host 'smithi033' with addr '172.21.15.33' 2024-04-25T01:26:49.912 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.317+0000 7f338bc78700 1 Processor -- start 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.317+0000 7f338bc78700 1 -- start start 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f338bc78700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f3384102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f338bc78700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33841031d0 con 0x7f3384102880 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f3389a14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f3384102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f3389a14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f3384102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51774/0 (socket says 172.21.15.33:51774) 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f3389a14700 1 -- 172.21.15.33:0/3609186026 learned_addr learned my addr 172.21.15.33:0/3609186026 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f3389a14700 1 -- 172.21.15.33:0/3609186026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3384103310 con 0x7f3384102880 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f3389a14700 1 --2- 172.21.15.33:0/3609186026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f3384102c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f33800098e0 tx=0x7f3380009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce8e59ca0d1bb6c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f3388a12700 1 -- 172.21.15.33:0/3609186026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f338000b860 con 0x7f3384102880 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f3388a12700 1 -- 172.21.15.33:0/3609186026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f338000be50 con 0x7f3384102880 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.318+0000 7f3388a12700 1 -- 172.21.15.33:0/3609186026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3380010380 con 0x7f3384102880 2024-04-25T01:26:49.913 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.319+0000 7f338bc78700 1 -- 172.21.15.33:0/3609186026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 msgr2=0x7f3384102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.319+0000 7f338bc78700 1 --2- 172.21.15.33:0/3609186026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f3384102c90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f33800098e0 tx=0x7f3380009250 comp rx=0 tx=0).stop 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.319+0000 7f338bc78700 1 -- 172.21.15.33:0/3609186026 shutdown_connections 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.319+0000 7f338bc78700 1 --2- 172.21.15.33:0/3609186026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f3384102c90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.319+0000 7f338bc78700 1 -- 172.21.15.33:0/3609186026 >> 172.21.15.33:0/3609186026 conn(0x7f33840fe060 msgr2=0x7f3384100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.319+0000 7f338bc78700 1 -- 172.21.15.33:0/3609186026 shutdown_connections 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.319+0000 7f338bc78700 1 -- 172.21.15.33:0/3609186026 wait complete. 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.320+0000 7f338bc78700 1 Processor -- start 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.320+0000 7f338bc78700 1 -- start start 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.320+0000 7f338bc78700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f33841115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.320+0000 7f338bc78700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3384111b30 con 0x7f3384102880 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.320+0000 7f3389a14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f33841115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:49.914 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.320+0000 7f3389a14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f33841115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51788/0 (socket says 172.21.15.33:51788) 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.320+0000 7f3389a14700 1 -- 172.21.15.33:0/2978811059 learned_addr learned my addr 172.21.15.33:0/2978811059 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.320+0000 7f3389a14700 1 -- 172.21.15.33:0/2978811059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33800094f0 con 0x7f3384102880 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.320+0000 7f3389a14700 1 --2- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f33841115f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f338000b440 tx=0x7f3380009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.321+0000 7f337a7fc700 1 -- 172.21.15.33:0/2978811059 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3380010760 con 0x7f3384102880 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.321+0000 7f337a7fc700 1 -- 172.21.15.33:0/2978811059 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f3380010d50 con 0x7f3384102880 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.321+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3384111d30 con 0x7f3384102880 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.321+0000 7f337a7fc700 1 -- 172.21.15.33:0/2978811059 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f338001ae20 con 0x7f3384102880 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.321+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33841128a0 con 0x7f3384102880 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.321+0000 7f337a7fc700 1 -- 172.21.15.33:0/2978811059 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f338002e440 con 0x7f3384102880 2024-04-25T01:26:49.915 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.321+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f338405f5c0 con 0x7f3384102880 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.322+0000 7f337a7fc700 1 --2- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f33700368a0 0x7f3370038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.322+0000 7f337a7fc700 1 -- 172.21.15.33:0/2978811059 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f338004a680 con 0x7f3384102880 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.322+0000 7f3389213700 1 --2- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f33700368a0 0x7f3370038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.322+0000 7f3389213700 1 --2- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f33700368a0 0x7f3370038d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3374009990 tx=0x7f3374006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.326+0000 7f337a7fc700 1 -- 172.21.15.33:0/2978811059 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3380018530 con 0x7f3384102880 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:47.526+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi033", "addr": "172.21.15.33", "target": ["mon-mgr", ""]}) v1 -- 0x7f33840ff200 con 0x7f33700368a0 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.406+0000 7f337a7fc700 1 -- 172.21.15.33:0/2978811059 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f33840ff200 con 0x7f33700368a0 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f33700368a0 msgr2=0x7f3370038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 --2- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f33700368a0 0x7f3370038d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3374009990 tx=0x7f3374006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 msgr2=0x7f33841115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 --2- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f33841115f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f338000b440 tx=0x7f3380009250 comp rx=0 tx=0).stop 2024-04-25T01:26:49.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 shutdown_connections 2024-04-25T01:26:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 --2- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f33700368a0 0x7f3370038d50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 --2- 172.21.15.33:0/2978811059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3384102880 0x7f33841115f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 >> 172.21.15.33:0/2978811059 conn(0x7f33840fe060 msgr2=0x7f33840feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 shutdown_connections 2024-04-25T01:26:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:49.409+0000 7f338bc78700 1 -- 172.21.15.33:0/2978811059 wait complete. 2024-04-25T01:26:49.917 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying mon service with default placement... 2024-04-25T01:26:50.977 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:50 smithi033 ceph-mon[26353]: Added host smithi033 2024-04-25T01:26:51.353 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T01:26:51.353 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.361+0000 7fd3bd150700 1 Processor -- start 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.361+0000 7fd3bd150700 1 -- start start 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.361+0000 7fd3bd150700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b81078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.361+0000 7fd3bd150700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8103290 con 0x7fd3b8076bd0 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.361+0000 7fd3b6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b81078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.361+0000 7fd3b6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b81078a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51802/0 (socket says 172.21.15.33:51802) 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.361+0000 7fd3b6d9d700 1 -- 172.21.15.33:0/187729280 learned_addr learned my addr 172.21.15.33:0/187729280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.361+0000 7fd3b6d9d700 1 -- 172.21.15.33:0/187729280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8107de0 con 0x7fd3b8076bd0 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3b6d9d700 1 --2- 172.21.15.33:0/187729280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b81078a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80098e0 tx=0x7fd3a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2efd78ae4a06e171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3b5d9b700 1 -- 172.21.15.33:0/187729280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3a800b860 con 0x7fd3b8076bd0 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3b5d9b700 1 -- 172.21.15.33:0/187729280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd3a800be50 con 0x7fd3b8076bd0 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3b5d9b700 1 -- 172.21.15.33:0/187729280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3a8010380 con 0x7fd3b8076bd0 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3bd150700 1 -- 172.21.15.33:0/187729280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 msgr2=0x7fd3b81078a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3bd150700 1 --2- 172.21.15.33:0/187729280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b81078a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd3a80098e0 tx=0x7fd3a8009250 comp rx=0 tx=0).stop 2024-04-25T01:26:51.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3bd150700 1 -- 172.21.15.33:0/187729280 shutdown_connections 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3bd150700 1 --2- 172.21.15.33:0/187729280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b81078a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3bd150700 1 -- 172.21.15.33:0/187729280 >> 172.21.15.33:0/187729280 conn(0x7fd3b80fcef0 msgr2=0x7fd3b80ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.362+0000 7fd3bd150700 1 -- 172.21.15.33:0/187729280 shutdown_connections 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.363+0000 7fd3bd150700 1 -- 172.21.15.33:0/187729280 wait complete. 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.363+0000 7fd3bd150700 1 Processor -- start 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.363+0000 7fd3bd150700 1 -- start start 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.363+0000 7fd3bd150700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b8113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.363+0000 7fd3bd150700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8113d40 con 0x7fd3b8076bd0 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd3b6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b8113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd3b6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b8113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51804/0 (socket says 172.21.15.33:51804) 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd3b6d9d700 1 -- 172.21.15.33:0/2886725450 learned_addr learned my addr 172.21.15.33:0/2886725450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd3b6d9d700 1 -- 172.21.15.33:0/2886725450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a80094f0 con 0x7fd3b8076bd0 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd3b6d9d700 1 --2- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b8113800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8006b00 tx=0x7fd3a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd39f7fe700 1 -- 172.21.15.33:0/2886725450 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3a80107c0 con 0x7fd3b8076bd0 2024-04-25T01:26:51.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b8113f40 con 0x7fd3b8076bd0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd39f7fe700 1 -- 172.21.15.33:0/2886725450 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd3a8010db0 con 0x7fd3b8076bd0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd39f7fe700 1 -- 172.21.15.33:0/2886725450 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd3a801acf0 con 0x7fd3b8076bd0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.364+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b8114ab0 con 0x7fd3b8076bd0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.365+0000 7fd39f7fe700 1 -- 172.21.15.33:0/2886725450 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fd3a8010920 con 0x7fd3b8076bd0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.365+0000 7fd39f7fe700 1 --2- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd3a0036850 0x7fd3a0038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.365+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b805f5c0 con 0x7fd3b8076bd0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.365+0000 7fd39f7fe700 1 -- 172.21.15.33:0/2886725450 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fd3a804a6a0 con 0x7fd3b8076bd0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.365+0000 7fd3b659c700 1 --2- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd3a0036850 0x7fd3a0038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.366+0000 7fd3b659c700 1 --2- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd3a0036850 0x7fd3a0038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009990 tx=0x7fd3ac006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.370+0000 7fd39f7fe700 1 -- 172.21.15.33:0/2886725450 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd3a801a530 con 0x7fd3b8076bd0 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.602+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3b80fe090 con 0x7fd3a0036850 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.606+0000 7fd39f7fe700 1 -- 172.21.15.33:0/2886725450 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd3b80fe090 con 0x7fd3a0036850 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd3a0036850 msgr2=0x7fd3a0038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 --2- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd3a0036850 0x7fd3a0038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009990 tx=0x7fd3ac006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:51.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 msgr2=0x7fd3b8113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:51.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 --2- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b8113800 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd3a8006b00 tx=0x7fd3a8009250 comp rx=0 tx=0).stop 2024-04-25T01:26:51.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 shutdown_connections 2024-04-25T01:26:51.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 --2- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fd3a0036850 0x7fd3a0038d00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 --2- 172.21.15.33:0/2886725450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3b8076bd0 0x7fd3b8113800 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:51.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 >> 172.21.15.33:0/2886725450 conn(0x7fd3b80fcef0 msgr2=0x7fd3b80fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:51.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 shutdown_connections 2024-04-25T01:26:51.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:50.608+0000 7fd3bd150700 1 -- 172.21.15.33:0/2886725450 wait complete. 2024-04-25T01:26:51.357 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying mgr service with default placement... 2024-04-25T01:26:51.596 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:51 smithi033 ceph-mon[26353]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:51.596 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:51 smithi033 ceph-mon[26353]: Saving service mon spec with placement count:5 2024-04-25T01:26:51.596 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:51 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:52.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T01:26:52.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.022+0000 7fc63ad39700 1 Processor -- start 2024-04-25T01:26:52.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.022+0000 7fc63ad39700 1 -- start start 2024-04-25T01:26:52.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.022+0000 7fc63ad39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc634102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:52.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.022+0000 7fc63ad39700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc634103160 con 0x7fc634102810 2024-04-25T01:26:52.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.022+0000 7fc638ad5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc634102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:52.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.022+0000 7fc638ad5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc634102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51818/0 (socket says 172.21.15.33:51818) 2024-04-25T01:26:52.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.022+0000 7fc638ad5700 1 -- 172.21.15.33:0/3052088625 learned_addr learned my addr 172.21.15.33:0/3052088625 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:52.873 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.023+0000 7fc638ad5700 1 -- 172.21.15.33:0/3052088625 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6341032a0 con 0x7fc634102810 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.023+0000 7fc638ad5700 1 --2- 172.21.15.33:0/3052088625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc634102c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098e0 tx=0x7fc628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceab853b82c55a2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.023+0000 7fc6337fe700 1 -- 172.21.15.33:0/3052088625 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc62800b860 con 0x7fc634102810 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.023+0000 7fc6337fe700 1 -- 172.21.15.33:0/3052088625 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc62800be50 con 0x7fc634102810 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.023+0000 7fc6337fe700 1 -- 172.21.15.33:0/3052088625 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc628010380 con 0x7fc634102810 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.024+0000 7fc63ad39700 1 -- 172.21.15.33:0/3052088625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 msgr2=0x7fc634102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.024+0000 7fc63ad39700 1 --2- 172.21.15.33:0/3052088625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc634102c20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc6280098e0 tx=0x7fc628009250 comp rx=0 tx=0).stop 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.024+0000 7fc63ad39700 1 -- 172.21.15.33:0/3052088625 shutdown_connections 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.024+0000 7fc63ad39700 1 --2- 172.21.15.33:0/3052088625 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc634102c20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.024+0000 7fc63ad39700 1 -- 172.21.15.33:0/3052088625 >> 172.21.15.33:0/3052088625 conn(0x7fc6340fdfd0 msgr2=0x7fc6341003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.024+0000 7fc63ad39700 1 -- 172.21.15.33:0/3052088625 shutdown_connections 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.024+0000 7fc63ad39700 1 -- 172.21.15.33:0/3052088625 wait complete. 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.024+0000 7fc63ad39700 1 Processor -- start 2024-04-25T01:26:52.874 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.024+0000 7fc63ad39700 1 -- start start 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc63ad39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc6341116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc63ad39700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc634111be0 con 0x7fc634102810 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc638ad5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc6341116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc638ad5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc6341116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51822/0 (socket says 172.21.15.33:51822) 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc638ad5700 1 -- 172.21.15.33:0/3337362068 learned_addr learned my addr 172.21.15.33:0/3337362068 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc638ad5700 1 -- 172.21.15.33:0/3337362068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6280094f0 con 0x7fc634102810 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc638ad5700 1 --2- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc6341116a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc628006b00 tx=0x7fc628009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc6317fa700 1 -- 172.21.15.33:0/3337362068 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc628010760 con 0x7fc634102810 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc6317fa700 1 -- 172.21.15.33:0/3337362068 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fc628010d50 con 0x7fc634102810 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc634111de0 con 0x7fc634102810 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc6317fa700 1 -- 172.21.15.33:0/3337362068 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc62801ad40 con 0x7fc634102810 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.025+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc634112950 con 0x7fc634102810 2024-04-25T01:26:52.875 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.026+0000 7fc6317fa700 1 -- 172.21.15.33:0/3337362068 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fc6280108c0 con 0x7fc634102810 2024-04-25T01:26:52.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.026+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63405f5c0 con 0x7fc634102810 2024-04-25T01:26:52.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.026+0000 7fc6317fa700 1 --2- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc624036850 0x7fc624038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:52.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.027+0000 7fc6317fa700 1 -- 172.21.15.33:0/3337362068 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc62804a540 con 0x7fc634102810 2024-04-25T01:26:52.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.027+0000 7fc633fff700 1 --2- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc624036850 0x7fc624038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:52.876 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.027+0000 7fc633fff700 1 --2- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc624036850 0x7fc624038d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc620009990 tx=0x7fc620006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.031+0000 7fc6317fa700 1 -- 172.21.15.33:0/3337362068 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc62802e960 con 0x7fc634102810 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.242+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fc6340ff200 con 0x7fc624036850 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.246+0000 7fc6317fa700 1 -- 172.21.15.33:0/3337362068 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc6340ff200 con 0x7fc624036850 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc624036850 msgr2=0x7fc624038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 --2- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc624036850 0x7fc624038d00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc620009990 tx=0x7fc620006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 msgr2=0x7fc6341116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 --2- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc6341116a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc628006b00 tx=0x7fc628009250 comp rx=0 tx=0).stop 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 shutdown_connections 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 --2- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc624036850 0x7fc624038d00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 --2- 172.21.15.33:0/3337362068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc634102810 0x7fc6341116a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 >> 172.21.15.33:0/3337362068 conn(0x7fc6340fdfd0 msgr2=0x7fc6340feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 shutdown_connections 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:52.248+0000 7fc63ad39700 1 -- 172.21.15.33:0/3337362068 wait complete. 2024-04-25T01:26:52.877 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying crash service with default placement... 2024-04-25T01:26:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:53 smithi033 ceph-mon[26353]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:53 smithi033 ceph-mon[26353]: Saving service mgr spec with placement count:2 2024-04-25T01:26:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:53 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:54.413 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.341+0000 7ffb3a33b700 1 Processor -- start 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.341+0000 7ffb3a33b700 1 -- start start 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.341+0000 7ffb3a33b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.341+0000 7ffb3a33b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34103160 con 0x7ffb34102810 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.341+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.341+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51824/0 (socket says 172.21.15.33:51824) 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.341+0000 7ffb33fff700 1 -- 172.21.15.33:0/1881456275 learned_addr learned my addr 172.21.15.33:0/1881456275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.341+0000 7ffb33fff700 1 -- 172.21.15.33:0/1881456275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb341032a0 con 0x7ffb34102810 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb33fff700 1 --2- 172.21.15.33:0/1881456275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34102c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098e0 tx=0x7ffb24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea7e4fec6ba2397 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb32ffd700 1 -- 172.21.15.33:0/1881456275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb2400b860 con 0x7ffb34102810 2024-04-25T01:26:54.414 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb32ffd700 1 -- 172.21.15.33:0/1881456275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb2400be50 con 0x7ffb34102810 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb32ffd700 1 -- 172.21.15.33:0/1881456275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb24010380 con 0x7ffb34102810 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb3a33b700 1 -- 172.21.15.33:0/1881456275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 msgr2=0x7ffb34102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb3a33b700 1 --2- 172.21.15.33:0/1881456275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34102c20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffb240098e0 tx=0x7ffb24009250 comp rx=0 tx=0).stop 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb3a33b700 1 -- 172.21.15.33:0/1881456275 shutdown_connections 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb3a33b700 1 --2- 172.21.15.33:0/1881456275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34102c20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb3a33b700 1 -- 172.21.15.33:0/1881456275 >> 172.21.15.33:0/1881456275 conn(0x7ffb340fdfd0 msgr2=0x7ffb341003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb3a33b700 1 -- 172.21.15.33:0/1881456275 shutdown_connections 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.342+0000 7ffb3a33b700 1 -- 172.21.15.33:0/1881456275 wait complete. 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.343+0000 7ffb3a33b700 1 Processor -- start 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.343+0000 7ffb3a33b700 1 -- start start 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.343+0000 7ffb3a33b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.343+0000 7ffb3a33b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb34079160 con 0x7ffb34102810 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.343+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:54.415 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.343+0000 7ffb33fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51828/0 (socket says 172.21.15.33:51828) 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.343+0000 7ffb33fff700 1 -- 172.21.15.33:0/3029612608 learned_addr learned my addr 172.21.15.33:0/3029612608 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.344+0000 7ffb33fff700 1 -- 172.21.15.33:0/3029612608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb240094f0 con 0x7ffb34102810 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.344+0000 7ffb33fff700 1 --2- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34078c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400ba90 tx=0x7ffb24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.344+0000 7ffb30ff9700 1 -- 172.21.15.33:0/3029612608 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb240107c0 con 0x7ffb34102810 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.344+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb340771d0 con 0x7ffb34102810 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.344+0000 7ffb30ff9700 1 -- 172.21.15.33:0/3029612608 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ffb24010db0 con 0x7ffb34102810 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.344+0000 7ffb30ff9700 1 -- 172.21.15.33:0/3029612608 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb2401add0 con 0x7ffb34102810 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.344+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb340776c0 con 0x7ffb34102810 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.345+0000 7ffb30ff9700 1 -- 172.21.15.33:0/3029612608 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ffb24010920 con 0x7ffb34102810 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.345+0000 7ffb30ff9700 1 --2- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7ffb1c036850 0x7ffb1c038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.345+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3404bed0 con 0x7ffb34102810 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.346+0000 7ffb30ff9700 1 -- 172.21.15.33:0/3029612608 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ffb2404a560 con 0x7ffb34102810 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.346+0000 7ffb337fe700 1 --2- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7ffb1c036850 0x7ffb1c038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:54.416 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.346+0000 7ffb337fe700 1 --2- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7ffb1c036850 0x7ffb1c038d00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009990 tx=0x7ffb28006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.350+0000 7ffb30ff9700 1 -- 172.21.15.33:0/3029612608 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffb2401a430 con 0x7ffb34102810 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.559+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb34077e80 con 0x7ffb1c036850 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.566+0000 7ffb30ff9700 1 -- 172.21.15.33:0/3029612608 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7ffb34077e80 con 0x7ffb1c036850 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7ffb1c036850 msgr2=0x7ffb1c038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 --2- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7ffb1c036850 0x7ffb1c038d00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009990 tx=0x7ffb28006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 msgr2=0x7ffb34078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 --2- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34078c20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffb2400ba90 tx=0x7ffb24009250 comp rx=0 tx=0).stop 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 shutdown_connections 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 --2- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7ffb1c036850 0x7ffb1c038d00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:54.417 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 --2- 172.21.15.33:0/3029612608 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb34102810 0x7ffb34078c20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:54.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 >> 172.21.15.33:0/3029612608 conn(0x7ffb340fdfd0 msgr2=0x7ffb340feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:54.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 shutdown_connections 2024-04-25T01:26:54.418 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:53.568+0000 7ffb3a33b700 1 -- 172.21.15.33:0/3029612608 wait complete. 2024-04-25T01:26:54.418 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying prometheus service with default placement... 2024-04-25T01:26:54.734 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:54 smithi033 ceph-mon[26353]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:54.734 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:54 smithi033 ceph-mon[26353]: Saving service crash spec with placement * 2024-04-25T01:26:54.734 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:54 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:54.734 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:54 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:54.734 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:54 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:55.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-25T01:26:55.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.855+0000 7f03c0ca1700 1 Processor -- start 2024-04-25T01:26:55.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.855+0000 7f03c0ca1700 1 -- start start 2024-04-25T01:26:55.948 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.855+0000 7f03c0ca1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc101040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.856+0000 7f03c0ca1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc101580 con 0x7f03bc100c30 2024-04-25T01:26:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.856+0000 7f03ba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc101040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.856+0000 7f03ba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc101040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51836/0 (socket says 172.21.15.33:51836) 2024-04-25T01:26:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.856+0000 7f03ba59c700 1 -- 172.21.15.33:0/394014765 learned_addr learned my addr 172.21.15.33:0/394014765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.856+0000 7f03ba59c700 1 -- 172.21.15.33:0/394014765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03bc1016c0 con 0x7f03bc100c30 2024-04-25T01:26:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.856+0000 7f03ba59c700 1 --2- 172.21.15.33:0/394014765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc101040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0098e0 tx=0x7f03ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfdab328dee896d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.856+0000 7f03b959a700 1 -- 172.21.15.33:0/394014765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03ac00b860 con 0x7f03bc100c30 2024-04-25T01:26:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.857+0000 7f03b959a700 1 -- 172.21.15.33:0/394014765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03ac00be90 con 0x7f03bc100c30 2024-04-25T01:26:55.949 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.857+0000 7f03b959a700 1 -- 172.21.15.33:0/394014765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03ac0103c0 con 0x7f03bc100c30 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.857+0000 7f03c0ca1700 1 -- 172.21.15.33:0/394014765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 msgr2=0x7f03bc101040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.857+0000 7f03c0ca1700 1 --2- 172.21.15.33:0/394014765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc101040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0098e0 tx=0x7f03ac009250 comp rx=0 tx=0).stop 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.857+0000 7f03c0ca1700 1 -- 172.21.15.33:0/394014765 shutdown_connections 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.857+0000 7f03c0ca1700 1 --2- 172.21.15.33:0/394014765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc101040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.857+0000 7f03c0ca1700 1 -- 172.21.15.33:0/394014765 >> 172.21.15.33:0/394014765 conn(0x7f03bc0fc690 msgr2=0x7f03bc0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.857+0000 7f03c0ca1700 1 -- 172.21.15.33:0/394014765 shutdown_connections 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.857+0000 7f03c0ca1700 1 -- 172.21.15.33:0/394014765 wait complete. 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.858+0000 7f03c0ca1700 1 Processor -- start 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.858+0000 7f03c0ca1700 1 -- start start 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.858+0000 7f03c0ca1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc103660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.858+0000 7f03c0ca1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03bc103ba0 con 0x7f03bc100c30 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.858+0000 7f03ba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:55.950 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.858+0000 7f03ba59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc103660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51844/0 (socket says 172.21.15.33:51844) 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.858+0000 7f03ba59c700 1 -- 172.21.15.33:0/4083165455 learned_addr learned my addr 172.21.15.33:0/4083165455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.859+0000 7f03ba59c700 1 -- 172.21.15.33:0/4083165455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac0094f0 con 0x7f03bc100c30 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.859+0000 7f03ba59c700 1 --2- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc103660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03ac006d40 tx=0x7f03ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.859+0000 7f03aaffd700 1 -- 172.21.15.33:0/4083165455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03ac010840 con 0x7f03bc100c30 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.859+0000 7f03aaffd700 1 -- 172.21.15.33:0/4083165455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03ac010e70 con 0x7f03bc100c30 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.859+0000 7f03aaffd700 1 -- 172.21.15.33:0/4083165455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03ac024ea0 con 0x7f03bc100c30 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.859+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03bc101c10 con 0x7f03bc100c30 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.859+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03bc102100 con 0x7f03bc100c30 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.860+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03bc04bed0 con 0x7f03bc100c30 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.861+0000 7f03aaffd700 1 -- 172.21.15.33:0/4083165455 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f03ac024780 con 0x7f03bc100c30 2024-04-25T01:26:55.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.861+0000 7f03aaffd700 1 --2- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f03a4036850 0x7f03a4038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.861+0000 7f03aaffd700 1 -- 172.21.15.33:0/4083165455 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f03ac04a3b0 con 0x7f03bc100c30 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.861+0000 7f03b9d9b700 1 --2- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f03a4036850 0x7f03a4038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.862+0000 7f03b9d9b700 1 --2- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f03a4036850 0x7f03a4038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009990 tx=0x7f03b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:54.868+0000 7f03aaffd700 1 -- 172.21.15.33:0/4083165455 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f03ac023980 con 0x7f03bc100c30 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.089+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f03bc1028c0 con 0x7f03a4036850 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.093+0000 7f03aaffd700 1 -- 172.21.15.33:0/4083165455 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f03bc1028c0 con 0x7f03a4036850 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f03a4036850 msgr2=0x7f03a4038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 --2- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f03a4036850 0x7f03a4038d00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009990 tx=0x7f03b0006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 msgr2=0x7f03bc103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 --2- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc103660 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03ac006d40 tx=0x7f03ac009250 comp rx=0 tx=0).stop 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 shutdown_connections 2024-04-25T01:26:55.952 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 --2- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f03a4036850 0x7f03a4038d00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:55.953 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 --2- 172.21.15.33:0/4083165455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03bc100c30 0x7f03bc103660 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:55.953 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 >> 172.21.15.33:0/4083165455 conn(0x7f03bc0fc690 msgr2=0x7f03bc0fd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:55.953 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 shutdown_connections 2024-04-25T01:26:55.953 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:55.095+0000 7f03c0ca1700 1 -- 172.21.15.33:0/4083165455 wait complete. 2024-04-25T01:26:55.953 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying grafana service with default placement... 2024-04-25T01:26:56.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:56 smithi033 ceph-mon[26353]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:56.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:56 smithi033 ceph-mon[26353]: Saving service prometheus spec with placement count:1 2024-04-25T01:26:56.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:56 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:57.457 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-25T01:26:57.457 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.632+0000 7fe45046c700 1 Processor -- start 2024-04-25T01:26:57.457 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.632+0000 7fe45046c700 1 -- start start 2024-04-25T01:26:57.457 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.632+0000 7fe45046c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe448102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:57.457 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.632+0000 7fe45046c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448103160 con 0x7fe448102810 2024-04-25T01:26:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.632+0000 7fe44e208700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe448102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.632+0000 7fe44e208700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe448102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38606/0 (socket says 172.21.15.33:38606) 2024-04-25T01:26:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.632+0000 7fe44e208700 1 -- 172.21.15.33:0/3474202997 learned_addr learned my addr 172.21.15.33:0/3474202997 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.633+0000 7fe44e208700 1 -- 172.21.15.33:0/3474202997 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4481032a0 con 0x7fe448102810 2024-04-25T01:26:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.633+0000 7fe44e208700 1 --2- 172.21.15.33:0/3474202997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe448102c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098e0 tx=0x7fe438009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecc776638340e846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.633+0000 7fe44d206700 1 -- 172.21.15.33:0/3474202997 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe43800b860 con 0x7fe448102810 2024-04-25T01:26:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.633+0000 7fe44d206700 1 -- 172.21.15.33:0/3474202997 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe43800be90 con 0x7fe448102810 2024-04-25T01:26:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.633+0000 7fe44d206700 1 -- 172.21.15.33:0/3474202997 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4380103c0 con 0x7fe448102810 2024-04-25T01:26:57.458 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.633+0000 7fe45046c700 1 -- 172.21.15.33:0/3474202997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 msgr2=0x7fe448102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.633+0000 7fe45046c700 1 --2- 172.21.15.33:0/3474202997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe448102c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe4380098e0 tx=0x7fe438009250 comp rx=0 tx=0).stop 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe45046c700 1 -- 172.21.15.33:0/3474202997 shutdown_connections 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe45046c700 1 --2- 172.21.15.33:0/3474202997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe448102c20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe45046c700 1 -- 172.21.15.33:0/3474202997 >> 172.21.15.33:0/3474202997 conn(0x7fe4480fdfd0 msgr2=0x7fe4481003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe45046c700 1 -- 172.21.15.33:0/3474202997 shutdown_connections 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe45046c700 1 -- 172.21.15.33:0/3474202997 wait complete. 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe45046c700 1 Processor -- start 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe45046c700 1 -- start start 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe45046c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe4481116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:57.459 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe45046c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448111c00 con 0x7fe448102810 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe44e208700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe4481116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.634+0000 7fe44e208700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe4481116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38610/0 (socket says 172.21.15.33:38610) 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.635+0000 7fe44e208700 1 -- 172.21.15.33:0/3403880184 learned_addr learned my addr 172.21.15.33:0/3403880184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.635+0000 7fe44e208700 1 -- 172.21.15.33:0/3403880184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4380094f0 con 0x7fe448102810 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.635+0000 7fe44e208700 1 --2- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe4481116c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe438011c10 tx=0x7fe438009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.635+0000 7fe43effd700 1 -- 172.21.15.33:0/3403880184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe438010890 con 0x7fe448102810 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.635+0000 7fe43effd700 1 -- 172.21.15.33:0/3403880184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe438010ec0 con 0x7fe448102810 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.635+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe448111e00 con 0x7fe448102810 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.635+0000 7fe43effd700 1 -- 172.21.15.33:0/3403880184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe4380243f0 con 0x7fe448102810 2024-04-25T01:26:57.460 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.635+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe448112970 con 0x7fe448102810 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.636+0000 7fe43effd700 1 -- 172.21.15.33:0/3403880184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fe4380109f0 con 0x7fe448102810 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.636+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44805f5c0 con 0x7fe448102810 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.636+0000 7fe43effd700 1 --2- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fe4340368a0 0x7fe434038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.636+0000 7fe43effd700 1 -- 172.21.15.33:0/3403880184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe43804a1e0 con 0x7fe448102810 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.636+0000 7fe44da07700 1 --2- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fe4340368a0 0x7fe434038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.637+0000 7fe44da07700 1 --2- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fe4340368a0 0x7fe434038d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe444009990 tx=0x7fe444006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.641+0000 7fe43effd700 1 -- 172.21.15.33:0/3403880184 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe43802b4d0 con 0x7fe448102810 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.849+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4480ff230 con 0x7fe4340368a0 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.855+0000 7fe43effd700 1 -- 172.21.15.33:0/3403880184 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fe4480ff230 con 0x7fe4340368a0 2024-04-25T01:26:57.461 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fe4340368a0 msgr2=0x7fe434038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 --2- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fe4340368a0 0x7fe434038d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe444009990 tx=0x7fe444006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 msgr2=0x7fe4481116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 --2- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe4481116c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe438011c10 tx=0x7fe438009250 comp rx=0 tx=0).stop 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 shutdown_connections 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 --2- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fe4340368a0 0x7fe434038d50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 --2- 172.21.15.33:0/3403880184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe448102810 0x7fe4481116c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 >> 172.21.15.33:0/3403880184 conn(0x7fe4480fdfd0 msgr2=0x7fe4480feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 shutdown_connections 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:56.857+0000 7fe45046c700 1 -- 172.21.15.33:0/3403880184 wait complete. 2024-04-25T01:26:57.462 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying node-exporter service with default placement... 2024-04-25T01:26:57.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:57 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:57.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:57 smithi033 ceph-mon[26353]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:57.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:57 smithi033 ceph-mon[26353]: Saving service grafana spec with placement count:1 2024-04-25T01:26:57.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:57 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:57.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:57 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:26:58.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-25T01:26:58.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.923+0000 7f50e2d1e700 1 Processor -- start 2024-04-25T01:26:58.798 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.923+0000 7f50e2d1e700 1 -- start start 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.923+0000 7f50e2d1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.924+0000 7f50e2d1e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc105980 con 0x7f50dc076c30 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.924+0000 7f50e0aba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.924+0000 7f50e0aba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38614/0 (socket says 172.21.15.33:38614) 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.924+0000 7f50e0aba700 1 -- 172.21.15.33:0/326201640 learned_addr learned my addr 172.21.15.33:0/326201640 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.924+0000 7f50e0aba700 1 -- 172.21.15.33:0/326201640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50dc105ac0 con 0x7f50dc076c30 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.924+0000 7f50e0aba700 1 --2- 172.21.15.33:0/326201640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc105440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f50d00098e0 tx=0x7f50d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4554599c373d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.925+0000 7f50db7fe700 1 -- 172.21.15.33:0/326201640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50d000b860 con 0x7f50dc076c30 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.925+0000 7f50db7fe700 1 -- 172.21.15.33:0/326201640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50d000be90 con 0x7f50dc076c30 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.925+0000 7f50db7fe700 1 -- 172.21.15.33:0/326201640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50d00103c0 con 0x7f50dc076c30 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.925+0000 7f50e2d1e700 1 -- 172.21.15.33:0/326201640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 msgr2=0x7f50dc105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:58.799 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.925+0000 7f50e2d1e700 1 --2- 172.21.15.33:0/326201640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc105440 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f50d00098e0 tx=0x7f50d0009250 comp rx=0 tx=0).stop 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.925+0000 7f50e2d1e700 1 -- 172.21.15.33:0/326201640 shutdown_connections 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.926+0000 7f50e2d1e700 1 --2- 172.21.15.33:0/326201640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc105440 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.926+0000 7f50e2d1e700 1 -- 172.21.15.33:0/326201640 >> 172.21.15.33:0/326201640 conn(0x7f50dc0fcef0 msgr2=0x7f50dc0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.926+0000 7f50e2d1e700 1 -- 172.21.15.33:0/326201640 shutdown_connections 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.926+0000 7f50e2d1e700 1 -- 172.21.15.33:0/326201640 wait complete. 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.926+0000 7f50e2d1e700 1 Processor -- start 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.926+0000 7f50e2d1e700 1 -- start start 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.926+0000 7f50e2d1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.926+0000 7f50e2d1e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc1032d0 con 0x7f50dc076c30 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50e0aba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50e0aba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38626/0 (socket says 172.21.15.33:38626) 2024-04-25T01:26:58.800 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50e0aba700 1 -- 172.21.15.33:0/3204690589 learned_addr learned my addr 172.21.15.33:0/3204690589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50e0aba700 1 -- 172.21.15.33:0/3204690589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d00094f0 con 0x7f50dc076c30 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50e0aba700 1 --2- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc102d90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50d0003080 tx=0x7f50d0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50d97fa700 1 -- 172.21.15.33:0/3204690589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50d0011920 con 0x7f50dc076c30 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc101340 con 0x7f50dc076c30 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50d97fa700 1 -- 172.21.15.33:0/3204690589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50d0010d10 con 0x7f50dc076c30 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc101830 con 0x7f50dc076c30 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.927+0000 7f50d97fa700 1 -- 172.21.15.33:0/3204690589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50d001ae50 con 0x7f50dc076c30 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.928+0000 7f50d97fa700 1 -- 172.21.15.33:0/3204690589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f50d0024500 con 0x7f50dc076c30 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.929+0000 7f50d97fa700 1 --2- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f50cc036850 0x7f50cc038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.929+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50dc04bed0 con 0x7f50dc076c30 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.929+0000 7f50d97fa700 1 -- 172.21.15.33:0/3204690589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f50d004a5d0 con 0x7f50dc076c30 2024-04-25T01:26:58.801 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.929+0000 7f50dbfff700 1 --2- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f50cc036850 0x7f50cc038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:26:58.802 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.929+0000 7f50dbfff700 1 --2- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f50cc036850 0x7f50cc038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009990 tx=0x7f50c8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:26:58.802 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:57.934+0000 7f50d97fa700 1 -- 172.21.15.33:0/3204690589 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f50d0018710 con 0x7f50dc076c30 2024-04-25T01:26:58.802 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.164+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f50dc101f40 con 0x7f50cc036850 2024-04-25T01:26:58.802 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.168+0000 7f50d97fa700 1 -- 172.21.15.33:0/3204690589 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f50dc101f40 con 0x7f50cc036850 2024-04-25T01:26:58.802 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f50cc036850 msgr2=0x7f50cc038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:58.802 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 --2- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f50cc036850 0x7f50cc038d00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f50c8009990 tx=0x7f50c8006e40 comp rx=0 tx=0).stop 2024-04-25T01:26:58.802 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 msgr2=0x7f50dc102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:26:58.803 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 --2- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc102d90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50d0003080 tx=0x7f50d0009250 comp rx=0 tx=0).stop 2024-04-25T01:26:58.803 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 shutdown_connections 2024-04-25T01:26:58.803 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 --2- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f50cc036850 0x7f50cc038d00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:58.803 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 --2- 172.21.15.33:0/3204690589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50dc076c30 0x7f50dc102d90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:26:58.803 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 >> 172.21.15.33:0/3204690589 conn(0x7f50dc0fcef0 msgr2=0x7f50dc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:26:58.803 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 shutdown_connections 2024-04-25T01:26:58.803 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:58.170+0000 7f50e2d1e700 1 -- 172.21.15.33:0/3204690589 wait complete. 2024-04-25T01:26:58.803 INFO:teuthology.orchestra.run.smithi033.stdout:Deploying alertmanager service with default placement... 2024-04-25T01:26:59.191 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:59 smithi033 ceph-mon[26353]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:26:59.191 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:59 smithi033 ceph-mon[26353]: Saving service node-exporter spec with placement * 2024-04-25T01:26:59.192 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:26:59 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:27:00.176 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-25T01:27:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.212+0000 7fc862eef700 1 Processor -- start 2024-04-25T01:27:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.212+0000 7fc862eef700 1 -- start start 2024-04-25T01:27:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.212+0000 7fc862eef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.212+0000 7fc862eef700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c103160 con 0x7fc85c102810 2024-04-25T01:27:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.212+0000 7fc860c8b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.212+0000 7fc860c8b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38632/0 (socket says 172.21.15.33:38632) 2024-04-25T01:27:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.212+0000 7fc860c8b700 1 -- 172.21.15.33:0/765349661 learned_addr learned my addr 172.21.15.33:0/765349661 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.212+0000 7fc860c8b700 1 -- 172.21.15.33:0/765349661 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c1032a0 con 0x7fc85c102810 2024-04-25T01:27:00.182 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.213+0000 7fc860c8b700 1 --2- 172.21.15.33:0/765349661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c102c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8500098e0 tx=0x7fc850009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b3fb159585ff62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.213+0000 7fc85b7fe700 1 -- 172.21.15.33:0/765349661 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc85000b860 con 0x7fc85c102810 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.213+0000 7fc85b7fe700 1 -- 172.21.15.33:0/765349661 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc85000be90 con 0x7fc85c102810 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.213+0000 7fc85b7fe700 1 -- 172.21.15.33:0/765349661 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8500103c0 con 0x7fc85c102810 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.213+0000 7fc862eef700 1 -- 172.21.15.33:0/765349661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 msgr2=0x7fc85c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.213+0000 7fc862eef700 1 --2- 172.21.15.33:0/765349661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c102c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc8500098e0 tx=0x7fc850009250 comp rx=0 tx=0).stop 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.213+0000 7fc862eef700 1 -- 172.21.15.33:0/765349661 shutdown_connections 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.213+0000 7fc862eef700 1 --2- 172.21.15.33:0/765349661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c102c20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.213+0000 7fc862eef700 1 -- 172.21.15.33:0/765349661 >> 172.21.15.33:0/765349661 conn(0x7fc85c0fdfd0 msgr2=0x7fc85c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.214+0000 7fc862eef700 1 -- 172.21.15.33:0/765349661 shutdown_connections 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.214+0000 7fc862eef700 1 -- 172.21.15.33:0/765349661 wait complete. 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.214+0000 7fc862eef700 1 Processor -- start 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.214+0000 7fc862eef700 1 -- start start 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.214+0000 7fc862eef700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c1116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:00.183 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.214+0000 7fc862eef700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c111c00 con 0x7fc85c102810 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc860c8b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c1116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc860c8b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c1116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38644/0 (socket says 172.21.15.33:38644) 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc860c8b700 1 -- 172.21.15.33:0/1841863709 learned_addr learned my addr 172.21.15.33:0/1841863709 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc860c8b700 1 -- 172.21.15.33:0/1841863709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8500094f0 con 0x7fc85c102810 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc860c8b700 1 --2- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c1116c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc850011c10 tx=0x7fc850009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc8597fa700 1 -- 172.21.15.33:0/1841863709 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc850010890 con 0x7fc85c102810 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc8597fa700 1 -- 172.21.15.33:0/1841863709 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc850010ec0 con 0x7fc85c102810 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c111e00 con 0x7fc85c102810 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc8597fa700 1 -- 172.21.15.33:0/1841863709 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc8500243f0 con 0x7fc85c102810 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.215+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c112970 con 0x7fc85c102810 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.216+0000 7fc8597fa700 1 -- 172.21.15.33:0/1841863709 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fc8500109f0 con 0x7fc85c102810 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.216+0000 7fc8597fa700 1 --2- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc84c0368a0 0x7fc84c038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.216+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c05f5c0 con 0x7fc85c102810 2024-04-25T01:27:00.184 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.216+0000 7fc8597fa700 1 -- 172.21.15.33:0/1841863709 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fc85004a1e0 con 0x7fc85c102810 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.216+0000 7fc85bfff700 1 --2- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc84c0368a0 0x7fc84c038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.217+0000 7fc85bfff700 1 --2- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc84c0368a0 0x7fc84c038d50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc848009990 tx=0x7fc848006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.221+0000 7fc8597fa700 1 -- 172.21.15.33:0/1841863709 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc85001a430 con 0x7fc85c102810 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.430+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fc85c0ff230 con 0x7fc84c0368a0 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.436+0000 7fc8597fa700 1 -- 172.21.15.33:0/1841863709 <== mgr.14118 v2:172.21.15.33:6800/3376891250 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fc85c0ff230 con 0x7fc84c0368a0 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc84c0368a0 msgr2=0x7fc84c038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 --2- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc84c0368a0 0x7fc84c038d50 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc848009990 tx=0x7fc848006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 msgr2=0x7fc85c1116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 --2- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c1116c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc850011c10 tx=0x7fc850009250 comp rx=0 tx=0).stop 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 shutdown_connections 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 --2- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fc84c0368a0 0x7fc84c038d50 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 --2- 172.21.15.33:0/1841863709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc85c102810 0x7fc85c1116c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 >> 172.21.15.33:0/1841863709 conn(0x7fc85c0fdfd0 msgr2=0x7fc85c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:00.185 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 shutdown_connections 2024-04-25T01:27:00.186 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:26:59.439+0000 7fc862eef700 1 -- 172.21.15.33:0/1841863709 wait complete. 2024-04-25T01:27:00.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:00 smithi033 ceph-mon[26353]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:27:00.819 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:00 smithi033 ceph-mon[26353]: Saving service alertmanager spec with placement count:1 2024-04-25T01:27:00.819 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:00 smithi033 ceph-mon[26353]: from='mgr.14118 172.21.15.33:0/3991189478' entity='mgr.smithi033.asemba' 2024-04-25T01:27:01.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:01 smithi033 ceph-mon[26353]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:01.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:01 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2738478962' entity='client.admin' 2024-04-25T01:27:01.725 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.005+0000 7fac8050e700 1 Processor -- start 2024-04-25T01:27:01.725 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.005+0000 7fac8050e700 1 -- start start 2024-04-25T01:27:01.725 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.005+0000 7fac8050e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac78102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:01.725 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.005+0000 7fac8050e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac781031d0 con 0x7fac78102880 2024-04-25T01:27:01.725 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.006+0000 7fac7e2aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac78102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.006+0000 7fac7e2aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac78102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38660/0 (socket says 172.21.15.33:38660) 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.006+0000 7fac7e2aa700 1 -- 172.21.15.33:0/2929174370 learned_addr learned my addr 172.21.15.33:0/2929174370 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.006+0000 7fac7e2aa700 1 -- 172.21.15.33:0/2929174370 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac78103310 con 0x7fac78102880 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.006+0000 7fac7e2aa700 1 --2- 172.21.15.33:0/2929174370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac78102c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac740098e0 tx=0x7fac74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0171213f4f4800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.006+0000 7fac7d2a8700 1 -- 172.21.15.33:0/2929174370 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac7400b860 con 0x7fac78102880 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.006+0000 7fac7d2a8700 1 -- 172.21.15.33:0/2929174370 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac7400be90 con 0x7fac78102880 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.006+0000 7fac7d2a8700 1 -- 172.21.15.33:0/2929174370 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac740103c0 con 0x7fac78102880 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.007+0000 7fac8050e700 1 -- 172.21.15.33:0/2929174370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 msgr2=0x7fac78102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.007+0000 7fac8050e700 1 --2- 172.21.15.33:0/2929174370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac78102c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fac740098e0 tx=0x7fac74009250 comp rx=0 tx=0).stop 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.007+0000 7fac8050e700 1 -- 172.21.15.33:0/2929174370 shutdown_connections 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.007+0000 7fac8050e700 1 --2- 172.21.15.33:0/2929174370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac78102c90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.007+0000 7fac8050e700 1 -- 172.21.15.33:0/2929174370 >> 172.21.15.33:0/2929174370 conn(0x7fac780fe060 msgr2=0x7fac78100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.007+0000 7fac8050e700 1 -- 172.21.15.33:0/2929174370 shutdown_connections 2024-04-25T01:27:01.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.007+0000 7fac8050e700 1 -- 172.21.15.33:0/2929174370 wait complete. 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.007+0000 7fac8050e700 1 Processor -- start 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.008+0000 7fac8050e700 1 -- start start 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.008+0000 7fac8050e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac781115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.008+0000 7fac8050e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac78111b00 con 0x7fac78102880 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.008+0000 7fac7e2aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac781115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.008+0000 7fac7e2aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac781115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38672/0 (socket says 172.21.15.33:38672) 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.008+0000 7fac7e2aa700 1 -- 172.21.15.33:0/2738478962 learned_addr learned my addr 172.21.15.33:0/2738478962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.008+0000 7fac7e2aa700 1 -- 172.21.15.33:0/2738478962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740094f0 con 0x7fac78102880 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.008+0000 7fac7e2aa700 1 --2- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac781115c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fac74011680 tx=0x7fac74009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.008+0000 7fac6effd700 1 -- 172.21.15.33:0/2738478962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac74010890 con 0x7fac78102880 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.009+0000 7fac6effd700 1 -- 172.21.15.33:0/2738478962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac74010ec0 con 0x7fac78102880 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.009+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac78111d00 con 0x7fac78102880 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.009+0000 7fac6effd700 1 -- 172.21.15.33:0/2738478962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fac740243f0 con 0x7fac78102880 2024-04-25T01:27:01.727 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.009+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac78112870 con 0x7fac78102880 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.010+0000 7fac6effd700 1 -- 172.21.15.33:0/2738478962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fac7401a8c0 con 0x7fac78102880 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.010+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7805f5c0 con 0x7fac78102880 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.010+0000 7fac6effd700 1 --2- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fac640368a0 0x7fac64038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.010+0000 7fac6effd700 1 -- 172.21.15.33:0/2738478962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fac7404a1f0 con 0x7fac78102880 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.010+0000 7fac7daa9700 1 --2- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fac640368a0 0x7fac64038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.010+0000 7fac7daa9700 1 --2- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fac640368a0 0x7fac64038d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac68009990 tx=0x7fac68006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.015+0000 7fac6effd700 1 -- 172.21.15.33:0/2738478962 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fac7402b8e0 con 0x7fac78102880 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.213+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fac7804bed0 con 0x7fac78102880 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.216+0000 7fac6effd700 1 -- 172.21.15.33:0/2738478962 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fac7402b2f0 con 0x7fac78102880 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.218+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fac640368a0 msgr2=0x7fac64038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.218+0000 7fac8050e700 1 --2- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fac640368a0 0x7fac64038d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fac68009990 tx=0x7fac68006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.218+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 msgr2=0x7fac781115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.218+0000 7fac8050e700 1 --2- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac781115c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fac74011680 tx=0x7fac74009250 comp rx=0 tx=0).stop 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.218+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 shutdown_connections 2024-04-25T01:27:01.728 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.218+0000 7fac8050e700 1 --2- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fac640368a0 0x7fac64038d50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:01.729 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.218+0000 7fac8050e700 1 --2- 172.21.15.33:0/2738478962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac78102880 0x7fac781115c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:01.729 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.218+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 >> 172.21.15.33:0/2738478962 conn(0x7fac780fe060 msgr2=0x7fac780feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:01.729 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.219+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 shutdown_connections 2024-04-25T01:27:01.729 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:01.219+0000 7fac8050e700 1 -- 172.21.15.33:0/2738478962 wait complete. 2024-04-25T01:27:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.170+0000 7fa3f12d6700 1 Processor -- start 2024-04-25T01:27:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.171+0000 7fa3f12d6700 1 -- start start 2024-04-25T01:27:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.171+0000 7fa3f12d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec105410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.171+0000 7fa3f12d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec105950 con 0x7fa3ec071310 2024-04-25T01:27:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.171+0000 7fa3eaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec105410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:02.916 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.171+0000 7fa3eaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec105410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38684/0 (socket says 172.21.15.33:38684) 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.171+0000 7fa3eaffd700 1 -- 172.21.15.33:0/675596903 learned_addr learned my addr 172.21.15.33:0/675596903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.171+0000 7fa3eaffd700 1 -- 172.21.15.33:0/675596903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec105a90 con 0x7fa3ec071310 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.171+0000 7fa3eaffd700 1 --2- 172.21.15.33:0/675596903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec105410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0098e0 tx=0x7fa3dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b90c0e92a8fae56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3e9ffb700 1 -- 172.21.15.33:0/675596903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3dc00b860 con 0x7fa3ec071310 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3e9ffb700 1 -- 172.21.15.33:0/675596903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3dc00be90 con 0x7fa3ec071310 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3e9ffb700 1 -- 172.21.15.33:0/675596903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3dc0103c0 con 0x7fa3ec071310 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3f12d6700 1 -- 172.21.15.33:0/675596903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 msgr2=0x7fa3ec105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3f12d6700 1 --2- 172.21.15.33:0/675596903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec105410 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0098e0 tx=0x7fa3dc009250 comp rx=0 tx=0).stop 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3f12d6700 1 -- 172.21.15.33:0/675596903 shutdown_connections 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3f12d6700 1 --2- 172.21.15.33:0/675596903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec105410 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3f12d6700 1 -- 172.21.15.33:0/675596903 >> 172.21.15.33:0/675596903 conn(0x7fa3ec0fcec0 msgr2=0x7fa3ec0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3f12d6700 1 -- 172.21.15.33:0/675596903 shutdown_connections 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.172+0000 7fa3f12d6700 1 -- 172.21.15.33:0/675596903 wait complete. 2024-04-25T01:27:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.173+0000 7fa3f12d6700 1 Processor -- start 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.173+0000 7fa3f12d6700 1 -- start start 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.173+0000 7fa3f12d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.173+0000 7fa3f12d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec1032a0 con 0x7fa3ec071310 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.173+0000 7fa3eaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.173+0000 7fa3eaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38694/0 (socket says 172.21.15.33:38694) 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.173+0000 7fa3eaffd700 1 -- 172.21.15.33:0/2643985698 learned_addr learned my addr 172.21.15.33:0/2643985698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.174+0000 7fa3eaffd700 1 -- 172.21.15.33:0/2643985698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0094f0 con 0x7fa3ec071310 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.174+0000 7fa3eaffd700 1 --2- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec102d60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0094c0 tx=0x7fa3dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.174+0000 7fa3d37fe700 1 -- 172.21.15.33:0/2643985698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3dc010890 con 0x7fa3ec071310 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.174+0000 7fa3d37fe700 1 -- 172.21.15.33:0/2643985698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3dc010ec0 con 0x7fa3ec071310 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.174+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec101310 con 0x7fa3ec071310 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.174+0000 7fa3d37fe700 1 -- 172.21.15.33:0/2643985698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa3dc0243f0 con 0x7fa3ec071310 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.174+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec101800 con 0x7fa3ec071310 2024-04-25T01:27:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.175+0000 7fa3d37fe700 1 -- 172.21.15.33:0/2643985698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fa3dc02d3f0 con 0x7fa3ec071310 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.175+0000 7fa3d37fe700 1 --2- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa3d40368a0 0x7fa3d4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.175+0000 7fa3d37fe700 1 -- 172.21.15.33:0/2643985698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa3dc04a4c0 con 0x7fa3ec071310 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.175+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec04bed0 con 0x7fa3ec071310 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.175+0000 7fa3ea7fc700 1 --2- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa3d40368a0 0x7fa3d4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.176+0000 7fa3ea7fc700 1 --2- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa3d40368a0 0x7fa3d4038d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009990 tx=0x7fa3e0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.180+0000 7fa3d37fe700 1 -- 172.21.15.33:0/2643985698 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa3dc02b8e0 con 0x7fa3ec071310 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.376+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa3ec101c80 con 0x7fa3ec071310 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.379+0000 7fa3d37fe700 1 -- 172.21.15.33:0/2643985698 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa3dc02b2f0 con 0x7fa3ec071310 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa3d40368a0 msgr2=0x7fa3d4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 --2- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa3d40368a0 0x7fa3d4038d50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0009990 tx=0x7fa3e0006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 msgr2=0x7fa3ec102d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 --2- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec102d60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc0094c0 tx=0x7fa3dc009250 comp rx=0 tx=0).stop 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 shutdown_connections 2024-04-25T01:27:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 --2- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa3d40368a0 0x7fa3d4038d50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:02.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 --2- 172.21.15.33:0/2643985698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3ec071310 0x7fa3ec102d60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:02.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 >> 172.21.15.33:0/2643985698 conn(0x7fa3ec0fcec0 msgr2=0x7fa3ec0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:02.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 shutdown_connections 2024-04-25T01:27:02.920 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:02.381+0000 7fa3f12d6700 1 -- 172.21.15.33:0/2643985698 wait complete. 2024-04-25T01:27:02.920 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling the dashboard module... 2024-04-25T01:27:03.613 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:03 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2643985698' entity='client.admin' 2024-04-25T01:27:03.613 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:03 smithi033 ceph-mon[26353]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:04.697 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:04 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1026797287' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T01:27:05.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.372+0000 7fa371ca1700 1 Processor -- start 2024-04-25T01:27:05.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.373+0000 7fa371ca1700 1 -- start start 2024-04-25T01:27:05.022 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.373+0000 7fa371ca1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c103080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.373+0000 7fa371ca1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c103650 con 0x7fa36c102c70 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.373+0000 7fa36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.373+0000 7fa36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38702/0 (socket says 172.21.15.33:38702) 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.373+0000 7fa36b7fe700 1 -- 172.21.15.33:0/4217470288 learned_addr learned my addr 172.21.15.33:0/4217470288 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.373+0000 7fa36b7fe700 1 -- 172.21.15.33:0/4217470288 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c103790 con 0x7fa36c102c70 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.373+0000 7fa36b7fe700 1 --2- 172.21.15.33:0/4217470288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c103080 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0098e0 tx=0x7fa35c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a598250938f106a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa36a7fc700 1 -- 172.21.15.33:0/4217470288 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa35c00b860 con 0x7fa36c102c70 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa36a7fc700 1 -- 172.21.15.33:0/4217470288 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa35c00be90 con 0x7fa36c102c70 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa36a7fc700 1 -- 172.21.15.33:0/4217470288 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa35c0103c0 con 0x7fa36c102c70 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa371ca1700 1 -- 172.21.15.33:0/4217470288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 msgr2=0x7fa36c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:05.023 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa371ca1700 1 --2- 172.21.15.33:0/4217470288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c103080 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0098e0 tx=0x7fa35c009250 comp rx=0 tx=0).stop 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa371ca1700 1 -- 172.21.15.33:0/4217470288 shutdown_connections 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa371ca1700 1 --2- 172.21.15.33:0/4217470288 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c103080 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa371ca1700 1 -- 172.21.15.33:0/4217470288 >> 172.21.15.33:0/4217470288 conn(0x7fa36c0fc6f0 msgr2=0x7fa36c0feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa371ca1700 1 -- 172.21.15.33:0/4217470288 shutdown_connections 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.374+0000 7fa371ca1700 1 -- 172.21.15.33:0/4217470288 wait complete. 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.375+0000 7fa371ca1700 1 Processor -- start 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.375+0000 7fa371ca1700 1 -- start start 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.375+0000 7fa371ca1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c113860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.375+0000 7fa371ca1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c113da0 con 0x7fa36c102c70 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.375+0000 7fa36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.375+0000 7fa36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c113860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38716/0 (socket says 172.21.15.33:38716) 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.375+0000 7fa36b7fe700 1 -- 172.21.15.33:0/1026797287 learned_addr learned my addr 172.21.15.33:0/1026797287 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.376+0000 7fa36b7fe700 1 -- 172.21.15.33:0/1026797287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0094f0 con 0x7fa36c102c70 2024-04-25T01:27:05.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.376+0000 7fa36b7fe700 1 --2- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c113860 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0090b0 tx=0x7fa35c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.376+0000 7fa353fff700 1 -- 172.21.15.33:0/1026797287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa35c010840 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.376+0000 7fa353fff700 1 -- 172.21.15.33:0/1026797287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa35c010e70 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.376+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c113fa0 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.376+0000 7fa353fff700 1 -- 172.21.15.33:0/1026797287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa35c024e90 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.376+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c114ac0 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.377+0000 7fa353fff700 1 -- 172.21.15.33:0/1026797287 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fa35c0109a0 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.377+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36c05f5c0 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.377+0000 7fa353fff700 1 --2- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa354036850 0x7fa354038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.377+0000 7fa353fff700 1 -- 172.21.15.33:0/1026797287 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa35c04a7d0 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.377+0000 7fa36affd700 1 --2- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa354036850 0x7fa354038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.378+0000 7fa36affd700 1 --2- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa354036850 0x7fa354038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa360009990 tx=0x7fa360006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.384+0000 7fa353fff700 1 -- 172.21.15.33:0/1026797287 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa35c02c520 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:03.634+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fa36c04bed0 con 0x7fa36c102c70 2024-04-25T01:27:05.025 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.392+0000 7fa353fff700 1 -- 172.21.15.33:0/1026797287 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fa35c024430 con 0x7fa36c102c70 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.393+0000 7fa353fff700 1 -- 172.21.15.33:0/1026797287 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fa35c00f4c0 con 0x7fa36c102c70 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa354036850 msgr2=0x7fa354038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 --2- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa354036850 0x7fa354038d00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa360009990 tx=0x7fa360006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 msgr2=0x7fa36c113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 --2- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c113860 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa35c0090b0 tx=0x7fa35c009250 comp rx=0 tx=0).stop 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 shutdown_connections 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 --2- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa354036850 0x7fa354038d00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 --2- 172.21.15.33:0/1026797287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa36c102c70 0x7fa36c113860 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 >> 172.21.15.33:0/1026797287 conn(0x7fa36c0fc6f0 msgr2=0x7fa36c0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 shutdown_connections 2024-04-25T01:27:05.026 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:04.396+0000 7fa371ca1700 1 -- 172.21.15.33:0/1026797287 wait complete. 2024-04-25T01:27:05.666 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:05 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1026797287' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T01:27:05.667 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:05 smithi033 ceph-mon[26353]: mgrmap e9: smithi033.asemba(active, since 23s) 2024-04-25T01:27:06.352 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:27:06.352 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T01:27:06.352 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T01:27:06.352 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "active_name": "smithi033.asemba", 2024-04-25T01:27:06.352 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T01:27:06.352 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:27:06.353 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.481+0000 7fa473010700 1 Processor -- start 2024-04-25T01:27:06.353 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.481+0000 7fa473010700 1 -- start start 2024-04-25T01:27:06.353 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.481+0000 7fa473010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c1054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:06.353 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.481+0000 7fa473010700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c071640 con 0x7fa46c071230 2024-04-25T01:27:06.353 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.481+0000 7fa470dac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c1054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:06.353 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.481+0000 7fa470dac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c1054f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38744/0 (socket says 172.21.15.33:38744) 2024-04-25T01:27:06.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.481+0000 7fa470dac700 1 -- 172.21.15.33:0/952258328 learned_addr learned my addr 172.21.15.33:0/952258328 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:06.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.481+0000 7fa470dac700 1 -- 172.21.15.33:0/952258328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c105a30 con 0x7fa46c071230 2024-04-25T01:27:06.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa470dac700 1 --2- 172.21.15.33:0/952258328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c1054f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4600098e0 tx=0x7fa460009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f531bcf157d5a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:06.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa46b7fe700 1 -- 172.21.15.33:0/952258328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa46000b860 con 0x7fa46c071230 2024-04-25T01:27:06.354 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa46b7fe700 1 -- 172.21.15.33:0/952258328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa46000be90 con 0x7fa46c071230 2024-04-25T01:27:06.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa46b7fe700 1 -- 172.21.15.33:0/952258328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4600103c0 con 0x7fa46c071230 2024-04-25T01:27:06.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa473010700 1 -- 172.21.15.33:0/952258328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 msgr2=0x7fa46c1054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:06.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa473010700 1 --2- 172.21.15.33:0/952258328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c1054f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4600098e0 tx=0x7fa460009250 comp rx=0 tx=0).stop 2024-04-25T01:27:06.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa473010700 1 -- 172.21.15.33:0/952258328 shutdown_connections 2024-04-25T01:27:06.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa473010700 1 --2- 172.21.15.33:0/952258328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c1054f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:06.355 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa473010700 1 -- 172.21.15.33:0/952258328 >> 172.21.15.33:0/952258328 conn(0x7fa46c0fcec0 msgr2=0x7fa46c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:06.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.482+0000 7fa473010700 1 -- 172.21.15.33:0/952258328 shutdown_connections 2024-04-25T01:27:06.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.483+0000 7fa473010700 1 -- 172.21.15.33:0/952258328 wait complete. 2024-04-25T01:27:06.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.483+0000 7fa473010700 1 Processor -- start 2024-04-25T01:27:06.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.483+0000 7fa473010700 1 -- start start 2024-04-25T01:27:06.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.483+0000 7fa473010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:06.356 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.483+0000 7fa473010700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa46c111b40 con 0x7fa46c071230 2024-04-25T01:27:06.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.483+0000 7fa470dac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:06.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.483+0000 7fa470dac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38758/0 (socket says 172.21.15.33:38758) 2024-04-25T01:27:06.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.484+0000 7fa470dac700 1 -- 172.21.15.33:0/494206835 learned_addr learned my addr 172.21.15.33:0/494206835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:06.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.484+0000 7fa470dac700 1 -- 172.21.15.33:0/494206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4600094f0 con 0x7fa46c071230 2024-04-25T01:27:06.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.484+0000 7fa470dac700 1 --2- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c111600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4600094c0 tx=0x7fa460009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:06.357 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.484+0000 7fa4697fa700 1 -- 172.21.15.33:0/494206835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa460010890 con 0x7fa46c071230 2024-04-25T01:27:06.358 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.484+0000 7fa4697fa700 1 -- 172.21.15.33:0/494206835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa460010ec0 con 0x7fa46c071230 2024-04-25T01:27:06.358 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.484+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa46c111d40 con 0x7fa46c071230 2024-04-25T01:27:06.358 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.484+0000 7fa4697fa700 1 -- 172.21.15.33:0/494206835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa4600243f0 con 0x7fa46c071230 2024-04-25T01:27:06.358 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.484+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa46c1128b0 con 0x7fa46c071230 2024-04-25T01:27:06.358 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.485+0000 7fa4697fa700 1 -- 172.21.15.33:0/494206835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fa46002d3f0 con 0x7fa46c071230 2024-04-25T01:27:06.358 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.485+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46c05f5c0 con 0x7fa46c071230 2024-04-25T01:27:06.359 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.485+0000 7fa4697fa700 1 --2- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa45c0368a0 0x7fa45c038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:06.359 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.485+0000 7fa4697fa700 1 -- 172.21.15.33:0/494206835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa46004a390 con 0x7fa46c071230 2024-04-25T01:27:06.359 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.485+0000 7fa46bfff700 1 -- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa45c0368a0 msgr2=0x7fa45c038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3376891250 2024-04-25T01:27:06.359 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.485+0000 7fa46bfff700 1 --2- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa45c0368a0 0x7fa45c038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:06.359 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.490+0000 7fa4697fa700 1 -- 172.21.15.33:0/494206835 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa46002b8e0 con 0x7fa46c071230 2024-04-25T01:27:06.360 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.686+0000 7fa46bfff700 1 -- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa45c0368a0 msgr2=0x7fa45c038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3376891250 2024-04-25T01:27:06.360 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.686+0000 7fa46bfff700 1 --2- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa45c0368a0 0x7fa45c038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:27:06.360 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.789+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fa46c112bf0 con 0x7fa46c071230 2024-04-25T01:27:06.360 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.789+0000 7fa4697fa700 1 -- 172.21.15.33:0/494206835 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fa46002c350 con 0x7fa46c071230 2024-04-25T01:27:06.360 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.791+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa45c0368a0 msgr2=0x7fa45c038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:27:06.360 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.791+0000 7fa473010700 1 --2- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa45c0368a0 0x7fa45c038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:06.361 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.791+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 msgr2=0x7fa46c111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:06.361 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.791+0000 7fa473010700 1 --2- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c111600 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4600094c0 tx=0x7fa460009250 comp rx=0 tx=0).stop 2024-04-25T01:27:06.361 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.792+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 shutdown_connections 2024-04-25T01:27:06.361 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.792+0000 7fa473010700 1 --2- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7fa45c0368a0 0x7fa45c038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:06.361 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.792+0000 7fa473010700 1 --2- 172.21.15.33:0/494206835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa46c071230 0x7fa46c111600 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:06.361 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.792+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 >> 172.21.15.33:0/494206835 conn(0x7fa46c0fcec0 msgr2=0x7fa46c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:06.361 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.792+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 shutdown_connections 2024-04-25T01:27:06.362 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:05.792+0000 7fa473010700 1 -- 172.21.15.33:0/494206835 wait complete. 2024-04-25T01:27:06.362 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for the mgr to restart... 2024-04-25T01:27:06.362 INFO:teuthology.orchestra.run.smithi033.stdout:Waiting for mgr epoch 9... 2024-04-25T01:27:06.596 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:06 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/494206835' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T01:27:11.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: Active manager daemon smithi033.asemba restarted 2024-04-25T01:27:11.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: Activating manager daemon smithi033.asemba 2024-04-25T01:27:11.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T01:27:11.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: mgrmap e10: smithi033.asemba(active, starting, since 0.00314336s) 2024-04-25T01:27:11.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:27:11.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:27:11.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:27:11.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:27:11.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:27:11.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: Manager daemon smithi033.asemba is now available 2024-04-25T01:27:11.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:27:11.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:27:11.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:27:11.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:11 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/trash_purge_schedule"}]: dispatch 2024-04-25T01:27:12.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout { 2024-04-25T01:27:12.891 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout } 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.846+0000 7f6441ab6700 1 Processor -- start 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.846+0000 7f6441ab6700 1 -- start start 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.847+0000 7f6441ab6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c1020d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.847+0000 7f6441ab6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c102610 con 0x7f643c101cc0 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.847+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c1020d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.847+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c1020d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53760/0 (socket says 172.21.15.33:53760) 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.847+0000 7f643b7fe700 1 -- 172.21.15.33:0/4195306533 learned_addr learned my addr 172.21.15.33:0/4195306533 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.847+0000 7f643b7fe700 1 -- 172.21.15.33:0/4195306533 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c102750 con 0x7f643c101cc0 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.847+0000 7f643b7fe700 1 --2- 172.21.15.33:0/4195306533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c1020d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f642c0098e0 tx=0x7f642c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c0e54dad1ba2fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f643a7fc700 1 -- 172.21.15.33:0/4195306533 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f642c00b860 con 0x7f643c101cc0 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f643a7fc700 1 -- 172.21.15.33:0/4195306533 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f642c00be90 con 0x7f643c101cc0 2024-04-25T01:27:12.892 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f643a7fc700 1 -- 172.21.15.33:0/4195306533 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f642c0103c0 con 0x7f643c101cc0 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f6441ab6700 1 -- 172.21.15.33:0/4195306533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 msgr2=0x7f643c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f6441ab6700 1 --2- 172.21.15.33:0/4195306533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c1020d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f642c0098e0 tx=0x7f642c009250 comp rx=0 tx=0).stop 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f6441ab6700 1 -- 172.21.15.33:0/4195306533 shutdown_connections 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f6441ab6700 1 --2- 172.21.15.33:0/4195306533 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c1020d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f6441ab6700 1 -- 172.21.15.33:0/4195306533 >> 172.21.15.33:0/4195306533 conn(0x7f643c0fc690 msgr2=0x7f643c0feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f6441ab6700 1 -- 172.21.15.33:0/4195306533 shutdown_connections 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.848+0000 7f6441ab6700 1 -- 172.21.15.33:0/4195306533 wait complete. 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.849+0000 7f6441ab6700 1 Processor -- start 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.849+0000 7f6441ab6700 1 -- start start 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.849+0000 7f6441ab6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c107a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.849+0000 7f6441ab6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c107f40 con 0x7f643c101cc0 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.849+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.849+0000 7f643b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53764/0 (socket says 172.21.15.33:53764) 2024-04-25T01:27:12.893 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.849+0000 7f643b7fe700 1 -- 172.21.15.33:0/612366866 learned_addr learned my addr 172.21.15.33:0/612366866 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.850+0000 7f643b7fe700 1 -- 172.21.15.33:0/612366866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642c0094f0 con 0x7f643c101cc0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.850+0000 7f643b7fe700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c107a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f642c006d40 tx=0x7f642c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.850+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f642c010840 con 0x7f643c101cc0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.850+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f642c010e70 con 0x7f643c101cc0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.850+0000 7f6441ab6700 1 -- 172.21.15.33:0/612366866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c105fb0 con 0x7f643c101cc0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.850+0000 7f6441ab6700 1 -- 172.21.15.33:0/612366866 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c1064a0 con 0x7f643c101cc0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.850+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f642c024ea0 con 0x7f643c101cc0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.851+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f642c01a4e0 con 0x7f643c101cc0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.852+0000 7f6423fff700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 0x7f6424038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.852+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 --> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6424039460 con 0x7f64240368a0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.852+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f642c04a9e0 con 0x7f643c101cc0 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.852+0000 7f643affd700 1 -- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 msgr2=0x7f6424038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3376891250 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:06.852+0000 7f643affd700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 0x7f6424038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:27:12.894 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:07.053+0000 7f643affd700 1 -- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 msgr2=0x7f6424038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3376891250 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:07.053+0000 7f643affd700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 0x7f6424038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:07.454+0000 7f643affd700 1 -- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 msgr2=0x7f6424038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3376891250 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:07.454+0000 7f643affd700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 0x7f6424038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:08.255+0000 7f643affd700 1 -- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 msgr2=0x7f6424038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3376891250 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:08.255+0000 7f643affd700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 0x7f6424038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:09.856+0000 7f643affd700 1 -- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 msgr2=0x7f6424038d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/3376891250 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:09.856+0000 7f643affd700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 0x7f6424038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:11.183+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7f642c01a830 con 0x7f643c101cc0 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:11.183+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 msgr2=0x7f6424038d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:11.183+0000 7f6423fff700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 0x7f6424038d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.184+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f642c030d80 con 0x7f643c101cc0 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.184+0000 7f6423fff700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f642403a090 0x7f642403c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.184+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 --> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6424039460 con 0x7f642403a090 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.185+0000 7f643affd700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f642403a090 0x7f642403c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.185+0000 7f643affd700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f642403a090 0x7f642403c470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64300039d0 tx=0x7f6430009350 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:12.895 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.186+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 <== mgr.14160 v2:172.21.15.33:6800/395184027 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f6424039460 con 0x7f642403a090 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.192+0000 7f6441ab6700 1 -- 172.21.15.33:0/612366866 --> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f643c04bed0 con 0x7f642403a090 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.192+0000 7f6423fff700 1 -- 172.21.15.33:0/612366866 <== mgr.14160 v2:172.21.15.33:6800/395184027 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f643c04bed0 con 0x7f642403a090 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 -- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f642403a090 msgr2=0x7f642403c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f642403a090 0x7f642403c470 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64300039d0 tx=0x7f6430009350 comp rx=0 tx=0).stop 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 -- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 msgr2=0x7f643c107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c107a00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f642c006d40 tx=0x7f642c009250 comp rx=0 tx=0).stop 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 -- 172.21.15.33:0/612366866 shutdown_connections 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f642403a090 0x7f642403c470 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f643c101cc0 0x7f643c107a00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 --2- 172.21.15.33:0/612366866 >> [v2:172.21.15.33:6800/3376891250,v1:172.21.15.33:6801/3376891250] conn(0x7f64240368a0 0x7f6424038d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 -- 172.21.15.33:0/612366866 >> 172.21.15.33:0/612366866 conn(0x7f643c0fc690 msgr2=0x7f643c0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 -- 172.21.15.33:0/612366866 shutdown_connections 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:12.193+0000 7f6441ab6700 1 -- 172.21.15.33:0/612366866 wait complete. 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:mgr epoch 9 is available 2024-04-25T01:27:12.896 INFO:teuthology.orchestra.run.smithi033.stdout:Generating a dashboard self-signed certificate... 2024-04-25T01:27:13.273 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:13 smithi033 ceph-mon[26353]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T01:27:13.273 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:13 smithi033 ceph-mon[26353]: mgrmap e11: smithi033.asemba(active, since 1.00567s) 2024-04-25T01:27:13.273 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:13 smithi033 ceph-mon[26353]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T01:27:13.273 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:13 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:14.120 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.293+0000 7fde6299d700 1 Processor -- start 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.293+0000 7fde6299d700 1 -- start start 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.293+0000 7fde6299d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c105430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.293+0000 7fde6299d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c105970 con 0x7fde5c0712f0 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.293+0000 7fde5bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c105430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.293+0000 7fde5bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c105430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53844/0 (socket says 172.21.15.33:53844) 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.293+0000 7fde5bfff700 1 -- 172.21.15.33:0/3156631847 learned_addr learned my addr 172.21.15.33:0/3156631847 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.294+0000 7fde5bfff700 1 -- 172.21.15.33:0/3156631847 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c105ab0 con 0x7fde5c0712f0 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.294+0000 7fde5bfff700 1 --2- 172.21.15.33:0/3156631847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c105430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0098e0 tx=0x7fde4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7dce2db0df843a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.294+0000 7fde5affd700 1 -- 172.21.15.33:0/3156631847 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde4c00b8f0 con 0x7fde5c0712f0 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.294+0000 7fde5affd700 1 -- 172.21.15.33:0/3156631847 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde4c011420 con 0x7fde5c0712f0 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.294+0000 7fde5affd700 1 -- 172.21.15.33:0/3156631847 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde4c010540 con 0x7fde5c0712f0 2024-04-25T01:27:14.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.295+0000 7fde6299d700 1 -- 172.21.15.33:0/3156631847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 msgr2=0x7fde5c105430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.295+0000 7fde6299d700 1 --2- 172.21.15.33:0/3156631847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c105430 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fde4c0098e0 tx=0x7fde4c009250 comp rx=0 tx=0).stop 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.295+0000 7fde6299d700 1 -- 172.21.15.33:0/3156631847 shutdown_connections 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.295+0000 7fde6299d700 1 --2- 172.21.15.33:0/3156631847 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c105430 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.295+0000 7fde6299d700 1 -- 172.21.15.33:0/3156631847 >> 172.21.15.33:0/3156631847 conn(0x7fde5c0fcec0 msgr2=0x7fde5c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.295+0000 7fde6299d700 1 -- 172.21.15.33:0/3156631847 shutdown_connections 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.296+0000 7fde6299d700 1 -- 172.21.15.33:0/3156631847 wait complete. 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.296+0000 7fde6299d700 1 Processor -- start 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.296+0000 7fde6299d700 1 -- start start 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.296+0000 7fde6299d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c111610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.296+0000 7fde6299d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c111b50 con 0x7fde5c0712f0 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.297+0000 7fde5bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c111610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.297+0000 7fde5bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c111610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53846/0 (socket says 172.21.15.33:53846) 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.297+0000 7fde5bfff700 1 -- 172.21.15.33:0/1581136312 learned_addr learned my addr 172.21.15.33:0/1581136312 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.297+0000 7fde5bfff700 1 -- 172.21.15.33:0/1581136312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0094f0 con 0x7fde5c0712f0 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.297+0000 7fde5bfff700 1 --2- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c111610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fde4c011e90 tx=0x7fde4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:14.122 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.297+0000 7fde58ff9700 1 -- 172.21.15.33:0/1581136312 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde4c00beb0 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.297+0000 7fde58ff9700 1 -- 172.21.15.33:0/1581136312 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde4c010c80 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.297+0000 7fde58ff9700 1 -- 172.21.15.33:0/1581136312 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde4c01ad40 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.297+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c111d50 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.298+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c1128c0 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.298+0000 7fde58ff9700 1 -- 172.21.15.33:0/1581136312 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7fde4c0107b0 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.299+0000 7fde58ff9700 1 --2- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fde44036850 0x7fde44038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.299+0000 7fde58ff9700 1 -- 172.21.15.33:0/1581136312 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fde4c04a370 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.299+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde5c05f5c0 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.299+0000 7fde5b7fe700 1 --2- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fde44036850 0x7fde44038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.299+0000 7fde5b7fe700 1 --2- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fde44036850 0x7fde44038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde50009990 tx=0x7fde50006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.308+0000 7fde58ff9700 1 -- 172.21.15.33:0/1581136312 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fde4c02d3f0 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.434+0000 7fde58ff9700 1 -- 172.21.15.33:0/1581136312 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fde4c02bd40 con 0x7fde5c0712f0 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.513+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 --> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fde5c0fe0d0 con 0x7fde44036850 2024-04-25T01:27:14.123 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.567+0000 7fde58ff9700 1 -- 172.21.15.33:0/1581136312 <== mgr.14160 v2:172.21.15.33:6800/395184027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fde5c0fe0d0 con 0x7fde44036850 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.569+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fde44036850 msgr2=0x7fde44038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.569+0000 7fde6299d700 1 --2- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fde44036850 0x7fde44038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde50009990 tx=0x7fde50006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.569+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 msgr2=0x7fde5c111610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.569+0000 7fde6299d700 1 --2- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c111610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fde4c011e90 tx=0x7fde4c009250 comp rx=0 tx=0).stop 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.569+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 shutdown_connections 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.569+0000 7fde6299d700 1 --2- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fde44036850 0x7fde44038d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.569+0000 7fde6299d700 1 --2- 172.21.15.33:0/1581136312 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde5c0712f0 0x7fde5c111610 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.569+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 >> 172.21.15.33:0/1581136312 conn(0x7fde5c0fcec0 msgr2=0x7fde5c0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.570+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 shutdown_connections 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:13.570+0000 7fde6299d700 1 -- 172.21.15.33:0/1581136312 wait complete. 2024-04-25T01:27:14.124 INFO:teuthology.orchestra.run.smithi033.stdout:Creating initial admin user... 2024-04-25T01:27:14.733 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:14 smithi033 ceph-mon[26353]: mgrmap e12: smithi033.asemba(active, since 2s) 2024-04-25T01:27:14.734 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:14 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:14.734 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:14 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:14.734 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:14 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:14.734 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:14 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:15.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:15 smithi033 ceph-mon[26353]: [25/Apr/2024:01:27:13] ENGINE Bus STARTING 2024-04-25T01:27:15.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:15 smithi033 ceph-mon[26353]: [25/Apr/2024:01:27:13] ENGINE Serving on https://172.21.15.33:7150 2024-04-25T01:27:15.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:15 smithi033 ceph-mon[26353]: [25/Apr/2024:01:27:13] ENGINE Bus STARTED 2024-04-25T01:27:15.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:15 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:16.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$q3UMT1iKO1rzLHR4kbBROu2/uVUJ/zo9ZzB/7oaeLeEehy9n.cfyC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714008435, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T01:27:16.139 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.594+0000 7f9edd5b0700 1 Processor -- start 2024-04-25T01:27:16.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.594+0000 7f9edd5b0700 1 -- start start 2024-04-25T01:27:16.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.594+0000 7f9edd5b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed8105400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:16.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.594+0000 7f9edd5b0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed8105940 con 0x7f9ed8076c50 2024-04-25T01:27:16.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.594+0000 7f9ed6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed8105400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:16.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.594+0000 7f9ed6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed8105400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53860/0 (socket says 172.21.15.33:53860) 2024-04-25T01:27:16.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.594+0000 7f9ed6ffd700 1 -- 172.21.15.33:0/26908958 learned_addr learned my addr 172.21.15.33:0/26908958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:16.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9ed6ffd700 1 -- 172.21.15.33:0/26908958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed8105a80 con 0x7f9ed8076c50 2024-04-25T01:27:16.140 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9ed6ffd700 1 --2- 172.21.15.33:0/26908958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed8105400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80098e0 tx=0x7f9ec8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4085a4230cf175 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9ed5ffb700 1 -- 172.21.15.33:0/26908958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ec800b860 con 0x7f9ed8076c50 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9ed5ffb700 1 -- 172.21.15.33:0/26908958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ec800be90 con 0x7f9ed8076c50 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9ed5ffb700 1 -- 172.21.15.33:0/26908958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ec80103c0 con 0x7f9ed8076c50 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9edd5b0700 1 -- 172.21.15.33:0/26908958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 msgr2=0x7f9ed8105400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9edd5b0700 1 --2- 172.21.15.33:0/26908958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed8105400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80098e0 tx=0x7f9ec8009250 comp rx=0 tx=0).stop 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9edd5b0700 1 -- 172.21.15.33:0/26908958 shutdown_connections 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9edd5b0700 1 --2- 172.21.15.33:0/26908958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed8105400 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.595+0000 7f9edd5b0700 1 -- 172.21.15.33:0/26908958 >> 172.21.15.33:0/26908958 conn(0x7f9ed80fcef0 msgr2=0x7f9ed80ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.596+0000 7f9edd5b0700 1 -- 172.21.15.33:0/26908958 shutdown_connections 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.596+0000 7f9edd5b0700 1 -- 172.21.15.33:0/26908958 wait complete. 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.596+0000 7f9edd5b0700 1 Processor -- start 2024-04-25T01:27:16.146 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.596+0000 7f9edd5b0700 1 -- start start 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.596+0000 7f9edd5b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed81115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.596+0000 7f9edd5b0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed8111b20 con 0x7f9ed8076c50 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9ed6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed81115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9ed6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed81115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53874/0 (socket says 172.21.15.33:53874) 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9ed6ffd700 1 -- 172.21.15.33:0/121662313 learned_addr learned my addr 172.21.15.33:0/121662313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9ed6ffd700 1 -- 172.21.15.33:0/121662313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80094f0 con 0x7f9ed8076c50 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9ed6ffd700 1 --2- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed81115e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8003080 tx=0x7f9ec8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9ecf7fe700 1 -- 172.21.15.33:0/121662313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ec8011920 con 0x7f9ed8076c50 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9ecf7fe700 1 -- 172.21.15.33:0/121662313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9ec8010d10 con 0x7f9ed8076c50 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed8111d20 con 0x7f9ed8076c50 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9ecf7fe700 1 -- 172.21.15.33:0/121662313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ec801ae50 con 0x7f9ed8076c50 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.597+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed8112890 con 0x7f9ed8076c50 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.598+0000 7f9ecf7fe700 1 -- 172.21.15.33:0/121662313 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f9ec8024480 con 0x7f9ed8076c50 2024-04-25T01:27:16.147 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.598+0000 7f9ecf7fe700 1 --2- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9ec40368a0 0x7f9ec4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.598+0000 7f9ecf7fe700 1 -- 172.21.15.33:0/121662313 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f9ec804a910 con 0x7f9ed8076c50 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.598+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed805f5c0 con 0x7f9ed8076c50 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.598+0000 7f9ed67fc700 1 --2- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9ec40368a0 0x7f9ec4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.599+0000 7f9ed67fc700 1 --2- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9ec40368a0 0x7f9ec4038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009990 tx=0x7f9ec0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.603+0000 7f9ecf7fe700 1 -- 172.21.15.33:0/121662313 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9ec8023b40 con 0x7f9ed8076c50 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:14.822+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 --> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9ed805f3b0 con 0x7f9ec40368a0 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.069+0000 7f9ecf7fe700 1 -- 172.21.15.33:0/121662313 <== mgr.14160 v2:172.21.15.33:6800/395184027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f9ed805f3b0 con 0x7f9ec40368a0 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9ec40368a0 msgr2=0x7f9ec4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 --2- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9ec40368a0 0x7f9ec4038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ec0009990 tx=0x7f9ec0006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 msgr2=0x7f9ed81115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 --2- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed81115e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8003080 tx=0x7f9ec8009250 comp rx=0 tx=0).stop 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 shutdown_connections 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 --2- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9ec40368a0 0x7f9ec4038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:16.148 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 --2- 172.21.15.33:0/121662313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9ed8076c50 0x7f9ed81115e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:16.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 >> 172.21.15.33:0/121662313 conn(0x7f9ed80fcef0 msgr2=0x7f9ed80fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:16.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 shutdown_connections 2024-04-25T01:27:16.149 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:15.071+0000 7f9edd5b0700 1 -- 172.21.15.33:0/121662313 wait complete. 2024-04-25T01:27:16.149 INFO:teuthology.orchestra.run.smithi033.stdout:Fetching dashboard port number... 2024-04-25T01:27:16.705 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:16 smithi033 ceph-mon[26353]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:27:17.517 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:17 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2156730469' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T01:27:17.526 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T01:27:17.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.750+0000 7f68bd740700 1 Processor -- start 2024-04-25T01:27:17.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.750+0000 7f68bd740700 1 -- start start 2024-04-25T01:27:17.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.750+0000 7f68bd740700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8105460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:17.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.750+0000 7f68bd740700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b81059a0 con 0x7f68b8076bf0 2024-04-25T01:27:17.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.750+0000 7f68b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8105460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:17.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.750+0000 7f68b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8105460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47120/0 (socket says 172.21.15.33:47120) 2024-04-25T01:27:17.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.750+0000 7f68b6ffd700 1 -- 172.21.15.33:0/3011749893 learned_addr learned my addr 172.21.15.33:0/3011749893 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:17.527 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.751+0000 7f68b6ffd700 1 -- 172.21.15.33:0/3011749893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b8105ae0 con 0x7f68b8076bf0 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.751+0000 7f68b6ffd700 1 --2- 172.21.15.33:0/3011749893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8105460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68b00098e0 tx=0x7f68b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94236712effce7fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.751+0000 7f68b5ffb700 1 -- 172.21.15.33:0/3011749893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b000b860 con 0x7f68b8076bf0 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.751+0000 7f68b5ffb700 1 -- 172.21.15.33:0/3011749893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b000be90 con 0x7f68b8076bf0 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.751+0000 7f68b5ffb700 1 -- 172.21.15.33:0/3011749893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b00103c0 con 0x7f68b8076bf0 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.752+0000 7f68bd740700 1 -- 172.21.15.33:0/3011749893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 msgr2=0x7f68b8105460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.752+0000 7f68bd740700 1 --2- 172.21.15.33:0/3011749893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8105460 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68b00098e0 tx=0x7f68b0009250 comp rx=0 tx=0).stop 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.752+0000 7f68bd740700 1 -- 172.21.15.33:0/3011749893 shutdown_connections 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.752+0000 7f68bd740700 1 --2- 172.21.15.33:0/3011749893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8105460 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.752+0000 7f68bd740700 1 -- 172.21.15.33:0/3011749893 >> 172.21.15.33:0/3011749893 conn(0x7f68b80fcef0 msgr2=0x7f68b80ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.752+0000 7f68bd740700 1 -- 172.21.15.33:0/3011749893 shutdown_connections 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.752+0000 7f68bd740700 1 -- 172.21.15.33:0/3011749893 wait complete. 2024-04-25T01:27:17.528 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.752+0000 7f68bd740700 1 Processor -- start 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.752+0000 7f68bd740700 1 -- start start 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68bd740700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8111640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68bd740700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b8111b80 con 0x7f68b8076bf0 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8111640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68b6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8111640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47134/0 (socket says 172.21.15.33:47134) 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68b6ffd700 1 -- 172.21.15.33:0/2156730469 learned_addr learned my addr 172.21.15.33:0/2156730469 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68b6ffd700 1 -- 172.21.15.33:0/2156730469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b00094f0 con 0x7f68b8076bf0 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68b6ffd700 1 --2- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8111640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68b0006d40 tx=0x7f68b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68af7fe700 1 -- 172.21.15.33:0/2156730469 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b0010840 con 0x7f68b8076bf0 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68af7fe700 1 -- 172.21.15.33:0/2156730469 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b0010e70 con 0x7f68b8076bf0 2024-04-25T01:27:17.529 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b8111d80 con 0x7f68b8076bf0 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68af7fe700 1 -- 172.21.15.33:0/2156730469 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b0024ea0 con 0x7f68b8076bf0 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.753+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b81128f0 con 0x7f68b8076bf0 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.754+0000 7f68af7fe700 1 -- 172.21.15.33:0/2156730469 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f68b00109a0 con 0x7f68b8076bf0 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.754+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b805f5c0 con 0x7f68b8076bf0 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.754+0000 7f68af7fe700 1 --2- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f68a4036850 0x7f68a4038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.754+0000 7f68af7fe700 1 -- 172.21.15.33:0/2156730469 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f68b004a800 con 0x7f68b8076bf0 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.755+0000 7f68b67fc700 1 --2- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f68a4036850 0x7f68a4038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.755+0000 7f68b67fc700 1 --2- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f68a4036850 0x7f68a4038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009990 tx=0x7f68a0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.759+0000 7f68af7fe700 1 -- 172.21.15.33:0/2156730469 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f68b002c6c0 con 0x7f68b8076bf0 2024-04-25T01:27:17.530 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.954+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f68b804bed0 con 0x7f68b8076bf0 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.955+0000 7f68af7fe700 1 -- 172.21.15.33:0/2156730469 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f68b002c8d0 con 0x7f68b8076bf0 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f68a4036850 msgr2=0x7f68a4038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 --2- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f68a4036850 0x7f68a4038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009990 tx=0x7f68a0006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 msgr2=0x7f68b8111640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 --2- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8111640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68b0006d40 tx=0x7f68b0009250 comp rx=0 tx=0).stop 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 shutdown_connections 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 --2- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f68a4036850 0x7f68a4038d00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 --2- 172.21.15.33:0/2156730469 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68b8076bf0 0x7f68b8111640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 >> 172.21.15.33:0/2156730469 conn(0x7f68b80fcef0 msgr2=0x7f68b80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 shutdown_connections 2024-04-25T01:27:17.531 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:16.957+0000 7f68bd740700 1 -- 172.21.15.33:0/2156730469 wait complete. 2024-04-25T01:27:17.532 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T01:27:17.532 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout disabled 2024-04-25T01:27:17.540 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T01:27:17.540 INFO:teuthology.orchestra.run.smithi033.stdout:systemctl: stdout inactive 2024-04-25T01:27:17.540 INFO:teuthology.orchestra.run.smithi033.stdout:firewalld.service is not enabled 2024-04-25T01:27:17.540 INFO:teuthology.orchestra.run.smithi033.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T01:27:17.542 INFO:teuthology.orchestra.run.smithi033.stdout:Ceph Dashboard is now available at: 2024-04-25T01:27:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: URL: https://smithi033:8443/ 2024-04-25T01:27:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: User: admin 2024-04-25T01:27:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: Password: pekct1zykv 2024-04-25T01:27:17.542 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:17.543 INFO:teuthology.orchestra.run.smithi033.stdout:Saving cluster configuration to /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config directory 2024-04-25T01:27:17.543 INFO:teuthology.orchestra.run.smithi033.stdout:Enabling autotune for osd_memory_target 2024-04-25T01:27:18.723 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:18 smithi033 ceph-mon[26353]: mgrmap e13: smithi033.asemba(active, since 6s) 2024-04-25T01:27:18.726 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.977+0000 7f9498185700 1 Processor -- start 2024-04-25T01:27:18.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.977+0000 7f9498185700 1 -- start start 2024-04-25T01:27:18.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.978+0000 7f9498185700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490076100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:18.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.978+0000 7f9498185700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490077f60 con 0x7f9490077b50 2024-04-25T01:27:18.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.978+0000 7f9495f21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490076100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:18.731 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.978+0000 7f9495f21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490076100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47136/0 (socket says 172.21.15.33:47136) 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.978+0000 7f9495f21700 1 -- 172.21.15.33:0/1011010691 learned_addr learned my addr 172.21.15.33:0/1011010691 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.978+0000 7f9495f21700 1 -- 172.21.15.33:0/1011010691 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9490076640 con 0x7f9490077b50 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.978+0000 7f9495f21700 1 --2- 172.21.15.33:0/1011010691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490076100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f948c0098e0 tx=0x7f948c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c54caf8347a6ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.978+0000 7f9494f1f700 1 -- 172.21.15.33:0/1011010691 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f948c00b860 con 0x7f9490077b50 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.979+0000 7f9494f1f700 1 -- 172.21.15.33:0/1011010691 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f948c00be90 con 0x7f9490077b50 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.979+0000 7f9494f1f700 1 -- 172.21.15.33:0/1011010691 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f948c0103c0 con 0x7f9490077b50 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.979+0000 7f9498185700 1 -- 172.21.15.33:0/1011010691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 msgr2=0x7f9490076100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.979+0000 7f9498185700 1 --2- 172.21.15.33:0/1011010691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490076100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f948c0098e0 tx=0x7f948c009250 comp rx=0 tx=0).stop 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.979+0000 7f9498185700 1 -- 172.21.15.33:0/1011010691 shutdown_connections 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.979+0000 7f9498185700 1 --2- 172.21.15.33:0/1011010691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490076100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:18.732 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.979+0000 7f9498185700 1 -- 172.21.15.33:0/1011010691 >> 172.21.15.33:0/1011010691 conn(0x7f94900fdf90 msgr2=0x7f9490100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.979+0000 7f9498185700 1 -- 172.21.15.33:0/1011010691 shutdown_connections 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.979+0000 7f9498185700 1 -- 172.21.15.33:0/1011010691 wait complete. 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.980+0000 7f9498185700 1 Processor -- start 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.980+0000 7f9498185700 1 -- start start 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.980+0000 7f9498185700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490113830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.980+0000 7f9498185700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490113d70 con 0x7f9490077b50 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.980+0000 7f9495f21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490113830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.980+0000 7f9495f21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490113830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47140/0 (socket says 172.21.15.33:47140) 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.980+0000 7f9495f21700 1 -- 172.21.15.33:0/247455153 learned_addr learned my addr 172.21.15.33:0/247455153 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.980+0000 7f9495f21700 1 -- 172.21.15.33:0/247455153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f948c0094f0 con 0x7f9490077b50 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.980+0000 7f9495f21700 1 --2- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490113830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f948c006d40 tx=0x7f948c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.981+0000 7f94867fc700 1 -- 172.21.15.33:0/247455153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f948c010840 con 0x7f9490077b50 2024-04-25T01:27:18.733 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.981+0000 7f94867fc700 1 -- 172.21.15.33:0/247455153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f948c010e70 con 0x7f9490077b50 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.981+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9490113f70 con 0x7f9490077b50 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.981+0000 7f94867fc700 1 -- 172.21.15.33:0/247455153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f948c024ea0 con 0x7f9490077b50 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.981+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9490114ae0 con 0x7f9490077b50 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.982+0000 7f94867fc700 1 -- 172.21.15.33:0/247455153 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f948c0109a0 con 0x7f9490077b50 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.982+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949005f5c0 con 0x7f9490077b50 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.982+0000 7f94867fc700 1 --2- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f947c036940 0x7f947c038df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.982+0000 7f94867fc700 1 -- 172.21.15.33:0/247455153 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f948c04a890 con 0x7f9490077b50 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.982+0000 7f9495720700 1 --2- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f947c036940 0x7f947c038df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.982+0000 7f9495720700 1 --2- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f947c036940 0x7f947c038df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9480009990 tx=0x7f9480006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:17.986+0000 7f94867fc700 1 -- 172.21.15.33:0/247455153 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f948c02e440 con 0x7f9490077b50 2024-04-25T01:27:18.734 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.182+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f949004bed0 con 0x7f9490077b50 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.182+0000 7f94867fc700 1 -- 172.21.15.33:0/247455153 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f948c02c7b0 con 0x7f9490077b50 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f947c036940 msgr2=0x7f947c038df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 --2- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f947c036940 0x7f947c038df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9480009990 tx=0x7f9480006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 msgr2=0x7f9490113830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 --2- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490113830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f948c006d40 tx=0x7f948c009250 comp rx=0 tx=0).stop 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 shutdown_connections 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 --2- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f947c036940 0x7f947c038df0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 --2- 172.21.15.33:0/247455153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9490077b50 0x7f9490113830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 >> 172.21.15.33:0/247455153 conn(0x7f94900fdf90 msgr2=0x7f94900feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 shutdown_connections 2024-04-25T01:27:18.735 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:18.184+0000 7f9498185700 1 -- 172.21.15.33:0/247455153 wait complete. 2024-04-25T01:27:20.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.172+0000 7fddd8693700 1 Processor -- start 2024-04-25T01:27:20.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.172+0000 7fddd8693700 1 -- start start 2024-04-25T01:27:20.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd8693700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:20.029 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd8693700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01031d0 con 0x7fddd0102880 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd642f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd642f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47144/0 (socket says 172.21.15.33:47144) 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd642f700 1 -- 172.21.15.33:0/1165202372 learned_addr learned my addr 172.21.15.33:0/1165202372 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd642f700 1 -- 172.21.15.33:0/1165202372 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd0103310 con 0x7fddd0102880 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd642f700 1 --2- 172.21.15.33:0/1165202372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0102c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0098e0 tx=0x7fddcc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98a86f4051db0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd542d700 1 -- 172.21.15.33:0/1165202372 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddcc00b860 con 0x7fddd0102880 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd542d700 1 -- 172.21.15.33:0/1165202372 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddcc00be90 con 0x7fddd0102880 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.173+0000 7fddd542d700 1 -- 172.21.15.33:0/1165202372 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddcc0103c0 con 0x7fddd0102880 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.174+0000 7fddd8693700 1 -- 172.21.15.33:0/1165202372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 msgr2=0x7fddd0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.174+0000 7fddd8693700 1 --2- 172.21.15.33:0/1165202372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0102c90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0098e0 tx=0x7fddcc009250 comp rx=0 tx=0).stop 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.174+0000 7fddd8693700 1 -- 172.21.15.33:0/1165202372 shutdown_connections 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.174+0000 7fddd8693700 1 --2- 172.21.15.33:0/1165202372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0102c90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:20.030 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.174+0000 7fddd8693700 1 -- 172.21.15.33:0/1165202372 >> 172.21.15.33:0/1165202372 conn(0x7fddd00fe060 msgr2=0x7fddd0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.174+0000 7fddd8693700 1 -- 172.21.15.33:0/1165202372 shutdown_connections 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.174+0000 7fddd8693700 1 -- 172.21.15.33:0/1165202372 wait complete. 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd8693700 1 Processor -- start 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd8693700 1 -- start start 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd8693700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd8693700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0079160 con 0x7fddd0102880 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd642f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd642f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47158/0 (socket says 172.21.15.33:47158) 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd642f700 1 -- 172.21.15.33:0/871488433 learned_addr learned my addr 172.21.15.33:0/871488433 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd642f700 1 -- 172.21.15.33:0/871488433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc0094f0 con 0x7fddd0102880 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd642f700 1 --2- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0078c20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0090b0 tx=0x7fddcc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddc2ffd700 1 -- 172.21.15.33:0/871488433 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddcc010840 con 0x7fddd0102880 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd00771d0 con 0x7fddd0102880 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddc2ffd700 1 -- 172.21.15.33:0/871488433 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddcc010e70 con 0x7fddd0102880 2024-04-25T01:27:20.031 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.175+0000 7fddc2ffd700 1 -- 172.21.15.33:0/871488433 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddcc024e40 con 0x7fddd0102880 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.176+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd00776c0 con 0x7fddd0102880 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.176+0000 7fddc2ffd700 1 -- 172.21.15.33:0/871488433 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fddcc01a530 con 0x7fddd0102880 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.176+0000 7fddc2ffd700 1 --2- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fddbc0368f0 0x7fddbc038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.176+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd004bed0 con 0x7fddd0102880 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.176+0000 7fddc2ffd700 1 -- 172.21.15.33:0/871488433 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fddcc04a9f0 con 0x7fddd0102880 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.177+0000 7fddd5c2e700 1 --2- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fddbc0368f0 0x7fddbc038da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.177+0000 7fddd5c2e700 1 --2- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fddbc0368f0 0x7fddbc038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009990 tx=0x7fddc4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.181+0000 7fddc2ffd700 1 -- 172.21.15.33:0/871488433 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fddcc018940 con 0x7fddd0102880 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.492+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fddd0077c00 con 0x7fddd0102880 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.494+0000 7fddc2ffd700 1 -- 172.21.15.33:0/871488433 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7fddcc02d3a0 con 0x7fddd0102880 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.496+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fddbc0368f0 msgr2=0x7fddbc038da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.496+0000 7fddd8693700 1 --2- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fddbc0368f0 0x7fddbc038da0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009990 tx=0x7fddc4006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.496+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 msgr2=0x7fddd0078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:20.032 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.496+0000 7fddd8693700 1 --2- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0078c20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fddcc0090b0 tx=0x7fddcc009250 comp rx=0 tx=0).stop 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.496+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 shutdown_connections 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.496+0000 7fddd8693700 1 --2- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fddbc0368f0 0x7fddbc038da0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.496+0000 7fddd8693700 1 --2- 172.21.15.33:0/871488433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fddd0102880 0x7fddd0078c20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.496+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 >> 172.21.15.33:0/871488433 conn(0x7fddd00fe060 msgr2=0x7fddd00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.496+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 shutdown_connections 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr 2024-04-25T01:27:19.497+0000 7fddd8693700 1 -- 172.21.15.33:0/871488433 wait complete. 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T01:27:20.033 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:20.034 INFO:teuthology.orchestra.run.smithi033.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T01:27:20.034 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:20.034 INFO:teuthology.orchestra.run.smithi033.stdout: ceph telemetry on 2024-04-25T01:27:20.034 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:20.034 INFO:teuthology.orchestra.run.smithi033.stdout:For more information see: 2024-04-25T01:27:20.034 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:20.034 INFO:teuthology.orchestra.run.smithi033.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-04-25T01:27:20.034 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:20.034 INFO:teuthology.orchestra.run.smithi033.stdout:Bootstrap complete. 2024-04-25T01:27:20.065 INFO:tasks.cephadm:Fetching config... 2024-04-25T01:27:20.065 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:27:20.065 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T01:27:20.082 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T01:27:20.082 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:27:20.082 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T01:27:20.140 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T01:27:20.140 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:27:20.140 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/keyring of=/dev/stdout 2024-04-25T01:27:20.204 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T01:27:20.205 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:27:20.205 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T01:27:20.260 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T01:27:20.260 DEBUG:teuthology.orchestra.run.smithi033:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5OwZBkWrg2DE5htToC59jXuSLRSt7WxyiLLwpS/l5uAkojIbAw5kYefVeR61Eckc0zciNGbm50ALIGZObQXeh58TBwgdXjeiumtkMzUzDnZOKhMpAhiEHkWsHkF4mscPDzn/oKHm/AAqf6xh7dNkfM2CyhFk5HEydyMp0p2wLyWBD79DDMihjBJ30lzD0HT6f+J6ndRHgSX4gTrcV/iCeHn0ZPLEsOdcEWzp4VJxZL5VZzZvYFL4kmEHNtKLkhxl5yCrLbNURSyBSnK74BDpcRhHxeeOsnsPKWh71A3ZzFGGavccQgnpjv+oQ3d2m7wx+PsibCFY6n4jraXLiufM2SjNB2OZlMrvPMcMihwTVq0hbkHFcHaEUbtKSSS36kXeDjfisXlAwPI2jnSzb9GxZVBCiT5aLFz31qs+rvN81dxog6wgF3mlkz3QCepRTCJp67PblwEmnFlbZ8CdKjgDffElAyqGBCPmzkL6YsOw2/iqmzAupnIfNor5p8lAOs40= ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T01:27:20.346 INFO:teuthology.orchestra.run.smithi033.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5OwZBkWrg2DE5htToC59jXuSLRSt7WxyiLLwpS/l5uAkojIbAw5kYefVeR61Eckc0zciNGbm50ALIGZObQXeh58TBwgdXjeiumtkMzUzDnZOKhMpAhiEHkWsHkF4mscPDzn/oKHm/AAqf6xh7dNkfM2CyhFk5HEydyMp0p2wLyWBD79DDMihjBJ30lzD0HT6f+J6ndRHgSX4gTrcV/iCeHn0ZPLEsOdcEWzp4VJxZL5VZzZvYFL4kmEHNtKLkhxl5yCrLbNURSyBSnK74BDpcRhHxeeOsnsPKWh71A3ZzFGGavccQgnpjv+oQ3d2m7wx+PsibCFY6n4jraXLiufM2SjNB2OZlMrvPMcMihwTVq0hbkHFcHaEUbtKSSS36kXeDjfisXlAwPI2jnSzb9GxZVBCiT5aLFz31qs+rvN81dxog6wgF3mlkz3QCepRTCJp67PblwEmnFlbZ8CdKjgDffElAyqGBCPmzkL6YsOw2/iqmzAupnIfNor5p8lAOs40= ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:27:20.359 DEBUG:teuthology.orchestra.run.smithi138:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5OwZBkWrg2DE5htToC59jXuSLRSt7WxyiLLwpS/l5uAkojIbAw5kYefVeR61Eckc0zciNGbm50ALIGZObQXeh58TBwgdXjeiumtkMzUzDnZOKhMpAhiEHkWsHkF4mscPDzn/oKHm/AAqf6xh7dNkfM2CyhFk5HEydyMp0p2wLyWBD79DDMihjBJ30lzD0HT6f+J6ndRHgSX4gTrcV/iCeHn0ZPLEsOdcEWzp4VJxZL5VZzZvYFL4kmEHNtKLkhxl5yCrLbNURSyBSnK74BDpcRhHxeeOsnsPKWh71A3ZzFGGavccQgnpjv+oQ3d2m7wx+PsibCFY6n4jraXLiufM2SjNB2OZlMrvPMcMihwTVq0hbkHFcHaEUbtKSSS36kXeDjfisXlAwPI2jnSzb9GxZVBCiT5aLFz31qs+rvN81dxog6wgF3mlkz3QCepRTCJp67PblwEmnFlbZ8CdKjgDffElAyqGBCPmzkL6YsOw2/iqmzAupnIfNor5p8lAOs40= ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T01:27:20.398 INFO:teuthology.orchestra.run.smithi138.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC5OwZBkWrg2DE5htToC59jXuSLRSt7WxyiLLwpS/l5uAkojIbAw5kYefVeR61Eckc0zciNGbm50ALIGZObQXeh58TBwgdXjeiumtkMzUzDnZOKhMpAhiEHkWsHkF4mscPDzn/oKHm/AAqf6xh7dNkfM2CyhFk5HEydyMp0p2wLyWBD79DDMihjBJ30lzD0HT6f+J6ndRHgSX4gTrcV/iCeHn0ZPLEsOdcEWzp4VJxZL5VZzZvYFL4kmEHNtKLkhxl5yCrLbNURSyBSnK74BDpcRhHxeeOsnsPKWh71A3ZzFGGavccQgnpjv+oQ3d2m7wx+PsibCFY6n4jraXLiufM2SjNB2OZlMrvPMcMihwTVq0hbkHFcHaEUbtKSSS36kXeDjfisXlAwPI2jnSzb9GxZVBCiT5aLFz31qs+rvN81dxog6wgF3mlkz3QCepRTCJp67PblwEmnFlbZ8CdKjgDffElAyqGBCPmzkL6YsOw2/iqmzAupnIfNor5p8lAOs40= ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:27:20.409 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T01:27:20.669 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:27:20.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:20 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/871488433' entity='client.admin' 2024-04-25T01:27:23.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.182+0000 7f943a6cb700 1 -- 172.21.15.33:0/3555049710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 msgr2=0x7f94340ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:23.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.182+0000 7f943a6cb700 1 --2- 172.21.15.33:0/3555049710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 0x7f94340ff630 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f941c0048c0 tx=0x7f941c009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:23.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.182+0000 7f943a6cb700 1 -- 172.21.15.33:0/3555049710 shutdown_connections 2024-04-25T01:27:23.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.182+0000 7f943a6cb700 1 --2- 172.21.15.33:0/3555049710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 0x7f94340ff630 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:23.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.182+0000 7f943a6cb700 1 -- 172.21.15.33:0/3555049710 >> 172.21.15.33:0/3555049710 conn(0x7f94340f70a0 msgr2=0x7f94340f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:23.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.182+0000 7f943a6cb700 1 -- 172.21.15.33:0/3555049710 shutdown_connections 2024-04-25T01:27:23.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.182+0000 7f943a6cb700 1 -- 172.21.15.33:0/3555049710 wait complete. 2024-04-25T01:27:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.182+0000 7f943a6cb700 1 Processor -- start 2024-04-25T01:27:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.183+0000 7f943a6cb700 1 -- start start 2024-04-25T01:27:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.183+0000 7f943a6cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 0x7f943410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.183+0000 7f943a6cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943410d5d0 con 0x7f9434065d30 2024-04-25T01:27:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.183+0000 7f9433fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 0x7f943410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.183+0000 7f9433fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 0x7f943410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47178/0 (socket says 172.21.15.33:47178) 2024-04-25T01:27:23.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.183+0000 7f9433fff700 1 -- 172.21.15.33:0/2217703774 learned_addr learned my addr 172.21.15.33:0/2217703774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.183+0000 7f9433fff700 1 -- 172.21.15.33:0/2217703774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c005040 con 0x7f9434065d30 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.183+0000 7f9433fff700 1 --2- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 0x7f943410d090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f941c009680 tx=0x7f941c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.183+0000 7f9430ff9700 1 -- 172.21.15.33:0/2217703774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f941c00bed0 con 0x7f9434065d30 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.184+0000 7f9430ff9700 1 -- 172.21.15.33:0/2217703774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f941c00b5c0 con 0x7f9434065d30 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.184+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943410d7d0 con 0x7f9434065d30 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.184+0000 7f9430ff9700 1 -- 172.21.15.33:0/2217703774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f941c01c5b0 con 0x7f9434065d30 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.184+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943410e340 con 0x7f9434065d30 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.185+0000 7f9430ff9700 1 -- 172.21.15.33:0/2217703774 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f941c01c710 con 0x7f9434065d30 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.185+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9434102f40 con 0x7f9434065d30 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.185+0000 7f9430ff9700 1 --2- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9420036880 0x7f9420038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.185+0000 7f9430ff9700 1 -- 172.21.15.33:0/2217703774 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f941c049ab0 con 0x7f9434065d30 2024-04-25T01:27:23.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.185+0000 7f94337fe700 1 --2- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9420036880 0x7f9420038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:23.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.186+0000 7f94337fe700 1 --2- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9420036880 0x7f9420038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9424009990 tx=0x7f9424006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:23.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.190+0000 7f9430ff9700 1 -- 172.21.15.33:0/2217703774 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f941c025780 con 0x7f9434065d30 2024-04-25T01:27:23.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:22 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:23.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:22 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:23.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:22 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:27:23.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:22 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:23.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:22 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:27:23.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:22 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-25T01:27:23.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:22 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:27:23.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:22 smithi033 ceph-mon[26353]: Deploying daemon crash.smithi033 on smithi033 2024-04-25T01:27:23.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.386+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f943410ee10 con 0x7f9434065d30 2024-04-25T01:27:23.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.391+0000 7f9430ff9700 1 -- 172.21.15.33:0/2217703774 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f941c011370 con 0x7f9434065d30 2024-04-25T01:27:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9420036880 msgr2=0x7f9420038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 --2- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9420036880 0x7f9420038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9424009990 tx=0x7f9424006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 msgr2=0x7f943410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 --2- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 0x7f943410d090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f941c009680 tx=0x7f941c009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 shutdown_connections 2024-04-25T01:27:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 --2- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f9420036880 0x7f9420038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 --2- 172.21.15.33:0/2217703774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9434065d30 0x7f943410d090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 >> 172.21.15.33:0/2217703774 conn(0x7f94340f70a0 msgr2=0x7f9434103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 shutdown_connections 2024-04-25T01:27:23.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:23.394+0000 7f943a6cb700 1 -- 172.21.15.33:0/2217703774 wait complete. 2024-04-25T01:27:24.014 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T01:27:24.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T01:27:24.607 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:27:24.622 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:24 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2217703774' entity='client.admin' 2024-04-25T01:27:27.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.022+0000 7f593e161700 1 -- 172.21.15.33:0/1146657909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 msgr2=0x7f59380fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.022+0000 7f593e161700 1 --2- 172.21.15.33:0/1146657909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 0x7f59380fe490 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5928009c60 tx=0x7f5928004d10 comp rx=0 tx=0).stop 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.022+0000 7f593e161700 1 -- 172.21.15.33:0/1146657909 shutdown_connections 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.022+0000 7f593e161700 1 --2- 172.21.15.33:0/1146657909 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 0x7f59380fe490 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.022+0000 7f593e161700 1 -- 172.21.15.33:0/1146657909 >> 172.21.15.33:0/1146657909 conn(0x7f59380f6820 msgr2=0x7f59380f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.022+0000 7f593e161700 1 -- 172.21.15.33:0/1146657909 shutdown_connections 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.022+0000 7f593e161700 1 -- 172.21.15.33:0/1146657909 wait complete. 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.023+0000 7f593e161700 1 Processor -- start 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.023+0000 7f593e161700 1 -- start start 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.023+0000 7f593e161700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 0x7f59380fd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.023+0000 7f593e161700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59380fdcd0 con 0x7f59380fe080 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.023+0000 7f59377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 0x7f59380fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.023+0000 7f59377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 0x7f59380fd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38356/0 (socket says 172.21.15.33:38356) 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.023+0000 7f59377fe700 1 -- 172.21.15.33:0/2622882564 learned_addr learned my addr 172.21.15.33:0/2622882564 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.023+0000 7f59377fe700 1 -- 172.21.15.33:0/2622882564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5928009940 con 0x7f59380fe080 2024-04-25T01:27:27.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.024+0000 7f59377fe700 1 --2- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 0x7f59380fd790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5928004e60 tx=0x7f5928004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.024+0000 7f592ffff700 1 -- 172.21.15.33:0/2622882564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5928012410 con 0x7f59380fe080 2024-04-25T01:27:27.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.024+0000 7f592ffff700 1 -- 172.21.15.33:0/2622882564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5928012a40 con 0x7f59380fe080 2024-04-25T01:27:27.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.024+0000 7f592ffff700 1 -- 172.21.15.33:0/2622882564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f592801ba40 con 0x7f59380fe080 2024-04-25T01:27:27.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.024+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59380fbda0 con 0x7f59380fe080 2024-04-25T01:27:27.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.024+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59380fc1d0 con 0x7f59380fe080 2024-04-25T01:27:27.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.025+0000 7f592ffff700 1 -- 172.21.15.33:0/2622882564 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f5928019040 con 0x7f59380fe080 2024-04-25T01:27:27.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.025+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593805f5c0 con 0x7f59380fe080 2024-04-25T01:27:27.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.025+0000 7f592ffff700 1 --2- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59240368d0 0x7f5924038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:27.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.025+0000 7f592ffff700 1 -- 172.21.15.33:0/2622882564 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5928049b70 con 0x7f59380fe080 2024-04-25T01:27:27.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.025+0000 7f5936ffd700 1 --2- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59240368d0 0x7f5924038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:27.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.026+0000 7f5936ffd700 1 --2- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59240368d0 0x7f5924038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5920009990 tx=0x7f5920006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:27.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.030+0000 7f592ffff700 1 -- 172.21.15.33:0/2622882564 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f592802b8d0 con 0x7f59380fe080 2024-04-25T01:27:27.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.254+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 --> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f59381064c0 con 0x7f59240368d0 2024-04-25T01:27:27.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.257+0000 7f592ffff700 1 -- 172.21.15.33:0/2622882564 <== mgr.14160 v2:172.21.15.33:6800/395184027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f59381064c0 con 0x7f59240368d0 2024-04-25T01:27:27.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59240368d0 msgr2=0x7f5924038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 --2- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59240368d0 0x7f5924038d80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5920009990 tx=0x7f5920006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 msgr2=0x7f59380fd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 --2- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 0x7f59380fd790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5928004e60 tx=0x7f5928004d10 comp rx=0 tx=0).stop 2024-04-25T01:27:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 shutdown_connections 2024-04-25T01:27:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 --2- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59240368d0 0x7f5924038d80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 --2- 172.21.15.33:0/2622882564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59380fe080 0x7f59380fd790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 >> 172.21.15.33:0/2622882564 conn(0x7f59380f6820 msgr2=0x7f5938103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 shutdown_connections 2024-04-25T01:27:27.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:27.259+0000 7f593e161700 1 -- 172.21.15.33:0/2622882564 wait complete. 2024-04-25T01:27:28.251 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi138 2024-04-25T01:27:28.251 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:27:28.251 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.conf 2024-04-25T01:27:28.270 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:27:28.271 DEBUG:teuthology.orchestra.run.smithi138:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:27:28.328 INFO:tasks.cephadm:Adding host smithi138 to orchestrator... 2024-04-25T01:27:28.328 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch host add smithi138 2024-04-25T01:27:28.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:28 smithi033 ceph-mon[26353]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:27:28.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:28 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:28.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:28 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:28.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:28 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:28.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:28 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:28.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:28 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:28.756 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:27:29.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:29 smithi033 ceph-mon[26353]: Deploying daemon node-exporter.smithi033 on smithi033 2024-04-25T01:27:30.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.358+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2815560826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 msgr2=0x7f59f00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.358+0000 7f59f5a7e700 1 --2- 172.21.15.33:0/2815560826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 0x7f59f00fce40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f59e00048c0 tx=0x7f59e0009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.358+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2815560826 shutdown_connections 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.358+0000 7f59f5a7e700 1 --2- 172.21.15.33:0/2815560826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 0x7f59f00fce40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.358+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2815560826 >> 172.21.15.33:0/2815560826 conn(0x7f59f00f8170 msgr2=0x7f59f00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.358+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2815560826 shutdown_connections 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.359+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2815560826 wait complete. 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.359+0000 7f59f5a7e700 1 Processor -- start 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.359+0000 7f59f5a7e700 1 -- start start 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.359+0000 7f59f5a7e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 0x7f59f010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.359+0000 7f59f5a7e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f010d5f0 con 0x7f59f00fca30 2024-04-25T01:27:30.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 0x7f59f010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59ef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 0x7f59f010d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38386/0 (socket says 172.21.15.33:38386) 2024-04-25T01:27:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59ef7fe700 1 -- 172.21.15.33:0/2520084228 learned_addr learned my addr 172.21.15.33:0/2520084228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59ef7fe700 1 -- 172.21.15.33:0/2520084228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59e0005040 con 0x7f59f00fca30 2024-04-25T01:27:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59ef7fe700 1 --2- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 0x7f59f010d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009680 tx=0x7f59e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59d7fff700 1 -- 172.21.15.33:0/2520084228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59e000b5c0 con 0x7f59f00fca30 2024-04-25T01:27:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59d7fff700 1 -- 172.21.15.33:0/2520084228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59e000be70 con 0x7f59f00fca30 2024-04-25T01:27:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59d7fff700 1 -- 172.21.15.33:0/2520084228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59e00135a0 con 0x7f59f00fca30 2024-04-25T01:27:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f010d7f0 con 0x7f59f00fca30 2024-04-25T01:27:30.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.360+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f010e360 con 0x7f59f00fca30 2024-04-25T01:27:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.361+0000 7f59d7fff700 1 -- 172.21.15.33:0/2520084228 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f59e001c430 con 0x7f59f00fca30 2024-04-25T01:27:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.361+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f0102f40 con 0x7f59f00fca30 2024-04-25T01:27:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.361+0000 7f59d7fff700 1 --2- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59d8036880 0x7f59d8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.361+0000 7f59d7fff700 1 -- 172.21.15.33:0/2520084228 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f59e0049390 con 0x7f59f00fca30 2024-04-25T01:27:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.361+0000 7f59eeffd700 1 --2- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59d8036880 0x7f59d8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:30.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.362+0000 7f59eeffd700 1 --2- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59d8036880 0x7f59d8038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009990 tx=0x7f59e4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:30.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.366+0000 7f59d7fff700 1 -- 172.21.15.33:0/2520084228 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f59e001c7d0 con 0x7f59f00fca30 2024-04-25T01:27:30.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:30.568+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 --> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}) v1 -- 0x7f59f010e5a0 con 0x7f59d8036880 2024-04-25T01:27:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:32 smithi033 ceph-mon[26353]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:27:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:32 smithi033 ceph-mon[26353]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:32 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:32 smithi033 ceph-mon[26353]: Deploying cephadm binary to smithi138 2024-04-25T01:27:32.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.612+0000 7f59d7fff700 1 -- 172.21.15.33:0/2520084228 <== mgr.14160 v2:172.21.15.33:6800/395184027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f59f010e5a0 con 0x7f59d8036880 2024-04-25T01:27:32.592 INFO:teuthology.orchestra.run.smithi033.stdout:Added host 'smithi138' with addr '172.21.15.138' 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.615+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59d8036880 msgr2=0x7f59d8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.615+0000 7f59f5a7e700 1 --2- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59d8036880 0x7f59d8038d30 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f59e4009990 tx=0x7f59e4006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.615+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 msgr2=0x7f59f010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.615+0000 7f59f5a7e700 1 --2- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 0x7f59f010d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009680 tx=0x7f59e0009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.615+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 shutdown_connections 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.615+0000 7f59f5a7e700 1 --2- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f59d8036880 0x7f59d8038d30 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.615+0000 7f59f5a7e700 1 --2- 172.21.15.33:0/2520084228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59f00fca30 0x7f59f010d0b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.615+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 >> 172.21.15.33:0/2520084228 conn(0x7f59f00f8170 msgr2=0x7f59f0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.615+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 shutdown_connections 2024-04-25T01:27:32.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:32.616+0000 7f59f5a7e700 1 -- 172.21.15.33:0/2520084228 wait complete. 2024-04-25T01:27:33.654 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T01:27:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:27:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:33 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:33 smithi033 ceph-mon[26353]: Added host smithi138 2024-04-25T01:27:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:33 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:33 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:33 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:33 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:33 smithi033 ceph-mon[26353]: Deploying daemon alertmanager.smithi033 on smithi033 2024-04-25T01:27:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:33 smithi033 ceph-mon[26353]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:35.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.832+0000 7f2cef097700 1 -- 172.21.15.33:0/2298965328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 msgr2=0x7f2ce80fce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.832+0000 7f2cef097700 1 --2- 172.21.15.33:0/2298965328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 0x7f2ce80fce30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0048c0 tx=0x7f2cdc009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.832+0000 7f2cef097700 1 -- 172.21.15.33:0/2298965328 shutdown_connections 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.832+0000 7f2cef097700 1 --2- 172.21.15.33:0/2298965328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 0x7f2ce80fce30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.832+0000 7f2cef097700 1 -- 172.21.15.33:0/2298965328 >> 172.21.15.33:0/2298965328 conn(0x7f2ce80f81a0 msgr2=0x7f2ce80fa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.832+0000 7f2cef097700 1 -- 172.21.15.33:0/2298965328 shutdown_connections 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.832+0000 7f2cef097700 1 -- 172.21.15.33:0/2298965328 wait complete. 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.832+0000 7f2cef097700 1 Processor -- start 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.832+0000 7f2cef097700 1 -- start start 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2cef097700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 0x7f2ce8071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:35.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2cef097700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8072240 con 0x7f2ce80fca20 2024-04-25T01:27:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2cece33700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 0x7f2ce8071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2cece33700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 0x7f2ce8071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54144/0 (socket says 172.21.15.33:54144) 2024-04-25T01:27:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2cece33700 1 -- 172.21.15.33:0/2387645931 learned_addr learned my addr 172.21.15.33:0/2387645931 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2cece33700 1 -- 172.21.15.33:0/2387645931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc005040 con 0x7f2ce80fca20 2024-04-25T01:27:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2cece33700 1 --2- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 0x7f2ce8071d00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0096b0 tx=0x7f2cdc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2ce57fa700 1 -- 172.21.15.33:0/2387645931 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cdc00bed0 con 0x7f2ce80fca20 2024-04-25T01:27:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce8070310 con 0x7f2ce80fca20 2024-04-25T01:27:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2ce57fa700 1 -- 172.21.15.33:0/2387645931 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cdc00b5c0 con 0x7f2ce80fca20 2024-04-25T01:27:35.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2ce57fa700 1 -- 172.21.15.33:0/2387645931 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2cdc01c5b0 con 0x7f2ce80fca20 2024-04-25T01:27:35.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.833+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce8070800 con 0x7f2ce80fca20 2024-04-25T01:27:35.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.834+0000 7f2ce57fa700 1 -- 172.21.15.33:0/2387645931 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f2cdc01c820 con 0x7f2ce80fca20 2024-04-25T01:27:35.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.834+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce805f5c0 con 0x7f2ce80fca20 2024-04-25T01:27:35.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.834+0000 7f2ce57fa700 1 --2- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f2cd8036880 0x7f2cd8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:35.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.834+0000 7f2ce57fa700 1 -- 172.21.15.33:0/2387645931 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f2cdc049bc0 con 0x7f2ce80fca20 2024-04-25T01:27:35.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.834+0000 7f2ce7fff700 1 --2- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f2cd8036880 0x7f2cd8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:35.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.835+0000 7f2ce7fff700 1 --2- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f2cd8036880 0x7f2cd8038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009990 tx=0x7f2cd4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:35.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:35.839+0000 7f2ce57fa700 1 -- 172.21.15.33:0/2387645931 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2cdc011d40 con 0x7f2ce80fca20 2024-04-25T01:27:36.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.042+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 --> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ce8106470 con 0x7f2cd8036880 2024-04-25T01:27:36.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.043+0000 7f2ce57fa700 1 -- 172.21.15.33:0/2387645931 <== mgr.14160 v2:172.21.15.33:6800/395184027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f2ce8106470 con 0x7f2cd8036880 2024-04-25T01:27:36.022 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:27:36.023 INFO:teuthology.orchestra.run.smithi033.stdout:[{"addr": "172.21.15.33", "hostname": "smithi033", "labels": [], "status": ""}, {"addr": "172.21.15.138", "hostname": "smithi138", "labels": [], "status": ""}] 2024-04-25T01:27:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f2cd8036880 msgr2=0x7f2cd8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 --2- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f2cd8036880 0x7f2cd8038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009990 tx=0x7f2cd4006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:36.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 msgr2=0x7f2ce8071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 --2- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 0x7f2ce8071d00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0096b0 tx=0x7f2cdc009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 shutdown_connections 2024-04-25T01:27:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 --2- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f2cd8036880 0x7f2cd8038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 --2- 172.21.15.33:0/2387645931 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ce80fca20 0x7f2ce8071d00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 >> 172.21.15.33:0/2387645931 conn(0x7f2ce80f81a0 msgr2=0x7f2ce8103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 shutdown_connections 2024-04-25T01:27:36.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:36.045+0000 7f2cef097700 1 -- 172.21.15.33:0/2387645931 wait complete. 2024-04-25T01:27:36.498 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:36 smithi033 ceph-mon[26353]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:36.498 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:36 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:37.127 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T01:27:37.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T01:27:37.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:37 smithi033 ceph-mon[26353]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:27:37.892 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:27:38.360 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:38 smithi033 ceph-mon[26353]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:40.359 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:40 smithi033 ceph-mon[26353]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:41.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.436+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1955272845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 msgr2=0x7f5a5c0fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.436+0000 7f5a61cbd700 1 --2- 172.21.15.33:0/1955272845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 0x7f5a5c0fe490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0048c0 tx=0x7f5a4c009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.436+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1955272845 shutdown_connections 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.436+0000 7f5a61cbd700 1 --2- 172.21.15.33:0/1955272845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 0x7f5a5c0fe490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.436+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1955272845 >> 172.21.15.33:0/1955272845 conn(0x7f5a5c0f6820 msgr2=0x7f5a5c0f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.436+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1955272845 shutdown_connections 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.436+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1955272845 wait complete. 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.437+0000 7f5a61cbd700 1 Processor -- start 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.437+0000 7f5a61cbd700 1 -- start start 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.437+0000 7f5a61cbd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 0x7f5a5c10d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.437+0000 7f5a61cbd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c10d5e0 con 0x7f5a5c0fe080 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.437+0000 7f5a5b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 0x7f5a5c10d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:41.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.437+0000 7f5a5b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 0x7f5a5c10d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54156/0 (socket says 172.21.15.33:54156) 2024-04-25T01:27:41.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.437+0000 7f5a5b7fe700 1 -- 172.21.15.33:0/1532800219 learned_addr learned my addr 172.21.15.33:0/1532800219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:27:41.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.438+0000 7f5a5b7fe700 1 -- 172.21.15.33:0/1532800219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4c005040 con 0x7f5a5c0fe080 2024-04-25T01:27:41.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.438+0000 7f5a5b7fe700 1 --2- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 0x7f5a5c10d0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0096b0 tx=0x7f5a4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.438+0000 7f5a43fff700 1 -- 172.21.15.33:0/1532800219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a4c00bed0 con 0x7f5a5c0fe080 2024-04-25T01:27:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.438+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c10d7e0 con 0x7f5a5c0fe080 2024-04-25T01:27:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.438+0000 7f5a43fff700 1 -- 172.21.15.33:0/1532800219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a4c00b5c0 con 0x7f5a5c0fe080 2024-04-25T01:27:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.438+0000 7f5a43fff700 1 -- 172.21.15.33:0/1532800219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a4c01c5b0 con 0x7f5a5c0fe080 2024-04-25T01:27:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.438+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c10e350 con 0x7f5a5c0fe080 2024-04-25T01:27:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.439+0000 7f5a43fff700 1 -- 172.21.15.33:0/1532800219 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f5a4c01c820 con 0x7f5a5c0fe080 2024-04-25T01:27:41.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.439+0000 7f5a43fff700 1 --2- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5a44036880 0x7f5a44038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:41.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.439+0000 7f5a43fff700 1 -- 172.21.15.33:0/1532800219 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5a4c049a00 con 0x7f5a5c0fe080 2024-04-25T01:27:41.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.439+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c102f50 con 0x7f5a5c0fe080 2024-04-25T01:27:41.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.439+0000 7f5a5affd700 1 --2- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5a44036880 0x7f5a44038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:41.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.439+0000 7f5a5affd700 1 --2- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5a44036880 0x7f5a44038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009990 tx=0x7f5a50006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:41.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.444+0000 7f5a43fff700 1 -- 172.21.15.33:0/1532800219 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5a4c023300 con 0x7f5a5c0fe080 2024-04-25T01:27:41.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:41.646+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5a5c05f5c0 con 0x7f5a5c0fe080 2024-04-25T01:27:42.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.252+0000 7f5a43fff700 1 -- 172.21.15.33:0/1532800219 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5a5c05f5c0 con 0x7f5a5c0fe080 2024-04-25T01:27:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5a44036880 msgr2=0x7f5a44038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 --2- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5a44036880 0x7f5a44038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009990 tx=0x7f5a50006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 msgr2=0x7f5a5c10d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:42.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 --2- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 0x7f5a5c10d0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c0096b0 tx=0x7f5a4c009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 shutdown_connections 2024-04-25T01:27:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 --2- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5a44036880 0x7f5a44038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 --2- 172.21.15.33:0/1532800219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a5c0fe080 0x7f5a5c10d0a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 >> 172.21.15.33:0/1532800219 conn(0x7f5a5c0f6820 msgr2=0x7f5a5c103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 shutdown_connections 2024-04-25T01:27:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:27:42.255+0000 7f5a61cbd700 1 -- 172.21.15.33:0/1532800219 wait complete. 2024-04-25T01:27:42.235 INFO:teuthology.orchestra.run.smithi033.stderr:adjusted tunables profile to default 2024-04-25T01:27:42.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:42 smithi033 ceph-mon[26353]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:42.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:42 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1532800219' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T01:27:42.806 INFO:tasks.cephadm:Adding mon.smithi033 on smithi033 2024-04-25T01:27:42.806 INFO:tasks.cephadm:Adding mon.smithi138 on smithi138 2024-04-25T01:27:42.807 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi033:172.21.15.33=smithi033;smithi138:172.21.15.138=smithi138' 2024-04-25T01:27:43.089 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:43.137 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:43.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:43 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1532800219' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T01:27:43.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:43 smithi033 ceph-mon[26353]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T01:27:44.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:44 smithi033 ceph-mon[26353]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:44.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:44 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:44.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:44 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:44.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:44 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:44.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:44 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:44.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:44 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:44.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:44 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:44.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:44 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-25T01:27:44.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:44 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:44.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.570+0000 7f52b845d700 1 -- 172.21.15.138:0/1595009572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 msgr2=0x7f52b00fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:44.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.570+0000 7f52b845d700 1 --2- 172.21.15.138:0/1595009572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 0x7f52b00fce40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0048c0 tx=0x7f52ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:44.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.570+0000 7f52b845d700 1 -- 172.21.15.138:0/1595009572 shutdown_connections 2024-04-25T01:27:44.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.570+0000 7f52b845d700 1 --2- 172.21.15.138:0/1595009572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 0x7f52b00fce40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:44.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.570+0000 7f52b845d700 1 -- 172.21.15.138:0/1595009572 >> 172.21.15.138:0/1595009572 conn(0x7f52b00f8170 msgr2=0x7f52b00fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:44.552 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.571+0000 7f52b845d700 1 -- 172.21.15.138:0/1595009572 shutdown_connections 2024-04-25T01:27:44.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.571+0000 7f52b845d700 1 -- 172.21.15.138:0/1595009572 wait complete. 2024-04-25T01:27:44.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.571+0000 7f52b845d700 1 Processor -- start 2024-04-25T01:27:44.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.571+0000 7f52b845d700 1 -- start start 2024-04-25T01:27:44.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.571+0000 7f52b845d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 0x7f52b010d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:44.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.571+0000 7f52b845d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b010d5d0 con 0x7f52b00fca30 2024-04-25T01:27:44.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.571+0000 7f52b61f9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 0x7f52b010d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:44.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.571+0000 7f52b61f9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 0x7f52b010d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:41352/0 (socket says 172.21.15.138:41352) 2024-04-25T01:27:44.553 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.571+0000 7f52b61f9700 1 -- 172.21.15.138:0/2718681650 learned_addr learned my addr 172.21.15.138:0/2718681650 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:27:44.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.572+0000 7f52b61f9700 1 -- 172.21.15.138:0/2718681650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac005040 con 0x7f52b00fca30 2024-04-25T01:27:44.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.572+0000 7f52b61f9700 1 --2- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 0x7f52b010d090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0096b0 tx=0x7f52ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:44.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.572+0000 7f52a6ffd700 1 -- 172.21.15.138:0/2718681650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52ac00bed0 con 0x7f52b00fca30 2024-04-25T01:27:44.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.572+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b010d7d0 con 0x7f52b00fca30 2024-04-25T01:27:44.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.572+0000 7f52a6ffd700 1 -- 172.21.15.138:0/2718681650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52ac00b5c0 con 0x7f52b00fca30 2024-04-25T01:27:44.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.572+0000 7f52a6ffd700 1 -- 172.21.15.138:0/2718681650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f52ac01c5b0 con 0x7f52b00fca30 2024-04-25T01:27:44.554 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.572+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b010e340 con 0x7f52b00fca30 2024-04-25T01:27:44.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.573+0000 7f52a6ffd700 1 -- 172.21.15.138:0/2718681650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f52ac013420 con 0x7f52b00fca30 2024-04-25T01:27:44.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.573+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b0102f40 con 0x7f52b00fca30 2024-04-25T01:27:44.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.573+0000 7f52a6ffd700 1 --2- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f529c0368d0 0x7f529c038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:44.555 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.573+0000 7f52a6ffd700 1 -- 172.21.15.138:0/2718681650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f52ac049ae0 con 0x7f52b00fca30 2024-04-25T01:27:44.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.573+0000 7f52b59f8700 1 --2- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f529c0368d0 0x7f529c038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:44.556 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.574+0000 7f52b59f8700 1 --2- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f529c0368d0 0x7f529c038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009990 tx=0x7f52a0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:44.557 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.578+0000 7f52a6ffd700 1 -- 172.21.15.138:0/2718681650 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f52ac025400 con 0x7f52b00fca30 2024-04-25T01:27:44.757 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.777+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 --> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi033:172.21.15.33=smithi033;smithi138:172.21.15.138=smithi138", "target": ["mon-mgr", ""]}) v1 -- 0x7f52b010e640 con 0x7f529c0368d0 2024-04-25T01:27:44.763 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.784+0000 7f52a6ffd700 1 -- 172.21.15.138:0/2718681650 <== mgr.14160 v2:172.21.15.33:6800/395184027 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f52b010e640 con 0x7f529c0368d0 2024-04-25T01:27:44.763 INFO:teuthology.orchestra.run.smithi138.stdout:Scheduled mon update... 2024-04-25T01:27:44.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f529c0368d0 msgr2=0x7f529c038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:44.765 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 --2- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f529c0368d0 0x7f529c038d80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009990 tx=0x7f52a0006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:44.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 msgr2=0x7f52b010d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:44.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 --2- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 0x7f52b010d090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0096b0 tx=0x7f52ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:44.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 shutdown_connections 2024-04-25T01:27:44.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 --2- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f529c0368d0 0x7f529c038d80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:44.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 --2- 172.21.15.138:0/2718681650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52b00fca30 0x7f52b010d090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:44.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 >> 172.21.15.138:0/2718681650 conn(0x7f52b00f8170 msgr2=0x7f52b0103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:44.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 shutdown_connections 2024-04-25T01:27:44.766 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:44.786+0000 7f52b845d700 1 -- 172.21.15.138:0/2718681650 wait complete. 2024-04-25T01:27:45.329 DEBUG:teuthology.orchestra.run.smithi138:mon.smithi138> sudo journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi138.service 2024-04-25T01:27:45.331 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:27:45.331 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:27:45.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:45 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-25T01:27:45.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:45 smithi033 ceph-mon[26353]: Deploying daemon grafana.smithi033 on smithi033 2024-04-25T01:27:45.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:45 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:45.613 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:45.660 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:46 smithi033 ceph-mon[26353]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi033:172.21.15.33=smithi033;smithi138:172.21.15.138=smithi138", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:27:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:46 smithi033 ceph-mon[26353]: Saving service mon spec with placement smithi033:172.21.15.33=smithi033;smithi138:172.21.15.138=smithi138;count:2 2024-04-25T01:27:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:46 smithi033 ceph-mon[26353]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:46 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.163+0000 7f37349de700 1 -- 172.21.15.138:0/3297263096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 msgr2=0x7f37300fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.163+0000 7f37349de700 1 --2- 172.21.15.138:0/3297263096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 0x7f37300fbd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3728009c60 tx=0x7f3728004d10 comp rx=0 tx=0).stop 2024-04-25T01:27:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.163+0000 7f37349de700 1 -- 172.21.15.138:0/3297263096 shutdown_connections 2024-04-25T01:27:47.146 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.163+0000 7f37349de700 1 --2- 172.21.15.138:0/3297263096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 0x7f37300fbd50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:47.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.163+0000 7f37349de700 1 -- 172.21.15.138:0/3297263096 >> 172.21.15.138:0/3297263096 conn(0x7f37300f70a0 msgr2=0x7f37300f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:47.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.163+0000 7f37349de700 1 -- 172.21.15.138:0/3297263096 shutdown_connections 2024-04-25T01:27:47.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.163+0000 7f37349de700 1 -- 172.21.15.138:0/3297263096 wait complete. 2024-04-25T01:27:47.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.164+0000 7f37349de700 1 Processor -- start 2024-04-25T01:27:47.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.164+0000 7f37349de700 1 -- start start 2024-04-25T01:27:47.147 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.164+0000 7f37349de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 0x7f373010f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:47.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.164+0000 7f37349de700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373010f7d0 con 0x7f37300fb940 2024-04-25T01:27:47.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.164+0000 7f372e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 0x7f373010f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:47.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.164+0000 7f372e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 0x7f373010f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:41378/0 (socket says 172.21.15.138:41378) 2024-04-25T01:27:47.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.164+0000 7f372e59c700 1 -- 172.21.15.138:0/1033456840 learned_addr learned my addr 172.21.15.138:0/1033456840 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:27:47.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.164+0000 7f372e59c700 1 -- 172.21.15.138:0/1033456840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3728009940 con 0x7f37300fb940 2024-04-25T01:27:47.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.165+0000 7f372e59c700 1 --2- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 0x7f373010f290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3728009500 tx=0x7f3728004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:47.148 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.165+0000 7f371effd700 1 -- 172.21.15.138:0/1033456840 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3728007c80 con 0x7f37300fb940 2024-04-25T01:27:47.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.165+0000 7f371effd700 1 -- 172.21.15.138:0/1033456840 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f372801d910 con 0x7f37300fb940 2024-04-25T01:27:47.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.165+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f373010f9d0 con 0x7f37300fb940 2024-04-25T01:27:47.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.165+0000 7f371effd700 1 -- 172.21.15.138:0/1033456840 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f37280129d0 con 0x7f37300fb940 2024-04-25T01:27:47.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.165+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3730110540 con 0x7f37300fb940 2024-04-25T01:27:47.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.166+0000 7f371effd700 1 -- 172.21.15.138:0/1033456840 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f3728019040 con 0x7f37300fb940 2024-04-25T01:27:47.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.166+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3730100d40 con 0x7f37300fb940 2024-04-25T01:27:47.149 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.166+0000 7f371effd700 1 --2- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f3718036870 0x7f3718038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:47.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.166+0000 7f371effd700 1 -- 172.21.15.138:0/1033456840 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f372804a3c0 con 0x7f37300fb940 2024-04-25T01:27:47.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.166+0000 7f372dd9b700 1 --2- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f3718036870 0x7f3718038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:47.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.166+0000 7f372dd9b700 1 --2- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f3718036870 0x7f3718038d20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3724009990 tx=0x7f3724006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:47.150 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.171+0000 7f371effd700 1 -- 172.21.15.138:0/1033456840 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f372802eab0 con 0x7f37300fb940 2024-04-25T01:27:47.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.428+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3730110ea0 con 0x7f37300fb940 2024-04-25T01:27:47.407 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.428+0000 7f371effd700 1 -- 172.21.15.138:0/1033456840 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f3728017020 con 0x7f37300fb940 2024-04-25T01:27:47.408 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:27:47.408 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:27:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f3718036870 msgr2=0x7f3718038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 --2- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f3718036870 0x7f3718038d20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3724009990 tx=0x7f3724006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 msgr2=0x7f373010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 --2- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 0x7f373010f290 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3728009500 tx=0x7f3728004d10 comp rx=0 tx=0).stop 2024-04-25T01:27:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 shutdown_connections 2024-04-25T01:27:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 --2- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f3718036870 0x7f3718038d20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 --2- 172.21.15.138:0/1033456840 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37300fb940 0x7f373010f290 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 >> 172.21.15.138:0/1033456840 conn(0x7f37300f70a0 msgr2=0x7f3730101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:47.410 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 shutdown_connections 2024-04-25T01:27:47.411 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:47.430+0000 7f37349de700 1 -- 172.21.15.138:0/1033456840 wait complete. 2024-04-25T01:27:47.411 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:27:48.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:48 smithi033 ceph-mon[26353]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:48.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:48 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/1033456840' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:27:49.001 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:27:49.002 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:27:49.256 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:49.303 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:50.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:50 smithi033 ceph-mon[26353]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 -- 172.21.15.138:0/680550048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 msgr2=0x7ff2b40fce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 --2- 172.21.15.138:0/680550048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 0x7ff2b40fce70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40048c0 tx=0x7ff2a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 -- 172.21.15.138:0/680550048 shutdown_connections 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 --2- 172.21.15.138:0/680550048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 0x7ff2b40fce70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 -- 172.21.15.138:0/680550048 >> 172.21.15.138:0/680550048 conn(0x7ff2b40f81c0 msgr2=0x7ff2b40fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 -- 172.21.15.138:0/680550048 shutdown_connections 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 -- 172.21.15.138:0/680550048 wait complete. 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 Processor -- start 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 -- start start 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.791+0000 7ff2b9895700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 0x7ff2b410d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff2b9895700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b410d5d0 con 0x7ff2b40fca60 2024-04-25T01:27:50.772 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 0x7ff2b410d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff2b2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 0x7ff2b410d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:35934/0 (socket says 172.21.15.138:35934) 2024-04-25T01:27:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff2b2ffd700 1 -- 172.21.15.138:0/3673079275 learned_addr learned my addr 172.21.15.138:0/3673079275 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:27:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff2b2ffd700 1 -- 172.21.15.138:0/3673079275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a4005040 con 0x7ff2b40fca60 2024-04-25T01:27:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff2b2ffd700 1 --2- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 0x7ff2b410d090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40096b0 tx=0x7ff2a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff29b7fe700 1 -- 172.21.15.138:0/3673079275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff2a400bed0 con 0x7ff2b40fca60 2024-04-25T01:27:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b410d7d0 con 0x7ff2b40fca60 2024-04-25T01:27:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff29b7fe700 1 -- 172.21.15.138:0/3673079275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2a400b5c0 con 0x7ff2b40fca60 2024-04-25T01:27:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff29b7fe700 1 -- 172.21.15.138:0/3673079275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff2a401c5b0 con 0x7ff2b40fca60 2024-04-25T01:27:50.773 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.792+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b410e340 con 0x7ff2b40fca60 2024-04-25T01:27:50.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.793+0000 7ff29b7fe700 1 -- 172.21.15.138:0/3673079275 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7ff2a4013420 con 0x7ff2b40fca60 2024-04-25T01:27:50.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.793+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b4102f70 con 0x7ff2b40fca60 2024-04-25T01:27:50.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.793+0000 7ff29b7fe700 1 --2- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7ff29c036880 0x7ff29c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:50.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.793+0000 7ff29b7fe700 1 -- 172.21.15.138:0/3673079275 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff2a4049ac0 con 0x7ff2b40fca60 2024-04-25T01:27:50.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.794+0000 7ff2b27fc700 1 --2- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7ff29c036880 0x7ff29c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:50.774 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.794+0000 7ff2b27fc700 1 --2- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7ff29c036880 0x7ff29c038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009990 tx=0x7ff2a8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:50.781 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:50.802+0000 7ff29b7fe700 1 -- 172.21.15.138:0/3673079275 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff2a4013a80 con 0x7ff2b40fca60 2024-04-25T01:27:51.041 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.062+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff2b410ed50 con 0x7ff2b40fca60 2024-04-25T01:27:51.042 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.063+0000 7ff29b7fe700 1 -- 172.21.15.138:0/3673079275 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff2a4011370 con 0x7ff2b40fca60 2024-04-25T01:27:51.042 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:27:51.042 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:27:51.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7ff29c036880 msgr2=0x7ff29c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:51.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 --2- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7ff29c036880 0x7ff29c038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009990 tx=0x7ff2a8006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:51.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 msgr2=0x7ff2b410d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:51.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 --2- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 0x7ff2b410d090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40096b0 tx=0x7ff2a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:51.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 shutdown_connections 2024-04-25T01:27:51.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 --2- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7ff29c036880 0x7ff29c038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:51.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 --2- 172.21.15.138:0/3673079275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2b40fca60 0x7ff2b410d090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:51.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 >> 172.21.15.138:0/3673079275 conn(0x7ff2b40f81c0 msgr2=0x7ff2b4103ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:51.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 shutdown_connections 2024-04-25T01:27:51.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:51.065+0000 7ff2b9895700 1 -- 172.21.15.138:0/3673079275 wait complete. 2024-04-25T01:27:51.045 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:27:51.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:51 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/3673079275' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:27:52.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:52 smithi033 ceph-mon[26353]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:52.664 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:27:52.664 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:27:52.908 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:52.956 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:54.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.476+0000 7fdece9a2700 1 -- 172.21.15.138:0/3908669130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 msgr2=0x7fdec80fc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:54.460 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.476+0000 7fdece9a2700 1 --2- 172.21.15.138:0/3908669130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 0x7fdec80fc010 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdec40048c0 tx=0x7fdec4009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.476+0000 7fdece9a2700 1 -- 172.21.15.138:0/3908669130 shutdown_connections 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.476+0000 7fdece9a2700 1 --2- 172.21.15.138:0/3908669130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 0x7fdec80fc010 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.476+0000 7fdece9a2700 1 -- 172.21.15.138:0/3908669130 >> 172.21.15.138:0/3908669130 conn(0x7fdec80f6670 msgr2=0x7fdec80f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.476+0000 7fdece9a2700 1 -- 172.21.15.138:0/3908669130 shutdown_connections 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.476+0000 7fdece9a2700 1 -- 172.21.15.138:0/3908669130 wait complete. 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdece9a2700 1 Processor -- start 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdece9a2700 1 -- start start 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdece9a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 0x7fdec81134e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdece9a2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8113a20 con 0x7fdec80fbc00 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdecd9a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 0x7fdec81134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdecd9a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 0x7fdec81134e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:35938/0 (socket says 172.21.15.138:35938) 2024-04-25T01:27:54.461 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdecd9a0700 1 -- 172.21.15.138:0/130168111 learned_addr learned my addr 172.21.15.138:0/130168111 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdecd9a0700 1 -- 172.21.15.138:0/130168111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec4005040 con 0x7fdec80fbc00 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdecd9a0700 1 --2- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 0x7fdec81134e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdec40096b0 tx=0x7fdec4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.477+0000 7fdebe7fc700 1 -- 172.21.15.138:0/130168111 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdec400bed0 con 0x7fdec80fbc00 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.478+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec8113c20 con 0x7fdec80fbc00 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.478+0000 7fdebe7fc700 1 -- 172.21.15.138:0/130168111 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdec400b5c0 con 0x7fdec80fbc00 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.478+0000 7fdebe7fc700 1 -- 172.21.15.138:0/130168111 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdec401c5b0 con 0x7fdec80fbc00 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.478+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec8114790 con 0x7fdec80fbc00 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.479+0000 7fdebe7fc700 1 -- 172.21.15.138:0/130168111 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7fdec4013420 con 0x7fdec80fbc00 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.479+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec8100b70 con 0x7fdec80fbc00 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.479+0000 7fdebe7fc700 1 --2- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fdeb4036880 0x7fdeb4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.479+0000 7fdebe7fc700 1 -- 172.21.15.138:0/130168111 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fdec4049ad0 con 0x7fdec80fbc00 2024-04-25T01:27:54.462 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.479+0000 7fdecd19f700 1 --2- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fdeb4036880 0x7fdeb4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:54.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.479+0000 7fdecd19f700 1 --2- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fdeb4036880 0x7fdeb4038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009990 tx=0x7fdeb8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:54.463 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.484+0000 7fdebe7fc700 1 -- 172.21.15.138:0/130168111 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdec4025400 con 0x7fdec80fbc00 2024-04-25T01:27:54.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:54 smithi033 ceph-mon[26353]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:54.725 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.746+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdec81151a0 con 0x7fdec80fbc00 2024-04-25T01:27:54.726 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.747+0000 7fdebe7fc700 1 -- 172.21.15.138:0/130168111 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fdec4004d30 con 0x7fdec80fbc00 2024-04-25T01:27:54.726 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:27:54.726 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:27:54.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fdeb4036880 msgr2=0x7fdeb4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:54.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 --2- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fdeb4036880 0x7fdeb4038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009990 tx=0x7fdeb8006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:54.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 msgr2=0x7fdec81134e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:54.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 --2- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 0x7fdec81134e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdec40096b0 tx=0x7fdec4009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:54.728 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 shutdown_connections 2024-04-25T01:27:54.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 --2- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fdeb4036880 0x7fdeb4038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:54.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 --2- 172.21.15.138:0/130168111 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdec80fbc00 0x7fdec81134e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:54.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 >> 172.21.15.138:0/130168111 conn(0x7fdec80f6670 msgr2=0x7fdec810a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:54.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 shutdown_connections 2024-04-25T01:27:54.729 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:54.749+0000 7fdece9a2700 1 -- 172.21.15.138:0/130168111 wait complete. 2024-04-25T01:27:54.729 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:27:55.529 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:55 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/130168111' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:27:56.424 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:27:56.424 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:27:56.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:56 smithi033 ceph-mon[26353]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:56.670 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:56.717 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:27:58.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.142+0000 7f660f7a7700 1 -- 172.21.15.138:0/849155580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 msgr2=0x7f66080fbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:58.122 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.142+0000 7f660f7a7700 1 --2- 172.21.15.138:0/849155580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 0x7f66080fbd40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f66040048c0 tx=0x7f6604009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.142+0000 7f660f7a7700 1 -- 172.21.15.138:0/849155580 shutdown_connections 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.142+0000 7f660f7a7700 1 --2- 172.21.15.138:0/849155580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 0x7f66080fbd40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.142+0000 7f660f7a7700 1 -- 172.21.15.138:0/849155580 >> 172.21.15.138:0/849155580 conn(0x7f66080f7070 msgr2=0x7f66080f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.142+0000 7f660f7a7700 1 -- 172.21.15.138:0/849155580 shutdown_connections 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.142+0000 7f660f7a7700 1 -- 172.21.15.138:0/849155580 wait complete. 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.142+0000 7f660f7a7700 1 Processor -- start 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.142+0000 7f660f7a7700 1 -- start start 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.143+0000 7f660f7a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 0x7f660810d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.143+0000 7f660f7a7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660810d600 con 0x7f66080fb930 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.143+0000 7f660d543700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 0x7f660810d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.143+0000 7f660d543700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 0x7f660810d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:35964/0 (socket says 172.21.15.138:35964) 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.143+0000 7f660d543700 1 -- 172.21.15.138:0/232786518 learned_addr learned my addr 172.21.15.138:0/232786518 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.143+0000 7f660d543700 1 -- 172.21.15.138:0/232786518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6604005040 con 0x7f66080fb930 2024-04-25T01:27:58.123 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.143+0000 7f660d543700 1 --2- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 0x7f660810d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66040096b0 tx=0x7f6604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.143+0000 7f65fdffb700 1 -- 172.21.15.138:0/232786518 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f660400bed0 con 0x7f66080fb930 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.143+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660810d800 con 0x7f66080fb930 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.144+0000 7f65fdffb700 1 -- 172.21.15.138:0/232786518 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f660400b5c0 con 0x7f66080fb930 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.144+0000 7f65fdffb700 1 -- 172.21.15.138:0/232786518 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f660401c5b0 con 0x7f66080fb930 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.144+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660810e370 con 0x7f66080fb930 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.144+0000 7f65fdffb700 1 -- 172.21.15.138:0/232786518 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f6604013420 con 0x7f66080fb930 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.145+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6608101e40 con 0x7f66080fb930 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.145+0000 7f65fdffb700 1 --2- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f65f4036880 0x7f65f4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.145+0000 7f65fdffb700 1 -- 172.21.15.138:0/232786518 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6604049ad0 con 0x7f66080fb930 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.145+0000 7f660cd42700 1 --2- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f65f4036880 0x7f65f4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:27:58.125 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.145+0000 7f660cd42700 1 --2- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f65f4036880 0x7f65f4038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009990 tx=0x7f65f8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:27:58.129 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.150+0000 7f65fdffb700 1 -- 172.21.15.138:0/232786518 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6604025400 con 0x7f66080fb930 2024-04-25T01:27:58.389 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.410+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f660810e5b0 con 0x7f66080fb930 2024-04-25T01:27:58.390 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.411+0000 7f65fdffb700 1 -- 172.21.15.138:0/232786518 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f6604004d30 con 0x7f66080fb930 2024-04-25T01:27:58.390 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:27:58.390 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:27:58.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f65f4036880 msgr2=0x7f65f4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:58.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 --2- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f65f4036880 0x7f65f4038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009990 tx=0x7f65f8006e40 comp rx=0 tx=0).stop 2024-04-25T01:27:58.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 msgr2=0x7f660810d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:27:58.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 --2- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 0x7f660810d0c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f66040096b0 tx=0x7f6604009e30 comp rx=0 tx=0).stop 2024-04-25T01:27:58.392 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 shutdown_connections 2024-04-25T01:27:58.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 --2- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f65f4036880 0x7f65f4038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 --2- 172.21.15.138:0/232786518 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66080fb930 0x7f660810d0c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:27:58.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 >> 172.21.15.138:0/232786518 conn(0x7f66080f7070 msgr2=0x7f6608065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:27:58.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 shutdown_connections 2024-04-25T01:27:58.393 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:27:58.413+0000 7f660f7a7700 1 -- 172.21.15.138:0/232786518 wait complete. 2024-04-25T01:27:58.393 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:27:58.481 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:58 smithi033 ceph-mon[26353]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:27:59.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:59 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/232786518' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:27:59.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:59 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:59.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:59 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:59.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:59 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:59.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:59 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:59.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:59 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:59.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:27:59 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:27:59.946 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:27:59.946 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:28:00.193 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:00.239 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:00.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:00 smithi033 ceph-mon[26353]: Deploying daemon prometheus.smithi033 on smithi033 2024-04-25T01:28:00.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:00 smithi033 ceph-mon[26353]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:01.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.731+0000 7f30f975e700 1 -- 172.21.15.138:0/1127550108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 msgr2=0x7f30f40fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:01.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.731+0000 7f30f975e700 1 --2- 172.21.15.138:0/1127550108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 0x7f30f40fce40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f30e80048c0 tx=0x7f30e8009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:01.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.731+0000 7f30f975e700 1 -- 172.21.15.138:0/1127550108 shutdown_connections 2024-04-25T01:28:01.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.731+0000 7f30f975e700 1 --2- 172.21.15.138:0/1127550108 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 0x7f30f40fce40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:01.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.731+0000 7f30f975e700 1 -- 172.21.15.138:0/1127550108 >> 172.21.15.138:0/1127550108 conn(0x7f30f40f8170 msgr2=0x7f30f40fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:01.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.731+0000 7f30f975e700 1 -- 172.21.15.138:0/1127550108 shutdown_connections 2024-04-25T01:28:01.710 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.731+0000 7f30f975e700 1 -- 172.21.15.138:0/1127550108 wait complete. 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.732+0000 7f30f975e700 1 Processor -- start 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.732+0000 7f30f975e700 1 -- start start 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.732+0000 7f30f975e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 0x7f30f4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.732+0000 7f30f975e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f4072240 con 0x7f30f40fca30 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.732+0000 7f30f2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 0x7f30f4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.732+0000 7f30f2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 0x7f30f4071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:50974/0 (socket says 172.21.15.138:50974) 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.732+0000 7f30f2ffd700 1 -- 172.21.15.138:0/2146137503 learned_addr learned my addr 172.21.15.138:0/2146137503 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.733+0000 7f30f2ffd700 1 -- 172.21.15.138:0/2146137503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e8005040 con 0x7f30f40fca30 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.733+0000 7f30f2ffd700 1 --2- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 0x7f30f4071d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30e80096b0 tx=0x7f30e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.733+0000 7f30db7fe700 1 -- 172.21.15.138:0/2146137503 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30e800bed0 con 0x7f30f40fca30 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.733+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f4070310 con 0x7f30f40fca30 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.733+0000 7f30db7fe700 1 -- 172.21.15.138:0/2146137503 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30e800b5c0 con 0x7f30f40fca30 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.733+0000 7f30db7fe700 1 -- 172.21.15.138:0/2146137503 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30e801c5b0 con 0x7f30f40fca30 2024-04-25T01:28:01.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.733+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f4070800 con 0x7f30f40fca30 2024-04-25T01:28:01.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.734+0000 7f30db7fe700 1 -- 172.21.15.138:0/2146137503 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f30e8013420 con 0x7f30f40fca30 2024-04-25T01:28:01.713 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.734+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f405f5c0 con 0x7f30f40fca30 2024-04-25T01:28:01.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.734+0000 7f30db7fe700 1 --2- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f30dc036880 0x7f30dc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:01.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.734+0000 7f30db7fe700 1 -- 172.21.15.138:0/2146137503 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f30e80499f0 con 0x7f30f40fca30 2024-04-25T01:28:01.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.735+0000 7f30f27fc700 1 --2- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f30dc036880 0x7f30dc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:01.714 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.735+0000 7f30f27fc700 1 --2- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f30dc036880 0x7f30dc038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009990 tx=0x7f30e4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:01.717 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:01.739+0000 7f30db7fe700 1 -- 172.21.15.138:0/2146137503 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f30e8025400 con 0x7f30f40fca30 2024-04-25T01:28:01.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.017+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f30f4106410 con 0x7f30f40fca30 2024-04-25T01:28:01.996 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.018+0000 7f30db7fe700 1 -- 172.21.15.138:0/2146137503 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f30e8004d30 con 0x7f30f40fca30 2024-04-25T01:28:01.996 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:28:01.997 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:28:01.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f30dc036880 msgr2=0x7f30dc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:01.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 --2- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f30dc036880 0x7f30dc038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f30e4009990 tx=0x7f30e4006e40 comp rx=0 tx=0).stop 2024-04-25T01:28:01.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 msgr2=0x7f30f4071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:01.998 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 --2- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 0x7f30f4071d00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f30e80096b0 tx=0x7f30e8009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:01.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 shutdown_connections 2024-04-25T01:28:01.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 --2- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f30dc036880 0x7f30dc038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:01.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 --2- 172.21.15.138:0/2146137503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30f40fca30 0x7f30f4071d00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:01.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 >> 172.21.15.138:0/2146137503 conn(0x7f30f40f8170 msgr2=0x7f30f4103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:01.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 shutdown_connections 2024-04-25T01:28:01.999 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:02.020+0000 7f30f975e700 1 -- 172.21.15.138:0/2146137503 wait complete. 2024-04-25T01:28:01.999 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:28:02.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:02 smithi033 ceph-mon[26353]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:02.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:02 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:28:02.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:02 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/2146137503' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:03.678 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:28:03.679 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:28:03.924 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:03.972 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:04.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:04 smithi033 ceph-mon[26353]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:05.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.467+0000 7f5da759e700 1 -- 172.21.15.138:0/3757856946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 msgr2=0x7f5da80ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:05.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.467+0000 7f5da759e700 1 --2- 172.21.15.138:0/3757856946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 0x7f5da80ff530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5d980048c0 tx=0x7f5d98009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:05.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.467+0000 7f5da759e700 1 -- 172.21.15.138:0/3757856946 shutdown_connections 2024-04-25T01:28:05.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.467+0000 7f5da759e700 1 --2- 172.21.15.138:0/3757856946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 0x7f5da80ff530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:05.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.467+0000 7f5da759e700 1 -- 172.21.15.138:0/3757856946 >> 172.21.15.138:0/3757856946 conn(0x7f5da80f6ee0 msgr2=0x7f5da80f9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:05.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.467+0000 7f5da759e700 1 -- 172.21.15.138:0/3757856946 shutdown_connections 2024-04-25T01:28:05.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.467+0000 7f5da759e700 1 -- 172.21.15.138:0/3757856946 wait complete. 2024-04-25T01:28:05.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.467+0000 7f5da759e700 1 Processor -- start 2024-04-25T01:28:05.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.467+0000 7f5da759e700 1 -- start start 2024-04-25T01:28:05.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.468+0000 7f5da759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 0x7f5da810cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:05.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.468+0000 7f5da759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da810d470 con 0x7f5da8066760 2024-04-25T01:28:05.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.468+0000 7f5da659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 0x7f5da810cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:05.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.468+0000 7f5da659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 0x7f5da810cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:50996/0 (socket says 172.21.15.138:50996) 2024-04-25T01:28:05.447 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.468+0000 7f5da659c700 1 -- 172.21.15.138:0/2482354367 learned_addr learned my addr 172.21.15.138:0/2482354367 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:28:05.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.468+0000 7f5da659c700 1 -- 172.21.15.138:0/2482354367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d98005040 con 0x7f5da8066760 2024-04-25T01:28:05.448 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.468+0000 7f5da659c700 1 --2- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 0x7f5da810cf30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800b3b0 tx=0x7f5d98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.469+0000 7f5d96ffd700 1 -- 172.21.15.138:0/2482354367 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d98004e60 con 0x7f5da8066760 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.469+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5da810d670 con 0x7f5da8066760 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.469+0000 7f5d96ffd700 1 -- 172.21.15.138:0/2482354367 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d98009650 con 0x7f5da8066760 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.469+0000 7f5d96ffd700 1 -- 172.21.15.138:0/2482354367 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5d9801c5b0 con 0x7f5da8066760 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.469+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da810e1e0 con 0x7f5da8066760 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.469+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da8102d80 con 0x7f5da8066760 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.470+0000 7f5d96ffd700 1 -- 172.21.15.138:0/2482354367 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f5d980097c0 con 0x7f5da8066760 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.470+0000 7f5d96ffd700 1 --2- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5d90036880 0x7f5d90038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.470+0000 7f5d96ffd700 1 -- 172.21.15.138:0/2482354367 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5d98049a10 con 0x7f5da8066760 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.470+0000 7f5da5d9b700 1 --2- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5d90036880 0x7f5d90038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:05.449 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.471+0000 7f5da5d9b700 1 --2- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5d90036880 0x7f5d90038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009990 tx=0x7f5d9c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:05.453 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.475+0000 7f5d96ffd700 1 -- 172.21.15.138:0/2482354367 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5d98011550 con 0x7f5da8066760 2024-04-25T01:28:05.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.733+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5da810ebf0 con 0x7f5da8066760 2024-04-25T01:28:05.712 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.734+0000 7f5d96ffd700 1 -- 172.21.15.138:0/2482354367 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f5d98011370 con 0x7f5da8066760 2024-04-25T01:28:05.713 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:28:05.713 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5d90036880 msgr2=0x7f5d90038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 --2- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5d90036880 0x7f5d90038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c009990 tx=0x7f5d9c006e40 comp rx=0 tx=0).stop 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 msgr2=0x7f5da810cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 --2- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 0x7f5da810cf30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5d9800b3b0 tx=0x7f5d98009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 shutdown_connections 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 --2- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f5d90036880 0x7f5d90038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 --2- 172.21.15.138:0/2482354367 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5da8066760 0x7f5da810cf30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 >> 172.21.15.138:0/2482354367 conn(0x7f5da80f6ee0 msgr2=0x7f5da80f7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 shutdown_connections 2024-04-25T01:28:05.715 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:05.736+0000 7f5da759e700 1 -- 172.21.15.138:0/2482354367 wait complete. 2024-04-25T01:28:05.716 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:28:06.465 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:06 smithi033 ceph-mon[26353]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:06.465 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:06 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/2482354367' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:07.311 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:28:07.311 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:28:07.559 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:07.607 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:08.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:08 smithi033 ceph-mon[26353]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:08.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/2031002032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 msgr2=0x7f4cd80fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:08.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 --2- 172.21.15.138:0/2031002032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 0x7f4cd80fe490 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80048c0 tx=0x7f4cc8009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:08.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/2031002032 shutdown_connections 2024-04-25T01:28:08.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 --2- 172.21.15.138:0/2031002032 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 0x7f4cd80fe490 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:08.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/2031002032 >> 172.21.15.138:0/2031002032 conn(0x7f4cd80f6820 msgr2=0x7f4cd80f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:08.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/2031002032 shutdown_connections 2024-04-25T01:28:08.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/2031002032 wait complete. 2024-04-25T01:28:08.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 Processor -- start 2024-04-25T01:28:08.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 -- start start 2024-04-25T01:28:08.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 0x7f4cd810d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:08.947 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.967+0000 7f4cdcd4b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cd810d5e0 con 0x7f4cd80fe080 2024-04-25T01:28:08.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.968+0000 7f4cd659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 0x7f4cd810d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:08.948 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.968+0000 7f4cd659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 0x7f4cd810d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:50636/0 (socket says 172.21.15.138:50636) 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.968+0000 7f4cd659c700 1 -- 172.21.15.138:0/1076557649 learned_addr learned my addr 172.21.15.138:0/1076557649 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.968+0000 7f4cd659c700 1 -- 172.21.15.138:0/1076557649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc8005040 con 0x7f4cd80fe080 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.968+0000 7f4cd659c700 1 --2- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 0x7f4cd810d0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80096b0 tx=0x7f4cc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.969+0000 7f4cc6ffd700 1 -- 172.21.15.138:0/1076557649 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cc800bed0 con 0x7f4cd80fe080 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.969+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cd810d7e0 con 0x7f4cd80fe080 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.969+0000 7f4cc6ffd700 1 -- 172.21.15.138:0/1076557649 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cc800b5c0 con 0x7f4cd80fe080 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.969+0000 7f4cc6ffd700 1 -- 172.21.15.138:0/1076557649 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4cc801c5b0 con 0x7f4cd80fe080 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.969+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cd810e350 con 0x7f4cd80fe080 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.969+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd8102f50 con 0x7f4cd80fe080 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.969+0000 7f4cc6ffd700 1 -- 172.21.15.138:0/1076557649 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 13) v1 ==== 43432+0+0 (secure 0 0 0) 0x7f4cc8013a80 con 0x7f4cd80fe080 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.970+0000 7f4cc6ffd700 1 --2- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f4cc0036880 0x7f4cc0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.970+0000 7f4cc6ffd700 1 -- 172.21.15.138:0/1076557649 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f4cc8049a90 con 0x7f4cd80fe080 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.970+0000 7f4cd5d9b700 1 --2- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f4cc0036880 0x7f4cc0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:08.949 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.970+0000 7f4cd5d9b700 1 --2- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f4cc0036880 0x7f4cc0038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009990 tx=0x7f4ccc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:08.953 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:08.975+0000 7f4cc6ffd700 1 -- 172.21.15.138:0/1076557649 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4cc8011940 con 0x7f4cd80fe080 2024-04-25T01:28:09.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.231+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4cd810ed60 con 0x7f4cd80fe080 2024-04-25T01:28:09.210 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.232+0000 7f4cc6ffd700 1 -- 172.21.15.138:0/1076557649 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f4cc8011370 con 0x7f4cd80fe080 2024-04-25T01:28:09.214 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:28:09.214 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:28:09.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f4cc0036880 msgr2=0x7f4cc0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:09.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 --2- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f4cc0036880 0x7f4cc0038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009990 tx=0x7f4ccc006e40 comp rx=0 tx=0).stop 2024-04-25T01:28:09.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 msgr2=0x7f4cd810d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:09.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 --2- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 0x7f4cd810d0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4cc80096b0 tx=0x7f4cc8009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:09.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 shutdown_connections 2024-04-25T01:28:09.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 --2- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7f4cc0036880 0x7f4cc0038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:09.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 --2- 172.21.15.138:0/1076557649 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4cd80fe080 0x7f4cd810d0a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:09.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 >> 172.21.15.138:0/1076557649 conn(0x7f4cd80f6820 msgr2=0x7f4cd8103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:09.215 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 shutdown_connections 2024-04-25T01:28:09.216 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:09.234+0000 7f4cdcd4b700 1 -- 172.21.15.138:0/1076557649 wait complete. 2024-04-25T01:28:09.216 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:28:09.573 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:09 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:28:09.573 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:09 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:28:09.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:09 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' 2024-04-25T01:28:09.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:09 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-25T01:28:09.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:09 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/1076557649' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:10.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:10 smithi033 ceph-mon[26353]: from='mgr.14160 172.21.15.33:0/1877527621' entity='mgr.smithi033.asemba' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-25T01:28:10.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:10 smithi033 ceph-mon[26353]: mgrmap e14: smithi033.asemba(active, since 58s) 2024-04-25T01:28:10.776 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:28:10.777 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:28:11.024 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:11.072 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:12.433 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.454+0000 7fe2a53cc700 1 -- 172.21.15.138:0/334446653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 msgr2=0x7fe2a00fc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.454+0000 7fe2a53cc700 1 --2- 172.21.15.138:0/334446653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 0x7fe2a00fc340 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe2940048c0 tx=0x7fe294009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.454+0000 7fe2a53cc700 1 -- 172.21.15.138:0/334446653 shutdown_connections 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.454+0000 7fe2a53cc700 1 --2- 172.21.15.138:0/334446653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 0x7fe2a00fc340 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.454+0000 7fe2a53cc700 1 -- 172.21.15.138:0/334446653 >> 172.21.15.138:0/334446653 conn(0x7fe2a00f6800 msgr2=0x7fe2a00f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.454+0000 7fe2a53cc700 1 -- 172.21.15.138:0/334446653 shutdown_connections 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.454+0000 7fe2a53cc700 1 -- 172.21.15.138:0/334446653 wait complete. 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.455+0000 7fe2a53cc700 1 Processor -- start 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.455+0000 7fe2a53cc700 1 -- start start 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.455+0000 7fe2a53cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 0x7fe2a010d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.455+0000 7fe2a53cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a010d640 con 0x7fe2a00fbf30 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.455+0000 7fe29effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 0x7fe2a010d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.455+0000 7fe29effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 0x7fe2a010d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:50652/0 (socket says 172.21.15.138:50652) 2024-04-25T01:28:12.434 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.455+0000 7fe29effd700 1 -- 172.21.15.138:0/1520156912 learned_addr learned my addr 172.21.15.138:0/1520156912 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.456+0000 7fe29effd700 1 -- 172.21.15.138:0/1520156912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe294005040 con 0x7fe2a00fbf30 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.456+0000 7fe29effd700 1 --2- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 0x7fe2a010d100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe294009680 tx=0x7fe294009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.456+0000 7fe2877fe700 1 -- 172.21.15.138:0/1520156912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe29400bed0 con 0x7fe2a00fbf30 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.456+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2a010d840 con 0x7fe2a00fbf30 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.456+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2a010e3b0 con 0x7fe2a00fbf30 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.456+0000 7fe2877fe700 1 -- 172.21.15.138:0/1520156912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe29400b5c0 con 0x7fe2a00fbf30 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.456+0000 7fe2877fe700 1 -- 172.21.15.138:0/1520156912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe2940255c0 con 0x7fe2a00fbf30 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.457+0000 7fe2877fe700 1 -- 172.21.15.138:0/1520156912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 14) v1 ==== 43446+0+0 (secure 0 0 0) 0x7fe294013420 con 0x7fe2a00fbf30 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.457+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2a0101e40 con 0x7fe2a00fbf30 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.457+0000 7fe2877fe700 1 --2- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fe2880368d0 0x7fe288038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.457+0000 7fe2877fe700 1 -- 172.21.15.138:0/1520156912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fe294049a30 con 0x7fe2a00fbf30 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.457+0000 7fe29e7fc700 1 -- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fe2880368d0 msgr2=0x7fe288038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/395184027 2024-04-25T01:28:12.436 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.457+0000 7fe29e7fc700 1 --2- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fe2880368d0 0x7fe288038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:12.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.462+0000 7fe2877fe700 1 -- 172.21.15.138:0/1520156912 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe2940123c0 con 0x7fe2a00fbf30 2024-04-25T01:28:12.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.658+0000 7fe29e7fc700 1 -- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fe2880368d0 msgr2=0x7fe288038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.33:6800/395184027 2024-04-25T01:28:12.636 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.658+0000 7fe29e7fc700 1 --2- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fe2880368d0 0x7fe288038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T01:28:12.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.726+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe2a005f5c0 con 0x7fe2a00fbf30 2024-04-25T01:28:12.705 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.727+0000 7fe2877fe700 1 -- 172.21.15.138:0/1520156912 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fe294023300 con 0x7fe2a00fbf30 2024-04-25T01:28:12.705 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:28:12.705 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:28:12.707 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fe2880368d0 msgr2=0x7fe288038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 --2- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fe2880368d0 0x7fe288038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 msgr2=0x7fe2a010d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 --2- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 0x7fe2a010d100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fe294009680 tx=0x7fe294009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 shutdown_connections 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 --2- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:6800/395184027,v1:172.21.15.33:6801/395184027] conn(0x7fe2880368d0 0x7fe288038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 --2- 172.21.15.138:0/1520156912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe2a00fbf30 0x7fe2a010d100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 >> 172.21.15.138:0/1520156912 conn(0x7fe2a00f6800 msgr2=0x7fe2a00fabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 shutdown_connections 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:12.729+0000 7fe2a53cc700 1 -- 172.21.15.138:0/1520156912 wait complete. 2024-04-25T01:28:12.708 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:28:13.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:12 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/1520156912' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:14.227 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:28:14.227 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:28:14.476 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:14.524 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:16.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.074+0000 7f21c1c49700 1 -- 172.21.15.138:0/2286148356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 msgr2=0x7f21bc0fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:16.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.074+0000 7f21c1c49700 1 --2- 172.21.15.138:0/2286148356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 0x7f21bc0fd430 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f21b40048c0 tx=0x7f21b4009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:16.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.074+0000 7f21c1c49700 1 -- 172.21.15.138:0/2286148356 shutdown_connections 2024-04-25T01:28:16.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.074+0000 7f21c1c49700 1 --2- 172.21.15.138:0/2286148356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 0x7f21bc0fd430 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:16.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.074+0000 7f21c1c49700 1 -- 172.21.15.138:0/2286148356 >> 172.21.15.138:0/2286148356 conn(0x7f21bc0f70a0 msgr2=0x7f21bc0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:16.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.074+0000 7f21c1c49700 1 -- 172.21.15.138:0/2286148356 shutdown_connections 2024-04-25T01:28:16.053 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.074+0000 7f21c1c49700 1 -- 172.21.15.138:0/2286148356 wait complete. 2024-04-25T01:28:16.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.075+0000 7f21c1c49700 1 Processor -- start 2024-04-25T01:28:16.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.075+0000 7f21c1c49700 1 -- start start 2024-04-25T01:28:16.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.075+0000 7f21c1c49700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 0x7f21bc101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:16.054 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.075+0000 7f21c1c49700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21bc101ed0 con 0x7f21bc0fd020 2024-04-25T01:28:16.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.075+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 0x7f21bc101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:16.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.075+0000 7f21bb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 0x7f21bc101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:50670/0 (socket says 172.21.15.138:50670) 2024-04-25T01:28:16.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.075+0000 7f21bb7fe700 1 -- 172.21.15.138:0/1213268790 learned_addr learned my addr 172.21.15.138:0/1213268790 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:28:16.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.076+0000 7f21bb7fe700 1 -- 172.21.15.138:0/1213268790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b4005040 con 0x7f21bc0fd020 2024-04-25T01:28:16.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.076+0000 7f21bb7fe700 1 --2- 172.21.15.138:0/1213268790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 0x7f21bc101990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21b40096b0 tx=0x7f21b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:16.055 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.076+0000 7f21a3fff700 1 -- 172.21.15.138:0/1213268790 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21b400bed0 con 0x7f21bc0fd020 2024-04-25T01:28:16.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.076+0000 7f21a3fff700 1 -- 172.21.15.138:0/1213268790 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21b400b5c0 con 0x7f21bc0fd020 2024-04-25T01:28:16.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.076+0000 7f21c1c49700 1 -- 172.21.15.138:0/1213268790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21bc0fffa0 con 0x7f21bc0fd020 2024-04-25T01:28:16.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.076+0000 7f21a3fff700 1 -- 172.21.15.138:0/1213268790 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21b401c5b0 con 0x7f21bc0fd020 2024-04-25T01:28:16.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.076+0000 7f21c1c49700 1 -- 172.21.15.138:0/1213268790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21bc100490 con 0x7f21bc0fd020 2024-04-25T01:28:16.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.077+0000 7f21a3fff700 1 -- 172.21.15.138:0/1213268790 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 15) v1 ==== 43170+0+0 (secure 0 0 0) 0x7f21b4013420 con 0x7f21bc0fd020 2024-04-25T01:28:16.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.077+0000 7f21c1c49700 1 -- 172.21.15.138:0/1213268790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21bc102f50 con 0x7f21bc0fd020 2024-04-25T01:28:16.056 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.077+0000 7f21a3fff700 1 -- 172.21.15.138:0/1213268790 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f21b4049680 con 0x7f21bc0fd020 2024-04-25T01:28:16.061 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.083+0000 7f21a3fff700 1 -- 172.21.15.138:0/1213268790 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f21b4004d30 con 0x7f21bc0fd020 2024-04-25T01:28:16.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.356+0000 7f21c1c49700 1 -- 172.21.15.138:0/1213268790 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f21bc05f5c0 con 0x7f21bc0fd020 2024-04-25T01:28:16.335 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.356+0000 7f21a3fff700 1 -- 172.21.15.138:0/1213268790 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7f21b4011370 con 0x7f21bc0fd020 2024-04-25T01:28:16.335 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:28:16.336 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:28:16.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.358+0000 7f21c1c49700 1 -- 172.21.15.138:0/1213268790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 msgr2=0x7f21bc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:16.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.358+0000 7f21c1c49700 1 --2- 172.21.15.138:0/1213268790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 0x7f21bc101990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21b40096b0 tx=0x7f21b4009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:16.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.358+0000 7f21c1c49700 1 -- 172.21.15.138:0/1213268790 shutdown_connections 2024-04-25T01:28:16.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.358+0000 7f21c1c49700 1 --2- 172.21.15.138:0/1213268790 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21bc0fd020 0x7f21bc101990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:16.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.358+0000 7f21c1c49700 1 -- 172.21.15.138:0/1213268790 >> 172.21.15.138:0/1213268790 conn(0x7f21bc0f70a0 msgr2=0x7f21bc103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:16.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.358+0000 7f21c1c49700 1 -- 172.21.15.138:0/1213268790 shutdown_connections 2024-04-25T01:28:16.337 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:16.358+0000 7f21c1c49700 1 -- 172.21.15.138:0/1213268790 wait complete. 2024-04-25T01:28:16.337 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: Active manager daemon smithi033.asemba restarted 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: Activating manager daemon smithi033.asemba 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: osdmap e5: 0 total, 0 up, 0 in 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: mgrmap e15: smithi033.asemba(active, starting, since 0.0053945s) 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:28:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: Manager daemon smithi033.asemba is now available 2024-04-25T01:28:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:28:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:28:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:28:17.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:28:17.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/trash_purge_schedule"}]: dispatch 2024-04-25T01:28:17.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:17 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/1213268790' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:17.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:17.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:17.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:17.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:17 smithi033 ceph-mon[26353]: mgrmap e16: smithi033.asemba(active, since 1.00749s) 2024-04-25T01:28:17.920 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:28:17.920 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:28:18.438 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:18 smithi033 ceph-mon[26353]: [25/Apr/2024:01:28:16] ENGINE Bus STARTING 2024-04-25T01:28:18.438 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:18 smithi033 ceph-mon[26353]: [25/Apr/2024:01:28:16] ENGINE Serving on https://172.21.15.33:7150 2024-04-25T01:28:18.438 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:18 smithi033 ceph-mon[26353]: [25/Apr/2024:01:28:16] ENGINE Bus STARTED 2024-04-25T01:28:18.438 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:18.490 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:18.881 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T01:28:19.938 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:19 smithi033 ceph-mon[26353]: mgrmap e17: smithi033.asemba(active, since 2s) 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd/host:smithi138", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:21.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd/host:smithi033", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:28:21.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:21.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:28:21.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.580+0000 7fcbe5949700 1 -- 172.21.15.138:0/2385674419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 msgr2=0x7fcbe00ffa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:21.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.580+0000 7fcbe5949700 1 --2- 172.21.15.138:0/2385674419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 0x7fcbe00ffa00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcbd00048c0 tx=0x7fcbd0009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:21.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.580+0000 7fcbe5949700 1 -- 172.21.15.138:0/2385674419 shutdown_connections 2024-04-25T01:28:21.569 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.580+0000 7fcbe5949700 1 --2- 172.21.15.138:0/2385674419 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 0x7fcbe00ffa00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.580+0000 7fcbe5949700 1 -- 172.21.15.138:0/2385674419 >> 172.21.15.138:0/2385674419 conn(0x7fcbe00f70a0 msgr2=0x7fcbe00f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.580+0000 7fcbe5949700 1 -- 172.21.15.138:0/2385674419 shutdown_connections 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.580+0000 7fcbe5949700 1 -- 172.21.15.138:0/2385674419 wait complete. 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.580+0000 7fcbe5949700 1 Processor -- start 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.580+0000 7fcbe5949700 1 -- start start 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.581+0000 7fcbe5949700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 0x7fcbe010d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.581+0000 7fcbe5949700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe010d5f0 con 0x7fcbe00fd620 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.581+0000 7fcbdeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 0x7fcbe010d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.581+0000 7fcbdeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 0x7fcbe010d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:43102/0 (socket says 172.21.15.138:43102) 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.581+0000 7fcbdeffd700 1 -- 172.21.15.138:0/3097679009 learned_addr learned my addr 172.21.15.138:0/3097679009 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.581+0000 7fcbdeffd700 1 -- 172.21.15.138:0/3097679009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0005040 con 0x7fcbe00fd620 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.581+0000 7fcbdeffd700 1 --2- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 0x7fcbe010d0b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009680 tx=0x7fcbd0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.582+0000 7fcbc77fe700 1 -- 172.21.15.138:0/3097679009 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbd000bed0 con 0x7fcbe00fd620 2024-04-25T01:28:21.570 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.582+0000 7fcbc77fe700 1 -- 172.21.15.138:0/3097679009 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbd000b5c0 con 0x7fcbe00fd620 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.582+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe010d7f0 con 0x7fcbe00fd620 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.582+0000 7fcbc77fe700 1 -- 172.21.15.138:0/3097679009 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcbd001c5b0 con 0x7fcbe00fd620 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.582+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe010e360 con 0x7fcbe00fd620 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.583+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe0102f30 con 0x7fcbe00fd620 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.583+0000 7fcbc77fe700 1 -- 172.21.15.138:0/3097679009 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7fcbd001c710 con 0x7fcbe00fd620 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.583+0000 7fcbc77fe700 1 --2- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcbc80369c0 0x7fcbc8038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.583+0000 7fcbc77fe700 1 -- 172.21.15.138:0/3097679009 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fcbd0049b20 con 0x7fcbe00fd620 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.583+0000 7fcbde7fc700 1 --2- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcbc80369c0 0x7fcbc8038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.584+0000 7fcbde7fc700 1 --2- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcbc80369c0 0x7fcbc8038e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009990 tx=0x7fcbd4006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:21.571 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.588+0000 7fcbc77fe700 1 -- 172.21.15.138:0/3097679009 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcbd0025400 con 0x7fcbe00fd620 2024-04-25T01:28:21.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.879+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcbe010ed70 con 0x7fcbe00fd620 2024-04-25T01:28:21.859 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.880+0000 7fcbc77fe700 1 -- 172.21.15.138:0/3097679009 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fcbd0011b60 con 0x7fcbe00fd620 2024-04-25T01:28:21.860 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:28:21.860 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:28:21.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcbc80369c0 msgr2=0x7fcbc8038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:21.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 --2- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcbc80369c0 0x7fcbc8038e70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcbd4009990 tx=0x7fcbd4006e40 comp rx=0 tx=0).stop 2024-04-25T01:28:21.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 msgr2=0x7fcbe010d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:21.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 --2- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 0x7fcbe010d0b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcbd0009680 tx=0x7fcbd0009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:21.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 shutdown_connections 2024-04-25T01:28:21.861 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 --2- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcbc80369c0 0x7fcbc8038e70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:21.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 --2- 172.21.15.138:0/3097679009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbe00fd620 0x7fcbe010d0b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:21.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 >> 172.21.15.138:0/3097679009 conn(0x7fcbe00f70a0 msgr2=0x7fcbe0103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:21.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 shutdown_connections 2024-04-25T01:28:21.862 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:21.882+0000 7fcbe5949700 1 -- 172.21.15.138:0/3097679009 wait complete. 2024-04-25T01:28:21.862 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/3097679009' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:22.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:28:22.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-25T01:28:22.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:23.458 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:28:23.458 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:28:23.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:23 smithi033 ceph-mon[26353]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:28:23.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:23 smithi033 ceph-mon[26353]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:28:23.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:23 smithi033 ceph-mon[26353]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:28:23.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:23 smithi033 ceph-mon[26353]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:28:23.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:23 smithi033 ceph-mon[26353]: Deploying daemon crash.smithi138 on smithi138 2024-04-25T01:28:23.715 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:28:25.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.349+0000 7ff77ec10700 1 -- 172.21.15.138:0/2270445099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 msgr2=0x7ff778101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:25.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.349+0000 7ff77ec10700 1 --2- 172.21.15.138:0/2270445099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 0x7ff778101da0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff770009c60 tx=0x7ff770004d10 comp rx=0 tx=0).stop 2024-04-25T01:28:25.329 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.349+0000 7ff77ec10700 1 -- 172.21.15.138:0/2270445099 shutdown_connections 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.349+0000 7ff77ec10700 1 --2- 172.21.15.138:0/2270445099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 0x7ff778101da0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.349+0000 7ff77ec10700 1 -- 172.21.15.138:0/2270445099 >> 172.21.15.138:0/2270445099 conn(0x7ff778071800 msgr2=0x7ff778071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.349+0000 7ff77ec10700 1 -- 172.21.15.138:0/2270445099 shutdown_connections 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.349+0000 7ff77ec10700 1 -- 172.21.15.138:0/2270445099 wait complete. 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.349+0000 7ff77ec10700 1 Processor -- start 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.349+0000 7ff77ec10700 1 -- start start 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.350+0000 7ff77ec10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 0x7ff778113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.350+0000 7ff77ec10700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7781136c0 con 0x7ff7781019d0 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.350+0000 7ff77c9ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 0x7ff778113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.350+0000 7ff77c9ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 0x7ff778113180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:43120/0 (socket says 172.21.15.138:43120) 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.350+0000 7ff77c9ac700 1 -- 172.21.15.138:0/524781801 learned_addr learned my addr 172.21.15.138:0/524781801 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:28:25.330 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.350+0000 7ff77c9ac700 1 -- 172.21.15.138:0/524781801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770009940 con 0x7ff7781019d0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.351+0000 7ff77c9ac700 1 --2- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 0x7ff778113180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff77000b740 tx=0x7ff770004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.351+0000 7ff7757fa700 1 -- 172.21.15.138:0/524781801 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff770007b30 con 0x7ff7781019d0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.351+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7781138c0 con 0x7ff7781019d0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.351+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff778113db0 con 0x7ff7781019d0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.351+0000 7ff7757fa700 1 -- 172.21.15.138:0/524781801 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff77001d910 con 0x7ff7781019d0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.351+0000 7ff7757fa700 1 -- 172.21.15.138:0/524781801 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff7700128b0 con 0x7ff7781019d0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.352+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7780fd0d0 con 0x7ff7781019d0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.352+0000 7ff7757fa700 1 -- 172.21.15.138:0/524781801 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7ff770019040 con 0x7ff7781019d0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.352+0000 7ff7757fa700 1 --2- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff764036970 0x7ff764038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.352+0000 7ff7757fa700 1 -- 172.21.15.138:0/524781801 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7ff77004a500 con 0x7ff7781019d0 2024-04-25T01:28:25.332 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.352+0000 7ff777fff700 1 --2- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff764036970 0x7ff764038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:25.333 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.353+0000 7ff777fff700 1 --2- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff764036970 0x7ff764038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff768009990 tx=0x7ff768006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:25.336 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.357+0000 7ff7757fa700 1 -- 172.21.15.138:0/524781801 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff77002eb40 con 0x7ff7781019d0 2024-04-25T01:28:25.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.626+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff77805f5c0 con 0x7ff7781019d0 2024-04-25T01:28:25.606 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.627+0000 7ff7757fa700 1 -- 172.21.15.138:0/524781801 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7ff770017020 con 0x7ff7781019d0 2024-04-25T01:28:25.606 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:28:25.607 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:28:25.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff764036970 msgr2=0x7ff764038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:25.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 --2- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff764036970 0x7ff764038e20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff768009990 tx=0x7ff768006e40 comp rx=0 tx=0).stop 2024-04-25T01:28:25.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 msgr2=0x7ff778113180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:25.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 --2- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 0x7ff778113180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff77000b740 tx=0x7ff770004d10 comp rx=0 tx=0).stop 2024-04-25T01:28:25.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 shutdown_connections 2024-04-25T01:28:25.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 --2- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff764036970 0x7ff764038e20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:25.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 --2- 172.21.15.138:0/524781801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7781019d0 0x7ff778113180 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:25.608 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 >> 172.21.15.138:0/524781801 conn(0x7ff778071800 msgr2=0x7ff7780650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:25.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 shutdown_connections 2024-04-25T01:28:25.609 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:25.629+0000 7ff77ec10700 1 -- 172.21.15.138:0/524781801 wait complete. 2024-04-25T01:28:25.609 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:28:25.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:25 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/524781801' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:27.494 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:28:27.494 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:28:27.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:27 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:27.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:27 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:27.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:27 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:27.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:27 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:27.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:27 smithi033 ceph-mon[26353]: Deploying daemon node-exporter.smithi138 on smithi138 2024-04-25T01:28:27.748 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:28:30.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.459+0000 7fd742209700 1 -- 172.21.15.138:0/1346115864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 msgr2=0x7fd73c101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:30.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.459+0000 7fd742209700 1 --2- 172.21.15.138:0/1346115864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 0x7fd73c101d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd72c0048c0 tx=0x7fd72c009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:30.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.459+0000 7fd742209700 1 -- 172.21.15.138:0/1346115864 shutdown_connections 2024-04-25T01:28:30.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.459+0000 7fd742209700 1 --2- 172.21.15.138:0/1346115864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 0x7fd73c101d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:30.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.459+0000 7fd742209700 1 -- 172.21.15.138:0/1346115864 >> 172.21.15.138:0/1346115864 conn(0x7fd73c0f74d0 msgr2=0x7fd73c0f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:30.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.459+0000 7fd742209700 1 -- 172.21.15.138:0/1346115864 shutdown_connections 2024-04-25T01:28:30.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.459+0000 7fd742209700 1 -- 172.21.15.138:0/1346115864 wait complete. 2024-04-25T01:28:30.439 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.460+0000 7fd742209700 1 Processor -- start 2024-04-25T01:28:30.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.460+0000 7fd742209700 1 -- start start 2024-04-25T01:28:30.440 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.460+0000 7fd742209700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 0x7fd73c112060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.460+0000 7fd742209700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73c1125a0 con 0x7fd73c1019b0 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.460+0000 7fd73b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 0x7fd73c112060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.460+0000 7fd73b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 0x7fd73c112060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.138:34708/0 (socket says 172.21.15.138:34708) 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.460+0000 7fd73b7fe700 1 -- 172.21.15.138:0/3084085259 learned_addr learned my addr 172.21.15.138:0/3084085259 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.461+0000 7fd73b7fe700 1 -- 172.21.15.138:0/3084085259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c005040 con 0x7fd73c1019b0 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.461+0000 7fd73b7fe700 1 --2- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 0x7fd73c112060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00b820 tx=0x7fd72c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.461+0000 7fd723fff700 1 -- 172.21.15.138:0/3084085259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd72c0093c0 con 0x7fd73c1019b0 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.461+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73c1127a0 con 0x7fd73c1019b0 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.461+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73c112c90 con 0x7fd73c1019b0 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.461+0000 7fd723fff700 1 -- 172.21.15.138:0/3084085259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd72c013420 con 0x7fd73c1019b0 2024-04-25T01:28:30.441 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.461+0000 7fd723fff700 1 -- 172.21.15.138:0/3084085259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd72c0255c0 con 0x7fd73c1019b0 2024-04-25T01:28:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.462+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7280052b0 con 0x7fd73c1019b0 2024-04-25T01:28:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.462+0000 7fd723fff700 1 -- 172.21.15.138:0/3084085259 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 17) v1 ==== 43489+0+0 (secure 0 0 0) 0x7fd72c025720 con 0x7fd73c1019b0 2024-04-25T01:28:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.462+0000 7fd723fff700 1 --2- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd724036970 0x7fd724038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.462+0000 7fd723fff700 1 -- 172.21.15.138:0/3084085259 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd72c049b40 con 0x7fd73c1019b0 2024-04-25T01:28:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.463+0000 7fd73affd700 1 --2- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd724036970 0x7fd724038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:30.442 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.463+0000 7fd73affd700 1 --2- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd724036970 0x7fd724038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd730009990 tx=0x7fd730006e40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:30.446 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.467+0000 7fd723fff700 1 -- 172.21.15.138:0/3084085259 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd72c024dc0 con 0x7fd73c1019b0 2024-04-25T01:28:30.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.760+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd728005560 con 0x7fd73c1019b0 2024-04-25T01:28:30.740 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.761+0000 7fd723fff700 1 -- 172.21.15.138:0/3084085259 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+739 (secure 0 0 0) 0x7fd72c024360 con 0x7fd73c1019b0 2024-04-25T01:28:30.740 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:28:30.741 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":1,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:26:02.814821Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd724036970 msgr2=0x7fd724038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 --2- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd724036970 0x7fd724038e20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd730009990 tx=0x7fd730006e40 comp rx=0 tx=0).stop 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 msgr2=0x7fd73c112060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 --2- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 0x7fd73c112060 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd72c00b820 tx=0x7fd72c009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 shutdown_connections 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 --2- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd724036970 0x7fd724038e20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 --2- 172.21.15.138:0/3084085259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd73c1019b0 0x7fd73c112060 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 >> 172.21.15.138:0/3084085259 conn(0x7fd73c0f74d0 msgr2=0x7fd73c0f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 shutdown_connections 2024-04-25T01:28:30.743 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:30.764+0000 7fd742209700 1 -- 172.21.15.138:0/3084085259 wait complete. 2024-04-25T01:28:30.744 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 1 2024-04-25T01:28:31.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:31.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:31.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:31.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:31.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:28:31.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T01:28:31.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:28:31.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:31.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: Deploying daemon mgr.smithi138.mmfhyi on smithi138 2024-04-25T01:28:31.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:31 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/3084085259' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:32.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:32 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:32 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:28:33.099 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T01:28:33.099 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T01:28:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:28:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:33.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:33 smithi033 ceph-mon[26353]: Deploying daemon mon.smithi138 on smithi138 2024-04-25T01:28:34.565 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:28:41.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:28:41.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: mon.smithi033 calling monitor election 2024-04-25T01:28:41.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: mon.smithi138 calling monitor election 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/2309471832' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/crt"}]: dispatch 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: mon.smithi033 is new leader, mons smithi033,smithi138 in quorum (ranks 0,1) 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: monmap e2: 2 mons at {smithi033=[v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0],smithi138=[v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: fsmap 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: osdmap e5: 0 total, 0 up, 0 in 2024-04-25T01:28:41.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: mgrmap e17: smithi033.asemba(active, since 25s) 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: overall HEALTH_OK 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: Standby manager daemon smithi138.mmfhyi started 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/2309471832' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/2309471832' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/key"}]: dispatch 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/2309471832' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:41.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:42.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:42.877 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:42 smithi138 ceph-mon[32977]: mgrmap e18: smithi033.asemba(active, since 25s), standbys: smithi138.mmfhyi 2024-04-25T01:28:42.877 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:42 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi138.mmfhyi", "id": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:28:42.877 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:42 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:28:42.877 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:42 smithi138 ceph-mon[32977]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:42.877 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:42 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:28:42.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:42 smithi033 ceph-mon[26353]: mgrmap e18: smithi033.asemba(active, since 25s), standbys: smithi138.mmfhyi 2024-04-25T01:28:42.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:42 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi138.mmfhyi", "id": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:28:42.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:42 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:28:42.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:42 smithi033 ceph-mon[26353]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:42.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:42 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:28:43.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.120+0000 7f538388b700 1 -- 172.21.15.138:0/4081077841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53680061f0 msgr2=0x7f5368008650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:43.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.120+0000 7f538388b700 1 --2- 172.21.15.138:0/4081077841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53680061f0 0x7f5368008650 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f537c065b40 tx=0x7f5378002040 comp rx=0 tx=0).stop 2024-04-25T01:28:43.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.120+0000 7f538388b700 1 -- 172.21.15.138:0/4081077841 shutdown_connections 2024-04-25T01:28:43.099 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.120+0000 7f538388b700 1 --2- 172.21.15.138:0/4081077841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53680061f0 0x7f5368008650 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:43.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.120+0000 7f538388b700 1 -- 172.21.15.138:0/4081077841 >> 172.21.15.138:0/4081077841 conn(0x7f537c0f74d0 msgr2=0x7f537c0f98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:43.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.120+0000 7f538388b700 1 -- 172.21.15.138:0/4081077841 shutdown_connections 2024-04-25T01:28:43.100 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.120+0000 7f538388b700 1 -- 172.21.15.138:0/4081077841 wait complete. 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f538388b700 1 Processor -- start 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f538388b700 1 -- start start 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f538388b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 0x7f537c1132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f538388b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c1137e0 0x7f537c10d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f538388b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c113df0 con 0x7f537c1137e0 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f538388b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f537c10d740 con 0x7f53680061f0 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5381627700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 0x7f537c1132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5381627700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 0x7f537c1132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:49130/0 (socket says 172.21.15.138:49130) 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5381627700 1 -- 172.21.15.138:0/1995859061 learned_addr learned my addr 172.21.15.138:0/1995859061 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5380e26700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c1137e0 0x7f537c10d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:43.102 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5381627700 1 -- 172.21.15.138:0/1995859061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 msgr2=0x7f537c1132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5381627700 1 -- 172.21.15.138:0/1995859061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 msgr2=0x7f537c1132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5381627700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 0x7f537c1132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5381627700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 0x7f537c1132a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5380e26700 1 -- 172.21.15.138:0/1995859061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 msgr2=0x7f537c1132a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5380e26700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 0x7f537c1132a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.121+0000 7f5380e26700 1 -- 172.21.15.138:0/1995859061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5378005040 con 0x7f537c1137e0 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.122+0000 7f5380e26700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c1137e0 0x7f537c10d200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b790 tx=0x7f536c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.122+0000 7f5371ffb700 1 -- 172.21.15.138:0/1995859061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f536c011810 con 0x7f537c1137e0 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.122+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f537c10da20 con 0x7f537c1137e0 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.122+0000 7f5371ffb700 1 -- 172.21.15.138:0/1995859061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f536c011e40 con 0x7f537c1137e0 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.122+0000 7f5371ffb700 1 -- 172.21.15.138:0/1995859061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f536c00f2e0 con 0x7f537c1137e0 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.122+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f537c10df60 con 0x7f537c1137e0 2024-04-25T01:28:43.103 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.123+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f537c102130 con 0x7f537c1137e0 2024-04-25T01:28:43.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.123+0000 7f5371ffb700 1 -- 172.21.15.138:0/1995859061 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f536c010380 con 0x7f537c1137e0 2024-04-25T01:28:43.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.124+0000 7f5371ffb700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5360069320 0x7f536006b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:43.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.124+0000 7f5371ffb700 1 -- 172.21.15.138:0/1995859061 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f536c086410 con 0x7f537c1137e0 2024-04-25T01:28:43.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.124+0000 7f5381627700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5360069320 0x7f536006b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:43.104 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.125+0000 7f5381627700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5360069320 0x7f536006b7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5378004f50 tx=0x7f5378002040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:43.108 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.129+0000 7f5371ffb700 1 -- 172.21.15.138:0/1995859061 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f536c057190 con 0x7f537c1137e0 2024-04-25T01:28:43.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.405+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f537c10e8f0 con 0x7f537c1137e0 2024-04-25T01:28:43.384 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.405+0000 7f5371ffb700 1 -- 172.21.15.138:0/1995859061 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1017 (secure 0 0 0) 0x7f536c015090 con 0x7f537c1137e0 2024-04-25T01:28:43.384 INFO:teuthology.orchestra.run.smithi138.stdout: 2024-04-25T01:28:43.384 INFO:teuthology.orchestra.run.smithi138.stdout:{"epoch":2,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","modified":"2024-04-25T01:28:36.545717Z","created":"2024-04-25T01:26:02.814821Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi033","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:3300","nonce":0},{"type":"v1","addr":"172.21.15.33:6789","nonce":0}]},"addr":"172.21.15.33:6789/0","public_addr":"172.21.15.33:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi138","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:3300","nonce":0},{"type":"v1","addr":"172.21.15.138:6789","nonce":0}]},"addr":"172.21.15.138:6789/0","public_addr":"172.21.15.138:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T01:28:43.386 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5360069320 msgr2=0x7f536006b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5360069320 0x7f536006b7d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5378004f50 tx=0x7f5378002040 comp rx=0 tx=0).stop 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c1137e0 msgr2=0x7f537c10d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c1137e0 0x7f537c10d200 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f536c00b790 tx=0x7f536c0091a0 comp rx=0 tx=0).stop 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 shutdown_connections 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53680061f0 0x7f537c1132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5360069320 0x7f536006b7d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 --2- 172.21.15.138:0/1995859061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f537c1137e0 0x7f537c10d200 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 >> 172.21.15.138:0/1995859061 conn(0x7f537c0f74d0 msgr2=0x7f537c106030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.407+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 shutdown_connections 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:28:43.408+0000 7f538388b700 1 -- 172.21.15.138:0/1995859061 wait complete. 2024-04-25T01:28:43.387 INFO:teuthology.orchestra.run.smithi138.stderr:dumped monmap epoch 2 2024-04-25T01:28:44.029 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T01:28:44.030 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T01:28:44.625 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.625 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/1995859061' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: Reconfiguring mon.smithi033 (unknown last config time)... 2024-04-25T01:28:44.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:28:44.627 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:28:44.627 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:44.627 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: Reconfiguring daemon mon.smithi033 on smithi033 2024-04-25T01:28:44.627 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:44 smithi033 ceph-mon[26353]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:44.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:44.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:28:44.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-25T01:28:44.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T01:28:44.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/1995859061' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T01:28:44.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:28:44.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: Reconfiguring mon.smithi033 (unknown last config time)... 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: Reconfiguring daemon mon.smithi033 on smithi033 2024-04-25T01:28:44.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:44 smithi138 ceph-mon[32977]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:44.855 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:28:46.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:46 smithi033 ceph-mon[26353]: Reconfiguring mgr.smithi033.asemba (unknown last config time)... 2024-04-25T01:28:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.asemba", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:28:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:28:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:46 smithi033 ceph-mon[26353]: Reconfiguring daemon mgr.smithi033.asemba on smithi033 2024-04-25T01:28:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:28:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:46.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:46.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:46.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:46 smithi138 ceph-mon[32977]: Reconfiguring mgr.smithi033.asemba (unknown last config time)... 2024-04-25T01:28:46.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.asemba", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:28:46.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:28:46.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:46.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:46 smithi138 ceph-mon[32977]: Reconfiguring daemon mgr.smithi033.asemba on smithi033 2024-04-25T01:28:46.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:28:46.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.352+0000 7f74b3a42700 1 -- 172.21.15.33:0/583340401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 msgr2=0x7f74ac1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.352+0000 7f74b3a42700 1 --2- 172.21.15.33:0/583340401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 0x7f74ac1031c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f74a8004940 tx=0x7f74a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:47.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.352+0000 7f74b3a42700 1 -- 172.21.15.33:0/583340401 shutdown_connections 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.352+0000 7f74b3a42700 1 --2- 172.21.15.33:0/583340401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac103790 0x7f74ac0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.352+0000 7f74b3a42700 1 --2- 172.21.15.33:0/583340401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 0x7f74ac1031c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.352+0000 7f74b3a42700 1 -- 172.21.15.33:0/583340401 >> 172.21.15.33:0/583340401 conn(0x7f74ac0f8980 msgr2=0x7f74ac0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b3a42700 1 -- 172.21.15.33:0/583340401 shutdown_connections 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b3a42700 1 -- 172.21.15.33:0/583340401 wait complete. 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b3a42700 1 Processor -- start 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b3a42700 1 -- start start 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b3a42700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 0x7f74ac113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b3a42700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac103790 0x7f74ac1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b3a42700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac114710 con 0x7f74ac102df0 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b3a42700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac10db20 con 0x7f74ac103790 2024-04-25T01:28:47.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b17de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 0x7f74ac113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b17de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 0x7f74ac113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60404/0 (socket says 172.21.15.33:60404) 2024-04-25T01:28:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.353+0000 7f74b17de700 1 -- 172.21.15.33:0/112155255 learned_addr learned my addr 172.21.15.33:0/112155255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:28:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74b0fdd700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac103790 0x7f74ac1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74b17de700 1 -- 172.21.15.33:0/112155255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac103790 msgr2=0x7f74ac1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74b17de700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac103790 0x7f74ac1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74b17de700 1 -- 172.21.15.33:0/112155255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8005040 con 0x7f74ac102df0 2024-04-25T01:28:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74b17de700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 0x7f74ac113b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f74a800b3b0 tx=0x7f74a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74a1ffb700 1 -- 172.21.15.33:0/112155255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74a800e040 con 0x7f74ac102df0 2024-04-25T01:28:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74a1ffb700 1 -- 172.21.15.33:0/112155255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f74a8013420 con 0x7f74ac102df0 2024-04-25T01:28:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac10dda0 con 0x7f74ac102df0 2024-04-25T01:28:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74a1ffb700 1 -- 172.21.15.33:0/112155255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74a801c630 con 0x7f74ac102df0 2024-04-25T01:28:47.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.354+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac10e360 con 0x7f74ac102df0 2024-04-25T01:28:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.356+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac100100 con 0x7f74ac102df0 2024-04-25T01:28:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.356+0000 7f74a1ffb700 1 -- 172.21.15.33:0/112155255 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f74a8013590 con 0x7f74ac102df0 2024-04-25T01:28:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.356+0000 7f74a1ffb700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7498069210 0x7f749806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.356+0000 7f74a1ffb700 1 -- 172.21.15.33:0/112155255 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f74a8087250 con 0x7f74ac102df0 2024-04-25T01:28:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.356+0000 7f74b0fdd700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7498069210 0x7f749806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:47.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.357+0000 7f74b0fdd700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7498069210 0x7f749806b6c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f74ac10f420 tx=0x7f749c0043c0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:47.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.361+0000 7f74a1ffb700 1 -- 172.21.15.33:0/112155255 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f74a8054660 con 0x7f74ac102df0 2024-04-25T01:28:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.561+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f74ac04bed0 con 0x7f74ac102df0 2024-04-25T01:28:47.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.561+0000 7f74a1ffb700 1 -- 172.21.15.33:0/112155255 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f74a8086a00 con 0x7f74ac102df0 2024-04-25T01:28:47.540 INFO:teuthology.orchestra.run.smithi033.stdout:# minimal ceph.conf for b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:28:47.540 INFO:teuthology.orchestra.run.smithi033.stdout:[global] 2024-04-25T01:28:47.540 INFO:teuthology.orchestra.run.smithi033.stdout: fsid = b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:28:47.541 INFO:teuthology.orchestra.run.smithi033.stdout: mon_host = [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] 2024-04-25T01:28:47.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.563+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7498069210 msgr2=0x7f749806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.563+0000 7f74b3a42700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7498069210 0x7f749806b6c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f74ac10f420 tx=0x7f749c0043c0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.563+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 msgr2=0x7f74ac113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:47.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.563+0000 7f74b3a42700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 0x7f74ac113b80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f74a800b3b0 tx=0x7f74a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:47.543 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:47 smithi033 ceph-mon[26353]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:47.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.563+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 shutdown_connections 2024-04-25T01:28:47.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.563+0000 7f74b3a42700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7498069210 0x7f749806b6c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.563+0000 7f74b3a42700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74ac102df0 0x7f74ac113b80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.563+0000 7f74b3a42700 1 --2- 172.21.15.33:0/112155255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f74ac103790 0x7f74ac1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.563+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 >> 172.21.15.33:0/112155255 conn(0x7f74ac0f8980 msgr2=0x7f74ac1073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.564+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 shutdown_connections 2024-04-25T01:28:47.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:47.564+0000 7f74b3a42700 1 -- 172.21.15.33:0/112155255 wait complete. 2024-04-25T01:28:47.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:47 smithi138 ceph-mon[32977]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:48.337 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T01:28:48.338 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:28:48.338 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T01:28:48.368 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:28:48.368 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:28:48.437 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:28:48.437 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T01:28:48.467 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:28:48.468 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:28:48.535 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T01:28:48.535 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:28:48.535 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:28:48.547 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:48 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/112155255' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:48.547 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:48 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:48.547 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:48 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:48.547 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:48 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:28:48.547 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:48 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:48.551 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:28:48.551 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_1 2024-04-25T01:28:48.607 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:28:48.607 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:28:48.607 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T01:28:48.607 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:28:48.607 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:28:48.607 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-25 01:28:07.647827758 +0000 2024-04-25T01:28:48.607 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-25 01:22:55.794086770 +0000 2024-04-25T01:28:48.607 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-25 01:22:55.794086770 +0000 2024-04-25T01:28:48.607 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-25 01:22:55.794086770 +0000 2024-04-25T01:28:48.608 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:28:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-25T01:28:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-25T01:28:48.673 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000252197 s, 2.0 MB/s 2024-04-25T01:28:48.675 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:28:48.732 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_2 2024-04-25T01:28:48.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:48 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/112155255' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:48.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:48 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:48.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:48 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:48.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:48 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:28:48.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:48 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:48.788 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:28:48.788 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:28:48.788 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T01:28:48.788 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:28:48.788 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:28:48.789 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-25 01:28:07.648827742 +0000 2024-04-25T01:28:48.789 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-25 01:22:56.130081254 +0000 2024-04-25T01:28:48.789 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-25 01:22:56.130081254 +0000 2024-04-25T01:28:48.789 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-25 01:22:56.130081254 +0000 2024-04-25T01:28:48.789 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:28:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-25T01:28:48.854 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-25T01:28:48.855 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000202881 s, 2.5 MB/s 2024-04-25T01:28:48.856 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:28:48.914 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_3 2024-04-25T01:28:48.968 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:28:48.969 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:28:48.969 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T01:28:48.969 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:28:48.969 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:28:48.969 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-25 01:28:07.648827742 +0000 2024-04-25T01:28:48.969 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-25 01:22:56.482075475 +0000 2024-04-25T01:28:48.969 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-25 01:22:56.482075475 +0000 2024-04-25T01:28:48.969 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-25 01:22:56.482075475 +0000 2024-04-25T01:28:48.969 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:28:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-25T01:28:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-25T01:28:49.032 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000214596 s, 2.4 MB/s 2024-04-25T01:28:49.033 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:28:49.090 DEBUG:teuthology.orchestra.run.smithi033:> stat /dev/vg_nvme/lv_4 2024-04-25T01:28:49.147 INFO:teuthology.orchestra.run.smithi033.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:28:49.147 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:28:49.147 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T01:28:49.147 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:28:49.147 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:28:49.147 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 2024-04-25 01:28:07.649827726 +0000 2024-04-25T01:28:49.147 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-25 01:22:56.819069943 +0000 2024-04-25T01:28:49.147 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-25 01:22:56.819069943 +0000 2024-04-25T01:28:49.147 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: 2024-04-25 01:22:56.819069943 +0000 2024-04-25T01:28:49.147 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:28:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records in 2024-04-25T01:28:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:1+0 records out 2024-04-25T01:28:49.212 INFO:teuthology.orchestra.run.smithi033.stderr:512 bytes copied, 0.000188129 s, 2.7 MB/s 2024-04-25T01:28:49.213 DEBUG:teuthology.orchestra.run.smithi033:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:28:49.269 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:28:49.270 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T01:28:49.288 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T01:28:49.288 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_1 2024-04-25T01:28:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T01:28:49.347 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:28:49.347 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T01:28:49.347 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:28:49.347 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:28:49.348 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 01:28:35.287700303 +0000 2024-04-25T01:28:49.348 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 01:22:51.718598451 +0000 2024-04-25T01:28:49.348 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 01:22:51.718598451 +0000 2024-04-25T01:28:49.348 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 01:22:51.718598451 +0000 2024-04-25T01:28:49.348 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T01:28:49.415 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T01:28:49.415 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T01:28:49.415 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000270982 s, 1.9 MB/s 2024-04-25T01:28:49.417 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T01:28:49.475 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_2 2024-04-25T01:28:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:49 smithi033 ceph-mon[26353]: Reconfiguring crash.smithi033 (monmap changed)... 2024-04-25T01:28:49.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:49 smithi033 ceph-mon[26353]: Reconfiguring daemon crash.smithi033 on smithi033 2024-04-25T01:28:49.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:49 smithi033 ceph-mon[26353]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T01:28:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:28:49.532 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T01:28:49.532 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:28:49.532 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:28:49.532 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 01:28:35.288700283 +0000 2024-04-25T01:28:49.532 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 01:22:52.055591419 +0000 2024-04-25T01:28:49.532 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 01:22:52.055591419 +0000 2024-04-25T01:28:49.532 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 01:22:52.055591419 +0000 2024-04-25T01:28:49.533 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T01:28:49.596 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:49 smithi138 ceph-mon[32977]: Reconfiguring crash.smithi033 (monmap changed)... 2024-04-25T01:28:49.596 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:49 smithi138 ceph-mon[32977]: Reconfiguring daemon crash.smithi033 on smithi033 2024-04-25T01:28:49.596 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:49 smithi138 ceph-mon[32977]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:49.598 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T01:28:49.599 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T01:28:49.599 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.00018268 s, 2.8 MB/s 2024-04-25T01:28:49.600 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T01:28:49.657 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_3 2024-04-25T01:28:49.714 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T01:28:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:28:49.715 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-25T01:28:49.715 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:28:49.715 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:28:49.715 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 01:28:35.288700283 +0000 2024-04-25T01:28:49.715 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 01:22:52.394584345 +0000 2024-04-25T01:28:49.715 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 01:22:52.394584345 +0000 2024-04-25T01:28:49.715 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 01:22:52.394584345 +0000 2024-04-25T01:28:49.715 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T01:28:49.780 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T01:28:49.780 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T01:28:49.780 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000216388 s, 2.4 MB/s 2024-04-25T01:28:49.782 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T01:28:49.839 DEBUG:teuthology.orchestra.run.smithi138:> stat /dev/vg_nvme/lv_4 2024-04-25T01:28:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T01:28:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T01:28:49.895 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T01:28:49.895 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:28:49.895 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T01:28:49.895 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 2024-04-25 01:28:35.288700283 +0000 2024-04-25T01:28:49.895 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 01:22:52.752576874 +0000 2024-04-25T01:28:49.895 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 01:22:52.752576874 +0000 2024-04-25T01:28:49.895 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: 2024-04-25 01:22:52.752576874 +0000 2024-04-25T01:28:49.895 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T01:28:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records in 2024-04-25T01:28:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:1+0 records out 2024-04-25T01:28:49.958 INFO:teuthology.orchestra.run.smithi138.stderr:512 bytes copied, 0.000173065 s, 3.0 MB/s 2024-04-25T01:28:49.959 DEBUG:teuthology.orchestra.run.smithi138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T01:28:50.016 INFO:tasks.cephadm:Deploying osd.0 on smithi033 with /dev/vg_nvme/lv_4... 2024-04-25T01:28:50.016 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T01:28:50.713 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:28:50.820 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:50 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:50.820 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:50 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:50.820 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:50 smithi033 ceph-mon[26353]: Reconfiguring alertmanager.smithi033 (dependencies changed)... 2024-04-25T01:28:50.821 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:50 smithi033 ceph-mon[26353]: Reconfiguring daemon alertmanager.smithi033 on smithi033 2024-04-25T01:28:50.821 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:50 smithi033 ceph-mon[26353]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:51.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:50 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:51.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:50 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:51.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:50 smithi138 ceph-mon[32977]: Reconfiguring alertmanager.smithi033 (dependencies changed)... 2024-04-25T01:28:51.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:50 smithi138 ceph-mon[32977]: Reconfiguring daemon alertmanager.smithi033 on smithi033 2024-04-25T01:28:51.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:50 smithi138 ceph-mon[32977]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:53.449 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:53 smithi033 ceph-mon[26353]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:53.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:53 smithi138 ceph-mon[32977]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:53.923 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T01:28:53.924 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T01:28:53.924 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-25T01:28:53.924 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-25T01:28:53.924 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0116924 s, 897 MB/s 2024-04-25T01:28:53.924 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 2024-04-25T01:28:53.924 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-25T01:28:54.664 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_4 2024-04-25T01:28:55.321 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:28:55.447 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:55 smithi033 ceph-mon[26353]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:55.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:55 smithi138 ceph-mon[32977]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.117+0000 7f71c1a95700 1 -- 172.21.15.33:0/3979370552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fce90 msgr2=0x7f71bc0fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.117+0000 7f71c1a95700 1 --2- 172.21.15.33:0/3979370552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fce90 0x7f71bc0fd2a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f71ac004940 tx=0x7f71ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:28:57.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.117+0000 7f71c1a95700 1 -- 172.21.15.33:0/3979370552 shutdown_connections 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.117+0000 7f71c1a95700 1 --2- 172.21.15.33:0/3979370552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71bc0fe090 0x7f71bc0fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.117+0000 7f71c1a95700 1 --2- 172.21.15.33:0/3979370552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fce90 0x7f71bc0fd2a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.117+0000 7f71c1a95700 1 -- 172.21.15.33:0/3979370552 >> 172.21.15.33:0/3979370552 conn(0x7f71bc0f85c0 msgr2=0x7f71bc0fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.117+0000 7f71c1a95700 1 -- 172.21.15.33:0/3979370552 shutdown_connections 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.117+0000 7f71c1a95700 1 -- 172.21.15.33:0/3979370552 wait complete. 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71c1a95700 1 Processor -- start 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71c1a95700 1 -- start start 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71c1a95700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71bc0fce90 0x7f71bc071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71c1a95700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fe090 0x7f71bc0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71c1a95700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc070860 con 0x7f71bc0fe090 2024-04-25T01:28:57.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71c1a95700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0709d0 con 0x7f71bc0fce90 2024-04-25T01:28:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fe090 0x7f71bc0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71baffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fe090 0x7f71bc0702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46686/0 (socket says 172.21.15.33:46686) 2024-04-25T01:28:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71baffd700 1 -- 172.21.15.33:0/1596240479 learned_addr learned my addr 172.21.15.33:0/1596240479 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:28:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.118+0000 7f71bb7fe700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71bc0fce90 0x7f71bc071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.119+0000 7f71baffd700 1 -- 172.21.15.33:0/1596240479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71bc0fce90 msgr2=0x7f71bc071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:28:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.119+0000 7f71baffd700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71bc0fce90 0x7f71bc071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:28:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.119+0000 7f71baffd700 1 -- 172.21.15.33:0/1596240479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71ac005040 con 0x7f71bc0fe090 2024-04-25T01:28:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.119+0000 7f71baffd700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fe090 0x7f71bc0702b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f71a4002a50 tx=0x7f71a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:57.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.119+0000 7f71b3fff700 1 -- 172.21.15.33:0/1596240479 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71a4007cc0 con 0x7f71bc0fe090 2024-04-25T01:28:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.119+0000 7f71b3fff700 1 -- 172.21.15.33:0/1596240479 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71a400f040 con 0x7f71bc0fe090 2024-04-25T01:28:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.119+0000 7f71b3fff700 1 -- 172.21.15.33:0/1596240479 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71a400da30 con 0x7f71bc0fe090 2024-04-25T01:28:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.119+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc070cb0 con 0x7f71bc0fe090 2024-04-25T01:28:57.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.119+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc115a80 con 0x7f71bc0fe090 2024-04-25T01:28:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.120+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc05f5c0 con 0x7f71bc0fe090 2024-04-25T01:28:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.120+0000 7f71b3fff700 1 -- 172.21.15.33:0/1596240479 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f71a400db90 con 0x7f71bc0fe090 2024-04-25T01:28:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.121+0000 7f71b3fff700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f71a8069210 0x7f71a806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:28:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.121+0000 7f71b3fff700 1 -- 172.21.15.33:0/1596240479 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f71a4085d50 con 0x7f71bc0fe090 2024-04-25T01:28:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.121+0000 7f71bb7fe700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f71a8069210 0x7f71a806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:28:57.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.121+0000 7f71bb7fe700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f71a8069210 0x7f71a806b6c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71bc071b70 tx=0x7f71ac009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:28:57.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.126+0000 7f71b3fff700 1 -- 172.21.15.33:0/1596240479 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f71a4089030 con 0x7f71bc0fe090 2024-04-25T01:28:57.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:57 smithi033 ceph-mon[26353]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:57.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:28:57.333+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f71bc102450 con 0x7f71a8069210 2024-04-25T01:28:57.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:57 smithi138 ceph-mon[32977]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:58.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:58 smithi033 ceph-mon[26353]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:28:58.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:58 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:28:58.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:58 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:28:58.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:58 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:58.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:58 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:58.171 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:58 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:58.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:58 smithi138 ceph-mon[32977]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:28:58.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:58 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:28:58.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:58 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:28:58.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:58 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:28:58.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:58 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:58.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:58 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:28:59.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:59 smithi033 ceph-mon[26353]: Reconfiguring grafana.smithi033 (dependencies changed)... 2024-04-25T01:28:59.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:59 smithi033 ceph-mon[26353]: Reconfiguring daemon grafana.smithi033 on smithi033 2024-04-25T01:28:59.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:28:59 smithi033 ceph-mon[26353]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:28:59.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:59 smithi138 ceph-mon[32977]: Reconfiguring grafana.smithi033 (dependencies changed)... 2024-04-25T01:28:59.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:59 smithi138 ceph-mon[32977]: Reconfiguring daemon grafana.smithi033 on smithi033 2024-04-25T01:28:59.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:28:59 smithi138 ceph-mon[32977]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:01.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:01 smithi138 ceph-mon[32977]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:01.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:29:01.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:01 smithi033 ceph-mon[26353]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:01.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:29:03.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:03 smithi033 ceph-mon[26353]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:03.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:03.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:03.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:03 smithi138 ceph-mon[32977]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:03.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:03.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:04 smithi033 ceph-mon[26353]: Reconfiguring prometheus.smithi033 (dependencies changed)... 2024-04-25T01:29:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:04 smithi033 ceph-mon[26353]: Reconfiguring daemon prometheus.smithi033 on smithi033 2024-04-25T01:29:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:04 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/3262829756' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68af93d4-7aa8-485f-826e-62a6e362907b"}]: dispatch 2024-04-25T01:29:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:04 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/3262829756' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68af93d4-7aa8-485f-826e-62a6e362907b"}]': finished 2024-04-25T01:29:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:04 smithi033 ceph-mon[26353]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T01:29:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:04 smithi138 ceph-mon[32977]: Reconfiguring prometheus.smithi033 (dependencies changed)... 2024-04-25T01:29:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:04 smithi138 ceph-mon[32977]: Reconfiguring daemon prometheus.smithi033 on smithi033 2024-04-25T01:29:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:04 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/3262829756' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68af93d4-7aa8-485f-826e-62a6e362907b"}]: dispatch 2024-04-25T01:29:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:04 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/3262829756' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68af93d4-7aa8-485f-826e-62a6e362907b"}]': finished 2024-04-25T01:29:04.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:04 smithi138 ceph-mon[32977]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T01:29:04.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:05.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:05 smithi033 ceph-mon[26353]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:05.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:05 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/347313486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:29:05.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:05 smithi138 ceph-mon[32977]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:05.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:05 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/347313486' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:29:07.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:07 smithi033 ceph-mon[26353]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:07 smithi138 ceph-mon[32977]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:09.463 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:09 smithi033 ceph-mon[26353]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:09.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:09 smithi138 ceph-mon[32977]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:11.341 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:11 smithi033 ceph-mon[26353]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:11.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:11 smithi138 ceph-mon[32977]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:13.398 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:13 smithi033 ceph-mon[26353]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:13.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:13 smithi138 ceph-mon[32977]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:15.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:14 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:15.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:14 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:15.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:14 smithi033 ceph-mon[26353]: Reconfiguring crash.smithi138 (monmap changed)... 2024-04-25T01:29:15.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:14 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:29:15.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:14 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:15.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:14 smithi033 ceph-mon[26353]: Reconfiguring daemon crash.smithi138 on smithi138 2024-04-25T01:29:15.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:14 smithi033 ceph-mon[26353]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:15.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:14 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:15.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:14 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:15.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:14 smithi138 ceph-mon[32977]: Reconfiguring crash.smithi138 (monmap changed)... 2024-04-25T01:29:15.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:14 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:29:15.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:14 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:15.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:14 smithi138 ceph-mon[32977]: Reconfiguring daemon crash.smithi138 on smithi138 2024-04-25T01:29:15.246 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:14 smithi138 ceph-mon[32977]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:16.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:16.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:16.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: Reconfiguring mgr.smithi138.mmfhyi (monmap changed)... 2024-04-25T01:29:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:29:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:29:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: Reconfiguring daemon mgr.smithi138.mmfhyi on smithi138 2024-04-25T01:29:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:29:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:29:16.551 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:16.551 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:16.551 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: Reconfiguring mgr.smithi138.mmfhyi (monmap changed)... 2024-04-25T01:29:16.552 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:29:16.552 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:29:16.552 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:16.552 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: Reconfiguring daemon mgr.smithi138.mmfhyi on smithi138 2024-04-25T01:29:16.552 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:29:16.552 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:16.552 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:29:17.479 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:17 smithi033 ceph-mon[26353]: Deploying daemon osd.0 on smithi033 2024-04-25T01:29:17.479 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:17 smithi033 ceph-mon[26353]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:17.480 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:17.480 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:17.480 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:29:17.480 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:29:17.480 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:17.523 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:17 smithi138 ceph-mon[32977]: Deploying daemon osd.0 on smithi033 2024-04-25T01:29:17.523 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:17 smithi138 ceph-mon[32977]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:17.523 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:17.523 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:17.523 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:29:17.524 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:29:17.524 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:18.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: Reconfiguring mon.smithi138 (monmap changed)... 2024-04-25T01:29:18.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: Reconfiguring daemon mon.smithi138 on smithi138 2024-04-25T01:29:18.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:29:18.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033:9093"}]: dispatch 2024-04-25T01:29:18.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:29:18.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033:3000"}]: dispatch 2024-04-25T01:29:18.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:29:18.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033:9095"}]: dispatch 2024-04-25T01:29:18.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:29:18.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: Reconfiguring mon.smithi138 (monmap changed)... 2024-04-25T01:29:18.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: Reconfiguring daemon mon.smithi138 on smithi138 2024-04-25T01:29:18.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:29:18.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033:9093"}]: dispatch 2024-04-25T01:29:18.568 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.569 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:29:18.569 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033:3000"}]: dispatch 2024-04-25T01:29:18.569 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.569 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:29:18.569 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033:9095"}]: dispatch 2024-04-25T01:29:18.569 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:18.569 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:29:19.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:19 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:29:19.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:19 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033:9093"}]: dispatch 2024-04-25T01:29:19.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:19 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:29:19.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:19 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033:3000"}]: dispatch 2024-04-25T01:29:19.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:19 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:29:19.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:19 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033:9095"}]: dispatch 2024-04-25T01:29:19.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:19 smithi033 ceph-mon[26353]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:19.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:19 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:29:19.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:19 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033:9093"}]: dispatch 2024-04-25T01:29:19.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:19 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:29:19.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:19 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033:3000"}]: dispatch 2024-04-25T01:29:19.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:19 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:29:19.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:19 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033:9095"}]: dispatch 2024-04-25T01:29:19.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:19 smithi138 ceph-mon[32977]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:20.929 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:20 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:20.929 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:20 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:20.929 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:20 smithi138 ceph-mon[32977]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:20.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:20 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:20.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:20 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:20.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:20 smithi033 ceph-mon[26353]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:23.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:23 smithi033 ceph-mon[26353]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:23.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:23 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:23.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:23 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:23.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:23 smithi138 ceph-mon[32977]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:23.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:23 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:23.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:23 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:24.525 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:24 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:24.525 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:24 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:24.525 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:24 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:24.526 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:24 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:29:24.526 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:24 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:24.526 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:24 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:29:24.526 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:24 smithi033 ceph-mon[26353]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:24.685 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:24 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:24.685 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:24 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:24.685 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:24 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:24.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:24 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:29:24.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:24 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:24.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:24 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:29:24.686 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:24 smithi138 ceph-mon[32977]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.359+0000 7f71b3fff700 1 -- 172.21.15.33:0/1596240479 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f71bc102450 con 0x7f71a8069210 2024-04-25T01:29:26.338 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 0 on host 'smithi033' 2024-04-25T01:29:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f71a8069210 msgr2=0x7f71a806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f71a8069210 0x7f71a806b6c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f71bc071b70 tx=0x7f71ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:29:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fe090 msgr2=0x7f71bc0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fe090 0x7f71bc0702b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f71a4002a50 tx=0x7f71a40046b0 comp rx=0 tx=0).stop 2024-04-25T01:29:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 shutdown_connections 2024-04-25T01:29:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71bc0fce90 0x7f71bc071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f71a8069210 0x7f71a806b6c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 --2- 172.21.15.33:0/1596240479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71bc0fe090 0x7f71bc0702b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 >> 172.21.15.33:0/1596240479 conn(0x7f71bc0f85c0 msgr2=0x7f71bc0f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:26.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.361+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 shutdown_connections 2024-04-25T01:29:26.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:26.362+0000 7f71c1a95700 1 -- 172.21.15.33:0/1596240479 wait complete. 2024-04-25T01:29:26.965 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:26.966 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:26.966 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:26 smithi033 ceph-mon[26353]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:26.966 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:26.966 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:26.966 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:26 smithi033 ceph-mon[26353]: from='osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:29:26.990 DEBUG:teuthology.orchestra.run.smithi033:osd.0> sudo journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T01:29:26.992 INFO:tasks.cephadm:Deploying osd.1 on smithi033 with /dev/vg_nvme/lv_3... 2024-04-25T01:29:26.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T01:29:27.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:27.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:27.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:26 smithi138 ceph-mon[32977]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:27.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:27.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:27.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:26 smithi138 ceph-mon[32977]: from='osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:29:27.735 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:29:28.489 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:28 smithi033 ceph-mon[26353]: from='osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:29:28.489 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:28 smithi033 ceph-mon[26353]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T01:29:28.489 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:28.490 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:28 smithi033 ceph-mon[26353]: from='osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:29:28.490 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:29:28 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[45018]: 2024-04-25T01:29:28.335+0000 7fced2f92700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T01:29:28.490 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:29:28 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[45018]: 2024-04-25T01:29:28.338+0000 7fcecfd91700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:29:28.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:28 smithi138 ceph-mon[32977]: from='osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:29:28.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:28 smithi138 ceph-mon[32977]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T01:29:28.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:28.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:28 smithi138 ceph-mon[32977]: from='osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:29:29.619 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:29 smithi033 ceph-mon[26353]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:29.619 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:29 smithi033 ceph-mon[26353]: from='osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-25T01:29:29.619 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:29 smithi033 ceph-mon[26353]: osdmap e8: 1 total, 0 up, 1 in 2024-04-25T01:29:29.619 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:29.619 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:29.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:29 smithi138 ceph-mon[32977]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:29.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:29 smithi138 ceph-mon[32977]: from='osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-25T01:29:29.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:29 smithi138 ceph-mon[32977]: osdmap e8: 1 total, 0 up, 1 in 2024-04-25T01:29:29.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:29.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T01:29:30.325 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T01:29:30.326 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-25T01:29:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-25T01:29:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0117771 s, 890 MB/s 2024-04-25T01:29:30.326 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-25T01:29:30.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:30 smithi033 ceph-mon[26353]: purged_snaps scrub starts 2024-04-25T01:29:30.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:30 smithi033 ceph-mon[26353]: purged_snaps scrub ok 2024-04-25T01:29:30.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:30 smithi033 ceph-mon[26353]: osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] boot 2024-04-25T01:29:30.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:30 smithi033 ceph-mon[26353]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T01:29:30.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:30 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:30.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:30 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T01:29:30.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:30 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T01:29:30.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:30 smithi138 ceph-mon[32977]: osd.0 [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] boot 2024-04-25T01:29:30.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:30 smithi138 ceph-mon[32977]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T01:29:30.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:30 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:29:31.462 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_3 2024-04-25T01:29:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:31 smithi033 ceph-mon[26353]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:31 smithi033 ceph-mon[26353]: osdmap e10: 1 total, 1 up, 1 in 2024-04-25T01:29:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:29:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:31 smithi138 ceph-mon[32977]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T01:29:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:31 smithi138 ceph-mon[32977]: osdmap e10: 1 total, 1 up, 1 in 2024-04-25T01:29:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:31 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:29:31.815 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:29:33.119 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:32 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:33.119 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:32 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:33.119 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:32 smithi033 ceph-mon[26353]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:33.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:32 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:33.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:32 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:33.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:32 smithi138 ceph-mon[32977]: pgmap v36: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.799+0000 7f5f755ce700 1 -- 172.21.15.33:0/1274753007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fce60 msgr2=0x7f5f700fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.799+0000 7f5f755ce700 1 --2- 172.21.15.33:0/1274753007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fce60 0x7f5f700fd270 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5f60004940 tx=0x7f5f60009e30 comp rx=0 tx=0).stop 2024-04-25T01:29:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.799+0000 7f5f755ce700 1 -- 172.21.15.33:0/1274753007 shutdown_connections 2024-04-25T01:29:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.799+0000 7f5f755ce700 1 --2- 172.21.15.33:0/1274753007 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700fdfd0 0x7f5f700fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.799+0000 7f5f755ce700 1 --2- 172.21.15.33:0/1274753007 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fce60 0x7f5f700fd270 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.799+0000 7f5f755ce700 1 -- 172.21.15.33:0/1274753007 >> 172.21.15.33:0/1274753007 conn(0x7f5f700f8540 msgr2=0x7f5f700fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.799+0000 7f5f755ce700 1 -- 172.21.15.33:0/1274753007 shutdown_connections 2024-04-25T01:29:34.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.799+0000 7f5f755ce700 1 -- 172.21.15.33:0/1274753007 wait complete. 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.800+0000 7f5f755ce700 1 Processor -- start 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.800+0000 7f5f755ce700 1 -- start start 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.800+0000 7f5f755ce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700fce60 0x7f5f7010d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.800+0000 7f5f755ce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fdfd0 0x7f5f7010de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.800+0000 7f5f755ce700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7010e500 con 0x7f5f700fdfd0 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.800+0000 7f5f755ce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f70112ea0 con 0x7f5f700fce60 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f6e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fdfd0 0x7f5f7010de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f6e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fdfd0 0x7f5f7010de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60666/0 (socket says 172.21.15.33:60666) 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f6e7fc700 1 -- 172.21.15.33:0/2661253259 learned_addr learned my addr 172.21.15.33:0/2661253259 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f6effd700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700fce60 0x7f5f7010d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f6e7fc700 1 -- 172.21.15.33:0/2661253259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700fce60 msgr2=0x7f5f7010d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f6e7fc700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700fce60 0x7f5f7010d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:34.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f6e7fc700 1 -- 172.21.15.33:0/2661253259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f60005040 con 0x7f5f700fdfd0 2024-04-25T01:29:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f6effd700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700fce60 0x7f5f7010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:29:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f6e7fc700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fdfd0 0x7f5f7010de70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b790 tx=0x7f5f58009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f677fe700 1 -- 172.21.15.33:0/2661253259 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5800f770 con 0x7f5f700fdfd0 2024-04-25T01:29:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f70113100 con 0x7f5f700fdfd0 2024-04-25T01:29:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f677fe700 1 -- 172.21.15.33:0/2661253259 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f5800fda0 con 0x7f5f700fdfd0 2024-04-25T01:29:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f677fe700 1 -- 172.21.15.33:0/2661253259 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5800d310 con 0x7f5f700fdfd0 2024-04-25T01:29:34.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.801+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f70070210 con 0x7f5f700fdfd0 2024-04-25T01:29:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.803+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7004c710 con 0x7f5f700fdfd0 2024-04-25T01:29:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.803+0000 7f5f677fe700 1 -- 172.21.15.33:0/2661253259 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f5f58010420 con 0x7f5f700fdfd0 2024-04-25T01:29:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.803+0000 7f5f677fe700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5f5c06d620 0x7f5f5c06fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:29:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.803+0000 7f5f677fe700 1 -- 172.21.15.33:0/2661253259 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7f5f58086300 con 0x7f5f700fdfd0 2024-04-25T01:29:34.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.803+0000 7f5f6effd700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5f5c06d620 0x7f5f5c06fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:29:34.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.804+0000 7f5f6effd700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5f5c06d620 0x7f5f5c06fad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000b3b0 tx=0x7f5f60009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:29:34.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:34.808+0000 7f5f677fe700 1 -- 172.21.15.33:0/2661253259 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5f58056f40 con 0x7f5f700fdfd0 2024-04-25T01:29:35.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:35.045+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f701022b0 con 0x7f5f5c06d620 2024-04-25T01:29:35.305 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:35 smithi033 ceph-mon[26353]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:35.305 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:35 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:29:35.305 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:35 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:29:35.305 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:35 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:35 smithi138 ceph-mon[32977]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:35 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:29:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:35 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:29:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:35 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:36.214 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:36 smithi033 ceph-mon[26353]: from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:29:36.215 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:36 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:36.215 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:36 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:36.215 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:36 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:29:36.215 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:36 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:36.215 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:36 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:36.215 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:36 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:29:36.215 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:36 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:36.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:36 smithi138 ceph-mon[32977]: from='client.14256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:29:36.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:36 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:36.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:36 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:36.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:36 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:29:36.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:36 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:36.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:36 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:36.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:36 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:29:36.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:36 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:37.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:37 smithi033 ceph-mon[26353]: Detected new or changed devices on smithi033 2024-04-25T01:29:37.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:37 smithi033 ceph-mon[26353]: Adjusting osd_memory_target on smithi033 to 12869M 2024-04-25T01:29:37.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:37 smithi033 ceph-mon[26353]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:37.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:37 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:29:37.423 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:37 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi033 2024-04-25T01:29:37.423 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:37 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi033 to 12869M 2024-04-25T01:29:37.423 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:37 smithi138 ceph-mon[32977]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:37.423 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:37 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:29:39.110 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:39 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:39.110 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:39 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:39.110 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:39 smithi033 ceph-mon[26353]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:39.110 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:39 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1170278772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fdb7e9b-1138-43db-a11f-3e30e286c1a9"}]: dispatch 2024-04-25T01:29:39.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:39 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1170278772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fdb7e9b-1138-43db-a11f-3e30e286c1a9"}]': finished 2024-04-25T01:29:39.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:39 smithi033 ceph-mon[26353]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T01:29:39.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:39 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:39.111 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:39 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/21456052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:29:39.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:39 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:39.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:39 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:39.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:39 smithi138 ceph-mon[32977]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:39.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:39 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1170278772' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fdb7e9b-1138-43db-a11f-3e30e286c1a9"}]: dispatch 2024-04-25T01:29:39.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:39 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1170278772' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fdb7e9b-1138-43db-a11f-3e30e286c1a9"}]': finished 2024-04-25T01:29:39.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:39 smithi138 ceph-mon[32977]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T01:29:39.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:39 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:39.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:39 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/21456052' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:29:41.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:41 smithi033 ceph-mon[26353]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:41.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:41.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:41.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:41.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:29:41.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:41 smithi138 ceph-mon[32977]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:41.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:29:41.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:43.441 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:43 smithi033 ceph-mon[26353]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:43.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:43 smithi138 ceph-mon[32977]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:45.418 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:45 smithi033 ceph-mon[26353]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:45.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:45 smithi138 ceph-mon[32977]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:46.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:29:46.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:46.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:29:46.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:29:46.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:29:46.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:29:47.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:47 smithi033 ceph-mon[26353]: Deploying daemon osd.1 on smithi033 2024-04-25T01:29:47.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:47 smithi033 ceph-mon[26353]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:47.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:47 smithi138 ceph-mon[32977]: Deploying daemon osd.1 on smithi033 2024-04-25T01:29:47.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:47 smithi138 ceph-mon[32977]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:49 smithi033 ceph-mon[26353]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:49.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:49 smithi138 ceph-mon[32977]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:51.379 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:51 smithi033 ceph-mon[26353]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:51.379 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:51 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:51.379 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:51 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:51.379 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:51 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:29:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:51 smithi138 ceph-mon[32977]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:51 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:51 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:51 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:29:53.198 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:53 smithi033 ceph-mon[26353]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:53.508 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:53 smithi138 ceph-mon[32977]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:54.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.628+0000 7f5f677fe700 1 -- 172.21.15.33:0/2661253259 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5f701022b0 con 0x7f5f5c06d620 2024-04-25T01:29:54.606 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 1 on host 'smithi033' 2024-04-25T01:29:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5f5c06d620 msgr2=0x7f5f5c06fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5f5c06d620 0x7f5f5c06fad0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000b3b0 tx=0x7f5f60009e30 comp rx=0 tx=0).stop 2024-04-25T01:29:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fdfd0 msgr2=0x7f5f7010de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:29:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fdfd0 0x7f5f7010de70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800b790 tx=0x7f5f58009250 comp rx=0 tx=0).stop 2024-04-25T01:29:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 shutdown_connections 2024-04-25T01:29:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5f700fce60 0x7f5f7010d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:54.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5f5c06d620 0x7f5f5c06fad0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 --2- 172.21.15.33:0/2661253259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5f700fdfd0 0x7f5f7010de70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:29:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 >> 172.21.15.33:0/2661253259 conn(0x7f5f700f8540 msgr2=0x7f5f700f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:29:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 shutdown_connections 2024-04-25T01:29:54.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:29:54.630+0000 7f5f755ce700 1 -- 172.21.15.33:0/2661253259 wait complete. 2024-04-25T01:29:55.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:55 smithi033 ceph-mon[26353]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:55.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:55 smithi033 ceph-mon[26353]: from='osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:29:55.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:55 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:55.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:55 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:55.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:55 smithi138 ceph-mon[32977]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:55.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:55 smithi138 ceph-mon[32977]: from='osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:29:55.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:55 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:55.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:55 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:55.548 DEBUG:teuthology.orchestra.run.smithi033:osd.1> sudo journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T01:29:55.551 INFO:tasks.cephadm:Deploying osd.2 on smithi033 with /dev/vg_nvme/lv_2... 2024-04-25T01:29:55.551 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T01:29:55.940 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:29:56.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:56 smithi033 ceph-mon[26353]: from='osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:29:56.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:56 smithi033 ceph-mon[26353]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T01:29:56.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:56 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:56.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:56 smithi033 ceph-mon[26353]: from='osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:29:56.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:56 smithi033 ceph-mon[26353]: from='osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-25T01:29:56.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:56 smithi033 ceph-mon[26353]: osdmap e13: 2 total, 1 up, 2 in 2024-04-25T01:29:56.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:56 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:56.396 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:29:56 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[53014]: 2024-04-25T01:29:56.188+0000 7fd36d984700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T01:29:56.396 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:29:56 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[53014]: 2024-04-25T01:29:56.193+0000 7fd367f7e700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:29:56.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:56 smithi138 ceph-mon[32977]: from='osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:29:56.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:56 smithi138 ceph-mon[32977]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T01:29:56.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:56 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:56.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:56 smithi138 ceph-mon[32977]: from='osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:29:56.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:56 smithi138 ceph-mon[32977]: from='osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-25T01:29:56.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:56 smithi138 ceph-mon[32977]: osdmap e13: 2 total, 1 up, 2 in 2024-04-25T01:29:56.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:56 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:57.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:57 smithi033 ceph-mon[26353]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:57.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:57.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:57.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:57 smithi138 ceph-mon[32977]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:57.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:57.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:58.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:58 smithi033 ceph-mon[26353]: purged_snaps scrub starts 2024-04-25T01:29:58.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:58 smithi033 ceph-mon[26353]: purged_snaps scrub ok 2024-04-25T01:29:58.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:58 smithi033 ceph-mon[26353]: osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] boot 2024-04-25T01:29:58.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:58 smithi033 ceph-mon[26353]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T01:29:58.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:58 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:58.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:58 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T01:29:58.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:58 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T01:29:58.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:58 smithi138 ceph-mon[32977]: osd.1 [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] boot 2024-04-25T01:29:58.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:58 smithi138 ceph-mon[32977]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T01:29:58.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:58 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:29:59.001 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T01:29:59.001 INFO:teuthology.orchestra.run.smithi033.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T01:29:59.001 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10+0 records in 2024-04-25T01:29:59.001 INFO:teuthology.orchestra.run.smithi033.stderr:10+0 records out 2024-04-25T01:29:59.001 INFO:teuthology.orchestra.run.smithi033.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112319 s, 934 MB/s 2024-04-25T01:29:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:--> Zapping successful for: 2024-04-25T01:29:59.317 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:59 smithi033 ceph-mon[26353]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:59.317 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:59 smithi033 ceph-mon[26353]: osdmap e15: 2 total, 2 up, 2 in 2024-04-25T01:29:59.317 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:59 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:59.317 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:29:59 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:59.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:59 smithi138 ceph-mon[32977]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T01:29:59.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:59 smithi138 ceph-mon[32977]: osdmap e15: 2 total, 2 up, 2 in 2024-04-25T01:29:59.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:59 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:59.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:29:59 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:29:59.606 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi033:vg_nvme/lv_2 2024-04-25T01:30:00.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:00 smithi033 ceph-mon[26353]: overall HEALTH_OK 2024-04-25T01:30:00.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:00 smithi138 ceph-mon[32977]: overall HEALTH_OK 2024-04-25T01:30:00.794 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:30:01.295 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:01 smithi033 ceph-mon[26353]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:01.295 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:30:01.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:01 smithi138 ceph-mon[32977]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:01.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:30:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 -- 172.21.15.33:0/3611835408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fe430 msgr2=0x7f80a40fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 --2- 172.21.15.33:0/3611835408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fe430 0x7f80a40fe840 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f809c009ce0 tx=0x7f809c004d10 comp rx=0 tx=0).stop 2024-04-25T01:30:02.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 -- 172.21.15.33:0/3611835408 shutdown_connections 2024-04-25T01:30:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 --2- 172.21.15.33:0/3611835408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40fed80 0x7f80a40faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 --2- 172.21.15.33:0/3611835408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fe430 0x7f80a40fe840 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 -- 172.21.15.33:0/3611835408 >> 172.21.15.33:0/3611835408 conn(0x7f80a40f6bd0 msgr2=0x7f80a40f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 -- 172.21.15.33:0/3611835408 shutdown_connections 2024-04-25T01:30:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 -- 172.21.15.33:0/3611835408 wait complete. 2024-04-25T01:30:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 Processor -- start 2024-04-25T01:30:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.946+0000 7f80ab952700 1 -- start start 2024-04-25T01:30:02.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80ab952700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40fe430 0x7f80a40fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80ab952700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fed80 0x7f80a40fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80ab952700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40fe040 con 0x7f80a40fed80 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80ab952700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40fc630 con 0x7f80a40fe430 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a8eed700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fed80 0x7f80a40fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a8eed700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fed80 0x7f80a40fc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56986/0 (socket says 172.21.15.33:56986) 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a8eed700 1 -- 172.21.15.33:0/3529684361 learned_addr learned my addr 172.21.15.33:0/3529684361 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a96ee700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40fe430 0x7f80a40fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a8eed700 1 -- 172.21.15.33:0/3529684361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40fe430 msgr2=0x7f80a40fda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a8eed700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40fe430 0x7f80a40fda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:02.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a8eed700 1 -- 172.21.15.33:0/3529684361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c009940 con 0x7f80a40fed80 2024-04-25T01:30:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a8eed700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fed80 0x7f80a40fc0f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f809800c830 tx=0x7f8098004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a1ffb700 1 -- 172.21.15.33:0/3529684361 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80980106f0 con 0x7f80a40fed80 2024-04-25T01:30:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a1ffb700 1 -- 172.21.15.33:0/3529684361 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8098010d20 con 0x7f80a40fed80 2024-04-25T01:30:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80a1ffb700 1 -- 172.21.15.33:0/3529684361 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80980113a0 con 0x7f80a40fed80 2024-04-25T01:30:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.947+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a40fc8e0 con 0x7f80a40fed80 2024-04-25T01:30:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.948+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a40fcea0 con 0x7f80a40fed80 2024-04-25T01:30:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.949+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a404c710 con 0x7f80a40fed80 2024-04-25T01:30:02.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.949+0000 7f80a1ffb700 1 -- 172.21.15.33:0/3529684361 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7f8098012400 con 0x7f80a40fed80 2024-04-25T01:30:02.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.949+0000 7f80a1ffb700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f808c069210 0x7f808c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.949+0000 7f80a1ffb700 1 -- 172.21.15.33:0/3529684361 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f8098086360 con 0x7f80a40fed80 2024-04-25T01:30:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.949+0000 7f80a96ee700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f808c069210 0x7f808c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.950+0000 7f80a96ee700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f808c069210 0x7f808c06b6c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f809c015040 tx=0x7f809c004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:02.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:02.954+0000 7f80a1ffb700 1 -- 172.21.15.33:0/3529684361 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8098056e90 con 0x7f80a40fed80 2024-04-25T01:30:03.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:03.162+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f80a4102640 con 0x7f808c069210 2024-04-25T01:30:03.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:03 smithi138 ceph-mon[32977]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:03.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:30:03.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:30:03.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:03.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:03 smithi033 ceph-mon[26353]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:03.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:30:03.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:30:03.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:04.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:30:04.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:04.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:04.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:04.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:04.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='client.14274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi033:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:30:04.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:04.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:04.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:04.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:05.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:05 smithi033 ceph-mon[26353]: Detected new or changed devices on smithi033 2024-04-25T01:30:05.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:05 smithi033 ceph-mon[26353]: Adjusting osd_memory_target on smithi033 to 6434M 2024-04-25T01:30:05.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:05 smithi033 ceph-mon[26353]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:05.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:05 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi033 2024-04-25T01:30:05.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:05 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi033 to 6434M 2024-04-25T01:30:05.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:05 smithi138 ceph-mon[32977]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:06.468 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:06 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1530681990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1239da3-566e-4df0-b5ea-df4c14959837"}]: dispatch 2024-04-25T01:30:06.468 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:06 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1530681990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1239da3-566e-4df0-b5ea-df4c14959837"}]': finished 2024-04-25T01:30:06.468 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:06 smithi033 ceph-mon[26353]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T01:30:06.469 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:06 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:06.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:06 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1530681990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1239da3-566e-4df0-b5ea-df4c14959837"}]: dispatch 2024-04-25T01:30:06.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:06 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1530681990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1239da3-566e-4df0-b5ea-df4c14959837"}]': finished 2024-04-25T01:30:06.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:06 smithi138 ceph-mon[32977]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T01:30:06.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:06 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:07.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:07 smithi033 ceph-mon[26353]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:07.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:07 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/3655755844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:30:07.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:07 smithi138 ceph-mon[32977]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:07.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:07 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/3655755844' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:30:09.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:09 smithi033 ceph-mon[26353]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:09.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:09.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:09.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:09.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:09.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:09.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:09 smithi138 ceph-mon[32977]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:09.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:09.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:09.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:09.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:09.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:11.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:11 smithi033 ceph-mon[26353]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:11.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:11 smithi138 ceph-mon[32977]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:13.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:13 smithi033 ceph-mon[26353]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:13.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:13 smithi138 ceph-mon[32977]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:14.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:14 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:30:14.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:14 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:14.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:14 smithi033 ceph-mon[26353]: Deploying daemon osd.2 on smithi033 2024-04-25T01:30:14.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:14 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:30:14.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:14 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:14.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:14 smithi138 ceph-mon[32977]: Deploying daemon osd.2 on smithi033 2024-04-25T01:30:15.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:15 smithi033 ceph-mon[26353]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:15.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:15 smithi138 ceph-mon[32977]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:16.380 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:30:16.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:30:17.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:17 smithi033 ceph-mon[26353]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:17.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:17 smithi138 ceph-mon[32977]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:19.466 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:19 smithi033 ceph-mon[26353]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:19.466 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:19 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:19.466 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:19 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:19.466 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:19 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:19.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:19 smithi138 ceph-mon[32977]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:19.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:19 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:19.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:19 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:19.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:19 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:21.473 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:21 smithi033 ceph-mon[26353]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:21.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:21 smithi138 ceph-mon[32977]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:22.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.925+0000 7f80a1ffb700 1 -- 172.21.15.33:0/3529684361 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f80a4102640 con 0x7f808c069210 2024-04-25T01:30:22.904 INFO:teuthology.orchestra.run.smithi033.stdout:Created osd(s) 2 on host 'smithi033' 2024-04-25T01:30:22.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f808c069210 msgr2=0x7f808c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:22.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f808c069210 0x7f808c06b6c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f809c015040 tx=0x7f809c004d10 comp rx=0 tx=0).stop 2024-04-25T01:30:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fed80 msgr2=0x7f80a40fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fed80 0x7f80a40fc0f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f809800c830 tx=0x7f8098004470 comp rx=0 tx=0).stop 2024-04-25T01:30:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 shutdown_connections 2024-04-25T01:30:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80a40fe430 0x7f80a40fda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f808c069210 0x7f808c06b6c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 --2- 172.21.15.33:0/3529684361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80a40fed80 0x7f80a40fc0f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 >> 172.21.15.33:0/3529684361 conn(0x7f80a40f6bd0 msgr2=0x7f80a40f82d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 shutdown_connections 2024-04-25T01:30:22.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:30:22.928+0000 7f80ab952700 1 -- 172.21.15.33:0/3529684361 wait complete. 2024-04-25T01:30:23.466 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:23 smithi033 ceph-mon[26353]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:23.467 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:23 smithi033 ceph-mon[26353]: from='osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:30:23.467 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:23 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:23.467 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:23 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:23.497 DEBUG:teuthology.orchestra.run.smithi033:osd.2> sudo journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T01:30:23.499 INFO:tasks.cephadm:Deploying osd.3 on smithi138 with /dev/vg_nvme/lv_4... 2024-04-25T01:30:23.500 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T01:30:23.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:23 smithi138 ceph-mon[32977]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:23.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:23 smithi138 ceph-mon[32977]: from='osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:30:23.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:23 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:23.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:23 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:23.758 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T01:30:24.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:24 smithi033 ceph-mon[26353]: from='osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:30:24.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:24 smithi033 ceph-mon[26353]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T01:30:24.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:24 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:24.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:24 smithi033 ceph-mon[26353]: from='osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:30:24.495 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:24 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[60541]: 2024-04-25T01:30:24.263+0000 7efe5027e700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T01:30:24.495 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:24 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[60541]: 2024-04-25T01:30:24.268+0000 7efe49075700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:30:24.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:24 smithi138 ceph-mon[32977]: from='osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:30:24.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:24 smithi138 ceph-mon[32977]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T01:30:24.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:24 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:24.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:24 smithi138 ceph-mon[32977]: from='osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:30:25.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:25 smithi138 ceph-mon[32977]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:25.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:25 smithi138 ceph-mon[32977]: from='osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-25T01:30:25.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:25 smithi138 ceph-mon[32977]: osdmap e18: 3 total, 2 up, 3 in 2024-04-25T01:30:25.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:25 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:25.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:25 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:25.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:25 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:25.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:25 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:25.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:25 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:25.707 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T01:30:25.707 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T01:30:25.707 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T01:30:25.707 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T01:30:25.707 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111481 s, 941 MB/s 2024-04-25T01:30:25.708 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T01:30:25.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:25 smithi033 ceph-mon[26353]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:25.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:25 smithi033 ceph-mon[26353]: from='osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]': finished 2024-04-25T01:30:25.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:25 smithi033 ceph-mon[26353]: osdmap e18: 3 total, 2 up, 3 in 2024-04-25T01:30:25.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:25 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:25.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:25 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:25.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:25 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:25.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:25 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:25.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:25 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:26.257 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_4 2024-04-25T01:30:26.553 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T01:30:26.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:26 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T01:30:26.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:26 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T01:30:26.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:26 smithi138 ceph-mon[32977]: osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] boot 2024-04-25T01:30:26.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:26 smithi138 ceph-mon[32977]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T01:30:26.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:26.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:26 smithi033 ceph-mon[26353]: purged_snaps scrub starts 2024-04-25T01:30:26.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:26 smithi033 ceph-mon[26353]: purged_snaps scrub ok 2024-04-25T01:30:26.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:26 smithi033 ceph-mon[26353]: osd.2 [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] boot 2024-04-25T01:30:26.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:26 smithi033 ceph-mon[26353]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T01:30:26.853 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:30:27.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:27 smithi138 ceph-mon[32977]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:27.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:27 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-25T01:30:27.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:27 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-25T01:30:27.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:27 smithi138 ceph-mon[32977]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T01:30:27.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:27 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:30:27.872 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:27 smithi033 ceph-mon[26353]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T01:30:27.872 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:27 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-25T01:30:27.872 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:27 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-25T01:30:27.872 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:27 smithi033 ceph-mon[26353]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T01:30:27.872 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:27 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T01:30:28.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.066+0000 7ff76b123700 1 -- 172.21.15.138:0/758073396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 msgr2=0x7ff7640fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:28.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.066+0000 7ff76b123700 1 --2- 172.21.15.138:0/758073396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 0x7ff7640fd3e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff750004940 tx=0x7ff750009e30 comp rx=0 tx=0).stop 2024-04-25T01:30:28.044 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.066+0000 7ff76b123700 1 -- 172.21.15.138:0/758073396 shutdown_connections 2024-04-25T01:30:28.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.066+0000 7ff76b123700 1 --2- 172.21.15.138:0/758073396 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 0x7ff7640fd3e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.066+0000 7ff76b123700 1 --2- 172.21.15.138:0/758073396 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640fbcc0 0x7ff7640fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.066+0000 7ff76b123700 1 -- 172.21.15.138:0/758073396 >> 172.21.15.138:0/758073396 conn(0x7ff7640f7440 msgr2=0x7ff7640f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:28.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.067+0000 7ff76b123700 1 -- 172.21.15.138:0/758073396 shutdown_connections 2024-04-25T01:30:28.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.067+0000 7ff76b123700 1 -- 172.21.15.138:0/758073396 wait complete. 2024-04-25T01:30:28.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.067+0000 7ff76b123700 1 Processor -- start 2024-04-25T01:30:28.045 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.067+0000 7ff76b123700 1 -- start start 2024-04-25T01:30:28.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.067+0000 7ff76b123700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640fbcc0 0x7ff76410d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:28.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.067+0000 7ff76b123700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 0x7ff76410de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:28.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.067+0000 7ff76b123700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76410e430 con 0x7ff7640fbcc0 2024-04-25T01:30:28.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.067+0000 7ff76b123700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76410e5a0 con 0x7ff7640fcf70 2024-04-25T01:30:28.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.067+0000 7ff763fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 0x7ff76410de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:28.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff763fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 0x7ff76410de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:46830/0 (socket says 172.21.15.138:46830) 2024-04-25T01:30:28.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff763fff700 1 -- 172.21.15.138:0/784362064 learned_addr learned my addr 172.21.15.138:0/784362064 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:30:28.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff768ebf700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640fbcc0 0x7ff76410d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:28.047 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff763fff700 1 -- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640fbcc0 msgr2=0x7ff76410d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff763fff700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640fbcc0 0x7ff76410d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff763fff700 1 -- 172.21.15.138:0/784362064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff750005040 con 0x7ff7640fcf70 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff768ebf700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640fbcc0 0x7ff76410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff763fff700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 0x7ff76410de30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff750013870 tx=0x7ff750009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff7617fa700 1 -- 172.21.15.138:0/784362064 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff75000e070 con 0x7ff7640fcf70 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff7617fa700 1 -- 172.21.15.138:0/784362064 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff750013de0 con 0x7ff7640fcf70 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff764112f50 con 0x7ff7640fcf70 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff7617fa700 1 -- 172.21.15.138:0/784362064 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff75001cea0 con 0x7ff7640fcf70 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.068+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff764113400 con 0x7ff7640fcf70 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.069+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff764101b00 con 0x7ff7640fcf70 2024-04-25T01:30:28.048 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.070+0000 7ff7617fa700 1 -- 172.21.15.138:0/784362064 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 18) v1 ==== 86573+0+0 (secure 0 0 0) 0x7ff75001c690 con 0x7ff7640fcf70 2024-04-25T01:30:28.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.070+0000 7ff7617fa700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff754069210 0x7ff75406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:28.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.070+0000 7ff7617fa700 1 -- 172.21.15.138:0/784362064 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3228+0+0 (secure 0 0 0) 0x7ff750087e90 con 0x7ff7640fcf70 2024-04-25T01:30:28.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.070+0000 7ff768ebf700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff754069210 0x7ff75406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:28.049 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.071+0000 7ff768ebf700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff754069210 0x7ff75406b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff758004fd0 tx=0x7ff75800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:28.052 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.075+0000 7ff7617fa700 1 -- 172.21.15.138:0/784362064 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff7500585f0 con 0x7ff7640fcf70 2024-04-25T01:30:28.244 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:28.267+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff76402b3e0 con 0x7ff754069210 2024-04-25T01:30:28.871 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:30:28.871 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:28 smithi138 ceph-mon[32977]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T01:30:28.871 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:28 smithi138 ceph-mon[32977]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:28.871 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:28 smithi138 ceph-mon[32977]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:30:28.871 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:30:28.871 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:30:28.871 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:28.888 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T01:30:28.888 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:28 smithi033 ceph-mon[26353]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T01:30:28.888 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:28 smithi033 ceph-mon[26353]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:28.888 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:28 smithi033 ceph-mon[26353]: from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:30:28.888 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:30:28.888 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:30:28.888 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:29.244 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:30:28 smithi033 sudo[65119]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:30:29.244 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:30:28 smithi033 sudo[65119]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:29.244 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:30:28 smithi033 sudo[65119]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:29.244 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:30:28 smithi033 sudo[65119]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:29.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T01:30:29.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi033 2024-04-25T01:30:29.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:29.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:29.711 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi033 to 4289M 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.712 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:29 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.743 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:30:29 smithi033 sudo[65122]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:30:29.744 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:30:29 smithi033 sudo[65122]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:29.744 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:30:29 smithi033 sudo[65122]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:29.744 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:30:29 smithi033 sudo[65122]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:29.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T01:30:29.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: Detected new or changed devices on smithi033 2024-04-25T01:30:29.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:29.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: Adjusting osd_memory_target on smithi033 to 4289M 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:29.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:29 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:30.244 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:30:29 smithi033 sudo[65157]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:30:30.244 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:30:29 smithi033 sudo[65157]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:30.244 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:30:29 smithi033 sudo[65157]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:30.244 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:30:29 smithi033 sudo[65157]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:30.559 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:30:30 smithi033 sudo[65160]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:30:30.560 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:30:30 smithi033 sudo[65160]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:30.560 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:30:30 smithi033 sudo[65160]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:30.560 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:30:30 smithi033 sudo[65160]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:30.563 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:30.584+0000 7ff7617fa700 1 -- 172.21.15.138:0/784362064 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff750054f30 con 0x7ff7640fcf70 2024-04-25T01:30:30.954 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:30 smithi138 ceph-mon[32977]: pgmap v77: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:30.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:30 smithi033 ceph-mon[26353]: pgmap v77: 1 pgs: 1 creating+activating; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:30.994 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:30 smithi033 sudo[65163]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:30:30.994 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:30 smithi033 sudo[65163]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:30.994 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:30 smithi033 sudo[65163]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:30.994 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:30 smithi033 sudo[65163]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:31.494 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:31 smithi033 sudo[65166]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:30:31.494 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:31 smithi033 sudo[65166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:31.494 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:31 smithi033 sudo[65166]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:31.494 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:30:31 smithi033 sudo[65166]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:31.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:31 smithi033 sudo[65169]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:30:31.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:31 smithi033 sudo[65169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:31.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:31 smithi033 sudo[65169]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:31.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:31 smithi033 sudo[65169]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:32.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 sudo[65172]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:30:32.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 sudo[65172]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:32.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 sudo[65172]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:32.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 sudo[65172]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 ceph-mon[26353]: mgrmap e19: smithi033.asemba(active, since 2m), standbys: smithi138.mmfhyi 2024-04-25T01:30:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/3751603840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abc58749-77ca-4476-9601-cdf89d3c1e78"}]: dispatch 2024-04-25T01:30:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 ceph-mon[26353]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abc58749-77ca-4476-9601-cdf89d3c1e78"}]: dispatch 2024-04-25T01:30:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 ceph-mon[26353]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abc58749-77ca-4476-9601-cdf89d3c1e78"}]': finished 2024-04-25T01:30:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 ceph-mon[26353]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T01:30:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:32.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:32 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:30:32.572 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 sudo[36682]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T01:30:32.572 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 sudo[36682]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:32.572 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 sudo[36682]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:32.572 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 sudo[36682]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:33.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 sudo[36869]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T01:30:33.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 sudo[36869]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T01:30:33.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 sudo[36869]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T01:30:33.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 sudo[36869]: pam_unix(sudo:session): session closed for user root 2024-04-25T01:30:33.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 ceph-mon[32977]: mgrmap e19: smithi033.asemba(active, since 2m), standbys: smithi138.mmfhyi 2024-04-25T01:30:33.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/3751603840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abc58749-77ca-4476-9601-cdf89d3c1e78"}]: dispatch 2024-04-25T01:30:33.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "abc58749-77ca-4476-9601-cdf89d3c1e78"}]: dispatch 2024-04-25T01:30:33.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "abc58749-77ca-4476-9601-cdf89d3c1e78"}]': finished 2024-04-25T01:30:33.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 ceph-mon[32977]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T01:30:33.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:33.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:32 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:30:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:30:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/3690104520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:30:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:30:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:30:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:30:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:30:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:30:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:30:33.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:30:33.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:33.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:30:33.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/3690104520' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:30:33.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:33.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:30:33.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:30:33.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:30:33.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T01:30:33.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:30:33.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:30:33.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T01:30:33.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:33 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:35.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:35 smithi033 ceph-mon[26353]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:35.496 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:35 smithi138 ceph-mon[32977]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:37.338 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:37 smithi138 ceph-mon[32977]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:37.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:37 smithi033 ceph-mon[26353]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:38.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:38 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:30:38.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:38 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:38.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:38 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:30:38.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:38 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:39.195 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:39 smithi138 ceph-mon[32977]: Deploying daemon osd.3 on smithi138 2024-04-25T01:30:39.195 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:39 smithi138 ceph-mon[32977]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:39.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:39 smithi033 ceph-mon[26353]: Deploying daemon osd.3 on smithi138 2024-04-25T01:30:39.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:39 smithi033 ceph-mon[26353]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:41.353 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:41 smithi138 ceph-mon[32977]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:41.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:41 smithi033 ceph-mon[26353]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:43.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:43 smithi033 ceph-mon[26353]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:43.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:43 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:43.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:43 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:43.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:43 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:43.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:43 smithi138 ceph-mon[32977]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:43.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:43 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:43.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:43 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:43.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:43 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:45.152 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.174+0000 7ff7617fa700 1 -- 172.21.15.138:0/784362064 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff76402b3e0 con 0x7ff754069210 2024-04-25T01:30:45.152 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 3 on host 'smithi138' 2024-04-25T01:30:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff754069210 msgr2=0x7ff75406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:45.153 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff754069210 0x7ff75406b6c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff758004fd0 tx=0x7ff75800b040 comp rx=0 tx=0).stop 2024-04-25T01:30:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 msgr2=0x7ff76410de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 0x7ff76410de30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff750013870 tx=0x7ff750009e30 comp rx=0 tx=0).stop 2024-04-25T01:30:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 shutdown_connections 2024-04-25T01:30:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff754069210 0x7ff75406b6c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff7640fbcc0 0x7ff76410d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 --2- 172.21.15.138:0/784362064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff7640fcf70 0x7ff76410de30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 >> 172.21.15.138:0/784362064 conn(0x7ff7640f7440 msgr2=0x7ff764066790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 shutdown_connections 2024-04-25T01:30:45.154 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:45.176+0000 7ff76b123700 1 -- 172.21.15.138:0/784362064 wait complete. 2024-04-25T01:30:45.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:45 smithi138 ceph-mon[32977]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:45.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:45 smithi033 ceph-mon[26353]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:46.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:46.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:46.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:30:46.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:46.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:46.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:30:46.627 DEBUG:teuthology.orchestra.run.smithi138:osd.3> sudo journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T01:30:46.629 INFO:tasks.cephadm:Deploying osd.4 on smithi138 with /dev/vg_nvme/lv_3... 2024-04-25T01:30:46.629 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T01:30:46.933 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T01:30:47.362 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:47 smithi138 ceph-mon[32977]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:47.362 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:47 smithi138 ceph-mon[32977]: from='osd.3 [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:30:47.363 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:47 smithi138 ceph-mon[32977]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:30:47.363 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:47 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:47.363 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:47 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:47.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:47 smithi033 ceph-mon[26353]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:47.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:47 smithi033 ceph-mon[26353]: from='osd.3 [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:30:47.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:47 smithi033 ceph-mon[26353]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:30:47.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:47 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:47.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:47 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:48.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:48 smithi138 ceph-mon[32977]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:30:48.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:48 smithi138 ceph-mon[32977]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T01:30:48.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:48 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:48.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:48 smithi138 ceph-mon[32977]: from='osd.3 [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:30:48.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:48 smithi138 ceph-mon[32977]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:30:48.270 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:30:48 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[38469]: 2024-04-25T01:30:48.192+0000 7f476c5e8700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T01:30:48.270 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:30:48 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[38469]: 2024-04-25T01:30:48.198+0000 7f47663e1700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:30:48.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:48 smithi033 ceph-mon[26353]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:30:48.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:48 smithi033 ceph-mon[26353]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T01:30:48.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:48 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:48.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:48 smithi033 ceph-mon[26353]: from='osd.3 [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:30:48.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:48 smithi033 ceph-mon[26353]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:30:49.359 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:49 smithi138 ceph-mon[32977]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:49.360 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:49 smithi138 ceph-mon[32977]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T01:30:49.360 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:49 smithi138 ceph-mon[32977]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T01:30:49.360 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:49 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:49.360 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:49 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:49.360 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:49 smithi138 ceph-mon[32977]: osd.3 [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] boot 2024-04-25T01:30:49.360 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:49 smithi138 ceph-mon[32977]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T01:30:49.360 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:49 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:49 smithi033 ceph-mon[26353]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T01:30:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:49 smithi033 ceph-mon[26353]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T01:30:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:49 smithi033 ceph-mon[26353]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T01:30:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:49 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:49 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:49 smithi033 ceph-mon[26353]: osd.3 [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] boot 2024-04-25T01:30:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:49 smithi033 ceph-mon[26353]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T01:30:49.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:49 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:30:49.818 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T01:30:49.818 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T01:30:49.818 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T01:30:49.818 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T01:30:49.818 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.029105 s, 360 MB/s 2024-04-25T01:30:49.819 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T01:30:50.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:50 smithi033 ceph-mon[26353]: purged_snaps scrub starts 2024-04-25T01:30:50.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:50 smithi033 ceph-mon[26353]: purged_snaps scrub ok 2024-04-25T01:30:50.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:50 smithi033 ceph-mon[26353]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T01:30:50.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:50 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T01:30:50.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:50 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T01:30:50.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:50 smithi138 ceph-mon[32977]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T01:30:50.618 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_3 2024-04-25T01:30:51.231 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T01:30:51.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:51 smithi033 ceph-mon[26353]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:30:51.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:51 smithi033 ceph-mon[26353]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T01:30:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:51 smithi138 ceph-mon[32977]: pgmap v91: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:30:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:51 smithi138 ceph-mon[32977]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T01:30:52.506 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi138 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 15936M 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:52 smithi138 ceph-mon[32977]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 -- 172.21.15.138:0/2913951896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fbcc0 msgr2=0x7f12900fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 --2- 172.21.15.138:0/2913951896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fbcc0 0x7f12900fc0d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1284004940 tx=0x7f1284009e30 comp rx=0 tx=0).stop 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 -- 172.21.15.138:0/2913951896 shutdown_connections 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 --2- 172.21.15.138:0/2913951896 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12900fcf70 0x7f12900fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 --2- 172.21.15.138:0/2913951896 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fbcc0 0x7f12900fc0d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 -- 172.21.15.138:0/2913951896 >> 172.21.15.138:0/2913951896 conn(0x7f12900f7420 msgr2=0x7f12900f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 -- 172.21.15.138:0/2913951896 shutdown_connections 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 -- 172.21.15.138:0/2913951896 wait complete. 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 Processor -- start 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.756+0000 7f129526c700 1 -- start start 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f129526c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12900fbcc0 0x7f1290110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:52.736 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f129526c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fcf70 0x7f1290112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f129526c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290066c50 con 0x7f12900fbcc0 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f129526c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1290066d90 con 0x7f12900fcf70 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f128e7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fcf70 0x7f1290112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f128e7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fcf70 0x7f1290112c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33834/0 (socket says 172.21.15.138:33834) 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f128e7fc700 1 -- 172.21.15.138:0/1416122550 learned_addr learned my addr 172.21.15.138:0/1416122550 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f128effd700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12900fbcc0 0x7f1290110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f128e7fc700 1 -- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12900fbcc0 msgr2=0x7f1290110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f128e7fc700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12900fbcc0 0x7f1290110730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f128e7fc700 1 -- 172.21.15.138:0/1416122550 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1284005040 con 0x7f12900fcf70 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f128e7fc700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fcf70 0x7f1290112c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f128000b790 tx=0x7f1280009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f12777fe700 1 -- 172.21.15.138:0/1416122550 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f128000f6f0 con 0x7f12900fcf70 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f12777fe700 1 -- 172.21.15.138:0/1416122550 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f128000fd20 con 0x7f12900fcf70 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12901132e0 con 0x7f12900fcf70 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.757+0000 7f12777fe700 1 -- 172.21.15.138:0/1416122550 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12800103a0 con 0x7f12900fcf70 2024-04-25T01:30:52.737 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.758+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1290113870 con 0x7f12900fcf70 2024-04-25T01:30:52.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.758+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129005f5c0 con 0x7f12900fcf70 2024-04-25T01:30:52.738 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.759+0000 7f12777fe700 1 -- 172.21.15.138:0/1416122550 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1280011400 con 0x7f12900fcf70 2024-04-25T01:30:52.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.759+0000 7f12777fe700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f12780692a0 0x7f127806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:30:52.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.760+0000 7f12777fe700 1 -- 172.21.15.138:0/1416122550 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f12800868e0 con 0x7f12900fcf70 2024-04-25T01:30:52.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.760+0000 7f128effd700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f12780692a0 0x7f127806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:30:52.739 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.760+0000 7f128effd700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f12780692a0 0x7f127806b750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1284004890 tx=0x7f1284009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:30:52.742 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.764+0000 7f12777fe700 1 -- 172.21.15.138:0/1416122550 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f12800532b0 con 0x7f12900fcf70 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: Detected new or changed devices on smithi138 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: Adjusting osd_memory_target on smithi138 to 15936M 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:30:52.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:52.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:30:52.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:30:52.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:52 smithi033 ceph-mon[26353]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T01:30:52.946 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:30:52.967+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1290101290 con 0x7f12780692a0 2024-04-25T01:30:53.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:53 smithi138 ceph-mon[32977]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:30:53.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:53 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:30:53.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:53 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:30:53.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:53 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:53.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:53 smithi138 ceph-mon[32977]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T01:30:53.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:53 smithi033 ceph-mon[26353]: pgmap v94: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:30:53.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:53 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:30:53.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:53 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:30:53.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:53 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:30:53.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:53 smithi033 ceph-mon[26353]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T01:30:54.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:54 smithi138 ceph-mon[32977]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:30:54.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:54 smithi033 ceph-mon[26353]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:30:55.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:55 smithi033 ceph-mon[26353]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:30:55.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:55 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/600572673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e421bd-52ba-47ad-8755-e1354f502ec5"}]: dispatch 2024-04-25T01:30:55.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:55 smithi033 ceph-mon[26353]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e421bd-52ba-47ad-8755-e1354f502ec5"}]: dispatch 2024-04-25T01:30:55.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:55 smithi033 ceph-mon[26353]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83e421bd-52ba-47ad-8755-e1354f502ec5"}]': finished 2024-04-25T01:30:55.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:55 smithi033 ceph-mon[26353]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T01:30:55.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:55 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:30:55.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:55 smithi138 ceph-mon[32977]: pgmap v97: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T01:30:55.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:55 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/600572673' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e421bd-52ba-47ad-8755-e1354f502ec5"}]: dispatch 2024-04-25T01:30:55.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:55 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "83e421bd-52ba-47ad-8755-e1354f502ec5"}]: dispatch 2024-04-25T01:30:55.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:55 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83e421bd-52ba-47ad-8755-e1354f502ec5"}]': finished 2024-04-25T01:30:55.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:55 smithi138 ceph-mon[32977]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T01:30:55.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:55 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:30:56.645 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:56 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/1495643531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:30:56.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:56 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/1495643531' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:30:57.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:57 smithi033 ceph-mon[26353]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T01:30:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:57 smithi138 ceph-mon[32977]: pgmap v99: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-04-25T01:30:59.380 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:30:59 smithi138 ceph-mon[32977]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T01:30:59.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:30:59 smithi033 ceph-mon[26353]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T01:31:01.692 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:01 smithi138 ceph-mon[32977]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T01:31:01.692 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:31:01.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:01 smithi033 ceph-mon[26353]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-04-25T01:31:01.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:31:02.685 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:02 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:31:02.685 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:02 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:02.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:02 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:31:02.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:02 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:03.669 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:03 smithi138 ceph-mon[32977]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:03.669 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:03 smithi138 ceph-mon[32977]: Deploying daemon osd.4 on smithi138 2024-04-25T01:31:03.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:03 smithi033 ceph-mon[26353]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:03.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:03 smithi033 ceph-mon[26353]: Deploying daemon osd.4 on smithi138 2024-04-25T01:31:05.629 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:05 smithi138 ceph-mon[32977]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:05.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:05 smithi033 ceph-mon[26353]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:07.604 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:07 smithi138 ceph-mon[32977]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:07.604 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:07.605 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:07.605 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:31:07.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:07 smithi033 ceph-mon[26353]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:07.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:07.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:07.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:31:09.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:09 smithi138 ceph-mon[32977]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:09.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:09 smithi033 ceph-mon[26353]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:10.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.684+0000 7f12777fe700 1 -- 172.21.15.138:0/1416122550 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1290101290 con 0x7f12780692a0 2024-04-25T01:31:10.662 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 4 on host 'smithi138' 2024-04-25T01:31:10.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f12780692a0 msgr2=0x7f127806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:10.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f12780692a0 0x7f127806b750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1284004890 tx=0x7f1284009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:10.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fcf70 msgr2=0x7f1290112c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:10.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fcf70 0x7f1290112c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f128000b790 tx=0x7f1280009250 comp rx=0 tx=0).stop 2024-04-25T01:31:10.664 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 shutdown_connections 2024-04-25T01:31:10.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f12780692a0 0x7f127806b750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:10.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f12900fbcc0 0x7f1290110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:10.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 --2- 172.21.15.138:0/1416122550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12900fcf70 0x7f1290112c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:10.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 >> 172.21.15.138:0/1416122550 conn(0x7f12900f7420 msgr2=0x7f12900f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:10.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 shutdown_connections 2024-04-25T01:31:10.665 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:10.686+0000 7f129526c700 1 -- 172.21.15.138:0/1416122550 wait complete. 2024-04-25T01:31:11.561 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:11 smithi138 ceph-mon[32977]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:11.561 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:11 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:11.561 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:11 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:11.561 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:11 smithi138 ceph-mon[32977]: from='osd.4 [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:31:11.561 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:11 smithi138 ceph-mon[32977]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:31:11.563 DEBUG:teuthology.orchestra.run.smithi138:osd.4> sudo journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T01:31:11.565 INFO:tasks.cephadm:Deploying osd.5 on smithi138 with /dev/vg_nvme/lv_2... 2024-04-25T01:31:11.565 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T01:31:11.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:11 smithi033 ceph-mon[26353]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:11.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:11 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:11.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:11 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:11.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:11 smithi033 ceph-mon[26353]: from='osd.4 [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:31:11.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:11 smithi033 ceph-mon[26353]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:31:11.873 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T01:31:12.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:12 smithi033 ceph-mon[26353]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:31:12.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:12 smithi033 ceph-mon[26353]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T01:31:12.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:12 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:31:12.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:12 smithi033 ceph-mon[26353]: from='osd.4 [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:31:12.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:12 smithi033 ceph-mon[26353]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:31:12.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:12 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:12.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:12 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:12.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:12 smithi033 ceph-mon[26353]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:13.002 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[45353]: 2024-04-25T01:31:12.693+0000 7fed7aaa5700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T01:31:13.002 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[45353]: 2024-04-25T01:31:12.699+0000 7fed7389c700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:31:13.003 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-mon[32977]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:31:13.003 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-mon[32977]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T01:31:13.003 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:31:13.003 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-mon[32977]: from='osd.4 [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:31:13.003 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-mon[32977]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:31:13.003 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:13.003 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:13.003 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:12 smithi138 ceph-mon[32977]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T01:31:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T01:31:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T01:31:13.686 INFO:teuthology.orchestra.run.smithi138.stderr: stderr: 10+0 records in 2024-04-25T01:31:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:10+0 records out 2024-04-25T01:31:13.686 INFO:teuthology.orchestra.run.smithi138.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0130399 s, 804 MB/s 2024-04-25T01:31:13.687 INFO:teuthology.orchestra.run.smithi138.stderr:--> Zapping successful for: 2024-04-25T01:31:13.971 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:13 smithi138 ceph-mon[32977]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T01:31:13.971 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:13 smithi138 ceph-mon[32977]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T01:31:13.971 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:13 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:31:13.971 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:13 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:31:13.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:13 smithi033 ceph-mon[26353]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T01:31:13.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:13 smithi033 ceph-mon[26353]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T01:31:13.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:13 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:31:13.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:13 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:31:14.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:14 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T01:31:14.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:14 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T01:31:14.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:14 smithi138 ceph-mon[32977]: osd.4 [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] boot 2024-04-25T01:31:14.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:14 smithi138 ceph-mon[32977]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T01:31:14.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:14 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:31:14.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:14 smithi138 ceph-mon[32977]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:14.978 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi138:vg_nvme/lv_2 2024-04-25T01:31:14.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:14 smithi033 ceph-mon[26353]: purged_snaps scrub starts 2024-04-25T01:31:14.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:14 smithi033 ceph-mon[26353]: purged_snaps scrub ok 2024-04-25T01:31:14.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:14 smithi033 ceph-mon[26353]: osd.4 [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] boot 2024-04-25T01:31:14.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:14 smithi033 ceph-mon[26353]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T01:31:14.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:14 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:31:14.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:14 smithi033 ceph-mon[26353]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:15.444 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T01:31:15.843 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:15 smithi138 ceph-mon[32977]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T01:31:15.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:15 smithi033 ceph-mon[26353]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T01:31:16.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:16 smithi033 ceph-mon[26353]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:16.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:31:17.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:16 smithi138 ceph-mon[32977]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:17.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.637+0000 7f63c3014700 1 -- 172.21.15.138:0/2854822387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 msgr2=0x7f63bc0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.637+0000 7f63c3014700 1 --2- 172.21.15.138:0/2854822387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 0x7f63bc0fc0c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63b0004940 tx=0x7f63b0009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.637+0000 7f63c3014700 1 -- 172.21.15.138:0/2854822387 shutdown_connections 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.637+0000 7f63c3014700 1 --2- 172.21.15.138:0/2854822387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc0fcf60 0x7f63bc0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.637+0000 7f63c3014700 1 --2- 172.21.15.138:0/2854822387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 0x7f63bc0fc0c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.637+0000 7f63c3014700 1 -- 172.21.15.138:0/2854822387 >> 172.21.15.138:0/2854822387 conn(0x7f63bc0f7430 msgr2=0x7f63bc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.637+0000 7f63c3014700 1 -- 172.21.15.138:0/2854822387 shutdown_connections 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.637+0000 7f63c3014700 1 -- 172.21.15.138:0/2854822387 wait complete. 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.637+0000 7f63c3014700 1 Processor -- start 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c3014700 1 -- start start 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c3014700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 0x7f63bc10fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:17.616 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c3014700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc0fcf60 0x7f63bc110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c3014700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc1107b0 con 0x7f63bc0fcf60 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c3014700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63bc110fa0 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c0db0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 0x7f63bc10fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c0db0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 0x7f63bc10fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:33846/0 (socket says 172.21.15.138:33846) 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c0db0700 1 -- 172.21.15.138:0/2586964045 learned_addr learned my addr 172.21.15.138:0/2586964045 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c0db0700 1 -- 172.21.15.138:0/2586964045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc0fcf60 msgr2=0x7f63bc110120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c0db0700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc0fcf60 0x7f63bc110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c0db0700 1 -- 172.21.15.138:0/2586964045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b0005040 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c0db0700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 0x7f63bc10fbe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f63b000b3b0 tx=0x7f63b0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63b97fa700 1 -- 172.21.15.138:0/2586964045 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b000e040 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63b97fa700 1 -- 172.21.15.138:0/2586964045 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63b000bcc0 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63b97fa700 1 -- 172.21.15.138:0/2586964045 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63b001c630 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.638+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63bc111220 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.617 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.639+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63bc1117e0 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.640+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63bc1018f0 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.640+0000 7f63b97fa700 1 -- 172.21.15.138:0/2586964045 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f63b0013980 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.618 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.640+0000 7f63b97fa700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f63ac0692a0 0x7f63ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:17.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.640+0000 7f63b97fa700 1 -- 172.21.15.138:0/2586964045 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f63b0087ad0 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.641+0000 7f63bbfff700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f63ac0692a0 0x7f63ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:17.619 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.641+0000 7f63bbfff700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f63ac0692a0 0x7f63ac06b750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006fd0 tx=0x7f63a8006db0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:17.622 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.645+0000 7f63b97fa700 1 -- 172.21.15.138:0/2586964045 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f63b0057d00 con 0x7f63bc0fbcb0 2024-04-25T01:31:17.821 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:17.844+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f63bc111a00 con 0x7f63ac0692a0 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: Detected new or changed devices on smithi138 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: Adjusting osd_memory_target on smithi138 to 7968M 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:31:17.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:17.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:31:17.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:17.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:31:17.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:17 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi138 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 7968M 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:18.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:31:18.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:18.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:31:18.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:17 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:18.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:18 smithi033 ceph-mon[26353]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:31:18.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:31:18.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:31:18.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:18.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:18 smithi033 ceph-mon[26353]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:19.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:18 smithi138 ceph-mon[32977]: from='client.24175 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi138:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:31:19.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T01:31:19.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T01:31:19.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:19.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:18 smithi138 ceph-mon[32977]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:21.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:21 smithi033 ceph-mon[26353]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:21.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:21 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/1597855134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5984-f087-408e-a84c-04f81f33af43"}]: dispatch 2024-04-25T01:31:21.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:21 smithi033 ceph-mon[26353]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5984-f087-408e-a84c-04f81f33af43"}]: dispatch 2024-04-25T01:31:21.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:21 smithi033 ceph-mon[26353]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "266e5984-f087-408e-a84c-04f81f33af43"}]': finished 2024-04-25T01:31:21.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:21 smithi033 ceph-mon[26353]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T01:31:21.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:21 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:21.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:21 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/2586743856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:31:21.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:21 smithi138 ceph-mon[32977]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:21.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:21 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/1597855134' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5984-f087-408e-a84c-04f81f33af43"}]: dispatch 2024-04-25T01:31:21.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:21 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "266e5984-f087-408e-a84c-04f81f33af43"}]: dispatch 2024-04-25T01:31:21.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:21 smithi138 ceph-mon[32977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "266e5984-f087-408e-a84c-04f81f33af43"}]': finished 2024-04-25T01:31:21.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:21 smithi138 ceph-mon[32977]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T01:31:21.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:21 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:21.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:21 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/2586743856' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T01:31:23.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:23 smithi033 ceph-mon[26353]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:23.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:23 smithi138 ceph-mon[32977]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:25.426 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:25 smithi138 ceph-mon[32977]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:25.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:25 smithi033 ceph-mon[26353]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:27.361 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:27 smithi138 ceph-mon[32977]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:27.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:27 smithi033 ceph-mon[26353]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:28.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:31:28.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:28.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:31:28.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:29.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:29 smithi138 ceph-mon[32977]: Deploying daemon osd.5 on smithi138 2024-04-25T01:31:29.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:29 smithi138 ceph-mon[32977]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:29.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:29 smithi033 ceph-mon[26353]: Deploying daemon osd.5 on smithi138 2024-04-25T01:31:29.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:29 smithi033 ceph-mon[26353]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:31.258 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:31 smithi138 ceph-mon[32977]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:31.259 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:31 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:31:31.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:31 smithi033 ceph-mon[26353]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:31.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:31:33.309 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:33 smithi138 ceph-mon[32977]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:33.309 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:33 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:33.309 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:33 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:31:33.309 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:33 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:33 smithi033 ceph-mon[26353]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:31:33.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:33 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:35.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:35 smithi033 ceph-mon[26353]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:35.501 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:35 smithi138 ceph-mon[32977]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:36.589 INFO:teuthology.orchestra.run.smithi138.stdout:Created osd(s) 5 on host 'smithi138' 2024-04-25T01:31:36.589 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.611+0000 7f63b97fa700 1 -- 172.21.15.138:0/2586964045 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f63bc111a00 con 0x7f63ac0692a0 2024-04-25T01:31:36.591 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.613+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f63ac0692a0 msgr2=0x7f63ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.613+0000 7f63c3014700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f63ac0692a0 0x7f63ac06b750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006fd0 tx=0x7f63a8006db0 comp rx=0 tx=0).stop 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.613+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 msgr2=0x7f63bc10fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.613+0000 7f63c3014700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 0x7f63bc10fbe0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f63b000b3b0 tx=0x7f63b0009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.613+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 shutdown_connections 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.613+0000 7f63c3014700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63bc0fbcb0 0x7f63bc10fbe0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.613+0000 7f63c3014700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f63ac0692a0 0x7f63ac06b750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.613+0000 7f63c3014700 1 --2- 172.21.15.138:0/2586964045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63bc0fcf60 0x7f63bc110120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.613+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 >> 172.21.15.138:0/2586964045 conn(0x7f63bc0f7430 msgr2=0x7f63bc0f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.614+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 shutdown_connections 2024-04-25T01:31:36.592 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:36.614+0000 7f63c3014700 1 -- 172.21.15.138:0/2586964045 wait complete. 2024-04-25T01:31:37.195 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:37 smithi138 ceph-mon[32977]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:37.195 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:37 smithi138 ceph-mon[32977]: from='osd.5 [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:31:37.195 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:37 smithi138 ceph-mon[32977]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:31:37.195 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:37 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:37.195 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:37 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:37.220 DEBUG:teuthology.orchestra.run.smithi138:osd.5> sudo journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T01:31:37.222 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-25T01:31:37.222 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T01:31:37.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:37 smithi033 ceph-mon[26353]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:37.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:37 smithi033 ceph-mon[26353]: from='osd.5 [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:31:37.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:37 smithi033 ceph-mon[26353]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:31:37.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:37 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:37.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:37 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:37.499 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:31:38.357 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:38 smithi138 ceph-mon[32977]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:31:38.357 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:38 smithi138 ceph-mon[32977]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T01:31:38.357 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:38 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:38.357 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:38 smithi138 ceph-mon[32977]: from='osd.5 [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:31:38.357 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:38 smithi138 ceph-mon[32977]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:31:38.357 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:38 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:38.358 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:38 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:38.358 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:31:38 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[51830]: 2024-04-25T01:31:38.178+0000 7fbd84517700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T01:31:38.358 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:31:38 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[51830]: 2024-04-25T01:31:38.188+0000 7fbd7f312700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:31:38.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:38 smithi033 ceph-mon[26353]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:31:38.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:38 smithi033 ceph-mon[26353]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T01:31:38.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:38 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:38.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:38 smithi033 ceph-mon[26353]: from='osd.5 [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:31:38.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:38 smithi033 ceph-mon[26353]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:31:38.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:38 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:38.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:38 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.243+0000 7f0fb3850700 1 -- 172.21.15.33:0/3468008495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fb460 msgr2=0x7f0fac0fb870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.243+0000 7f0fb3850700 1 --2- 172.21.15.33:0/3468008495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fb460 0x7f0fac0fb870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0fa8004a50 tx=0x7f0fa800af40 comp rx=0 tx=0).stop 2024-04-25T01:31:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.243+0000 7f0fb3850700 1 -- 172.21.15.33:0/3468008495 shutdown_connections 2024-04-25T01:31:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.243+0000 7f0fb3850700 1 --2- 172.21.15.33:0/3468008495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac0fc710 0x7f0fac0fcb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.243+0000 7f0fb3850700 1 --2- 172.21.15.33:0/3468008495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fb460 0x7f0fac0fb870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.243+0000 7f0fb3850700 1 -- 172.21.15.33:0/3468008495 >> 172.21.15.33:0/3468008495 conn(0x7f0fac0f6be0 msgr2=0x7f0fac0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.243+0000 7f0fb3850700 1 -- 172.21.15.33:0/3468008495 shutdown_connections 2024-04-25T01:31:39.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.243+0000 7f0fb3850700 1 -- 172.21.15.33:0/3468008495 wait complete. 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb3850700 1 Processor -- start 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb3850700 1 -- start start 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb3850700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac0fb460 0x7f0fac10b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb3850700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fc710 0x7f0fac10bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb3850700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac10c250 con 0x7f0fac0fc710 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb3850700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fac10c3c0 con 0x7f0fac0fb460 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb0deb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fc710 0x7f0fac10bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb0deb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fc710 0x7f0fac10bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42076/0 (socket says 172.21.15.33:42076) 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb0deb700 1 -- 172.21.15.33:0/4236494143 learned_addr learned my addr 172.21.15.33:0/4236494143 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:31:39.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb15ec700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac0fb460 0x7f0fac10b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb0deb700 1 -- 172.21.15.33:0/4236494143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac0fb460 msgr2=0x7f0fac10b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb0deb700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac0fb460 0x7f0fac10b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb0deb700 1 -- 172.21.15.33:0/4236494143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa8005040 con 0x7f0fac0fc710 2024-04-25T01:31:39.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.244+0000 7f0fb0deb700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fc710 0x7f0fac10bc50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00fce0 tx=0x7f0f9c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:39.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.245+0000 7f0fa1ffb700 1 -- 172.21.15.33:0/4236494143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f9c007ea0 con 0x7f0fac0fc710 2024-04-25T01:31:39.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.245+0000 7f0fa1ffb700 1 -- 172.21.15.33:0/4236494143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f9c00dda0 con 0x7f0fac0fc710 2024-04-25T01:31:39.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.245+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fac110e20 con 0x7f0fac0fc710 2024-04-25T01:31:39.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.245+0000 7f0fa1ffb700 1 -- 172.21.15.33:0/4236494143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f9c0099b0 con 0x7f0fac0fc710 2024-04-25T01:31:39.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.245+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fac1113b0 con 0x7f0fac0fc710 2024-04-25T01:31:39.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.246+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fac04c710 con 0x7f0fac0fc710 2024-04-25T01:31:39.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.246+0000 7f0fa1ffb700 1 -- 172.21.15.33:0/4236494143 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0f9c00d8d0 con 0x7f0fac0fc710 2024-04-25T01:31:39.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.247+0000 7f0fa1ffb700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0f980692a0 0x7f0f9806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:39.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.247+0000 7f0fa1ffb700 1 -- 172.21.15.33:0/4236494143 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0f9c089620 con 0x7f0fac0fc710 2024-04-25T01:31:39.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.247+0000 7f0fb15ec700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0f980692a0 0x7f0f9806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:39.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.247+0000 7f0fb15ec700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0f980692a0 0x7f0f9806b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800c3b0 tx=0x7f0fa800af40 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:39.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.251+0000 7f0fa1ffb700 1 -- 172.21.15.33:0/4236494143 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0f9c059600 con 0x7f0fac0fc710 2024-04-25T01:31:39.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:39 smithi138 ceph-mon[32977]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:39.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:39 smithi138 ceph-mon[32977]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T01:31:39.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:39 smithi138 ceph-mon[32977]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T01:31:39.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:39 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:39.241 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:39 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:39.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.445+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f0fac1036c0 con 0x7f0fac0fc710 2024-04-25T01:31:39.423 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:39 smithi033 ceph-mon[26353]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:39.423 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:39 smithi033 ceph-mon[26353]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]': finished 2024-04-25T01:31:39.423 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:39 smithi033 ceph-mon[26353]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T01:31:39.423 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:39 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:39.423 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:39 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:39.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.445+0000 7f0fa1ffb700 1 -- 172.21.15.33:0/4236494143 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f0f9c059190 con 0x7f0fac0fc710 2024-04-25T01:31:39.424 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:31:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.447+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0f980692a0 msgr2=0x7f0f9806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.447+0000 7f0fb3850700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0f980692a0 0x7f0f9806b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800c3b0 tx=0x7f0fa800af40 comp rx=0 tx=0).stop 2024-04-25T01:31:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.447+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fc710 msgr2=0x7f0fac10bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:39.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.447+0000 7f0fb3850700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fc710 0x7f0fac10bc50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00fce0 tx=0x7f0f9c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.447+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 shutdown_connections 2024-04-25T01:31:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.447+0000 7f0fb3850700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fac0fb460 0x7f0fac10b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.447+0000 7f0fb3850700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0f980692a0 0x7f0f9806b750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.447+0000 7f0fb3850700 1 --2- 172.21.15.33:0/4236494143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fac0fc710 0x7f0fac10bc50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.447+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 >> 172.21.15.33:0/4236494143 conn(0x7f0fac0f6be0 msgr2=0x7f0fac0f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.448+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 shutdown_connections 2024-04-25T01:31:39.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:39.448+0000 7f0fb3850700 1 -- 172.21.15.33:0/4236494143 wait complete. 2024-04-25T01:31:40.041 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1714008699,"num_in_osds":6,"osd_in_since":1714008680,"num_remapped_pgs":0} 2024-04-25T01:31:40.042 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T01:31:40.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:40 smithi138 ceph-mon[32977]: purged_snaps scrub starts 2024-04-25T01:31:40.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:40 smithi138 ceph-mon[32977]: purged_snaps scrub ok 2024-04-25T01:31:40.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:40 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:40.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:40 smithi138 ceph-mon[32977]: osd.5 [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] boot 2024-04-25T01:31:40.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:40 smithi138 ceph-mon[32977]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T01:31:40.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:40 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:40.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:40 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/4236494143' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:31:40.318 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:31:40.433 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:40 smithi033 ceph-mon[26353]: purged_snaps scrub starts 2024-04-25T01:31:40.433 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:40 smithi033 ceph-mon[26353]: purged_snaps scrub ok 2024-04-25T01:31:40.433 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:40 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:40.433 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:40 smithi033 ceph-mon[26353]: osd.5 [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] boot 2024-04-25T01:31:40.433 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:40 smithi033 ceph-mon[26353]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T01:31:40.434 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:40 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:31:40.434 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:40 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/4236494143' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T01:31:41.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:41.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:31:41.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:41 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:41.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:31:41.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:31:41.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:31:41.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:31:41.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:41 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 -- 172.21.15.33:0/3180558303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c103780 msgr2=0x7f173c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 --2- 172.21.15.33:0/3180558303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c103780 0x7f173c0fcdf0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1734009ce0 tx=0x7f1734004d10 comp rx=0 tx=0).stop 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 -- 172.21.15.33:0/3180558303 shutdown_connections 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 --2- 172.21.15.33:0/3180558303 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c103780 0x7f173c0fcdf0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 --2- 172.21.15.33:0/3180558303 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f173c102de0 0x7f173c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 -- 172.21.15.33:0/3180558303 >> 172.21.15.33:0/3180558303 conn(0x7f173c0f8990 msgr2=0x7f173c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 -- 172.21.15.33:0/3180558303 shutdown_connections 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 -- 172.21.15.33:0/3180558303 wait complete. 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 Processor -- start 2024-04-25T01:31:41.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.985+0000 7f1742d24700 1 -- start start 2024-04-25T01:31:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1742d24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c102de0 0x7f173c113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1742d24700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f173c103780 0x7f173c114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1742d24700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c114690 con 0x7f173c102de0 2024-04-25T01:31:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1742d24700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173c10dc70 con 0x7f173c103780 2024-04-25T01:31:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1740ac0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c102de0 0x7f173c113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1740ac0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c102de0 0x7f173c113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42092/0 (socket says 172.21.15.33:42092) 2024-04-25T01:31:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1740ac0700 1 -- 172.21.15.33:0/1546034910 learned_addr learned my addr 172.21.15.33:0/1546034910 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:31:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f173bfff700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f173c103780 0x7f173c114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:41.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1740ac0700 1 -- 172.21.15.33:0/1546034910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f173c103780 msgr2=0x7f173c114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1740ac0700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f173c103780 0x7f173c114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1740ac0700 1 -- 172.21.15.33:0/1546034910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734009940 con 0x7f173c102de0 2024-04-25T01:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f1740ac0700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c102de0 0x7f173c113b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c830 tx=0x7f172c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.986+0000 7f17397fa700 1 -- 172.21.15.33:0/1546034910 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f172c00a760 con 0x7f173c102de0 2024-04-25T01:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.987+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173c10df50 con 0x7f173c102de0 2024-04-25T01:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.987+0000 7f17397fa700 1 -- 172.21.15.33:0/1546034910 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f172c00ad90 con 0x7f173c102de0 2024-04-25T01:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.987+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173c10e510 con 0x7f173c102de0 2024-04-25T01:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.987+0000 7f17397fa700 1 -- 172.21.15.33:0/1546034910 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f172c0113a0 con 0x7f173c102de0 2024-04-25T01:31:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.988+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173c0fe530 con 0x7f173c102de0 2024-04-25T01:31:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.988+0000 7f17397fa700 1 -- 172.21.15.33:0/1546034910 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f172c012400 con 0x7f173c102de0 2024-04-25T01:31:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.988+0000 7f17397fa700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f17240692a0 0x7f172406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.989+0000 7f17397fa700 1 -- 172.21.15.33:0/1546034910 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f172c086dc0 con 0x7f173c102de0 2024-04-25T01:31:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.989+0000 7f173bfff700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f17240692a0 0x7f172406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.989+0000 7f173bfff700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f17240692a0 0x7f172406b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f173c10f3a0 tx=0x7f1734004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:41.993+0000 7f17397fa700 1 -- 172.21.15.33:0/1546034910 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f172c056f60 con 0x7f173c102de0 2024-04-25T01:31:42.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.200+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f173c04c710 con 0x7f173c102de0 2024-04-25T01:31:42.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.201+0000 7f17397fa700 1 -- 172.21.15.33:0/1546034910 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11057 (secure 0 0 0) 0x7f172c056af0 con 0x7f173c102de0 2024-04-25T01:31:42.179 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:31:42.179 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":40,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","created":"2024-04-25T01:26:05.674330+0000","modified":"2024-04-25T01:31:40.182361+0000","last_up_change":"2024-04-25T01:31:39.177371+0000","last_in_change":"2024-04-25T01:31:20.199327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:30:26.558656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"68af93d4-7aa8-485f-826e-62a6e362907b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6803","nonce":505851393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6805","nonce":505851393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6809","nonce":505851393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6807","nonce":505851393}]},"public_addr":"172.21.15.33:6803/505851393","cluster_addr":"172.21.15.33:6805/505851393","heartbeat_back_addr":"172.21.15.33:6809/505851393","heartbeat_front_addr":"172.21.15.33:6807/505851393","state":["exists","up"]},{"osd":1,"uuid":"1fdb7e9b-1138-43db-a11f-3e30e286c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6811","nonce":835394340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6813","nonce":835394340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6817","nonce":835394340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6815","nonce":835394340}]},"public_addr":"172.21.15.33:6811/835394340","cluster_addr":"172.21.15.33:6813/835394340","heartbeat_back_addr":"172.21.15.33:6817/835394340","heartbeat_front_addr":"172.21.15.33:6815/835394340","state":["exists","up"]},{"osd":2,"uuid":"b1239da3-566e-4df0-b5ea-df4c14959837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6819","nonce":4233054842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6821","nonce":4233054842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6825","nonce":4233054842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6823","nonce":4233054842}]},"public_addr":"172.21.15.33:6819/4233054842","cluster_addr":"172.21.15.33:6821/4233054842","heartbeat_back_addr":"172.21.15.33:6825/4233054842","heartbeat_front_addr":"172.21.15.33:6823/4233054842","state":["exists","up"]},{"osd":3,"uuid":"abc58749-77ca-4476-9601-cdf89d3c1e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6801","nonce":461435998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6803","nonce":461435998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6807","nonce":461435998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6805","nonce":461435998}]},"public_addr":"172.21.15.138:6801/461435998","cluster_addr":"172.21.15.138:6803/461435998","heartbeat_back_addr":"172.21.15.138:6807/461435998","heartbeat_front_addr":"172.21.15.138:6805/461435998","state":["exists","up"]},{"osd":4,"uuid":"83e421bd-52ba-47ad-8755-e1354f502ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6809","nonce":1618213578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6811","nonce":1618213578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6815","nonce":1618213578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6813","nonce":1618213578}]},"public_addr":"172.21.15.138:6809/1618213578","cluster_addr":"172.21.15.138:6811/1618213578","heartbeat_back_addr":"172.21.15.138:6815/1618213578","heartbeat_front_addr":"172.21.15.138:6813/1618213578","state":["exists","up"]},{"osd":5,"uuid":"266e5984-f087-408e-a84c-04f81f33af43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6817","nonce":1128834056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6819","nonce":1128834056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6823","nonce":1128834056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6821","nonce":1128834056}]},"public_addr":"172.21.15.138:6817/1128834056","cluster_addr":"172.21.15.138:6819/1128834056","heartbeat_back_addr":"172.21.15.138:6823/1128834056","heartbeat_front_addr":"172.21.15.138:6821/1128834056","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:27.636323+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:55.618887+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:23.307858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:47.331937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:12.159932+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:37.317499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2566719862":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1401659678":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/751781866":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6800/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/4160574345":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/2117172792":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/1131828704":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2112653252":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6800/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/686286320":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6801/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1869074265":"2024-04-26T01:27:11.182264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:31:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f17240692a0 msgr2=0x7f172406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f17240692a0 0x7f172406b750 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f173c10f3a0 tx=0x7f1734004d10 comp rx=0 tx=0).stop 2024-04-25T01:31:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c102de0 msgr2=0x7f173c113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c102de0 0x7f173c113b00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f172c00c830 tx=0x7f172c004470 comp rx=0 tx=0).stop 2024-04-25T01:31:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 shutdown_connections 2024-04-25T01:31:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f17240692a0 0x7f172406b750 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f173c102de0 0x7f173c113b00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:42.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 --2- 172.21.15.33:0/1546034910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f173c103780 0x7f173c114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 >> 172.21.15.33:0/1546034910 conn(0x7f173c0f8990 msgr2=0x7f173c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 shutdown_connections 2024-04-25T01:31:42.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:42.204+0000 7f1742d24700 1 -- 172.21.15.33:0/1546034910 wait complete. 2024-04-25T01:31:42.471 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:42 smithi033 ceph-mon[26353]: Detected new or changed devices on smithi138 2024-04-25T01:31:42.471 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:42 smithi033 ceph-mon[26353]: Adjusting osd_memory_target on smithi138 to 5312M 2024-04-25T01:31:42.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:42 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi138 2024-04-25T01:31:42.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:42 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 5312M 2024-04-25T01:31:43.062 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:30:26.558656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-04-25T01:31:43.063 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T01:31:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:31:43.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:43 smithi033 ceph-mon[26353]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:43.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:43 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1546034910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:31:43.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:43 smithi138 ceph-mon[32977]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:43.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:43 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1546034910' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:31:45.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 -- 172.21.15.33:0/4057677317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 msgr2=0x7fcaf4109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:45.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 --2- 172.21.15.33:0/4057677317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 0x7fcaf4109150 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcae4004970 tx=0x7fcae4009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:45.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 -- 172.21.15.33:0/4057677317 shutdown_connections 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 --2- 172.21.15.33:0/4057677317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 0x7fcaf4109150 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 --2- 172.21.15.33:0/4057677317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf40702a0 0x7fcaf4070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 -- 172.21.15.33:0/4057677317 >> 172.21.15.33:0/4057677317 conn(0x7fcaf40f8990 msgr2=0x7fcaf40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 -- 172.21.15.33:0/4057677317 shutdown_connections 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 -- 172.21.15.33:0/4057677317 wait complete. 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 Processor -- start 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.034+0000 7fcafc794700 1 -- start start 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcafc794700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf40702a0 0x7fcaf4115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcafc794700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 0x7fcaf4116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcafc794700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf41166e0 con 0x7fcaf4070bb0 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcafc794700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf410fcc0 con 0x7fcaf40702a0 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaf9d2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 0x7fcaf4116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaf9d2f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 0x7fcaf4116090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42100/0 (socket says 172.21.15.33:42100) 2024-04-25T01:31:45.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaf9d2f700 1 -- 172.21.15.33:0/828022071 learned_addr learned my addr 172.21.15.33:0/828022071 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcafa530700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf40702a0 0x7fcaf4115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaf9d2f700 1 -- 172.21.15.33:0/828022071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf40702a0 msgr2=0x7fcaf4115b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaf9d2f700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf40702a0 0x7fcaf4115b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaf9d2f700 1 -- 172.21.15.33:0/828022071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae4005040 con 0x7fcaf4070bb0 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcafa530700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf40702a0 0x7fcaf4115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaf9d2f700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 0x7fcaf4116090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcae4000c00 tx=0x7fcae4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaeaffd700 1 -- 172.21.15.33:0/828022071 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae400e040 con 0x7fcaf4070bb0 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaf410ff40 con 0x7fcaf4070bb0 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaeaffd700 1 -- 172.21.15.33:0/828022071 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcae400bc10 con 0x7fcaf4070bb0 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcaeaffd700 1 -- 172.21.15.33:0/828022071 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcae401c630 con 0x7fcaf4070bb0 2024-04-25T01:31:45.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.035+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf4110480 con 0x7fcaf4070bb0 2024-04-25T01:31:45.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.036+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf40719e0 con 0x7fcaf4070bb0 2024-04-25T01:31:45.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.037+0000 7fcaeaffd700 1 -- 172.21.15.33:0/828022071 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fcae400bd80 con 0x7fcaf4070bb0 2024-04-25T01:31:45.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.037+0000 7fcaeaffd700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcae00692a0 0x7fcae006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:45.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.037+0000 7fcaeaffd700 1 -- 172.21.15.33:0/828022071 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fcae4088130 con 0x7fcaf4070bb0 2024-04-25T01:31:45.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.037+0000 7fcafa530700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcae00692a0 0x7fcae006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:45.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.038+0000 7fcafa530700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcae00692a0 0x7fcae006b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0007fd0 tx=0x7fcaf0009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:45.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.042+0000 7fcaeaffd700 1 -- 172.21.15.33:0/828022071 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcae4058230 con 0x7fcaf4070bb0 2024-04-25T01:31:45.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.235+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcaf41111a0 con 0x7fcaf4070bb0 2024-04-25T01:31:45.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.235+0000 7fcaeaffd700 1 -- 172.21.15.33:0/828022071 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcae4013e60 con 0x7fcaf4070bb0 2024-04-25T01:31:45.213 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcae00692a0 msgr2=0x7fcae006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcae00692a0 0x7fcae006b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0007fd0 tx=0x7fcaf0009040 comp rx=0 tx=0).stop 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 msgr2=0x7fcaf4116090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 0x7fcaf4116090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcae4000c00 tx=0x7fcae4009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 shutdown_connections 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcaf40702a0 0x7fcaf4115b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcae00692a0 0x7fcae006b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 --2- 172.21.15.33:0/828022071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcaf4070bb0 0x7fcaf4116090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 >> 172.21.15.33:0/828022071 conn(0x7fcaf40f8990 msgr2=0x7fcaf40f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 shutdown_connections 2024-04-25T01:31:45.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:45.237+0000 7fcafc794700 1 -- 172.21.15.33:0/828022071 wait complete. 2024-04-25T01:31:45.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:45 smithi033 ceph-mon[26353]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:45.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:45 smithi138 ceph-mon[32977]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:45.915 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T01:31:45.916 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T01:31:46.197 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:31:46.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:46 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/828022071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:31:46.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:31:46.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:46 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/828022071' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:31:46.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:31:47.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:47 smithi033 ceph-mon[26353]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:47.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:47 smithi138 ceph-mon[32977]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:47.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 -- 172.21.15.33:0/1355867039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 msgr2=0x7f82740fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:47.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 --2- 172.21.15.33:0/1355867039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 0x7f82740fe460 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8268004940 tx=0x7f8268009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:47.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 -- 172.21.15.33:0/1355867039 shutdown_connections 2024-04-25T01:31:47.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 --2- 172.21.15.33:0/1355867039 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 0x7f82740fe460 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 --2- 172.21.15.33:0/1355867039 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fce80 0x7f82740fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 -- 172.21.15.33:0/1355867039 >> 172.21.15.33:0/1355867039 conn(0x7f82740f8580 msgr2=0x7f82740fa990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 -- 172.21.15.33:0/1355867039 shutdown_connections 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 -- 172.21.15.33:0/1355867039 wait complete. 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 Processor -- start 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.018+0000 7f827c381700 1 -- start start 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827c381700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fce80 0x7f827410d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827c381700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 0x7f827410ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827c381700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827410e3b0 con 0x7f82740fdff0 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827c381700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827410e520 con 0x7f82740fce80 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827991c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 0x7f827410ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827991c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 0x7f827410ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55396/0 (socket says 172.21.15.33:55396) 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827991c700 1 -- 172.21.15.33:0/2269402284 learned_addr learned my addr 172.21.15.33:0/2269402284 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:31:47.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827a11d700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fce80 0x7f827410d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:47.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827991c700 1 -- 172.21.15.33:0/2269402284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fce80 msgr2=0x7f827410d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:47.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827991c700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fce80 0x7f827410d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:47.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827991c700 1 -- 172.21.15.33:0/2269402284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8268005040 con 0x7f82740fdff0 2024-04-25T01:31:47.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827991c700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 0x7f827410ddb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8268013870 tx=0x7f8268009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:47.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f8272ffd700 1 -- 172.21.15.33:0/2269402284 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8268021070 con 0x7f82740fdff0 2024-04-25T01:31:47.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f8272ffd700 1 -- 172.21.15.33:0/2269402284 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f826801c400 con 0x7f82740fdff0 2024-04-25T01:31:47.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8274112f20 con 0x7f82740fdff0 2024-04-25T01:31:47.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f8272ffd700 1 -- 172.21.15.33:0/2269402284 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f826800e070 con 0x7f82740fdff0 2024-04-25T01:31:47.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.019+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8274113430 con 0x7f82740fdff0 2024-04-25T01:31:48.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.020+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f827404c710 con 0x7f82740fdff0 2024-04-25T01:31:48.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.021+0000 7f8272ffd700 1 -- 172.21.15.33:0/2269402284 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f826801ca20 con 0x7f82740fdff0 2024-04-25T01:31:48.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.021+0000 7f8272ffd700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f825c0692a0 0x7f825c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:48.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.021+0000 7f8272ffd700 1 -- 172.21.15.33:0/2269402284 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8268088240 con 0x7f82740fdff0 2024-04-25T01:31:48.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.021+0000 7f827a11d700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f825c0692a0 0x7f825c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:48.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.022+0000 7f827a11d700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f825c0692a0 0x7f825c06b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8264004b50 tx=0x7f826400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:48.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.026+0000 7f8272ffd700 1 -- 172.21.15.33:0/2269402284 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8268058340 con 0x7f82740fdff0 2024-04-25T01:31:48.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.299+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8274113650 con 0x7f82740fdff0 2024-04-25T01:31:48.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.304+0000 7f8272ffd700 1 -- 172.21.15.33:0/2269402284 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8268004490 con 0x7f82740fdff0 2024-04-25T01:31:48.282 INFO:teuthology.orchestra.run.smithi033.stdout:[client.0] 2024-04-25T01:31:48.282 INFO:teuthology.orchestra.run.smithi033.stdout: key = AQCEsilmGHTpERAA0OvgPaRPwFXU3VpqF+SWWQ== 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f825c0692a0 msgr2=0x7f825c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f825c0692a0 0x7f825c06b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8264004b50 tx=0x7f826400b040 comp rx=0 tx=0).stop 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 msgr2=0x7f827410ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 0x7f827410ddb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8268013870 tx=0x7f8268009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 shutdown_connections 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82740fce80 0x7f827410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f825c0692a0 0x7f825c06b750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 --2- 172.21.15.33:0/2269402284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82740fdff0 0x7f827410ddb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 >> 172.21.15.33:0/2269402284 conn(0x7f82740f8580 msgr2=0x7f82740f9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:48.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.306+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 shutdown_connections 2024-04-25T01:31:48.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:48.307+0000 7f827c381700 1 -- 172.21.15.33:0/2269402284 wait complete. 2024-04-25T01:31:48.840 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:31:48.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T01:31:48.840 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T01:31:48.878 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T01:31:49.140 INFO:teuthology.orchestra.run.smithi138.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T01:31:49.470 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:49 smithi138 ceph-mon[32977]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:49.471 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:49 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2269402284' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:31:49.471 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:49 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2269402284' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:31:49.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:49 smithi033 ceph-mon[26353]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:49.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:49 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2269402284' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:31:49.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:49 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2269402284' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:31:50.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.674+0000 7faa65ad1700 1 -- 172.21.15.138:0/3133413434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 msgr2=0x7faa600faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:50.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.674+0000 7faa65ad1700 1 --2- 172.21.15.138:0/3133413434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 0x7faa600faea0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faa54004940 tx=0x7faa54009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:50.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.674+0000 7faa65ad1700 1 -- 172.21.15.138:0/3133413434 shutdown_connections 2024-04-25T01:31:50.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.674+0000 7faa65ad1700 1 --2- 172.21.15.138:0/3133413434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 0x7faa600faea0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:50.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.674+0000 7faa65ad1700 1 --2- 172.21.15.138:0/3133413434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa600fe410 0x7faa600fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:50.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.674+0000 7faa65ad1700 1 -- 172.21.15.138:0/3133413434 >> 172.21.15.138:0/3133413434 conn(0x7faa600f6bb0 msgr2=0x7faa600f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:50.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.674+0000 7faa65ad1700 1 -- 172.21.15.138:0/3133413434 shutdown_connections 2024-04-25T01:31:50.653 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.674+0000 7faa65ad1700 1 -- 172.21.15.138:0/3133413434 wait complete. 2024-04-25T01:31:50.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.675+0000 7faa65ad1700 1 Processor -- start 2024-04-25T01:31:50.654 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.675+0000 7faa65ad1700 1 -- start start 2024-04-25T01:31:50.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.675+0000 7faa65ad1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa600fe410 0x7faa6010d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:50.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.675+0000 7faa65ad1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 0x7faa6010de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:50.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.675+0000 7faa65ad1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6010e4a0 con 0x7faa600fe410 2024-04-25T01:31:50.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.675+0000 7faa65ad1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa60112e40 con 0x7faa600fed60 2024-04-25T01:31:50.655 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.675+0000 7faa5effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 0x7faa6010de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa5effd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 0x7faa6010de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.138:60992/0 (socket says 172.21.15.138:60992) 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa5effd700 1 -- 172.21.15.138:0/3732704130 learned_addr learned my addr 172.21.15.138:0/3732704130 (peer_addr_for_me v2:172.21.15.138:0/0) 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa5f7fe700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa600fe410 0x7faa6010d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa5effd700 1 -- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa600fe410 msgr2=0x7faa6010d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa5effd700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa600fe410 0x7faa6010d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa5effd700 1 -- 172.21.15.138:0/3732704130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa54005040 con 0x7faa600fed60 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa5effd700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 0x7faa6010de10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faa5400b3b0 tx=0x7faa54009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa47fff700 1 -- 172.21.15.138:0/3732704130 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa5400e040 con 0x7faa600fed60 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa47fff700 1 -- 172.21.15.138:0/3732704130 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa5400bcc0 con 0x7faa600fed60 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa60113040 con 0x7faa600fed60 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa47fff700 1 -- 172.21.15.138:0/3732704130 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa5401c630 con 0x7faa600fed60 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.676+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa601135b0 con 0x7faa600fed60 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.677+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6004c710 con 0x7faa600fed60 2024-04-25T01:31:50.656 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.678+0000 7faa47fff700 1 -- 172.21.15.138:0/3732704130 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7faa54013980 con 0x7faa600fed60 2024-04-25T01:31:50.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.678+0000 7faa47fff700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa480692a0 0x7faa4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:50.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.678+0000 7faa47fff700 1 -- 172.21.15.138:0/3732704130 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7faa54087bd0 con 0x7faa600fed60 2024-04-25T01:31:50.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.678+0000 7faa5f7fe700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa480692a0 0x7faa4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:50.657 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.679+0000 7faa5f7fe700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa480692a0 0x7faa4806b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa50006fd0 tx=0x7faa50008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:50.662 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.682+0000 7faa47fff700 1 -- 172.21.15.138:0/3732704130 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faa54057cd0 con 0x7faa600fed60 2024-04-25T01:31:50.916 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.938+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7faa601139e0 con 0x7faa600fed60 2024-04-25T01:31:50.921 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.943+0000 7faa47fff700 1 -- 172.21.15.138:0/3732704130 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7faa54057860 con 0x7faa600fed60 2024-04-25T01:31:50.921 INFO:teuthology.orchestra.run.smithi138.stdout:[client.1] 2024-04-25T01:31:50.922 INFO:teuthology.orchestra.run.smithi138.stdout: key = AQCGsilmx6AbOBAAyI03Rx6jtZhXTTFr8aIrBA== 2024-04-25T01:31:50.923 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.945+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa480692a0 msgr2=0x7faa4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa480692a0 0x7faa4806b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa50006fd0 tx=0x7faa50008040 comp rx=0 tx=0).stop 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 msgr2=0x7faa6010de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 0x7faa6010de10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7faa5400b3b0 tx=0x7faa54009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 shutdown_connections 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa480692a0 0x7faa4806b750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa600fe410 0x7faa6010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 --2- 172.21.15.138:0/3732704130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa600fed60 0x7faa6010de10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 >> 172.21.15.138:0/3732704130 conn(0x7faa600f6bb0 msgr2=0x7faa600f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 shutdown_connections 2024-04-25T01:31:50.924 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:31:50.946+0000 7faa65ad1700 1 -- 172.21.15.138:0/3732704130 wait complete. 2024-04-25T01:31:51.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:51 smithi033 ceph-mon[26353]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:51.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:51 smithi033 ceph-mon[26353]: from='client.? 172.21.15.138:0/3732704130' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:31:51.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:51 smithi033 ceph-mon[26353]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:31:51.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:51 smithi033 ceph-mon[26353]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:31:51.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:51 smithi138 ceph-mon[32977]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:51 smithi138 ceph-mon[32977]: from='client.? 172.21.15.138:0/3732704130' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:31:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:51 smithi138 ceph-mon[32977]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T01:31:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:51 smithi138 ceph-mon[32977]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T01:31:51.548 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:31:51.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-25T01:31:51.548 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T01:31:51.583 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T01:31:51.584 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T01:31:51.584 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T01:31:51.862 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:31:53.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:53 smithi033 ceph-mon[26353]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:53.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:53 smithi138 ceph-mon[32977]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.648+0000 7fa8f2207700 1 -- 172.21.15.33:0/983504652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fbf40 msgr2=0x7fa8ec101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:53.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.648+0000 7fa8f2207700 1 --2- 172.21.15.33:0/983504652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fbf40 0x7fa8ec101a30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0004940 tx=0x7fa8e0009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.648+0000 7fa8f2207700 1 -- 172.21.15.33:0/983504652 shutdown_connections 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.648+0000 7fa8f2207700 1 --2- 172.21.15.33:0/983504652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fbf40 0x7fa8ec101a30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.648+0000 7fa8f2207700 1 --2- 172.21.15.33:0/983504652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8ec0fb5a0 0x7fa8ec0fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.648+0000 7fa8f2207700 1 -- 172.21.15.33:0/983504652 >> 172.21.15.33:0/983504652 conn(0x7fa8ec073350 msgr2=0x7fa8ec073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.648+0000 7fa8f2207700 1 -- 172.21.15.33:0/983504652 shutdown_connections 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.648+0000 7fa8f2207700 1 -- 172.21.15.33:0/983504652 wait complete. 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8f2207700 1 Processor -- start 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8f2207700 1 -- start start 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8f2207700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fb5a0 0x7fa8ec0fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8f2207700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8ec0fbf40 0x7fa8ec0fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:53.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8f2207700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec0fe480 con 0x7fa8ec0fb5a0 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8f2207700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ec0fcfb0 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8eb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fb5a0 0x7fa8ec0fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8eb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fb5a0 0x7fa8ec0fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55418/0 (socket says 172.21.15.33:55418) 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8eb7fe700 1 -- 172.21.15.33:0/1881838899 learned_addr learned my addr 172.21.15.33:0/1881838899 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8eaffd700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8ec0fbf40 0x7fa8ec0fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8eaffd700 1 -- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fb5a0 msgr2=0x7fa8ec0fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8eaffd700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fb5a0 0x7fa8ec0fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8eaffd700 1 -- 172.21.15.33:0/1881838899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e0005040 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.649+0000 7fa8eb7fe700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fb5a0 0x7fa8ec0fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.650+0000 7fa8eaffd700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8ec0fbf40 0x7fa8ec0fca70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0701d0 tx=0x7fa8e0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.650+0000 7fa8d3fff700 1 -- 172.21.15.33:0/1881838899 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8e000e070 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.650+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ec0fd230 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.650+0000 7fa8d3fff700 1 -- 172.21.15.33:0/1881838899 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8e0013c00 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.650+0000 7fa8d3fff700 1 -- 172.21.15.33:0/1881838899 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8e000b670 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.650+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ec11a030 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.651+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ec0fe670 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.651+0000 7fa8d3fff700 1 -- 172.21.15.33:0/1881838899 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa8e000b8a0 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.652+0000 7fa8d3fff700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa8d40692a0 0x7fa8d406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:53.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.652+0000 7fa8d3fff700 1 -- 172.21.15.33:0/1881838899 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fa8e0088510 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.652+0000 7fa8eb7fe700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa8d40692a0 0x7fa8d406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:53.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.652+0000 7fa8eb7fe700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa8d40692a0 0x7fa8d406b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0041d0 tx=0x7fa8dc00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:53.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.656+0000 7fa8d3fff700 1 -- 172.21.15.33:0/1881838899 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa8e0054ae0 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.908+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa8ec109f00 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.911+0000 7fa8d3fff700 1 -- 172.21.15.33:0/1881838899 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167178 (secure 0 0 0) 0x7fa8e0058100 con 0x7fa8ec0fbf40 2024-04-25T01:31:53.892 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa8d40692a0 msgr2=0x7fa8d406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa8d40692a0 0x7fa8d406b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc0041d0 tx=0x7fa8dc00b040 comp rx=0 tx=0).stop 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8ec0fbf40 msgr2=0x7fa8ec0fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8ec0fbf40 0x7fa8ec0fca70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0701d0 tx=0x7fa8e0009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 shutdown_connections 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa8d40692a0 0x7fa8d406b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8ec0fb5a0 0x7fa8ec0fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 --2- 172.21.15.33:0/1881838899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8ec0fbf40 0x7fa8ec0fca70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 >> 172.21.15.33:0/1881838899 conn(0x7fa8ec073350 msgr2=0x7fa8ec100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:53.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 shutdown_connections 2024-04-25T01:31:53.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:53.915+0000 7fa8f2207700 1 -- 172.21.15.33:0/1881838899 wait complete. 2024-04-25T01:31:54.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:54 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1881838899' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T01:31:54.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:54 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1881838899' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T01:31:54.527 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":19,"flags":0,"active_gid":14208,"active_name":"smithi033.asemba","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6800","nonce":1387801224},{"type":"v1","addr":"172.21.15.33:6801","nonce":1387801224}]},"active_addr":"172.21.15.33:6801/1387801224","active_change":"2024-04-25T01:28:16.045680+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14226,"name":"smithi138.mmfhyi","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.33:8443/","prometheus":"http://172.21.15.33:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":3334291936}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":454449040}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2062772238}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.33:0","nonce":2952222145}]}]} 2024-04-25T01:31:54.531 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T01:31:54.531 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T01:31:54.531 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T01:31:54.801 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:31:55.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:55 smithi033 ceph-mon[26353]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:55.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:55 smithi138 ceph-mon[32977]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.428+0000 7f866945e700 1 -- 172.21.15.33:0/4084377048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 msgr2=0x7f86640fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.428+0000 7f866945e700 1 --2- 172.21.15.33:0/4084377048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 0x7f86640fcdf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f864c004940 tx=0x7f864c009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.428+0000 7f866945e700 1 -- 172.21.15.33:0/4084377048 shutdown_connections 2024-04-25T01:31:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.428+0000 7f866945e700 1 --2- 172.21.15.33:0/4084377048 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 0x7f86640fcdf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.428+0000 7f866945e700 1 --2- 172.21.15.33:0/4084377048 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8664102de0 0x7f86641031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.428+0000 7f866945e700 1 -- 172.21.15.33:0/4084377048 >> 172.21.15.33:0/4084377048 conn(0x7f86640f8990 msgr2=0x7f86640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.428+0000 7f866945e700 1 -- 172.21.15.33:0/4084377048 shutdown_connections 2024-04-25T01:31:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.428+0000 7f866945e700 1 -- 172.21.15.33:0/4084377048 wait complete. 2024-04-25T01:31:56.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.428+0000 7f866945e700 1 Processor -- start 2024-04-25T01:31:56.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f866945e700 1 -- start start 2024-04-25T01:31:56.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f866945e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8664102de0 0x7f8664113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:56.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f866945e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 0x7f8664113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:56.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f866945e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8664114520 con 0x7f8664103780 2024-04-25T01:31:56.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f866945e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866410db20 con 0x7f8664102de0 2024-04-25T01:31:56.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f86627fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 0x7f8664113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:56.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f86627fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 0x7f8664113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42168/0 (socket says 172.21.15.33:42168) 2024-04-25T01:31:56.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f86627fc700 1 -- 172.21.15.33:0/1185821731 learned_addr learned my addr 172.21.15.33:0/1185821731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:31:56.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f8662ffd700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8664102de0 0x7f8664113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f86627fc700 1 -- 172.21.15.33:0/1185821731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8664102de0 msgr2=0x7f8664113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f86627fc700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8664102de0 0x7f8664113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f86627fc700 1 -- 172.21.15.33:0/1185821731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c005040 con 0x7f8664103780 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.429+0000 7f86627fc700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 0x7f8664113ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f864c009100 tx=0x7f864c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.430+0000 7f865b7fe700 1 -- 172.21.15.33:0/1185821731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864c00e040 con 0x7f8664103780 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.430+0000 7f865b7fe700 1 -- 172.21.15.33:0/1185821731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f864c009af0 con 0x7f8664103780 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.430+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866410dda0 con 0x7f8664103780 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.430+0000 7f865b7fe700 1 -- 172.21.15.33:0/1185821731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f864c01c3f0 con 0x7f8664103780 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.430+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866410e360 con 0x7f8664103780 2024-04-25T01:31:56.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.431+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86640fe530 con 0x7f8664103780 2024-04-25T01:31:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.431+0000 7f865b7fe700 1 -- 172.21.15.33:0/1185821731 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f864c00baf0 con 0x7f8664103780 2024-04-25T01:31:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.431+0000 7f865b7fe700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f86500692a0 0x7f865006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.432+0000 7f865b7fe700 1 -- 172.21.15.33:0/1185821731 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f864c0880e0 con 0x7f8664103780 2024-04-25T01:31:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.432+0000 7f8662ffd700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f86500692a0 0x7f865006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.432+0000 7f8662ffd700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f86500692a0 0x7f865006b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8654004eb0 tx=0x7f865400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:56.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.436+0000 7f865b7fe700 1 -- 172.21.15.33:0/1185821731 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f864c0581e0 con 0x7f8664103780 2024-04-25T01:31:56.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.632+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8664109ec0 con 0x7f8664103780 2024-04-25T01:31:56.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.633+0000 7f865b7fe700 1 -- 172.21.15.33:0/1185821731 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11057 (secure 0 0 0) 0x7f864c057d70 con 0x7f8664103780 2024-04-25T01:31:56.615 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:31:56.615 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":40,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","created":"2024-04-25T01:26:05.674330+0000","modified":"2024-04-25T01:31:40.182361+0000","last_up_change":"2024-04-25T01:31:39.177371+0000","last_in_change":"2024-04-25T01:31:20.199327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:30:26.558656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"68af93d4-7aa8-485f-826e-62a6e362907b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6803","nonce":505851393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6805","nonce":505851393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6809","nonce":505851393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6807","nonce":505851393}]},"public_addr":"172.21.15.33:6803/505851393","cluster_addr":"172.21.15.33:6805/505851393","heartbeat_back_addr":"172.21.15.33:6809/505851393","heartbeat_front_addr":"172.21.15.33:6807/505851393","state":["exists","up"]},{"osd":1,"uuid":"1fdb7e9b-1138-43db-a11f-3e30e286c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6811","nonce":835394340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6813","nonce":835394340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6817","nonce":835394340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6815","nonce":835394340}]},"public_addr":"172.21.15.33:6811/835394340","cluster_addr":"172.21.15.33:6813/835394340","heartbeat_back_addr":"172.21.15.33:6817/835394340","heartbeat_front_addr":"172.21.15.33:6815/835394340","state":["exists","up"]},{"osd":2,"uuid":"b1239da3-566e-4df0-b5ea-df4c14959837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6819","nonce":4233054842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6821","nonce":4233054842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6825","nonce":4233054842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6823","nonce":4233054842}]},"public_addr":"172.21.15.33:6819/4233054842","cluster_addr":"172.21.15.33:6821/4233054842","heartbeat_back_addr":"172.21.15.33:6825/4233054842","heartbeat_front_addr":"172.21.15.33:6823/4233054842","state":["exists","up"]},{"osd":3,"uuid":"abc58749-77ca-4476-9601-cdf89d3c1e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6801","nonce":461435998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6803","nonce":461435998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6807","nonce":461435998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6805","nonce":461435998}]},"public_addr":"172.21.15.138:6801/461435998","cluster_addr":"172.21.15.138:6803/461435998","heartbeat_back_addr":"172.21.15.138:6807/461435998","heartbeat_front_addr":"172.21.15.138:6805/461435998","state":["exists","up"]},{"osd":4,"uuid":"83e421bd-52ba-47ad-8755-e1354f502ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6809","nonce":1618213578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6811","nonce":1618213578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6815","nonce":1618213578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6813","nonce":1618213578}]},"public_addr":"172.21.15.138:6809/1618213578","cluster_addr":"172.21.15.138:6811/1618213578","heartbeat_back_addr":"172.21.15.138:6815/1618213578","heartbeat_front_addr":"172.21.15.138:6813/1618213578","state":["exists","up"]},{"osd":5,"uuid":"266e5984-f087-408e-a84c-04f81f33af43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6817","nonce":1128834056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6819","nonce":1128834056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6823","nonce":1128834056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6821","nonce":1128834056}]},"public_addr":"172.21.15.138:6817/1128834056","cluster_addr":"172.21.15.138:6819/1128834056","heartbeat_back_addr":"172.21.15.138:6823/1128834056","heartbeat_front_addr":"172.21.15.138:6821/1128834056","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:27.636323+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:55.618887+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:23.307858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:47.331937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:12.159932+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:37.317499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2566719862":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1401659678":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/751781866":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6800/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/4160574345":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/2117172792":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/1131828704":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2112653252":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6800/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/686286320":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6801/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1869074265":"2024-04-26T01:27:11.182264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:31:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f86500692a0 msgr2=0x7f865006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:56.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f86500692a0 0x7f865006b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8654004eb0 tx=0x7f865400b040 comp rx=0 tx=0).stop 2024-04-25T01:31:56.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 msgr2=0x7f8664113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:56.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 0x7f8664113ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f864c009100 tx=0x7f864c009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:56.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 shutdown_connections 2024-04-25T01:31:56.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8664102de0 0x7f8664113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:56.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f86500692a0 0x7f865006b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:56.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 --2- 172.21.15.33:0/1185821731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8664103780 0x7f8664113ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:56.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 >> 172.21.15.33:0/1185821731 conn(0x7f86640f8990 msgr2=0x7f86640f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:56.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 shutdown_connections 2024-04-25T01:31:56.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:56.635+0000 7f866945e700 1 -- 172.21.15.33:0/1185821731 wait complete. 2024-04-25T01:31:57.289 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T01:31:57.289 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T01:31:57.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:57 smithi033 ceph-mon[26353]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:57.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:57 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1185821731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:31:57.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:57 smithi138 ceph-mon[32977]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:57.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:57 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1185821731' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:31:57.557 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:31:59.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.385+0000 7f4df10ec700 1 -- 172.21.15.33:0/3013314785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec101d00 msgr2=0x7f4dec1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.385+0000 7f4df10ec700 1 --2- 172.21.15.33:0/3013314785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec101d00 0x7f4dec1020d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc004940 tx=0x7f4ddc009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:59.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.385+0000 7f4df10ec700 1 -- 172.21.15.33:0/3013314785 shutdown_connections 2024-04-25T01:31:59.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.385+0000 7f4df10ec700 1 --2- 172.21.15.33:0/3013314785 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec1026a0 0x7f4dec0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.385+0000 7f4df10ec700 1 --2- 172.21.15.33:0/3013314785 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec101d00 0x7f4dec1020d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.385+0000 7f4df10ec700 1 -- 172.21.15.33:0/3013314785 >> 172.21.15.33:0/3013314785 conn(0x7f4dec0f7870 msgr2=0x7f4dec0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.386+0000 7f4df10ec700 1 -- 172.21.15.33:0/3013314785 shutdown_connections 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.386+0000 7f4df10ec700 1 -- 172.21.15.33:0/3013314785 wait complete. 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.386+0000 7f4df10ec700 1 Processor -- start 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.386+0000 7f4df10ec700 1 -- start start 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.386+0000 7f4df10ec700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec101d00 0x7f4dec102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.386+0000 7f4df10ec700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec1026a0 0x7f4dec1030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.386+0000 7f4df10ec700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec104b40 con 0x7f4dec1026a0 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.386+0000 7f4df10ec700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec1035e0 con 0x7f4dec101d00 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec1026a0 0x7f4dec1030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec1026a0 0x7f4dec1030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42184/0 (socket says 172.21.15.33:42184) 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dea59c700 1 -- 172.21.15.33:0/990742328 learned_addr learned my addr 172.21.15.33:0/990742328 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dead9d700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec101d00 0x7f4dec102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dea59c700 1 -- 172.21.15.33:0/990742328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec101d00 msgr2=0x7f4dec102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dea59c700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec101d00 0x7f4dec102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dea59c700 1 -- 172.21.15.33:0/990742328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc005040 con 0x7f4dec1026a0 2024-04-25T01:31:59.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dea59c700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec1026a0 0x7f4dec1030a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4de000b790 tx=0x7f4de0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:59.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dd37fe700 1 -- 172.21.15.33:0/990742328 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de000f6f0 con 0x7f4dec1026a0 2024-04-25T01:31:59.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dd37fe700 1 -- 172.21.15.33:0/990742328 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4de000fd20 con 0x7f4dec1026a0 2024-04-25T01:31:59.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec1038c0 con 0x7f4dec1026a0 2024-04-25T01:31:59.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4dd37fe700 1 -- 172.21.15.33:0/990742328 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de00103a0 con 0x7f4dec1026a0 2024-04-25T01:31:59.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.387+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec11a200 con 0x7f4dec1026a0 2024-04-25T01:31:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.388+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec0fd450 con 0x7f4dec1026a0 2024-04-25T01:31:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.389+0000 7f4dd37fe700 1 -- 172.21.15.33:0/990742328 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4de0011400 con 0x7f4dec1026a0 2024-04-25T01:31:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.389+0000 7f4dd37fe700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4dd40692a0 0x7f4dd406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:31:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.389+0000 7f4dd37fe700 1 -- 172.21.15.33:0/990742328 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4de0086c00 con 0x7f4dec1026a0 2024-04-25T01:31:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.389+0000 7f4dead9d700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4dd40692a0 0x7f4dd406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:31:59.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.390+0000 7f4dead9d700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4dd40692a0 0x7f4dd406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc004f90 tx=0x7f4ddc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:31:59.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.394+0000 7f4dd37fe700 1 -- 172.21.15.33:0/990742328 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4de008c020 con 0x7f4dec1026a0 2024-04-25T01:31:59.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:31:59 smithi033 ceph-mon[26353]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:59.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:31:59 smithi138 ceph-mon[32977]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:31:59.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.591+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4dec04c710 con 0x7f4dec1026a0 2024-04-25T01:31:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.591+0000 7f4dd37fe700 1 -- 172.21.15.33:0/990742328 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11057 (secure 0 0 0) 0x7f4de0053270 con 0x7f4dec1026a0 2024-04-25T01:31:59.568 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:31:59.568 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":40,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","created":"2024-04-25T01:26:05.674330+0000","modified":"2024-04-25T01:31:40.182361+0000","last_up_change":"2024-04-25T01:31:39.177371+0000","last_in_change":"2024-04-25T01:31:20.199327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:30:26.558656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"68af93d4-7aa8-485f-826e-62a6e362907b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6803","nonce":505851393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6805","nonce":505851393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6809","nonce":505851393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6807","nonce":505851393}]},"public_addr":"172.21.15.33:6803/505851393","cluster_addr":"172.21.15.33:6805/505851393","heartbeat_back_addr":"172.21.15.33:6809/505851393","heartbeat_front_addr":"172.21.15.33:6807/505851393","state":["exists","up"]},{"osd":1,"uuid":"1fdb7e9b-1138-43db-a11f-3e30e286c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6811","nonce":835394340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6813","nonce":835394340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6817","nonce":835394340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6815","nonce":835394340}]},"public_addr":"172.21.15.33:6811/835394340","cluster_addr":"172.21.15.33:6813/835394340","heartbeat_back_addr":"172.21.15.33:6817/835394340","heartbeat_front_addr":"172.21.15.33:6815/835394340","state":["exists","up"]},{"osd":2,"uuid":"b1239da3-566e-4df0-b5ea-df4c14959837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6819","nonce":4233054842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6821","nonce":4233054842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6825","nonce":4233054842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6823","nonce":4233054842}]},"public_addr":"172.21.15.33:6819/4233054842","cluster_addr":"172.21.15.33:6821/4233054842","heartbeat_back_addr":"172.21.15.33:6825/4233054842","heartbeat_front_addr":"172.21.15.33:6823/4233054842","state":["exists","up"]},{"osd":3,"uuid":"abc58749-77ca-4476-9601-cdf89d3c1e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6801","nonce":461435998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6803","nonce":461435998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6807","nonce":461435998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6805","nonce":461435998}]},"public_addr":"172.21.15.138:6801/461435998","cluster_addr":"172.21.15.138:6803/461435998","heartbeat_back_addr":"172.21.15.138:6807/461435998","heartbeat_front_addr":"172.21.15.138:6805/461435998","state":["exists","up"]},{"osd":4,"uuid":"83e421bd-52ba-47ad-8755-e1354f502ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6809","nonce":1618213578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6811","nonce":1618213578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6815","nonce":1618213578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6813","nonce":1618213578}]},"public_addr":"172.21.15.138:6809/1618213578","cluster_addr":"172.21.15.138:6811/1618213578","heartbeat_back_addr":"172.21.15.138:6815/1618213578","heartbeat_front_addr":"172.21.15.138:6813/1618213578","state":["exists","up"]},{"osd":5,"uuid":"266e5984-f087-408e-a84c-04f81f33af43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6817","nonce":1128834056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6819","nonce":1128834056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6823","nonce":1128834056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6821","nonce":1128834056}]},"public_addr":"172.21.15.138:6817/1128834056","cluster_addr":"172.21.15.138:6819/1128834056","heartbeat_back_addr":"172.21.15.138:6823/1128834056","heartbeat_front_addr":"172.21.15.138:6821/1128834056","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:27.636323+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:55.618887+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:23.307858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:47.331937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:12.159932+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:37.317499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2566719862":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1401659678":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/751781866":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6800/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/4160574345":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/2117172792":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/1131828704":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2112653252":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6800/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/686286320":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6801/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1869074265":"2024-04-26T01:27:11.182264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:31:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.593+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4dd40692a0 msgr2=0x7f4dd406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.593+0000 7f4df10ec700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4dd40692a0 0x7f4dd406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc004f90 tx=0x7f4ddc009e30 comp rx=0 tx=0).stop 2024-04-25T01:31:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.593+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec1026a0 msgr2=0x7f4dec1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:31:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.593+0000 7f4df10ec700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec1026a0 0x7f4dec1030a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4de000b790 tx=0x7f4de0009220 comp rx=0 tx=0).stop 2024-04-25T01:31:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.593+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 shutdown_connections 2024-04-25T01:31:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.593+0000 7f4df10ec700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4dec101d00 0x7f4dec102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.593+0000 7f4df10ec700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4dd40692a0 0x7f4dd406b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.593+0000 7f4df10ec700 1 --2- 172.21.15.33:0/990742328 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4dec1026a0 0x7f4dec1030a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:31:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.593+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 >> 172.21.15.33:0/990742328 conn(0x7f4dec0f7870 msgr2=0x7f4dec0f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:31:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.594+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 shutdown_connections 2024-04-25T01:31:59.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:31:59.594+0000 7f4df10ec700 1 -- 172.21.15.33:0/990742328 wait complete. 2024-04-25T01:32:00.303 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T01:32:00.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T01:32:00.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T01:32:00.304 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T01:32:00.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T01:32:00.305 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T01:32:00.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:00 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/990742328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:32:00.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:00 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/990742328' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:32:00.632 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:00.649 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:01.368 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:01.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:01 smithi033 ceph-mon[26353]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:01.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:32:01.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:01 smithi138 ceph-mon[32977]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:01.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:32:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:03.483 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:03 smithi033 ceph-mon[26353]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:03.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:03 smithi138 ceph-mon[32977]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.005+0000 7efe96325700 1 -- 172.21.15.33:0/2144520248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 msgr2=0x7efe900fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.005+0000 7efe96325700 1 --2- 172.21.15.33:0/2144520248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 0x7efe900fbd20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efe84004940 tx=0x7efe84009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.005+0000 7efe96325700 1 -- 172.21.15.33:0/2144520248 shutdown_connections 2024-04-25T01:32:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.005+0000 7efe96325700 1 --2- 172.21.15.33:0/2144520248 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 0x7efe900fbd20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.005+0000 7efe96325700 1 --2- 172.21.15.33:0/2144520248 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe90101d10 0x7efe901020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.005+0000 7efe96325700 1 -- 172.21.15.33:0/2144520248 >> 172.21.15.33:0/2144520248 conn(0x7efe900f7860 msgr2=0x7efe900f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.006+0000 7efe96325700 1 -- 172.21.15.33:0/2144520248 shutdown_connections 2024-04-25T01:32:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.006+0000 7efe96325700 1 -- 172.21.15.33:0/2144520248 wait complete. 2024-04-25T01:32:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.006+0000 7efe96325700 1 Processor -- start 2024-04-25T01:32:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.006+0000 7efe96325700 1 -- start start 2024-04-25T01:32:03.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.006+0000 7efe96325700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe90101d10 0x7efe9010eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe96325700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 0x7efe9010f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe96325700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9010f5d0 con 0x7efe901026b0 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe96325700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9010f710 con 0x7efe90101d10 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 0x7efe9010f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 0x7efe9010f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42206/0 (socket says 172.21.15.33:42206) 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8f7fe700 1 -- 172.21.15.33:0/880856204 learned_addr learned my addr 172.21.15.33:0/880856204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8ffff700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe90101d10 0x7efe9010eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8f7fe700 1 -- 172.21.15.33:0/880856204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe90101d10 msgr2=0x7efe9010eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8f7fe700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe90101d10 0x7efe9010eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8f7fe700 1 -- 172.21.15.33:0/880856204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84005040 con 0x7efe901026b0 2024-04-25T01:32:03.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8f7fe700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 0x7efe9010f090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efe84009100 tx=0x7efe84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8cff9700 1 -- 172.21.15.33:0/880856204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe8400e040 con 0x7efe901026b0 2024-04-25T01:32:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8cff9700 1 -- 172.21.15.33:0/880856204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe84004290 con 0x7efe901026b0 2024-04-25T01:32:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9010f990 con 0x7efe901026b0 2024-04-25T01:32:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe8cff9700 1 -- 172.21.15.33:0/880856204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe8401c3f0 con 0x7efe901026b0 2024-04-25T01:32:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.007+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe90117f60 con 0x7efe901026b0 2024-04-25T01:32:03.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.008+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe900f9180 con 0x7efe901026b0 2024-04-25T01:32:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.009+0000 7efe8cff9700 1 -- 172.21.15.33:0/880856204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7efe84013790 con 0x7efe901026b0 2024-04-25T01:32:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.009+0000 7efe8cff9700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7efe780692a0 0x7efe7806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.009+0000 7efe8cff9700 1 -- 172.21.15.33:0/880856204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7efe84087fc0 con 0x7efe901026b0 2024-04-25T01:32:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.009+0000 7efe8ffff700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7efe780692a0 0x7efe7806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:03.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.009+0000 7efe8cff9700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] conn(0x7efe7806ef70 0x7efe78071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.009+0000 7efe8cff9700 1 -- 172.21.15.33:0/880856204 --> [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe78071a30 con 0x7efe7806ef70 2024-04-25T01:32:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.009+0000 7efe8cff9700 1 -- 172.21.15.33:0/880856204 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe8408a570 con 0x7efe901026b0 2024-04-25T01:32:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.010+0000 7efe948c2700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] conn(0x7efe7806ef70 0x7efe78071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.010+0000 7efe8ffff700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7efe780692a0 0x7efe7806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efe80006fd0 tx=0x7efe80008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.010+0000 7efe948c2700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] conn(0x7efe7806ef70 0x7efe78071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.011+0000 7efe8cff9700 1 -- 172.21.15.33:0/880856204 <== osd.1 v2:172.21.15.33:6810/835394340 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7efe78071a30 con 0x7efe7806ef70 2024-04-25T01:32:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 --> [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe900fd460 con 0x7efe7806ef70 2024-04-25T01:32:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe8cff9700 1 -- 172.21.15.33:0/880856204 <== osd.1 v2:172.21.15.33:6810/835394340 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efe900fd460 con 0x7efe7806ef70 2024-04-25T01:32:04.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] conn(0x7efe7806ef70 msgr2=0x7efe78071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:04.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] conn(0x7efe7806ef70 0x7efe78071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:04.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7efe780692a0 msgr2=0x7efe7806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7efe780692a0 0x7efe7806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efe80006fd0 tx=0x7efe80008040 comp rx=0 tx=0).stop 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 msgr2=0x7efe9010f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 0x7efe9010f090 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efe84009100 tx=0x7efe84009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 shutdown_connections 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6810/835394340,v1:172.21.15.33:6811/835394340] conn(0x7efe7806ef70 0x7efe78071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe90101d10 0x7efe9010eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7efe780692a0 0x7efe7806b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 --2- 172.21.15.33:0/880856204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe901026b0 0x7efe9010f090 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 >> 172.21.15.33:0/880856204 conn(0x7efe900f7860 msgr2=0x7efe900f8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 shutdown_connections 2024-04-25T01:32:04.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:04.026+0000 7efe96325700 1 -- 172.21.15.33:0/880856204 wait complete. 2024-04-25T01:32:05.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:05 smithi033 ceph-mon[26353]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:05.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:05 smithi138 ceph-mon[32977]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:05.747 INFO:teuthology.orchestra.run.smithi033.stdout:60129542172 2024-04-25T01:32:05.747 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.081+0000 7ffa98afa700 1 -- 172.21.15.33:0/467344837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94101cd0 msgr2=0x7ffa941020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.081+0000 7ffa98afa700 1 --2- 172.21.15.33:0/467344837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94101cd0 0x7ffa941020a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c004940 tx=0x7ffa8c009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.081+0000 7ffa98afa700 1 -- 172.21.15.33:0/467344837 shutdown_connections 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.081+0000 7ffa98afa700 1 --2- 172.21.15.33:0/467344837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa94102670 0x7ffa940fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.081+0000 7ffa98afa700 1 --2- 172.21.15.33:0/467344837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94101cd0 0x7ffa941020a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.081+0000 7ffa98afa700 1 -- 172.21.15.33:0/467344837 >> 172.21.15.33:0/467344837 conn(0x7ffa94071460 msgr2=0x7ffa94071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.081+0000 7ffa98afa700 1 -- 172.21.15.33:0/467344837 shutdown_connections 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.081+0000 7ffa98afa700 1 -- 172.21.15.33:0/467344837 wait complete. 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.082+0000 7ffa98afa700 1 Processor -- start 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.082+0000 7ffa98afa700 1 -- start start 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.082+0000 7ffa98afa700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa94101cd0 0x7ffa94113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:07.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.082+0000 7ffa98afa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94102670 0x7ffa94114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.082+0000 7ffa98afa700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa94114670 con 0x7ffa94102670 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.082+0000 7ffa98afa700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9410da60 con 0x7ffa94101cd0 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.082+0000 7ffa91d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94102670 0x7ffa94114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.082+0000 7ffa91d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94102670 0x7ffa94114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34622/0 (socket says 172.21.15.33:34622) 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.082+0000 7ffa91d9b700 1 -- 172.21.15.33:0/2724274571 learned_addr learned my addr 172.21.15.33:0/2724274571 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa9259c700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa94101cd0 0x7ffa94113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa91d9b700 1 -- 172.21.15.33:0/2724274571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa94101cd0 msgr2=0x7ffa94113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa91d9b700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa94101cd0 0x7ffa94113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa91d9b700 1 -- 172.21.15.33:0/2724274571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa8c005040 con 0x7ffa94102670 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa91d9b700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94102670 0x7ffa94114020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0044e0 tx=0x7ffa7c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa8affd700 1 -- 172.21.15.33:0/2724274571 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa7c007cc0 con 0x7ffa94102670 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9410dce0 con 0x7ffa94102670 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa8affd700 1 -- 172.21.15.33:0/2724274571 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa7c00f040 con 0x7ffa94102670 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa8affd700 1 -- 172.21.15.33:0/2724274571 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa7c00ca50 con 0x7ffa94102670 2024-04-25T01:32:07.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.083+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9410e220 con 0x7ffa94102670 2024-04-25T01:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.084+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa9410e700 con 0x7ffa94102670 2024-04-25T01:32:07.450 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:07 smithi033 ceph-mon[26353]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.085+0000 7ffa8affd700 1 -- 172.21.15.33:0/2724274571 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ffa7c00cbb0 con 0x7ffa94102670 2024-04-25T01:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.085+0000 7ffa8affd700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa800692a0 0x7ffa8006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:07.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.085+0000 7ffa8affd700 1 -- 172.21.15.33:0/2724274571 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffa7c0868b0 con 0x7ffa94102670 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.085+0000 7ffa9259c700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa800692a0 0x7ffa8006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.085+0000 7ffa8affd700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] conn(0x7ffa8006ef70 0x7ffa80071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.085+0000 7ffa8affd700 1 -- 172.21.15.33:0/2724274571 --> [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa80071a30 con 0x7ffa8006ef70 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.085+0000 7ffa8affd700 1 -- 172.21.15.33:0/2724274571 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa7c08b220 con 0x7ffa94102670 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.085+0000 7ffa92d9d700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] conn(0x7ffa8006ef70 0x7ffa80071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.086+0000 7ffa9259c700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa800692a0 0x7ffa8006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009b30 tx=0x7ffa8c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.086+0000 7ffa92d9d700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] conn(0x7ffa8006ef70 0x7ffa80071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.087+0000 7ffa8affd700 1 -- 172.21.15.33:0/2724274571 <== osd.0 v2:172.21.15.33:6802/505851393 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ffa80071a30 con 0x7ffa8006ef70 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.103+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 --> [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa9405e660 con 0x7ffa8006ef70 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.103+0000 7ffa8affd700 1 -- 172.21.15.33:0/2724274571 <== osd.0 v2:172.21.15.33:6802/505851393 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffa9405e660 con 0x7ffa8006ef70 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.103+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] conn(0x7ffa8006ef70 msgr2=0x7ffa80071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.103+0000 7ffa98afa700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] conn(0x7ffa8006ef70 0x7ffa80071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa800692a0 msgr2=0x7ffa8006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa800692a0 0x7ffa8006b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009b30 tx=0x7ffa8c009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:07.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94102670 msgr2=0x7ffa94114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:07.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94102670 0x7ffa94114020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0044e0 tx=0x7ffa7c00b040 comp rx=0 tx=0).stop 2024-04-25T01:32:07.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 shutdown_connections 2024-04-25T01:32:07.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6802/505851393,v1:172.21.15.33:6803/505851393] conn(0x7ffa8006ef70 0x7ffa80071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa94101cd0 0x7ffa94113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa800692a0 0x7ffa8006b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 --2- 172.21.15.33:0/2724274571 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa94102670 0x7ffa94114020 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:07.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 >> 172.21.15.33:0/2724274571 conn(0x7ffa94071460 msgr2=0x7ffa940f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:07.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 shutdown_connections 2024-04-25T01:32:07.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:07.104+0000 7ffa98afa700 1 -- 172.21.15.33:0/2724274571 wait complete. 2024-04-25T01:32:07.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:07 smithi138 ceph-mon[32977]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 -- 172.21.15.33:0/1091460756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 msgr2=0x7ff1180fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1091460756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 0x7ff1180fbcf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff108004940 tx=0x7ff108009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 -- 172.21.15.33:0/1091460756 shutdown_connections 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1091460756 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 0x7ff1180fbcf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1091460756 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff118101ce0 0x7ff1181020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 -- 172.21.15.33:0/1091460756 >> 172.21.15.33:0/1091460756 conn(0x7ff1180f7890 msgr2=0x7ff1180f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 -- 172.21.15.33:0/1091460756 shutdown_connections 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 -- 172.21.15.33:0/1091460756 wait complete. 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 Processor -- start 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 -- start start 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.640+0000 7ff11ff15700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff118101ce0 0x7ff11810eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:08.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11ff15700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 0x7ff11810f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11ff15700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11810f720 con 0x7ff118102680 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11ff15700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11810f860 con 0x7ff118101ce0 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11d4b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 0x7ff11810f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11d4b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 0x7ff11810f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34632/0 (socket says 172.21.15.33:34632) 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11d4b0700 1 -- 172.21.15.33:0/1860869136 learned_addr learned my addr 172.21.15.33:0/1860869136 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11dcb1700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff118101ce0 0x7ff11810eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11d4b0700 1 -- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff118101ce0 msgr2=0x7ff11810eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11d4b0700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff118101ce0 0x7ff11810eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11d4b0700 1 -- 172.21.15.33:0/1860869136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff108005040 con 0x7ff118102680 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11dcb1700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff118101ce0 0x7ff11810eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:32:08.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11d4b0700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 0x7ff11810f1e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff108009100 tx=0x7ff108009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff10e7fc700 1 -- 172.21.15.33:0/1860869136 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff10800e040 con 0x7ff118102680 2024-04-25T01:32:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff10e7fc700 1 -- 172.21.15.33:0/1860869136 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff108013420 con 0x7ff118102680 2024-04-25T01:32:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff118117b50 con 0x7ff118102680 2024-04-25T01:32:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff10e7fc700 1 -- 172.21.15.33:0/1860869136 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff10801c610 con 0x7ff118102680 2024-04-25T01:32:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.641+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1181180b0 con 0x7ff118102680 2024-04-25T01:32:08.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.642+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1180f9230 con 0x7ff118102680 2024-04-25T01:32:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.643+0000 7ff10e7fc700 1 -- 172.21.15.33:0/1860869136 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff108013590 con 0x7ff118102680 2024-04-25T01:32:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.643+0000 7ff10e7fc700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff1040692a0 0x7ff10406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.643+0000 7ff11dcb1700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff1040692a0 0x7ff10406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.643+0000 7ff10e7fc700 1 -- 172.21.15.33:0/1860869136 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff1080880e0 con 0x7ff118102680 2024-04-25T01:32:08.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.644+0000 7ff10e7fc700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] conn(0x7ff10406ef70 0x7ff104071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.644+0000 7ff10e7fc700 1 -- 172.21.15.33:0/1860869136 --> [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff104071a30 con 0x7ff10406ef70 2024-04-25T01:32:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.644+0000 7ff10e7fc700 1 -- 172.21.15.33:0/1860869136 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff10808a6d0 con 0x7ff118102680 2024-04-25T01:32:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.644+0000 7ff11dcb1700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff1040692a0 0x7ff10406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1140041d0 tx=0x7ff114009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.644+0000 7ff11e4b2700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] conn(0x7ff10406ef70 0x7ff104071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.644+0000 7ff11e4b2700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] conn(0x7ff10406ef70 0x7ff104071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:08.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.645+0000 7ff10e7fc700 1 -- 172.21.15.33:0/1860869136 <== osd.3 v2:172.21.15.138:6800/461435998 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff104071a30 con 0x7ff10406ef70 2024-04-25T01:32:08.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.660+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 --> [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff1180fd430 con 0x7ff10406ef70 2024-04-25T01:32:08.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff10e7fc700 1 -- 172.21.15.33:0/1860869136 <== osd.3 v2:172.21.15.138:6800/461435998 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff1180fd430 con 0x7ff10406ef70 2024-04-25T01:32:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] conn(0x7ff10406ef70 msgr2=0x7ff104071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] conn(0x7ff10406ef70 0x7ff104071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff1040692a0 msgr2=0x7ff10406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff1040692a0 0x7ff10406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff1140041d0 tx=0x7ff114009040 comp rx=0 tx=0).stop 2024-04-25T01:32:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 msgr2=0x7ff11810f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 0x7ff11810f1e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff108009100 tx=0x7ff108009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 shutdown_connections 2024-04-25T01:32:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:6800/461435998,v1:172.21.15.138:6801/461435998] conn(0x7ff10406ef70 0x7ff104071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff118101ce0 0x7ff11810eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff1040692a0 0x7ff10406b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 --2- 172.21.15.33:0/1860869136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff118102680 0x7ff11810f1e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 >> 172.21.15.33:0/1860869136 conn(0x7ff1180f7890 msgr2=0x7ff1180f8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 shutdown_connections 2024-04-25T01:32:08.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.661+0000 7ff11ff15700 1 -- 172.21.15.33:0/1860869136 wait complete. 2024-04-25T01:32:08.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.864+0000 7fc7ac940700 1 -- 172.21.15.33:0/4260620225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 msgr2=0x7fc7a80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.864+0000 7fc7ac940700 1 --2- 172.21.15.33:0/4260620225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 0x7fc7a80fcdf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc79c004940 tx=0x7fc79c009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.865+0000 7fc7ac940700 1 -- 172.21.15.33:0/4260620225 shutdown_connections 2024-04-25T01:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.865+0000 7fc7ac940700 1 --2- 172.21.15.33:0/4260620225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 0x7fc7a80fcdf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.865+0000 7fc7ac940700 1 --2- 172.21.15.33:0/4260620225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8102de0 0x7fc7a81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.865+0000 7fc7ac940700 1 -- 172.21.15.33:0/4260620225 >> 172.21.15.33:0/4260620225 conn(0x7fc7a80f8990 msgr2=0x7fc7a80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.865+0000 7fc7ac940700 1 -- 172.21.15.33:0/4260620225 shutdown_connections 2024-04-25T01:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.865+0000 7fc7ac940700 1 -- 172.21.15.33:0/4260620225 wait complete. 2024-04-25T01:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.865+0000 7fc7ac940700 1 Processor -- start 2024-04-25T01:32:08.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.865+0000 7fc7ac940700 1 -- start start 2024-04-25T01:32:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7ac940700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8102de0 0x7fc7a8113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7ac940700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 0x7fc7a8113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7ac940700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a81144e0 con 0x7fc7a8103780 2024-04-25T01:32:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7ac940700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a810dae0 con 0x7fc7a8102de0 2024-04-25T01:32:08.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7a5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 0x7fc7a8113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7a5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 0x7fc7a8113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34654/0 (socket says 172.21.15.33:34654) 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7a5d9b700 1 -- 172.21.15.33:0/3425202063 learned_addr learned my addr 172.21.15.33:0/3425202063 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7a659c700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8102de0 0x7fc7a8113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7a5d9b700 1 -- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8102de0 msgr2=0x7fc7a8113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7a5d9b700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8102de0 0x7fc7a8113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7a5d9b700 1 -- 172.21.15.33:0/3425202063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79c005040 con 0x7fc7a8103780 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7a659c700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8102de0 0x7fc7a8113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7a5d9b700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 0x7fc7a8113e90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009100 tx=0x7fc79c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc796ffd700 1 -- 172.21.15.33:0/3425202063 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc79c00e040 con 0x7fc7a8103780 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc796ffd700 1 -- 172.21.15.33:0/3425202063 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc79c013420 con 0x7fc7a8103780 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a810dd60 con 0x7fc7a8103780 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc796ffd700 1 -- 172.21.15.33:0/3425202063 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc79c01c630 con 0x7fc7a8103780 2024-04-25T01:32:08.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.866+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a810e320 con 0x7fc7a8103780 2024-04-25T01:32:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.867+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7a810e850 con 0x7fc7a8103780 2024-04-25T01:32:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.868+0000 7fc796ffd700 1 -- 172.21.15.33:0/3425202063 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc79c013590 con 0x7fc7a8103780 2024-04-25T01:32:08.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.868+0000 7fc796ffd700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc7900692a0 0x7fc79006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.868+0000 7fc796ffd700 1 -- 172.21.15.33:0/3425202063 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc79c088a50 con 0x7fc7a8103780 2024-04-25T01:32:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.868+0000 7fc7a659c700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc7900692a0 0x7fc79006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.869+0000 7fc796ffd700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] conn(0x7fc79006ef70 0x7fc790071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.869+0000 7fc796ffd700 1 -- 172.21.15.33:0/3425202063 --> [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc790071a30 con 0x7fc79006ef70 2024-04-25T01:32:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.869+0000 7fc796ffd700 1 -- 172.21.15.33:0/3425202063 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc79c088e50 con 0x7fc7a8103780 2024-04-25T01:32:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.869+0000 7fc7a659c700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc7900692a0 0x7fc79006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc79800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.869+0000 7fc7a6d9d700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] conn(0x7fc79006ef70 0x7fc790071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.869+0000 7fc7a6d9d700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] conn(0x7fc79006ef70 0x7fc790071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:08.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.870+0000 7fc796ffd700 1 -- 172.21.15.33:0/3425202063 <== osd.5 v2:172.21.15.138:6816/1128834056 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fc790071a30 con 0x7fc79006ef70 2024-04-25T01:32:08.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.886+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 --> [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc7a80fa2e0 con 0x7fc79006ef70 2024-04-25T01:32:08.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.886+0000 7fc796ffd700 1 -- 172.21.15.33:0/3425202063 <== osd.5 v2:172.21.15.138:6816/1128834056 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc7a80fa2e0 con 0x7fc79006ef70 2024-04-25T01:32:08.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] conn(0x7fc79006ef70 msgr2=0x7fc790071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] conn(0x7fc79006ef70 0x7fc790071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc7900692a0 msgr2=0x7fc79006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc7900692a0 0x7fc79006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc798009fd0 tx=0x7fc79800b040 comp rx=0 tx=0).stop 2024-04-25T01:32:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 msgr2=0x7fc7a8113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 0x7fc7a8113e90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc79c009100 tx=0x7fc79c009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:08.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 shutdown_connections 2024-04-25T01:32:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:6816/1128834056,v1:172.21.15.138:6817/1128834056] conn(0x7fc79006ef70 0x7fc790071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc7a8102de0 0x7fc7a8113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc7900692a0 0x7fc79006b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 --2- 172.21.15.33:0/3425202063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc7a8103780 0x7fc7a8113e90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 >> 172.21.15.33:0/3425202063 conn(0x7fc7a80f8990 msgr2=0x7fc7a80f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 shutdown_connections 2024-04-25T01:32:08.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:08.887+0000 7fc7ac940700 1 -- 172.21.15.33:0/3425202063 wait complete. 2024-04-25T01:32:09.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:09 smithi033 ceph-mon[26353]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:09.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:09 smithi138 ceph-mon[32977]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:09.920 INFO:teuthology.orchestra.run.smithi033.stdout:111669149714 2024-04-25T01:32:09.921 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T01:32:10.282 INFO:teuthology.orchestra.run.smithi033.stdout:38654705698 2024-04-25T01:32:10.282 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T01:32:11.162 INFO:teuthology.orchestra.run.smithi033.stdout:167503724553 2024-04-25T01:32:11.163 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T01:32:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.455+0000 7f1f79791700 1 -- 172.21.15.33:0/3763303952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 msgr2=0x7f1f740fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.455+0000 7f1f79791700 1 --2- 172.21.15.33:0/3763303952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 0x7f1f740fb600 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1f68004940 tx=0x7f1f68009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.455+0000 7f1f79791700 1 -- 172.21.15.33:0/3763303952 shutdown_connections 2024-04-25T01:32:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.455+0000 7f1f79791700 1 --2- 172.21.15.33:0/3763303952 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 0x7f1f740fb600 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.455+0000 7f1f79791700 1 --2- 172.21.15.33:0/3763303952 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74101cd0 0x7f1f741020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.455+0000 7f1f79791700 1 -- 172.21.15.33:0/3763303952 >> 172.21.15.33:0/3763303952 conn(0x7f1f74071460 msgr2=0x7f1f74071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.456+0000 7f1f79791700 1 -- 172.21.15.33:0/3763303952 shutdown_connections 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.456+0000 7f1f79791700 1 -- 172.21.15.33:0/3763303952 wait complete. 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f79791700 1 Processor -- start 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f79791700 1 -- start start 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f79791700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74101cd0 0x7f1f740fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f79791700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 0x7f1f740fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f79791700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740fd2b0 con 0x7f1f74102670 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f79791700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740fd3f0 con 0x7f1f74101cd0 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f727fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 0x7f1f740fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f727fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 0x7f1f740fcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34668/0 (socket says 172.21.15.33:34668) 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f727fc700 1 -- 172.21.15.33:0/1792551911 learned_addr learned my addr 172.21.15.33:0/1792551911 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f72ffd700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74101cd0 0x7f1f740fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:11.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f727fc700 1 -- 172.21.15.33:0/1792551911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74101cd0 msgr2=0x7f1f740fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f727fc700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74101cd0 0x7f1f740fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.457+0000 7f1f727fc700 1 -- 172.21.15.33:0/1792551911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f68005040 con 0x7f1f74102670 2024-04-25T01:32:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.458+0000 7f1f727fc700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 0x7f1f740fcd20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f68000c00 tx=0x7f1f68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.458+0000 7f1f5b7fe700 1 -- 172.21.15.33:0/1792551911 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f68021070 con 0x7f1f74102670 2024-04-25T01:32:11.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.458+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740fd640 con 0x7f1f74102670 2024-04-25T01:32:11.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.458+0000 7f1f5b7fe700 1 -- 172.21.15.33:0/1792551911 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f68013c00 con 0x7f1f74102670 2024-04-25T01:32:11.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.458+0000 7f1f5b7fe700 1 -- 172.21.15.33:0/1792551911 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f6800e070 con 0x7f1f74102670 2024-04-25T01:32:11.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.458+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7411a1a0 con 0x7f1f74102670 2024-04-25T01:32:11.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.459+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1f740fa2f0 con 0x7f1f74102670 2024-04-25T01:32:11.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.460+0000 7f1f5b7fe700 1 -- 172.21.15.33:0/1792551911 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1f6801c400 con 0x7f1f74102670 2024-04-25T01:32:11.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.460+0000 7f1f5b7fe700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1f5c0692a0 0x7f1f5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:11.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.460+0000 7f1f5b7fe700 1 -- 172.21.15.33:0/1792551911 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1f68088240 con 0x7f1f74102670 2024-04-25T01:32:11.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.460+0000 7f1f72ffd700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1f5c0692a0 0x7f1f5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:11.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.460+0000 7f1f5b7fe700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] conn(0x7f1f5c06ef70 0x7f1f5c071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:11.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.460+0000 7f1f5b7fe700 1 -- 172.21.15.33:0/1792551911 --> [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1f5c071a30 con 0x7f1f5c06ef70 2024-04-25T01:32:11.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.460+0000 7f1f5b7fe700 1 -- 172.21.15.33:0/1792551911 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f6808a830 con 0x7f1f74102670 2024-04-25T01:32:11.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.461+0000 7f1f737fe700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] conn(0x7f1f5c06ef70 0x7f1f5c071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:11.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.461+0000 7f1f72ffd700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1f5c0692a0 0x7f1f5c06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009b80 tx=0x7f1f6400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:11.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.461+0000 7f1f737fe700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] conn(0x7f1f5c06ef70 0x7f1f5c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:11.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.462+0000 7f1f5b7fe700 1 -- 172.21.15.33:0/1792551911 <== osd.2 v2:172.21.15.33:6818/4233054842 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f1f5c071a30 con 0x7f1f5c06ef70 2024-04-25T01:32:11.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.480+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 --> [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1f74065200 con 0x7f1f5c06ef70 2024-04-25T01:32:11.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.480+0000 7f1f5b7fe700 1 -- 172.21.15.33:0/1792551911 <== osd.2 v2:172.21.15.33:6818/4233054842 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1f74065200 con 0x7f1f5c06ef70 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.480+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] conn(0x7f1f5c06ef70 msgr2=0x7f1f5c071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.480+0000 7f1f79791700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] conn(0x7f1f5c06ef70 0x7f1f5c071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1f5c0692a0 msgr2=0x7f1f5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1f5c0692a0 0x7f1f5c06b750 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009b80 tx=0x7f1f6400b040 comp rx=0 tx=0).stop 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 msgr2=0x7f1f740fcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 0x7f1f740fcd20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1f68000c00 tx=0x7f1f68009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 shutdown_connections 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f74101cd0 0x7f1f740fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6818/4233054842,v1:172.21.15.33:6819/4233054842] conn(0x7f1f5c06ef70 0x7f1f5c071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1f5c0692a0 0x7f1f5c06b750 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 --2- 172.21.15.33:0/1792551911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f74102670 0x7f1f740fcd20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 >> 172.21.15.33:0/1792551911 conn(0x7f1f74071460 msgr2=0x7f1f740f9920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 shutdown_connections 2024-04-25T01:32:11.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:11.481+0000 7f1f79791700 1 -- 172.21.15.33:0/1792551911 wait complete. 2024-04-25T01:32:11.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:11 smithi033 ceph-mon[26353]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:11.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:11 smithi138 ceph-mon[32977]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:12.492 INFO:teuthology.orchestra.run.smithi033.stdout:81604378648 2024-04-25T01:32:12.492 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T01:32:13.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:13 smithi033 ceph-mon[26353]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:13.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:13 smithi138 ceph-mon[32977]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:14.062 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.518+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1316942595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 msgr2=0x7ff4f0106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.518+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1316942595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 0x7ff4f0106dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0004940 tx=0x7ff4e0009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.518+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1316942595 shutdown_connections 2024-04-25T01:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.518+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1316942595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 0x7ff4f0106dc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.518+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1316942595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f00fdde0 0x7ff4f00fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.518+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1316942595 >> 172.21.15.33:0/1316942595 conn(0x7ff4f00f7850 msgr2=0x7ff4f00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.518+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1316942595 shutdown_connections 2024-04-25T01:32:14.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.518+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1316942595 wait complete. 2024-04-25T01:32:14.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.519+0000 7ff4f84ee700 1 Processor -- start 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.519+0000 7ff4f84ee700 1 -- start start 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.519+0000 7ff4f84ee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f00fdde0 0x7ff4f0113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.519+0000 7ff4f84ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 0x7ff4f0114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.519+0000 7ff4f84ee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f01146e0 con 0x7ff4f00fe780 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.519+0000 7ff4f84ee700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f010dad0 con 0x7ff4f00fdde0 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.520+0000 7ff4f5a89700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 0x7ff4f0114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.520+0000 7ff4f5a89700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 0x7ff4f0114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34694/0 (socket says 172.21.15.33:34694) 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.520+0000 7ff4f5a89700 1 -- 172.21.15.33:0/1805799338 learned_addr learned my addr 172.21.15.33:0/1805799338 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.520+0000 7ff4f628a700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f00fdde0 0x7ff4f0113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.520+0000 7ff4f5a89700 1 -- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f00fdde0 msgr2=0x7ff4f0113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.520+0000 7ff4f5a89700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f00fdde0 0x7ff4f0113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.520+0000 7ff4f5a89700 1 -- 172.21.15.33:0/1805799338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e0005040 con 0x7ff4f00fe780 2024-04-25T01:32:14.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.520+0000 7ff4f5a89700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 0x7ff4f0114090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009380 tx=0x7ff4e0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.520+0000 7ff4e6ffd700 1 -- 172.21.15.33:0/1805799338 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4e000e040 con 0x7ff4f00fe780 2024-04-25T01:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.521+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f010dd50 con 0x7ff4f00fe780 2024-04-25T01:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.521+0000 7ff4e6ffd700 1 -- 172.21.15.33:0/1805799338 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4e0009af0 con 0x7ff4f00fe780 2024-04-25T01:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.521+0000 7ff4e6ffd700 1 -- 172.21.15.33:0/1805799338 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4e001c630 con 0x7ff4f00fe780 2024-04-25T01:32:14.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.521+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f010e290 con 0x7ff4f00fe780 2024-04-25T01:32:14.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.521+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff4f010e870 con 0x7ff4f00fe780 2024-04-25T01:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.522+0000 7ff4e6ffd700 1 -- 172.21.15.33:0/1805799338 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff4e000bbc0 con 0x7ff4f00fe780 2024-04-25T01:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.523+0000 7ff4e6ffd700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff4dc0692a0 0x7ff4dc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.523+0000 7ff4e6ffd700 1 -- 172.21.15.33:0/1805799338 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff4e0088c00 con 0x7ff4f00fe780 2024-04-25T01:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.523+0000 7ff4f628a700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff4dc0692a0 0x7ff4dc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.523+0000 7ff4e6ffd700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] conn(0x7ff4dc06ef70 0x7ff4dc071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.523+0000 7ff4e6ffd700 1 -- 172.21.15.33:0/1805799338 --> [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff4dc071a30 con 0x7ff4dc06ef70 2024-04-25T01:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.523+0000 7ff4e6ffd700 1 -- 172.21.15.33:0/1805799338 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4e0088fe0 con 0x7ff4f00fe780 2024-04-25T01:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.523+0000 7ff4f6a8b700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] conn(0x7ff4dc06ef70 0x7ff4dc071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:14.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.523+0000 7ff4f628a700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff4dc0692a0 0x7ff4dc06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec007fd0 tx=0x7ff4ec009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.524+0000 7ff4f6a8b700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] conn(0x7ff4dc06ef70 0x7ff4dc071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:14.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.525+0000 7ff4e6ffd700 1 -- 172.21.15.33:0/1805799338 <== osd.4 v2:172.21.15.138:6808/1618213578 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7ff4dc071a30 con 0x7ff4dc06ef70 2024-04-25T01:32:14.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.540+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 --> [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff4f00f96e0 con 0x7ff4dc06ef70 2024-04-25T01:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.540+0000 7ff4e6ffd700 1 -- 172.21.15.33:0/1805799338 <== osd.4 v2:172.21.15.138:6808/1618213578 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff4f00f96e0 con 0x7ff4dc06ef70 2024-04-25T01:32:14.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.540+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] conn(0x7ff4dc06ef70 msgr2=0x7ff4dc071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.540+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] conn(0x7ff4dc06ef70 0x7ff4dc071380 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff4dc0692a0 msgr2=0x7ff4dc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff4dc0692a0 0x7ff4dc06b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec007fd0 tx=0x7ff4ec009040 comp rx=0 tx=0).stop 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 msgr2=0x7ff4f0114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 0x7ff4f0114090 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff4e0009380 tx=0x7ff4e0009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 shutdown_connections 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:6808/1618213578,v1:172.21.15.138:6809/1618213578] conn(0x7ff4dc06ef70 0x7ff4dc071380 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4f00fdde0 0x7ff4f0113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff4dc0692a0 0x7ff4dc06b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 --2- 172.21.15.33:0/1805799338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4f00fe780 0x7ff4f0114090 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 >> 172.21.15.33:0/1805799338 conn(0x7ff4f00f7850 msgr2=0x7ff4f00f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 shutdown_connections 2024-04-25T01:32:14.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.541+0000 7ff4f84ee700 1 -- 172.21.15.33:0/1805799338 wait complete. 2024-04-25T01:32:15.361 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:15 smithi033 ceph-mon[26353]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:15.361 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:15 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1876292268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T01:32:15.385 INFO:teuthology.orchestra.run.smithi033.stdout:146028888079 2024-04-25T01:32:15.386 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T01:32:15.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:15 smithi138 ceph-mon[32977]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:15.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:15 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1876292268' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T01:32:15.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.755+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1735192441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 msgr2=0x7ff8c80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:15.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.755+0000 7ff8cd12e700 1 --2- 172.21.15.33:0/1735192441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 0x7ff8c80fcdf0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc004940 tx=0x7ff8bc009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:15.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.755+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1735192441 shutdown_connections 2024-04-25T01:32:15.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.755+0000 7ff8cd12e700 1 --2- 172.21.15.33:0/1735192441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 0x7ff8c80fcdf0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:15.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.755+0000 7ff8cd12e700 1 --2- 172.21.15.33:0/1735192441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102de0 0x7ff8c81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:15.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.755+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1735192441 >> 172.21.15.33:0/1735192441 conn(0x7ff8c80f8990 msgr2=0x7ff8c80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:15.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.755+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1735192441 shutdown_connections 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.755+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1735192441 wait complete. 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8cd12e700 1 Processor -- start 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8cd12e700 1 -- start start 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8cd12e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102de0 0x7ff8c81138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8cd12e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 0x7ff8c8113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8cd12e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8114460 con 0x7ff8c8103780 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8cd12e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c810da60 con 0x7ff8c8102de0 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 0x7ff8c8113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8c659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 0x7ff8c8113e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34698/0 (socket says 172.21.15.33:34698) 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8c659c700 1 -- 172.21.15.33:0/1876292268 learned_addr learned my addr 172.21.15.33:0/1876292268 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:15.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.756+0000 7ff8c6d9d700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102de0 0x7ff8c81138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stdout:60129542174 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.757+0000 7ff8c659c700 1 -- 172.21.15.33:0/1876292268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102de0 msgr2=0x7ff8c81138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.757+0000 7ff8c659c700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102de0 0x7ff8c81138d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.757+0000 7ff8c659c700 1 -- 172.21.15.33:0/1876292268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc005040 con 0x7ff8c8103780 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.757+0000 7ff8c659c700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 0x7ff8c8113e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc000c00 tx=0x7ff8bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.757+0000 7ff8af7fe700 1 -- 172.21.15.33:0/1876292268 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8bc021070 con 0x7ff8c8103780 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.757+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c810dce0 con 0x7ff8c8103780 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.757+0000 7ff8af7fe700 1 -- 172.21.15.33:0/1876292268 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8bc013c00 con 0x7ff8c8103780 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.757+0000 7ff8af7fe700 1 -- 172.21.15.33:0/1876292268 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8bc00e070 con 0x7ff8c8103780 2024-04-25T01:32:15.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.757+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c810e1c0 con 0x7ff8c8103780 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.758+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c80fe530 con 0x7ff8c8103780 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.758+0000 7ff8af7fe700 1 -- 172.21.15.33:0/1876292268 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff8bc01ca60 con 0x7ff8c8103780 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.759+0000 7ff8af7fe700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff8b00692a0 0x7ff8b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.759+0000 7ff8c6d9d700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff8b00692a0 0x7ff8b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.759+0000 7ff8af7fe700 1 -- 172.21.15.33:0/1876292268 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ff8bc0882b0 con 0x7ff8c8103780 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.759+0000 7ff8c6d9d700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff8b00692a0 0x7ff8b006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009b80 tx=0x7ff8b800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.763+0000 7ff8af7fe700 1 -- 172.21.15.33:0/1876292268 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff8bc0583b0 con 0x7ff8c8103780 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.965+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff8c8109ec0 con 0x7ff8c8103780 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.965+0000 7ff8af7fe700 1 -- 172.21.15.33:0/1876292268 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff8bc004490 con 0x7ff8c8103780 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff8b00692a0 msgr2=0x7ff8b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff8b00692a0 0x7ff8b006b750 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009b80 tx=0x7ff8b800b040 comp rx=0 tx=0).stop 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 msgr2=0x7ff8c8113e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 0x7ff8c8113e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc000c00 tx=0x7ff8bc009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:15.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 shutdown_connections 2024-04-25T01:32:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102de0 0x7ff8c81138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff8b00692a0 0x7ff8b006b750 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 --2- 172.21.15.33:0/1876292268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8103780 0x7ff8c8113e10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 >> 172.21.15.33:0/1876292268 conn(0x7ff8c80f8990 msgr2=0x7ff8c80f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 shutdown_connections 2024-04-25T01:32:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:14.967+0000 7ff8cd12e700 1 -- 172.21.15.33:0/1876292268 wait complete. 2024-04-25T01:32:16.114 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542172 got 60129542174 for osd.1 2024-04-25T01:32:16.114 DEBUG:teuthology.parallel:result is None 2024-04-25T01:32:16.136 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:16.137 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:16.342 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:32:16.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:32:16.782 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:17.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:17 smithi033 ceph-mon[26353]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:17.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:17 smithi138 ceph-mon[32977]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:19.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:19 smithi033 ceph-mon[26353]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:19.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:19 smithi138 ceph-mon[32977]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.616+0000 7feba5010700 1 -- 172.21.15.33:0/610932018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba00fbd40 msgr2=0x7feba00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.616+0000 7feba5010700 1 --2- 172.21.15.33:0/610932018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba00fbd40 0x7feba00fc1b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feb94004940 tx=0x7feb94009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.616+0000 7feba5010700 1 -- 172.21.15.33:0/610932018 shutdown_connections 2024-04-25T01:32:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.616+0000 7feba5010700 1 --2- 172.21.15.33:0/610932018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba00fbd40 0x7feba00fc1b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.616+0000 7feba5010700 1 --2- 172.21.15.33:0/610932018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba0101d90 0x7feba0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.616+0000 7feba5010700 1 -- 172.21.15.33:0/610932018 >> 172.21.15.33:0/610932018 conn(0x7feba0071d00 msgr2=0x7feba0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.616+0000 7feba5010700 1 -- 172.21.15.33:0/610932018 shutdown_connections 2024-04-25T01:32:20.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.616+0000 7feba5010700 1 -- 172.21.15.33:0/610932018 wait complete. 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feba5010700 1 Processor -- start 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feba5010700 1 -- start start 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feba5010700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00fbd40 0x7feba010ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feba5010700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba0101d90 0x7feba010f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feba5010700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba010f700 con 0x7feba0101d90 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feba5010700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba010f840 con 0x7feba00fbd40 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feb9e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba0101d90 0x7feba010f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feb9e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba0101d90 0x7feba010f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59922/0 (socket says 172.21.15.33:59922) 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feb9e59c700 1 -- 172.21.15.33:0/2917597171 learned_addr learned my addr 172.21.15.33:0/2917597171 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.617+0000 7feb9ed9d700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00fbd40 0x7feba010ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:20.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.618+0000 7feb9e59c700 1 -- 172.21.15.33:0/2917597171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00fbd40 msgr2=0x7feba010ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.618+0000 7feb9e59c700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00fbd40 0x7feba010ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.618+0000 7feb9e59c700 1 -- 172.21.15.33:0/2917597171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb94005040 con 0x7feba0101d90 2024-04-25T01:32:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.618+0000 7feb9e59c700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba0101d90 0x7feba010f1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feb94009b30 tx=0x7feb94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.618+0000 7feb877fe700 1 -- 172.21.15.33:0/2917597171 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb94021070 con 0x7feba0101d90 2024-04-25T01:32:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.618+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba0117b30 con 0x7feba0101d90 2024-04-25T01:32:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.618+0000 7feb877fe700 1 -- 172.21.15.33:0/2917597171 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb94013c00 con 0x7feba0101d90 2024-04-25T01:32:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.618+0000 7feb877fe700 1 -- 172.21.15.33:0/2917597171 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9400e070 con 0x7feba0101d90 2024-04-25T01:32:20.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.618+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba0117fc0 con 0x7feba0101d90 2024-04-25T01:32:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.619+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba00fd430 con 0x7feba0101d90 2024-04-25T01:32:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.620+0000 7feb877fe700 1 -- 172.21.15.33:0/2917597171 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7feb9401cb60 con 0x7feba0101d90 2024-04-25T01:32:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.620+0000 7feb877fe700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb880692a0 0x7feb8806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.620+0000 7feb877fe700 1 -- 172.21.15.33:0/2917597171 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7feb940882d0 con 0x7feba0101d90 2024-04-25T01:32:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.620+0000 7feb9ed9d700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb880692a0 0x7feb8806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:20.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.620+0000 7feb9ed9d700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb880692a0 0x7feb8806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb90001470 tx=0x7feb9000b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:20.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.625+0000 7feb877fe700 1 -- 172.21.15.33:0/2917597171 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feb94054940 con 0x7feba0101d90 2024-04-25T01:32:20.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.832+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7feba005f5c0 con 0x7feba0101d90 2024-04-25T01:32:20.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.833+0000 7feb877fe700 1 -- 172.21.15.33:0/2917597171 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feb94004490 con 0x7feba0101d90 2024-04-25T01:32:20.811 INFO:teuthology.orchestra.run.smithi033.stdout:167503724555 2024-04-25T01:32:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb880692a0 msgr2=0x7feb8806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb880692a0 0x7feb8806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb90001470 tx=0x7feb9000b040 comp rx=0 tx=0).stop 2024-04-25T01:32:20.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba0101d90 msgr2=0x7feba010f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:20.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba0101d90 0x7feba010f1c0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feb94009b30 tx=0x7feb94009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:20.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 shutdown_connections 2024-04-25T01:32:20.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feba00fbd40 0x7feba010ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:20.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb880692a0 0x7feb8806b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:20.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 --2- 172.21.15.33:0/2917597171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feba0101d90 0x7feba010f1c0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:20.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 >> 172.21.15.33:0/2917597171 conn(0x7feba0071d00 msgr2=0x7feba00f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:20.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 shutdown_connections 2024-04-25T01:32:20.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:20.835+0000 7feba5010700 1 -- 172.21.15.33:0/2917597171 wait complete. 2024-04-25T01:32:21.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.467+0000 7f5b63a43700 1 -- 172.21.15.33:0/3688467926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c102df0 msgr2=0x7f5b5c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.467+0000 7f5b63a43700 1 --2- 172.21.15.33:0/3688467926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c102df0 0x7f5b5c1031c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c004940 tx=0x7f5b4c009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.467+0000 7f5b63a43700 1 -- 172.21.15.33:0/3688467926 shutdown_connections 2024-04-25T01:32:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.467+0000 7f5b63a43700 1 --2- 172.21.15.33:0/3688467926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b5c103790 0x7f5b5c0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.467+0000 7f5b63a43700 1 --2- 172.21.15.33:0/3688467926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c102df0 0x7f5b5c1031c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.467+0000 7f5b63a43700 1 -- 172.21.15.33:0/3688467926 >> 172.21.15.33:0/3688467926 conn(0x7f5b5c0f8980 msgr2=0x7f5b5c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.467+0000 7f5b63a43700 1 -- 172.21.15.33:0/3688467926 shutdown_connections 2024-04-25T01:32:21.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.467+0000 7f5b63a43700 1 -- 172.21.15.33:0/3688467926 wait complete. 2024-04-25T01:32:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b63a43700 1 Processor -- start 2024-04-25T01:32:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b63a43700 1 -- start start 2024-04-25T01:32:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b63a43700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b5c102df0 0x7f5b5c113c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b63a43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c103790 0x7f5b5c114140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b63a43700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c114790 con 0x7f5b5c103790 2024-04-25T01:32:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b63a43700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b5c10db80 con 0x7f5b5c102df0 2024-04-25T01:32:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b60fde700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c103790 0x7f5b5c114140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:21.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b617df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b5c102df0 0x7f5b5c113c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b617df700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b5c102df0 0x7f5b5c113c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:48656/0 (socket says 172.21.15.33:48656) 2024-04-25T01:32:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b60fde700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c103790 0x7f5b5c114140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59938/0 (socket says 172.21.15.33:59938) 2024-04-25T01:32:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b617df700 1 -- 172.21.15.33:0/734522559 learned_addr learned my addr 172.21.15.33:0/734522559 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b60fde700 1 -- 172.21.15.33:0/734522559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b5c102df0 msgr2=0x7f5b5c113c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b60fde700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b5c102df0 0x7f5b5c113c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b60fde700 1 -- 172.21.15.33:0/734522559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b4c005040 con 0x7f5b5c103790 2024-04-25T01:32:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b617df700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b5c102df0 0x7f5b5c113c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:32:21.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.468+0000 7f5b60fde700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c103790 0x7f5b5c114140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b50002840 tx=0x7f5b50004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.469+0000 7f5b59ffb700 1 -- 172.21.15.33:0/734522559 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b50007cc0 con 0x7f5b5c103790 2024-04-25T01:32:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.469+0000 7f5b59ffb700 1 -- 172.21.15.33:0/734522559 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b5000f040 con 0x7f5b5c103790 2024-04-25T01:32:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.469+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b5c10de60 con 0x7f5b5c103790 2024-04-25T01:32:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.469+0000 7f5b59ffb700 1 -- 172.21.15.33:0/734522559 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b50013a10 con 0x7f5b5c103790 2024-04-25T01:32:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.469+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b5c10e420 con 0x7f5b5c103790 2024-04-25T01:32:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.470+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c100100 con 0x7f5b5c103790 2024-04-25T01:32:21.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.470+0000 7f5b59ffb700 1 -- 172.21.15.33:0/734522559 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5b5001c400 con 0x7f5b5c103790 2024-04-25T01:32:21.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.471+0000 7f5b59ffb700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b440692a0 0x7f5b4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:21.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.471+0000 7f5b59ffb700 1 -- 172.21.15.33:0/734522559 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5b50086480 con 0x7f5b5c103790 2024-04-25T01:32:21.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.471+0000 7f5b617df700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b440692a0 0x7f5b4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:21.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.471+0000 7f5b617df700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b440692a0 0x7f5b4406b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009fd0 tx=0x7f5b4c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:21.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.475+0000 7f5b59ffb700 1 -- 172.21.15.33:0/734522559 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5b5004d250 con 0x7f5b5c103790 2024-04-25T01:32:21.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:21 smithi033 ceph-mon[26353]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:21.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:21 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2917597171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T01:32:21.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:21 smithi138 ceph-mon[32977]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:21.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:21 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2917597171' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T01:32:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.674+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5b5c10f1e0 con 0x7f5b5c103790 2024-04-25T01:32:21.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.675+0000 7f5b59ffb700 1 -- 172.21.15.33:0/734522559 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5b50056500 con 0x7f5b5c103790 2024-04-25T01:32:21.653 INFO:teuthology.orchestra.run.smithi033.stdout:38654705700 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b440692a0 msgr2=0x7f5b4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b440692a0 0x7f5b4406b750 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c009fd0 tx=0x7f5b4c009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c103790 msgr2=0x7f5b5c114140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c103790 0x7f5b5c114140 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5b50002840 tx=0x7f5b50004570 comp rx=0 tx=0).stop 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 shutdown_connections 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b5c102df0 0x7f5b5c113c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b440692a0 0x7f5b4406b750 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 --2- 172.21.15.33:0/734522559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b5c103790 0x7f5b5c114140 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 >> 172.21.15.33:0/734522559 conn(0x7f5b5c0f8980 msgr2=0x7f5b5c0facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 shutdown_connections 2024-04-25T01:32:21.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:21.677+0000 7f5b63a43700 1 -- 172.21.15.33:0/734522559 wait complete. 2024-04-25T01:32:22.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:22 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/734522559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T01:32:22.504 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:22 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/734522559' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T01:32:23.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:23 smithi033 ceph-mon[26353]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:23.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:23 smithi138 ceph-mon[32977]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:23.900 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724555 for osd.5 2024-04-25T01:32:23.901 DEBUG:teuthology.parallel:result is None 2024-04-25T01:32:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.333+0000 7f3002739700 1 -- 172.21.15.33:0/3194645239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 msgr2=0x7f2ffc0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.333+0000 7f3002739700 1 --2- 172.21.15.33:0/3194645239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 0x7f2ffc0fc1b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0004940 tx=0x7f2ff0009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.333+0000 7f3002739700 1 -- 172.21.15.33:0/3194645239 shutdown_connections 2024-04-25T01:32:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.333+0000 7f3002739700 1 --2- 172.21.15.33:0/3194645239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 0x7f2ffc0fc1b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.333+0000 7f3002739700 1 --2- 172.21.15.33:0/3194645239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc101d90 0x7f2ffc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.333+0000 7f3002739700 1 -- 172.21.15.33:0/3194645239 >> 172.21.15.33:0/3194645239 conn(0x7f2ffc071d00 msgr2=0x7f2ffc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.333+0000 7f3002739700 1 -- 172.21.15.33:0/3194645239 shutdown_connections 2024-04-25T01:32:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.333+0000 7f3002739700 1 -- 172.21.15.33:0/3194645239 wait complete. 2024-04-25T01:32:24.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.333+0000 7f3002739700 1 Processor -- start 2024-04-25T01:32:24.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f3002739700 1 -- start start 2024-04-25T01:32:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f3002739700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 0x7f2ffc113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f3002739700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc101d90 0x7f2ffc1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f3002739700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc114720 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f3002739700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ffc10db10 con 0x7f2ffc101d90 2024-04-25T01:32:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f2ffbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 0x7f2ffc113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f2ffbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 0x7f2ffc113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59956/0 (socket says 172.21.15.33:59956) 2024-04-25T01:32:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f2ffbfff700 1 -- 172.21.15.33:0/3226509400 learned_addr learned my addr 172.21.15.33:0/3226509400 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f2ffb7fe700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc101d90 0x7f2ffc1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:24.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f2ffbfff700 1 -- 172.21.15.33:0/3226509400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc101d90 msgr2=0x7f2ffc1140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f2ffbfff700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc101d90 0x7f2ffc1140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f2ffbfff700 1 -- 172.21.15.33:0/3226509400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0005040 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.334+0000 7f2ffb7fe700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc101d90 0x7f2ffc1140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.335+0000 7f2ffbfff700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 0x7f2ffc113b90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c8c0 tx=0x7f2fec0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.335+0000 7f2ff8ff9700 1 -- 172.21.15.33:0/3226509400 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fec00a980 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.335+0000 7f2ff8ff9700 1 -- 172.21.15.33:0/3226509400 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fec00ce60 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.335+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ffc10ddf0 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.335+0000 7f2ff8ff9700 1 -- 172.21.15.33:0/3226509400 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fec0115e0 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.335+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ffc10e3b0 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.336+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ffc0fd430 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.336+0000 7f2ff8ff9700 1 -- 172.21.15.33:0/3226509400 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f2fec012400 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.336+0000 7f2ff8ff9700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f2fe40692a0 0x7f2fe406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.337+0000 7f2ff8ff9700 1 -- 172.21.15.33:0/3226509400 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f2fec086f80 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.337+0000 7f2ffb7fe700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f2fe40692a0 0x7f2fe406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:24.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.337+0000 7f2ffb7fe700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f2fe40692a0 0x7f2fe406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000b3b0 tx=0x7f2ff0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:24.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.341+0000 7f2ff8ff9700 1 -- 172.21.15.33:0/3226509400 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2fec057080 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.538+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f2ffc108df0 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.538+0000 7f2ff8ff9700 1 -- 172.21.15.33:0/3226509400 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2fec0126b0 con 0x7f2ffc0fbd40 2024-04-25T01:32:24.517 INFO:teuthology.orchestra.run.smithi033.stdout:146028888081 2024-04-25T01:32:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.540+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f2fe40692a0 msgr2=0x7f2fe406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.540+0000 7f3002739700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f2fe40692a0 0x7f2fe406b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ff000b3b0 tx=0x7f2ff0009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.540+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 msgr2=0x7f2ffc113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.541+0000 7f3002739700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 0x7f2ffc113b90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00c8c0 tx=0x7f2fec0044c0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.541+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 shutdown_connections 2024-04-25T01:32:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.541+0000 7f3002739700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f2fe40692a0 0x7f2fe406b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.541+0000 7f3002739700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2ffc0fbd40 0x7f2ffc113b90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.541+0000 7f3002739700 1 --2- 172.21.15.33:0/3226509400 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2ffc101d90 0x7f2ffc1140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.541+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 >> 172.21.15.33:0/3226509400 conn(0x7f2ffc071d00 msgr2=0x7f2ffc0fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.541+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 shutdown_connections 2024-04-25T01:32:24.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.541+0000 7f3002739700 1 -- 172.21.15.33:0/3226509400 wait complete. 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 -- 172.21.15.33:0/2647318349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec80fbd40 msgr2=0x7f3ec80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 --2- 172.21.15.33:0/2647318349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec80fbd40 0x7f3ec80fc1b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc004940 tx=0x7f3ebc009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 -- 172.21.15.33:0/2647318349 shutdown_connections 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 --2- 172.21.15.33:0/2647318349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec80fbd40 0x7f3ec80fc1b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 --2- 172.21.15.33:0/2647318349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ec8101d90 0x7f3ec8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 -- 172.21.15.33:0/2647318349 >> 172.21.15.33:0/2647318349 conn(0x7f3ec8071d00 msgr2=0x7f3ec8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 -- 172.21.15.33:0/2647318349 shutdown_connections 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 -- 172.21.15.33:0/2647318349 wait complete. 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 Processor -- start 2024-04-25T01:32:24.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.593+0000 7f3ecdd62700 1 -- start start 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ecdd62700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ec80fbd40 0x7f3ec8113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ecdd62700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec8101d90 0x7f3ec81140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ecdd62700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8114740 con 0x7f3ec8101d90 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ecdd62700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec810db30 con 0x7f3ec80fbd40 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ec6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec8101d90 0x7f3ec81140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ec6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec8101d90 0x7f3ec81140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59970/0 (socket says 172.21.15.33:59970) 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ec6ffd700 1 -- 172.21.15.33:0/57009530 learned_addr learned my addr 172.21.15.33:0/57009530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ec6ffd700 1 -- 172.21.15.33:0/57009530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ec80fbd40 msgr2=0x7f3ec8113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ec6ffd700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ec80fbd40 0x7f3ec8113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ec6ffd700 1 -- 172.21.15.33:0/57009530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc005040 con 0x7f3ec8101d90 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ec6ffd700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec8101d90 0x7f3ec81140f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00b3b0 tx=0x7f3ebc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3eaffff700 1 -- 172.21.15.33:0/57009530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ebc00e040 con 0x7f3ec8101d90 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3eaffff700 1 -- 172.21.15.33:0/57009530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ebc00bbc0 con 0x7f3ec8101d90 2024-04-25T01:32:24.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec810ddb0 con 0x7f3ec8101d90 2024-04-25T01:32:24.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3eaffff700 1 -- 172.21.15.33:0/57009530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ebc01c3f0 con 0x7f3ec8101d90 2024-04-25T01:32:24.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.594+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec810e370 con 0x7f3ec8101d90 2024-04-25T01:32:24.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.595+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec80fd430 con 0x7f3ec8101d90 2024-04-25T01:32:24.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.596+0000 7f3eaffff700 1 -- 172.21.15.33:0/57009530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f3ebc00bd30 con 0x7f3ec8101d90 2024-04-25T01:32:24.836 INFO:teuthology.orchestra.run.smithi033.stdout:81604378650 2024-04-25T01:32:24.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.596+0000 7f3eaffff700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f3eb00692a0 0x7f3eb006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.596+0000 7f3eaffff700 1 -- 172.21.15.33:0/57009530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3ebc0881b0 con 0x7f3ec8101d90 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.596+0000 7f3ec77fe700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f3eb00692a0 0x7f3eb006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.597+0000 7f3ec77fe700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f3eb00692a0 0x7f3eb006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8004b70 tx=0x7f3eb8007cc0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.600+0000 7f3eaffff700 1 -- 172.21.15.33:0/57009530 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3ebc0582e0 con 0x7f3ec8101d90 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.806+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ec8108df0 con 0x7f3ec8101d90 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.806+0000 7f3eaffff700 1 -- 172.21.15.33:0/57009530 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3ebc057e70 con 0x7f3ec8101d90 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f3eb00692a0 msgr2=0x7f3eb006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f3eb00692a0 0x7f3eb006b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8004b70 tx=0x7f3eb8007cc0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec8101d90 msgr2=0x7f3ec81140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec8101d90 0x7f3ec81140f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc00b3b0 tx=0x7f3ebc009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 shutdown_connections 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ec80fbd40 0x7f3ec8113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f3eb00692a0 0x7f3eb006b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 --2- 172.21.15.33:0/57009530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ec8101d90 0x7f3ec81140f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:24.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 >> 172.21.15.33:0/57009530 conn(0x7f3ec8071d00 msgr2=0x7f3ec80fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:24.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 shutdown_connections 2024-04-25T01:32:24.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:24.808+0000 7f3ecdd62700 1 -- 172.21.15.33:0/57009530 wait complete. 2024-04-25T01:32:25.475 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705698 got 38654705700 for osd.0 2024-04-25T01:32:25.475 DEBUG:teuthology.parallel:result is None 2024-04-25T01:32:25.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:25 smithi138 ceph-mon[32977]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:25.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:25 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/3226509400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T01:32:25.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:25 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/57009530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T01:32:25.671 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:25 smithi033 ceph-mon[26353]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:25.672 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:25 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/3226509400' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T01:32:25.672 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:25 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/57009530' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T01:32:26.088 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378650 for osd.2 2024-04-25T01:32:26.088 DEBUG:teuthology.parallel:result is None 2024-04-25T01:32:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.300+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/2581537091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e780fbd40 msgr2=0x7f4e780fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.300+0000 7f4e7d3ca700 1 --2- 172.21.15.33:0/2581537091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e780fbd40 0x7f4e780fc1b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4e68004940 tx=0x7f4e68009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:26.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.300+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/2581537091 shutdown_connections 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.300+0000 7f4e7d3ca700 1 --2- 172.21.15.33:0/2581537091 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e780fbd40 0x7f4e780fc1b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.300+0000 7f4e7d3ca700 1 --2- 172.21.15.33:0/2581537091 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e78101d90 0x7f4e78102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.300+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/2581537091 >> 172.21.15.33:0/2581537091 conn(0x7f4e78071d00 msgr2=0x7f4e78072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.300+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/2581537091 shutdown_connections 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.300+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/2581537091 wait complete. 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e7d3ca700 1 Processor -- start 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e7d3ca700 1 -- start start 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e7d3ca700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e780fbd40 0x7f4e78113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e7d3ca700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e78101d90 0x7f4e781140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e7d3ca700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e78114740 con 0x7f4e78101d90 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e7d3ca700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7810db30 con 0x7f4e780fbd40 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e767fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e78101d90 0x7f4e781140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e767fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e78101d90 0x7f4e781140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44470/0 (socket says 172.21.15.33:44470) 2024-04-25T01:32:26.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e767fc700 1 -- 172.21.15.33:0/1867992695 learned_addr learned my addr 172.21.15.33:0/1867992695 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e76ffd700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e780fbd40 0x7f4e78113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e767fc700 1 -- 172.21.15.33:0/1867992695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e780fbd40 msgr2=0x7f4e78113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e767fc700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e780fbd40 0x7f4e78113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.301+0000 7f4e767fc700 1 -- 172.21.15.33:0/1867992695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e68005040 con 0x7f4e78101d90 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.302+0000 7f4e767fc700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e78101d90 0x7f4e781140f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800b8b0 tx=0x7f4e68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.302+0000 7f4e5f7fe700 1 -- 172.21.15.33:0/1867992695 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6800e040 con 0x7f4e78101d90 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.302+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e7810ddb0 con 0x7f4e78101d90 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.302+0000 7f4e5f7fe700 1 -- 172.21.15.33:0/1867992695 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e68004290 con 0x7f4e78101d90 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.302+0000 7f4e5f7fe700 1 -- 172.21.15.33:0/1867992695 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e6801c3f0 con 0x7f4e78101d90 2024-04-25T01:32:26.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.302+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e7810e370 con 0x7f4e78101d90 2024-04-25T01:32:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.303+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e780fd430 con 0x7f4e78101d90 2024-04-25T01:32:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.303+0000 7f4e5f7fe700 1 -- 172.21.15.33:0/1867992695 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4e68013820 con 0x7f4e78101d90 2024-04-25T01:32:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.304+0000 7f4e5f7fe700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4e600692a0 0x7f4e6006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.304+0000 7f4e5f7fe700 1 -- 172.21.15.33:0/1867992695 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4e68088110 con 0x7f4e78101d90 2024-04-25T01:32:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.304+0000 7f4e76ffd700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4e600692a0 0x7f4e6006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:26.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.304+0000 7f4e76ffd700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4e600692a0 0x7f4e6006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c004f40 tx=0x7f4e6c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.308+0000 7f4e5f7fe700 1 -- 172.21.15.33:0/1867992695 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4e68058210 con 0x7f4e78101d90 2024-04-25T01:32:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.506+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4e78108df0 con 0x7f4e78101d90 2024-04-25T01:32:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.506+0000 7f4e5f7fe700 1 -- 172.21.15.33:0/1867992695 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e68057da0 con 0x7f4e78101d90 2024-04-25T01:32:26.484 INFO:teuthology.orchestra.run.smithi033.stdout:111669149718 2024-04-25T01:32:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.508+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4e600692a0 msgr2=0x7f4e6006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.508+0000 7f4e7d3ca700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4e600692a0 0x7f4e6006b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e6c004f40 tx=0x7f4e6c00b040 comp rx=0 tx=0).stop 2024-04-25T01:32:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.509+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e78101d90 msgr2=0x7f4e781140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.509+0000 7f4e7d3ca700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e78101d90 0x7f4e781140f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4e6800b8b0 tx=0x7f4e68009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.509+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 shutdown_connections 2024-04-25T01:32:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.509+0000 7f4e7d3ca700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e780fbd40 0x7f4e78113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.509+0000 7f4e7d3ca700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4e600692a0 0x7f4e6006b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.509+0000 7f4e7d3ca700 1 --2- 172.21.15.33:0/1867992695 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e78101d90 0x7f4e781140f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.509+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 >> 172.21.15.33:0/1867992695 conn(0x7f4e78071d00 msgr2=0x7f4e78065600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.509+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 shutdown_connections 2024-04-25T01:32:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:26.509+0000 7f4e7d3ca700 1 -- 172.21.15.33:0/1867992695 wait complete. 2024-04-25T01:32:26.798 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888079 got 146028888081 for osd.4 2024-04-25T01:32:26.798 DEBUG:teuthology.parallel:result is None 2024-04-25T01:32:27.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:27 smithi033 ceph-mon[26353]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:27.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:27 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1867992695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T01:32:27.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:27 smithi138 ceph-mon[32977]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:27.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:27 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1867992695' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T01:32:27.556 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149714 got 111669149718 for osd.3 2024-04-25T01:32:27.556 DEBUG:teuthology.parallel:result is None 2024-04-25T01:32:27.556 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T01:32:27.556 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T01:32:27.826 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:29.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.477+0000 7f35cb66f700 1 -- 172.21.15.33:0/2623661733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 msgr2=0x7f35c41020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:29.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.477+0000 7f35cb66f700 1 --2- 172.21.15.33:0/2623661733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 0x7f35c41020e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f35bc009ce0 tx=0x7f35bc004d10 comp rx=0 tx=0).stop 2024-04-25T01:32:29.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.477+0000 7f35cb66f700 1 -- 172.21.15.33:0/2623661733 shutdown_connections 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.477+0000 7f35cb66f700 1 --2- 172.21.15.33:0/2623661733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c41026b0 0x7f35c40fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.477+0000 7f35cb66f700 1 --2- 172.21.15.33:0/2623661733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 0x7f35c41020e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.477+0000 7f35cb66f700 1 -- 172.21.15.33:0/2623661733 >> 172.21.15.33:0/2623661733 conn(0x7f35c40f7860 msgr2=0x7f35c40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.477+0000 7f35cb66f700 1 -- 172.21.15.33:0/2623661733 shutdown_connections 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.477+0000 7f35cb66f700 1 -- 172.21.15.33:0/2623661733 wait complete. 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.477+0000 7f35cb66f700 1 Processor -- start 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35cb66f700 1 -- start start 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35cb66f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 0x7f35c410ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35cb66f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c41026b0 0x7f35c410f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35cb66f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c410f760 con 0x7f35c4101d10 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35cb66f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c410f8d0 con 0x7f35c41026b0 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35c940b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 0x7f35c410ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35c940b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 0x7f35c410ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44490/0 (socket says 172.21.15.33:44490) 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35c940b700 1 -- 172.21.15.33:0/3399538776 learned_addr learned my addr 172.21.15.33:0/3399538776 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:29.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35c8c0a700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c41026b0 0x7f35c410f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35c940b700 1 -- 172.21.15.33:0/3399538776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c41026b0 msgr2=0x7f35c410f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35c940b700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c41026b0 0x7f35c410f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35c940b700 1 -- 172.21.15.33:0/3399538776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc009940 con 0x7f35c4101d10 2024-04-25T01:32:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.478+0000 7f35c940b700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 0x7f35c410ece0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35bc003400 tx=0x7f35bc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.479+0000 7f35c1ffb700 1 -- 172.21.15.33:0/3399538776 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35bc00e040 con 0x7f35c4101d10 2024-04-25T01:32:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.479+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c410fb50 con 0x7f35c4101d10 2024-04-25T01:32:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.479+0000 7f35c1ffb700 1 -- 172.21.15.33:0/3399538776 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35bc004610 con 0x7f35c4101d10 2024-04-25T01:32:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.479+0000 7f35c1ffb700 1 -- 172.21.15.33:0/3399538776 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35bc01c640 con 0x7f35c4101d10 2024-04-25T01:32:29.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.479+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c4118050 con 0x7f35c4101d10 2024-04-25T01:32:29.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.480+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c40fd460 con 0x7f35c4101d10 2024-04-25T01:32:29.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.480+0000 7f35c1ffb700 1 -- 172.21.15.33:0/3399538776 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f35bc012a70 con 0x7f35c4101d10 2024-04-25T01:32:29.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.481+0000 7f35c1ffb700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f35ac0692a0 0x7f35ac06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:29.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.481+0000 7f35c1ffb700 1 -- 172.21.15.33:0/3399538776 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f35bc0886c0 con 0x7f35c4101d10 2024-04-25T01:32:29.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.481+0000 7f35c8c0a700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f35ac0692a0 0x7f35ac06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:29.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.481+0000 7f35c8c0a700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f35ac0692a0 0x7f35ac06b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f35c4066a60 tx=0x7f35b800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:29.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.485+0000 7f35c1ffb700 1 -- 172.21.15.33:0/3399538776 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f35bc054cb0 con 0x7f35c4101d10 2024-04-25T01:32:29.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:29 smithi138 ceph-mon[32977]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:29.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.676+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f35c410fb50 con 0x7f35ac0692a0 2024-04-25T01:32:29.654 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:29 smithi033 ceph-mon[26353]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:29.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.679+0000 7f35c1ffb700 1 -- 172.21.15.33:0/3399538776 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28286 (secure 0 0 0) 0x7f35c410fb50 con 0x7f35ac0692a0 2024-04-25T01:32:29.657 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:32:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f35ac0692a0 msgr2=0x7f35ac06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f35ac0692a0 0x7f35ac06b750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f35c4066a60 tx=0x7f35b800b040 comp rx=0 tx=0).stop 2024-04-25T01:32:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 msgr2=0x7f35c410ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 0x7f35c410ece0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f35bc003400 tx=0x7f35bc004d10 comp rx=0 tx=0).stop 2024-04-25T01:32:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 shutdown_connections 2024-04-25T01:32:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f35ac0692a0 0x7f35ac06b750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35c4101d10 0x7f35c410ece0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 --2- 172.21.15.33:0/3399538776 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35c41026b0 0x7f35c410f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 >> 172.21.15.33:0/3399538776 conn(0x7f35c40f7860 msgr2=0x7f35c40f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 shutdown_connections 2024-04-25T01:32:29.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:29.681+0000 7f35cb66f700 1 -- 172.21.15.33:0/3399538776 wait complete. 2024-04-25T01:32:29.660 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:32:30.399 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-04-25T01:32:28.091926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788036,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682876,"statfs":{"total":575970213888,"available":574139265024,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002133"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:31:39.180391+0000","last_change":"2024-04-25T01:31:13.693030+0000","last_active":"2024-04-25T01:31:39.180391+0000","last_peered":"2024-04-25T01:31:39.180391+0000","last_clean":"2024-04-25T01:31:39.180391+0000","last_became_active":"2024-04-25T01:31:13.692615+0000","last_became_peered":"2024-04-25T01:31:13.692615+0000","last_unstale":"2024-04-25T01:31:39.180391+0000","last_undegraded":"2024-04-25T01:31:39.180391+0000","last_fullsized":"2024-04-25T01:31:39.180391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:30:26.559030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:30:26.559030+0000","last_clean_scrub_stamp":"2024-04-25T01:30:26.559030+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:21.077418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]}]},{"osd":4,"up_from":34,"seq":146028888081,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.432},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.41999999999999998}]},{"osd":3,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.30299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149718,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 01:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Apr 25 01:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.66200000000000003}]},{"osd":2,"last update":"Thu Apr 25 01:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.502}]},{"osd":4,"last update":"Thu Apr 25 01:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378651,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 01:31:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.38}]},{"osd":1,"last update":"Thu Apr 25 01:31:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.32600000000000001}]},{"osd":3,"last update":"Thu Apr 25 01:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.73399999999999999}]},{"osd":4,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.58999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 01:32:04 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35199999999999998,"15min":0.34799999999999998},"min":{"1min":0.27500000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47799999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.307},{"interface":"front","average":{"1min":0.374,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Apr 25 01:31:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Apr 25 01:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.50800000000000001}]},{"osd":4,"last update":"Thu Apr 25 01:32:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 01:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.53100000000000003,"5min":0.626,"15min":0.626},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.375,"15min":0.377},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.502,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Apr 25 01:31:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.439},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.55700000000000005}]},{"osd":3,"last update":"Thu Apr 25 01:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.505},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53300000000000003}]},{"osd":4,"last update":"Thu Apr 25 01:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:32:30.400 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T01:32:30.670 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:31.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:31 smithi033 ceph-mon[26353]: from='client.14402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:32:31.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:31 smithi033 ceph-mon[26353]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:31.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:32:31.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:31 smithi138 ceph-mon[32977]: from='client.14402 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:32:31.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:31 smithi138 ceph-mon[32977]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:31.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:31 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:32:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.352+0000 7feb4fbdd700 1 -- 172.21.15.33:0/4242827833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 msgr2=0x7feb481020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.352+0000 7feb4fbdd700 1 --2- 172.21.15.33:0/4242827833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 0x7feb481020e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7feb44004940 tx=0x7feb44009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.353+0000 7feb4fbdd700 1 -- 172.21.15.33:0/4242827833 shutdown_connections 2024-04-25T01:32:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.353+0000 7feb4fbdd700 1 --2- 172.21.15.33:0/4242827833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb481026b0 0x7feb480fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.353+0000 7feb4fbdd700 1 --2- 172.21.15.33:0/4242827833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 0x7feb481020e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.353+0000 7feb4fbdd700 1 -- 172.21.15.33:0/4242827833 >> 172.21.15.33:0/4242827833 conn(0x7feb480f7860 msgr2=0x7feb480f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.353+0000 7feb4fbdd700 1 -- 172.21.15.33:0/4242827833 shutdown_connections 2024-04-25T01:32:32.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.353+0000 7feb4fbdd700 1 -- 172.21.15.33:0/4242827833 wait complete. 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.353+0000 7feb4fbdd700 1 Processor -- start 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.353+0000 7feb4fbdd700 1 -- start start 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4fbdd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 0x7feb481128e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4fbdd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb481026b0 0x7feb48112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4fbdd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb48113470 con 0x7feb48101d10 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4fbdd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4810ca70 con 0x7feb481026b0 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4d979700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 0x7feb481128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4d979700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 0x7feb481128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44502/0 (socket says 172.21.15.33:44502) 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4d979700 1 -- 172.21.15.33:0/2168236101 learned_addr learned my addr 172.21.15.33:0/2168236101 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4d178700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb481026b0 0x7feb48112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4d979700 1 -- 172.21.15.33:0/2168236101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb481026b0 msgr2=0x7feb48112e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4d979700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb481026b0 0x7feb48112e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4d979700 1 -- 172.21.15.33:0/2168236101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44005040 con 0x7feb48101d10 2024-04-25T01:32:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4d178700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb481026b0 0x7feb48112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:32:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4d979700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 0x7feb481128e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feb44009100 tx=0x7feb44009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb3e7fc700 1 -- 172.21.15.33:0/2168236101 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4400e040 con 0x7feb48101d10 2024-04-25T01:32:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb3e7fc700 1 -- 172.21.15.33:0/2168236101 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb44013420 con 0x7feb48101d10 2024-04-25T01:32:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb3e7fc700 1 -- 172.21.15.33:0/2168236101 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4401c630 con 0x7feb48101d10 2024-04-25T01:32:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4810ccf0 con 0x7feb48101d10 2024-04-25T01:32:32.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.354+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4810d260 con 0x7feb48101d10 2024-04-25T01:32:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.356+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb480fd460 con 0x7feb48101d10 2024-04-25T01:32:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.356+0000 7feb3e7fc700 1 -- 172.21.15.33:0/2168236101 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7feb44013590 con 0x7feb48101d10 2024-04-25T01:32:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.356+0000 7feb3e7fc700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb340692a0 0x7feb3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.356+0000 7feb3e7fc700 1 -- 172.21.15.33:0/2168236101 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7feb44087aa0 con 0x7feb48101d10 2024-04-25T01:32:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.357+0000 7feb4d178700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb340692a0 0x7feb3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:32.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.357+0000 7feb4d178700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb340692a0 0x7feb3406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb4810dc30 tx=0x7feb38009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:32.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.361+0000 7feb3e7fc700 1 -- 172.21.15.33:0/2168236101 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7feb44057ba0 con 0x7feb48101d10 2024-04-25T01:32:32.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.550+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feb48108e10 con 0x7feb340692a0 2024-04-25T01:32:32.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.552+0000 7feb3e7fc700 1 -- 172.21.15.33:0/2168236101 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28257 (secure 0 0 0) 0x7feb48108e10 con 0x7feb340692a0 2024-04-25T01:32:32.529 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:32:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.553+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb340692a0 msgr2=0x7feb3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.553+0000 7feb4fbdd700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb340692a0 0x7feb3406b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feb4810dc30 tx=0x7feb38009040 comp rx=0 tx=0).stop 2024-04-25T01:32:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.553+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 msgr2=0x7feb481128e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.553+0000 7feb4fbdd700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 0x7feb481128e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feb44009100 tx=0x7feb44009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.554+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 shutdown_connections 2024-04-25T01:32:32.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.554+0000 7feb4fbdd700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7feb340692a0 0x7feb3406b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.554+0000 7feb4fbdd700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb48101d10 0x7feb481128e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.554+0000 7feb4fbdd700 1 --2- 172.21.15.33:0/2168236101 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb481026b0 0x7feb48112e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:32.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.554+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 >> 172.21.15.33:0/2168236101 conn(0x7feb480f7860 msgr2=0x7feb480f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:32.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.554+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 shutdown_connections 2024-04-25T01:32:32.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:32.554+0000 7feb4fbdd700 1 -- 172.21.15.33:0/2168236101 wait complete. 2024-04-25T01:32:32.532 INFO:teuthology.orchestra.run.smithi033.stderr:dumped all 2024-04-25T01:32:33.155 INFO:teuthology.orchestra.run.smithi033.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-04-25T01:32:32.092808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788036,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682876,"statfs":{"total":575970213888,"available":574139265024,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002480"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T01:31:39.180391+0000","last_change":"2024-04-25T01:31:13.693030+0000","last_active":"2024-04-25T01:31:39.180391+0000","last_peered":"2024-04-25T01:31:39.180391+0000","last_clean":"2024-04-25T01:31:39.180391+0000","last_became_active":"2024-04-25T01:31:13.692615+0000","last_became_peered":"2024-04-25T01:31:13.692615+0000","last_unstale":"2024-04-25T01:31:39.180391+0000","last_undegraded":"2024-04-25T01:31:39.180391+0000","last_fullsized":"2024-04-25T01:31:39.180391+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T01:30:26.559030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T01:30:26.559030+0000","last_clean_scrub_stamp":"2024-04-25T01:30:26.559030+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:21.077418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297716,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447436,"statfs":{"total":95995035648,"available":95690174464,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.32800000000000001}]},{"osd":1,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.47899999999999998}]},{"osd":2,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.436},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.374}]},{"osd":3,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.45500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 01:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.37},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.35199999999999998}]},{"osd":1,"last update":"Thu Apr 25 01:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.434},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Apr 25 01:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Apr 25 01:32:13 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.40200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 01:31:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Apr 25 01:31:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.25700000000000001}]},{"osd":3,"last update":"Thu Apr 25 01:31:49 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.61399999999999999}]},{"osd":4,"last update":"Thu Apr 25 01:32:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298300,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446852,"statfs":{"total":95995035648,"available":95689576448,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 01:32:04 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.35199999999999998,"15min":0.34799999999999998},"min":{"1min":0.27500000000000002,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.47799999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"last":0.317},{"interface":"front","average":{"1min":0.374,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"last":0.34699999999999998}]},{"osd":2,"last update":"Thu Apr 25 01:31:30 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Apr 25 01:31:52 2024","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.497},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Apr 25 01:32:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 01:32:00 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36199999999999999,"15min":0.35999999999999999},"min":{"1min":0.17499999999999999,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.53100000000000003,"5min":0.626,"15min":0.626},"last":0.38},{"interface":"front","average":{"1min":0.36599999999999999,"5min":0.375,"15min":0.377},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.502,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Apr 25 01:31:29 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"last":0.44600000000000001}]},{"osd":3,"last update":"Thu Apr 25 01:31:50 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.72499999999999998}]},{"osd":4,"last update":"Thu Apr 25 01:32:15 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.73899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T01:32:33.156 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T01:32:33.156 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T01:32:33.157 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T01:32:33.157 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T01:32:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:33.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:33 smithi033 ceph-mon[26353]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:33.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:33 smithi138 ceph-mon[32977]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:34.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:34 smithi138 ceph-mon[32977]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:32:34.648 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:34 smithi033 ceph-mon[26353]: from='client.14406 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T01:32:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.168+0000 7f5321378700 1 -- 172.21.15.33:0/3655175455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 msgr2=0x7f531c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:35.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.168+0000 7f5321378700 1 --2- 172.21.15.33:0/3655175455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 0x7f531c0fcdf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5310004940 tx=0x7f5310009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.168+0000 7f5321378700 1 -- 172.21.15.33:0/3655175455 shutdown_connections 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.168+0000 7f5321378700 1 --2- 172.21.15.33:0/3655175455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 0x7f531c0fcdf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.168+0000 7f5321378700 1 --2- 172.21.15.33:0/3655175455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f531c102de0 0x7f531c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.168+0000 7f5321378700 1 -- 172.21.15.33:0/3655175455 >> 172.21.15.33:0/3655175455 conn(0x7f531c0f8990 msgr2=0x7f531c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.168+0000 7f5321378700 1 -- 172.21.15.33:0/3655175455 shutdown_connections 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.168+0000 7f5321378700 1 -- 172.21.15.33:0/3655175455 wait complete. 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f5321378700 1 Processor -- start 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f5321378700 1 -- start start 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f5321378700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f531c102de0 0x7f531c113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f5321378700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 0x7f531c113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f5321378700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c114490 con 0x7f531c103780 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f5321378700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c10da90 con 0x7f531c102de0 2024-04-25T01:32:35.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f531a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 0x7f531c113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f531a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 0x7f531c113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44516/0 (socket says 172.21.15.33:44516) 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f531a7fc700 1 -- 172.21.15.33:0/617161702 learned_addr learned my addr 172.21.15.33:0/617161702 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.169+0000 7f531affd700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f531c102de0 0x7f531c113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.170+0000 7f531a7fc700 1 -- 172.21.15.33:0/617161702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f531c102de0 msgr2=0x7f531c113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.170+0000 7f531a7fc700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f531c102de0 0x7f531c113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.170+0000 7f531a7fc700 1 -- 172.21.15.33:0/617161702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5310005040 con 0x7f531c103780 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.170+0000 7f531a7fc700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 0x7f531c113e40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5310009180 tx=0x7f5310009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.170+0000 7f53037fe700 1 -- 172.21.15.33:0/617161702 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f531000e040 con 0x7f531c103780 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.170+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c10dd10 con 0x7f531c103780 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.170+0000 7f53037fe700 1 -- 172.21.15.33:0/617161702 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5310013420 con 0x7f531c103780 2024-04-25T01:32:35.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.170+0000 7f53037fe700 1 -- 172.21.15.33:0/617161702 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f531001c610 con 0x7f531c103780 2024-04-25T01:32:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.170+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c10e2d0 con 0x7f531c103780 2024-04-25T01:32:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.171+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c0fe530 con 0x7f531c103780 2024-04-25T01:32:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.171+0000 7f53037fe700 1 -- 172.21.15.33:0/617161702 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5310013a00 con 0x7f531c103780 2024-04-25T01:32:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.172+0000 7f53037fe700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f53040692a0 0x7f530406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.172+0000 7f53037fe700 1 -- 172.21.15.33:0/617161702 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5310087f80 con 0x7f531c103780 2024-04-25T01:32:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.172+0000 7f531affd700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f53040692a0 0x7f530406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.172+0000 7f531affd700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f53040692a0 0x7f530406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f530c009fd0 tx=0x7f530c00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:35.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.176+0000 7f53037fe700 1 -- 172.21.15.33:0/617161702 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5310058080 con 0x7f531c103780 2024-04-25T01:32:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.414+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f531c109ec0 con 0x7f531c103780 2024-04-25T01:32:35.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.415+0000 7f53037fe700 1 -- 172.21.15.33:0/617161702 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5310057c10 con 0x7f531c103780 2024-04-25T01:32:35.393 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:32:35.393 INFO:teuthology.orchestra.run.smithi033.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T01:32:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.416+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f53040692a0 msgr2=0x7f530406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.416+0000 7f5321378700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f53040692a0 0x7f530406b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f530c009fd0 tx=0x7f530c00b040 comp rx=0 tx=0).stop 2024-04-25T01:32:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.417+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 msgr2=0x7f531c113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.417+0000 7f5321378700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 0x7f531c113e40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f5310009180 tx=0x7f5310009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.417+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 shutdown_connections 2024-04-25T01:32:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.417+0000 7f5321378700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f531c102de0 0x7f531c113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:35.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.417+0000 7f5321378700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f53040692a0 0x7f530406b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.417+0000 7f5321378700 1 --2- 172.21.15.33:0/617161702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f531c103780 0x7f531c113e40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.417+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 >> 172.21.15.33:0/617161702 conn(0x7f531c0f8990 msgr2=0x7f531c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.417+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 shutdown_connections 2024-04-25T01:32:35.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:35.417+0000 7f5321378700 1 -- 172.21.15.33:0/617161702 wait complete. 2024-04-25T01:32:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:35 smithi138 ceph-mon[32977]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:35.697 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:35 smithi033 ceph-mon[26353]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:36.002 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T01:32:36.002 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T01:32:36.003 INFO:teuthology.run_tasks:Running task print... 2024-04-25T01:32:36.011 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-04-25T01:32:36.011 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:32:36.024 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:32:36.025 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-25T01:32:36.296 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:36.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:36 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/617161702' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T01:32:36.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:36 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/617161702' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T01:32:37.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:37 smithi138 ceph-mon[32977]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:37.615 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:37 smithi033 ceph-mon[26353]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/1436944838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 msgr2=0x7fdc040fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 --2- 172.21.15.33:0/1436944838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 0x7fdc040fd270 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4004940 tx=0x7fdbf4009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:37.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/1436944838 shutdown_connections 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 --2- 172.21.15.33:0/1436944838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc040fdfd0 0x7fdc040fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 --2- 172.21.15.33:0/1436944838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 0x7fdc040fd270 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/1436944838 >> 172.21.15.33:0/1436944838 conn(0x7fdc040f8540 msgr2=0x7fdc040fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/1436944838 shutdown_connections 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/1436944838 wait complete. 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 Processor -- start 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.973+0000 7fdc0a7bb700 1 -- start start 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc0a7bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 0x7fdc04071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc0a7bb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc040fdfd0 0x7fdc040702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc0a7bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc04072200 con 0x7fdc040fce60 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc0a7bb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc040707f0 con 0x7fdc040fdfd0 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc03fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 0x7fdc04071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc03fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 0x7fdc04071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44998/0 (socket says 172.21.15.33:44998) 2024-04-25T01:32:37.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc03fff700 1 -- 172.21.15.33:0/4207779448 learned_addr learned my addr 172.21.15.33:0/4207779448 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc037fe700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc040fdfd0 0x7fdc040702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc03fff700 1 -- 172.21.15.33:0/4207779448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc040fdfd0 msgr2=0x7fdc040702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc03fff700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc040fdfd0 0x7fdc040702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc03fff700 1 -- 172.21.15.33:0/4207779448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf4005040 con 0x7fdc040fce60 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc037fe700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc040fdfd0 0x7fdc040702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc03fff700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 0x7fdc04071c00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009150 tx=0x7fdbf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc00ff9700 1 -- 172.21.15.33:0/4207779448 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf400e040 con 0x7fdc040fce60 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc00ff9700 1 -- 172.21.15.33:0/4207779448 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbf400bbf0 con 0x7fdc040fce60 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc00ff9700 1 -- 172.21.15.33:0/4207779448 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbf401c630 con 0x7fdc040fce60 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc04070a40 con 0x7fdc040fce60 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.974+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc04071000 con 0x7fdc040fce60 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.975+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0404c710 con 0x7fdc040fce60 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.976+0000 7fdc00ff9700 1 -- 172.21.15.33:0/4207779448 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fdbf400bd60 con 0x7fdc040fce60 2024-04-25T01:32:37.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.976+0000 7fdc00ff9700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fdbec0692a0 0x7fdbec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:37.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.976+0000 7fdc00ff9700 1 -- 172.21.15.33:0/4207779448 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fdbf4088240 con 0x7fdc040fce60 2024-04-25T01:32:37.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.976+0000 7fdc037fe700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fdbec0692a0 0x7fdbec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:37.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.977+0000 7fdc037fe700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fdbec0692a0 0x7fdbec06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009fd0 tx=0x7fdbf800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:37.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:37.981+0000 7fdc00ff9700 1 -- 172.21.15.33:0/4207779448 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdbf4058340 con 0x7fdc040fce60 2024-04-25T01:32:38.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.175+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fdc041058c0 con 0x7fdc040fce60 2024-04-25T01:32:38.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.180+0000 7fdc00ff9700 1 -- 172.21.15.33:0/4207779448 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fdbf4057ed0 con 0x7fdc040fce60 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fdbec0692a0 msgr2=0x7fdbec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fdbec0692a0 0x7fdbec06b750 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009fd0 tx=0x7fdbf800b040 comp rx=0 tx=0).stop 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 msgr2=0x7fdc04071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 0x7fdc04071c00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009150 tx=0x7fdbf4009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 shutdown_connections 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fdbec0692a0 0x7fdbec06b750 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdc040fce60 0x7fdc04071c00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 --2- 172.21.15.33:0/4207779448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdc040fdfd0 0x7fdc040702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 >> 172.21.15.33:0/4207779448 conn(0x7fdc040f8540 msgr2=0x7fdc040f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 shutdown_connections 2024-04-25T01:32:38.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:38.182+0000 7fdc0a7bb700 1 -- 172.21.15.33:0/4207779448 wait complete. 2024-04-25T01:32:38.846 INFO:teuthology.run_tasks:Running task print... 2024-04-25T01:32:38.857 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-25T01:32:38.858 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:32:38.888 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:32:38.888 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T01:32:39.159 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:39.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:39 smithi033 ceph-mon[26353]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:39.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:39 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/4207779448' entity='client.admin' 2024-04-25T01:32:39.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:39 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:32:39.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:39 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:32:39.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:39 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:32:39.413 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:39 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:32:39.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:39 smithi138 ceph-mon[32977]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:39.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:39 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/4207779448' entity='client.admin' 2024-04-25T01:32:39.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:39 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:32:39.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:39 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:32:39.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:39 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:32:39.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:39 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:32:40.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 -- 172.21.15.33:0/1280347365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0662c0 msgr2=0x7ffa6c0ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:40.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 --2- 172.21.15.33:0/1280347365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0662c0 0x7ffa6c0ff9a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffa60004940 tx=0x7ffa60009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:40.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 -- 172.21.15.33:0/1280347365 shutdown_connections 2024-04-25T01:32:40.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 --2- 172.21.15.33:0/1280347365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa6c0ffee0 0x7ffa6c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:40.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 --2- 172.21.15.33:0/1280347365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0662c0 0x7ffa6c0ff9a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:40.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 -- 172.21.15.33:0/1280347365 >> 172.21.15.33:0/1280347365 conn(0x7ffa6c0f7430 msgr2=0x7ffa6c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:40.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 -- 172.21.15.33:0/1280347365 shutdown_connections 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 -- 172.21.15.33:0/1280347365 wait complete. 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 Processor -- start 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.852+0000 7ffa72e11700 1 -- start start 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa72e11700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa6c0662c0 0x7ffa6c0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa72e11700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0ffee0 0x7ffa6c0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa72e11700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0fbe50 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa72e11700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6c0fbfc0 con 0x7ffa6c0662c0 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0ffee0 0x7ffa6c0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0ffee0 0x7ffa6c0fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45010/0 (socket says 172.21.15.33:45010) 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa6bfff700 1 -- 172.21.15.33:0/3690997992 learned_addr learned my addr 172.21.15.33:0/3690997992 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa70bad700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa6c0662c0 0x7ffa6c0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa6bfff700 1 -- 172.21.15.33:0/3690997992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa6c0662c0 msgr2=0x7ffa6c0fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:40.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa6bfff700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa6c0662c0 0x7ffa6c0fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa6bfff700 1 -- 172.21.15.33:0/3690997992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60005040 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa6bfff700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0ffee0 0x7ffa6c0fb8a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffa58002a50 tx=0x7ffa580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa697fa700 1 -- 172.21.15.33:0/3690997992 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa58007cc0 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa697fa700 1 -- 172.21.15.33:0/3690997992 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa5800f040 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa697fa700 1 -- 172.21.15.33:0/3690997992 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa5800da30 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.853+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa6c0fc2a0 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.854+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa6c115a50 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.855+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa6c05f5c0 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.855+0000 7ffa697fa700 1 -- 172.21.15.33:0/3690997992 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ffa5800db90 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.855+0000 7ffa697fa700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa5c0692a0 0x7ffa5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.855+0000 7ffa697fa700 1 -- 172.21.15.33:0/3690997992 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7ffa58086b60 con 0x7ffa6c0ffee0 2024-04-25T01:32:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.855+0000 7ffa70bad700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa5c0692a0 0x7ffa5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:40.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.856+0000 7ffa70bad700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa5c0692a0 0x7ffa5c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0fd160 tx=0x7ffa60009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:40.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:40.860+0000 7ffa697fa700 1 -- 172.21.15.33:0/3690997992 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffa580531d0 con 0x7ffa6c0ffee0 2024-04-25T01:32:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.070+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa6c0f8f20 con 0x7ffa5c0692a0 2024-04-25T01:32:41.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.071+0000 7ffa697fa700 1 -- 172.21.15.33:0/3690997992 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ffa6c0f8f20 con 0x7ffa5c0692a0 2024-04-25T01:32:41.048 INFO:teuthology.orchestra.run.smithi033.stdout:Backend: cephadm 2024-04-25T01:32:41.049 INFO:teuthology.orchestra.run.smithi033.stdout:Available: Yes 2024-04-25T01:32:41.049 INFO:teuthology.orchestra.run.smithi033.stdout:Paused: No 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.073+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa5c0692a0 msgr2=0x7ffa5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.073+0000 7ffa72e11700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa5c0692a0 0x7ffa5c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffa6c0fd160 tx=0x7ffa60009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.073+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0ffee0 msgr2=0x7ffa6c0fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.073+0000 7ffa72e11700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0ffee0 0x7ffa6c0fb8a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ffa58002a50 tx=0x7ffa580046b0 comp rx=0 tx=0).stop 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.073+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 shutdown_connections 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.073+0000 7ffa72e11700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa6c0662c0 0x7ffa6c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.073+0000 7ffa72e11700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ffa5c0692a0 0x7ffa5c06b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.073+0000 7ffa72e11700 1 --2- 172.21.15.33:0/3690997992 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa6c0ffee0 0x7ffa6c0fb8a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.073+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 >> 172.21.15.33:0/3690997992 conn(0x7ffa6c0f7430 msgr2=0x7ffa6c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.074+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 shutdown_connections 2024-04-25T01:32:41.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:41.074+0000 7ffa72e11700 1 -- 172.21.15.33:0/3690997992 wait complete. 2024-04-25T01:32:41.278 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:41 smithi033 ceph-mon[26353]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:41.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:41 smithi138 ceph-mon[32977]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:41.681 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T01:32:41.952 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:42.368 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:42 smithi033 ceph-mon[26353]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:42.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:42 smithi138 ceph-mon[32977]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:43.325 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:43 smithi033 ceph-mon[26353]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:43.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:43 smithi138 ceph-mon[32977]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:43.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.685+0000 7f8a9f662700 1 -- 172.21.15.33:0/1729776239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 msgr2=0x7f8a98100160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:43.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.685+0000 7f8a9f662700 1 --2- 172.21.15.33:0/1729776239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 0x7f8a98100160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8a88004940 tx=0x7f8a88009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:43.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.685+0000 7f8a9f662700 1 -- 172.21.15.33:0/1729776239 shutdown_connections 2024-04-25T01:32:43.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.685+0000 7f8a9f662700 1 --2- 172.21.15.33:0/1729776239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 0x7f8a98100160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.685+0000 7f8a9f662700 1 --2- 172.21.15.33:0/1729776239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a98066f30 0x7f8a980fd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.685+0000 7f8a9f662700 1 -- 172.21.15.33:0/1729776239 >> 172.21.15.33:0/1729776239 conn(0x7f8a980f7440 msgr2=0x7f8a980f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.685+0000 7f8a9f662700 1 -- 172.21.15.33:0/1729776239 shutdown_connections 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.685+0000 7f8a9f662700 1 -- 172.21.15.33:0/1729776239 wait complete. 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9f662700 1 Processor -- start 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9f662700 1 -- start start 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9f662700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a98066f30 0x7f8a98113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9f662700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 0x7f8a98114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9f662700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98114b10 con 0x7f8a980fdce0 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9f662700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a981194b0 con 0x7f8a98066f30 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9cbfd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 0x7f8a98114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9cbfd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 0x7f8a98114480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45026/0 (socket says 172.21.15.33:45026) 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9cbfd700 1 -- 172.21.15.33:0/130627067 learned_addr learned my addr 172.21.15.33:0/130627067 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:43.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9d3fe700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a98066f30 0x7f8a98113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:43.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9cbfd700 1 -- 172.21.15.33:0/130627067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a98066f30 msgr2=0x7f8a98113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:43.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9cbfd700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a98066f30 0x7f8a98113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:43.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9cbfd700 1 -- 172.21.15.33:0/130627067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a88005040 con 0x7f8a980fdce0 2024-04-25T01:32:43.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.686+0000 7f8a9cbfd700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 0x7f8a98114480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800b3b0 tx=0x7f8a88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:43.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.687+0000 7f8a8dffb700 1 -- 172.21.15.33:0/130627067 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a8800e040 con 0x7f8a980fdce0 2024-04-25T01:32:43.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.687+0000 7f8a8dffb700 1 -- 172.21.15.33:0/130627067 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a88004290 con 0x7f8a980fdce0 2024-04-25T01:32:43.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.687+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a981196b0 con 0x7f8a980fdce0 2024-04-25T01:32:43.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.687+0000 7f8a8dffb700 1 -- 172.21.15.33:0/130627067 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a8801c630 con 0x7f8a980fdce0 2024-04-25T01:32:43.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.687+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a98119c20 con 0x7f8a980fdce0 2024-04-25T01:32:43.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.688+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a9804c710 con 0x7f8a980fdce0 2024-04-25T01:32:43.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.688+0000 7f8a8dffb700 1 -- 172.21.15.33:0/130627067 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8a8800bbf0 con 0x7f8a980fdce0 2024-04-25T01:32:43.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.689+0000 7f8a8dffb700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8a840692f0 0x7f8a8406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:43.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.689+0000 7f8a8dffb700 1 -- 172.21.15.33:0/130627067 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f8a88088350 con 0x7f8a980fdce0 2024-04-25T01:32:43.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.689+0000 7f8a9d3fe700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8a840692f0 0x7f8a8406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:43.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.689+0000 7f8a9d3fe700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8a840692f0 0x7f8a8406b7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8a94006fd0 tx=0x7f8a94008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:43.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.693+0000 7f8a8dffb700 1 -- 172.21.15.33:0/130627067 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8a880583b0 con 0x7f8a980fdce0 2024-04-25T01:32:43.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.898+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a980f8f30 con 0x7f8a840692f0 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.906+0000 7f8a8dffb700 1 -- 172.21.15.33:0/130627067 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f8a980f8f30 con 0x7f8a840692f0 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (3m) 2m ago 5m 25.9M - 0.25.0 c8568f914cd2 50a86f263578 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (5m) 2m ago 5m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (4m) 66s ago 4m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (3m) 2m ago 4m 80.6M - 9.4.7 954c08fa6188 4533d743079b 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:9283 running (6m) 2m ago 6m 469M - 17.2.7-902-gc61472c8 0de407c1b3af 22f7bb21dc39 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283 running (4m) 66s ago 4m 430M - 17.2.7-902-gc61472c8 0de407c1b3af 2440b5c4f74d 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (6m) 2m ago 6m 41.0M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (4m) 66s ago 4m 41.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:32:43.884 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (5m) 2m ago 5m 15.9M - 1.5.0 0da6a335fe13 c2f75e5aaf0f 2024-04-25T01:32:43.885 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (4m) 66s ago 4m 15.7M - 1.5.0 0da6a335fe13 ce8fbd90510d 2024-04-25T01:32:43.885 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (3m) 2m ago 3m 36.1M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:32:43.885 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (2m) 2m ago 2m 38.0M 4289M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:32:43.885 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (2m) 2m ago 2m 10.6M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:32:43.885 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (2m) 66s ago 2m 41.4M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:32:43.885 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (97s) 66s ago 96s 36.5M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:32:43.885 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (71s) 66s ago 71s 13.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:32:43.885 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (3m) 2m ago 4m 37.9M - 2.43.0 a07b618ecd1d 9109b23faf81 2024-04-25T01:32:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.908+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8a840692f0 msgr2=0x7f8a8406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.908+0000 7f8a9f662700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8a840692f0 0x7f8a8406b7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8a94006fd0 tx=0x7f8a94008040 comp rx=0 tx=0).stop 2024-04-25T01:32:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.908+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 msgr2=0x7f8a98114480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.908+0000 7f8a9f662700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 0x7f8a98114480 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800b3b0 tx=0x7f8a88009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.909+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 shutdown_connections 2024-04-25T01:32:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.909+0000 7f8a9f662700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a98066f30 0x7f8a98113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.909+0000 7f8a9f662700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8a840692f0 0x7f8a8406b7a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.909+0000 7f8a9f662700 1 --2- 172.21.15.33:0/130627067 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a980fdce0 0x7f8a98114480 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:43.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.909+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 >> 172.21.15.33:0/130627067 conn(0x7f8a980f7440 msgr2=0x7f8a980f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.909+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 shutdown_connections 2024-04-25T01:32:43.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:43.909+0000 7f8a9f662700 1 -- 172.21.15.33:0/130627067 wait complete. 2024-04-25T01:32:44.550 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T01:32:44.816 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:45.192 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:45 smithi033 ceph-mon[26353]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:45.192 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:45 smithi033 ceph-mon[26353]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:45.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:45 smithi138 ceph-mon[32977]: from='client.14422 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:45.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:45 smithi138 ceph-mon[32977]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:46.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:32:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.501+0000 7f049ea53700 1 -- 172.21.15.33:0/586920003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 msgr2=0x7f04980fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:46.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.501+0000 7f049ea53700 1 --2- 172.21.15.33:0/586920003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 0x7f04980fe830 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0488004940 tx=0x7f0488009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.501+0000 7f049ea53700 1 -- 172.21.15.33:0/586920003 shutdown_connections 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.501+0000 7f049ea53700 1 --2- 172.21.15.33:0/586920003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04980fed70 0x7f04980faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.501+0000 7f049ea53700 1 --2- 172.21.15.33:0/586920003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 0x7f04980fe830 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.501+0000 7f049ea53700 1 -- 172.21.15.33:0/586920003 >> 172.21.15.33:0/586920003 conn(0x7f04980f6be0 msgr2=0x7f04980f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.501+0000 7f049ea53700 1 -- 172.21.15.33:0/586920003 shutdown_connections 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.501+0000 7f049ea53700 1 -- 172.21.15.33:0/586920003 wait complete. 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.501+0000 7f049ea53700 1 Processor -- start 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f049ea53700 1 -- start start 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f049ea53700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 0x7f049810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f049ea53700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04980fed70 0x7f049810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f049ea53700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049810e4e0 con 0x7f04980fe420 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f049ea53700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0498112e80 con 0x7f04980fed70 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f0497fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 0x7f049810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f0497fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 0x7f049810d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40476/0 (socket says 172.21.15.33:40476) 2024-04-25T01:32:46.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f0497fff700 1 -- 172.21.15.33:0/459388732 learned_addr learned my addr 172.21.15.33:0/459388732 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f04977fe700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04980fed70 0x7f049810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f0497fff700 1 -- 172.21.15.33:0/459388732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04980fed70 msgr2=0x7f049810de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f0497fff700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04980fed70 0x7f049810de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f0497fff700 1 -- 172.21.15.33:0/459388732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488005040 con 0x7f04980fe420 2024-04-25T01:32:46.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.502+0000 7f0497fff700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 0x7f049810d910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f048800b3b0 tx=0x7f0488009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.503+0000 7f0494ff9700 1 -- 172.21.15.33:0/459388732 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f048800e040 con 0x7f04980fe420 2024-04-25T01:32:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.503+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0498113080 con 0x7f04980fe420 2024-04-25T01:32:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.503+0000 7f0494ff9700 1 -- 172.21.15.33:0/459388732 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f048800bcc0 con 0x7f04980fe420 2024-04-25T01:32:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.503+0000 7f0494ff9700 1 -- 172.21.15.33:0/459388732 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f048801c630 con 0x7f04980fe420 2024-04-25T01:32:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.503+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04981135f0 con 0x7f04980fe420 2024-04-25T01:32:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.504+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049804c710 con 0x7f04980fe420 2024-04-25T01:32:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.504+0000 7f0494ff9700 1 -- 172.21.15.33:0/459388732 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0488013420 con 0x7f04980fe420 2024-04-25T01:32:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.505+0000 7f0494ff9700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f04840692a0 0x7f048406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.505+0000 7f0494ff9700 1 -- 172.21.15.33:0/459388732 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0488087ca0 con 0x7f04980fe420 2024-04-25T01:32:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.505+0000 7f04977fe700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f04840692a0 0x7f048406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:46.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.505+0000 7f04977fe700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f04840692a0 0x7f048406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0480006fd0 tx=0x7f0480008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:46.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.509+0000 7f0494ff9700 1 -- 172.21.15.33:0/459388732 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f048808c070 con 0x7f04980fe420 2024-04-25T01:32:46.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:32:46.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.714+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f0498102470 con 0x7f04840692a0 2024-04-25T01:32:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.718+0000 7f0494ff9700 1 -- 172.21.15.33:0/459388732 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1170 (secure 0 0 0) 0x7f0498102470 con 0x7f04840692a0 2024-04-25T01:32:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T01:32:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-25T01:32:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:crash 2/2 2m ago 5m * 2024-04-25T01:32:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-04-25T01:32:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:mgr 2/2 2m ago 5m count:2 2024-04-25T01:32:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:mon 2/2 2m ago 5m smithi033:172.21.15.33=smithi033;smithi138:172.21.15.138=smithi138;count:2 2024-04-25T01:32:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-25T01:32:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:osd 6 2m ago - 2024-04-25T01:32:46.696 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.720+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f04840692a0 msgr2=0x7f048406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.720+0000 7f049ea53700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f04840692a0 0x7f048406b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0480006fd0 tx=0x7f0480008040 comp rx=0 tx=0).stop 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.720+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 msgr2=0x7f049810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.720+0000 7f049ea53700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 0x7f049810d910 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f048800b3b0 tx=0x7f0488009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.720+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 shutdown_connections 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.720+0000 7f049ea53700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f04840692a0 0x7f048406b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.720+0000 7f049ea53700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04980fe420 0x7f049810d910 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.720+0000 7f049ea53700 1 --2- 172.21.15.33:0/459388732 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04980fed70 0x7f049810de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.720+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 >> 172.21.15.33:0/459388732 conn(0x7f04980f6be0 msgr2=0x7f04980f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.721+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 shutdown_connections 2024-04-25T01:32:46.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:46.721+0000 7f049ea53700 1 -- 172.21.15.33:0/459388732 wait complete. 2024-04-25T01:32:47.346 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:47 smithi033 ceph-mon[26353]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:47.376 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T01:32:47.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:47 smithi138 ceph-mon[32977]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:47.643 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:48.421 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:48 smithi033 ceph-mon[26353]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:48.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:48 smithi138 ceph-mon[32977]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:49.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:49 smithi033 ceph-mon[26353]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.279+0000 7f1fd3080700 1 -- 172.21.15.33:0/344010259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 msgr2=0x7f1fcc102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.279+0000 7f1fd3080700 1 --2- 172.21.15.33:0/344010259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 0x7f1fcc102370 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8004940 tx=0x7f1fb8009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.280+0000 7f1fd3080700 1 -- 172.21.15.33:0/344010259 shutdown_connections 2024-04-25T01:32:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.280+0000 7f1fd3080700 1 --2- 172.21.15.33:0/344010259 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 0x7f1fcc102370 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.280+0000 7f1fd3080700 1 --2- 172.21.15.33:0/344010259 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fcc066ed0 0x7f1fcc0ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.280+0000 7f1fd3080700 1 -- 172.21.15.33:0/344010259 >> 172.21.15.33:0/344010259 conn(0x7f1fcc0f7480 msgr2=0x7f1fcc0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:49.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.280+0000 7f1fd3080700 1 -- 172.21.15.33:0/344010259 shutdown_connections 2024-04-25T01:32:49.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.280+0000 7f1fd3080700 1 -- 172.21.15.33:0/344010259 wait complete. 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.280+0000 7f1fd3080700 1 Processor -- start 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.280+0000 7f1fd3080700 1 -- start start 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fd3080700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fcc066ed0 0x7f1fcc10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fd3080700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 0x7f1fcc10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fd3080700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc10e540 con 0x7f1fcc0fff90 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fd3080700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc112ee0 con 0x7f1fcc066ed0 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fcbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 0x7f1fcc10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fcbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 0x7f1fcc10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40502/0 (socket says 172.21.15.33:40502) 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fcbfff700 1 -- 172.21.15.33:0/3795081731 learned_addr learned my addr 172.21.15.33:0/3795081731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fd0e1c700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fcc066ed0 0x7f1fcc10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fcbfff700 1 -- 172.21.15.33:0/3795081731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fcc066ed0 msgr2=0x7f1fcc10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fcbfff700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fcc066ed0 0x7f1fcc10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:49.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fcbfff700 1 -- 172.21.15.33:0/3795081731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb8005040 con 0x7f1fcc0fff90 2024-04-25T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fcbfff700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 0x7f1fcc10deb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800b3b0 tx=0x7f1fb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fc97fa700 1 -- 172.21.15.33:0/3795081731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fb800e040 con 0x7f1fcc0fff90 2024-04-25T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc1130e0 con 0x7f1fcc0fff90 2024-04-25T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fc97fa700 1 -- 172.21.15.33:0/3795081731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fb800bcc0 con 0x7f1fcc0fff90 2024-04-25T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fc97fa700 1 -- 172.21.15.33:0/3795081731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fb801c630 con 0x7f1fcc0fff90 2024-04-25T01:32:49.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.281+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc113650 con 0x7f1fcc0fff90 2024-04-25T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.282+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc04c710 con 0x7f1fcc0fff90 2024-04-25T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.283+0000 7f1fc97fa700 1 -- 172.21.15.33:0/3795081731 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f1fb8013980 con 0x7f1fcc0fff90 2024-04-25T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.283+0000 7f1fc97fa700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1fbc0692a0 0x7f1fbc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.283+0000 7f1fc97fa700 1 -- 172.21.15.33:0/3795081731 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1fb8087c80 con 0x7f1fcc0fff90 2024-04-25T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.283+0000 7f1fd0e1c700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1fbc0692a0 0x7f1fbc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:49.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.284+0000 7f1fd0e1c700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1fbc0692a0 0x7f1fbc06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0006fd0 tx=0x7f1fc0008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:49.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.288+0000 7f1fc97fa700 1 -- 172.21.15.33:0/3795081731 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1fb8057d00 con 0x7f1fcc0fff90 2024-04-25T01:32:49.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.489+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fcc0f8f70 con 0x7f1fbc0692a0 2024-04-25T01:32:49.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.490+0000 7f1fc97fa700 1 -- 172.21.15.33:0/3795081731 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f1fcc0f8f70 con 0x7f1fbc0692a0 2024-04-25T01:32:49.468 INFO:teuthology.orchestra.run.smithi033.stdout:HOST ADDR LABELS STATUS 2024-04-25T01:32:49.468 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 172.21.15.33 2024-04-25T01:32:49.468 INFO:teuthology.orchestra.run.smithi033.stdout:smithi138 172.21.15.138 2024-04-25T01:32:49.468 INFO:teuthology.orchestra.run.smithi033.stdout:2 hosts in cluster 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.492+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1fbc0692a0 msgr2=0x7f1fbc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.492+0000 7f1fd3080700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1fbc0692a0 0x7f1fbc06b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0006fd0 tx=0x7f1fc0008040 comp rx=0 tx=0).stop 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.492+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 msgr2=0x7f1fcc10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.492+0000 7f1fd3080700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 0x7f1fcc10deb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800b3b0 tx=0x7f1fb8009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.493+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 shutdown_connections 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.493+0000 7f1fd3080700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1fcc066ed0 0x7f1fcc10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.493+0000 7f1fd3080700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f1fbc0692a0 0x7f1fbc06b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.493+0000 7f1fd3080700 1 --2- 172.21.15.33:0/3795081731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1fcc0fff90 0x7f1fcc10deb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.493+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 >> 172.21.15.33:0/3795081731 conn(0x7f1fcc0f7480 msgr2=0x7f1fcc0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:49.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.493+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 shutdown_connections 2024-04-25T01:32:49.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:49.493+0000 7f1fd3080700 1 -- 172.21.15.33:0/3795081731 wait complete. 2024-04-25T01:32:49.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:49 smithi138 ceph-mon[32977]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:50.107 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T01:32:50.376 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:51.485 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:51 smithi033 ceph-mon[26353]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:51.485 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:51 smithi033 ceph-mon[26353]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:51.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:51 smithi138 ceph-mon[32977]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:51.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:51 smithi138 ceph-mon[32977]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:52.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.025+0000 7f52190de700 1 -- 172.21.15.33:0/3661044626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 msgr2=0x7f52140fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:52.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.025+0000 7f52190de700 1 --2- 172.21.15.33:0/3661044626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 0x7f52140fd3e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5208004940 tx=0x7f5208009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:52.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.025+0000 7f52190de700 1 -- 172.21.15.33:0/3661044626 shutdown_connections 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.025+0000 7f52190de700 1 --2- 172.21.15.33:0/3661044626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 0x7f52140fd3e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.025+0000 7f52190de700 1 --2- 172.21.15.33:0/3661044626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fbcc0 0x7f52140fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.025+0000 7f52190de700 1 -- 172.21.15.33:0/3661044626 >> 172.21.15.33:0/3661044626 conn(0x7f52140f7420 msgr2=0x7f52140f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.026+0000 7f52190de700 1 -- 172.21.15.33:0/3661044626 shutdown_connections 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.026+0000 7f52190de700 1 -- 172.21.15.33:0/3661044626 wait complete. 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.026+0000 7f52190de700 1 Processor -- start 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.026+0000 7f52190de700 1 -- start start 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.026+0000 7f52190de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fbcc0 0x7f52140ffc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.026+0000 7f52190de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 0x7f52140fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.026+0000 7f52190de700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5214100200 con 0x7f52140fcf70 2024-04-25T01:32:52.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.026+0000 7f52190de700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140fe7f0 con 0x7f52140fbcc0 2024-04-25T01:32:52.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.026+0000 7f521259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 0x7f52140fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f521259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 0x7f52140fe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40512/0 (socket says 172.21.15.33:40512) 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f521259c700 1 -- 172.21.15.33:0/2526951809 learned_addr learned my addr 172.21.15.33:0/2526951809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f5212d9d700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fbcc0 0x7f52140ffc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f521259c700 1 -- 172.21.15.33:0/2526951809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fbcc0 msgr2=0x7f52140ffc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f521259c700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fbcc0 0x7f52140ffc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f521259c700 1 -- 172.21.15.33:0/2526951809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208005040 con 0x7f52140fcf70 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f521259c700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 0x7f52140fe2b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f520800b3b0 tx=0x7f5208009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f51fb7fe700 1 -- 172.21.15.33:0/2526951809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520800e040 con 0x7f52140fcf70 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f51fb7fe700 1 -- 172.21.15.33:0/2526951809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f520800bcc0 con 0x7f52140fcf70 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52140fea40 con 0x7f52140fcf70 2024-04-25T01:32:52.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f51fb7fe700 1 -- 172.21.15.33:0/2526951809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520801c630 con 0x7f52140fcf70 2024-04-25T01:32:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.027+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52140ff000 con 0x7f52140fcf70 2024-04-25T01:32:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.028+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5214101b00 con 0x7f52140fcf70 2024-04-25T01:32:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.029+0000 7f51fb7fe700 1 -- 172.21.15.33:0/2526951809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5208013980 con 0x7f52140fcf70 2024-04-25T01:32:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.029+0000 7f51fb7fe700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f51fc0692f0 0x7f51fc06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.029+0000 7f51fb7fe700 1 -- 172.21.15.33:0/2526951809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5208087d20 con 0x7f52140fcf70 2024-04-25T01:32:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.029+0000 7f5212d9d700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f51fc0692f0 0x7f51fc06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:52.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.030+0000 7f5212d9d700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f51fc0692f0 0x7f51fc06b7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5204007fd0 tx=0x7f5204009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:52.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.034+0000 7f51fb7fe700 1 -- 172.21.15.33:0/2526951809 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f520808d020 con 0x7f52140fcf70 2024-04-25T01:32:52.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.238+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f521402b310 con 0x7f51fc0692f0 2024-04-25T01:32:52.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.240+0000 7f51fb7fe700 1 -- 172.21.15.33:0/2526951809 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f521402b310 con 0x7f51fc0692f0 2024-04-25T01:32:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T01:32:52.217 INFO:teuthology.orchestra.run.smithi033.stdout:smithi033 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330008Q400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-25T01:32:52.218 INFO:teuthology.orchestra.run.smithi033.stdout:smithi138 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6161003C400BGN 372G No 71s ago Has a FileSystem, LVM detected 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f51fc0692f0 msgr2=0x7f51fc06b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f51fc0692f0 0x7f51fc06b7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5204007fd0 tx=0x7f5204009040 comp rx=0 tx=0).stop 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 msgr2=0x7f52140fe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 0x7f52140fe2b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f520800b3b0 tx=0x7f5208009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 shutdown_connections 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fbcc0 0x7f52140ffc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f51fc0692f0 0x7f51fc06b7a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 --2- 172.21.15.33:0/2526951809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fcf70 0x7f52140fe2b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 >> 172.21.15.33:0/2526951809 conn(0x7f52140f7420 msgr2=0x7f5214066830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 shutdown_connections 2024-04-25T01:32:52.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:52.242+0000 7f52190de700 1 -- 172.21.15.33:0/2526951809 wait complete. 2024-04-25T01:32:52.778 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:32:52.788 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:32:52.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-25T01:32:53.052 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:53.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:53 smithi033 ceph-mon[26353]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:53.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:53 smithi033 ceph-mon[26353]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:53.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:53 smithi138 ceph-mon[32977]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:53.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:53 smithi138 ceph-mon[32977]: from='client.14434 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.725+0000 7fd809082700 1 -- 172.21.15.33:0/4227260075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 msgr2=0x7fd804102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.725+0000 7fd809082700 1 --2- 172.21.15.33:0/4227260075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 0x7fd804102350 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8004940 tx=0x7fd7f8009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.725+0000 7fd809082700 1 -- 172.21.15.33:0/4227260075 shutdown_connections 2024-04-25T01:32:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.725+0000 7fd809082700 1 --2- 172.21.15.33:0/4227260075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 0x7fd804102350 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.725+0000 7fd809082700 1 --2- 172.21.15.33:0/4227260075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd804066ed0 0x7fd8040ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.725+0000 7fd809082700 1 -- 172.21.15.33:0/4227260075 >> 172.21.15.33:0/4227260075 conn(0x7fd8040f7440 msgr2=0x7fd8040f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.725+0000 7fd809082700 1 -- 172.21.15.33:0/4227260075 shutdown_connections 2024-04-25T01:32:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.725+0000 7fd809082700 1 -- 172.21.15.33:0/4227260075 wait complete. 2024-04-25T01:32:54.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.726+0000 7fd809082700 1 Processor -- start 2024-04-25T01:32:54.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.726+0000 7fd809082700 1 -- start start 2024-04-25T01:32:54.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.726+0000 7fd809082700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd804066ed0 0x7fd8040fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:54.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.726+0000 7fd809082700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 0x7fd8040fb930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:54.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.726+0000 7fd809082700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040fbe70 con 0x7fd8040fff70 2024-04-25T01:32:54.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.726+0000 7fd809082700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8040fbfe0 con 0x7fd804066ed0 2024-04-25T01:32:54.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.726+0000 7fd80259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 0x7fd8040fb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.726+0000 7fd80259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 0x7fd8040fb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40532/0 (socket says 172.21.15.33:40532) 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.726+0000 7fd80259c700 1 -- 172.21.15.33:0/979753919 learned_addr learned my addr 172.21.15.33:0/979753919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd802d9d700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd804066ed0 0x7fd8040fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd80259c700 1 -- 172.21.15.33:0/979753919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd804066ed0 msgr2=0x7fd8040fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd80259c700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd804066ed0 0x7fd8040fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd80259c700 1 -- 172.21.15.33:0/979753919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f8005040 con 0x7fd8040fff70 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd802d9d700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd804066ed0 0x7fd8040fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd80259c700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 0x7fd8040fb930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009fd0 tx=0x7fd7f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd7eb7fe700 1 -- 172.21.15.33:0/979753919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f800e040 con 0x7fd8040fff70 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd7eb7fe700 1 -- 172.21.15.33:0/979753919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7f800bcc0 con 0x7fd8040fff70 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8040fc260 con 0x7fd8040fff70 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd7eb7fe700 1 -- 172.21.15.33:0/979753919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7f801c630 con 0x7fd8040fff70 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.727+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd804115ac0 con 0x7fd8040fff70 2024-04-25T01:32:54.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.728+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80404c710 con 0x7fd8040fff70 2024-04-25T01:32:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.729+0000 7fd7eb7fe700 1 -- 172.21.15.33:0/979753919 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fd7f8013980 con 0x7fd8040fff70 2024-04-25T01:32:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.729+0000 7fd7eb7fe700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd7ec0692a0 0x7fd7ec06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:32:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.729+0000 7fd7eb7fe700 1 -- 172.21.15.33:0/979753919 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd7f8088170 con 0x7fd8040fff70 2024-04-25T01:32:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.729+0000 7fd802d9d700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd7ec0692a0 0x7fd7ec06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:32:54.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.729+0000 7fd802d9d700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd7ec0692a0 0x7fd7ec06b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006fd0 tx=0x7fd7f4008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:32:54.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.733+0000 7fd7eb7fe700 1 -- 172.21.15.33:0/979753919 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd7f80581f0 con 0x7fd8040fff70 2024-04-25T01:32:54.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:54.964+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8040f8f30 con 0x7fd7ec0692a0 2024-04-25T01:32:55.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:55 smithi033 ceph-mon[26353]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:55.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:55 smithi138 ceph-mon[32977]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.492+0000 7fd7eb7fe700 1 -- 172.21.15.33:0/979753919 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd8040f8f30 con 0x7fd7ec0692a0 2024-04-25T01:32:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.495+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd7ec0692a0 msgr2=0x7fd7ec06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.495+0000 7fd809082700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd7ec0692a0 0x7fd7ec06b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4006fd0 tx=0x7fd7f4008040 comp rx=0 tx=0).stop 2024-04-25T01:32:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.495+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 msgr2=0x7fd8040fb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:32:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.495+0000 7fd809082700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 0x7fd8040fb930 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd7f8009fd0 tx=0x7fd7f8009e30 comp rx=0 tx=0).stop 2024-04-25T01:32:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.495+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 shutdown_connections 2024-04-25T01:32:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.495+0000 7fd809082700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd804066ed0 0x7fd8040fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.495+0000 7fd809082700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fd7ec0692a0 0x7fd7ec06b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:56.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.495+0000 7fd809082700 1 --2- 172.21.15.33:0/979753919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8040fff70 0x7fd8040fb930 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:32:56.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.495+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 >> 172.21.15.33:0/979753919 conn(0x7fd8040f7440 msgr2=0x7fd8040f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:32:56.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.496+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 shutdown_connections 2024-04-25T01:32:56.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:32:56.496+0000 7fd809082700 1 -- 172.21.15.33:0/979753919 wait complete. 2024-04-25T01:32:56.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:56 smithi033 ceph-mon[26353]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:56.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:56 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-25T01:32:56.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:56 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-25T01:32:56.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:56 smithi033 ceph-mon[26353]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T01:32:56.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:56 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-25T01:32:56.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:56 smithi033 ceph-mon[26353]: pgmap v169: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:56.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:56 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[26349]: 2024-04-25T01:32:56.469+0000 7f5d29a8d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:32:56.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:56 smithi138 ceph-mon[32977]: from='client.14438 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:32:56.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:56 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-25T01:32:56.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:56 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-25T01:32:56.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:56 smithi138 ceph-mon[32977]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T01:32:56.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:56 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-25T01:32:56.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:56 smithi138 ceph-mon[32977]: pgmap v169: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:57.151 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T01:32:57.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: fsmap cephfs:0 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: Saving service mds.cephfs spec with placement count:4 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:32:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:32:57.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.qqhhcz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:32:57.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.qqhhcz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:32:57.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:32:57.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:57 smithi033 ceph-mon[26353]: Deploying daemon mds.cephfs.smithi033.qqhhcz on smithi033 2024-04-25T01:32:57.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T01:32:57.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-25T01:32:57.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: fsmap cephfs:0 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: Saving service mds.cephfs spec with placement count:4 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.qqhhcz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:32:57.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.qqhhcz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:32:57.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:32:57.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:57 smithi138 ceph-mon[32977]: Deploying daemon mds.cephfs.smithi033.qqhhcz on smithi033 2024-04-25T01:32:58.495 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:32:58.625 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:58 smithi033 ceph-mon[26353]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T01:32:58.626 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:58 smithi033 ceph-mon[26353]: pgmap v173: 65 pgs: 5 creating+peering, 1 active+clean, 59 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:58.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:58 smithi138 ceph-mon[32977]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T01:32:58.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:58 smithi138 ceph-mon[32977]: pgmap v173: 65 pgs: 5 creating+peering, 1 active+clean, 59 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:32:59.647 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:59 smithi033 ceph-mon[26353]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:32:59.647 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:32:59 smithi033 ceph-mon[26353]: osdmap e45: 6 total, 6 up, 6 in 2024-04-25T01:32:59.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:59 smithi138 ceph-mon[32977]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-25T01:32:59.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:32:59 smithi138 ceph-mon[32977]: osdmap e45: 6 total, 6 up, 6 in 2024-04-25T01:33:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.168+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/2138231049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 msgr2=0x7fe7d40fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.168+0000 7fe7d8e4b700 1 --2- 172.21.15.33:0/2138231049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 0x7fe7d40fc0d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4009ce0 tx=0x7fe7c4004d10 comp rx=0 tx=0).stop 2024-04-25T01:33:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.168+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/2138231049 shutdown_connections 2024-04-25T01:33:00.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.168+0000 7fe7d8e4b700 1 --2- 172.21.15.33:0/2138231049 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7d40fcf70 0x7fe7d40fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:00.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.168+0000 7fe7d8e4b700 1 --2- 172.21.15.33:0/2138231049 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 0x7fe7d40fc0d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:00.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.168+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/2138231049 >> 172.21.15.33:0/2138231049 conn(0x7fe7d40f7440 msgr2=0x7fe7d40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:00.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.168+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/2138231049 shutdown_connections 2024-04-25T01:33:00.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.168+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/2138231049 wait complete. 2024-04-25T01:33:00.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.169+0000 7fe7d8e4b700 1 Processor -- start 2024-04-25T01:33:00.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.169+0000 7fe7d8e4b700 1 -- start start 2024-04-25T01:33:00.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.169+0000 7fe7d8e4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 0x7fe7d4066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:00.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.169+0000 7fe7d8e4b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7d40fcf70 0x7fe7d41128b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:00.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.169+0000 7fe7d8e4b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d4112df0 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.169+0000 7fe7d8e4b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d4112f60 con 0x7fe7d40fcf70 2024-04-25T01:33:00.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.169+0000 7fe7d259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 0x7fe7d4066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:00.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.169+0000 7fe7d259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 0x7fe7d4066cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41338/0 (socket says 172.21.15.33:41338) 2024-04-25T01:33:00.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.169+0000 7fe7d259c700 1 -- 172.21.15.33:0/945435937 learned_addr learned my addr 172.21.15.33:0/945435937 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:00.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7d1d9b700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7d40fcf70 0x7fe7d41128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:00.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7d259c700 1 -- 172.21.15.33:0/945435937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7d40fcf70 msgr2=0x7fe7d41128b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:00.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7d259c700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7d40fcf70 0x7fe7d41128b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:00.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7d259c700 1 -- 172.21.15.33:0/945435937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c4009940 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7d1d9b700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7d40fcf70 0x7fe7d41128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:33:00.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7d259c700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 0x7fe7d4066cb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4004740 tx=0x7fe7c4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:00.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7c2ffd700 1 -- 172.21.15.33:0/945435937 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7c400e040 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7c2ffd700 1 -- 172.21.15.33:0/945435937 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7c40128e0 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d41131e0 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7c2ffd700 1 -- 172.21.15.33:0/945435937 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7c401ca80 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.170+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d41137a0 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.171+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d404c710 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.172+0000 7fe7c2ffd700 1 -- 172.21.15.33:0/945435937 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fe7c4012a50 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.172+0000 7fe7c2ffd700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fe7bc0692a0 0x7fe7bc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:00.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.172+0000 7fe7c2ffd700 1 -- 172.21.15.33:0/945435937 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe7c4088db0 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.172+0000 7fe7d1d9b700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fe7bc0692a0 0x7fe7bc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:00.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.173+0000 7fe7d1d9b700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fe7bc0692a0 0x7fe7bc06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8006fd0 tx=0x7fe7c8008040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:00.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.177+0000 7fe7c2ffd700 1 -- 172.21.15.33:0/945435937 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe7c4058b70 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.421+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe7d4113c70 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.422+0000 7fe7c2ffd700 1 -- 172.21.15.33:0/945435937 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fe7c4058700 con 0x7fe7d40fbcc0 2024-04-25T01:33:00.399 INFO:teuthology.orchestra.run.smithi033.stdout:e2 2024-04-25T01:33:00.399 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:33:00.399 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:33:00.399 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:33:00.399 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:33:00.399 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 2 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:32:56.470391+0000 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:33:00.400 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 1 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:in 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 0 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:33:00.401 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:33:00.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fe7bc0692a0 msgr2=0x7fe7bc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:00.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fe7bc0692a0 0x7fe7bc06b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8006fd0 tx=0x7fe7c8008040 comp rx=0 tx=0).stop 2024-04-25T01:33:00.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 msgr2=0x7fe7d4066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:00.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 0x7fe7d4066cb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe7c4004740 tx=0x7fe7c4004d10 comp rx=0 tx=0).stop 2024-04-25T01:33:00.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 shutdown_connections 2024-04-25T01:33:00.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fe7bc0692a0 0x7fe7bc06b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:00.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7d40fbcc0 0x7fe7d4066cb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:00.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 --2- 172.21.15.33:0/945435937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7d40fcf70 0x7fe7d41128b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:00.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 >> 172.21.15.33:0/945435937 conn(0x7fe7d40f7440 msgr2=0x7fe7d40f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:00.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 shutdown_connections 2024-04-25T01:33:00.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:00.424+0000 7fe7d8e4b700 1 -- 172.21.15.33:0/945435937 wait complete. 2024-04-25T01:33:00.403 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 2 2024-04-25T01:33:00.696 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:00 smithi033 ceph-mon[26353]: pgmap v175: 65 pgs: 5 creating+peering, 34 active+clean, 26 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:33:00.696 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:00 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/945435937' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:33:00.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:00 smithi138 ceph-mon[32977]: pgmap v175: 65 pgs: 5 creating+peering, 34 active+clean, 26 unknown; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T01:33:00.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:00 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/945435937' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:33:01.300 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:33:01.310 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:33:01.310 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-04-25T01:33:01.574 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:01.982 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.jwzxlz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.jwzxlz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: Deploying daemon mds.cephfs.smithi138.jwzxlz on smithi138 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: daemon mds.cephfs.smithi033.qqhhcz assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: Cluster is now healthy 2024-04-25T01:33:01.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] up:boot 2024-04-25T01:33:01.984 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: fsmap cephfs:1 {0=cephfs.smithi033.qqhhcz=up:creating} 2024-04-25T01:33:01.984 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:33:01.984 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: daemon mds.cephfs.smithi033.qqhhcz is now active in filesystem cephfs as rank 0 2024-04-25T01:33:01.984 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:33:01.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:01.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:01.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:01.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.jwzxlz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.jwzxlz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: Deploying daemon mds.cephfs.smithi138.jwzxlz on smithi138 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: daemon mds.cephfs.smithi033.qqhhcz assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: Cluster is now healthy 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] up:boot 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: fsmap cephfs:1 {0=cephfs.smithi033.qqhhcz=up:creating} 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: daemon mds.cephfs.smithi033.qqhhcz is now active in filesystem cephfs as rank 0 2024-04-25T01:33:01.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:33:03.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:02 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] up:active 2024-04-25T01:33:03.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:02 smithi138 ceph-mon[32977]: fsmap cephfs:1 {0=cephfs.smithi033.qqhhcz=up:active} 2024-04-25T01:33:03.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:02 smithi138 ceph-mon[32977]: pgmap v176: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 2 op/s 2024-04-25T01:33:03.068 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:02 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] up:active 2024-04-25T01:33:03.068 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:02 smithi033 ceph-mon[26353]: fsmap cephfs:1 {0=cephfs.smithi033.qqhhcz=up:active} 2024-04-25T01:33:03.068 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:02 smithi033 ceph-mon[26353]: pgmap v176: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 2 op/s 2024-04-25T01:33:03.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.114+0000 7f029a1d4700 1 -- 172.21.15.33:0/4263590481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fcf70 msgr2=0x7f02940fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.114+0000 7f029a1d4700 1 --2- 172.21.15.33:0/4263590481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fcf70 0x7f02940fd3e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0288004940 tx=0x7f0288009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.114+0000 7f029a1d4700 1 -- 172.21.15.33:0/4263590481 shutdown_connections 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.114+0000 7f029a1d4700 1 --2- 172.21.15.33:0/4263590481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fcf70 0x7f02940fd3e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.114+0000 7f029a1d4700 1 --2- 172.21.15.33:0/4263590481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02940fbcc0 0x7f02940fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.114+0000 7f029a1d4700 1 -- 172.21.15.33:0/4263590481 >> 172.21.15.33:0/4263590481 conn(0x7f02940f7440 msgr2=0x7f02940f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.114+0000 7f029a1d4700 1 -- 172.21.15.33:0/4263590481 shutdown_connections 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.114+0000 7f029a1d4700 1 -- 172.21.15.33:0/4263590481 wait complete. 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f029a1d4700 1 Processor -- start 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f029a1d4700 1 -- start start 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f029a1d4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fbcc0 0x7f0294066d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f029a1d4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02940fcf70 0x7f02941126c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f029a1d4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0294112c90 con 0x7f02940fbcc0 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f029a1d4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0294112dd0 con 0x7f02940fcf70 2024-04-25T01:33:03.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f02937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fbcc0 0x7f0294066d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f02937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fbcc0 0x7f0294066d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41372/0 (socket says 172.21.15.33:41372) 2024-04-25T01:33:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f02937fe700 1 -- 172.21.15.33:0/30057345 learned_addr learned my addr 172.21.15.33:0/30057345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.115+0000 7f0292ffd700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02940fcf70 0x7f02941126c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.116+0000 7f02937fe700 1 -- 172.21.15.33:0/30057345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02940fcf70 msgr2=0x7f02941126c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.116+0000 7f02937fe700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02940fcf70 0x7f02941126c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:03.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.116+0000 7f02937fe700 1 -- 172.21.15.33:0/30057345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0288005040 con 0x7f02940fbcc0 2024-04-25T01:33:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.116+0000 7f02937fe700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fbcc0 0x7f0294066d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f028400b790 tx=0x7f0284009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.116+0000 7f027bfff700 1 -- 172.21.15.33:0/30057345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f028400f770 con 0x7f02940fbcc0 2024-04-25T01:33:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.116+0000 7f027bfff700 1 -- 172.21.15.33:0/30057345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f028400fda0 con 0x7f02940fbcc0 2024-04-25T01:33:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.116+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02941130b0 con 0x7f02940fbcc0 2024-04-25T01:33:03.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.116+0000 7f027bfff700 1 -- 172.21.15.33:0/30057345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f028400d310 con 0x7f02940fbcc0 2024-04-25T01:33:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.116+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0294113670 con 0x7f02940fbcc0 2024-04-25T01:33:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.117+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029404c710 con 0x7f02940fbcc0 2024-04-25T01:33:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.117+0000 7f027bfff700 1 -- 172.21.15.33:0/30057345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0284010420 con 0x7f02940fbcc0 2024-04-25T01:33:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.118+0000 7f027bfff700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f027c0692a0 0x7f027c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.118+0000 7f027bfff700 1 -- 172.21.15.33:0/30057345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f02840871c0 con 0x7f02940fbcc0 2024-04-25T01:33:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.118+0000 7f0292ffd700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f027c0692a0 0x7f027c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:03.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.118+0000 7f0292ffd700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f027c0692a0 0x7f027c06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f028800b3b0 tx=0x7f0288009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:03.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.122+0000 7f027bfff700 1 -- 172.21.15.33:0/30057345 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0284056f80 con 0x7f02940fbcc0 2024-04-25T01:33:03.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.366+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f0294113240 con 0x7f02940fbcc0 2024-04-25T01:33:03.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.934+0000 7f027bfff700 1 -- 172.21.15.33:0/30057345 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v5) v1 ==== 105+0+0 (secure 0 0 0) 0x7f0294113240 con 0x7f02940fbcc0 2024-04-25T01:33:03.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f027c0692a0 msgr2=0x7f027c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f027c0692a0 0x7f027c06b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f028800b3b0 tx=0x7f0288009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fbcc0 msgr2=0x7f0294066d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fbcc0 0x7f0294066d10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f028400b790 tx=0x7f0284009250 comp rx=0 tx=0).stop 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 shutdown_connections 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f027c0692a0 0x7f027c06b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02940fbcc0 0x7f0294066d10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 --2- 172.21.15.33:0/30057345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02940fcf70 0x7f02941126c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 >> 172.21.15.33:0/30057345 conn(0x7f02940f7440 msgr2=0x7f02940f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 shutdown_connections 2024-04-25T01:33:03.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:03.937+0000 7f029a1d4700 1 -- 172.21.15.33:0/30057345 wait complete. 2024-04-25T01:33:04.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:03 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/30057345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-25T01:33:04.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:04.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:04.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:04.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hobkqr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:33:04.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hobkqr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:33:04.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:03 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:04.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:03 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/30057345' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-25T01:33:04.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:04.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:04.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:04.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hobkqr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:33:04.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hobkqr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:33:04.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:03 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:05.160 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: Deploying daemon mds.cephfs.smithi033.hobkqr on smithi033 2024-04-25T01:33:05.160 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:33:05.160 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/30057345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-25T01:33:05.160 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] up:boot 2024-04-25T01:33:05.160 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: daemon mds.cephfs.smithi138.jwzxlz assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T01:33:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-25T01:33:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:33:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: fsmap cephfs:1 {0=cephfs.smithi033.qqhhcz=up:active} 1 up:standby 2024-04-25T01:33:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:33:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:creating} 2024-04-25T01:33:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: daemon mds.cephfs.smithi138.jwzxlz is now active in filesystem cephfs as rank 1 2024-04-25T01:33:05.161 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:04 smithi033 ceph-mon[26353]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 403 B/s wr, 3 op/s 2024-04-25T01:33:05.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: Deploying daemon mds.cephfs.smithi033.hobkqr on smithi033 2024-04-25T01:33:05.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T01:33:05.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/30057345' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-25T01:33:05.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] up:boot 2024-04-25T01:33:05.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: daemon mds.cephfs.smithi138.jwzxlz assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T01:33:05.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-25T01:33:05.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T01:33:05.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: fsmap cephfs:1 {0=cephfs.smithi033.qqhhcz=up:active} 1 up:standby 2024-04-25T01:33:05.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:33:05.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:creating} 2024-04-25T01:33:05.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: daemon mds.cephfs.smithi138.jwzxlz is now active in filesystem cephfs as rank 1 2024-04-25T01:33:05.269 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:04 smithi138 ceph-mon[32977]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 403 B/s wr, 3 op/s 2024-04-25T01:33:05.705 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:33:05.720 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:33:05.720 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-04-25T01:33:05.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:05 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] up:active 2024-04-25T01:33:05.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:05 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2024-04-25T01:33:06.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:05 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] up:active 2024-04-25T01:33:06.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:05 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2024-04-25T01:33:06.269 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:07.439 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s wr, 8 op/s 2024-04-25T01:33:07.439 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:07.439 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:07.439 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:07.439 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.rvxnby", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:33:07.439 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.rvxnby", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:33:07.440 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:07.440 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: Deploying daemon mds.cephfs.smithi138.rvxnby on smithi138 2024-04-25T01:33:07.440 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-25T01:33:07.440 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: Cluster is now healthy 2024-04-25T01:33:07.440 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] up:boot 2024-04-25T01:33:07.440 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 1 up:standby 2024-04-25T01:33:07.440 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:07 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:33:07.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s wr, 8 op/s 2024-04-25T01:33:07.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.rvxnby", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.rvxnby", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: Deploying daemon mds.cephfs.smithi138.rvxnby on smithi138 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: Cluster is now healthy 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] up:boot 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 1 up:standby 2024-04-25T01:33:07.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:07 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:33:07.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.846+0000 7ff0d356c700 1 -- 172.21.15.33:0/805376151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fce60 msgr2=0x7ff0cc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:07.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.846+0000 7ff0d356c700 1 --2- 172.21.15.33:0/805376151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fce60 0x7ff0cc0fd270 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8004940 tx=0x7ff0c8009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:07.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.846+0000 7ff0d356c700 1 -- 172.21.15.33:0/805376151 shutdown_connections 2024-04-25T01:33:07.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.846+0000 7ff0d356c700 1 --2- 172.21.15.33:0/805376151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc0fdfd0 0x7ff0cc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:07.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.846+0000 7ff0d356c700 1 --2- 172.21.15.33:0/805376151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fce60 0x7ff0cc0fd270 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:07.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.846+0000 7ff0d356c700 1 -- 172.21.15.33:0/805376151 >> 172.21.15.33:0/805376151 conn(0x7ff0cc0f8540 msgr2=0x7ff0cc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d356c700 1 -- 172.21.15.33:0/805376151 shutdown_connections 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d356c700 1 -- 172.21.15.33:0/805376151 wait complete. 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d356c700 1 Processor -- start 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d356c700 1 -- start start 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d356c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc0fce60 0x7ff0cc10d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d356c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fdfd0 0x7ff0cc10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d356c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc10e3f0 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d356c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc10e560 con 0x7ff0cc0fce60 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d0b07700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fdfd0 0x7ff0cc10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d0b07700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fdfd0 0x7ff0cc10ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55998/0 (socket says 172.21.15.33:55998) 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d0b07700 1 -- 172.21.15.33:0/2714640171 learned_addr learned my addr 172.21.15.33:0/2714640171 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.847+0000 7ff0d1308700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc0fce60 0x7ff0cc10d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0d0b07700 1 -- 172.21.15.33:0/2714640171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc0fce60 msgr2=0x7ff0cc10d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0d0b07700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc0fce60 0x7ff0cc10d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0d0b07700 1 -- 172.21.15.33:0/2714640171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c8005040 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0d1308700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc0fce60 0x7ff0cc10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0d0b07700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fdfd0 0x7ff0cc10ddf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00eb40 tx=0x7ff0bc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0c1ffb700 1 -- 172.21.15.33:0/2714640171 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0bc007cc0 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0c1ffb700 1 -- 172.21.15.33:0/2714640171 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0bc00f040 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc112f60 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0c1ffb700 1 -- 172.21.15.33:0/2714640171 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0bc00ca10 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.848+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc113470 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.849+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0cc04c710 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.850+0000 7ff0c1ffb700 1 -- 172.21.15.33:0/2714640171 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff0bc00cb70 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.850+0000 7ff0c1ffb700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff0b80692a0 0x7ff0b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.850+0000 7ff0c1ffb700 1 -- 172.21.15.33:0/2714640171 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff0bc086c80 con 0x7ff0cc0fdfd0 2024-04-25T01:33:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.850+0000 7ff0d1308700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff0b80692a0 0x7ff0b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:07.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.850+0000 7ff0d1308700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff0b80692a0 0x7ff0b806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009b30 tx=0x7ff0c8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:07.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:07.855+0000 7ff0c1ffb700 1 -- 172.21.15.33:0/2714640171 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff0bc04e250 con 0x7ff0cc0fdfd0 2024-04-25T01:33:08.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.096+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7ff0cc113710 con 0x7ff0cc0fdfd0 2024-04-25T01:33:08.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.283+0000 7ff0c1ffb700 1 -- 172.21.15.33:0/2714640171 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7ff0bc056af0 con 0x7ff0cc0fdfd0 2024-04-25T01:33:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff0b80692a0 msgr2=0x7ff0b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff0b80692a0 0x7ff0b806b750 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009b30 tx=0x7ff0c8009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fdfd0 msgr2=0x7ff0cc10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fdfd0 0x7ff0cc10ddf0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00eb40 tx=0x7ff0bc00b040 comp rx=0 tx=0).stop 2024-04-25T01:33:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 shutdown_connections 2024-04-25T01:33:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0cc0fce60 0x7ff0cc10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff0b80692a0 0x7ff0b806b750 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:08.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 --2- 172.21.15.33:0/2714640171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0cc0fdfd0 0x7ff0cc10ddf0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:08.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 >> 172.21.15.33:0/2714640171 conn(0x7ff0cc0f8540 msgr2=0x7ff0cc0f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:08.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 shutdown_connections 2024-04-25T01:33:08.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:08.286+0000 7ff0d356c700 1 -- 172.21.15.33:0/2714640171 wait complete. 2024-04-25T01:33:08.398 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:08 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2714640171' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T01:33:08.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:08 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2714640171' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T01:33:08.949 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:33:08.962 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:33:08.963 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-04-25T01:33:09.498 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:09.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-25T01:33:09.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2714640171' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T01:33:09.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 1 up:standby 2024-04-25T01:33:09.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 1 up:standby-replay 2024-04-25T01:33:09.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:09 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s wr, 7 op/s 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2714640171' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 1 up:standby 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 1 up:standby-replay 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:09.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:09 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:33:10.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:10 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] up:boot 2024-04-25T01:33:10.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:10 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] up:active 2024-04-25T01:33:10.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:10 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 1 up:standby-replay 1 up:standby 2024-04-25T01:33:10.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:10 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:33:10.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:10 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:33:10.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:10 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] up:boot 2024-04-25T01:33:10.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:10 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] up:active 2024-04-25T01:33:10.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:10 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 1 up:standby-replay 1 up:standby 2024-04-25T01:33:10.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:10 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:33:10.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:10 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:33:11.560 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:11 smithi033 ceph-mon[26353]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 528 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-25T01:33:11.560 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:11 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] up:standby-replay 2024-04-25T01:33:11.560 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:11 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:33:11.561 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:11 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:11.561 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:11 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:11.561 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:11 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.606+0000 7faa425e1700 1 -- 172.21.15.33:0/34850024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fbcc0 msgr2=0x7faa3c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.606+0000 7faa425e1700 1 --2- 172.21.15.33:0/34850024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fbcc0 0x7faa3c0fc0d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faa2c004940 tx=0x7faa2c009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 -- 172.21.15.33:0/34850024 shutdown_connections 2024-04-25T01:33:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 --2- 172.21.15.33:0/34850024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa3c0fcf70 0x7faa3c0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:11.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 --2- 172.21.15.33:0/34850024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fbcc0 0x7faa3c0fc0d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 -- 172.21.15.33:0/34850024 >> 172.21.15.33:0/34850024 conn(0x7faa3c0f7440 msgr2=0x7faa3c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 -- 172.21.15.33:0/34850024 shutdown_connections 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 -- 172.21.15.33:0/34850024 wait complete. 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 Processor -- start 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 -- start start 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa3c0fbcc0 0x7faa3c066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fcf70 0x7faa3c112740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c112c80 con 0x7faa3c0fcf70 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.607+0000 7faa425e1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3c112dc0 con 0x7faa3c0fbcc0 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa3b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fcf70 0x7faa3c112740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa3b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fcf70 0x7faa3c112740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56014/0 (socket says 172.21.15.33:56014) 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa3b7fe700 1 -- 172.21.15.33:0/74195853 learned_addr learned my addr 172.21.15.33:0/74195853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:11.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa3bfff700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa3c0fbcc0 0x7faa3c066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa3b7fe700 1 -- 172.21.15.33:0/74195853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa3c0fbcc0 msgr2=0x7faa3c066cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa3b7fe700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa3c0fbcc0 0x7faa3c066cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa3b7fe700 1 -- 172.21.15.33:0/74195853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2c005040 con 0x7faa3c0fcf70 2024-04-25T01:33:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa3bfff700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa3c0fbcc0 0x7faa3c066cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:33:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa3b7fe700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fcf70 0x7faa3c112740 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa30002a50 tx=0x7faa300046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:11.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa38ff9700 1 -- 172.21.15.33:0/74195853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa30007cc0 con 0x7faa3c0fcf70 2024-04-25T01:33:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c113020 con 0x7faa3c0fcf70 2024-04-25T01:33:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa38ff9700 1 -- 172.21.15.33:0/74195853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa3000f040 con 0x7faa3c0fcf70 2024-04-25T01:33:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa38ff9700 1 -- 172.21.15.33:0/74195853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa30009640 con 0x7faa3c0fcf70 2024-04-25T01:33:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.608+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3c1135b0 con 0x7faa3c0fcf70 2024-04-25T01:33:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.609+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa3c04c710 con 0x7faa3c0fcf70 2024-04-25T01:33:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.610+0000 7faa38ff9700 1 -- 172.21.15.33:0/74195853 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7faa300097c0 con 0x7faa3c0fcf70 2024-04-25T01:33:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.610+0000 7faa38ff9700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa240692a0 0x7faa2406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.610+0000 7faa38ff9700 1 -- 172.21.15.33:0/74195853 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7faa30086eb0 con 0x7faa3c0fcf70 2024-04-25T01:33:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.610+0000 7faa3bfff700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa240692a0 0x7faa2406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.611+0000 7faa3bfff700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa240692a0 0x7faa2406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00b3b0 tx=0x7faa2c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.615+0000 7faa38ff9700 1 -- 172.21.15.33:0/74195853 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7faa30056c70 con 0x7faa3c0fcf70 2024-04-25T01:33:11.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:11 smithi138 ceph-mon[32977]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 528 B/s rd, 2.1 KiB/s wr, 7 op/s 2024-04-25T01:33:11.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:11 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] up:standby-replay 2024-04-25T01:33:11.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:11 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:33:11.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:11 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:11.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:11 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:11.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:11 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:11.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:11.862+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7faa3c1047b0 con 0x7faa3c0fcf70 2024-04-25T01:33:12.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.309+0000 7faa38ff9700 1 -- 172.21.15.33:0/74195853 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v14) v1 ==== 133+0+0 (secure 0 0 0) 0x7faa30056800 con 0x7faa3c0fcf70 2024-04-25T01:33:12.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.311+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa240692a0 msgr2=0x7faa2406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.311+0000 7faa425e1700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa240692a0 0x7faa2406b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faa2c00b3b0 tx=0x7faa2c009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.311+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fcf70 msgr2=0x7faa3c112740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.311+0000 7faa425e1700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fcf70 0x7faa3c112740 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7faa30002a50 tx=0x7faa300046b0 comp rx=0 tx=0).stop 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.311+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 shutdown_connections 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.311+0000 7faa425e1700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa3c0fbcc0 0x7faa3c066cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.311+0000 7faa425e1700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7faa240692a0 0x7faa2406b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.311+0000 7faa425e1700 1 --2- 172.21.15.33:0/74195853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa3c0fcf70 0x7faa3c112740 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.311+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 >> 172.21.15.33:0/74195853 conn(0x7faa3c0f7440 msgr2=0x7faa3c0f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.312+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 shutdown_connections 2024-04-25T01:33:12.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:12.312+0000 7faa425e1700 1 -- 172.21.15.33:0/74195853 wait complete. 2024-04-25T01:33:12.290 INFO:teuthology.orchestra.run.smithi033.stderr:inline data disabled 2024-04-25T01:33:12.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:12 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/74195853' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-25T01:33:12.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:12 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/74195853' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-25T01:33:12.971 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T01:33:12.982 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:33:12.982 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T01:33:13.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:13 smithi033 ceph-mon[26353]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T01:33:13.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:13 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/74195853' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-25T01:33:13.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:13 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] up:active 2024-04-25T01:33:13.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:13 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:33:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:13.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:13 smithi138 ceph-mon[32977]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 938 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T01:33:13.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:13 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/74195853' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-25T01:33:13.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:13 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] up:active 2024-04-25T01:33:13.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:13 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:33:14.632 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:14 smithi138 ceph-mon[32977]: mds.? [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] up:standby-replay 2024-04-25T01:33:14.633 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:14 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:33:14.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:14 smithi033 ceph-mon[26353]: mds.? [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] up:standby-replay 2024-04-25T01:33:14.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:14 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:33:15.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:15 smithi033 ceph-mon[26353]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T01:33:15.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:15 smithi138 ceph-mon[32977]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T01:33:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.085+0000 7f8b25721700 1 -- 172.21.15.33:0/1472991057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 msgr2=0x7f8b200faf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.085+0000 7f8b25721700 1 --2- 172.21.15.33:0/1472991057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 0x7f8b200faf80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8b10004940 tx=0x7f8b10009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:16.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.085+0000 7f8b25721700 1 -- 172.21.15.33:0/1472991057 shutdown_connections 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.085+0000 7f8b25721700 1 --2- 172.21.15.33:0/1472991057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 0x7f8b200faf80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.085+0000 7f8b25721700 1 --2- 172.21.15.33:0/1472991057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b200fe4f0 0x7f8b200fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.085+0000 7f8b25721700 1 -- 172.21.15.33:0/1472991057 >> 172.21.15.33:0/1472991057 conn(0x7f8b200f6bd0 msgr2=0x7f8b200f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.085+0000 7f8b25721700 1 -- 172.21.15.33:0/1472991057 shutdown_connections 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.085+0000 7f8b25721700 1 -- 172.21.15.33:0/1472991057 wait complete. 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b25721700 1 Processor -- start 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b25721700 1 -- start start 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b25721700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b200fe4f0 0x7f8b200fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b25721700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 0x7f8b200fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b25721700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200fe100 con 0x7f8b200fee40 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b25721700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b200fc6f0 con 0x7f8b200fe4f0 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b1e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 0x7f8b200fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b1e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 0x7f8b200fc1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58288/0 (socket says 172.21.15.33:58288) 2024-04-25T01:33:16.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/1229261249 learned_addr learned my addr 172.21.15.33:0/1229261249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b1effd700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b200fe4f0 0x7f8b200fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/1229261249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b200fe4f0 msgr2=0x7f8b200fdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b1e7fc700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b200fe4f0 0x7f8b200fdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.086+0000 7f8b1e7fc700 1 -- 172.21.15.33:0/1229261249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b10005040 con 0x7f8b200fee40 2024-04-25T01:33:16.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.087+0000 7f8b1e7fc700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 0x7f8b200fc1b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000b3b0 tx=0x7f8b10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.087+0000 7f8b177fe700 1 -- 172.21.15.33:0/1229261249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1000e040 con 0x7f8b200fee40 2024-04-25T01:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.087+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b200fc940 con 0x7f8b200fee40 2024-04-25T01:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.087+0000 7f8b177fe700 1 -- 172.21.15.33:0/1229261249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b1000bbf0 con 0x7f8b200fee40 2024-04-25T01:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.087+0000 7f8b177fe700 1 -- 172.21.15.33:0/1229261249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b1001c630 con 0x7f8b200fee40 2024-04-25T01:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.087+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b200fcf00 con 0x7f8b200fee40 2024-04-25T01:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.088+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b2004c710 con 0x7f8b200fee40 2024-04-25T01:33:16.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.088+0000 7f8b177fe700 1 -- 172.21.15.33:0/1229261249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8b10013420 con 0x7f8b200fee40 2024-04-25T01:33:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.089+0000 7f8b177fe700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b0c0692a0 0x7f8b0c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.089+0000 7f8b177fe700 1 -- 172.21.15.33:0/1229261249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8b10087e70 con 0x7f8b200fee40 2024-04-25T01:33:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.089+0000 7f8b1effd700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b0c0692a0 0x7f8b0c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:16.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.089+0000 7f8b1effd700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b0c0692a0 0x7f8b0c06b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b08004f40 tx=0x7f8b0800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:16.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.093+0000 7f8b177fe700 1 -- 172.21.15.33:0/1229261249 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8b10057c30 con 0x7f8b200fee40 2024-04-25T01:33:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.352+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8b20105900 con 0x7f8b200fee40 2024-04-25T01:33:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.353+0000 7f8b177fe700 1 -- 172.21.15.33:0/1229261249 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f8b100577c0 con 0x7f8b200fee40 2024-04-25T01:33:16.330 INFO:teuthology.orchestra.run.smithi033.stdout:e15 2024-04-25T01:33:16.330 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 15 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:33:13.308700+0000 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:33:16.331 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:33:16.332 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:33:16.333 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:33:16.333 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:33:16.333 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:33:16.333 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:33:16.333 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:33:16.333 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b0c0692a0 msgr2=0x7f8b0c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b0c0692a0 0x7f8b0c06b750 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b08004f40 tx=0x7f8b0800b040 comp rx=0 tx=0).stop 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 msgr2=0x7f8b200fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 0x7f8b200fc1b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000b3b0 tx=0x7f8b10009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 shutdown_connections 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b200fe4f0 0x7f8b200fdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b0c0692a0 0x7f8b0c06b750 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 --2- 172.21.15.33:0/1229261249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b200fee40 0x7f8b200fc1b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 >> 172.21.15.33:0/1229261249 conn(0x7f8b200f6bd0 msgr2=0x7f8b200f82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:16.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 shutdown_connections 2024-04-25T01:33:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:16.355+0000 7f8b25721700 1 -- 172.21.15.33:0/1229261249 wait complete. 2024-04-25T01:33:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 15 2024-04-25T01:33:16.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:33:16.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:33:17.335 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-25T01:33:17.620 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:17.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:17 smithi033 ceph-mon[26353]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:33:17.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:17 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1229261249' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:33:17.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:17 smithi138 ceph-mon[32977]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T01:33:17.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:17 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1229261249' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:33:18.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:18.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:18.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:18.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:33:18.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:18.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:18 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:33:18.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:18 smithi033 ceph-mon[26353]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T01:33:18.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:18.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:18.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:18.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:33:18.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:18.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:18 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:33:18.966 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:18 smithi138 ceph-mon[32977]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T01:33:19.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 -- 172.21.15.33:0/183013289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fdfd0 msgr2=0x7fcef40fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 --2- 172.21.15.33:0/183013289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fdfd0 0x7fcef40fe440 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcee4004940 tx=0x7fcee4009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 -- 172.21.15.33:0/183013289 shutdown_connections 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 --2- 172.21.15.33:0/183013289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fdfd0 0x7fcef40fe440 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 --2- 172.21.15.33:0/183013289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef40fce60 0x7fcef40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 -- 172.21.15.33:0/183013289 >> 172.21.15.33:0/183013289 conn(0x7fcef40f8540 msgr2=0x7fcef40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 -- 172.21.15.33:0/183013289 shutdown_connections 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 -- 172.21.15.33:0/183013289 wait complete. 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 Processor -- start 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 -- start start 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fce60 0x7fcef410d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef40fdfd0 0x7fcef410de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef410e430 con 0x7fcef40fce60 2024-04-25T01:33:19.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.284+0000 7fcefc460700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef410e5a0 con 0x7fcef40fdfd0 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcefa1fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fce60 0x7fcef410d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcefa1fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fce60 0x7fcef410d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58306/0 (socket says 172.21.15.33:58306) 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcefa1fc700 1 -- 172.21.15.33:0/3007271313 learned_addr learned my addr 172.21.15.33:0/3007271313 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcef99fb700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef40fdfd0 0x7fcef410de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcefa1fc700 1 -- 172.21.15.33:0/3007271313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef40fdfd0 msgr2=0x7fcef410de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcefa1fc700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef40fdfd0 0x7fcef410de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcefa1fc700 1 -- 172.21.15.33:0/3007271313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcee4005040 con 0x7fcef40fce60 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcefa1fc700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fce60 0x7fcef410d8f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcef0002a50 tx=0x7fcef00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fceeaffd700 1 -- 172.21.15.33:0/3007271313 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcef0007cc0 con 0x7fcef40fce60 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef4112fa0 con 0x7fcef40fce60 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fceeaffd700 1 -- 172.21.15.33:0/3007271313 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcef000f040 con 0x7fcef40fce60 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fceeaffd700 1 -- 172.21.15.33:0/3007271313 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcef0009640 con 0x7fcef40fce60 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.285+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef4113530 con 0x7fcef40fce60 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.286+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef404c710 con 0x7fcef40fce60 2024-04-25T01:33:19.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.286+0000 7fceeaffd700 1 -- 172.21.15.33:0/3007271313 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fcef00097c0 con 0x7fcef40fce60 2024-04-25T01:33:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.287+0000 7fceeaffd700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcee00692a0 0x7fcee006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.287+0000 7fceeaffd700 1 -- 172.21.15.33:0/3007271313 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fcef0086f80 con 0x7fcef40fce60 2024-04-25T01:33:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.287+0000 7fcef99fb700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcee00692a0 0x7fcee006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.287+0000 7fcef99fb700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcee00692a0 0x7fcee006b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcee400b3b0 tx=0x7fcee4009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.292+0000 7fceeaffd700 1 -- 172.21.15.33:0/3007271313 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcef0056cc0 con 0x7fcef40fce60 2024-04-25T01:33:19.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.536+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcef41058c0 con 0x7fcef40fce60 2024-04-25T01:33:19.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.537+0000 7fceeaffd700 1 -- 172.21.15.33:0/3007271313 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4723 (secure 0 0 0) 0x7fcef0056850 con 0x7fcef40fce60 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.538+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcee00692a0 msgr2=0x7fcee006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.538+0000 7fcefc460700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcee00692a0 0x7fcee006b750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcee400b3b0 tx=0x7fcee4009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.538+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fce60 msgr2=0x7fcef410d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.538+0000 7fcefc460700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fce60 0x7fcef410d8f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcef0002a50 tx=0x7fcef00046b0 comp rx=0 tx=0).stop 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.539+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 shutdown_connections 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.539+0000 7fcefc460700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fcee00692a0 0x7fcee006b750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.539+0000 7fcefc460700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcef40fce60 0x7fcef410d8f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.539+0000 7fcefc460700 1 --2- 172.21.15.33:0/3007271313 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcef40fdfd0 0x7fcef410de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.539+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 >> 172.21.15.33:0/3007271313 conn(0x7fcef40f8540 msgr2=0x7fcef40f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.539+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 shutdown_connections 2024-04-25T01:33:19.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:19.539+0000 7fcefc460700 1 -- 172.21.15.33:0/3007271313 wait complete. 2024-04-25T01:33:19.517 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 15 2024-04-25T01:33:19.526 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:33:19.856 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:19 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/3007271313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:33:20.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:19 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/3007271313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:33:20.593 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-25T01:33:20.869 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:20 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:20.869 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:20 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:20.869 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:20 smithi033 ceph-mon[26353]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 341 B/s wr, 4 op/s 2024-04-25T01:33:21.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:20 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:21.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:20 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:21.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:20 smithi138 ceph-mon[32977]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 341 B/s wr, 4 op/s 2024-04-25T01:33:21.382 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:23.301 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:23 smithi033 ceph-mon[26353]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T01:33:23.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:23 smithi138 ceph-mon[32977]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T01:33:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.237+0000 7f178b288700 1 -- 172.21.15.33:0/1894213268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fce60 msgr2=0x7f17840fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.237+0000 7f178b288700 1 --2- 172.21.15.33:0/1894213268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fce60 0x7f17840fd270 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1774009ce0 tx=0x7f1774004d10 comp rx=0 tx=0).stop 2024-04-25T01:33:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.237+0000 7f178b288700 1 -- 172.21.15.33:0/1894213268 shutdown_connections 2024-04-25T01:33:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.237+0000 7f178b288700 1 --2- 172.21.15.33:0/1894213268 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17840fdfd0 0x7f17840fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.237+0000 7f178b288700 1 --2- 172.21.15.33:0/1894213268 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fce60 0x7f17840fd270 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.237+0000 7f178b288700 1 -- 172.21.15.33:0/1894213268 >> 172.21.15.33:0/1894213268 conn(0x7f17840f8540 msgr2=0x7f17840fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.237+0000 7f178b288700 1 -- 172.21.15.33:0/1894213268 shutdown_connections 2024-04-25T01:33:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.237+0000 7f178b288700 1 -- 172.21.15.33:0/1894213268 wait complete. 2024-04-25T01:33:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f178b288700 1 Processor -- start 2024-04-25T01:33:24.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f178b288700 1 -- start start 2024-04-25T01:33:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f178b288700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17840fce60 0x7f1784071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f178b288700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fdfd0 0x7f17840702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f178b288700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1784072200 con 0x7f17840fdfd0 2024-04-25T01:33:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f178b288700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17840707f0 con 0x7f17840fce60 2024-04-25T01:33:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f1788823700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fdfd0 0x7f17840702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f1788823700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fdfd0 0x7f17840702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58322/0 (socket says 172.21.15.33:58322) 2024-04-25T01:33:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f1788823700 1 -- 172.21.15.33:0/347970057 learned_addr learned my addr 172.21.15.33:0/347970057 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:24.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f1789024700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17840fce60 0x7f1784071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.238+0000 7f1788823700 1 -- 172.21.15.33:0/347970057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17840fce60 msgr2=0x7f1784071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.239+0000 7f1788823700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17840fce60 0x7f1784071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.239+0000 7f1788823700 1 -- 172.21.15.33:0/347970057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1774009940 con 0x7f17840fdfd0 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.239+0000 7f1789024700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17840fce60 0x7f1784071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.239+0000 7f1788823700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fdfd0 0x7f17840702b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f177800eb40 tx=0x7f177800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.239+0000 7f1781ffb700 1 -- 172.21.15.33:0/347970057 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1778007cc0 con 0x7f17840fdfd0 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.239+0000 7f1781ffb700 1 -- 172.21.15.33:0/347970057 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f177800f040 con 0x7f17840fdfd0 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.239+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1784070a40 con 0x7f17840fdfd0 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.239+0000 7f1781ffb700 1 -- 172.21.15.33:0/347970057 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f177800ca10 con 0x7f17840fdfd0 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.239+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1784071000 con 0x7f17840fdfd0 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.240+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178404c710 con 0x7f17840fdfd0 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.240+0000 7f1781ffb700 1 -- 172.21.15.33:0/347970057 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f177800cc80 con 0x7f17840fdfd0 2024-04-25T01:33:24.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.241+0000 7f1781ffb700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f176c0692a0 0x7f176c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:24.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.241+0000 7f1781ffb700 1 -- 172.21.15.33:0/347970057 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f1778086d90 con 0x7f17840fdfd0 2024-04-25T01:33:24.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.241+0000 7f1789024700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f176c0692a0 0x7f176c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:24.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.241+0000 7f1789024700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f176c0692a0 0x7f176c06b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1774009560 tx=0x7f1774004d10 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:24.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.245+0000 7f1781ffb700 1 -- 172.21.15.33:0/347970057 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f177804e250 con 0x7f17840fdfd0 2024-04-25T01:33:24.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.525+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f17841058c0 con 0x7f17840fdfd0 2024-04-25T01:33:24.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.526+0000 7f1781ffb700 1 -- 172.21.15.33:0/347970057 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v15) v1 ==== 78+0+99 (secure 0 0 0) 0x7f1778056b80 con 0x7f17840fdfd0 2024-04-25T01:33:24.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f176c0692a0 msgr2=0x7f176c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:24.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f176c0692a0 0x7f176c06b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1774009560 tx=0x7f1774004d10 comp rx=0 tx=0).stop 2024-04-25T01:33:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fdfd0 msgr2=0x7f17840702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fdfd0 0x7f17840702b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f177800eb40 tx=0x7f177800b040 comp rx=0 tx=0).stop 2024-04-25T01:33:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 shutdown_connections 2024-04-25T01:33:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17840fce60 0x7f1784071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f176c0692a0 0x7f176c06b750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 --2- 172.21.15.33:0/347970057 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17840fdfd0 0x7f17840702b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 >> 172.21.15.33:0/347970057 conn(0x7f17840f8540 msgr2=0x7f17840f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 shutdown_connections 2024-04-25T01:33:24.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:24.528+0000 7f178b288700 1 -- 172.21.15.33:0/347970057 wait complete. 2024-04-25T01:33:24.516 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:33:25.418 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:25 smithi033 ceph-mon[26353]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T01:33:25.418 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:25 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/347970057' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-25T01:33:25.444 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-25T01:33:25.576 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T01:33:25.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:25 smithi138 ceph-mon[32977]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-04-25T01:33:25.577 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:25 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/347970057' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-25T01:33:25.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.727+0000 7f97c2f5d640 1 -- 172.21.15.33:0/3727444648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 msgr2=0x7f97bc0fd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:25.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.727+0000 7f97c2f5d640 1 --2- 172.21.15.33:0/3727444648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 0x7f97bc0fd8c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f97ac0048d0 tx=0x7f97ac009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:25.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 -- 172.21.15.33:0/3727444648 shutdown_connections 2024-04-25T01:33:25.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 --2- 172.21.15.33:0/3727444648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 0x7f97bc0fd8c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:25.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 --2- 172.21.15.33:0/3727444648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97bc0fc180 0x7f97bc0fc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:25.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 -- 172.21.15.33:0/3727444648 >> 172.21.15.33:0/3727444648 conn(0x7f97bc0f7960 msgr2=0x7f97bc0f9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:25.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 -- 172.21.15.33:0/3727444648 shutdown_connections 2024-04-25T01:33:25.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 -- 172.21.15.33:0/3727444648 wait complete. 2024-04-25T01:33:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 Processor -- start 2024-04-25T01:33:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 -- start start 2024-04-25T01:33:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97bc0fc180 0x7f97bc0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 0x7f97bc06f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc071740 con 0x7f97bc0fd450 2024-04-25T01:33:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.728+0000 7f97c2f5d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97bc06fd00 con 0x7f97bc0fc180 2024-04-25T01:33:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 0x7f97bc06f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97bbfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 0x7f97bc06f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57896/0 (socket says 172.21.15.33:57896) 2024-04-25T01:33:25.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97bbfff640 1 -- 172.21.15.33:0/2038769979 learned_addr learned my addr 172.21.15.33:0/2038769979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97c0cf9640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97bc0fc180 0x7f97bc0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97bbfff640 1 -- 172.21.15.33:0/2038769979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97bc0fc180 msgr2=0x7f97bc0710f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97bbfff640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97bc0fc180 0x7f97bc0710f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97bbfff640 1 -- 172.21.15.33:0/2038769979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac005040 con 0x7f97bc0fd450 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97c0cf9640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97bc0fc180 0x7f97bc0710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97bbfff640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 0x7f97bc06f7c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f97ac000c00 tx=0x7f97ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97b97fa640 1 -- 172.21.15.33:0/2038769979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97ac0094c0 con 0x7f97bc0fd450 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc06ff80 con 0x7f97bc0fd450 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97b97fa640 1 -- 172.21.15.33:0/2038769979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97ac032e90 con 0x7f97bc0fd450 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97b97fa640 1 -- 172.21.15.33:0/2038769979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97ac0435f0 con 0x7f97bc0fd450 2024-04-25T01:33:25.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.729+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97bc070440 con 0x7f97bc0fd450 2024-04-25T01:33:25.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.731+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97bc04bcc0 con 0x7f97bc0fd450 2024-04-25T01:33:25.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.731+0000 7f97b97fa640 1 -- 172.21.15.33:0/2038769979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f97ac02f070 con 0x7f97bc0fd450 2024-04-25T01:33:25.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.732+0000 7f97b97fa640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f97a8069210 0x7f97a806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:25.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.732+0000 7f97b97fa640 1 -- 172.21.15.33:0/2038769979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f97ac0a91c0 con 0x7f97bc0fd450 2024-04-25T01:33:25.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.732+0000 7f97c0cf9640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f97a8069210 0x7f97a806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:25.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.732+0000 7f97c0cf9640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f97a8069210 0x7f97a806b6c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f97a4004d60 tx=0x7f97a400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:25.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.736+0000 7f97b97fa640 1 -- 172.21.15.33:0/2038769979 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f97ac075410 con 0x7f97bc0fd450 2024-04-25T01:33:25.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.930+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97bc104cd0 con 0x7f97bc0fd450 2024-04-25T01:33:25.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.930+0000 7f97b97fa640 1 -- 172.21.15.33:0/2038769979 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14242 (secure 0 0 0) 0x7f97ac078a30 con 0x7f97bc0fd450 2024-04-25T01:33:25.908 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:33:25.908 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":45,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","created":"2024-04-25T01:26:05.674330+0000","modified":"2024-04-25T01:32:58.476828+0000","last_up_change":"2024-04-25T01:31:39.177371+0000","last_in_change":"2024-04-25T01:31:20.199327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:30:26.558656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-25T01:32:55.402156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-25T01:32:55.894097+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"68af93d4-7aa8-485f-826e-62a6e362907b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6803","nonce":505851393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6805","nonce":505851393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6809","nonce":505851393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":505851393},{"type":"v1","addr":"172.21.15.33:6807","nonce":505851393}]},"public_addr":"172.21.15.33:6803/505851393","cluster_addr":"172.21.15.33:6805/505851393","heartbeat_back_addr":"172.21.15.33:6809/505851393","heartbeat_front_addr":"172.21.15.33:6807/505851393","state":["exists","up"]},{"osd":1,"uuid":"1fdb7e9b-1138-43db-a11f-3e30e286c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6811","nonce":835394340}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6813","nonce":835394340}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6817","nonce":835394340}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":835394340},{"type":"v1","addr":"172.21.15.33:6815","nonce":835394340}]},"public_addr":"172.21.15.33:6811/835394340","cluster_addr":"172.21.15.33:6813/835394340","heartbeat_back_addr":"172.21.15.33:6817/835394340","heartbeat_front_addr":"172.21.15.33:6815/835394340","state":["exists","up"]},{"osd":2,"uuid":"b1239da3-566e-4df0-b5ea-df4c14959837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6819","nonce":4233054842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6821","nonce":4233054842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6825","nonce":4233054842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4233054842},{"type":"v1","addr":"172.21.15.33:6823","nonce":4233054842}]},"public_addr":"172.21.15.33:6819/4233054842","cluster_addr":"172.21.15.33:6821/4233054842","heartbeat_back_addr":"172.21.15.33:6825/4233054842","heartbeat_front_addr":"172.21.15.33:6823/4233054842","state":["exists","up"]},{"osd":3,"uuid":"abc58749-77ca-4476-9601-cdf89d3c1e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6801","nonce":461435998}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6803","nonce":461435998}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6807","nonce":461435998}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":461435998},{"type":"v1","addr":"172.21.15.138:6805","nonce":461435998}]},"public_addr":"172.21.15.138:6801/461435998","cluster_addr":"172.21.15.138:6803/461435998","heartbeat_back_addr":"172.21.15.138:6807/461435998","heartbeat_front_addr":"172.21.15.138:6805/461435998","state":["exists","up"]},{"osd":4,"uuid":"83e421bd-52ba-47ad-8755-e1354f502ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6809","nonce":1618213578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6811","nonce":1618213578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6815","nonce":1618213578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":1618213578},{"type":"v1","addr":"172.21.15.138:6813","nonce":1618213578}]},"public_addr":"172.21.15.138:6809/1618213578","cluster_addr":"172.21.15.138:6811/1618213578","heartbeat_back_addr":"172.21.15.138:6815/1618213578","heartbeat_front_addr":"172.21.15.138:6813/1618213578","state":["exists","up"]},{"osd":5,"uuid":"266e5984-f087-408e-a84c-04f81f33af43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6817","nonce":1128834056}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6819","nonce":1128834056}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6823","nonce":1128834056}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":1128834056},{"type":"v1","addr":"172.21.15.138:6821","nonce":1128834056}]},"public_addr":"172.21.15.138:6817/1128834056","cluster_addr":"172.21.15.138:6819/1128834056","heartbeat_back_addr":"172.21.15.138:6823/1128834056","heartbeat_front_addr":"172.21.15.138:6821/1128834056","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:27.636323+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:55.618887+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:23.307858+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:47.331937+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:12.159932+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:37.317499+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.33:6800/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2566719862":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1401659678":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/751781866":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6800/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/4160574345":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/2117172792":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/1131828704":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2112653252":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6800/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:0/686286320":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6801/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1869074265":"2024-04-26T01:27:11.182264+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T01:33:25.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.932+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f97a8069210 msgr2=0x7f97a806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f97a8069210 0x7f97a806b6c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f97a4004d60 tx=0x7f97a400b040 comp rx=0 tx=0).stop 2024-04-25T01:33:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 msgr2=0x7f97bc06f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 0x7f97bc06f7c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f97ac000c00 tx=0x7f97ac009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 shutdown_connections 2024-04-25T01:33:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f97a8069210 0x7f97a806b6c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97bc0fd450 0x7f97bc06f7c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:25.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 --2- 172.21.15.33:0/2038769979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97bc0fc180 0x7f97bc0710f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:25.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 >> 172.21.15.33:0/2038769979 conn(0x7f97bc0f7960 msgr2=0x7f97bc0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:25.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 shutdown_connections 2024-04-25T01:33:25.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:25.933+0000 7f97c2f5d640 1 -- 172.21.15.33:0/2038769979 wait complete. 2024-04-25T01:33:25.921 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:30:26.558656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-25T01:32:55.402156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-25T01:32:55.894097+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T01:33:25.922 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T01:33:26.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 -- 172.21.15.33:0/1340687030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 msgr2=0x7f90f80fc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 --2- 172.21.15.33:0/1340687030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 0x7f90f80fc790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f90ec0048d0 tx=0x7f90ec009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 -- 172.21.15.33:0/1340687030 shutdown_connections 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 --2- 172.21.15.33:0/1340687030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 0x7f90f80fc790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 --2- 172.21.15.33:0/1340687030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f80fb0f0 0x7f90f80fb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 -- 172.21.15.33:0/1340687030 >> 172.21.15.33:0/1340687030 conn(0x7f90f80f6830 msgr2=0x7f90f80f8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 -- 172.21.15.33:0/1340687030 shutdown_connections 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 -- 172.21.15.33:0/1340687030 wait complete. 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 Processor -- start 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 -- start start 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.047+0000 7f90fd55d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f80fb0f0 0x7f90f810cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90fd55d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 0x7f90f810d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90fd55d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f810d7e0 con 0x7f90f80fc320 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90fd55d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f81121a0 con 0x7f90f80fb0f0 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90f67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 0x7f90f810d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90f67fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 0x7f90f810d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57922/0 (socket says 172.21.15.33:57922) 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90f67fc640 1 -- 172.21.15.33:0/2145413934 learned_addr learned my addr 172.21.15.33:0/2145413934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90f6ffd640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f80fb0f0 0x7f90f810cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90f67fc640 1 -- 172.21.15.33:0/2145413934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f80fb0f0 msgr2=0x7f90f810cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90f67fc640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f80fb0f0 0x7f90f810cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90f67fc640 1 -- 172.21.15.33:0/2145413934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec005040 con 0x7f90f80fc320 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90f6ffd640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f80fb0f0 0x7f90f810cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90f67fc640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 0x7f90f810d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f90ec007da0 tx=0x7f90ec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90df7fe640 1 -- 172.21.15.33:0/2145413934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ec0094c0 con 0x7f90f80fc320 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90df7fe640 1 -- 172.21.15.33:0/2145413934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90ec009620 con 0x7f90f80fc320 2024-04-25T01:33:26.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f81122e0 con 0x7f90f80fc320 2024-04-25T01:33:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90df7fe640 1 -- 172.21.15.33:0/2145413934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ec03b620 con 0x7f90f80fc320 2024-04-25T01:33:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.048+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f81127b0 con 0x7f90f80fc320 2024-04-25T01:33:26.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.050+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f804bcc0 con 0x7f90f80fc320 2024-04-25T01:33:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.050+0000 7f90df7fe640 1 -- 172.21.15.33:0/2145413934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f90ec02f070 con 0x7f90f80fc320 2024-04-25T01:33:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.050+0000 7f90df7fe640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f90e0069210 0x7f90e006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.050+0000 7f90f6ffd640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f90e0069210 0x7f90e006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.050+0000 7f90df7fe640 1 -- 172.21.15.33:0/2145413934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f90ec0a97e0 con 0x7f90f80fc320 2024-04-25T01:33:26.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.051+0000 7f90f6ffd640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f90e0069210 0x7f90e006b6c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90f810e110 tx=0x7f90e8009ef0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:26.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.055+0000 7f90df7fe640 1 -- 172.21.15.33:0/2145413934 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f90ec0759f0 con 0x7f90f80fc320 2024-04-25T01:33:26.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.232+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90f8112c50 con 0x7f90f80fc320 2024-04-25T01:33:26.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.232+0000 7f90df7fe640 1 -- 172.21.15.33:0/2145413934 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90ec0a9020 con 0x7f90f80fc320 2024-04-25T01:33:26.209 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T01:33:26.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f90e0069210 msgr2=0x7f90e006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f90e0069210 0x7f90e006b6c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f90f810e110 tx=0x7f90e8009ef0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 msgr2=0x7f90f810d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 0x7f90f810d190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f90ec007da0 tx=0x7f90ec009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 shutdown_connections 2024-04-25T01:33:26.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f90e0069210 0x7f90e006b6c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fc320 0x7f90f810d190 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 --2- 172.21.15.33:0/2145413934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f80fb0f0 0x7f90f810cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 >> 172.21.15.33:0/2145413934 conn(0x7f90f80f6830 msgr2=0x7f90f80654f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:26.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 shutdown_connections 2024-04-25T01:33:26.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.234+0000 7f90fd55d640 1 -- 172.21.15.33:0/2145413934 wait complete. 2024-04-25T01:33:26.223 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-25T01:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.345+0000 7f8b97b5d640 1 -- 172.21.15.33:0/2643033390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b90065c20 msgr2=0x7f8b900fefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.345+0000 7f8b97b5d640 1 --2- 172.21.15.33:0/2643033390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b90065c20 0x7f8b900fefe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0048d0 tx=0x7f8b8c009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 -- 172.21.15.33:0/2643033390 shutdown_connections 2024-04-25T01:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 --2- 172.21.15.33:0/2643033390 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b900ff5b0 0x7f8b90101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 --2- 172.21.15.33:0/2643033390 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b90065c20 0x7f8b900fefe0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 -- 172.21.15.33:0/2643033390 >> 172.21.15.33:0/2643033390 conn(0x7f8b900f69b0 msgr2=0x7f8b900f8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 -- 172.21.15.33:0/2643033390 shutdown_connections 2024-04-25T01:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 -- 172.21.15.33:0/2643033390 wait complete. 2024-04-25T01:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 Processor -- start 2024-04-25T01:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 -- start start 2024-04-25T01:33:26.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b90065c20 0x7f8b9010cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b900ff5b0 0x7f8b9010d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9010dac0 con 0x7f8b900ff5b0 2024-04-25T01:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b97b5d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90112470 con 0x7f8b90065c20 2024-04-25T01:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.346+0000 7f8b950f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b900ff5b0 0x7f8b9010d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b950f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b900ff5b0 0x7f8b9010d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57930/0 (socket says 172.21.15.33:57930) 2024-04-25T01:33:26.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b950f8640 1 -- 172.21.15.33:0/312050689 learned_addr learned my addr 172.21.15.33:0/312050689 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b958f9640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b90065c20 0x7f8b9010cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b950f8640 1 -- 172.21.15.33:0/312050689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b90065c20 msgr2=0x7f8b9010cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b950f8640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b90065c20 0x7f8b9010cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b950f8640 1 -- 172.21.15.33:0/312050689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8c005040 con 0x7f8b900ff5b0 2024-04-25T01:33:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b950f8640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b900ff5b0 0x7f8b9010d470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8b80002870 tx=0x7f8b80004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:26.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b867fc640 1 -- 172.21.15.33:0/312050689 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b80007c30 con 0x7f8b900ff5b0 2024-04-25T01:33:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b90112610 con 0x7f8b900ff5b0 2024-04-25T01:33:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b867fc640 1 -- 172.21.15.33:0/312050689 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b8000f040 con 0x7f8b900ff5b0 2024-04-25T01:33:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b867fc640 1 -- 172.21.15.33:0/312050689 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b800095f0 con 0x7f8b900ff5b0 2024-04-25T01:33:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.347+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b90112b00 con 0x7f8b900ff5b0 2024-04-25T01:33:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.348+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9004bcc0 con 0x7f8b900ff5b0 2024-04-25T01:33:26.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.349+0000 7f8b867fc640 1 -- 172.21.15.33:0/312050689 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f8b8000d6b0 con 0x7f8b900ff5b0 2024-04-25T01:33:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.349+0000 7f8b867fc640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b7c069210 0x7f8b7c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.349+0000 7f8b867fc640 1 -- 172.21.15.33:0/312050689 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8b80086990 con 0x7f8b900ff5b0 2024-04-25T01:33:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.349+0000 7f8b958f9640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b7c069210 0x7f8b7c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.350+0000 7f8b958f9640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b7c069210 0x7f8b7c06b6c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0317e0 tx=0x7f8b8c009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:26.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.353+0000 7f8b867fc640 1 -- 172.21.15.33:0/312050689 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8b80052cf0 con 0x7f8b900ff5b0 2024-04-25T01:33:26.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.526+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f8b90112dc0 con 0x7f8b900ff5b0 2024-04-25T01:33:26.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.526+0000 7f8b867fc640 1 -- 172.21.15.33:0/312050689 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f8b80017020 con 0x7f8b900ff5b0 2024-04-25T01:33:26.504 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:33:26.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b7c069210 msgr2=0x7f8b7c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b7c069210 0x7f8b7c06b6c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0317e0 tx=0x7f8b8c009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b900ff5b0 msgr2=0x7f8b9010d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b900ff5b0 0x7f8b9010d470 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8b80002870 tx=0x7f8b80004600 comp rx=0 tx=0).stop 2024-04-25T01:33:26.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 shutdown_connections 2024-04-25T01:33:26.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f8b7c069210 0x7f8b7c06b6c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b900ff5b0 0x7f8b9010d470 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 --2- 172.21.15.33:0/312050689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b90065c20 0x7f8b9010cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 >> 172.21.15.33:0/312050689 conn(0x7f8b900f69b0 msgr2=0x7f8b900f6df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:26.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 shutdown_connections 2024-04-25T01:33:26.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.528+0000 7f8b97b5d640 1 -- 172.21.15.33:0/312050689 wait complete. 2024-04-25T01:33:26.516 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-25T01:33:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 -- 172.21.15.33:0/3239206013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 msgr2=0x7f094c0fc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 --2- 172.21.15.33:0/3239206013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 0x7f094c0fc610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f093c0048d0 tx=0x7f093c009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 -- 172.21.15.33:0/3239206013 shutdown_connections 2024-04-25T01:33:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 --2- 172.21.15.33:0/3239206013 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f094c0fd430 0x7f094c0fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 --2- 172.21.15.33:0/3239206013 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 0x7f094c0fc610 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 -- 172.21.15.33:0/3239206013 >> 172.21.15.33:0/3239206013 conn(0x7f094c0f7900 msgr2=0x7f094c0f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 -- 172.21.15.33:0/3239206013 shutdown_connections 2024-04-25T01:33:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 -- 172.21.15.33:0/3239206013 wait complete. 2024-04-25T01:33:26.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 Processor -- start 2024-04-25T01:33:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 -- start start 2024-04-25T01:33:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 0x7f094c0710f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f094c0fd430 0x7f094c06f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c071740 con 0x7f094c0fc200 2024-04-25T01:33:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.639+0000 7f095371d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c06fd00 con 0x7f094c0fd430 2024-04-25T01:33:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f09514b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 0x7f094c0710f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f09514b9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 0x7f094c0710f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57946/0 (socket says 172.21.15.33:57946) 2024-04-25T01:33:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f09514b9640 1 -- 172.21.15.33:0/1664029592 learned_addr learned my addr 172.21.15.33:0/1664029592 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f0950cb8640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f094c0fd430 0x7f094c06f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f09514b9640 1 -- 172.21.15.33:0/1664029592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f094c0fd430 msgr2=0x7f094c06f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f09514b9640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f094c0fd430 0x7f094c06f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f09514b9640 1 -- 172.21.15.33:0/1664029592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c005040 con 0x7f094c0fc200 2024-04-25T01:33:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f09514b9640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 0x7f094c0710f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f093c02bda0 tx=0x7f093c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f0941ffb640 1 -- 172.21.15.33:0/1664029592 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c0094c0 con 0x7f094c0fc200 2024-04-25T01:33:26.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f0941ffb640 1 -- 172.21.15.33:0/1664029592 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f093c009620 con 0x7f094c0fc200 2024-04-25T01:33:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094c06ff80 con 0x7f094c0fc200 2024-04-25T01:33:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.640+0000 7f0941ffb640 1 -- 172.21.15.33:0/1664029592 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f093c03b640 con 0x7f094c0fc200 2024-04-25T01:33:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.641+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c0704a0 con 0x7f094c0fc200 2024-04-25T01:33:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.642+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094c1054f0 con 0x7f094c0fc200 2024-04-25T01:33:26.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.642+0000 7f0941ffb640 1 -- 172.21.15.33:0/1664029592 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f093c02f070 con 0x7f094c0fc200 2024-04-25T01:33:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.642+0000 7f0941ffb640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0938069210 0x7f093806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.642+0000 7f0941ffb640 1 -- 172.21.15.33:0/1664029592 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f093c0a96e0 con 0x7f094c0fc200 2024-04-25T01:33:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.642+0000 7f0950cb8640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0938069210 0x7f093806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.643+0000 7f0950cb8640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0938069210 0x7f093806b6c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f094c070ea0 tx=0x7f094800b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:26.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.647+0000 7f0941ffb640 1 -- 172.21.15.33:0/1664029592 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f093c075990 con 0x7f094c0fc200 2024-04-25T01:33:26.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.821+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f094c04bcc0 con 0x7f094c0fc200 2024-04-25T01:33:26.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.821+0000 7f0941ffb640 1 -- 172.21.15.33:0/1664029592 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f093c078fb0 con 0x7f094c0fc200 2024-04-25T01:33:26.798 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T01:33:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0938069210 msgr2=0x7f093806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0938069210 0x7f093806b6c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f094c070ea0 tx=0x7f094800b040 comp rx=0 tx=0).stop 2024-04-25T01:33:26.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 msgr2=0x7f094c0710f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 0x7f094c0710f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f093c02bda0 tx=0x7f093c009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 shutdown_connections 2024-04-25T01:33:26.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0938069210 0x7f093806b6c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f094c0fd430 0x7f094c06f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 --2- 172.21.15.33:0/1664029592 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f094c0fc200 0x7f094c0710f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 >> 172.21.15.33:0/1664029592 conn(0x7f094c0f7900 msgr2=0x7f094c0f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:26.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 shutdown_connections 2024-04-25T01:33:26.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.823+0000 7f095371d640 1 -- 172.21.15.33:0/1664029592 wait complete. 2024-04-25T01:33:26.811 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:26.811 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:26.811 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:26.811 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:33:26.811 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:26 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:26.812 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:26 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2038769979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:33:26.812 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:26 smithi033 ceph-mon[26353]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-04-25T01:33:26.812 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:26 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2145413934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:33:26.812 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:26 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/312050689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T01:33:26.813 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T01:33:26.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.933+0000 7f2877910640 1 -- 172.21.15.33:0/2689361350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fd7c0 msgr2=0x7f28700fdbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.933+0000 7f2877910640 1 --2- 172.21.15.33:0/2689361350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fd7c0 0x7f28700fdbd0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f28600048d0 tx=0x7f2860009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 -- 172.21.15.33:0/2689361350 shutdown_connections 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 --2- 172.21.15.33:0/2689361350 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28700fe110 0x7f28700fa270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 --2- 172.21.15.33:0/2689361350 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fd7c0 0x7f28700fdbd0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 -- 172.21.15.33:0/2689361350 >> 172.21.15.33:0/2689361350 conn(0x7f28700f5fc0 msgr2=0x7f28700f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 -- 172.21.15.33:0/2689361350 shutdown_connections 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 -- 172.21.15.33:0/2689361350 wait complete. 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 Processor -- start 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 -- start start 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28700fd7c0 0x7f287010ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fe110 0x7f287010d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287010d880 con 0x7f28700fe110 2024-04-25T01:33:26.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2877910640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2870112240 con 0x7f28700fd7c0 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2874eab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fe110 0x7f287010d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2874eab640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fe110 0x7f287010d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57958/0 (socket says 172.21.15.33:57958) 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.934+0000 7f2874eab640 1 -- 172.21.15.33:0/212179028 learned_addr learned my addr 172.21.15.33:0/212179028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f28756ac640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28700fd7c0 0x7f287010ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f2874eab640 1 -- 172.21.15.33:0/212179028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28700fd7c0 msgr2=0x7f287010ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f2874eab640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28700fd7c0 0x7f287010ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f2874eab640 1 -- 172.21.15.33:0/212179028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2860005040 con 0x7f28700fe110 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f2874eab640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fe110 0x7f287010d200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f286c00c790 tx=0x7f286c0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f2865ffb640 1 -- 172.21.15.33:0/212179028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286c00a820 con 0x7f28700fe110 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f2865ffb640 1 -- 172.21.15.33:0/212179028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f286c00ae50 con 0x7f28700fe110 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28701123e0 con 0x7f28700fe110 2024-04-25T01:33:26.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f2865ffb640 1 -- 172.21.15.33:0/212179028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f286c0113a0 con 0x7f28700fe110 2024-04-25T01:33:26.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.935+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28701128d0 con 0x7f28700fe110 2024-04-25T01:33:26.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.936+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f287004bcc0 con 0x7f28700fe110 2024-04-25T01:33:26.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.937+0000 7f2865ffb640 1 -- 172.21.15.33:0/212179028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f286c012400 con 0x7f28700fe110 2024-04-25T01:33:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.937+0000 7f2865ffb640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f285c069210 0x7f285c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.937+0000 7f2865ffb640 1 -- 172.21.15.33:0/212179028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f286c0871e0 con 0x7f28700fe110 2024-04-25T01:33:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.937+0000 7f28756ac640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f285c069210 0x7f285c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:26.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.938+0000 7f28756ac640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f285c069210 0x7f285c06b6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f2860009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:26.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:26.941+0000 7f2865ffb640 1 -- 172.21.15.33:0/212179028 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f286c053670 con 0x7f28700fe110 2024-04-25T01:33:27.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:27.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:27.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:33:27.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:33:27.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:26 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:33:27.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:26 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2038769979' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T01:33:27.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:26 smithi138 ceph-mon[32977]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 4 op/s 2024-04-25T01:33:27.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:26 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2145413934' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T01:33:27.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:26 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/312050689' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T01:33:27.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.156+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2870112bd0 con 0x7f28700fe110 2024-04-25T01:33:27.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.157+0000 7f2865ffb640 1 -- 172.21.15.33:0/212179028 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4723 (secure 0 0 0) 0x7f286c056c90 con 0x7f28700fe110 2024-04-25T01:33:27.134 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:33:27.134 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":15,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:33:13.308700+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14446,"mds_1":24279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14446":{"gid":14446,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6827/3303942324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3303942324},{"type":"v1","addr":"172.21.15.33:6827","nonce":3303942324}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14458":{"gid":14458,"name":"cephfs.smithi033.hobkqr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.33:6829/3968098059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3968098059},{"type":"v1","addr":"172.21.15.33:6829","nonce":3968098059}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24279":{"gid":24279,"name":"cephfs.smithi138.jwzxlz","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/3887358693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3887358693},{"type":"v1","addr":"172.21.15.138:6825","nonce":3887358693}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi138.rvxnby","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.138:6827/4019073381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4019073381},{"type":"v1","addr":"172.21.15.138:6827","nonce":4019073381}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-04-25T01:33:27.134 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 15 2024-04-25T01:33:27.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f285c069210 msgr2=0x7f285c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:27.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f285c069210 0x7f285c06b6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2860000c00 tx=0x7f2860009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:27.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fe110 msgr2=0x7f287010d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:27.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fe110 0x7f287010d200 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f286c00c790 tx=0x7f286c0043c0 comp rx=0 tx=0).stop 2024-04-25T01:33:27.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 shutdown_connections 2024-04-25T01:33:27.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f285c069210 0x7f285c06b6c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:27.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28700fe110 0x7f287010d200 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:27.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 --2- 172.21.15.33:0/212179028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28700fd7c0 0x7f287010ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:27.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 >> 172.21.15.33:0/212179028 conn(0x7f28700f5fc0 msgr2=0x7f28700f7690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:27.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 shutdown_connections 2024-04-25T01:33:27.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:27.159+0000 7f2877910640 1 -- 172.21.15.33:0/212179028 wait complete. 2024-04-25T01:33:27.147 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 2, 'flags': 50} 2024-04-25T01:33:27.147 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T01:33:27.183 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T01:33:27.183 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-25T01:33:27.183 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T01:33:27.183 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:33:27.183 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:27.183 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T01:33:27.184 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T01:33:27.184 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:27.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:27.184 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T01:33:27.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:27.204 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link delete ceph-brx 2024-04-25T01:33:27.279 INFO:teuthology.orchestra.run.smithi138.stderr:Cannot find device "ceph-brx" 2024-04-25T01:33:27.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:33:27.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:27.281 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-25T01:33:27.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:27.298 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link delete ceph-brx 2024-04-25T01:33:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:Cannot find device "ceph-brx" 2024-04-25T01:33:27.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:33:27.367 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T01:33:27.368 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T01:33:27.677 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:28.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:27 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1664029592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T01:33:28.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:27 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/212179028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:33:28.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:27 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1664029592' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T01:33:28.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:27 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/212179028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T01:33:29.044 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:28 smithi033 ceph-mon[26353]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:33:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.269+0000 7fc3d801c700 1 -- 172.21.15.33:0/3623089267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0065640 msgr2=0x7fc3d0065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.269+0000 7fc3d801c700 1 --2- 172.21.15.33:0/3623089267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0065640 0x7fc3d0065ab0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4004940 tx=0x7fc3c4009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.269+0000 7fc3d801c700 1 -- 172.21.15.33:0/3623089267 shutdown_connections 2024-04-25T01:33:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.269+0000 7fc3d801c700 1 --2- 172.21.15.33:0/3623089267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0065640 0x7fc3d0065ab0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:29.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.269+0000 7fc3d801c700 1 --2- 172.21.15.33:0/3623089267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0102e50 0x7fc3d0103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.269+0000 7fc3d801c700 1 -- 172.21.15.33:0/3623089267 >> 172.21.15.33:0/3623089267 conn(0x7fc3d0071d00 msgr2=0x7fc3d0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.269+0000 7fc3d801c700 1 -- 172.21.15.33:0/3623089267 shutdown_connections 2024-04-25T01:33:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.269+0000 7fc3d801c700 1 -- 172.21.15.33:0/3623089267 wait complete. 2024-04-25T01:33:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.270+0000 7fc3d801c700 1 Processor -- start 2024-04-25T01:33:29.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.270+0000 7fc3d801c700 1 -- start start 2024-04-25T01:33:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.270+0000 7fc3d801c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0065640 0x7fc3d00fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.270+0000 7fc3d801c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0102e50 0x7fc3d00fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.270+0000 7fc3d801c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00fc880 con 0x7fc3d0102e50 2024-04-25T01:33:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.270+0000 7fc3d801c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00fc9c0 con 0x7fc3d0065640 2024-04-25T01:33:29.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d55b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0102e50 0x7fc3d00fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d55b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0102e50 0x7fc3d00fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57972/0 (socket says 172.21.15.33:57972) 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d55b7700 1 -- 172.21.15.33:0/4054576236 learned_addr learned my addr 172.21.15.33:0/4054576236 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d5db8700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0065640 0x7fc3d00fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d55b7700 1 -- 172.21.15.33:0/4054576236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0065640 msgr2=0x7fc3d00fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d55b7700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0065640 0x7fc3d00fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d55b7700 1 -- 172.21.15.33:0/4054576236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c4005040 con 0x7fc3d0102e50 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d55b7700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0102e50 0x7fc3d00fc2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009150 tx=0x7fc3c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3ce7fc700 1 -- 172.21.15.33:0/4054576236 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c400e040 con 0x7fc3d0102e50 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d00fcc10 con 0x7fc3d0102e50 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3ce7fc700 1 -- 172.21.15.33:0/4054576236 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3c4013420 con 0x7fc3d0102e50 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3ce7fc700 1 -- 172.21.15.33:0/4054576236 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3c401c610 con 0x7fc3d0102e50 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.271+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d011a2a0 con 0x7fc3d0102e50 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.272+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d0100100 con 0x7fc3d0102e50 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.273+0000 7fc3ce7fc700 1 -- 172.21.15.33:0/4054576236 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc3c400baf0 con 0x7fc3d0102e50 2024-04-25T01:33:29.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.273+0000 7fc3ce7fc700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc3b80692b0 0x7fc3b806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.273+0000 7fc3ce7fc700 1 -- 172.21.15.33:0/4054576236 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc3c40882e0 con 0x7fc3d0102e50 2024-04-25T01:33:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.273+0000 7fc3d5db8700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc3b80692b0 0x7fc3b806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:29.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.274+0000 7fc3d5db8700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc3b80692b0 0x7fc3b806b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0007a40 tx=0x7fc3c0009da0 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:29.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.278+0000 7fc3ce7fc700 1 -- 172.21.15.33:0/4054576236 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc3c40546b0 con 0x7fc3d0102e50 2024-04-25T01:33:29.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:28 smithi138 ceph-mon[32977]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:33:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.523+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc3d00fd500 con 0x7fc3d0102e50 2024-04-25T01:33:29.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.524+0000 7fc3ce7fc700 1 -- 172.21.15.33:0/4054576236 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7fc3c4057cd0 con 0x7fc3d0102e50 2024-04-25T01:33:29.501 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-25T01:33:29.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc3b80692b0 msgr2=0x7fc3b806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:29.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc3b80692b0 0x7fc3b806b760 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0007a40 tx=0x7fc3c0009da0 comp rx=0 tx=0).stop 2024-04-25T01:33:29.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0102e50 msgr2=0x7fc3d00fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:29.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0102e50 0x7fc3d00fc2f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009150 tx=0x7fc3c4009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 shutdown_connections 2024-04-25T01:33:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3d0065640 0x7fc3d00fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc3b80692b0 0x7fc3b806b760 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 --2- 172.21.15.33:0/4054576236 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3d0102e50 0x7fc3d00fc2f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 >> 172.21.15.33:0/4054576236 conn(0x7fc3d0071d00 msgr2=0x7fc3d00fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 shutdown_connections 2024-04-25T01:33:29.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:29.526+0000 7fc3d801c700 1 -- 172.21.15.33:0/4054576236 wait complete. 2024-04-25T01:33:30.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:29 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/4054576236' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:33:30.187 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:33:30.187 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:33:30.187 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi033.front.sepia.ceph.com 2024-04-25T01:33:30.187 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T01:33:30.187 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:30.187 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:33:30.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:33:30.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:33:30.187 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:33:30.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:30.188 DEBUG:teuthology.orchestra.run.smithi033:> ip addr 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 ::1/128 scope host 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f0 2024-04-25T01:33:30.206 INFO:teuthology.orchestra.run.smithi033.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether 0c:c4:7a:6b:fd:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp7s0f1 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f0 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: inet 172.21.15.33/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft 42274sec preferred_lft 42274sec 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: inet6 fe80::ae1f:6bff:fea5:d294/64 scope link noprefixroute 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: link/ether ac:1f:6b:a5:d2:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:33:30.207 INFO:teuthology.orchestra.run.smithi033.stdout: altname enp1s0f1 2024-04-25T01:33:30.208 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:33:30.208 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:30.208 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:33:30.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:33:30.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr flush dev ceph-brx 2024-04-25T01:33:30.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set ceph-brx up 2024-04-25T01:33:30.208 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:33:30.208 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:33:30.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:29 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/4054576236' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:33:30.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:30.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:30.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:30.397 DEBUG:teuthology.orchestra.run.smithi033:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:33:30.422 INFO:teuthology.orchestra.run.smithi033.stdout:1 2024-04-25T01:33:30.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:30.424 DEBUG:teuthology.orchestra.run.smithi033:> route 2024-04-25T01:33:30.504 INFO:teuthology.orchestra.run.smithi033.stdout:Kernel IP routing table 2024-04-25T01:33:30.504 INFO:teuthology.orchestra.run.smithi033.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:33:30.504 INFO:teuthology.orchestra.run.smithi033.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T01:33:30.504 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T01:33:30.504 INFO:teuthology.orchestra.run.smithi033.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:33:30.505 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:30.505 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:33:30.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T01:33:30.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T01:33:30.505 DEBUG:teuthology.orchestra.run.smithi033:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T01:33:30.505 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:33:30.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:30.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:30.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:30.791 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list 2024-04-25T01:33:30.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:30.807 DEBUG:teuthology.orchestra.run.smithi033:> ip netns list-id 2024-04-25T01:33:30.875 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:30.876 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:33:30.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:33:30.876 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T01:33:30.876 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:33:30.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:30.992 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:30 smithi033 ceph-mon[26353]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:33:31.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:31.013 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T01:33:31.013 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:31.014 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:33:31.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T01:33:31.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:33:31.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T01:33:31.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T01:33:31.014 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T01:33:31.014 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:33:31.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:31.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:31.206 DEBUG:teuthology.orchestra.run.smithi033:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:31.206 DEBUG:teuthology.orchestra.run.smithi033:> set -e 2024-04-25T01:33:31.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set brx.0 up 2024-04-25T01:33:31.206 DEBUG:teuthology.orchestra.run.smithi033:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:33:31.206 DEBUG:teuthology.orchestra.run.smithi033:> ') 2024-04-25T01:33:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:31.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:30 smithi138 ceph-mon[32977]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:33:31.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:31.286 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T01:33:31.286 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:33:31.287 DEBUG:teuthology.orchestra.run.smithi033:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:31.304 INFO:teuthology.orchestra.run.smithi033.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T01:33:31.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:33:31.304 DEBUG:teuthology.orchestra.run.smithi033:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:31.362 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:33:31.431 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:33:31.492 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:33:31.493 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:33:31.493 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:33:31.493 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:33:31.493 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:33:31.493 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:33:31.493 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:33:31.493 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:33:31.493 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:33:31.494 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:33:31.495 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:33:31.495 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:33:31.495 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:33:31.495 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:33:31.495 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:33:31.495 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:33:31.495 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:33:31.495 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-04-25T01:33:31.496 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/9e6d06b16fe8b64e6e31c6da7e19615b8d8fdfb369e61b78a349554b09116ef1/merged 2024-04-25T01:33:31.496 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/73cd3cd5b6c1a692a8f0d17e6cb8f282e05e4caf02d2a2d42b9da85d16cf2f31/merged 2024-04-25T01:33:31.496 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-04-25T01:33:31.496 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:33:31.496 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8a26ccce2ffd3c7255171bba0918b2aa84b4367dc8517c5eb4a1e690175b30e3/merged 2024-04-25T01:33:31.496 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/985244f92a87f7bcc1f21b6a2611341e1e1730b597379111f71aff0c56bf135e/merged 2024-04-25T01:33:31.496 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/7fad4ed1d306e062e00024a94110dabf3d2350da09fa2469d00c3949300ae952/merged 2024-04-25T01:33:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/2a9aa08d01335b31ae51b9469a237a4ace780f2295bb0ab8d2da6010a3a29bc7/merged 2024-04-25T01:33:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/95e42a99443df648126bd6cadaaf273aee5885effc3cdeb58a2aff29697cef58/merged 2024-04-25T01:33:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/28e62f3cc9f1a0921217084da3df85fea4f6a1c4f521b7f0a8c8752f74a53a5c/merged 2024-04-25T01:33:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/2029f66ff8df76e9c340bc52acc256f721af1b30ea058e2080ad35868e943d82/merged 2024-04-25T01:33:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b58604c38d831b783401e0ff06ec78e76c4c17d92b29a857199e42d2b05858e5/merged 2024-04-25T01:33:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c0b1b81f0a24c024004834ca7734ce16e73c990e4d7c4990df2808e3784bd636/merged 2024-04-25T01:33:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c06b264c12f8f12ec0f70f999c5127e629866b7f2dab911cd744dc506bbe3afe/merged 2024-04-25T01:33:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:33:31.497 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:33:31.498 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:33:31.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:31.499 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:33:31.556 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:33:31.556 DEBUG:teuthology.orchestra.run.smithi033:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T01:33:31.603 DEBUG:teuthology.orchestra.run.smithi033:> sudo modprobe fuse 2024-04-25T01:33:31.632 DEBUG:teuthology.orchestra.run.smithi033:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/proc 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/sys 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/dev 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/security 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/shm 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/pts 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/run 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/cgroup 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/pstore 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/bpf 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/ 2024-04-25T01:33:31.692 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/selinux 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/mqueue 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/dev/hugepages 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/debug 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/tracing 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/kernel/config 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/sys/fs/fuse/connections 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/1000 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/ceph 2024-04-25T01:33:31.693 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/9e6d06b16fe8b64e6e31c6da7e19615b8d8fdfb369e61b78a349554b09116ef1/merged 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/73cd3cd5b6c1a692a8f0d17e6cb8f282e05e4caf02d2a2d42b9da85d16cf2f31/merged 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/run/user/0 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/8a26ccce2ffd3c7255171bba0918b2aa84b4367dc8517c5eb4a1e690175b30e3/merged 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/985244f92a87f7bcc1f21b6a2611341e1e1730b597379111f71aff0c56bf135e/merged 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/7fad4ed1d306e062e00024a94110dabf3d2350da09fa2469d00c3949300ae952/merged 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/2a9aa08d01335b31ae51b9469a237a4ace780f2295bb0ab8d2da6010a3a29bc7/merged 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/95e42a99443df648126bd6cadaaf273aee5885effc3cdeb58a2aff29697cef58/merged 2024-04-25T01:33:31.694 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/28e62f3cc9f1a0921217084da3df85fea4f6a1c4f521b7f0a8c8752f74a53a5c/merged 2024-04-25T01:33:31.695 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/2029f66ff8df76e9c340bc52acc256f721af1b30ea058e2080ad35868e943d82/merged 2024-04-25T01:33:31.695 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/b58604c38d831b783401e0ff06ec78e76c4c17d92b29a857199e42d2b05858e5/merged 2024-04-25T01:33:31.695 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c0b1b81f0a24c024004834ca7734ce16e73c990e4d7c4990df2808e3784bd636/merged 2024-04-25T01:33:31.695 INFO:teuthology.orchestra.run.smithi033.stdout:/var/lib/containers/storage/overlay/c06b264c12f8f12ec0f70f999c5127e629866b7f2dab911cd744dc506bbe3afe/merged 2024-04-25T01:33:31.695 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns 2024-04-25T01:33:31.695 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:33:31.695 INFO:teuthology.orchestra.run.smithi033.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T01:33:31.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:31.695 DEBUG:teuthology.orchestra.run.smithi033:> ls /sys/fs/fuse/connections 2024-04-25T01:33:31.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:2024-04-25T01:33:31.720+0000 7fc7ef8c0300 -1 init, newargv = 0x5624a38b28b0 newargc=15 2024-04-25T01:33:31.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[74113]: starting ceph client 2024-04-25T01:33:31.704 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[74113]: starting fuse 2024-04-25T01:33:31.714 INFO:teuthology.orchestra.run.smithi033.stdout:145 2024-04-25T01:33:31.714 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [145] 2024-04-25T01:33:31.714 DEBUG:teuthology.orchestra.run.smithi033:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:33:31.714 DEBUG:teuthology.orchestra.run.smithi033:> import glob 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> import re 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> import os 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> import subprocess 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> def _find_admin_socket(client_name): 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> files = glob.glob(asok_path) 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> # Given a non-glob path, it better be there 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> if "*" not in asok_path: 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> assert(len(files) == 1) 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> return files[0] 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> for f in files: 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:33:31.715 DEBUG:teuthology.orchestra.run.smithi033:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:33:31.716 DEBUG:teuthology.orchestra.run.smithi033:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:33:31.716 DEBUG:teuthology.orchestra.run.smithi033:> contents = proc_f.read() 2024-04-25T01:33:31.716 DEBUG:teuthology.orchestra.run.smithi033:> if mountpoint in contents: 2024-04-25T01:33:31.716 DEBUG:teuthology.orchestra.run.smithi033:> return f 2024-04-25T01:33:31.716 DEBUG:teuthology.orchestra.run.smithi033:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:33:31.716 DEBUG:teuthology.orchestra.run.smithi033:> 2024-04-25T01:33:31.716 DEBUG:teuthology.orchestra.run.smithi033:> print(_find_admin_socket("client.0")) 2024-04-25T01:33:31.716 DEBUG:teuthology.orchestra.run.smithi033:> ' 2024-04-25T01:33:31.839 INFO:teuthology.orchestra.run.smithi033.stdout:/var/run/ceph/ceph-client.0.74113.asok 2024-04-25T01:33:31.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:31.850 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.74113.asok 2024-04-25T01:33:31.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:31.851 DEBUG:teuthology.orchestra.run.smithi033:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.74113.asok status 2024-04-25T01:33:31.913 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:33:31.984 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:33:31.984 INFO:teuthology.orchestra.run.smithi033.stdout: "metadata": { 2024-04-25T01:33:31.984 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_sha1": "c61472c83ce76869b0fe076c19ed2f4bcc3fb4af", 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "entity_id": "0", 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "hostname": "smithi033", 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "pid": "74113", 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "root": "/" 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_count": 0, 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "dentry_pinned_count": 0, 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "id": 14506, 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "inst": { 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "name": { 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "client", 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: "num": 14506 2024-04-25T01:33:31.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 899480764 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": { 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "type": "v1", 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "nonce": 899480764 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "inst_str": "client.14506 192.168.0.1:0/899480764", 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "addr_str": "192.168.0.1:0/899480764", 2024-04-25T01:33:31.986 INFO:teuthology.orchestra.run.smithi033.stdout: "inode_count": 1, 2024-04-25T01:33:31.987 INFO:teuthology.orchestra.run.smithi033.stdout: "mds_epoch": 15, 2024-04-25T01:33:31.987 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch": 45, 2024-04-25T01:33:31.987 INFO:teuthology.orchestra.run.smithi033.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:33:31.987 INFO:teuthology.orchestra.run.smithi033.stdout: "blocklisted": false, 2024-04-25T01:33:31.987 INFO:teuthology.orchestra.run.smithi033.stdout: "fs_name": "cephfs" 2024-04-25T01:33:31.987 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:33:31.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T01:33:32.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:31 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:33:32.299 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:32.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:32 smithi033 ceph-mon[26353]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:33:33.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:32 smithi138 ceph-mon[32977]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.856+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1186351283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fbf40 msgr2=0x7f4e08101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.856+0000 7f4e0dd1d700 1 --2- 172.21.15.33:0/1186351283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fbf40 0x7f4e08101a30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc004940 tx=0x7f4dfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.856+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1186351283 shutdown_connections 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.856+0000 7f4e0dd1d700 1 --2- 172.21.15.33:0/1186351283 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fbf40 0x7f4e08101a30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.856+0000 7f4e0dd1d700 1 --2- 172.21.15.33:0/1186351283 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e080fb5a0 0x7f4e080fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.856+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1186351283 >> 172.21.15.33:0/1186351283 conn(0x7f4e08073350 msgr2=0x7f4e08073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.856+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1186351283 shutdown_connections 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.856+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1186351283 wait complete. 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e0dd1d700 1 Processor -- start 2024-04-25T01:33:33.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e0dd1d700 1 -- start start 2024-04-25T01:33:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e0dd1d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fb5a0 0x7f4e080fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e0dd1d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e080fbf40 0x7f4e080fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e0dd1d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e080fe480 con 0x7f4e080fb5a0 2024-04-25T01:33:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e0dd1d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e080fcfb0 con 0x7f4e080fbf40 2024-04-25T01:33:33.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fb5a0 0x7f4e080fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fb5a0 0x7f4e080fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:57992/0 (socket says 172.21.15.33:57992) 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e077fe700 1 -- 172.21.15.33:0/1920009541 learned_addr learned my addr 172.21.15.33:0/1920009541 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.857+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e080fbf40 0x7f4e080fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.858+0000 7f4e077fe700 1 -- 172.21.15.33:0/1920009541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e080fbf40 msgr2=0x7f4e080fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.858+0000 7f4e077fe700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e080fbf40 0x7f4e080fca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.858+0000 7f4e077fe700 1 -- 172.21.15.33:0/1920009541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dfc005040 con 0x7f4e080fb5a0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.858+0000 7f4e077fe700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fb5a0 0x7f4e080fc530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4df800eb40 tx=0x7f4df800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.858+0000 7f4deffff700 1 -- 172.21.15.33:0/1920009541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df8007cc0 con 0x7f4e080fb5a0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.858+0000 7f4deffff700 1 -- 172.21.15.33:0/1920009541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4df800f040 con 0x7f4e080fb5a0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.858+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e080fd230 con 0x7f4e080fb5a0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.858+0000 7f4deffff700 1 -- 172.21.15.33:0/1920009541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4df8009620 con 0x7f4e080fb5a0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.858+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0811a030 con 0x7f4e080fb5a0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.859+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0804c710 con 0x7f4e080fb5a0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.859+0000 7f4deffff700 1 -- 172.21.15.33:0/1920009541 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4df8009780 con 0x7f4e080fb5a0 2024-04-25T01:33:33.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.860+0000 7f4deffff700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4df00692a0 0x7f4df006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.860+0000 7f4deffff700 1 -- 172.21.15.33:0/1920009541 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4df8086d30 con 0x7f4e080fb5a0 2024-04-25T01:33:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.860+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4df00692a0 0x7f4df006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:33.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.860+0000 7f4e06ffd700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4df00692a0 0x7f4df006b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc004890 tx=0x7f4dfc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:33.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:33.864+0000 7f4deffff700 1 -- 172.21.15.33:0/1920009541 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4df804e250 con 0x7f4e080fb5a0 2024-04-25T01:33:34.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.106+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4e08109f00 con 0x7f4e080fb5a0 2024-04-25T01:33:34.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.106+0000 7f4deffff700 1 -- 172.21.15.33:0/1920009541 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+83 (secure 0 0 0) 0x7f4df8053110 con 0x7f4e080fb5a0 2024-04-25T01:33:34.083 INFO:teuthology.orchestra.run.smithi033.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-25T01:33:34.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4df00692a0 msgr2=0x7f4df006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:34.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4df00692a0 0x7f4df006b750 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc004890 tx=0x7f4dfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:34.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fb5a0 msgr2=0x7f4e080fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:34.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fb5a0 0x7f4e080fc530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4df800eb40 tx=0x7f4df800b040 comp rx=0 tx=0).stop 2024-04-25T01:33:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 shutdown_connections 2024-04-25T01:33:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4df00692a0 0x7f4df006b750 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4e080fb5a0 0x7f4e080fc530 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 --2- 172.21.15.33:0/1920009541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4e080fbf40 0x7f4e080fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 >> 172.21.15.33:0/1920009541 conn(0x7f4e08073350 msgr2=0x7f4e08100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 shutdown_connections 2024-04-25T01:33:34.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:34.108+0000 7f4e0dd1d700 1 -- 172.21.15.33:0/1920009541 wait complete. 2024-04-25T01:33:34.422 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:34 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1920009541' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:33:34.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:34 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1920009541' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T01:33:34.700 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T01:33:34.700 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T01:33:34.700 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi138.front.sepia.ceph.com 2024-04-25T01:33:34.700 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T01:33:34.700 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:34.700 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T01:33:34.700 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T01:33:34.700 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T01:33:34.700 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T01:33:34.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:34.701 DEBUG:teuthology.orchestra.run.smithi138:> ip addr 2024-04-25T01:33:34.719 INFO:teuthology.orchestra.run.smithi138.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 ::1/128 scope host 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7a brd ff:ff:ff:ff:ff:ff 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f0 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:d9:36:7b brd ff:ff:ff:ff:ff:ff 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: altname enp7s0f1 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T01:33:34.720 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:70 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:33:34.721 INFO:teuthology.orchestra.run.smithi138.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T01:33:34.721 INFO:teuthology.orchestra.run.smithi138.stdout: link/ether 0c:c4:7a:88:6f:71 brd ff:ff:ff:ff:ff:ff 2024-04-25T01:33:34.721 INFO:teuthology.orchestra.run.smithi138.stdout: inet 172.21.15.138/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T01:33:34.721 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft 42304sec preferred_lft 42304sec 2024-04-25T01:33:34.721 INFO:teuthology.orchestra.run.smithi138.stdout: inet6 fe80::2ddd:cf7d:fc75:8d44/64 scope link noprefixroute 2024-04-25T01:33:34.721 INFO:teuthology.orchestra.run.smithi138.stdout: valid_lft forever preferred_lft forever 2024-04-25T01:33:34.721 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T01:33:34.721 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:34.721 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:33:34.721 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add name ceph-brx type bridge 2024-04-25T01:33:34.721 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr flush dev ceph-brx 2024-04-25T01:33:34.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set ceph-brx up 2024-04-25T01:33:34.722 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T01:33:34.722 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:33:34.813 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:34.904 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:34.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:34.910 DEBUG:teuthology.orchestra.run.smithi138:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T01:33:34.934 INFO:teuthology.orchestra.run.smithi138.stdout:1 2024-04-25T01:33:34.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:34.936 DEBUG:teuthology.orchestra.run.smithi138:> route 2024-04-25T01:33:35.007 INFO:teuthology.orchestra.run.smithi138.stdout:Kernel IP routing table 2024-04-25T01:33:35.007 INFO:teuthology.orchestra.run.smithi138.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T01:33:35.007 INFO:teuthology.orchestra.run.smithi138.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T01:33:35.007 INFO:teuthology.orchestra.run.smithi138.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T01:33:35.008 INFO:teuthology.orchestra.run.smithi138.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T01:33:35.008 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:35.008 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:33:35.008 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T01:33:35.008 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T01:33:35.008 DEBUG:teuthology.orchestra.run.smithi138:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T01:33:35.008 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:33:35.095 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:35.292 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:35.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:35.299 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list 2024-04-25T01:33:35.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:35.315 DEBUG:teuthology.orchestra.run.smithi138:> ip netns list-id 2024-04-25T01:33:35.370 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:35.371 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:33:35.371 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:33:35.371 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T01:33:35.371 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:33:35.458 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:35.466 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:35 smithi138 ceph-mon[32977]: pgmap v192: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T01:33:35.482 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:35.488 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T01:33:35.488 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:35.488 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:33:35.488 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T01:33:35.488 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T01:33:35.489 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T01:33:35.489 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T01:33:35.489 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T01:33:35.489 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:33:35.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:35 smithi033 ceph-mon[26353]: pgmap v192: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-04-25T01:33:35.536 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:35.614 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:35.620 DEBUG:teuthology.orchestra.run.smithi138:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T01:33:35.621 DEBUG:teuthology.orchestra.run.smithi138:> set -e 2024-04-25T01:33:35.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set brx.0 up 2024-04-25T01:33:35.621 DEBUG:teuthology.orchestra.run.smithi138:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T01:33:35.621 DEBUG:teuthology.orchestra.run.smithi138:> ') 2024-04-25T01:33:35.668 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T01:33:35.688 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:35.696 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T01:33:35.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:33:35.696 DEBUG:teuthology.orchestra.run.smithi138:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:35.710 INFO:teuthology.orchestra.run.smithi138.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T01:33:35.710 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T01:33:35.710 DEBUG:teuthology.orchestra.run.smithi138:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:35.764 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T01:33:35.829 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:33:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T01:33:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T01:33:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T01:33:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T01:33:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T01:33:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T01:33:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T01:33:35.887 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:33:35.888 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/0 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/9418b847a0cd708b5fc35f187db8f895f5835c6ff65a00ab8af6036bc874aba5/merged 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/c81744b04e972300421f9c59308c442f1fca62d595403d444d3dfdb1b5f6f1b3/merged 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/8b645ed6fc8334e7fc8f9212625599673346d7b503266f14759844863bc607a9/merged 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/6994e64a80463e9088e246db40493ae703384fac085a1b8467381048437aa960/merged 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/b4672fb7bc2dce97b006e90cd4893dd01d72de9468b70687f22e21b9e605c41c/merged 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/c5eac862eafde698ab60b1de3ff3cd2257344e520259947c58a7c9a531d180ee/merged 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/a2c328df44ef24a496c6936eec3560ec8094faf3932fd5f9868c74ebc156d31e/merged 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/dad4091ea1854b8d854799ed8ceedb905db20034ea1194259afa3593d278aed1/merged 2024-04-25T01:33:35.889 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/44e98f7ecc8de9592c9cffc0d3054d8cba81c9fb9a3b0d6fe9ccda4aa76e063f/merged 2024-04-25T01:33:35.890 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T01:33:35.890 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:33:35.890 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:33:35.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:35.890 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T01:33:35.942 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T01:33:35.942 DEBUG:teuthology.orchestra.run.smithi138:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T01:33:35.984 DEBUG:teuthology.orchestra.run.smithi138:> sudo modprobe fuse 2024-04-25T01:33:36.008 DEBUG:teuthology.orchestra.run.smithi138:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T01:33:36.063 INFO:teuthology.orchestra.run.smithi138.stdout:/proc 2024-04-25T01:33:36.063 INFO:teuthology.orchestra.run.smithi138.stdout:/sys 2024-04-25T01:33:36.063 INFO:teuthology.orchestra.run.smithi138.stdout:/dev 2024-04-25T01:33:36.063 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/security 2024-04-25T01:33:36.063 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/shm 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/pts 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/run 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/cgroup 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/pstore 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/bpf 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/ 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/selinux 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/mqueue 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/dev/hugepages 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/debug 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/tracing 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/kernel/config 2024-04-25T01:33:36.064 INFO:teuthology.orchestra.run.smithi138.stdout:/sys/fs/fuse/connections 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/1000 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/ceph 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/run/user/0 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/9418b847a0cd708b5fc35f187db8f895f5835c6ff65a00ab8af6036bc874aba5/merged 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/c81744b04e972300421f9c59308c442f1fca62d595403d444d3dfdb1b5f6f1b3/merged 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/8b645ed6fc8334e7fc8f9212625599673346d7b503266f14759844863bc607a9/merged 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/6994e64a80463e9088e246db40493ae703384fac085a1b8467381048437aa960/merged 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/b4672fb7bc2dce97b006e90cd4893dd01d72de9468b70687f22e21b9e605c41c/merged 2024-04-25T01:33:36.065 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/c5eac862eafde698ab60b1de3ff3cd2257344e520259947c58a7c9a531d180ee/merged 2024-04-25T01:33:36.066 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/a2c328df44ef24a496c6936eec3560ec8094faf3932fd5f9868c74ebc156d31e/merged 2024-04-25T01:33:36.066 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/dad4091ea1854b8d854799ed8ceedb905db20034ea1194259afa3593d278aed1/merged 2024-04-25T01:33:36.066 INFO:teuthology.orchestra.run.smithi138.stdout:/var/lib/containers/storage/overlay/44e98f7ecc8de9592c9cffc0d3054d8cba81c9fb9a3b0d6fe9ccda4aa76e063f/merged 2024-04-25T01:33:36.066 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns 2024-04-25T01:33:36.066 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:33:36.066 INFO:teuthology.orchestra.run.smithi138.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T01:33:36.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:36.066 DEBUG:teuthology.orchestra.run.smithi138:> ls /sys/fs/fuse/connections 2024-04-25T01:33:36.094 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:2024-04-25T01:33:36.116+0000 7fdaa0e3d300 -1 init, newargv = 0x559c532747a0 newargc=15 2024-04-25T01:33:36.094 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[58277]: starting ceph client 2024-04-25T01:33:36.101 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi138.stderr:ceph-fuse[58277]: starting fuse 2024-04-25T01:33:36.110 INFO:teuthology.orchestra.run.smithi138.stdout:112 2024-04-25T01:33:36.110 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [112] 2024-04-25T01:33:36.110 DEBUG:teuthology.orchestra.run.smithi138:> sudo stdin-killer -- python3 -c ' 2024-04-25T01:33:36.110 DEBUG:teuthology.orchestra.run.smithi138:> import glob 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> import re 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> import os 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> import subprocess 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> def _find_admin_socket(client_name): 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> files = glob.glob(asok_path) 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> # Given a non-glob path, it better be there 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> if "*" not in asok_path: 2024-04-25T01:33:36.111 DEBUG:teuthology.orchestra.run.smithi138:> assert(len(files) == 1) 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> return files[0] 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> for f in files: 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> contents = proc_f.read() 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> if mountpoint in contents: 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> return f 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T01:33:36.112 DEBUG:teuthology.orchestra.run.smithi138:> 2024-04-25T01:33:36.113 DEBUG:teuthology.orchestra.run.smithi138:> print(_find_admin_socket("client.1")) 2024-04-25T01:33:36.113 DEBUG:teuthology.orchestra.run.smithi138:> ' 2024-04-25T01:33:36.226 INFO:teuthology.orchestra.run.smithi138.stdout:/var/run/ceph/ceph-client.1.58277.asok 2024-04-25T01:33:36.229 INFO:teuthology.orchestra.run.smithi138.stderr:2024-04-25T01:33:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T01:33:36.237 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.58277.asok 2024-04-25T01:33:36.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:36.237 DEBUG:teuthology.orchestra.run.smithi138:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.58277.asok status 2024-04-25T01:33:36.367 INFO:teuthology.orchestra.run.smithi138.stdout:{ 2024-04-25T01:33:36.367 INFO:teuthology.orchestra.run.smithi138.stdout: "metadata": { 2024-04-25T01:33:36.367 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_sha1": "c61472c83ce76869b0fe076c19ed2f4bcc3fb4af", 2024-04-25T01:33:36.367 INFO:teuthology.orchestra.run.smithi138.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-25T01:33:36.367 INFO:teuthology.orchestra.run.smithi138.stdout: "entity_id": "1", 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "hostname": "smithi138", 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "pid": "58277", 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "root": "/" 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_count": 0, 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "dentry_pinned_count": 0, 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "id": 24333, 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "inst": { 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "name": { 2024-04-25T01:33:36.368 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "client", 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "num": 24333 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 193185086 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: } 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": { 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "type": "v1", 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "addr": "192.168.0.1:0", 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "nonce": 193185086 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: }, 2024-04-25T01:33:36.369 INFO:teuthology.orchestra.run.smithi138.stdout: "inst_str": "client.24333 192.168.0.1:0/193185086", 2024-04-25T01:33:36.370 INFO:teuthology.orchestra.run.smithi138.stdout: "addr_str": "192.168.0.1:0/193185086", 2024-04-25T01:33:36.370 INFO:teuthology.orchestra.run.smithi138.stdout: "inode_count": 1, 2024-04-25T01:33:36.370 INFO:teuthology.orchestra.run.smithi138.stdout: "mds_epoch": 15, 2024-04-25T01:33:36.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch": 45, 2024-04-25T01:33:36.370 INFO:teuthology.orchestra.run.smithi138.stdout: "osd_epoch_barrier": 0, 2024-04-25T01:33:36.370 INFO:teuthology.orchestra.run.smithi138.stdout: "blocklisted": false, 2024-04-25T01:33:36.370 INFO:teuthology.orchestra.run.smithi138.stdout: "fs_name": "cephfs" 2024-04-25T01:33:36.370 INFO:teuthology.orchestra.run.smithi138.stdout:} 2024-04-25T01:33:36.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:36.375 DEBUG:teuthology.orchestra.run.smithi033:> stat --file-system '--printf=%T 2024-04-25T01:33:36.375 DEBUG:teuthology.orchestra.run.smithi033:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:36.396 INFO:teuthology.orchestra.run.smithi033.stdout:fuseblk 2024-04-25T01:33:36.396 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:36.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:36.397 DEBUG:teuthology.orchestra.run.smithi033:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:36.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:36.473 DEBUG:teuthology.orchestra.run.smithi138:> stat --file-system '--printf=%T 2024-04-25T01:33:36.473 DEBUG:teuthology.orchestra.run.smithi138:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:36.489 INFO:teuthology.orchestra.run.smithi138.stdout:fuseblk 2024-04-25T01:33:36.489 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:36.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T01:33:36.490 DEBUG:teuthology.orchestra.run.smithi138:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:37.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:37 smithi033 ceph-mon[26353]: pgmap v193: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T01:33:37.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:37 smithi138 ceph-mon[32977]: pgmap v193: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T01:33:39.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:39 smithi033 ceph-mon[26353]: pgmap v194: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T01:33:39.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:39 smithi138 ceph-mon[32977]: pgmap v194: 65 pgs: 65 active+clean; 581 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 341 B/s wr, 3 op/s 2024-04-25T01:33:40.729 INFO:teuthology.run_tasks:Running task print... 2024-04-25T01:33:40.743 INFO:teuthology.task.print:**** done client 2024-04-25T01:33:40.743 INFO:teuthology.run_tasks:Running task parallel... 2024-04-25T01:33:40.761 INFO:teuthology.task.parallel:starting parallel... 2024-04-25T01:33:40.762 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-25T01:33:40.763 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-25T01:33:40.763 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:33:40.764 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-25T01:33:40.764 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-25T01:33:40.765 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-25T01:33:40.770 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:33:40.770 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T01:33:40.770 INFO:tasks.workunit:timeout=3h 2024-04-25T01:33:40.770 INFO:tasks.workunit:cleanup=True 2024-04-25T01:33:40.770 DEBUG:teuthology.orchestra.run.smithi033:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T01:33:40.792 INFO:teuthology.orchestra.run.smithi033.stdout:Device: 91h/145d Inode: 1 Links: 2 2024-04-25T01:33:40.792 INFO:teuthology.orchestra.run.smithi033.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:33:40.792 INFO:teuthology.orchestra.run.smithi033.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T01:33:40.792 INFO:teuthology.orchestra.run.smithi033.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T01:33:40.792 INFO:teuthology.orchestra.run.smithi033.stdout:Modify: 2024-04-25 01:33:00.922461724 +0000 2024-04-25T01:33:40.792 INFO:teuthology.orchestra.run.smithi033.stdout:Change: 2024-04-25 01:33:36.493370368 +0000 2024-04-25T01:33:40.792 INFO:teuthology.orchestra.run.smithi033.stdout: Birth: - 2024-04-25T01:33:40.793 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T01:33:40.793 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T01:33:40.866 DEBUG:teuthology.orchestra.run.smithi138:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:40.886 INFO:teuthology.orchestra.run.smithi138.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:40.887 INFO:teuthology.orchestra.run.smithi138.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T01:33:40.887 INFO:teuthology.orchestra.run.smithi138.stdout:Device: 70h/112d Inode: 1 Links: 3 2024-04-25T01:33:40.887 INFO:teuthology.orchestra.run.smithi138.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T01:33:40.887 INFO:teuthology.orchestra.run.smithi138.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T01:33:40.887 INFO:teuthology.orchestra.run.smithi138.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T01:33:40.887 INFO:teuthology.orchestra.run.smithi138.stdout:Modify: 2024-04-25 01:33:40.882561346 +0000 2024-04-25T01:33:40.887 INFO:teuthology.orchestra.run.smithi138.stdout:Change: 2024-04-25 01:33:40.882561346 +0000 2024-04-25T01:33:40.887 INFO:teuthology.orchestra.run.smithi138.stdout: Birth: - 2024-04-25T01:33:40.887 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-25T01:33:40.887 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-25T01:33:40.966 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:33:40.966 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:33:40.984 INFO:tasks.workunit.client.0.smithi033.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T01:33:41.026 INFO:tasks.workunit.client.1.smithi138.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-25T01:33:41.053 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:41.468 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:41 smithi033 ceph-mon[26353]: pgmap v195: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T01:33:41.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:41 smithi138 ceph-mon[32977]: pgmap v195: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T01:33:42.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.785+0000 7f6d10a02700 1 -- 172.21.15.33:0/494096506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fcc00 msgr2=0x7f6d0c0fd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.785+0000 7f6d10a02700 1 --2- 172.21.15.33:0/494096506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fcc00 0x7f6d0c0fd010 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc004940 tx=0x7f6cfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.785+0000 7f6d10a02700 1 -- 172.21.15.33:0/494096506 shutdown_connections 2024-04-25T01:33:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.785+0000 7f6d10a02700 1 --2- 172.21.15.33:0/494096506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c0fdeb0 0x7f6d0c0fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.785+0000 7f6d10a02700 1 --2- 172.21.15.33:0/494096506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fcc00 0x7f6d0c0fd010 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.785+0000 7f6d10a02700 1 -- 172.21.15.33:0/494096506 >> 172.21.15.33:0/494096506 conn(0x7f6d0c0f8340 msgr2=0x7f6d0c0fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.785+0000 7f6d10a02700 1 -- 172.21.15.33:0/494096506 shutdown_connections 2024-04-25T01:33:42.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.785+0000 7f6d10a02700 1 -- 172.21.15.33:0/494096506 wait complete. 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.786+0000 7f6d10a02700 1 Processor -- start 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.786+0000 7f6d10a02700 1 -- start start 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.786+0000 7f6d10a02700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c0fcc00 0x7f6d0c10d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.786+0000 7f6d10a02700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fdeb0 0x7f6d0c10dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.786+0000 7f6d10a02700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c10e410 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.786+0000 7f6d10a02700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0c10ec00 con 0x7f6d0c0fcc00 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.786+0000 7f6d0a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fdeb0 0x7f6d0c10dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.786+0000 7f6d0a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fdeb0 0x7f6d0c10dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58166/0 (socket says 172.21.15.33:58166) 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.786+0000 7f6d0a59c700 1 -- 172.21.15.33:0/167371086 learned_addr learned my addr 172.21.15.33:0/167371086 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6d0ad9d700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c0fcc00 0x7f6d0c10d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6d0a59c700 1 -- 172.21.15.33:0/167371086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c0fcc00 msgr2=0x7f6d0c10d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6d0a59c700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c0fcc00 0x7f6d0c10d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6d0a59c700 1 -- 172.21.15.33:0/167371086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc005040 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6d0a59c700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fdeb0 0x7f6d0c10dd80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b790 tx=0x7f6d00009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:42.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6cf37fe700 1 -- 172.21.15.33:0/167371086 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d0000f6f0 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c10eee0 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6cf37fe700 1 -- 172.21.15.33:0/167371086 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d0000fd20 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6cf37fe700 1 -- 172.21.15.33:0/167371086 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d000103a0 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.787+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0c10f4a0 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.788+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0c0fc000 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.789+0000 7f6cf37fe700 1 -- 172.21.15.33:0/167371086 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f6d00011400 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.789+0000 7f6cf37fe700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6cf40692a0 0x7f6cf406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.789+0000 7f6cf37fe700 1 -- 172.21.15.33:0/167371086 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6d00086fe0 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.789+0000 7f6d0ad9d700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6cf40692a0 0x7f6cf406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:42.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.790+0000 7f6d0ad9d700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6cf40692a0 0x7f6cf406b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00b970 tx=0x7f6cfc009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:42.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.794+0000 7f6cf37fe700 1 -- 172.21.15.33:0/167371086 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6d00056d40 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.988+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f6d0c04bed0 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.988+0000 7f6cf37fe700 1 -- 172.21.15.33:0/167371086 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f6d000568d0 con 0x7f6d0c0fdeb0 2024-04-25T01:33:42.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.991+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6cf40692a0 msgr2=0x7f6cf406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.991+0000 7f6d10a02700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6cf40692a0 0x7f6cf406b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00b970 tx=0x7f6cfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.991+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fdeb0 msgr2=0x7f6d0c10dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.991+0000 7f6d10a02700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fdeb0 0x7f6d0c10dd80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000b790 tx=0x7f6d00009250 comp rx=0 tx=0).stop 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.992+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 shutdown_connections 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.992+0000 7f6d10a02700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d0c0fcc00 0x7f6d0c10d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.992+0000 7f6d10a02700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6cf40692a0 0x7f6cf406b750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.992+0000 7f6d10a02700 1 --2- 172.21.15.33:0/167371086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d0c0fdeb0 0x7f6d0c10dd80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.992+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 >> 172.21.15.33:0/167371086 conn(0x7f6d0c0f8340 msgr2=0x7f6d0c0f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.992+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 shutdown_connections 2024-04-25T01:33:42.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:42.992+0000 7f6d10a02700 1 -- 172.21.15.33:0/167371086 wait complete. 2024-04-25T01:33:43.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:43 smithi033 ceph-mon[26353]: pgmap v196: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-25T01:33:43.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:43 smithi138 ceph-mon[32977]: pgmap v196: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-25T01:33:43.666 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-25T01:33:43.976 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:45.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:45 smithi138 ceph-mon[32977]: pgmap v197: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-25T01:33:45.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:45 smithi033 ceph-mon[26353]: pgmap v197: 65 pgs: 65 active+clean; 582 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-25T01:33:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 -- 172.21.15.33:0/2444496643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 msgr2=0x7f622c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 --2- 172.21.15.33:0/2444496643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 0x7f622c0fd270 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f621c004940 tx=0x7f621c009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 -- 172.21.15.33:0/2444496643 shutdown_connections 2024-04-25T01:33:45.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 --2- 172.21.15.33:0/2444496643 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f622c0fdfd0 0x7f622c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 --2- 172.21.15.33:0/2444496643 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 0x7f622c0fd270 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 -- 172.21.15.33:0/2444496643 >> 172.21.15.33:0/2444496643 conn(0x7f622c0f8540 msgr2=0x7f622c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 -- 172.21.15.33:0/2444496643 shutdown_connections 2024-04-25T01:33:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 -- 172.21.15.33:0/2444496643 wait complete. 2024-04-25T01:33:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 Processor -- start 2024-04-25T01:33:45.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.793+0000 7f6231ba1700 1 -- start start 2024-04-25T01:33:45.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f6231ba1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 0x7f622c10d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f6231ba1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f622c0fdfd0 0x7f622c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f6231ba1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c10e480 con 0x7f622c0fce60 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f6231ba1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c10e5f0 con 0x7f622c0fdfd0 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f622b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 0x7f622c10d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f622b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 0x7f622c10d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37436/0 (socket says 172.21.15.33:37436) 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f622b7fe700 1 -- 172.21.15.33:0/3111718766 learned_addr learned my addr 172.21.15.33:0/3111718766 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f622b7fe700 1 -- 172.21.15.33:0/3111718766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f622c0fdfd0 msgr2=0x7f622c10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f622affd700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f622c0fdfd0 0x7f622c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f622b7fe700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f622c0fdfd0 0x7f622c10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f622b7fe700 1 -- 172.21.15.33:0/3111718766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621c005040 con 0x7f622c0fce60 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f622affd700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f622c0fdfd0 0x7f622c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.794+0000 7f622b7fe700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 0x7f622c10d940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f621c009150 tx=0x7f621c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.795+0000 7f6213fff700 1 -- 172.21.15.33:0/3111718766 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f621c00e040 con 0x7f622c0fce60 2024-04-25T01:33:45.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.795+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c112ff0 con 0x7f622c0fce60 2024-04-25T01:33:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.795+0000 7f6213fff700 1 -- 172.21.15.33:0/3111718766 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f621c00bbf0 con 0x7f622c0fce60 2024-04-25T01:33:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.795+0000 7f6213fff700 1 -- 172.21.15.33:0/3111718766 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f621c01c630 con 0x7f622c0fce60 2024-04-25T01:33:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.795+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c113520 con 0x7f622c0fce60 2024-04-25T01:33:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.796+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c04c710 con 0x7f622c0fce60 2024-04-25T01:33:45.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.796+0000 7f6213fff700 1 -- 172.21.15.33:0/3111718766 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f621c00bd60 con 0x7f622c0fce60 2024-04-25T01:33:45.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.797+0000 7f6213fff700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f62140692a0 0x7f621406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:45.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.797+0000 7f6213fff700 1 -- 172.21.15.33:0/3111718766 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f621c0885b0 con 0x7f622c0fce60 2024-04-25T01:33:45.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.797+0000 7f622affd700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f62140692a0 0x7f621406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:45.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.797+0000 7f622affd700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f62140692a0 0x7f621406b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6220007fd0 tx=0x7f6220009040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:45.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.801+0000 7f6213fff700 1 -- 172.21.15.33:0/3111718766 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f621c058420 con 0x7f622c0fce60 2024-04-25T01:33:45.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.998+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f622c113ae0 con 0x7f622c0fce60 2024-04-25T01:33:45.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:45.998+0000 7f6213fff700 1 -- 172.21.15.33:0/3111718766 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f621c057fb0 con 0x7f622c0fce60 2024-04-25T01:33:45.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.000+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f62140692a0 msgr2=0x7f621406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:45.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.000+0000 7f6231ba1700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f62140692a0 0x7f621406b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6220007fd0 tx=0x7f6220009040 comp rx=0 tx=0).stop 2024-04-25T01:33:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.000+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 msgr2=0x7f622c10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.000+0000 7f6231ba1700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 0x7f622c10d940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f621c009150 tx=0x7f621c009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.000+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 shutdown_connections 2024-04-25T01:33:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.000+0000 7f6231ba1700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f62140692a0 0x7f621406b750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.000+0000 7f6231ba1700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f622c0fce60 0x7f622c10d940 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.000+0000 7f6231ba1700 1 --2- 172.21.15.33:0/3111718766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f622c0fdfd0 0x7f622c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.000+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 >> 172.21.15.33:0/3111718766 conn(0x7f622c0f8540 msgr2=0x7f622c0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.001+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 shutdown_connections 2024-04-25T01:33:45.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:46.001+0000 7f6231ba1700 1 -- 172.21.15.33:0/3111718766 wait complete. 2024-04-25T01:33:46.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:33:46.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:33:46.828 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-25T01:33:47.103 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:47.475 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:47 smithi033 ceph-mon[26353]: pgmap v198: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T01:33:47.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:47 smithi138 ceph-mon[32977]: pgmap v198: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.845+0000 7fce4dc42700 1 -- 172.21.15.33:0/390205197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480662c0 msgr2=0x7fce480ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.845+0000 7fce4dc42700 1 --2- 172.21.15.33:0/390205197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480662c0 0x7fce480ff9a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fce38004940 tx=0x7fce38009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.845+0000 7fce4dc42700 1 -- 172.21.15.33:0/390205197 shutdown_connections 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.845+0000 7fce4dc42700 1 --2- 172.21.15.33:0/390205197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce480ffee0 0x7fce48102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.845+0000 7fce4dc42700 1 --2- 172.21.15.33:0/390205197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480662c0 0x7fce480ff9a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.845+0000 7fce4dc42700 1 -- 172.21.15.33:0/390205197 >> 172.21.15.33:0/390205197 conn(0x7fce480f7430 msgr2=0x7fce480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.845+0000 7fce4dc42700 1 -- 172.21.15.33:0/390205197 shutdown_connections 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.845+0000 7fce4dc42700 1 -- 172.21.15.33:0/390205197 wait complete. 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.846+0000 7fce4dc42700 1 Processor -- start 2024-04-25T01:33:48.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.846+0000 7fce4dc42700 1 -- start start 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.846+0000 7fce4dc42700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce480662c0 0x7fce4810daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.846+0000 7fce4dc42700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480ffee0 0x7fce4810dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.846+0000 7fce4dc42700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4810e670 con 0x7fce480ffee0 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.846+0000 7fce4dc42700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4810edc0 con 0x7fce480662c0 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce46ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480ffee0 0x7fce4810dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce46ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480ffee0 0x7fce4810dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37446/0 (socket says 172.21.15.33:37446) 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce46ffd700 1 -- 172.21.15.33:0/718430877 learned_addr learned my addr 172.21.15.33:0/718430877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce477fe700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce480662c0 0x7fce4810daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce46ffd700 1 -- 172.21.15.33:0/718430877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce480662c0 msgr2=0x7fce4810daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce46ffd700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce480662c0 0x7fce4810daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce46ffd700 1 -- 172.21.15.33:0/718430877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38005040 con 0x7fce480ffee0 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce477fe700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce480662c0 0x7fce4810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce46ffd700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480ffee0 0x7fce4810dfe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b790 tx=0x7fce3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:48.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce2ffff700 1 -- 172.21.15.33:0/718430877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3c00f770 con 0x7fce480ffee0 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce2ffff700 1 -- 172.21.15.33:0/718430877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce3c00fda0 con 0x7fce480ffee0 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4810f0a0 con 0x7fce480ffee0 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce2ffff700 1 -- 172.21.15.33:0/718430877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3c00d310 con 0x7fce480ffee0 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.847+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4810f660 con 0x7fce480ffee0 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.848+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4805f5c0 con 0x7fce480ffee0 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.849+0000 7fce2ffff700 1 -- 172.21.15.33:0/718430877 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fce3c010420 con 0x7fce480ffee0 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.849+0000 7fce2ffff700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fce300692a0 0x7fce3006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.849+0000 7fce2ffff700 1 -- 172.21.15.33:0/718430877 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fce3c0870b0 con 0x7fce480ffee0 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.849+0000 7fce477fe700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fce300692a0 0x7fce3006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:48.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.850+0000 7fce477fe700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fce300692a0 0x7fce3006b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fce3800b970 tx=0x7fce38009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:48.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:48.854+0000 7fce2ffff700 1 -- 172.21.15.33:0/718430877 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fce3c056f10 con 0x7fce480ffee0 2024-04-25T01:33:49.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.049+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fce4804bed0 con 0x7fce480ffee0 2024-04-25T01:33:49.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.050+0000 7fce2ffff700 1 -- 172.21.15.33:0/718430877 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fce4804bed0 con 0x7fce480ffee0 2024-04-25T01:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fce300692a0 msgr2=0x7fce3006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fce300692a0 0x7fce3006b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fce3800b970 tx=0x7fce38009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480ffee0 msgr2=0x7fce4810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480ffee0 0x7fce4810dfe0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b790 tx=0x7fce3c009250 comp rx=0 tx=0).stop 2024-04-25T01:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 shutdown_connections 2024-04-25T01:33:49.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce480662c0 0x7fce4810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fce300692a0 0x7fce3006b750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 --2- 172.21.15.33:0/718430877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce480ffee0 0x7fce4810dfe0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 >> 172.21.15.33:0/718430877 conn(0x7fce480f7430 msgr2=0x7fce480f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 shutdown_connections 2024-04-25T01:33:49.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:49.052+0000 7fce4dc42700 1 -- 172.21.15.33:0/718430877 wait complete. 2024-04-25T01:33:49.361 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:49 smithi033 ceph-mon[26353]: pgmap v199: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-25T01:33:49.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:49 smithi138 ceph-mon[32977]: pgmap v199: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-25T01:33:49.673 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-25T01:33:49.941 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:33:51.327 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:51 smithi033 ceph-mon[26353]: pgmap v200: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T01:33:51.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:51 smithi138 ceph-mon[32977]: pgmap v200: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 853 B/s wr, 5 op/s 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.669+0000 7f25beef9700 1 -- 172.21.15.33:0/1260391578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 msgr2=0x7f25b80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.669+0000 7f25beef9700 1 --2- 172.21.15.33:0/1260391578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 0x7f25b80fe440 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f25a4004940 tx=0x7f25a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.669+0000 7f25beef9700 1 -- 172.21.15.33:0/1260391578 shutdown_connections 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.669+0000 7f25beef9700 1 --2- 172.21.15.33:0/1260391578 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 0x7f25b80fe440 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.669+0000 7f25beef9700 1 --2- 172.21.15.33:0/1260391578 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fce60 0x7f25b80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.669+0000 7f25beef9700 1 -- 172.21.15.33:0/1260391578 >> 172.21.15.33:0/1260391578 conn(0x7f25b80f8540 msgr2=0x7f25b80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.669+0000 7f25beef9700 1 -- 172.21.15.33:0/1260391578 shutdown_connections 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.669+0000 7f25beef9700 1 -- 172.21.15.33:0/1260391578 wait complete. 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.669+0000 7f25beef9700 1 Processor -- start 2024-04-25T01:33:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25beef9700 1 -- start start 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25beef9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fce60 0x7f25b8071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25beef9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 0x7f25b80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25beef9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8072200 con 0x7f25b80fdfd0 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25beef9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b80707f0 con 0x7f25b80fce60 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 0x7f25b80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 0x7f25b80702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37470/0 (socket says 172.21.15.33:37470) 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25b7fff700 1 -- 172.21.15.33:0/3979210806 learned_addr learned my addr 172.21.15.33:0/3979210806 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25bcc95700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fce60 0x7f25b8071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25b7fff700 1 -- 172.21.15.33:0/3979210806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fce60 msgr2=0x7f25b8071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25b7fff700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fce60 0x7f25b8071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.670+0000 7f25b7fff700 1 -- 172.21.15.33:0/3979210806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a4005040 con 0x7f25b80fdfd0 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.671+0000 7f25b7fff700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 0x7f25b80702b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f25a400b3b0 tx=0x7f25a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.671+0000 7f25b57fa700 1 -- 172.21.15.33:0/3979210806 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a400e040 con 0x7f25b80fdfd0 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.671+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b8070a40 con 0x7f25b80fdfd0 2024-04-25T01:33:51.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.671+0000 7f25b57fa700 1 -- 172.21.15.33:0/3979210806 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25a400bbf0 con 0x7f25b80fdfd0 2024-04-25T01:33:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.671+0000 7f25b57fa700 1 -- 172.21.15.33:0/3979210806 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a401c630 con 0x7f25b80fdfd0 2024-04-25T01:33:51.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.671+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b8071000 con 0x7f25b80fdfd0 2024-04-25T01:33:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.672+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b804c710 con 0x7f25b80fdfd0 2024-04-25T01:33:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.672+0000 7f25b57fa700 1 -- 172.21.15.33:0/3979210806 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f25a400bd60 con 0x7f25b80fdfd0 2024-04-25T01:33:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.673+0000 7f25b57fa700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f25a80692a0 0x7f25a806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:33:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.673+0000 7f25b57fa700 1 -- 172.21.15.33:0/3979210806 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f25a4087e50 con 0x7f25b80fdfd0 2024-04-25T01:33:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.673+0000 7f25bcc95700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f25a80692a0 0x7f25a806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:33:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.673+0000 7f25bcc95700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f25a80692a0 0x7f25a806b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f25ac004f40 tx=0x7f25ac00b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:33:51.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.677+0000 7f25b57fa700 1 -- 172.21.15.33:0/3979210806 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f25a4057c10 con 0x7f25b80fdfd0 2024-04-25T01:33:51.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:33:51.890+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f25b8102340 con 0x7f25a80692a0 2024-04-25T01:33:53.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:53 smithi033 ceph-mon[26353]: from='client.14528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:33:53.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:53 smithi033 ceph-mon[26353]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-25T01:33:53.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:53 smithi138 ceph-mon[32977]: from='client.14528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:33:53.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:53 smithi138 ceph-mon[32977]: pgmap v201: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-04-25T01:33:55.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:55 smithi033 ceph-mon[26353]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-25T01:33:55.507 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:55 smithi138 ceph-mon[32977]: pgmap v202: 65 pgs: 65 active+clean; 587 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 767 B/s wr, 4 op/s 2024-04-25T01:33:57.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:57 smithi033 ceph-mon[26353]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T01:33:57.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:57 smithi138 ceph-mon[32977]: pgmap v203: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 5 op/s 2024-04-25T01:33:59.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:33:59 smithi033 ceph-mon[26353]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-25T01:33:59.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:33:59 smithi138 ceph-mon[32977]: pgmap v204: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 597 B/s wr, 4 op/s 2024-04-25T01:34:01.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:01 smithi033 ceph-mon[26353]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T01:34:01.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:34:01.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:01 smithi138 ceph-mon[32977]: pgmap v205: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-04-25T01:34:01.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:34:03.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:03 smithi033 ceph-mon[26353]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-25T01:34:03.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:03 smithi138 ceph-mon[32977]: pgmap v206: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-25T01:34:05.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:05 smithi138 ceph-mon[32977]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-25T01:34:05.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:05 smithi033 ceph-mon[26353]: pgmap v207: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-04-25T01:34:07.242 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:06 smithi033 ceph-mon[26353]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 426 B/s wr, 4 op/s 2024-04-25T01:34:07.268 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:06 smithi138 ceph-mon[32977]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.8 KiB/s rd, 426 B/s wr, 4 op/s 2024-04-25T01:34:09.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:09 smithi033 ceph-mon[26353]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:09.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:09 smithi138 ceph-mon[32977]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:11.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:11 smithi033 ceph-mon[26353]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:11.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:11 smithi138 ceph-mon[32977]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:13.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:13 smithi033 ceph-mon[26353]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:13.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:13 smithi138 ceph-mon[32977]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:15.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:15 smithi033 ceph-mon[26353]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:15.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:15 smithi138 ceph-mon[32977]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:16.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:34:16.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:34:17.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:17 smithi033 ceph-mon[26353]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:34:17.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:17 smithi138 ceph-mon[32977]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:34:19.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:19 smithi033 ceph-mon[26353]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:19.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:19 smithi138 ceph-mon[32977]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:21.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:21 smithi033 ceph-mon[26353]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:21.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:21 smithi138 ceph-mon[32977]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:21.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.559+0000 7f25b57fa700 1 -- 172.21.15.33:0/3979210806 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f25b8102340 con 0x7f25a80692a0 2024-04-25T01:34:21.537 INFO:teuthology.orchestra.run.smithi033.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:34:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.561+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f25a80692a0 msgr2=0x7f25a806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:21.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.561+0000 7f25beef9700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f25a80692a0 0x7f25a806b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f25ac004f40 tx=0x7f25ac00b040 comp rx=0 tx=0).stop 2024-04-25T01:34:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.561+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 msgr2=0x7f25b80702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.561+0000 7f25beef9700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 0x7f25b80702b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f25a400b3b0 tx=0x7f25a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.562+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 shutdown_connections 2024-04-25T01:34:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.562+0000 7f25beef9700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fce60 0x7f25b8071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.562+0000 7f25beef9700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f25a80692a0 0x7f25a806b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.562+0000 7f25beef9700 1 --2- 172.21.15.33:0/3979210806 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fdfd0 0x7f25b80702b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.562+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 >> 172.21.15.33:0/3979210806 conn(0x7f25b80f8540 msgr2=0x7f25b80f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.562+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 shutdown_connections 2024-04-25T01:34:21.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:21.562+0000 7f25beef9700 1 -- 172.21.15.33:0/3979210806 wait complete. 2024-04-25T01:34:22.280 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-25T01:34:22.601 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:34:22.896 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:22 smithi033 ceph-mon[26353]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:34:22.896 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:22.896 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:34:22.896 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:34:22.896 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:34:22.896 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:22 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:22.896 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:22 smithi033 ceph-mon[26353]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:22.896 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:22 smithi033 ceph-mon[26353]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:34:23.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:22 smithi138 ceph-mon[32977]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:34:23.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:22 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:23.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:22 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:34:23.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:22 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:34:23.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:22 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:34:23.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:22 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:23.018 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:22 smithi138 ceph-mon[32977]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:23.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:22 smithi138 ceph-mon[32977]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:34:25.432 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:25 smithi033 ceph-mon[26353]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:25.432 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:25 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:25.432 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:25 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:34:25.432 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:25 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:25.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:25 smithi138 ceph-mon[32977]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:25.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:25 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:25.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:25 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:34:25.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:25 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:26.242 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:26 smithi033 ceph-mon[26353]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (unknown) 2024-04-25T01:34:26.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:26 smithi033 ceph-mon[26353]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-25T01:34:26.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:26 smithi033 ceph-mon[26353]: Upgrade: Need to upgrade myself (mgr.smithi033.asemba) 2024-04-25T01:34:26.243 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:26 smithi033 ceph-mon[26353]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 on smithi138 2024-04-25T01:34:26.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:26 smithi138 ceph-mon[32977]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (unknown) 2024-04-25T01:34:26.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:26 smithi138 ceph-mon[32977]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-25T01:34:26.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:26 smithi138 ceph-mon[32977]: Upgrade: Need to upgrade myself (mgr.smithi033.asemba) 2024-04-25T01:34:26.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:26 smithi138 ceph-mon[32977]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 on smithi138 2024-04-25T01:34:27.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:27 smithi033 ceph-mon[26353]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:34:27.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:27 smithi138 ceph-mon[32977]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:34:28.873 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.021+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/899360346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fcf70 msgr2=0x7fa2e40fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.021+0000 7fa2ea5ae700 1 --2- 172.21.15.33:0/899360346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fcf70 0x7fa2e40fd3e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8004940 tx=0x7fa2d8009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.021+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/899360346 shutdown_connections 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.021+0000 7fa2ea5ae700 1 --2- 172.21.15.33:0/899360346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fcf70 0x7fa2e40fd3e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.021+0000 7fa2ea5ae700 1 --2- 172.21.15.33:0/899360346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e40fbcc0 0x7fa2e40fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.021+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/899360346 >> 172.21.15.33:0/899360346 conn(0x7fa2e40f7420 msgr2=0x7fa2e40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.021+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/899360346 shutdown_connections 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.021+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/899360346 wait complete. 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.022+0000 7fa2ea5ae700 1 Processor -- start 2024-04-25T01:34:28.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.022+0000 7fa2ea5ae700 1 -- start start 2024-04-25T01:34:28.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.022+0000 7fa2ea5ae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fbcc0 0x7fa2e4116750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.022+0000 7fa2ea5ae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e40fcf70 0x7fa2e4116cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.022+0000 7fa2ea5ae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e4115450 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.022+0000 7fa2ea5ae700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e41155c0 con 0x7fa2e40fcf70 2024-04-25T01:34:28.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.022+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fbcc0 0x7fa2e4116750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.023+0000 7fa2e3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fbcc0 0x7fa2e4116750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40562/0 (socket says 172.21.15.33:40562) 2024-04-25T01:34:28.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.023+0000 7fa2e3fff700 1 -- 172.21.15.33:0/1020343853 learned_addr learned my addr 172.21.15.33:0/1020343853 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:34:28.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.023+0000 7fa2e37fe700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e40fcf70 0x7fa2e4116cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.023+0000 7fa2e3fff700 1 -- 172.21.15.33:0/1020343853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e40fcf70 msgr2=0x7fa2e4116cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.023+0000 7fa2e3fff700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e40fcf70 0x7fa2e4116cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.023+0000 7fa2e3fff700 1 -- 172.21.15.33:0/1020343853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d8005040 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.023+0000 7fa2e3fff700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fbcc0 0x7fa2e4116750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400ed00 tx=0x7fa2d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.023+0000 7fa2e0ff9700 1 -- 172.21.15.33:0/1020343853 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d400ee60 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.024+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e4115840 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.024+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e4115d50 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.024+0000 7fa2e0ff9700 1 -- 172.21.15.33:0/1020343853 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2d400f040 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.024+0000 7fa2e0ff9700 1 -- 172.21.15.33:0/1020343853 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2d4009580 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.025+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d00052e0 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.026+0000 7fa2e0ff9700 1 -- 172.21.15.33:0/1020343853 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fa2d40096e0 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.026+0000 7fa2e0ff9700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa2cc0692a0 0x7fa2cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.026+0000 7fa2e0ff9700 1 -- 172.21.15.33:0/1020343853 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa2d4086d80 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.028+0000 7fa2e37fe700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa2cc0692a0 0x7fa2cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.029+0000 7fa2e37fe700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa2cc0692a0 0x7fa2cc06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009b30 tx=0x7fa2d8009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.031+0000 7fa2e0ff9700 1 -- 172.21.15.33:0/1020343853 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa2d404e250 con 0x7fa2e40fbcc0 2024-04-25T01:34:28.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.395+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2d0000bf0 con 0x7fa2cc0692a0 2024-04-25T01:34:28.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.396+0000 7fa2e0ff9700 1 -- 172.21.15.33:0/1020343853 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fa2d0000bf0 con 0x7fa2cc0692a0 2024-04-25T01:34:28.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.398+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa2cc0692a0 msgr2=0x7fa2cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.398+0000 7fa2ea5ae700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa2cc0692a0 0x7fa2cc06b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009b30 tx=0x7fa2d8009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:28.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.398+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fbcc0 msgr2=0x7fa2e4116750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.398+0000 7fa2ea5ae700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fbcc0 0x7fa2e4116750 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa2d400ed00 tx=0x7fa2d400b040 comp rx=0 tx=0).stop 2024-04-25T01:34:28.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.398+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 shutdown_connections 2024-04-25T01:34:28.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.398+0000 7fa2ea5ae700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fa2cc0692a0 0x7fa2cc06b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.398+0000 7fa2ea5ae700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2e40fbcc0 0x7fa2e4116750 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.398+0000 7fa2ea5ae700 1 --2- 172.21.15.33:0/1020343853 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2e40fcf70 0x7fa2e4116cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.398+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 >> 172.21.15.33:0/1020343853 conn(0x7fa2e40f7420 msgr2=0x7fa2e40f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.399+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 shutdown_connections 2024-04-25T01:34:28.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.399+0000 7fa2ea5ae700 1 -- 172.21.15.33:0/1020343853 wait complete. 2024-04-25T01:34:28.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.561+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1366642564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffef0 msgr2=0x7f5c34102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.561+0000 7f5c3c1cd700 1 --2- 172.21.15.33:0/1366642564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffef0 0x7f5c34102310 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5c24004940 tx=0x7f5c24009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:28.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.562+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1366642564 shutdown_connections 2024-04-25T01:34:28.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.562+0000 7f5c3c1cd700 1 --2- 172.21.15.33:0/1366642564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c340ffef0 0x7f5c34102310 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.562+0000 7f5c3c1cd700 1 --2- 172.21.15.33:0/1366642564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c34066b50 0x7f5c340ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.562+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1366642564 >> 172.21.15.33:0/1366642564 conn(0x7f5c340f7420 msgr2=0x7f5c340f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.562+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1366642564 shutdown_connections 2024-04-25T01:34:28.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.562+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1366642564 wait complete. 2024-04-25T01:34:28.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.562+0000 7f5c3c1cd700 1 Processor -- start 2024-04-25T01:34:28.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c3c1cd700 1 -- start start 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c3c1cd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c34066b50 0x7f5c3410dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c3c1cd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3410e120 0x7f5c3410ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c3c1cd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3410e5e0 con 0x7f5c3410e120 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c3c1cd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3410e720 con 0x7f5c34066b50 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c39768700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3410e120 0x7f5c3410ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c39f69700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c34066b50 0x7f5c3410dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c39f69700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c34066b50 0x7f5c3410dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:35104/0 (socket says 172.21.15.33:35104) 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c39f69700 1 -- 172.21.15.33:0/1280864432 learned_addr learned my addr 172.21.15.33:0/1280864432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c39f69700 1 -- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3410e120 msgr2=0x7f5c3410ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.563+0000 7f5c39f69700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3410e120 0x7f5c3410ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.564+0000 7f5c39f69700 1 -- 172.21.15.33:0/1280864432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c24005040 con 0x7f5c34066b50 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.564+0000 7f5c39768700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3410e120 0x7f5c3410ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:34:28.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.564+0000 7f5c39f69700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c34066b50 0x7f5c3410dbe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c30002a50 tx=0x7f5c300046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.564+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1280864432 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c30007cc0 con 0x7f5c34066b50 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.564+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3410f580 con 0x7f5c34066b50 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.564+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3410fb10 con 0x7f5c34066b50 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.564+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1280864432 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c3000f040 con 0x7f5c34066b50 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.564+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1280864432 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c3000db30 con 0x7f5c34066b50 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.565+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c180052e0 con 0x7f5c34066b50 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.566+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1280864432 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5c3000dd30 con 0x7f5c34066b50 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.567+0000 7f5c2a7fc700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5c200692a0 0x7f5c2006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.567+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1280864432 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5c30086df0 con 0x7f5c34066b50 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.567+0000 7f5c39768700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5c200692a0 0x7f5c2006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.567+0000 7f5c39768700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5c200692a0 0x7f5c2006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c24000c00 tx=0x7f5c24009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:28.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.571+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1280864432 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5c30056bb0 con 0x7f5c34066b50 2024-04-25T01:34:28.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.970+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c18000bf0 con 0x7f5c200692a0 2024-04-25T01:34:28.947 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.971+0000 7f5c2a7fc700 1 -- 172.21.15.33:0/1280864432 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f5c18000bf0 con 0x7f5c200692a0 2024-04-25T01:34:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5c200692a0 msgr2=0x7f5c2006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5c200692a0 0x7f5c2006b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5c24000c00 tx=0x7f5c24009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c34066b50 msgr2=0x7f5c3410dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c34066b50 0x7f5c3410dbe0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5c30002a50 tx=0x7f5c300046b0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 shutdown_connections 2024-04-25T01:34:28.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c34066b50 0x7f5c3410dbe0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5c200692a0 0x7f5c2006b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 --2- 172.21.15.33:0/1280864432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c3410e120 0x7f5c3410ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 >> 172.21.15.33:0/1280864432 conn(0x7f5c340f7420 msgr2=0x7f5c340f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 shutdown_connections 2024-04-25T01:34:28.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:28.974+0000 7f5c3c1cd700 1 -- 172.21.15.33:0/1280864432 wait complete. 2024-04-25T01:34:29.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.122+0000 7f5b8eebe700 1 -- 172.21.15.33:0/3659150779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880fcf10 msgr2=0x7f5b880fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.122+0000 7f5b8eebe700 1 --2- 172.21.15.33:0/3659150779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880fcf10 0x7f5b880fd320 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c004940 tx=0x7f5b7c009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:29.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.122+0000 7f5b8eebe700 1 -- 172.21.15.33:0/3659150779 shutdown_connections 2024-04-25T01:34:29.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.122+0000 7f5b8eebe700 1 --2- 172.21.15.33:0/3659150779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fe080 0x7f5b880fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.122+0000 7f5b8eebe700 1 --2- 172.21.15.33:0/3659150779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880fcf10 0x7f5b880fd320 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.122+0000 7f5b8eebe700 1 -- 172.21.15.33:0/3659150779 >> 172.21.15.33:0/3659150779 conn(0x7f5b880f8560 msgr2=0x7f5b880fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:29.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.122+0000 7f5b8eebe700 1 -- 172.21.15.33:0/3659150779 shutdown_connections 2024-04-25T01:34:29.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.122+0000 7f5b8eebe700 1 -- 172.21.15.33:0/3659150779 wait complete. 2024-04-25T01:34:29.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.123+0000 7f5b8eebe700 1 Processor -- start 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.123+0000 7f5b8eebe700 1 -- start start 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.123+0000 7f5b8eebe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fe080 0x7f5b88071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.123+0000 7f5b8eebe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880702b0 0x7f5b88070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b8eebe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b88070c60 con 0x7f5b880702b0 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b8eebe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b88070da0 con 0x7f5b880fe080 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880702b0 0x7f5b88070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880702b0 0x7f5b88070720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40580/0 (socket says 172.21.15.33:40580) 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b87fff700 1 -- 172.21.15.33:0/604858630 learned_addr learned my addr 172.21.15.33:0/604858630 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b8cc5a700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fe080 0x7f5b88071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b8cc5a700 1 -- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880702b0 msgr2=0x7f5b88070720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b8cc5a700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880702b0 0x7f5b88070720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b8cc5a700 1 -- 172.21.15.33:0/604858630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c005040 con 0x7f5b880fe080 2024-04-25T01:34:29.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b87fff700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880702b0 0x7f5b88070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:34:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.124+0000 7f5b8cc5a700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fe080 0x7f5b88071de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c008000 tx=0x7f5b7c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.125+0000 7f5b857fa700 1 -- 172.21.15.33:0/604858630 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b7c00e070 con 0x7f5b880fe080 2024-04-25T01:34:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.125+0000 7f5b857fa700 1 -- 172.21.15.33:0/604858630 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b7c01c400 con 0x7f5b880fe080 2024-04-25T01:34:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.125+0000 7f5b857fa700 1 -- 172.21.15.33:0/604858630 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b7c00b650 con 0x7f5b880fe080 2024-04-25T01:34:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.125+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b88071020 con 0x7f5b880fe080 2024-04-25T01:34:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.125+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b88115da0 con 0x7f5b880fe080 2024-04-25T01:34:29.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.126+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8805f5c0 con 0x7f5b880fe080 2024-04-25T01:34:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.127+0000 7f5b857fa700 1 -- 172.21.15.33:0/604858630 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f5b7c01ca00 con 0x7f5b880fe080 2024-04-25T01:34:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.127+0000 7f5b857fa700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b780692a0 0x7f5b7806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.127+0000 7f5b857fa700 1 -- 172.21.15.33:0/604858630 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5b7c0883f0 con 0x7f5b880fe080 2024-04-25T01:34:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.127+0000 7f5b87fff700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b780692a0 0x7f5b7806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:29.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.128+0000 7f5b87fff700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b780692a0 0x7f5b7806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5b740041d0 tx=0x7f5b7400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:29.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.133+0000 7f5b857fa700 1 -- 172.21.15.33:0/604858630 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5b7c054700 con 0x7f5b880fe080 2024-04-25T01:34:29.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.418+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b88102530 con 0x7f5b780692a0 2024-04-25T01:34:29.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.428+0000 7f5b857fa700 1 -- 172.21.15.33:0/604858630 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f5b88102530 con 0x7f5b780692a0 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (5m) 63s ago 6m 24.8M - 0.25.0 c8568f914cd2 50a86f263578 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (7m) 63s ago 7m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (6m) 69s ago 6m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (5m) 63s ago 6m 82.9M - 9.4.7 954c08fa6188 4533d743079b 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (83s) 63s ago 83s 14.7M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (89s) 63s ago 88s 16.2M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (86s) 69s ago 85s 17.0M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (81s) 69s ago 80s 14.5M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:34:29.405 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:9283 running (8m) 63s ago 8m 481M - 17.2.7-902-gc61472c8 0de407c1b3af 22f7bb21dc39 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283 running (5m) 69s ago 5m 431M - 17.2.7-902-gc61472c8 0de407c1b3af 2440b5c4f74d 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (8m) 63s ago 8m 46.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (5m) 69s ago 5m 38.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (6m) 63s ago 6m 16.0M - 1.5.0 0da6a335fe13 c2f75e5aaf0f 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 69s ago 5m 16.0M - 1.5.0 0da6a335fe13 ce8fbd90510d 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (5m) 63s ago 5m 46.3M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (4m) 63s ago 4m 48.1M 4289M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (4m) 63s ago 4m 43.6M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (3m) 69s ago 3m 46.6M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (3m) 69s ago 3m 42.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (2m) 69s ago 2m 43.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:34:29.406 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (5m) 63s ago 6m 56.5M - 2.43.0 a07b618ecd1d 9109b23faf81 2024-04-25T01:34:29.407 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:29 smithi033 ceph-mon[26353]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:29.407 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:29 smithi033 ceph-mon[26353]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.430+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b780692a0 msgr2=0x7f5b7806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.430+0000 7f5b8eebe700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b780692a0 0x7f5b7806b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5b740041d0 tx=0x7f5b7400b040 comp rx=0 tx=0).stop 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.430+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fe080 msgr2=0x7f5b88071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.430+0000 7f5b8eebe700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fe080 0x7f5b88071de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c008000 tx=0x7f5b7c009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.431+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 shutdown_connections 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.431+0000 7f5b8eebe700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b880fe080 0x7f5b88071de0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.431+0000 7f5b8eebe700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f5b780692a0 0x7f5b7806b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.431+0000 7f5b8eebe700 1 --2- 172.21.15.33:0/604858630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b880702b0 0x7f5b88070720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.431+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 >> 172.21.15.33:0/604858630 conn(0x7f5b880f8560 msgr2=0x7f5b880f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.431+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 shutdown_connections 2024-04-25T01:34:29.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.431+0000 7f5b8eebe700 1 -- 172.21.15.33:0/604858630 wait complete. 2024-04-25T01:34:29.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:29 smithi138 ceph-mon[32977]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:29.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:29 smithi138 ceph-mon[32977]: from='client.14532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:34:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.577+0000 7f0d77898700 1 -- 172.21.15.33:0/802456923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fdfd0 msgr2=0x7f0d700fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.577+0000 7f0d77898700 1 --2- 172.21.15.33:0/802456923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fdfd0 0x7f0d700fe440 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d60004940 tx=0x7f0d60009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.577+0000 7f0d77898700 1 -- 172.21.15.33:0/802456923 shutdown_connections 2024-04-25T01:34:29.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.577+0000 7f0d77898700 1 --2- 172.21.15.33:0/802456923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fdfd0 0x7f0d700fe440 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.577+0000 7f0d77898700 1 --2- 172.21.15.33:0/802456923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d700fce60 0x7f0d700fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.577+0000 7f0d77898700 1 -- 172.21.15.33:0/802456923 >> 172.21.15.33:0/802456923 conn(0x7f0d700f8540 msgr2=0x7f0d700fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.578+0000 7f0d77898700 1 -- 172.21.15.33:0/802456923 shutdown_connections 2024-04-25T01:34:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.578+0000 7f0d77898700 1 -- 172.21.15.33:0/802456923 wait complete. 2024-04-25T01:34:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.578+0000 7f0d77898700 1 Processor -- start 2024-04-25T01:34:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.578+0000 7f0d77898700 1 -- start start 2024-04-25T01:34:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.578+0000 7f0d77898700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fce60 0x7f0d7010d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:29.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d77898700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d700fdfd0 0x7f0d7010df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d77898700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7010e510 con 0x7f0d700fce60 2024-04-25T01:34:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d77898700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7010ed60 con 0x7f0d700fdfd0 2024-04-25T01:34:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d75634700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fce60 0x7f0d7010d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d75634700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fce60 0x7f0d7010d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40606/0 (socket says 172.21.15.33:40606) 2024-04-25T01:34:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d75634700 1 -- 172.21.15.33:0/2208001423 learned_addr learned my addr 172.21.15.33:0/2208001423 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:34:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d74e33700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d700fdfd0 0x7f0d7010df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:29.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d75634700 1 -- 172.21.15.33:0/2208001423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d700fdfd0 msgr2=0x7f0d7010df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d75634700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d700fdfd0 0x7f0d7010df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d75634700 1 -- 172.21.15.33:0/2208001423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d60005040 con 0x7f0d700fce60 2024-04-25T01:34:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.579+0000 7f0d75634700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fce60 0x7f0d7010d9d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c002a50 tx=0x7f0d6c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.580+0000 7f0d65ffb700 1 -- 172.21.15.33:0/2208001423 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d6c007cc0 con 0x7f0d700fce60 2024-04-25T01:34:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.580+0000 7f0d65ffb700 1 -- 172.21.15.33:0/2208001423 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d6c00f040 con 0x7f0d700fce60 2024-04-25T01:34:29.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.580+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7010f040 con 0x7f0d700fce60 2024-04-25T01:34:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.580+0000 7f0d65ffb700 1 -- 172.21.15.33:0/2208001423 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d6c00da30 con 0x7f0d700fce60 2024-04-25T01:34:29.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.580+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7010f580 con 0x7f0d700fce60 2024-04-25T01:34:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.581+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7005f5c0 con 0x7f0d700fce60 2024-04-25T01:34:29.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.582+0000 7f0d65ffb700 1 -- 172.21.15.33:0/2208001423 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f0d6c00db90 con 0x7f0d700fce60 2024-04-25T01:34:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.582+0000 7f0d65ffb700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0d5c0692b0 0x7f0d5c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.583+0000 7f0d65ffb700 1 -- 172.21.15.33:0/2208001423 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0d6c086f30 con 0x7f0d700fce60 2024-04-25T01:34:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.583+0000 7f0d74e33700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0d5c0692b0 0x7f0d5c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:29.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.583+0000 7f0d74e33700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0d5c0692b0 0x7f0d5c06b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009fd0 tx=0x7f0d60009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:29.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.587+0000 7f0d65ffb700 1 -- 172.21.15.33:0/2208001423 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0d6c056c70 con 0x7f0d700fce60 2024-04-25T01:34:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.869+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0d7010f900 con 0x7f0d700fce60 2024-04-25T01:34:29.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.870+0000 7f0d65ffb700 1 -- 172.21.15.33:0/2208001423 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f0d6c056800 con 0x7f0d700fce60 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:34:29.847 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:34:29.848 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:34:29.848 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:34:29.848 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-25T01:34:29.848 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:34:29.848 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:34:29.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.872+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0d5c0692b0 msgr2=0x7f0d5c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.872+0000 7f0d77898700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0d5c0692b0 0x7f0d5c06b760 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0d60009fd0 tx=0x7f0d60009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.872+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fce60 msgr2=0x7f0d7010d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.872+0000 7f0d77898700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fce60 0x7f0d7010d9d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c002a50 tx=0x7f0d6c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.873+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 shutdown_connections 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.873+0000 7f0d77898700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f0d5c0692b0 0x7f0d5c06b760 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.873+0000 7f0d77898700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0d700fce60 0x7f0d7010d9d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.873+0000 7f0d77898700 1 --2- 172.21.15.33:0/2208001423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0d700fdfd0 0x7f0d7010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.873+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 >> 172.21.15.33:0/2208001423 conn(0x7f0d700f8540 msgr2=0x7f0d700f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.873+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 shutdown_connections 2024-04-25T01:34:29.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.873+0000 7f0d77898700 1 -- 172.21.15.33:0/2208001423 wait complete. 2024-04-25T01:34:29.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.982+0000 7f7eb192b700 1 -- 172.21.15.33:0/3785038479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac0fffa0 msgr2=0x7f7eac102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.982+0000 7f7eb192b700 1 --2- 172.21.15.33:0/3785038479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac0fffa0 0x7f7eac102380 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0004940 tx=0x7f7ea0009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:29.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.982+0000 7f7eb192b700 1 -- 172.21.15.33:0/3785038479 shutdown_connections 2024-04-25T01:34:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.982+0000 7f7eb192b700 1 --2- 172.21.15.33:0/3785038479 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac0fffa0 0x7f7eac102380 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.982+0000 7f7eb192b700 1 --2- 172.21.15.33:0/3785038479 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac066200 0x7f7eac0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.982+0000 7f7eb192b700 1 -- 172.21.15.33:0/3785038479 >> 172.21.15.33:0/3785038479 conn(0x7f7eac0f7430 msgr2=0x7f7eac0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.982+0000 7f7eb192b700 1 -- 172.21.15.33:0/3785038479 shutdown_connections 2024-04-25T01:34:29.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.982+0000 7f7eb192b700 1 -- 172.21.15.33:0/3785038479 wait complete. 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.983+0000 7f7eb192b700 1 Processor -- start 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.983+0000 7f7eb192b700 1 -- start start 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.983+0000 7f7eb192b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac066200 0x7f7eac10fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.983+0000 7f7eb192b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac110300 0x7f7eac111070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.983+0000 7f7eb192b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac1107c0 con 0x7f7eac066200 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.983+0000 7f7eb192b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac110930 con 0x7f7eac110300 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7eaaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac066200 0x7f7eac10fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7eaaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac066200 0x7f7eac10fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40622/0 (socket says 172.21.15.33:40622) 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7eaa7fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac110300 0x7f7eac111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7eaaffd700 1 -- 172.21.15.33:0/2530976003 learned_addr learned my addr 172.21.15.33:0/2530976003 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7eaaffd700 1 -- 172.21.15.33:0/2530976003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac110300 msgr2=0x7f7eac111070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:29.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7eaaffd700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac110300 0x7f7eac111070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7eaaffd700 1 -- 172.21.15.33:0/2530976003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0005040 con 0x7f7eac066200 2024-04-25T01:34:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7eaaffd700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac066200 0x7f7eac10fdc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00eb70 tx=0x7f7e9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7e937fe700 1 -- 172.21.15.33:0/2530976003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e9c007cc0 con 0x7f7eac066200 2024-04-25T01:34:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.984+0000 7f7e937fe700 1 -- 172.21.15.33:0/2530976003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e9c00f040 con 0x7f7eac066200 2024-04-25T01:34:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.985+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eac1116f0 con 0x7f7eac066200 2024-04-25T01:34:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.985+0000 7f7e937fe700 1 -- 172.21.15.33:0/2530976003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e9c00cb20 con 0x7f7eac066200 2024-04-25T01:34:29.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.985+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac111c30 con 0x7f7eac066200 2024-04-25T01:34:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.986+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eac0fe6c0 con 0x7f7eac066200 2024-04-25T01:34:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.987+0000 7f7e937fe700 1 -- 172.21.15.33:0/2530976003 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7e9c00cc80 con 0x7f7eac066200 2024-04-25T01:34:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.987+0000 7f7e937fe700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7e940692a0 0x7f7e9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:34:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.987+0000 7f7e937fe700 1 -- 172.21.15.33:0/2530976003 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f7e9c086c40 con 0x7f7eac066200 2024-04-25T01:34:29.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.988+0000 7f7eaa7fc700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7e940692a0 0x7f7e9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:34:29.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.988+0000 7f7eaa7fc700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7e940692a0 0x7f7e9406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009fd0 tx=0x7f7ea0009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:34:29.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:29.992+0000 7f7e937fe700 1 -- 172.21.15.33:0/2530976003 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7e9c04e250 con 0x7f7eac066200 2024-04-25T01:34:30.159 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:30 smithi033 ceph-mon[26353]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:34:30.159 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:30 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2208001423' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:34:30.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.275+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7eac1058a0 con 0x7f7e940692a0 2024-04-25T01:34:30.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.277+0000 7f7e937fe700 1 -- 172.21.15.33:0/2530976003 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f7eac1058a0 con 0x7f7e940692a0 2024-04-25T01:34:30.254 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:34:30.254 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:34:30.254 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:34:30.254 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-25T01:34:30.254 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [], 2024-04-25T01:34:30.254 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "0/2 daemons upgraded", 2024-04-25T01:34:30.255 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 image on host smithi138", 2024-04-25T01:34:30.255 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:34:30.255 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:34:30.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7e940692a0 msgr2=0x7f7e9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:30.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7e940692a0 0x7f7e9406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009fd0 tx=0x7f7ea0009e30 comp rx=0 tx=0).stop 2024-04-25T01:34:30.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac066200 msgr2=0x7f7eac10fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:34:30.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac066200 0x7f7eac10fdc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00eb70 tx=0x7f7e9c00b040 comp rx=0 tx=0).stop 2024-04-25T01:34:30.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 shutdown_connections 2024-04-25T01:34:30.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7e940692a0 0x7f7e9406b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:30.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7eac066200 0x7f7eac10fdc0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:30.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 --2- 172.21.15.33:0/2530976003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7eac110300 0x7f7eac111070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:34:30.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 >> 172.21.15.33:0/2530976003 conn(0x7f7eac0f7430 msgr2=0x7f7eac0fb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:34:30.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.279+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 shutdown_connections 2024-04-25T01:34:30.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:34:30.280+0000 7f7eb192b700 1 -- 172.21.15.33:0/2530976003 wait complete. 2024-04-25T01:34:30.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:30 smithi138 ceph-mon[32977]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:34:30.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:30 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2208001423' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:34:31.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:31 smithi033 ceph-mon[26353]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:34:31.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:31 smithi033 ceph-mon[26353]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:31.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:31 smithi033 ceph-mon[26353]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:34:31.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:34:31.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:31 smithi138 ceph-mon[32977]: from='client.24351 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:34:31.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:31 smithi138 ceph-mon[32977]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:31.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:31 smithi138 ceph-mon[32977]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:34:31.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:31 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:34:33.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:33 smithi033 ceph-mon[26353]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:33.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:33 smithi138 ceph-mon[32977]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:35.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:35 smithi033 ceph-mon[26353]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:35 smithi138 ceph-mon[32977]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:37.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:37 smithi033 ceph-mon[26353]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:34:37.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:37 smithi138 ceph-mon[32977]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:34:39.242 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:39 smithi033 ceph-mon[26353]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:39.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:39 smithi138 ceph-mon[32977]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:41.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:41 smithi033 ceph-mon[26353]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:41.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:41 smithi138 ceph-mon[32977]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:43.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:43 smithi033 ceph-mon[26353]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:43.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:43 smithi138 ceph-mon[32977]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:45.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:45 smithi033 ceph-mon[26353]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:45.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:45 smithi138 ceph-mon[32977]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:46.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:46 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:34:46.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:46 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:34:47.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:47 smithi033 ceph-mon[26353]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:34:47.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:47 smithi138 ceph-mon[32977]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:34:49.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:49 smithi033 ceph-mon[26353]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:49.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:49 smithi138 ceph-mon[32977]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:51.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:51 smithi033 ceph-mon[26353]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:51.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:51 smithi138 ceph-mon[32977]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:34:51.870 INFO:tasks.workunit.client.0.smithi033.stderr:Updating files: 96% (12017/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr: git switch -c 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-25T01:34:52.649 INFO:tasks.workunit.client.0.smithi033.stderr:Or undo this operation with: 2024-04-25T01:34:52.650 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-25T01:34:52.650 INFO:tasks.workunit.client.0.smithi033.stderr: git switch - 2024-04-25T01:34:52.650 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-25T01:34:52.650 INFO:tasks.workunit.client.0.smithi033.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T01:34:52.650 INFO:tasks.workunit.client.0.smithi033.stderr: 2024-04-25T01:34:52.650 INFO:tasks.workunit.client.0.smithi033.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T01:34:52.653 DEBUG:teuthology.orchestra.run.smithi033:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T01:34:52.690 INFO:tasks.workunit.client.0.smithi033.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T01:34:52.693 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:34:52.693 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T01:34:53.335 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T01:34:53.389 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T01:34:53.430 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T01:34:53.432 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:34:53.432 INFO:tasks.workunit.client.0.smithi033.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T01:34:53.472 INFO:tasks.workunit.client.0.smithi033.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T01:34:53.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:53 smithi138 ceph-mon[32977]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:53.522 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T01:34:53.522 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T01:34:53.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:53 smithi033 ceph-mon[26353]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:34:53.579 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-25T01:34:53.580 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-25T01:34:53.580 DEBUG:teuthology.orchestra.run.smithi033:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-25T01:34:53.644 INFO:tasks.workunit.client.0.smithi033.stderr:+ mkdir -p fsstress 2024-04-25T01:34:53.646 INFO:tasks.workunit.client.0.smithi033.stderr:+ pushd fsstress 2024-04-25T01:34:53.647 INFO:tasks.workunit.client.0.smithi033.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-25T01:34:53.648 INFO:tasks.workunit.client.0.smithi033.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-25T01:34:54.224 INFO:tasks.workunit.client.0.smithi033.stderr:+ tar xzf ltp-full.tgz 2024-04-25T01:34:55.267 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:55 smithi138 ceph-mon[32977]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:55.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:55 smithi033 ceph-mon[26353]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:34:57.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:57 smithi033 ceph-mon[26353]: pgmap v233: 65 pgs: 65 active+clean; 9.4 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 764 KiB/s wr, 21 op/s 2024-04-25T01:34:57.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:57 smithi138 ceph-mon[32977]: pgmap v233: 65 pgs: 65 active+clean; 9.4 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 764 KiB/s wr, 21 op/s 2024-04-25T01:34:58.154 INFO:tasks.workunit.client.1.smithi138.stderr:Updating files: 92% (11527/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T01:34:58.472 INFO:tasks.workunit.client.1.smithi138.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T01:34:58.472 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-04-25T01:34:58.472 INFO:tasks.workunit.client.1.smithi138.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T01:34:58.472 INFO:tasks.workunit.client.1.smithi138.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr: git switch -c 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr:Or undo this operation with: 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr: git switch - 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-04-25T01:34:58.473 INFO:tasks.workunit.client.1.smithi138.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T01:34:58.474 INFO:tasks.workunit.client.1.smithi138.stderr: 2024-04-25T01:34:58.474 INFO:tasks.workunit.client.1.smithi138.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T01:34:58.476 DEBUG:teuthology.orchestra.run.smithi138:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-25T01:34:58.576 INFO:tasks.workunit.client.1.smithi138.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T01:34:58.579 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-25T01:34:58.579 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T01:34:59.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:59 smithi033 ceph-mon[26353]: pgmap v234: 65 pgs: 65 active+clean; 9.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 787 KiB/s wr, 27 op/s 2024-04-25T01:34:59.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:59 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:59.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:59 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:34:59.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:59 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:34:59.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:34:59 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:34:59.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:59 smithi138 ceph-mon[32977]: pgmap v234: 65 pgs: 65 active+clean; 9.6 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 787 KiB/s wr, 27 op/s 2024-04-25T01:34:59.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:59 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:34:59.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:59 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:34:59.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:59 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:34:59.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:34:59 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:35:00.027 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T01:35:00.142 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T01:35:00.185 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-25T01:35:00.186 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-25T01:35:00.186 INFO:tasks.workunit.client.1.smithi138.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T01:35:00.229 INFO:tasks.workunit.client.1.smithi138.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-25T01:35:00.233 DEBUG:teuthology.orchestra.run.smithi138:> set -ex 2024-04-25T01:35:00.233 DEBUG:teuthology.orchestra.run.smithi138:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-25T01:35:00.248 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-25T01:35:00.249 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-25T01:35:00.249 DEBUG:teuthology.orchestra.run.smithi138:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-25T01:35:00.315 INFO:tasks.workunit.client.1.smithi138.stderr:+ mkdir -p fsstress 2024-04-25T01:35:00.317 INFO:tasks.workunit.client.1.smithi138.stderr:+ pushd fsstress 2024-04-25T01:35:00.319 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-25T01:35:00.319 INFO:tasks.workunit.client.1.smithi138.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-25T01:35:00.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.413+0000 7f4c39024700 1 -- 172.21.15.33:0/1194734927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c34100210 msgr2=0x7f4c34100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.413+0000 7f4c39024700 1 --2- 172.21.15.33:0/1194734927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c34100210 0x7f4c34100680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f4c28004940 tx=0x7f4c28009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:00.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.413+0000 7f4c39024700 1 -- 172.21.15.33:0/1194734927 shutdown_connections 2024-04-25T01:35:00.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.413+0000 7f4c39024700 1 --2- 172.21.15.33:0/1194734927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c34100210 0x7f4c34100680 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.413+0000 7f4c39024700 1 --2- 172.21.15.33:0/1194734927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c340fef60 0x7f4c340ff370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.413+0000 7f4c39024700 1 -- 172.21.15.33:0/1194734927 >> 172.21.15.33:0/1194734927 conn(0x7f4c340fa730 msgr2=0x7f4c340fcb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:00.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.413+0000 7f4c39024700 1 -- 172.21.15.33:0/1194734927 shutdown_connections 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.413+0000 7f4c39024700 1 -- 172.21.15.33:0/1194734927 wait complete. 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.414+0000 7f4c39024700 1 Processor -- start 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.414+0000 7f4c39024700 1 -- start start 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.414+0000 7f4c39024700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340fef60 0x7f4c3410fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.414+0000 7f4c39024700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34100210 0x7f4c34110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.414+0000 7f4c39024700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34110640 con 0x7f4c340fef60 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.414+0000 7f4c39024700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34110e90 con 0x7f4c34100210 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.415+0000 7f4c32d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340fef60 0x7f4c3410fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.415+0000 7f4c32d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340fef60 0x7f4c3410fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52608/0 (socket says 172.21.15.33:52608) 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.415+0000 7f4c32d9d700 1 -- 172.21.15.33:0/256441060 learned_addr learned my addr 172.21.15.33:0/256441060 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:00.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.415+0000 7f4c3259c700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34100210 0x7f4c34110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.415+0000 7f4c32d9d700 1 -- 172.21.15.33:0/256441060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34100210 msgr2=0x7f4c34110040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.415+0000 7f4c32d9d700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34100210 0x7f4c34110040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.415+0000 7f4c32d9d700 1 -- 172.21.15.33:0/256441060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c28005040 con 0x7f4c340fef60 2024-04-25T01:35:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.415+0000 7f4c32d9d700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340fef60 0x7f4c3410fb00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400eb40 tx=0x7f4c2400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.416+0000 7f4c1b7fe700 1 -- 172.21.15.33:0/256441060 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c24007cc0 con 0x7f4c340fef60 2024-04-25T01:35:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.416+0000 7f4c1b7fe700 1 -- 172.21.15.33:0/256441060 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c2400f040 con 0x7f4c340fef60 2024-04-25T01:35:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.416+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c34111110 con 0x7f4c340fef60 2024-04-25T01:35:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.416+0000 7f4c1b7fe700 1 -- 172.21.15.33:0/256441060 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2400ca10 con 0x7f4c340fef60 2024-04-25T01:35:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.416+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c34111650 con 0x7f4c340fef60 2024-04-25T01:35:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.417+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c340fe360 con 0x7f4c340fef60 2024-04-25T01:35:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.418+0000 7f4c1b7fe700 1 -- 172.21.15.33:0/256441060 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f4c2400cb70 con 0x7f4c340fef60 2024-04-25T01:35:00.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.418+0000 7f4c1b7fe700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4c1c0692a0 0x7f4c1c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.418+0000 7f4c1b7fe700 1 -- 172.21.15.33:0/256441060 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f4c24086ca0 con 0x7f4c340fef60 2024-04-25T01:35:00.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.419+0000 7f4c3259c700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4c1c0692a0 0x7f4c1c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.420+0000 7f4c3259c700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4c1c0692a0 0x7f4c1c06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800b0d0 tx=0x7f4c28009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:00.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.425+0000 7f4c1b7fe700 1 -- 172.21.15.33:0/256441060 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4c2404e250 con 0x7f4c340fef60 2024-04-25T01:35:00.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:00 smithi033 ceph-mon[26353]: Upgrade: Updating mgr.smithi138.mmfhyi 2024-04-25T01:35:00.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:00 smithi033 ceph-mon[26353]: Deploying daemon mgr.smithi138.mmfhyi on smithi138 2024-04-25T01:35:00.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:00 smithi138 ceph-mon[32977]: Upgrade: Updating mgr.smithi138.mmfhyi 2024-04-25T01:35:00.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:00 smithi138 ceph-mon[32977]: Deploying daemon mgr.smithi138.mmfhyi on smithi138 2024-04-25T01:35:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.714+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c34111930 con 0x7f4c1c0692a0 2024-04-25T01:35:00.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.715+0000 7f4c1b7fe700 1 -- 172.21.15.33:0/256441060 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f4c34111930 con 0x7f4c1c0692a0 2024-04-25T01:35:00.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.717+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4c1c0692a0 msgr2=0x7f4c1c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.717+0000 7f4c39024700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4c1c0692a0 0x7f4c1c06b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800b0d0 tx=0x7f4c28009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:00.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.717+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340fef60 msgr2=0x7f4c3410fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.717+0000 7f4c39024700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340fef60 0x7f4c3410fb00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400eb40 tx=0x7f4c2400b040 comp rx=0 tx=0).stop 2024-04-25T01:35:00.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.717+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 shutdown_connections 2024-04-25T01:35:00.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.717+0000 7f4c39024700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f4c1c0692a0 0x7f4c1c06b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.717+0000 7f4c39024700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c340fef60 0x7f4c3410fb00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.717+0000 7f4c39024700 1 --2- 172.21.15.33:0/256441060 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c34100210 0x7f4c34110040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.717+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 >> 172.21.15.33:0/256441060 conn(0x7f4c340fa730 msgr2=0x7f4c340fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:00.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.718+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 shutdown_connections 2024-04-25T01:35:00.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.718+0000 7f4c39024700 1 -- 172.21.15.33:0/256441060 wait complete. 2024-04-25T01:35:00.706 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.862+0000 7fc98f2a2700 1 -- 172.21.15.33:0/3792494698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9880ffee0 msgr2=0x7fc988102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.862+0000 7fc98f2a2700 1 --2- 172.21.15.33:0/3792494698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9880ffee0 0x7fc988102300 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc978004940 tx=0x7fc978009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.862+0000 7fc98f2a2700 1 -- 172.21.15.33:0/3792494698 shutdown_connections 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.862+0000 7fc98f2a2700 1 --2- 172.21.15.33:0/3792494698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9880ffee0 0x7fc988102300 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.862+0000 7fc98f2a2700 1 --2- 172.21.15.33:0/3792494698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9880662c0 0x7fc9880ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.862+0000 7fc98f2a2700 1 -- 172.21.15.33:0/3792494698 >> 172.21.15.33:0/3792494698 conn(0x7fc9880f7430 msgr2=0x7fc9880f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.863+0000 7fc98f2a2700 1 -- 172.21.15.33:0/3792494698 shutdown_connections 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.863+0000 7fc98f2a2700 1 -- 172.21.15.33:0/3792494698 wait complete. 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.863+0000 7fc98f2a2700 1 Processor -- start 2024-04-25T01:35:00.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.863+0000 7fc98f2a2700 1 -- start start 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.863+0000 7fc98f2a2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9880662c0 0x7fc98810da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98f2a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9880ffee0 0x7fc98810dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98f2a2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98810e5a0 con 0x7fc9880ffee0 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98f2a2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98810ed50 con 0x7fc9880662c0 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98d03e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9880662c0 0x7fc98810da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98d03e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9880662c0 0x7fc98810da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:48336/0 (socket says 172.21.15.33:48336) 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98d03e700 1 -- 172.21.15.33:0/4114595814 learned_addr learned my addr 172.21.15.33:0/4114595814 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98c83d700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9880ffee0 0x7fc98810dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98d03e700 1 -- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9880ffee0 msgr2=0x7fc98810dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98d03e700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9880ffee0 0x7fc98810dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98d03e700 1 -- 172.21.15.33:0/4114595814 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc978005040 con 0x7fc9880662c0 2024-04-25T01:35:00.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98c83d700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9880ffee0 0x7fc98810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.864+0000 7fc98d03e700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9880662c0 0x7fc98810da60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc98400eb40 tx=0x7fc98400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.865+0000 7fc97dffb700 1 -- 172.21.15.33:0/4114595814 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc984007cc0 con 0x7fc9880662c0 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.865+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98810efd0 con 0x7fc9880662c0 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.865+0000 7fc97dffb700 1 -- 172.21.15.33:0/4114595814 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc98400f040 con 0x7fc9880662c0 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.865+0000 7fc97dffb700 1 -- 172.21.15.33:0/4114595814 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc98400cad0 con 0x7fc9880662c0 2024-04-25T01:35:00.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.865+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98810f510 con 0x7fc9880662c0 2024-04-25T01:35:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.866+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98805f5c0 con 0x7fc9880662c0 2024-04-25T01:35:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.866+0000 7fc97dffb700 1 -- 172.21.15.33:0/4114595814 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7fc98400cc30 con 0x7fc9880662c0 2024-04-25T01:35:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.867+0000 7fc97dffb700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc9740692a0 0x7fc97406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.867+0000 7fc97dffb700 1 -- 172.21.15.33:0/4114595814 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fc984086c30 con 0x7fc9880662c0 2024-04-25T01:35:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.867+0000 7fc98c83d700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc9740692a0 0x7fc97406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:00.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.867+0000 7fc98c83d700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc9740692a0 0x7fc97406b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc97800b0d0 tx=0x7fc978009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:00.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:00.875+0000 7fc97dffb700 1 -- 172.21.15.33:0/4114595814 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc9840569f0 con 0x7fc9880662c0 2024-04-25T01:35:01.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.180+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc98810f890 con 0x7fc9740692a0 2024-04-25T01:35:01.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.181+0000 7fc97dffb700 1 -- 172.21.15.33:0/4114595814 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fc98810f890 con 0x7fc9740692a0 2024-04-25T01:35:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.184+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc9740692a0 msgr2=0x7fc97406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.184+0000 7fc98f2a2700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc9740692a0 0x7fc97406b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc97800b0d0 tx=0x7fc978009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.184+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9880662c0 msgr2=0x7fc98810da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.184+0000 7fc98f2a2700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9880662c0 0x7fc98810da60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc98400eb40 tx=0x7fc98400b040 comp rx=0 tx=0).stop 2024-04-25T01:35:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.184+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 shutdown_connections 2024-04-25T01:35:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.184+0000 7fc98f2a2700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9880662c0 0x7fc98810da60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.184+0000 7fc98f2a2700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7fc9740692a0 0x7fc97406b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.184+0000 7fc98f2a2700 1 --2- 172.21.15.33:0/4114595814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9880ffee0 0x7fc98810dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.184+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 >> 172.21.15.33:0/4114595814 conn(0x7fc9880f7430 msgr2=0x7fc9880f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.185+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 shutdown_connections 2024-04-25T01:35:01.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.185+0000 7fc98f2a2700 1 -- 172.21.15.33:0/4114595814 wait complete. 2024-04-25T01:35:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.331+0000 7ff39dca4700 1 -- 172.21.15.33:0/565724190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fe090 msgr2=0x7ff3980fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.331+0000 7ff39dca4700 1 --2- 172.21.15.33:0/565724190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fe090 0x7ff3980fe500 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff380004940 tx=0x7ff380009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.331+0000 7ff39dca4700 1 -- 172.21.15.33:0/565724190 shutdown_connections 2024-04-25T01:35:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.331+0000 7ff39dca4700 1 --2- 172.21.15.33:0/565724190 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fe090 0x7ff3980fe500 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.331+0000 7ff39dca4700 1 --2- 172.21.15.33:0/565724190 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3980fce90 0x7ff3980fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.331+0000 7ff39dca4700 1 -- 172.21.15.33:0/565724190 >> 172.21.15.33:0/565724190 conn(0x7ff3980f85c0 msgr2=0x7ff3980fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.331+0000 7ff39dca4700 1 -- 172.21.15.33:0/565724190 shutdown_connections 2024-04-25T01:35:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.331+0000 7ff39dca4700 1 -- 172.21.15.33:0/565724190 wait complete. 2024-04-25T01:35:01.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.331+0000 7ff39dca4700 1 Processor -- start 2024-04-25T01:35:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.332+0000 7ff39dca4700 1 -- start start 2024-04-25T01:35:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.332+0000 7ff39dca4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fce90 0x7ff39810d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.332+0000 7ff39dca4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3980fe090 0x7ff39810def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.332+0000 7ff39dca4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39810e4f0 con 0x7ff3980fce90 2024-04-25T01:35:01.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.332+0000 7ff39dca4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39810ed40 con 0x7ff3980fe090 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.332+0000 7ff3977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fce90 0x7ff39810d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.332+0000 7ff3977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fce90 0x7ff39810d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52636/0 (socket says 172.21.15.33:52636) 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.332+0000 7ff3977fe700 1 -- 172.21.15.33:0/220909228 learned_addr learned my addr 172.21.15.33:0/220909228 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.332+0000 7ff396ffd700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3980fe090 0x7ff39810def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.333+0000 7ff3977fe700 1 -- 172.21.15.33:0/220909228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3980fe090 msgr2=0x7ff39810def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.333+0000 7ff3977fe700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3980fe090 0x7ff39810def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.333+0000 7ff3977fe700 1 -- 172.21.15.33:0/220909228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff380005040 con 0x7ff3980fce90 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.333+0000 7ff3977fe700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fce90 0x7ff39810d9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff38800eb70 tx=0x7ff38800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.333+0000 7ff38ffff700 1 -- 172.21.15.33:0/220909228 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff388007cc0 con 0x7ff3980fce90 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.334+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39810efc0 con 0x7ff3980fce90 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.334+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39810f500 con 0x7ff3980fce90 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.334+0000 7ff38ffff700 1 -- 172.21.15.33:0/220909228 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff38800f040 con 0x7ff3980fce90 2024-04-25T01:35:01.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.334+0000 7ff38ffff700 1 -- 172.21.15.33:0/220909228 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff38800cbd0 con 0x7ff3980fce90 2024-04-25T01:35:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.335+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3780052e0 con 0x7ff3980fce90 2024-04-25T01:35:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.336+0000 7ff38ffff700 1 -- 172.21.15.33:0/220909228 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7ff388009830 con 0x7ff3980fce90 2024-04-25T01:35:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.336+0000 7ff38ffff700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff38406d780 0x7ff38406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.336+0000 7ff38ffff700 1 -- 172.21.15.33:0/220909228 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff388086bf0 con 0x7ff3980fce90 2024-04-25T01:35:01.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.336+0000 7ff396ffd700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff38406d780 0x7ff38406fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.337+0000 7ff396ffd700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff38406d780 0x7ff38406fc30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff380009b30 tx=0x7ff380009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.343+0000 7ff38ffff700 1 -- 172.21.15.33:0/220909228 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff38804e250 con 0x7ff3980fce90 2024-04-25T01:35:01.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:01 smithi033 ceph-mon[26353]: pgmap v235: 65 pgs: 65 active+clean; 17 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-04-25T01:35:01.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:01 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:01.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:01 smithi138 ceph-mon[32977]: pgmap v235: 65 pgs: 65 active+clean; 17 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 1.4 MiB/s wr, 82 op/s 2024-04-25T01:35:01.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:01 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:01.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.666+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff378000bf0 con 0x7ff38406d780 2024-04-25T01:35:01.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.677+0000 7ff38ffff700 1 -- 172.21.15.33:0/220909228 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7ff378000bf0 con 0x7ff38406d780 2024-04-25T01:35:01.655 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:35:01.655 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (6m) 95s ago 7m 24.8M - 0.25.0 c8568f914cd2 50a86f263578 2024-04-25T01:35:01.655 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (7m) 95s ago 7m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:35:01.655 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (6m) 101s ago 6m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:35:01.655 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (5m) 95s ago 7m 82.9M - 9.4.7 954c08fa6188 4533d743079b 2024-04-25T01:35:01.655 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (115s) 95s ago 115s 14.7M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:35:01.655 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (2m) 95s ago 2m 16.2M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:35:01.655 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (118s) 101s ago 118s 17.0M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:35:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (113s) 101s ago 113s 14.5M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:35:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:9283 running (8m) 95s ago 8m 481M - 17.2.7-902-gc61472c8 0de407c1b3af 22f7bb21dc39 2024-04-25T01:35:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283 running (6m) 101s ago 6m 431M - 17.2.7-902-gc61472c8 0de407c1b3af 2440b5c4f74d 2024-04-25T01:35:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (8m) 95s ago 8m 46.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:35:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (6m) 101s ago 6m 38.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:35:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (7m) 95s ago 7m 16.0M - 1.5.0 0da6a335fe13 c2f75e5aaf0f 2024-04-25T01:35:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (6m) 101s ago 6m 16.0M - 1.5.0 0da6a335fe13 ce8fbd90510d 2024-04-25T01:35:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (5m) 95s ago 5m 46.3M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:35:01.657 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (5m) 95s ago 5m 48.1M 4289M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:35:01.657 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (4m) 95s ago 4m 43.6M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:35:01.657 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (4m) 101s ago 4m 46.6M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:35:01.657 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (3m) 101s ago 3m 42.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:35:01.657 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (3m) 101s ago 3m 43.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:35:01.657 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (5m) 95s ago 6m 56.5M - 2.43.0 a07b618ecd1d 9109b23faf81 2024-04-25T01:35:01.658 INFO:tasks.workunit.client.1.smithi138.stderr:+ tar xzf ltp-full.tgz 2024-04-25T01:35:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff38406d780 msgr2=0x7ff38406fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff38406d780 0x7ff38406fc30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff380009b30 tx=0x7ff380009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fce90 msgr2=0x7ff39810d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fce90 0x7ff39810d9b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff38800eb70 tx=0x7ff38800b040 comp rx=0 tx=0).stop 2024-04-25T01:35:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 shutdown_connections 2024-04-25T01:35:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7ff38406d780 0x7ff38406fc30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff3980fce90 0x7ff39810d9b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 --2- 172.21.15.33:0/220909228 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff3980fe090 0x7ff39810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 >> 172.21.15.33:0/220909228 conn(0x7ff3980f85c0 msgr2=0x7ff3980f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 shutdown_connections 2024-04-25T01:35:01.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.680+0000 7ff39dca4700 1 -- 172.21.15.33:0/220909228 wait complete. 2024-04-25T01:35:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.856+0000 7eff2a622700 1 -- 172.21.15.33:0/703430902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 msgr2=0x7eff24104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.856+0000 7eff2a622700 1 --2- 172.21.15.33:0/703430902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 0x7eff24104720 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7eff18004940 tx=0x7eff18009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.856+0000 7eff2a622700 1 -- 172.21.15.33:0/703430902 shutdown_connections 2024-04-25T01:35:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.856+0000 7eff2a622700 1 --2- 172.21.15.33:0/703430902 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 0x7eff24104720 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.856+0000 7eff2a622700 1 --2- 172.21.15.33:0/703430902 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff24066260 0x7eff24101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.856+0000 7eff2a622700 1 -- 172.21.15.33:0/703430902 >> 172.21.15.33:0/703430902 conn(0x7eff240f7430 msgr2=0x7eff240f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.856+0000 7eff2a622700 1 -- 172.21.15.33:0/703430902 shutdown_connections 2024-04-25T01:35:01.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.856+0000 7eff2a622700 1 -- 172.21.15.33:0/703430902 wait complete. 2024-04-25T01:35:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff2a622700 1 Processor -- start 2024-04-25T01:35:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff2a622700 1 -- start start 2024-04-25T01:35:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff2a622700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff24066260 0x7eff2410fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff2a622700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 0x7eff24110130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff2a622700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff24110730 con 0x7eff24102340 2024-04-25T01:35:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff2a622700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff24110f30 con 0x7eff24066260 2024-04-25T01:35:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 0x7eff24110130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 0x7eff24110130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52664/0 (socket says 172.21.15.33:52664) 2024-04-25T01:35:01.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff23fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff24066260 0x7eff2410fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.857+0000 7eff237fe700 1 -- 172.21.15.33:0/1796245944 learned_addr learned my addr 172.21.15.33:0/1796245944 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:01.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.858+0000 7eff237fe700 1 -- 172.21.15.33:0/1796245944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff24066260 msgr2=0x7eff2410fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:01.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.858+0000 7eff237fe700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff24066260 0x7eff2410fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:01.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.858+0000 7eff237fe700 1 -- 172.21.15.33:0/1796245944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff18005040 con 0x7eff24102340 2024-04-25T01:35:01.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.859+0000 7eff23fff700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff24066260 0x7eff2410fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:35:01.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.859+0000 7eff237fe700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 0x7eff24110130 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7eff1800b0d0 tx=0x7eff18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.859+0000 7eff20ff9700 1 -- 172.21.15.33:0/1796245944 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff1800e030 con 0x7eff24102340 2024-04-25T01:35:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.859+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff241111b0 con 0x7eff24102340 2024-04-25T01:35:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.859+0000 7eff20ff9700 1 -- 172.21.15.33:0/1796245944 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff1801c400 con 0x7eff24102340 2024-04-25T01:35:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.859+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff241116f0 con 0x7eff24102340 2024-04-25T01:35:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.859+0000 7eff20ff9700 1 -- 172.21.15.33:0/1796245944 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff1800b630 con 0x7eff24102340 2024-04-25T01:35:01.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.860+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff100052e0 con 0x7eff24102340 2024-04-25T01:35:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.861+0000 7eff20ff9700 1 -- 172.21.15.33:0/1796245944 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7eff18013af0 con 0x7eff24102340 2024-04-25T01:35:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.861+0000 7eff20ff9700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7eff0c0692a0 0x7eff0c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.861+0000 7eff23fff700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7eff0c0692a0 0x7eff0c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:01.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.862+0000 7eff20ff9700 1 -- 172.21.15.33:0/1796245944 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7eff18088af0 con 0x7eff24102340 2024-04-25T01:35:01.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.862+0000 7eff23fff700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7eff0c0692a0 0x7eff0c06b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7eff14009b80 tx=0x7eff1400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:01.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:01.870+0000 7eff20ff9700 1 -- 172.21.15.33:0/1796245944 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7eff18058890 con 0x7eff24102340 2024-04-25T01:35:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.261+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7eff10005da0 con 0x7eff24102340 2024-04-25T01:35:02.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.262+0000 7eff20ff9700 1 -- 172.21.15.33:0/1796245944 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7eff18058420 con 0x7eff24102340 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:35:02.239 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:35:02.240 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:35:02.240 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:35:02.240 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-25T01:35:02.240 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:35:02.240 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7eff0c0692a0 msgr2=0x7eff0c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7eff0c0692a0 0x7eff0c06b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7eff14009b80 tx=0x7eff1400b040 comp rx=0 tx=0).stop 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 msgr2=0x7eff24110130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 0x7eff24110130 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7eff1800b0d0 tx=0x7eff18009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 shutdown_connections 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff24066260 0x7eff2410fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7eff0c0692a0 0x7eff0c06b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 --2- 172.21.15.33:0/1796245944 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff24102340 0x7eff24110130 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 >> 172.21.15.33:0/1796245944 conn(0x7eff240f7430 msgr2=0x7eff240f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:02.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 shutdown_connections 2024-04-25T01:35:02.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.265+0000 7eff2a622700 1 -- 172.21.15.33:0/1796245944 wait complete. 2024-04-25T01:35:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.421+0000 7f7fc04d0700 1 -- 172.21.15.33:0/2732341172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 msgr2=0x7f7fb81023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.421+0000 7f7fc04d0700 1 --2- 172.21.15.33:0/2732341172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 0x7f7fb81023b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8004940 tx=0x7f7fa8009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.421+0000 7f7fc04d0700 1 -- 172.21.15.33:0/2732341172 shutdown_connections 2024-04-25T01:35:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.421+0000 7f7fc04d0700 1 --2- 172.21.15.33:0/2732341172 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 0x7f7fb81023b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.421+0000 7f7fc04d0700 1 --2- 172.21.15.33:0/2732341172 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fb8066af0 0x7f7fb80ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.421+0000 7f7fc04d0700 1 -- 172.21.15.33:0/2732341172 >> 172.21.15.33:0/2732341172 conn(0x7f7fb80f74c0 msgr2=0x7f7fb80f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.421+0000 7f7fc04d0700 1 -- 172.21.15.33:0/2732341172 shutdown_connections 2024-04-25T01:35:02.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.421+0000 7f7fc04d0700 1 -- 172.21.15.33:0/2732341172 wait complete. 2024-04-25T01:35:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fc04d0700 1 Processor -- start 2024-04-25T01:35:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fc04d0700 1 -- start start 2024-04-25T01:35:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fc04d0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fb8066af0 0x7f7fb810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fc04d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 0x7f7fb810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fc04d0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb810e550 con 0x7f7fb80fffd0 2024-04-25T01:35:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fc04d0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb810ed50 con 0x7f7fb8066af0 2024-04-25T01:35:02.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fbda6b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 0x7f7fb810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fbe26c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fb8066af0 0x7f7fb810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fbda6b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 0x7f7fb810df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52672/0 (socket says 172.21.15.33:52672) 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.422+0000 7f7fbda6b700 1 -- 172.21.15.33:0/80187453 learned_addr learned my addr 172.21.15.33:0/80187453 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.423+0000 7f7fbda6b700 1 -- 172.21.15.33:0/80187453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fb8066af0 msgr2=0x7f7fb810da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.423+0000 7f7fbda6b700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fb8066af0 0x7f7fb810da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.423+0000 7f7fbda6b700 1 -- 172.21.15.33:0/80187453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa8005040 con 0x7f7fb80fffd0 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.423+0000 7f7fbe26c700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fb8066af0 0x7f7fb810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.423+0000 7f7fbda6b700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 0x7f7fb810df50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8013870 tx=0x7f7fa8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.423+0000 7f7faeffd700 1 -- 172.21.15.33:0/80187453 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa800e070 con 0x7f7fb80fffd0 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.424+0000 7f7faeffd700 1 -- 172.21.15.33:0/80187453 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fa8013de0 con 0x7f7fb80fffd0 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.424+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb810efd0 con 0x7f7fb80fffd0 2024-04-25T01:35:02.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.424+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb810f4b0 con 0x7f7fb80fffd0 2024-04-25T01:35:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.424+0000 7f7faeffd700 1 -- 172.21.15.33:0/80187453 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa801ce70 con 0x7f7fb80fffd0 2024-04-25T01:35:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.425+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c0052e0 con 0x7f7fb80fffd0 2024-04-25T01:35:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.426+0000 7f7faeffd700 1 -- 172.21.15.33:0/80187453 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 19) v1 ==== 86629+0+0 (secure 0 0 0) 0x7f7fa8013910 con 0x7f7fb80fffd0 2024-04-25T01:35:02.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.426+0000 7f7faeffd700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7fa4069360 0x7f7fa406b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.426+0000 7f7faeffd700 1 -- 172.21.15.33:0/80187453 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f7fa80886a0 con 0x7f7fb80fffd0 2024-04-25T01:35:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.426+0000 7f7fbe26c700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7fa4069360 0x7f7fa406b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:02.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.427+0000 7f7fbe26c700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7fa4069360 0x7f7fa406b810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009b80 tx=0x7f7fb400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:02.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.433+0000 7f7faeffd700 1 -- 172.21.15.33:0/80187453 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7fa8058500 con 0x7f7fb80fffd0 2024-04-25T01:35:02.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:02 smithi033 ceph-mon[26353]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:02.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:02 smithi033 ceph-mon[26353]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:02.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:02 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:35:02.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:02 smithi138 ceph-mon[32977]: from='client.14552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:02.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:02 smithi138 ceph-mon[32977]: from='client.24367 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:02.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:02 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:35:02.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.795+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f9c000bf0 con 0x7f7fa4069360 2024-04-25T01:35:02.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.796+0000 7f7faeffd700 1 -- 172.21.15.33:0/80187453 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f7f9c000bf0 con 0x7f7fa4069360 2024-04-25T01:35:02.773 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:35:02.773 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:35:02.773 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:35:02.773 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-25T01:35:02.773 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [], 2024-04-25T01:35:02.774 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "0/2 daemons upgraded", 2024-04-25T01:35:02.774 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mgr daemons", 2024-04-25T01:35:02.774 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:35:02.774 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:35:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.799+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7fa4069360 msgr2=0x7f7fa406b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.799+0000 7f7fc04d0700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7fa4069360 0x7f7fa406b810 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009b80 tx=0x7f7fb400b040 comp rx=0 tx=0).stop 2024-04-25T01:35:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.799+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 msgr2=0x7f7fb810df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.799+0000 7f7fc04d0700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 0x7f7fb810df50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8013870 tx=0x7f7fa8009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.799+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 shutdown_connections 2024-04-25T01:35:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.800+0000 7f7fc04d0700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fb8066af0 0x7f7fb810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.800+0000 7f7fc04d0700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f7fa4069360 0x7f7fa406b810 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.800+0000 7f7fc04d0700 1 --2- 172.21.15.33:0/80187453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fb80fffd0 0x7f7fb810df50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.800+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 >> 172.21.15.33:0/80187453 conn(0x7f7fb80f74c0 msgr2=0x7f7fb80f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.800+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 shutdown_connections 2024-04-25T01:35:02.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:02.800+0000 7f7fc04d0700 1 -- 172.21.15.33:0/80187453 wait complete. 2024-04-25T01:35:03.485 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:03 smithi033 ceph-mon[26353]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:03.485 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:03 smithi033 ceph-mon[26353]: pgmap v236: 65 pgs: 65 active+clean; 20 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-04-25T01:35:03.485 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:03 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1796245944' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:35:03.498 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:03 smithi138 ceph-mon[32977]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:03.498 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:03 smithi138 ceph-mon[32977]: pgmap v236: 65 pgs: 65 active+clean; 20 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 250 KiB/s rd, 1.7 MiB/s wr, 152 op/s 2024-04-25T01:35:03.499 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:03 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1796245944' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:35:04.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:04 smithi033 ceph-mon[26353]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:04.511 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:04 smithi138 ceph-mon[32977]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:05.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:05 smithi033 ceph-mon[26353]: pgmap v237: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 603 KiB/s rd, 2.7 MiB/s wr, 196 op/s 2024-04-25T01:35:05.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:05 smithi138 ceph-mon[32977]: pgmap v237: 65 pgs: 65 active+clean; 32 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 603 KiB/s rd, 2.7 MiB/s wr, 196 op/s 2024-04-25T01:35:06.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:06 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:06.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:06 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:06.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:06 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:06.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:06 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:06.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:06 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:06.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:06 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:07.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:07 smithi138 ceph-mon[32977]: pgmap v238: 65 pgs: 65 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 697 KiB/s rd, 3.7 MiB/s wr, 297 op/s 2024-04-25T01:35:07.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:07 smithi033 ceph-mon[26353]: pgmap v238: 65 pgs: 65 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 697 KiB/s rd, 3.7 MiB/s wr, 297 op/s 2024-04-25T01:35:09.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:09 smithi138 ceph-mon[32977]: pgmap v239: 65 pgs: 65 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 696 KiB/s rd, 3.0 MiB/s wr, 298 op/s 2024-04-25T01:35:09.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:09 smithi033 ceph-mon[26353]: pgmap v239: 65 pgs: 65 active+clean; 43 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 696 KiB/s rd, 3.0 MiB/s wr, 298 op/s 2024-04-25T01:35:11.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:11 smithi138 ceph-mon[32977]: pgmap v240: 65 pgs: 65 active+clean; 57 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 361 op/s 2024-04-25T01:35:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:11 smithi033 ceph-mon[26353]: pgmap v240: 65 pgs: 65 active+clean; 57 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 361 op/s 2024-04-25T01:35:13.713 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:13 smithi138 ceph-mon[32977]: pgmap v241: 65 pgs: 65 active+clean; 64 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 383 op/s 2024-04-25T01:35:13.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:13 smithi033 ceph-mon[26353]: pgmap v241: 65 pgs: 65 active+clean; 64 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 383 op/s 2024-04-25T01:35:14.629 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:14 smithi138 ceph-mon[32977]: Standby manager daemon smithi138.mmfhyi restarted 2024-04-25T01:35:14.629 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:14 smithi138 ceph-mon[32977]: Standby manager daemon smithi138.mmfhyi started 2024-04-25T01:35:14.630 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:14 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/crt"}]: dispatch 2024-04-25T01:35:14.630 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:14 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:35:14.630 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:14 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/key"}]: dispatch 2024-04-25T01:35:14.630 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:14 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:35:14.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:14 smithi033 ceph-mon[26353]: Standby manager daemon smithi138.mmfhyi restarted 2024-04-25T01:35:14.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:14 smithi033 ceph-mon[26353]: Standby manager daemon smithi138.mmfhyi started 2024-04-25T01:35:14.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:14 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/crt"}]: dispatch 2024-04-25T01:35:14.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:14 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:35:14.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:14 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/key"}]: dispatch 2024-04-25T01:35:14.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:14 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:35:15.679 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:15 smithi138 ceph-mon[32977]: pgmap v242: 65 pgs: 65 active+clean; 70 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 358 op/s 2024-04-25T01:35:15.679 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:15 smithi138 ceph-mon[32977]: mgrmap e20: smithi033.asemba(active, since 6m), standbys: smithi138.mmfhyi 2024-04-25T01:35:15.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:15 smithi033 ceph-mon[26353]: pgmap v242: 65 pgs: 65 active+clean; 70 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 4.4 MiB/s wr, 358 op/s 2024-04-25T01:35:15.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:15 smithi033 ceph-mon[26353]: mgrmap e20: smithi033.asemba(active, since 6m), standbys: smithi138.mmfhyi 2024-04-25T01:35:16.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:16 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:16.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:16 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:17.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:17 smithi033 ceph-mon[26353]: pgmap v243: 65 pgs: 65 active+clean; 83 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 464 op/s 2024-04-25T01:35:17.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:17 smithi138 ceph-mon[32977]: pgmap v243: 65 pgs: 65 active+clean; 83 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.5 MiB/s wr, 464 op/s 2024-04-25T01:35:19.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:19 smithi033 ceph-mon[26353]: pgmap v244: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-04-25T01:35:19.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:19 smithi138 ceph-mon[32977]: pgmap v244: 65 pgs: 65 active+clean; 85 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 3.8 MiB/s wr, 398 op/s 2024-04-25T01:35:21.644 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:21 smithi138 ceph-mon[32977]: pgmap v245: 65 pgs: 65 active+clean; 96 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 487 op/s 2024-04-25T01:35:21.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:21 smithi033 ceph-mon[26353]: pgmap v245: 65 pgs: 65 active+clean; 96 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 487 op/s 2024-04-25T01:35:23.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:23 smithi033 ceph-mon[26353]: pgmap v246: 65 pgs: 65 active+clean; 110 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 546 op/s 2024-04-25T01:35:23.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:23 smithi138 ceph-mon[32977]: pgmap v246: 65 pgs: 65 active+clean; 110 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 546 op/s 2024-04-25T01:35:24.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:24 smithi033 ceph-mon[26353]: pgmap v247: 65 pgs: 65 active+clean; 115 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 522 op/s 2024-04-25T01:35:24.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:24 smithi138 ceph-mon[32977]: pgmap v247: 65 pgs: 65 active+clean; 115 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 522 op/s 2024-04-25T01:35:27.444 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:27 smithi033 ceph-mon[26353]: pgmap v248: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 607 op/s 2024-04-25T01:35:27.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:27 smithi138 ceph-mon[32977]: pgmap v248: 65 pgs: 65 active+clean; 129 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 607 op/s 2024-04-25T01:35:28.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:28.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:28.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:28.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:28 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:28.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:28.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:28.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:28.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:28 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:29.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:29 smithi138 ceph-mon[32977]: pgmap v249: 65 pgs: 65 active+clean; 131 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-04-25T01:35:29.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:29 smithi033 ceph-mon[26353]: pgmap v249: 65 pgs: 65 active+clean; 131 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 485 op/s 2024-04-25T01:35:31.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:31 smithi138 ceph-mon[32977]: pgmap v250: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 547 op/s 2024-04-25T01:35:31.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:31 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:31.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:31 smithi033 ceph-mon[26353]: pgmap v250: 65 pgs: 65 active+clean; 144 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 547 op/s 2024-04-25T01:35:31.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:31 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.960+0000 7f6ae0060700 1 -- 172.21.15.33:0/1433731144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ad80fdfc0 msgr2=0x7f6ad80fe430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:32.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.960+0000 7f6ae0060700 1 --2- 172.21.15.33:0/1433731144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ad80fdfc0 0x7f6ad80fe430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8004940 tx=0x7f6ac8009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.960+0000 7f6ae0060700 1 -- 172.21.15.33:0/1433731144 shutdown_connections 2024-04-25T01:35:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.960+0000 7f6ae0060700 1 --2- 172.21.15.33:0/1433731144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ad80fdfc0 0x7f6ad80fe430 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.960+0000 7f6ae0060700 1 --2- 172.21.15.33:0/1433731144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad80fce50 0x7f6ad80fd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.960+0000 7f6ae0060700 1 -- 172.21.15.33:0/1433731144 >> 172.21.15.33:0/1433731144 conn(0x7f6ad80f8510 msgr2=0x7f6ad80fa940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.960+0000 7f6ae0060700 1 -- 172.21.15.33:0/1433731144 shutdown_connections 2024-04-25T01:35:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.960+0000 7f6ae0060700 1 -- 172.21.15.33:0/1433731144 wait complete. 2024-04-25T01:35:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.961+0000 7f6ae0060700 1 Processor -- start 2024-04-25T01:35:32.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.961+0000 7f6ae0060700 1 -- start start 2024-04-25T01:35:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.961+0000 7f6ae0060700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad80fce50 0x7f6ad8113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.961+0000 7f6ae0060700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ad80fdfc0 0x7f6ad8114480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.961+0000 7f6ae0060700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8114a80 con 0x7f6ad80fce50 2024-04-25T01:35:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.961+0000 7f6ae0060700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad8114bf0 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.962+0000 7f6adddfc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad80fce50 0x7f6ad8113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:32.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.962+0000 7f6add5fb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ad80fdfc0 0x7f6ad8114480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.962+0000 7f6adddfc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad80fce50 0x7f6ad8113f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41624/0 (socket says 172.21.15.33:41624) 2024-04-25T01:35:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.962+0000 7f6adddfc700 1 -- 172.21.15.33:0/3203966022 learned_addr learned my addr 172.21.15.33:0/3203966022 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.962+0000 7f6add5fb700 1 -- 172.21.15.33:0/3203966022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad80fce50 msgr2=0x7f6ad8113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.962+0000 7f6add5fb700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad80fce50 0x7f6ad8113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.962+0000 7f6add5fb700 1 -- 172.21.15.33:0/3203966022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ac8005040 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.963+0000 7f6add5fb700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ad80fdfc0 0x7f6ad8114480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b0d0 tx=0x7f6ac8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.963+0000 7f6ace7fc700 1 -- 172.21.15.33:0/3203966022 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac800e070 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.963+0000 7f6ace7fc700 1 -- 172.21.15.33:0/3203966022 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ac8013de0 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.964+0000 7f6ace7fc700 1 -- 172.21.15.33:0/3203966022 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac800be30 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.964+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad81195a0 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.964+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad8119b10 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.965+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad8101950 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.966+0000 7f6ace7fc700 1 -- 172.21.15.33:0/3203966022 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) v1 ==== 89277+0+0 (secure 0 0 0) 0x7f6ac800b680 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.967+0000 7f6ace7fc700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6ac406b730 0x7f6ac406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.967+0000 7f6ace7fc700 1 -- 172.21.15.33:0/3203966022 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6ac808a8d0 con 0x7f6ad80fdfc0 2024-04-25T01:35:32.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.967+0000 7f6adddfc700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6ac406b730 0x7f6ac406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:32.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.968+0000 7f6adddfc700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6ac406b730 0x7f6ac406dbe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4008fd0 tx=0x7f6ad400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:32.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:32.974+0000 7f6ace7fc700 1 -- 172.21.15.33:0/3203966022 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6ac805a670 con 0x7f6ad80fdfc0 2024-04-25T01:35:33.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.307+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ad802b3b0 con 0x7f6ac406b730 2024-04-25T01:35:33.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.309+0000 7f6ace7fc700 1 -- 172.21.15.33:0/3203966022 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f6ad802b3b0 con 0x7f6ac406b730 2024-04-25T01:35:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.311+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6ac406b730 msgr2=0x7f6ac406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:33.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.311+0000 7f6ae0060700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6ac406b730 0x7f6ac406dbe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ad4008fd0 tx=0x7f6ad400b040 comp rx=0 tx=0).stop 2024-04-25T01:35:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.311+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ad80fdfc0 msgr2=0x7f6ad8114480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.311+0000 7f6ae0060700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ad80fdfc0 0x7f6ad8114480 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6ac800b0d0 tx=0x7f6ac8009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.311+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 shutdown_connections 2024-04-25T01:35:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.311+0000 7f6ae0060700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f6ac406b730 0x7f6ac406dbe0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.312+0000 7f6ae0060700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6ad80fce50 0x7f6ad8113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.312+0000 7f6ae0060700 1 --2- 172.21.15.33:0/3203966022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6ad80fdfc0 0x7f6ad8114480 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.312+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 >> 172.21.15.33:0/3203966022 conn(0x7f6ad80f8510 msgr2=0x7f6ad80f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.312+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 shutdown_connections 2024-04-25T01:35:33.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.312+0000 7f6ae0060700 1 -- 172.21.15.33:0/3203966022 wait complete. 2024-04-25T01:35:33.303 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:35:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.446+0000 7f00253a9700 1 -- 172.21.15.33:0/1570044088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 msgr2=0x7f00200fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.446+0000 7f00253a9700 1 --2- 172.21.15.33:0/1570044088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 0x7f00200fc0c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f0010004940 tx=0x7f0010009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.446+0000 7f00253a9700 1 -- 172.21.15.33:0/1570044088 shutdown_connections 2024-04-25T01:35:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.446+0000 7f00253a9700 1 --2- 172.21.15.33:0/1570044088 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00200fcf60 0x7f00200fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.446+0000 7f00253a9700 1 --2- 172.21.15.33:0/1570044088 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 0x7f00200fc0c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.446+0000 7f00253a9700 1 -- 172.21.15.33:0/1570044088 >> 172.21.15.33:0/1570044088 conn(0x7f00200f7430 msgr2=0x7f00200f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.446+0000 7f00253a9700 1 -- 172.21.15.33:0/1570044088 shutdown_connections 2024-04-25T01:35:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.446+0000 7f00253a9700 1 -- 172.21.15.33:0/1570044088 wait complete. 2024-04-25T01:35:33.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.447+0000 7f00253a9700 1 Processor -- start 2024-04-25T01:35:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.447+0000 7f00253a9700 1 -- start start 2024-04-25T01:35:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.447+0000 7f00253a9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 0x7f0020110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.447+0000 7f00253a9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00200fcf60 0x7f0020112bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.447+0000 7f00253a9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0020066340 con 0x7f00200fbcb0 2024-04-25T01:35:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.447+0000 7f00253a9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0020066480 con 0x7f00200fcf60 2024-04-25T01:35:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.448+0000 7f001effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 0x7f0020110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.448+0000 7f001effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 0x7f0020110680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41648/0 (socket says 172.21.15.33:41648) 2024-04-25T01:35:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.448+0000 7f001effd700 1 -- 172.21.15.33:0/944925384 learned_addr learned my addr 172.21.15.33:0/944925384 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:33.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.448+0000 7f001e7fc700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00200fcf60 0x7f0020112bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.448+0000 7f001effd700 1 -- 172.21.15.33:0/944925384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00200fcf60 msgr2=0x7f0020112bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.448+0000 7f001effd700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00200fcf60 0x7f0020112bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.448+0000 7f001effd700 1 -- 172.21.15.33:0/944925384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0010005040 con 0x7f00200fbcb0 2024-04-25T01:35:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.448+0000 7f001effd700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 0x7f0020110680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f001000b0d0 tx=0x7f0010009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.448+0000 7f00077fe700 1 -- 172.21.15.33:0/944925384 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001000e030 con 0x7f00200fbcb0 2024-04-25T01:35:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.449+0000 7f00077fe700 1 -- 172.21.15.33:0/944925384 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f001001c400 con 0x7f00200fbcb0 2024-04-25T01:35:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.449+0000 7f00077fe700 1 -- 172.21.15.33:0/944925384 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f001000b630 con 0x7f00200fbcb0 2024-04-25T01:35:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.449+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0020113170 con 0x7f00200fbcb0 2024-04-25T01:35:33.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.449+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00201136e0 con 0x7f00200fbcb0 2024-04-25T01:35:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.450+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002005f5c0 con 0x7f00200fbcb0 2024-04-25T01:35:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.451+0000 7f00077fe700 1 -- 172.21.15.33:0/944925384 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 20) v1 ==== 89277+0+0 (secure 0 0 0) 0x7f001001c570 con 0x7f00200fbcb0 2024-04-25T01:35:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.451+0000 7f00077fe700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f000806b740 0x7f000806dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:33.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.451+0000 7f00077fe700 1 -- 172.21.15.33:0/944925384 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f001008bc70 con 0x7f00200fbcb0 2024-04-25T01:35:33.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.452+0000 7f001e7fc700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f000806b740 0x7f000806dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:33.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.452+0000 7f001e7fc700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f000806b740 0x7f000806dbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0014009b80 tx=0x7f001400b040 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:33.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.456+0000 7f00077fe700 1 -- 172.21.15.33:0/944925384 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0010057fc0 con 0x7f00200fbcb0 2024-04-25T01:35:33.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:33 smithi138 ceph-mon[32977]: pgmap v251: 65 pgs: 65 active+clean; 148 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-04-25T01:35:33.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.744+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0020101250 con 0x7f000806b740 2024-04-25T01:35:33.721 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:33 smithi033 ceph-mon[26353]: pgmap v251: 65 pgs: 65 active+clean; 148 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.7 MiB/s wr, 554 op/s 2024-04-25T01:35:33.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.745+0000 7f00077fe700 1 -- 172.21.15.33:0/944925384 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f0020101250 con 0x7f000806b740 2024-04-25T01:35:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f000806b740 msgr2=0x7f000806dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:33.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f000806b740 0x7f000806dbf0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0014009b80 tx=0x7f001400b040 comp rx=0 tx=0).stop 2024-04-25T01:35:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 msgr2=0x7f0020110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 0x7f0020110680 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f001000b0d0 tx=0x7f0010009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 shutdown_connections 2024-04-25T01:35:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f000806b740 0x7f000806dbf0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00200fbcb0 0x7f0020110680 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 --2- 172.21.15.33:0/944925384 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00200fcf60 0x7f0020112bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 >> 172.21.15.33:0/944925384 conn(0x7f00200f7430 msgr2=0x7f00200f8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 shutdown_connections 2024-04-25T01:35:33.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.748+0000 7f00253a9700 1 -- 172.21.15.33:0/944925384 wait complete. 2024-04-25T01:35:33.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.895+0000 7f2546f86700 1 -- 172.21.15.33:0/3160756978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fcf60 msgr2=0x7f25400fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:33.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.895+0000 7f2546f86700 1 --2- 172.21.15.33:0/3160756978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fcf60 0x7f25400fd3d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f252c004940 tx=0x7f252c009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:33.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.896+0000 7f2546f86700 1 -- 172.21.15.33:0/3160756978 shutdown_connections 2024-04-25T01:35:33.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.896+0000 7f2546f86700 1 --2- 172.21.15.33:0/3160756978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fcf60 0x7f25400fd3d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.896+0000 7f2546f86700 1 --2- 172.21.15.33:0/3160756978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25400fbcb0 0x7f25400fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.896+0000 7f2546f86700 1 -- 172.21.15.33:0/3160756978 >> 172.21.15.33:0/3160756978 conn(0x7f25400f7430 msgr2=0x7f25400f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:33.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.896+0000 7f2546f86700 1 -- 172.21.15.33:0/3160756978 shutdown_connections 2024-04-25T01:35:33.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.896+0000 7f2546f86700 1 -- 172.21.15.33:0/3160756978 wait complete. 2024-04-25T01:35:33.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.897+0000 7f2546f86700 1 Processor -- start 2024-04-25T01:35:33.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.897+0000 7f2546f86700 1 -- start start 2024-04-25T01:35:33.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.897+0000 7f2546f86700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fbcb0 0x7f254010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.897+0000 7f2546f86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25400fcf60 0x7f25401100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.898+0000 7f2546f86700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25401106e0 con 0x7f25400fcf60 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.898+0000 7f2546f86700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2540110f30 con 0x7f25400fbcb0 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.898+0000 7f2544d22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fbcb0 0x7f254010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.898+0000 7f2544d22700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fbcb0 0x7f254010fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:57874/0 (socket says 172.21.15.33:57874) 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.898+0000 7f2544d22700 1 -- 172.21.15.33:0/1947960258 learned_addr learned my addr 172.21.15.33:0/1947960258 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.898+0000 7f2544d22700 1 -- 172.21.15.33:0/1947960258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25400fcf60 msgr2=0x7f25401100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.898+0000 7f2544d22700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25400fcf60 0x7f25401100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.898+0000 7f2544d22700 1 -- 172.21.15.33:0/1947960258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c005040 con 0x7f25400fbcb0 2024-04-25T01:35:33.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.898+0000 7f2544d22700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fbcb0 0x7f254010fba0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f253400eb40 tx=0x7f253400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:33.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.899+0000 7f253d7fa700 1 -- 172.21.15.33:0/1947960258 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2534007cc0 con 0x7f25400fbcb0 2024-04-25T01:35:33.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.899+0000 7f253d7fa700 1 -- 172.21.15.33:0/1947960258 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f253400f040 con 0x7f25400fbcb0 2024-04-25T01:35:33.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.899+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25401111b0 con 0x7f25400fbcb0 2024-04-25T01:35:33.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.899+0000 7f253d7fa700 1 -- 172.21.15.33:0/1947960258 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2534007cc0 con 0x7f25400fbcb0 2024-04-25T01:35:33.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.899+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25401116f0 con 0x7f25400fbcb0 2024-04-25T01:35:33.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.900+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25401018f0 con 0x7f25400fbcb0 2024-04-25T01:35:33.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.901+0000 7f253d7fa700 1 -- 172.21.15.33:0/1947960258 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 20) v1 ==== 89277+0+0 (secure 0 0 0) 0x7f253400c680 con 0x7f25400fbcb0 2024-04-25T01:35:33.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.902+0000 7f253d7fa700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f253006b6d0 0x7f253006db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:33.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.902+0000 7f253d7fa700 1 -- 172.21.15.33:0/1947960258 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2534089910 con 0x7f25400fbcb0 2024-04-25T01:35:33.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.902+0000 7f253ffff700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f253006b6d0 0x7f253006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:33.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.903+0000 7f253ffff700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f253006b6d0 0x7f253006db80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f252c013870 tx=0x7f252c009e30 comp rx=0 tx=0).ready entity=mgr.14208 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:33.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:33.908+0000 7f253d7fa700 1 -- 172.21.15.33:0/1947960258 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2534059700 con 0x7f25400fbcb0 2024-04-25T01:35:34.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.239+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 --> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2540111910 con 0x7f253006b6d0 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.252+0000 7f253d7fa700 1 -- 172.21.15.33:0/1947960258 <== mgr.14208 v2:172.21.15.33:6800/1387801224 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f2540111910 con 0x7f253006b6d0 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (6m) 2m ago 7m 24.8M - 0.25.0 c8568f914cd2 50a86f263578 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (8m) 2m ago 8m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (7m) 6s ago 7m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (6m) 2m ago 7m 82.9M - 9.4.7 954c08fa6188 4533d743079b 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (2m) 2m ago 2m 14.7M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (2m) 2m ago 2m 16.2M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (2m) 6s ago 2m 28.0M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (2m) 6s ago 2m 23.2M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:35:34.229 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:9283 running (9m) 2m ago 9m 481M - 17.2.7-902-gc61472c8 0de407c1b3af 22f7bb21dc39 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283 running (31s) 6s ago 7m 96.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8ce48ef47e1e 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (9m) 2m ago 9m 46.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (6m) 6s ago 6m 39.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (8m) 2m ago 8m 16.0M - 1.5.0 0da6a335fe13 c2f75e5aaf0f 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 6s ago 7m 16.4M - 1.5.0 0da6a335fe13 ce8fbd90510d 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (6m) 2m ago 6m 46.3M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (5m) 2m ago 5m 48.1M 4289M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (5m) 2m ago 5m 43.6M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (4m) 6s ago 4m 72.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (4m) 6s ago 4m 74.4M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (4m) 6s ago 4m 75.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:35:34.230 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (6m) 2m ago 7m 56.5M - 2.43.0 a07b618ecd1d 9109b23faf81 2024-04-25T01:35:34.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f253006b6d0 msgr2=0x7f253006db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:34.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f253006b6d0 0x7f253006db80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f252c013870 tx=0x7f252c009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:34.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fbcb0 msgr2=0x7f254010fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:34.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fbcb0 0x7f254010fba0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f253400eb40 tx=0x7f253400b040 comp rx=0 tx=0).stop 2024-04-25T01:35:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 shutdown_connections 2024-04-25T01:35:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25400fbcb0 0x7f254010fba0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.33:6800/1387801224,v1:172.21.15.33:6801/1387801224] conn(0x7f253006b6d0 0x7f253006db80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 --2- 172.21.15.33:0/1947960258 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25400fcf60 0x7f25401100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 >> 172.21.15.33:0/1947960258 conn(0x7f25400f7430 msgr2=0x7f25400f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 shutdown_connections 2024-04-25T01:35:34.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.256+0000 7f2546f86700 1 -- 172.21.15.33:0/1947960258 wait complete. 2024-04-25T01:35:34.306 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:34 smithi138 ceph-mon[32977]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:34.306 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:34 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:34.307 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:34 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:34.307 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:34 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:35:34.307 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:34 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:35:34.307 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:34 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.413+0000 7ff9ffff8700 1 -- 172.21.15.33:0/2932269973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f80fbcb0 msgr2=0x7ff9f80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.413+0000 7ff9ffff8700 1 --2- 172.21.15.33:0/2932269973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f80fbcb0 0x7ff9f80fc0c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4004940 tx=0x7ff9f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.413+0000 7ff9ffff8700 1 -- 172.21.15.33:0/2932269973 shutdown_connections 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.413+0000 7ff9ffff8700 1 --2- 172.21.15.33:0/2932269973 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f80fcf60 0x7ff9f80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.413+0000 7ff9ffff8700 1 --2- 172.21.15.33:0/2932269973 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f80fbcb0 0x7ff9f80fc0c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.413+0000 7ff9ffff8700 1 -- 172.21.15.33:0/2932269973 >> 172.21.15.33:0/2932269973 conn(0x7ff9f80f7430 msgr2=0x7ff9f80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.413+0000 7ff9ffff8700 1 -- 172.21.15.33:0/2932269973 shutdown_connections 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.413+0000 7ff9ffff8700 1 -- 172.21.15.33:0/2932269973 wait complete. 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.414+0000 7ff9ffff8700 1 Processor -- start 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.414+0000 7ff9ffff8700 1 -- start start 2024-04-25T01:35:34.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.414+0000 7ff9ffff8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f80fbcb0 0x7ff9f810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.414+0000 7ff9ffff8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f80fcf60 0x7ff9f810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.414+0000 7ff9ffff8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f810e4b0 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.414+0000 7ff9ffff8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f810e620 con 0x7ff9f80fcf60 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.414+0000 7ff9fdd94700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f80fbcb0 0x7ff9f810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.414+0000 7ff9fdd94700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f80fbcb0 0x7ff9f810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41692/0 (socket says 172.21.15.33:41692) 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.414+0000 7ff9fdd94700 1 -- 172.21.15.33:0/858623757 learned_addr learned my addr 172.21.15.33:0/858623757 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.415+0000 7ff9fd593700 1 --2- 172.21.15.33:0/858623757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f80fcf60 0x7ff9f810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.415+0000 7ff9fdd94700 1 -- 172.21.15.33:0/858623757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f80fcf60 msgr2=0x7ff9f810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.415+0000 7ff9fdd94700 1 --2- 172.21.15.33:0/858623757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f80fcf60 0x7ff9f810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.415+0000 7ff9fdd94700 1 -- 172.21.15.33:0/858623757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f4005040 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.415+0000 7ff9fdd94700 1 --2- 172.21.15.33:0/858623757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f80fbcb0 0x7ff9f810d970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4013870 tx=0x7ff9f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.415+0000 7ff9ee7fc700 1 -- 172.21.15.33:0/858623757 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f400e070 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.415+0000 7ff9ee7fc700 1 -- 172.21.15.33:0/858623757 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9f401c400 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.415+0000 7ff9ffff8700 1 -- 172.21.15.33:0/858623757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f8112f80 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.416+0000 7ff9ee7fc700 1 -- 172.21.15.33:0/858623757 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f4026640 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.416+0000 7ff9ffff8700 1 -- 172.21.15.33:0/858623757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f8113320 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.417+0000 7ff9ee7fc700 1 -- 172.21.15.33:0/858623757 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 45818+0+0 (secure 0 0 0) 0x7ff9f40267a0 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.417+0000 7ff9ffff8700 1 -- 172.21.15.33:0/858623757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f8101af0 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.417+0000 7ff9ee7fc700 1 -- 172.21.15.33:0/858623757 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff9f404dc40 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.426+0000 7ff9ee7fc700 1 -- 172.21.15.33:0/858623757 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff9f401b540 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:34 smithi033 ceph-mon[26353]: from='client.24387 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:34.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:34 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:34.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:34 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:34.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:34 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:35:34.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:34 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:35:34.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:34 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' 2024-04-25T01:35:34.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.923+0000 7ff9ffff8700 1 -- 172.21.15.33:0/858623757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff9f804c710 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.924+0000 7ff9ee7fc700 1 -- 172.21.15.33:0/858623757 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7ff9f401b360 con 0x7ff9f80fbcb0 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:35:34.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:35:34.911 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:35:34.911 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:35:34.911 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:35:34.911 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:35:34.911 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-25T01:35:34.911 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-25T01:35:34.911 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:35:34.911 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:35:34.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.927+0000 7ff9ffff8700 1 -- 172.21.15.33:0/858623757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f80fbcb0 msgr2=0x7ff9f810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:34.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.927+0000 7ff9ffff8700 1 --2- 172.21.15.33:0/858623757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f80fbcb0 0x7ff9f810d970 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4013870 tx=0x7ff9f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:34.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.927+0000 7ff9ffff8700 1 -- 172.21.15.33:0/858623757 shutdown_connections 2024-04-25T01:35:34.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.927+0000 7ff9ffff8700 1 --2- 172.21.15.33:0/858623757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9f80fcf60 0x7ff9f810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:34.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.927+0000 7ff9ffff8700 1 --2- 172.21.15.33:0/858623757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9f80fbcb0 0x7ff9f810d970 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:34.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.927+0000 7ff9ffff8700 1 -- 172.21.15.33:0/858623757 >> 172.21.15.33:0/858623757 conn(0x7ff9f80f7430 msgr2=0x7ff9f8066740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:34.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.928+0000 7ff9ffff8700 1 -- 172.21.15.33:0/858623757 shutdown_connections 2024-04-25T01:35:34.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:34.928+0000 7ff9ffff8700 1 -- 172.21.15.33:0/858623757 wait complete. 2024-04-25T01:35:35.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.104+0000 7f8a471ce700 1 -- 172.21.15.33:0/3876551949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400662c0 msgr2=0x7f8a400ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:35.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.104+0000 7f8a471ce700 1 --2- 172.21.15.33:0/3876551949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400662c0 0x7f8a400ff9a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8a34004940 tx=0x7f8a34009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:35.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.104+0000 7f8a471ce700 1 -- 172.21.15.33:0/3876551949 shutdown_connections 2024-04-25T01:35:35.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.104+0000 7f8a471ce700 1 --2- 172.21.15.33:0/3876551949 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a400ffee0 0x7f8a40102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:35.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.104+0000 7f8a471ce700 1 --2- 172.21.15.33:0/3876551949 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400662c0 0x7f8a400ff9a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:35.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.104+0000 7f8a471ce700 1 -- 172.21.15.33:0/3876551949 >> 172.21.15.33:0/3876551949 conn(0x7f8a400f7430 msgr2=0x7f8a400f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:35.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.104+0000 7f8a471ce700 1 -- 172.21.15.33:0/3876551949 shutdown_connections 2024-04-25T01:35:35.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.104+0000 7f8a471ce700 1 -- 172.21.15.33:0/3876551949 wait complete. 2024-04-25T01:35:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.105+0000 7f8a471ce700 1 Processor -- start 2024-04-25T01:35:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.105+0000 7f8a471ce700 1 -- start start 2024-04-25T01:35:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.105+0000 7f8a471ce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a400662c0 0x7f8a400fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.105+0000 7f8a471ce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400ffee0 0x7f8a400fd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.105+0000 7f8a471ce700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a400fb890 con 0x7f8a400ffee0 2024-04-25T01:35:35.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.105+0000 7f8a471ce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a400fb9d0 con 0x7f8a400662c0 2024-04-25T01:35:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.106+0000 7f8a3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400ffee0 0x7f8a400fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.106+0000 7f8a3ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400ffee0 0x7f8a400fd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41696/0 (socket says 172.21.15.33:41696) 2024-04-25T01:35:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.106+0000 7f8a3ffff700 1 -- 172.21.15.33:0/4198735553 learned_addr learned my addr 172.21.15.33:0/4198735553 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:35:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.106+0000 7f8a44f6a700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a400662c0 0x7f8a400fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.106+0000 7f8a3ffff700 1 -- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a400662c0 msgr2=0x7f8a400fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:35.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.106+0000 7f8a3ffff700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a400662c0 0x7f8a400fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.106+0000 7f8a3ffff700 1 -- 172.21.15.33:0/4198735553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a34005040 con 0x7f8a400ffee0 2024-04-25T01:35:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.107+0000 7f8a3ffff700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400ffee0 0x7f8a400fd440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c002a50 tx=0x7f8a2c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.107+0000 7f8a3d7fa700 1 -- 172.21.15.33:0/4198735553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a2c007cc0 con 0x7f8a400ffee0 2024-04-25T01:35:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.107+0000 7f8a3d7fa700 1 -- 172.21.15.33:0/4198735553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a2c00f040 con 0x7f8a400ffee0 2024-04-25T01:35:35.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.107+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a400fbcb0 con 0x7f8a400ffee0 2024-04-25T01:35:35.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.107+0000 7f8a3d7fa700 1 -- 172.21.15.33:0/4198735553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a2c007cc0 con 0x7f8a400ffee0 2024-04-25T01:35:35.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.108+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a400fc220 con 0x7f8a400ffee0 2024-04-25T01:35:35.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.108+0000 7f8a3d7fa700 1 -- 172.21.15.33:0/4198735553 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 21) v1 ==== 45818+0+0 (secure 0 0 0) 0x7f8a2c00d8d0 con 0x7f8a400ffee0 2024-04-25T01:35:35.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.109+0000 7f8a3d7fa700 1 -- 172.21.15.33:0/4198735553 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f8a2c04b450 con 0x7f8a400ffee0 2024-04-25T01:35:35.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.110+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4005f5c0 con 0x7f8a400ffee0 2024-04-25T01:35:35.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.116+0000 7f8a3d7fa700 1 -- 172.21.15.33:0/4198735553 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8a2c0112f0 con 0x7f8a400ffee0 2024-04-25T01:35:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.320+0000 7f8a3d7fa700 1 -- 172.21.15.33:0/4198735553 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mgrmap(e 22) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f8a2c01ca10 con 0x7f8a400ffee0 2024-04-25T01:35:35.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.320+0000 7f8a3d7fa700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f8a3003dcb0 0x7f8a30040090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:35:35.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.328+0000 7f8a44f6a700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f8a3003dcb0 0x7f8a30040090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:35:35.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.330+0000 7f8a44f6a700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f8a3003dcb0 0x7f8a30040090 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a400fcc90 tx=0x7f8a34009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:35:35.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.457+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a40068070 con 0x7f8a3003dcb0 2024-04-25T01:35:35.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.472+0000 7f8a3d7fa700 1 -- 172.21.15.33:0/4198735553 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f8a40068070 con 0x7f8a3003dcb0 2024-04-25T01:35:35.449 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:35:35.449 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:35:35.449 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:35:35.449 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-25T01:35:35.450 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [], 2024-04-25T01:35:35.450 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "1/2 daemons upgraded", 2024-04-25T01:35:35.450 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "", 2024-04-25T01:35:35.450 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:35:35.450 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:35:35.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f8a3003dcb0 msgr2=0x7f8a30040090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f8a3003dcb0 0x7f8a30040090 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8a400fcc90 tx=0x7f8a34009e30 comp rx=0 tx=0).stop 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400ffee0 msgr2=0x7f8a400fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400ffee0 0x7f8a400fd440 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c002a50 tx=0x7f8a2c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 shutdown_connections 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f8a3003dcb0 0x7f8a30040090 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a400662c0 0x7f8a400fcf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 --2- 172.21.15.33:0/4198735553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a400ffee0 0x7f8a400fd440 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 >> 172.21.15.33:0/4198735553 conn(0x7f8a400f7430 msgr2=0x7f8a400f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 shutdown_connections 2024-04-25T01:35:35.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:35:35.475+0000 7f8a471ce700 1 -- 172.21.15.33:0/4198735553 wait complete. 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr fail", "who": "smithi033.asemba"}]: dispatch 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: Activating manager daemon smithi138.mmfhyi 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: osdmap e46: 6 total, 6 up, 6 in 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "mgr fail", "who": "smithi033.asemba"}]': finished 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: mgrmap e21: smithi138.mmfhyi(active, starting, since 0.00958278s) 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:35:35.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr metadata", "who": "smithi138.mmfhyi", "id": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: Manager daemon smithi138.mmfhyi is now available 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:35.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.mmfhyi/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:35:35.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.mmfhyi/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:35:35.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.mmfhyi/trash_purge_schedule"}]: dispatch 2024-04-25T01:35:35.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.mmfhyi/trash_purge_schedule"}]: dispatch 2024-04-25T01:35:35.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:35 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/858623757' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:35:35.582 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:35.582 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr fail", "who": "smithi033.asemba"}]: dispatch 2024-04-25T01:35:35.582 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: Activating manager daemon smithi138.mmfhyi 2024-04-25T01:35:35.582 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: osdmap e46: 6 total, 6 up, 6 in 2024-04-25T01:35:35.582 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.14208 172.21.15.33:0/2522588271' entity='mgr.smithi033.asemba' cmd='[{"prefix": "mgr fail", "who": "smithi033.asemba"}]': finished 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: mgrmap e21: smithi138.mmfhyi(active, starting, since 0.00958278s) 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr metadata", "who": "smithi138.mmfhyi", "id": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:35:35.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: Manager daemon smithi138.mmfhyi is now available 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.mmfhyi/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.mmfhyi/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.mmfhyi/trash_purge_schedule"}]: dispatch 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi138.mmfhyi/trash_purge_schedule"}]: dispatch 2024-04-25T01:35:35.584 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:35 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/858623757' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:35:36.634 INFO:tasks.workunit.client.0.smithi033.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-25T01:35:36.638 INFO:tasks.workunit.client.0.smithi033.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-25T01:35:36.638 INFO:tasks.workunit.client.0.smithi033.stderr:+ make 2024-04-25T01:35:36.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:36 smithi033 ceph-mon[26353]: Deploying cephadm binary to smithi138 2024-04-25T01:35:36.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:36 smithi033 ceph-mon[26353]: Deploying cephadm binary to smithi033 2024-04-25T01:35:36.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:36 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:36.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:36 smithi033 ceph-mon[26353]: mgrmap e22: smithi138.mmfhyi(active, since 1.01165s) 2024-04-25T01:35:36.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:36 smithi033 ceph-mon[26353]: pgmap v3: 65 pgs: 65 active+clean; 182 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:35:36.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:36 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:36.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:36 smithi033 ceph-mon[26353]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:36.754 INFO:tasks.workunit.client.0.smithi033.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-25T01:35:36.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:36 smithi138 ceph-mon[32977]: Deploying cephadm binary to smithi138 2024-04-25T01:35:36.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:36 smithi138 ceph-mon[32977]: Deploying cephadm binary to smithi033 2024-04-25T01:35:36.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:36 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:36.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:36 smithi138 ceph-mon[32977]: mgrmap e22: smithi138.mmfhyi(active, since 1.01165s) 2024-04-25T01:35:36.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:36 smithi138 ceph-mon[32977]: pgmap v3: 65 pgs: 65 active+clean; 182 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:35:36.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:36 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:36.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:36 smithi138 ceph-mon[32977]: from='client.14612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:35:37.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:37 smithi033 ceph-mon[26353]: [25/Apr/2024:01:35:35] ENGINE Bus STARTING 2024-04-25T01:35:37.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:37 smithi033 ceph-mon[26353]: [25/Apr/2024:01:35:35] ENGINE Serving on http://172.21.15.138:8765 2024-04-25T01:35:37.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:37 smithi033 ceph-mon[26353]: [25/Apr/2024:01:35:36] ENGINE Serving on https://172.21.15.138:7150 2024-04-25T01:35:37.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:37 smithi033 ceph-mon[26353]: [25/Apr/2024:01:35:36] ENGINE Bus STARTED 2024-04-25T01:35:37.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:37 smithi033 ceph-mon[26353]: pgmap v4: 65 pgs: 65 active+clean; 182 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:35:37.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:37 smithi033 ceph-mon[26353]: mgrmap e23: smithi138.mmfhyi(active, since 2s) 2024-04-25T01:35:37.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:37 smithi138 ceph-mon[32977]: [25/Apr/2024:01:35:35] ENGINE Bus STARTING 2024-04-25T01:35:37.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:37 smithi138 ceph-mon[32977]: [25/Apr/2024:01:35:35] ENGINE Serving on http://172.21.15.138:8765 2024-04-25T01:35:37.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:37 smithi138 ceph-mon[32977]: [25/Apr/2024:01:35:36] ENGINE Serving on https://172.21.15.138:7150 2024-04-25T01:35:37.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:37 smithi138 ceph-mon[32977]: [25/Apr/2024:01:35:36] ENGINE Bus STARTED 2024-04-25T01:35:37.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:37 smithi138 ceph-mon[32977]: pgmap v4: 65 pgs: 65 active+clean; 182 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:35:37.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:37 smithi138 ceph-mon[32977]: mgrmap e23: smithi138.mmfhyi(active, since 2s) 2024-04-25T01:35:38.313 INFO:tasks.workunit.client.0.smithi033.stderr:++ readlink -f fsstress 2024-04-25T01:35:38.314 INFO:tasks.workunit.client.0.smithi033.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-25T01:35:38.315 INFO:tasks.workunit.client.0.smithi033.stderr:+ popd 2024-04-25T01:35:38.316 INFO:tasks.workunit.client.0.smithi033.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-25T01:35:38.316 INFO:tasks.workunit.client.0.smithi033.stderr:+ popd 2024-04-25T01:35:38.317 INFO:tasks.workunit.client.0.smithi033.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-25T01:35:38.317 INFO:tasks.workunit.client.0.smithi033.stderr:++ mktemp -d -p . 2024-04-25T01:35:38.320 INFO:tasks.workunit.client.0.smithi033.stderr:+ T=./tmp.IsQvjW8rMQ 2024-04-25T01:35:38.320 INFO:tasks.workunit.client.0.smithi033.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.IsQvjW8rMQ -l 1 -n 1000 -p 10 -v 2024-04-25T01:35:38.324 INFO:tasks.workunit.client.0.smithi033.stdout:seed = 1714218688 2024-04-25T01:35:38.328 INFO:tasks.workunit.client.0.smithi033.stdout:0/0: truncate - no filename 2024-04-25T01:35:38.328 INFO:tasks.workunit.client.0.smithi033.stdout:0/1: write - no filename 2024-04-25T01:35:38.328 INFO:tasks.workunit.client.0.smithi033.stdout:0/2: dread - no filename 2024-04-25T01:35:38.329 INFO:tasks.workunit.client.0.smithi033.stdout:0/3: truncate - no filename 2024-04-25T01:35:38.329 INFO:tasks.workunit.client.0.smithi033.stdout:0/4: write - no filename 2024-04-25T01:35:38.329 INFO:tasks.workunit.client.0.smithi033.stdout:0/5: write - no filename 2024-04-25T01:35:38.329 INFO:tasks.workunit.client.0.smithi033.stdout:0/6: truncate - no filename 2024-04-25T01:35:38.329 INFO:tasks.workunit.client.0.smithi033.stdout:0/7: dwrite - no filename 2024-04-25T01:35:38.330 INFO:tasks.workunit.client.0.smithi033.stdout:1/0: write - no filename 2024-04-25T01:35:38.330 INFO:tasks.workunit.client.0.smithi033.stdout:0/8: symlink l0 0 2024-04-25T01:35:38.333 INFO:tasks.workunit.client.0.smithi033.stdout:1/1: creat f0 x:0 0 0 2024-04-25T01:35:38.334 INFO:tasks.workunit.client.0.smithi033.stdout:1/2: truncate f0 833997 0 2024-04-25T01:35:38.334 INFO:tasks.workunit.client.0.smithi033.stdout:3/0: dwrite - no filename 2024-04-25T01:35:38.334 INFO:tasks.workunit.client.0.smithi033.stdout:3/1: chown . 122189 1 2024-04-25T01:35:38.334 INFO:tasks.workunit.client.0.smithi033.stdout:3/2: read - no filename 2024-04-25T01:35:38.335 INFO:tasks.workunit.client.0.smithi033.stdout:3/3: rmdir - no directory 2024-04-25T01:35:38.335 INFO:tasks.workunit.client.0.smithi033.stdout:3/4: rename - no filename 2024-04-25T01:35:38.336 INFO:tasks.workunit.client.0.smithi033.stdout:2/0: creat f0 x:0 0 0 2024-04-25T01:35:38.338 INFO:tasks.workunit.client.0.smithi033.stdout:4/0: dread - no filename 2024-04-25T01:35:38.338 INFO:tasks.workunit.client.0.smithi033.stdout:4/1: rename - no filename 2024-04-25T01:35:38.338 INFO:tasks.workunit.client.0.smithi033.stdout:4/2: fsync - no filename 2024-04-25T01:35:38.338 INFO:tasks.workunit.client.0.smithi033.stdout:4/3: dread - no filename 2024-04-25T01:35:38.338 INFO:tasks.workunit.client.0.smithi033.stdout:4/4: rename - no filename 2024-04-25T01:35:38.339 INFO:tasks.workunit.client.0.smithi033.stdout:4/5: chown . 0 1 2024-04-25T01:35:38.340 INFO:tasks.workunit.client.0.smithi033.stdout:2/1: creat f1 x:0 0 0 2024-04-25T01:35:38.341 INFO:tasks.workunit.client.0.smithi033.stdout:3/5: mknod c0 0 2024-04-25T01:35:38.342 INFO:tasks.workunit.client.0.smithi033.stdout:5/0: dread - no filename 2024-04-25T01:35:38.343 INFO:tasks.workunit.client.0.smithi033.stdout:5/1: write - no filename 2024-04-25T01:35:38.343 INFO:tasks.workunit.client.0.smithi033.stdout:5/2: truncate - no filename 2024-04-25T01:35:38.344 INFO:tasks.workunit.client.0.smithi033.stdout:2/2: creat f2 x:0 0 0 2024-04-25T01:35:38.346 INFO:tasks.workunit.client.0.smithi033.stdout:3/6: symlink l1 0 2024-04-25T01:35:38.347 INFO:tasks.workunit.client.0.smithi033.stdout:6/0: fdatasync - no filename 2024-04-25T01:35:38.349 INFO:tasks.workunit.client.0.smithi033.stdout:4/6: mknod c0 0 2024-04-25T01:35:38.349 INFO:tasks.workunit.client.0.smithi033.stdout:4/7: truncate - no filename 2024-04-25T01:35:38.350 INFO:tasks.workunit.client.0.smithi033.stdout:2/3: mknod c3 0 2024-04-25T01:35:38.351 INFO:tasks.workunit.client.0.smithi033.stdout:3/7: mknod c2 0 2024-04-25T01:35:38.353 INFO:tasks.workunit.client.0.smithi033.stdout:3/8: readlink l1 0 2024-04-25T01:35:38.354 INFO:tasks.workunit.client.0.smithi033.stdout:5/3: creat f0 x:0 0 0 2024-04-25T01:35:38.355 INFO:tasks.workunit.client.0.smithi033.stdout:7/0: write - no filename 2024-04-25T01:35:38.356 INFO:tasks.workunit.client.0.smithi033.stdout:7/1: chown . 1 1 2024-04-25T01:35:38.358 INFO:tasks.workunit.client.0.smithi033.stdout:3/9: creat f3 x:0 0 0 2024-04-25T01:35:38.359 INFO:tasks.workunit.client.0.smithi033.stdout:3/10: dread - f3 zero size 2024-04-25T01:35:38.361 INFO:tasks.workunit.client.0.smithi033.stdout:6/1: symlink l0 0 2024-04-25T01:35:38.361 INFO:tasks.workunit.client.0.smithi033.stdout:6/2: truncate - no filename 2024-04-25T01:35:38.362 INFO:tasks.workunit.client.0.smithi033.stdout:2/4: dwrite f0 [0,4194304] 0 2024-04-25T01:35:38.363 INFO:tasks.workunit.client.0.smithi033.stdout:5/4: unlink f0 0 2024-04-25T01:35:38.363 INFO:tasks.workunit.client.0.smithi033.stdout:5/5: unlink - no file 2024-04-25T01:35:38.363 INFO:tasks.workunit.client.0.smithi033.stdout:5/6: dwrite - no filename 2024-04-25T01:35:38.363 INFO:tasks.workunit.client.0.smithi033.stdout:5/7: rmdir - no directory 2024-04-25T01:35:38.363 INFO:tasks.workunit.client.0.smithi033.stdout:5/8: chown . 0 1 2024-04-25T01:35:38.367 INFO:tasks.workunit.client.0.smithi033.stdout:3/11: symlink l4 0 2024-04-25T01:35:38.368 INFO:tasks.workunit.client.0.smithi033.stdout:3/12: fdatasync f3 0 2024-04-25T01:35:38.370 INFO:tasks.workunit.client.0.smithi033.stdout:9/0: rename - no filename 2024-04-25T01:35:38.370 INFO:tasks.workunit.client.0.smithi033.stdout:9/1: write - no filename 2024-04-25T01:35:38.370 INFO:tasks.workunit.client.0.smithi033.stdout:5/9: creat f1 x:0 0 0 2024-04-25T01:35:38.371 INFO:tasks.workunit.client.0.smithi033.stdout:5/10: readlink - no filename 2024-04-25T01:35:38.372 INFO:tasks.workunit.client.0.smithi033.stdout:4/8: link c0 c1 0 2024-04-25T01:35:38.372 INFO:tasks.workunit.client.0.smithi033.stdout:5/11: write f1 [848000,66541] 0 2024-04-25T01:35:38.373 INFO:tasks.workunit.client.0.smithi033.stdout:7/2: creat f0 x:0 0 0 2024-04-25T01:35:38.374 INFO:tasks.workunit.client.0.smithi033.stdout:5/12: write f1 [547619,33930] 0 2024-04-25T01:35:38.377 INFO:tasks.workunit.client.0.smithi033.stdout:8/0: symlink l0 0 2024-04-25T01:35:38.380 INFO:tasks.workunit.client.0.smithi033.stdout:3/13: mknod c5 0 2024-04-25T01:35:38.381 INFO:tasks.workunit.client.0.smithi033.stdout:2/5: link f0 f4 0 2024-04-25T01:35:38.383 INFO:tasks.workunit.client.0.smithi033.stdout:3/14: chown l1 643497 1 2024-04-25T01:35:38.383 INFO:tasks.workunit.client.0.smithi033.stdout:3/15: rmdir - no directory 2024-04-25T01:35:38.384 INFO:tasks.workunit.client.0.smithi033.stdout:4/9: mknod c2 0 2024-04-25T01:35:38.386 INFO:tasks.workunit.client.0.smithi033.stdout:7/3: dwrite f0 [0,4194304] 0 2024-04-25T01:35:38.388 INFO:tasks.workunit.client.0.smithi033.stdout:5/13: mknod c2 0 2024-04-25T01:35:38.389 INFO:tasks.workunit.client.0.smithi033.stdout:5/14: write f1 [862403,46759] 0 2024-04-25T01:35:38.390 INFO:tasks.workunit.client.0.smithi033.stdout:5/15: chown c2 0 1 2024-04-25T01:35:38.391 INFO:tasks.workunit.client.0.smithi033.stdout:5/16: write f1 [42016,96446] 0 2024-04-25T01:35:38.396 INFO:tasks.workunit.client.0.smithi033.stdout:9/2: mkdir d0 0 2024-04-25T01:35:38.396 INFO:tasks.workunit.client.0.smithi033.stdout:9/3: link - no file 2024-04-25T01:35:38.399 INFO:tasks.workunit.client.0.smithi033.stdout:3/16: mknod c6 0 2024-04-25T01:35:38.400 INFO:tasks.workunit.client.0.smithi033.stdout:3/17: dread - f3 zero size 2024-04-25T01:35:38.401 INFO:tasks.workunit.client.0.smithi033.stdout:7/4: creat f1 x:0 0 0 2024-04-25T01:35:38.405 INFO:tasks.workunit.client.0.smithi033.stdout:2/6: dwrite f0 [0,4194304] 0 2024-04-25T01:35:38.406 INFO:tasks.workunit.client.0.smithi033.stdout:2/7: write f2 [330870,20912] 0 2024-04-25T01:35:38.407 INFO:tasks.workunit.client.0.smithi033.stdout:5/17: mknod c3 0 2024-04-25T01:35:38.410 INFO:tasks.workunit.client.0.smithi033.stdout:4/10: stat c1 0 2024-04-25T01:35:38.410 INFO:tasks.workunit.client.0.smithi033.stdout:4/11: dwrite - no filename 2024-04-25T01:35:38.411 INFO:tasks.workunit.client.0.smithi033.stdout:3/18: symlink l7 0 2024-04-25T01:35:38.413 INFO:tasks.workunit.client.0.smithi033.stdout:7/5: mknod c2 0 2024-04-25T01:35:38.413 INFO:tasks.workunit.client.0.smithi033.stdout:7/6: truncate f1 198828 0 2024-04-25T01:35:38.415 INFO:tasks.workunit.client.0.smithi033.stdout:2/8: creat f5 x:0 0 0 2024-04-25T01:35:38.416 INFO:tasks.workunit.client.0.smithi033.stdout:5/18: mknod c4 0 2024-04-25T01:35:38.417 INFO:tasks.workunit.client.0.smithi033.stdout:5/19: stat f1 0 2024-04-25T01:35:38.417 INFO:tasks.workunit.client.0.smithi033.stdout:5/20: readlink - no filename 2024-04-25T01:35:38.418 INFO:tasks.workunit.client.0.smithi033.stdout:5/21: write f1 [472455,88977] 0 2024-04-25T01:35:38.418 INFO:tasks.workunit.client.0.smithi033.stdout:9/4: mknod d0/c1 0 2024-04-25T01:35:38.420 INFO:tasks.workunit.client.0.smithi033.stdout:4/12: rename c2 to c3 0 2024-04-25T01:35:38.421 INFO:tasks.workunit.client.0.smithi033.stdout:3/19: mkdir d8 0 2024-04-25T01:35:38.425 INFO:tasks.workunit.client.0.smithi033.stdout:9/5: rmdir d0 39 2024-04-25T01:35:38.425 INFO:tasks.workunit.client.0.smithi033.stdout:9/6: dread - no filename 2024-04-25T01:35:38.426 INFO:tasks.workunit.client.0.smithi033.stdout:9/7: dwrite - no filename 2024-04-25T01:35:38.426 INFO:tasks.workunit.client.0.smithi033.stdout:9/8: dread - no filename 2024-04-25T01:35:38.426 INFO:tasks.workunit.client.0.smithi033.stdout:9/9: fdatasync - no filename 2024-04-25T01:35:38.426 INFO:tasks.workunit.client.0.smithi033.stdout:9/10: dread - no filename 2024-04-25T01:35:38.429 INFO:tasks.workunit.client.0.smithi033.stdout:2/9: link f2 f6 0 2024-04-25T01:35:38.435 INFO:tasks.workunit.client.0.smithi033.stdout:4/13: link c1 c4 0 2024-04-25T01:35:38.438 INFO:tasks.workunit.client.0.smithi033.stdout:2/10: dwrite f1 [0,4194304] 0 2024-04-25T01:35:38.438 INFO:tasks.workunit.client.0.smithi033.stdout:4/14: creat f5 x:0 0 0 2024-04-25T01:35:38.440 INFO:tasks.workunit.client.0.smithi033.stdout:4/15: mkdir d6 0 2024-04-25T01:35:38.440 INFO:tasks.workunit.client.0.smithi033.stdout:4/16: readlink - no filename 2024-04-25T01:35:38.443 INFO:tasks.workunit.client.0.smithi033.stdout:2/11: link f2 f7 0 2024-04-25T01:35:38.443 INFO:tasks.workunit.client.0.smithi033.stdout:2/12: truncate f7 754400 0 2024-04-25T01:35:38.445 INFO:tasks.workunit.client.0.smithi033.stdout:2/13: rename f5 to f8 0 2024-04-25T01:35:38.445 INFO:tasks.workunit.client.0.smithi033.stdout:2/14: rmdir - no directory 2024-04-25T01:35:38.619 INFO:tasks.workunit.client.0.smithi033.stdout:0/9: getdents . 0 2024-04-25T01:35:38.623 INFO:tasks.workunit.client.0.smithi033.stdout:1/3: getdents . 0 2024-04-25T01:35:38.631 INFO:tasks.workunit.client.0.smithi033.stdout:1/4: dwrite f0 [0,4194304] 0 2024-04-25T01:35:38.631 INFO:tasks.workunit.client.0.smithi033.stdout:1/5: stat f0 0 2024-04-25T01:35:38.632 INFO:tasks.workunit.client.0.smithi033.stdout:1/6: chown f0 742362 1 2024-04-25T01:35:38.633 INFO:tasks.workunit.client.0.smithi033.stdout:1/7: chown f0 6583 1 2024-04-25T01:35:38.644 INFO:tasks.workunit.client.0.smithi033.stdout:3/20: fsync f3 0 2024-04-25T01:35:38.657 INFO:tasks.workunit.client.0.smithi033.stdout:8/1: getdents . 0 2024-04-25T01:35:38.657 INFO:tasks.workunit.client.0.smithi033.stdout:8/2: fsync - no filename 2024-04-25T01:35:38.682 INFO:tasks.workunit.client.0.smithi033.stdout:5/22: truncate f1 95646 0 2024-04-25T01:35:38.682 INFO:tasks.workunit.client.0.smithi033.stdout:4/17: fsync f5 0 2024-04-25T01:35:38.688 INFO:tasks.workunit.client.0.smithi033.stdout:0/10: creat f1 x:0 0 0 2024-04-25T01:35:38.692 INFO:tasks.workunit.client.0.smithi033.stdout:9/11: rmdir d0 39 2024-04-25T01:35:38.692 INFO:tasks.workunit.client.0.smithi033.stdout:9/12: dwrite - no filename 2024-04-25T01:35:38.694 INFO:tasks.workunit.client.0.smithi033.stdout:4/18: unlink f5 0 2024-04-25T01:35:38.694 INFO:tasks.workunit.client.0.smithi033.stdout:4/19: fsync - no filename 2024-04-25T01:35:38.696 INFO:tasks.workunit.client.0.smithi033.stdout:2/15: rename f2 to f9 0 2024-04-25T01:35:38.697 INFO:tasks.workunit.client.0.smithi033.stdout:2/16: chown f9 5408350 1 2024-04-25T01:35:38.697 INFO:tasks.workunit.client.0.smithi033.stdout:0/11: mknod c2 0 2024-04-25T01:35:38.698 INFO:tasks.workunit.client.0.smithi033.stdout:3/21: symlink d8/l9 0 2024-04-25T01:35:38.699 INFO:tasks.workunit.client.0.smithi033.stdout:8/3: getdents . 0 2024-04-25T01:35:38.699 INFO:tasks.workunit.client.0.smithi033.stdout:8/4: write - no filename 2024-04-25T01:35:38.700 INFO:tasks.workunit.client.0.smithi033.stdout:3/22: write f3 [42739,61196] 0 2024-04-25T01:35:38.701 INFO:tasks.workunit.client.0.smithi033.stdout:3/23: write f3 [1126711,22232] 0 2024-04-25T01:35:38.701 INFO:tasks.workunit.client.0.smithi033.stdout:3/24: readlink l7 0 2024-04-25T01:35:38.706 INFO:tasks.workunit.client.0.smithi033.stdout:2/17: mkdir da 0 2024-04-25T01:35:38.708 INFO:tasks.workunit.client.0.smithi033.stdout:0/12: rename c2 to c3 0 2024-04-25T01:35:38.709 INFO:tasks.workunit.client.0.smithi033.stdout:8/5: mknod c1 0 2024-04-25T01:35:38.711 INFO:tasks.workunit.client.0.smithi033.stdout:9/13: mkdir d0/d2 0 2024-04-25T01:35:38.711 INFO:tasks.workunit.client.0.smithi033.stdout:9/14: readlink - no filename 2024-04-25T01:35:38.711 INFO:tasks.workunit.client.0.smithi033.stdout:9/15: dwrite - no filename 2024-04-25T01:35:38.714 INFO:tasks.workunit.client.0.smithi033.stdout:0/13: mknod c4 0 2024-04-25T01:35:38.716 INFO:tasks.workunit.client.0.smithi033.stdout:8/6: mknod c2 0 2024-04-25T01:35:38.718 INFO:tasks.workunit.client.0.smithi033.stdout:8/7: chown c1 25371 1 2024-04-25T01:35:38.718 INFO:tasks.workunit.client.0.smithi033.stdout:8/8: write - no filename 2024-04-25T01:35:38.718 INFO:tasks.workunit.client.0.smithi033.stdout:8/9: fsync - no filename 2024-04-25T01:35:38.718 INFO:tasks.workunit.client.0.smithi033.stdout:8/10: fsync - no filename 2024-04-25T01:35:38.719 INFO:tasks.workunit.client.0.smithi033.stdout:9/16: mkdir d0/d3 0 2024-04-25T01:35:38.719 INFO:tasks.workunit.client.0.smithi033.stdout:9/17: write - no filename 2024-04-25T01:35:38.719 INFO:tasks.workunit.client.0.smithi033.stdout:9/18: dread - no filename 2024-04-25T01:35:38.719 INFO:tasks.workunit.client.0.smithi033.stdout:9/19: dwrite - no filename 2024-04-25T01:35:38.720 INFO:tasks.workunit.client.0.smithi033.stdout:9/20: chown d0/c1 693 1 2024-04-25T01:35:38.721 INFO:tasks.workunit.client.0.smithi033.stdout:9/21: fdatasync - no filename 2024-04-25T01:35:38.722 INFO:tasks.workunit.client.0.smithi033.stdout:2/18: creat da/fb x:0 0 0 2024-04-25T01:35:38.723 INFO:tasks.workunit.client.0.smithi033.stdout:8/11: symlink l3 0 2024-04-25T01:35:38.724 INFO:tasks.workunit.client.0.smithi033.stdout:0/14: dwrite f1 [0,4194304] 0 2024-04-25T01:35:38.725 INFO:tasks.workunit.client.0.smithi033.stdout:2/19: truncate da/fb 334568 0 2024-04-25T01:35:38.728 INFO:tasks.workunit.client.0.smithi033.stdout:4/20: link c3 d6/c7 0 2024-04-25T01:35:38.728 INFO:tasks.workunit.client.0.smithi033.stdout:4/21: write - no filename 2024-04-25T01:35:38.731 INFO:tasks.workunit.client.0.smithi033.stdout:8/12: mknod c4 0 2024-04-25T01:35:38.735 INFO:tasks.workunit.client.0.smithi033.stdout:2/20: unlink da/fb 0 2024-04-25T01:35:38.736 INFO:tasks.workunit.client.0.smithi033.stdout:2/21: read f7 [723231,117684] 0 2024-04-25T01:35:38.737 INFO:tasks.workunit.client.0.smithi033.stdout:9/22: symlink d0/d3/l4 0 2024-04-25T01:35:38.738 INFO:tasks.workunit.client.0.smithi033.stdout:8/13: unlink c4 0 2024-04-25T01:35:38.742 INFO:tasks.workunit.client.0.smithi033.stdout:4/22: creat d6/f8 x:0 0 0 2024-04-25T01:35:38.743 INFO:tasks.workunit.client.0.smithi033.stdout:4/23: write d6/f8 [569156,58587] 0 2024-04-25T01:35:38.746 INFO:tasks.workunit.client.0.smithi033.stdout:2/22: creat da/fc x:0 0 0 2024-04-25T01:35:38.747 INFO:tasks.workunit.client.0.smithi033.stdout:9/23: creat d0/f5 x:0 0 0 2024-04-25T01:35:38.749 INFO:tasks.workunit.client.0.smithi033.stdout:9/24: chown d0/d2 1 1 2024-04-25T01:35:38.749 INFO:tasks.workunit.client.0.smithi033.stdout:8/14: creat f5 x:0 0 0 2024-04-25T01:35:38.750 INFO:tasks.workunit.client.0.smithi033.stdout:4/24: creat d6/f9 x:0 0 0 2024-04-25T01:35:38.752 INFO:tasks.workunit.client.0.smithi033.stdout:8/15: mkdir d6 0 2024-04-25T01:35:38.754 INFO:tasks.workunit.client.0.smithi033.stdout:9/25: creat d0/f6 x:0 0 0 2024-04-25T01:35:38.754 INFO:tasks.workunit.client.0.smithi033.stdout:2/23: dwrite f4 [0,4194304] 0 2024-04-25T01:35:38.756 INFO:tasks.workunit.client.0.smithi033.stdout:4/25: stat c4 0 2024-04-25T01:35:38.757 INFO:tasks.workunit.client.0.smithi033.stdout:9/26: write d0/f6 [46391,111383] 0 2024-04-25T01:35:38.758 INFO:tasks.workunit.client.0.smithi033.stdout:2/24: creat da/fd x:0 0 0 2024-04-25T01:35:38.759 INFO:tasks.workunit.client.0.smithi033.stdout:4/26: mkdir d6/da 0 2024-04-25T01:35:38.763 INFO:tasks.workunit.client.0.smithi033.stdout:8/16: dwrite f5 [0,4194304] 0 2024-04-25T01:35:38.763 INFO:tasks.workunit.client.0.smithi033.stdout:2/25: creat da/fe x:0 0 0 2024-04-25T01:35:38.765 INFO:tasks.workunit.client.0.smithi033.stdout:9/27: getdents d0/d2 0 2024-04-25T01:35:38.768 INFO:tasks.workunit.client.0.smithi033.stdout:4/27: mknod d6/da/cb 0 2024-04-25T01:35:38.769 INFO:tasks.workunit.client.0.smithi033.stdout:2/26: creat da/ff x:0 0 0 2024-04-25T01:35:38.773 INFO:tasks.workunit.client.0.smithi033.stdout:4/28: symlink d6/da/lc 0 2024-04-25T01:35:38.774 INFO:tasks.workunit.client.0.smithi033.stdout:8/17: symlink d6/l7 0 2024-04-25T01:35:38.775 INFO:tasks.workunit.client.0.smithi033.stdout:8/18: chown c2 524 1 2024-04-25T01:35:38.778 INFO:tasks.workunit.client.0.smithi033.stdout:4/29: creat d6/da/fd x:0 0 0 2024-04-25T01:35:38.779 INFO:tasks.workunit.client.0.smithi033.stdout:8/19: creat d6/f8 x:0 0 0 2024-04-25T01:35:38.780 INFO:tasks.workunit.client.0.smithi033.stdout:4/30: mknod d6/da/ce 0 2024-04-25T01:35:38.781 INFO:tasks.workunit.client.0.smithi033.stdout:8/20: creat d6/f9 x:0 0 0 2024-04-25T01:35:38.782 INFO:tasks.workunit.client.0.smithi033.stdout:8/21: read f5 [2518050,130117] 0 2024-04-25T01:35:38.785 INFO:tasks.workunit.client.0.smithi033.stdout:4/31: link d6/f9 d6/da/ff 0 2024-04-25T01:35:38.786 INFO:tasks.workunit.client.0.smithi033.stdout:8/22: creat d6/fa x:0 0 0 2024-04-25T01:35:38.788 INFO:tasks.workunit.client.0.smithi033.stdout:4/32: write d6/f9 [68674,25784] 0 2024-04-25T01:35:38.789 INFO:tasks.workunit.client.0.smithi033.stdout:8/23: mkdir d6/db 0 2024-04-25T01:35:38.790 INFO:tasks.workunit.client.0.smithi033.stdout:8/24: write d6/f8 [306290,28843] 0 2024-04-25T01:35:38.795 INFO:tasks.workunit.client.0.smithi033.stdout:4/33: rename d6/c7 to d6/da/c10 0 2024-04-25T01:35:38.795 INFO:tasks.workunit.client.0.smithi033.stdout:8/25: dread d6/f8 [0,4194304] 0 2024-04-25T01:35:38.795 INFO:tasks.workunit.client.0.smithi033.stdout:4/34: readlink d6/da/lc 0 2024-04-25T01:35:38.800 INFO:tasks.workunit.client.0.smithi033.stdout:4/35: dread d6/f8 [0,4194304] 0 2024-04-25T01:35:38.804 INFO:tasks.workunit.client.0.smithi033.stdout:4/36: write d6/da/fd [393553,96838] 0 2024-04-25T01:35:38.805 INFO:tasks.workunit.client.0.smithi033.stdout:8/26: dwrite d6/f8 [0,4194304] 0 2024-04-25T01:35:38.806 INFO:tasks.workunit.client.0.smithi033.stdout:8/27: dread - d6/fa zero size 2024-04-25T01:35:38.812 INFO:tasks.workunit.client.0.smithi033.stdout:4/37: rename c3 to d6/c11 0 2024-04-25T01:35:38.813 INFO:tasks.workunit.client.0.smithi033.stdout:8/28: creat d6/db/fc x:0 0 0 2024-04-25T01:35:38.814 INFO:tasks.workunit.client.0.smithi033.stdout:4/38: truncate d6/da/ff 603152 0 2024-04-25T01:35:38.815 INFO:tasks.workunit.client.0.smithi033.stdout:4/39: read d6/da/fd [441571,53474] 0 2024-04-25T01:35:38.815 INFO:tasks.workunit.client.0.smithi033.stdout:8/29: symlink d6/ld 0 2024-04-25T01:35:38.817 INFO:tasks.workunit.client.0.smithi033.stdout:4/40: write d6/f8 [112562,124013] 0 2024-04-25T01:35:38.818 INFO:tasks.workunit.client.0.smithi033.stdout:8/30: creat d6/fe x:0 0 0 2024-04-25T01:35:38.819 INFO:tasks.workunit.client.0.smithi033.stdout:4/41: truncate d6/f9 1110574 0 2024-04-25T01:35:38.819 INFO:tasks.workunit.client.0.smithi033.stdout:4/42: chown d6/da 490872580 1 2024-04-25T01:35:38.820 INFO:tasks.workunit.client.0.smithi033.stdout:4/43: chown d6/da/ff 1315 1 2024-04-25T01:35:38.821 INFO:tasks.workunit.client.0.smithi033.stdout:8/31: mknod d6/db/cf 0 2024-04-25T01:35:38.821 INFO:tasks.workunit.client.0.smithi033.stdout:8/32: chown c2 445283754 1 2024-04-25T01:35:38.823 INFO:tasks.workunit.client.0.smithi033.stdout:8/33: truncate d6/fa 549374 0 2024-04-25T01:35:38.826 INFO:tasks.workunit.client.0.smithi033.stdout:8/34: mkdir d6/d10 0 2024-04-25T01:35:38.827 INFO:tasks.workunit.client.0.smithi033.stdout:8/35: fdatasync d6/fa 0 2024-04-25T01:35:38.830 INFO:tasks.workunit.client.0.smithi033.stdout:8/36: write d6/fe [842375,63375] 0 2024-04-25T01:35:38.830 INFO:tasks.workunit.client.0.smithi033.stdout:4/44: dwrite d6/f8 [0,4194304] 0 2024-04-25T01:35:38.832 INFO:tasks.workunit.client.0.smithi033.stdout:8/37: truncate d6/db/fc 860969 0 2024-04-25T01:35:38.832 INFO:tasks.workunit.client.0.smithi033.stdout:4/45: mkdir d6/d12 0 2024-04-25T01:35:38.833 INFO:tasks.workunit.client.0.smithi033.stdout:8/38: chown d6/db/fc 48228 1 2024-04-25T01:35:38.837 INFO:tasks.workunit.client.0.smithi033.stdout:8/39: unlink c1 0 2024-04-25T01:35:38.839 INFO:tasks.workunit.client.0.smithi033.stdout:4/46: creat d6/d12/f13 x:0 0 0 2024-04-25T01:35:38.840 INFO:tasks.workunit.client.0.smithi033.stdout:8/40: rename d6/l7 to d6/l11 0 2024-04-25T01:35:38.843 INFO:tasks.workunit.client.0.smithi033.stdout:8/41: dread d6/fa [0,4194304] 0 2024-04-25T01:35:38.843 INFO:tasks.workunit.client.0.smithi033.stdout:8/42: fdatasync d6/f9 0 2024-04-25T01:35:38.846 INFO:tasks.workunit.client.0.smithi033.stdout:8/43: creat d6/d10/f12 x:0 0 0 2024-04-25T01:35:38.871 INFO:tasks.workunit.client.0.smithi033.stdout:0/15: fsync f1 0 2024-04-25T01:35:38.871 INFO:tasks.workunit.client.0.smithi033.stdout:1/8: write f0 [4903141,85090] 0 2024-04-25T01:35:38.872 INFO:tasks.workunit.client.0.smithi033.stdout:1/9: stat f0 0 2024-04-25T01:35:38.873 INFO:tasks.workunit.client.0.smithi033.stdout:1/10: chown f0 5 1 2024-04-25T01:35:38.873 INFO:tasks.workunit.client.0.smithi033.stdout:1/11: chown f0 271618097 1 2024-04-25T01:35:38.874 INFO:tasks.workunit.client.0.smithi033.stdout:1/12: write f0 [4651888,58996] 0 2024-04-25T01:35:38.878 INFO:tasks.workunit.client.0.smithi033.stdout:1/13: unlink f0 0 2024-04-25T01:35:38.878 INFO:tasks.workunit.client.0.smithi033.stdout:1/14: rename - no filename 2024-04-25T01:35:38.878 INFO:tasks.workunit.client.0.smithi033.stdout:1/15: chown . 31355841 1 2024-04-25T01:35:38.879 INFO:tasks.workunit.client.0.smithi033.stdout:3/25: getdents d8 0 2024-04-25T01:35:38.881 INFO:tasks.workunit.client.0.smithi033.stdout:3/26: creat d8/fa x:0 0 0 2024-04-25T01:35:38.886 INFO:tasks.workunit.client.0.smithi033.stdout:0/16: rename c3 to c5 0 2024-04-25T01:35:38.887 INFO:tasks.workunit.client.0.smithi033.stdout:3/27: dread f3 [0,4194304] 0 2024-04-25T01:35:38.887 INFO:tasks.workunit.client.0.smithi033.stdout:3/28: stat f3 0 2024-04-25T01:35:38.888 INFO:tasks.workunit.client.0.smithi033.stdout:3/29: dread - d8/fa zero size 2024-04-25T01:35:38.891 INFO:tasks.workunit.client.0.smithi033.stdout:0/17: dread f1 [0,4194304] 0 2024-04-25T01:35:38.897 INFO:tasks.workunit.client.0.smithi033.stdout:3/30: dwrite d8/fa [0,4194304] 0 2024-04-25T01:35:38.901 INFO:tasks.workunit.client.0.smithi033.stdout:0/18: dwrite f1 [0,4194304] 0 2024-04-25T01:35:38.902 INFO:tasks.workunit.client.0.smithi033.stdout:0/19: chown c4 79 1 2024-04-25T01:35:38.903 INFO:tasks.workunit.client.0.smithi033.stdout:9/28: fsync d0/f6 0 2024-04-25T01:35:38.906 INFO:tasks.workunit.client.0.smithi033.stdout:9/29: symlink d0/d3/l7 0 2024-04-25T01:35:38.909 INFO:tasks.workunit.client.0.smithi033.stdout:2/27: rmdir da 39 2024-04-25T01:35:38.910 INFO:tasks.workunit.client.0.smithi033.stdout:2/28: truncate f8 470297 0 2024-04-25T01:35:38.917 INFO:tasks.workunit.client.0.smithi033.stdout:8/44: fsync d6/db/fc 0 2024-04-25T01:35:38.921 INFO:tasks.workunit.client.0.smithi033.stdout:0/20: fdatasync f1 0 2024-04-25T01:35:38.922 INFO:tasks.workunit.client.0.smithi033.stdout:4/47: getdents d6 0 2024-04-25T01:35:38.923 INFO:tasks.workunit.client.0.smithi033.stdout:0/21: rename l0 to l6 0 2024-04-25T01:35:38.926 INFO:tasks.workunit.client.0.smithi033.stdout:4/48: rename d6/da to d6/d14 0 2024-04-25T01:35:38.927 INFO:tasks.workunit.client.0.smithi033.stdout:0/22: dread f1 [0,4194304] 0 2024-04-25T01:35:38.932 INFO:tasks.workunit.client.0.smithi033.stdout:0/23: unlink f1 0 2024-04-25T01:35:38.932 INFO:tasks.workunit.client.0.smithi033.stdout:0/24: dwrite - no filename 2024-04-25T01:35:38.932 INFO:tasks.workunit.client.0.smithi033.stdout:0/25: dread - no filename 2024-04-25T01:35:38.932 INFO:tasks.workunit.client.0.smithi033.stdout:8/45: rmdir d6/d10 39 2024-04-25T01:35:38.934 INFO:tasks.workunit.client.0.smithi033.stdout:5/23: write f1 [430720,112706] 0 2024-04-25T01:35:38.934 INFO:tasks.workunit.client.0.smithi033.stdout:8/46: dread d6/fa [0,4194304] 0 2024-04-25T01:35:38.935 INFO:tasks.workunit.client.0.smithi033.stdout:5/24: write f1 [1096444,97300] 0 2024-04-25T01:35:38.935 INFO:tasks.workunit.client.0.smithi033.stdout:5/25: stat c3 0 2024-04-25T01:35:38.936 INFO:tasks.workunit.client.0.smithi033.stdout:0/26: mkdir d7 0 2024-04-25T01:35:38.937 INFO:tasks.workunit.client.0.smithi033.stdout:5/26: fdatasync f1 0 2024-04-25T01:35:38.939 INFO:tasks.workunit.client.0.smithi033.stdout:1/16: getdents . 0 2024-04-25T01:35:38.942 INFO:tasks.workunit.client.0.smithi033.stdout:5/27: creat f5 x:0 0 0 2024-04-25T01:35:38.944 INFO:tasks.workunit.client.0.smithi033.stdout:3/31: write d8/fa [5089653,120689] 0 2024-04-25T01:35:38.946 INFO:tasks.workunit.client.0.smithi033.stdout:1/17: creat f1 x:0 0 0 2024-04-25T01:35:38.946 INFO:tasks.workunit.client.0.smithi033.stdout:1/18: write f1 [891172,42428] 0 2024-04-25T01:35:38.947 INFO:tasks.workunit.client.0.smithi033.stdout:5/28: symlink l6 0 2024-04-25T01:35:38.949 INFO:tasks.workunit.client.0.smithi033.stdout:0/27: unlink c5 0 2024-04-25T01:35:38.949 INFO:tasks.workunit.client.0.smithi033.stdout:0/28: write - no filename 2024-04-25T01:35:38.950 INFO:tasks.workunit.client.0.smithi033.stdout:8/47: creat d6/f13 x:0 0 0 2024-04-25T01:35:38.952 INFO:tasks.workunit.client.0.smithi033.stdout:3/32: write f3 [1981820,116270] 0 2024-04-25T01:35:38.952 INFO:tasks.workunit.client.0.smithi033.stdout:1/19: dread f1 [0,4194304] 0 2024-04-25T01:35:38.956 INFO:tasks.workunit.client.0.smithi033.stdout:5/29: rename f1 to f7 0 2024-04-25T01:35:38.957 INFO:tasks.workunit.client.0.smithi033.stdout:5/30: readlink l6 0 2024-04-25T01:35:38.961 INFO:tasks.workunit.client.0.smithi033.stdout:4/49: dwrite d6/d14/fd [0,4194304] 0 2024-04-25T01:35:38.962 INFO:tasks.workunit.client.0.smithi033.stdout:1/20: mkdir d2 0 2024-04-25T01:35:38.965 INFO:tasks.workunit.client.0.smithi033.stdout:0/29: mkdir d7/d8 0 2024-04-25T01:35:38.966 INFO:tasks.workunit.client.0.smithi033.stdout:5/31: mknod c8 0 2024-04-25T01:35:38.969 INFO:tasks.workunit.client.0.smithi033.stdout:8/48: symlink d6/d10/l14 0 2024-04-25T01:35:38.970 INFO:tasks.workunit.client.0.smithi033.stdout:4/50: stat d6/d14/c10 0 2024-04-25T01:35:38.972 INFO:tasks.workunit.client.0.smithi033.stdout:3/33: link l7 d8/lb 0 2024-04-25T01:35:38.972 INFO:tasks.workunit.client.0.smithi033.stdout:0/30: readlink l6 0 2024-04-25T01:35:38.972 INFO:tasks.workunit.client.0.smithi033.stdout:8/49: rmdir d6/d10 39 2024-04-25T01:35:38.975 INFO:tasks.workunit.client.0.smithi033.stdout:1/21: link f1 d2/f3 0 2024-04-25T01:35:38.976 INFO:tasks.workunit.client.0.smithi033.stdout:5/32: dwrite f5 [0,4194304] 0 2024-04-25T01:35:38.976 INFO:tasks.workunit.client.0.smithi033.stdout:3/34: write f3 [1600425,5970] 0 2024-04-25T01:35:38.986 INFO:tasks.workunit.client.0.smithi033.stdout:3/35: mknod d8/cc 0 2024-04-25T01:35:38.986 INFO:tasks.workunit.client.0.smithi033.stdout:4/51: dwrite d6/d12/f13 [0,4194304] 0 2024-04-25T01:35:38.992 INFO:tasks.workunit.client.0.smithi033.stdout:1/22: symlink d2/l4 0 2024-04-25T01:35:38.996 INFO:tasks.workunit.client.0.smithi033.stdout:3/36: mkdir d8/dd 0 2024-04-25T01:35:38.996 INFO:tasks.workunit.client.0.smithi033.stdout:4/52: dwrite d6/f9 [0,4194304] 0 2024-04-25T01:35:38.999 INFO:tasks.workunit.client.0.smithi033.stdout:0/31: rmdir d7/d8 0 2024-04-25T01:35:39.002 INFO:tasks.workunit.client.0.smithi033.stdout:8/50: creat d6/f15 x:0 0 0 2024-04-25T01:35:39.006 INFO:tasks.workunit.client.0.smithi033.stdout:1/23: mkdir d2/d5 0 2024-04-25T01:35:39.007 INFO:tasks.workunit.client.0.smithi033.stdout:5/33: write f5 [4269641,56710] 0 2024-04-25T01:35:39.008 INFO:tasks.workunit.client.0.smithi033.stdout:4/53: dwrite d6/f8 [0,4194304] 0 2024-04-25T01:35:39.015 INFO:tasks.workunit.client.0.smithi033.stdout:5/34: creat f9 x:0 0 0 2024-04-25T01:35:39.017 INFO:tasks.workunit.client.0.smithi033.stdout:8/51: dwrite d6/fe [0,4194304] 0 2024-04-25T01:35:39.021 INFO:tasks.workunit.client.0.smithi033.stdout:1/24: creat d2/d5/f6 x:0 0 0 2024-04-25T01:35:39.021 INFO:tasks.workunit.client.0.smithi033.stdout:1/25: stat f1 0 2024-04-25T01:35:39.021 INFO:tasks.workunit.client.0.smithi033.stdout:1/26: chown d2/d5 3137 1 2024-04-25T01:35:39.022 INFO:tasks.workunit.client.0.smithi033.stdout:5/35: mknod ca 0 2024-04-25T01:35:39.023 INFO:tasks.workunit.client.0.smithi033.stdout:5/36: rmdir - no directory 2024-04-25T01:35:39.024 INFO:tasks.workunit.client.0.smithi033.stdout:8/52: rmdir d6/db 39 2024-04-25T01:35:39.025 INFO:tasks.workunit.client.0.smithi033.stdout:3/37: link l7 d8/dd/le 0 2024-04-25T01:35:39.026 INFO:tasks.workunit.client.0.smithi033.stdout:3/38: fsync d8/fa 0 2024-04-25T01:35:39.026 INFO:tasks.workunit.client.0.smithi033.stdout:1/27: mknod d2/c7 0 2024-04-25T01:35:39.027 INFO:tasks.workunit.client.0.smithi033.stdout:5/37: fdatasync f7 0 2024-04-25T01:35:39.031 INFO:tasks.workunit.client.0.smithi033.stdout:8/53: symlink d6/l16 0 2024-04-25T01:35:39.035 INFO:tasks.workunit.client.0.smithi033.stdout:8/54: rename d6 to d6/db/d17 22 2024-04-25T01:35:39.037 INFO:tasks.workunit.client.0.smithi033.stdout:4/54: write d6/d14/ff [4594334,114693] 0 2024-04-25T01:35:39.038 INFO:tasks.workunit.client.0.smithi033.stdout:3/39: link d8/lb d8/lf 0 2024-04-25T01:35:39.038 INFO:tasks.workunit.client.0.smithi033.stdout:1/28: dwrite d2/d5/f6 [0,4194304] 0 2024-04-25T01:35:39.039 INFO:tasks.workunit.client.0.smithi033.stdout:5/38: link c2 cb 0 2024-04-25T01:35:39.041 INFO:tasks.workunit.client.0.smithi033.stdout:5/39: write f7 [718348,143] 0 2024-04-25T01:35:39.045 INFO:tasks.workunit.client.0.smithi033.stdout:3/40: readlink d8/lf 0 2024-04-25T01:35:39.049 INFO:tasks.workunit.client.0.smithi033.stdout:3/41: write d8/fa [829951,129737] 0 2024-04-25T01:35:39.050 INFO:tasks.workunit.client.0.smithi033.stdout:4/55: dread d6/d14/ff [0,4194304] 0 2024-04-25T01:35:39.050 INFO:tasks.workunit.client.0.smithi033.stdout:1/29: mknod d2/d5/c8 0 2024-04-25T01:35:39.055 INFO:tasks.workunit.client.0.smithi033.stdout:5/40: dwrite f9 [0,4194304] 0 2024-04-25T01:35:39.060 INFO:tasks.workunit.client.0.smithi033.stdout:3/42: truncate d8/fa 5045135 0 2024-04-25T01:35:39.061 INFO:tasks.workunit.client.0.smithi033.stdout:5/41: dread f9 [0,4194304] 0 2024-04-25T01:35:39.061 INFO:tasks.workunit.client.0.smithi033.stdout:1/30: getdents d2 0 2024-04-25T01:35:39.064 INFO:tasks.workunit.client.0.smithi033.stdout:1/31: write d2/d5/f6 [4443737,64675] 0 2024-04-25T01:35:39.065 INFO:tasks.workunit.client.0.smithi033.stdout:5/42: rename f9 to fc 0 2024-04-25T01:35:39.066 INFO:tasks.workunit.client.0.smithi033.stdout:3/43: dread f3 [0,4194304] 0 2024-04-25T01:35:39.067 INFO:tasks.workunit.client.0.smithi033.stdout:1/32: dread d2/f3 [0,4194304] 0 2024-04-25T01:35:39.072 INFO:tasks.workunit.client.0.smithi033.stdout:1/33: mknod d2/c9 0 2024-04-25T01:35:39.073 INFO:tasks.workunit.client.0.smithi033.stdout:3/44: write d8/fa [1603714,48538] 0 2024-04-25T01:35:39.074 INFO:tasks.workunit.client.0.smithi033.stdout:3/45: chown l4 2 1 2024-04-25T01:35:39.074 INFO:tasks.workunit.client.0.smithi033.stdout:3/46: chown l1 471697097 1 2024-04-25T01:35:39.075 INFO:tasks.workunit.client.0.smithi033.stdout:3/47: read d8/fa [540229,207] 0 2024-04-25T01:35:39.076 INFO:tasks.workunit.client.0.smithi033.stdout:1/34: rename d2/d5/f6 to d2/d5/fa 0 2024-04-25T01:35:39.078 INFO:tasks.workunit.client.0.smithi033.stdout:3/48: fdatasync d8/fa 0 2024-04-25T01:35:39.083 INFO:tasks.workunit.client.0.smithi033.stdout:3/49: rename l4 to d8/dd/l10 0 2024-04-25T01:35:39.085 INFO:tasks.workunit.client.0.smithi033.stdout:5/43: truncate f7 1094631 0 2024-04-25T01:35:39.087 INFO:tasks.workunit.client.0.smithi033.stdout:3/50: stat d8/lf 0 2024-04-25T01:35:39.088 INFO:tasks.workunit.client.0.smithi033.stdout:3/51: chown l1 1125989 1 2024-04-25T01:35:39.089 INFO:tasks.workunit.client.0.smithi033.stdout:1/35: link d2/c7 d2/cb 0 2024-04-25T01:35:39.089 INFO:tasks.workunit.client.0.smithi033.stdout:3/52: read f3 [1986703,52477] 0 2024-04-25T01:35:39.097 INFO:tasks.workunit.client.0.smithi033.stdout:5/44: truncate f5 1800113 0 2024-04-25T01:35:39.098 INFO:tasks.workunit.client.0.smithi033.stdout:3/53: dwrite f3 [0,4194304] 0 2024-04-25T01:35:39.107 INFO:tasks.workunit.client.0.smithi033.stdout:5/45: dwrite fc [0,4194304] 0 2024-04-25T01:35:39.107 INFO:tasks.workunit.client.0.smithi033.stdout:3/54: truncate f3 3049242 0 2024-04-25T01:35:39.108 INFO:tasks.workunit.client.0.smithi033.stdout:3/55: readlink l1 0 2024-04-25T01:35:39.108 INFO:tasks.workunit.client.0.smithi033.stdout:5/46: write fc [4690487,102085] 0 2024-04-25T01:35:39.111 INFO:tasks.workunit.client.0.smithi033.stdout:5/47: mknod cd 0 2024-04-25T01:35:39.112 INFO:tasks.workunit.client.0.smithi033.stdout:3/56: truncate d8/fa 4343032 0 2024-04-25T01:35:39.118 INFO:tasks.workunit.client.0.smithi033.stdout:5/48: dwrite fc [0,4194304] 0 2024-04-25T01:35:39.121 INFO:tasks.workunit.client.0.smithi033.stdout:5/49: mknod ce 0 2024-04-25T01:35:39.122 INFO:tasks.workunit.client.0.smithi033.stdout:3/57: write f3 [4009822,97716] 0 2024-04-25T01:35:39.122 INFO:tasks.workunit.client.0.smithi033.stdout:5/50: chown l6 18147568 1 2024-04-25T01:35:39.123 INFO:tasks.workunit.client.0.smithi033.stdout:5/51: chown ce 142 1 2024-04-25T01:35:39.125 INFO:tasks.workunit.client.0.smithi033.stdout:3/58: mkdir d8/dd/d11 0 2024-04-25T01:35:39.128 INFO:tasks.workunit.client.0.smithi033.stdout:5/52: read f5 [1433895,8327] 0 2024-04-25T01:35:39.129 INFO:tasks.workunit.client.0.smithi033.stdout:5/53: chown l6 14982 1 2024-04-25T01:35:39.129 INFO:tasks.workunit.client.0.smithi033.stdout:3/59: chown d8/dd/l10 115 1 2024-04-25T01:35:39.132 INFO:tasks.workunit.client.0.smithi033.stdout:5/54: fsync f7 0 2024-04-25T01:35:39.135 INFO:tasks.workunit.client.0.smithi033.stdout:5/55: creat ff x:0 0 0 2024-04-25T01:35:39.135 INFO:tasks.workunit.client.0.smithi033.stdout:5/56: chown ff 22304 1 2024-04-25T01:35:39.136 INFO:tasks.workunit.client.0.smithi033.stdout:5/57: mknod c10 0 2024-04-25T01:35:39.137 INFO:tasks.workunit.client.0.smithi033.stdout:5/58: dread - ff zero size 2024-04-25T01:35:39.137 INFO:tasks.workunit.client.0.smithi033.stdout:5/59: rmdir - no directory 2024-04-25T01:35:39.137 INFO:tasks.workunit.client.0.smithi033.stdout:5/60: rmdir - no directory 2024-04-25T01:35:39.139 INFO:tasks.workunit.client.0.smithi033.stdout:5/61: symlink l11 0 2024-04-25T01:35:39.141 INFO:tasks.workunit.client.0.smithi033.stdout:5/62: dread f5 [0,4194304] 0 2024-04-25T01:35:39.141 INFO:tasks.workunit.client.0.smithi033.stdout:5/63: write ff [228830,78128] 0 2024-04-25T01:35:39.145 INFO:tasks.workunit.client.0.smithi033.stdout:5/64: truncate f7 2104274 0 2024-04-25T01:35:39.150 INFO:tasks.workunit.client.0.smithi033.stdout:5/65: truncate ff 28707 0 2024-04-25T01:35:39.155 INFO:tasks.workunit.client.0.smithi033.stdout:5/66: write f7 [533703,11918] 0 2024-04-25T01:35:39.156 INFO:tasks.workunit.client.0.smithi033.stdout:5/67: write f7 [2539429,36729] 0 2024-04-25T01:35:39.160 INFO:tasks.workunit.client.0.smithi033.stdout:5/68: getdents . 0 2024-04-25T01:35:39.162 INFO:tasks.workunit.client.0.smithi033.stdout:5/69: mknod c12 0 2024-04-25T01:35:39.163 INFO:tasks.workunit.client.0.smithi033.stdout:5/70: read f5 [1890,112912] 0 2024-04-25T01:35:39.163 INFO:tasks.workunit.client.0.smithi033.stdout:5/71: fsync fc 0 2024-04-25T01:35:39.171 INFO:tasks.workunit.client.0.smithi033.stdout:5/72: getdents . 0 2024-04-25T01:35:39.173 INFO:tasks.workunit.client.0.smithi033.stdout:5/73: symlink l13 0 2024-04-25T01:35:39.176 INFO:tasks.workunit.client.0.smithi033.stdout:5/74: dread fc [0,4194304] 0 2024-04-25T01:35:39.179 INFO:tasks.workunit.client.0.smithi033.stdout:5/75: creat f14 x:0 0 0 2024-04-25T01:35:39.181 INFO:tasks.workunit.client.0.smithi033.stdout:5/76: creat f15 x:0 0 0 2024-04-25T01:35:39.182 INFO:tasks.workunit.client.0.smithi033.stdout:5/77: creat f16 x:0 0 0 2024-04-25T01:35:39.184 INFO:tasks.workunit.client.0.smithi033.stdout:5/78: creat f17 x:0 0 0 2024-04-25T01:35:39.184 INFO:tasks.workunit.client.0.smithi033.stdout:5/79: chown f5 1966098478 1 2024-04-25T01:35:39.185 INFO:tasks.workunit.client.0.smithi033.stdout:5/80: read - f17 zero size 2024-04-25T01:35:39.193 INFO:tasks.workunit.client.0.smithi033.stdout:5/81: getdents . 0 2024-04-25T01:35:39.195 INFO:tasks.workunit.client.0.smithi033.stdout:5/82: mknod c18 0 2024-04-25T01:35:39.201 INFO:tasks.workunit.client.0.smithi033.stdout:5/83: dwrite fc [0,4194304] 0 2024-04-25T01:35:39.203 INFO:tasks.workunit.client.0.smithi033.stdout:5/84: symlink l19 0 2024-04-25T01:35:39.207 INFO:tasks.workunit.client.0.smithi033.stdout:5/85: rename c2 to c1a 0 2024-04-25T01:35:39.211 INFO:tasks.workunit.client.0.smithi033.stdout:5/86: dread fc [0,4194304] 0 2024-04-25T01:35:39.211 INFO:tasks.workunit.client.0.smithi033.stdout:5/87: read - f15 zero size 2024-04-25T01:35:39.212 INFO:tasks.workunit.client.0.smithi033.stdout:5/88: fdatasync f7 0 2024-04-25T01:35:39.220 INFO:tasks.workunit.client.0.smithi033.stdout:5/89: truncate fc 4497774 0 2024-04-25T01:35:39.226 INFO:tasks.workunit.client.0.smithi033.stdout:5/90: dread fc [0,4194304] 0 2024-04-25T01:35:39.227 INFO:tasks.workunit.client.0.smithi033.stdout:5/91: write f15 [969041,81053] 0 2024-04-25T01:35:39.229 INFO:tasks.workunit.client.0.smithi033.stdout:5/92: dread ff [0,4194304] 0 2024-04-25T01:35:39.236 INFO:tasks.workunit.client.0.smithi033.stdout:5/93: dwrite f7 [0,4194304] 0 2024-04-25T01:35:39.238 INFO:tasks.workunit.client.0.smithi033.stdout:5/94: creat f1b x:0 0 0 2024-04-25T01:35:39.253 INFO:tasks.workunit.client.0.smithi033.stdout:5/95: dwrite f5 [0,4194304] 0 2024-04-25T01:35:39.253 INFO:tasks.workunit.client.0.smithi033.stdout:5/96: fsync f16 0 2024-04-25T01:35:39.256 INFO:tasks.workunit.client.0.smithi033.stdout:5/97: mkdir d1c 0 2024-04-25T01:35:39.258 INFO:tasks.workunit.client.0.smithi033.stdout:5/98: truncate ff 109995 0 2024-04-25T01:35:39.267 INFO:tasks.workunit.client.0.smithi033.stdout:5/99: dwrite f14 [0,4194304] 0 2024-04-25T01:35:39.269 INFO:tasks.workunit.client.0.smithi033.stdout:5/100: creat d1c/f1d x:0 0 0 2024-04-25T01:35:39.271 INFO:tasks.workunit.client.0.smithi033.stdout:5/101: creat d1c/f1e x:0 0 0 2024-04-25T01:35:39.279 INFO:tasks.workunit.client.0.smithi033.stdout:5/102: dwrite f5 [4194304,4194304] 0 2024-04-25T01:35:39.283 INFO:tasks.workunit.client.0.smithi033.stdout:5/103: creat d1c/f1f x:0 0 0 2024-04-25T01:35:39.285 INFO:tasks.workunit.client.0.smithi033.stdout:5/104: creat d1c/f20 x:0 0 0 2024-04-25T01:35:39.287 INFO:tasks.workunit.client.0.smithi033.stdout:5/105: mkdir d1c/d21 0 2024-04-25T01:35:39.287 INFO:tasks.workunit.client.0.smithi033.stdout:5/106: stat f5 0 2024-04-25T01:35:39.295 INFO:tasks.workunit.client.0.smithi033.stdout:5/107: dwrite f15 [0,4194304] 0 2024-04-25T01:35:39.295 INFO:tasks.workunit.client.0.smithi033.stdout:5/108: chown f15 409 1 2024-04-25T01:35:39.297 INFO:tasks.workunit.client.0.smithi033.stdout:5/109: creat d1c/f22 x:0 0 0 2024-04-25T01:35:39.299 INFO:tasks.workunit.client.0.smithi033.stdout:5/110: unlink c3 0 2024-04-25T01:35:39.300 INFO:tasks.workunit.client.0.smithi033.stdout:5/111: creat d1c/f23 x:0 0 0 2024-04-25T01:35:39.301 INFO:tasks.workunit.client.0.smithi033.stdout:5/112: stat f17 0 2024-04-25T01:35:39.302 INFO:tasks.workunit.client.0.smithi033.stdout:5/113: getdents d1c/d21 0 2024-04-25T01:35:39.304 INFO:tasks.workunit.client.0.smithi033.stdout:5/114: symlink d1c/l24 0 2024-04-25T01:35:39.314 INFO:tasks.workunit.client.0.smithi033.stdout:5/115: getdents d1c 0 2024-04-25T01:35:39.316 INFO:tasks.workunit.client.0.smithi033.stdout:5/116: symlink d1c/l25 0 2024-04-25T01:35:39.319 INFO:tasks.workunit.client.0.smithi033.stdout:5/117: dread ff [0,4194304] 0 2024-04-25T01:35:39.319 INFO:tasks.workunit.client.0.smithi033.stdout:5/118: chown f1b 74815 1 2024-04-25T01:35:39.320 INFO:tasks.workunit.client.0.smithi033.stdout:5/119: fdatasync f15 0 2024-04-25T01:35:39.321 INFO:tasks.workunit.client.0.smithi033.stdout:5/120: dread - d1c/f20 zero size 2024-04-25T01:35:39.321 INFO:tasks.workunit.client.0.smithi033.stdout:5/121: chown f7 6379 1 2024-04-25T01:35:39.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:39 smithi033 ceph-mon[26353]: pgmap v5: 65 pgs: 65 active+clean; 182 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:35:39.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:39 smithi033 ceph-mon[26353]: mgrmap e24: smithi138.mmfhyi(active, since 4s) 2024-04-25T01:35:39.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:39 smithi138 ceph-mon[32977]: pgmap v5: 65 pgs: 65 active+clean; 182 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:35:39.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:39 smithi138 ceph-mon[32977]: mgrmap e24: smithi138.mmfhyi(active, since 4s) 2024-04-25T01:35:40.236 INFO:tasks.workunit.client.0.smithi033.stdout:6/3: sync 2024-04-25T01:35:40.236 INFO:tasks.workunit.client.0.smithi033.stdout:6/4: write - no filename 2024-04-25T01:35:40.236 INFO:tasks.workunit.client.0.smithi033.stdout:6/5: fdatasync - no filename 2024-04-25T01:35:40.236 INFO:tasks.workunit.client.0.smithi033.stdout:6/6: truncate - no filename 2024-04-25T01:35:40.238 INFO:tasks.workunit.client.0.smithi033.stdout:6/7: creat f1 x:0 0 0 2024-04-25T01:35:40.245 INFO:tasks.workunit.client.0.smithi033.stdout:6/8: dwrite f1 [0,4194304] 0 2024-04-25T01:35:40.247 INFO:tasks.workunit.client.0.smithi033.stdout:6/9: symlink l2 0 2024-04-25T01:35:40.249 INFO:tasks.workunit.client.0.smithi033.stdout:6/10: creat f3 x:0 0 0 2024-04-25T01:35:40.348 INFO:tasks.workunit.client.0.smithi033.stdout:6/11: truncate f1 1981885 0 2024-04-25T01:35:40.350 INFO:tasks.workunit.client.0.smithi033.stdout:6/12: mkdir d4 0 2024-04-25T01:35:40.354 INFO:tasks.workunit.client.0.smithi033.stdout:6/13: dread f1 [0,4194304] 0 2024-04-25T01:35:40.357 INFO:tasks.workunit.client.0.smithi033.stdout:6/14: creat d4/f5 x:0 0 0 2024-04-25T01:35:40.357 INFO:tasks.workunit.client.0.smithi033.stdout:6/15: stat l2 0 2024-04-25T01:35:40.358 INFO:tasks.workunit.client.0.smithi033.stdout:6/16: read f1 [1335481,113234] 0 2024-04-25T01:35:40.359 INFO:tasks.workunit.client.0.smithi033.stdout:7/7: sync 2024-04-25T01:35:40.359 INFO:tasks.workunit.client.0.smithi033.stdout:8/55: sync 2024-04-25T01:35:40.359 INFO:tasks.workunit.client.0.smithi033.stdout:2/29: sync 2024-04-25T01:35:40.359 INFO:tasks.workunit.client.0.smithi033.stdout:4/56: sync 2024-04-25T01:35:40.359 INFO:tasks.workunit.client.0.smithi033.stdout:0/32: sync 2024-04-25T01:35:40.359 INFO:tasks.workunit.client.0.smithi033.stdout:0/33: fsync - no filename 2024-04-25T01:35:40.359 INFO:tasks.workunit.client.0.smithi033.stdout:0/34: write - no filename 2024-04-25T01:35:40.359 INFO:tasks.workunit.client.0.smithi033.stdout:9/30: sync 2024-04-25T01:35:40.359 INFO:tasks.workunit.client.0.smithi033.stdout:5/122: sync 2024-04-25T01:35:40.360 INFO:tasks.workunit.client.0.smithi033.stdout:3/60: sync 2024-04-25T01:35:40.360 INFO:tasks.workunit.client.0.smithi033.stdout:1/36: sync 2024-04-25T01:35:40.361 INFO:tasks.workunit.client.0.smithi033.stdout:2/30: fsync da/ff 0 2024-04-25T01:35:40.362 INFO:tasks.workunit.client.0.smithi033.stdout:6/17: truncate d4/f5 562097 0 2024-04-25T01:35:40.362 INFO:tasks.workunit.client.0.smithi033.stdout:9/31: write d0/f6 [1033120,125360] 0 2024-04-25T01:35:40.362 INFO:tasks.workunit.client.0.smithi033.stdout:3/61: rmdir d8 39 2024-04-25T01:35:40.363 INFO:tasks.workunit.client.0.smithi033.stdout:3/62: stat f3 0 2024-04-25T01:35:40.364 INFO:tasks.workunit.client.0.smithi033.stdout:3/63: write f3 [2228388,104344] 0 2024-04-25T01:35:40.365 INFO:tasks.workunit.client.0.smithi033.stdout:8/56: creat d6/db/f18 x:0 0 0 2024-04-25T01:35:40.366 INFO:tasks.workunit.client.0.smithi033.stdout:8/57: read - d6/f9 zero size 2024-04-25T01:35:40.369 INFO:tasks.workunit.client.0.smithi033.stdout:2/31: rmdir da 39 2024-04-25T01:35:40.369 INFO:tasks.workunit.client.0.smithi033.stdout:2/32: readlink - no filename 2024-04-25T01:35:40.371 INFO:tasks.workunit.client.0.smithi033.stdout:2/33: truncate f8 632662 0 2024-04-25T01:35:40.372 INFO:tasks.workunit.client.0.smithi033.stdout:9/32: unlink d0/f5 0 2024-04-25T01:35:40.378 INFO:tasks.workunit.client.0.smithi033.stdout:9/33: write d0/f6 [1909447,38646] 0 2024-04-25T01:35:40.379 INFO:tasks.workunit.client.0.smithi033.stdout:1/37: rename d2/c7 to d2/d5/cc 0 2024-04-25T01:35:40.380 INFO:tasks.workunit.client.0.smithi033.stdout:8/58: dwrite d6/f9 [0,4194304] 0 2024-04-25T01:35:40.382 INFO:tasks.workunit.client.0.smithi033.stdout:9/34: chown d0/d3 310 1 2024-04-25T01:35:40.385 INFO:tasks.workunit.client.0.smithi033.stdout:6/18: rename d4/f5 to d4/f6 0 2024-04-25T01:35:40.385 INFO:tasks.workunit.client.0.smithi033.stdout:4/57: dwrite d6/d14/ff [0,4194304] 0 2024-04-25T01:35:40.387 INFO:tasks.workunit.client.0.smithi033.stdout:5/123: dwrite ff [0,4194304] 0 2024-04-25T01:35:40.388 INFO:tasks.workunit.client.0.smithi033.stdout:9/35: chown d0/f6 2926090 1 2024-04-25T01:35:40.388 INFO:tasks.workunit.client.0.smithi033.stdout:3/64: rmdir d8 39 2024-04-25T01:35:40.389 INFO:tasks.workunit.client.0.smithi033.stdout:2/34: truncate da/fd 987456 0 2024-04-25T01:35:40.390 INFO:tasks.workunit.client.0.smithi033.stdout:9/36: truncate d0/f6 2910011 0 2024-04-25T01:35:40.391 INFO:tasks.workunit.client.0.smithi033.stdout:9/37: stat d0/d3 0 2024-04-25T01:35:40.392 INFO:tasks.workunit.client.0.smithi033.stdout:1/38: creat d2/d5/fd x:0 0 0 2024-04-25T01:35:40.396 INFO:tasks.workunit.client.0.smithi033.stdout:6/19: creat d4/f7 x:0 0 0 2024-04-25T01:35:40.401 INFO:tasks.workunit.client.0.smithi033.stdout:4/58: rename d6/d14/ce to d6/c15 0 2024-04-25T01:35:40.405 INFO:tasks.workunit.client.0.smithi033.stdout:2/35: creat da/f10 x:0 0 0 2024-04-25T01:35:40.406 INFO:tasks.workunit.client.0.smithi033.stdout:9/38: creat d0/f8 x:0 0 0 2024-04-25T01:35:40.407 INFO:tasks.workunit.client.0.smithi033.stdout:1/39: truncate d2/f3 653297 0 2024-04-25T01:35:40.410 INFO:tasks.workunit.client.0.smithi033.stdout:6/20: unlink f1 0 2024-04-25T01:35:40.412 INFO:tasks.workunit.client.0.smithi033.stdout:4/59: dwrite d6/f8 [4194304,4194304] 0 2024-04-25T01:35:40.413 INFO:tasks.workunit.client.0.smithi033.stdout:5/124: creat d1c/d21/f26 x:0 0 0 2024-04-25T01:35:40.414 INFO:tasks.workunit.client.0.smithi033.stdout:4/60: chown d6/d14/ff 0 1 2024-04-25T01:35:40.415 INFO:tasks.workunit.client.0.smithi033.stdout:9/39: creat d0/f9 x:0 0 0 2024-04-25T01:35:40.416 INFO:tasks.workunit.client.0.smithi033.stdout:2/36: write f8 [66092,102094] 0 2024-04-25T01:35:40.418 INFO:tasks.workunit.client.0.smithi033.stdout:1/40: creat d2/d5/fe x:0 0 0 2024-04-25T01:35:40.419 INFO:tasks.workunit.client.0.smithi033.stdout:4/61: write d6/d14/fd [4721728,56000] 0 2024-04-25T01:35:40.421 INFO:tasks.workunit.client.0.smithi033.stdout:8/59: link d6/ld d6/l19 0 2024-04-25T01:35:40.423 INFO:tasks.workunit.client.0.smithi033.stdout:2/37: fsync f9 0 2024-04-25T01:35:40.424 INFO:tasks.workunit.client.0.smithi033.stdout:9/40: mknod d0/d3/ca 0 2024-04-25T01:35:40.425 INFO:tasks.workunit.client.0.smithi033.stdout:6/21: dwrite d4/f7 [0,4194304] 0 2024-04-25T01:35:40.426 INFO:tasks.workunit.client.0.smithi033.stdout:2/38: write f1 [1867475,130306] 0 2024-04-25T01:35:40.431 INFO:tasks.workunit.client.0.smithi033.stdout:3/65: getdents d8 0 2024-04-25T01:35:40.438 INFO:tasks.workunit.client.0.smithi033.stdout:3/66: dread f3 [0,4194304] 0 2024-04-25T01:35:40.439 INFO:tasks.workunit.client.0.smithi033.stdout:6/22: mknod d4/c8 0 2024-04-25T01:35:40.439 INFO:tasks.workunit.client.0.smithi033.stdout:5/125: fsync d1c/d21/f26 0 2024-04-25T01:35:40.441 INFO:tasks.workunit.client.0.smithi033.stdout:9/41: creat d0/d3/fb x:0 0 0 2024-04-25T01:35:40.441 INFO:tasks.workunit.client.0.smithi033.stdout:5/126: dread - d1c/d21/f26 zero size 2024-04-25T01:35:40.442 INFO:tasks.workunit.client.0.smithi033.stdout:2/39: creat da/f11 x:0 0 0 2024-04-25T01:35:40.443 INFO:tasks.workunit.client.0.smithi033.stdout:2/40: chown da/f11 114700230 1 2024-04-25T01:35:40.446 INFO:tasks.workunit.client.0.smithi033.stdout:2/41: read f7 [671794,102094] 0 2024-04-25T01:35:40.447 INFO:tasks.workunit.client.0.smithi033.stdout:3/67: dread d8/fa [0,4194304] 0 2024-04-25T01:35:40.448 INFO:tasks.workunit.client.0.smithi033.stdout:8/60: unlink d6/d10/f12 0 2024-04-25T01:35:40.452 INFO:tasks.workunit.client.0.smithi033.stdout:6/23: write d4/f7 [4984575,26353] 0 2024-04-25T01:35:40.453 INFO:tasks.workunit.client.0.smithi033.stdout:9/42: creat d0/fc x:0 0 0 2024-04-25T01:35:40.455 INFO:tasks.workunit.client.0.smithi033.stdout:1/41: chown f1 3655 1 2024-04-25T01:35:40.458 INFO:tasks.workunit.client.0.smithi033.stdout:8/61: dread d6/db/fc [0,4194304] 0 2024-04-25T01:35:40.459 INFO:tasks.workunit.client.0.smithi033.stdout:5/127: dwrite f17 [0,4194304] 0 2024-04-25T01:35:40.460 INFO:tasks.workunit.client.0.smithi033.stdout:2/42: creat da/f12 x:0 0 0 2024-04-25T01:35:40.462 INFO:tasks.workunit.client.0.smithi033.stdout:2/43: chown da/ff 1700 1 2024-04-25T01:35:40.464 INFO:tasks.workunit.client.0.smithi033.stdout:3/68: dwrite f3 [0,4194304] 0 2024-04-25T01:35:40.464 INFO:tasks.workunit.client.0.smithi033.stdout:4/62: getdents d6 0 2024-04-25T01:35:40.469 INFO:tasks.workunit.client.0.smithi033.stdout:9/43: rename d0/d3 to d0/d2/dd 0 2024-04-25T01:35:40.471 INFO:tasks.workunit.client.0.smithi033.stdout:8/62: creat d6/db/f1a x:0 0 0 2024-04-25T01:35:40.471 INFO:tasks.workunit.client.0.smithi033.stdout:9/44: chown d0/f6 65308 1 2024-04-25T01:35:40.472 INFO:tasks.workunit.client.0.smithi033.stdout:8/63: read f5 [779775,88341] 0 2024-04-25T01:35:40.474 INFO:tasks.workunit.client.0.smithi033.stdout:1/42: dwrite d2/d5/fa [0,4194304] 0 2024-04-25T01:35:40.494 INFO:tasks.workunit.client.0.smithi033.stdout:8/64: dwrite d6/fe [0,4194304] 0 2024-04-25T01:35:40.495 INFO:tasks.workunit.client.0.smithi033.stdout:8/65: write d6/f13 [975920,88372] 0 2024-04-25T01:35:40.496 INFO:tasks.workunit.client.0.smithi033.stdout:8/66: read d6/fe [571118,1208] 0 2024-04-25T01:35:40.597 INFO:tasks.workunit.client.0.smithi033.stdout:6/24: link d4/f7 d4/f9 0 2024-04-25T01:35:40.598 INFO:tasks.workunit.client.0.smithi033.stdout:3/69: truncate d8/fa 2182474 0 2024-04-25T01:35:40.598 INFO:tasks.workunit.client.0.smithi033.stdout:6/25: chown d4/f6 6242523 1 2024-04-25T01:35:40.599 INFO:tasks.workunit.client.0.smithi033.stdout:5/128: rename l6 to d1c/d21/l27 0 2024-04-25T01:35:40.662 INFO:tasks.workunit.client.0.smithi033.stdout:0/35: sync 2024-04-25T01:35:40.663 INFO:tasks.workunit.client.0.smithi033.stdout:7/8: sync 2024-04-25T01:35:40.663 INFO:tasks.workunit.client.0.smithi033.stdout:7/9: rmdir - no directory 2024-04-25T01:35:40.663 INFO:tasks.workunit.client.0.smithi033.stdout:0/36: chown l6 0 1 2024-04-25T01:35:40.819 INFO:tasks.workunit.client.0.smithi033.stdout:3/70: mknod d8/c12 0 2024-04-25T01:35:40.820 INFO:tasks.workunit.client.0.smithi033.stdout:6/26: symlink d4/la 0 2024-04-25T01:35:40.823 INFO:tasks.workunit.client.0.smithi033.stdout:5/129: symlink d1c/l28 0 2024-04-25T01:35:40.825 INFO:tasks.workunit.client.0.smithi033.stdout:7/10: creat f3 x:0 0 0 2024-04-25T01:35:40.825 INFO:tasks.workunit.client.0.smithi033.stdout:7/11: chown f1 27548983 1 2024-04-25T01:35:40.826 INFO:tasks.workunit.client.0.smithi033.stdout:7/12: chown f3 1036425124 1 2024-04-25T01:35:40.827 INFO:tasks.workunit.client.0.smithi033.stdout:7/13: write f0 [190785,50268] 0 2024-04-25T01:35:40.827 INFO:tasks.workunit.client.0.smithi033.stdout:3/71: mknod d8/c13 0 2024-04-25T01:35:40.827 INFO:tasks.workunit.client.0.smithi033.stdout:7/14: chown f1 7144965 1 2024-04-25T01:35:40.830 INFO:tasks.workunit.client.0.smithi033.stdout:7/15: dread f1 [0,4194304] 0 2024-04-25T01:35:40.830 INFO:tasks.workunit.client.0.smithi033.stdout:5/130: rmdir d1c 39 2024-04-25T01:35:40.832 INFO:tasks.workunit.client.0.smithi033.stdout:9/45: getdents d0/d2 0 2024-04-25T01:35:40.833 INFO:tasks.workunit.client.0.smithi033.stdout:2/44: sync 2024-04-25T01:35:40.833 INFO:tasks.workunit.client.0.smithi033.stdout:2/45: fsync da/f12 0 2024-04-25T01:35:40.834 INFO:tasks.workunit.client.0.smithi033.stdout:2/46: dread - da/fe zero size 2024-04-25T01:35:40.837 INFO:tasks.workunit.client.0.smithi033.stdout:8/67: rename l3 to d6/l1b 0 2024-04-25T01:35:40.839 INFO:tasks.workunit.client.0.smithi033.stdout:5/131: dread - d1c/f1d zero size 2024-04-25T01:35:40.840 INFO:tasks.workunit.client.0.smithi033.stdout:5/132: fdatasync d1c/f20 0 2024-04-25T01:35:40.840 INFO:tasks.workunit.client.0.smithi033.stdout:7/16: mkdir d4 0 2024-04-25T01:35:40.842 INFO:tasks.workunit.client.0.smithi033.stdout:1/43: getdents d2 0 2024-04-25T01:35:40.846 INFO:tasks.workunit.client.0.smithi033.stdout:2/47: creat da/f13 x:0 0 0 2024-04-25T01:35:40.847 INFO:tasks.workunit.client.0.smithi033.stdout:8/68: dwrite d6/f15 [0,4194304] 0 2024-04-25T01:35:40.849 INFO:tasks.workunit.client.0.smithi033.stdout:7/17: dread f0 [0,4194304] 0 2024-04-25T01:35:40.850 INFO:tasks.workunit.client.0.smithi033.stdout:7/18: stat f0 0 2024-04-25T01:35:40.853 INFO:tasks.workunit.client.0.smithi033.stdout:5/133: dwrite f7 [0,4194304] 0 2024-04-25T01:35:40.855 INFO:tasks.workunit.client.0.smithi033.stdout:2/48: truncate f6 1081138 0 2024-04-25T01:35:40.857 INFO:tasks.workunit.client.0.smithi033.stdout:2/49: write da/f13 [159971,65740] 0 2024-04-25T01:35:40.859 INFO:tasks.workunit.client.0.smithi033.stdout:8/69: rename d6/fa to d6/db/f1c 0 2024-04-25T01:35:40.860 INFO:tasks.workunit.client.0.smithi033.stdout:8/70: write d6/f9 [3230140,121326] 0 2024-04-25T01:35:40.861 INFO:tasks.workunit.client.0.smithi033.stdout:5/134: rmdir d1c 39 2024-04-25T01:35:40.861 INFO:tasks.workunit.client.0.smithi033.stdout:5/135: fdatasync f1b 0 2024-04-25T01:35:40.864 INFO:tasks.workunit.client.0.smithi033.stdout:2/50: symlink da/l14 0 2024-04-25T01:35:40.868 INFO:tasks.workunit.client.0.smithi033.stdout:7/19: link f3 d4/f5 0 2024-04-25T01:35:40.871 INFO:tasks.workunit.client.0.smithi033.stdout:5/136: write d1c/d21/f26 [834495,116367] 0 2024-04-25T01:35:40.872 INFO:tasks.workunit.client.0.smithi033.stdout:2/51: creat da/f15 x:0 0 0 2024-04-25T01:35:40.873 INFO:tasks.workunit.client.0.smithi033.stdout:2/52: chown c3 16672605 1 2024-04-25T01:35:40.874 INFO:tasks.workunit.client.0.smithi033.stdout:7/20: write f0 [794091,24115] 0 2024-04-25T01:35:40.874 INFO:tasks.workunit.client.0.smithi033.stdout:7/21: chown d4 240066517 1 2024-04-25T01:35:40.874 INFO:tasks.workunit.client.0.smithi033.stdout:2/53: truncate da/fd 1511963 0 2024-04-25T01:35:40.877 INFO:tasks.workunit.client.0.smithi033.stdout:7/22: write f3 [553810,98091] 0 2024-04-25T01:35:40.881 INFO:tasks.workunit.client.0.smithi033.stdout:5/137: link ff d1c/d21/f29 0 2024-04-25T01:35:40.882 INFO:tasks.workunit.client.0.smithi033.stdout:5/138: write f5 [6246002,52817] 0 2024-04-25T01:35:40.955 INFO:tasks.workunit.client.0.smithi033.stdout:5/139: getdents d1c 0 2024-04-25T01:35:40.961 INFO:tasks.workunit.client.0.smithi033.stdout:7/23: truncate f1 1075255 0 2024-04-25T01:35:40.962 INFO:tasks.workunit.client.0.smithi033.stdout:5/140: dwrite fc [0,4194304] 0 2024-04-25T01:35:40.963 INFO:tasks.workunit.client.0.smithi033.stdout:5/141: chown l13 6289 1 2024-04-25T01:35:40.963 INFO:tasks.workunit.client.0.smithi033.stdout:9/46: truncate d0/f6 1899447 0 2024-04-25T01:35:40.964 INFO:tasks.workunit.client.0.smithi033.stdout:5/142: chown c8 763557861 1 2024-04-25T01:35:40.964 INFO:tasks.workunit.client.0.smithi033.stdout:9/47: dread - d0/d2/dd/fb zero size 2024-04-25T01:35:40.965 INFO:tasks.workunit.client.0.smithi033.stdout:5/143: dread - d1c/f23 zero size 2024-04-25T01:35:40.969 INFO:tasks.workunit.client.0.smithi033.stdout:5/144: dread ff [0,4194304] 0 2024-04-25T01:35:40.970 INFO:tasks.workunit.client.0.smithi033.stdout:9/48: creat d0/d2/dd/fe x:0 0 0 2024-04-25T01:35:40.972 INFO:tasks.workunit.client.0.smithi033.stdout:5/145: mkdir d1c/d2a 0 2024-04-25T01:35:40.974 INFO:tasks.workunit.client.0.smithi033.stdout:9/49: mkdir d0/d2/dd/df 0 2024-04-25T01:35:40.975 INFO:tasks.workunit.client.0.smithi033.stdout:2/54: getdents da 0 2024-04-25T01:35:40.976 INFO:tasks.workunit.client.0.smithi033.stdout:8/71: truncate d6/db/f1c 498501 0 2024-04-25T01:35:40.976 INFO:tasks.workunit.client.0.smithi033.stdout:9/50: write d0/d2/dd/fe [689343,127845] 0 2024-04-25T01:35:40.978 INFO:tasks.workunit.client.0.smithi033.stdout:2/55: symlink da/l16 0 2024-04-25T01:35:40.978 INFO:tasks.workunit.client.0.smithi033.stdout:2/56: chown f8 72 1 2024-04-25T01:35:40.979 INFO:tasks.workunit.client.0.smithi033.stdout:8/72: readlink d6/ld 0 2024-04-25T01:35:40.980 INFO:tasks.workunit.client.0.smithi033.stdout:2/57: read f8 [54380,74544] 0 2024-04-25T01:35:40.981 INFO:tasks.workunit.client.0.smithi033.stdout:5/146: link f16 d1c/f2b 0 2024-04-25T01:35:40.984 INFO:tasks.workunit.client.0.smithi033.stdout:8/73: dread d6/f9 [0,4194304] 0 2024-04-25T01:35:40.987 INFO:tasks.workunit.client.0.smithi033.stdout:2/58: chown f6 1545235 1 2024-04-25T01:35:40.995 INFO:tasks.workunit.client.0.smithi033.stdout:2/59: dwrite da/f11 [0,4194304] 0 2024-04-25T01:35:40.996 INFO:tasks.workunit.client.0.smithi033.stdout:2/60: dread - da/fc zero size 2024-04-25T01:35:40.997 INFO:tasks.workunit.client.0.smithi033.stdout:3/72: write d8/fa [2309678,126510] 0 2024-04-25T01:35:40.999 INFO:tasks.workunit.client.0.smithi033.stdout:3/73: write d8/fa [250704,43730] 0 2024-04-25T01:35:41.000 INFO:tasks.workunit.client.0.smithi033.stdout:9/51: read d0/f6 [680503,38501] 0 2024-04-25T01:35:41.000 INFO:tasks.workunit.client.0.smithi033.stdout:3/74: read f3 [1936169,12740] 0 2024-04-25T01:35:41.000 INFO:tasks.workunit.client.0.smithi033.stdout:7/24: truncate d4/f5 438556 0 2024-04-25T01:35:41.001 INFO:tasks.workunit.client.0.smithi033.stdout:2/61: rmdir da 39 2024-04-25T01:35:41.001 INFO:tasks.workunit.client.0.smithi033.stdout:7/25: fsync f0 0 2024-04-25T01:35:41.001 INFO:tasks.workunit.client.0.smithi033.stdout:9/52: readlink d0/d2/dd/l4 0 2024-04-25T01:35:41.003 INFO:tasks.workunit.client.0.smithi033.stdout:0/37: sync 2024-04-25T01:35:41.003 INFO:tasks.workunit.client.0.smithi033.stdout:4/63: sync 2024-04-25T01:35:41.008 INFO:tasks.workunit.client.0.smithi033.stdout:4/64: dread d6/d14/fd [0,4194304] 0 2024-04-25T01:35:41.010 INFO:tasks.workunit.client.0.smithi033.stdout:2/62: write f8 [240166,93607] 0 2024-04-25T01:35:41.016 INFO:tasks.workunit.client.0.smithi033.stdout:9/53: symlink d0/d2/l10 0 2024-04-25T01:35:41.018 INFO:tasks.workunit.client.0.smithi033.stdout:0/38: unlink c4 0 2024-04-25T01:35:41.018 INFO:tasks.workunit.client.0.smithi033.stdout:0/39: chown l6 6237 1 2024-04-25T01:35:41.018 INFO:tasks.workunit.client.0.smithi033.stdout:0/40: dwrite - no filename 2024-04-25T01:35:41.019 INFO:tasks.workunit.client.0.smithi033.stdout:5/147: write ff [63685,3495] 0 2024-04-25T01:35:41.024 INFO:tasks.workunit.client.0.smithi033.stdout:3/75: mknod d8/dd/d11/c14 0 2024-04-25T01:35:41.026 INFO:tasks.workunit.client.0.smithi033.stdout:9/54: mkdir d0/d11 0 2024-04-25T01:35:41.031 INFO:tasks.workunit.client.0.smithi033.stdout:5/148: creat d1c/d21/f2c x:0 0 0 2024-04-25T01:35:41.033 INFO:tasks.workunit.client.0.smithi033.stdout:3/76: creat d8/dd/f15 x:0 0 0 2024-04-25T01:35:41.035 INFO:tasks.workunit.client.0.smithi033.stdout:9/55: mknod d0/c12 0 2024-04-25T01:35:41.036 INFO:tasks.workunit.client.0.smithi033.stdout:7/26: fsync d4/f5 0 2024-04-25T01:35:41.037 INFO:tasks.workunit.client.0.smithi033.stdout:7/27: truncate d4/f5 707708 0 2024-04-25T01:35:41.039 INFO:tasks.workunit.client.0.smithi033.stdout:5/149: symlink d1c/d21/l2d 0 2024-04-25T01:35:41.041 INFO:tasks.workunit.client.0.smithi033.stdout:3/77: write f3 [1318170,38024] 0 2024-04-25T01:35:41.043 INFO:tasks.workunit.client.0.smithi033.stdout:0/41: link l6 d7/l9 0 2024-04-25T01:35:41.047 INFO:tasks.workunit.client.0.smithi033.stdout:9/56: getdents d0/d11 0 2024-04-25T01:35:41.051 INFO:tasks.workunit.client.0.smithi033.stdout:7/28: symlink d4/l6 0 2024-04-25T01:35:41.052 INFO:tasks.workunit.client.0.smithi033.stdout:5/150: dwrite d1c/f1e [0,4194304] 0 2024-04-25T01:35:41.056 INFO:tasks.workunit.client.0.smithi033.stdout:2/63: getdents da 0 2024-04-25T01:35:41.064 INFO:tasks.workunit.client.0.smithi033.stdout:0/42: chown l6 2457 1 2024-04-25T01:35:41.064 INFO:tasks.workunit.client.0.smithi033.stdout:0/43: dread - no filename 2024-04-25T01:35:41.065 INFO:tasks.workunit.client.0.smithi033.stdout:9/57: dwrite d0/d2/dd/fe [0,4194304] 0 2024-04-25T01:35:41.065 INFO:tasks.workunit.client.0.smithi033.stdout:4/65: truncate d6/d14/ff 1672145 0 2024-04-25T01:35:41.069 INFO:tasks.workunit.client.0.smithi033.stdout:2/64: dwrite da/f11 [0,4194304] 0 2024-04-25T01:35:41.070 INFO:tasks.workunit.client.0.smithi033.stdout:2/65: write da/f12 [439784,32713] 0 2024-04-25T01:35:41.070 INFO:tasks.workunit.client.0.smithi033.stdout:0/44: mkdir d7/da 0 2024-04-25T01:35:41.072 INFO:tasks.workunit.client.0.smithi033.stdout:4/66: creat d6/f16 x:0 0 0 2024-04-25T01:35:41.075 INFO:tasks.workunit.client.0.smithi033.stdout:2/66: dread f8 [0,4194304] 0 2024-04-25T01:35:41.075 INFO:tasks.workunit.client.0.smithi033.stdout:5/151: link d1c/f20 d1c/f2e 0 2024-04-25T01:35:41.076 INFO:tasks.workunit.client.0.smithi033.stdout:2/67: chown da/f12 93028 1 2024-04-25T01:35:41.076 INFO:tasks.workunit.client.0.smithi033.stdout:5/152: dread - d1c/f2e zero size 2024-04-25T01:35:41.077 INFO:tasks.workunit.client.0.smithi033.stdout:5/153: dread - d1c/f1f zero size 2024-04-25T01:35:41.077 INFO:tasks.workunit.client.0.smithi033.stdout:2/68: write da/f11 [992188,75795] 0 2024-04-25T01:35:41.079 INFO:tasks.workunit.client.0.smithi033.stdout:3/78: dwrite f3 [0,4194304] 0 2024-04-25T01:35:41.081 INFO:tasks.workunit.client.0.smithi033.stdout:4/67: creat d6/d12/f17 x:0 0 0 2024-04-25T01:35:41.083 INFO:tasks.workunit.client.0.smithi033.stdout:7/29: truncate f3 1334337 0 2024-04-25T01:35:41.086 INFO:tasks.workunit.client.0.smithi033.stdout:9/58: link d0/f8 d0/d2/dd/f13 0 2024-04-25T01:35:41.087 INFO:tasks.workunit.client.0.smithi033.stdout:9/59: write d0/f9 [42841,33090] 0 2024-04-25T01:35:41.089 INFO:tasks.workunit.client.0.smithi033.stdout:2/69: creat da/f17 x:0 0 0 2024-04-25T01:35:41.091 INFO:tasks.workunit.client.0.smithi033.stdout:0/45: creat d7/da/fb x:0 0 0 2024-04-25T01:35:41.093 INFO:tasks.workunit.client.0.smithi033.stdout:0/46: truncate d7/da/fb 637869 0 2024-04-25T01:35:41.094 INFO:tasks.workunit.client.0.smithi033.stdout:0/47: write d7/da/fb [302731,42575] 0 2024-04-25T01:35:41.096 INFO:tasks.workunit.client.0.smithi033.stdout:9/60: rename d0/d2/dd/fb to d0/d11/f14 0 2024-04-25T01:35:41.097 INFO:tasks.workunit.client.0.smithi033.stdout:2/70: fsync f9 0 2024-04-25T01:35:41.106 INFO:tasks.workunit.client.0.smithi033.stdout:0/48: mknod d7/da/cc 0 2024-04-25T01:35:41.108 INFO:tasks.workunit.client.0.smithi033.stdout:0/49: write d7/da/fb [274161,11436] 0 2024-04-25T01:35:41.109 INFO:tasks.workunit.client.0.smithi033.stdout:9/61: write d0/d2/dd/f13 [417112,63474] 0 2024-04-25T01:35:41.110 INFO:tasks.workunit.client.0.smithi033.stdout:7/30: fsync d4/f5 0 2024-04-25T01:35:41.116 INFO:tasks.workunit.client.0.smithi033.stdout:0/50: rename d7/da/cc to d7/cd 0 2024-04-25T01:35:41.121 INFO:tasks.workunit.client.0.smithi033.stdout:9/62: unlink d0/d11/f14 0 2024-04-25T01:35:41.123 INFO:tasks.workunit.client.0.smithi033.stdout:7/31: dread f1 [0,4194304] 0 2024-04-25T01:35:41.127 INFO:tasks.workunit.client.0.smithi033.stdout:0/51: dwrite d7/da/fb [0,4194304] 0 2024-04-25T01:35:41.127 INFO:tasks.workunit.client.0.smithi033.stdout:3/79: truncate f3 2087743 0 2024-04-25T01:35:41.129 INFO:tasks.workunit.client.0.smithi033.stdout:9/63: creat d0/d2/dd/f15 x:0 0 0 2024-04-25T01:35:41.130 INFO:tasks.workunit.client.0.smithi033.stdout:7/32: creat d4/f7 x:0 0 0 2024-04-25T01:35:41.135 INFO:tasks.workunit.client.0.smithi033.stdout:2/71: getdents da 0 2024-04-25T01:35:41.135 INFO:tasks.workunit.client.0.smithi033.stdout:7/33: chown f0 28253 1 2024-04-25T01:35:41.135 INFO:tasks.workunit.client.0.smithi033.stdout:9/64: chown d0/d2/dd/df 318866934 1 2024-04-25T01:35:41.136 INFO:tasks.workunit.client.0.smithi033.stdout:2/72: chown f7 1519 1 2024-04-25T01:35:41.137 INFO:tasks.workunit.client.0.smithi033.stdout:4/68: dwrite d6/f9 [0,4194304] 0 2024-04-25T01:35:41.142 INFO:tasks.workunit.client.0.smithi033.stdout:7/34: creat d4/f8 x:0 0 0 2024-04-25T01:35:41.145 INFO:tasks.workunit.client.0.smithi033.stdout:0/52: getdents d7/da 0 2024-04-25T01:35:41.146 INFO:tasks.workunit.client.0.smithi033.stdout:3/80: dwrite d8/dd/f15 [0,4194304] 0 2024-04-25T01:35:41.147 INFO:tasks.workunit.client.0.smithi033.stdout:4/69: creat d6/f18 x:0 0 0 2024-04-25T01:35:41.151 INFO:tasks.workunit.client.0.smithi033.stdout:0/53: readlink l6 0 2024-04-25T01:35:41.152 INFO:tasks.workunit.client.0.smithi033.stdout:7/35: rename d4/f8 to d4/f9 0 2024-04-25T01:35:41.155 INFO:tasks.workunit.client.0.smithi033.stdout:4/70: mknod d6/d12/c19 0 2024-04-25T01:35:41.158 INFO:tasks.workunit.client.0.smithi033.stdout:7/36: chown d4/f7 4004287 1 2024-04-25T01:35:41.158 INFO:tasks.workunit.client.0.smithi033.stdout:9/65: dwrite d0/fc [0,4194304] 0 2024-04-25T01:35:41.161 INFO:tasks.workunit.client.0.smithi033.stdout:2/73: dwrite da/f10 [0,4194304] 0 2024-04-25T01:35:41.161 INFO:tasks.workunit.client.0.smithi033.stdout:4/71: write d6/f16 [765494,40597] 0 2024-04-25T01:35:41.167 INFO:tasks.workunit.client.0.smithi033.stdout:7/37: fdatasync f1 0 2024-04-25T01:35:41.168 INFO:tasks.workunit.client.0.smithi033.stdout:0/54: rename d7/da/fb to d7/fe 0 2024-04-25T01:35:41.169 INFO:tasks.workunit.client.0.smithi033.stdout:9/66: creat d0/d11/f16 x:0 0 0 2024-04-25T01:35:41.172 INFO:tasks.workunit.client.0.smithi033.stdout:2/74: rename da/f10 to da/f18 0 2024-04-25T01:35:41.173 INFO:tasks.workunit.client.0.smithi033.stdout:3/81: link d8/cc d8/c16 0 2024-04-25T01:35:41.174 INFO:tasks.workunit.client.0.smithi033.stdout:3/82: chown d8/dd/l10 124507 1 2024-04-25T01:35:41.177 INFO:tasks.workunit.client.0.smithi033.stdout:1/44: sync 2024-04-25T01:35:41.177 INFO:tasks.workunit.client.0.smithi033.stdout:0/55: mkdir d7/da/df 0 2024-04-25T01:35:41.179 INFO:tasks.workunit.client.0.smithi033.stdout:9/67: creat d0/d2/dd/f17 x:0 0 0 2024-04-25T01:35:41.180 INFO:tasks.workunit.client.0.smithi033.stdout:9/68: fsync d0/f9 0 2024-04-25T01:35:41.180 INFO:tasks.workunit.client.0.smithi033.stdout:6/27: sync 2024-04-25T01:35:41.183 INFO:tasks.workunit.client.0.smithi033.stdout:3/83: mknod d8/c17 0 2024-04-25T01:35:41.184 INFO:tasks.workunit.client.0.smithi033.stdout:0/56: rmdir d7/da 39 2024-04-25T01:35:41.185 INFO:tasks.workunit.client.0.smithi033.stdout:9/69: creat d0/d11/f18 x:0 0 0 2024-04-25T01:35:41.186 INFO:tasks.workunit.client.0.smithi033.stdout:6/28: mknod d4/cb 0 2024-04-25T01:35:41.190 INFO:tasks.workunit.client.0.smithi033.stdout:9/70: dread d0/d2/dd/fe [0,4194304] 0 2024-04-25T01:35:41.191 INFO:tasks.workunit.client.0.smithi033.stdout:1/45: fdatasync d2/f3 0 2024-04-25T01:35:41.194 INFO:tasks.workunit.client.0.smithi033.stdout:3/84: rename d8/c13 to d8/dd/d11/c18 0 2024-04-25T01:35:41.196 INFO:tasks.workunit.client.0.smithi033.stdout:4/72: write d6/d14/fd [388867,900] 0 2024-04-25T01:35:41.196 INFO:tasks.workunit.client.0.smithi033.stdout:0/57: unlink d7/l9 0 2024-04-25T01:35:41.198 INFO:tasks.workunit.client.0.smithi033.stdout:4/73: chown d6/d14 1229362 1 2024-04-25T01:35:41.200 INFO:tasks.workunit.client.0.smithi033.stdout:6/29: mknod d4/cc 0 2024-04-25T01:35:41.206 INFO:tasks.workunit.client.0.smithi033.stdout:4/74: truncate d6/f16 1404972 0 2024-04-25T01:35:41.208 INFO:tasks.workunit.client.0.smithi033.stdout:0/58: write d7/fe [3707869,49499] 0 2024-04-25T01:35:41.209 INFO:tasks.workunit.client.0.smithi033.stdout:9/71: dwrite d0/d2/dd/f13 [0,4194304] 0 2024-04-25T01:35:41.209 INFO:tasks.workunit.client.0.smithi033.stdout:6/30: read d4/f6 [556404,1048] 0 2024-04-25T01:35:41.212 INFO:tasks.workunit.client.0.smithi033.stdout:2/75: truncate f0 1093394 0 2024-04-25T01:35:41.214 INFO:tasks.workunit.client.0.smithi033.stdout:1/46: dwrite d2/d5/fd [0,4194304] 0 2024-04-25T01:35:41.219 INFO:tasks.workunit.client.0.smithi033.stdout:4/75: creat d6/d12/f1a x:0 0 0 2024-04-25T01:35:41.220 INFO:tasks.workunit.client.0.smithi033.stdout:9/72: symlink d0/d2/l19 0 2024-04-25T01:35:41.223 INFO:tasks.workunit.client.0.smithi033.stdout:7/38: dwrite f3 [0,4194304] 0 2024-04-25T01:35:41.225 INFO:tasks.workunit.client.0.smithi033.stdout:2/76: mkdir da/d19 0 2024-04-25T01:35:41.228 INFO:tasks.workunit.client.0.smithi033.stdout:2/77: truncate da/fc 8706 0 2024-04-25T01:35:41.229 INFO:tasks.workunit.client.0.smithi033.stdout:3/85: rename d8/dd/d11/c18 to d8/dd/d11/c19 0 2024-04-25T01:35:41.233 INFO:tasks.workunit.client.0.smithi033.stdout:9/73: symlink d0/d2/l1a 0 2024-04-25T01:35:41.234 INFO:tasks.workunit.client.0.smithi033.stdout:7/39: mknod d4/ca 0 2024-04-25T01:35:41.235 INFO:tasks.workunit.client.0.smithi033.stdout:2/78: mknod da/c1a 0 2024-04-25T01:35:41.236 INFO:tasks.workunit.client.0.smithi033.stdout:3/86: rename c5 to d8/c1a 0 2024-04-25T01:35:41.238 INFO:tasks.workunit.client.0.smithi033.stdout:3/87: read f3 [1929372,49437] 0 2024-04-25T01:35:41.238 INFO:tasks.workunit.client.0.smithi033.stdout:0/59: creat d7/da/df/f10 x:0 0 0 2024-04-25T01:35:41.239 INFO:tasks.workunit.client.0.smithi033.stdout:9/74: rmdir d0/d2 39 2024-04-25T01:35:41.240 INFO:tasks.workunit.client.0.smithi033.stdout:2/79: write da/f17 [918068,20687] 0 2024-04-25T01:35:41.240 INFO:tasks.workunit.client.0.smithi033.stdout:4/76: dwrite d6/d14/ff [0,4194304] 0 2024-04-25T01:35:41.241 INFO:tasks.workunit.client.0.smithi033.stdout:3/88: fsync d8/fa 0 2024-04-25T01:35:41.243 INFO:tasks.workunit.client.0.smithi033.stdout:6/31: getdents d4 0 2024-04-25T01:35:41.248 INFO:tasks.workunit.client.0.smithi033.stdout:2/80: rmdir da 39 2024-04-25T01:35:41.249 INFO:tasks.workunit.client.0.smithi033.stdout:9/75: chown d0/d2/dd/fe 13118 1 2024-04-25T01:35:41.251 INFO:tasks.workunit.client.0.smithi033.stdout:7/40: dwrite d4/f5 [0,4194304] 0 2024-04-25T01:35:41.253 INFO:tasks.workunit.client.0.smithi033.stdout:4/77: dread d6/d12/f13 [0,4194304] 0 2024-04-25T01:35:41.253 INFO:tasks.workunit.client.0.smithi033.stdout:6/32: symlink d4/ld 0 2024-04-25T01:35:41.257 INFO:tasks.workunit.client.0.smithi033.stdout:1/47: write f1 [892533,23405] 0 2024-04-25T01:35:41.259 INFO:tasks.workunit.client.0.smithi033.stdout:1/48: stat d2/d5/fe 0 2024-04-25T01:35:41.261 INFO:tasks.workunit.client.0.smithi033.stdout:2/81: write da/fd [257885,67329] 0 2024-04-25T01:35:41.267 INFO:tasks.workunit.client.0.smithi033.stdout:7/41: rmdir d4 39 2024-04-25T01:35:41.269 INFO:tasks.workunit.client.0.smithi033.stdout:8/74: sync 2024-04-25T01:35:41.269 INFO:tasks.workunit.client.0.smithi033.stdout:6/33: symlink d4/le 0 2024-04-25T01:35:41.269 INFO:tasks.workunit.client.0.smithi033.stdout:4/78: dwrite d6/f9 [0,4194304] 0 2024-04-25T01:35:41.270 INFO:tasks.workunit.client.0.smithi033.stdout:1/49: dread d2/d5/fd [0,4194304] 0 2024-04-25T01:35:41.272 INFO:tasks.workunit.client.0.smithi033.stdout:4/79: read d6/d14/fd [4303364,103134] 0 2024-04-25T01:35:41.273 INFO:tasks.workunit.client.0.smithi033.stdout:4/80: dread - d6/d12/f17 zero size 2024-04-25T01:35:41.275 INFO:tasks.workunit.client.0.smithi033.stdout:1/50: dread f1 [0,4194304] 0 2024-04-25T01:35:41.275 INFO:tasks.workunit.client.0.smithi033.stdout:4/81: fsync d6/d12/f17 0 2024-04-25T01:35:41.276 INFO:tasks.workunit.client.0.smithi033.stdout:4/82: readlink d6/d14/lc 0 2024-04-25T01:35:41.282 INFO:tasks.workunit.client.0.smithi033.stdout:9/76: mkdir d0/d2/dd/df/d1b 0 2024-04-25T01:35:41.284 INFO:tasks.workunit.client.0.smithi033.stdout:9/77: chown d0/c1 2 1 2024-04-25T01:35:41.286 INFO:tasks.workunit.client.0.smithi033.stdout:1/51: dwrite d2/d5/fe [0,4194304] 0 2024-04-25T01:35:41.288 INFO:tasks.workunit.client.0.smithi033.stdout:6/34: symlink d4/lf 0 2024-04-25T01:35:41.290 INFO:tasks.workunit.client.0.smithi033.stdout:6/35: write f3 [601441,102197] 0 2024-04-25T01:35:41.292 INFO:tasks.workunit.client.0.smithi033.stdout:3/89: getdents d8/dd 0 2024-04-25T01:35:41.293 INFO:tasks.workunit.client.0.smithi033.stdout:4/83: symlink d6/l1b 0 2024-04-25T01:35:41.294 INFO:tasks.workunit.client.0.smithi033.stdout:0/60: getdents d7/da/df 0 2024-04-25T01:35:41.295 INFO:tasks.workunit.client.0.smithi033.stdout:4/84: write d6/d12/f1a [977755,68928] 0 2024-04-25T01:35:41.298 INFO:tasks.workunit.client.0.smithi033.stdout:7/42: creat d4/fb x:0 0 0 2024-04-25T01:35:41.300 INFO:tasks.workunit.client.0.smithi033.stdout:7/43: readlink d4/l6 0 2024-04-25T01:35:41.302 INFO:tasks.workunit.client.0.smithi033.stdout:3/90: write f3 [2615204,8374] 0 2024-04-25T01:35:41.303 INFO:tasks.workunit.client.0.smithi033.stdout:2/82: rmdir da/d19 0 2024-04-25T01:35:41.303 INFO:tasks.workunit.client.0.smithi033.stdout:6/36: dread f3 [0,4194304] 0 2024-04-25T01:35:41.307 INFO:tasks.workunit.client.0.smithi033.stdout:0/61: symlink d7/da/l11 0 2024-04-25T01:35:41.310 INFO:tasks.workunit.client.0.smithi033.stdout:8/75: creat d6/f1d x:0 0 0 2024-04-25T01:35:41.311 INFO:tasks.workunit.client.0.smithi033.stdout:4/85: dread d6/d12/f1a [0,4194304] 0 2024-04-25T01:35:41.311 INFO:tasks.workunit.client.0.smithi033.stdout:3/91: creat d8/f1b x:0 0 0 2024-04-25T01:35:41.312 INFO:tasks.workunit.client.0.smithi033.stdout:6/37: dread d4/f7 [0,4194304] 0 2024-04-25T01:35:41.313 INFO:tasks.workunit.client.0.smithi033.stdout:2/83: mknod da/c1b 0 2024-04-25T01:35:41.318 INFO:tasks.workunit.client.0.smithi033.stdout:0/62: symlink d7/da/df/l12 0 2024-04-25T01:35:41.323 INFO:tasks.workunit.client.0.smithi033.stdout:2/84: mknod da/c1c 0 2024-04-25T01:35:41.325 INFO:tasks.workunit.client.0.smithi033.stdout:1/52: truncate f1 672622 0 2024-04-25T01:35:41.326 INFO:tasks.workunit.client.0.smithi033.stdout:0/63: dread d7/fe [0,4194304] 0 2024-04-25T01:35:41.326 INFO:tasks.workunit.client.0.smithi033.stdout:6/38: rename d4/le to d4/l10 0 2024-04-25T01:35:41.328 INFO:tasks.workunit.client.0.smithi033.stdout:9/78: getdents d0/d11 0 2024-04-25T01:35:41.328 INFO:tasks.workunit.client.0.smithi033.stdout:0/64: read - d7/da/df/f10 zero size 2024-04-25T01:35:41.329 INFO:tasks.workunit.client.0.smithi033.stdout:4/86: dwrite d6/d12/f13 [0,4194304] 0 2024-04-25T01:35:41.331 INFO:tasks.workunit.client.0.smithi033.stdout:4/87: chown d6/f8 89284070 1 2024-04-25T01:35:41.332 INFO:tasks.workunit.client.0.smithi033.stdout:6/39: dread f3 [0,4194304] 0 2024-04-25T01:35:41.333 INFO:tasks.workunit.client.0.smithi033.stdout:2/85: symlink da/l1d 0 2024-04-25T01:35:41.334 INFO:tasks.workunit.client.0.smithi033.stdout:9/79: write d0/d2/dd/f13 [1004719,65886] 0 2024-04-25T01:35:41.335 INFO:tasks.workunit.client.0.smithi033.stdout:1/53: unlink d2/d5/c8 0 2024-04-25T01:35:41.337 INFO:tasks.workunit.client.0.smithi033.stdout:1/54: chown d2/d5/fe 69 1 2024-04-25T01:35:41.337 INFO:tasks.workunit.client.0.smithi033.stdout:0/65: creat d7/da/f13 x:0 0 0 2024-04-25T01:35:41.338 INFO:tasks.workunit.client.0.smithi033.stdout:8/76: creat d6/f1e x:0 0 0 2024-04-25T01:35:41.342 INFO:tasks.workunit.client.0.smithi033.stdout:9/80: dread - d0/d2/dd/f17 zero size 2024-04-25T01:35:41.342 INFO:tasks.workunit.client.0.smithi033.stdout:4/88: mknod d6/d14/c1c 0 2024-04-25T01:35:41.342 INFO:tasks.workunit.client.0.smithi033.stdout:8/77: chown d6/f15 3655122 1 2024-04-25T01:35:41.343 INFO:tasks.workunit.client.0.smithi033.stdout:0/66: mkdir d7/da/df/d14 0 2024-04-25T01:35:41.344 INFO:tasks.workunit.client.0.smithi033.stdout:6/40: write f3 [1314095,110136] 0 2024-04-25T01:35:41.345 INFO:tasks.workunit.client.0.smithi033.stdout:6/41: stat d4/ld 0 2024-04-25T01:35:41.348 INFO:tasks.workunit.client.0.smithi033.stdout:2/86: dwrite da/f17 [0,4194304] 0 2024-04-25T01:35:41.349 INFO:tasks.workunit.client.0.smithi033.stdout:4/89: mkdir d6/d14/d1d 0 2024-04-25T01:35:41.349 INFO:tasks.workunit.client.0.smithi033.stdout:6/42: truncate f3 1436424 0 2024-04-25T01:35:41.352 INFO:tasks.workunit.client.0.smithi033.stdout:0/67: chown l6 626389365 1 2024-04-25T01:35:41.353 INFO:tasks.workunit.client.0.smithi033.stdout:1/55: rename f1 to d2/ff 0 2024-04-25T01:35:41.357 INFO:tasks.workunit.client.0.smithi033.stdout:8/78: creat d6/d10/f1f x:0 0 0 2024-04-25T01:35:41.357 INFO:tasks.workunit.client.0.smithi033.stdout:8/79: dread - d6/f1e zero size 2024-04-25T01:35:41.358 INFO:tasks.workunit.client.0.smithi033.stdout:9/81: mknod d0/c1c 0 2024-04-25T01:35:41.360 INFO:tasks.workunit.client.0.smithi033.stdout:4/90: creat d6/f1e x:0 0 0 2024-04-25T01:35:41.361 INFO:tasks.workunit.client.0.smithi033.stdout:6/43: rename d4/f7 to d4/f11 0 2024-04-25T01:35:41.367 INFO:tasks.workunit.client.0.smithi033.stdout:1/56: unlink d2/c9 0 2024-04-25T01:35:41.369 INFO:tasks.workunit.client.0.smithi033.stdout:8/80: symlink d6/db/l20 0 2024-04-25T01:35:41.371 INFO:tasks.workunit.client.0.smithi033.stdout:9/82: symlink d0/d2/dd/df/l1d 0 2024-04-25T01:35:41.373 INFO:tasks.workunit.client.0.smithi033.stdout:4/91: dwrite d6/f1e [0,4194304] 0 2024-04-25T01:35:41.375 INFO:tasks.workunit.client.0.smithi033.stdout:0/68: creat d7/da/df/d14/f15 x:0 0 0 2024-04-25T01:35:41.377 INFO:tasks.workunit.client.0.smithi033.stdout:8/81: dread d6/f15 [0,4194304] 0 2024-04-25T01:35:41.383 INFO:tasks.workunit.client.0.smithi033.stdout:9/83: symlink d0/d2/dd/l1e 0 2024-04-25T01:35:41.384 INFO:tasks.workunit.client.0.smithi033.stdout:4/92: creat d6/f1f x:0 0 0 2024-04-25T01:35:41.385 INFO:tasks.workunit.client.0.smithi033.stdout:0/69: creat d7/da/df/d14/f16 x:0 0 0 2024-04-25T01:35:41.395 INFO:tasks.workunit.client.0.smithi033.stdout:4/93: dread d6/f16 [0,4194304] 0 2024-04-25T01:35:41.396 INFO:tasks.workunit.client.0.smithi033.stdout:9/84: link d0/f9 d0/d2/f1f 0 2024-04-25T01:35:41.398 INFO:tasks.workunit.client.0.smithi033.stdout:8/82: creat d6/f21 x:0 0 0 2024-04-25T01:35:41.400 INFO:tasks.workunit.client.0.smithi033.stdout:1/57: truncate d2/d5/fa 1503303 0 2024-04-25T01:35:41.400 INFO:tasks.workunit.client.0.smithi033.stdout:9/85: stat d0/d11/f18 0 2024-04-25T01:35:41.402 INFO:tasks.workunit.client.0.smithi033.stdout:4/94: truncate d6/f18 468899 0 2024-04-25T01:35:41.403 INFO:tasks.workunit.client.0.smithi033.stdout:6/44: dwrite d4/f6 [0,4194304] 0 2024-04-25T01:35:41.406 INFO:tasks.workunit.client.0.smithi033.stdout:0/70: dwrite d7/da/df/f10 [0,4194304] 0 2024-04-25T01:35:41.412 INFO:tasks.workunit.client.0.smithi033.stdout:4/95: write d6/d12/f17 [1002437,41295] 0 2024-04-25T01:35:41.418 INFO:tasks.workunit.client.0.smithi033.stdout:4/96: chown d6/f8 65332 1 2024-04-25T01:35:41.422 INFO:tasks.workunit.client.0.smithi033.stdout:0/71: dread - d7/da/df/d14/f15 zero size 2024-04-25T01:35:41.425 INFO:tasks.workunit.client.0.smithi033.stdout:8/83: dwrite d6/d10/f1f [0,4194304] 0 2024-04-25T01:35:41.425 INFO:tasks.workunit.client.0.smithi033.stdout:1/58: write d2/d5/fa [711834,85694] 0 2024-04-25T01:35:41.431 INFO:tasks.workunit.client.0.smithi033.stdout:8/84: fsync d6/f8 0 2024-04-25T01:35:41.432 INFO:tasks.workunit.client.0.smithi033.stdout:1/59: read d2/d5/fd [540903,29206] 0 2024-04-25T01:35:41.433 INFO:tasks.workunit.client.0.smithi033.stdout:0/72: creat d7/da/df/f17 x:0 0 0 2024-04-25T01:35:41.436 INFO:tasks.workunit.client.0.smithi033.stdout:6/45: dwrite d4/f6 [0,4194304] 0 2024-04-25T01:35:41.436 INFO:tasks.workunit.client.0.smithi033.stdout:8/85: chown d6/l19 7441 1 2024-04-25T01:35:41.437 INFO:tasks.workunit.client.0.smithi033.stdout:1/60: mknod d2/c10 0 2024-04-25T01:35:41.440 INFO:tasks.workunit.client.0.smithi033.stdout:0/73: chown d7/da/df/d14/f15 1083937 1 2024-04-25T01:35:41.440 INFO:tasks.workunit.client.0.smithi033.stdout:0/74: chown d7 0 1 2024-04-25T01:35:41.441 INFO:tasks.workunit.client.0.smithi033.stdout:4/97: dwrite d6/d14/ff [0,4194304] 0 2024-04-25T01:35:41.442 INFO:tasks.workunit.client.0.smithi033.stdout:1/61: mkdir d2/d11 0 2024-04-25T01:35:41.445 INFO:tasks.workunit.client.0.smithi033.stdout:8/86: rmdir d6 39 2024-04-25T01:35:41.445 INFO:tasks.workunit.client.0.smithi033.stdout:9/86: dwrite d0/f9 [0,4194304] 0 2024-04-25T01:35:41.445 INFO:tasks.workunit.client.0.smithi033.stdout:1/62: chown d2/d5/fe 58133 1 2024-04-25T01:35:41.445 INFO:tasks.workunit.client.0.smithi033.stdout:4/98: chown d6/d14/c10 113 1 2024-04-25T01:35:41.446 INFO:tasks.workunit.client.0.smithi033.stdout:0/75: fdatasync d7/fe 0 2024-04-25T01:35:41.449 INFO:tasks.workunit.client.0.smithi033.stdout:6/46: dread d4/f9 [0,4194304] 0 2024-04-25T01:35:41.452 INFO:tasks.workunit.client.0.smithi033.stdout:8/87: dread - d6/f1d zero size 2024-04-25T01:35:41.454 INFO:tasks.workunit.client.0.smithi033.stdout:1/63: dread d2/f3 [0,4194304] 0 2024-04-25T01:35:41.454 INFO:tasks.workunit.client.0.smithi033.stdout:8/88: chown d6/l19 7 1 2024-04-25T01:35:41.460 INFO:tasks.workunit.client.0.smithi033.stdout:1/64: rmdir d2 39 2024-04-25T01:35:41.461 INFO:tasks.workunit.client.0.smithi033.stdout:8/89: write d6/db/fc [680495,25839] 0 2024-04-25T01:35:41.461 INFO:tasks.workunit.client.0.smithi033.stdout:4/99: write d6/d14/fd [5433371,43868] 0 2024-04-25T01:35:41.465 INFO:tasks.workunit.client.0.smithi033.stdout:8/90: readlink d6/l11 0 2024-04-25T01:35:41.468 INFO:tasks.workunit.client.0.smithi033.stdout:6/47: dwrite f3 [0,4194304] 0 2024-04-25T01:35:41.468 INFO:tasks.workunit.client.0.smithi033.stdout:5/154: sync 2024-04-25T01:35:41.469 INFO:tasks.workunit.client.0.smithi033.stdout:4/100: unlink d6/d12/f13 0 2024-04-25T01:35:41.474 INFO:tasks.workunit.client.0.smithi033.stdout:4/101: chown d6/c11 17 1 2024-04-25T01:35:41.475 INFO:tasks.workunit.client.0.smithi033.stdout:5/155: truncate d1c/d21/f2c 194849 0 2024-04-25T01:35:41.476 INFO:tasks.workunit.client.0.smithi033.stdout:8/91: mknod d6/db/c22 0 2024-04-25T01:35:41.476 INFO:tasks.workunit.client.0.smithi033.stdout:6/48: creat d4/f12 x:0 0 0 2024-04-25T01:35:41.478 INFO:tasks.workunit.client.0.smithi033.stdout:6/49: fsync d4/f6 0 2024-04-25T01:35:41.479 INFO:tasks.workunit.client.0.smithi033.stdout:5/156: symlink d1c/d21/l2f 0 2024-04-25T01:35:41.481 INFO:tasks.workunit.client.0.smithi033.stdout:8/92: mkdir d6/d23 0 2024-04-25T01:35:41.482 INFO:tasks.workunit.client.0.smithi033.stdout:6/50: symlink d4/l13 0 2024-04-25T01:35:41.482 INFO:tasks.workunit.client.0.smithi033.stdout:8/93: readlink d6/ld 0 2024-04-25T01:35:41.483 INFO:tasks.workunit.client.0.smithi033.stdout:1/65: dwrite d2/d5/fe [0,4194304] 0 2024-04-25T01:35:41.484 INFO:tasks.workunit.client.0.smithi033.stdout:6/51: stat d4/la 0 2024-04-25T01:35:41.485 INFO:tasks.workunit.client.0.smithi033.stdout:1/66: chown d2/d11 26641 1 2024-04-25T01:35:41.486 INFO:tasks.workunit.client.0.smithi033.stdout:5/157: link c10 d1c/d2a/c30 0 2024-04-25T01:35:41.490 INFO:tasks.workunit.client.0.smithi033.stdout:8/94: rename d6/d10/f1f to d6/db/f24 0 2024-04-25T01:35:41.508 INFO:tasks.workunit.client.0.smithi033.stdout:4/102: dwrite d6/d12/f1a [0,4194304] 0 2024-04-25T01:35:41.514 INFO:tasks.workunit.client.0.smithi033.stdout:4/103: chown d6/d14/c10 732419 1 2024-04-25T01:35:41.516 INFO:tasks.workunit.client.0.smithi033.stdout:1/67: dwrite d2/d5/fa [0,4194304] 0 2024-04-25T01:35:41.517 INFO:tasks.workunit.client.0.smithi033.stdout:5/158: dwrite f7 [0,4194304] 0 2024-04-25T01:35:41.521 INFO:tasks.workunit.client.0.smithi033.stdout:1/68: dread d2/ff [0,4194304] 0 2024-04-25T01:35:41.530 INFO:tasks.workunit.client.0.smithi033.stdout:1/69: dread d2/d5/fd [0,4194304] 0 2024-04-25T01:35:41.643 INFO:tasks.workunit.client.0.smithi033.stdout:6/52: link d4/f9 d4/f14 0 2024-04-25T01:35:41.644 INFO:tasks.workunit.client.0.smithi033.stdout:6/53: truncate d4/f9 5958241 0 2024-04-25T01:35:41.644 INFO:tasks.workunit.client.0.smithi033.stdout:8/95: unlink d6/db/cf 0 2024-04-25T01:35:41.647 INFO:tasks.workunit.client.0.smithi033.stdout:6/54: read f3 [1017275,97799] 0 2024-04-25T01:35:41.647 INFO:tasks.workunit.client.0.smithi033.stdout:5/159: rmdir d1c/d2a 39 2024-04-25T01:35:41.648 INFO:tasks.workunit.client.0.smithi033.stdout:8/96: fdatasync d6/db/fc 0 2024-04-25T01:35:41.648 INFO:tasks.workunit.client.0.smithi033.stdout:4/104: getdents d6/d14/d1d 0 2024-04-25T01:35:41.650 INFO:tasks.workunit.client.0.smithi033.stdout:4/105: write d6/f8 [1126555,10193] 0 2024-04-25T01:35:41.653 INFO:tasks.workunit.client.0.smithi033.stdout:6/55: mkdir d4/d15 0 2024-04-25T01:35:41.656 INFO:tasks.workunit.client.0.smithi033.stdout:8/97: symlink d6/db/l25 0 2024-04-25T01:35:41.657 INFO:tasks.workunit.client.0.smithi033.stdout:1/70: link d2/ff d2/f12 0 2024-04-25T01:35:41.660 INFO:tasks.workunit.client.0.smithi033.stdout:4/106: dwrite d6/d12/f1a [0,4194304] 0 2024-04-25T01:35:41.661 INFO:tasks.workunit.client.0.smithi033.stdout:6/56: write f3 [2606466,88059] 0 2024-04-25T01:35:41.664 INFO:tasks.workunit.client.0.smithi033.stdout:5/160: symlink d1c/l31 0 2024-04-25T01:35:41.665 INFO:tasks.workunit.client.0.smithi033.stdout:5/161: dread - d1c/f2b zero size 2024-04-25T01:35:41.671 INFO:tasks.workunit.client.0.smithi033.stdout:6/57: rename d4/c8 to d4/d15/c16 0 2024-04-25T01:35:41.675 INFO:tasks.workunit.client.0.smithi033.stdout:1/71: creat d2/d11/f13 x:0 0 0 2024-04-25T01:35:41.677 INFO:tasks.workunit.client.0.smithi033.stdout:8/98: link d6/f21 d6/db/f26 0 2024-04-25T01:35:41.677 INFO:tasks.workunit.client.0.smithi033.stdout:8/99: chown d6/l19 512209284 1 2024-04-25T01:35:41.678 INFO:tasks.workunit.client.0.smithi033.stdout:4/107: creat d6/d14/d1d/f20 x:0 0 0 2024-04-25T01:35:41.680 INFO:tasks.workunit.client.0.smithi033.stdout:6/58: symlink d4/d15/l17 0 2024-04-25T01:35:41.683 INFO:tasks.workunit.client.0.smithi033.stdout:1/72: creat d2/d5/f14 x:0 0 0 2024-04-25T01:35:41.684 INFO:tasks.workunit.client.0.smithi033.stdout:4/108: symlink d6/d12/l21 0 2024-04-25T01:35:41.686 INFO:tasks.workunit.client.0.smithi033.stdout:8/100: dwrite d6/f1e [0,4194304] 0 2024-04-25T01:35:41.687 INFO:tasks.workunit.client.0.smithi033.stdout:5/162: creat d1c/f32 x:0 0 0 2024-04-25T01:35:41.687 INFO:tasks.workunit.client.0.smithi033.stdout:1/73: truncate d2/d11/f13 973725 0 2024-04-25T01:35:41.688 INFO:tasks.workunit.client.0.smithi033.stdout:6/59: creat d4/f18 x:0 0 0 2024-04-25T01:35:41.689 INFO:tasks.workunit.client.0.smithi033.stdout:5/163: write d1c/f32 [385222,24909] 0 2024-04-25T01:35:41.690 INFO:tasks.workunit.client.0.smithi033.stdout:6/60: chown d4/f11 5792 1 2024-04-25T01:35:41.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:41 smithi033 ceph-mon[26353]: pgmap v6: 65 pgs: 65 active+clean; 182 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:35:41.691 INFO:tasks.workunit.client.0.smithi033.stdout:4/109: creat d6/d14/f22 x:0 0 0 2024-04-25T01:35:41.692 INFO:tasks.workunit.client.0.smithi033.stdout:6/61: chown d4/cc 138239670 1 2024-04-25T01:35:41.693 INFO:tasks.workunit.client.0.smithi033.stdout:6/62: write d4/f6 [3434072,38099] 0 2024-04-25T01:35:41.694 INFO:tasks.workunit.client.0.smithi033.stdout:8/101: unlink d6/ld 0 2024-04-25T01:35:41.696 INFO:tasks.workunit.client.0.smithi033.stdout:6/63: chown d4/la 216 1 2024-04-25T01:35:41.696 INFO:tasks.workunit.client.0.smithi033.stdout:1/74: dread d2/d5/fa [0,4194304] 0 2024-04-25T01:35:41.697 INFO:tasks.workunit.client.0.smithi033.stdout:8/102: truncate d6/f1e 5019265 0 2024-04-25T01:35:41.697 INFO:tasks.workunit.client.0.smithi033.stdout:5/164: symlink d1c/d21/l33 0 2024-04-25T01:35:41.700 INFO:tasks.workunit.client.0.smithi033.stdout:4/110: dread d6/f1e [0,4194304] 0 2024-04-25T01:35:41.703 INFO:tasks.workunit.client.0.smithi033.stdout:6/64: creat d4/f19 x:0 0 0 2024-04-25T01:35:41.705 INFO:tasks.workunit.client.0.smithi033.stdout:8/103: creat d6/d10/f27 x:0 0 0 2024-04-25T01:35:41.708 INFO:tasks.workunit.client.0.smithi033.stdout:4/111: dread d6/d14/fd [4194304,4194304] 0 2024-04-25T01:35:41.711 INFO:tasks.workunit.client.0.smithi033.stdout:5/165: dwrite d1c/d21/f2c [0,4194304] 0 2024-04-25T01:35:41.714 INFO:tasks.workunit.client.0.smithi033.stdout:5/166: write d1c/d21/f26 [85673,102827] 0 2024-04-25T01:35:41.714 INFO:tasks.workunit.client.0.smithi033.stdout:8/104: unlink d6/l11 0 2024-04-25T01:35:41.716 INFO:tasks.workunit.client.0.smithi033.stdout:4/112: dread d6/d12/f1a [0,4194304] 0 2024-04-25T01:35:41.716 INFO:tasks.workunit.client.0.smithi033.stdout:4/113: stat d6/d14/ff 0 2024-04-25T01:35:41.717 INFO:tasks.workunit.client.0.smithi033.stdout:4/114: fdatasync d6/f8 0 2024-04-25T01:35:41.723 INFO:tasks.workunit.client.0.smithi033.stdout:5/167: creat d1c/d2a/f34 x:0 0 0 2024-04-25T01:35:41.730 INFO:tasks.workunit.client.0.smithi033.stdout:4/115: creat d6/d12/f23 x:0 0 0 2024-04-25T01:35:41.732 INFO:tasks.workunit.client.0.smithi033.stdout:8/105: dwrite d6/f15 [0,4194304] 0 2024-04-25T01:35:41.739 INFO:tasks.workunit.client.0.smithi033.stdout:4/116: rename d6/d14/fd to d6/d14/d1d/f24 0 2024-04-25T01:35:41.744 INFO:tasks.workunit.client.0.smithi033.stdout:8/106: rmdir d6/d23 0 2024-04-25T01:35:41.752 INFO:tasks.workunit.client.0.smithi033.stdout:8/107: dwrite d6/f8 [0,4194304] 0 2024-04-25T01:35:41.759 INFO:tasks.workunit.client.0.smithi033.stdout:8/108: dread d6/f1e [4194304,4194304] 0 2024-04-25T01:35:41.760 INFO:tasks.workunit.client.0.smithi033.stdout:8/109: creat d6/d10/f28 x:0 0 0 2024-04-25T01:35:41.764 INFO:tasks.workunit.client.0.smithi033.stdout:8/110: symlink d6/db/l29 0 2024-04-25T01:35:41.767 INFO:tasks.workunit.client.0.smithi033.stdout:8/111: dread d6/db/f1c [0,4194304] 0 2024-04-25T01:35:41.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:41 smithi138 ceph-mon[32977]: pgmap v6: 65 pgs: 65 active+clean; 182 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:35:41.772 INFO:tasks.workunit.client.0.smithi033.stdout:8/112: rename d6/db/c22 to d6/c2a 0 2024-04-25T01:35:41.774 INFO:tasks.workunit.client.0.smithi033.stdout:8/113: creat d6/d10/f2b x:0 0 0 2024-04-25T01:35:41.777 INFO:tasks.workunit.client.0.smithi033.stdout:8/114: symlink d6/db/l2c 0 2024-04-25T01:35:41.780 INFO:tasks.workunit.client.0.smithi033.stdout:8/115: creat d6/db/f2d x:0 0 0 2024-04-25T01:35:41.781 INFO:tasks.workunit.client.0.smithi033.stdout:8/116: chown l0 1245 1 2024-04-25T01:35:41.783 INFO:tasks.workunit.client.0.smithi033.stdout:8/117: mkdir d6/d10/d2e 0 2024-04-25T01:35:41.789 INFO:tasks.workunit.client.0.smithi033.stdout:8/118: dread d6/f8 [0,4194304] 0 2024-04-25T01:35:41.791 INFO:tasks.workunit.client.0.smithi033.stdout:8/119: symlink d6/d10/l2f 0 2024-04-25T01:35:41.795 INFO:tasks.workunit.client.0.smithi033.stdout:8/120: link d6/db/f1a d6/d10/d2e/f30 0 2024-04-25T01:35:41.797 INFO:tasks.workunit.client.0.smithi033.stdout:8/121: write d6/db/f24 [1933842,100080] 0 2024-04-25T01:35:41.801 INFO:tasks.workunit.client.0.smithi033.stdout:8/122: creat d6/db/f31 x:0 0 0 2024-04-25T01:35:41.803 INFO:tasks.workunit.client.0.smithi033.stdout:8/123: write d6/db/f18 [690616,35204] 0 2024-04-25T01:35:41.807 INFO:tasks.workunit.client.0.smithi033.stdout:8/124: rename d6/fe to d6/d10/f32 0 2024-04-25T01:35:41.809 INFO:tasks.workunit.client.0.smithi033.stdout:8/125: truncate d6/f9 1396588 0 2024-04-25T01:35:41.812 INFO:tasks.workunit.client.0.smithi033.stdout:8/126: unlink d6/db/f1c 0 2024-04-25T01:35:41.813 INFO:tasks.workunit.client.0.smithi033.stdout:7/44: sync 2024-04-25T01:35:41.817 INFO:tasks.workunit.client.0.smithi033.stdout:7/45: rmdir d4 39 2024-04-25T01:35:41.819 INFO:tasks.workunit.client.0.smithi033.stdout:7/46: unlink f1 0 2024-04-25T01:35:41.821 INFO:tasks.workunit.client.0.smithi033.stdout:7/47: chown d4/f5 13662 1 2024-04-25T01:35:41.825 INFO:tasks.workunit.client.0.smithi033.stdout:7/48: dread f3 [0,4194304] 0 2024-04-25T01:35:41.826 INFO:tasks.workunit.client.0.smithi033.stdout:7/49: write d4/fb [810330,121599] 0 2024-04-25T01:35:41.828 INFO:tasks.workunit.client.0.smithi033.stdout:7/50: fsync d4/f5 0 2024-04-25T01:35:41.832 INFO:tasks.workunit.client.0.smithi033.stdout:7/51: rename d4/l6 to d4/lc 0 2024-04-25T01:35:41.832 INFO:tasks.workunit.client.0.smithi033.stdout:7/52: chown f0 170814 1 2024-04-25T01:35:41.863 INFO:tasks.workunit.client.0.smithi033.stdout:1/75: rmdir d2/d5 39 2024-04-25T01:35:41.866 INFO:tasks.workunit.client.0.smithi033.stdout:1/76: symlink d2/l15 0 2024-04-25T01:35:41.867 INFO:tasks.workunit.client.0.smithi033.stdout:1/77: rmdir d2/d11 39 2024-04-25T01:35:41.868 INFO:tasks.workunit.client.0.smithi033.stdout:1/78: readlink d2/l15 0 2024-04-25T01:35:41.873 INFO:tasks.workunit.client.0.smithi033.stdout:1/79: creat d2/f16 x:0 0 0 2024-04-25T01:35:41.874 INFO:tasks.workunit.client.0.smithi033.stdout:6/65: getdents d4 0 2024-04-25T01:35:41.877 INFO:tasks.workunit.client.0.smithi033.stdout:6/66: mkdir d4/d1a 0 2024-04-25T01:35:41.878 INFO:tasks.workunit.client.0.smithi033.stdout:6/67: write d4/f19 [714740,50316] 0 2024-04-25T01:35:41.884 INFO:tasks.workunit.client.0.smithi033.stdout:5/168: rmdir d1c 39 2024-04-25T01:35:41.886 INFO:tasks.workunit.client.0.smithi033.stdout:1/80: dwrite d2/d5/fd [0,4194304] 0 2024-04-25T01:35:41.887 INFO:tasks.workunit.client.0.smithi033.stdout:5/169: truncate d1c/f22 978139 0 2024-04-25T01:35:41.890 INFO:tasks.workunit.client.0.smithi033.stdout:5/170: read - d1c/f23 zero size 2024-04-25T01:35:41.890 INFO:tasks.workunit.client.0.smithi033.stdout:1/81: write d2/d5/fd [4508831,75382] 0 2024-04-25T01:35:41.894 INFO:tasks.workunit.client.0.smithi033.stdout:1/82: mknod d2/d5/c17 0 2024-04-25T01:35:41.894 INFO:tasks.workunit.client.0.smithi033.stdout:1/83: chown d2/d5/fd 441565364 1 2024-04-25T01:35:41.895 INFO:tasks.workunit.client.0.smithi033.stdout:1/84: readlink d2/l4 0 2024-04-25T01:35:41.895 INFO:tasks.workunit.client.0.smithi033.stdout:5/171: unlink c10 0 2024-04-25T01:35:41.898 INFO:tasks.workunit.client.0.smithi033.stdout:1/85: chown d2/d11/f13 95010 1 2024-04-25T01:35:41.899 INFO:tasks.workunit.client.0.smithi033.stdout:1/86: write d2/f16 [321509,113020] 0 2024-04-25T01:35:41.900 INFO:tasks.workunit.client.0.smithi033.stdout:5/172: symlink d1c/l35 0 2024-04-25T01:35:41.901 INFO:tasks.workunit.client.0.smithi033.stdout:5/173: chown d1c/f2e 2 1 2024-04-25T01:35:41.903 INFO:tasks.workunit.client.0.smithi033.stdout:1/87: chown d2/d5/f14 24 1 2024-04-25T01:35:41.904 INFO:tasks.workunit.client.0.smithi033.stdout:5/174: symlink d1c/d21/l36 0 2024-04-25T01:35:41.911 INFO:tasks.workunit.client.0.smithi033.stdout:1/88: dwrite d2/d5/f14 [0,4194304] 0 2024-04-25T01:35:41.913 INFO:tasks.workunit.client.0.smithi033.stdout:8/127: truncate d6/db/f24 2380057 0 2024-04-25T01:35:41.918 INFO:tasks.workunit.client.0.smithi033.stdout:1/89: getdents d2/d11 0 2024-04-25T01:35:41.920 INFO:tasks.workunit.client.0.smithi033.stdout:6/68: getdents d4 0 2024-04-25T01:35:41.923 INFO:tasks.workunit.client.0.smithi033.stdout:6/69: creat d4/f1b x:0 0 0 2024-04-25T01:35:41.924 INFO:tasks.workunit.client.0.smithi033.stdout:1/90: unlink d2/f12 0 2024-04-25T01:35:41.928 INFO:tasks.workunit.client.0.smithi033.stdout:1/91: rename d2/d5/fa to d2/d5/f18 0 2024-04-25T01:35:41.929 INFO:tasks.workunit.client.0.smithi033.stdout:1/92: fdatasync d2/d5/fe 0 2024-04-25T01:35:41.938 INFO:tasks.workunit.client.0.smithi033.stdout:8/128: stat d6/d10/f28 0 2024-04-25T01:35:41.946 INFO:tasks.workunit.client.0.smithi033.stdout:8/129: dwrite d6/f15 [4194304,4194304] 0 2024-04-25T01:35:41.948 INFO:tasks.workunit.client.0.smithi033.stdout:8/130: dread - d6/f1d zero size 2024-04-25T01:35:41.951 INFO:tasks.workunit.client.0.smithi033.stdout:8/131: stat d6/db/f2d 0 2024-04-25T01:35:41.951 INFO:tasks.workunit.client.0.smithi033.stdout:8/132: dread - d6/db/f31 zero size 2024-04-25T01:35:41.952 INFO:tasks.workunit.client.0.smithi033.stdout:8/133: stat d6/db/l25 0 2024-04-25T01:35:41.957 INFO:tasks.workunit.client.0.smithi033.stdout:8/134: dread d6/f1e [0,4194304] 0 2024-04-25T01:35:41.958 INFO:tasks.workunit.client.0.smithi033.stdout:8/135: mknod d6/db/c33 0 2024-04-25T01:35:41.959 INFO:tasks.workunit.client.0.smithi033.stdout:8/136: dread - d6/f21 zero size 2024-04-25T01:35:41.964 INFO:tasks.workunit.client.0.smithi033.stdout:8/137: truncate d6/d10/f32 2091590 0 2024-04-25T01:35:41.967 INFO:tasks.workunit.client.0.smithi033.stdout:8/138: creat d6/f34 x:0 0 0 2024-04-25T01:35:41.970 INFO:tasks.workunit.client.0.smithi033.stdout:8/139: symlink d6/d10/d2e/l35 0 2024-04-25T01:35:41.978 INFO:tasks.workunit.client.0.smithi033.stdout:8/140: unlink d6/d10/f32 0 2024-04-25T01:35:41.979 INFO:tasks.workunit.client.0.smithi033.stdout:8/141: rmdir d6/d10/d2e 39 2024-04-25T01:35:41.983 INFO:tasks.workunit.client.0.smithi033.stdout:8/142: creat d6/d10/f36 x:0 0 0 2024-04-25T01:35:41.984 INFO:tasks.workunit.client.0.smithi033.stdout:8/143: mknod d6/d10/c37 0 2024-04-25T01:35:41.986 INFO:tasks.workunit.client.0.smithi033.stdout:8/144: fdatasync d6/db/f24 0 2024-04-25T01:35:41.987 INFO:tasks.workunit.client.0.smithi033.stdout:3/92: sync 2024-04-25T01:35:41.987 INFO:tasks.workunit.client.0.smithi033.stdout:2/87: sync 2024-04-25T01:35:41.989 INFO:tasks.workunit.client.0.smithi033.stdout:8/145: chown d6/d10/f27 24 1 2024-04-25T01:35:41.989 INFO:tasks.workunit.client.0.smithi033.stdout:2/88: mknod da/c1e 0 2024-04-25T01:35:41.996 INFO:tasks.workunit.client.0.smithi033.stdout:2/89: rename da/l16 to da/l1f 0 2024-04-25T01:35:41.998 INFO:tasks.workunit.client.0.smithi033.stdout:8/146: dread d6/f8 [0,4194304] 0 2024-04-25T01:35:41.998 INFO:tasks.workunit.client.0.smithi033.stdout:2/90: write da/fd [1039325,38385] 0 2024-04-25T01:35:41.999 INFO:tasks.workunit.client.0.smithi033.stdout:2/91: readlink da/l14 0 2024-04-25T01:35:42.000 INFO:tasks.workunit.client.0.smithi033.stdout:3/93: rename d8/c16 to d8/dd/c1c 0 2024-04-25T01:35:42.005 INFO:tasks.workunit.client.0.smithi033.stdout:8/147: unlink d6/d10/l2f 0 2024-04-25T01:35:42.007 INFO:tasks.workunit.client.0.smithi033.stdout:2/92: creat da/f20 x:0 0 0 2024-04-25T01:35:42.007 INFO:tasks.workunit.client.0.smithi033.stdout:3/94: mkdir d8/dd/d1d 0 2024-04-25T01:35:42.010 INFO:tasks.workunit.client.0.smithi033.stdout:2/93: unlink da/fc 0 2024-04-25T01:35:42.012 INFO:tasks.workunit.client.0.smithi033.stdout:3/95: mkdir d8/dd/d11/d1e 0 2024-04-25T01:35:42.013 INFO:tasks.workunit.client.0.smithi033.stdout:3/96: fdatasync d8/f1b 0 2024-04-25T01:35:42.014 INFO:tasks.workunit.client.0.smithi033.stdout:3/97: write d8/fa [864049,128862] 0 2024-04-25T01:35:42.016 INFO:tasks.workunit.client.0.smithi033.stdout:2/94: symlink da/l21 0 2024-04-25T01:35:42.017 INFO:tasks.workunit.client.0.smithi033.stdout:3/98: symlink d8/dd/l1f 0 2024-04-25T01:35:42.020 INFO:tasks.workunit.client.0.smithi033.stdout:3/99: symlink d8/l20 0 2024-04-25T01:35:42.022 INFO:tasks.workunit.client.0.smithi033.stdout:8/148: rmdir d6/d10 39 2024-04-25T01:35:42.026 INFO:tasks.workunit.client.0.smithi033.stdout:2/95: dwrite da/f13 [0,4194304] 0 2024-04-25T01:35:42.030 INFO:tasks.workunit.client.0.smithi033.stdout:3/100: dread d8/dd/f15 [0,4194304] 0 2024-04-25T01:35:42.030 INFO:tasks.workunit.client.0.smithi033.stdout:2/96: fsync f9 0 2024-04-25T01:35:42.031 INFO:tasks.workunit.client.0.smithi033.stdout:3/101: readlink d8/l20 0 2024-04-25T01:35:42.031 INFO:tasks.workunit.client.0.smithi033.stdout:2/97: chown f7 3212411 1 2024-04-25T01:35:42.034 INFO:tasks.workunit.client.0.smithi033.stdout:2/98: creat da/f22 x:0 0 0 2024-04-25T01:35:42.036 INFO:tasks.workunit.client.0.smithi033.stdout:2/99: readlink da/l21 0 2024-04-25T01:35:42.038 INFO:tasks.workunit.client.0.smithi033.stdout:8/149: rename d6/d10/c37 to d6/c38 0 2024-04-25T01:35:42.042 INFO:tasks.workunit.client.0.smithi033.stdout:3/102: link d8/fa d8/dd/d11/d1e/f21 0 2024-04-25T01:35:42.042 INFO:tasks.workunit.client.0.smithi033.stdout:3/103: chown d8/dd/d11 1872 1 2024-04-25T01:35:42.042 INFO:tasks.workunit.client.0.smithi033.stdout:8/150: stat d6/f13 0 2024-04-25T01:35:42.043 INFO:tasks.workunit.client.0.smithi033.stdout:3/104: fdatasync f3 0 2024-04-25T01:35:42.043 INFO:tasks.workunit.client.0.smithi033.stdout:2/100: rmdir da 39 2024-04-25T01:35:42.054 INFO:tasks.workunit.client.0.smithi033.stdout:8/151: dwrite d6/db/f2d [0,4194304] 0 2024-04-25T01:35:42.058 INFO:tasks.workunit.client.0.smithi033.stdout:2/101: getdents da 0 2024-04-25T01:35:42.059 INFO:tasks.workunit.client.0.smithi033.stdout:8/152: fdatasync d6/db/f26 0 2024-04-25T01:35:42.062 INFO:tasks.workunit.client.0.smithi033.stdout:2/102: getdents da 0 2024-04-25T01:35:42.063 INFO:tasks.workunit.client.0.smithi033.stdout:3/105: dwrite d8/fa [0,4194304] 0 2024-04-25T01:35:42.064 INFO:tasks.workunit.client.0.smithi033.stdout:2/103: write da/fe [156077,84754] 0 2024-04-25T01:35:42.068 INFO:tasks.workunit.client.0.smithi033.stdout:8/153: symlink d6/l39 0 2024-04-25T01:35:42.069 INFO:tasks.workunit.client.0.smithi033.stdout:3/106: unlink d8/c12 0 2024-04-25T01:35:42.071 INFO:tasks.workunit.client.0.smithi033.stdout:3/107: write f3 [779635,69970] 0 2024-04-25T01:35:42.073 INFO:tasks.workunit.client.0.smithi033.stdout:2/104: dread f8 [0,4194304] 0 2024-04-25T01:35:42.076 INFO:tasks.workunit.client.0.smithi033.stdout:3/108: write d8/f1b [971410,74137] 0 2024-04-25T01:35:42.077 INFO:tasks.workunit.client.0.smithi033.stdout:2/105: mkdir da/d23 0 2024-04-25T01:35:42.079 INFO:tasks.workunit.client.0.smithi033.stdout:8/154: dwrite d6/d10/f2b [0,4194304] 0 2024-04-25T01:35:42.086 INFO:tasks.workunit.client.0.smithi033.stdout:2/106: symlink da/d23/l24 0 2024-04-25T01:35:42.087 INFO:tasks.workunit.client.0.smithi033.stdout:3/109: symlink d8/dd/d11/d1e/l22 0 2024-04-25T01:35:42.088 INFO:tasks.workunit.client.0.smithi033.stdout:3/110: fdatasync f3 0 2024-04-25T01:35:42.090 INFO:tasks.workunit.client.0.smithi033.stdout:8/155: dwrite d6/db/f2d [0,4194304] 0 2024-04-25T01:35:42.090 INFO:tasks.workunit.client.0.smithi033.stdout:2/107: write da/f13 [2220600,124931] 0 2024-04-25T01:35:42.096 INFO:tasks.workunit.client.0.smithi033.stdout:3/111: dread d8/dd/f15 [0,4194304] 0 2024-04-25T01:35:42.097 INFO:tasks.workunit.client.0.smithi033.stdout:8/156: creat d6/d10/d2e/f3a x:0 0 0 2024-04-25T01:35:42.097 INFO:tasks.workunit.client.0.smithi033.stdout:8/157: stat d6/f21 0 2024-04-25T01:35:42.097 INFO:tasks.workunit.client.0.smithi033.stdout:3/112: readlink d8/dd/d11/d1e/l22 0 2024-04-25T01:35:42.098 INFO:tasks.workunit.client.0.smithi033.stdout:8/158: chown d6/d10/f28 1 1 2024-04-25T01:35:42.100 INFO:tasks.workunit.client.0.smithi033.stdout:8/159: stat d6/c2a 0 2024-04-25T01:35:42.101 INFO:tasks.workunit.client.0.smithi033.stdout:3/113: creat d8/f23 x:0 0 0 2024-04-25T01:35:42.102 INFO:tasks.workunit.client.0.smithi033.stdout:8/160: chown d6/l16 6 1 2024-04-25T01:35:42.103 INFO:tasks.workunit.client.0.smithi033.stdout:8/161: chown d6/db/f24 47164013 1 2024-04-25T01:35:42.104 INFO:tasks.workunit.client.0.smithi033.stdout:3/114: creat d8/dd/d11/f24 x:0 0 0 2024-04-25T01:35:42.106 INFO:tasks.workunit.client.0.smithi033.stdout:3/115: truncate d8/dd/d11/f24 1004846 0 2024-04-25T01:35:42.109 INFO:tasks.workunit.client.0.smithi033.stdout:3/116: readlink d8/dd/l10 0 2024-04-25T01:35:42.111 INFO:tasks.workunit.client.0.smithi033.stdout:3/117: fsync d8/fa 0 2024-04-25T01:35:42.112 INFO:tasks.workunit.client.0.smithi033.stdout:8/162: dwrite d6/f13 [0,4194304] 0 2024-04-25T01:35:42.114 INFO:tasks.workunit.client.0.smithi033.stdout:3/118: write d8/dd/d11/d1e/f21 [654344,48349] 0 2024-04-25T01:35:42.121 INFO:tasks.workunit.client.0.smithi033.stdout:8/163: creat d6/d10/f3b x:0 0 0 2024-04-25T01:35:42.122 INFO:tasks.workunit.client.0.smithi033.stdout:8/164: fdatasync f5 0 2024-04-25T01:35:42.124 INFO:tasks.workunit.client.0.smithi033.stdout:8/165: chown d6/d10/d2e/f30 1028 1 2024-04-25T01:35:42.126 INFO:tasks.workunit.client.0.smithi033.stdout:3/119: dwrite d8/dd/f15 [0,4194304] 0 2024-04-25T01:35:42.129 INFO:tasks.workunit.client.0.smithi033.stdout:3/120: creat d8/dd/f25 x:0 0 0 2024-04-25T01:35:42.177 INFO:tasks.workunit.client.0.smithi033.stdout:4/117: sync 2024-04-25T01:35:42.177 INFO:tasks.workunit.client.0.smithi033.stdout:7/53: sync 2024-04-25T01:35:42.177 INFO:tasks.workunit.client.0.smithi033.stdout:0/76: sync 2024-04-25T01:35:42.177 INFO:tasks.workunit.client.0.smithi033.stdout:9/87: sync 2024-04-25T01:35:42.177 INFO:tasks.workunit.client.0.smithi033.stdout:9/88: chown d0/c1c 782264 1 2024-04-25T01:35:42.178 INFO:tasks.workunit.client.0.smithi033.stdout:9/89: readlink d0/d2/l1a 0 2024-04-25T01:35:42.179 INFO:tasks.workunit.client.0.smithi033.stdout:7/54: stat d4/lc 0 2024-04-25T01:35:42.179 INFO:tasks.workunit.client.0.smithi033.stdout:7/55: fdatasync f3 0 2024-04-25T01:35:42.180 INFO:tasks.workunit.client.0.smithi033.stdout:9/90: chown d0/d2/f1f 58079 1 2024-04-25T01:35:42.182 INFO:tasks.workunit.client.0.smithi033.stdout:4/118: write d6/f8 [8788985,113688] 0 2024-04-25T01:35:42.182 INFO:tasks.workunit.client.0.smithi033.stdout:7/56: read - d4/f7 zero size 2024-04-25T01:35:42.184 INFO:tasks.workunit.client.0.smithi033.stdout:0/77: rename d7/da/l11 to d7/l18 0 2024-04-25T01:35:42.185 INFO:tasks.workunit.client.0.smithi033.stdout:0/78: readlink d7/da/df/l12 0 2024-04-25T01:35:42.186 INFO:tasks.workunit.client.0.smithi033.stdout:0/79: chown d7 4116439 1 2024-04-25T01:35:42.189 INFO:tasks.workunit.client.0.smithi033.stdout:7/57: mkdir d4/dd 0 2024-04-25T01:35:42.191 INFO:tasks.workunit.client.0.smithi033.stdout:4/119: dread d6/f16 [0,4194304] 0 2024-04-25T01:35:42.191 INFO:tasks.workunit.client.0.smithi033.stdout:7/58: fdatasync d4/fb 0 2024-04-25T01:35:42.200 INFO:tasks.workunit.client.0.smithi033.stdout:0/80: dwrite d7/da/f13 [0,4194304] 0 2024-04-25T01:35:42.201 INFO:tasks.workunit.client.0.smithi033.stdout:4/120: getdents d6 0 2024-04-25T01:35:42.204 INFO:tasks.workunit.client.0.smithi033.stdout:9/91: dwrite d0/d2/dd/fe [4194304,4194304] 0 2024-04-25T01:35:42.208 INFO:tasks.workunit.client.0.smithi033.stdout:7/59: dread f3 [0,4194304] 0 2024-04-25T01:35:42.208 INFO:tasks.workunit.client.0.smithi033.stdout:0/81: write d7/da/df/d14/f16 [41164,113718] 0 2024-04-25T01:35:42.210 INFO:tasks.workunit.client.0.smithi033.stdout:7/60: write f3 [5065611,35455] 0 2024-04-25T01:35:42.212 INFO:tasks.workunit.client.0.smithi033.stdout:4/121: mknod d6/d12/c25 0 2024-04-25T01:35:42.212 INFO:tasks.workunit.client.0.smithi033.stdout:0/82: dread - d7/da/df/f17 zero size 2024-04-25T01:35:42.214 INFO:tasks.workunit.client.0.smithi033.stdout:7/61: mknod d4/ce 0 2024-04-25T01:35:42.215 INFO:tasks.workunit.client.0.smithi033.stdout:4/122: creat d6/d12/f26 x:0 0 0 2024-04-25T01:35:42.216 INFO:tasks.workunit.client.0.smithi033.stdout:0/83: rmdir d7 39 2024-04-25T01:35:42.217 INFO:tasks.workunit.client.0.smithi033.stdout:7/62: rmdir d4 39 2024-04-25T01:35:42.218 INFO:tasks.workunit.client.0.smithi033.stdout:9/92: dwrite d0/f9 [0,4194304] 0 2024-04-25T01:35:42.219 INFO:tasks.workunit.client.0.smithi033.stdout:4/123: chown d6/d12/f1a 366335 1 2024-04-25T01:35:42.225 INFO:tasks.workunit.client.0.smithi033.stdout:9/93: mkdir d0/d2/dd/d20 0 2024-04-25T01:35:42.225 INFO:tasks.workunit.client.0.smithi033.stdout:9/94: stat d0/c1c 0 2024-04-25T01:35:42.226 INFO:tasks.workunit.client.0.smithi033.stdout:7/63: write d4/f9 [547579,27523] 0 2024-04-25T01:35:42.227 INFO:tasks.workunit.client.0.smithi033.stdout:4/124: rmdir d6/d14 39 2024-04-25T01:35:42.228 INFO:tasks.workunit.client.0.smithi033.stdout:0/84: creat d7/da/df/f19 x:0 0 0 2024-04-25T01:35:42.230 INFO:tasks.workunit.client.0.smithi033.stdout:0/85: read - d7/da/df/d14/f15 zero size 2024-04-25T01:35:42.231 INFO:tasks.workunit.client.0.smithi033.stdout:7/64: creat d4/dd/ff x:0 0 0 2024-04-25T01:35:42.232 INFO:tasks.workunit.client.0.smithi033.stdout:9/95: symlink d0/d2/dd/df/d1b/l21 0 2024-04-25T01:35:42.232 INFO:tasks.workunit.client.0.smithi033.stdout:7/65: fdatasync f3 0 2024-04-25T01:35:42.234 INFO:tasks.workunit.client.0.smithi033.stdout:4/125: rmdir d6/d14/d1d 39 2024-04-25T01:35:42.234 INFO:tasks.workunit.client.0.smithi033.stdout:0/86: mknod d7/da/c1a 0 2024-04-25T01:35:42.235 INFO:tasks.workunit.client.0.smithi033.stdout:0/87: fdatasync d7/da/f13 0 2024-04-25T01:35:42.236 INFO:tasks.workunit.client.0.smithi033.stdout:7/66: mkdir d4/d10 0 2024-04-25T01:35:42.238 INFO:tasks.workunit.client.0.smithi033.stdout:9/96: mknod d0/d2/dd/df/d1b/c22 0 2024-04-25T01:35:42.239 INFO:tasks.workunit.client.0.smithi033.stdout:4/126: dread d6/f18 [0,4194304] 0 2024-04-25T01:35:42.240 INFO:tasks.workunit.client.0.smithi033.stdout:9/97: write d0/f8 [4202504,28360] 0 2024-04-25T01:35:42.243 INFO:tasks.workunit.client.0.smithi033.stdout:7/67: creat d4/f11 x:0 0 0 2024-04-25T01:35:42.250 INFO:tasks.workunit.client.0.smithi033.stdout:7/68: symlink d4/l12 0 2024-04-25T01:35:42.251 INFO:tasks.workunit.client.0.smithi033.stdout:7/69: write d4/f5 [5651530,79437] 0 2024-04-25T01:35:42.256 INFO:tasks.workunit.client.0.smithi033.stdout:9/98: creat d0/f23 x:0 0 0 2024-04-25T01:35:42.258 INFO:tasks.workunit.client.0.smithi033.stdout:0/88: truncate d7/da/df/f10 4145092 0 2024-04-25T01:35:42.260 INFO:tasks.workunit.client.0.smithi033.stdout:9/99: creat d0/d2/dd/f24 x:0 0 0 2024-04-25T01:35:42.261 INFO:tasks.workunit.client.0.smithi033.stdout:4/127: write d6/d14/d1d/f24 [3019468,70317] 0 2024-04-25T01:35:42.262 INFO:tasks.workunit.client.0.smithi033.stdout:7/70: getdents d4/dd 0 2024-04-25T01:35:42.265 INFO:tasks.workunit.client.0.smithi033.stdout:4/128: chown d6/d12/c25 5 1 2024-04-25T01:35:42.267 INFO:tasks.workunit.client.0.smithi033.stdout:9/100: rename d0/d2/dd/df to d0/d2/dd/df/d1b/d25 22 2024-04-25T01:35:42.267 INFO:tasks.workunit.client.0.smithi033.stdout:0/89: mkdir d7/d1b 0 2024-04-25T01:35:42.269 INFO:tasks.workunit.client.0.smithi033.stdout:9/101: write d0/d2/f1f [4030121,53788] 0 2024-04-25T01:35:42.272 INFO:tasks.workunit.client.0.smithi033.stdout:0/90: rename d7 to d7/da/df/d14/d1c 22 2024-04-25T01:35:42.273 INFO:tasks.workunit.client.0.smithi033.stdout:7/71: dwrite d4/f9 [0,4194304] 0 2024-04-25T01:35:42.274 INFO:tasks.workunit.client.0.smithi033.stdout:0/91: write d7/da/df/f19 [223307,108761] 0 2024-04-25T01:35:42.275 INFO:tasks.workunit.client.0.smithi033.stdout:9/102: mknod d0/d2/dd/df/c26 0 2024-04-25T01:35:42.275 INFO:tasks.workunit.client.0.smithi033.stdout:7/72: write f0 [4597699,32705] 0 2024-04-25T01:35:42.278 INFO:tasks.workunit.client.0.smithi033.stdout:4/129: write d6/f16 [1707249,117560] 0 2024-04-25T01:35:42.279 INFO:tasks.workunit.client.0.smithi033.stdout:7/73: mkdir d4/d10/d13 0 2024-04-25T01:35:42.284 INFO:tasks.workunit.client.0.smithi033.stdout:9/103: dwrite d0/f23 [0,4194304] 0 2024-04-25T01:35:42.284 INFO:tasks.workunit.client.0.smithi033.stdout:0/92: getdents d7/da/df/d14 0 2024-04-25T01:35:42.285 INFO:tasks.workunit.client.0.smithi033.stdout:4/130: rename d6/d14 to d6/d14/d27 22 2024-04-25T01:35:42.287 INFO:tasks.workunit.client.0.smithi033.stdout:9/104: fsync d0/f9 0 2024-04-25T01:35:42.292 INFO:tasks.workunit.client.0.smithi033.stdout:4/131: creat d6/d14/d1d/f28 x:0 0 0 2024-04-25T01:35:42.293 INFO:tasks.workunit.client.0.smithi033.stdout:9/105: creat d0/d11/f27 x:0 0 0 2024-04-25T01:35:42.297 INFO:tasks.workunit.client.0.smithi033.stdout:7/74: dwrite d4/f11 [0,4194304] 0 2024-04-25T01:35:42.299 INFO:tasks.workunit.client.0.smithi033.stdout:9/106: dread - d0/d2/dd/f24 zero size 2024-04-25T01:35:42.304 INFO:tasks.workunit.client.0.smithi033.stdout:0/93: dwrite d7/da/df/d14/f16 [0,4194304] 0 2024-04-25T01:35:42.305 INFO:tasks.workunit.client.0.smithi033.stdout:9/107: read - d0/d2/dd/f15 zero size 2024-04-25T01:35:42.308 INFO:tasks.workunit.client.0.smithi033.stdout:0/94: dread - d7/da/df/d14/f15 zero size 2024-04-25T01:35:42.309 INFO:tasks.workunit.client.0.smithi033.stdout:4/132: symlink d6/d14/l29 0 2024-04-25T01:35:42.311 INFO:tasks.workunit.client.0.smithi033.stdout:9/108: unlink d0/f6 0 2024-04-25T01:35:42.312 INFO:tasks.workunit.client.0.smithi033.stdout:7/75: creat d4/f14 x:0 0 0 2024-04-25T01:35:42.313 INFO:tasks.workunit.client.0.smithi033.stdout:0/95: mknod d7/da/df/c1d 0 2024-04-25T01:35:42.320 INFO:tasks.workunit.client.0.smithi033.stdout:0/96: symlink d7/da/df/d14/l1e 0 2024-04-25T01:35:42.321 INFO:tasks.workunit.client.0.smithi033.stdout:7/76: dread d4/f5 [4194304,4194304] 0 2024-04-25T01:35:42.322 INFO:tasks.workunit.client.0.smithi033.stdout:9/109: dwrite d0/d2/f1f [0,4194304] 0 2024-04-25T01:35:42.322 INFO:tasks.workunit.client.0.smithi033.stdout:4/133: getdents d6 0 2024-04-25T01:35:42.326 INFO:tasks.workunit.client.0.smithi033.stdout:9/110: dread - d0/d11/f27 zero size 2024-04-25T01:35:42.327 INFO:tasks.workunit.client.0.smithi033.stdout:9/111: readlink d0/d2/dd/df/d1b/l21 0 2024-04-25T01:35:42.331 INFO:tasks.workunit.client.0.smithi033.stdout:0/97: dread d7/da/f13 [0,4194304] 0 2024-04-25T01:35:42.332 INFO:tasks.workunit.client.0.smithi033.stdout:9/112: rename d0/d2/dd/d20 to d0/d2/dd/d20/d28 22 2024-04-25T01:35:42.333 INFO:tasks.workunit.client.0.smithi033.stdout:0/98: write d7/da/df/f19 [966812,52054] 0 2024-04-25T01:35:42.333 INFO:tasks.workunit.client.0.smithi033.stdout:9/113: fsync d0/d11/f16 0 2024-04-25T01:35:42.339 INFO:tasks.workunit.client.0.smithi033.stdout:9/114: unlink d0/d2/dd/l7 0 2024-04-25T01:35:42.340 INFO:tasks.workunit.client.0.smithi033.stdout:9/115: dread - d0/d2/dd/f15 zero size 2024-04-25T01:35:42.340 INFO:tasks.workunit.client.0.smithi033.stdout:0/99: dread d7/da/df/f10 [0,4194304] 0 2024-04-25T01:35:42.342 INFO:tasks.workunit.client.0.smithi033.stdout:4/134: write d6/d12/f1a [292107,95659] 0 2024-04-25T01:35:42.348 INFO:tasks.workunit.client.0.smithi033.stdout:4/135: mknod d6/c2a 0 2024-04-25T01:35:42.351 INFO:tasks.workunit.client.0.smithi033.stdout:7/77: dwrite d4/f9 [4194304,4194304] 0 2024-04-25T01:35:42.354 INFO:tasks.workunit.client.0.smithi033.stdout:7/78: read f0 [1175020,1461] 0 2024-04-25T01:35:42.356 INFO:tasks.workunit.client.0.smithi033.stdout:9/116: dwrite d0/d2/dd/f17 [0,4194304] 0 2024-04-25T01:35:42.362 INFO:tasks.workunit.client.0.smithi033.stdout:7/79: chown d4/f9 0 1 2024-04-25T01:35:42.365 INFO:tasks.workunit.client.0.smithi033.stdout:0/100: dwrite d7/da/df/f10 [4194304,4194304] 0 2024-04-25T01:35:42.365 INFO:tasks.workunit.client.0.smithi033.stdout:5/175: sync 2024-04-25T01:35:42.368 INFO:tasks.workunit.client.0.smithi033.stdout:5/176: fdatasync ff 0 2024-04-25T01:35:42.368 INFO:tasks.workunit.client.0.smithi033.stdout:0/101: fsync d7/da/df/d14/f16 0 2024-04-25T01:35:42.371 INFO:tasks.workunit.client.0.smithi033.stdout:0/102: chown d7/da/f13 1311 1 2024-04-25T01:35:42.371 INFO:tasks.workunit.client.0.smithi033.stdout:9/117: getdents d0/d2/dd 0 2024-04-25T01:35:42.373 INFO:tasks.workunit.client.0.smithi033.stdout:7/80: dread d4/f9 [4194304,4194304] 0 2024-04-25T01:35:42.375 INFO:tasks.workunit.client.0.smithi033.stdout:4/136: creat d6/d14/f2b x:0 0 0 2024-04-25T01:35:42.377 INFO:tasks.workunit.client.0.smithi033.stdout:9/118: unlink d0/c1c 0 2024-04-25T01:35:42.378 INFO:tasks.workunit.client.0.smithi033.stdout:9/119: fdatasync d0/d2/dd/fe 0 2024-04-25T01:35:42.378 INFO:tasks.workunit.client.0.smithi033.stdout:0/103: creat d7/da/f1f x:0 0 0 2024-04-25T01:35:42.381 INFO:tasks.workunit.client.0.smithi033.stdout:7/81: creat d4/d10/f15 x:0 0 0 2024-04-25T01:35:42.384 INFO:tasks.workunit.client.0.smithi033.stdout:4/137: truncate d6/d12/f23 762056 0 2024-04-25T01:35:42.385 INFO:tasks.workunit.client.0.smithi033.stdout:9/120: dread d0/f9 [0,4194304] 0 2024-04-25T01:35:42.386 INFO:tasks.workunit.client.0.smithi033.stdout:5/177: creat d1c/f37 x:0 0 0 2024-04-25T01:35:42.387 INFO:tasks.workunit.client.0.smithi033.stdout:0/104: dread d7/da/df/d14/f16 [0,4194304] 0 2024-04-25T01:35:42.388 INFO:tasks.workunit.client.0.smithi033.stdout:7/82: write f0 [5102482,8824] 0 2024-04-25T01:35:42.392 INFO:tasks.workunit.client.0.smithi033.stdout:9/121: dread d0/d2/f1f [0,4194304] 0 2024-04-25T01:35:42.398 INFO:tasks.workunit.client.0.smithi033.stdout:0/105: dwrite d7/da/df/d14/f16 [0,4194304] 0 2024-04-25T01:35:42.400 INFO:tasks.workunit.client.0.smithi033.stdout:4/138: write d6/f18 [1417065,16819] 0 2024-04-25T01:35:42.402 INFO:tasks.workunit.client.0.smithi033.stdout:7/83: truncate d4/f11 2604322 0 2024-04-25T01:35:42.406 INFO:tasks.workunit.client.0.smithi033.stdout:0/106: symlink d7/l20 0 2024-04-25T01:35:42.407 INFO:tasks.workunit.client.0.smithi033.stdout:4/139: dread d6/f16 [0,4194304] 0 2024-04-25T01:35:42.411 INFO:tasks.workunit.client.0.smithi033.stdout:7/84: dread f0 [0,4194304] 0 2024-04-25T01:35:42.413 INFO:tasks.workunit.client.0.smithi033.stdout:0/107: rename d7/l18 to d7/da/l21 0 2024-04-25T01:35:42.415 INFO:tasks.workunit.client.0.smithi033.stdout:0/108: fsync d7/da/df/f17 0 2024-04-25T01:35:42.416 INFO:tasks.workunit.client.0.smithi033.stdout:0/109: dread - d7/da/df/f17 zero size 2024-04-25T01:35:42.419 INFO:tasks.workunit.client.0.smithi033.stdout:0/110: unlink d7/da/df/c1d 0 2024-04-25T01:35:42.421 INFO:tasks.workunit.client.0.smithi033.stdout:0/111: write d7/da/f1f [835686,88218] 0 2024-04-25T01:35:42.422 INFO:tasks.workunit.client.0.smithi033.stdout:0/112: chown d7/da/df/f10 0 1 2024-04-25T01:35:42.424 INFO:tasks.workunit.client.0.smithi033.stdout:4/140: truncate d6/f16 706800 0 2024-04-25T01:35:42.427 INFO:tasks.workunit.client.0.smithi033.stdout:4/141: write d6/d12/f23 [1237819,52793] 0 2024-04-25T01:35:42.428 INFO:tasks.workunit.client.0.smithi033.stdout:4/142: stat d6/d14/cb 0 2024-04-25T01:35:42.428 INFO:tasks.workunit.client.0.smithi033.stdout:0/113: creat d7/da/df/d14/f22 x:0 0 0 2024-04-25T01:35:42.429 INFO:tasks.workunit.client.0.smithi033.stdout:7/85: truncate f3 5535121 0 2024-04-25T01:35:42.432 INFO:tasks.workunit.client.0.smithi033.stdout:4/143: creat d6/d14/d1d/f2c x:0 0 0 2024-04-25T01:35:42.432 INFO:tasks.workunit.client.0.smithi033.stdout:4/144: stat d6 0 2024-04-25T01:35:42.436 INFO:tasks.workunit.client.0.smithi033.stdout:7/86: mknod d4/d10/d13/c16 0 2024-04-25T01:35:42.436 INFO:tasks.workunit.client.0.smithi033.stdout:4/145: chown d6/c15 8857 1 2024-04-25T01:35:42.444 INFO:tasks.workunit.client.0.smithi033.stdout:0/114: dwrite d7/da/df/d14/f22 [0,4194304] 0 2024-04-25T01:35:42.446 INFO:tasks.workunit.client.0.smithi033.stdout:7/87: dread d4/fb [0,4194304] 0 2024-04-25T01:35:42.448 INFO:tasks.workunit.client.0.smithi033.stdout:7/88: chown d4/fb 5356 1 2024-04-25T01:35:42.450 INFO:tasks.workunit.client.0.smithi033.stdout:7/89: stat d4/d10 0 2024-04-25T01:35:42.457 INFO:tasks.workunit.client.0.smithi033.stdout:0/115: read d7/da/df/d14/f16 [2139029,980] 0 2024-04-25T01:35:42.457 INFO:tasks.workunit.client.0.smithi033.stdout:4/146: dwrite d6/d14/d1d/f24 [0,4194304] 0 2024-04-25T01:35:42.462 INFO:tasks.workunit.client.0.smithi033.stdout:0/116: mkdir d7/da/d23 0 2024-04-25T01:35:42.467 INFO:tasks.workunit.client.0.smithi033.stdout:7/90: dwrite d4/f14 [0,4194304] 0 2024-04-25T01:35:42.469 INFO:tasks.workunit.client.0.smithi033.stdout:4/147: rmdir d6/d14/d1d 39 2024-04-25T01:35:42.476 INFO:tasks.workunit.client.0.smithi033.stdout:0/117: fsync d7/fe 0 2024-04-25T01:35:42.480 INFO:tasks.workunit.client.0.smithi033.stdout:4/148: mknod d6/d14/d1d/c2d 0 2024-04-25T01:35:42.481 INFO:tasks.workunit.client.0.smithi033.stdout:0/118: creat d7/da/df/d14/f24 x:0 0 0 2024-04-25T01:35:42.486 INFO:tasks.workunit.client.0.smithi033.stdout:4/149: link d6/d14/d1d/f2c d6/d14/d1d/f2e 0 2024-04-25T01:35:42.490 INFO:tasks.workunit.client.0.smithi033.stdout:7/91: write d4/f11 [3333551,78740] 0 2024-04-25T01:35:42.491 INFO:tasks.workunit.client.0.smithi033.stdout:0/119: getdents d7/da 0 2024-04-25T01:35:42.496 INFO:tasks.workunit.client.0.smithi033.stdout:0/120: symlink d7/l25 0 2024-04-25T01:35:42.497 INFO:tasks.workunit.client.0.smithi033.stdout:4/150: dwrite d6/f1e [0,4194304] 0 2024-04-25T01:35:42.500 INFO:tasks.workunit.client.0.smithi033.stdout:7/92: dread f3 [0,4194304] 0 2024-04-25T01:35:42.503 INFO:tasks.workunit.client.0.smithi033.stdout:0/121: symlink d7/l26 0 2024-04-25T01:35:42.504 INFO:tasks.workunit.client.0.smithi033.stdout:4/151: fsync d6/d14/d1d/f2e 0 2024-04-25T01:35:42.520 INFO:tasks.workunit.client.0.smithi033.stdout:0/122: dwrite d7/da/df/d14/f24 [0,4194304] 0 2024-04-25T01:35:42.521 INFO:tasks.workunit.client.0.smithi033.stdout:7/93: dwrite d4/f11 [0,4194304] 0 2024-04-25T01:35:42.524 INFO:tasks.workunit.client.0.smithi033.stdout:7/94: stat d4/f7 0 2024-04-25T01:35:42.526 INFO:tasks.workunit.client.0.smithi033.stdout:2/108: sync 2024-04-25T01:35:42.526 INFO:tasks.workunit.client.0.smithi033.stdout:6/70: sync 2024-04-25T01:35:42.534 INFO:tasks.workunit.client.0.smithi033.stdout:6/71: dwrite f3 [4194304,4194304] 0 2024-04-25T01:35:42.537 INFO:tasks.workunit.client.0.smithi033.stdout:4/152: creat d6/d12/f2f x:0 0 0 2024-04-25T01:35:42.541 INFO:tasks.workunit.client.0.smithi033.stdout:4/153: fdatasync d6/d14/d1d/f20 0 2024-04-25T01:35:42.542 INFO:tasks.workunit.client.0.smithi033.stdout:7/95: write d4/f9 [9015409,122559] 0 2024-04-25T01:35:42.543 INFO:tasks.workunit.client.0.smithi033.stdout:0/123: rename d7/da/df/d14/f16 to d7/da/d23/f27 0 2024-04-25T01:35:42.543 INFO:tasks.workunit.client.0.smithi033.stdout:7/96: chown d4/ce 23070 1 2024-04-25T01:35:42.544 INFO:tasks.workunit.client.0.smithi033.stdout:0/124: chown d7/da/df/d14/f15 0 1 2024-04-25T01:35:42.545 INFO:tasks.workunit.client.0.smithi033.stdout:0/125: fdatasync d7/da/df/f10 0 2024-04-25T01:35:42.546 INFO:tasks.workunit.client.0.smithi033.stdout:2/109: creat da/d23/f25 x:0 0 0 2024-04-25T01:35:42.547 INFO:tasks.workunit.client.0.smithi033.stdout:6/72: mknod d4/c1c 0 2024-04-25T01:35:42.547 INFO:tasks.workunit.client.0.smithi033.stdout:0/126: dread - d7/da/df/f17 zero size 2024-04-25T01:35:42.552 INFO:tasks.workunit.client.0.smithi033.stdout:7/97: creat d4/f17 x:0 0 0 2024-04-25T01:35:42.553 INFO:tasks.workunit.client.0.smithi033.stdout:2/110: fsync f8 0 2024-04-25T01:35:42.556 INFO:tasks.workunit.client.0.smithi033.stdout:7/98: mknod d4/c18 0 2024-04-25T01:35:42.557 INFO:tasks.workunit.client.0.smithi033.stdout:2/111: read f1 [3560989,54360] 0 2024-04-25T01:35:42.561 INFO:tasks.workunit.client.0.smithi033.stdout:7/99: write d4/fb [1926013,92097] 0 2024-04-25T01:35:42.566 INFO:tasks.workunit.client.0.smithi033.stdout:0/127: dread d7/da/df/d14/f22 [0,4194304] 0 2024-04-25T01:35:42.566 INFO:tasks.workunit.client.0.smithi033.stdout:7/100: chown d4/d10 7307990 1 2024-04-25T01:35:42.566 INFO:tasks.workunit.client.0.smithi033.stdout:6/73: rmdir d4/d1a 0 2024-04-25T01:35:42.566 INFO:tasks.workunit.client.0.smithi033.stdout:0/128: stat d7/da/df/f19 0 2024-04-25T01:35:42.566 INFO:tasks.workunit.client.0.smithi033.stdout:2/112: creat da/f26 x:0 0 0 2024-04-25T01:35:42.566 INFO:tasks.workunit.client.0.smithi033.stdout:2/113: write da/f11 [3030810,39765] 0 2024-04-25T01:35:42.570 INFO:tasks.workunit.client.0.smithi033.stdout:6/74: fdatasync d4/f11 0 2024-04-25T01:35:42.578 INFO:tasks.workunit.client.0.smithi033.stdout:2/114: dwrite da/fd [0,4194304] 0 2024-04-25T01:35:42.578 INFO:tasks.workunit.client.0.smithi033.stdout:6/75: dread f3 [0,4194304] 0 2024-04-25T01:35:42.581 INFO:tasks.workunit.client.0.smithi033.stdout:6/76: write d4/f19 [927358,105605] 0 2024-04-25T01:35:42.581 INFO:tasks.workunit.client.0.smithi033.stdout:7/101: getdents d4/d10/d13 0 2024-04-25T01:35:42.582 INFO:tasks.workunit.client.0.smithi033.stdout:6/77: chown d4/cc 120 1 2024-04-25T01:35:42.583 INFO:tasks.workunit.client.0.smithi033.stdout:6/78: truncate d4/f12 910470 0 2024-04-25T01:35:42.598 INFO:tasks.workunit.client.0.smithi033.stdout:7/102: link d4/ca d4/c19 0 2024-04-25T01:35:42.601 INFO:tasks.workunit.client.0.smithi033.stdout:3/121: sync 2024-04-25T01:35:42.601 INFO:tasks.workunit.client.0.smithi033.stdout:1/93: sync 2024-04-25T01:35:42.601 INFO:tasks.workunit.client.0.smithi033.stdout:8/166: sync 2024-04-25T01:35:42.602 INFO:tasks.workunit.client.0.smithi033.stdout:0/129: truncate d7/da/d23/f27 4884423 0 2024-04-25T01:35:42.603 INFO:tasks.workunit.client.0.smithi033.stdout:6/79: dwrite f3 [4194304,4194304] 0 2024-04-25T01:35:42.603 INFO:tasks.workunit.client.0.smithi033.stdout:7/103: chown d4/f9 33 1 2024-04-25T01:35:42.608 INFO:tasks.workunit.client.0.smithi033.stdout:3/122: mknod d8/dd/c26 0 2024-04-25T01:35:42.609 INFO:tasks.workunit.client.0.smithi033.stdout:1/94: truncate d2/d5/fd 4759852 0 2024-04-25T01:35:42.609 INFO:tasks.workunit.client.0.smithi033.stdout:4/154: dwrite d6/d14/d1d/f2c [0,4194304] 0 2024-04-25T01:35:42.609 INFO:tasks.workunit.client.0.smithi033.stdout:0/130: read d7/da/df/f10 [5064739,7727] 0 2024-04-25T01:35:42.609 INFO:tasks.workunit.client.0.smithi033.stdout:7/104: symlink d4/d10/l1a 0 2024-04-25T01:35:42.613 INFO:tasks.workunit.client.0.smithi033.stdout:1/95: mknod d2/d11/c19 0 2024-04-25T01:35:42.613 INFO:tasks.workunit.client.0.smithi033.stdout:3/123: dread - d8/f23 zero size 2024-04-25T01:35:42.613 INFO:tasks.workunit.client.0.smithi033.stdout:0/131: chown d7/da/c1a 15369304 1 2024-04-25T01:35:42.619 INFO:tasks.workunit.client.0.smithi033.stdout:1/96: rename d2/d11/f13 to d2/d11/f1a 0 2024-04-25T01:35:42.620 INFO:tasks.workunit.client.0.smithi033.stdout:8/167: dwrite d6/d10/f2b [4194304,4194304] 0 2024-04-25T01:35:42.622 INFO:tasks.workunit.client.0.smithi033.stdout:1/97: chown d2/d5/f14 3664 1 2024-04-25T01:35:42.623 INFO:tasks.workunit.client.0.smithi033.stdout:1/98: truncate d2/d11/f1a 1072548 0 2024-04-25T01:35:42.625 INFO:tasks.workunit.client.0.smithi033.stdout:6/80: write d4/f14 [6188984,22483] 0 2024-04-25T01:35:42.628 INFO:tasks.workunit.client.0.smithi033.stdout:0/132: creat d7/da/df/f28 x:0 0 0 2024-04-25T01:35:42.629 INFO:tasks.workunit.client.0.smithi033.stdout:3/124: mkdir d8/dd/d11/d1e/d27 0 2024-04-25T01:35:42.630 INFO:tasks.workunit.client.0.smithi033.stdout:3/125: dread - d8/f23 zero size 2024-04-25T01:35:42.631 INFO:tasks.workunit.client.0.smithi033.stdout:7/105: getdents d4/dd 0 2024-04-25T01:35:42.635 INFO:tasks.workunit.client.0.smithi033.stdout:3/126: write d8/dd/f25 [180102,414] 0 2024-04-25T01:35:42.637 INFO:tasks.workunit.client.0.smithi033.stdout:2/115: dwrite f4 [0,4194304] 0 2024-04-25T01:35:42.640 INFO:tasks.workunit.client.0.smithi033.stdout:6/81: mkdir d4/d1d 0 2024-04-25T01:35:42.641 INFO:tasks.workunit.client.0.smithi033.stdout:2/116: write da/ff [42185,36796] 0 2024-04-25T01:35:42.641 INFO:tasks.workunit.client.0.smithi033.stdout:2/117: stat da/f22 0 2024-04-25T01:35:42.642 INFO:tasks.workunit.client.0.smithi033.stdout:6/82: write d4/f6 [1468699,67626] 0 2024-04-25T01:35:42.642 INFO:tasks.workunit.client.0.smithi033.stdout:6/83: readlink l2 0 2024-04-25T01:35:42.643 INFO:tasks.workunit.client.0.smithi033.stdout:2/118: chown da/d23/l24 26958103 1 2024-04-25T01:35:42.645 INFO:tasks.workunit.client.0.smithi033.stdout:7/106: creat d4/f1b x:0 0 0 2024-04-25T01:35:42.646 INFO:tasks.workunit.client.0.smithi033.stdout:7/107: readlink d4/l12 0 2024-04-25T01:35:42.648 INFO:tasks.workunit.client.0.smithi033.stdout:4/155: write d6/f16 [1623794,82225] 0 2024-04-25T01:35:42.655 INFO:tasks.workunit.client.0.smithi033.stdout:6/84: dwrite d4/f9 [4194304,4194304] 0 2024-04-25T01:35:42.662 INFO:tasks.workunit.client.0.smithi033.stdout:8/168: creat d6/f3c x:0 0 0 2024-04-25T01:35:42.662 INFO:tasks.workunit.client.0.smithi033.stdout:5/178: sync 2024-04-25T01:35:42.662 INFO:tasks.workunit.client.0.smithi033.stdout:9/122: sync 2024-04-25T01:35:42.663 INFO:tasks.workunit.client.0.smithi033.stdout:3/127: mknod d8/dd/d1d/c28 0 2024-04-25T01:35:42.663 INFO:tasks.workunit.client.0.smithi033.stdout:2/119: fsync da/f18 0 2024-04-25T01:35:42.665 INFO:tasks.workunit.client.0.smithi033.stdout:4/156: mknod d6/d14/d1d/c30 0 2024-04-25T01:35:42.666 INFO:tasks.workunit.client.0.smithi033.stdout:7/108: dwrite d4/f11 [4194304,4194304] 0 2024-04-25T01:35:42.666 INFO:tasks.workunit.client.0.smithi033.stdout:0/133: rename d7/fe to d7/f29 0 2024-04-25T01:35:42.666 INFO:tasks.workunit.client.0.smithi033.stdout:2/120: dread - da/f22 zero size 2024-04-25T01:35:42.672 INFO:tasks.workunit.client.0.smithi033.stdout:1/99: dwrite d2/f3 [0,4194304] 0 2024-04-25T01:35:42.673 INFO:tasks.workunit.client.0.smithi033.stdout:7/109: chown d4/f17 80 1 2024-04-25T01:35:42.675 INFO:tasks.workunit.client.0.smithi033.stdout:6/85: symlink d4/d1d/l1e 0 2024-04-25T01:35:42.680 INFO:tasks.workunit.client.0.smithi033.stdout:2/121: dread - da/f15 zero size 2024-04-25T01:35:42.682 INFO:tasks.workunit.client.0.smithi033.stdout:9/123: creat d0/f29 x:0 0 0 2024-04-25T01:35:42.683 INFO:tasks.workunit.client.0.smithi033.stdout:6/86: readlink l0 0 2024-04-25T01:35:42.685 INFO:tasks.workunit.client.0.smithi033.stdout:0/134: dread d7/da/df/d14/f24 [0,4194304] 0 2024-04-25T01:35:42.686 INFO:tasks.workunit.client.0.smithi033.stdout:9/124: chown d0/d2/dd/d20 3128 1 2024-04-25T01:35:42.689 INFO:tasks.workunit.client.0.smithi033.stdout:2/122: chown da/l14 837 1 2024-04-25T01:35:42.689 INFO:tasks.workunit.client.0.smithi033.stdout:9/125: dread - d0/d2/dd/f15 zero size 2024-04-25T01:35:42.691 INFO:tasks.workunit.client.0.smithi033.stdout:4/157: creat d6/d14/f31 x:0 0 0 2024-04-25T01:35:42.691 INFO:tasks.workunit.client.0.smithi033.stdout:1/100: write d2/d5/f14 [4343119,122022] 0 2024-04-25T01:35:42.692 INFO:tasks.workunit.client.0.smithi033.stdout:2/123: dread - da/f22 zero size 2024-04-25T01:35:42.693 INFO:tasks.workunit.client.0.smithi033.stdout:3/128: dwrite d8/dd/d11/f24 [0,4194304] 0 2024-04-25T01:35:42.695 INFO:tasks.workunit.client.0.smithi033.stdout:9/126: symlink d0/d2/dd/df/d1b/l2a 0 2024-04-25T01:35:42.697 INFO:tasks.workunit.client.0.smithi033.stdout:7/110: link d4/f5 d4/dd/f1c 0 2024-04-25T01:35:42.697 INFO:tasks.workunit.client.0.smithi033.stdout:1/101: mkdir d2/d5/d1b 0 2024-04-25T01:35:42.699 INFO:tasks.workunit.client.0.smithi033.stdout:7/111: read - d4/f17 zero size 2024-04-25T01:35:42.700 INFO:tasks.workunit.client.0.smithi033.stdout:9/127: dread - d0/d11/f18 zero size 2024-04-25T01:35:42.701 INFO:tasks.workunit.client.0.smithi033.stdout:1/102: write d2/ff [1646967,75784] 0 2024-04-25T01:35:42.701 INFO:tasks.workunit.client.0.smithi033.stdout:4/158: write d6/f1f [275868,4489] 0 2024-04-25T01:35:42.702 INFO:tasks.workunit.client.0.smithi033.stdout:2/124: mkdir da/d27 0 2024-04-25T01:35:42.704 INFO:tasks.workunit.client.0.smithi033.stdout:9/128: readlink d0/d2/dd/df/d1b/l21 0 2024-04-25T01:35:42.705 INFO:tasks.workunit.client.0.smithi033.stdout:4/159: chown d6/d14/d1d/c30 945 1 2024-04-25T01:35:42.706 INFO:tasks.workunit.client.0.smithi033.stdout:0/135: dread d7/da/f13 [0,4194304] 0 2024-04-25T01:35:42.706 INFO:tasks.workunit.client.0.smithi033.stdout:3/129: creat d8/dd/d11/d1e/f29 x:0 0 0 2024-04-25T01:35:42.709 INFO:tasks.workunit.client.0.smithi033.stdout:3/130: write d8/dd/f25 [417152,6106] 0 2024-04-25T01:35:42.709 INFO:tasks.workunit.client.0.smithi033.stdout:0/136: chown d7/da/df/l12 123 1 2024-04-25T01:35:42.709 INFO:tasks.workunit.client.0.smithi033.stdout:2/125: creat da/f28 x:0 0 0 2024-04-25T01:35:42.710 INFO:tasks.workunit.client.0.smithi033.stdout:2/126: readlink da/l1f 0 2024-04-25T01:35:42.711 INFO:tasks.workunit.client.0.smithi033.stdout:1/103: mknod d2/d5/c1c 0 2024-04-25T01:35:42.711 INFO:tasks.workunit.client.0.smithi033.stdout:2/127: dread - da/f15 zero size 2024-04-25T01:35:42.713 INFO:tasks.workunit.client.0.smithi033.stdout:3/131: fsync d8/dd/f25 0 2024-04-25T01:35:42.713 INFO:tasks.workunit.client.0.smithi033.stdout:7/112: rename d4/f1b to d4/f1d 0 2024-04-25T01:35:42.713 INFO:tasks.workunit.client.0.smithi033.stdout:2/128: stat da/fd 0 2024-04-25T01:35:42.715 INFO:tasks.workunit.client.0.smithi033.stdout:9/129: fsync d0/f29 0 2024-04-25T01:35:42.715 INFO:tasks.workunit.client.0.smithi033.stdout:1/104: write d2/d5/fd [695912,94908] 0 2024-04-25T01:35:42.716 INFO:tasks.workunit.client.0.smithi033.stdout:7/113: read - d4/dd/ff zero size 2024-04-25T01:35:42.716 INFO:tasks.workunit.client.0.smithi033.stdout:4/160: creat d6/d12/f32 x:0 0 0 2024-04-25T01:35:42.720 INFO:tasks.workunit.client.0.smithi033.stdout:0/137: mkdir d7/da/df/d2a 0 2024-04-25T01:35:42.721 INFO:tasks.workunit.client.0.smithi033.stdout:6/87: truncate d4/f14 2193450 0 2024-04-25T01:35:42.729 INFO:tasks.workunit.client.0.smithi033.stdout:4/161: chown d6/l1b 12 1 2024-04-25T01:35:42.730 INFO:tasks.workunit.client.0.smithi033.stdout:3/132: rmdir d8/dd/d11/d1e 39 2024-04-25T01:35:42.732 INFO:tasks.workunit.client.0.smithi033.stdout:0/138: symlink d7/da/d23/l2b 0 2024-04-25T01:35:42.734 INFO:tasks.workunit.client.0.smithi033.stdout:0/139: dread - d7/da/df/f17 zero size 2024-04-25T01:35:42.735 INFO:tasks.workunit.client.0.smithi033.stdout:0/140: write d7/da/f13 [53400,72032] 0 2024-04-25T01:35:42.738 INFO:tasks.workunit.client.0.smithi033.stdout:0/141: write d7/da/df/d14/f15 [879267,30305] 0 2024-04-25T01:35:42.738 INFO:tasks.workunit.client.0.smithi033.stdout:6/88: creat d4/d1d/f1f x:0 0 0 2024-04-25T01:35:42.741 INFO:tasks.workunit.client.0.smithi033.stdout:4/162: unlink d6/d14/c10 0 2024-04-25T01:35:42.741 INFO:tasks.workunit.client.0.smithi033.stdout:6/89: dread - d4/d1d/f1f zero size 2024-04-25T01:35:42.743 INFO:tasks.workunit.client.0.smithi033.stdout:3/133: mknod d8/c2a 0 2024-04-25T01:35:42.745 INFO:tasks.workunit.client.0.smithi033.stdout:7/114: dwrite d4/f1d [0,4194304] 0 2024-04-25T01:35:42.748 INFO:tasks.workunit.client.0.smithi033.stdout:9/130: getdents d0/d2/dd/df 0 2024-04-25T01:35:42.751 INFO:tasks.workunit.client.0.smithi033.stdout:1/105: getdents d2 0 2024-04-25T01:35:42.751 INFO:tasks.workunit.client.0.smithi033.stdout:2/129: dread f1 [0,4194304] 0 2024-04-25T01:35:42.752 INFO:tasks.workunit.client.0.smithi033.stdout:1/106: rename d2 to d2/d5/d1b/d1d 22 2024-04-25T01:35:42.753 INFO:tasks.workunit.client.0.smithi033.stdout:1/107: readlink d2/l4 0 2024-04-25T01:35:42.754 INFO:tasks.workunit.client.0.smithi033.stdout:6/90: creat d4/d15/f20 x:0 0 0 2024-04-25T01:35:42.756 INFO:tasks.workunit.client.0.smithi033.stdout:3/134: creat d8/dd/d1d/f2b x:0 0 0 2024-04-25T01:35:42.758 INFO:tasks.workunit.client.0.smithi033.stdout:9/131: creat d0/d2/dd/df/f2b x:0 0 0 2024-04-25T01:35:42.758 INFO:tasks.workunit.client.0.smithi033.stdout:0/142: mknod d7/d1b/c2c 0 2024-04-25T01:35:42.761 INFO:tasks.workunit.client.0.smithi033.stdout:1/108: creat d2/d11/f1e x:0 0 0 2024-04-25T01:35:42.763 INFO:tasks.workunit.client.0.smithi033.stdout:9/132: chown d0/d11/f16 348692866 1 2024-04-25T01:35:42.763 INFO:tasks.workunit.client.0.smithi033.stdout:6/91: dread d4/f19 [0,4194304] 0 2024-04-25T01:35:42.763 INFO:tasks.workunit.client.0.smithi033.stdout:2/130: mknod da/d27/c29 0 2024-04-25T01:35:42.765 INFO:tasks.workunit.client.0.smithi033.stdout:2/131: fdatasync da/f26 0 2024-04-25T01:35:42.768 INFO:tasks.workunit.client.0.smithi033.stdout:3/135: dwrite d8/f1b [0,4194304] 0 2024-04-25T01:35:42.769 INFO:tasks.workunit.client.0.smithi033.stdout:1/109: mknod d2/d5/c1f 0 2024-04-25T01:35:42.775 INFO:tasks.workunit.client.0.smithi033.stdout:6/92: unlink d4/f18 0 2024-04-25T01:35:42.775 INFO:tasks.workunit.client.0.smithi033.stdout:4/163: getdents d6/d14 0 2024-04-25T01:35:42.778 INFO:tasks.workunit.client.0.smithi033.stdout:9/133: mknod d0/c2c 0 2024-04-25T01:35:42.781 INFO:tasks.workunit.client.0.smithi033.stdout:4/164: fdatasync d6/d12/f26 0 2024-04-25T01:35:42.783 INFO:tasks.workunit.client.0.smithi033.stdout:3/136: creat d8/dd/f2c x:0 0 0 2024-04-25T01:35:42.783 INFO:tasks.workunit.client.0.smithi033.stdout:1/110: link d2/f3 d2/f20 0 2024-04-25T01:35:42.784 INFO:tasks.workunit.client.0.smithi033.stdout:2/132: dwrite da/f12 [0,4194304] 0 2024-04-25T01:35:42.787 INFO:tasks.workunit.client.0.smithi033.stdout:6/93: dwrite d4/d15/f20 [0,4194304] 0 2024-04-25T01:35:42.787 INFO:tasks.workunit.client.0.smithi033.stdout:9/134: write d0/d11/f27 [652501,82867] 0 2024-04-25T01:35:42.791 INFO:tasks.workunit.client.0.smithi033.stdout:6/94: chown l0 186 1 2024-04-25T01:35:42.793 INFO:tasks.workunit.client.0.smithi033.stdout:2/133: truncate da/d23/f25 629678 0 2024-04-25T01:35:42.797 INFO:tasks.workunit.client.0.smithi033.stdout:0/143: dwrite d7/f29 [4194304,4194304] 0 2024-04-25T01:35:42.799 INFO:tasks.workunit.client.0.smithi033.stdout:2/134: write da/fd [3562205,115753] 0 2024-04-25T01:35:42.805 INFO:tasks.workunit.client.0.smithi033.stdout:4/165: write d6/f16 [483531,126236] 0 2024-04-25T01:35:42.810 INFO:tasks.workunit.client.0.smithi033.stdout:3/137: dwrite d8/dd/d11/d1e/f29 [0,4194304] 0 2024-04-25T01:35:42.813 INFO:tasks.workunit.client.0.smithi033.stdout:2/135: rename da/c1c to da/d23/c2a 0 2024-04-25T01:35:42.813 INFO:tasks.workunit.client.0.smithi033.stdout:9/135: creat d0/f2d x:0 0 0 2024-04-25T01:35:42.816 INFO:tasks.workunit.client.0.smithi033.stdout:2/136: symlink da/d23/l2b 0 2024-04-25T01:35:42.818 INFO:tasks.workunit.client.0.smithi033.stdout:1/111: mknod d2/c21 0 2024-04-25T01:35:42.819 INFO:tasks.workunit.client.0.smithi033.stdout:0/144: mkdir d7/d2d 0 2024-04-25T01:35:42.820 INFO:tasks.workunit.client.0.smithi033.stdout:9/136: read d0/d2/dd/f13 [3426441,88270] 0 2024-04-25T01:35:42.822 INFO:tasks.workunit.client.0.smithi033.stdout:3/138: chown d8/dd/d11/c19 9102 1 2024-04-25T01:35:42.824 INFO:tasks.workunit.client.0.smithi033.stdout:6/95: getdents d4 0 2024-04-25T01:35:42.825 INFO:tasks.workunit.client.0.smithi033.stdout:3/139: truncate d8/dd/d11/d1e/f21 4775723 0 2024-04-25T01:35:42.826 INFO:tasks.workunit.client.0.smithi033.stdout:3/140: chown d8/dd/d11/d1e/f21 40957700 1 2024-04-25T01:35:42.829 INFO:tasks.workunit.client.0.smithi033.stdout:2/137: mkdir da/d2c 0 2024-04-25T01:35:42.830 INFO:tasks.workunit.client.0.smithi033.stdout:2/138: chown da/d2c 37 1 2024-04-25T01:35:42.831 INFO:tasks.workunit.client.0.smithi033.stdout:4/166: truncate d6/d14/d1d/f2c 3793842 0 2024-04-25T01:35:42.833 INFO:tasks.workunit.client.0.smithi033.stdout:9/137: creat d0/d2/dd/df/d1b/f2e x:0 0 0 2024-04-25T01:35:42.835 INFO:tasks.workunit.client.0.smithi033.stdout:3/141: dwrite d8/dd/d11/f24 [4194304,4194304] 0 2024-04-25T01:35:42.836 INFO:tasks.workunit.client.0.smithi033.stdout:4/167: dread d6/f1f [0,4194304] 0 2024-04-25T01:35:42.837 INFO:tasks.workunit.client.0.smithi033.stdout:0/145: mknod d7/d2d/c2e 0 2024-04-25T01:35:42.839 INFO:tasks.workunit.client.0.smithi033.stdout:2/139: creat da/d27/f2d x:0 0 0 2024-04-25T01:35:42.840 INFO:tasks.workunit.client.0.smithi033.stdout:0/146: truncate d7/da/df/f17 950614 0 2024-04-25T01:35:42.841 INFO:tasks.workunit.client.0.smithi033.stdout:3/142: dread d8/dd/d11/f24 [0,4194304] 0 2024-04-25T01:35:42.844 INFO:tasks.workunit.client.0.smithi033.stdout:0/147: write d7/da/df/f17 [1032041,66981] 0 2024-04-25T01:35:42.850 INFO:tasks.workunit.client.0.smithi033.stdout:2/140: dwrite da/ff [0,4194304] 0 2024-04-25T01:35:42.854 INFO:tasks.workunit.client.0.smithi033.stdout:9/138: dwrite d0/f9 [0,4194304] 0 2024-04-25T01:35:42.861 INFO:tasks.workunit.client.0.smithi033.stdout:2/141: dwrite da/f28 [0,4194304] 0 2024-04-25T01:35:42.864 INFO:tasks.workunit.client.0.smithi033.stdout:3/143: creat d8/dd/d1d/f2d x:0 0 0 2024-04-25T01:35:42.864 INFO:tasks.workunit.client.0.smithi033.stdout:1/112: truncate d2/d5/f18 1367497 0 2024-04-25T01:35:42.870 INFO:tasks.workunit.client.0.smithi033.stdout:6/96: getdents d4/d15 0 2024-04-25T01:35:42.872 INFO:tasks.workunit.client.0.smithi033.stdout:2/142: symlink da/d23/l2e 0 2024-04-25T01:35:42.876 INFO:tasks.workunit.client.0.smithi033.stdout:2/143: write da/d27/f2d [361322,49684] 0 2024-04-25T01:35:42.878 INFO:tasks.workunit.client.0.smithi033.stdout:9/139: creat d0/f2f x:0 0 0 2024-04-25T01:35:42.878 INFO:tasks.workunit.client.0.smithi033.stdout:6/97: write d4/f19 [2002126,32652] 0 2024-04-25T01:35:42.879 INFO:tasks.workunit.client.0.smithi033.stdout:2/144: chown da/f26 23577 1 2024-04-25T01:35:42.880 INFO:tasks.workunit.client.0.smithi033.stdout:3/144: dwrite d8/dd/f15 [4194304,4194304] 0 2024-04-25T01:35:42.881 INFO:tasks.workunit.client.0.smithi033.stdout:1/113: rename d2/c21 to d2/c22 0 2024-04-25T01:35:42.883 INFO:tasks.workunit.client.0.smithi033.stdout:0/148: dwrite d7/da/df/d14/f24 [0,4194304] 0 2024-04-25T01:35:42.891 INFO:tasks.workunit.client.0.smithi033.stdout:9/140: mknod d0/c30 0 2024-04-25T01:35:42.895 INFO:tasks.workunit.client.0.smithi033.stdout:0/149: mknod d7/d2d/c2f 0 2024-04-25T01:35:42.900 INFO:tasks.workunit.client.0.smithi033.stdout:3/145: creat d8/dd/d11/d1e/d27/f2e x:0 0 0 2024-04-25T01:35:42.900 INFO:tasks.workunit.client.0.smithi033.stdout:2/145: dwrite da/d23/f25 [0,4194304] 0 2024-04-25T01:35:42.902 INFO:tasks.workunit.client.0.smithi033.stdout:6/98: dwrite d4/f1b [0,4194304] 0 2024-04-25T01:35:42.903 INFO:tasks.workunit.client.0.smithi033.stdout:0/150: write d7/da/df/d14/f15 [678794,42329] 0 2024-04-25T01:35:42.907 INFO:tasks.workunit.client.0.smithi033.stdout:1/114: dwrite d2/f16 [0,4194304] 0 2024-04-25T01:35:42.908 INFO:tasks.workunit.client.0.smithi033.stdout:3/146: write d8/dd/d1d/f2b [984151,12300] 0 2024-04-25T01:35:42.909 INFO:tasks.workunit.client.0.smithi033.stdout:2/146: creat da/f2f x:0 0 0 2024-04-25T01:35:42.910 INFO:tasks.workunit.client.0.smithi033.stdout:9/141: link d0/d2/dd/l1e d0/d2/dd/l31 0 2024-04-25T01:35:42.911 INFO:tasks.workunit.client.0.smithi033.stdout:8/169: sync 2024-04-25T01:35:42.913 INFO:tasks.workunit.client.0.smithi033.stdout:2/147: readlink da/d23/l2b 0 2024-04-25T01:35:42.919 INFO:tasks.workunit.client.0.smithi033.stdout:8/170: symlink d6/d10/d2e/l3d 0 2024-04-25T01:35:42.921 INFO:tasks.workunit.client.0.smithi033.stdout:0/151: dwrite d7/da/df/f28 [0,4194304] 0 2024-04-25T01:35:42.923 INFO:tasks.workunit.client.0.smithi033.stdout:3/147: dwrite d8/dd/f25 [0,4194304] 0 2024-04-25T01:35:42.925 INFO:tasks.workunit.client.0.smithi033.stdout:1/115: chown d2/d5/f18 125919879 1 2024-04-25T01:35:42.926 INFO:tasks.workunit.client.0.smithi033.stdout:0/152: write d7/da/df/f19 [1940761,125148] 0 2024-04-25T01:35:42.928 INFO:tasks.workunit.client.0.smithi033.stdout:9/142: getdents d0 0 2024-04-25T01:35:42.931 INFO:tasks.workunit.client.0.smithi033.stdout:4/168: write d6/d14/d1d/f2e [197487,128434] 0 2024-04-25T01:35:42.937 INFO:tasks.workunit.client.0.smithi033.stdout:3/148: rmdir d8/dd 39 2024-04-25T01:35:42.939 INFO:tasks.workunit.client.0.smithi033.stdout:6/99: write d4/f14 [1125295,61524] 0 2024-04-25T01:35:42.940 INFO:tasks.workunit.client.0.smithi033.stdout:1/116: truncate d2/d11/f1e 668954 0 2024-04-25T01:35:42.942 INFO:tasks.workunit.client.0.smithi033.stdout:0/153: symlink d7/da/df/d14/l30 0 2024-04-25T01:35:42.943 INFO:tasks.workunit.client.0.smithi033.stdout:4/169: fsync d6/f1f 0 2024-04-25T01:35:42.946 INFO:tasks.workunit.client.0.smithi033.stdout:4/170: write d6/d14/d1d/f28 [40064,90031] 0 2024-04-25T01:35:42.947 INFO:tasks.workunit.client.0.smithi033.stdout:6/100: unlink d4/f19 0 2024-04-25T01:35:42.947 INFO:tasks.workunit.client.0.smithi033.stdout:4/171: chown d6/d14/d1d/c2d 257998540 1 2024-04-25T01:35:42.948 INFO:tasks.workunit.client.0.smithi033.stdout:6/101: truncate d4/d1d/f1f 231651 0 2024-04-25T01:35:42.951 INFO:tasks.workunit.client.0.smithi033.stdout:3/149: creat d8/dd/f2f x:0 0 0 2024-04-25T01:35:42.952 INFO:tasks.workunit.client.0.smithi033.stdout:6/102: dread d4/f12 [0,4194304] 0 2024-04-25T01:35:42.955 INFO:tasks.workunit.client.0.smithi033.stdout:1/117: unlink d2/l4 0 2024-04-25T01:35:42.956 INFO:tasks.workunit.client.0.smithi033.stdout:2/148: truncate da/f13 3328395 0 2024-04-25T01:35:42.956 INFO:tasks.workunit.client.0.smithi033.stdout:1/118: fdatasync d2/d5/fe 0 2024-04-25T01:35:42.957 INFO:tasks.workunit.client.0.smithi033.stdout:4/172: rename d6/d14/d1d/f24 to d6/d14/f33 0 2024-04-25T01:35:42.958 INFO:tasks.workunit.client.0.smithi033.stdout:3/150: rename d8/dd to d8/dd/d30 22 2024-04-25T01:35:42.961 INFO:tasks.workunit.client.0.smithi033.stdout:6/103: mknod d4/d15/c21 0 2024-04-25T01:35:42.965 INFO:tasks.workunit.client.0.smithi033.stdout:1/119: symlink d2/d11/l23 0 2024-04-25T01:35:42.967 INFO:tasks.workunit.client.0.smithi033.stdout:3/151: mknod d8/dd/d1d/c31 0 2024-04-25T01:35:42.969 INFO:tasks.workunit.client.0.smithi033.stdout:6/104: mknod d4/c22 0 2024-04-25T01:35:42.969 INFO:tasks.workunit.client.0.smithi033.stdout:3/152: write d8/dd/d11/d1e/d27/f2e [814397,104654] 0 2024-04-25T01:35:42.975 INFO:tasks.workunit.client.0.smithi033.stdout:6/105: mknod d4/c23 0 2024-04-25T01:35:42.977 INFO:tasks.workunit.client.0.smithi033.stdout:0/154: truncate d7/da/df/d14/f22 2717235 0 2024-04-25T01:35:42.977 INFO:tasks.workunit.client.0.smithi033.stdout:6/106: chown d4/d1d/f1f 91001 1 2024-04-25T01:35:42.980 INFO:tasks.workunit.client.0.smithi033.stdout:3/153: dwrite d8/dd/f2c [0,4194304] 0 2024-04-25T01:35:42.982 INFO:tasks.workunit.client.0.smithi033.stdout:3/154: write f3 [1924111,71693] 0 2024-04-25T01:35:42.984 INFO:tasks.workunit.client.0.smithi033.stdout:0/155: unlink d7/da/f1f 0 2024-04-25T01:35:42.986 INFO:tasks.workunit.client.0.smithi033.stdout:4/173: rmdir d6/d14/d1d 39 2024-04-25T01:35:42.987 INFO:tasks.workunit.client.0.smithi033.stdout:2/149: write da/f13 [3106184,117096] 0 2024-04-25T01:35:42.989 INFO:tasks.workunit.client.0.smithi033.stdout:3/155: mkdir d8/dd/d11/d1e/d32 0 2024-04-25T01:35:42.994 INFO:tasks.workunit.client.0.smithi033.stdout:1/120: write d2/d5/f18 [1204004,75236] 0 2024-04-25T01:35:42.999 INFO:tasks.workunit.client.0.smithi033.stdout:4/174: chown d6/d14/cb 8107279 1 2024-04-25T01:35:42.999 INFO:tasks.workunit.client.0.smithi033.stdout:2/150: mkdir da/d2c/d30 0 2024-04-25T01:35:43.001 INFO:tasks.workunit.client.0.smithi033.stdout:6/107: getdents d4/d15 0 2024-04-25T01:35:43.002 INFO:tasks.workunit.client.0.smithi033.stdout:1/121: symlink d2/d5/d1b/l24 0 2024-04-25T01:35:43.003 INFO:tasks.workunit.client.0.smithi033.stdout:4/175: write d6/f18 [1250565,59364] 0 2024-04-25T01:35:43.005 INFO:tasks.workunit.client.0.smithi033.stdout:1/122: fsync d2/f16 0 2024-04-25T01:35:43.005 INFO:tasks.workunit.client.0.smithi033.stdout:3/156: dwrite d8/dd/f2c [4194304,4194304] 0 2024-04-25T01:35:43.009 INFO:tasks.workunit.client.0.smithi033.stdout:4/176: fdatasync d6/f8 0 2024-04-25T01:35:43.013 INFO:tasks.workunit.client.0.smithi033.stdout:1/123: creat d2/d5/f25 x:0 0 0 2024-04-25T01:35:43.015 INFO:tasks.workunit.client.0.smithi033.stdout:6/108: dwrite f3 [0,4194304] 0 2024-04-25T01:35:43.016 INFO:tasks.workunit.client.0.smithi033.stdout:1/124: read d2/f16 [3790168,63507] 0 2024-04-25T01:35:43.022 INFO:tasks.workunit.client.0.smithi033.stdout:6/109: mkdir d4/d24 0 2024-04-25T01:35:43.022 INFO:tasks.workunit.client.0.smithi033.stdout:3/157: dwrite d8/dd/d1d/f2d [0,4194304] 0 2024-04-25T01:35:43.026 INFO:tasks.workunit.client.0.smithi033.stdout:6/110: mknod d4/d15/c25 0 2024-04-25T01:35:43.027 INFO:tasks.workunit.client.0.smithi033.stdout:3/158: fsync d8/dd/f2c 0 2024-04-25T01:35:43.029 INFO:tasks.workunit.client.0.smithi033.stdout:3/159: write d8/dd/d11/d1e/f21 [5785995,84821] 0 2024-04-25T01:35:43.029 INFO:tasks.workunit.client.0.smithi033.stdout:1/125: creat d2/f26 x:0 0 0 2024-04-25T01:35:43.030 INFO:tasks.workunit.client.0.smithi033.stdout:4/177: getdents d6/d14/d1d 0 2024-04-25T01:35:43.031 INFO:tasks.workunit.client.0.smithi033.stdout:4/178: read d6/f1f [266929,4471] 0 2024-04-25T01:35:43.032 INFO:tasks.workunit.client.0.smithi033.stdout:1/126: write d2/d5/fe [4634973,39771] 0 2024-04-25T01:35:43.034 INFO:tasks.workunit.client.0.smithi033.stdout:6/111: rename d4/la to d4/d15/l26 0 2024-04-25T01:35:43.039 INFO:tasks.workunit.client.0.smithi033.stdout:4/179: symlink d6/l34 0 2024-04-25T01:35:43.040 INFO:tasks.workunit.client.0.smithi033.stdout:3/160: dread d8/dd/d1d/f2b [0,4194304] 0 2024-04-25T01:35:43.040 INFO:tasks.workunit.client.0.smithi033.stdout:6/112: symlink d4/d24/l27 0 2024-04-25T01:35:43.042 INFO:tasks.workunit.client.0.smithi033.stdout:1/127: dwrite d2/d5/fe [0,4194304] 0 2024-04-25T01:35:43.049 INFO:tasks.workunit.client.0.smithi033.stdout:3/161: symlink d8/l33 0 2024-04-25T01:35:43.050 INFO:tasks.workunit.client.0.smithi033.stdout:1/128: fsync d2/f20 0 2024-04-25T01:35:43.068 INFO:tasks.workunit.client.0.smithi033.stdout:5/179: sync 2024-04-25T01:35:43.068 INFO:tasks.workunit.client.0.smithi033.stdout:7/115: sync 2024-04-25T01:35:43.070 INFO:tasks.workunit.client.0.smithi033.stdout:7/116: readlink d4/d10/l1a 0 2024-04-25T01:35:43.071 INFO:tasks.workunit.client.0.smithi033.stdout:6/113: dwrite d4/f9 [0,4194304] 0 2024-04-25T01:35:43.074 INFO:tasks.workunit.client.0.smithi033.stdout:1/129: symlink d2/d5/d1b/l27 0 2024-04-25T01:35:43.074 INFO:tasks.workunit.client.0.smithi033.stdout:5/180: rmdir d1c/d21 39 2024-04-25T01:35:43.083 INFO:tasks.workunit.client.0.smithi033.stdout:7/117: link c2 d4/dd/c1e 0 2024-04-25T01:35:43.083 INFO:tasks.workunit.client.0.smithi033.stdout:1/130: dwrite d2/d5/f14 [0,4194304] 0 2024-04-25T01:35:43.085 INFO:tasks.workunit.client.0.smithi033.stdout:4/180: dread d6/d12/f1a [0,4194304] 0 2024-04-25T01:35:43.085 INFO:tasks.workunit.client.0.smithi033.stdout:5/181: creat d1c/f38 x:0 0 0 2024-04-25T01:35:43.186 INFO:tasks.workunit.client.0.smithi033.stdout:1/131: rmdir d2/d5 39 2024-04-25T01:35:43.186 INFO:tasks.workunit.client.0.smithi033.stdout:3/162: truncate d8/dd/d11/d1e/f29 2621359 0 2024-04-25T01:35:43.187 INFO:tasks.workunit.client.0.smithi033.stdout:7/118: mkdir d4/d10/d13/d1f 0 2024-04-25T01:35:43.189 INFO:tasks.workunit.client.0.smithi033.stdout:7/119: stat d4/f7 0 2024-04-25T01:35:43.190 INFO:tasks.workunit.client.0.smithi033.stdout:5/182: creat d1c/d21/f39 x:0 0 0 2024-04-25T01:35:43.190 INFO:tasks.workunit.client.0.smithi033.stdout:1/132: read - d2/d5/f25 zero size 2024-04-25T01:35:43.192 INFO:tasks.workunit.client.0.smithi033.stdout:1/133: chown d2/d11/f1e 744917585 1 2024-04-25T01:35:43.193 INFO:tasks.workunit.client.0.smithi033.stdout:7/120: write d4/f9 [223191,129380] 0 2024-04-25T01:35:43.194 INFO:tasks.workunit.client.0.smithi033.stdout:4/181: dwrite d6/f1e [0,4194304] 0 2024-04-25T01:35:43.197 INFO:tasks.workunit.client.0.smithi033.stdout:5/183: write f14 [3800541,46760] 0 2024-04-25T01:35:43.197 INFO:tasks.workunit.client.0.smithi033.stdout:3/163: rmdir d8/dd/d11/d1e/d32 0 2024-04-25T01:35:43.199 INFO:tasks.workunit.client.0.smithi033.stdout:7/121: fdatasync d4/f9 0 2024-04-25T01:35:43.201 INFO:tasks.workunit.client.0.smithi033.stdout:7/122: fdatasync d4/f1d 0 2024-04-25T01:35:43.205 INFO:tasks.workunit.client.0.smithi033.stdout:1/134: dwrite d2/d5/fd [0,4194304] 0 2024-04-25T01:35:43.207 INFO:tasks.workunit.client.0.smithi033.stdout:1/135: write d2/f16 [5194552,19754] 0 2024-04-25T01:35:43.208 INFO:tasks.workunit.client.0.smithi033.stdout:3/164: creat d8/dd/f34 x:0 0 0 2024-04-25T01:35:43.211 INFO:tasks.workunit.client.0.smithi033.stdout:5/184: rename d1c/f20 to d1c/d21/f3a 0 2024-04-25T01:35:43.214 INFO:tasks.workunit.client.0.smithi033.stdout:1/136: mkdir d2/d11/d28 0 2024-04-25T01:35:43.218 INFO:tasks.workunit.client.0.smithi033.stdout:7/123: creat d4/d10/d13/d1f/f20 x:0 0 0 2024-04-25T01:35:43.220 INFO:tasks.workunit.client.0.smithi033.stdout:5/185: dwrite d1c/d21/f26 [0,4194304] 0 2024-04-25T01:35:43.221 INFO:tasks.workunit.client.0.smithi033.stdout:5/186: chown d1c/d21/f39 161255667 1 2024-04-25T01:35:43.221 INFO:tasks.workunit.client.0.smithi033.stdout:4/182: mkdir d6/d14/d35 0 2024-04-25T01:35:43.222 INFO:tasks.workunit.client.0.smithi033.stdout:4/183: dread - d6/d12/f2f zero size 2024-04-25T01:35:43.224 INFO:tasks.workunit.client.0.smithi033.stdout:3/165: symlink d8/dd/d11/l35 0 2024-04-25T01:35:43.225 INFO:tasks.workunit.client.0.smithi033.stdout:7/124: chown d4/dd/c1e 254 1 2024-04-25T01:35:43.229 INFO:tasks.workunit.client.0.smithi033.stdout:3/166: creat d8/dd/d11/d1e/d27/f36 x:0 0 0 2024-04-25T01:35:43.229 INFO:tasks.workunit.client.0.smithi033.stdout:7/125: symlink d4/d10/l21 0 2024-04-25T01:35:43.231 INFO:tasks.workunit.client.0.smithi033.stdout:3/167: write d8/dd/d11/d1e/f21 [1261573,57775] 0 2024-04-25T01:35:43.232 INFO:tasks.workunit.client.0.smithi033.stdout:1/137: creat d2/f29 x:0 0 0 2024-04-25T01:35:43.233 INFO:tasks.workunit.client.0.smithi033.stdout:1/138: chown d2/d5/fe 29760529 1 2024-04-25T01:35:43.234 INFO:tasks.workunit.client.0.smithi033.stdout:1/139: dread - d2/d5/f25 zero size 2024-04-25T01:35:43.234 INFO:tasks.workunit.client.0.smithi033.stdout:5/187: symlink d1c/d2a/l3b 0 2024-04-25T01:35:43.235 INFO:tasks.workunit.client.0.smithi033.stdout:5/188: dread - d1c/f37 zero size 2024-04-25T01:35:43.238 INFO:tasks.workunit.client.0.smithi033.stdout:5/189: write d1c/f32 [379127,19927] 0 2024-04-25T01:35:43.238 INFO:tasks.workunit.client.0.smithi033.stdout:5/190: dread - d1c/f1f zero size 2024-04-25T01:35:43.238 INFO:tasks.workunit.client.0.smithi033.stdout:1/140: truncate d2/ff 3620951 0 2024-04-25T01:35:43.246 INFO:tasks.workunit.client.0.smithi033.stdout:5/191: dwrite d1c/f1d [0,4194304] 0 2024-04-25T01:35:43.249 INFO:tasks.workunit.client.0.smithi033.stdout:4/184: write d6/f1f [1164362,117536] 0 2024-04-25T01:35:43.250 INFO:tasks.workunit.client.0.smithi033.stdout:5/192: creat d1c/f3c x:0 0 0 2024-04-25T01:35:43.252 INFO:tasks.workunit.client.0.smithi033.stdout:7/126: write d4/dd/f1c [5022865,96045] 0 2024-04-25T01:35:43.253 INFO:tasks.workunit.client.0.smithi033.stdout:4/185: write d6/d12/f1a [1507113,120533] 0 2024-04-25T01:35:43.257 INFO:tasks.workunit.client.0.smithi033.stdout:1/141: unlink d2/f3 0 2024-04-25T01:35:43.260 INFO:tasks.workunit.client.0.smithi033.stdout:1/142: creat d2/d11/f2a x:0 0 0 2024-04-25T01:35:43.261 INFO:tasks.workunit.client.0.smithi033.stdout:5/193: mkdir d1c/d2a/d3d 0 2024-04-25T01:35:43.263 INFO:tasks.workunit.client.0.smithi033.stdout:4/186: symlink d6/d14/d35/l36 0 2024-04-25T01:35:43.265 INFO:tasks.workunit.client.0.smithi033.stdout:1/143: rename d2/f16 to d2/d5/d1b/f2b 0 2024-04-25T01:35:43.265 INFO:tasks.workunit.client.0.smithi033.stdout:1/144: stat d2/d11/f1a 0 2024-04-25T01:35:43.266 INFO:tasks.workunit.client.0.smithi033.stdout:5/194: readlink d1c/d21/l27 0 2024-04-25T01:35:43.267 INFO:tasks.workunit.client.0.smithi033.stdout:7/127: link d4/d10/l1a d4/d10/l22 0 2024-04-25T01:35:43.271 INFO:tasks.workunit.client.0.smithi033.stdout:4/187: symlink d6/d14/l37 0 2024-04-25T01:35:43.277 INFO:tasks.workunit.client.0.smithi033.stdout:5/195: dwrite d1c/f38 [0,4194304] 0 2024-04-25T01:35:43.279 INFO:tasks.workunit.client.0.smithi033.stdout:7/128: dwrite f0 [0,4194304] 0 2024-04-25T01:35:43.282 INFO:tasks.workunit.client.0.smithi033.stdout:1/145: getdents d2/d5/d1b 0 2024-04-25T01:35:43.289 INFO:tasks.workunit.client.0.smithi033.stdout:1/146: unlink d2/ff 0 2024-04-25T01:35:43.291 INFO:tasks.workunit.client.0.smithi033.stdout:7/129: dwrite d4/d10/d13/d1f/f20 [0,4194304] 0 2024-04-25T01:35:43.292 INFO:tasks.workunit.client.0.smithi033.stdout:7/130: fsync d4/dd/ff 0 2024-04-25T01:35:43.293 INFO:tasks.workunit.client.0.smithi033.stdout:1/147: readlink d2/l15 0 2024-04-25T01:35:43.296 INFO:tasks.workunit.client.0.smithi033.stdout:7/131: mkdir d4/d23 0 2024-04-25T01:35:43.297 INFO:tasks.workunit.client.0.smithi033.stdout:7/132: readlink d4/d10/l21 0 2024-04-25T01:35:43.298 INFO:tasks.workunit.client.0.smithi033.stdout:1/148: creat d2/d11/d28/f2c x:0 0 0 2024-04-25T01:35:43.299 INFO:tasks.workunit.client.0.smithi033.stdout:1/149: dread - d2/f29 zero size 2024-04-25T01:35:43.301 INFO:tasks.workunit.client.0.smithi033.stdout:7/133: creat d4/d10/d13/d1f/f24 x:0 0 0 2024-04-25T01:35:43.303 INFO:tasks.workunit.client.0.smithi033.stdout:1/150: symlink d2/d11/l2d 0 2024-04-25T01:35:43.309 INFO:tasks.workunit.client.0.smithi033.stdout:7/134: dwrite d4/d10/d13/d1f/f24 [0,4194304] 0 2024-04-25T01:35:43.310 INFO:tasks.workunit.client.0.smithi033.stdout:4/188: dread d6/f8 [4194304,4194304] 0 2024-04-25T01:35:43.312 INFO:tasks.workunit.client.0.smithi033.stdout:1/151: fsync d2/d11/d28/f2c 0 2024-04-25T01:35:43.322 INFO:tasks.workunit.client.0.smithi033.stdout:7/135: getdents d4/d10/d13/d1f 0 2024-04-25T01:35:43.323 INFO:tasks.workunit.client.0.smithi033.stdout:9/143: sync 2024-04-25T01:35:43.323 INFO:tasks.workunit.client.0.smithi033.stdout:8/171: sync 2024-04-25T01:35:43.324 INFO:tasks.workunit.client.0.smithi033.stdout:7/136: rmdir d4/d10/d13 39 2024-04-25T01:35:43.324 INFO:tasks.workunit.client.0.smithi033.stdout:8/172: readlink l0 0 2024-04-25T01:35:43.325 INFO:tasks.workunit.client.0.smithi033.stdout:7/137: chown d4 22 1 2024-04-25T01:35:43.326 INFO:tasks.workunit.client.0.smithi033.stdout:4/189: dwrite d6/d14/f31 [0,4194304] 0 2024-04-25T01:35:43.330 INFO:tasks.workunit.client.0.smithi033.stdout:9/144: fdatasync d0/d2/dd/f13 0 2024-04-25T01:35:43.331 INFO:tasks.workunit.client.0.smithi033.stdout:4/190: mkdir d6/d12/d38 0 2024-04-25T01:35:43.331 INFO:tasks.workunit.client.0.smithi033.stdout:1/152: dwrite d2/d5/fe [4194304,4194304] 0 2024-04-25T01:35:43.332 INFO:tasks.workunit.client.0.smithi033.stdout:9/145: dread - d0/d2/dd/f24 zero size 2024-04-25T01:35:43.337 INFO:tasks.workunit.client.0.smithi033.stdout:8/173: creat d6/f3e x:0 0 0 2024-04-25T01:35:43.337 INFO:tasks.workunit.client.0.smithi033.stdout:9/146: write d0/d11/f27 [1280835,48151] 0 2024-04-25T01:35:43.341 INFO:tasks.workunit.client.0.smithi033.stdout:4/191: mkdir d6/d14/d35/d39 0 2024-04-25T01:35:43.342 INFO:tasks.workunit.client.0.smithi033.stdout:4/192: fdatasync d6/d14/d1d/f20 0 2024-04-25T01:35:43.343 INFO:tasks.workunit.client.0.smithi033.stdout:9/147: creat d0/d2/f32 x:0 0 0 2024-04-25T01:35:43.343 INFO:tasks.workunit.client.0.smithi033.stdout:7/138: dread d4/fb [0,4194304] 0 2024-04-25T01:35:43.344 INFO:tasks.workunit.client.0.smithi033.stdout:4/193: dread - d6/d12/f2f zero size 2024-04-25T01:35:43.344 INFO:tasks.workunit.client.0.smithi033.stdout:1/153: chown d2/d11/d28/f2c 149967068 1 2024-04-25T01:35:43.346 INFO:tasks.workunit.client.0.smithi033.stdout:8/174: creat d6/d10/f3f x:0 0 0 2024-04-25T01:35:43.353 INFO:tasks.workunit.client.0.smithi033.stdout:1/154: mkdir d2/d2e 0 2024-04-25T01:35:43.355 INFO:tasks.workunit.client.0.smithi033.stdout:7/139: dwrite d4/fb [0,4194304] 0 2024-04-25T01:35:43.356 INFO:tasks.workunit.client.0.smithi033.stdout:9/148: link d0/d2/dd/f24 d0/f33 0 2024-04-25T01:35:43.357 INFO:tasks.workunit.client.0.smithi033.stdout:8/175: creat d6/f40 x:0 0 0 2024-04-25T01:35:43.358 INFO:tasks.workunit.client.0.smithi033.stdout:4/194: dread d6/f16 [0,4194304] 0 2024-04-25T01:35:43.366 INFO:tasks.workunit.client.0.smithi033.stdout:8/176: creat d6/f41 x:0 0 0 2024-04-25T01:35:43.366 INFO:tasks.workunit.client.0.smithi033.stdout:7/140: dwrite d4/dd/ff [0,4194304] 0 2024-04-25T01:35:43.376 INFO:tasks.workunit.client.0.smithi033.stdout:7/141: dread f0 [0,4194304] 0 2024-04-25T01:35:43.385 INFO:tasks.workunit.client.0.smithi033.stdout:9/149: rename d0/c12 to d0/d2/dd/d20/c34 0 2024-04-25T01:35:43.391 INFO:tasks.workunit.client.0.smithi033.stdout:8/177: dwrite d6/d10/d2e/f30 [0,4194304] 0 2024-04-25T01:35:43.392 INFO:tasks.workunit.client.0.smithi033.stdout:7/142: rename d4/ce to d4/d10/d13/c25 0 2024-04-25T01:35:43.392 INFO:tasks.workunit.client.0.smithi033.stdout:8/178: symlink d6/d10/d2e/l42 0 2024-04-25T01:35:43.393 INFO:tasks.workunit.client.0.smithi033.stdout:8/179: write d6/d10/f28 [647478,7598] 0 2024-04-25T01:35:43.395 INFO:tasks.workunit.client.0.smithi033.stdout:8/180: truncate d6/d10/f28 1489149 0 2024-04-25T01:35:43.396 INFO:tasks.workunit.client.0.smithi033.stdout:9/150: readlink d0/d2/l19 0 2024-04-25T01:35:43.397 INFO:tasks.workunit.client.0.smithi033.stdout:8/181: dread - d6/f21 zero size 2024-04-25T01:35:43.398 INFO:tasks.workunit.client.0.smithi033.stdout:7/143: creat d4/d10/f26 x:0 0 0 2024-04-25T01:35:43.399 INFO:tasks.workunit.client.0.smithi033.stdout:4/195: dread d6/d14/f33 [0,4194304] 0 2024-04-25T01:35:43.403 INFO:tasks.workunit.client.0.smithi033.stdout:4/196: mkdir d6/d14/d35/d3a 0 2024-04-25T01:35:43.404 INFO:tasks.workunit.client.0.smithi033.stdout:8/182: creat d6/d10/f43 x:0 0 0 2024-04-25T01:35:43.407 INFO:tasks.workunit.client.0.smithi033.stdout:7/144: dwrite d4/d10/f26 [0,4194304] 0 2024-04-25T01:35:43.407 INFO:tasks.workunit.client.0.smithi033.stdout:4/197: mkdir d6/d14/d1d/d3b 0 2024-04-25T01:35:43.408 INFO:tasks.workunit.client.0.smithi033.stdout:8/183: truncate d6/d10/f36 1009334 0 2024-04-25T01:35:43.409 INFO:tasks.workunit.client.0.smithi033.stdout:8/184: dread - d6/db/f26 zero size 2024-04-25T01:35:43.411 INFO:tasks.workunit.client.0.smithi033.stdout:4/198: symlink d6/l3c 0 2024-04-25T01:35:43.413 INFO:tasks.workunit.client.0.smithi033.stdout:7/145: getdents d4/d23 0 2024-04-25T01:35:43.421 INFO:tasks.workunit.client.0.smithi033.stdout:8/185: dread d6/d10/f28 [0,4194304] 0 2024-04-25T01:35:43.421 INFO:tasks.workunit.client.0.smithi033.stdout:4/199: dwrite d6/d12/f17 [0,4194304] 0 2024-04-25T01:35:43.422 INFO:tasks.workunit.client.0.smithi033.stdout:7/146: dread d4/d10/d13/d1f/f24 [0,4194304] 0 2024-04-25T01:35:43.422 INFO:tasks.workunit.client.0.smithi033.stdout:8/186: chown d6/db/l25 71 1 2024-04-25T01:35:43.429 INFO:tasks.workunit.client.0.smithi033.stdout:4/200: read d6/d12/f23 [91464,100628] 0 2024-04-25T01:35:43.429 INFO:tasks.workunit.client.0.smithi033.stdout:7/147: symlink d4/d23/l27 0 2024-04-25T01:35:43.429 INFO:tasks.workunit.client.0.smithi033.stdout:4/201: truncate d6/f16 2710654 0 2024-04-25T01:35:43.433 INFO:tasks.workunit.client.0.smithi033.stdout:4/202: mkdir d6/d12/d3d 0 2024-04-25T01:35:43.434 INFO:tasks.workunit.client.0.smithi033.stdout:4/203: chown d6/d14/cb 0 1 2024-04-25T01:35:43.434 INFO:tasks.workunit.client.0.smithi033.stdout:9/151: dread d0/f23 [0,4194304] 0 2024-04-25T01:35:43.435 INFO:tasks.workunit.client.0.smithi033.stdout:4/204: chown d6/d14/d35/d39 208121455 1 2024-04-25T01:35:43.435 INFO:tasks.workunit.client.0.smithi033.stdout:8/187: creat d6/d10/f44 x:0 0 0 2024-04-25T01:35:43.439 INFO:tasks.workunit.client.0.smithi033.stdout:7/148: getdents d4/d10 0 2024-04-25T01:35:43.441 INFO:tasks.workunit.client.0.smithi033.stdout:4/205: dread d6/d12/f23 [0,4194304] 0 2024-04-25T01:35:43.444 INFO:tasks.workunit.client.0.smithi033.stdout:9/152: dread d0/d11/f27 [0,4194304] 0 2024-04-25T01:35:43.445 INFO:tasks.workunit.client.0.smithi033.stdout:4/206: chown d6/c11 391643305 1 2024-04-25T01:35:43.446 INFO:tasks.workunit.client.0.smithi033.stdout:8/188: getdents d6/d10/d2e 0 2024-04-25T01:35:43.448 INFO:tasks.workunit.client.0.smithi033.stdout:9/153: creat d0/d11/f35 x:0 0 0 2024-04-25T01:35:43.450 INFO:tasks.workunit.client.0.smithi033.stdout:9/154: write d0/d11/f18 [173669,33702] 0 2024-04-25T01:35:43.451 INFO:tasks.workunit.client.0.smithi033.stdout:7/149: truncate d4/dd/ff 1169198 0 2024-04-25T01:35:43.457 INFO:tasks.workunit.client.0.smithi033.stdout:9/155: rename d0/d2/dd/df/c26 to d0/d2/dd/df/d1b/c36 0 2024-04-25T01:35:43.458 INFO:tasks.workunit.client.0.smithi033.stdout:4/207: dwrite d6/f18 [0,4194304] 0 2024-04-25T01:35:43.461 INFO:tasks.workunit.client.0.smithi033.stdout:9/156: mkdir d0/d2/dd/df/d37 0 2024-04-25T01:35:43.462 INFO:tasks.workunit.client.0.smithi033.stdout:8/189: dwrite d6/db/f26 [0,4194304] 0 2024-04-25T01:35:43.464 INFO:tasks.workunit.client.0.smithi033.stdout:4/208: fdatasync d6/d14/d1d/f2e 0 2024-04-25T01:35:43.467 INFO:tasks.workunit.client.0.smithi033.stdout:8/190: unlink c2 0 2024-04-25T01:35:43.469 INFO:tasks.workunit.client.0.smithi033.stdout:7/150: write d4/dd/ff [490079,39425] 0 2024-04-25T01:35:43.469 INFO:tasks.workunit.client.0.smithi033.stdout:7/151: fsync d4/f17 0 2024-04-25T01:35:43.471 INFO:tasks.workunit.client.0.smithi033.stdout:9/157: unlink d0/d2/dd/l31 0 2024-04-25T01:35:43.473 INFO:tasks.workunit.client.0.smithi033.stdout:4/209: dwrite d6/f1f [0,4194304] 0 2024-04-25T01:35:43.473 INFO:tasks.workunit.client.0.smithi033.stdout:7/152: chown f3 6574 1 2024-04-25T01:35:43.473 INFO:tasks.workunit.client.0.smithi033.stdout:9/158: readlink d0/d2/dd/df/d1b/l2a 0 2024-04-25T01:35:43.475 INFO:tasks.workunit.client.0.smithi033.stdout:7/153: readlink d4/d23/l27 0 2024-04-25T01:35:43.478 INFO:tasks.workunit.client.0.smithi033.stdout:4/210: write d6/d14/f22 [621103,75069] 0 2024-04-25T01:35:43.480 INFO:tasks.workunit.client.0.smithi033.stdout:4/211: fsync d6/d14/d1d/f28 0 2024-04-25T01:35:43.482 INFO:tasks.workunit.client.0.smithi033.stdout:7/154: dwrite d4/dd/f1c [0,4194304] 0 2024-04-25T01:35:43.483 INFO:tasks.workunit.client.0.smithi033.stdout:7/155: dread - d4/f17 zero size 2024-04-25T01:35:43.513 INFO:tasks.workunit.client.0.smithi033.stdout:0/156: sync 2024-04-25T01:35:43.513 INFO:tasks.workunit.client.0.smithi033.stdout:2/151: sync 2024-04-25T01:35:43.513 INFO:tasks.workunit.client.0.smithi033.stdout:6/114: sync 2024-04-25T01:35:43.513 INFO:tasks.workunit.client.0.smithi033.stdout:3/168: sync 2024-04-25T01:35:43.521 INFO:tasks.workunit.client.0.smithi033.stdout:0/157: chown d7/da/c1a 38 1 2024-04-25T01:35:43.524 INFO:tasks.workunit.client.0.smithi033.stdout:3/169: dread d8/dd/d1d/f2b [0,4194304] 0 2024-04-25T01:35:43.524 INFO:tasks.workunit.client.0.smithi033.stdout:0/158: write d7/da/df/d14/f24 [4938493,90971] 0 2024-04-25T01:35:43.526 INFO:tasks.workunit.client.0.smithi033.stdout:4/212: link d6/d12/f26 d6/d14/d1d/d3b/f3e 0 2024-04-25T01:35:43.528 INFO:tasks.workunit.client.0.smithi033.stdout:2/152: dread da/f12 [0,4194304] 0 2024-04-25T01:35:43.531 INFO:tasks.workunit.client.0.smithi033.stdout:7/156: getdents d4/d10/d13 0 2024-04-25T01:35:43.535 INFO:tasks.workunit.client.0.smithi033.stdout:3/170: creat d8/f37 x:0 0 0 2024-04-25T01:35:43.537 INFO:tasks.workunit.client.0.smithi033.stdout:2/153: mkdir da/d27/d31 0 2024-04-25T01:35:43.539 INFO:tasks.workunit.client.0.smithi033.stdout:0/159: dwrite d7/da/df/f10 [8388608,4194304] 0 2024-04-25T01:35:43.539 INFO:tasks.workunit.client.0.smithi033.stdout:3/171: fsync d8/dd/f2c 0 2024-04-25T01:35:43.540 INFO:tasks.workunit.client.0.smithi033.stdout:2/154: fdatasync da/f15 0 2024-04-25T01:35:43.540 INFO:tasks.workunit.client.0.smithi033.stdout:7/157: rename d4/d10/d13/c16 to d4/d10/d13/d1f/c28 0 2024-04-25T01:35:43.541 INFO:tasks.workunit.client.0.smithi033.stdout:4/213: rmdir d6/d12/d3d 0 2024-04-25T01:35:43.544 INFO:tasks.workunit.client.0.smithi033.stdout:0/160: stat d7/da/df/f10 0 2024-04-25T01:35:43.545 INFO:tasks.workunit.client.0.smithi033.stdout:3/172: chown d8/f37 3772896 1 2024-04-25T01:35:43.551 INFO:tasks.workunit.client.0.smithi033.stdout:9/159: dwrite d0/f8 [4194304,4194304] 0 2024-04-25T01:35:43.553 INFO:tasks.workunit.client.0.smithi033.stdout:0/161: mknod d7/da/c31 0 2024-04-25T01:35:43.553 INFO:tasks.workunit.client.0.smithi033.stdout:3/173: write d8/dd/f2c [9210926,85399] 0 2024-04-25T01:35:43.553 INFO:tasks.workunit.client.0.smithi033.stdout:6/115: dwrite d4/f12 [0,4194304] 0 2024-04-25T01:35:43.553 INFO:tasks.workunit.client.0.smithi033.stdout:7/158: creat d4/d10/f29 x:0 0 0 2024-04-25T01:35:43.565 INFO:tasks.workunit.client.0.smithi033.stdout:6/116: rename l2 to d4/d15/l28 0 2024-04-25T01:35:43.565 INFO:tasks.workunit.client.0.smithi033.stdout:3/174: rename d8 to d8/dd/d11/d1e/d27/d38 22 2024-04-25T01:35:43.566 INFO:tasks.workunit.client.0.smithi033.stdout:2/155: dwrite da/f28 [0,4194304] 0 2024-04-25T01:35:43.566 INFO:tasks.workunit.client.0.smithi033.stdout:7/159: mknod d4/dd/c2a 0 2024-04-25T01:35:43.567 INFO:tasks.workunit.client.0.smithi033.stdout:4/214: dwrite d6/f9 [0,4194304] 0 2024-04-25T01:35:43.570 INFO:tasks.workunit.client.0.smithi033.stdout:7/160: readlink d4/d10/l1a 0 2024-04-25T01:35:43.571 INFO:tasks.workunit.client.0.smithi033.stdout:0/162: getdents d7/d1b 0 2024-04-25T01:35:43.572 INFO:tasks.workunit.client.0.smithi033.stdout:3/175: read d8/dd/d11/d1e/f21 [2904244,24802] 0 2024-04-25T01:35:43.574 INFO:tasks.workunit.client.0.smithi033.stdout:0/163: fsync d7/da/df/d14/f24 0 2024-04-25T01:35:43.574 INFO:tasks.workunit.client.0.smithi033.stdout:3/176: chown l7 15 1 2024-04-25T01:35:43.576 INFO:tasks.workunit.client.0.smithi033.stdout:6/117: unlink d4/cb 0 2024-04-25T01:35:43.579 INFO:tasks.workunit.client.0.smithi033.stdout:7/161: creat d4/d23/f2b x:0 0 0 2024-04-25T01:35:43.581 INFO:tasks.workunit.client.0.smithi033.stdout:0/164: dread d7/da/df/f28 [0,4194304] 0 2024-04-25T01:35:43.584 INFO:tasks.workunit.client.0.smithi033.stdout:4/215: symlink d6/d14/d1d/d3b/l3f 0 2024-04-25T01:35:43.585 INFO:tasks.workunit.client.0.smithi033.stdout:4/216: chown d6/f8 531214812 1 2024-04-25T01:35:43.586 INFO:tasks.workunit.client.0.smithi033.stdout:6/118: mkdir d4/d24/d29 0 2024-04-25T01:35:43.587 INFO:tasks.workunit.client.0.smithi033.stdout:4/217: write d6/d12/f2f [472290,95180] 0 2024-04-25T01:35:43.588 INFO:tasks.workunit.client.0.smithi033.stdout:7/162: rmdir d4/d10/d13/d1f 39 2024-04-25T01:35:43.589 INFO:tasks.workunit.client.0.smithi033.stdout:9/160: write d0/f33 [876542,92949] 0 2024-04-25T01:35:43.589 INFO:tasks.workunit.client.0.smithi033.stdout:7/163: stat d4/d10/l21 0 2024-04-25T01:35:43.589 INFO:tasks.workunit.client.0.smithi033.stdout:4/218: read d6/d12/f23 [1197999,74523] 0 2024-04-25T01:35:43.590 INFO:tasks.workunit.client.0.smithi033.stdout:7/164: chown d4/d10 466363 1 2024-04-25T01:35:43.594 INFO:tasks.workunit.client.0.smithi033.stdout:4/219: chown d6/d14/d1d/d3b 45 1 2024-04-25T01:35:43.595 INFO:tasks.workunit.client.0.smithi033.stdout:0/165: creat d7/d1b/f32 x:0 0 0 2024-04-25T01:35:43.595 INFO:tasks.workunit.client.0.smithi033.stdout:2/156: write f6 [2021264,104476] 0 2024-04-25T01:35:43.596 INFO:tasks.workunit.client.0.smithi033.stdout:6/119: mkdir d4/d2a 0 2024-04-25T01:35:43.598 INFO:tasks.workunit.client.0.smithi033.stdout:6/120: dread d4/d1d/f1f [0,4194304] 0 2024-04-25T01:35:43.599 INFO:tasks.workunit.client.0.smithi033.stdout:3/177: link f3 d8/dd/d11/f39 0 2024-04-25T01:35:43.600 INFO:tasks.workunit.client.0.smithi033.stdout:6/121: write d4/f14 [2157962,41492] 0 2024-04-25T01:35:43.600 INFO:tasks.workunit.client.0.smithi033.stdout:1/155: sync 2024-04-25T01:35:43.600 INFO:tasks.workunit.client.0.smithi033.stdout:5/196: sync 2024-04-25T01:35:43.601 INFO:tasks.workunit.client.0.smithi033.stdout:6/122: stat d4/lf 0 2024-04-25T01:35:43.602 INFO:tasks.workunit.client.0.smithi033.stdout:9/161: creat d0/d2/dd/d20/f38 x:0 0 0 2024-04-25T01:35:43.603 INFO:tasks.workunit.client.0.smithi033.stdout:1/156: chown d2/d5/d1b/l27 6 1 2024-04-25T01:35:43.604 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:43 smithi033 ceph-mon[26353]: pgmap v7: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 20 MiB/s wr, 276 op/s 2024-04-25T01:35:43.605 INFO:tasks.workunit.client.0.smithi033.stdout:4/220: rename d6/f1e to d6/d12/f40 0 2024-04-25T01:35:43.607 INFO:tasks.workunit.client.0.smithi033.stdout:2/157: mkdir da/d32 0 2024-04-25T01:35:43.663 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:43 smithi138 ceph-mon[32977]: pgmap v7: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 20 MiB/s wr, 276 op/s 2024-04-25T01:35:43.716 INFO:tasks.workunit.client.0.smithi033.stdout:9/162: dwrite d0/f33 [0,4194304] 0 2024-04-25T01:35:43.717 INFO:tasks.workunit.client.0.smithi033.stdout:7/165: truncate d4/d10/f26 1452586 0 2024-04-25T01:35:43.718 INFO:tasks.workunit.client.0.smithi033.stdout:0/166: dread d7/da/f13 [0,4194304] 0 2024-04-25T01:35:43.720 INFO:tasks.workunit.client.0.smithi033.stdout:4/221: creat d6/d12/d38/f41 x:0 0 0 2024-04-25T01:35:43.721 INFO:tasks.workunit.client.0.smithi033.stdout:0/167: write d7/da/df/d14/f15 [1162486,112844] 0 2024-04-25T01:35:43.722 INFO:tasks.workunit.client.0.smithi033.stdout:2/158: link da/f15 da/d2c/f33 0 2024-04-25T01:35:43.723 INFO:tasks.workunit.client.0.smithi033.stdout:2/159: chown da/d32 1 1 2024-04-25T01:35:43.726 INFO:tasks.workunit.client.0.smithi033.stdout:7/166: unlink d4/f1d 0 2024-04-25T01:35:43.726 INFO:tasks.workunit.client.0.smithi033.stdout:7/167: read d4/fb [1324589,13870] 0 2024-04-25T01:35:43.727 INFO:tasks.workunit.client.0.smithi033.stdout:7/168: readlink d4/l12 0 2024-04-25T01:35:43.728 INFO:tasks.workunit.client.0.smithi033.stdout:9/163: symlink d0/d2/dd/df/d1b/l39 0 2024-04-25T01:35:43.729 INFO:tasks.workunit.client.0.smithi033.stdout:3/178: write d8/dd/d11/d1e/f29 [1392944,9086] 0 2024-04-25T01:35:43.730 INFO:tasks.workunit.client.0.smithi033.stdout:5/197: write d1c/d21/f3a [506525,38530] 0 2024-04-25T01:35:43.731 INFO:tasks.workunit.client.0.smithi033.stdout:3/179: chown l7 30490852 1 2024-04-25T01:35:43.731 INFO:tasks.workunit.client.0.smithi033.stdout:1/157: write d2/f20 [1360987,78586] 0 2024-04-25T01:35:43.732 INFO:tasks.workunit.client.0.smithi033.stdout:4/222: rmdir d6/d14/d35 39 2024-04-25T01:35:43.734 INFO:tasks.workunit.client.0.smithi033.stdout:3/180: chown d8/f1b 391 1 2024-04-25T01:35:43.734 INFO:tasks.workunit.client.0.smithi033.stdout:4/223: write d6/d12/d38/f41 [145181,123727] 0 2024-04-25T01:35:43.737 INFO:tasks.workunit.client.0.smithi033.stdout:2/160: dread f1 [0,4194304] 0 2024-04-25T01:35:43.738 INFO:tasks.workunit.client.0.smithi033.stdout:2/161: write f9 [2768458,114683] 0 2024-04-25T01:35:43.739 INFO:tasks.workunit.client.0.smithi033.stdout:9/164: creat d0/d2/dd/df/f3a x:0 0 0 2024-04-25T01:35:43.740 INFO:tasks.workunit.client.0.smithi033.stdout:5/198: mkdir d1c/d21/d3e 0 2024-04-25T01:35:43.744 INFO:tasks.workunit.client.0.smithi033.stdout:3/181: mknod d8/dd/d1d/c3a 0 2024-04-25T01:35:43.746 INFO:tasks.workunit.client.0.smithi033.stdout:4/224: symlink d6/d12/l42 0 2024-04-25T01:35:43.747 INFO:tasks.workunit.client.0.smithi033.stdout:0/168: mkdir d7/d33 0 2024-04-25T01:35:43.749 INFO:tasks.workunit.client.0.smithi033.stdout:1/158: dwrite d2/f26 [0,4194304] 0 2024-04-25T01:35:43.749 INFO:tasks.workunit.client.0.smithi033.stdout:0/169: write d7/da/df/f17 [2026345,126936] 0 2024-04-25T01:35:43.751 INFO:tasks.workunit.client.0.smithi033.stdout:2/162: rename da/d23/f25 to da/f34 0 2024-04-25T01:35:43.757 INFO:tasks.workunit.client.0.smithi033.stdout:3/182: creat d8/dd/d11/d1e/f3b x:0 0 0 2024-04-25T01:35:43.757 INFO:tasks.workunit.client.0.smithi033.stdout:4/225: fdatasync d6/d14/f2b 0 2024-04-25T01:35:43.758 INFO:tasks.workunit.client.0.smithi033.stdout:4/226: write d6/f1f [3865188,4909] 0 2024-04-25T01:35:43.760 INFO:tasks.workunit.client.0.smithi033.stdout:4/227: write d6/d14/f31 [4830355,22632] 0 2024-04-25T01:35:43.761 INFO:tasks.workunit.client.0.smithi033.stdout:7/169: truncate d4/dd/ff 766477 0 2024-04-25T01:35:43.762 INFO:tasks.workunit.client.0.smithi033.stdout:7/170: readlink d4/d10/l1a 0 2024-04-25T01:35:43.765 INFO:tasks.workunit.client.0.smithi033.stdout:3/183: unlink d8/l20 0 2024-04-25T01:35:43.770 INFO:tasks.workunit.client.0.smithi033.stdout:1/159: dwrite d2/d11/f1a [0,4194304] 0 2024-04-25T01:35:43.772 INFO:tasks.workunit.client.0.smithi033.stdout:2/163: creat da/d32/f35 x:0 0 0 2024-04-25T01:35:43.773 INFO:tasks.workunit.client.0.smithi033.stdout:2/164: truncate da/f15 391506 0 2024-04-25T01:35:43.773 INFO:tasks.workunit.client.0.smithi033.stdout:5/199: creat d1c/d2a/d3d/f3f x:0 0 0 2024-04-25T01:35:43.774 INFO:tasks.workunit.client.0.smithi033.stdout:8/191: sync 2024-04-25T01:35:43.775 INFO:tasks.workunit.client.0.smithi033.stdout:3/184: mkdir d8/dd/d11/d1e/d27/d3c 0 2024-04-25T01:35:43.775 INFO:tasks.workunit.client.0.smithi033.stdout:8/192: fsync d6/f40 0 2024-04-25T01:35:43.777 INFO:tasks.workunit.client.0.smithi033.stdout:8/193: read d6/f1e [4695943,77428] 0 2024-04-25T01:35:43.778 INFO:tasks.workunit.client.0.smithi033.stdout:4/228: mknod d6/c43 0 2024-04-25T01:35:43.780 INFO:tasks.workunit.client.0.smithi033.stdout:8/194: chown d6/d10/d2e/l35 0 1 2024-04-25T01:35:43.782 INFO:tasks.workunit.client.0.smithi033.stdout:5/200: dwrite d1c/f38 [0,4194304] 0 2024-04-25T01:35:43.784 INFO:tasks.workunit.client.0.smithi033.stdout:5/201: fsync f1b 0 2024-04-25T01:35:43.784 INFO:tasks.workunit.client.0.smithi033.stdout:3/185: mkdir d8/dd/d3d 0 2024-04-25T01:35:43.785 INFO:tasks.workunit.client.0.smithi033.stdout:3/186: write d8/dd/f25 [4803268,57176] 0 2024-04-25T01:35:43.787 INFO:tasks.workunit.client.0.smithi033.stdout:0/170: getdents d7 0 2024-04-25T01:35:43.788 INFO:tasks.workunit.client.0.smithi033.stdout:1/160: mknod d2/d2e/c2f 0 2024-04-25T01:35:43.788 INFO:tasks.workunit.client.0.smithi033.stdout:3/187: chown d8/dd/d11/d1e/d27/d3c 190933237 1 2024-04-25T01:35:43.792 INFO:tasks.workunit.client.0.smithi033.stdout:8/195: rmdir d6 39 2024-04-25T01:35:43.793 INFO:tasks.workunit.client.0.smithi033.stdout:2/165: write da/d2c/f33 [244904,100320] 0 2024-04-25T01:35:43.794 INFO:tasks.workunit.client.0.smithi033.stdout:2/166: write da/d32/f35 [56878,68567] 0 2024-04-25T01:35:43.795 INFO:tasks.workunit.client.0.smithi033.stdout:2/167: stat da/f18 0 2024-04-25T01:35:43.798 INFO:tasks.workunit.client.0.smithi033.stdout:2/168: dread - da/f26 zero size 2024-04-25T01:35:43.798 INFO:tasks.workunit.client.0.smithi033.stdout:2/169: stat f1 0 2024-04-25T01:35:43.802 INFO:tasks.workunit.client.0.smithi033.stdout:5/202: unlink d1c/d2a/c30 0 2024-04-25T01:35:43.802 INFO:tasks.workunit.client.0.smithi033.stdout:5/203: chown cd 164491183 1 2024-04-25T01:35:43.805 INFO:tasks.workunit.client.0.smithi033.stdout:2/170: rename da/f11 to da/d2c/f36 0 2024-04-25T01:35:43.805 INFO:tasks.workunit.client.0.smithi033.stdout:7/171: getdents d4/d10/d13/d1f 0 2024-04-25T01:35:43.806 INFO:tasks.workunit.client.0.smithi033.stdout:4/229: truncate d6/f1f 984771 0 2024-04-25T01:35:43.807 INFO:tasks.workunit.client.0.smithi033.stdout:4/230: read - d6/d12/f26 zero size 2024-04-25T01:35:43.808 INFO:tasks.workunit.client.0.smithi033.stdout:3/188: symlink d8/dd/d11/l3e 0 2024-04-25T01:35:43.811 INFO:tasks.workunit.client.0.smithi033.stdout:5/204: creat d1c/d2a/d3d/f40 x:0 0 0 2024-04-25T01:35:43.813 INFO:tasks.workunit.client.0.smithi033.stdout:7/172: dwrite d4/f11 [0,4194304] 0 2024-04-25T01:35:43.814 INFO:tasks.workunit.client.0.smithi033.stdout:2/171: unlink f6 0 2024-04-25T01:35:43.821 INFO:tasks.workunit.client.0.smithi033.stdout:0/171: truncate d7/f29 1968487 0 2024-04-25T01:35:43.825 INFO:tasks.workunit.client.0.smithi033.stdout:8/196: mkdir d6/d45 0 2024-04-25T01:35:43.826 INFO:tasks.workunit.client.0.smithi033.stdout:4/231: dread d6/d14/ff [0,4194304] 0 2024-04-25T01:35:43.827 INFO:tasks.workunit.client.0.smithi033.stdout:4/232: fsync d6/d14/d1d/f2c 0 2024-04-25T01:35:43.828 INFO:tasks.workunit.client.0.smithi033.stdout:5/205: mknod d1c/d2a/d3d/c41 0 2024-04-25T01:35:43.831 INFO:tasks.workunit.client.0.smithi033.stdout:1/161: getdents d2 0 2024-04-25T01:35:43.836 INFO:tasks.workunit.client.0.smithi033.stdout:3/189: symlink d8/dd/d11/d1e/d27/d3c/l3f 0 2024-04-25T01:35:43.838 INFO:tasks.workunit.client.0.smithi033.stdout:8/197: creat d6/db/f46 x:0 0 0 2024-04-25T01:35:43.838 INFO:tasks.workunit.client.0.smithi033.stdout:0/172: dwrite d7/da/f13 [0,4194304] 0 2024-04-25T01:35:43.839 INFO:tasks.workunit.client.0.smithi033.stdout:8/198: dread - d6/db/f46 zero size 2024-04-25T01:35:43.841 INFO:tasks.workunit.client.0.smithi033.stdout:0/173: write d7/da/df/f10 [12722324,84846] 0 2024-04-25T01:35:43.842 INFO:tasks.workunit.client.0.smithi033.stdout:8/199: fsync d6/db/f26 0 2024-04-25T01:35:43.843 INFO:tasks.workunit.client.0.smithi033.stdout:8/200: stat d6/f8 0 2024-04-25T01:35:43.844 INFO:tasks.workunit.client.0.smithi033.stdout:8/201: write d6/db/fc [1216014,12519] 0 2024-04-25T01:35:43.845 INFO:tasks.workunit.client.0.smithi033.stdout:8/202: readlink l0 0 2024-04-25T01:35:43.845 INFO:tasks.workunit.client.0.smithi033.stdout:8/203: dread - d6/d10/f3b zero size 2024-04-25T01:35:43.848 INFO:tasks.workunit.client.0.smithi033.stdout:1/162: chown d2/d5/cc 0 1 2024-04-25T01:35:43.850 INFO:tasks.workunit.client.0.smithi033.stdout:3/190: mkdir d8/dd/d40 0 2024-04-25T01:35:43.851 INFO:tasks.workunit.client.0.smithi033.stdout:7/173: truncate d4/f14 4179418 0 2024-04-25T01:35:43.855 INFO:tasks.workunit.client.0.smithi033.stdout:8/204: dwrite d6/d10/d2e/f30 [0,4194304] 0 2024-04-25T01:35:43.857 INFO:tasks.workunit.client.0.smithi033.stdout:1/163: creat d2/d11/f30 x:0 0 0 2024-04-25T01:35:43.866 INFO:tasks.workunit.client.0.smithi033.stdout:3/191: symlink d8/dd/d40/l41 0 2024-04-25T01:35:43.866 INFO:tasks.workunit.client.0.smithi033.stdout:2/172: write f9 [1736893,14641] 0 2024-04-25T01:35:43.868 INFO:tasks.workunit.client.0.smithi033.stdout:1/164: rename d2/d5/c17 to d2/c31 0 2024-04-25T01:35:43.869 INFO:tasks.workunit.client.0.smithi033.stdout:5/206: link l19 d1c/d2a/l42 0 2024-04-25T01:35:43.870 INFO:tasks.workunit.client.0.smithi033.stdout:5/207: write f7 [4344720,74091] 0 2024-04-25T01:35:43.871 INFO:tasks.workunit.client.0.smithi033.stdout:4/233: getdents d6/d14 0 2024-04-25T01:35:43.875 INFO:tasks.workunit.client.0.smithi033.stdout:7/174: dread d4/f9 [0,4194304] 0 2024-04-25T01:35:43.875 INFO:tasks.workunit.client.0.smithi033.stdout:2/173: unlink da/f28 0 2024-04-25T01:35:43.877 INFO:tasks.workunit.client.0.smithi033.stdout:7/175: truncate f0 6066648 0 2024-04-25T01:35:43.880 INFO:tasks.workunit.client.0.smithi033.stdout:3/192: symlink d8/dd/d11/l42 0 2024-04-25T01:35:43.883 INFO:tasks.workunit.client.0.smithi033.stdout:4/234: dwrite d6/f8 [0,4194304] 0 2024-04-25T01:35:43.884 INFO:tasks.workunit.client.0.smithi033.stdout:3/193: read f3 [2375973,79484] 0 2024-04-25T01:35:43.886 INFO:tasks.workunit.client.0.smithi033.stdout:3/194: chown d8/lb 3 1 2024-04-25T01:35:43.888 INFO:tasks.workunit.client.0.smithi033.stdout:8/205: dwrite d6/d10/f28 [0,4194304] 0 2024-04-25T01:35:43.891 INFO:tasks.workunit.client.0.smithi033.stdout:8/206: write d6/db/f26 [4828741,68026] 0 2024-04-25T01:35:43.894 INFO:tasks.workunit.client.0.smithi033.stdout:3/195: creat d8/dd/d40/f43 x:0 0 0 2024-04-25T01:35:43.953 INFO:tasks.workunit.client.0.smithi033.stdout:6/123: sync 2024-04-25T01:35:43.953 INFO:tasks.workunit.client.0.smithi033.stdout:5/208: rename c1a to d1c/d2a/c43 0 2024-04-25T01:35:43.956 INFO:tasks.workunit.client.0.smithi033.stdout:8/207: getdents d6/d45 0 2024-04-25T01:35:43.957 INFO:tasks.workunit.client.0.smithi033.stdout:2/174: dread da/d2c/f36 [0,4194304] 0 2024-04-25T01:35:43.958 INFO:tasks.workunit.client.0.smithi033.stdout:3/196: symlink d8/dd/d11/d1e/d27/d3c/l44 0 2024-04-25T01:35:43.960 INFO:tasks.workunit.client.0.smithi033.stdout:6/124: rename d4/d1d/l1e to d4/d2a/l2b 0 2024-04-25T01:35:43.961 INFO:tasks.workunit.client.0.smithi033.stdout:1/165: dread d2/d5/f14 [0,4194304] 0 2024-04-25T01:35:43.962 INFO:tasks.workunit.client.0.smithi033.stdout:2/175: fsync da/f12 0 2024-04-25T01:35:43.962 INFO:tasks.workunit.client.0.smithi033.stdout:6/125: chown d4/d2a 130789 1 2024-04-25T01:35:43.963 INFO:tasks.workunit.client.0.smithi033.stdout:8/208: mknod d6/d45/c47 0 2024-04-25T01:35:43.964 INFO:tasks.workunit.client.0.smithi033.stdout:6/126: fdatasync d4/f9 0 2024-04-25T01:35:43.967 INFO:tasks.workunit.client.0.smithi033.stdout:8/209: write d6/f3c [821895,89762] 0 2024-04-25T01:35:43.967 INFO:tasks.workunit.client.0.smithi033.stdout:7/176: dread d4/f11 [4194304,4194304] 0 2024-04-25T01:35:43.969 INFO:tasks.workunit.client.0.smithi033.stdout:5/209: dwrite d1c/f38 [4194304,4194304] 0 2024-04-25T01:35:43.974 INFO:tasks.workunit.client.0.smithi033.stdout:3/197: dread d8/dd/d1d/f2d [0,4194304] 0 2024-04-25T01:35:43.977 INFO:tasks.workunit.client.0.smithi033.stdout:7/177: creat d4/f2c x:0 0 0 2024-04-25T01:35:43.977 INFO:tasks.workunit.client.0.smithi033.stdout:5/210: creat d1c/d2a/d3d/f44 x:0 0 0 2024-04-25T01:35:43.978 INFO:tasks.workunit.client.0.smithi033.stdout:8/210: mkdir d6/d10/d48 0 2024-04-25T01:35:43.981 INFO:tasks.workunit.client.0.smithi033.stdout:8/211: dread - d6/d10/f3b zero size 2024-04-25T01:35:43.981 INFO:tasks.workunit.client.0.smithi033.stdout:2/176: dwrite da/d27/f2d [0,4194304] 0 2024-04-25T01:35:43.982 INFO:tasks.workunit.client.0.smithi033.stdout:6/127: dwrite d4/d15/f20 [4194304,4194304] 0 2024-04-25T01:35:43.987 INFO:tasks.workunit.client.0.smithi033.stdout:2/177: rename da/c1b to da/c37 0 2024-04-25T01:35:43.989 INFO:tasks.workunit.client.0.smithi033.stdout:8/212: mkdir d6/d49 0 2024-04-25T01:35:43.993 INFO:tasks.workunit.client.0.smithi033.stdout:2/178: mknod da/d27/d31/c38 0 2024-04-25T01:35:43.994 INFO:tasks.workunit.client.0.smithi033.stdout:8/213: creat d6/d10/d2e/f4a x:0 0 0 2024-04-25T01:35:43.996 INFO:tasks.workunit.client.0.smithi033.stdout:7/178: dwrite d4/fb [0,4194304] 0 2024-04-25T01:35:43.997 INFO:tasks.workunit.client.0.smithi033.stdout:6/128: dwrite d4/f6 [0,4194304] 0 2024-04-25T01:35:44.002 INFO:tasks.workunit.client.0.smithi033.stdout:2/179: creat da/d27/d31/f39 x:0 0 0 2024-04-25T01:35:44.003 INFO:tasks.workunit.client.0.smithi033.stdout:8/214: getdents d6/d49 0 2024-04-25T01:35:44.003 INFO:tasks.workunit.client.0.smithi033.stdout:7/179: mknod d4/d10/c2d 0 2024-04-25T01:35:44.006 INFO:tasks.workunit.client.0.smithi033.stdout:2/180: chown da/f17 112 1 2024-04-25T01:35:44.006 INFO:tasks.workunit.client.0.smithi033.stdout:6/129: creat d4/d24/d29/f2c x:0 0 0 2024-04-25T01:35:44.006 INFO:tasks.workunit.client.0.smithi033.stdout:8/215: chown d6/f3e 34641 1 2024-04-25T01:35:44.008 INFO:tasks.workunit.client.0.smithi033.stdout:8/216: chown d6/f34 0 1 2024-04-25T01:35:44.009 INFO:tasks.workunit.client.0.smithi033.stdout:8/217: fdatasync d6/f3c 0 2024-04-25T01:35:44.016 INFO:tasks.workunit.client.0.smithi033.stdout:5/211: dread fc [0,4194304] 0 2024-04-25T01:35:44.021 INFO:tasks.workunit.client.0.smithi033.stdout:6/130: dwrite d4/d15/f20 [4194304,4194304] 0 2024-04-25T01:35:44.021 INFO:tasks.workunit.client.0.smithi033.stdout:2/181: dwrite da/f13 [0,4194304] 0 2024-04-25T01:35:44.024 INFO:tasks.workunit.client.0.smithi033.stdout:2/182: stat da/d2c/f33 0 2024-04-25T01:35:44.027 INFO:tasks.workunit.client.0.smithi033.stdout:8/218: dwrite d6/d10/d2e/f3a [0,4194304] 0 2024-04-25T01:35:44.029 INFO:tasks.workunit.client.0.smithi033.stdout:7/180: truncate d4/dd/f1c 3673619 0 2024-04-25T01:35:44.033 INFO:tasks.workunit.client.0.smithi033.stdout:7/181: chown d4/d10/l22 0 1 2024-04-25T01:35:44.035 INFO:tasks.workunit.client.0.smithi033.stdout:6/131: dread d4/f11 [0,4194304] 0 2024-04-25T01:35:44.037 INFO:tasks.workunit.client.0.smithi033.stdout:5/212: dwrite f1b [0,4194304] 0 2024-04-25T01:35:44.038 INFO:tasks.workunit.client.0.smithi033.stdout:6/132: write d4/d15/f20 [7320794,106129] 0 2024-04-25T01:35:44.038 INFO:tasks.workunit.client.0.smithi033.stdout:7/182: rename d4/dd to d4/d2e 0 2024-04-25T01:35:44.040 INFO:tasks.workunit.client.0.smithi033.stdout:7/183: truncate d4/d23/f2b 193114 0 2024-04-25T01:35:44.043 INFO:tasks.workunit.client.0.smithi033.stdout:7/184: write d4/f11 [5380494,100267] 0 2024-04-25T01:35:44.044 INFO:tasks.workunit.client.0.smithi033.stdout:5/213: getdents d1c/d2a/d3d 0 2024-04-25T01:35:44.045 INFO:tasks.workunit.client.0.smithi033.stdout:7/185: truncate d4/f2c 835927 0 2024-04-25T01:35:44.045 INFO:tasks.workunit.client.0.smithi033.stdout:5/214: chown f15 883056162 1 2024-04-25T01:35:44.049 INFO:tasks.workunit.client.0.smithi033.stdout:5/215: unlink d1c/l35 0 2024-04-25T01:35:44.050 INFO:tasks.workunit.client.0.smithi033.stdout:6/133: read d4/d15/f20 [3709157,56662] 0 2024-04-25T01:35:44.053 INFO:tasks.workunit.client.0.smithi033.stdout:5/216: truncate d1c/d2a/d3d/f44 862404 0 2024-04-25T01:35:44.054 INFO:tasks.workunit.client.0.smithi033.stdout:6/134: unlink d4/d2a/l2b 0 2024-04-25T01:35:44.055 INFO:tasks.workunit.client.0.smithi033.stdout:5/217: creat d1c/d21/f45 x:0 0 0 2024-04-25T01:35:44.057 INFO:tasks.workunit.client.0.smithi033.stdout:6/135: rename d4/d24/d29/f2c to d4/f2d 0 2024-04-25T01:35:44.062 INFO:tasks.workunit.client.0.smithi033.stdout:6/136: rename d4/d15/l26 to d4/d1d/l2e 0 2024-04-25T01:35:44.064 INFO:tasks.workunit.client.0.smithi033.stdout:6/137: creat d4/f2f x:0 0 0 2024-04-25T01:35:44.066 INFO:tasks.workunit.client.0.smithi033.stdout:6/138: unlink d4/f9 0 2024-04-25T01:35:44.066 INFO:tasks.workunit.client.0.smithi033.stdout:8/219: write d6/f9 [1792594,91210] 0 2024-04-25T01:35:44.067 INFO:tasks.workunit.client.0.smithi033.stdout:7/186: dread d4/f14 [0,4194304] 0 2024-04-25T01:35:44.068 INFO:tasks.workunit.client.0.smithi033.stdout:8/220: truncate d6/d10/f3b 585827 0 2024-04-25T01:35:44.070 INFO:tasks.workunit.client.0.smithi033.stdout:8/221: mknod d6/db/c4b 0 2024-04-25T01:35:44.071 INFO:tasks.workunit.client.0.smithi033.stdout:8/222: readlink d6/db/l20 0 2024-04-25T01:35:44.071 INFO:tasks.workunit.client.0.smithi033.stdout:7/187: rmdir d4/d10/d13 39 2024-04-25T01:35:44.074 INFO:tasks.workunit.client.0.smithi033.stdout:8/223: mknod d6/d45/c4c 0 2024-04-25T01:35:44.075 INFO:tasks.workunit.client.0.smithi033.stdout:5/218: truncate d1c/d21/f2c 3425433 0 2024-04-25T01:35:44.079 INFO:tasks.workunit.client.0.smithi033.stdout:7/188: getdents d4 0 2024-04-25T01:35:44.085 INFO:tasks.workunit.client.0.smithi033.stdout:6/139: fsync d4/f11 0 2024-04-25T01:35:44.085 INFO:tasks.workunit.client.0.smithi033.stdout:8/224: dwrite d6/db/f31 [0,4194304] 0 2024-04-25T01:35:44.086 INFO:tasks.workunit.client.0.smithi033.stdout:5/219: dwrite d1c/f1f [0,4194304] 0 2024-04-25T01:35:44.094 INFO:tasks.workunit.client.0.smithi033.stdout:5/220: creat d1c/d2a/d3d/f46 x:0 0 0 2024-04-25T01:35:44.095 INFO:tasks.workunit.client.0.smithi033.stdout:5/221: write f15 [3180600,71248] 0 2024-04-25T01:35:44.096 INFO:tasks.workunit.client.0.smithi033.stdout:8/225: rename d6/d10/f44 to d6/d10/d48/f4d 0 2024-04-25T01:35:44.101 INFO:tasks.workunit.client.0.smithi033.stdout:7/189: link d4/d10/l21 d4/d10/d13/l2f 0 2024-04-25T01:35:44.102 INFO:tasks.workunit.client.0.smithi033.stdout:5/222: creat d1c/d2a/f47 x:0 0 0 2024-04-25T01:35:44.104 INFO:tasks.workunit.client.0.smithi033.stdout:7/190: write f0 [3988860,62138] 0 2024-04-25T01:35:44.105 INFO:tasks.workunit.client.0.smithi033.stdout:6/140: write d4/d1d/f1f [257216,65445] 0 2024-04-25T01:35:44.106 INFO:tasks.workunit.client.0.smithi033.stdout:8/226: dwrite d6/db/f1a [0,4194304] 0 2024-04-25T01:35:44.107 INFO:tasks.workunit.client.0.smithi033.stdout:6/141: fdatasync d4/f2d 0 2024-04-25T01:35:44.113 INFO:tasks.workunit.client.0.smithi033.stdout:7/191: rename d4/d10/l22 to d4/d10/d13/l30 0 2024-04-25T01:35:44.117 INFO:tasks.workunit.client.0.smithi033.stdout:7/192: mknod d4/c31 0 2024-04-25T01:35:44.120 INFO:tasks.workunit.client.0.smithi033.stdout:7/193: creat d4/f32 x:0 0 0 2024-04-25T01:35:44.121 INFO:tasks.workunit.client.0.smithi033.stdout:6/142: link d4/f14 d4/d2a/f30 0 2024-04-25T01:35:44.122 INFO:tasks.workunit.client.0.smithi033.stdout:8/227: truncate d6/f21 3033785 0 2024-04-25T01:35:44.122 INFO:tasks.workunit.client.0.smithi033.stdout:6/143: write d4/f2f [983641,113934] 0 2024-04-25T01:35:44.125 INFO:tasks.workunit.client.0.smithi033.stdout:5/223: dread d1c/f1d [0,4194304] 0 2024-04-25T01:35:44.125 INFO:tasks.workunit.client.0.smithi033.stdout:7/194: creat d4/d10/d13/f33 x:0 0 0 2024-04-25T01:35:44.126 INFO:tasks.workunit.client.0.smithi033.stdout:5/224: write d1c/d21/f45 [23188,36144] 0 2024-04-25T01:35:44.126 INFO:tasks.workunit.client.0.smithi033.stdout:6/144: rename d4/f2f to d4/d2a/f31 0 2024-04-25T01:35:44.130 INFO:tasks.workunit.client.0.smithi033.stdout:7/195: rename d4/d10/d13/d1f/f20 to d4/f34 0 2024-04-25T01:35:44.131 INFO:tasks.workunit.client.0.smithi033.stdout:6/145: mknod d4/d1d/c32 0 2024-04-25T01:35:44.136 INFO:tasks.workunit.client.0.smithi033.stdout:7/196: creat d4/d23/f35 x:0 0 0 2024-04-25T01:35:44.137 INFO:tasks.workunit.client.0.smithi033.stdout:6/146: creat d4/d24/d29/f33 x:0 0 0 2024-04-25T01:35:44.206 INFO:tasks.workunit.client.0.smithi033.stdout:9/165: sync 2024-04-25T01:35:44.216 INFO:tasks.workunit.client.0.smithi033.stdout:5/225: creat d1c/f48 x:0 0 0 2024-04-25T01:35:44.216 INFO:tasks.workunit.client.0.smithi033.stdout:7/197: rmdir d4 39 2024-04-25T01:35:44.218 INFO:tasks.workunit.client.0.smithi033.stdout:9/166: unlink d0/f8 0 2024-04-25T01:35:44.220 INFO:tasks.workunit.client.0.smithi033.stdout:7/198: chown d4/d10 58025 1 2024-04-25T01:35:44.264 INFO:tasks.workunit.client.0.smithi033.stdout:0/174: sync 2024-04-25T01:35:44.264 INFO:tasks.workunit.client.0.smithi033.stdout:6/147: dwrite d4/f2d [0,4194304] 0 2024-04-25T01:35:44.265 INFO:tasks.workunit.client.0.smithi033.stdout:5/226: mkdir d1c/d2a/d49 0 2024-04-25T01:35:44.267 INFO:tasks.workunit.client.0.smithi033.stdout:8/228: dwrite d6/db/f26 [0,4194304] 0 2024-04-25T01:35:44.270 INFO:tasks.workunit.client.0.smithi033.stdout:7/199: creat d4/d10/d13/d1f/f36 x:0 0 0 2024-04-25T01:35:44.274 INFO:tasks.workunit.client.0.smithi033.stdout:6/148: dwrite d4/d1d/f1f [0,4194304] 0 2024-04-25T01:35:44.275 INFO:tasks.workunit.client.0.smithi033.stdout:8/229: mknod d6/d10/d2e/c4e 0 2024-04-25T01:35:44.282 INFO:tasks.workunit.client.0.smithi033.stdout:5/227: truncate f1b 1170707 0 2024-04-25T01:35:44.284 INFO:tasks.workunit.client.0.smithi033.stdout:8/230: creat d6/d10/d48/f4f x:0 0 0 2024-04-25T01:35:44.285 INFO:tasks.workunit.client.0.smithi033.stdout:9/167: fdatasync d0/d2/dd/d20/f38 0 2024-04-25T01:35:44.286 INFO:tasks.workunit.client.0.smithi033.stdout:5/228: dread f7 [4194304,4194304] 0 2024-04-25T01:35:44.287 INFO:tasks.workunit.client.0.smithi033.stdout:7/200: rename d4/d10/d13/l2f to d4/d10/d13/d1f/l37 0 2024-04-25T01:35:44.289 INFO:tasks.workunit.client.0.smithi033.stdout:0/175: link d7/da/l21 d7/da/d23/l34 0 2024-04-25T01:35:44.289 INFO:tasks.workunit.client.0.smithi033.stdout:7/201: fdatasync d4/d10/d13/f33 0 2024-04-25T01:35:44.291 INFO:tasks.workunit.client.0.smithi033.stdout:7/202: write d4/d10/d13/d1f/f36 [18680,62288] 0 2024-04-25T01:35:44.296 INFO:tasks.workunit.client.0.smithi033.stdout:8/231: link d6/d10/d2e/f4a d6/d45/f50 0 2024-04-25T01:35:44.299 INFO:tasks.workunit.client.0.smithi033.stdout:9/168: stat d0/d2/dd/df/d37 0 2024-04-25T01:35:44.299 INFO:tasks.workunit.client.0.smithi033.stdout:8/232: chown d6/d10/f3f 64109 1 2024-04-25T01:35:44.299 INFO:tasks.workunit.client.0.smithi033.stdout:7/203: mknod d4/d23/c38 0 2024-04-25T01:35:44.299 INFO:tasks.workunit.client.0.smithi033.stdout:6/149: link d4/f14 d4/d2a/f34 0 2024-04-25T01:35:44.299 INFO:tasks.workunit.client.0.smithi033.stdout:6/150: write d4/f6 [3613040,102246] 0 2024-04-25T01:35:44.300 INFO:tasks.workunit.client.0.smithi033.stdout:0/176: creat d7/da/df/d2a/f35 x:0 0 0 2024-04-25T01:35:44.302 INFO:tasks.workunit.client.0.smithi033.stdout:9/169: creat d0/d2/dd/df/f3b x:0 0 0 2024-04-25T01:35:44.302 INFO:tasks.workunit.client.0.smithi033.stdout:7/204: rmdir d4 39 2024-04-25T01:35:44.302 INFO:tasks.workunit.client.0.smithi033.stdout:0/177: chown d7/da/df/d2a 113838 1 2024-04-25T01:35:44.303 INFO:tasks.workunit.client.0.smithi033.stdout:6/151: mkdir d4/d24/d29/d35 0 2024-04-25T01:35:44.306 INFO:tasks.workunit.client.0.smithi033.stdout:6/152: mkdir d4/d24/d36 0 2024-04-25T01:35:44.309 INFO:tasks.workunit.client.0.smithi033.stdout:7/205: creat d4/f39 x:0 0 0 2024-04-25T01:35:44.309 INFO:tasks.workunit.client.0.smithi033.stdout:0/178: creat d7/d33/f36 x:0 0 0 2024-04-25T01:35:44.311 INFO:tasks.workunit.client.0.smithi033.stdout:6/153: mkdir d4/d24/d36/d37 0 2024-04-25T01:35:44.311 INFO:tasks.workunit.client.0.smithi033.stdout:7/206: readlink d4/d23/l27 0 2024-04-25T01:35:44.312 INFO:tasks.workunit.client.0.smithi033.stdout:5/229: dread f7 [0,4194304] 0 2024-04-25T01:35:44.312 INFO:tasks.workunit.client.0.smithi033.stdout:9/170: dwrite d0/d11/f18 [0,4194304] 0 2024-04-25T01:35:44.313 INFO:tasks.workunit.client.0.smithi033.stdout:6/154: write d4/f1b [1364707,46233] 0 2024-04-25T01:35:44.313 INFO:tasks.workunit.client.0.smithi033.stdout:7/207: fdatasync d4/d10/f15 0 2024-04-25T01:35:44.315 INFO:tasks.workunit.client.0.smithi033.stdout:5/230: chown d1c/d2a/l3b 4954992 1 2024-04-25T01:35:44.319 INFO:tasks.workunit.client.0.smithi033.stdout:0/179: dread d7/da/df/f10 [8388608,4194304] 0 2024-04-25T01:35:44.320 INFO:tasks.workunit.client.0.smithi033.stdout:7/208: dread d4/f14 [0,4194304] 0 2024-04-25T01:35:44.323 INFO:tasks.workunit.client.0.smithi033.stdout:0/180: rename d7/da to d7/da/df/d2a/d37 22 2024-04-25T01:35:44.323 INFO:tasks.workunit.client.0.smithi033.stdout:7/209: dread - d4/d10/f29 zero size 2024-04-25T01:35:44.324 INFO:tasks.workunit.client.0.smithi033.stdout:9/171: unlink d0/d2/dd/df/d1b/c36 0 2024-04-25T01:35:44.325 INFO:tasks.workunit.client.0.smithi033.stdout:0/181: chown d7/da/df/f28 128387 1 2024-04-25T01:35:44.326 INFO:tasks.workunit.client.0.smithi033.stdout:0/182: rename d7/d2d to d7/d2d/d38 22 2024-04-25T01:35:44.328 INFO:tasks.workunit.client.0.smithi033.stdout:7/210: creat d4/f3a x:0 0 0 2024-04-25T01:35:44.332 INFO:tasks.workunit.client.0.smithi033.stdout:5/231: rmdir d1c/d2a/d49 0 2024-04-25T01:35:44.333 INFO:tasks.workunit.client.0.smithi033.stdout:5/232: dread - d1c/d2a/d3d/f40 zero size 2024-04-25T01:35:44.336 INFO:tasks.workunit.client.0.smithi033.stdout:7/211: rmdir d4/d10 39 2024-04-25T01:35:44.337 INFO:tasks.workunit.client.0.smithi033.stdout:6/155: dread d4/f1b [0,4194304] 0 2024-04-25T01:35:44.338 INFO:tasks.workunit.client.0.smithi033.stdout:9/172: rename d0/f2f to d0/d2/dd/df/d37/f3c 0 2024-04-25T01:35:44.341 INFO:tasks.workunit.client.0.smithi033.stdout:5/233: write d1c/f1d [4763362,29146] 0 2024-04-25T01:35:44.342 INFO:tasks.workunit.client.0.smithi033.stdout:5/234: readlink d1c/l31 0 2024-04-25T01:35:44.343 INFO:tasks.workunit.client.0.smithi033.stdout:9/173: dread d0/d11/f18 [0,4194304] 0 2024-04-25T01:35:44.345 INFO:tasks.workunit.client.0.smithi033.stdout:9/174: chown d0/d11 0 1 2024-04-25T01:35:44.345 INFO:tasks.workunit.client.0.smithi033.stdout:9/175: readlink d0/d2/l19 0 2024-04-25T01:35:44.346 INFO:tasks.workunit.client.0.smithi033.stdout:9/176: stat d0/d2/l19 0 2024-04-25T01:35:44.352 INFO:tasks.workunit.client.0.smithi033.stdout:5/235: unlink d1c/f3c 0 2024-04-25T01:35:44.353 INFO:tasks.workunit.client.0.smithi033.stdout:9/177: mkdir d0/d2/dd/df/d1b/d3d 0 2024-04-25T01:35:44.358 INFO:tasks.workunit.client.0.smithi033.stdout:5/236: chown d1c/d2a/f34 5998175 1 2024-04-25T01:35:44.358 INFO:tasks.workunit.client.0.smithi033.stdout:0/183: truncate d7/da/df/f10 2211725 0 2024-04-25T01:35:44.360 INFO:tasks.workunit.client.0.smithi033.stdout:9/178: creat d0/d2/dd/df/f3e x:0 0 0 2024-04-25T01:35:44.362 INFO:tasks.workunit.client.0.smithi033.stdout:9/179: dread - d0/d2/dd/df/d37/f3c zero size 2024-04-25T01:35:44.362 INFO:tasks.workunit.client.0.smithi033.stdout:5/237: mkdir d1c/d21/d4a 0 2024-04-25T01:35:44.363 INFO:tasks.workunit.client.0.smithi033.stdout:5/238: write d1c/d21/f3a [405474,88905] 0 2024-04-25T01:35:44.364 INFO:tasks.workunit.client.0.smithi033.stdout:7/212: creat d4/d10/f3b x:0 0 0 2024-04-25T01:35:44.364 INFO:tasks.workunit.client.0.smithi033.stdout:5/239: readlink d1c/l24 0 2024-04-25T01:35:44.366 INFO:tasks.workunit.client.0.smithi033.stdout:0/184: mknod d7/da/df/c39 0 2024-04-25T01:35:44.367 INFO:tasks.workunit.client.0.smithi033.stdout:6/156: getdents d4/d2a 0 2024-04-25T01:35:44.368 INFO:tasks.workunit.client.0.smithi033.stdout:6/157: write d4/f2d [692247,89840] 0 2024-04-25T01:35:44.369 INFO:tasks.workunit.client.0.smithi033.stdout:9/180: creat d0/d2/dd/df/d37/f3f x:0 0 0 2024-04-25T01:35:44.370 INFO:tasks.workunit.client.0.smithi033.stdout:9/181: dread - d0/d2/dd/d20/f38 zero size 2024-04-25T01:35:44.375 INFO:tasks.workunit.client.0.smithi033.stdout:0/185: creat d7/da/d23/f3a x:0 0 0 2024-04-25T01:35:44.379 INFO:tasks.workunit.client.0.smithi033.stdout:0/186: dread d7/da/f13 [0,4194304] 0 2024-04-25T01:35:44.380 INFO:tasks.workunit.client.0.smithi033.stdout:9/182: rename d0/d2/l19 to d0/d11/l40 0 2024-04-25T01:35:44.381 INFO:tasks.workunit.client.0.smithi033.stdout:0/187: truncate d7/d1b/f32 401081 0 2024-04-25T01:35:44.383 INFO:tasks.workunit.client.0.smithi033.stdout:0/188: read d7/d1b/f32 [355113,76085] 0 2024-04-25T01:35:44.384 INFO:tasks.workunit.client.0.smithi033.stdout:0/189: chown d7/d1b/c2c 265 1 2024-04-25T01:35:44.385 INFO:tasks.workunit.client.0.smithi033.stdout:6/158: mknod d4/d24/d36/d37/c38 0 2024-04-25T01:35:44.386 INFO:tasks.workunit.client.0.smithi033.stdout:6/159: read d4/f2d [104055,20922] 0 2024-04-25T01:35:44.387 INFO:tasks.workunit.client.0.smithi033.stdout:6/160: chown d4/cc 40732119 1 2024-04-25T01:35:44.391 INFO:tasks.workunit.client.0.smithi033.stdout:7/213: creat d4/d10/d13/f3c x:0 0 0 2024-04-25T01:35:44.394 INFO:tasks.workunit.client.0.smithi033.stdout:0/190: mknod d7/da/c3b 0 2024-04-25T01:35:44.395 INFO:tasks.workunit.client.0.smithi033.stdout:5/240: dwrite d1c/d21/f2c [0,4194304] 0 2024-04-25T01:35:44.399 INFO:tasks.workunit.client.0.smithi033.stdout:6/161: creat d4/d2a/f39 x:0 0 0 2024-04-25T01:35:44.400 INFO:tasks.workunit.client.0.smithi033.stdout:7/214: unlink d4/f14 0 2024-04-25T01:35:44.401 INFO:tasks.workunit.client.0.smithi033.stdout:7/215: write d4/f9 [9535891,61832] 0 2024-04-25T01:35:44.402 INFO:tasks.workunit.client.0.smithi033.stdout:9/183: dread d0/f9 [0,4194304] 0 2024-04-25T01:35:44.410 INFO:tasks.workunit.client.0.smithi033.stdout:5/241: link d1c/d21/f45 d1c/d21/d3e/f4b 0 2024-04-25T01:35:44.410 INFO:tasks.workunit.client.0.smithi033.stdout:0/191: dread d7/da/df/f19 [0,4194304] 0 2024-04-25T01:35:44.410 INFO:tasks.workunit.client.0.smithi033.stdout:5/242: write f15 [1347626,52510] 0 2024-04-25T01:35:44.411 INFO:tasks.workunit.client.0.smithi033.stdout:7/216: creat d4/f3d x:0 0 0 2024-04-25T01:35:44.413 INFO:tasks.workunit.client.0.smithi033.stdout:6/162: dwrite d4/f2d [0,4194304] 0 2024-04-25T01:35:44.417 INFO:tasks.workunit.client.0.smithi033.stdout:0/192: symlink d7/da/l3c 0 2024-04-25T01:35:44.418 INFO:tasks.workunit.client.0.smithi033.stdout:9/184: link d0/d2/f1f d0/d2/dd/f41 0 2024-04-25T01:35:44.420 INFO:tasks.workunit.client.0.smithi033.stdout:7/217: creat d4/d2e/f3e x:0 0 0 2024-04-25T01:35:44.422 INFO:tasks.workunit.client.0.smithi033.stdout:6/163: dwrite f3 [4194304,4194304] 0 2024-04-25T01:35:44.422 INFO:tasks.workunit.client.0.smithi033.stdout:5/243: creat d1c/d21/d4a/f4c x:0 0 0 2024-04-25T01:35:44.425 INFO:tasks.workunit.client.0.smithi033.stdout:0/193: creat d7/d2d/f3d x:0 0 0 2024-04-25T01:35:44.429 INFO:tasks.workunit.client.0.smithi033.stdout:7/218: mkdir d4/d23/d3f 0 2024-04-25T01:35:44.430 INFO:tasks.workunit.client.0.smithi033.stdout:7/219: stat d4/d10/d13/d1f/f24 0 2024-04-25T01:35:44.432 INFO:tasks.workunit.client.0.smithi033.stdout:6/164: symlink d4/d1d/l3a 0 2024-04-25T01:35:44.433 INFO:tasks.workunit.client.0.smithi033.stdout:9/185: mkdir d0/d2/dd/df/d1b/d3d/d42 0 2024-04-25T01:35:44.435 INFO:tasks.workunit.client.0.smithi033.stdout:5/244: link d1c/d21/f29 d1c/d2a/d3d/f4d 0 2024-04-25T01:35:44.439 INFO:tasks.workunit.client.0.smithi033.stdout:9/186: rename d0/d2/dd/df/d37 to d0/d2/d43 0 2024-04-25T01:35:44.441 INFO:tasks.workunit.client.0.smithi033.stdout:7/220: symlink d4/d10/l40 0 2024-04-25T01:35:44.442 INFO:tasks.workunit.client.0.smithi033.stdout:5/245: dread f1b [0,4194304] 0 2024-04-25T01:35:44.442 INFO:tasks.workunit.client.0.smithi033.stdout:6/165: getdents d4/d24/d29/d35 0 2024-04-25T01:35:44.443 INFO:tasks.workunit.client.0.smithi033.stdout:9/187: mknod d0/d2/dd/df/d1b/d3d/c44 0 2024-04-25T01:35:44.446 INFO:tasks.workunit.client.0.smithi033.stdout:5/246: creat d1c/d21/d4a/f4e x:0 0 0 2024-04-25T01:35:44.450 INFO:tasks.workunit.client.0.smithi033.stdout:5/247: symlink d1c/d21/l4f 0 2024-04-25T01:35:44.450 INFO:tasks.workunit.client.0.smithi033.stdout:0/194: fsync d7/d2d/f3d 0 2024-04-25T01:35:44.451 INFO:tasks.workunit.client.0.smithi033.stdout:9/188: mkdir d0/d2/dd/d45 0 2024-04-25T01:35:44.452 INFO:tasks.workunit.client.0.smithi033.stdout:0/195: truncate d7/da/df/f19 2538818 0 2024-04-25T01:35:44.454 INFO:tasks.workunit.client.0.smithi033.stdout:7/221: rename d4/d10/f15 to d4/f41 0 2024-04-25T01:35:44.455 INFO:tasks.workunit.client.0.smithi033.stdout:7/222: write d4/d23/f2b [4112,70235] 0 2024-04-25T01:35:44.456 INFO:tasks.workunit.client.0.smithi033.stdout:6/166: rmdir d4/d24/d29/d35 0 2024-04-25T01:35:44.458 INFO:tasks.workunit.client.0.smithi033.stdout:9/189: fdatasync d0/d11/f27 0 2024-04-25T01:35:44.459 INFO:tasks.workunit.client.0.smithi033.stdout:0/196: creat d7/d33/f3e x:0 0 0 2024-04-25T01:35:44.460 INFO:tasks.workunit.client.0.smithi033.stdout:5/248: mkdir d1c/d2a/d50 0 2024-04-25T01:35:44.461 INFO:tasks.workunit.client.0.smithi033.stdout:6/167: rmdir d4/d24 39 2024-04-25T01:35:44.463 INFO:tasks.workunit.client.0.smithi033.stdout:0/197: mknod d7/da/d23/c3f 0 2024-04-25T01:35:44.466 INFO:tasks.workunit.client.0.smithi033.stdout:7/223: link d4/d10/d13/f3c d4/d10/d13/d1f/f42 0 2024-04-25T01:35:44.467 INFO:tasks.workunit.client.0.smithi033.stdout:9/190: symlink d0/l46 0 2024-04-25T01:35:44.468 INFO:tasks.workunit.client.0.smithi033.stdout:5/249: dwrite d1c/f22 [0,4194304] 0 2024-04-25T01:35:44.471 INFO:tasks.workunit.client.0.smithi033.stdout:7/224: read d4/f2c [35061,39806] 0 2024-04-25T01:35:44.471 INFO:tasks.workunit.client.0.smithi033.stdout:6/168: creat d4/d24/f3b x:0 0 0 2024-04-25T01:35:44.473 INFO:tasks.workunit.client.0.smithi033.stdout:9/191: rename d0/d11/f35 to d0/d11/f47 0 2024-04-25T01:35:44.474 INFO:tasks.workunit.client.0.smithi033.stdout:9/192: read - d0/d2/dd/df/f3a zero size 2024-04-25T01:35:44.484 INFO:tasks.workunit.client.0.smithi033.stdout:5/250: creat d1c/f51 x:0 0 0 2024-04-25T01:35:44.484 INFO:tasks.workunit.client.0.smithi033.stdout:6/169: dread d4/d2a/f34 [0,4194304] 0 2024-04-25T01:35:44.486 INFO:tasks.workunit.client.0.smithi033.stdout:9/193: dread d0/d2/dd/f24 [0,4194304] 0 2024-04-25T01:35:44.486 INFO:tasks.workunit.client.0.smithi033.stdout:6/170: truncate d4/d24/f3b 526445 0 2024-04-25T01:35:44.490 INFO:tasks.workunit.client.0.smithi033.stdout:7/225: rename d4/f2c to d4/f43 0 2024-04-25T01:35:44.492 INFO:tasks.workunit.client.0.smithi033.stdout:6/171: dread f3 [4194304,4194304] 0 2024-04-25T01:35:44.495 INFO:tasks.workunit.client.0.smithi033.stdout:9/194: dwrite d0/d11/f27 [0,4194304] 0 2024-04-25T01:35:44.497 INFO:tasks.workunit.client.0.smithi033.stdout:6/172: dread d4/d24/f3b [0,4194304] 0 2024-04-25T01:35:44.498 INFO:tasks.workunit.client.0.smithi033.stdout:7/226: dread d4/f11 [4194304,4194304] 0 2024-04-25T01:35:44.499 INFO:tasks.workunit.client.0.smithi033.stdout:9/195: dread - d0/d2/dd/f15 zero size 2024-04-25T01:35:44.500 INFO:tasks.workunit.client.0.smithi033.stdout:6/173: symlink d4/d24/d36/d37/l3c 0 2024-04-25T01:35:44.500 INFO:tasks.workunit.client.0.smithi033.stdout:9/196: chown d0/d2/dd/df/d1b/d3d 125798433 1 2024-04-25T01:35:44.501 INFO:tasks.workunit.client.0.smithi033.stdout:7/227: mknod d4/d23/c44 0 2024-04-25T01:35:44.502 INFO:tasks.workunit.client.0.smithi033.stdout:9/197: dread - d0/d2/dd/df/d1b/f2e zero size 2024-04-25T01:35:44.503 INFO:tasks.workunit.client.0.smithi033.stdout:6/174: creat d4/d15/f3d x:0 0 0 2024-04-25T01:35:44.504 INFO:tasks.workunit.client.0.smithi033.stdout:5/251: dread d1c/d21/f26 [0,4194304] 0 2024-04-25T01:35:44.504 INFO:tasks.workunit.client.0.smithi033.stdout:9/198: creat d0/d2/f48 x:0 0 0 2024-04-25T01:35:44.505 INFO:tasks.workunit.client.0.smithi033.stdout:7/228: mkdir d4/d23/d3f/d45 0 2024-04-25T01:35:44.506 INFO:tasks.workunit.client.0.smithi033.stdout:5/252: fdatasync d1c/d21/f26 0 2024-04-25T01:35:44.506 INFO:tasks.workunit.client.0.smithi033.stdout:7/229: write d4/d2e/f3e [195379,68683] 0 2024-04-25T01:35:44.507 INFO:tasks.workunit.client.0.smithi033.stdout:9/199: rmdir d0/d2/dd/df/d1b 39 2024-04-25T01:35:44.509 INFO:tasks.workunit.client.0.smithi033.stdout:5/253: mkdir d1c/d2a/d50/d52 0 2024-04-25T01:35:44.511 INFO:tasks.workunit.client.0.smithi033.stdout:7/230: rename d4/d10/d13/f3c to d4/d10/d13/d1f/f46 0 2024-04-25T01:35:44.512 INFO:tasks.workunit.client.0.smithi033.stdout:9/200: dread d0/d11/f18 [0,4194304] 0 2024-04-25T01:35:44.512 INFO:tasks.workunit.client.0.smithi033.stdout:5/254: mkdir d1c/d53 0 2024-04-25T01:35:44.514 INFO:tasks.workunit.client.0.smithi033.stdout:9/201: write d0/d2/dd/d20/f38 [218298,54479] 0 2024-04-25T01:35:44.514 INFO:tasks.workunit.client.0.smithi033.stdout:5/255: fdatasync d1c/d21/d3e/f4b 0 2024-04-25T01:35:44.520 INFO:tasks.workunit.client.0.smithi033.stdout:9/202: creat d0/d2/dd/df/d1b/d3d/d42/f49 x:0 0 0 2024-04-25T01:35:44.523 INFO:tasks.workunit.client.0.smithi033.stdout:9/203: link d0/d2/dd/df/d1b/c22 d0/d2/dd/df/d1b/d3d/d42/c4a 0 2024-04-25T01:35:44.525 INFO:tasks.workunit.client.0.smithi033.stdout:9/204: mkdir d0/d2/d43/d4b 0 2024-04-25T01:35:44.526 INFO:tasks.workunit.client.0.smithi033.stdout:9/205: creat d0/d11/f4c x:0 0 0 2024-04-25T01:35:44.538 INFO:tasks.workunit.client.0.smithi033.stdout:1/166: sync 2024-04-25T01:35:44.538 INFO:tasks.workunit.client.0.smithi033.stdout:4/235: sync 2024-04-25T01:35:44.542 INFO:tasks.workunit.client.0.smithi033.stdout:4/236: write d6/d12/f32 [743186,14598] 0 2024-04-25T01:35:44.547 INFO:tasks.workunit.client.0.smithi033.stdout:1/167: dread d2/d11/f1e [0,4194304] 0 2024-04-25T01:35:44.552 INFO:tasks.workunit.client.0.smithi033.stdout:7/231: dread d4/d2e/ff [0,4194304] 0 2024-04-25T01:35:44.553 INFO:tasks.workunit.client.0.smithi033.stdout:7/232: stat d4/d10/l40 0 2024-04-25T01:35:44.556 INFO:tasks.workunit.client.0.smithi033.stdout:1/168: dwrite d2/d5/f25 [0,4194304] 0 2024-04-25T01:35:44.559 INFO:tasks.workunit.client.0.smithi033.stdout:1/169: rmdir d2/d11/d28 39 2024-04-25T01:35:44.560 INFO:tasks.workunit.client.0.smithi033.stdout:6/175: truncate d4/d24/f3b 623049 0 2024-04-25T01:35:44.562 INFO:tasks.workunit.client.0.smithi033.stdout:7/233: creat d4/d10/d13/f47 x:0 0 0 2024-04-25T01:35:44.564 INFO:tasks.workunit.client.0.smithi033.stdout:6/176: mkdir d4/d1d/d3e 0 2024-04-25T01:35:44.566 INFO:tasks.workunit.client.0.smithi033.stdout:6/177: write d4/d2a/f39 [99554,33027] 0 2024-04-25T01:35:44.579 INFO:tasks.workunit.client.0.smithi033.stdout:5/256: dwrite f16 [0,4194304] 0 2024-04-25T01:35:44.580 INFO:tasks.workunit.client.0.smithi033.stdout:1/170: truncate d2/d5/fd 1662233 0 2024-04-25T01:35:44.581 INFO:tasks.workunit.client.0.smithi033.stdout:6/178: write d4/d2a/f34 [3815340,76442] 0 2024-04-25T01:35:44.585 INFO:tasks.workunit.client.0.smithi033.stdout:4/237: dwrite d6/f9 [4194304,4194304] 0 2024-04-25T01:35:44.591 INFO:tasks.workunit.client.0.smithi033.stdout:5/257: creat d1c/d2a/d50/d52/f54 x:0 0 0 2024-04-25T01:35:44.592 INFO:tasks.workunit.client.0.smithi033.stdout:6/179: rename d4/d15/f3d to d4/d2a/f3f 0 2024-04-25T01:35:44.594 INFO:tasks.workunit.client.0.smithi033.stdout:1/171: getdents d2/d2e 0 2024-04-25T01:35:44.596 INFO:tasks.workunit.client.0.smithi033.stdout:6/180: symlink d4/l40 0 2024-04-25T01:35:44.597 INFO:tasks.workunit.client.0.smithi033.stdout:1/172: stat d2/d11/d28/f2c 0 2024-04-25T01:35:44.600 INFO:tasks.workunit.client.0.smithi033.stdout:5/258: dwrite d1c/f32 [0,4194304] 0 2024-04-25T01:35:44.603 INFO:tasks.workunit.client.0.smithi033.stdout:6/181: dread d4/f2d [0,4194304] 0 2024-04-25T01:35:44.604 INFO:tasks.workunit.client.0.smithi033.stdout:5/259: mknod d1c/d2a/d50/c55 0 2024-04-25T01:35:44.606 INFO:tasks.workunit.client.0.smithi033.stdout:5/260: truncate d1c/f2e 1358485 0 2024-04-25T01:35:44.610 INFO:tasks.workunit.client.0.smithi033.stdout:5/261: mknod d1c/c56 0 2024-04-25T01:35:44.615 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:44 smithi138 ceph-mon[32977]: Standby manager daemon smithi033.asemba started 2024-04-25T01:35:44.615 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:44 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.33:0/2120145286' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.asemba/crt"}]: dispatch 2024-04-25T01:35:44.615 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:44 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.33:0/2120145286' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:35:44.615 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:44 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.33:0/2120145286' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.asemba/key"}]: dispatch 2024-04-25T01:35:44.615 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:44 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.33:0/2120145286' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:35:44.616 INFO:tasks.workunit.client.0.smithi033.stdout:1/173: dwrite d2/d5/d1b/f2b [0,4194304] 0 2024-04-25T01:35:44.616 INFO:tasks.workunit.client.0.smithi033.stdout:5/262: creat d1c/d53/f57 x:0 0 0 2024-04-25T01:35:44.616 INFO:tasks.workunit.client.0.smithi033.stdout:6/182: dwrite d4/d2a/f39 [0,4194304] 0 2024-04-25T01:35:44.616 INFO:tasks.workunit.client.0.smithi033.stdout:1/174: chown d2/d11/l23 62 1 2024-04-25T01:35:44.616 INFO:tasks.workunit.client.0.smithi033.stdout:1/175: chown d2/d2e/c2f 535009875 1 2024-04-25T01:35:44.617 INFO:tasks.workunit.client.0.smithi033.stdout:5/263: mknod d1c/d2a/c58 0 2024-04-25T01:35:44.617 INFO:tasks.workunit.client.0.smithi033.stdout:5/264: readlink d1c/d21/l2d 0 2024-04-25T01:35:44.618 INFO:tasks.workunit.client.0.smithi033.stdout:1/176: rmdir d2/d11/d28 39 2024-04-25T01:35:44.619 INFO:tasks.workunit.client.0.smithi033.stdout:6/183: mkdir d4/d1d/d3e/d41 0 2024-04-25T01:35:44.619 INFO:tasks.workunit.client.0.smithi033.stdout:6/184: chown d4/d15/f20 1403 1 2024-04-25T01:35:44.622 INFO:tasks.workunit.client.0.smithi033.stdout:6/185: mkdir d4/d24/d36/d42 0 2024-04-25T01:35:44.623 INFO:tasks.workunit.client.0.smithi033.stdout:6/186: chown d4/f12 57675182 1 2024-04-25T01:35:44.624 INFO:tasks.workunit.client.0.smithi033.stdout:1/177: creat d2/f32 x:0 0 0 2024-04-25T01:35:44.627 INFO:tasks.workunit.client.0.smithi033.stdout:6/187: rename l0 to d4/d24/d36/l43 0 2024-04-25T01:35:44.633 INFO:tasks.workunit.client.0.smithi033.stdout:1/178: mkdir d2/d11/d28/d33 0 2024-04-25T01:35:44.634 INFO:tasks.workunit.client.0.smithi033.stdout:5/265: truncate f14 729784 0 2024-04-25T01:35:44.637 INFO:tasks.workunit.client.0.smithi033.stdout:6/188: dwrite d4/d2a/f3f [0,4194304] 0 2024-04-25T01:35:44.638 INFO:tasks.workunit.client.0.smithi033.stdout:1/179: unlink d2/d5/d1b/f2b 0 2024-04-25T01:35:44.639 INFO:tasks.workunit.client.0.smithi033.stdout:1/180: write d2/d5/f18 [447980,94446] 0 2024-04-25T01:35:44.640 INFO:tasks.workunit.client.0.smithi033.stdout:6/189: creat d4/d24/d36/d37/f44 x:0 0 0 2024-04-25T01:35:44.642 INFO:tasks.workunit.client.0.smithi033.stdout:1/181: unlink d2/d5/c1c 0 2024-04-25T01:35:44.643 INFO:tasks.workunit.client.0.smithi033.stdout:6/190: symlink d4/d2a/l45 0 2024-04-25T01:35:44.644 INFO:tasks.workunit.client.0.smithi033.stdout:1/182: chown d2/c31 30091578 1 2024-04-25T01:35:44.646 INFO:tasks.workunit.client.0.smithi033.stdout:6/191: rename d4/c23 to d4/d1d/d3e/c46 0 2024-04-25T01:35:44.647 INFO:tasks.workunit.client.0.smithi033.stdout:1/183: fsync d2/d5/f14 0 2024-04-25T01:35:44.663 INFO:tasks.workunit.client.0.smithi033.stdout:6/192: rename d4/d1d/d3e/c46 to d4/d24/d29/c47 0 2024-04-25T01:35:44.668 INFO:tasks.workunit.client.0.smithi033.stdout:6/193: dread d4/f6 [0,4194304] 0 2024-04-25T01:35:44.670 INFO:tasks.workunit.client.0.smithi033.stdout:6/194: write d4/d2a/f30 [4073100,28800] 0 2024-04-25T01:35:44.674 INFO:tasks.workunit.client.0.smithi033.stdout:6/195: link d4/d2a/f39 d4/d15/f48 0 2024-04-25T01:35:44.675 INFO:tasks.workunit.client.0.smithi033.stdout:6/196: write d4/d2a/f34 [4572325,119469] 0 2024-04-25T01:35:44.677 INFO:tasks.workunit.client.0.smithi033.stdout:6/197: write d4/f2d [2279250,113438] 0 2024-04-25T01:35:44.683 INFO:tasks.workunit.client.0.smithi033.stdout:6/198: dwrite d4/f2d [0,4194304] 0 2024-04-25T01:35:44.687 INFO:tasks.workunit.client.0.smithi033.stdout:6/199: unlink d4/d2a/f31 0 2024-04-25T01:35:44.694 INFO:tasks.workunit.client.0.smithi033.stdout:6/200: fsync d4/d2a/f39 0 2024-04-25T01:35:44.696 INFO:tasks.workunit.client.0.smithi033.stdout:6/201: read d4/f12 [3334945,3654] 0 2024-04-25T01:35:44.699 INFO:tasks.workunit.client.0.smithi033.stdout:6/202: creat d4/d15/f49 x:0 0 0 2024-04-25T01:35:44.700 INFO:tasks.workunit.client.0.smithi033.stdout:6/203: chown d4/d24/d29/f33 4037128 1 2024-04-25T01:35:44.702 INFO:tasks.workunit.client.0.smithi033.stdout:6/204: mkdir d4/d1d/d3e/d4a 0 2024-04-25T01:35:44.703 INFO:tasks.workunit.client.0.smithi033.stdout:6/205: rmdir d4/d2a 39 2024-04-25T01:35:44.705 INFO:tasks.workunit.client.0.smithi033.stdout:6/206: creat d4/d1d/f4b x:0 0 0 2024-04-25T01:35:44.707 INFO:tasks.workunit.client.0.smithi033.stdout:6/207: mknod d4/d15/c4c 0 2024-04-25T01:35:44.710 INFO:tasks.workunit.client.0.smithi033.stdout:6/208: dread d4/d15/f48 [0,4194304] 0 2024-04-25T01:35:44.714 INFO:tasks.workunit.client.0.smithi033.stdout:6/209: rename d4/d15/c25 to d4/d24/d36/d42/c4d 0 2024-04-25T01:35:44.715 INFO:tasks.workunit.client.0.smithi033.stdout:6/210: mkdir d4/d1d/d3e/d41/d4e 0 2024-04-25T01:35:44.721 INFO:tasks.workunit.client.0.smithi033.stdout:6/211: dwrite d4/f11 [0,4194304] 0 2024-04-25T01:35:44.723 INFO:tasks.workunit.client.0.smithi033.stdout:6/212: creat d4/d1d/d3e/d4a/f4f x:0 0 0 2024-04-25T01:35:44.725 INFO:tasks.workunit.client.0.smithi033.stdout:6/213: creat d4/d1d/f50 x:0 0 0 2024-04-25T01:35:44.726 INFO:tasks.workunit.client.0.smithi033.stdout:6/214: readlink d4/d24/d36/l43 0 2024-04-25T01:35:44.727 INFO:tasks.workunit.client.0.smithi033.stdout:6/215: symlink d4/d15/l51 0 2024-04-25T01:35:44.729 INFO:tasks.workunit.client.0.smithi033.stdout:6/216: creat d4/d1d/f52 x:0 0 0 2024-04-25T01:35:44.730 INFO:tasks.workunit.client.0.smithi033.stdout:6/217: symlink d4/d1d/d3e/l53 0 2024-04-25T01:35:44.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:44 smithi033 ceph-mon[26353]: Standby manager daemon smithi033.asemba started 2024-04-25T01:35:44.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:44 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.33:0/2120145286' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.asemba/crt"}]: dispatch 2024-04-25T01:35:44.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:44 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.33:0/2120145286' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:35:44.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:44 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.33:0/2120145286' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.asemba/key"}]: dispatch 2024-04-25T01:35:44.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:44 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.33:0/2120145286' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:35:44.765 INFO:tasks.workunit.client.0.smithi033.stdout:8/233: sync 2024-04-25T01:35:44.766 INFO:tasks.workunit.client.0.smithi033.stdout:2/183: sync 2024-04-25T01:35:44.766 INFO:tasks.workunit.client.0.smithi033.stdout:3/198: sync 2024-04-25T01:35:44.766 INFO:tasks.workunit.client.0.smithi033.stdout:2/184: readlink da/d23/l2b 0 2024-04-25T01:35:44.766 INFO:tasks.workunit.client.0.smithi033.stdout:2/185: chown da/d32 193 1 2024-04-25T01:35:44.773 INFO:tasks.workunit.client.0.smithi033.stdout:3/199: creat d8/dd/d11/d1e/f45 x:0 0 0 2024-04-25T01:35:44.777 INFO:tasks.workunit.client.0.smithi033.stdout:6/218: dwrite d4/d15/f48 [0,4194304] 0 2024-04-25T01:35:44.778 INFO:tasks.workunit.client.0.smithi033.stdout:3/200: symlink d8/dd/d40/l46 0 2024-04-25T01:35:44.779 INFO:tasks.workunit.client.0.smithi033.stdout:8/234: dwrite d6/d10/f28 [0,4194304] 0 2024-04-25T01:35:44.780 INFO:tasks.workunit.client.0.smithi033.stdout:2/186: dwrite da/f15 [0,4194304] 0 2024-04-25T01:35:44.781 INFO:tasks.workunit.client.0.smithi033.stdout:2/187: chown c3 0 1 2024-04-25T01:35:44.783 INFO:tasks.workunit.client.0.smithi033.stdout:6/219: mkdir d4/d1d/d3e/d41/d54 0 2024-04-25T01:35:44.785 INFO:tasks.workunit.client.0.smithi033.stdout:6/220: truncate d4/d1d/f52 529093 0 2024-04-25T01:35:44.787 INFO:tasks.workunit.client.0.smithi033.stdout:8/235: rename d6/d10/d48 to d6/d45/d51 0 2024-04-25T01:35:44.788 INFO:tasks.workunit.client.0.smithi033.stdout:6/221: creat d4/d24/d29/f55 x:0 0 0 2024-04-25T01:35:44.792 INFO:tasks.workunit.client.0.smithi033.stdout:2/188: rename f0 to da/d2c/d30/f3a 0 2024-04-25T01:35:44.794 INFO:tasks.workunit.client.0.smithi033.stdout:6/222: fsync d4/d2a/f30 0 2024-04-25T01:35:44.796 INFO:tasks.workunit.client.0.smithi033.stdout:2/189: rmdir da/d2c 39 2024-04-25T01:35:44.798 INFO:tasks.workunit.client.0.smithi033.stdout:3/201: write d8/fa [4343530,70624] 0 2024-04-25T01:35:44.802 INFO:tasks.workunit.client.0.smithi033.stdout:6/223: link d4/d1d/d3e/l53 d4/l56 0 2024-04-25T01:35:44.807 INFO:tasks.workunit.client.0.smithi033.stdout:2/190: dread da/d2c/f36 [0,4194304] 0 2024-04-25T01:35:44.807 INFO:tasks.workunit.client.0.smithi033.stdout:3/202: dwrite d8/dd/f34 [0,4194304] 0 2024-04-25T01:35:44.807 INFO:tasks.workunit.client.0.smithi033.stdout:6/224: creat d4/d1d/d3e/d41/d54/f57 x:0 0 0 2024-04-25T01:35:44.809 INFO:tasks.workunit.client.0.smithi033.stdout:2/191: stat da/d2c/d30 0 2024-04-25T01:35:44.810 INFO:tasks.workunit.client.0.smithi033.stdout:8/236: rename d6/d10 to d6/d45/d52 0 2024-04-25T01:35:44.812 INFO:tasks.workunit.client.0.smithi033.stdout:6/225: rmdir d4/d15 39 2024-04-25T01:35:44.812 INFO:tasks.workunit.client.0.smithi033.stdout:8/237: fdatasync d6/d45/d52/f36 0 2024-04-25T01:35:44.813 INFO:tasks.workunit.client.0.smithi033.stdout:8/238: readlink d6/db/l29 0 2024-04-25T01:35:44.817 INFO:tasks.workunit.client.0.smithi033.stdout:6/226: creat d4/d1d/d3e/d41/f58 x:0 0 0 2024-04-25T01:35:44.819 INFO:tasks.workunit.client.0.smithi033.stdout:8/239: symlink d6/d45/d52/d2e/l53 0 2024-04-25T01:35:44.823 INFO:tasks.workunit.client.0.smithi033.stdout:6/227: fsync d4/d15/f49 0 2024-04-25T01:35:44.824 INFO:tasks.workunit.client.0.smithi033.stdout:3/203: write d8/dd/d11/f39 [2202888,27452] 0 2024-04-25T01:35:44.826 INFO:tasks.workunit.client.0.smithi033.stdout:2/192: getdents da/d27 0 2024-04-25T01:35:44.827 INFO:tasks.workunit.client.0.smithi033.stdout:8/240: creat d6/d45/d52/f54 x:0 0 0 2024-04-25T01:35:44.828 INFO:tasks.workunit.client.0.smithi033.stdout:2/193: chown da/d23 50968 1 2024-04-25T01:35:44.833 INFO:tasks.workunit.client.0.smithi033.stdout:3/204: dread d8/fa [4194304,4194304] 0 2024-04-25T01:35:44.834 INFO:tasks.workunit.client.0.smithi033.stdout:2/194: creat da/d2c/d30/f3b x:0 0 0 2024-04-25T01:35:44.834 INFO:tasks.workunit.client.0.smithi033.stdout:6/228: dwrite d4/d2a/f30 [0,4194304] 0 2024-04-25T01:35:44.838 INFO:tasks.workunit.client.0.smithi033.stdout:6/229: fsync d4/d1d/d3e/d41/f58 0 2024-04-25T01:35:44.838 INFO:tasks.workunit.client.0.smithi033.stdout:3/205: mkdir d8/dd/d11/d1e/d27/d3c/d47 0 2024-04-25T01:35:44.840 INFO:tasks.workunit.client.0.smithi033.stdout:8/241: dwrite d6/f21 [4194304,4194304] 0 2024-04-25T01:35:44.840 INFO:tasks.workunit.client.0.smithi033.stdout:6/230: chown d4/l10 0 1 2024-04-25T01:35:44.841 INFO:tasks.workunit.client.0.smithi033.stdout:2/195: dread f8 [0,4194304] 0 2024-04-25T01:35:44.842 INFO:tasks.workunit.client.0.smithi033.stdout:2/196: write da/f2f [449807,25927] 0 2024-04-25T01:35:44.842 INFO:tasks.workunit.client.0.smithi033.stdout:6/231: chown d4/d15/f20 1918965 1 2024-04-25T01:35:44.842 INFO:tasks.workunit.client.0.smithi033.stdout:8/242: stat d6/d45/d52/d2e/l35 0 2024-04-25T01:35:44.843 INFO:tasks.workunit.client.0.smithi033.stdout:3/206: creat d8/dd/d11/d1e/d27/f48 x:0 0 0 2024-04-25T01:35:44.844 INFO:tasks.workunit.client.0.smithi033.stdout:8/243: readlink d6/l39 0 2024-04-25T01:35:44.845 INFO:tasks.workunit.client.0.smithi033.stdout:2/197: write f7 [2267374,27918] 0 2024-04-25T01:35:44.850 INFO:tasks.workunit.client.0.smithi033.stdout:8/244: mkdir d6/d45/d51/d55 0 2024-04-25T01:35:44.851 INFO:tasks.workunit.client.0.smithi033.stdout:3/207: symlink d8/dd/d1d/l49 0 2024-04-25T01:35:44.852 INFO:tasks.workunit.client.0.smithi033.stdout:3/208: dread - d8/f23 zero size 2024-04-25T01:35:44.853 INFO:tasks.workunit.client.0.smithi033.stdout:8/245: creat d6/d45/d52/d2e/f56 x:0 0 0 2024-04-25T01:35:44.854 INFO:tasks.workunit.client.0.smithi033.stdout:3/209: mknod d8/dd/c4a 0 2024-04-25T01:35:44.857 INFO:tasks.workunit.client.0.smithi033.stdout:3/210: symlink d8/dd/d11/d1e/d27/l4b 0 2024-04-25T01:35:44.861 INFO:tasks.workunit.client.0.smithi033.stdout:8/246: dwrite d6/d45/d51/f4f [0,4194304] 0 2024-04-25T01:35:44.862 INFO:tasks.workunit.client.0.smithi033.stdout:3/211: symlink d8/dd/d11/d1e/d27/d3c/d47/l4c 0 2024-04-25T01:35:44.862 INFO:tasks.workunit.client.0.smithi033.stdout:6/232: write d4/f12 [1180102,92677] 0 2024-04-25T01:35:44.865 INFO:tasks.workunit.client.0.smithi033.stdout:6/233: dread - d4/d1d/d3e/d41/d54/f57 zero size 2024-04-25T01:35:44.866 INFO:tasks.workunit.client.0.smithi033.stdout:2/198: truncate da/f2f 347384 0 2024-04-25T01:35:44.867 INFO:tasks.workunit.client.0.smithi033.stdout:6/234: write d4/f12 [3714649,126295] 0 2024-04-25T01:35:44.870 INFO:tasks.workunit.client.0.smithi033.stdout:2/199: write da/d32/f35 [171133,123711] 0 2024-04-25T01:35:44.871 INFO:tasks.workunit.client.0.smithi033.stdout:8/247: getdents d6/d45/d51 0 2024-04-25T01:35:44.874 INFO:tasks.workunit.client.0.smithi033.stdout:3/212: dwrite d8/dd/d11/d1e/f29 [0,4194304] 0 2024-04-25T01:35:44.879 INFO:tasks.workunit.client.0.smithi033.stdout:6/235: dread d4/d2a/f34 [0,4194304] 0 2024-04-25T01:35:44.881 INFO:tasks.workunit.client.0.smithi033.stdout:8/248: dwrite d6/f3c [0,4194304] 0 2024-04-25T01:35:44.883 INFO:tasks.workunit.client.0.smithi033.stdout:6/236: fsync d4/f1b 0 2024-04-25T01:35:44.885 INFO:tasks.workunit.client.0.smithi033.stdout:8/249: read - d6/f1d zero size 2024-04-25T01:35:44.889 INFO:tasks.workunit.client.0.smithi033.stdout:8/250: creat d6/d45/f57 x:0 0 0 2024-04-25T01:35:44.890 INFO:tasks.workunit.client.0.smithi033.stdout:6/237: dread f3 [4194304,4194304] 0 2024-04-25T01:35:44.892 INFO:tasks.workunit.client.0.smithi033.stdout:2/200: dwrite da/f2f [0,4194304] 0 2024-04-25T01:35:44.894 INFO:tasks.workunit.client.0.smithi033.stdout:2/201: write da/ff [35218,81101] 0 2024-04-25T01:35:44.896 INFO:tasks.workunit.client.0.smithi033.stdout:6/238: dread d4/f1b [0,4194304] 0 2024-04-25T01:35:44.899 INFO:tasks.workunit.client.0.smithi033.stdout:2/202: dread da/d2c/f33 [0,4194304] 0 2024-04-25T01:35:44.900 INFO:tasks.workunit.client.0.smithi033.stdout:6/239: rmdir d4/d1d 39 2024-04-25T01:35:44.904 INFO:tasks.workunit.client.0.smithi033.stdout:6/240: symlink d4/l59 0 2024-04-25T01:35:44.907 INFO:tasks.workunit.client.0.smithi033.stdout:2/203: dwrite f9 [0,4194304] 0 2024-04-25T01:35:44.908 INFO:tasks.workunit.client.0.smithi033.stdout:2/204: fdatasync da/f12 0 2024-04-25T01:35:44.911 INFO:tasks.workunit.client.0.smithi033.stdout:2/205: mknod da/d23/c3c 0 2024-04-25T01:35:44.913 INFO:tasks.workunit.client.0.smithi033.stdout:8/251: dread d6/db/f24 [0,4194304] 0 2024-04-25T01:35:44.915 INFO:tasks.workunit.client.0.smithi033.stdout:8/252: chown d6/l1b 851080 1 2024-04-25T01:35:44.916 INFO:tasks.workunit.client.0.smithi033.stdout:8/253: truncate d6/db/f18 895578 0 2024-04-25T01:35:44.917 INFO:tasks.workunit.client.0.smithi033.stdout:8/254: write d6/f3e [671354,67519] 0 2024-04-25T01:35:44.919 INFO:tasks.workunit.client.0.smithi033.stdout:6/241: truncate d4/d1d/f52 80551 0 2024-04-25T01:35:44.920 INFO:tasks.workunit.client.0.smithi033.stdout:8/255: creat d6/f58 x:0 0 0 2024-04-25T01:35:44.920 INFO:tasks.workunit.client.0.smithi033.stdout:8/256: rename d6 to d6/d49/d59 22 2024-04-25T01:35:44.921 INFO:tasks.workunit.client.0.smithi033.stdout:6/242: mknod d4/d1d/d3e/d41/c5a 0 2024-04-25T01:35:44.921 INFO:tasks.workunit.client.0.smithi033.stdout:8/257: fsync d6/d45/d52/d2e/f30 0 2024-04-25T01:35:44.922 INFO:tasks.workunit.client.0.smithi033.stdout:6/243: creat d4/d24/f5b x:0 0 0 2024-04-25T01:35:44.928 INFO:tasks.workunit.client.0.smithi033.stdout:8/258: dwrite d6/d45/d52/f3f [0,4194304] 0 2024-04-25T01:35:44.932 INFO:tasks.workunit.client.0.smithi033.stdout:8/259: creat d6/d49/f5a x:0 0 0 2024-04-25T01:35:44.934 INFO:tasks.workunit.client.0.smithi033.stdout:2/206: dread f4 [0,4194304] 0 2024-04-25T01:35:44.935 INFO:tasks.workunit.client.0.smithi033.stdout:8/260: creat d6/d45/f5b x:0 0 0 2024-04-25T01:35:44.937 INFO:tasks.workunit.client.0.smithi033.stdout:2/207: unlink f9 0 2024-04-25T01:35:44.940 INFO:tasks.workunit.client.0.smithi033.stdout:2/208: rename f1 to da/d27/d31/f3d 0 2024-04-25T01:35:44.942 INFO:tasks.workunit.client.0.smithi033.stdout:8/261: creat d6/d45/d51/d55/f5c x:0 0 0 2024-04-25T01:35:44.947 INFO:tasks.workunit.client.0.smithi033.stdout:2/209: dwrite da/f12 [0,4194304] 0 2024-04-25T01:35:44.949 INFO:tasks.workunit.client.0.smithi033.stdout:8/262: dwrite d6/db/f18 [0,4194304] 0 2024-04-25T01:35:44.957 INFO:tasks.workunit.client.0.smithi033.stdout:2/210: link da/l1f da/d32/l3e 0 2024-04-25T01:35:44.957 INFO:tasks.workunit.client.0.smithi033.stdout:8/263: mkdir d6/d45/d51/d5d 0 2024-04-25T01:35:44.959 INFO:tasks.workunit.client.0.smithi033.stdout:2/211: mknod da/d27/c3f 0 2024-04-25T01:35:44.959 INFO:tasks.workunit.client.0.smithi033.stdout:8/264: creat d6/d45/d52/d2e/f5e x:0 0 0 2024-04-25T01:35:44.960 INFO:tasks.workunit.client.0.smithi033.stdout:2/212: mkdir da/d27/d31/d40 0 2024-04-25T01:35:44.962 INFO:tasks.workunit.client.0.smithi033.stdout:2/213: truncate da/d32/f35 969866 0 2024-04-25T01:35:44.964 INFO:tasks.workunit.client.0.smithi033.stdout:2/214: mkdir da/d32/d41 0 2024-04-25T01:35:44.965 INFO:tasks.workunit.client.0.smithi033.stdout:2/215: chown da/d23/l2e 3331 1 2024-04-25T01:35:44.966 INFO:tasks.workunit.client.0.smithi033.stdout:8/265: dwrite d6/d45/d52/f28 [4194304,4194304] 0 2024-04-25T01:35:44.966 INFO:tasks.workunit.client.0.smithi033.stdout:2/216: dread - da/d27/d31/f39 zero size 2024-04-25T01:35:44.972 INFO:tasks.workunit.client.0.smithi033.stdout:2/217: unlink da/l1f 0 2024-04-25T01:35:44.988 INFO:tasks.workunit.client.0.smithi033.stdout:2/218: getdents da/d32 0 2024-04-25T01:35:45.000 INFO:tasks.workunit.client.0.smithi033.stdout:2/219: rmdir da/d32/d41 0 2024-04-25T01:35:45.010 INFO:tasks.workunit.client.0.smithi033.stdout:2/220: creat da/f42 x:0 0 0 2024-04-25T01:35:45.026 INFO:tasks.workunit.client.0.smithi033.stdout:2/221: dwrite da/d32/f35 [0,4194304] 0 2024-04-25T01:35:45.028 INFO:tasks.workunit.client.0.smithi033.stdout:2/222: write da/fe [794319,40159] 0 2024-04-25T01:35:45.028 INFO:tasks.workunit.client.0.smithi033.stdout:2/223: mkdir da/d23/d43 0 2024-04-25T01:35:45.031 INFO:tasks.workunit.client.0.smithi033.stdout:2/224: dwrite da/d2c/d30/f3b [0,4194304] 0 2024-04-25T01:35:45.038 INFO:tasks.workunit.client.0.smithi033.stdout:2/225: dread f7 [0,4194304] 0 2024-04-25T01:35:45.043 INFO:tasks.workunit.client.0.smithi033.stdout:2/226: dwrite da/f34 [0,4194304] 0 2024-04-25T01:35:45.050 INFO:tasks.workunit.client.0.smithi033.stdout:2/227: dwrite da/d27/f2d [0,4194304] 0 2024-04-25T01:35:45.053 INFO:tasks.workunit.client.0.smithi033.stdout:2/228: creat da/d27/f44 x:0 0 0 2024-04-25T01:35:45.057 INFO:tasks.workunit.client.0.smithi033.stdout:2/229: creat da/f45 x:0 0 0 2024-04-25T01:35:45.059 INFO:tasks.workunit.client.0.smithi033.stdout:2/230: creat da/d27/d31/f46 x:0 0 0 2024-04-25T01:35:45.065 INFO:tasks.workunit.client.0.smithi033.stdout:2/231: dwrite da/f42 [0,4194304] 0 2024-04-25T01:35:45.065 INFO:tasks.workunit.client.0.smithi033.stdout:2/232: chown da/fe 131214084 1 2024-04-25T01:35:45.067 INFO:tasks.workunit.client.0.smithi033.stdout:2/233: creat da/d27/f47 x:0 0 0 2024-04-25T01:35:45.070 INFO:tasks.workunit.client.0.smithi033.stdout:2/234: creat da/d2c/f48 x:0 0 0 2024-04-25T01:35:45.072 INFO:tasks.workunit.client.0.smithi033.stdout:2/235: creat da/d27/f49 x:0 0 0 2024-04-25T01:35:45.073 INFO:tasks.workunit.client.0.smithi033.stdout:2/236: write da/d27/f2d [2091722,63458] 0 2024-04-25T01:35:45.076 INFO:tasks.workunit.client.0.smithi033.stdout:2/237: mkdir da/d4a 0 2024-04-25T01:35:45.076 INFO:tasks.workunit.client.0.smithi033.stdout:2/238: chown da/f26 131 1 2024-04-25T01:35:45.077 INFO:tasks.workunit.client.0.smithi033.stdout:2/239: write da/f17 [1411797,57328] 0 2024-04-25T01:35:45.084 INFO:tasks.workunit.client.0.smithi033.stdout:2/240: dwrite da/d2c/f33 [0,4194304] 0 2024-04-25T01:35:45.086 INFO:tasks.workunit.client.0.smithi033.stdout:2/241: creat da/d23/f4b x:0 0 0 2024-04-25T01:35:45.089 INFO:tasks.workunit.client.0.smithi033.stdout:2/242: symlink da/l4c 0 2024-04-25T01:35:45.091 INFO:tasks.workunit.client.0.smithi033.stdout:2/243: symlink da/d4a/l4d 0 2024-04-25T01:35:45.092 INFO:tasks.workunit.client.0.smithi033.stdout:2/244: write da/d27/f2d [2809214,114839] 0 2024-04-25T01:35:45.092 INFO:tasks.workunit.client.0.smithi033.stdout:2/245: readlink da/l1d 0 2024-04-25T01:35:45.098 INFO:tasks.workunit.client.0.smithi033.stdout:2/246: dwrite da/d2c/f36 [0,4194304] 0 2024-04-25T01:35:45.102 INFO:tasks.workunit.client.0.smithi033.stdout:2/247: rename da/d23/l2b to da/d2c/l4e 0 2024-04-25T01:35:45.102 INFO:tasks.workunit.client.0.smithi033.stdout:2/248: readlink da/l1d 0 2024-04-25T01:35:45.108 INFO:tasks.workunit.client.0.smithi033.stdout:2/249: dwrite da/d23/f4b [0,4194304] 0 2024-04-25T01:35:45.115 INFO:tasks.workunit.client.0.smithi033.stdout:2/250: getdents da/d23 0 2024-04-25T01:35:45.117 INFO:tasks.workunit.client.0.smithi033.stdout:2/251: creat da/d27/d31/f4f x:0 0 0 2024-04-25T01:35:45.127 INFO:tasks.workunit.client.0.smithi033.stdout:2/252: truncate da/d23/f4b 1053404 0 2024-04-25T01:35:45.129 INFO:tasks.workunit.client.0.smithi033.stdout:2/253: mkdir da/d50 0 2024-04-25T01:35:45.131 INFO:tasks.workunit.client.0.smithi033.stdout:2/254: dread f8 [0,4194304] 0 2024-04-25T01:35:45.132 INFO:tasks.workunit.client.0.smithi033.stdout:2/255: mknod da/d27/c51 0 2024-04-25T01:35:45.134 INFO:tasks.workunit.client.0.smithi033.stdout:2/256: symlink da/d27/d31/d40/l52 0 2024-04-25T01:35:45.135 INFO:tasks.workunit.client.0.smithi033.stdout:2/257: creat da/d2c/d30/f53 x:0 0 0 2024-04-25T01:35:45.135 INFO:tasks.workunit.client.0.smithi033.stdout:2/258: readlink da/d23/l2e 0 2024-04-25T01:35:45.142 INFO:tasks.workunit.client.0.smithi033.stdout:2/259: dwrite da/f26 [0,4194304] 0 2024-04-25T01:35:45.146 INFO:tasks.workunit.client.0.smithi033.stdout:2/260: unlink da/d27/f44 0 2024-04-25T01:35:45.151 INFO:tasks.workunit.client.0.smithi033.stdout:0/198: sync 2024-04-25T01:35:45.153 INFO:tasks.workunit.client.0.smithi033.stdout:2/261: getdents da/d27 0 2024-04-25T01:35:45.154 INFO:tasks.workunit.client.0.smithi033.stdout:0/199: symlink d7/da/d23/l40 0 2024-04-25T01:35:45.156 INFO:tasks.workunit.client.0.smithi033.stdout:0/200: mkdir d7/da/d23/d41 0 2024-04-25T01:35:45.158 INFO:tasks.workunit.client.0.smithi033.stdout:2/262: getdents da/d2c 0 2024-04-25T01:35:45.159 INFO:tasks.workunit.client.0.smithi033.stdout:0/201: creat d7/da/d23/f42 x:0 0 0 2024-04-25T01:35:45.160 INFO:tasks.workunit.client.0.smithi033.stdout:2/263: write da/d27/d31/f46 [273496,121769] 0 2024-04-25T01:35:45.162 INFO:tasks.workunit.client.0.smithi033.stdout:0/202: fsync d7/d1b/f32 0 2024-04-25T01:35:45.164 INFO:tasks.workunit.client.0.smithi033.stdout:0/203: mkdir d7/d2d/d43 0 2024-04-25T01:35:45.167 INFO:tasks.workunit.client.0.smithi033.stdout:0/204: getdents d7/da/df/d2a 0 2024-04-25T01:35:45.171 INFO:tasks.workunit.client.0.smithi033.stdout:0/205: write d7/da/df/f28 [3290404,38378] 0 2024-04-25T01:35:45.172 INFO:tasks.workunit.client.0.smithi033.stdout:0/206: write d7/d2d/f3d [769043,39265] 0 2024-04-25T01:35:45.175 INFO:tasks.workunit.client.0.smithi033.stdout:0/207: mkdir d7/d2d/d43/d44 0 2024-04-25T01:35:45.229 INFO:tasks.workunit.client.0.smithi033.stdout:2/264: dread da/f17 [0,4194304] 0 2024-04-25T01:35:45.229 INFO:tasks.workunit.client.0.smithi033.stdout:0/208: chown d7/da/df/d14/f22 23524 1 2024-04-25T01:35:45.230 INFO:tasks.workunit.client.0.smithi033.stdout:0/209: write d7/da/df/f19 [1891930,30325] 0 2024-04-25T01:35:45.231 INFO:tasks.workunit.client.0.smithi033.stdout:0/210: read d7/da/df/f19 [1313798,45245] 0 2024-04-25T01:35:45.235 INFO:tasks.workunit.client.0.smithi033.stdout:0/211: mkdir d7/da/df/d14/d45 0 2024-04-25T01:35:45.236 INFO:tasks.workunit.client.0.smithi033.stdout:0/212: creat d7/f46 x:0 0 0 2024-04-25T01:35:45.240 INFO:tasks.workunit.client.0.smithi033.stdout:2/265: dwrite da/d27/d31/f3d [0,4194304] 0 2024-04-25T01:35:45.245 INFO:tasks.workunit.client.0.smithi033.stdout:2/266: dread da/d27/d31/f46 [0,4194304] 0 2024-04-25T01:35:45.245 INFO:tasks.workunit.client.0.smithi033.stdout:0/213: creat d7/da/df/d14/f47 x:0 0 0 2024-04-25T01:35:45.245 INFO:tasks.workunit.client.0.smithi033.stdout:2/267: readlink da/l1d 0 2024-04-25T01:35:45.246 INFO:tasks.workunit.client.0.smithi033.stdout:0/214: read - d7/da/df/d2a/f35 zero size 2024-04-25T01:35:45.246 INFO:tasks.workunit.client.0.smithi033.stdout:0/215: chown d7/da/df/d14 1678501 1 2024-04-25T01:35:45.246 INFO:tasks.workunit.client.0.smithi033.stdout:2/268: write da/d27/d31/f46 [1324201,1120] 0 2024-04-25T01:35:45.250 INFO:tasks.workunit.client.0.smithi033.stdout:0/216: creat d7/da/f48 x:0 0 0 2024-04-25T01:35:45.252 INFO:tasks.workunit.client.0.smithi033.stdout:2/269: dread da/f15 [0,4194304] 0 2024-04-25T01:35:45.252 INFO:tasks.workunit.client.0.smithi033.stdout:2/270: chown da/d4a 32303834 1 2024-04-25T01:35:45.258 INFO:tasks.workunit.client.0.smithi033.stdout:0/217: dwrite d7/da/df/d14/f24 [0,4194304] 0 2024-04-25T01:35:45.260 INFO:tasks.workunit.client.0.smithi033.stdout:2/271: dwrite da/f22 [0,4194304] 0 2024-04-25T01:35:45.261 INFO:tasks.workunit.client.0.smithi033.stdout:0/218: truncate d7/da/d23/f3a 524893 0 2024-04-25T01:35:45.261 INFO:tasks.workunit.client.0.smithi033.stdout:2/272: readlink da/d23/l2e 0 2024-04-25T01:35:45.266 INFO:tasks.workunit.client.0.smithi033.stdout:2/273: creat da/d27/d31/d40/f54 x:0 0 0 2024-04-25T01:35:45.270 INFO:tasks.workunit.client.0.smithi033.stdout:2/274: mknod da/d27/d31/d40/c55 0 2024-04-25T01:35:45.273 INFO:tasks.workunit.client.0.smithi033.stdout:2/275: creat da/d23/f56 x:0 0 0 2024-04-25T01:35:45.273 INFO:tasks.workunit.client.0.smithi033.stdout:2/276: readlink da/d23/l24 0 2024-04-25T01:35:45.274 INFO:tasks.workunit.client.0.smithi033.stdout:2/277: write da/d27/f47 [286008,79726] 0 2024-04-25T01:35:45.277 INFO:tasks.workunit.client.0.smithi033.stdout:2/278: creat da/d23/f57 x:0 0 0 2024-04-25T01:35:45.279 INFO:tasks.workunit.client.0.smithi033.stdout:2/279: creat da/d50/f58 x:0 0 0 2024-04-25T01:35:45.280 INFO:tasks.workunit.client.0.smithi033.stdout:2/280: link da/d23/f56 da/d27/f59 0 2024-04-25T01:35:45.281 INFO:tasks.workunit.client.0.smithi033.stdout:2/281: dread f8 [0,4194304] 0 2024-04-25T01:35:45.288 INFO:tasks.workunit.client.0.smithi033.stdout:2/282: dwrite da/d23/f4b [0,4194304] 0 2024-04-25T01:35:45.291 INFO:tasks.workunit.client.0.smithi033.stdout:2/283: creat da/d27/d31/f5a x:0 0 0 2024-04-25T01:35:45.294 INFO:tasks.workunit.client.0.smithi033.stdout:2/284: rmdir da/d2c/d30 39 2024-04-25T01:35:45.294 INFO:tasks.workunit.client.0.smithi033.stdout:2/285: dread - da/d27/f49 zero size 2024-04-25T01:35:45.296 INFO:tasks.workunit.client.0.smithi033.stdout:2/286: write da/d23/f4b [4917509,110725] 0 2024-04-25T01:35:45.296 INFO:tasks.workunit.client.0.smithi033.stdout:9/206: sync 2024-04-25T01:35:45.299 INFO:tasks.workunit.client.0.smithi033.stdout:2/287: unlink da/d27/c51 0 2024-04-25T01:35:45.301 INFO:tasks.workunit.client.0.smithi033.stdout:9/207: creat d0/d2/dd/df/d1b/f4d x:0 0 0 2024-04-25T01:35:45.301 INFO:tasks.workunit.client.0.smithi033.stdout:2/288: unlink da/d27/f49 0 2024-04-25T01:35:45.304 INFO:tasks.workunit.client.0.smithi033.stdout:9/208: mkdir d0/d2/dd/d20/d4e 0 2024-04-25T01:35:45.307 INFO:tasks.workunit.client.0.smithi033.stdout:2/289: mkdir da/d5b 0 2024-04-25T01:35:45.310 INFO:tasks.workunit.client.0.smithi033.stdout:9/209: creat d0/d2/dd/d20/d4e/f4f x:0 0 0 2024-04-25T01:35:45.311 INFO:tasks.workunit.client.0.smithi033.stdout:9/210: stat d0/d2/dd/d45 0 2024-04-25T01:35:45.323 INFO:tasks.workunit.client.0.smithi033.stdout:2/290: dwrite da/f17 [4194304,4194304] 0 2024-04-25T01:35:45.323 INFO:tasks.workunit.client.0.smithi033.stdout:9/211: dwrite d0/d11/f47 [0,4194304] 0 2024-04-25T01:35:45.323 INFO:tasks.workunit.client.0.smithi033.stdout:2/291: write da/fd [3464762,56742] 0 2024-04-25T01:35:45.327 INFO:tasks.workunit.client.0.smithi033.stdout:9/212: rmdir d0/d11 39 2024-04-25T01:35:45.330 INFO:tasks.workunit.client.0.smithi033.stdout:2/292: creat da/d27/f5c x:0 0 0 2024-04-25T01:35:45.332 INFO:tasks.workunit.client.0.smithi033.stdout:9/213: fsync d0/d11/f4c 0 2024-04-25T01:35:45.336 INFO:tasks.workunit.client.0.smithi033.stdout:9/214: unlink d0/l46 0 2024-04-25T01:35:45.338 INFO:tasks.workunit.client.0.smithi033.stdout:9/215: dread - d0/d2/dd/df/f3e zero size 2024-04-25T01:35:45.339 INFO:tasks.workunit.client.0.smithi033.stdout:2/293: mkdir da/d5d 0 2024-04-25T01:35:45.339 INFO:tasks.workunit.client.0.smithi033.stdout:2/294: chown da/d2c/l4e 3 1 2024-04-25T01:35:45.342 INFO:tasks.workunit.client.0.smithi033.stdout:2/295: truncate da/d27/d31/f4f 615440 0 2024-04-25T01:35:45.347 INFO:tasks.workunit.client.0.smithi033.stdout:9/216: dwrite d0/d2/dd/df/f3a [0,4194304] 0 2024-04-25T01:35:45.349 INFO:tasks.workunit.client.0.smithi033.stdout:9/217: chown d0/d2/dd/df/f3b 1 1 2024-04-25T01:35:45.351 INFO:tasks.workunit.client.0.smithi033.stdout:2/296: write da/d23/f56 [343695,13057] 0 2024-04-25T01:35:45.352 INFO:tasks.workunit.client.0.smithi033.stdout:9/218: creat d0/d2/dd/df/d1b/f50 x:0 0 0 2024-04-25T01:35:45.353 INFO:tasks.workunit.client.0.smithi033.stdout:2/297: write da/f22 [3903693,116474] 0 2024-04-25T01:35:45.356 INFO:tasks.workunit.client.0.smithi033.stdout:2/298: mkdir da/d5b/d5e 0 2024-04-25T01:35:45.357 INFO:tasks.workunit.client.0.smithi033.stdout:2/299: read - da/d27/d31/d40/f54 zero size 2024-04-25T01:35:45.361 INFO:tasks.workunit.client.0.smithi033.stdout:2/300: unlink da/f26 0 2024-04-25T01:35:45.365 INFO:tasks.workunit.client.0.smithi033.stdout:2/301: rmdir da/d5d 0 2024-04-25T01:35:45.373 INFO:tasks.workunit.client.0.smithi033.stdout:2/302: truncate da/d23/f4b 1007696 0 2024-04-25T01:35:45.380 INFO:tasks.workunit.client.0.smithi033.stdout:2/303: truncate da/d2c/f36 332819 0 2024-04-25T01:35:45.394 INFO:tasks.workunit.client.0.smithi033.stdout:2/304: creat da/d27/d31/d40/f5f x:0 0 0 2024-04-25T01:35:45.394 INFO:tasks.workunit.client.0.smithi033.stdout:2/305: stat da/d5b/d5e 0 2024-04-25T01:35:45.397 INFO:tasks.workunit.client.0.smithi033.stdout:2/306: dread da/f42 [0,4194304] 0 2024-04-25T01:35:45.402 INFO:tasks.workunit.client.0.smithi033.stdout:2/307: rename da/d32/l3e to da/d50/l60 0 2024-04-25T01:35:45.406 INFO:tasks.workunit.client.0.smithi033.stdout:2/308: link f7 da/d27/d31/f61 0 2024-04-25T01:35:45.411 INFO:tasks.workunit.client.0.smithi033.stdout:2/309: rmdir da/d32 39 2024-04-25T01:35:45.414 INFO:tasks.workunit.client.0.smithi033.stdout:2/310: creat da/d23/d43/f62 x:0 0 0 2024-04-25T01:35:45.415 INFO:tasks.workunit.client.0.smithi033.stdout:2/311: fdatasync da/f42 0 2024-04-25T01:35:45.416 INFO:tasks.workunit.client.0.smithi033.stdout:2/312: write da/d27/d31/d40/f5f [126486,88862] 0 2024-04-25T01:35:45.423 INFO:tasks.workunit.client.0.smithi033.stdout:2/313: dwrite da/d23/f57 [0,4194304] 0 2024-04-25T01:35:45.427 INFO:tasks.workunit.client.0.smithi033.stdout:2/314: fsync f7 0 2024-04-25T01:35:45.428 INFO:tasks.workunit.client.0.smithi033.stdout:2/315: write da/ff [2414220,43734] 0 2024-04-25T01:35:45.436 INFO:tasks.workunit.client.0.smithi033.stdout:2/316: truncate da/fd 1700055 0 2024-04-25T01:35:45.444 INFO:tasks.workunit.client.0.smithi033.stdout:2/317: rename da/d2c/d30/f3a to da/d5b/f63 0 2024-04-25T01:35:45.444 INFO:tasks.workunit.client.0.smithi033.stdout:2/318: dread da/d27/f59 [0,4194304] 0 2024-04-25T01:35:45.444 INFO:tasks.workunit.client.0.smithi033.stdout:2/319: unlink da/d27/d31/f39 0 2024-04-25T01:35:45.446 INFO:tasks.workunit.client.0.smithi033.stdout:2/320: creat da/d27/d31/d40/f64 x:0 0 0 2024-04-25T01:35:45.453 INFO:tasks.workunit.client.0.smithi033.stdout:2/321: dwrite da/d2c/f48 [0,4194304] 0 2024-04-25T01:35:45.464 INFO:tasks.workunit.client.0.smithi033.stdout:2/322: dwrite da/d27/f59 [0,4194304] 0 2024-04-25T01:35:45.469 INFO:tasks.workunit.client.0.smithi033.stdout:2/323: unlink da/l14 0 2024-04-25T01:35:45.471 INFO:tasks.workunit.client.0.smithi033.stdout:2/324: truncate da/d27/d31/d40/f64 687551 0 2024-04-25T01:35:45.473 INFO:tasks.workunit.client.0.smithi033.stdout:2/325: mknod da/d27/d31/c65 0 2024-04-25T01:35:45.475 INFO:tasks.workunit.client.0.smithi033.stdout:2/326: creat da/d23/d43/f66 x:0 0 0 2024-04-25T01:35:45.482 INFO:tasks.workunit.client.0.smithi033.stdout:2/327: dwrite da/d27/d31/f5a [0,4194304] 0 2024-04-25T01:35:45.483 INFO:tasks.workunit.client.0.smithi033.stdout:2/328: fdatasync da/f2f 0 2024-04-25T01:35:45.487 INFO:tasks.workunit.client.0.smithi033.stdout:2/329: mknod da/d27/d31/c67 0 2024-04-25T01:35:45.493 INFO:tasks.workunit.client.0.smithi033.stdout:2/330: read da/ff [2880311,107293] 0 2024-04-25T01:35:45.506 INFO:tasks.workunit.client.0.smithi033.stdout:2/331: dwrite f7 [0,4194304] 0 2024-04-25T01:35:45.511 INFO:tasks.workunit.client.0.smithi033.stdout:2/332: creat da/d32/f68 x:0 0 0 2024-04-25T01:35:45.513 INFO:tasks.workunit.client.0.smithi033.stdout:2/333: rmdir da/d27/d31/d40 39 2024-04-25T01:35:45.521 INFO:tasks.workunit.client.0.smithi033.stdout:7/234: sync 2024-04-25T01:35:45.521 INFO:tasks.workunit.client.0.smithi033.stdout:4/238: sync 2024-04-25T01:35:45.521 INFO:tasks.workunit.client.0.smithi033.stdout:5/266: sync 2024-04-25T01:35:45.526 INFO:tasks.workunit.client.0.smithi033.stdout:7/235: rename d4/f3a to d4/d2e/f48 0 2024-04-25T01:35:45.527 INFO:tasks.workunit.client.0.smithi033.stdout:2/334: fdatasync da/d27/d31/f61 0 2024-04-25T01:35:45.528 INFO:tasks.workunit.client.0.smithi033.stdout:7/236: write d4/f3d [265297,24438] 0 2024-04-25T01:35:45.531 INFO:tasks.workunit.client.0.smithi033.stdout:5/267: dwrite d1c/d2a/d50/d52/f54 [0,4194304] 0 2024-04-25T01:35:45.532 INFO:tasks.workunit.client.0.smithi033.stdout:7/237: chown d4/fb 33205 1 2024-04-25T01:35:45.535 INFO:tasks.workunit.client.0.smithi033.stdout:5/268: fsync d1c/d2a/f47 0 2024-04-25T01:35:45.537 INFO:tasks.workunit.client.0.smithi033.stdout:7/238: write d4/d10/d13/f47 [506763,32610] 0 2024-04-25T01:35:45.539 INFO:tasks.workunit.client.0.smithi033.stdout:4/239: dwrite d6/d14/d1d/d3b/f3e [0,4194304] 0 2024-04-25T01:35:45.542 INFO:tasks.workunit.client.0.smithi033.stdout:4/240: chown d6/d12/l21 16242 1 2024-04-25T01:35:45.544 INFO:tasks.workunit.client.0.smithi033.stdout:2/335: dwrite da/f18 [4194304,4194304] 0 2024-04-25T01:35:45.545 INFO:tasks.workunit.client.0.smithi033.stdout:4/241: chown d6/d14/d1d/f2e 412665 1 2024-04-25T01:35:45.546 INFO:tasks.workunit.client.0.smithi033.stdout:2/336: readlink da/d50/l60 0 2024-04-25T01:35:45.548 INFO:tasks.workunit.client.0.smithi033.stdout:4/242: chown d6/d14/d1d/c2d 273 1 2024-04-25T01:35:45.551 INFO:tasks.workunit.client.0.smithi033.stdout:2/337: truncate da/d27/d31/f46 2223468 0 2024-04-25T01:35:45.552 INFO:tasks.workunit.client.0.smithi033.stdout:5/269: dwrite ff [0,4194304] 0 2024-04-25T01:35:45.553 INFO:tasks.workunit.client.0.smithi033.stdout:4/243: mkdir d6/d14/d35/d44 0 2024-04-25T01:35:45.556 INFO:tasks.workunit.client.0.smithi033.stdout:2/338: creat da/d27/f69 x:0 0 0 2024-04-25T01:35:45.557 INFO:tasks.workunit.client.0.smithi033.stdout:7/239: dread d4/f9 [0,4194304] 0 2024-04-25T01:35:45.558 INFO:tasks.workunit.client.0.smithi033.stdout:7/240: chown d4/lc 17899 1 2024-04-25T01:35:45.558 INFO:tasks.workunit.client.0.smithi033.stdout:5/270: dread f16 [0,4194304] 0 2024-04-25T01:35:45.562 INFO:tasks.workunit.client.0.smithi033.stdout:7/241: write d4/d2e/f48 [762728,83638] 0 2024-04-25T01:35:45.562 INFO:tasks.workunit.client.0.smithi033.stdout:2/339: chown da/d32/f35 166 1 2024-04-25T01:35:45.567 INFO:tasks.workunit.client.0.smithi033.stdout:5/271: dread d1c/f2e [0,4194304] 0 2024-04-25T01:35:45.567 INFO:tasks.workunit.client.0.smithi033.stdout:7/242: getdents d4/d10/d13 0 2024-04-25T01:35:45.568 INFO:tasks.workunit.client.0.smithi033.stdout:7/243: dread - d4/f39 zero size 2024-04-25T01:35:45.570 INFO:tasks.workunit.client.0.smithi033.stdout:7/244: chown f3 62 1 2024-04-25T01:35:45.570 INFO:tasks.workunit.client.0.smithi033.stdout:6/244: sync 2024-04-25T01:35:45.571 INFO:tasks.workunit.client.0.smithi033.stdout:0/219: sync 2024-04-25T01:35:45.571 INFO:tasks.workunit.client.0.smithi033.stdout:1/184: sync 2024-04-25T01:35:45.571 INFO:tasks.workunit.client.0.smithi033.stdout:8/266: sync 2024-04-25T01:35:45.571 INFO:tasks.workunit.client.0.smithi033.stdout:3/213: sync 2024-04-25T01:35:45.571 INFO:tasks.workunit.client.0.smithi033.stdout:9/219: sync 2024-04-25T01:35:45.573 INFO:tasks.workunit.client.0.smithi033.stdout:8/267: rename d6 to d6/d49/d5f 22 2024-04-25T01:35:45.574 INFO:tasks.workunit.client.0.smithi033.stdout:7/245: mknod d4/d2e/c49 0 2024-04-25T01:35:45.577 INFO:tasks.workunit.client.0.smithi033.stdout:1/185: mkdir d2/d2e/d34 0 2024-04-25T01:35:45.578 INFO:tasks.workunit.client.0.smithi033.stdout:8/268: chown d6/db/f18 1420833292 1 2024-04-25T01:35:45.580 INFO:tasks.workunit.client.0.smithi033.stdout:4/244: dread d6/d14/f22 [0,4194304] 0 2024-04-25T01:35:45.580 INFO:tasks.workunit.client.0.smithi033.stdout:0/220: unlink d7/da/df/f10 0 2024-04-25T01:35:45.581 INFO:tasks.workunit.client.0.smithi033.stdout:7/246: mknod d4/d23/d3f/c4a 0 2024-04-25T01:35:45.586 INFO:tasks.workunit.client.0.smithi033.stdout:1/186: fdatasync d2/d11/f1e 0 2024-04-25T01:35:45.590 INFO:tasks.workunit.client.0.smithi033.stdout:8/269: chown d6/d45/f50 40328870 1 2024-04-25T01:35:45.591 INFO:tasks.workunit.client.0.smithi033.stdout:4/245: write d6/d12/d38/f41 [1080595,103146] 0 2024-04-25T01:35:45.594 INFO:tasks.workunit.client.0.smithi033.stdout:9/220: dwrite d0/d2/dd/df/d1b/f50 [0,4194304] 0 2024-04-25T01:35:45.595 INFO:tasks.workunit.client.0.smithi033.stdout:8/270: chown d6/c2a 27 1 2024-04-25T01:35:45.596 INFO:tasks.workunit.client.0.smithi033.stdout:3/214: dwrite d8/dd/f2f [0,4194304] 0 2024-04-25T01:35:45.597 INFO:tasks.workunit.client.0.smithi033.stdout:7/247: symlink d4/d10/l4b 0 2024-04-25T01:35:45.599 INFO:tasks.workunit.client.0.smithi033.stdout:1/187: symlink d2/d5/l35 0 2024-04-25T01:35:45.602 INFO:tasks.workunit.client.0.smithi033.stdout:6/245: dwrite d4/f11 [0,4194304] 0 2024-04-25T01:35:45.604 INFO:tasks.workunit.client.0.smithi033.stdout:5/272: dwrite f17 [0,4194304] 0 2024-04-25T01:35:45.604 INFO:tasks.workunit.client.0.smithi033.stdout:4/246: mkdir d6/d14/d35/d44/d45 0 2024-04-25T01:35:45.606 INFO:tasks.workunit.client.0.smithi033.stdout:3/215: readlink d8/dd/d11/d1e/d27/d3c/l3f 0 2024-04-25T01:35:45.609 INFO:tasks.workunit.client.0.smithi033.stdout:8/271: creat d6/f60 x:0 0 0 2024-04-25T01:35:45.609 INFO:tasks.workunit.client.0.smithi033.stdout:0/221: getdents d7/da/d23 0 2024-04-25T01:35:45.609 INFO:tasks.workunit.client.0.smithi033.stdout:9/221: creat d0/d2/dd/f51 x:0 0 0 2024-04-25T01:35:45.612 INFO:tasks.workunit.client.0.smithi033.stdout:5/273: mkdir d1c/d2a/d3d/d59 0 2024-04-25T01:35:45.615 INFO:tasks.workunit.client.0.smithi033.stdout:1/188: rename d2/d5/fe to d2/d11/d28/d33/f36 0 2024-04-25T01:35:45.616 INFO:tasks.workunit.client.0.smithi033.stdout:0/222: stat d7/da/df/d14/l1e 0 2024-04-25T01:35:45.619 INFO:tasks.workunit.client.0.smithi033.stdout:7/248: dwrite d4/d10/d13/d1f/f24 [0,4194304] 0 2024-04-25T01:35:45.619 INFO:tasks.workunit.client.0.smithi033.stdout:8/272: creat d6/db/f61 x:0 0 0 2024-04-25T01:35:45.621 INFO:tasks.workunit.client.0.smithi033.stdout:3/216: rmdir d8/dd/d11/d1e/d27/d3c 39 2024-04-25T01:35:45.624 INFO:tasks.workunit.client.0.smithi033.stdout:1/189: symlink d2/d11/d28/l37 0 2024-04-25T01:35:45.625 INFO:tasks.workunit.client.0.smithi033.stdout:1/190: chown d2 30524 1 2024-04-25T01:35:45.627 INFO:tasks.workunit.client.0.smithi033.stdout:3/217: fsync d8/dd/d11/d1e/f3b 0 2024-04-25T01:35:45.629 INFO:tasks.workunit.client.0.smithi033.stdout:5/274: truncate d1c/d2a/d3d/f4d 4339954 0 2024-04-25T01:35:45.630 INFO:tasks.workunit.client.0.smithi033.stdout:6/246: dwrite d4/d1d/f50 [0,4194304] 0 2024-04-25T01:35:45.631 INFO:tasks.workunit.client.0.smithi033.stdout:3/218: chown d8/dd/d11/d1e 52589 1 2024-04-25T01:35:45.638 INFO:tasks.workunit.client.0.smithi033.stdout:9/222: creat d0/d2/dd/f52 x:0 0 0 2024-04-25T01:35:45.638 INFO:tasks.workunit.client.0.smithi033.stdout:5/275: read f5 [1582114,104931] 0 2024-04-25T01:35:45.639 INFO:tasks.workunit.client.0.smithi033.stdout:8/273: creat d6/d45/d51/d5d/f62 x:0 0 0 2024-04-25T01:35:45.640 INFO:tasks.workunit.client.0.smithi033.stdout:5/276: write d1c/d2a/f47 [56037,19661] 0 2024-04-25T01:35:45.643 INFO:tasks.workunit.client.0.smithi033.stdout:0/223: dwrite d7/da/d23/f3a [0,4194304] 0 2024-04-25T01:35:45.644 INFO:tasks.workunit.client.0.smithi033.stdout:0/224: fsync d7/da/f48 0 2024-04-25T01:35:45.644 INFO:tasks.workunit.client.0.smithi033.stdout:8/274: readlink d6/d45/d52/d2e/l42 0 2024-04-25T01:35:45.647 INFO:tasks.workunit.client.0.smithi033.stdout:5/277: write d1c/d21/d4a/f4c [576583,91342] 0 2024-04-25T01:35:45.647 INFO:tasks.workunit.client.0.smithi033.stdout:6/247: mkdir d4/d1d/d3e/d41/d5c 0 2024-04-25T01:35:45.650 INFO:tasks.workunit.client.0.smithi033.stdout:4/247: dread d6/f9 [0,4194304] 0 2024-04-25T01:35:45.654 INFO:tasks.workunit.client.0.smithi033.stdout:9/223: mknod d0/d2/dd/df/d1b/d3d/d42/c53 0 2024-04-25T01:35:45.656 INFO:tasks.workunit.client.0.smithi033.stdout:8/275: unlink d6/d45/f50 0 2024-04-25T01:35:45.657 INFO:tasks.workunit.client.0.smithi033.stdout:1/191: dread d2/d5/f18 [0,4194304] 0 2024-04-25T01:35:45.659 INFO:tasks.workunit.client.0.smithi033.stdout:7/249: link d4/d2e/ff d4/d2e/f4c 0 2024-04-25T01:35:45.662 INFO:tasks.workunit.client.0.smithi033.stdout:4/248: dread d6/f16 [0,4194304] 0 2024-04-25T01:35:45.663 INFO:tasks.workunit.client.0.smithi033.stdout:7/250: stat d4/d10/d13/d1f/f42 0 2024-04-25T01:35:45.664 INFO:tasks.workunit.client.0.smithi033.stdout:9/224: truncate d0/d11/f18 3076527 0 2024-04-25T01:35:45.666 INFO:tasks.workunit.client.0.smithi033.stdout:6/248: creat d4/d1d/d3e/d41/d5c/f5d x:0 0 0 2024-04-25T01:35:45.667 INFO:tasks.workunit.client.0.smithi033.stdout:7/251: dread d4/f3d [0,4194304] 0 2024-04-25T01:35:45.672 INFO:tasks.workunit.client.0.smithi033.stdout:8/276: link d6/f8 d6/d45/d51/d5d/f63 0 2024-04-25T01:35:45.674 INFO:tasks.workunit.client.0.smithi033.stdout:3/219: rename d8/dd/d11/d1e/d27/d3c/d47 to d8/dd/d11/d1e/d27/d4d 0 2024-04-25T01:35:45.675 INFO:tasks.workunit.client.0.smithi033.stdout:7/252: fsync d4/f41 0 2024-04-25T01:35:45.676 INFO:tasks.workunit.client.0.smithi033.stdout:4/249: mkdir d6/d14/d35/d44/d45/d46 0 2024-04-25T01:35:45.678 INFO:tasks.workunit.client.0.smithi033.stdout:0/225: rename d7/da/d23/f27 to d7/d2d/d43/d44/f49 0 2024-04-25T01:35:45.679 INFO:tasks.workunit.client.0.smithi033.stdout:3/220: creat d8/dd/d40/f4e x:0 0 0 2024-04-25T01:35:45.680 INFO:tasks.workunit.client.0.smithi033.stdout:0/226: truncate d7/da/f48 383881 0 2024-04-25T01:35:45.680 INFO:tasks.workunit.client.0.smithi033.stdout:7/253: creat d4/d23/d3f/f4d x:0 0 0 2024-04-25T01:35:45.680 INFO:tasks.workunit.client.0.smithi033.stdout:3/221: write f3 [2250789,12348] 0 2024-04-25T01:35:45.681 INFO:tasks.workunit.client.0.smithi033.stdout:9/225: creat d0/d2/dd/f54 x:0 0 0 2024-04-25T01:35:45.682 INFO:tasks.workunit.client.0.smithi033.stdout:3/222: fdatasync d8/f1b 0 2024-04-25T01:35:45.683 INFO:tasks.workunit.client.0.smithi033.stdout:1/192: rename d2/d2e/c2f to d2/d11/d28/c38 0 2024-04-25T01:35:45.685 INFO:tasks.workunit.client.0.smithi033.stdout:4/250: mknod d6/d14/d35/d39/c47 0 2024-04-25T01:35:45.686 INFO:tasks.workunit.client.0.smithi033.stdout:4/251: chown d6/d12/c25 3 1 2024-04-25T01:35:45.686 INFO:tasks.workunit.client.0.smithi033.stdout:7/254: mknod d4/d2e/c4e 0 2024-04-25T01:35:45.687 INFO:tasks.workunit.client.0.smithi033.stdout:4/252: fsync d6/f18 0 2024-04-25T01:35:45.690 INFO:tasks.workunit.client.0.smithi033.stdout:7/255: write d4/f39 [161719,82020] 0 2024-04-25T01:35:45.690 INFO:tasks.workunit.client.0.smithi033.stdout:4/253: read d6/d12/f26 [979689,61509] 0 2024-04-25T01:35:45.693 INFO:tasks.workunit.client.0.smithi033.stdout:1/193: write d2/d5/f18 [1522763,110300] 0 2024-04-25T01:35:45.694 INFO:tasks.workunit.client.0.smithi033.stdout:3/223: mkdir d8/dd/d3d/d4f 0 2024-04-25T01:35:45.696 INFO:tasks.workunit.client.0.smithi033.stdout:9/226: fsync d0/f23 0 2024-04-25T01:35:45.697 INFO:tasks.workunit.client.0.smithi033.stdout:1/194: write d2/d11/f2a [71913,116108] 0 2024-04-25T01:35:45.698 INFO:tasks.workunit.client.0.smithi033.stdout:0/227: dwrite d7/d33/f3e [0,4194304] 0 2024-04-25T01:35:45.701 INFO:tasks.workunit.client.0.smithi033.stdout:9/227: truncate d0/d2/dd/f52 523534 0 2024-04-25T01:35:45.702 INFO:tasks.workunit.client.0.smithi033.stdout:7/256: creat d4/d2e/f4f x:0 0 0 2024-04-25T01:35:45.706 INFO:tasks.workunit.client.0.smithi033.stdout:8/277: rename d6/db/l29 to d6/l64 0 2024-04-25T01:35:45.710 INFO:tasks.workunit.client.0.smithi033.stdout:0/228: mknod d7/d2d/c4a 0 2024-04-25T01:35:45.711 INFO:tasks.workunit.client.0.smithi033.stdout:7/257: rmdir d4/d10/d13 39 2024-04-25T01:35:45.712 INFO:tasks.workunit.client.0.smithi033.stdout:0/229: dread - d7/da/df/d14/f47 zero size 2024-04-25T01:35:45.713 INFO:tasks.workunit.client.0.smithi033.stdout:7/258: fdatasync d4/d10/f3b 0 2024-04-25T01:35:45.713 INFO:tasks.workunit.client.0.smithi033.stdout:9/228: rmdir d0/d2/dd/df/d1b/d3d 39 2024-04-25T01:35:45.716 INFO:tasks.workunit.client.0.smithi033.stdout:3/224: mknod d8/dd/d11/c50 0 2024-04-25T01:35:45.717 INFO:tasks.workunit.client.0.smithi033.stdout:8/278: dwrite d6/d45/d52/f36 [0,4194304] 0 2024-04-25T01:35:45.717 INFO:tasks.workunit.client.0.smithi033.stdout:7/259: chown d4/d2e/f48 76 1 2024-04-25T01:35:45.718 INFO:tasks.workunit.client.0.smithi033.stdout:4/254: dread d6/d14/d1d/f2e [0,4194304] 0 2024-04-25T01:35:45.720 INFO:tasks.workunit.client.0.smithi033.stdout:8/279: dread - d6/d45/d52/f27 zero size 2024-04-25T01:35:45.724 INFO:tasks.workunit.client.0.smithi033.stdout:1/195: rename d2/c10 to d2/d11/d28/d33/c39 0 2024-04-25T01:35:45.725 INFO:tasks.workunit.client.0.smithi033.stdout:1/196: chown d2/d5/f25 404 1 2024-04-25T01:35:45.727 INFO:tasks.workunit.client.0.smithi033.stdout:3/225: dread d8/dd/d1d/f2b [0,4194304] 0 2024-04-25T01:35:45.728 INFO:tasks.workunit.client.0.smithi033.stdout:4/255: mkdir d6/d14/d48 0 2024-04-25T01:35:45.730 INFO:tasks.workunit.client.0.smithi033.stdout:3/226: read d8/dd/d11/d1e/d27/f2e [583866,90157] 0 2024-04-25T01:35:45.731 INFO:tasks.workunit.client.0.smithi033.stdout:3/227: chown d8/l9 230655 1 2024-04-25T01:35:45.731 INFO:tasks.workunit.client.0.smithi033.stdout:7/260: dread d4/d10/d13/d1f/f24 [0,4194304] 0 2024-04-25T01:35:45.733 INFO:tasks.workunit.client.0.smithi033.stdout:3/228: write d8/dd/d11/f39 [185144,23312] 0 2024-04-25T01:35:45.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:45 smithi033 ceph-mon[26353]: pgmap v8: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 214 op/s 2024-04-25T01:35:45.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:45 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:35:45.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:45 smithi033 ceph-mon[26353]: mgrmap e25: smithi138.mmfhyi(active, since 10s), standbys: smithi033.asemba 2024-04-25T01:35:45.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:45 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:45.754 INFO:tasks.workunit.client.0.smithi033.stdout:9/229: mkdir d0/d2/d43/d4b/d55 0 2024-04-25T01:35:45.755 INFO:tasks.workunit.client.0.smithi033.stdout:8/280: creat d6/d45/d51/d5d/f65 x:0 0 0 2024-04-25T01:35:45.757 INFO:tasks.workunit.client.0.smithi033.stdout:1/197: mknod d2/d5/d1b/c3a 0 2024-04-25T01:35:45.764 INFO:tasks.workunit.client.0.smithi033.stdout:7/261: mkdir d4/d2e/d50 0 2024-04-25T01:35:45.765 INFO:tasks.workunit.client.0.smithi033.stdout:1/198: dwrite d2/d5/f25 [0,4194304] 0 2024-04-25T01:35:45.766 INFO:tasks.workunit.client.0.smithi033.stdout:1/199: chown d2/d5/cc 627 1 2024-04-25T01:35:45.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:45 smithi138 ceph-mon[32977]: pgmap v8: 65 pgs: 65 active+clean; 291 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 16 MiB/s wr, 214 op/s 2024-04-25T01:35:45.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:45 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:35:45.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:45 smithi138 ceph-mon[32977]: mgrmap e25: smithi138.mmfhyi(active, since 10s), standbys: smithi033.asemba 2024-04-25T01:35:45.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:45 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:45.769 INFO:tasks.workunit.client.0.smithi033.stdout:9/230: fdatasync d0/f29 0 2024-04-25T01:35:45.770 INFO:tasks.workunit.client.0.smithi033.stdout:8/281: symlink d6/d45/d51/l66 0 2024-04-25T01:35:45.774 INFO:tasks.workunit.client.0.smithi033.stdout:3/229: dwrite d8/dd/d11/f24 [4194304,4194304] 0 2024-04-25T01:35:45.774 INFO:tasks.workunit.client.0.smithi033.stdout:8/282: truncate d6/d45/d51/d5d/f62 992307 0 2024-04-25T01:35:45.778 INFO:tasks.workunit.client.0.smithi033.stdout:8/283: write d6/d45/d52/d2e/f30 [2990117,106365] 0 2024-04-25T01:35:45.781 INFO:tasks.workunit.client.0.smithi033.stdout:4/256: chown d6/d14/d1d/f2c 16367 1 2024-04-25T01:35:45.781 INFO:tasks.workunit.client.0.smithi033.stdout:0/230: dwrite d7/d2d/d43/d44/f49 [0,4194304] 0 2024-04-25T01:35:45.781 INFO:tasks.workunit.client.0.smithi033.stdout:8/284: write d6/f41 [217969,44410] 0 2024-04-25T01:35:45.784 INFO:tasks.workunit.client.0.smithi033.stdout:4/257: read d6/d12/f32 [94260,96091] 0 2024-04-25T01:35:45.786 INFO:tasks.workunit.client.0.smithi033.stdout:3/230: rmdir d8/dd/d11/d1e/d27 39 2024-04-25T01:35:45.786 INFO:tasks.workunit.client.0.smithi033.stdout:4/258: write d6/d12/f1a [3277148,105966] 0 2024-04-25T01:35:45.792 INFO:tasks.workunit.client.0.smithi033.stdout:1/200: symlink d2/d11/l3b 0 2024-04-25T01:35:45.792 INFO:tasks.workunit.client.0.smithi033.stdout:1/201: stat d2/d11/d28 0 2024-04-25T01:35:45.843 INFO:tasks.workunit.client.0.smithi033.stdout:7/262: link d4/d23/c38 d4/d23/d3f/c51 0 2024-04-25T01:35:45.843 INFO:tasks.workunit.client.0.smithi033.stdout:8/285: symlink d6/d45/d52/l67 0 2024-04-25T01:35:45.846 INFO:tasks.workunit.client.0.smithi033.stdout:1/202: creat d2/d5/f3c x:0 0 0 2024-04-25T01:35:45.848 INFO:tasks.workunit.client.0.smithi033.stdout:4/259: fdatasync d6/d14/ff 0 2024-04-25T01:35:45.850 INFO:tasks.workunit.client.0.smithi033.stdout:1/203: dread d2/d11/f1e [0,4194304] 0 2024-04-25T01:35:45.851 INFO:tasks.workunit.client.0.smithi033.stdout:7/263: unlink d4/d2e/c2a 0 2024-04-25T01:35:45.852 INFO:tasks.workunit.client.0.smithi033.stdout:8/286: dwrite d6/d45/d51/f4f [0,4194304] 0 2024-04-25T01:35:45.853 INFO:tasks.workunit.client.0.smithi033.stdout:7/264: chown d4/d10/d13/d1f/f24 5 1 2024-04-25T01:35:45.855 INFO:tasks.workunit.client.0.smithi033.stdout:3/231: creat d8/f51 x:0 0 0 2024-04-25T01:35:45.856 INFO:tasks.workunit.client.0.smithi033.stdout:3/232: write d8/f51 [764942,11718] 0 2024-04-25T01:35:45.856 INFO:tasks.workunit.client.0.smithi033.stdout:3/233: stat c2 0 2024-04-25T01:35:45.857 INFO:tasks.workunit.client.0.smithi033.stdout:3/234: rename d8 to d8/dd/d3d/d52 22 2024-04-25T01:35:45.859 INFO:tasks.workunit.client.0.smithi033.stdout:7/265: creat d4/d10/d13/f52 x:0 0 0 2024-04-25T01:35:45.862 INFO:tasks.workunit.client.0.smithi033.stdout:1/204: mknod d2/d2e/d34/c3d 0 2024-04-25T01:35:45.864 INFO:tasks.workunit.client.0.smithi033.stdout:3/235: creat d8/dd/d40/f53 x:0 0 0 2024-04-25T01:35:45.869 INFO:tasks.workunit.client.0.smithi033.stdout:4/260: creat d6/d14/d35/d3a/f49 x:0 0 0 2024-04-25T01:35:45.870 INFO:tasks.workunit.client.0.smithi033.stdout:4/261: rename d6/d14/d35 to d6/d14/d35/d44/d4a 22 2024-04-25T01:35:45.871 INFO:tasks.workunit.client.0.smithi033.stdout:3/236: mknod d8/dd/d1d/c54 0 2024-04-25T01:35:45.873 INFO:tasks.workunit.client.0.smithi033.stdout:4/262: rmdir d6/d14/d35 39 2024-04-25T01:35:45.878 INFO:tasks.workunit.client.0.smithi033.stdout:3/237: creat d8/dd/d11/f55 x:0 0 0 2024-04-25T01:35:45.880 INFO:tasks.workunit.client.0.smithi033.stdout:3/238: truncate d8/dd/f2f 4354834 0 2024-04-25T01:35:45.881 INFO:tasks.workunit.client.0.smithi033.stdout:3/239: truncate d8/dd/f2c 10175503 0 2024-04-25T01:35:45.883 INFO:tasks.workunit.client.0.smithi033.stdout:3/240: rmdir d8/dd/d11/d1e 39 2024-04-25T01:35:45.886 INFO:tasks.workunit.client.0.smithi033.stdout:3/241: dread - d8/dd/d11/d1e/d27/f36 zero size 2024-04-25T01:35:45.886 INFO:tasks.workunit.client.0.smithi033.stdout:3/242: readlink d8/lf 0 2024-04-25T01:35:45.887 INFO:tasks.workunit.client.0.smithi033.stdout:3/243: chown d8/dd/d11 541 1 2024-04-25T01:35:45.888 INFO:tasks.workunit.client.0.smithi033.stdout:3/244: fdatasync d8/dd/f34 0 2024-04-25T01:35:45.891 INFO:tasks.workunit.client.0.smithi033.stdout:7/266: truncate d4/d10/d13/d1f/f24 3751277 0 2024-04-25T01:35:45.892 INFO:tasks.workunit.client.0.smithi033.stdout:3/245: symlink d8/dd/d11/d1e/l56 0 2024-04-25T01:35:45.893 INFO:tasks.workunit.client.0.smithi033.stdout:8/287: truncate d6/db/f26 161590 0 2024-04-25T01:35:45.894 INFO:tasks.workunit.client.0.smithi033.stdout:7/267: symlink d4/d23/l53 0 2024-04-25T01:35:45.896 INFO:tasks.workunit.client.0.smithi033.stdout:1/205: truncate d2/d5/f18 1282372 0 2024-04-25T01:35:45.905 INFO:tasks.workunit.client.0.smithi033.stdout:4/263: dwrite d6/d14/d1d/d3b/f3e [0,4194304] 0 2024-04-25T01:35:45.911 INFO:tasks.workunit.client.0.smithi033.stdout:8/288: dread d6/f15 [0,4194304] 0 2024-04-25T01:35:45.911 INFO:tasks.workunit.client.0.smithi033.stdout:7/268: truncate d4/d2e/f48 840474 0 2024-04-25T01:35:45.916 INFO:tasks.workunit.client.0.smithi033.stdout:4/264: rename d6/d14/l29 to d6/d14/l4b 0 2024-04-25T01:35:45.918 INFO:tasks.workunit.client.0.smithi033.stdout:8/289: dread d6/db/fc [0,4194304] 0 2024-04-25T01:35:45.919 INFO:tasks.workunit.client.0.smithi033.stdout:8/290: chown d6/db/l20 254 1 2024-04-25T01:35:45.919 INFO:tasks.workunit.client.0.smithi033.stdout:4/265: fsync d6/d14/f2b 0 2024-04-25T01:35:45.920 INFO:tasks.workunit.client.0.smithi033.stdout:7/269: link d4/f43 d4/d23/f54 0 2024-04-25T01:35:45.921 INFO:tasks.workunit.client.0.smithi033.stdout:8/291: fsync d6/db/f24 0 2024-04-25T01:35:45.925 INFO:tasks.workunit.client.0.smithi033.stdout:8/292: unlink d6/d45/d52/d2e/l35 0 2024-04-25T01:35:45.926 INFO:tasks.workunit.client.0.smithi033.stdout:4/266: rmdir d6/d14/d35/d3a 39 2024-04-25T01:35:45.929 INFO:tasks.workunit.client.0.smithi033.stdout:8/293: unlink f5 0 2024-04-25T01:35:45.929 INFO:tasks.workunit.client.0.smithi033.stdout:8/294: chown d6/db/c33 191058 1 2024-04-25T01:35:45.934 INFO:tasks.workunit.client.0.smithi033.stdout:4/267: write d6/d14/f33 [2055290,9560] 0 2024-04-25T01:35:45.935 INFO:tasks.workunit.client.0.smithi033.stdout:4/268: write d6/d14/f33 [6099433,102218] 0 2024-04-25T01:35:45.936 INFO:tasks.workunit.client.0.smithi033.stdout:4/269: chown d6/c2a 63674 1 2024-04-25T01:35:45.937 INFO:tasks.workunit.client.0.smithi033.stdout:4/270: read - d6/d14/f2b zero size 2024-04-25T01:35:45.940 INFO:tasks.workunit.client.0.smithi033.stdout:8/295: link d6/db/c4b d6/c68 0 2024-04-25T01:35:45.940 INFO:tasks.workunit.client.0.smithi033.stdout:8/296: chown d6/d45/d52 216913462 1 2024-04-25T01:35:45.943 INFO:tasks.workunit.client.0.smithi033.stdout:8/297: symlink d6/d45/l69 0 2024-04-25T01:35:45.944 INFO:tasks.workunit.client.0.smithi033.stdout:7/270: link d4/d2e/f48 d4/d2e/d50/f55 0 2024-04-25T01:35:45.944 INFO:tasks.workunit.client.0.smithi033.stdout:8/298: chown d6/f15 23468205 1 2024-04-25T01:35:45.950 INFO:tasks.workunit.client.0.smithi033.stdout:4/271: truncate d6/d12/f2f 183397 0 2024-04-25T01:35:45.950 INFO:tasks.workunit.client.0.smithi033.stdout:8/299: rmdir d6/d45 39 2024-04-25T01:35:45.951 INFO:tasks.workunit.client.0.smithi033.stdout:4/272: fsync d6/f16 0 2024-04-25T01:35:45.954 INFO:tasks.workunit.client.0.smithi033.stdout:4/273: fdatasync d6/d14/ff 0 2024-04-25T01:35:45.956 INFO:tasks.workunit.client.0.smithi033.stdout:8/300: unlink d6/f21 0 2024-04-25T01:35:45.957 INFO:tasks.workunit.client.0.smithi033.stdout:7/271: dwrite d4/d2e/f1c [0,4194304] 0 2024-04-25T01:35:45.959 INFO:tasks.workunit.client.0.smithi033.stdout:4/274: unlink d6/d14/c1c 0 2024-04-25T01:35:45.963 INFO:tasks.workunit.client.0.smithi033.stdout:7/272: creat d4/d10/d13/f56 x:0 0 0 2024-04-25T01:35:45.967 INFO:tasks.workunit.client.0.smithi033.stdout:8/301: dwrite d6/d45/f57 [0,4194304] 0 2024-04-25T01:35:45.967 INFO:tasks.workunit.client.0.smithi033.stdout:7/273: mkdir d4/d57 0 2024-04-25T01:35:45.973 INFO:tasks.workunit.client.0.smithi033.stdout:8/302: creat d6/d45/d51/d5d/f6a x:0 0 0 2024-04-25T01:35:45.973 INFO:tasks.workunit.client.0.smithi033.stdout:4/275: dwrite d6/d14/d1d/f2e [4194304,4194304] 0 2024-04-25T01:35:45.981 INFO:tasks.workunit.client.0.smithi033.stdout:7/274: dwrite d4/f41 [0,4194304] 0 2024-04-25T01:35:45.981 INFO:tasks.workunit.client.0.smithi033.stdout:8/303: rename d6/l16 to d6/db/l6b 0 2024-04-25T01:35:45.982 INFO:tasks.workunit.client.0.smithi033.stdout:4/276: dread d6/d14/ff [0,4194304] 0 2024-04-25T01:35:45.983 INFO:tasks.workunit.client.0.smithi033.stdout:8/304: chown d6/d45/c4c 28 1 2024-04-25T01:35:45.985 INFO:tasks.workunit.client.0.smithi033.stdout:7/275: chown d4/d23/l27 729098251 1 2024-04-25T01:35:45.985 INFO:tasks.workunit.client.0.smithi033.stdout:8/305: fsync d6/d45/d52/d2e/f56 0 2024-04-25T01:35:45.988 INFO:tasks.workunit.client.0.smithi033.stdout:7/276: chown d4/d10/d13/f33 25391 1 2024-04-25T01:35:45.990 INFO:tasks.workunit.client.0.smithi033.stdout:4/277: fsync d6/f1f 0 2024-04-25T01:35:45.990 INFO:tasks.workunit.client.0.smithi033.stdout:8/306: dread d6/f15 [0,4194304] 0 2024-04-25T01:35:45.995 INFO:tasks.workunit.client.0.smithi033.stdout:8/307: mknod d6/d49/c6c 0 2024-04-25T01:35:45.998 INFO:tasks.workunit.client.0.smithi033.stdout:7/277: dwrite d4/d2e/f3e [0,4194304] 0 2024-04-25T01:35:46.000 INFO:tasks.workunit.client.0.smithi033.stdout:4/278: mkdir d6/d14/d35/d3a/d4c 0 2024-04-25T01:35:46.006 INFO:tasks.workunit.client.0.smithi033.stdout:4/279: truncate d6/d12/f32 835264 0 2024-04-25T01:35:46.010 INFO:tasks.workunit.client.0.smithi033.stdout:4/280: mkdir d6/d14/d35/d3a/d4d 0 2024-04-25T01:35:46.016 INFO:tasks.workunit.client.0.smithi033.stdout:4/281: rename d6/d12/c19 to d6/d14/d35/d3a/d4d/c4e 0 2024-04-25T01:35:46.018 INFO:tasks.workunit.client.0.smithi033.stdout:7/278: getdents d4/d10/d13/d1f 0 2024-04-25T01:35:46.019 INFO:tasks.workunit.client.0.smithi033.stdout:7/279: truncate d4/f17 982842 0 2024-04-25T01:35:46.023 INFO:tasks.workunit.client.0.smithi033.stdout:4/282: unlink d6/c2a 0 2024-04-25T01:35:46.023 INFO:tasks.workunit.client.0.smithi033.stdout:7/280: symlink d4/d23/d3f/l58 0 2024-04-25T01:35:46.025 INFO:tasks.workunit.client.0.smithi033.stdout:7/281: mknod d4/d10/d13/c59 0 2024-04-25T01:35:46.028 INFO:tasks.workunit.client.0.smithi033.stdout:7/282: unlink d4/f34 0 2024-04-25T01:35:46.042 INFO:tasks.workunit.client.0.smithi033.stdout:7/283: dread d4/d10/d13/d1f/f24 [0,4194304] 0 2024-04-25T01:35:46.047 INFO:tasks.workunit.client.0.smithi033.stdout:7/284: read d4/d10/d13/f47 [373005,43208] 0 2024-04-25T01:35:46.051 INFO:tasks.workunit.client.0.smithi033.stdout:8/308: dwrite d6/db/f24 [0,4194304] 0 2024-04-25T01:35:46.052 INFO:tasks.workunit.client.0.smithi033.stdout:4/283: truncate d6/d14/f31 3828513 0 2024-04-25T01:35:46.052 INFO:tasks.workunit.client.0.smithi033.stdout:8/309: fsync d6/f3c 0 2024-04-25T01:35:46.054 INFO:tasks.workunit.client.0.smithi033.stdout:4/284: rmdir d6/d14/d1d 39 2024-04-25T01:35:46.055 INFO:tasks.workunit.client.0.smithi033.stdout:7/285: dwrite d4/f39 [0,4194304] 0 2024-04-25T01:35:46.057 INFO:tasks.workunit.client.0.smithi033.stdout:4/285: write d6/d14/f2b [618859,114936] 0 2024-04-25T01:35:46.057 INFO:tasks.workunit.client.0.smithi033.stdout:8/310: creat d6/d45/f6d x:0 0 0 2024-04-25T01:35:46.057 INFO:tasks.workunit.client.0.smithi033.stdout:7/286: write d4/d10/f3b [462860,75013] 0 2024-04-25T01:35:46.058 INFO:tasks.workunit.client.0.smithi033.stdout:4/286: chown d6 2127 1 2024-04-25T01:35:46.060 INFO:tasks.workunit.client.0.smithi033.stdout:4/287: write d6/f9 [7671814,50251] 0 2024-04-25T01:35:46.067 INFO:tasks.workunit.client.0.smithi033.stdout:8/311: dwrite d6/f1d [0,4194304] 0 2024-04-25T01:35:46.074 INFO:tasks.workunit.client.0.smithi033.stdout:8/312: mkdir d6/d45/d51/d5d/d6e 0 2024-04-25T01:35:46.077 INFO:tasks.workunit.client.0.smithi033.stdout:7/287: rename d4/d23/d3f/c51 to d4/c5a 0 2024-04-25T01:35:46.078 INFO:tasks.workunit.client.0.smithi033.stdout:4/288: link d6/d14/l4b d6/l4f 0 2024-04-25T01:35:46.079 INFO:tasks.workunit.client.0.smithi033.stdout:8/313: creat d6/d45/d51/d5d/f6f x:0 0 0 2024-04-25T01:35:46.081 INFO:tasks.workunit.client.0.smithi033.stdout:7/288: unlink d4/d23/f35 0 2024-04-25T01:35:46.083 INFO:tasks.workunit.client.0.smithi033.stdout:4/289: chown d6/d12/f2f 69430 1 2024-04-25T01:35:46.083 INFO:tasks.workunit.client.0.smithi033.stdout:4/290: readlink d6/d12/l21 0 2024-04-25T01:35:46.087 INFO:tasks.workunit.client.0.smithi033.stdout:7/289: creat d4/d23/d3f/d45/f5b x:0 0 0 2024-04-25T01:35:46.092 INFO:tasks.workunit.client.0.smithi033.stdout:4/291: dread d6/d14/d1d/f2c [4194304,4194304] 0 2024-04-25T01:35:46.092 INFO:tasks.workunit.client.0.smithi033.stdout:7/290: fsync d4/d10/d13/d1f/f46 0 2024-04-25T01:35:46.095 INFO:tasks.workunit.client.0.smithi033.stdout:4/292: read d6/d14/f2b [544648,128863] 0 2024-04-25T01:35:46.095 INFO:tasks.workunit.client.0.smithi033.stdout:4/293: chown d6/d12/f26 2 1 2024-04-25T01:35:46.097 INFO:tasks.workunit.client.0.smithi033.stdout:8/314: dread d6/d45/d52/d2e/f3a [0,4194304] 0 2024-04-25T01:35:46.099 INFO:tasks.workunit.client.0.smithi033.stdout:7/291: truncate d4/d10/d13/f47 871663 0 2024-04-25T01:35:46.100 INFO:tasks.workunit.client.0.smithi033.stdout:7/292: chown d4/c31 49667 1 2024-04-25T01:35:46.103 INFO:tasks.workunit.client.0.smithi033.stdout:8/315: mkdir d6/d70 0 2024-04-25T01:35:46.104 INFO:tasks.workunit.client.0.smithi033.stdout:4/294: dwrite d6/d12/f26 [4194304,4194304] 0 2024-04-25T01:35:46.107 INFO:tasks.workunit.client.0.smithi033.stdout:4/295: fsync d6/d14/f33 0 2024-04-25T01:35:46.107 INFO:tasks.workunit.client.0.smithi033.stdout:8/316: write d6/d45/d51/f4f [580214,6532] 0 2024-04-25T01:35:46.109 INFO:tasks.workunit.client.0.smithi033.stdout:4/296: chown d6/d12/f2f 1 1 2024-04-25T01:35:46.109 INFO:tasks.workunit.client.0.smithi033.stdout:8/317: read d6/f3e [1678,67180] 0 2024-04-25T01:35:46.110 INFO:tasks.workunit.client.0.smithi033.stdout:7/293: dwrite d4/d10/d13/d1f/f36 [0,4194304] 0 2024-04-25T01:35:46.112 INFO:tasks.workunit.client.0.smithi033.stdout:8/318: chown d6/d45/d52/f3f 0 1 2024-04-25T01:35:46.113 INFO:tasks.workunit.client.0.smithi033.stdout:4/297: unlink d6/d14/lc 0 2024-04-25T01:35:46.114 INFO:tasks.workunit.client.0.smithi033.stdout:8/319: symlink d6/d45/d52/d2e/l71 0 2024-04-25T01:35:46.119 INFO:tasks.workunit.client.0.smithi033.stdout:8/320: symlink d6/d45/d51/d5d/d6e/l72 0 2024-04-25T01:35:46.121 INFO:tasks.workunit.client.0.smithi033.stdout:7/294: dwrite d4/d10/d13/f56 [0,4194304] 0 2024-04-25T01:35:46.121 INFO:tasks.workunit.client.0.smithi033.stdout:8/321: dread - d6/d45/d52/f54 zero size 2024-04-25T01:35:46.123 INFO:tasks.workunit.client.0.smithi033.stdout:8/322: fsync d6/d45/d51/d5d/f65 0 2024-04-25T01:35:46.124 INFO:tasks.workunit.client.0.smithi033.stdout:4/298: dwrite d6/d12/f26 [4194304,4194304] 0 2024-04-25T01:35:46.126 INFO:tasks.workunit.client.0.smithi033.stdout:7/295: symlink d4/d57/l5c 0 2024-04-25T01:35:46.126 INFO:tasks.workunit.client.0.smithi033.stdout:4/299: write d6/d14/d35/d3a/f49 [319708,28632] 0 2024-04-25T01:35:46.127 INFO:tasks.workunit.client.0.smithi033.stdout:8/323: creat d6/d45/f73 x:0 0 0 2024-04-25T01:35:46.128 INFO:tasks.workunit.client.0.smithi033.stdout:8/324: truncate d6/f58 521144 0 2024-04-25T01:35:46.129 INFO:tasks.workunit.client.0.smithi033.stdout:7/296: creat d4/d57/f5d x:0 0 0 2024-04-25T01:35:46.134 INFO:tasks.workunit.client.0.smithi033.stdout:7/297: creat d4/d23/d3f/d45/f5e x:0 0 0 2024-04-25T01:35:46.136 INFO:tasks.workunit.client.0.smithi033.stdout:8/325: rename d6/d45/d51/d5d/d6e/l72 to d6/d45/d51/d55/l74 0 2024-04-25T01:35:46.142 INFO:tasks.workunit.client.0.smithi033.stdout:7/298: link c2 d4/d23/d3f/c5f 0 2024-04-25T01:35:46.144 INFO:tasks.workunit.client.0.smithi033.stdout:8/326: rename d6/db/c33 to d6/d49/c75 0 2024-04-25T01:35:46.145 INFO:tasks.workunit.client.0.smithi033.stdout:8/327: dread - d6/d45/f73 zero size 2024-04-25T01:35:46.146 INFO:tasks.workunit.client.0.smithi033.stdout:7/299: creat d4/d23/d3f/f60 x:0 0 0 2024-04-25T01:35:46.147 INFO:tasks.workunit.client.0.smithi033.stdout:8/328: creat d6/d45/f76 x:0 0 0 2024-04-25T01:35:46.148 INFO:tasks.workunit.client.0.smithi033.stdout:8/329: dread - d6/f34 zero size 2024-04-25T01:35:46.149 INFO:tasks.workunit.client.0.smithi033.stdout:7/300: symlink d4/d57/l61 0 2024-04-25T01:35:46.152 INFO:tasks.workunit.client.0.smithi033.stdout:4/300: dwrite d6/d12/f32 [0,4194304] 0 2024-04-25T01:35:46.155 INFO:tasks.workunit.client.0.smithi033.stdout:7/301: creat d4/d10/d13/f62 x:0 0 0 2024-04-25T01:35:46.156 INFO:tasks.workunit.client.0.smithi033.stdout:2/340: sync 2024-04-25T01:35:46.157 INFO:tasks.workunit.client.0.smithi033.stdout:4/301: creat d6/d14/d1d/d3b/f50 x:0 0 0 2024-04-25T01:35:46.157 INFO:tasks.workunit.client.0.smithi033.stdout:7/302: write d4/d10/f3b [494390,1997] 0 2024-04-25T01:35:46.159 INFO:tasks.workunit.client.0.smithi033.stdout:8/330: dwrite d6/d45/d51/d5d/f62 [0,4194304] 0 2024-04-25T01:35:46.164 INFO:tasks.workunit.client.0.smithi033.stdout:4/302: symlink d6/d14/d1d/d3b/l51 0 2024-04-25T01:35:46.164 INFO:tasks.workunit.client.0.smithi033.stdout:2/341: dread da/d27/d31/d40/f5f [0,4194304] 0 2024-04-25T01:35:46.165 INFO:tasks.workunit.client.0.smithi033.stdout:2/342: chown da/l1d 10 1 2024-04-25T01:35:46.166 INFO:tasks.workunit.client.0.smithi033.stdout:2/343: chown da/f13 1957147 1 2024-04-25T01:35:46.167 INFO:tasks.workunit.client.0.smithi033.stdout:7/303: rmdir d4/d10/d13 39 2024-04-25T01:35:46.171 INFO:tasks.workunit.client.0.smithi033.stdout:7/304: rmdir d4/d57 39 2024-04-25T01:35:46.174 INFO:tasks.workunit.client.0.smithi033.stdout:7/305: readlink d4/d57/l61 0 2024-04-25T01:35:46.174 INFO:tasks.workunit.client.0.smithi033.stdout:8/331: getdents d6/d45/d52/d2e 0 2024-04-25T01:35:46.174 INFO:tasks.workunit.client.0.smithi033.stdout:4/303: dwrite d6/d12/f40 [0,4194304] 0 2024-04-25T01:35:46.175 INFO:tasks.workunit.client.0.smithi033.stdout:7/306: readlink d4/l12 0 2024-04-25T01:35:46.178 INFO:tasks.workunit.client.0.smithi033.stdout:7/307: dread - d4/f7 zero size 2024-04-25T01:35:46.179 INFO:tasks.workunit.client.0.smithi033.stdout:7/308: readlink d4/lc 0 2024-04-25T01:35:46.184 INFO:tasks.workunit.client.0.smithi033.stdout:8/332: creat d6/d45/d51/d5d/d6e/f77 x:0 0 0 2024-04-25T01:35:46.184 INFO:tasks.workunit.client.0.smithi033.stdout:4/304: link d6/d12/f40 d6/d14/d1d/d3b/f52 0 2024-04-25T01:35:46.187 INFO:tasks.workunit.client.0.smithi033.stdout:8/333: mknod d6/d45/d51/d5d/d6e/c78 0 2024-04-25T01:35:46.188 INFO:tasks.workunit.client.0.smithi033.stdout:7/309: dwrite d4/d23/d3f/d45/f5e [0,4194304] 0 2024-04-25T01:35:46.190 INFO:tasks.workunit.client.0.smithi033.stdout:4/305: creat d6/d14/f53 x:0 0 0 2024-04-25T01:35:46.192 INFO:tasks.workunit.client.0.smithi033.stdout:7/310: fdatasync d4/d10/d13/d1f/f36 0 2024-04-25T01:35:46.193 INFO:tasks.workunit.client.0.smithi033.stdout:2/344: dread da/d27/f2d [0,4194304] 0 2024-04-25T01:35:46.194 INFO:tasks.workunit.client.0.smithi033.stdout:8/334: link d6/db/f24 d6/d45/d51/f79 0 2024-04-25T01:35:46.194 INFO:tasks.workunit.client.0.smithi033.stdout:2/345: chown da/d27/c29 101653 1 2024-04-25T01:35:46.195 INFO:tasks.workunit.client.0.smithi033.stdout:2/346: dread - da/d50/f58 zero size 2024-04-25T01:35:46.202 INFO:tasks.workunit.client.0.smithi033.stdout:8/335: dwrite d6/f3c [0,4194304] 0 2024-04-25T01:35:46.204 INFO:tasks.workunit.client.0.smithi033.stdout:8/336: fdatasync d6/d45/d51/d5d/f65 0 2024-04-25T01:35:46.205 INFO:tasks.workunit.client.0.smithi033.stdout:2/347: rename da/d23 to da/d50/d6a 0 2024-04-25T01:35:46.206 INFO:tasks.workunit.client.0.smithi033.stdout:4/306: link d6/c15 d6/d14/d35/d44/c54 0 2024-04-25T01:35:46.207 INFO:tasks.workunit.client.0.smithi033.stdout:2/348: rmdir da/d32 39 2024-04-25T01:35:46.208 INFO:tasks.workunit.client.0.smithi033.stdout:4/307: truncate d6/d14/d35/d3a/f49 973092 0 2024-04-25T01:35:46.209 INFO:tasks.workunit.client.0.smithi033.stdout:8/337: mknod d6/d45/d51/d55/c7a 0 2024-04-25T01:35:46.211 INFO:tasks.workunit.client.0.smithi033.stdout:8/338: stat d6/db/f18 0 2024-04-25T01:35:46.212 INFO:tasks.workunit.client.0.smithi033.stdout:8/339: chown d6/d45/d52/f28 274 1 2024-04-25T01:35:46.213 INFO:tasks.workunit.client.0.smithi033.stdout:4/308: rename d6/d12/f2f to d6/d14/d48/f55 0 2024-04-25T01:35:46.215 INFO:tasks.workunit.client.0.smithi033.stdout:2/349: symlink da/l6b 0 2024-04-25T01:35:46.217 INFO:tasks.workunit.client.0.smithi033.stdout:8/340: readlink d6/db/l20 0 2024-04-25T01:35:46.218 INFO:tasks.workunit.client.0.smithi033.stdout:4/309: creat d6/d14/d35/d3a/f56 x:0 0 0 2024-04-25T01:35:46.220 INFO:tasks.workunit.client.0.smithi033.stdout:7/311: dwrite d4/d10/d13/d1f/f42 [0,4194304] 0 2024-04-25T01:35:46.222 INFO:tasks.workunit.client.0.smithi033.stdout:2/350: rename da/d4a to da/d50/d6a/d43/d6c 0 2024-04-25T01:35:46.227 INFO:tasks.workunit.client.0.smithi033.stdout:2/351: fdatasync da/d50/d6a/d43/f66 0 2024-04-25T01:35:46.227 INFO:tasks.workunit.client.0.smithi033.stdout:7/312: creat d4/d2e/f63 x:0 0 0 2024-04-25T01:35:46.227 INFO:tasks.workunit.client.0.smithi033.stdout:8/341: dread d6/f15 [4194304,4194304] 0 2024-04-25T01:35:46.228 INFO:tasks.workunit.client.0.smithi033.stdout:7/313: mknod d4/d23/c64 0 2024-04-25T01:35:46.228 INFO:tasks.workunit.client.0.smithi033.stdout:2/352: chown da/d27/d31/d40/f5f 0 1 2024-04-25T01:35:46.233 INFO:tasks.workunit.client.0.smithi033.stdout:2/353: creat da/d27/f6d x:0 0 0 2024-04-25T01:35:46.234 INFO:tasks.workunit.client.0.smithi033.stdout:7/314: truncate d4/d10/d13/d1f/f24 2271710 0 2024-04-25T01:35:46.234 INFO:tasks.workunit.client.0.smithi033.stdout:4/310: dwrite d6/d12/f26 [4194304,4194304] 0 2024-04-25T01:35:46.237 INFO:tasks.workunit.client.0.smithi033.stdout:4/311: write d6/d12/f32 [2267458,46965] 0 2024-04-25T01:35:46.238 INFO:tasks.workunit.client.0.smithi033.stdout:8/342: dread d6/db/f24 [0,4194304] 0 2024-04-25T01:35:46.244 INFO:tasks.workunit.client.0.smithi033.stdout:4/312: dread d6/d14/d1d/f2e [0,4194304] 0 2024-04-25T01:35:46.246 INFO:tasks.workunit.client.0.smithi033.stdout:2/354: truncate da/d5b/f63 3602921 0 2024-04-25T01:35:46.251 INFO:tasks.workunit.client.0.smithi033.stdout:2/355: write da/f13 [4734254,99867] 0 2024-04-25T01:35:46.251 INFO:tasks.workunit.client.0.smithi033.stdout:4/313: mkdir d6/d14/d35/d44/d45/d46/d57 0 2024-04-25T01:35:46.251 INFO:tasks.workunit.client.0.smithi033.stdout:4/314: stat d6/f9 0 2024-04-25T01:35:46.251 INFO:tasks.workunit.client.0.smithi033.stdout:2/356: chown da/d27/d31/d40/f64 4722 1 2024-04-25T01:35:46.251 INFO:tasks.workunit.client.0.smithi033.stdout:4/315: read d6/d12/f23 [795813,47057] 0 2024-04-25T01:35:46.251 INFO:tasks.workunit.client.0.smithi033.stdout:8/343: dread d6/f58 [0,4194304] 0 2024-04-25T01:35:46.251 INFO:tasks.workunit.client.0.smithi033.stdout:2/357: symlink da/d2c/d30/l6e 0 2024-04-25T01:35:46.252 INFO:tasks.workunit.client.0.smithi033.stdout:2/358: readlink da/d50/d6a/l24 0 2024-04-25T01:35:46.253 INFO:tasks.workunit.client.0.smithi033.stdout:4/316: rmdir d6/d12 39 2024-04-25T01:35:46.255 INFO:tasks.workunit.client.0.smithi033.stdout:8/344: unlink d6/f8 0 2024-04-25T01:35:46.257 INFO:tasks.workunit.client.0.smithi033.stdout:4/317: fsync d6/d12/f1a 0 2024-04-25T01:35:46.258 INFO:tasks.workunit.client.0.smithi033.stdout:2/359: rmdir da/d32 39 2024-04-25T01:35:46.259 INFO:tasks.workunit.client.0.smithi033.stdout:7/315: truncate d4/f39 2833686 0 2024-04-25T01:35:46.262 INFO:tasks.workunit.client.0.smithi033.stdout:4/318: creat d6/d12/f58 x:0 0 0 2024-04-25T01:35:46.264 INFO:tasks.workunit.client.0.smithi033.stdout:4/319: write d6/f8 [6621330,41399] 0 2024-04-25T01:35:46.264 INFO:tasks.workunit.client.0.smithi033.stdout:2/360: dread da/d50/d6a/f56 [0,4194304] 0 2024-04-25T01:35:46.265 INFO:tasks.workunit.client.0.smithi033.stdout:4/320: chown d6/d14/ff 183 1 2024-04-25T01:35:46.266 INFO:tasks.workunit.client.0.smithi033.stdout:4/321: chown d6/d12/f23 15237456 1 2024-04-25T01:35:46.267 INFO:tasks.workunit.client.0.smithi033.stdout:7/316: mkdir d4/d2e/d65 0 2024-04-25T01:35:46.268 INFO:tasks.workunit.client.0.smithi033.stdout:4/322: write d6/d14/d1d/d3b/f3e [3884064,75097] 0 2024-04-25T01:35:46.269 INFO:tasks.workunit.client.0.smithi033.stdout:8/345: creat d6/f7b x:0 0 0 2024-04-25T01:35:46.270 INFO:tasks.workunit.client.0.smithi033.stdout:8/346: dread - d6/d45/d51/d55/f5c zero size 2024-04-25T01:35:46.271 INFO:tasks.workunit.client.0.smithi033.stdout:8/347: truncate d6/d45/d52/f2b 8652165 0 2024-04-25T01:35:46.272 INFO:tasks.workunit.client.0.smithi033.stdout:8/348: chown d6/d45/d51/d5d 0 1 2024-04-25T01:35:46.279 INFO:tasks.workunit.client.0.smithi033.stdout:4/323: mknod d6/d14/d35/d39/c59 0 2024-04-25T01:35:46.281 INFO:tasks.workunit.client.0.smithi033.stdout:2/361: creat da/d5b/d5e/f6f x:0 0 0 2024-04-25T01:35:46.291 INFO:tasks.workunit.client.0.smithi033.stdout:8/349: mknod d6/c7c 0 2024-04-25T01:35:46.293 INFO:tasks.workunit.client.0.smithi033.stdout:4/324: mknod d6/d14/d35/c5a 0 2024-04-25T01:35:46.297 INFO:tasks.workunit.client.0.smithi033.stdout:8/350: symlink d6/d45/d51/l7d 0 2024-04-25T01:35:46.297 INFO:tasks.workunit.client.0.smithi033.stdout:7/317: write d4/d10/f26 [2298054,26025] 0 2024-04-25T01:35:46.300 INFO:tasks.workunit.client.0.smithi033.stdout:8/351: mknod d6/d45/d51/c7e 0 2024-04-25T01:35:46.302 INFO:tasks.workunit.client.0.smithi033.stdout:2/362: rename da/f2f to da/d32/f70 0 2024-04-25T01:35:46.302 INFO:tasks.workunit.client.0.smithi033.stdout:2/363: chown da/d27 534653127 1 2024-04-25T01:35:46.303 INFO:tasks.workunit.client.0.smithi033.stdout:7/318: creat d4/d23/d3f/d45/f66 x:0 0 0 2024-04-25T01:35:46.304 INFO:tasks.workunit.client.0.smithi033.stdout:8/352: creat d6/d45/f7f x:0 0 0 2024-04-25T01:35:46.306 INFO:tasks.workunit.client.0.smithi033.stdout:4/325: write d6/f1f [157055,56611] 0 2024-04-25T01:35:46.306 INFO:tasks.workunit.client.0.smithi033.stdout:8/353: read d6/db/f18 [4018167,44398] 0 2024-04-25T01:35:46.308 INFO:tasks.workunit.client.0.smithi033.stdout:7/319: read d4/f39 [303866,106317] 0 2024-04-25T01:35:46.310 INFO:tasks.workunit.client.0.smithi033.stdout:7/320: chown d4/d10/d13/f56 29 1 2024-04-25T01:35:46.314 INFO:tasks.workunit.client.0.smithi033.stdout:4/326: dwrite d6/d12/f58 [0,4194304] 0 2024-04-25T01:35:46.317 INFO:tasks.workunit.client.0.smithi033.stdout:8/354: mkdir d6/db/d80 0 2024-04-25T01:35:46.318 INFO:tasks.workunit.client.0.smithi033.stdout:2/364: symlink da/d32/l71 0 2024-04-25T01:35:46.321 INFO:tasks.workunit.client.0.smithi033.stdout:7/321: mkdir d4/d10/d13/d67 0 2024-04-25T01:35:46.321 INFO:tasks.workunit.client.0.smithi033.stdout:8/355: dread - d6/db/f61 zero size 2024-04-25T01:35:46.322 INFO:tasks.workunit.client.0.smithi033.stdout:2/365: mkdir da/d5b/d72 0 2024-04-25T01:35:46.323 INFO:tasks.workunit.client.0.smithi033.stdout:2/366: readlink da/d27/d31/d40/l52 0 2024-04-25T01:35:46.325 INFO:tasks.workunit.client.0.smithi033.stdout:7/322: creat d4/d2e/d65/f68 x:0 0 0 2024-04-25T01:35:46.326 INFO:tasks.workunit.client.0.smithi033.stdout:4/327: getdents d6/d12/d38 0 2024-04-25T01:35:46.330 INFO:tasks.workunit.client.0.smithi033.stdout:4/328: stat d6/d14/d35/d44/d45/d46 0 2024-04-25T01:35:46.330 INFO:tasks.workunit.client.0.smithi033.stdout:2/367: rename da/d32/l71 to da/d50/l73 0 2024-04-25T01:35:46.330 INFO:tasks.workunit.client.0.smithi033.stdout:8/356: chown d6/d45/d51/d5d/d6e 0 1 2024-04-25T01:35:46.331 INFO:tasks.workunit.client.0.smithi033.stdout:8/357: write d6/f34 [790453,28391] 0 2024-04-25T01:35:46.331 INFO:tasks.workunit.client.0.smithi033.stdout:7/323: creat d4/d2e/d65/f69 x:0 0 0 2024-04-25T01:35:46.331 INFO:tasks.workunit.client.0.smithi033.stdout:4/329: read d6/d14/d35/d3a/f49 [336385,99194] 0 2024-04-25T01:35:46.334 INFO:tasks.workunit.client.0.smithi033.stdout:8/358: write d6/db/f24 [968041,83186] 0 2024-04-25T01:35:46.334 INFO:tasks.workunit.client.0.smithi033.stdout:2/368: creat da/f74 x:0 0 0 2024-04-25T01:35:46.335 INFO:tasks.workunit.client.0.smithi033.stdout:7/324: mknod d4/d10/c6a 0 2024-04-25T01:35:46.337 INFO:tasks.workunit.client.0.smithi033.stdout:4/330: creat d6/d14/d35/d3a/d4d/f5b x:0 0 0 2024-04-25T01:35:46.342 INFO:tasks.workunit.client.0.smithi033.stdout:8/359: creat d6/d45/d51/d5d/f81 x:0 0 0 2024-04-25T01:35:46.344 INFO:tasks.workunit.client.0.smithi033.stdout:4/331: dread d6/d14/d1d/d3b/f3e [4194304,4194304] 0 2024-04-25T01:35:46.346 INFO:tasks.workunit.client.0.smithi033.stdout:4/332: fsync d6/d14/d1d/d3b/f52 0 2024-04-25T01:35:46.347 INFO:tasks.workunit.client.0.smithi033.stdout:4/333: stat d6/d14/d35/l36 0 2024-04-25T01:35:46.355 INFO:tasks.workunit.client.0.smithi033.stdout:4/334: getdents d6/d14/d35/d44/d45/d46 0 2024-04-25T01:35:46.355 INFO:tasks.workunit.client.0.smithi033.stdout:7/325: dread f0 [0,4194304] 0 2024-04-25T01:35:46.362 INFO:tasks.workunit.client.0.smithi033.stdout:7/326: rename d4/d10/d13 to d4/d2e/d6b 0 2024-04-25T01:35:46.363 INFO:tasks.workunit.client.0.smithi033.stdout:2/369: dread da/f18 [0,4194304] 0 2024-04-25T01:35:46.470 INFO:tasks.workunit.client.0.smithi033.stdout:4/335: truncate d6/d12/d38/f41 1049069 0 2024-04-25T01:35:46.471 INFO:tasks.workunit.client.0.smithi033.stdout:7/327: truncate d4/d23/d3f/d45/f66 15021 0 2024-04-25T01:35:46.472 INFO:tasks.workunit.client.0.smithi033.stdout:8/360: dread d6/d45/d52/f2b [4194304,4194304] 0 2024-04-25T01:35:46.473 INFO:tasks.workunit.client.0.smithi033.stdout:7/328: truncate d4/d23/d3f/f4d 888526 0 2024-04-25T01:35:46.476 INFO:tasks.workunit.client.0.smithi033.stdout:7/329: mkdir d4/d23/d3f/d45/d6c 0 2024-04-25T01:35:46.476 INFO:tasks.workunit.client.0.smithi033.stdout:8/361: dread d6/db/fc [0,4194304] 0 2024-04-25T01:35:46.486 INFO:tasks.workunit.client.0.smithi033.stdout:7/330: truncate d4/d23/d3f/d45/f5e 2039736 0 2024-04-25T01:35:46.487 INFO:tasks.workunit.client.0.smithi033.stdout:8/362: dwrite d6/f7b [0,4194304] 0 2024-04-25T01:35:46.487 INFO:tasks.workunit.client.0.smithi033.stdout:4/336: dwrite d6/d14/d48/f55 [0,4194304] 0 2024-04-25T01:35:46.488 INFO:tasks.workunit.client.0.smithi033.stdout:7/331: chown d4/d23/l27 10054 1 2024-04-25T01:35:46.489 INFO:tasks.workunit.client.0.smithi033.stdout:8/363: chown d6/d45/d51/d5d/d6e/c78 1629758 1 2024-04-25T01:35:46.491 INFO:tasks.workunit.client.0.smithi033.stdout:8/364: write d6/f1d [2853257,112263] 0 2024-04-25T01:35:46.497 INFO:tasks.workunit.client.0.smithi033.stdout:4/337: dwrite d6/d14/d1d/d3b/f50 [0,4194304] 0 2024-04-25T01:35:46.500 INFO:tasks.workunit.client.0.smithi033.stdout:8/365: rename d6/f3c to d6/d45/f82 0 2024-04-25T01:35:46.504 INFO:tasks.workunit.client.0.smithi033.stdout:4/338: dread d6/d12/f23 [0,4194304] 0 2024-04-25T01:35:46.504 INFO:tasks.workunit.client.0.smithi033.stdout:8/366: write d6/d45/d51/f4d [307574,130890] 0 2024-04-25T01:35:46.508 INFO:tasks.workunit.client.0.smithi033.stdout:4/339: mknod d6/d12/d38/c5c 0 2024-04-25T01:35:46.510 INFO:tasks.workunit.client.0.smithi033.stdout:8/367: rmdir d6/d70 0 2024-04-25T01:35:46.512 INFO:tasks.workunit.client.0.smithi033.stdout:4/340: creat d6/d12/d38/f5d x:0 0 0 2024-04-25T01:35:46.513 INFO:tasks.workunit.client.0.smithi033.stdout:4/341: write d6/d12/d38/f5d [835060,87641] 0 2024-04-25T01:35:46.515 INFO:tasks.workunit.client.0.smithi033.stdout:4/342: mkdir d6/d14/d35/d3a/d4d/d5e 0 2024-04-25T01:35:46.516 INFO:tasks.workunit.client.0.smithi033.stdout:4/343: stat d6/d12/d38/f5d 0 2024-04-25T01:35:46.518 INFO:tasks.workunit.client.0.smithi033.stdout:4/344: mkdir d6/d14/d1d/d5f 0 2024-04-25T01:35:46.528 INFO:tasks.workunit.client.0.smithi033.stdout:4/345: dwrite d6/d12/f58 [4194304,4194304] 0 2024-04-25T01:35:46.528 INFO:tasks.workunit.client.0.smithi033.stdout:7/332: truncate d4/d23/d3f/d45/f5e 2865288 0 2024-04-25T01:35:46.529 INFO:tasks.workunit.client.0.smithi033.stdout:8/368: write d6/f58 [1017244,88435] 0 2024-04-25T01:35:46.531 INFO:tasks.workunit.client.0.smithi033.stdout:8/369: write d6/d45/f7f [764666,59009] 0 2024-04-25T01:35:46.538 INFO:tasks.workunit.client.0.smithi033.stdout:4/346: dwrite d6/d12/f40 [0,4194304] 0 2024-04-25T01:35:46.539 INFO:tasks.workunit.client.0.smithi033.stdout:4/347: chown d6/d14/ff 7 1 2024-04-25T01:35:46.546 INFO:tasks.workunit.client.0.smithi033.stdout:7/333: dwrite d4/d2e/d50/f55 [0,4194304] 0 2024-04-25T01:35:46.547 INFO:tasks.workunit.client.0.smithi033.stdout:8/370: truncate d6/d45/d51/d5d/f62 655605 0 2024-04-25T01:35:46.549 INFO:tasks.workunit.client.0.smithi033.stdout:4/348: creat d6/d14/d35/d3a/d4d/d5e/f60 x:0 0 0 2024-04-25T01:35:46.550 INFO:tasks.workunit.client.0.smithi033.stdout:7/334: mkdir d4/d2e/d6d 0 2024-04-25T01:35:46.551 INFO:tasks.workunit.client.0.smithi033.stdout:7/335: truncate d4/f7 281361 0 2024-04-25T01:35:46.551 INFO:tasks.workunit.client.0.smithi033.stdout:7/336: readlink d4/lc 0 2024-04-25T01:35:46.552 INFO:tasks.workunit.client.0.smithi033.stdout:8/371: rename d6/d45/d52/d2e/l3d to d6/d45/d51/d55/l83 0 2024-04-25T01:35:46.554 INFO:tasks.workunit.client.0.smithi033.stdout:4/349: mkdir d6/d14/d48/d61 0 2024-04-25T01:35:46.556 INFO:tasks.workunit.client.0.smithi033.stdout:8/372: mkdir d6/d45/d51/d55/d84 0 2024-04-25T01:35:46.558 INFO:tasks.workunit.client.0.smithi033.stdout:7/337: dwrite d4/fb [0,4194304] 0 2024-04-25T01:35:46.560 INFO:tasks.workunit.client.0.smithi033.stdout:4/350: link d6/f1f d6/d14/d35/d44/d45/d46/d57/f62 0 2024-04-25T01:35:46.561 INFO:tasks.workunit.client.0.smithi033.stdout:8/373: creat d6/d45/d51/d5d/d6e/f85 x:0 0 0 2024-04-25T01:35:46.566 INFO:tasks.workunit.client.0.smithi033.stdout:7/338: unlink d4/d23/f2b 0 2024-04-25T01:35:46.570 INFO:tasks.workunit.client.0.smithi033.stdout:7/339: write d4/d23/d3f/f4d [1468877,53350] 0 2024-04-25T01:35:46.571 INFO:tasks.workunit.client.0.smithi033.stdout:4/351: dwrite d6/f9 [0,4194304] 0 2024-04-25T01:35:46.573 INFO:tasks.workunit.client.0.smithi033.stdout:8/374: dwrite d6/d45/d52/f3b [0,4194304] 0 2024-04-25T01:35:46.573 INFO:tasks.workunit.client.0.smithi033.stdout:7/340: dread - d4/d2e/d65/f69 zero size 2024-04-25T01:35:46.577 INFO:tasks.workunit.client.0.smithi033.stdout:4/352: creat d6/d14/d1d/f63 x:0 0 0 2024-04-25T01:35:46.579 INFO:tasks.workunit.client.0.smithi033.stdout:4/353: write d6/d14/f33 [3513687,61648] 0 2024-04-25T01:35:46.580 INFO:tasks.workunit.client.0.smithi033.stdout:8/375: mknod d6/d45/d52/d2e/c86 0 2024-04-25T01:35:46.584 INFO:tasks.workunit.client.0.smithi033.stdout:7/341: creat d4/d2e/d50/f6e x:0 0 0 2024-04-25T01:35:46.587 INFO:tasks.workunit.client.0.smithi033.stdout:8/376: unlink d6/d45/d52/d2e/l53 0 2024-04-25T01:35:46.589 INFO:tasks.workunit.client.0.smithi033.stdout:4/354: mkdir d6/d14/d35/d44/d45/d46/d64 0 2024-04-25T01:35:46.591 INFO:tasks.workunit.client.0.smithi033.stdout:7/342: creat d4/d23/d3f/f6f x:0 0 0 2024-04-25T01:35:46.594 INFO:tasks.workunit.client.0.smithi033.stdout:8/377: mkdir d6/d45/d52/d2e/d87 0 2024-04-25T01:35:46.600 INFO:tasks.workunit.client.0.smithi033.stdout:8/378: symlink d6/db/l88 0 2024-04-25T01:35:46.600 INFO:tasks.workunit.client.0.smithi033.stdout:7/343: dwrite d4/d23/d3f/f60 [0,4194304] 0 2024-04-25T01:35:46.600 INFO:tasks.workunit.client.0.smithi033.stdout:8/379: dread - d6/f60 zero size 2024-04-25T01:35:46.601 INFO:tasks.workunit.client.0.smithi033.stdout:7/344: write d4/d23/d3f/f6f [267092,70833] 0 2024-04-25T01:35:46.603 INFO:tasks.workunit.client.0.smithi033.stdout:8/380: read - d6/d45/f6d zero size 2024-04-25T01:35:46.604 INFO:tasks.workunit.client.0.smithi033.stdout:8/381: write d6/f41 [660820,12891] 0 2024-04-25T01:35:46.606 INFO:tasks.workunit.client.0.smithi033.stdout:7/345: mknod d4/d2e/d6b/d1f/c70 0 2024-04-25T01:35:46.607 INFO:tasks.workunit.client.0.smithi033.stdout:8/382: dread d6/f58 [0,4194304] 0 2024-04-25T01:35:46.611 INFO:tasks.workunit.client.0.smithi033.stdout:7/346: dread d4/d23/d3f/f60 [0,4194304] 0 2024-04-25T01:35:46.612 INFO:tasks.workunit.client.0.smithi033.stdout:8/383: mkdir d6/db/d80/d89 0 2024-04-25T01:35:46.613 INFO:tasks.workunit.client.0.smithi033.stdout:8/384: dread - d6/f60 zero size 2024-04-25T01:35:46.619 INFO:tasks.workunit.client.0.smithi033.stdout:5/278: sync 2024-04-25T01:35:46.619 INFO:tasks.workunit.client.0.smithi033.stdout:7/347: link d4/d2e/d6b/f62 d4/d2e/d6b/d1f/f71 0 2024-04-25T01:35:46.620 INFO:tasks.workunit.client.0.smithi033.stdout:5/279: chown d1c/d2a/d3d/d59 0 1 2024-04-25T01:35:46.620 INFO:tasks.workunit.client.0.smithi033.stdout:7/348: fdatasync d4/d23/d3f/f4d 0 2024-04-25T01:35:46.624 INFO:tasks.workunit.client.0.smithi033.stdout:5/280: mknod d1c/d2a/d50/d52/c5a 0 2024-04-25T01:35:46.624 INFO:tasks.workunit.client.0.smithi033.stdout:8/385: dwrite d6/d45/f82 [0,4194304] 0 2024-04-25T01:35:46.625 INFO:tasks.workunit.client.0.smithi033.stdout:5/281: dread - d1c/d2a/f34 zero size 2024-04-25T01:35:46.627 INFO:tasks.workunit.client.0.smithi033.stdout:5/282: write d1c/d21/f29 [1884222,110851] 0 2024-04-25T01:35:46.629 INFO:tasks.workunit.client.0.smithi033.stdout:7/349: rename d4/d10/l1a to d4/d23/d3f/d45/d6c/l72 0 2024-04-25T01:35:46.630 INFO:tasks.workunit.client.0.smithi033.stdout:8/386: symlink d6/d45/d52/d2e/l8a 0 2024-04-25T01:35:46.633 INFO:tasks.workunit.client.0.smithi033.stdout:5/283: unlink f16 0 2024-04-25T01:35:46.634 INFO:tasks.workunit.client.0.smithi033.stdout:5/284: readlink d1c/d21/l36 0 2024-04-25T01:35:46.635 INFO:tasks.workunit.client.0.smithi033.stdout:7/350: write d4/d2e/d6b/f62 [803655,71126] 0 2024-04-25T01:35:46.637 INFO:tasks.workunit.client.0.smithi033.stdout:5/285: truncate d1c/f23 245960 0 2024-04-25T01:35:46.639 INFO:tasks.workunit.client.0.smithi033.stdout:8/387: dwrite d6/f60 [0,4194304] 0 2024-04-25T01:35:46.642 INFO:tasks.workunit.client.0.smithi033.stdout:5/286: unlink d1c/d21/l36 0 2024-04-25T01:35:46.646 INFO:tasks.workunit.client.0.smithi033.stdout:7/351: getdents d4/d57 0 2024-04-25T01:35:46.648 INFO:tasks.workunit.client.0.smithi033.stdout:5/287: mkdir d1c/d2a/d50/d5b 0 2024-04-25T01:35:46.658 INFO:tasks.workunit.client.0.smithi033.stdout:8/388: dwrite d6/d45/d51/d5d/f6f [0,4194304] 0 2024-04-25T01:35:46.658 INFO:tasks.workunit.client.0.smithi033.stdout:7/352: dread d4/d23/d3f/f4d [0,4194304] 0 2024-04-25T01:35:46.658 INFO:tasks.workunit.client.0.smithi033.stdout:5/288: rename cb to d1c/d53/c5c 0 2024-04-25T01:35:46.659 INFO:tasks.workunit.client.0.smithi033.stdout:8/389: creat d6/d45/f8b x:0 0 0 2024-04-25T01:35:46.665 INFO:tasks.workunit.client.0.smithi033.stdout:8/390: fdatasync d6/f58 0 2024-04-25T01:35:46.665 INFO:tasks.workunit.client.0.smithi033.stdout:5/289: mknod d1c/d21/d3e/c5d 0 2024-04-25T01:35:46.667 INFO:tasks.workunit.client.0.smithi033.stdout:8/391: rmdir d6/db 39 2024-04-25T01:35:46.674 INFO:tasks.workunit.client.0.smithi033.stdout:7/353: truncate d4/d23/d3f/f6f 169854 0 2024-04-25T01:35:46.675 INFO:tasks.workunit.client.0.smithi033.stdout:7/354: truncate d4/d2e/f4f 225488 0 2024-04-25T01:35:46.677 INFO:tasks.workunit.client.0.smithi033.stdout:7/355: unlink d4/f7 0 2024-04-25T01:35:46.678 INFO:tasks.workunit.client.0.smithi033.stdout:5/290: link f1b d1c/f5e 0 2024-04-25T01:35:46.681 INFO:tasks.workunit.client.0.smithi033.stdout:7/356: mkdir d4/d23/d3f/d45/d73 0 2024-04-25T01:35:46.687 INFO:tasks.workunit.client.0.smithi033.stdout:5/291: dwrite d1c/f22 [0,4194304] 0 2024-04-25T01:35:46.690 INFO:tasks.workunit.client.0.smithi033.stdout:7/357: fdatasync d4/d2e/ff 0 2024-04-25T01:35:46.693 INFO:tasks.workunit.client.0.smithi033.stdout:7/358: dread d4/d10/f3b [0,4194304] 0 2024-04-25T01:35:46.694 INFO:tasks.workunit.client.0.smithi033.stdout:5/292: symlink d1c/d21/d3e/l5f 0 2024-04-25T01:35:46.695 INFO:tasks.workunit.client.0.smithi033.stdout:7/359: chown d4/d2e/d6b/d1f/f71 0 1 2024-04-25T01:35:46.704 INFO:tasks.workunit.client.0.smithi033.stdout:7/360: getdents d4/d23/d3f 0 2024-04-25T01:35:46.707 INFO:tasks.workunit.client.0.smithi033.stdout:7/361: mknod d4/d2e/d6b/c74 0 2024-04-25T01:35:46.708 INFO:tasks.workunit.client.0.smithi033.stdout:5/293: truncate d1c/f1f 2330103 0 2024-04-25T01:35:46.710 INFO:tasks.workunit.client.0.smithi033.stdout:5/294: fsync d1c/d2a/d3d/f44 0 2024-04-25T01:35:46.723 INFO:tasks.workunit.client.0.smithi033.stdout:5/295: dwrite d1c/f1e [0,4194304] 0 2024-04-25T01:35:46.724 INFO:tasks.workunit.client.0.smithi033.stdout:5/296: dread - d1c/d21/d4a/f4e zero size 2024-04-25T01:35:46.732 INFO:tasks.workunit.client.0.smithi033.stdout:5/297: write f5 [6379999,122664] 0 2024-04-25T01:35:46.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:46 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:46.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:46 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:46.937 INFO:tasks.workunit.client.0.smithi033.stdout:9/231: sync 2024-04-25T01:35:46.937 INFO:tasks.workunit.client.0.smithi033.stdout:6/249: sync 2024-04-25T01:35:46.938 INFO:tasks.workunit.client.0.smithi033.stdout:9/232: fdatasync d0/d2/dd/f52 0 2024-04-25T01:35:46.938 INFO:tasks.workunit.client.0.smithi033.stdout:6/250: dread - d4/d15/f49 zero size 2024-04-25T01:35:46.939 INFO:tasks.workunit.client.0.smithi033.stdout:6/251: rmdir d4/d1d 39 2024-04-25T01:35:46.943 INFO:tasks.workunit.client.0.smithi033.stdout:9/233: link d0/d11/f27 d0/d2/dd/f56 0 2024-04-25T01:35:46.945 INFO:tasks.workunit.client.0.smithi033.stdout:9/234: mknod d0/d2/dd/d20/d4e/c57 0 2024-04-25T01:35:46.949 INFO:tasks.workunit.client.0.smithi033.stdout:6/252: truncate d4/f14 2119085 0 2024-04-25T01:35:46.951 INFO:tasks.workunit.client.0.smithi033.stdout:9/235: dwrite d0/d2/dd/d20/f38 [0,4194304] 0 2024-04-25T01:35:46.953 INFO:tasks.workunit.client.0.smithi033.stdout:9/236: dread - d0/d2/d43/f3f zero size 2024-04-25T01:35:46.954 INFO:tasks.workunit.client.0.smithi033.stdout:9/237: truncate d0/f29 479619 0 2024-04-25T01:35:46.954 INFO:tasks.workunit.client.0.smithi033.stdout:6/253: truncate d4/d1d/f52 395149 0 2024-04-25T01:35:46.955 INFO:tasks.workunit.client.0.smithi033.stdout:6/254: stat d4/d1d/d3e/d41/c5a 0 2024-04-25T01:35:46.955 INFO:tasks.workunit.client.0.smithi033.stdout:9/238: dread - d0/d2/dd/f15 zero size 2024-04-25T01:35:46.956 INFO:tasks.workunit.client.0.smithi033.stdout:9/239: readlink d0/d2/dd/df/d1b/l39 0 2024-04-25T01:35:46.957 INFO:tasks.workunit.client.0.smithi033.stdout:6/255: chown d4/d1d/f52 160 1 2024-04-25T01:35:46.959 INFO:tasks.workunit.client.0.smithi033.stdout:6/256: symlink d4/d24/d36/d37/l5e 0 2024-04-25T01:35:46.962 INFO:tasks.workunit.client.0.smithi033.stdout:9/240: dwrite d0/d2/d43/f3f [0,4194304] 0 2024-04-25T01:35:46.966 INFO:tasks.workunit.client.0.smithi033.stdout:6/257: rename d4/d1d/d3e/d41/d5c/f5d to d4/d15/f5f 0 2024-04-25T01:35:46.970 INFO:tasks.workunit.client.0.smithi033.stdout:9/241: dwrite d0/d2/dd/d20/f38 [0,4194304] 0 2024-04-25T01:35:46.972 INFO:tasks.workunit.client.0.smithi033.stdout:6/258: mknod d4/d1d/d3e/d41/d54/c60 0 2024-04-25T01:35:46.978 INFO:tasks.workunit.client.0.smithi033.stdout:6/259: unlink d4/d24/f3b 0 2024-04-25T01:35:46.980 INFO:tasks.workunit.client.0.smithi033.stdout:9/242: dwrite d0/d2/dd/fe [4194304,4194304] 0 2024-04-25T01:35:46.982 INFO:tasks.workunit.client.0.smithi033.stdout:6/260: mknod d4/d1d/d3e/c61 0 2024-04-25T01:35:46.983 INFO:tasks.workunit.client.0.smithi033.stdout:9/243: write d0/d2/dd/d20/f38 [5091238,125775] 0 2024-04-25T01:35:46.984 INFO:tasks.workunit.client.0.smithi033.stdout:9/244: chown d0/d2/f1f 17330 1 2024-04-25T01:35:46.985 INFO:tasks.workunit.client.0.smithi033.stdout:9/245: stat d0/d2/dd/f56 0 2024-04-25T01:35:46.986 INFO:tasks.workunit.client.0.smithi033.stdout:6/261: read d4/f12 [3434870,87074] 0 2024-04-25T01:35:46.991 INFO:tasks.workunit.client.0.smithi033.stdout:6/262: rename d4/d15/f49 to d4/d1d/d3e/f62 0 2024-04-25T01:35:46.992 INFO:tasks.workunit.client.0.smithi033.stdout:9/246: dwrite d0/d2/dd/df/f3b [0,4194304] 0 2024-04-25T01:35:46.994 INFO:tasks.workunit.client.0.smithi033.stdout:6/263: mkdir d4/d24/d63 0 2024-04-25T01:35:46.997 INFO:tasks.workunit.client.0.smithi033.stdout:9/247: mkdir d0/d2/dd/d45/d58 0 2024-04-25T01:35:47.001 INFO:tasks.workunit.client.0.smithi033.stdout:9/248: rename d0/d2/dd/df/d1b/l39 to d0/d2/d43/d4b/l59 0 2024-04-25T01:35:47.001 INFO:tasks.workunit.client.0.smithi033.stdout:9/249: stat d0/d2/d43/d4b/d55 0 2024-04-25T01:35:47.004 INFO:tasks.workunit.client.0.smithi033.stdout:9/250: creat d0/d2/dd/d20/f5a x:0 0 0 2024-04-25T01:35:47.008 INFO:tasks.workunit.client.0.smithi033.stdout:9/251: link d0/f33 d0/d2/d43/d4b/f5b 0 2024-04-25T01:35:47.010 INFO:tasks.workunit.client.0.smithi033.stdout:9/252: creat d0/d2/dd/df/f5c x:0 0 0 2024-04-25T01:35:47.043 INFO:tasks.workunit.client.0.smithi033.stdout:9/253: fsync d0/d2/dd/f13 0 2024-04-25T01:35:47.049 INFO:tasks.workunit.client.0.smithi033.stdout:9/254: fsync d0/d2/dd/f24 0 2024-04-25T01:35:47.066 INFO:tasks.workunit.client.0.smithi033.stdout:9/255: dread d0/d2/dd/df/f3a [0,4194304] 0 2024-04-25T01:35:47.067 INFO:tasks.workunit.client.0.smithi033.stdout:9/256: chown d0/d2/dd/df/d1b/d3d/d42 3802 1 2024-04-25T01:35:47.109 INFO:tasks.workunit.client.0.smithi033.stdout:0/231: sync 2024-04-25T01:35:47.111 INFO:tasks.workunit.client.0.smithi033.stdout:0/232: creat d7/d1b/f4b x:0 0 0 2024-04-25T01:35:47.113 INFO:tasks.workunit.client.0.smithi033.stdout:0/233: mkdir d7/d2d/d4c 0 2024-04-25T01:35:47.114 INFO:tasks.workunit.client.0.smithi033.stdout:0/234: mkdir d7/d2d/d4d 0 2024-04-25T01:35:47.115 INFO:tasks.workunit.client.0.smithi033.stdout:0/235: write d7/d33/f36 [200416,6302] 0 2024-04-25T01:35:47.123 INFO:tasks.workunit.client.0.smithi033.stdout:0/236: dread d7/da/df/f17 [0,4194304] 0 2024-04-25T01:35:47.127 INFO:tasks.workunit.client.0.smithi033.stdout:0/237: truncate d7/d33/f3e 1970744 0 2024-04-25T01:35:47.129 INFO:tasks.workunit.client.0.smithi033.stdout:0/238: creat d7/f4e x:0 0 0 2024-04-25T01:35:47.133 INFO:tasks.workunit.client.0.smithi033.stdout:0/239: mknod d7/d2d/d43/d44/c4f 0 2024-04-25T01:35:47.135 INFO:tasks.workunit.client.0.smithi033.stdout:0/240: fdatasync d7/f29 0 2024-04-25T01:35:47.139 INFO:tasks.workunit.client.0.smithi033.stdout:0/241: mkdir d7/da/df/d50 0 2024-04-25T01:35:47.211 INFO:tasks.workunit.client.0.smithi033.stdout:4/355: sync 2024-04-25T01:35:47.211 INFO:tasks.workunit.client.0.smithi033.stdout:1/206: sync 2024-04-25T01:35:47.211 INFO:tasks.workunit.client.0.smithi033.stdout:2/370: sync 2024-04-25T01:35:47.211 INFO:tasks.workunit.client.0.smithi033.stdout:6/264: sync 2024-04-25T01:35:47.212 INFO:tasks.workunit.client.0.smithi033.stdout:8/392: sync 2024-04-25T01:35:47.212 INFO:tasks.workunit.client.0.smithi033.stdout:7/362: sync 2024-04-25T01:35:47.212 INFO:tasks.workunit.client.0.smithi033.stdout:3/246: sync 2024-04-25T01:35:47.212 INFO:tasks.workunit.client.0.smithi033.stdout:5/298: sync 2024-04-25T01:35:47.213 INFO:tasks.workunit.client.0.smithi033.stdout:2/371: chown f8 49653 1 2024-04-25T01:35:47.214 INFO:tasks.workunit.client.0.smithi033.stdout:1/207: write d2/f26 [2431084,18678] 0 2024-04-25T01:35:47.215 INFO:tasks.workunit.client.0.smithi033.stdout:5/299: write d1c/d2a/f47 [999098,12543] 0 2024-04-25T01:35:47.215 INFO:tasks.workunit.client.0.smithi033.stdout:6/265: creat d4/d24/f64 x:0 0 0 2024-04-25T01:35:47.217 INFO:tasks.workunit.client.0.smithi033.stdout:8/393: mknod d6/d45/d51/c8c 0 2024-04-25T01:35:47.220 INFO:tasks.workunit.client.0.smithi033.stdout:4/356: mkdir d6/d14/d35/d44/d45/d46/d57/d65 0 2024-04-25T01:35:47.221 INFO:tasks.workunit.client.0.smithi033.stdout:6/266: symlink d4/d1d/d3e/l65 0 2024-04-25T01:35:47.272 INFO:tasks.workunit.client.0.smithi033.stdout:7/363: rename d4/d23/f54 to d4/d2e/d6d/f75 0 2024-04-25T01:35:47.273 INFO:tasks.workunit.client.0.smithi033.stdout:8/394: dread d6/d45/d52/f3b [0,4194304] 0 2024-04-25T01:35:47.277 INFO:tasks.workunit.client.0.smithi033.stdout:7/364: write d4/d10/f26 [3160378,80631] 0 2024-04-25T01:35:47.278 INFO:tasks.workunit.client.0.smithi033.stdout:5/300: rename d1c/d2a/d50/c55 to d1c/d21/c60 0 2024-04-25T01:35:47.279 INFO:tasks.workunit.client.0.smithi033.stdout:6/267: mknod d4/c66 0 2024-04-25T01:35:47.281 INFO:tasks.workunit.client.0.smithi033.stdout:4/357: dwrite d6/d14/d1d/d3b/f50 [4194304,4194304] 0 2024-04-25T01:35:47.284 INFO:tasks.workunit.client.0.smithi033.stdout:5/301: creat d1c/d21/d4a/f61 x:0 0 0 2024-04-25T01:35:47.286 INFO:tasks.workunit.client.0.smithi033.stdout:5/302: write d1c/d21/d4a/f61 [337132,99584] 0 2024-04-25T01:35:47.287 INFO:tasks.workunit.client.0.smithi033.stdout:5/303: dread - d1c/d21/f39 zero size 2024-04-25T01:35:47.288 INFO:tasks.workunit.client.0.smithi033.stdout:8/395: getdents d6/d45/d51/d55 0 2024-04-25T01:35:47.288 INFO:tasks.workunit.client.0.smithi033.stdout:5/304: fdatasync d1c/f1e 0 2024-04-25T01:35:47.288 INFO:tasks.workunit.client.0.smithi033.stdout:4/358: truncate d6/d14/f22 726917 0 2024-04-25T01:35:47.290 INFO:tasks.workunit.client.0.smithi033.stdout:4/359: write d6/d12/f17 [530185,38877] 0 2024-04-25T01:35:47.291 INFO:tasks.workunit.client.0.smithi033.stdout:6/268: getdents d4/d24/d63 0 2024-04-25T01:35:47.293 INFO:tasks.workunit.client.0.smithi033.stdout:5/305: creat d1c/d21/d4a/f62 x:0 0 0 2024-04-25T01:35:47.296 INFO:tasks.workunit.client.0.smithi033.stdout:8/396: rmdir d6/d45/d51/d5d/d6e 39 2024-04-25T01:35:47.297 INFO:tasks.workunit.client.0.smithi033.stdout:4/360: chown d6/d14/cb 11342042 1 2024-04-25T01:35:47.297 INFO:tasks.workunit.client.0.smithi033.stdout:3/247: dread d8/dd/f15 [0,4194304] 0 2024-04-25T01:35:47.298 INFO:tasks.workunit.client.0.smithi033.stdout:6/269: creat d4/d24/d36/d37/f67 x:0 0 0 2024-04-25T01:35:47.300 INFO:tasks.workunit.client.0.smithi033.stdout:0/242: sync 2024-04-25T01:35:47.300 INFO:tasks.workunit.client.0.smithi033.stdout:9/257: sync 2024-04-25T01:35:47.300 INFO:tasks.workunit.client.0.smithi033.stdout:5/306: unlink d1c/d21/d4a/f62 0 2024-04-25T01:35:47.301 INFO:tasks.workunit.client.0.smithi033.stdout:2/372: dread da/f34 [0,4194304] 0 2024-04-25T01:35:47.304 INFO:tasks.workunit.client.0.smithi033.stdout:9/258: chown d0/d2/f1f 114 1 2024-04-25T01:35:47.304 INFO:tasks.workunit.client.0.smithi033.stdout:6/270: symlink d4/d2a/l68 0 2024-04-25T01:35:47.305 INFO:tasks.workunit.client.0.smithi033.stdout:0/243: mkdir d7/da/df/d2a/d51 0 2024-04-25T01:35:47.305 INFO:tasks.workunit.client.0.smithi033.stdout:7/365: truncate d4/d10/f26 2412883 0 2024-04-25T01:35:47.307 INFO:tasks.workunit.client.0.smithi033.stdout:8/397: symlink d6/db/d80/l8d 0 2024-04-25T01:35:47.310 INFO:tasks.workunit.client.0.smithi033.stdout:5/307: creat d1c/d21/f63 x:0 0 0 2024-04-25T01:35:47.310 INFO:tasks.workunit.client.0.smithi033.stdout:7/366: dread - d4/d2e/d6b/f52 zero size 2024-04-25T01:35:47.313 INFO:tasks.workunit.client.0.smithi033.stdout:8/398: fsync d6/d45/d51/f79 0 2024-04-25T01:35:47.313 INFO:tasks.workunit.client.0.smithi033.stdout:2/373: dread da/d27/d31/d40/f64 [0,4194304] 0 2024-04-25T01:35:47.314 INFO:tasks.workunit.client.0.smithi033.stdout:3/248: dwrite d8/f37 [0,4194304] 0 2024-04-25T01:35:47.314 INFO:tasks.workunit.client.0.smithi033.stdout:9/259: mknod d0/d11/c5d 0 2024-04-25T01:35:47.318 INFO:tasks.workunit.client.0.smithi033.stdout:5/308: chown d1c/d2a/d3d/f3f 13 1 2024-04-25T01:35:47.318 INFO:tasks.workunit.client.0.smithi033.stdout:0/244: creat d7/da/df/d2a/d51/f52 x:0 0 0 2024-04-25T01:35:47.318 INFO:tasks.workunit.client.0.smithi033.stdout:7/367: chown d4/d2e/d6b/d67 1 1 2024-04-25T01:35:47.318 INFO:tasks.workunit.client.0.smithi033.stdout:2/374: readlink da/d50/l60 0 2024-04-25T01:35:47.319 INFO:tasks.workunit.client.0.smithi033.stdout:9/260: chown d0/d2/d43/d4b/d55 6916 1 2024-04-25T01:35:47.319 INFO:tasks.workunit.client.0.smithi033.stdout:2/375: fsync da/f42 0 2024-04-25T01:35:47.320 INFO:tasks.workunit.client.0.smithi033.stdout:4/361: dwrite d6/d12/f26 [4194304,4194304] 0 2024-04-25T01:35:47.322 INFO:tasks.workunit.client.0.smithi033.stdout:3/249: creat d8/dd/f57 x:0 0 0 2024-04-25T01:35:47.323 INFO:tasks.workunit.client.0.smithi033.stdout:2/376: fdatasync da/f18 0 2024-04-25T01:35:47.324 INFO:tasks.workunit.client.0.smithi033.stdout:8/399: write d6/d45/d51/d5d/f6a [404014,80029] 0 2024-04-25T01:35:47.326 INFO:tasks.workunit.client.0.smithi033.stdout:8/400: read d6/d45/d52/d2e/f3a [3168570,44020] 0 2024-04-25T01:35:47.328 INFO:tasks.workunit.client.0.smithi033.stdout:7/368: creat d4/d2e/d65/f76 x:0 0 0 2024-04-25T01:35:47.330 INFO:tasks.workunit.client.0.smithi033.stdout:7/369: dread - d4/f32 zero size 2024-04-25T01:35:47.331 INFO:tasks.workunit.client.0.smithi033.stdout:2/377: creat da/d2c/d30/f75 x:0 0 0 2024-04-25T01:35:47.334 INFO:tasks.workunit.client.0.smithi033.stdout:8/401: mknod d6/d45/c8e 0 2024-04-25T01:35:47.335 INFO:tasks.workunit.client.0.smithi033.stdout:0/245: link d7/da/l3c d7/da/d23/d41/l53 0 2024-04-25T01:35:47.337 INFO:tasks.workunit.client.0.smithi033.stdout:8/402: chown d6/d45/d51/d5d/f81 246385 1 2024-04-25T01:35:47.339 INFO:tasks.workunit.client.0.smithi033.stdout:3/250: mknod d8/dd/d3d/d4f/c58 0 2024-04-25T01:35:47.343 INFO:tasks.workunit.client.0.smithi033.stdout:4/362: dread d6/f8 [0,4194304] 0 2024-04-25T01:35:47.343 INFO:tasks.workunit.client.0.smithi033.stdout:5/309: dread d1c/d21/f26 [0,4194304] 0 2024-04-25T01:35:47.343 INFO:tasks.workunit.client.0.smithi033.stdout:7/370: dread d4/f3d [0,4194304] 0 2024-04-25T01:35:47.344 INFO:tasks.workunit.client.0.smithi033.stdout:8/403: creat d6/d45/d51/f8f x:0 0 0 2024-04-25T01:35:47.346 INFO:tasks.workunit.client.0.smithi033.stdout:0/246: creat d7/d2d/d4d/f54 x:0 0 0 2024-04-25T01:35:47.349 INFO:tasks.workunit.client.0.smithi033.stdout:5/310: creat d1c/d21/d4a/f64 x:0 0 0 2024-04-25T01:35:47.351 INFO:tasks.workunit.client.0.smithi033.stdout:7/371: creat d4/d23/d3f/f77 x:0 0 0 2024-04-25T01:35:47.352 INFO:tasks.workunit.client.0.smithi033.stdout:2/378: write da/d27/f59 [518770,45771] 0 2024-04-25T01:35:47.354 INFO:tasks.workunit.client.0.smithi033.stdout:3/251: creat d8/f59 x:0 0 0 2024-04-25T01:35:47.356 INFO:tasks.workunit.client.0.smithi033.stdout:8/404: symlink d6/d45/d52/l90 0 2024-04-25T01:35:47.357 INFO:tasks.workunit.client.0.smithi033.stdout:3/252: read d8/dd/d11/f24 [2222274,44378] 0 2024-04-25T01:35:47.357 INFO:tasks.workunit.client.0.smithi033.stdout:7/372: mkdir d4/d2e/d6b/d1f/d78 0 2024-04-25T01:35:47.357 INFO:tasks.workunit.client.0.smithi033.stdout:0/247: symlink d7/d2d/d4c/l55 0 2024-04-25T01:35:47.359 INFO:tasks.workunit.client.0.smithi033.stdout:2/379: mkdir da/d5b/d5e/d76 0 2024-04-25T01:35:47.361 INFO:tasks.workunit.client.0.smithi033.stdout:4/363: rename d6/l34 to d6/d14/d35/l66 0 2024-04-25T01:35:47.361 INFO:tasks.workunit.client.0.smithi033.stdout:5/311: mknod d1c/d2a/c65 0 2024-04-25T01:35:47.364 INFO:tasks.workunit.client.0.smithi033.stdout:4/364: chown d6/d14/d1d/f63 0 1 2024-04-25T01:35:47.364 INFO:tasks.workunit.client.0.smithi033.stdout:5/312: chown d1c/d2a/d50/d5b 19795995 1 2024-04-25T01:35:47.367 INFO:tasks.workunit.client.0.smithi033.stdout:3/253: dwrite d8/dd/d11/d1e/d27/f36 [0,4194304] 0 2024-04-25T01:35:47.367 INFO:tasks.workunit.client.0.smithi033.stdout:0/248: rename d7/d2d/d4d/f54 to d7/da/df/f56 0 2024-04-25T01:35:47.369 INFO:tasks.workunit.client.0.smithi033.stdout:8/405: creat d6/db/d80/d89/f91 x:0 0 0 2024-04-25T01:35:47.370 INFO:tasks.workunit.client.0.smithi033.stdout:1/208: sync 2024-04-25T01:35:47.371 INFO:tasks.workunit.client.0.smithi033.stdout:5/313: creat d1c/d2a/d50/d52/f66 x:0 0 0 2024-04-25T01:35:47.377 INFO:tasks.workunit.client.0.smithi033.stdout:3/254: mknod d8/dd/d11/d1e/d27/d4d/c5a 0 2024-04-25T01:35:47.381 INFO:tasks.workunit.client.0.smithi033.stdout:7/373: getdents d4/d23/d3f/d45 0 2024-04-25T01:35:47.382 INFO:tasks.workunit.client.0.smithi033.stdout:8/406: mknod d6/d45/d52/c92 0 2024-04-25T01:35:47.384 INFO:tasks.workunit.client.0.smithi033.stdout:8/407: fdatasync d6/d45/f57 0 2024-04-25T01:35:47.385 INFO:tasks.workunit.client.0.smithi033.stdout:3/255: dwrite d8/dd/f34 [0,4194304] 0 2024-04-25T01:35:47.387 INFO:tasks.workunit.client.0.smithi033.stdout:5/314: symlink d1c/d2a/l67 0 2024-04-25T01:35:47.388 INFO:tasks.workunit.client.0.smithi033.stdout:4/365: link d6/d12/d38/c5c d6/d12/d38/c67 0 2024-04-25T01:35:47.389 INFO:tasks.workunit.client.0.smithi033.stdout:4/366: chown d6/d14/d35/d44/d45 762922 1 2024-04-25T01:35:47.390 INFO:tasks.workunit.client.0.smithi033.stdout:7/374: creat d4/d23/d3f/f79 x:0 0 0 2024-04-25T01:35:47.391 INFO:tasks.workunit.client.0.smithi033.stdout:5/315: dread d1c/d21/d3e/f4b [0,4194304] 0 2024-04-25T01:35:47.393 INFO:tasks.workunit.client.1.smithi138.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-25T01:35:47.394 INFO:tasks.workunit.client.0.smithi033.stdout:0/249: link l6 d7/l57 0 2024-04-25T01:35:47.395 INFO:tasks.workunit.client.0.smithi033.stdout:0/250: chown d7/d2d/d43/d44/f49 16634 1 2024-04-25T01:35:47.397 INFO:tasks.workunit.client.0.smithi033.stdout:4/367: creat d6/d14/d35/d44/d45/f68 x:0 0 0 2024-04-25T01:35:47.398 INFO:tasks.workunit.client.0.smithi033.stdout:1/209: write d2/d5/f18 [1314201,28870] 0 2024-04-25T01:35:47.401 INFO:tasks.workunit.client.0.smithi033.stdout:8/408: mknod d6/d45/d52/d2e/d87/c93 0 2024-04-25T01:35:47.401 INFO:tasks.workunit.client.0.smithi033.stdout:8/409: dread - d6/d49/f5a zero size 2024-04-25T01:35:47.401 INFO:tasks.workunit.client.0.smithi033.stdout:3/256: symlink d8/dd/d11/l5b 0 2024-04-25T01:35:47.403 INFO:tasks.workunit.client.0.smithi033.stdout:0/251: write d7/d1b/f4b [419606,95297] 0 2024-04-25T01:35:47.404 INFO:tasks.workunit.client.0.smithi033.stdout:1/210: rmdir d2/d11/d28/d33 39 2024-04-25T01:35:47.408 INFO:tasks.workunit.client.0.smithi033.stdout:7/375: creat d4/f7a x:0 0 0 2024-04-25T01:35:47.409 INFO:tasks.workunit.client.0.smithi033.stdout:0/252: unlink d7/da/f13 0 2024-04-25T01:35:47.412 INFO:tasks.workunit.client.0.smithi033.stdout:7/376: truncate d4/d10/f29 453421 0 2024-04-25T01:35:47.414 INFO:tasks.workunit.client.0.smithi033.stdout:5/316: truncate f7 2501885 0 2024-04-25T01:35:47.414 INFO:tasks.workunit.client.0.smithi033.stdout:8/410: dwrite d6/db/f46 [0,4194304] 0 2024-04-25T01:35:47.417 INFO:tasks.workunit.client.0.smithi033.stdout:4/368: write d6/d12/d38/f41 [253529,107475] 0 2024-04-25T01:35:47.422 INFO:tasks.workunit.client.0.smithi033.stdout:0/253: rename d7/da/d23/f42 to d7/da/f58 0 2024-04-25T01:35:47.423 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-25T01:35:47.424 INFO:tasks.workunit.client.1.smithi138.stderr:+ make 2024-04-25T01:35:47.424 INFO:tasks.workunit.client.0.smithi033.stdout:1/211: creat d2/d11/d28/d33/f3e x:0 0 0 2024-04-25T01:35:47.425 INFO:tasks.workunit.client.0.smithi033.stdout:1/212: chown d2/d11/f1e 10150 1 2024-04-25T01:35:47.425 INFO:tasks.workunit.client.0.smithi033.stdout:3/257: creat d8/f5c x:0 0 0 2024-04-25T01:35:47.429 INFO:tasks.workunit.client.0.smithi033.stdout:0/254: rename d7/da/df/d2a/f35 to d7/da/d23/d41/f59 0 2024-04-25T01:35:47.431 INFO:tasks.workunit.client.0.smithi033.stdout:7/377: symlink d4/d2e/d6b/d1f/d78/l7b 0 2024-04-25T01:35:47.432 INFO:tasks.workunit.client.0.smithi033.stdout:1/213: dwrite d2/d5/f25 [0,4194304] 0 2024-04-25T01:35:47.432 INFO:tasks.workunit.client.0.smithi033.stdout:5/317: symlink d1c/d53/l68 0 2024-04-25T01:35:47.434 INFO:tasks.workunit.client.0.smithi033.stdout:8/411: link d6/d45/f6d d6/d45/d51/f94 0 2024-04-25T01:35:47.434 INFO:tasks.workunit.client.0.smithi033.stdout:5/318: write d1c/d2a/d3d/f4d [692873,70809] 0 2024-04-25T01:35:47.435 INFO:tasks.workunit.client.0.smithi033.stdout:4/369: creat d6/d14/d35/d44/d45/d46/d64/f69 x:0 0 0 2024-04-25T01:35:47.439 INFO:tasks.workunit.client.0.smithi033.stdout:3/258: rename d8/lb to d8/dd/d3d/d4f/l5d 0 2024-04-25T01:35:47.440 INFO:tasks.workunit.client.0.smithi033.stdout:7/378: creat d4/d23/d3f/d45/f7c x:0 0 0 2024-04-25T01:35:47.441 INFO:tasks.workunit.client.0.smithi033.stdout:7/379: stat d4/d2e/d6b/f56 0 2024-04-25T01:35:47.441 INFO:tasks.workunit.client.0.smithi033.stdout:1/214: unlink d2/d5/f25 0 2024-04-25T01:35:47.445 INFO:tasks.workunit.client.0.smithi033.stdout:8/412: mkdir d6/d45/d52/d2e/d95 0 2024-04-25T01:35:47.447 INFO:tasks.workunit.client.0.smithi033.stdout:8/413: readlink d6/d45/d52/d2e/l71 0 2024-04-25T01:35:47.448 INFO:tasks.workunit.client.0.smithi033.stdout:4/370: symlink d6/l6a 0 2024-04-25T01:35:47.449 INFO:tasks.workunit.client.0.smithi033.stdout:3/259: symlink d8/l5e 0 2024-04-25T01:35:47.450 INFO:tasks.workunit.client.0.smithi033.stdout:1/215: dwrite d2/f32 [0,4194304] 0 2024-04-25T01:35:47.451 INFO:tasks.workunit.client.0.smithi033.stdout:1/216: dread - d2/d5/f3c zero size 2024-04-25T01:35:47.452 INFO:tasks.workunit.client.0.smithi033.stdout:0/255: truncate d7/d33/f3e 1736393 0 2024-04-25T01:35:47.453 INFO:tasks.workunit.client.0.smithi033.stdout:7/380: link d4/d2e/d6b/d1f/f42 d4/d2e/d6b/d67/f7d 0 2024-04-25T01:35:47.453 INFO:tasks.workunit.client.0.smithi033.stdout:0/256: write d7/d2d/f3d [613156,11178] 0 2024-04-25T01:35:47.453 INFO:tasks.workunit.client.0.smithi033.stdout:1/217: dread - d2/d11/d28/f2c zero size 2024-04-25T01:35:47.455 INFO:tasks.workunit.client.0.smithi033.stdout:4/371: rmdir d6/d12/d38 39 2024-04-25T01:35:47.456 INFO:tasks.workunit.client.0.smithi033.stdout:1/218: dread - d2/d11/f30 zero size 2024-04-25T01:35:47.458 INFO:tasks.workunit.client.0.smithi033.stdout:7/381: fdatasync d4/d10/f3b 0 2024-04-25T01:35:47.459 INFO:tasks.workunit.client.0.smithi033.stdout:7/382: chown d4/d2e/d6b/f62 197 1 2024-04-25T01:35:47.460 INFO:tasks.workunit.client.0.smithi033.stdout:4/372: symlink d6/d14/d48/l6b 0 2024-04-25T01:35:47.463 INFO:tasks.workunit.client.0.smithi033.stdout:3/260: link d8/dd/f57 d8/dd/d1d/f5f 0 2024-04-25T01:35:47.465 INFO:tasks.workunit.client.0.smithi033.stdout:0/257: rename d7/f29 to d7/da/d23/f5a 0 2024-04-25T01:35:47.466 INFO:tasks.workunit.client.0.smithi033.stdout:0/258: stat d7/d2d/c2f 0 2024-04-25T01:35:47.466 INFO:tasks.workunit.client.0.smithi033.stdout:0/259: readlink d7/da/d23/l2b 0 2024-04-25T01:35:47.467 INFO:tasks.workunit.client.0.smithi033.stdout:1/219: symlink d2/d2e/l3f 0 2024-04-25T01:35:47.468 INFO:tasks.workunit.client.0.smithi033.stdout:0/260: write d7/da/d23/f3a [421573,34596] 0 2024-04-25T01:35:47.474 INFO:tasks.workunit.client.0.smithi033.stdout:7/383: symlink d4/d23/d3f/d45/d73/l7e 0 2024-04-25T01:35:47.474 INFO:tasks.workunit.client.0.smithi033.stdout:1/220: creat d2/d5/d1b/f40 x:0 0 0 2024-04-25T01:35:47.475 INFO:tasks.workunit.client.0.smithi033.stdout:1/221: chown d2/d2e/d34 2472759 1 2024-04-25T01:35:47.478 INFO:tasks.workunit.client.0.smithi033.stdout:3/261: symlink d8/dd/d11/d1e/l60 0 2024-04-25T01:35:47.479 INFO:tasks.workunit.client.0.smithi033.stdout:3/262: fsync d8/dd/d40/f43 0 2024-04-25T01:35:47.481 INFO:tasks.workunit.client.0.smithi033.stdout:5/319: fsync f7 0 2024-04-25T01:35:47.482 INFO:tasks.workunit.client.0.smithi033.stdout:1/222: write d2/d11/d28/d33/f36 [430165,2289] 0 2024-04-25T01:35:47.484 INFO:tasks.workunit.client.0.smithi033.stdout:0/261: rename d7/da/l3c to d7/da/d23/l5b 0 2024-04-25T01:35:47.484 INFO:tasks.workunit.client.0.smithi033.stdout:7/384: dread d4/f11 [0,4194304] 0 2024-04-25T01:35:47.488 INFO:tasks.workunit.client.0.smithi033.stdout:4/373: getdents d6/d14/d35/d39 0 2024-04-25T01:35:47.489 INFO:tasks.workunit.client.0.smithi033.stdout:5/320: mkdir d1c/d21/d4a/d69 0 2024-04-25T01:35:47.493 INFO:tasks.workunit.client.0.smithi033.stdout:7/385: dwrite d4/d2e/d6b/f33 [0,4194304] 0 2024-04-25T01:35:47.498 INFO:tasks.workunit.client.0.smithi033.stdout:3/263: mknod d8/dd/d11/c61 0 2024-04-25T01:35:47.500 INFO:tasks.workunit.client.0.smithi033.stdout:4/374: symlink d6/l6c 0 2024-04-25T01:35:47.501 INFO:tasks.workunit.client.0.smithi033.stdout:7/386: mkdir d4/d2e/d7f 0 2024-04-25T01:35:47.504 INFO:tasks.workunit.client.0.smithi033.stdout:4/375: dread d6/d14/d1d/d3b/f52 [0,4194304] 0 2024-04-25T01:35:47.505 INFO:tasks.workunit.client.0.smithi033.stdout:1/223: rename d2/d11/d28/l37 to d2/d11/l41 0 2024-04-25T01:35:47.516 INFO:tasks.workunit.client.0.smithi033.stdout:0/262: dwrite d7/da/df/d14/f22 [0,4194304] 0 2024-04-25T01:35:47.520 INFO:tasks.workunit.client.0.smithi033.stdout:5/321: truncate d1c/f2b 2004573 0 2024-04-25T01:35:47.521 INFO:tasks.workunit.client.0.smithi033.stdout:4/376: rmdir d6/d14/d48/d61 0 2024-04-25T01:35:47.528 INFO:tasks.workunit.client.0.smithi033.stdout:5/322: dread d1c/d2a/d3d/f4d [4194304,4194304] 0 2024-04-25T01:35:47.528 INFO:tasks.workunit.client.0.smithi033.stdout:0/263: rename d7/da/d23/f3a to d7/d2d/d4d/f5c 0 2024-04-25T01:35:47.529 INFO:tasks.workunit.client.0.smithi033.stdout:7/387: write d4/d2e/d6b/d1f/f24 [2454343,23688] 0 2024-04-25T01:35:47.529 INFO:tasks.workunit.client.0.smithi033.stdout:3/264: dwrite d8/dd/d1d/f5f [0,4194304] 0 2024-04-25T01:35:47.531 INFO:tasks.workunit.client.0.smithi033.stdout:5/323: chown c12 365 1 2024-04-25T01:35:47.531 INFO:tasks.workunit.client.0.smithi033.stdout:0/264: fsync d7/da/df/f28 0 2024-04-25T01:35:47.532 INFO:tasks.workunit.client.0.smithi033.stdout:3/265: fdatasync d8/dd/d11/d1e/d27/f48 0 2024-04-25T01:35:47.537 INFO:tasks.workunit.client.0.smithi033.stdout:1/224: link d2/d11/f1e d2/d11/f42 0 2024-04-25T01:35:47.542 INFO:tasks.workunit.client.0.smithi033.stdout:0/265: dread d7/d2d/f3d [0,4194304] 0 2024-04-25T01:35:47.543 INFO:tasks.workunit.client.0.smithi033.stdout:7/388: mknod d4/d2e/d6b/d1f/c80 0 2024-04-25T01:35:47.544 INFO:tasks.workunit.client.0.smithi033.stdout:7/389: chown d4/d2e/f4c 3 1 2024-04-25T01:35:47.544 INFO:tasks.workunit.client.0.smithi033.stdout:7/390: stat d4/d2e/d50/f55 0 2024-04-25T01:35:47.545 INFO:tasks.workunit.client.0.smithi033.stdout:3/266: creat d8/dd/d11/d1e/d27/d4d/f62 x:0 0 0 2024-04-25T01:35:47.546 INFO:tasks.workunit.client.0.smithi033.stdout:3/267: stat d8/dd/d3d/d4f 0 2024-04-25T01:35:47.547 INFO:tasks.workunit.client.0.smithi033.stdout:1/225: symlink d2/d2e/l43 0 2024-04-25T01:35:47.549 INFO:tasks.workunit.client.0.smithi033.stdout:0/266: creat d7/da/f5d x:0 0 0 2024-04-25T01:35:47.551 INFO:tasks.workunit.client.0.smithi033.stdout:7/391: write d4/f11 [3245723,99003] 0 2024-04-25T01:35:47.553 INFO:tasks.workunit.client.0.smithi033.stdout:4/377: dread d6/d12/f1a [0,4194304] 0 2024-04-25T01:35:47.554 INFO:tasks.workunit.client.0.smithi033.stdout:3/268: mkdir d8/dd/d3d/d63 0 2024-04-25T01:35:47.555 INFO:tasks.workunit.client.0.smithi033.stdout:5/324: link d1c/f32 d1c/d2a/d50/f6a 0 2024-04-25T01:35:47.556 INFO:tasks.workunit.client.0.smithi033.stdout:0/267: chown d7/l57 47204570 1 2024-04-25T01:35:47.557 INFO:tasks.workunit.client.0.smithi033.stdout:7/392: mknod d4/d23/c81 0 2024-04-25T01:35:47.559 INFO:tasks.workunit.client.0.smithi033.stdout:5/325: mknod d1c/d21/d4a/c6b 0 2024-04-25T01:35:47.560 INFO:tasks.workunit.client.0.smithi033.stdout:0/268: chown d7/d33/f3e 14821 1 2024-04-25T01:35:47.561 INFO:tasks.workunit.client.0.smithi033.stdout:4/378: mkdir d6/d14/d6d 0 2024-04-25T01:35:47.563 INFO:tasks.workunit.client.0.smithi033.stdout:5/326: creat d1c/d2a/d50/d52/f6c x:0 0 0 2024-04-25T01:35:47.563 INFO:tasks.workunit.client.0.smithi033.stdout:4/379: fdatasync d6/d14/d35/d3a/f56 0 2024-04-25T01:35:47.563 INFO:tasks.workunit.client.0.smithi033.stdout:1/226: dread d2/f26 [0,4194304] 0 2024-04-25T01:35:47.564 INFO:tasks.workunit.client.0.smithi033.stdout:3/269: rename d8/dd/f2f to d8/dd/d11/d1e/d27/f64 0 2024-04-25T01:35:47.566 INFO:tasks.workunit.client.0.smithi033.stdout:5/327: rmdir d1c/d21/d3e 39 2024-04-25T01:35:47.567 INFO:tasks.workunit.client.0.smithi033.stdout:5/328: stat c8 0 2024-04-25T01:35:47.567 INFO:tasks.workunit.client.0.smithi033.stdout:3/270: write d8/dd/d11/d1e/d27/d4d/f62 [210738,130047] 0 2024-04-25T01:35:47.567 INFO:tasks.workunit.client.0.smithi033.stdout:4/380: write d6/d14/d35/d3a/d4d/d5e/f60 [274515,14041] 0 2024-04-25T01:35:47.570 INFO:tasks.workunit.client.0.smithi033.stdout:0/269: dread d7/da/df/f19 [0,4194304] 0 2024-04-25T01:35:47.571 INFO:tasks.workunit.client.0.smithi033.stdout:3/271: fsync d8/dd/d40/f4e 0 2024-04-25T01:35:47.571 INFO:tasks.workunit.client.0.smithi033.stdout:3/272: stat d8/l5e 0 2024-04-25T01:35:47.573 INFO:tasks.workunit.client.0.smithi033.stdout:3/273: write d8/dd/d40/f43 [1008385,108866] 0 2024-04-25T01:35:47.575 INFO:tasks.workunit.client.0.smithi033.stdout:1/227: getdents d2/d2e 0 2024-04-25T01:35:47.577 INFO:tasks.workunit.client.0.smithi033.stdout:5/329: creat d1c/d21/f6d x:0 0 0 2024-04-25T01:35:47.586 INFO:tasks.workunit.client.0.smithi033.stdout:4/381: dread d6/d12/f17 [0,4194304] 0 2024-04-25T01:35:47.587 INFO:tasks.workunit.client.0.smithi033.stdout:1/228: dwrite d2/d11/d28/d33/f36 [4194304,4194304] 0 2024-04-25T01:35:47.588 INFO:tasks.workunit.client.0.smithi033.stdout:5/330: dwrite d1c/d21/f63 [0,4194304] 0 2024-04-25T01:35:47.588 INFO:tasks.workunit.client.0.smithi033.stdout:1/229: chown d2/d5/d1b/l27 132478105 1 2024-04-25T01:35:47.591 INFO:tasks.workunit.client.0.smithi033.stdout:4/382: symlink d6/d14/d1d/d3b/l6e 0 2024-04-25T01:35:47.592 INFO:tasks.workunit.client.0.smithi033.stdout:3/274: dread d8/fa [0,4194304] 0 2024-04-25T01:35:47.593 INFO:tasks.workunit.client.0.smithi033.stdout:4/383: write d6/d14/f53 [598450,14749] 0 2024-04-25T01:35:47.594 INFO:tasks.workunit.client.0.smithi033.stdout:1/230: chown d2/d11/f1e 24678026 1 2024-04-25T01:35:47.594 INFO:tasks.workunit.client.0.smithi033.stdout:1/231: stat d2/d11/f1e 0 2024-04-25T01:35:47.597 INFO:tasks.workunit.client.0.smithi033.stdout:5/331: dread d1c/d2a/f47 [0,4194304] 0 2024-04-25T01:35:47.598 INFO:tasks.workunit.client.0.smithi033.stdout:5/332: chown d1c/d2a 988 1 2024-04-25T01:35:47.598 INFO:tasks.workunit.client.0.smithi033.stdout:1/232: symlink d2/d11/d28/l44 0 2024-04-25T01:35:47.604 INFO:tasks.workunit.client.0.smithi033.stdout:3/275: dwrite d8/dd/d11/d1e/d27/d4d/f62 [0,4194304] 0 2024-04-25T01:35:47.609 INFO:tasks.workunit.client.0.smithi033.stdout:5/333: dwrite d1c/f51 [0,4194304] 0 2024-04-25T01:35:47.609 INFO:tasks.workunit.client.0.smithi033.stdout:3/276: truncate d8/dd/d11/d1e/d27/f48 832135 0 2024-04-25T01:35:47.617 INFO:tasks.workunit.client.0.smithi033.stdout:5/334: dwrite f5 [4194304,4194304] 0 2024-04-25T01:35:47.630 INFO:tasks.workunit.client.0.smithi033.stdout:1/233: dread d2/d11/d28/d33/f36 [0,4194304] 0 2024-04-25T01:35:47.631 INFO:tasks.workunit.client.0.smithi033.stdout:1/234: truncate d2/f26 4985952 0 2024-04-25T01:35:47.634 INFO:tasks.workunit.client.0.smithi033.stdout:1/235: dread d2/f26 [0,4194304] 0 2024-04-25T01:35:47.635 INFO:tasks.workunit.client.0.smithi033.stdout:1/236: write d2/d11/f1a [3995355,73607] 0 2024-04-25T01:35:47.675 INFO:tasks.workunit.client.0.smithi033.stdout:6/271: sync 2024-04-25T01:35:47.675 INFO:tasks.workunit.client.0.smithi033.stdout:2/380: sync 2024-04-25T01:35:47.675 INFO:tasks.workunit.client.0.smithi033.stdout:9/261: sync 2024-04-25T01:35:47.676 INFO:tasks.workunit.client.0.smithi033.stdout:6/272: chown d4/d24/d63 16541 1 2024-04-25T01:35:47.676 INFO:tasks.workunit.client.0.smithi033.stdout:2/381: fsync da/d50/d6a/f56 0 2024-04-25T01:35:47.676 INFO:tasks.workunit.client.0.smithi033.stdout:9/262: stat d0/d2/dd/df/d1b/d3d/d42/f49 0 2024-04-25T01:35:47.677 INFO:tasks.workunit.client.0.smithi033.stdout:9/263: dread - d0/d2/dd/df/f2b zero size 2024-04-25T01:35:47.678 INFO:tasks.workunit.client.0.smithi033.stdout:9/264: chown d0/d2/dd/df 170 1 2024-04-25T01:35:47.683 INFO:tasks.workunit.client.0.smithi033.stdout:9/265: dread d0/d2/dd/f41 [0,4194304] 0 2024-04-25T01:35:47.698 INFO:tasks.workunit.client.0.smithi033.stdout:9/266: dread d0/d2/dd/f13 [4194304,4194304] 0 2024-04-25T01:35:47.699 INFO:tasks.workunit.client.0.smithi033.stdout:9/267: read d0/f29 [356680,98687] 0 2024-04-25T01:35:47.710 INFO:tasks.workunit.client.0.smithi033.stdout:3/277: rmdir d8/dd/d11 39 2024-04-25T01:35:47.712 INFO:tasks.workunit.client.0.smithi033.stdout:5/335: write d1c/f37 [187166,73911] 0 2024-04-25T01:35:47.715 INFO:tasks.workunit.client.0.smithi033.stdout:1/237: rename d2/d11/l3b to d2/d5/d1b/l45 0 2024-04-25T01:35:47.716 INFO:tasks.workunit.client.0.smithi033.stdout:1/238: fdatasync d2/d5/f3c 0 2024-04-25T01:35:47.717 INFO:tasks.workunit.client.0.smithi033.stdout:6/273: creat d4/d1d/f69 x:0 0 0 2024-04-25T01:35:47.719 INFO:tasks.workunit.client.0.smithi033.stdout:2/382: mkdir da/d27/d31/d40/d77 0 2024-04-25T01:35:47.719 INFO:tasks.workunit.client.0.smithi033.stdout:6/274: write d4/d2a/f3f [4176214,96634] 0 2024-04-25T01:35:47.719 INFO:tasks.workunit.client.0.smithi033.stdout:1/239: dread d2/d11/f2a [0,4194304] 0 2024-04-25T01:35:47.725 INFO:tasks.workunit.client.0.smithi033.stdout:3/278: fsync d8/dd/d11/d1e/f3b 0 2024-04-25T01:35:47.725 INFO:tasks.workunit.client.0.smithi033.stdout:5/336: mknod d1c/d2a/d3d/c6e 0 2024-04-25T01:35:47.735 INFO:tasks.workunit.client.0.smithi033.stdout:3/279: rename d8/dd/f57 to d8/dd/d40/f65 0 2024-04-25T01:35:47.736 INFO:tasks.workunit.client.0.smithi033.stdout:2/383: mknod da/d2c/c78 0 2024-04-25T01:35:47.736 INFO:tasks.workunit.client.0.smithi033.stdout:9/268: dwrite d0/f9 [0,4194304] 0 2024-04-25T01:35:47.738 INFO:tasks.workunit.client.0.smithi033.stdout:6/275: dread f3 [4194304,4194304] 0 2024-04-25T01:35:47.740 INFO:tasks.workunit.client.0.smithi033.stdout:1/240: dwrite d2/f20 [0,4194304] 0 2024-04-25T01:35:47.741 INFO:tasks.workunit.client.0.smithi033.stdout:3/280: write d8/dd/d11/d1e/f29 [3808219,24490] 0 2024-04-25T01:35:47.741 INFO:tasks.workunit.client.0.smithi033.stdout:5/337: creat d1c/f6f x:0 0 0 2024-04-25T01:35:47.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:47 smithi033 ceph-mon[26353]: pgmap v9: 65 pgs: 65 active+clean; 778 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 12 MiB/s rd, 89 MiB/s wr, 390 op/s 2024-04-25T01:35:47.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:47 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:47.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:47 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:47.743 INFO:tasks.workunit.client.0.smithi033.stdout:2/384: chown da/d50/d6a/d43/d6c/l4d 3169233 1 2024-04-25T01:35:47.746 INFO:tasks.workunit.client.0.smithi033.stdout:9/269: symlink d0/d2/d43/d4b/l5e 0 2024-04-25T01:35:47.746 INFO:tasks.workunit.client.0.smithi033.stdout:1/241: mkdir d2/d5/d1b/d46 0 2024-04-25T01:35:47.752 INFO:tasks.workunit.client.0.smithi033.stdout:1/242: dread d2/d11/f2a [0,4194304] 0 2024-04-25T01:35:47.752 INFO:tasks.workunit.client.0.smithi033.stdout:5/338: dread d1c/d21/d3e/f4b [0,4194304] 0 2024-04-25T01:35:47.753 INFO:tasks.workunit.client.0.smithi033.stdout:3/281: unlink d8/dd/d11/d1e/d27/f2e 0 2024-04-25T01:35:47.754 INFO:tasks.workunit.client.0.smithi033.stdout:9/270: mkdir d0/d2/dd/d5f 0 2024-04-25T01:35:47.754 INFO:tasks.workunit.client.0.smithi033.stdout:6/276: dwrite d4/f2d [4194304,4194304] 0 2024-04-25T01:35:47.755 INFO:tasks.workunit.client.0.smithi033.stdout:3/282: chown d8/dd/d11/d1e/l22 0 1 2024-04-25T01:35:47.756 INFO:tasks.workunit.client.0.smithi033.stdout:6/277: write d4/d2a/f3f [318356,20830] 0 2024-04-25T01:35:47.756 INFO:tasks.workunit.client.0.smithi033.stdout:5/339: mkdir d1c/d21/d3e/d70 0 2024-04-25T01:35:47.759 INFO:tasks.workunit.client.0.smithi033.stdout:9/271: write d0/d2/d43/f3c [81167,49056] 0 2024-04-25T01:35:47.760 INFO:tasks.workunit.client.0.smithi033.stdout:6/278: rmdir d4/d1d/d3e/d41 39 2024-04-25T01:35:47.761 INFO:tasks.workunit.client.0.smithi033.stdout:6/279: dread - d4/d1d/f69 zero size 2024-04-25T01:35:47.762 INFO:tasks.workunit.client.0.smithi033.stdout:6/280: dread - d4/d24/d36/d37/f67 zero size 2024-04-25T01:35:47.766 INFO:tasks.workunit.client.0.smithi033.stdout:9/272: mknod d0/d11/c60 0 2024-04-25T01:35:47.766 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:47 smithi138 ceph-mon[32977]: pgmap v9: 65 pgs: 65 active+clean; 778 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 12 MiB/s rd, 89 MiB/s wr, 390 op/s 2024-04-25T01:35:47.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:47 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:47.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:47 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:35:47.768 INFO:tasks.workunit.client.0.smithi033.stdout:1/243: getdents d2/d2e/d34 0 2024-04-25T01:35:47.769 INFO:tasks.workunit.client.0.smithi033.stdout:1/244: chown d2/d2e/l3f 51075838 1 2024-04-25T01:35:47.770 INFO:tasks.workunit.client.0.smithi033.stdout:9/273: read d0/d11/f27 [1052387,81334] 0 2024-04-25T01:35:47.771 INFO:tasks.workunit.client.0.smithi033.stdout:6/281: unlink d4/d1d/f4b 0 2024-04-25T01:35:47.771 INFO:tasks.workunit.client.0.smithi033.stdout:6/282: read - d4/d1d/d3e/f62 zero size 2024-04-25T01:35:47.775 INFO:tasks.workunit.client.0.smithi033.stdout:2/385: dread da/d2c/f48 [0,4194304] 0 2024-04-25T01:35:47.779 INFO:tasks.workunit.client.0.smithi033.stdout:1/245: rename d2/d5/f3c to d2/d5/d1b/d46/f47 0 2024-04-25T01:35:47.779 INFO:tasks.workunit.client.0.smithi033.stdout:1/246: chown d2/d5/d1b 1964325 1 2024-04-25T01:35:47.782 INFO:tasks.workunit.client.0.smithi033.stdout:9/274: write d0/d2/dd/df/f3a [4864741,31622] 0 2024-04-25T01:35:47.783 INFO:tasks.workunit.client.0.smithi033.stdout:3/283: link d8/dd/d1d/c3a d8/dd/d11/c66 0 2024-04-25T01:35:47.785 INFO:tasks.workunit.client.0.smithi033.stdout:2/386: creat da/d5b/f79 x:0 0 0 2024-04-25T01:35:47.788 INFO:tasks.workunit.client.0.smithi033.stdout:1/247: symlink d2/d5/l48 0 2024-04-25T01:35:47.788 INFO:tasks.workunit.client.0.smithi033.stdout:5/340: link l19 d1c/d21/l71 0 2024-04-25T01:35:47.790 INFO:tasks.workunit.client.0.smithi033.stdout:9/275: mknod d0/d2/dd/df/d1b/d3d/d42/c61 0 2024-04-25T01:35:47.790 INFO:tasks.workunit.client.0.smithi033.stdout:9/276: dread - d0/d2/f48 zero size 2024-04-25T01:35:47.791 INFO:tasks.workunit.client.0.smithi033.stdout:2/387: creat da/d27/f7a x:0 0 0 2024-04-25T01:35:47.793 INFO:tasks.workunit.client.0.smithi033.stdout:2/388: write da/f22 [154574,26489] 0 2024-04-25T01:35:47.794 INFO:tasks.workunit.client.0.smithi033.stdout:9/277: read d0/f23 [2558378,86099] 0 2024-04-25T01:35:47.795 INFO:tasks.workunit.client.0.smithi033.stdout:2/389: readlink da/d2c/l4e 0 2024-04-25T01:35:47.795 INFO:tasks.workunit.client.0.smithi033.stdout:3/284: rename d8/dd/le to d8/dd/d3d/d63/l67 0 2024-04-25T01:35:47.803 INFO:tasks.workunit.client.0.smithi033.stdout:5/341: write d1c/f32 [2557011,124706] 0 2024-04-25T01:35:47.806 INFO:tasks.workunit.client.0.smithi033.stdout:9/278: dwrite d0/d2/dd/f41 [0,4194304] 0 2024-04-25T01:35:47.807 INFO:tasks.workunit.client.0.smithi033.stdout:1/248: dread d2/d5/f14 [0,4194304] 0 2024-04-25T01:35:47.808 INFO:tasks.workunit.client.0.smithi033.stdout:2/390: dwrite da/d27/f6d [0,4194304] 0 2024-04-25T01:35:47.810 INFO:tasks.workunit.client.0.smithi033.stdout:1/249: chown d2/d11/f1e 4103740 1 2024-04-25T01:35:47.811 INFO:tasks.workunit.client.0.smithi033.stdout:5/342: rename d1c/d21/f29 to d1c/d21/d4a/f72 0 2024-04-25T01:35:47.812 INFO:tasks.workunit.client.0.smithi033.stdout:9/279: mknod d0/d2/d43/d4b/c62 0 2024-04-25T01:35:47.817 INFO:tasks.workunit.client.0.smithi033.stdout:1/250: mkdir d2/d2e/d49 0 2024-04-25T01:35:47.818 INFO:tasks.workunit.client.0.smithi033.stdout:1/251: readlink d2/d5/l35 0 2024-04-25T01:35:47.819 INFO:tasks.workunit.client.0.smithi033.stdout:9/280: rename d0/d2/dd/df/d1b/d3d/d42/f49 to d0/d2/d43/d4b/f63 0 2024-04-25T01:35:47.820 INFO:tasks.workunit.client.0.smithi033.stdout:2/391: creat da/d27/d31/d40/d77/f7b x:0 0 0 2024-04-25T01:35:47.820 INFO:tasks.workunit.client.0.smithi033.stdout:5/343: getdents d1c/d21/d3e/d70 0 2024-04-25T01:35:47.822 INFO:tasks.workunit.client.0.smithi033.stdout:1/252: unlink d2/d5/c1f 0 2024-04-25T01:35:47.822 INFO:tasks.workunit.client.0.smithi033.stdout:5/344: write d1c/d21/f2c [2489575,98434] 0 2024-04-25T01:35:47.827 INFO:tasks.workunit.client.0.smithi033.stdout:1/253: truncate d2/d11/f1e 1394671 0 2024-04-25T01:35:47.827 INFO:tasks.workunit.client.0.smithi033.stdout:9/281: dread d0/d2/dd/f13 [4194304,4194304] 0 2024-04-25T01:35:47.830 INFO:tasks.workunit.client.0.smithi033.stdout:2/392: dwrite da/d27/f69 [0,4194304] 0 2024-04-25T01:35:47.834 INFO:tasks.workunit.client.0.smithi033.stdout:9/282: mknod d0/d2/dd/d5f/c64 0 2024-04-25T01:35:47.834 INFO:tasks.workunit.client.0.smithi033.stdout:5/345: dwrite d1c/d21/d4a/f61 [0,4194304] 0 2024-04-25T01:35:47.835 INFO:tasks.workunit.client.0.smithi033.stdout:2/393: rmdir da/d2c/d30 39 2024-04-25T01:35:47.866 INFO:tasks.workunit.client.0.smithi033.stdout:8/414: sync 2024-04-25T01:35:47.867 INFO:tasks.workunit.client.0.smithi033.stdout:9/283: mknod d0/d2/dd/d45/d58/c65 0 2024-04-25T01:35:47.867 INFO:tasks.workunit.client.0.smithi033.stdout:8/415: fdatasync d6/d45/d51/d55/f5c 0 2024-04-25T01:35:47.868 INFO:tasks.workunit.client.0.smithi033.stdout:8/416: dread - d6/d45/d51/f94 zero size 2024-04-25T01:35:47.870 INFO:tasks.workunit.client.0.smithi033.stdout:2/394: rename da/d32 to da/d5b/d5e/d7c 0 2024-04-25T01:35:47.873 INFO:tasks.workunit.client.0.smithi033.stdout:9/284: creat d0/d2/dd/d45/f66 x:0 0 0 2024-04-25T01:35:47.874 INFO:tasks.workunit.client.0.smithi033.stdout:9/285: write d0/d2/dd/df/f3e [805469,25741] 0 2024-04-25T01:35:47.876 INFO:tasks.workunit.client.0.smithi033.stdout:9/286: write d0/d2/f1f [2827372,78773] 0 2024-04-25T01:35:47.882 INFO:tasks.workunit.client.0.smithi033.stdout:2/395: write da/d2c/d30/f3b [5018444,41777] 0 2024-04-25T01:35:47.882 INFO:tasks.workunit.client.0.smithi033.stdout:5/346: truncate fc 3123247 0 2024-04-25T01:35:47.884 INFO:tasks.workunit.client.0.smithi033.stdout:5/347: readlink d1c/l25 0 2024-04-25T01:35:47.887 INFO:tasks.workunit.client.0.smithi033.stdout:1/254: dwrite d2/d11/f42 [0,4194304] 0 2024-04-25T01:35:47.889 INFO:tasks.workunit.client.0.smithi033.stdout:8/417: truncate d6/f41 597228 0 2024-04-25T01:35:47.890 INFO:tasks.workunit.client.0.smithi033.stdout:2/396: mknod da/d5b/d5e/d76/c7d 0 2024-04-25T01:35:47.891 INFO:tasks.workunit.client.0.smithi033.stdout:8/418: chown d6/db/d80/l8d 285 1 2024-04-25T01:35:47.892 INFO:tasks.workunit.client.0.smithi033.stdout:2/397: write da/d50/d6a/f57 [3507341,43763] 0 2024-04-25T01:35:47.893 INFO:tasks.workunit.client.0.smithi033.stdout:1/255: dread d2/f32 [0,4194304] 0 2024-04-25T01:35:47.893 INFO:tasks.workunit.client.0.smithi033.stdout:2/398: stat da/d5b/d5e/f6f 0 2024-04-25T01:35:47.896 INFO:tasks.workunit.client.0.smithi033.stdout:9/287: truncate d0/d2/d43/f3f 2582480 0 2024-04-25T01:35:47.899 INFO:tasks.workunit.client.0.smithi033.stdout:8/419: mknod d6/db/d80/c96 0 2024-04-25T01:35:47.904 INFO:tasks.workunit.client.0.smithi033.stdout:1/256: dwrite d2/d11/d28/d33/f3e [0,4194304] 0 2024-04-25T01:35:47.906 INFO:tasks.workunit.client.0.smithi033.stdout:2/399: symlink da/d27/d31/d40/l7e 0 2024-04-25T01:35:47.906 INFO:tasks.workunit.client.0.smithi033.stdout:5/348: creat d1c/d2a/f73 x:0 0 0 2024-04-25T01:35:47.910 INFO:tasks.workunit.client.0.smithi033.stdout:8/420: symlink d6/d45/d51/l97 0 2024-04-25T01:35:47.912 INFO:tasks.workunit.client.0.smithi033.stdout:1/257: mknod d2/d5/d1b/c4a 0 2024-04-25T01:35:47.914 INFO:tasks.workunit.client.0.smithi033.stdout:2/400: read da/d27/d31/d40/f64 [524903,124724] 0 2024-04-25T01:35:47.914 INFO:tasks.workunit.client.0.smithi033.stdout:5/349: dwrite d1c/f32 [0,4194304] 0 2024-04-25T01:35:47.914 INFO:tasks.workunit.client.0.smithi033.stdout:1/258: write d2/d5/d1b/f40 [213166,10181] 0 2024-04-25T01:35:47.918 INFO:tasks.workunit.client.0.smithi033.stdout:8/421: creat d6/d45/d52/d2e/f98 x:0 0 0 2024-04-25T01:35:47.919 INFO:tasks.workunit.client.0.smithi033.stdout:5/350: unlink d1c/d21/f45 0 2024-04-25T01:35:47.925 INFO:tasks.workunit.client.0.smithi033.stdout:5/351: symlink d1c/d2a/d3d/d59/l74 0 2024-04-25T01:35:47.926 INFO:tasks.workunit.client.0.smithi033.stdout:5/352: write d1c/d21/d4a/f4c [576541,124870] 0 2024-04-25T01:35:47.928 INFO:tasks.workunit.client.0.smithi033.stdout:9/288: dread d0/d2/d43/d4b/f5b [0,4194304] 0 2024-04-25T01:35:47.928 INFO:tasks.workunit.client.0.smithi033.stdout:8/422: rename d6/f9 to d6/f99 0 2024-04-25T01:35:47.929 INFO:tasks.workunit.client.0.smithi033.stdout:1/259: link d2/d5/d1b/l27 d2/d11/l4b 0 2024-04-25T01:35:47.930 INFO:tasks.workunit.client.0.smithi033.stdout:5/353: dread d1c/d2a/f47 [0,4194304] 0 2024-04-25T01:35:47.931 INFO:tasks.workunit.client.0.smithi033.stdout:8/423: creat d6/d45/d51/f9a x:0 0 0 2024-04-25T01:35:47.932 INFO:tasks.workunit.client.0.smithi033.stdout:9/289: mknod d0/d2/dd/d45/c67 0 2024-04-25T01:35:47.934 INFO:tasks.workunit.client.0.smithi033.stdout:9/290: write d0/d2/dd/d45/f66 [362407,89660] 0 2024-04-25T01:35:47.935 INFO:tasks.workunit.client.0.smithi033.stdout:5/354: write d1c/f1d [5493658,21305] 0 2024-04-25T01:35:47.940 INFO:tasks.workunit.client.0.smithi033.stdout:5/355: mknod d1c/d2a/d3d/c75 0 2024-04-25T01:35:47.941 INFO:tasks.workunit.client.0.smithi033.stdout:9/291: creat d0/d2/dd/d20/d4e/f68 x:0 0 0 2024-04-25T01:35:47.942 INFO:tasks.workunit.client.0.smithi033.stdout:8/424: dwrite d6/d45/d51/f4d [0,4194304] 0 2024-04-25T01:35:47.942 INFO:tasks.workunit.client.0.smithi033.stdout:1/260: dwrite d2/f26 [0,4194304] 0 2024-04-25T01:35:47.945 INFO:tasks.workunit.client.0.smithi033.stdout:5/356: mknod d1c/d21/d4a/c76 0 2024-04-25T01:35:47.947 INFO:tasks.workunit.client.0.smithi033.stdout:8/425: symlink d6/d49/l9b 0 2024-04-25T01:35:47.948 INFO:tasks.workunit.client.0.smithi033.stdout:1/261: write d2/d11/d28/d33/f36 [9327309,104701] 0 2024-04-25T01:35:47.949 INFO:tasks.workunit.client.0.smithi033.stdout:8/426: truncate d6/db/f61 885878 0 2024-04-25T01:35:47.949 INFO:tasks.workunit.client.0.smithi033.stdout:5/357: creat d1c/d2a/d50/d5b/f77 x:0 0 0 2024-04-25T01:35:47.953 INFO:tasks.workunit.client.0.smithi033.stdout:1/262: unlink d2/d11/d28/c38 0 2024-04-25T01:35:48.009 INFO:tasks.workunit.client.1.smithi138.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-25T01:35:48.012 INFO:tasks.workunit.client.0.smithi033.stdout:7/393: sync 2024-04-25T01:35:48.013 INFO:tasks.workunit.client.0.smithi033.stdout:7/394: write d4/d2e/d6b/f56 [2196026,48846] 0 2024-04-25T01:35:48.024 INFO:tasks.workunit.client.0.smithi033.stdout:8/427: truncate d6/f99 1406 0 2024-04-25T01:35:48.025 INFO:tasks.workunit.client.0.smithi033.stdout:5/358: truncate d1c/d21/d4a/f4c 110973 0 2024-04-25T01:35:48.028 INFO:tasks.workunit.client.0.smithi033.stdout:8/428: dread d6/db/f61 [0,4194304] 0 2024-04-25T01:35:48.030 INFO:tasks.workunit.client.0.smithi033.stdout:1/263: dwrite d2/f32 [0,4194304] 0 2024-04-25T01:35:48.035 INFO:tasks.workunit.client.0.smithi033.stdout:1/264: rmdir d2/d5/d1b/d46 39 2024-04-25T01:35:48.038 INFO:tasks.workunit.client.0.smithi033.stdout:8/429: mknod d6/d45/d51/d5d/c9c 0 2024-04-25T01:35:48.040 INFO:tasks.workunit.client.0.smithi033.stdout:1/265: chown d2/d5/d1b/l45 17 1 2024-04-25T01:35:48.042 INFO:tasks.workunit.client.0.smithi033.stdout:8/430: mkdir d6/d45/d51/d5d/d9d 0 2024-04-25T01:35:48.044 INFO:tasks.workunit.client.0.smithi033.stdout:1/266: mknod d2/d2e/c4c 0 2024-04-25T01:35:48.047 INFO:tasks.workunit.client.0.smithi033.stdout:5/359: dread d1c/d2a/d50/d52/f54 [0,4194304] 0 2024-04-25T01:35:48.047 INFO:tasks.workunit.client.0.smithi033.stdout:8/431: getdents d6/d45/d52/d2e/d95 0 2024-04-25T01:35:48.049 INFO:tasks.workunit.client.0.smithi033.stdout:8/432: mkdir d6/d45/d51/d9e 0 2024-04-25T01:35:48.052 INFO:tasks.workunit.client.0.smithi033.stdout:8/433: chown d6/d45/d51/d55/c7a 98395 1 2024-04-25T01:35:48.055 INFO:tasks.workunit.client.0.smithi033.stdout:1/267: dwrite d2/f29 [0,4194304] 0 2024-04-25T01:35:48.057 INFO:tasks.workunit.client.0.smithi033.stdout:8/434: mknod d6/d45/d51/c9f 0 2024-04-25T01:35:48.057 INFO:tasks.workunit.client.0.smithi033.stdout:5/360: dwrite d1c/d2a/d50/d52/f54 [0,4194304] 0 2024-04-25T01:35:48.059 INFO:tasks.workunit.client.0.smithi033.stdout:5/361: truncate d1c/d2a/d50/d5b/f77 751944 0 2024-04-25T01:35:48.060 INFO:tasks.workunit.client.0.smithi033.stdout:1/268: read d2/d5/f18 [510759,109687] 0 2024-04-25T01:35:48.065 INFO:tasks.workunit.client.0.smithi033.stdout:5/362: link d1c/f38 d1c/d2a/d3d/f78 0 2024-04-25T01:35:48.068 INFO:tasks.workunit.client.0.smithi033.stdout:5/363: write d1c/d2a/f47 [2033032,40623] 0 2024-04-25T01:35:48.069 INFO:tasks.workunit.client.0.smithi033.stdout:1/269: link d2/d2e/l43 d2/d11/d28/l4d 0 2024-04-25T01:35:48.072 INFO:tasks.workunit.client.0.smithi033.stdout:1/270: chown d2/d5/d1b/d46/f47 1373810466 1 2024-04-25T01:35:48.075 INFO:tasks.workunit.client.0.smithi033.stdout:5/364: dwrite d1c/d21/f63 [0,4194304] 0 2024-04-25T01:35:48.078 INFO:tasks.workunit.client.0.smithi033.stdout:5/365: symlink d1c/d21/l79 0 2024-04-25T01:35:48.078 INFO:tasks.workunit.client.0.smithi033.stdout:5/366: chown d1c/d21 195199724 1 2024-04-25T01:35:48.079 INFO:tasks.workunit.client.0.smithi033.stdout:5/367: dread - d1c/d21/f39 zero size 2024-04-25T01:35:48.081 INFO:tasks.workunit.client.0.smithi033.stdout:5/368: getdents d1c/d21/d4a/d69 0 2024-04-25T01:35:48.086 INFO:tasks.workunit.client.0.smithi033.stdout:5/369: getdents d1c/d21/d4a 0 2024-04-25T01:35:48.088 INFO:tasks.workunit.client.0.smithi033.stdout:5/370: mkdir d1c/d21/d3e/d70/d7a 0 2024-04-25T01:35:48.088 INFO:tasks.workunit.client.0.smithi033.stdout:5/371: fsync d1c/d21/d4a/f61 0 2024-04-25T01:35:48.090 INFO:tasks.workunit.client.0.smithi033.stdout:5/372: stat d1c/d2a/c43 0 2024-04-25T01:35:48.095 INFO:tasks.workunit.client.0.smithi033.stdout:5/373: dwrite d1c/d2a/d50/d52/f66 [0,4194304] 0 2024-04-25T01:35:48.097 INFO:tasks.workunit.client.0.smithi033.stdout:5/374: fsync d1c/d21/f63 0 2024-04-25T01:35:48.099 INFO:tasks.workunit.client.0.smithi033.stdout:5/375: creat d1c/d21/d3e/d70/d7a/f7b x:0 0 0 2024-04-25T01:35:48.101 INFO:tasks.workunit.client.0.smithi033.stdout:5/376: creat d1c/d21/d4a/d69/f7c x:0 0 0 2024-04-25T01:35:48.174 INFO:tasks.workunit.client.0.smithi033.stdout:0/270: sync 2024-04-25T01:35:48.185 INFO:tasks.workunit.client.0.smithi033.stdout:0/271: dread d7/d2d/d4d/f5c [0,4194304] 0 2024-04-25T01:35:48.188 INFO:tasks.workunit.client.0.smithi033.stdout:0/272: read d7/da/d23/f5a [1745298,108992] 0 2024-04-25T01:35:48.193 INFO:tasks.workunit.client.0.smithi033.stdout:0/273: dwrite d7/da/df/d14/f22 [0,4194304] 0 2024-04-25T01:35:48.194 INFO:tasks.workunit.client.0.smithi033.stdout:0/274: chown d7/d2d/d4c 8934 1 2024-04-25T01:35:48.269 INFO:tasks.workunit.client.0.smithi033.stdout:0/275: dread d7/da/df/d14/f24 [0,4194304] 0 2024-04-25T01:35:48.270 INFO:tasks.workunit.client.0.smithi033.stdout:0/276: truncate d7/da/df/d2a/d51/f52 308615 0 2024-04-25T01:35:48.272 INFO:tasks.workunit.client.0.smithi033.stdout:0/277: rename d7/da/d23/d41 to d7/d2d/d5e 0 2024-04-25T01:35:48.273 INFO:tasks.workunit.client.0.smithi033.stdout:0/278: readlink d7/l25 0 2024-04-25T01:35:48.274 INFO:tasks.workunit.client.0.smithi033.stdout:0/279: mknod d7/d2d/d4d/c5f 0 2024-04-25T01:35:48.275 INFO:tasks.workunit.client.0.smithi033.stdout:0/280: dread - d7/f46 zero size 2024-04-25T01:35:48.275 INFO:tasks.workunit.client.0.smithi033.stdout:0/281: chown d7/da/d23 167880 1 2024-04-25T01:35:48.310 INFO:tasks.workunit.client.0.smithi033.stdout:4/384: sync 2024-04-25T01:35:48.312 INFO:tasks.workunit.client.0.smithi033.stdout:4/385: truncate d6/d14/d1d/d3b/f3e 9095265 0 2024-04-25T01:35:48.312 INFO:tasks.workunit.client.0.smithi033.stdout:4/386: chown d6/f8 718 1 2024-04-25T01:35:48.312 INFO:tasks.workunit.client.0.smithi033.stdout:4/387: write d6/d12/f58 [8228582,64721] 0 2024-04-25T01:35:48.317 INFO:tasks.workunit.client.0.smithi033.stdout:4/388: getdents d6/d14/d35/d3a 0 2024-04-25T01:35:48.331 INFO:tasks.workunit.client.0.smithi033.stdout:4/389: dread d6/d14/f31 [0,4194304] 0 2024-04-25T01:35:48.332 INFO:tasks.workunit.client.0.smithi033.stdout:4/390: creat d6/d14/d1d/f6f x:0 0 0 2024-04-25T01:35:48.333 INFO:tasks.workunit.client.0.smithi033.stdout:4/391: chown d6/d14/d35 190557049 1 2024-04-25T01:35:48.333 INFO:tasks.workunit.client.0.smithi033.stdout:4/392: chown d6/d12/f32 23930 1 2024-04-25T01:35:48.336 INFO:tasks.workunit.client.0.smithi033.stdout:4/393: getdents d6/d14/d35/d3a/d4d/d5e 0 2024-04-25T01:35:48.337 INFO:tasks.workunit.client.0.smithi033.stdout:4/394: chown d6/d14/d1d 0 1 2024-04-25T01:35:48.341 INFO:tasks.workunit.client.0.smithi033.stdout:4/395: rename d6/d14/d35/d39 to d6/d14/d35/d44/d45/d46/d57/d70 0 2024-04-25T01:35:48.342 INFO:tasks.workunit.client.0.smithi033.stdout:4/396: readlink d6/d14/l37 0 2024-04-25T01:35:48.342 INFO:tasks.workunit.client.0.smithi033.stdout:4/397: read - d6/d14/d1d/f63 zero size 2024-04-25T01:35:48.405 INFO:tasks.workunit.client.0.smithi033.stdout:3/285: sync 2024-04-25T01:35:48.406 INFO:tasks.workunit.client.0.smithi033.stdout:6/283: sync 2024-04-25T01:35:48.406 INFO:tasks.workunit.client.0.smithi033.stdout:6/284: chown d4/f12 84190 1 2024-04-25T01:35:48.406 INFO:tasks.workunit.client.0.smithi033.stdout:3/286: write d8/f51 [1813910,2235] 0 2024-04-25T01:35:48.410 INFO:tasks.workunit.client.0.smithi033.stdout:6/285: getdents d4/d1d/d3e/d41 0 2024-04-25T01:35:48.411 INFO:tasks.workunit.client.0.smithi033.stdout:6/286: fdatasync d4/d2a/f3f 0 2024-04-25T01:35:48.414 INFO:tasks.workunit.client.0.smithi033.stdout:6/287: rename d4/d24/d36/l43 to d4/d24/d63/l6a 0 2024-04-25T01:35:48.422 INFO:tasks.workunit.client.0.smithi033.stdout:3/287: dread d8/dd/d1d/f2d [0,4194304] 0 2024-04-25T01:35:48.424 INFO:tasks.workunit.client.0.smithi033.stdout:3/288: mknod d8/dd/d11/d1e/d27/c68 0 2024-04-25T01:35:48.426 INFO:tasks.workunit.client.0.smithi033.stdout:3/289: rename d8/dd/f25 to d8/dd/d3d/d4f/f69 0 2024-04-25T01:35:48.432 INFO:tasks.workunit.client.0.smithi033.stdout:3/290: dwrite d8/dd/d11/d1e/f45 [0,4194304] 0 2024-04-25T01:35:48.435 INFO:tasks.workunit.client.0.smithi033.stdout:3/291: symlink d8/dd/d40/l6a 0 2024-04-25T01:35:48.439 INFO:tasks.workunit.client.0.smithi033.stdout:3/292: dread d8/dd/d1d/f2b [0,4194304] 0 2024-04-25T01:35:48.440 INFO:tasks.workunit.client.0.smithi033.stdout:3/293: write d8/dd/d1d/f2b [1759102,129888] 0 2024-04-25T01:35:48.446 INFO:tasks.workunit.client.0.smithi033.stdout:3/294: rename d8/dd/d3d/d4f/l5d to d8/dd/d11/d1e/d27/l6b 0 2024-04-25T01:35:48.447 INFO:tasks.workunit.client.0.smithi033.stdout:3/295: write d8/f51 [2011584,92034] 0 2024-04-25T01:35:48.450 INFO:tasks.workunit.client.0.smithi033.stdout:3/296: dread d8/dd/d1d/f2d [0,4194304] 0 2024-04-25T01:35:48.453 INFO:tasks.workunit.client.0.smithi033.stdout:3/297: creat d8/dd/d11/d1e/d27/d4d/f6c x:0 0 0 2024-04-25T01:35:48.454 INFO:tasks.workunit.client.0.smithi033.stdout:3/298: symlink d8/dd/d3d/d4f/l6d 0 2024-04-25T01:35:48.456 INFO:tasks.workunit.client.0.smithi033.stdout:3/299: mkdir d8/dd/d11/d1e/d27/d3c/d6e 0 2024-04-25T01:35:48.457 INFO:tasks.workunit.client.0.smithi033.stdout:3/300: unlink f3 0 2024-04-25T01:35:48.458 INFO:tasks.workunit.client.0.smithi033.stdout:3/301: write d8/f51 [618509,6737] 0 2024-04-25T01:35:48.461 INFO:tasks.workunit.client.0.smithi033.stdout:3/302: creat d8/dd/d3d/d4f/f6f x:0 0 0 2024-04-25T01:35:48.461 INFO:tasks.workunit.client.0.smithi033.stdout:3/303: chown c6 23 1 2024-04-25T01:35:48.462 INFO:tasks.workunit.client.0.smithi033.stdout:3/304: dread - d8/dd/d11/f55 zero size 2024-04-25T01:35:48.469 INFO:tasks.workunit.client.0.smithi033.stdout:3/305: dwrite d8/dd/d40/f4e [0,4194304] 0 2024-04-25T01:35:48.476 INFO:tasks.workunit.client.0.smithi033.stdout:1/271: sync 2024-04-25T01:35:48.477 INFO:tasks.workunit.client.0.smithi033.stdout:9/292: sync 2024-04-25T01:35:48.477 INFO:tasks.workunit.client.0.smithi033.stdout:0/282: sync 2024-04-25T01:35:48.477 INFO:tasks.workunit.client.0.smithi033.stdout:8/435: sync 2024-04-25T01:35:48.477 INFO:tasks.workunit.client.0.smithi033.stdout:4/398: sync 2024-04-25T01:35:48.477 INFO:tasks.workunit.client.0.smithi033.stdout:7/395: sync 2024-04-25T01:35:48.477 INFO:tasks.workunit.client.0.smithi033.stdout:5/377: sync 2024-04-25T01:35:48.477 INFO:tasks.workunit.client.0.smithi033.stdout:2/401: sync 2024-04-25T01:35:48.477 INFO:tasks.workunit.client.0.smithi033.stdout:3/306: creat d8/dd/d11/d1e/d27/f70 x:0 0 0 2024-04-25T01:35:48.481 INFO:tasks.workunit.client.0.smithi033.stdout:4/399: chown d6/d14/d35/c5a 23 1 2024-04-25T01:35:48.482 INFO:tasks.workunit.client.0.smithi033.stdout:9/293: dread d0/d2/d43/f3c [0,4194304] 0 2024-04-25T01:35:48.482 INFO:tasks.workunit.client.0.smithi033.stdout:0/283: write d7/da/df/f19 [2284221,87088] 0 2024-04-25T01:35:48.484 INFO:tasks.workunit.client.0.smithi033.stdout:3/307: fdatasync d8/dd/d40/f65 0 2024-04-25T01:35:48.485 INFO:tasks.workunit.client.0.smithi033.stdout:2/402: mknod da/d50/d6a/d43/d6c/c7f 0 2024-04-25T01:35:48.486 INFO:tasks.workunit.client.0.smithi033.stdout:1/272: link d2/d11/d28/d33/f36 d2/d2e/d34/f4e 0 2024-04-25T01:35:48.487 INFO:tasks.workunit.client.0.smithi033.stdout:3/308: write d8/f51 [294982,4758] 0 2024-04-25T01:35:48.488 INFO:tasks.workunit.client.0.smithi033.stdout:0/284: creat d7/d1b/f60 x:0 0 0 2024-04-25T01:35:48.489 INFO:tasks.workunit.client.0.smithi033.stdout:5/378: dwrite f17 [4194304,4194304] 0 2024-04-25T01:35:48.489 INFO:tasks.workunit.client.0.smithi033.stdout:9/294: mknod d0/d2/d43/c69 0 2024-04-25T01:35:48.493 INFO:tasks.workunit.client.0.smithi033.stdout:4/400: mknod d6/d14/d35/d3a/d4d/d5e/c71 0 2024-04-25T01:35:48.498 INFO:tasks.workunit.client.0.smithi033.stdout:3/309: dread d8/dd/d40/f65 [0,4194304] 0 2024-04-25T01:35:48.498 INFO:tasks.workunit.client.0.smithi033.stdout:8/436: truncate d6/d45/d51/d5d/f63 527114 0 2024-04-25T01:35:48.499 INFO:tasks.workunit.client.0.smithi033.stdout:0/285: mkdir d7/d2d/d5e/d61 0 2024-04-25T01:35:48.500 INFO:tasks.workunit.client.0.smithi033.stdout:0/286: chown d7/d1b 23 1 2024-04-25T01:35:48.500 INFO:tasks.workunit.client.0.smithi033.stdout:3/310: dread d8/dd/d40/f65 [0,4194304] 0 2024-04-25T01:35:48.501 INFO:tasks.workunit.client.0.smithi033.stdout:5/379: stat d1c/f48 0 2024-04-25T01:35:48.501 INFO:tasks.workunit.client.0.smithi033.stdout:6/288: sync 2024-04-25T01:35:48.502 INFO:tasks.workunit.client.0.smithi033.stdout:3/311: write d8/dd/d11/d1e/d27/f70 [340408,111750] 0 2024-04-25T01:35:48.502 INFO:tasks.workunit.client.0.smithi033.stdout:9/295: creat d0/d2/dd/d45/f6a x:0 0 0 2024-04-25T01:35:48.503 INFO:tasks.workunit.client.0.smithi033.stdout:2/403: write f8 [104385,50924] 0 2024-04-25T01:35:48.504 INFO:tasks.workunit.client.0.smithi033.stdout:2/404: write da/d27/f59 [67106,2393] 0 2024-04-25T01:35:48.505 INFO:tasks.workunit.client.0.smithi033.stdout:4/401: unlink d6/l6a 0 2024-04-25T01:35:48.507 INFO:tasks.workunit.client.0.smithi033.stdout:9/296: fdatasync d0/d2/dd/df/d1b/f2e 0 2024-04-25T01:35:48.508 INFO:tasks.workunit.client.0.smithi033.stdout:4/402: write d6/d14/d1d/f28 [19887,22547] 0 2024-04-25T01:35:48.510 INFO:tasks.workunit.client.0.smithi033.stdout:7/396: truncate d4/d2e/d6b/f33 392558 0 2024-04-25T01:35:48.510 INFO:tasks.workunit.client.0.smithi033.stdout:4/403: readlink d6/d12/l21 0 2024-04-25T01:35:48.510 INFO:tasks.workunit.client.0.smithi033.stdout:0/287: rmdir d7/d2d/d4c 39 2024-04-25T01:35:48.510 INFO:tasks.workunit.client.0.smithi033.stdout:9/297: chown d0/d2/dd/df/l1d 2 1 2024-04-25T01:35:48.512 INFO:tasks.workunit.client.0.smithi033.stdout:2/405: dread da/d27/f6d [0,4194304] 0 2024-04-25T01:35:48.512 INFO:tasks.workunit.client.0.smithi033.stdout:0/288: chown d7/d2d/d4d/c5f 557213721 1 2024-04-25T01:35:48.514 INFO:tasks.workunit.client.0.smithi033.stdout:1/273: creat d2/f4f x:0 0 0 2024-04-25T01:35:48.515 INFO:tasks.workunit.client.0.smithi033.stdout:8/437: symlink d6/d45/d51/d5d/d9d/la0 0 2024-04-25T01:35:48.515 INFO:tasks.workunit.client.0.smithi033.stdout:2/406: chown da/d27/f6d 105278 1 2024-04-25T01:35:48.517 INFO:tasks.workunit.client.0.smithi033.stdout:7/397: creat d4/d2e/d6b/d1f/f82 x:0 0 0 2024-04-25T01:35:48.518 INFO:tasks.workunit.client.0.smithi033.stdout:5/380: link d1c/d2a/d50/d5b/f77 d1c/d2a/d50/f7d 0 2024-04-25T01:35:48.519 INFO:tasks.workunit.client.0.smithi033.stdout:9/298: rmdir d0/d2/dd/d5f 39 2024-04-25T01:35:48.519 INFO:tasks.workunit.client.0.smithi033.stdout:6/289: dwrite d4/f12 [0,4194304] 0 2024-04-25T01:35:48.520 INFO:tasks.workunit.client.0.smithi033.stdout:5/381: readlink d1c/d2a/d3d/d59/l74 0 2024-04-25T01:35:48.521 INFO:tasks.workunit.client.0.smithi033.stdout:4/404: dread d6/d14/f2b [0,4194304] 0 2024-04-25T01:35:48.522 INFO:tasks.workunit.client.0.smithi033.stdout:0/289: rmdir d7/d2d/d4d 39 2024-04-25T01:35:48.525 INFO:tasks.workunit.client.0.smithi033.stdout:1/274: creat d2/d2e/d34/f50 x:0 0 0 2024-04-25T01:35:48.526 INFO:tasks.workunit.client.0.smithi033.stdout:4/405: write d6/d14/ff [7608580,120151] 0 2024-04-25T01:35:48.527 INFO:tasks.workunit.client.0.smithi033.stdout:0/290: chown d7/d1b/c2c 6 1 2024-04-25T01:35:48.530 INFO:tasks.workunit.client.0.smithi033.stdout:8/438: mknod d6/db/d80/d89/ca1 0 2024-04-25T01:35:48.531 INFO:tasks.workunit.client.0.smithi033.stdout:6/290: symlink d4/d24/d63/l6b 0 2024-04-25T01:35:48.531 INFO:tasks.workunit.client.0.smithi033.stdout:5/382: chown d1c/d2a/d50/d52/c5a 438964805 1 2024-04-25T01:35:48.532 INFO:tasks.workunit.client.0.smithi033.stdout:4/406: readlink d6/d12/l42 0 2024-04-25T01:35:48.533 INFO:tasks.workunit.client.0.smithi033.stdout:6/291: stat d4/d24/l27 0 2024-04-25T01:35:48.533 INFO:tasks.workunit.client.0.smithi033.stdout:2/407: unlink da/d5b/d5e/d7c/f35 0 2024-04-25T01:35:48.534 INFO:tasks.workunit.client.0.smithi033.stdout:9/299: symlink d0/d2/dd/d5f/l6b 0 2024-04-25T01:35:48.534 INFO:tasks.workunit.client.0.smithi033.stdout:0/291: chown d7/da/df/f56 71213 1 2024-04-25T01:35:48.535 INFO:tasks.workunit.client.0.smithi033.stdout:7/398: dwrite d4/d23/d3f/d45/f7c [0,4194304] 0 2024-04-25T01:35:48.537 INFO:tasks.workunit.client.0.smithi033.stdout:6/292: write d4/d24/d29/f33 [334248,48466] 0 2024-04-25T01:35:48.540 INFO:tasks.workunit.client.0.smithi033.stdout:6/293: write d4/d1d/f50 [4076412,54795] 0 2024-04-25T01:35:48.543 INFO:tasks.workunit.client.0.smithi033.stdout:6/294: chown d4/d1d/d3e/d41/d4e 14904 1 2024-04-25T01:35:48.543 INFO:tasks.workunit.client.0.smithi033.stdout:0/292: chown d7/d2d/d4c/l55 1415 1 2024-04-25T01:35:48.547 INFO:tasks.workunit.client.0.smithi033.stdout:2/408: mkdir da/d2c/d30/d80 0 2024-04-25T01:35:48.549 INFO:tasks.workunit.client.0.smithi033.stdout:9/300: dwrite d0/d2/f1f [0,4194304] 0 2024-04-25T01:35:48.550 INFO:tasks.workunit.client.0.smithi033.stdout:2/409: write da/d27/d31/f61 [4412364,3232] 0 2024-04-25T01:35:48.552 INFO:tasks.workunit.client.0.smithi033.stdout:6/295: creat d4/d24/d36/f6c x:0 0 0 2024-04-25T01:35:48.552 INFO:tasks.workunit.client.0.smithi033.stdout:1/275: link d2/d5/d1b/l45 d2/d11/d28/d33/l51 0 2024-04-25T01:35:48.553 INFO:tasks.workunit.client.0.smithi033.stdout:6/296: fsync d4/d24/f64 0 2024-04-25T01:35:48.555 INFO:tasks.workunit.client.0.smithi033.stdout:1/276: chown d2/d5/cc 151000913 1 2024-04-25T01:35:48.558 INFO:tasks.workunit.client.0.smithi033.stdout:6/297: creat d4/d1d/d3e/d41/d5c/f6d x:0 0 0 2024-04-25T01:35:48.561 INFO:tasks.workunit.client.0.smithi033.stdout:6/298: write d4/d24/d29/f55 [1038102,128315] 0 2024-04-25T01:35:48.561 INFO:tasks.workunit.client.0.smithi033.stdout:1/277: write d2/d5/d1b/d46/f47 [390396,85492] 0 2024-04-25T01:35:48.563 INFO:tasks.workunit.client.0.smithi033.stdout:9/301: dwrite d0/d11/f47 [0,4194304] 0 2024-04-25T01:35:48.564 INFO:tasks.workunit.client.0.smithi033.stdout:6/299: write d4/d24/d29/f33 [1349978,13761] 0 2024-04-25T01:35:48.565 INFO:tasks.workunit.client.0.smithi033.stdout:1/278: truncate d2/d5/d1b/d46/f47 1353289 0 2024-04-25T01:35:48.566 INFO:tasks.workunit.client.0.smithi033.stdout:9/302: truncate d0/d2/dd/d20/f38 5498003 0 2024-04-25T01:35:48.568 INFO:tasks.workunit.client.0.smithi033.stdout:1/279: write d2/f26 [1165545,49075] 0 2024-04-25T01:35:48.569 INFO:tasks.workunit.client.0.smithi033.stdout:6/300: rename d4/cc to d4/d24/d63/c6e 0 2024-04-25T01:35:48.570 INFO:tasks.workunit.client.0.smithi033.stdout:1/280: write d2/f29 [3014002,15057] 0 2024-04-25T01:35:48.570 INFO:tasks.workunit.client.0.smithi033.stdout:9/303: creat d0/d2/f6c x:0 0 0 2024-04-25T01:35:48.573 INFO:tasks.workunit.client.0.smithi033.stdout:1/281: stat d2/d11/f2a 0 2024-04-25T01:35:48.574 INFO:tasks.workunit.client.0.smithi033.stdout:7/399: dread d4/f5 [0,4194304] 0 2024-04-25T01:35:48.575 INFO:tasks.workunit.client.0.smithi033.stdout:0/293: dread d7/da/df/f28 [0,4194304] 0 2024-04-25T01:35:48.575 INFO:tasks.workunit.client.0.smithi033.stdout:6/301: creat d4/d24/d63/f6f x:0 0 0 2024-04-25T01:35:48.577 INFO:tasks.workunit.client.0.smithi033.stdout:9/304: rename d0/d2/dd/df/d1b/f50 to d0/d2/d43/d4b/d55/f6d 0 2024-04-25T01:35:48.578 INFO:tasks.workunit.client.0.smithi033.stdout:7/400: write d4/d2e/f1c [1221759,117132] 0 2024-04-25T01:35:48.579 INFO:tasks.workunit.client.0.smithi033.stdout:0/294: symlink d7/d33/l62 0 2024-04-25T01:35:48.580 INFO:tasks.workunit.client.0.smithi033.stdout:0/295: fsync d7/d1b/f60 0 2024-04-25T01:35:48.582 INFO:tasks.workunit.client.0.smithi033.stdout:0/296: write d7/da/df/d14/f22 [2523341,78668] 0 2024-04-25T01:35:48.583 INFO:tasks.workunit.client.0.smithi033.stdout:6/302: mkdir d4/d1d/d3e/d70 0 2024-04-25T01:35:48.584 INFO:tasks.workunit.client.0.smithi033.stdout:1/282: creat d2/d11/f52 x:0 0 0 2024-04-25T01:35:48.585 INFO:tasks.workunit.client.0.smithi033.stdout:7/401: symlink d4/d2e/d6b/l83 0 2024-04-25T01:35:48.586 INFO:tasks.workunit.client.0.smithi033.stdout:0/297: fsync d7/da/f58 0 2024-04-25T01:35:48.590 INFO:tasks.workunit.client.0.smithi033.stdout:7/402: getdents d4/d2e/d7f 0 2024-04-25T01:35:48.592 INFO:tasks.workunit.client.0.smithi033.stdout:1/283: dwrite d2/f26 [0,4194304] 0 2024-04-25T01:35:48.592 INFO:tasks.workunit.client.0.smithi033.stdout:7/403: write d4/d2e/d50/f6e [652834,7937] 0 2024-04-25T01:35:48.597 INFO:tasks.workunit.client.0.smithi033.stdout:1/284: chown d2/d11/d28/d33/l51 2264 1 2024-04-25T01:35:48.598 INFO:tasks.workunit.client.0.smithi033.stdout:7/404: fsync d4/d2e/d6b/f47 0 2024-04-25T01:35:48.602 INFO:tasks.workunit.client.0.smithi033.stdout:7/405: unlink d4/d2e/f3e 0 2024-04-25T01:35:48.610 INFO:tasks.workunit.client.0.smithi033.stdout:7/406: link d4/d23/d3f/d45/d73/l7e d4/d2e/d6b/l84 0 2024-04-25T01:35:48.616 INFO:tasks.workunit.client.0.smithi033.stdout:7/407: mknod d4/d2e/d7f/c85 0 2024-04-25T01:35:48.617 INFO:tasks.workunit.client.0.smithi033.stdout:7/408: fsync d4/d2e/d6b/d1f/f42 0 2024-04-25T01:35:48.618 INFO:tasks.workunit.client.0.smithi033.stdout:7/409: write d4/fb [1883837,118020] 0 2024-04-25T01:35:48.626 INFO:tasks.workunit.client.0.smithi033.stdout:6/303: dread d4/d24/d29/f33 [0,4194304] 0 2024-04-25T01:35:48.627 INFO:tasks.workunit.client.0.smithi033.stdout:8/439: write d6/d45/f6d [480172,48495] 0 2024-04-25T01:35:48.628 INFO:tasks.workunit.client.0.smithi033.stdout:7/410: dread d4/d2e/d6b/f62 [0,4194304] 0 2024-04-25T01:35:48.629 INFO:tasks.workunit.client.0.smithi033.stdout:6/304: symlink d4/d2a/l71 0 2024-04-25T01:35:48.633 INFO:tasks.workunit.client.0.smithi033.stdout:8/440: mknod d6/d45/d52/d2e/d87/ca2 0 2024-04-25T01:35:48.634 INFO:tasks.workunit.client.0.smithi033.stdout:6/305: unlink d4/l10 0 2024-04-25T01:35:48.635 INFO:tasks.workunit.client.0.smithi033.stdout:5/383: dwrite f14 [0,4194304] 0 2024-04-25T01:35:48.637 INFO:tasks.workunit.client.0.smithi033.stdout:2/410: truncate da/fe 808578 0 2024-04-25T01:35:48.637 INFO:tasks.workunit.client.0.smithi033.stdout:6/306: fdatasync d4/f1b 0 2024-04-25T01:35:48.637 INFO:tasks.workunit.client.0.smithi033.stdout:3/312: dread d8/dd/d11/f24 [4194304,4194304] 0 2024-04-25T01:35:48.639 INFO:tasks.workunit.client.0.smithi033.stdout:6/307: dread - d4/d24/d36/d37/f67 zero size 2024-04-25T01:35:48.644 INFO:tasks.workunit.client.0.smithi033.stdout:5/384: mkdir d1c/d21/d3e/d70/d7a/d7e 0 2024-04-25T01:35:48.647 INFO:tasks.workunit.client.0.smithi033.stdout:6/308: dwrite d4/d24/f64 [0,4194304] 0 2024-04-25T01:35:48.647 INFO:tasks.workunit.client.0.smithi033.stdout:4/407: dread d6/d14/d48/f55 [0,4194304] 0 2024-04-25T01:35:48.648 INFO:tasks.workunit.client.0.smithi033.stdout:6/309: stat d4/d2a 0 2024-04-25T01:35:48.649 INFO:tasks.workunit.client.0.smithi033.stdout:2/411: mkdir da/d27/d31/d81 0 2024-04-25T01:35:48.650 INFO:tasks.workunit.client.0.smithi033.stdout:2/412: chown da/d27/d31/d40/l52 502 1 2024-04-25T01:35:48.651 INFO:tasks.workunit.client.0.smithi033.stdout:2/413: fsync f8 0 2024-04-25T01:35:48.653 INFO:tasks.workunit.client.0.smithi033.stdout:2/414: readlink da/d27/d31/d40/l7e 0 2024-04-25T01:35:48.656 INFO:tasks.workunit.client.0.smithi033.stdout:4/408: fsync d6/d14/d1d/f20 0 2024-04-25T01:35:48.664 INFO:tasks.workunit.client.0.smithi033.stdout:6/310: creat d4/d1d/d3e/d70/f72 x:0 0 0 2024-04-25T01:35:48.666 INFO:tasks.workunit.client.0.smithi033.stdout:6/311: chown d4/d15/f5f 33324567 1 2024-04-25T01:35:48.666 INFO:tasks.workunit.client.0.smithi033.stdout:3/313: getdents d8/dd/d3d/d4f 0 2024-04-25T01:35:48.667 INFO:tasks.workunit.client.0.smithi033.stdout:6/312: readlink d4/d15/l17 0 2024-04-25T01:35:48.668 INFO:tasks.workunit.client.0.smithi033.stdout:2/415: unlink da/d5b/d5e/d7c/f70 0 2024-04-25T01:35:48.668 INFO:tasks.workunit.client.0.smithi033.stdout:0/298: dwrite d7/d1b/f32 [0,4194304] 0 2024-04-25T01:35:48.675 INFO:tasks.workunit.client.0.smithi033.stdout:3/314: creat d8/dd/d3d/d63/f71 x:0 0 0 2024-04-25T01:35:48.676 INFO:tasks.workunit.client.0.smithi033.stdout:2/416: chown da/d5b/f63 13 1 2024-04-25T01:35:48.679 INFO:tasks.workunit.client.0.smithi033.stdout:7/411: truncate d4/d2e/d6b/d1f/f46 4927843 0 2024-04-25T01:35:48.680 INFO:tasks.workunit.client.0.smithi033.stdout:0/299: creat d7/d2d/d43/d44/f63 x:0 0 0 2024-04-25T01:35:48.680 INFO:tasks.workunit.client.0.smithi033.stdout:7/412: read d4/d2e/d6b/f47 [590809,18422] 0 2024-04-25T01:35:48.681 INFO:tasks.workunit.client.0.smithi033.stdout:0/300: truncate d7/d1b/f60 423034 0 2024-04-25T01:35:48.684 INFO:tasks.workunit.client.0.smithi033.stdout:3/315: rmdir d8/dd/d11/d1e 39 2024-04-25T01:35:48.685 INFO:tasks.workunit.client.0.smithi033.stdout:7/413: dread d4/d2e/d50/f6e [0,4194304] 0 2024-04-25T01:35:48.687 INFO:tasks.workunit.client.0.smithi033.stdout:3/316: truncate d8/dd/d3d/d63/f71 430846 0 2024-04-25T01:35:48.689 INFO:tasks.workunit.client.0.smithi033.stdout:5/385: getdents d1c/d21/d3e/d70/d7a 0 2024-04-25T01:35:48.691 INFO:tasks.workunit.client.0.smithi033.stdout:5/386: chown d1c/d21/d4a/f64 0 1 2024-04-25T01:35:48.691 INFO:tasks.workunit.client.0.smithi033.stdout:8/441: write d6/d45/d52/d2e/f4a [388938,19760] 0 2024-04-25T01:35:48.691 INFO:tasks.workunit.client.0.smithi033.stdout:5/387: chown d1c/d2a/f47 9 1 2024-04-25T01:35:48.692 INFO:tasks.workunit.client.0.smithi033.stdout:4/409: link d6/d14/d35/l66 d6/d14/d1d/d5f/l72 0 2024-04-25T01:35:48.694 INFO:tasks.workunit.client.0.smithi033.stdout:7/414: fdatasync d4/d2e/f4c 0 2024-04-25T01:35:48.695 INFO:tasks.workunit.client.0.smithi033.stdout:6/313: rmdir d4/d1d/d3e/d41/d4e 0 2024-04-25T01:35:48.696 INFO:tasks.workunit.client.0.smithi033.stdout:3/317: chown d8/dd/d11/d1e/d27/f70 4714 1 2024-04-25T01:35:48.697 INFO:tasks.workunit.client.0.smithi033.stdout:2/417: link da/d27/d31/f5a da/d27/d31/d40/f82 0 2024-04-25T01:35:48.698 INFO:tasks.workunit.client.0.smithi033.stdout:8/442: creat d6/db/d80/fa3 x:0 0 0 2024-04-25T01:35:48.702 INFO:tasks.workunit.client.0.smithi033.stdout:2/418: dwrite da/d27/d31/f46 [0,4194304] 0 2024-04-25T01:35:48.711 INFO:tasks.workunit.client.0.smithi033.stdout:5/388: creat d1c/d21/d4a/f7f x:0 0 0 2024-04-25T01:35:48.719 INFO:tasks.workunit.client.0.smithi033.stdout:3/318: write d8/dd/f15 [4377526,122298] 0 2024-04-25T01:35:48.719 INFO:tasks.workunit.client.0.smithi033.stdout:4/410: dwrite d6/d14/d48/f55 [0,4194304] 0 2024-04-25T01:35:48.722 INFO:tasks.workunit.client.0.smithi033.stdout:5/389: symlink d1c/d2a/d50/d52/l80 0 2024-04-25T01:35:48.722 INFO:tasks.workunit.client.0.smithi033.stdout:5/390: write f14 [724845,32659] 0 2024-04-25T01:35:48.726 INFO:tasks.workunit.client.0.smithi033.stdout:0/301: rename d7/da/df/d2a to d7/d2d/d64 0 2024-04-25T01:35:48.728 INFO:tasks.workunit.client.0.smithi033.stdout:7/415: unlink d4/d2e/d6b/f33 0 2024-04-25T01:35:48.735 INFO:tasks.workunit.client.0.smithi033.stdout:7/416: dwrite d4/f11 [4194304,4194304] 0 2024-04-25T01:35:48.736 INFO:tasks.workunit.client.0.smithi033.stdout:8/443: rename d6/d45/c8e to d6/d45/d52/d2e/d95/ca4 0 2024-04-25T01:35:48.737 INFO:tasks.workunit.client.0.smithi033.stdout:7/417: truncate d4/d10/f3b 1413165 0 2024-04-25T01:35:48.738 INFO:tasks.workunit.client.0.smithi033.stdout:0/302: mknod d7/d2d/d64/d51/c65 0 2024-04-25T01:35:48.740 INFO:tasks.workunit.client.0.smithi033.stdout:3/319: symlink d8/dd/d11/d1e/d27/l72 0 2024-04-25T01:35:48.742 INFO:tasks.workunit.client.0.smithi033.stdout:2/419: creat da/d27/d31/f83 x:0 0 0 2024-04-25T01:35:48.743 INFO:tasks.workunit.client.0.smithi033.stdout:2/420: fsync da/d27/d31/f46 0 2024-04-25T01:35:48.744 INFO:tasks.workunit.client.0.smithi033.stdout:4/411: mknod d6/d14/d1d/d5f/c73 0 2024-04-25T01:35:48.746 INFO:tasks.workunit.client.0.smithi033.stdout:6/314: getdents d4/d24/d36/d37 0 2024-04-25T01:35:48.747 INFO:tasks.workunit.client.0.smithi033.stdout:6/315: write d4/d1d/d3e/d70/f72 [316361,97410] 0 2024-04-25T01:35:48.747 INFO:tasks.workunit.client.0.smithi033.stdout:8/444: symlink d6/db/d80/la5 0 2024-04-25T01:35:48.752 INFO:tasks.workunit.client.0.smithi033.stdout:2/421: creat da/d5b/d5e/d76/f84 x:0 0 0 2024-04-25T01:35:48.752 INFO:tasks.workunit.client.0.smithi033.stdout:4/412: creat d6/d14/d1d/d3b/f74 x:0 0 0 2024-04-25T01:35:48.754 INFO:tasks.workunit.client.0.smithi033.stdout:5/391: creat d1c/d21/f81 x:0 0 0 2024-04-25T01:35:48.756 INFO:tasks.workunit.client.0.smithi033.stdout:7/418: unlink d4/d23/d3f/d45/d73/l7e 0 2024-04-25T01:35:48.757 INFO:tasks.workunit.client.0.smithi033.stdout:6/316: symlink d4/d24/l73 0 2024-04-25T01:35:48.758 INFO:tasks.workunit.client.0.smithi033.stdout:8/445: mknod d6/d45/d51/d55/ca6 0 2024-04-25T01:35:48.761 INFO:tasks.workunit.client.0.smithi033.stdout:7/419: mkdir d4/d2e/d6b/d1f/d78/d86 0 2024-04-25T01:35:48.762 INFO:tasks.workunit.client.0.smithi033.stdout:7/420: readlink d4/d23/l53 0 2024-04-25T01:35:48.762 INFO:tasks.workunit.client.0.smithi033.stdout:6/317: creat d4/d1d/d3e/d41/d54/f74 x:0 0 0 2024-04-25T01:35:48.763 INFO:tasks.workunit.client.0.smithi033.stdout:8/446: fdatasync d6/f1e 0 2024-04-25T01:35:48.764 INFO:tasks.workunit.client.0.smithi033.stdout:8/447: stat d6/d45/d51/d55/c7a 0 2024-04-25T01:35:48.765 INFO:tasks.workunit.client.0.smithi033.stdout:3/320: creat d8/dd/d11/d1e/d27/f73 x:0 0 0 2024-04-25T01:35:48.766 INFO:tasks.workunit.client.0.smithi033.stdout:8/448: write d6/d45/f6d [641650,58888] 0 2024-04-25T01:35:48.766 INFO:tasks.workunit.client.0.smithi033.stdout:3/321: chown d8/dd/d40/f53 1830513 1 2024-04-25T01:35:48.767 INFO:tasks.workunit.client.0.smithi033.stdout:4/413: link d6/f8 d6/d14/d35/d44/d45/f75 0 2024-04-25T01:35:48.768 INFO:tasks.workunit.client.0.smithi033.stdout:3/322: chown d8/dd/d11/d1e/f3b 0 1 2024-04-25T01:35:48.770 INFO:tasks.workunit.client.0.smithi033.stdout:7/421: write d4/d23/d3f/f60 [4065084,113036] 0 2024-04-25T01:35:48.773 INFO:tasks.workunit.client.0.smithi033.stdout:2/422: creat da/f85 x:0 0 0 2024-04-25T01:35:48.774 INFO:tasks.workunit.client.0.smithi033.stdout:2/423: chown da 410 1 2024-04-25T01:35:48.777 INFO:tasks.workunit.client.0.smithi033.stdout:8/449: write d6/d45/d51/d5d/f65 [505922,58845] 0 2024-04-25T01:35:48.780 INFO:tasks.workunit.client.0.smithi033.stdout:5/392: write d1c/f2e [1497920,14149] 0 2024-04-25T01:35:48.786 INFO:tasks.workunit.client.0.smithi033.stdout:7/422: dwrite d4/f9 [0,4194304] 0 2024-04-25T01:35:48.786 INFO:tasks.workunit.client.0.smithi033.stdout:2/424: unlink da/d2c/d30/l6e 0 2024-04-25T01:35:48.792 INFO:tasks.workunit.client.0.smithi033.stdout:4/414: link d6/d12/f58 d6/d14/d35/d3a/d4d/d5e/f76 0 2024-04-25T01:35:48.796 INFO:tasks.workunit.client.0.smithi033.stdout:5/393: creat d1c/d21/d3e/d70/d7a/f82 x:0 0 0 2024-04-25T01:35:48.798 INFO:tasks.workunit.client.0.smithi033.stdout:2/425: creat da/d27/f86 x:0 0 0 2024-04-25T01:35:48.799 INFO:tasks.workunit.client.0.smithi033.stdout:6/318: getdents d4/d1d/d3e/d4a 0 2024-04-25T01:35:48.800 INFO:tasks.workunit.client.0.smithi033.stdout:8/450: dwrite d6/f1e [4194304,4194304] 0 2024-04-25T01:35:48.803 INFO:tasks.workunit.client.0.smithi033.stdout:7/423: mknod d4/d2e/d6b/c87 0 2024-04-25T01:35:48.806 INFO:tasks.workunit.client.0.smithi033.stdout:3/323: truncate d8/dd/d11/d1e/f21 3409936 0 2024-04-25T01:35:48.807 INFO:tasks.workunit.client.0.smithi033.stdout:6/319: symlink d4/d1d/d3e/d41/d54/l75 0 2024-04-25T01:35:48.808 INFO:tasks.workunit.client.0.smithi033.stdout:5/394: dwrite f14 [0,4194304] 0 2024-04-25T01:35:48.809 INFO:tasks.workunit.client.0.smithi033.stdout:6/320: chown d4/d1d/d3e 76090 1 2024-04-25T01:35:48.809 INFO:tasks.workunit.client.0.smithi033.stdout:4/415: getdents d6/d14/d48 0 2024-04-25T01:35:48.810 INFO:tasks.workunit.client.0.smithi033.stdout:2/426: mkdir da/d5b/d72/d87 0 2024-04-25T01:35:48.813 INFO:tasks.workunit.client.0.smithi033.stdout:6/321: rmdir d4/d24/d36/d42 39 2024-04-25T01:35:48.814 INFO:tasks.workunit.client.0.smithi033.stdout:6/322: readlink d4/d1d/d3e/d41/d54/l75 0 2024-04-25T01:35:48.819 INFO:tasks.workunit.client.0.smithi033.stdout:5/395: mkdir d1c/d2a/d3d/d83 0 2024-04-25T01:35:48.820 INFO:tasks.workunit.client.0.smithi033.stdout:5/396: write d1c/d21/f6d [910885,98407] 0 2024-04-25T01:35:48.824 INFO:tasks.workunit.client.0.smithi033.stdout:2/427: dwrite da/d50/d6a/d43/f62 [0,4194304] 0 2024-04-25T01:35:48.835 INFO:tasks.workunit.client.0.smithi033.stdout:2/428: creat da/d27/d31/d40/f88 x:0 0 0 2024-04-25T01:35:48.836 INFO:tasks.workunit.client.0.smithi033.stdout:2/429: write da/d27/d31/d40/f54 [851123,27207] 0 2024-04-25T01:35:48.838 INFO:tasks.workunit.client.0.smithi033.stdout:3/324: rename d8/dd/l10 to d8/dd/d11/d1e/l74 0 2024-04-25T01:35:48.839 INFO:tasks.workunit.client.0.smithi033.stdout:4/416: creat d6/d14/d6d/f77 x:0 0 0 2024-04-25T01:35:48.842 INFO:tasks.workunit.client.0.smithi033.stdout:5/397: unlink d1c/d21/d3e/d70/d7a/f82 0 2024-04-25T01:35:48.843 INFO:tasks.workunit.client.0.smithi033.stdout:5/398: truncate d1c/d2a/d3d/f3f 822073 0 2024-04-25T01:35:48.844 INFO:tasks.workunit.client.0.smithi033.stdout:2/430: creat da/d5b/d5e/f89 x:0 0 0 2024-04-25T01:35:48.846 INFO:tasks.workunit.client.0.smithi033.stdout:6/323: rename d4/l40 to d4/d24/d29/l76 0 2024-04-25T01:35:48.848 INFO:tasks.workunit.client.0.smithi033.stdout:6/324: truncate d4/d1d/f69 879773 0 2024-04-25T01:35:48.850 INFO:tasks.workunit.client.0.smithi033.stdout:5/399: creat d1c/d2a/d3d/d59/f84 x:0 0 0 2024-04-25T01:35:48.851 INFO:tasks.workunit.client.0.smithi033.stdout:7/424: mknod d4/d2e/d6b/d1f/d78/d86/c88 0 2024-04-25T01:35:48.854 INFO:tasks.workunit.client.0.smithi033.stdout:2/431: rename da/d2c/d30 to da/d27/d31/d40/d8a 0 2024-04-25T01:35:48.855 INFO:tasks.workunit.client.0.smithi033.stdout:3/325: dread d8/f1b [0,4194304] 0 2024-04-25T01:35:48.856 INFO:tasks.workunit.client.0.smithi033.stdout:4/417: creat d6/d14/d35/d3a/d4c/f78 x:0 0 0 2024-04-25T01:35:48.858 INFO:tasks.workunit.client.0.smithi033.stdout:6/325: creat d4/d24/d63/f77 x:0 0 0 2024-04-25T01:35:48.862 INFO:tasks.workunit.client.0.smithi033.stdout:2/432: symlink da/d27/d31/d40/d8a/l8b 0 2024-04-25T01:35:48.868 INFO:tasks.workunit.client.0.smithi033.stdout:3/326: dwrite d8/dd/d11/d1e/d27/d4d/f62 [0,4194304] 0 2024-04-25T01:35:48.869 INFO:tasks.workunit.client.0.smithi033.stdout:4/418: dwrite d6/d14/d35/d3a/f56 [0,4194304] 0 2024-04-25T01:35:48.870 INFO:tasks.workunit.client.0.smithi033.stdout:3/327: fsync d8/dd/d40/f53 0 2024-04-25T01:35:48.870 INFO:tasks.workunit.client.0.smithi033.stdout:4/419: chown d6/d14/ff 1976953 1 2024-04-25T01:35:48.871 INFO:tasks.workunit.client.0.smithi033.stdout:2/433: symlink da/d5b/d5e/d7c/l8c 0 2024-04-25T01:35:48.875 INFO:tasks.workunit.client.0.smithi033.stdout:6/326: rename d4/d24/d63/l6b to d4/d1d/d3e/l78 0 2024-04-25T01:35:48.880 INFO:tasks.workunit.client.0.smithi033.stdout:2/434: mkdir da/d8d 0 2024-04-25T01:35:48.881 INFO:tasks.workunit.client.0.smithi033.stdout:6/327: mknod d4/d1d/d3e/d4a/c79 0 2024-04-25T01:35:48.881 INFO:tasks.workunit.client.0.smithi033.stdout:2/435: chown da/d27/d31/c38 2 1 2024-04-25T01:35:48.882 INFO:tasks.workunit.client.0.smithi033.stdout:5/400: dwrite d1c/d2a/d3d/f78 [0,4194304] 0 2024-04-25T01:35:48.885 INFO:tasks.workunit.client.0.smithi033.stdout:6/328: symlink d4/d1d/d3e/d41/l7a 0 2024-04-25T01:35:48.886 INFO:tasks.workunit.client.0.smithi033.stdout:5/401: mknod d1c/d2a/d50/c85 0 2024-04-25T01:35:48.887 INFO:tasks.workunit.client.0.smithi033.stdout:3/328: rename d8/dd/d11/d1e/d27/f36 to d8/dd/d11/d1e/f75 0 2024-04-25T01:35:48.892 INFO:tasks.workunit.client.0.smithi033.stdout:3/329: mkdir d8/dd/d3d/d63/d76 0 2024-04-25T01:35:48.893 INFO:tasks.workunit.client.0.smithi033.stdout:5/402: dread d1c/f23 [0,4194304] 0 2024-04-25T01:35:48.897 INFO:tasks.workunit.client.0.smithi033.stdout:3/330: symlink d8/dd/d3d/d63/l77 0 2024-04-25T01:35:48.898 INFO:tasks.workunit.client.0.smithi033.stdout:4/420: truncate d6/d14/d1d/f2c 6671230 0 2024-04-25T01:35:48.899 INFO:tasks.workunit.client.0.smithi033.stdout:4/421: dread - d6/d14/d1d/f20 zero size 2024-04-25T01:35:48.901 INFO:tasks.workunit.client.0.smithi033.stdout:5/403: rename d1c/d21/d4a/d69 to d1c/d2a/d50/d5b/d86 0 2024-04-25T01:35:48.905 INFO:tasks.workunit.client.0.smithi033.stdout:2/436: write da/d2c/f36 [710029,112073] 0 2024-04-25T01:35:48.907 INFO:tasks.workunit.client.0.smithi033.stdout:3/331: symlink d8/dd/d11/d1e/d27/d3c/d6e/l78 0 2024-04-25T01:35:48.908 INFO:tasks.workunit.client.0.smithi033.stdout:6/329: dread d4/d15/f48 [0,4194304] 0 2024-04-25T01:35:48.908 INFO:tasks.workunit.client.0.smithi033.stdout:4/422: symlink d6/d12/l79 0 2024-04-25T01:35:48.909 INFO:tasks.workunit.client.0.smithi033.stdout:5/404: mknod d1c/d2a/c87 0 2024-04-25T01:35:48.910 INFO:tasks.workunit.client.0.smithi033.stdout:2/437: truncate da/d5b/f63 676646 0 2024-04-25T01:35:48.915 INFO:tasks.workunit.client.0.smithi033.stdout:6/330: creat d4/d1d/d3e/d41/d54/f7b x:0 0 0 2024-04-25T01:35:48.915 INFO:tasks.workunit.client.0.smithi033.stdout:5/405: mknod d1c/d2a/d50/d52/c88 0 2024-04-25T01:35:48.920 INFO:tasks.workunit.client.0.smithi033.stdout:4/423: dwrite d6/d12/f23 [0,4194304] 0 2024-04-25T01:35:48.923 INFO:tasks.workunit.client.0.smithi033.stdout:5/406: mkdir d1c/d21/d3e/d70/d89 0 2024-04-25T01:35:48.925 INFO:tasks.workunit.client.0.smithi033.stdout:5/407: write d1c/d2a/d50/d52/f66 [2329730,107656] 0 2024-04-25T01:35:48.929 INFO:tasks.workunit.client.0.smithi033.stdout:2/438: creat da/f8e x:0 0 0 2024-04-25T01:35:48.930 INFO:tasks.workunit.client.0.smithi033.stdout:4/424: rename d6/d14/d35/d44/d45/f68 to d6/d14/d1d/f7a 0 2024-04-25T01:35:48.933 INFO:tasks.workunit.client.0.smithi033.stdout:3/332: write d8/dd/d11/f39 [124007,64171] 0 2024-04-25T01:35:48.933 INFO:tasks.workunit.client.0.smithi033.stdout:5/408: dwrite d1c/d21/d4a/f64 [0,4194304] 0 2024-04-25T01:35:48.945 INFO:tasks.workunit.client.0.smithi033.stdout:3/333: write d8/dd/d40/f65 [380460,100825] 0 2024-04-25T01:35:48.950 INFO:tasks.workunit.client.0.smithi033.stdout:6/331: write d4/d2a/f34 [2025653,105518] 0 2024-04-25T01:35:48.950 INFO:tasks.workunit.client.0.smithi033.stdout:5/409: mknod d1c/d21/d3e/d70/c8a 0 2024-04-25T01:35:48.951 INFO:tasks.workunit.client.0.smithi033.stdout:6/332: fdatasync d4/d24/d63/f6f 0 2024-04-25T01:35:48.952 INFO:tasks.workunit.client.0.smithi033.stdout:3/334: unlink d8/dd/d11/d1e/d27/c68 0 2024-04-25T01:35:48.952 INFO:tasks.workunit.client.0.smithi033.stdout:5/410: read d1c/f51 [1956500,66483] 0 2024-04-25T01:35:48.953 INFO:tasks.workunit.client.0.smithi033.stdout:4/425: dread d6/f1f [0,4194304] 0 2024-04-25T01:35:48.955 INFO:tasks.workunit.client.0.smithi033.stdout:2/439: creat da/d27/d31/d40/d8a/d80/f8f x:0 0 0 2024-04-25T01:35:48.958 INFO:tasks.workunit.client.0.smithi033.stdout:6/333: dread d4/d15/f48 [0,4194304] 0 2024-04-25T01:35:48.959 INFO:tasks.workunit.client.0.smithi033.stdout:3/335: symlink d8/dd/d3d/d4f/l79 0 2024-04-25T01:35:48.960 INFO:tasks.workunit.client.0.smithi033.stdout:3/336: chown d8/dd/d11 5 1 2024-04-25T01:35:48.961 INFO:tasks.workunit.client.0.smithi033.stdout:3/337: chown d8/dd/d1d/f2b 159 1 2024-04-25T01:35:48.962 INFO:tasks.workunit.client.0.smithi033.stdout:4/426: rename d6/d12/f17 to d6/d14/d35/d44/d45/d46/d57/d65/f7b 0 2024-04-25T01:35:48.962 INFO:tasks.workunit.client.0.smithi033.stdout:5/411: dwrite d1c/d2a/d3d/f40 [0,4194304] 0 2024-04-25T01:35:48.963 INFO:tasks.workunit.client.0.smithi033.stdout:3/338: fdatasync d8/dd/d11/d1e/d27/f70 0 2024-04-25T01:35:48.964 INFO:tasks.workunit.client.0.smithi033.stdout:2/440: creat da/d5b/d5e/d76/f90 x:0 0 0 2024-04-25T01:35:48.964 INFO:tasks.workunit.client.0.smithi033.stdout:5/412: dread - d1c/d21/d4a/f4e zero size 2024-04-25T01:35:48.965 INFO:tasks.workunit.client.0.smithi033.stdout:5/413: fdatasync d1c/d2a/f47 0 2024-04-25T01:35:48.966 INFO:tasks.workunit.client.0.smithi033.stdout:3/339: write d8/f37 [3786914,13538] 0 2024-04-25T01:35:48.967 INFO:tasks.workunit.client.0.smithi033.stdout:6/334: mkdir d4/d24/d7c 0 2024-04-25T01:35:48.968 INFO:tasks.workunit.client.0.smithi033.stdout:6/335: readlink d4/l59 0 2024-04-25T01:35:48.969 INFO:tasks.workunit.client.0.smithi033.stdout:6/336: dread - d4/d1d/d3e/f62 zero size 2024-04-25T01:35:48.976 INFO:tasks.workunit.client.0.smithi033.stdout:2/441: rename da/d27/f6d to da/d5b/d5e/f91 0 2024-04-25T01:35:48.977 INFO:tasks.workunit.client.0.smithi033.stdout:6/337: dwrite d4/d24/d63/f6f [0,4194304] 0 2024-04-25T01:35:48.978 INFO:tasks.workunit.client.0.smithi033.stdout:3/340: chown d8/dd/d11/l42 10343066 1 2024-04-25T01:35:48.978 INFO:tasks.workunit.client.0.smithi033.stdout:6/338: chown d4/d15/c16 394426 1 2024-04-25T01:35:48.979 INFO:tasks.workunit.client.0.smithi033.stdout:4/427: write d6/d12/f40 [2983607,46559] 0 2024-04-25T01:35:48.980 INFO:tasks.workunit.client.0.smithi033.stdout:5/414: symlink d1c/d21/d3e/l8b 0 2024-04-25T01:35:48.984 INFO:tasks.workunit.client.0.smithi033.stdout:3/341: chown d8/dd/d11/c50 19 1 2024-04-25T01:35:48.987 INFO:tasks.workunit.client.0.smithi033.stdout:4/428: mkdir d6/d14/d35/d3a/d4d/d5e/d7c 0 2024-04-25T01:35:48.988 INFO:tasks.workunit.client.0.smithi033.stdout:2/442: mkdir da/d5b/d5e/d7c/d92 0 2024-04-25T01:35:48.988 INFO:tasks.workunit.client.0.smithi033.stdout:6/339: dwrite d4/d1d/d3e/d70/f72 [0,4194304] 0 2024-04-25T01:35:48.992 INFO:tasks.workunit.client.0.smithi033.stdout:6/340: mknod d4/d1d/d3e/d4a/c7d 0 2024-04-25T01:35:48.993 INFO:tasks.workunit.client.0.smithi033.stdout:4/429: mknod d6/d14/d35/d44/d45/d46/d57/c7d 0 2024-04-25T01:35:48.999 INFO:tasks.workunit.client.0.smithi033.stdout:6/341: rename d4/d15/l17 to d4/d15/l7e 0 2024-04-25T01:35:49.000 INFO:tasks.workunit.client.0.smithi033.stdout:3/342: dwrite d8/dd/d40/f43 [0,4194304] 0 2024-04-25T01:35:49.002 INFO:tasks.workunit.client.0.smithi033.stdout:4/430: creat d6/d12/d38/f7e x:0 0 0 2024-04-25T01:35:49.005 INFO:tasks.workunit.client.0.smithi033.stdout:6/342: write d4/d2a/f30 [2157242,62897] 0 2024-04-25T01:35:49.005 INFO:tasks.workunit.client.0.smithi033.stdout:5/415: dwrite f7 [0,4194304] 0 2024-04-25T01:35:49.005 INFO:tasks.workunit.client.0.smithi033.stdout:2/443: dwrite da/d50/d6a/d43/f66 [0,4194304] 0 2024-04-25T01:35:49.009 INFO:tasks.workunit.client.0.smithi033.stdout:5/416: write d1c/f22 [4444927,53807] 0 2024-04-25T01:35:49.009 INFO:tasks.workunit.client.0.smithi033.stdout:3/343: symlink d8/dd/d1d/l7a 0 2024-04-25T01:35:49.009 INFO:tasks.workunit.client.0.smithi033.stdout:2/444: dread - da/d27/d31/d40/d77/f7b zero size 2024-04-25T01:35:49.012 INFO:tasks.workunit.client.0.smithi033.stdout:6/343: write d4/d15/f5f [690219,36294] 0 2024-04-25T01:35:49.014 INFO:tasks.workunit.client.0.smithi033.stdout:6/344: write d4/f14 [2336518,49059] 0 2024-04-25T01:35:49.017 INFO:tasks.workunit.client.0.smithi033.stdout:5/417: rmdir d1c 39 2024-04-25T01:35:49.018 INFO:tasks.workunit.client.0.smithi033.stdout:2/445: mknod da/d5b/d72/c93 0 2024-04-25T01:35:49.020 INFO:tasks.workunit.client.0.smithi033.stdout:3/344: mknod d8/dd/d40/c7b 0 2024-04-25T01:35:49.022 INFO:tasks.workunit.client.0.smithi033.stdout:3/345: write d8/dd/d3d/d4f/f6f [920119,48311] 0 2024-04-25T01:35:49.023 INFO:tasks.workunit.client.0.smithi033.stdout:5/418: chown d1c/d21/l79 48338 1 2024-04-25T01:35:49.025 INFO:tasks.workunit.client.0.smithi033.stdout:6/345: rename d4/d24/d36/d37 to d4/d1d/d3e/d41/d5c/d7f 0 2024-04-25T01:35:49.026 INFO:tasks.workunit.client.0.smithi033.stdout:2/446: creat da/d27/d31/d40/d8a/f94 x:0 0 0 2024-04-25T01:35:49.031 INFO:tasks.workunit.client.0.smithi033.stdout:4/431: dread d6/f8 [4194304,4194304] 0 2024-04-25T01:35:49.037 INFO:tasks.workunit.client.0.smithi033.stdout:3/346: dwrite d8/f23 [0,4194304] 0 2024-04-25T01:35:49.041 INFO:tasks.workunit.client.0.smithi033.stdout:5/419: write d1c/d2a/d50/d5b/f77 [561708,113322] 0 2024-04-25T01:35:49.045 INFO:tasks.workunit.client.0.smithi033.stdout:6/346: dwrite d4/d2a/f30 [0,4194304] 0 2024-04-25T01:35:49.047 INFO:tasks.workunit.client.0.smithi033.stdout:3/347: unlink d8/dd/d40/f4e 0 2024-04-25T01:35:49.049 INFO:tasks.workunit.client.0.smithi033.stdout:2/447: dwrite da/d5b/d5e/f6f [0,4194304] 0 2024-04-25T01:35:49.049 INFO:tasks.workunit.client.0.smithi033.stdout:5/420: rmdir d1c/d2a/d3d 39 2024-04-25T01:35:49.050 INFO:tasks.workunit.client.0.smithi033.stdout:2/448: readlink da/d50/l60 0 2024-04-25T01:35:49.050 INFO:tasks.workunit.client.0.smithi033.stdout:6/347: rmdir d4/d1d/d3e 39 2024-04-25T01:35:49.051 INFO:tasks.workunit.client.0.smithi033.stdout:2/449: fsync da/f45 0 2024-04-25T01:35:49.052 INFO:tasks.workunit.client.0.smithi033.stdout:3/348: rmdir d8/dd/d3d 39 2024-04-25T01:35:49.059 INFO:tasks.workunit.client.0.smithi033.stdout:2/450: rmdir da/d27/d31/d40 39 2024-04-25T01:35:49.062 INFO:tasks.workunit.client.0.smithi033.stdout:5/421: dwrite d1c/d2a/d3d/f40 [0,4194304] 0 2024-04-25T01:35:49.064 INFO:tasks.workunit.client.0.smithi033.stdout:5/422: readlink d1c/d2a/l67 0 2024-04-25T01:35:49.064 INFO:tasks.workunit.client.0.smithi033.stdout:2/451: creat da/d5b/d72/f95 x:0 0 0 2024-04-25T01:35:49.068 INFO:tasks.workunit.client.0.smithi033.stdout:3/349: rename d8/dd/d11/d1e/d27 to d8/dd/d3d/d63/d7c 0 2024-04-25T01:35:49.073 INFO:tasks.workunit.client.0.smithi033.stdout:5/423: rename d1c/f38 to d1c/d21/d4a/f8c 0 2024-04-25T01:35:49.073 INFO:tasks.workunit.client.0.smithi033.stdout:2/452: dwrite da/f18 [4194304,4194304] 0 2024-04-25T01:35:49.075 INFO:tasks.workunit.client.0.smithi033.stdout:4/432: dread d6/d14/d1d/d3b/f52 [0,4194304] 0 2024-04-25T01:35:49.076 INFO:tasks.workunit.client.0.smithi033.stdout:5/424: write d1c/d21/f81 [720429,47465] 0 2024-04-25T01:35:49.078 INFO:tasks.workunit.client.0.smithi033.stdout:5/425: stat l13 0 2024-04-25T01:35:49.078 INFO:tasks.workunit.client.0.smithi033.stdout:3/350: rename d8/dd/d1d/f5f to d8/dd/d3d/d63/d7c/d3c/f7d 0 2024-04-25T01:35:49.079 INFO:tasks.workunit.client.0.smithi033.stdout:4/433: truncate d6/d12/f32 4202264 0 2024-04-25T01:35:49.080 INFO:tasks.workunit.client.0.smithi033.stdout:2/453: symlink da/d5b/d5e/d76/l96 0 2024-04-25T01:35:49.083 INFO:tasks.workunit.client.0.smithi033.stdout:3/351: creat d8/dd/d1d/f7e x:0 0 0 2024-04-25T01:35:49.084 INFO:tasks.workunit.client.0.smithi033.stdout:5/426: mkdir d1c/d2a/d50/d8d 0 2024-04-25T01:35:49.085 INFO:tasks.workunit.client.0.smithi033.stdout:4/434: mknod d6/d14/d35/d3a/d4d/d5e/c7f 0 2024-04-25T01:35:49.089 INFO:tasks.workunit.client.0.smithi033.stdout:3/352: creat d8/dd/d11/f7f x:0 0 0 2024-04-25T01:35:49.089 INFO:tasks.workunit.client.0.smithi033.stdout:2/454: dread da/d27/f47 [0,4194304] 0 2024-04-25T01:35:49.090 INFO:tasks.workunit.client.0.smithi033.stdout:5/427: creat d1c/d21/f8e x:0 0 0 2024-04-25T01:35:49.091 INFO:tasks.workunit.client.0.smithi033.stdout:5/428: write d1c/d53/f57 [999983,6413] 0 2024-04-25T01:35:49.092 INFO:tasks.workunit.client.0.smithi033.stdout:2/455: fdatasync da/d27/d31/d40/d8a/f94 0 2024-04-25T01:35:49.093 INFO:tasks.workunit.client.0.smithi033.stdout:6/348: dread d4/d15/f20 [4194304,4194304] 0 2024-04-25T01:35:49.098 INFO:tasks.workunit.client.0.smithi033.stdout:6/349: rename d4/f11 to d4/d24/d7c/f80 0 2024-04-25T01:35:49.100 INFO:tasks.workunit.client.0.smithi033.stdout:5/429: creat d1c/d21/f8f x:0 0 0 2024-04-25T01:35:49.101 INFO:tasks.workunit.client.0.smithi033.stdout:5/430: write d1c/d2a/d50/f7d [1658318,20756] 0 2024-04-25T01:35:49.104 INFO:tasks.workunit.client.0.smithi033.stdout:4/435: write d6/d14/f31 [4799334,44345] 0 2024-04-25T01:35:49.111 INFO:tasks.workunit.client.0.smithi033.stdout:4/436: rename d6/d14/d35/d44/d45/d46/d57/d65 to d6/d14/d1d/d80 0 2024-04-25T01:35:49.111 INFO:tasks.workunit.client.0.smithi033.stdout:2/456: truncate da/d2c/f48 3486625 0 2024-04-25T01:35:49.111 INFO:tasks.workunit.client.0.smithi033.stdout:4/437: dread - d6/d14/d35/d44/d45/d46/d64/f69 zero size 2024-04-25T01:35:49.112 INFO:tasks.workunit.client.0.smithi033.stdout:4/438: stat d6/d12/l79 0 2024-04-25T01:35:49.114 INFO:tasks.workunit.client.0.smithi033.stdout:5/431: truncate d1c/d21/f81 533478 0 2024-04-25T01:35:49.114 INFO:tasks.workunit.client.0.smithi033.stdout:4/439: rename d6/d14/d35/d44/d45 to d6/d14/d35/d44/d45/d81 22 2024-04-25T01:35:49.119 INFO:tasks.workunit.client.0.smithi033.stdout:4/440: truncate d6/d14/d1d/f6f 1035275 0 2024-04-25T01:35:49.119 INFO:tasks.workunit.client.0.smithi033.stdout:2/457: dwrite da/d27/f69 [0,4194304] 0 2024-04-25T01:35:49.122 INFO:tasks.workunit.client.0.smithi033.stdout:5/432: rename d1c/f48 to d1c/d2a/d50/f90 0 2024-04-25T01:35:49.125 INFO:tasks.workunit.client.0.smithi033.stdout:4/441: truncate d6/d14/f2b 177966 0 2024-04-25T01:35:49.127 INFO:tasks.workunit.client.0.smithi033.stdout:5/433: rename d1c/d2a/d50/d5b/f77 to d1c/d2a/d50/f91 0 2024-04-25T01:35:49.128 INFO:tasks.workunit.client.0.smithi033.stdout:4/442: dread d6/d14/d1d/f6f [0,4194304] 0 2024-04-25T01:35:49.130 INFO:tasks.workunit.client.0.smithi033.stdout:4/443: rename d6 to d6/d14/d35/d44/d45/d82 22 2024-04-25T01:35:49.131 INFO:tasks.workunit.client.0.smithi033.stdout:4/444: readlink d6/d14/d35/l36 0 2024-04-25T01:35:49.131 INFO:tasks.workunit.client.0.smithi033.stdout:4/445: write d6/d14/f31 [675283,129124] 0 2024-04-25T01:35:49.133 INFO:tasks.workunit.client.0.smithi033.stdout:4/446: write d6/d14/d1d/d3b/f74 [691388,50995] 0 2024-04-25T01:35:49.133 INFO:tasks.workunit.client.0.smithi033.stdout:5/434: truncate d1c/d21/d4a/f61 3020494 0 2024-04-25T01:35:49.134 INFO:tasks.workunit.client.0.smithi033.stdout:4/447: readlink d6/l6c 0 2024-04-25T01:35:49.134 INFO:tasks.workunit.client.0.smithi033.stdout:5/435: write f14 [2705464,1231] 0 2024-04-25T01:35:49.136 INFO:tasks.workunit.client.0.smithi033.stdout:5/436: chown d1c/d2a/d50/d52/f66 3658 1 2024-04-25T01:35:49.137 INFO:tasks.workunit.client.0.smithi033.stdout:4/448: fsync d6/d14/d35/d3a/f49 0 2024-04-25T01:35:49.137 INFO:tasks.workunit.client.0.smithi033.stdout:5/437: write d1c/f1d [6234298,30892] 0 2024-04-25T01:35:49.139 INFO:tasks.workunit.client.0.smithi033.stdout:5/438: chown d1c/d2a/d50/f6a 2494 1 2024-04-25T01:35:49.139 INFO:tasks.workunit.client.0.smithi033.stdout:5/439: stat f7 0 2024-04-25T01:35:49.141 INFO:tasks.workunit.client.0.smithi033.stdout:4/449: link d6/d12/f40 d6/d14/d6d/f83 0 2024-04-25T01:35:49.142 INFO:tasks.workunit.client.0.smithi033.stdout:5/440: rmdir d1c/d21/d3e/d70/d7a 39 2024-04-25T01:35:49.142 INFO:tasks.workunit.client.0.smithi033.stdout:4/450: chown d6/l6c 438543753 1 2024-04-25T01:35:49.143 INFO:tasks.workunit.client.0.smithi033.stdout:5/441: write d1c/f37 [57940,49521] 0 2024-04-25T01:35:49.144 INFO:tasks.workunit.client.0.smithi033.stdout:4/451: mkdir d6/d12/d84 0 2024-04-25T01:35:49.146 INFO:tasks.workunit.client.0.smithi033.stdout:5/442: rename d1c/d2a/d50/f6a to d1c/d2a/d3d/f92 0 2024-04-25T01:35:49.148 INFO:tasks.workunit.client.0.smithi033.stdout:4/452: getdents d6/d12/d84 0 2024-04-25T01:35:49.150 INFO:tasks.workunit.client.0.smithi033.stdout:4/453: truncate d6/d12/d38/f7e 747970 0 2024-04-25T01:35:49.151 INFO:tasks.workunit.client.0.smithi033.stdout:5/443: rename ca to d1c/d21/c93 0 2024-04-25T01:35:49.152 INFO:tasks.workunit.client.0.smithi033.stdout:4/454: creat d6/d14/d48/f85 x:0 0 0 2024-04-25T01:35:49.155 INFO:tasks.workunit.client.0.smithi033.stdout:5/444: truncate d1c/d21/f63 2966128 0 2024-04-25T01:35:49.158 INFO:tasks.workunit.client.0.smithi033.stdout:4/455: getdents d6/d14/d35/d3a/d4c 0 2024-04-25T01:35:49.160 INFO:tasks.workunit.client.0.smithi033.stdout:5/445: symlink d1c/d21/d3e/d70/d7a/d7e/l94 0 2024-04-25T01:35:49.160 INFO:tasks.workunit.client.0.smithi033.stdout:4/456: symlink d6/d12/d38/l86 0 2024-04-25T01:35:49.166 INFO:tasks.workunit.client.0.smithi033.stdout:5/446: getdents d1c/d21/d3e/d70 0 2024-04-25T01:35:49.168 INFO:tasks.workunit.client.0.smithi033.stdout:5/447: creat d1c/d2a/d3d/f95 x:0 0 0 2024-04-25T01:35:49.170 INFO:tasks.workunit.client.0.smithi033.stdout:5/448: symlink d1c/d21/d3e/d70/l96 0 2024-04-25T01:35:49.171 INFO:tasks.workunit.client.0.smithi033.stdout:5/449: chown d1c/d2a/d3d/f3f 119 1 2024-04-25T01:35:49.171 INFO:tasks.workunit.client.0.smithi033.stdout:5/450: fdatasync d1c/d53/f57 0 2024-04-25T01:35:49.172 INFO:tasks.workunit.client.0.smithi033.stdout:5/451: fdatasync d1c/d2a/d3d/f3f 0 2024-04-25T01:35:49.174 INFO:tasks.workunit.client.0.smithi033.stdout:5/452: symlink d1c/d21/d3e/l97 0 2024-04-25T01:35:49.176 INFO:tasks.workunit.client.0.smithi033.stdout:5/453: creat d1c/f98 x:0 0 0 2024-04-25T01:35:49.182 INFO:tasks.workunit.client.0.smithi033.stdout:5/454: dwrite d1c/d21/d4a/f8c [0,4194304] 0 2024-04-25T01:35:49.183 INFO:tasks.workunit.client.0.smithi033.stdout:5/455: stat d1c/d2a/d3d/c41 0 2024-04-25T01:35:49.183 INFO:tasks.workunit.client.0.smithi033.stdout:5/456: dread - d1c/d2a/d3d/f95 zero size 2024-04-25T01:35:49.185 INFO:tasks.workunit.client.0.smithi033.stdout:5/457: creat d1c/d53/f99 x:0 0 0 2024-04-25T01:35:49.186 INFO:tasks.workunit.client.0.smithi033.stdout:5/458: write d1c/d21/f8e [767844,122517] 0 2024-04-25T01:35:49.187 INFO:tasks.workunit.client.0.smithi033.stdout:5/459: fsync d1c/d21/f8f 0 2024-04-25T01:35:49.254 INFO:tasks.workunit.client.0.smithi033.stdout:9/305: sync 2024-04-25T01:35:49.318 INFO:tasks.workunit.client.0.smithi033.stdout:1/285: sync 2024-04-25T01:35:49.318 INFO:tasks.workunit.client.0.smithi033.stdout:0/303: sync 2024-04-25T01:35:49.318 INFO:tasks.workunit.client.0.smithi033.stdout:0/304: chown d7/d2d/d5e 14983975 1 2024-04-25T01:35:49.318 INFO:tasks.workunit.client.0.smithi033.stdout:1/286: write d2/d11/f52 [991584,49246] 0 2024-04-25T01:35:49.318 INFO:tasks.workunit.client.0.smithi033.stdout:2/458: write da/d27/d31/f5a [1363875,97421] 0 2024-04-25T01:35:49.318 INFO:tasks.workunit.client.0.smithi033.stdout:2/459: chown da/l21 58 1 2024-04-25T01:35:49.318 INFO:tasks.workunit.client.0.smithi033.stdout:0/305: rmdir d7/da/d23 39 2024-04-25T01:35:49.323 INFO:tasks.workunit.client.0.smithi033.stdout:0/306: rename d7/da/df/d14/f22 to d7/d2d/d5e/d61/f66 0 2024-04-25T01:35:49.324 INFO:tasks.workunit.client.0.smithi033.stdout:2/460: dread da/d27/d31/d40/f5f [0,4194304] 0 2024-04-25T01:35:49.325 INFO:tasks.workunit.client.0.smithi033.stdout:1/287: truncate d2/d11/d28/d33/f36 2588689 0 2024-04-25T01:35:49.326 INFO:tasks.workunit.client.0.smithi033.stdout:2/461: readlink da/d2c/l4e 0 2024-04-25T01:35:49.331 INFO:tasks.workunit.client.0.smithi033.stdout:9/306: dread d0/d2/dd/f56 [0,4194304] 0 2024-04-25T01:35:49.331 INFO:tasks.workunit.client.0.smithi033.stdout:9/307: readlink d0/d2/l1a 0 2024-04-25T01:35:49.332 INFO:tasks.workunit.client.0.smithi033.stdout:0/307: dwrite d7/da/f58 [0,4194304] 0 2024-04-25T01:35:49.332 INFO:tasks.workunit.client.0.smithi033.stdout:2/462: unlink da/d5b/d5e/f91 0 2024-04-25T01:35:49.333 INFO:tasks.workunit.client.0.smithi033.stdout:9/308: fdatasync d0/d2/dd/d20/d4e/f4f 0 2024-04-25T01:35:49.335 INFO:tasks.workunit.client.0.smithi033.stdout:2/463: read da/d27/d31/d40/f82 [753148,81552] 0 2024-04-25T01:35:49.338 INFO:tasks.workunit.client.0.smithi033.stdout:2/464: truncate da/d5b/d5e/d7c/f68 348927 0 2024-04-25T01:35:49.339 INFO:tasks.workunit.client.0.smithi033.stdout:1/288: link d2/d11/f52 d2/d11/d28/d33/f53 0 2024-04-25T01:35:49.340 INFO:tasks.workunit.client.0.smithi033.stdout:2/465: write da/d27/f7a [390405,41804] 0 2024-04-25T01:35:49.341 INFO:tasks.workunit.client.0.smithi033.stdout:2/466: chown da/d27/c29 104952558 1 2024-04-25T01:35:49.346 INFO:tasks.workunit.client.0.smithi033.stdout:1/289: unlink d2/d11/f52 0 2024-04-25T01:35:49.346 INFO:tasks.workunit.client.0.smithi033.stdout:2/467: dread da/f18 [4194304,4194304] 0 2024-04-25T01:35:49.347 INFO:tasks.workunit.client.0.smithi033.stdout:1/290: write d2/f26 [4889824,124593] 0 2024-04-25T01:35:49.351 INFO:tasks.workunit.client.0.smithi033.stdout:0/308: creat d7/d2d/f67 x:0 0 0 2024-04-25T01:35:49.352 INFO:tasks.workunit.client.0.smithi033.stdout:4/457: rmdir d6 39 2024-04-25T01:35:49.353 INFO:tasks.workunit.client.0.smithi033.stdout:6/350: sync 2024-04-25T01:35:49.353 INFO:tasks.workunit.client.0.smithi033.stdout:8/451: sync 2024-04-25T01:35:49.353 INFO:tasks.workunit.client.0.smithi033.stdout:3/353: sync 2024-04-25T01:35:49.354 INFO:tasks.workunit.client.0.smithi033.stdout:7/425: sync 2024-04-25T01:35:49.359 INFO:tasks.workunit.client.0.smithi033.stdout:0/309: dread d7/d33/f3e [0,4194304] 0 2024-04-25T01:35:49.360 INFO:tasks.workunit.client.0.smithi033.stdout:2/468: mknod da/c97 0 2024-04-25T01:35:49.362 INFO:tasks.workunit.client.0.smithi033.stdout:0/310: write d7/d2d/d43/d44/f63 [735919,1703] 0 2024-04-25T01:35:49.362 INFO:tasks.workunit.client.0.smithi033.stdout:0/311: fsync d7/d1b/f32 0 2024-04-25T01:35:49.363 INFO:tasks.workunit.client.0.smithi033.stdout:6/351: chown d4/d1d/d3e/d41/d54/f7b 1 1 2024-04-25T01:35:49.374 INFO:tasks.workunit.client.0.smithi033.stdout:0/312: chown d7/d2d/d4d 149628985 1 2024-04-25T01:35:49.378 INFO:tasks.workunit.client.0.smithi033.stdout:5/460: mknod d1c/d21/d3e/d70/d7a/d7e/c9a 0 2024-04-25T01:35:49.379 INFO:tasks.workunit.client.0.smithi033.stdout:9/309: write d0/d2/dd/f17 [1436723,53673] 0 2024-04-25T01:35:49.381 INFO:tasks.workunit.client.0.smithi033.stdout:1/291: rename d2/f4f to d2/d5/d1b/f54 0 2024-04-25T01:35:49.383 INFO:tasks.workunit.client.0.smithi033.stdout:0/313: mkdir d7/d2d/d5e/d68 0 2024-04-25T01:35:49.384 INFO:tasks.workunit.client.0.smithi033.stdout:7/426: creat d4/d10/f89 x:0 0 0 2024-04-25T01:35:49.385 INFO:tasks.workunit.client.0.smithi033.stdout:0/314: truncate d7/da/df/d14/f47 257614 0 2024-04-25T01:35:49.385 INFO:tasks.workunit.client.0.smithi033.stdout:7/427: dread - d4/d2e/d65/f68 zero size 2024-04-25T01:35:49.386 INFO:tasks.workunit.client.0.smithi033.stdout:7/428: read f3 [2099371,87980] 0 2024-04-25T01:35:49.392 INFO:tasks.workunit.client.0.smithi033.stdout:9/310: mkdir d0/d2/dd/df/d1b/d3d/d6e 0 2024-04-25T01:35:49.393 INFO:tasks.workunit.client.0.smithi033.stdout:8/452: dwrite d6/db/f18 [0,4194304] 0 2024-04-25T01:35:49.393 INFO:tasks.workunit.client.0.smithi033.stdout:9/311: chown d0/d2/dd/d20 9 1 2024-04-25T01:35:49.395 INFO:tasks.workunit.client.0.smithi033.stdout:3/354: rename d8/c17 to d8/dd/d3d/d63/d7c/d3c/d6e/c80 0 2024-04-25T01:35:49.400 INFO:tasks.workunit.client.0.smithi033.stdout:2/469: creat da/d5b/d5e/d7c/f98 x:0 0 0 2024-04-25T01:35:49.400 INFO:tasks.workunit.client.0.smithi033.stdout:2/470: readlink da/d2c/l4e 0 2024-04-25T01:35:49.401 INFO:tasks.workunit.client.0.smithi033.stdout:3/355: dread d8/dd/d3d/d63/d7c/d4d/f62 [0,4194304] 0 2024-04-25T01:35:49.402 INFO:tasks.workunit.client.0.smithi033.stdout:2/471: fsync da/d27/d31/f4f 0 2024-04-25T01:35:49.403 INFO:tasks.workunit.client.0.smithi033.stdout:0/315: creat d7/da/df/d14/f69 x:0 0 0 2024-04-25T01:35:49.403 INFO:tasks.workunit.client.0.smithi033.stdout:2/472: dread - da/d27/d31/d40/f88 zero size 2024-04-25T01:35:49.404 INFO:tasks.workunit.client.0.smithi033.stdout:7/429: mkdir d4/d23/d3f/d45/d73/d8a 0 2024-04-25T01:35:49.406 INFO:tasks.workunit.client.0.smithi033.stdout:7/430: chown d4/d2e/d6b/d1f/d78/d86/c88 40187 1 2024-04-25T01:35:49.407 INFO:tasks.workunit.client.0.smithi033.stdout:5/461: write d1c/f23 [182181,33313] 0 2024-04-25T01:35:49.409 INFO:tasks.workunit.client.0.smithi033.stdout:4/458: getdents d6 0 2024-04-25T01:35:49.410 INFO:tasks.workunit.client.0.smithi033.stdout:4/459: chown d6/d14/d35/d44/d45/d46 1998 1 2024-04-25T01:35:49.411 INFO:tasks.workunit.client.0.smithi033.stdout:8/453: unlink d6/d45/f73 0 2024-04-25T01:35:49.412 INFO:tasks.workunit.client.0.smithi033.stdout:6/352: rename d4/f14 to d4/d1d/d3e/d41/d5c/d7f/f81 0 2024-04-25T01:35:49.413 INFO:tasks.workunit.client.0.smithi033.stdout:9/312: rename d0/d2/dd/d20 to d0/d2/dd/d20/d6f 22 2024-04-25T01:35:49.413 INFO:tasks.workunit.client.0.smithi033.stdout:1/292: mknod d2/d11/d28/c55 0 2024-04-25T01:35:49.414 INFO:tasks.workunit.client.0.smithi033.stdout:4/460: dread d6/d14/f22 [0,4194304] 0 2024-04-25T01:35:49.415 INFO:tasks.workunit.client.0.smithi033.stdout:6/353: truncate d4/d1d/d3e/d4a/f4f 235231 0 2024-04-25T01:35:49.416 INFO:tasks.workunit.client.0.smithi033.stdout:3/356: mknod d8/dd/d3d/d63/d7c/d3c/d6e/c81 0 2024-04-25T01:35:49.417 INFO:tasks.workunit.client.0.smithi033.stdout:6/354: write d4/d1d/d3e/d41/d54/f7b [296586,1580] 0 2024-04-25T01:35:49.418 INFO:tasks.workunit.client.0.smithi033.stdout:2/473: creat da/d50/d6a/d43/f99 x:0 0 0 2024-04-25T01:35:49.418 INFO:tasks.workunit.client.0.smithi033.stdout:3/357: dread - d8/dd/d3d/d63/d7c/d4d/f6c zero size 2024-04-25T01:35:49.420 INFO:tasks.workunit.client.0.smithi033.stdout:6/355: write d4/d1d/d3e/d41/d5c/d7f/f44 [1041777,116051] 0 2024-04-25T01:35:49.421 INFO:tasks.workunit.client.0.smithi033.stdout:3/358: chown d8/dd/d3d/d4f/f6f 162 1 2024-04-25T01:35:49.421 INFO:tasks.workunit.client.0.smithi033.stdout:7/431: dread d4/d2e/d50/f6e [0,4194304] 0 2024-04-25T01:35:49.421 INFO:tasks.workunit.client.0.smithi033.stdout:8/454: mknod d6/db/d80/d89/ca7 0 2024-04-25T01:35:49.422 INFO:tasks.workunit.client.0.smithi033.stdout:7/432: stat d4/d2e/d6b/d1f/c28 0 2024-04-25T01:35:49.423 INFO:tasks.workunit.client.0.smithi033.stdout:1/293: mknod d2/d2e/d34/c56 0 2024-04-25T01:35:49.425 INFO:tasks.workunit.client.0.smithi033.stdout:4/461: mkdir d6/d14/d35/d44/d45/d46/d57/d70/d87 0 2024-04-25T01:35:49.426 INFO:tasks.workunit.client.0.smithi033.stdout:0/316: mknod d7/da/df/d14/d45/c6a 0 2024-04-25T01:35:49.428 INFO:tasks.workunit.client.0.smithi033.stdout:5/462: mkdir d1c/d21/d3e/d70/d9b 0 2024-04-25T01:35:49.429 INFO:tasks.workunit.client.0.smithi033.stdout:2/474: symlink da/d5b/d5e/d76/l9a 0 2024-04-25T01:35:49.430 INFO:tasks.workunit.client.0.smithi033.stdout:6/356: symlink d4/d1d/d3e/d4a/l82 0 2024-04-25T01:35:49.432 INFO:tasks.workunit.client.0.smithi033.stdout:0/317: dwrite d7/d33/f36 [0,4194304] 0 2024-04-25T01:35:49.434 INFO:tasks.workunit.client.0.smithi033.stdout:9/313: link d0/d2/dd/f15 d0/d2/dd/df/d1b/d3d/f70 0 2024-04-25T01:35:49.436 INFO:tasks.workunit.client.0.smithi033.stdout:1/294: fsync d2/d5/f18 0 2024-04-25T01:35:49.437 INFO:tasks.workunit.client.0.smithi033.stdout:4/462: stat d6/d14/d1d/f2e 0 2024-04-25T01:35:49.439 INFO:tasks.workunit.client.0.smithi033.stdout:2/475: creat da/d27/d31/d40/d77/f9b x:0 0 0 2024-04-25T01:35:49.442 INFO:tasks.workunit.client.0.smithi033.stdout:2/476: chown da/d27/d31/c67 350197 1 2024-04-25T01:35:49.446 INFO:tasks.workunit.client.0.smithi033.stdout:1/295: dwrite d2/f26 [4194304,4194304] 0 2024-04-25T01:35:49.449 INFO:tasks.workunit.client.0.smithi033.stdout:7/433: link d4/l12 d4/d2e/d6b/d1f/l8b 0 2024-04-25T01:35:49.452 INFO:tasks.workunit.client.0.smithi033.stdout:6/357: creat d4/d24/d36/d42/f83 x:0 0 0 2024-04-25T01:35:49.453 INFO:tasks.workunit.client.0.smithi033.stdout:6/358: fdatasync d4/d1d/f1f 0 2024-04-25T01:35:49.454 INFO:tasks.workunit.client.0.smithi033.stdout:0/318: creat d7/da/df/d50/f6b x:0 0 0 2024-04-25T01:35:49.459 INFO:tasks.workunit.client.0.smithi033.stdout:5/463: rename d1c/d21/l2d to d1c/d21/d3e/d70/l9c 0 2024-04-25T01:35:49.461 INFO:tasks.workunit.client.0.smithi033.stdout:6/359: dwrite d4/f12 [4194304,4194304] 0 2024-04-25T01:35:49.461 INFO:tasks.workunit.client.0.smithi033.stdout:4/463: creat d6/d12/d84/f88 x:0 0 0 2024-04-25T01:35:49.462 INFO:tasks.workunit.client.0.smithi033.stdout:5/464: read d1c/f1e [1689716,123167] 0 2024-04-25T01:35:49.463 INFO:tasks.workunit.client.0.smithi033.stdout:4/464: fdatasync d6/d14/d6d/f77 0 2024-04-25T01:35:49.465 INFO:tasks.workunit.client.0.smithi033.stdout:4/465: fsync d6/d14/d48/f85 0 2024-04-25T01:35:49.466 INFO:tasks.workunit.client.0.smithi033.stdout:7/434: symlink d4/d2e/d7f/l8c 0 2024-04-25T01:35:49.466 INFO:tasks.workunit.client.0.smithi033.stdout:8/455: getdents d6/d45/d51 0 2024-04-25T01:35:49.466 INFO:tasks.workunit.client.0.smithi033.stdout:6/360: write d4/d1d/d3e/d70/f72 [2779428,51439] 0 2024-04-25T01:35:49.469 INFO:tasks.workunit.client.0.smithi033.stdout:3/359: truncate d8/dd/d3d/d63/d7c/d4d/f62 3024741 0 2024-04-25T01:35:49.470 INFO:tasks.workunit.client.0.smithi033.stdout:8/456: chown d6/db/f18 69784 1 2024-04-25T01:35:49.473 INFO:tasks.workunit.client.0.smithi033.stdout:2/477: dread da/d27/d31/f5a [0,4194304] 0 2024-04-25T01:35:49.473 INFO:tasks.workunit.client.0.smithi033.stdout:6/361: truncate d4/d24/d36/d42/f83 963372 0 2024-04-25T01:35:49.473 INFO:tasks.workunit.client.0.smithi033.stdout:2/478: fdatasync f7 0 2024-04-25T01:35:49.475 INFO:tasks.workunit.client.0.smithi033.stdout:2/479: write da/d5b/d5e/f6f [4493699,16877] 0 2024-04-25T01:35:49.475 INFO:tasks.workunit.client.0.smithi033.stdout:2/480: stat da/ff 0 2024-04-25T01:35:49.475 INFO:tasks.workunit.client.0.smithi033.stdout:7/435: mknod d4/d23/d3f/d45/c8d 0 2024-04-25T01:35:49.476 INFO:tasks.workunit.client.0.smithi033.stdout:7/436: read - d4/d57/f5d zero size 2024-04-25T01:35:49.478 INFO:tasks.workunit.client.0.smithi033.stdout:7/437: read d4/d2e/f4c [337022,18494] 0 2024-04-25T01:35:49.478 INFO:tasks.workunit.client.0.smithi033.stdout:0/319: mkdir d7/d2d/d5e/d68/d6c 0 2024-04-25T01:35:49.480 INFO:tasks.workunit.client.0.smithi033.stdout:3/360: write d8/dd/d3d/d63/d7c/d3c/f7d [2711899,123804] 0 2024-04-25T01:35:49.481 INFO:tasks.workunit.client.0.smithi033.stdout:3/361: stat d8/dd/d1d/f7e 0 2024-04-25T01:35:49.484 INFO:tasks.workunit.client.0.smithi033.stdout:1/296: unlink d2/d11/d28/d33/f36 0 2024-04-25T01:35:49.487 INFO:tasks.workunit.client.0.smithi033.stdout:9/314: truncate d0/d2/dd/df/f3a 961133 0 2024-04-25T01:35:49.489 INFO:tasks.workunit.client.0.smithi033.stdout:0/320: dread d7/d2d/d4d/f5c [0,4194304] 0 2024-04-25T01:35:49.490 INFO:tasks.workunit.client.0.smithi033.stdout:0/321: chown d7/d2d/d43/d44/c4f 80723 1 2024-04-25T01:35:49.492 INFO:tasks.workunit.client.0.smithi033.stdout:0/322: truncate d7/d2d/d43/d44/f63 1192381 0 2024-04-25T01:35:49.492 INFO:tasks.workunit.client.0.smithi033.stdout:6/362: symlink d4/l84 0 2024-04-25T01:35:49.493 INFO:tasks.workunit.client.0.smithi033.stdout:3/362: symlink d8/dd/d3d/d4f/l82 0 2024-04-25T01:35:49.494 INFO:tasks.workunit.client.0.smithi033.stdout:3/363: dread - d8/dd/d11/f7f zero size 2024-04-25T01:35:49.497 INFO:tasks.workunit.client.0.smithi033.stdout:0/323: dread d7/d1b/f32 [0,4194304] 0 2024-04-25T01:35:49.499 INFO:tasks.workunit.client.0.smithi033.stdout:5/465: rename d1c/d21/d3e/l8b to d1c/d2a/l9d 0 2024-04-25T01:35:49.501 INFO:tasks.workunit.client.0.smithi033.stdout:2/481: unlink f4 0 2024-04-25T01:35:49.501 INFO:tasks.workunit.client.0.smithi033.stdout:4/466: link c0 d6/d14/d35/d44/d45/d46/d57/d70/c89 0 2024-04-25T01:35:49.502 INFO:tasks.workunit.client.0.smithi033.stdout:7/438: mknod d4/c8e 0 2024-04-25T01:35:49.504 INFO:tasks.workunit.client.0.smithi033.stdout:6/363: symlink d4/d1d/d3e/d70/l85 0 2024-04-25T01:35:49.508 INFO:tasks.workunit.client.0.smithi033.stdout:1/297: mknod d2/d11/c57 0 2024-04-25T01:35:49.509 INFO:tasks.workunit.client.0.smithi033.stdout:0/324: creat d7/d2d/d64/f6d x:0 0 0 2024-04-25T01:35:49.510 INFO:tasks.workunit.client.0.smithi033.stdout:0/325: write d7/da/df/f19 [1411859,79605] 0 2024-04-25T01:35:49.511 INFO:tasks.workunit.client.0.smithi033.stdout:9/315: rename d0/d11/c5d to d0/d2/dd/d20/d4e/c71 0 2024-04-25T01:35:49.513 INFO:tasks.workunit.client.0.smithi033.stdout:8/457: truncate d6/f34 794877 0 2024-04-25T01:35:49.513 INFO:tasks.workunit.client.0.smithi033.stdout:0/326: read d7/d2d/d43/d44/f63 [498910,46565] 0 2024-04-25T01:35:49.514 INFO:tasks.workunit.client.0.smithi033.stdout:8/458: chown d6/d45/d51/d5d 10498681 1 2024-04-25T01:35:49.514 INFO:tasks.workunit.client.0.smithi033.stdout:4/467: readlink d6/l4f 0 2024-04-25T01:35:49.519 INFO:tasks.workunit.client.0.smithi033.stdout:6/364: creat d4/d1d/d3e/d70/f86 x:0 0 0 2024-04-25T01:35:49.522 INFO:tasks.workunit.client.0.smithi033.stdout:0/327: dwrite d7/da/df/d14/f47 [0,4194304] 0 2024-04-25T01:35:49.525 INFO:tasks.workunit.client.0.smithi033.stdout:0/328: write d7/da/df/d14/f69 [803866,101535] 0 2024-04-25T01:35:49.528 INFO:tasks.workunit.client.0.smithi033.stdout:9/316: mknod d0/d2/dd/d20/d4e/c72 0 2024-04-25T01:35:49.539 INFO:tasks.workunit.client.0.smithi033.stdout:8/459: dread d6/d45/d52/f3b [0,4194304] 0 2024-04-25T01:35:49.542 INFO:tasks.workunit.client.0.smithi033.stdout:5/466: truncate d1c/d21/d4a/f64 3080209 0 2024-04-25T01:35:49.543 INFO:tasks.workunit.client.0.smithi033.stdout:5/467: fdatasync d1c/d2a/f47 0 2024-04-25T01:35:49.544 INFO:tasks.workunit.client.0.smithi033.stdout:6/365: chown d4/d24/d63/c6e 60 1 2024-04-25T01:35:49.544 INFO:tasks.workunit.client.0.smithi033.stdout:2/482: write da/d2c/f48 [808584,22895] 0 2024-04-25T01:35:49.545 INFO:tasks.workunit.client.0.smithi033.stdout:5/468: dread d1c/f23 [0,4194304] 0 2024-04-25T01:35:49.547 INFO:tasks.workunit.client.0.smithi033.stdout:2/483: read da/d27/d31/f3d [845130,92655] 0 2024-04-25T01:35:49.548 INFO:tasks.workunit.client.0.smithi033.stdout:7/439: write d4/d2e/d6b/d1f/f46 [4317823,27830] 0 2024-04-25T01:35:49.548 INFO:tasks.workunit.client.0.smithi033.stdout:0/329: write d7/da/df/f17 [306770,54931] 0 2024-04-25T01:35:49.549 INFO:tasks.workunit.client.0.smithi033.stdout:9/317: mkdir d0/d2/d43/d73 0 2024-04-25T01:35:49.551 INFO:tasks.workunit.client.0.smithi033.stdout:0/330: write d7/d2d/d43/d44/f49 [241138,27064] 0 2024-04-25T01:35:49.552 INFO:tasks.workunit.client.0.smithi033.stdout:9/318: readlink d0/d2/d43/d4b/l5e 0 2024-04-25T01:35:49.556 INFO:tasks.workunit.client.0.smithi033.stdout:8/460: dread d6/d45/d52/d2e/f30 [0,4194304] 0 2024-04-25T01:35:49.557 INFO:tasks.workunit.client.0.smithi033.stdout:8/461: chown d6/db/f61 0 1 2024-04-25T01:35:49.559 INFO:tasks.workunit.client.0.smithi033.stdout:3/364: write d8/dd/d3d/d63/d7c/d4d/f62 [1770325,10199] 0 2024-04-25T01:35:49.563 INFO:tasks.workunit.client.0.smithi033.stdout:5/469: creat d1c/d2a/d3d/d59/f9e x:0 0 0 2024-04-25T01:35:49.565 INFO:tasks.workunit.client.0.smithi033.stdout:7/440: mkdir d4/d23/d3f/d45/d73/d8f 0 2024-04-25T01:35:49.567 INFO:tasks.workunit.client.0.smithi033.stdout:0/331: creat d7/d2d/d4c/f6e x:0 0 0 2024-04-25T01:35:49.569 INFO:tasks.workunit.client.0.smithi033.stdout:9/319: write d0/d2/d43/d4b/f63 [208949,56574] 0 2024-04-25T01:35:49.569 INFO:tasks.workunit.client.0.smithi033.stdout:4/468: creat d6/d14/d35/f8a x:0 0 0 2024-04-25T01:35:49.573 INFO:tasks.workunit.client.0.smithi033.stdout:8/462: rename d6/d45/d52/d2e/l8a to d6/d45/d52/d2e/la8 0 2024-04-25T01:35:49.573 INFO:tasks.workunit.client.0.smithi033.stdout:7/441: dwrite d4/d23/d3f/f79 [0,4194304] 0 2024-04-25T01:35:49.575 INFO:tasks.workunit.client.0.smithi033.stdout:1/298: getdents d2/d5 0 2024-04-25T01:35:49.577 INFO:tasks.workunit.client.0.smithi033.stdout:5/470: readlink d1c/d2a/l42 0 2024-04-25T01:35:49.583 INFO:tasks.workunit.client.0.smithi033.stdout:0/332: symlink d7/d2d/d64/l6f 0 2024-04-25T01:35:49.586 INFO:tasks.workunit.client.0.smithi033.stdout:9/320: mkdir d0/d2/dd/df/d1b/d3d/d74 0 2024-04-25T01:35:49.587 INFO:tasks.workunit.client.0.smithi033.stdout:8/463: creat d6/d45/d51/d55/fa9 x:0 0 0 2024-04-25T01:35:49.587 INFO:tasks.workunit.client.0.smithi033.stdout:6/366: rename d4/d1d/d3e/d4a/c79 to d4/d1d/c87 0 2024-04-25T01:35:49.589 INFO:tasks.workunit.client.0.smithi033.stdout:2/484: creat da/d27/d31/f9c x:0 0 0 2024-04-25T01:35:49.591 INFO:tasks.workunit.client.0.smithi033.stdout:1/299: dwrite d2/d5/f18 [0,4194304] 0 2024-04-25T01:35:49.591 INFO:tasks.workunit.client.0.smithi033.stdout:5/471: creat d1c/d2a/d50/d52/f9f x:0 0 0 2024-04-25T01:35:49.592 INFO:tasks.workunit.client.0.smithi033.stdout:7/442: dwrite d4/d23/d3f/f77 [0,4194304] 0 2024-04-25T01:35:49.593 INFO:tasks.workunit.client.0.smithi033.stdout:6/367: chown d4/d1d/d3e/d41/d54/l75 375420355 1 2024-04-25T01:35:49.596 INFO:tasks.workunit.client.0.smithi033.stdout:8/464: read d6/d45/d51/f79 [2029143,28870] 0 2024-04-25T01:35:49.597 INFO:tasks.workunit.client.0.smithi033.stdout:7/443: truncate d4/d23/d3f/d45/f66 895133 0 2024-04-25T01:35:49.601 INFO:tasks.workunit.client.0.smithi033.stdout:3/365: creat d8/dd/f83 x:0 0 0 2024-04-25T01:35:49.601 INFO:tasks.workunit.client.0.smithi033.stdout:5/472: dread f14 [0,4194304] 0 2024-04-25T01:35:49.603 INFO:tasks.workunit.client.0.smithi033.stdout:3/366: write d8/dd/d11/f39 [1201745,46012] 0 2024-04-25T01:35:49.605 INFO:tasks.workunit.client.0.smithi033.stdout:0/333: mkdir d7/d2d/d5e/d61/d70 0 2024-04-25T01:35:49.606 INFO:tasks.workunit.client.0.smithi033.stdout:9/321: mknod d0/d2/dd/d45/c75 0 2024-04-25T01:35:49.607 INFO:tasks.workunit.client.0.smithi033.stdout:7/444: dread f0 [4194304,4194304] 0 2024-04-25T01:35:49.608 INFO:tasks.workunit.client.0.smithi033.stdout:2/485: mknod da/d5b/d5e/c9d 0 2024-04-25T01:35:49.613 INFO:tasks.workunit.client.0.smithi033.stdout:8/465: mkdir d6/d45/d51/d55/daa 0 2024-04-25T01:35:49.616 INFO:tasks.workunit.client.0.smithi033.stdout:5/473: dread - d1c/d21/d3e/d70/d7a/f7b zero size 2024-04-25T01:35:49.620 INFO:tasks.workunit.client.0.smithi033.stdout:5/474: write d1c/d21/d3e/d70/d7a/f7b [790807,26616] 0 2024-04-25T01:35:49.622 INFO:tasks.workunit.client.0.smithi033.stdout:8/466: dwrite d6/d45/f82 [0,4194304] 0 2024-04-25T01:35:49.622 INFO:tasks.workunit.client.0.smithi033.stdout:7/445: mkdir d4/d23/d3f/d45/d90 0 2024-04-25T01:35:49.623 INFO:tasks.workunit.client.0.smithi033.stdout:2/486: mknod da/d5b/d5e/d76/c9e 0 2024-04-25T01:35:49.624 INFO:tasks.workunit.client.0.smithi033.stdout:5/475: chown d1c/d21/d3e/d70/d7a/f7b 443950 1 2024-04-25T01:35:49.625 INFO:tasks.workunit.client.0.smithi033.stdout:4/469: getdents d6/d14/d35/d44/d45 0 2024-04-25T01:35:49.625 INFO:tasks.workunit.client.0.smithi033.stdout:2/487: chown da/l21 366172 1 2024-04-25T01:35:49.630 INFO:tasks.workunit.client.0.smithi033.stdout:6/368: creat d4/d1d/f88 x:0 0 0 2024-04-25T01:35:49.630 INFO:tasks.workunit.client.0.smithi033.stdout:0/334: dwrite d7/d2d/d4d/f5c [0,4194304] 0 2024-04-25T01:35:49.632 INFO:tasks.workunit.client.0.smithi033.stdout:1/300: rename d2/d2e/c4c to d2/d11/d28/d33/c58 0 2024-04-25T01:35:49.634 INFO:tasks.workunit.client.0.smithi033.stdout:8/467: creat d6/db/d80/d89/fab x:0 0 0 2024-04-25T01:35:49.636 INFO:tasks.workunit.client.0.smithi033.stdout:2/488: creat da/d5b/d5e/f9f x:0 0 0 2024-04-25T01:35:49.640 INFO:tasks.workunit.client.0.smithi033.stdout:6/369: mknod d4/d15/c89 0 2024-04-25T01:35:49.644 INFO:tasks.workunit.client.0.smithi033.stdout:2/489: dwrite da/d27/f5c [0,4194304] 0 2024-04-25T01:35:49.648 INFO:tasks.workunit.client.0.smithi033.stdout:0/335: symlink d7/d2d/d5e/d68/d6c/l71 0 2024-04-25T01:35:49.648 INFO:tasks.workunit.client.0.smithi033.stdout:0/336: write d7/d2d/f67 [479300,19076] 0 2024-04-25T01:35:49.650 INFO:tasks.workunit.client.0.smithi033.stdout:6/370: mknod d4/d24/d7c/c8a 0 2024-04-25T01:35:49.650 INFO:tasks.workunit.client.0.smithi033.stdout:0/337: write d7/d2d/d4c/f6e [311269,7982] 0 2024-04-25T01:35:49.652 INFO:tasks.workunit.client.0.smithi033.stdout:3/367: truncate d8/dd/d40/f43 812282 0 2024-04-25T01:35:49.653 INFO:tasks.workunit.client.0.smithi033.stdout:0/338: truncate d7/da/df/d50/f6b 599026 0 2024-04-25T01:35:49.654 INFO:tasks.workunit.client.0.smithi033.stdout:9/322: rename d0/fc to d0/d2/dd/d20/d4e/f76 0 2024-04-25T01:35:49.654 INFO:tasks.workunit.client.0.smithi033.stdout:5/476: symlink d1c/d21/d3e/d70/d89/la0 0 2024-04-25T01:35:49.657 INFO:tasks.workunit.client.0.smithi033.stdout:1/301: mkdir d2/d11/d28/d33/d59 0 2024-04-25T01:35:49.659 INFO:tasks.workunit.client.0.smithi033.stdout:6/371: dread d4/d15/f48 [0,4194304] 0 2024-04-25T01:35:49.663 INFO:tasks.workunit.client.0.smithi033.stdout:6/372: dread - d4/d1d/d3e/d70/f86 zero size 2024-04-25T01:35:49.665 INFO:tasks.workunit.client.0.smithi033.stdout:3/368: dwrite d8/dd/d1d/f2b [0,4194304] 0 2024-04-25T01:35:49.666 INFO:tasks.workunit.client.0.smithi033.stdout:5/477: creat d1c/d2a/d50/d5b/fa1 x:0 0 0 2024-04-25T01:35:49.669 INFO:tasks.workunit.client.0.smithi033.stdout:1/302: dread d2/d5/f18 [0,4194304] 0 2024-04-25T01:35:49.670 INFO:tasks.workunit.client.0.smithi033.stdout:3/369: dread - d8/dd/d11/f7f zero size 2024-04-25T01:35:49.673 INFO:tasks.workunit.client.0.smithi033.stdout:7/446: getdents d4/d2e/d50 0 2024-04-25T01:35:49.675 INFO:tasks.workunit.client.0.smithi033.stdout:1/303: write d2/d5/f18 [4874423,107322] 0 2024-04-25T01:35:49.675 INFO:tasks.workunit.client.0.smithi033.stdout:8/468: link d6/d45/f5b d6/db/fac 0 2024-04-25T01:35:49.678 INFO:tasks.workunit.client.0.smithi033.stdout:3/370: write d8/dd/d3d/d63/d7c/f48 [1547436,77002] 0 2024-04-25T01:35:49.678 INFO:tasks.workunit.client.0.smithi033.stdout:4/470: dwrite d6/d14/d35/d44/d45/d46/d57/f62 [0,4194304] 0 2024-04-25T01:35:49.688 INFO:tasks.workunit.client.0.smithi033.stdout:0/339: dwrite d7/d2d/f3d [0,4194304] 0 2024-04-25T01:35:49.688 INFO:tasks.workunit.client.0.smithi033.stdout:8/469: fsync d6/d45/d52/f3b 0 2024-04-25T01:35:49.692 INFO:tasks.workunit.client.0.smithi033.stdout:0/340: chown d7/d2d/f3d 1456524 1 2024-04-25T01:35:49.695 INFO:tasks.workunit.client.0.smithi033.stdout:5/478: rename d1c/d21/f81 to d1c/d2a/d50/d52/fa2 0 2024-04-25T01:35:49.697 INFO:tasks.workunit.client.0.smithi033.stdout:2/490: getdents da 0 2024-04-25T01:35:49.697 INFO:tasks.workunit.client.0.smithi033.stdout:0/341: dread d7/d1b/f60 [0,4194304] 0 2024-04-25T01:35:49.698 INFO:tasks.workunit.client.0.smithi033.stdout:7/447: fsync d4/f43 0 2024-04-25T01:35:49.698 INFO:tasks.workunit.client.0.smithi033.stdout:9/323: dwrite d0/d2/dd/df/d1b/d3d/f70 [0,4194304] 0 2024-04-25T01:35:49.698 INFO:tasks.workunit.client.0.smithi033.stdout:8/470: symlink d6/d45/d52/d2e/d95/lad 0 2024-04-25T01:35:49.699 INFO:tasks.workunit.client.0.smithi033.stdout:3/371: chown d8/c1a 408404171 1 2024-04-25T01:35:49.700 INFO:tasks.workunit.client.0.smithi033.stdout:2/491: chown da/d50/d6a/l24 8 1 2024-04-25T01:35:49.702 INFO:tasks.workunit.client.0.smithi033.stdout:2/492: dread - da/d50/f58 zero size 2024-04-25T01:35:49.704 INFO:tasks.workunit.client.0.smithi033.stdout:3/372: write d8/dd/d3d/d63/f71 [964052,25739] 0 2024-04-25T01:35:49.704 INFO:tasks.workunit.client.0.smithi033.stdout:2/493: chown da/d50/d6a/l24 337 1 2024-04-25T01:35:49.705 INFO:tasks.workunit.client.0.smithi033.stdout:4/471: rename d6/d14/d1d/d3b/f74 to d6/d14/d35/d3a/d4c/f8b 0 2024-04-25T01:35:49.706 INFO:tasks.workunit.client.0.smithi033.stdout:3/373: write d8/dd/d3d/d63/d7c/d3c/f7d [1955424,62906] 0 2024-04-25T01:35:49.707 INFO:tasks.workunit.client.0.smithi033.stdout:6/373: truncate d4/d1d/d3e/d70/f72 1920827 0 2024-04-25T01:35:49.710 INFO:tasks.workunit.client.0.smithi033.stdout:7/448: mknod d4/d57/c91 0 2024-04-25T01:35:49.711 INFO:tasks.workunit.client.0.smithi033.stdout:7/449: stat d4/d23/d3f 0 2024-04-25T01:35:49.712 INFO:tasks.workunit.client.0.smithi033.stdout:9/324: creat d0/d2/dd/d20/f77 x:0 0 0 2024-04-25T01:35:49.714 INFO:tasks.workunit.client.0.smithi033.stdout:0/342: fdatasync d7/d2d/d5e/f59 0 2024-04-25T01:35:49.715 INFO:tasks.workunit.client.0.smithi033.stdout:3/374: dwrite d8/f37 [0,4194304] 0 2024-04-25T01:35:49.717 INFO:tasks.workunit.client.0.smithi033.stdout:3/375: chown c0 105185766 1 2024-04-25T01:35:49.717 INFO:tasks.workunit.client.0.smithi033.stdout:2/494: creat da/d50/d6a/d43/fa0 x:0 0 0 2024-04-25T01:35:49.726 INFO:tasks.workunit.client.0.smithi033.stdout:0/343: truncate d7/d2d/d64/d51/f52 520357 0 2024-04-25T01:35:49.726 INFO:tasks.workunit.client.0.smithi033.stdout:9/325: creat d0/d2/dd/d20/d4e/f78 x:0 0 0 2024-04-25T01:35:49.726 INFO:tasks.workunit.client.0.smithi033.stdout:2/495: creat da/d27/d31/d40/fa1 x:0 0 0 2024-04-25T01:35:49.726 INFO:tasks.workunit.client.0.smithi033.stdout:7/450: dwrite d4/d2e/d65/f68 [0,4194304] 0 2024-04-25T01:35:49.730 INFO:tasks.workunit.client.0.smithi033.stdout:9/326: rmdir d0/d2/dd/d5f 39 2024-04-25T01:35:49.730 INFO:tasks.workunit.client.0.smithi033.stdout:7/451: chown d4/d23/d3f/d45/f7c 15514637 1 2024-04-25T01:35:49.733 INFO:tasks.workunit.client.0.smithi033.stdout:3/376: rename d8/dd/d3d/d63/d7c/d3c/d6e/c81 to d8/dd/c84 0 2024-04-25T01:35:49.734 INFO:tasks.workunit.client.0.smithi033.stdout:6/374: dread d4/f12 [0,4194304] 0 2024-04-25T01:35:49.734 INFO:tasks.workunit.client.0.smithi033.stdout:2/496: creat da/d27/d31/fa2 x:0 0 0 2024-04-25T01:35:49.735 INFO:tasks.workunit.client.0.smithi033.stdout:0/344: dwrite d7/d2d/d43/d44/f49 [0,4194304] 0 2024-04-25T01:35:49.735 INFO:tasks.workunit.client.0.smithi033.stdout:9/327: mknod d0/d2/dd/d20/d4e/c79 0 2024-04-25T01:35:49.738 INFO:tasks.workunit.client.0.smithi033.stdout:0/345: write d7/f4e [735971,120862] 0 2024-04-25T01:35:49.738 INFO:tasks.workunit.client.0.smithi033.stdout:6/375: chown d4/d1d/d3e/l78 247 1 2024-04-25T01:35:49.739 INFO:tasks.workunit.client.0.smithi033.stdout:9/328: symlink d0/d2/d43/l7a 0 2024-04-25T01:35:49.739 INFO:tasks.workunit.client.0.smithi033.stdout:6/376: stat d4/f2d 0 2024-04-25T01:35:49.741 INFO:tasks.workunit.client.0.smithi033.stdout:0/346: chown d7/d33/f3e 7056 1 2024-04-25T01:35:49.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:49 smithi033 ceph-mon[26353]: pgmap v10: 65 pgs: 65 active+clean; 778 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 11 MiB/s rd, 82 MiB/s wr, 357 op/s 2024-04-25T01:35:49.742 INFO:tasks.workunit.client.0.smithi033.stdout:3/377: mknod d8/dd/d3d/d63/d7c/c85 0 2024-04-25T01:35:49.742 INFO:tasks.workunit.client.0.smithi033.stdout:7/452: dwrite d4/d23/d3f/f77 [0,4194304] 0 2024-04-25T01:35:49.745 INFO:tasks.workunit.client.0.smithi033.stdout:7/453: chown d4/d23 71 1 2024-04-25T01:35:49.747 INFO:tasks.workunit.client.0.smithi033.stdout:3/378: mkdir d8/dd/d3d/d63/d7c/d3c/d86 0 2024-04-25T01:35:49.747 INFO:tasks.workunit.client.0.smithi033.stdout:0/347: chown d7/d2d/d5e/d68/d6c/l71 2 1 2024-04-25T01:35:49.748 INFO:tasks.workunit.client.0.smithi033.stdout:2/497: dwrite da/d5b/f79 [0,4194304] 0 2024-04-25T01:35:49.749 INFO:tasks.workunit.client.0.smithi033.stdout:2/498: readlink da/l6b 0 2024-04-25T01:35:49.749 INFO:tasks.workunit.client.0.smithi033.stdout:3/379: stat d8/dd/d3d/d63/d7c/l72 0 2024-04-25T01:35:49.750 INFO:tasks.workunit.client.0.smithi033.stdout:6/377: rename d4/d15/f20 to d4/d1d/d3e/d41/d5c/f8b 0 2024-04-25T01:35:49.753 INFO:tasks.workunit.client.0.smithi033.stdout:0/348: link d7/da/df/f28 d7/da/df/f72 0 2024-04-25T01:35:49.756 INFO:tasks.workunit.client.0.smithi033.stdout:2/499: rename da/d2c/f36 to da/d50/d6a/fa3 0 2024-04-25T01:35:49.757 INFO:tasks.workunit.client.0.smithi033.stdout:3/380: mknod d8/dd/d3d/d63/d7c/d4d/c87 0 2024-04-25T01:35:49.758 INFO:tasks.workunit.client.0.smithi033.stdout:3/381: write d8/dd/d40/f53 [493690,92537] 0 2024-04-25T01:35:49.759 INFO:tasks.workunit.client.0.smithi033.stdout:0/349: mknod d7/d2d/d64/c73 0 2024-04-25T01:35:49.759 INFO:tasks.workunit.client.0.smithi033.stdout:3/382: truncate d8/f51 2252422 0 2024-04-25T01:35:49.760 INFO:tasks.workunit.client.0.smithi033.stdout:0/350: read d7/d2d/d43/d44/f49 [1893249,122483] 0 2024-04-25T01:35:49.760 INFO:tasks.workunit.client.0.smithi033.stdout:2/500: rename da/d5b/d5e/d76/l96 to da/d5b/la4 0 2024-04-25T01:35:49.764 INFO:tasks.workunit.client.0.smithi033.stdout:2/501: creat da/d27/d31/d40/d8a/fa5 x:0 0 0 2024-04-25T01:35:49.765 INFO:tasks.workunit.client.0.smithi033.stdout:2/502: fdatasync da/d27/d31/d40/d8a/f53 0 2024-04-25T01:35:49.766 INFO:tasks.workunit.client.0.smithi033.stdout:0/351: symlink d7/d2d/l74 0 2024-04-25T01:35:49.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:49 smithi138 ceph-mon[32977]: pgmap v10: 65 pgs: 65 active+clean; 778 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 11 MiB/s rd, 82 MiB/s wr, 357 op/s 2024-04-25T01:35:49.767 INFO:tasks.workunit.client.0.smithi033.stdout:3/383: getdents d8/dd/d3d 0 2024-04-25T01:35:49.768 INFO:tasks.workunit.client.0.smithi033.stdout:3/384: chown d8/dd/d40/f65 41740358 1 2024-04-25T01:35:49.774 INFO:tasks.workunit.client.0.smithi033.stdout:3/385: dwrite d8/dd/f83 [0,4194304] 0 2024-04-25T01:35:49.815 INFO:tasks.workunit.client.0.smithi033.stdout:7/454: read d4/d23/d3f/f4d [346182,41586] 0 2024-04-25T01:35:49.829 INFO:tasks.workunit.client.0.smithi033.stdout:4/472: write d6/d14/d35/d3a/f49 [644841,64294] 0 2024-04-25T01:35:49.834 INFO:tasks.workunit.client.0.smithi033.stdout:5/479: dwrite ff [0,4194304] 0 2024-04-25T01:35:49.837 INFO:tasks.workunit.client.0.smithi033.stdout:4/473: link d6/d14/l4b d6/d14/d1d/d80/l8c 0 2024-04-25T01:35:49.839 INFO:tasks.workunit.client.0.smithi033.stdout:5/480: mknod d1c/d21/d3e/d70/d7a/ca3 0 2024-04-25T01:35:49.842 INFO:tasks.workunit.client.0.smithi033.stdout:9/329: rmdir d0/d2 39 2024-04-25T01:35:49.845 INFO:tasks.workunit.client.0.smithi033.stdout:4/474: dwrite d6/d14/d48/f55 [0,4194304] 0 2024-04-25T01:35:49.849 INFO:tasks.workunit.client.0.smithi033.stdout:4/475: mknod d6/d14/d35/d3a/c8d 0 2024-04-25T01:35:49.850 INFO:tasks.workunit.client.0.smithi033.stdout:2/503: dread da/fd [0,4194304] 0 2024-04-25T01:35:49.854 INFO:tasks.workunit.client.0.smithi033.stdout:9/330: rmdir d0/d2/dd/df/d1b/d3d/d74 0 2024-04-25T01:35:49.855 INFO:tasks.workunit.client.0.smithi033.stdout:9/331: read d0/d2/dd/f15 [4121054,87145] 0 2024-04-25T01:35:49.856 INFO:tasks.workunit.client.0.smithi033.stdout:4/476: creat d6/d14/d35/d3a/d4d/d5e/d7c/f8e x:0 0 0 2024-04-25T01:35:49.857 INFO:tasks.workunit.client.0.smithi033.stdout:6/378: truncate d4/d2a/f34 1714206 0 2024-04-25T01:35:49.858 INFO:tasks.workunit.client.0.smithi033.stdout:2/504: creat da/d27/d31/d81/fa6 x:0 0 0 2024-04-25T01:35:49.861 INFO:tasks.workunit.client.0.smithi033.stdout:9/332: symlink d0/d11/l7b 0 2024-04-25T01:35:49.862 INFO:tasks.workunit.client.0.smithi033.stdout:9/333: fsync d0/d2/dd/fe 0 2024-04-25T01:35:49.862 INFO:tasks.workunit.client.0.smithi033.stdout:0/352: truncate d7/d33/f36 905386 0 2024-04-25T01:35:49.864 INFO:tasks.workunit.client.0.smithi033.stdout:0/353: write d7/da/df/f17 [2210206,108707] 0 2024-04-25T01:35:49.865 INFO:tasks.workunit.client.0.smithi033.stdout:2/505: rename da/d5b/d72/c93 to da/d27/ca7 0 2024-04-25T01:35:49.868 INFO:tasks.workunit.client.0.smithi033.stdout:0/354: dread d7/d2d/f67 [0,4194304] 0 2024-04-25T01:35:49.868 INFO:tasks.workunit.client.0.smithi033.stdout:9/334: mkdir d0/d2/dd/df/d1b/d3d/d7c 0 2024-04-25T01:35:49.870 INFO:tasks.workunit.client.0.smithi033.stdout:2/506: mkdir da/d5b/d5e/da8 0 2024-04-25T01:35:49.871 INFO:tasks.workunit.client.0.smithi033.stdout:3/386: dread d8/dd/f34 [0,4194304] 0 2024-04-25T01:35:49.872 INFO:tasks.workunit.client.0.smithi033.stdout:0/355: symlink d7/da/df/d14/d45/l75 0 2024-04-25T01:35:49.874 INFO:tasks.workunit.client.0.smithi033.stdout:9/335: creat d0/d2/dd/d20/f7d x:0 0 0 2024-04-25T01:35:49.878 INFO:tasks.workunit.client.0.smithi033.stdout:0/356: symlink d7/d2d/d4c/l76 0 2024-04-25T01:35:49.881 INFO:tasks.workunit.client.0.smithi033.stdout:2/507: truncate da/f42 2225471 0 2024-04-25T01:35:49.881 INFO:tasks.workunit.client.0.smithi033.stdout:7/455: dwrite d4/d23/d3f/d45/f5e [0,4194304] 0 2024-04-25T01:35:49.883 INFO:tasks.workunit.client.0.smithi033.stdout:5/481: truncate d1c/d21/f8e 40318 0 2024-04-25T01:35:49.884 INFO:tasks.workunit.client.0.smithi033.stdout:5/482: read - d1c/d2a/f34 zero size 2024-04-25T01:35:49.884 INFO:tasks.workunit.client.0.smithi033.stdout:5/483: write d1c/f1e [3956989,36811] 0 2024-04-25T01:35:49.885 INFO:tasks.workunit.client.0.smithi033.stdout:9/336: rename d0/d2/dd/d45 to d0/d2/dd/df/d1b/d3d/d7e 0 2024-04-25T01:35:49.887 INFO:tasks.workunit.client.0.smithi033.stdout:9/337: write d0/d2/dd/d20/f77 [488518,60316] 0 2024-04-25T01:35:49.931 INFO:tasks.workunit.client.0.smithi033.stdout:2/508: truncate da/fd 340228 0 2024-04-25T01:35:49.931 INFO:tasks.workunit.client.0.smithi033.stdout:7/456: dwrite d4/d2e/d6b/d1f/f42 [0,4194304] 0 2024-04-25T01:35:49.931 INFO:tasks.workunit.client.0.smithi033.stdout:3/387: rmdir d8/dd/d3d/d63/d76 0 2024-04-25T01:35:49.931 INFO:tasks.workunit.client.0.smithi033.stdout:0/357: rename d7/da/df/d14/l1e to d7/d33/l77 0 2024-04-25T01:35:49.932 INFO:tasks.workunit.client.0.smithi033.stdout:2/509: dread - da/d5b/d72/f95 zero size 2024-04-25T01:35:49.932 INFO:tasks.workunit.client.0.smithi033.stdout:6/379: write d4/d1d/f52 [537519,123073] 0 2024-04-25T01:35:49.934 INFO:tasks.workunit.client.0.smithi033.stdout:6/380: chown d4/d15/f48 0 1 2024-04-25T01:35:49.939 INFO:tasks.workunit.client.0.smithi033.stdout:7/457: unlink d4/f5 0 2024-04-25T01:35:49.941 INFO:tasks.workunit.client.0.smithi033.stdout:3/388: stat d8/dd/d11/c19 0 2024-04-25T01:35:49.943 INFO:tasks.workunit.client.0.smithi033.stdout:3/389: chown d8/dd/d11/d1e 4929809 1 2024-04-25T01:35:49.944 INFO:tasks.workunit.client.0.smithi033.stdout:2/510: symlink da/d50/d6a/d43/la9 0 2024-04-25T01:35:49.946 INFO:tasks.workunit.client.0.smithi033.stdout:2/511: write da/d27/f7a [594450,95988] 0 2024-04-25T01:35:49.947 INFO:tasks.workunit.client.0.smithi033.stdout:2/512: dread - da/d50/f58 zero size 2024-04-25T01:35:49.949 INFO:tasks.workunit.client.0.smithi033.stdout:6/381: creat d4/d24/d29/f8c x:0 0 0 2024-04-25T01:35:49.955 INFO:tasks.workunit.client.0.smithi033.stdout:9/338: dread d0/d2/dd/d20/d4e/f76 [0,4194304] 0 2024-04-25T01:35:49.958 INFO:tasks.workunit.client.0.smithi033.stdout:5/484: write d1c/f2b [84346,80187] 0 2024-04-25T01:35:49.960 INFO:tasks.workunit.client.0.smithi033.stdout:6/382: creat d4/d1d/d3e/d41/d54/f8d x:0 0 0 2024-04-25T01:35:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:6/383: chown d4/d2a 1 1 2024-04-25T01:35:49.961 INFO:tasks.workunit.client.0.smithi033.stdout:7/458: fsync d4/d2e/f1c 0 2024-04-25T01:35:49.962 INFO:tasks.workunit.client.0.smithi033.stdout:7/459: read f0 [4416522,107999] 0 2024-04-25T01:35:49.969 INFO:tasks.workunit.client.0.smithi033.stdout:6/384: dwrite d4/d1d/d3e/d41/d5c/d7f/f44 [0,4194304] 0 2024-04-25T01:35:49.971 INFO:tasks.workunit.client.0.smithi033.stdout:7/460: dread d4/d23/d3f/f77 [0,4194304] 0 2024-04-25T01:35:49.974 INFO:tasks.workunit.client.0.smithi033.stdout:0/358: write d7/d33/f36 [762261,62871] 0 2024-04-25T01:35:49.977 INFO:tasks.workunit.client.0.smithi033.stdout:2/513: link da/d27/d31/c65 da/d8d/caa 0 2024-04-25T01:35:49.979 INFO:tasks.workunit.client.0.smithi033.stdout:5/485: symlink d1c/d21/d3e/d70/d9b/la4 0 2024-04-25T01:35:49.981 INFO:tasks.workunit.client.0.smithi033.stdout:6/385: write d4/d2a/f39 [1472742,34271] 0 2024-04-25T01:35:49.983 INFO:tasks.workunit.client.0.smithi033.stdout:0/359: fdatasync d7/da/df/f72 0 2024-04-25T01:35:49.983 INFO:tasks.workunit.client.0.smithi033.stdout:5/486: read d1c/d21/d4a/f4c [106374,96266] 0 2024-04-25T01:35:49.985 INFO:tasks.workunit.client.0.smithi033.stdout:3/390: dwrite d8/dd/d40/f43 [0,4194304] 0 2024-04-25T01:35:49.985 INFO:tasks.workunit.client.0.smithi033.stdout:6/386: write d4/d24/d29/f8c [804042,57807] 0 2024-04-25T01:35:49.988 INFO:tasks.workunit.client.0.smithi033.stdout:3/391: read - d8/dd/d11/d1e/f3b zero size 2024-04-25T01:35:49.988 INFO:tasks.workunit.client.0.smithi033.stdout:0/360: stat d7/da/d23/c3f 0 2024-04-25T01:35:49.989 INFO:tasks.workunit.client.0.smithi033.stdout:5/487: creat d1c/d21/d3e/d70/d89/fa5 x:0 0 0 2024-04-25T01:35:49.991 INFO:tasks.workunit.client.0.smithi033.stdout:5/488: chown d1c/d21/l4f 12659 1 2024-04-25T01:35:49.992 INFO:tasks.workunit.client.0.smithi033.stdout:3/392: fsync d8/dd/f34 0 2024-04-25T01:35:49.992 INFO:tasks.workunit.client.0.smithi033.stdout:3/393: readlink d8/dd/d1d/l7a 0 2024-04-25T01:35:49.993 INFO:tasks.workunit.client.0.smithi033.stdout:6/387: symlink d4/d1d/d3e/l8e 0 2024-04-25T01:35:49.995 INFO:tasks.workunit.client.0.smithi033.stdout:0/361: symlink d7/d2d/d43/d44/l78 0 2024-04-25T01:35:49.997 INFO:tasks.workunit.client.0.smithi033.stdout:6/388: fsync d4/d24/d29/f33 0 2024-04-25T01:35:49.999 INFO:tasks.workunit.client.0.smithi033.stdout:0/362: fsync d7/d1b/f60 0 2024-04-25T01:35:50.001 INFO:tasks.workunit.client.0.smithi033.stdout:6/389: creat d4/d24/f8f x:0 0 0 2024-04-25T01:35:50.003 INFO:tasks.workunit.client.0.smithi033.stdout:0/363: mknod d7/da/df/d50/c79 0 2024-04-25T01:35:50.005 INFO:tasks.workunit.client.0.smithi033.stdout:3/394: link d8/dd/d11/f7f d8/f88 0 2024-04-25T01:35:50.005 INFO:tasks.workunit.client.0.smithi033.stdout:3/395: readlink d8/dd/d1d/l7a 0 2024-04-25T01:35:50.005 INFO:tasks.workunit.client.0.smithi033.stdout:5/489: rename d1c/d21 to d1c/d2a/d50/d52/da6 0 2024-04-25T01:35:50.009 INFO:tasks.workunit.client.0.smithi033.stdout:3/396: creat d8/dd/d3d/d63/d7c/d4d/f89 x:0 0 0 2024-04-25T01:35:50.011 INFO:tasks.workunit.client.0.smithi033.stdout:5/490: write d1c/d2a/d50/f7d [2503876,122706] 0 2024-04-25T01:35:50.013 INFO:tasks.workunit.client.0.smithi033.stdout:0/364: link d7/da/df/c39 d7/d2d/d5e/d61/c7a 0 2024-04-25T01:35:50.020 INFO:tasks.workunit.client.0.smithi033.stdout:3/397: dwrite d8/dd/d40/f53 [0,4194304] 0 2024-04-25T01:35:50.021 INFO:tasks.workunit.client.0.smithi033.stdout:3/398: stat d8/dd/f15 0 2024-04-25T01:35:50.023 INFO:tasks.workunit.client.0.smithi033.stdout:5/491: dwrite d1c/d2a/d50/f91 [0,4194304] 0 2024-04-25T01:35:50.025 INFO:tasks.workunit.client.0.smithi033.stdout:5/492: write d1c/d2a/f34 [463471,22165] 0 2024-04-25T01:35:50.026 INFO:tasks.workunit.client.0.smithi033.stdout:3/399: fsync d8/fa 0 2024-04-25T01:35:50.027 INFO:tasks.workunit.client.0.smithi033.stdout:5/493: write d1c/d2a/d50/d52/da6/d4a/f8c [9214699,101161] 0 2024-04-25T01:35:50.029 INFO:tasks.workunit.client.0.smithi033.stdout:3/400: chown d8/dd/d3d/d63/d7c/d3c/d6e 46680 1 2024-04-25T01:35:50.029 INFO:tasks.workunit.client.0.smithi033.stdout:3/401: stat d8 0 2024-04-25T01:35:50.031 INFO:tasks.workunit.client.0.smithi033.stdout:0/365: dread d7/da/df/f72 [0,4194304] 0 2024-04-25T01:35:50.031 INFO:tasks.workunit.client.0.smithi033.stdout:5/494: rmdir d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e 39 2024-04-25T01:35:50.034 INFO:tasks.workunit.client.0.smithi033.stdout:5/495: write d1c/d2a/d50/d52/da6/d4a/f7f [354719,60520] 0 2024-04-25T01:35:50.034 INFO:tasks.workunit.client.0.smithi033.stdout:0/366: creat d7/d2d/d43/f7b x:0 0 0 2024-04-25T01:35:50.036 INFO:tasks.workunit.client.0.smithi033.stdout:3/402: rename d8/dd/d3d/d63/d7c/f48 to d8/dd/d3d/d63/d7c/d3c/d86/f8a 0 2024-04-25T01:35:50.043 INFO:tasks.workunit.client.0.smithi033.stdout:0/367: rename d7/d2d/c2f to d7/c7c 0 2024-04-25T01:35:50.049 INFO:tasks.workunit.client.0.smithi033.stdout:0/368: dwrite d7/f4e [0,4194304] 0 2024-04-25T01:35:50.052 INFO:tasks.workunit.client.0.smithi033.stdout:5/496: dwrite fc [0,4194304] 0 2024-04-25T01:35:50.052 INFO:tasks.workunit.client.0.smithi033.stdout:0/369: chown d7/l20 794 1 2024-04-25T01:35:50.056 INFO:tasks.workunit.client.0.smithi033.stdout:0/370: unlink d7/da/df/f28 0 2024-04-25T01:35:50.056 INFO:tasks.workunit.client.0.smithi033.stdout:5/497: dread d1c/d2a/d50/d52/da6/f6d [0,4194304] 0 2024-04-25T01:35:50.057 INFO:tasks.workunit.client.0.smithi033.stdout:5/498: fdatasync d1c/d53/f99 0 2024-04-25T01:35:50.058 INFO:tasks.workunit.client.0.smithi033.stdout:0/371: creat d7/da/df/d50/f7d x:0 0 0 2024-04-25T01:35:50.058 INFO:tasks.workunit.client.0.smithi033.stdout:0/372: chown d7/da/c31 46905 1 2024-04-25T01:35:50.060 INFO:tasks.workunit.client.0.smithi033.stdout:5/499: mkdir d1c/d2a/d50/d52/da6/d3e/d70/d89/da7 0 2024-04-25T01:35:50.062 INFO:tasks.workunit.client.0.smithi033.stdout:0/373: mkdir d7/d7e 0 2024-04-25T01:35:50.062 INFO:tasks.workunit.client.0.smithi033.stdout:0/374: stat d7/l20 0 2024-04-25T01:35:50.064 INFO:tasks.workunit.client.0.smithi033.stdout:5/500: rename d1c/d2a/d3d/f3f to d1c/d2a/d50/d5b/fa8 0 2024-04-25T01:35:50.066 INFO:tasks.workunit.client.0.smithi033.stdout:0/375: creat d7/d2d/d64/d51/f7f x:0 0 0 2024-04-25T01:35:50.067 INFO:tasks.workunit.client.0.smithi033.stdout:5/501: fdatasync d1c/d2a/d3d/f46 0 2024-04-25T01:35:50.071 INFO:tasks.workunit.client.0.smithi033.stdout:5/502: unlink d1c/d2a/d50/d52/da6/d3e/d70/l96 0 2024-04-25T01:35:50.073 INFO:tasks.workunit.client.0.smithi033.stdout:0/376: getdents d7/d2d/d4d 0 2024-04-25T01:35:50.076 INFO:tasks.workunit.client.0.smithi033.stdout:0/377: mkdir d7/da/d80 0 2024-04-25T01:35:50.076 INFO:tasks.workunit.client.0.smithi033.stdout:0/378: fdatasync d7/da/f58 0 2024-04-25T01:35:50.078 INFO:tasks.workunit.client.0.smithi033.stdout:0/379: write d7/da/f48 [1378981,48687] 0 2024-04-25T01:35:50.079 INFO:tasks.workunit.client.0.smithi033.stdout:0/380: truncate d7/da/f5d 85749 0 2024-04-25T01:35:50.084 INFO:tasks.workunit.client.0.smithi033.stdout:0/381: mknod d7/d2d/d4c/c81 0 2024-04-25T01:35:50.085 INFO:tasks.workunit.client.0.smithi033.stdout:0/382: unlink d7/l20 0 2024-04-25T01:35:50.086 INFO:tasks.workunit.client.0.smithi033.stdout:0/383: fdatasync d7/d2d/d5e/f59 0 2024-04-25T01:35:50.088 INFO:tasks.workunit.client.0.smithi033.stdout:0/384: creat d7/d2d/d4d/f82 x:0 0 0 2024-04-25T01:35:50.089 INFO:tasks.workunit.client.0.smithi033.stdout:0/385: creat d7/d2d/d5e/f83 x:0 0 0 2024-04-25T01:35:50.090 INFO:tasks.workunit.client.0.smithi033.stdout:0/386: creat d7/d2d/d5e/f84 x:0 0 0 2024-04-25T01:35:50.146 INFO:tasks.workunit.client.0.smithi033.stdout:3/403: dread d8/dd/f15 [0,4194304] 0 2024-04-25T01:35:50.148 INFO:tasks.workunit.client.0.smithi033.stdout:3/404: creat d8/dd/d11/f8b x:0 0 0 2024-04-25T01:35:50.153 INFO:tasks.workunit.client.0.smithi033.stdout:3/405: dwrite d8/dd/f83 [0,4194304] 0 2024-04-25T01:35:50.170 INFO:tasks.workunit.client.0.smithi033.stdout:3/406: rename d8/dd/d11/d1e/l60 to d8/dd/d3d/d63/d7c/d4d/l8c 0 2024-04-25T01:35:50.175 INFO:tasks.workunit.client.0.smithi033.stdout:3/407: rename d8/dd/d11/f55 to d8/dd/d3d/d63/d7c/d4d/f8d 0 2024-04-25T01:35:50.178 INFO:tasks.workunit.client.0.smithi033.stdout:3/408: creat d8/dd/d11/f8e x:0 0 0 2024-04-25T01:35:50.178 INFO:tasks.workunit.client.0.smithi033.stdout:3/409: chown l1 48787 1 2024-04-25T01:35:50.179 INFO:tasks.workunit.client.0.smithi033.stdout:3/410: write d8/dd/d11/f39 [1908557,69296] 0 2024-04-25T01:35:50.190 INFO:tasks.workunit.client.0.smithi033.stdout:3/411: dread d8/dd/d11/f24 [4194304,4194304] 0 2024-04-25T01:35:50.201 INFO:tasks.workunit.client.0.smithi033.stdout:3/412: dwrite d8/dd/d40/f53 [4194304,4194304] 0 2024-04-25T01:35:50.321 INFO:tasks.workunit.client.1.smithi138.stderr:++ readlink -f fsstress 2024-04-25T01:35:50.322 INFO:tasks.workunit.client.1.smithi138.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-25T01:35:50.322 INFO:tasks.workunit.client.1.smithi138.stderr:+ popd 2024-04-25T01:35:50.323 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-25T01:35:50.323 INFO:tasks.workunit.client.1.smithi138.stderr:+ popd 2024-04-25T01:35:50.324 INFO:tasks.workunit.client.1.smithi138.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-25T01:35:50.324 INFO:tasks.workunit.client.1.smithi138.stderr:++ mktemp -d -p . 2024-04-25T01:35:50.327 INFO:tasks.workunit.client.1.smithi138.stderr:+ T=./tmp.ebWhjI7yKi 2024-04-25T01:35:50.327 INFO:tasks.workunit.client.1.smithi138.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.ebWhjI7yKi -l 1 -n 1000 -p 10 -v 2024-04-25T01:35:50.329 INFO:tasks.workunit.client.1.smithi138.stdout:seed = 1714215139 2024-04-25T01:35:50.333 INFO:tasks.workunit.client.1.smithi138.stdout:0/0: chown . 21091 1 2024-04-25T01:35:50.333 INFO:tasks.workunit.client.1.smithi138.stdout:0/1: fsync - no filename 2024-04-25T01:35:50.333 INFO:tasks.workunit.client.1.smithi138.stdout:0/2: readlink - no filename 2024-04-25T01:35:50.334 INFO:tasks.workunit.client.1.smithi138.stdout:1/0: dwrite - no filename 2024-04-25T01:35:50.334 INFO:tasks.workunit.client.1.smithi138.stdout:0/3: creat f0 x:0 0 0 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/0: read - no filename 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/1: dwrite - no filename 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/2: chown . 1017460852 1 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/3: write - no filename 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/4: rename - no filename 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:0/4: creat f1 x:0 0 0 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:1/1: mknod c0 0 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:3/0: truncate - no filename 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:3/1: write - no filename 2024-04-25T01:35:50.339 INFO:tasks.workunit.client.1.smithi138.stdout:3/2: dread - no filename 2024-04-25T01:35:50.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/5: mkdir d0 0 2024-04-25T01:35:50.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/6: chown d0 2075 1 2024-04-25T01:35:50.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/7: chown d0 861 1 2024-04-25T01:35:50.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/8: dwrite - no filename 2024-04-25T01:35:50.341 INFO:tasks.workunit.client.1.smithi138.stdout:2/9: chown d0 1 1 2024-04-25T01:35:50.341 INFO:tasks.workunit.client.1.smithi138.stdout:2/10: truncate - no filename 2024-04-25T01:35:50.341 INFO:tasks.workunit.client.1.smithi138.stdout:2/11: truncate - no filename 2024-04-25T01:35:50.341 INFO:tasks.workunit.client.1.smithi138.stdout:2/12: stat d0 0 2024-04-25T01:35:50.341 INFO:tasks.workunit.client.1.smithi138.stdout:4/0: dread - no filename 2024-04-25T01:35:50.341 INFO:tasks.workunit.client.1.smithi138.stdout:4/1: dwrite - no filename 2024-04-25T01:35:50.341 INFO:tasks.workunit.client.1.smithi138.stdout:3/3: mknod c0 0 2024-04-25T01:35:50.341 INFO:tasks.workunit.client.1.smithi138.stdout:3/4: dread - no filename 2024-04-25T01:35:50.342 INFO:tasks.workunit.client.1.smithi138.stdout:5/0: chown . 9 1 2024-04-25T01:35:50.343 INFO:tasks.workunit.client.1.smithi138.stdout:5/1: dwrite - no filename 2024-04-25T01:35:50.343 INFO:tasks.workunit.client.1.smithi138.stdout:5/2: link - no file 2024-04-25T01:35:50.345 INFO:tasks.workunit.client.1.smithi138.stdout:3/5: creat f1 x:0 0 0 2024-04-25T01:35:50.347 INFO:tasks.workunit.client.1.smithi138.stdout:6/0: write - no filename 2024-04-25T01:35:50.347 INFO:tasks.workunit.client.1.smithi138.stdout:6/1: chown . 47948117 1 2024-04-25T01:35:50.347 INFO:tasks.workunit.client.1.smithi138.stdout:6/2: dread - no filename 2024-04-25T01:35:50.347 INFO:tasks.workunit.client.1.smithi138.stdout:6/3: unlink - no file 2024-04-25T01:35:50.347 INFO:tasks.workunit.client.1.smithi138.stdout:6/4: chown . 14 1 2024-04-25T01:35:50.348 INFO:tasks.workunit.client.1.smithi138.stdout:2/13: creat d0/f1 x:0 0 0 2024-04-25T01:35:50.348 INFO:tasks.workunit.client.1.smithi138.stdout:4/2: mkdir d0 0 2024-04-25T01:35:50.348 INFO:tasks.workunit.client.1.smithi138.stdout:4/3: truncate - no filename 2024-04-25T01:35:50.348 INFO:tasks.workunit.client.1.smithi138.stdout:4/4: truncate - no filename 2024-04-25T01:35:50.349 INFO:tasks.workunit.client.1.smithi138.stdout:2/14: write d0/f1 [738678,126991] 0 2024-04-25T01:35:50.349 INFO:tasks.workunit.client.1.smithi138.stdout:4/5: chown d0 295 1 2024-04-25T01:35:50.349 INFO:tasks.workunit.client.1.smithi138.stdout:3/6: symlink l2 0 2024-04-25T01:35:50.350 INFO:tasks.workunit.client.1.smithi138.stdout:3/7: write f1 [349776,68386] 0 2024-04-25T01:35:50.351 INFO:tasks.workunit.client.1.smithi138.stdout:7/0: fdatasync - no filename 2024-04-25T01:35:50.351 INFO:tasks.workunit.client.1.smithi138.stdout:7/1: stat - no entries 2024-04-25T01:35:50.351 INFO:tasks.workunit.client.1.smithi138.stdout:7/2: dread - no filename 2024-04-25T01:35:50.352 INFO:tasks.workunit.client.1.smithi138.stdout:2/15: creat d0/f2 x:0 0 0 2024-04-25T01:35:50.353 INFO:tasks.workunit.client.1.smithi138.stdout:2/16: truncate d0/f1 1530910 0 2024-04-25T01:35:50.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/5: getdents . 0 2024-04-25T01:35:50.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/6: truncate - no filename 2024-04-25T01:35:50.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/7: unlink - no file 2024-04-25T01:35:50.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/8: rmdir - no directory 2024-04-25T01:35:50.355 INFO:tasks.workunit.client.1.smithi138.stdout:6/9: rename - no filename 2024-04-25T01:35:50.355 INFO:tasks.workunit.client.1.smithi138.stdout:8/0: chown . 1933 1 2024-04-25T01:35:50.356 INFO:tasks.workunit.client.1.smithi138.stdout:3/8: creat f3 x:0 0 0 2024-04-25T01:35:50.357 INFO:tasks.workunit.client.1.smithi138.stdout:4/6: symlink d0/l1 0 2024-04-25T01:35:50.357 INFO:tasks.workunit.client.1.smithi138.stdout:3/9: chown l2 260236659 1 2024-04-25T01:35:50.360 INFO:tasks.workunit.client.1.smithi138.stdout:3/10: write f1 [1310423,3882] 0 2024-04-25T01:35:50.361 INFO:tasks.workunit.client.1.smithi138.stdout:3/11: rmdir - no directory 2024-04-25T01:35:50.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/0: rename - no filename 2024-04-25T01:35:50.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/1: truncate - no filename 2024-04-25T01:35:50.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/2: dwrite - no filename 2024-04-25T01:35:50.361 INFO:tasks.workunit.client.1.smithi138.stdout:7/3: symlink l0 0 2024-04-25T01:35:50.363 INFO:tasks.workunit.client.1.smithi138.stdout:4/7: unlink d0/l1 0 2024-04-25T01:35:50.363 INFO:tasks.workunit.client.1.smithi138.stdout:4/8: write - no filename 2024-04-25T01:35:50.364 INFO:tasks.workunit.client.1.smithi138.stdout:6/10: getdents . 0 2024-04-25T01:35:50.364 INFO:tasks.workunit.client.1.smithi138.stdout:2/17: dwrite d0/f2 [0,4194304] 0 2024-04-25T01:35:50.364 INFO:tasks.workunit.client.1.smithi138.stdout:4/9: chown d0 41540 1 2024-04-25T01:35:50.365 INFO:tasks.workunit.client.1.smithi138.stdout:4/10: write - no filename 2024-04-25T01:35:50.367 INFO:tasks.workunit.client.1.smithi138.stdout:8/1: creat f0 x:0 0 0 2024-04-25T01:35:50.369 INFO:tasks.workunit.client.1.smithi138.stdout:3/12: symlink l4 0 2024-04-25T01:35:50.371 INFO:tasks.workunit.client.1.smithi138.stdout:7/4: symlink l1 0 2024-04-25T01:35:50.371 INFO:tasks.workunit.client.1.smithi138.stdout:7/5: dwrite - no filename 2024-04-25T01:35:50.373 INFO:tasks.workunit.client.1.smithi138.stdout:4/11: mknod d0/c2 0 2024-04-25T01:35:50.375 INFO:tasks.workunit.client.1.smithi138.stdout:2/18: unlink d0/f1 0 2024-04-25T01:35:50.375 INFO:tasks.workunit.client.1.smithi138.stdout:2/19: chown d0/f2 1657 1 2024-04-25T01:35:50.376 INFO:tasks.workunit.client.1.smithi138.stdout:9/3: mknod c0 0 2024-04-25T01:35:50.376 INFO:tasks.workunit.client.1.smithi138.stdout:9/4: fsync - no filename 2024-04-25T01:35:50.377 INFO:tasks.workunit.client.1.smithi138.stdout:9/5: stat c0 0 2024-04-25T01:35:50.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/6: mknod c2 0 2024-04-25T01:35:50.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/7: dread - no filename 2024-04-25T01:35:50.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/8: truncate - no filename 2024-04-25T01:35:50.379 INFO:tasks.workunit.client.1.smithi138.stdout:6/11: symlink l0 0 2024-04-25T01:35:50.379 INFO:tasks.workunit.client.1.smithi138.stdout:6/12: read - no filename 2024-04-25T01:35:50.379 INFO:tasks.workunit.client.1.smithi138.stdout:6/13: write - no filename 2024-04-25T01:35:50.379 INFO:tasks.workunit.client.1.smithi138.stdout:6/14: dread - no filename 2024-04-25T01:35:50.381 INFO:tasks.workunit.client.1.smithi138.stdout:8/2: link f0 f1 0 2024-04-25T01:35:50.383 INFO:tasks.workunit.client.1.smithi138.stdout:2/20: unlink d0/f2 0 2024-04-25T01:35:50.383 INFO:tasks.workunit.client.1.smithi138.stdout:2/21: unlink - no file 2024-04-25T01:35:50.383 INFO:tasks.workunit.client.1.smithi138.stdout:2/22: truncate - no filename 2024-04-25T01:35:50.384 INFO:tasks.workunit.client.1.smithi138.stdout:9/6: creat f1 x:0 0 0 2024-04-25T01:35:50.386 INFO:tasks.workunit.client.1.smithi138.stdout:6/15: creat f1 x:0 0 0 2024-04-25T01:35:50.386 INFO:tasks.workunit.client.1.smithi138.stdout:8/3: unlink f1 0 2024-04-25T01:35:50.386 INFO:tasks.workunit.client.1.smithi138.stdout:8/4: rmdir - no directory 2024-04-25T01:35:50.387 INFO:tasks.workunit.client.1.smithi138.stdout:2/23: mknod d0/c3 0 2024-04-25T01:35:50.388 INFO:tasks.workunit.client.1.smithi138.stdout:2/24: chown d0 25110984 1 2024-04-25T01:35:50.390 INFO:tasks.workunit.client.1.smithi138.stdout:9/7: creat f2 x:0 0 0 2024-04-25T01:35:50.390 INFO:tasks.workunit.client.1.smithi138.stdout:9/8: dread - f2 zero size 2024-04-25T01:35:50.391 INFO:tasks.workunit.client.1.smithi138.stdout:6/16: creat f2 x:0 0 0 2024-04-25T01:35:50.391 INFO:tasks.workunit.client.1.smithi138.stdout:6/17: chown l0 8 1 2024-04-25T01:35:50.393 INFO:tasks.workunit.client.1.smithi138.stdout:2/25: creat d0/f4 x:0 0 0 2024-04-25T01:35:50.394 INFO:tasks.workunit.client.1.smithi138.stdout:9/9: mknod c3 0 2024-04-25T01:35:50.394 INFO:tasks.workunit.client.1.smithi138.stdout:9/10: chown c3 344476158 1 2024-04-25T01:35:50.397 INFO:tasks.workunit.client.1.smithi138.stdout:6/18: mknod c3 0 2024-04-25T01:35:50.401 INFO:tasks.workunit.client.1.smithi138.stdout:6/19: write f1 [286944,27597] 0 2024-04-25T01:35:50.408 INFO:tasks.workunit.client.1.smithi138.stdout:2/26: dwrite d0/f4 [0,4194304] 0 2024-04-25T01:35:50.522 INFO:tasks.workunit.client.0.smithi033.stdout:1/304: sync 2024-04-25T01:35:50.525 INFO:tasks.workunit.client.0.smithi033.stdout:1/305: unlink d2/d5/d1b/l27 0 2024-04-25T01:35:50.527 INFO:tasks.workunit.client.0.smithi033.stdout:1/306: creat d2/d5/d1b/f5a x:0 0 0 2024-04-25T01:35:50.527 INFO:tasks.workunit.client.0.smithi033.stdout:1/307: chown d2/d5/f18 851 1 2024-04-25T01:35:50.533 INFO:tasks.workunit.client.0.smithi033.stdout:1/308: dwrite d2/f26 [4194304,4194304] 0 2024-04-25T01:35:50.536 INFO:tasks.workunit.client.0.smithi033.stdout:1/309: read d2/d5/d1b/f40 [148531,71689] 0 2024-04-25T01:35:50.542 INFO:tasks.workunit.client.0.smithi033.stdout:1/310: dread d2/d5/f18 [0,4194304] 0 2024-04-25T01:35:50.546 INFO:tasks.workunit.client.0.smithi033.stdout:1/311: symlink d2/d11/d28/l5b 0 2024-04-25T01:35:50.547 INFO:tasks.workunit.client.0.smithi033.stdout:1/312: mknod d2/d2e/d49/c5c 0 2024-04-25T01:35:50.548 INFO:tasks.workunit.client.0.smithi033.stdout:1/313: symlink d2/d11/d28/d33/d59/l5d 0 2024-04-25T01:35:50.554 INFO:tasks.workunit.client.0.smithi033.stdout:1/314: dwrite d2/d11/f1a [4194304,4194304] 0 2024-04-25T01:35:50.589 INFO:tasks.workunit.client.0.smithi033.stdout:0/387: sync 2024-04-25T01:35:50.589 INFO:tasks.workunit.client.0.smithi033.stdout:3/413: sync 2024-04-25T01:35:50.589 INFO:tasks.workunit.client.0.smithi033.stdout:7/461: sync 2024-04-25T01:35:50.589 INFO:tasks.workunit.client.0.smithi033.stdout:4/477: sync 2024-04-25T01:35:50.589 INFO:tasks.workunit.client.0.smithi033.stdout:8/471: sync 2024-04-25T01:35:50.590 INFO:tasks.workunit.client.0.smithi033.stdout:5/503: sync 2024-04-25T01:35:50.590 INFO:tasks.workunit.client.0.smithi033.stdout:6/390: sync 2024-04-25T01:35:50.590 INFO:tasks.workunit.client.0.smithi033.stdout:9/339: sync 2024-04-25T01:35:50.590 INFO:tasks.workunit.client.0.smithi033.stdout:2/514: sync 2024-04-25T01:35:50.590 INFO:tasks.workunit.client.0.smithi033.stdout:7/462: read - d4/d57/f5d zero size 2024-04-25T01:35:50.591 INFO:tasks.workunit.client.0.smithi033.stdout:6/391: chown d4/d15/c16 0 1 2024-04-25T01:35:50.592 INFO:tasks.workunit.client.0.smithi033.stdout:3/414: read d8/dd/f34 [1177902,17665] 0 2024-04-25T01:35:50.592 INFO:tasks.workunit.client.0.smithi033.stdout:0/388: write d7/d33/f36 [265205,51392] 0 2024-04-25T01:35:50.594 INFO:tasks.workunit.client.0.smithi033.stdout:4/478: creat d6/d14/d1d/d5f/f8f x:0 0 0 2024-04-25T01:35:50.596 INFO:tasks.workunit.client.0.smithi033.stdout:0/389: stat d7/d2d/d5e/d61 0 2024-04-25T01:35:50.596 INFO:tasks.workunit.client.0.smithi033.stdout:2/515: unlink da/d27/d31/fa2 0 2024-04-25T01:35:50.597 INFO:tasks.workunit.client.0.smithi033.stdout:8/472: mkdir d6/d45/d52/dae 0 2024-04-25T01:35:50.598 INFO:tasks.workunit.client.0.smithi033.stdout:2/516: read da/d27/d31/d40/f82 [3883723,67149] 0 2024-04-25T01:35:50.599 INFO:tasks.workunit.client.0.smithi033.stdout:7/463: creat d4/d23/d3f/d45/d6c/f92 x:0 0 0 2024-04-25T01:35:50.601 INFO:tasks.workunit.client.0.smithi033.stdout:3/415: dwrite d8/dd/f83 [0,4194304] 0 2024-04-25T01:35:50.601 INFO:tasks.workunit.client.0.smithi033.stdout:9/340: mknod d0/d2/dd/df/c7f 0 2024-04-25T01:35:50.603 INFO:tasks.workunit.client.0.smithi033.stdout:6/392: mknod d4/d1d/d3e/d41/d5c/c90 0 2024-04-25T01:35:50.605 INFO:tasks.workunit.client.0.smithi033.stdout:8/473: creat d6/db/d80/d89/faf x:0 0 0 2024-04-25T01:35:50.605 INFO:tasks.workunit.client.1.smithi138.stdout:9/11: dwrite f1 [0,4194304] 0 2024-04-25T01:35:50.606 INFO:tasks.workunit.client.1.smithi138.stdout:8/5: dwrite f0 [0,4194304] 0 2024-04-25T01:35:50.606 INFO:tasks.workunit.client.0.smithi033.stdout:2/517: creat da/d27/d31/d40/d8a/d80/fab x:0 0 0 2024-04-25T01:35:50.609 INFO:tasks.workunit.client.1.smithi138.stdout:9/12: write f1 [2198624,718] 0 2024-04-25T01:35:50.609 INFO:tasks.workunit.client.0.smithi033.stdout:4/479: mknod d6/d12/c90 0 2024-04-25T01:35:50.611 INFO:tasks.workunit.client.1.smithi138.stdout:9/13: dread - f2 zero size 2024-04-25T01:35:50.612 INFO:tasks.workunit.client.0.smithi033.stdout:5/504: creat d1c/d2a/d50/d52/da6/d3e/d70/d7a/fa9 x:0 0 0 2024-04-25T01:35:50.612 INFO:tasks.workunit.client.1.smithi138.stdout:6/20: dwrite f1 [0,4194304] 0 2024-04-25T01:35:50.612 INFO:tasks.workunit.client.0.smithi033.stdout:6/393: symlink d4/d2a/l91 0 2024-04-25T01:35:50.613 INFO:tasks.workunit.client.0.smithi033.stdout:9/341: mknod d0/d2/dd/df/d1b/d3d/d42/c80 0 2024-04-25T01:35:50.614 INFO:tasks.workunit.client.0.smithi033.stdout:7/464: dwrite d4/d2e/f48 [0,4194304] 0 2024-04-25T01:35:50.615 INFO:tasks.workunit.client.0.smithi033.stdout:8/474: rmdir d6/d49 39 2024-04-25T01:35:50.617 INFO:tasks.workunit.client.0.smithi033.stdout:4/480: creat d6/d12/d38/f91 x:0 0 0 2024-04-25T01:35:50.620 INFO:tasks.workunit.client.0.smithi033.stdout:3/416: dwrite d8/dd/d11/d1e/f29 [0,4194304] 0 2024-04-25T01:35:50.620 INFO:tasks.workunit.client.0.smithi033.stdout:4/481: chown d6/d14/d1d/f20 312333851 1 2024-04-25T01:35:50.621 INFO:tasks.workunit.client.0.smithi033.stdout:3/417: fdatasync d8/f51 0 2024-04-25T01:35:50.621 INFO:tasks.workunit.client.0.smithi033.stdout:4/482: chown d6/d12/f1a 60770348 1 2024-04-25T01:35:50.621 INFO:tasks.workunit.client.0.smithi033.stdout:2/518: mknod da/d5b/d5e/da8/cac 0 2024-04-25T01:35:50.622 INFO:tasks.workunit.client.0.smithi033.stdout:6/394: dread - d4/d24/f5b zero size 2024-04-25T01:35:50.623 INFO:tasks.workunit.client.0.smithi033.stdout:2/519: read da/d27/f5c [1528400,122699] 0 2024-04-25T01:35:50.623 INFO:tasks.workunit.client.0.smithi033.stdout:5/505: symlink d1c/d2a/d50/d5b/laa 0 2024-04-25T01:35:50.624 INFO:tasks.workunit.client.0.smithi033.stdout:2/520: readlink da/l6b 0 2024-04-25T01:35:50.625 INFO:tasks.workunit.client.0.smithi033.stdout:7/465: mkdir d4/d2e/d6b/d1f/d78/d93 0 2024-04-25T01:35:50.626 INFO:tasks.workunit.client.0.smithi033.stdout:7/466: read - d4/d2e/d6b/d1f/f82 zero size 2024-04-25T01:35:50.628 INFO:tasks.workunit.client.0.smithi033.stdout:4/483: readlink d6/d14/d1d/d80/l8c 0 2024-04-25T01:35:50.631 INFO:tasks.workunit.client.0.smithi033.stdout:4/484: read d6/d14/d1d/d3b/f3e [2769540,12916] 0 2024-04-25T01:35:50.631 INFO:tasks.workunit.client.0.smithi033.stdout:8/475: mkdir d6/d45/d52/db0 0 2024-04-25T01:35:50.633 INFO:tasks.workunit.client.0.smithi033.stdout:2/521: creat da/d50/fad x:0 0 0 2024-04-25T01:35:50.635 INFO:tasks.workunit.client.0.smithi033.stdout:8/476: write d6/d45/d51/f94 [1153389,2202] 0 2024-04-25T01:35:50.636 INFO:tasks.workunit.client.0.smithi033.stdout:2/522: chown da/d5b/d5e/d7c/f68 95 1 2024-04-25T01:35:50.637 INFO:tasks.workunit.client.0.smithi033.stdout:6/395: mknod d4/c92 0 2024-04-25T01:35:50.638 INFO:tasks.workunit.client.0.smithi033.stdout:2/523: write da/d27/d31/d40/fa1 [547896,12444] 0 2024-04-25T01:35:50.639 INFO:tasks.workunit.client.0.smithi033.stdout:4/485: dread d6/d12/d38/f5d [0,4194304] 0 2024-04-25T01:35:50.639 INFO:tasks.workunit.client.0.smithi033.stdout:6/396: truncate d4/d1d/d3e/d70/f86 135784 0 2024-04-25T01:35:50.640 INFO:tasks.workunit.client.0.smithi033.stdout:5/506: mkdir d1c/d2a/d50/d52/da6/d3e/d70/d7a/dab 0 2024-04-25T01:35:50.641 INFO:tasks.workunit.client.0.smithi033.stdout:5/507: fdatasync d1c/d2a/d3d/f4d 0 2024-04-25T01:35:50.643 INFO:tasks.workunit.client.0.smithi033.stdout:3/418: rename d8/f88 to d8/dd/f8f 0 2024-04-25T01:35:50.644 INFO:tasks.workunit.client.0.smithi033.stdout:3/419: stat d8 0 2024-04-25T01:35:50.647 INFO:tasks.workunit.client.0.smithi033.stdout:2/524: truncate da/d27/d31/f3d 1129448 0 2024-04-25T01:35:50.647 INFO:tasks.workunit.client.0.smithi033.stdout:4/486: dwrite d6/d12/d38/f41 [0,4194304] 0 2024-04-25T01:35:50.649 INFO:tasks.workunit.client.0.smithi033.stdout:6/397: mkdir d4/d1d/d3e/d41/d93 0 2024-04-25T01:35:50.650 INFO:tasks.workunit.client.0.smithi033.stdout:7/467: creat d4/d10/f94 x:0 0 0 2024-04-25T01:35:50.653 INFO:tasks.workunit.client.0.smithi033.stdout:8/477: rename d6/db/fac to d6/d45/d51/d5d/d6e/fb1 0 2024-04-25T01:35:50.656 INFO:tasks.workunit.client.0.smithi033.stdout:6/398: creat d4/d15/f94 x:0 0 0 2024-04-25T01:35:50.658 INFO:tasks.workunit.client.0.smithi033.stdout:5/508: truncate d1c/d2a/d50/d52/da6/f26 3480181 0 2024-04-25T01:35:50.658 INFO:tasks.workunit.client.0.smithi033.stdout:7/468: mknod d4/d2e/d50/c95 0 2024-04-25T01:35:50.659 INFO:tasks.workunit.client.0.smithi033.stdout:4/487: symlink d6/d14/l92 0 2024-04-25T01:35:50.661 INFO:tasks.workunit.client.0.smithi033.stdout:8/478: mknod d6/d45/d51/d5d/d6e/cb2 0 2024-04-25T01:35:50.662 INFO:tasks.workunit.client.0.smithi033.stdout:6/399: creat d4/d1d/d3e/d70/f95 x:0 0 0 2024-04-25T01:35:50.667 INFO:tasks.workunit.client.0.smithi033.stdout:4/488: creat d6/d14/d1d/d3b/f93 x:0 0 0 2024-04-25T01:35:50.669 INFO:tasks.workunit.client.0.smithi033.stdout:4/489: write d6/d14/d35/d3a/d4d/d5e/f60 [920034,104811] 0 2024-04-25T01:35:50.671 INFO:tasks.workunit.client.0.smithi033.stdout:6/400: creat d4/d24/d63/f96 x:0 0 0 2024-04-25T01:35:50.671 INFO:tasks.workunit.client.0.smithi033.stdout:2/525: getdents da/d27/d31/d81 0 2024-04-25T01:35:50.672 INFO:tasks.workunit.client.0.smithi033.stdout:2/526: fdatasync da/d27/f69 0 2024-04-25T01:35:50.673 INFO:tasks.workunit.client.0.smithi033.stdout:7/469: symlink d4/d23/d3f/d45/d73/d8f/l96 0 2024-04-25T01:35:50.675 INFO:tasks.workunit.client.0.smithi033.stdout:6/401: chown d4/d1d/d3e/l53 91 1 2024-04-25T01:35:50.676 INFO:tasks.workunit.client.1.smithi138.stdout:9/14: mkdir d4 0 2024-04-25T01:35:50.677 INFO:tasks.workunit.client.1.smithi138.stdout:9/15: stat d4 0 2024-04-25T01:35:50.677 INFO:tasks.workunit.client.0.smithi033.stdout:4/490: mkdir d6/d14/d35/d3a/d4d/d5e/d7c/d94 0 2024-04-25T01:35:50.679 INFO:tasks.workunit.client.1.smithi138.stdout:6/21: mknod c4 0 2024-04-25T01:35:50.681 INFO:tasks.workunit.client.0.smithi033.stdout:3/420: dread d8/dd/f2c [8388608,4194304] 0 2024-04-25T01:35:50.682 INFO:tasks.workunit.client.0.smithi033.stdout:6/402: mkdir d4/d24/d36/d42/d97 0 2024-04-25T01:35:50.682 INFO:tasks.workunit.client.0.smithi033.stdout:7/470: dwrite d4/d2e/d6b/d1f/f42 [0,4194304] 0 2024-04-25T01:35:50.683 INFO:tasks.workunit.client.1.smithi138.stdout:6/22: creat f5 x:0 0 0 2024-04-25T01:35:50.683 INFO:tasks.workunit.client.1.smithi138.stdout:6/23: rmdir - no directory 2024-04-25T01:35:50.684 INFO:tasks.workunit.client.0.smithi033.stdout:7/471: stat d4/d2e/d50/f55 0 2024-04-25T01:35:50.684 INFO:tasks.workunit.client.0.smithi033.stdout:6/403: chown d4/d1d/f52 18952 1 2024-04-25T01:35:50.685 INFO:tasks.workunit.client.1.smithi138.stdout:9/16: creat d4/f5 x:0 0 0 2024-04-25T01:35:50.688 INFO:tasks.workunit.client.1.smithi138.stdout:9/17: mknod d4/c6 0 2024-04-25T01:35:50.688 INFO:tasks.workunit.client.1.smithi138.stdout:9/18: chown f2 17900 1 2024-04-25T01:35:50.689 INFO:tasks.workunit.client.0.smithi033.stdout:7/472: creat d4/d23/d3f/d45/d6c/f97 x:0 0 0 2024-04-25T01:35:50.691 INFO:tasks.workunit.client.0.smithi033.stdout:6/404: fsync d4/d1d/d3e/d41/d54/f57 0 2024-04-25T01:35:50.694 INFO:tasks.workunit.client.0.smithi033.stdout:3/421: dwrite d8/dd/d11/d1e/f75 [4194304,4194304] 0 2024-04-25T01:35:50.696 INFO:tasks.workunit.client.0.smithi033.stdout:6/405: mknod d4/d24/d7c/c98 0 2024-04-25T01:35:50.696 INFO:tasks.workunit.client.1.smithi138.stdout:9/19: dwrite f1 [0,4194304] 0 2024-04-25T01:35:50.700 INFO:tasks.workunit.client.0.smithi033.stdout:6/406: read d4/d1d/f52 [472160,84347] 0 2024-04-25T01:35:50.700 INFO:tasks.workunit.client.1.smithi138.stdout:9/20: mkdir d4/d7 0 2024-04-25T01:35:50.701 INFO:tasks.workunit.client.0.smithi033.stdout:3/422: creat d8/dd/d40/f90 x:0 0 0 2024-04-25T01:35:50.702 INFO:tasks.workunit.client.0.smithi033.stdout:6/407: creat d4/d24/f99 x:0 0 0 2024-04-25T01:35:50.705 INFO:tasks.workunit.client.0.smithi033.stdout:6/408: creat d4/d24/d63/f9a x:0 0 0 2024-04-25T01:35:50.706 INFO:tasks.workunit.client.0.smithi033.stdout:6/409: chown d4/d15/l51 2 1 2024-04-25T01:35:50.706 INFO:tasks.workunit.client.1.smithi138.stdout:9/21: rmdir d4/d7 0 2024-04-25T01:35:50.707 INFO:tasks.workunit.client.0.smithi033.stdout:6/410: chown d4/d1d/c32 96324 1 2024-04-25T01:35:50.708 INFO:tasks.workunit.client.1.smithi138.stdout:9/22: mkdir d4/d8 0 2024-04-25T01:35:50.709 INFO:tasks.workunit.client.0.smithi033.stdout:3/423: rename c0 to d8/dd/c91 0 2024-04-25T01:35:50.710 INFO:tasks.workunit.client.1.smithi138.stdout:9/23: mknod d4/c9 0 2024-04-25T01:35:50.710 INFO:tasks.workunit.client.0.smithi033.stdout:6/411: mknod d4/d2a/c9b 0 2024-04-25T01:35:50.711 INFO:tasks.workunit.client.1.smithi138.stdout:9/24: mknod d4/ca 0 2024-04-25T01:35:50.713 INFO:tasks.workunit.client.0.smithi033.stdout:3/424: rename d8/dd/d11/d1e to d8/dd/d3d/d63/d7c/d3c/d86/d92 0 2024-04-25T01:35:50.715 INFO:tasks.workunit.client.0.smithi033.stdout:3/425: readlink d8/dd/d3d/d63/d7c/d3c/d6e/l78 0 2024-04-25T01:35:50.715 INFO:tasks.workunit.client.0.smithi033.stdout:6/412: symlink d4/d24/l9c 0 2024-04-25T01:35:50.716 INFO:tasks.workunit.client.0.smithi033.stdout:1/315: truncate d2/f32 3384819 0 2024-04-25T01:35:50.736 INFO:tasks.workunit.client.0.smithi033.stdout:4/491: dread d6/d12/f32 [0,4194304] 0 2024-04-25T01:35:50.737 INFO:tasks.workunit.client.0.smithi033.stdout:4/492: fdatasync d6/d14/d48/f55 0 2024-04-25T01:35:50.738 INFO:tasks.workunit.client.0.smithi033.stdout:4/493: chown d6/d14/d1d/f28 1657097 1 2024-04-25T01:35:50.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:50 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:50.742 INFO:tasks.workunit.client.1.smithi138.stdout:0/5: fsync f0 0 2024-04-25T01:35:50.742 INFO:tasks.workunit.client.1.smithi138.stdout:0/6: read - f0 zero size 2024-04-25T01:35:50.743 INFO:tasks.workunit.client.1.smithi138.stdout:1/2: getdents . 0 2024-04-25T01:35:50.744 INFO:tasks.workunit.client.1.smithi138.stdout:1/3: stat c0 0 2024-04-25T01:35:50.744 INFO:tasks.workunit.client.1.smithi138.stdout:1/4: rmdir - no directory 2024-04-25T01:35:50.745 INFO:tasks.workunit.client.1.smithi138.stdout:0/7: mknod c2 0 2024-04-25T01:35:50.745 INFO:tasks.workunit.client.1.smithi138.stdout:0/8: dread - f1 zero size 2024-04-25T01:35:50.747 INFO:tasks.workunit.client.1.smithi138.stdout:0/9: mknod c3 0 2024-04-25T01:35:50.749 INFO:tasks.workunit.client.1.smithi138.stdout:0/10: mkdir d4 0 2024-04-25T01:35:50.750 INFO:tasks.workunit.client.1.smithi138.stdout:1/5: link c0 c1 0 2024-04-25T01:35:50.755 INFO:tasks.workunit.client.1.smithi138.stdout:1/6: creat f2 x:0 0 0 2024-04-25T01:35:50.755 INFO:tasks.workunit.client.1.smithi138.stdout:1/7: chown f2 120 1 2024-04-25T01:35:50.756 INFO:tasks.workunit.client.1.smithi138.stdout:1/8: fdatasync f2 0 2024-04-25T01:35:50.759 INFO:tasks.workunit.client.0.smithi033.stdout:2/527: rmdir da/d27 39 2024-04-25T01:35:50.760 INFO:tasks.workunit.client.1.smithi138.stdout:1/9: dwrite f2 [0,4194304] 0 2024-04-25T01:35:50.760 INFO:tasks.workunit.client.1.smithi138.stdout:3/13: getdents . 0 2024-04-25T01:35:50.762 INFO:tasks.workunit.client.1.smithi138.stdout:4/12: rmdir d0 39 2024-04-25T01:35:50.764 INFO:tasks.workunit.client.0.smithi033.stdout:4/494: dwrite d6/d14/d1d/f7a [0,4194304] 0 2024-04-25T01:35:50.766 INFO:tasks.workunit.client.1.smithi138.stdout:4/13: stat d0/c2 0 2024-04-25T01:35:50.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:50 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:35:50.767 INFO:tasks.workunit.client.1.smithi138.stdout:4/14: fdatasync - no filename 2024-04-25T01:35:50.771 INFO:tasks.workunit.client.1.smithi138.stdout:1/10: dread f2 [0,4194304] 0 2024-04-25T01:35:50.771 INFO:tasks.workunit.client.1.smithi138.stdout:3/14: dwrite f3 [0,4194304] 0 2024-04-25T01:35:50.772 INFO:tasks.workunit.client.1.smithi138.stdout:4/15: mknod d0/c3 0 2024-04-25T01:35:50.775 INFO:tasks.workunit.client.0.smithi033.stdout:4/495: rename d6/d14/d48/l6b to d6/d14/d35/d3a/d4d/d5e/d7c/l95 0 2024-04-25T01:35:50.775 INFO:tasks.workunit.client.1.smithi138.stdout:1/11: write f2 [606322,48628] 0 2024-04-25T01:35:50.778 INFO:tasks.workunit.client.1.smithi138.stdout:3/15: creat f5 x:0 0 0 2024-04-25T01:35:50.780 INFO:tasks.workunit.client.0.smithi033.stdout:4/496: creat d6/d14/d6d/f96 x:0 0 0 2024-04-25T01:35:50.784 INFO:tasks.workunit.client.1.smithi138.stdout:4/16: creat d0/f4 x:0 0 0 2024-04-25T01:35:50.786 INFO:tasks.workunit.client.1.smithi138.stdout:6/24: fsync f2 0 2024-04-25T01:35:50.787 INFO:tasks.workunit.client.0.smithi033.stdout:4/497: symlink d6/d14/d6d/l97 0 2024-04-25T01:35:50.787 INFO:tasks.workunit.client.1.smithi138.stdout:2/27: getdents d0 0 2024-04-25T01:35:50.788 INFO:tasks.workunit.client.1.smithi138.stdout:4/17: symlink d0/l5 0 2024-04-25T01:35:50.789 INFO:tasks.workunit.client.1.smithi138.stdout:1/12: dwrite f2 [0,4194304] 0 2024-04-25T01:35:50.790 INFO:tasks.workunit.client.0.smithi033.stdout:4/498: symlink d6/d14/d35/d3a/d4d/l98 0 2024-04-25T01:35:50.791 INFO:tasks.workunit.client.0.smithi033.stdout:4/499: chown d6/d14/d48 0 1 2024-04-25T01:35:50.792 INFO:tasks.workunit.client.1.smithi138.stdout:4/18: readlink d0/l5 0 2024-04-25T01:35:50.793 INFO:tasks.workunit.client.1.smithi138.stdout:8/6: write f0 [4775368,118316] 0 2024-04-25T01:35:50.793 INFO:tasks.workunit.client.0.smithi033.stdout:8/479: truncate d6/d45/d52/d2e/f3a 1904990 0 2024-04-25T01:35:50.795 INFO:tasks.workunit.client.0.smithi033.stdout:5/509: dwrite d1c/f5e [0,4194304] 0 2024-04-25T01:35:50.795 INFO:tasks.workunit.client.1.smithi138.stdout:4/19: fdatasync d0/f4 0 2024-04-25T01:35:50.796 INFO:tasks.workunit.client.1.smithi138.stdout:3/16: dwrite f3 [0,4194304] 0 2024-04-25T01:35:50.799 INFO:tasks.workunit.client.0.smithi033.stdout:4/500: creat d6/d14/d48/f99 x:0 0 0 2024-04-25T01:35:50.805 INFO:tasks.workunit.client.1.smithi138.stdout:9/25: rmdir d4 39 2024-04-25T01:35:50.807 INFO:tasks.workunit.client.0.smithi033.stdout:5/510: dwrite d1c/d53/f57 [0,4194304] 0 2024-04-25T01:35:50.807 INFO:tasks.workunit.client.1.smithi138.stdout:4/20: dread - d0/f4 zero size 2024-04-25T01:35:50.809 INFO:tasks.workunit.client.1.smithi138.stdout:3/17: rename f3 to f6 0 2024-04-25T01:35:50.812 INFO:tasks.workunit.client.1.smithi138.stdout:6/25: dwrite f2 [0,4194304] 0 2024-04-25T01:35:50.813 INFO:tasks.workunit.client.0.smithi033.stdout:5/511: symlink d1c/d2a/d50/d5b/d86/lac 0 2024-04-25T01:35:50.816 INFO:tasks.workunit.client.1.smithi138.stdout:4/21: write d0/f4 [126243,129700] 0 2024-04-25T01:35:50.817 INFO:tasks.workunit.client.1.smithi138.stdout:1/13: dwrite f2 [0,4194304] 0 2024-04-25T01:35:50.818 INFO:tasks.workunit.client.0.smithi033.stdout:4/501: dread d6/d14/f31 [4194304,4194304] 0 2024-04-25T01:35:50.818 INFO:tasks.workunit.client.1.smithi138.stdout:2/28: dwrite d0/f4 [0,4194304] 0 2024-04-25T01:35:50.820 INFO:tasks.workunit.client.1.smithi138.stdout:3/18: rename c0 to c7 0 2024-04-25T01:35:50.821 INFO:tasks.workunit.client.1.smithi138.stdout:8/7: dread f0 [0,4194304] 0 2024-04-25T01:35:50.822 INFO:tasks.workunit.client.1.smithi138.stdout:3/19: dread - f5 zero size 2024-04-25T01:35:50.822 INFO:tasks.workunit.client.1.smithi138.stdout:6/26: creat f6 x:0 0 0 2024-04-25T01:35:50.826 INFO:tasks.workunit.client.1.smithi138.stdout:2/29: rmdir d0 39 2024-04-25T01:35:50.829 INFO:tasks.workunit.client.1.smithi138.stdout:8/8: creat f2 x:0 0 0 2024-04-25T01:35:50.831 INFO:tasks.workunit.client.1.smithi138.stdout:8/9: truncate f2 789532 0 2024-04-25T01:35:50.831 INFO:tasks.workunit.client.1.smithi138.stdout:1/14: dwrite f2 [0,4194304] 0 2024-04-25T01:35:50.832 INFO:tasks.workunit.client.1.smithi138.stdout:2/30: creat d0/f5 x:0 0 0 2024-04-25T01:35:50.833 INFO:tasks.workunit.client.1.smithi138.stdout:2/31: stat d0/f4 0 2024-04-25T01:35:50.834 INFO:tasks.workunit.client.1.smithi138.stdout:8/10: mkdir d3 0 2024-04-25T01:35:50.834 INFO:tasks.workunit.client.1.smithi138.stdout:1/15: mkdir d3 0 2024-04-25T01:35:50.834 INFO:tasks.workunit.client.1.smithi138.stdout:2/32: dread - d0/f5 zero size 2024-04-25T01:35:50.835 INFO:tasks.workunit.client.1.smithi138.stdout:6/27: dwrite f6 [0,4194304] 0 2024-04-25T01:35:50.837 INFO:tasks.workunit.client.0.smithi033.stdout:0/390: sync 2024-04-25T01:35:50.837 INFO:tasks.workunit.client.0.smithi033.stdout:0/391: dread - d7/d2d/d5e/f83 zero size 2024-04-25T01:35:50.837 INFO:tasks.workunit.client.1.smithi138.stdout:6/28: write f6 [1543890,35641] 0 2024-04-25T01:35:50.839 INFO:tasks.workunit.client.0.smithi033.stdout:8/480: dread d6/d45/d51/d5d/f63 [0,4194304] 0 2024-04-25T01:35:50.841 INFO:tasks.workunit.client.1.smithi138.stdout:6/29: creat f7 x:0 0 0 2024-04-25T01:35:50.845 INFO:tasks.workunit.client.0.smithi033.stdout:5/512: creat d1c/d53/fad x:0 0 0 2024-04-25T01:35:50.846 INFO:tasks.workunit.client.1.smithi138.stdout:6/30: rename f2 to f8 0 2024-04-25T01:35:50.847 INFO:tasks.workunit.client.0.smithi033.stdout:4/502: fsync d6/d14/d35/d44/d45/f75 0 2024-04-25T01:35:50.848 INFO:tasks.workunit.client.0.smithi033.stdout:4/503: readlink d6/d14/l92 0 2024-04-25T01:35:50.848 INFO:tasks.workunit.client.1.smithi138.stdout:6/31: chown f5 14 1 2024-04-25T01:35:50.850 INFO:tasks.workunit.client.1.smithi138.stdout:8/11: dwrite f2 [0,4194304] 0 2024-04-25T01:35:50.850 INFO:tasks.workunit.client.1.smithi138.stdout:1/16: dwrite f2 [4194304,4194304] 0 2024-04-25T01:35:50.851 INFO:tasks.workunit.client.0.smithi033.stdout:6/413: rmdir d4/d24/d63 39 2024-04-25T01:35:50.851 INFO:tasks.workunit.client.1.smithi138.stdout:6/32: chown c3 7937 1 2024-04-25T01:35:50.852 INFO:tasks.workunit.client.1.smithi138.stdout:3/20: truncate f6 2290563 0 2024-04-25T01:35:50.852 INFO:tasks.workunit.client.1.smithi138.stdout:9/26: truncate f1 784252 0 2024-04-25T01:35:50.854 INFO:tasks.workunit.client.0.smithi033.stdout:4/504: dread d6/d14/d1d/f28 [0,4194304] 0 2024-04-25T01:35:50.856 INFO:tasks.workunit.client.0.smithi033.stdout:4/505: write d6/d14/d35/d44/d45/d46/d57/f62 [4727463,24304] 0 2024-04-25T01:35:50.858 INFO:tasks.workunit.client.1.smithi138.stdout:6/33: mknod c9 0 2024-04-25T01:35:50.859 INFO:tasks.workunit.client.0.smithi033.stdout:4/506: dread - d6/d14/d35/d3a/d4d/d5e/d7c/f8e zero size 2024-04-25T01:35:50.862 INFO:tasks.workunit.client.1.smithi138.stdout:8/12: write f0 [3661548,116156] 0 2024-04-25T01:35:50.862 INFO:tasks.workunit.client.1.smithi138.stdout:3/21: mknod c8 0 2024-04-25T01:35:50.865 INFO:tasks.workunit.client.1.smithi138.stdout:1/17: mkdir d3/d4 0 2024-04-25T01:35:50.867 INFO:tasks.workunit.client.1.smithi138.stdout:4/22: dread d0/f4 [0,4194304] 0 2024-04-25T01:35:50.872 INFO:tasks.workunit.client.1.smithi138.stdout:9/27: symlink d4/d8/lb 0 2024-04-25T01:35:50.873 INFO:tasks.workunit.client.0.smithi033.stdout:3/426: truncate d8/dd/d1d/f2b 519309 0 2024-04-25T01:35:50.874 INFO:tasks.workunit.client.1.smithi138.stdout:9/28: write f2 [29127,28647] 0 2024-04-25T01:35:50.876 INFO:tasks.workunit.client.1.smithi138.stdout:4/23: mkdir d0/d6 0 2024-04-25T01:35:50.877 INFO:tasks.workunit.client.0.smithi033.stdout:6/414: mkdir d4/d24/d36/d42/d9d 0 2024-04-25T01:35:50.877 INFO:tasks.workunit.client.1.smithi138.stdout:3/22: dwrite f5 [0,4194304] 0 2024-04-25T01:35:50.877 INFO:tasks.workunit.client.1.smithi138.stdout:8/13: symlink d3/l4 0 2024-04-25T01:35:50.877 INFO:tasks.workunit.client.1.smithi138.stdout:9/29: chown d4/ca 2220 1 2024-04-25T01:35:50.879 INFO:tasks.workunit.client.1.smithi138.stdout:4/24: truncate d0/f4 884628 0 2024-04-25T01:35:50.879 INFO:tasks.workunit.client.0.smithi033.stdout:8/481: creat d6/d45/d52/dae/fb3 x:0 0 0 2024-04-25T01:35:50.880 INFO:tasks.workunit.client.1.smithi138.stdout:9/30: truncate f2 324279 0 2024-04-25T01:35:50.880 INFO:tasks.workunit.client.1.smithi138.stdout:1/18: mknod d3/d4/c5 0 2024-04-25T01:35:50.882 INFO:tasks.workunit.client.1.smithi138.stdout:4/25: write d0/f4 [163856,29702] 0 2024-04-25T01:35:50.884 INFO:tasks.workunit.client.0.smithi033.stdout:0/392: dwrite d7/da/df/f17 [0,4194304] 0 2024-04-25T01:35:50.884 INFO:tasks.workunit.client.0.smithi033.stdout:4/507: mknod d6/d14/d1d/d3b/c9a 0 2024-04-25T01:35:50.885 INFO:tasks.workunit.client.1.smithi138.stdout:1/19: dread f2 [0,4194304] 0 2024-04-25T01:35:50.886 INFO:tasks.workunit.client.1.smithi138.stdout:8/14: link f0 d3/f5 0 2024-04-25T01:35:50.887 INFO:tasks.workunit.client.1.smithi138.stdout:9/31: chown f1 128732 1 2024-04-25T01:35:50.888 INFO:tasks.workunit.client.0.smithi033.stdout:4/508: chown d6/d12/f58 76389 1 2024-04-25T01:35:50.890 INFO:tasks.workunit.client.1.smithi138.stdout:8/15: mknod d3/c6 0 2024-04-25T01:35:50.893 INFO:tasks.workunit.client.1.smithi138.stdout:4/26: dwrite d0/f4 [0,4194304] 0 2024-04-25T01:35:50.894 INFO:tasks.workunit.client.1.smithi138.stdout:8/16: read d3/f5 [431070,91121] 0 2024-04-25T01:35:50.895 INFO:tasks.workunit.client.0.smithi033.stdout:0/393: mknod d7/d2d/d4c/c85 0 2024-04-25T01:35:50.896 INFO:tasks.workunit.client.1.smithi138.stdout:9/32: rename d4/f5 to d4/d8/fc 0 2024-04-25T01:35:50.896 INFO:tasks.workunit.client.0.smithi033.stdout:0/394: stat d7/d33 0 2024-04-25T01:35:50.896 INFO:tasks.workunit.client.1.smithi138.stdout:1/20: dread f2 [4194304,4194304] 0 2024-04-25T01:35:50.897 INFO:tasks.workunit.client.1.smithi138.stdout:8/17: rmdir d3 39 2024-04-25T01:35:50.900 INFO:tasks.workunit.client.1.smithi138.stdout:1/21: symlink d3/d4/l6 0 2024-04-25T01:35:50.901 INFO:tasks.workunit.client.0.smithi033.stdout:5/513: dread d1c/d2a/d50/d52/da6/d4a/f64 [0,4194304] 0 2024-04-25T01:35:50.901 INFO:tasks.workunit.client.0.smithi033.stdout:4/509: link d6/f18 d6/d14/d35/d44/d45/d46/f9b 0 2024-04-25T01:35:50.902 INFO:tasks.workunit.client.1.smithi138.stdout:1/22: unlink d3/d4/l6 0 2024-04-25T01:35:50.904 INFO:tasks.workunit.client.1.smithi138.stdout:1/23: symlink d3/l7 0 2024-04-25T01:35:50.905 INFO:tasks.workunit.client.1.smithi138.stdout:1/24: readlink d3/l7 0 2024-04-25T01:35:50.905 INFO:tasks.workunit.client.0.smithi033.stdout:0/395: dwrite d7/d1b/f32 [0,4194304] 0 2024-04-25T01:35:50.906 INFO:tasks.workunit.client.1.smithi138.stdout:1/25: rmdir d3 39 2024-04-25T01:35:50.906 INFO:tasks.workunit.client.1.smithi138.stdout:2/33: fsync d0/f4 0 2024-04-25T01:35:50.907 INFO:tasks.workunit.client.0.smithi033.stdout:0/396: dread - d7/d2d/d5e/f59 zero size 2024-04-25T01:35:50.907 INFO:tasks.workunit.client.1.smithi138.stdout:2/34: write d0/f5 [136502,93978] 0 2024-04-25T01:35:50.908 INFO:tasks.workunit.client.1.smithi138.stdout:2/35: read d0/f4 [1449391,96380] 0 2024-04-25T01:35:50.910 INFO:tasks.workunit.client.1.smithi138.stdout:2/36: creat d0/f6 x:0 0 0 2024-04-25T01:35:50.913 INFO:tasks.workunit.client.0.smithi033.stdout:0/397: write d7/d1b/f32 [3827509,95602] 0 2024-04-25T01:35:50.914 INFO:tasks.workunit.client.0.smithi033.stdout:5/514: dwrite d1c/f37 [0,4194304] 0 2024-04-25T01:35:50.918 INFO:tasks.workunit.client.0.smithi033.stdout:9/342: sync 2024-04-25T01:35:50.920 INFO:tasks.workunit.client.0.smithi033.stdout:9/343: dread - d0/d2/dd/d20/d4e/f78 zero size 2024-04-25T01:35:50.920 INFO:tasks.workunit.client.0.smithi033.stdout:5/515: readlink d1c/d2a/d50/d52/da6/d3e/d70/l9c 0 2024-04-25T01:35:50.922 INFO:tasks.workunit.client.0.smithi033.stdout:9/344: write d0/d2/dd/df/d1b/f2e [567916,35520] 0 2024-04-25T01:35:50.925 INFO:tasks.workunit.client.0.smithi033.stdout:5/516: mkdir d1c/d2a/d50/d52/da6/d3e/d70/dae 0 2024-04-25T01:35:50.926 INFO:tasks.workunit.client.1.smithi138.stdout:2/37: fdatasync d0/f5 0 2024-04-25T01:35:50.926 INFO:tasks.workunit.client.0.smithi033.stdout:8/482: dread d6/f1d [0,4194304] 0 2024-04-25T01:35:50.926 INFO:tasks.workunit.client.1.smithi138.stdout:2/38: read - d0/f6 zero size 2024-04-25T01:35:50.927 INFO:tasks.workunit.client.0.smithi033.stdout:9/345: read d0/d11/f47 [3926480,119186] 0 2024-04-25T01:35:50.928 INFO:tasks.workunit.client.1.smithi138.stdout:2/39: chown d0/c3 329123119 1 2024-04-25T01:35:50.928 INFO:tasks.workunit.client.1.smithi138.stdout:8/18: mknod d3/c7 0 2024-04-25T01:35:50.931 INFO:tasks.workunit.client.1.smithi138.stdout:1/26: unlink d3/l7 0 2024-04-25T01:35:50.931 INFO:tasks.workunit.client.1.smithi138.stdout:1/27: rename d3 to d3/d4/d8 22 2024-04-25T01:35:50.933 INFO:tasks.workunit.client.1.smithi138.stdout:8/19: mkdir d3/d8 0 2024-04-25T01:35:50.935 INFO:tasks.workunit.client.1.smithi138.stdout:1/28: unlink d3/d4/c5 0 2024-04-25T01:35:50.936 INFO:tasks.workunit.client.1.smithi138.stdout:8/20: symlink d3/l9 0 2024-04-25T01:35:50.939 INFO:tasks.workunit.client.1.smithi138.stdout:1/29: write f2 [6349782,54827] 0 2024-04-25T01:35:50.941 INFO:tasks.workunit.client.1.smithi138.stdout:1/30: write f2 [2831476,45801] 0 2024-04-25T01:35:50.941 INFO:tasks.workunit.client.1.smithi138.stdout:3/23: write f6 [1619305,103000] 0 2024-04-25T01:35:50.941 INFO:tasks.workunit.client.1.smithi138.stdout:3/24: chown f1 4 1 2024-04-25T01:35:50.946 INFO:tasks.workunit.client.1.smithi138.stdout:8/21: mkdir d3/d8/da 0 2024-04-25T01:35:50.946 INFO:tasks.workunit.client.1.smithi138.stdout:9/33: dread f1 [0,4194304] 0 2024-04-25T01:35:50.949 INFO:tasks.workunit.client.1.smithi138.stdout:3/25: dwrite f6 [0,4194304] 0 2024-04-25T01:35:50.950 INFO:tasks.workunit.client.0.smithi033.stdout:8/483: dread d6/d45/d52/f3b [0,4194304] 0 2024-04-25T01:35:50.956 INFO:tasks.workunit.client.1.smithi138.stdout:3/26: dread f6 [0,4194304] 0 2024-04-25T01:35:50.956 INFO:tasks.workunit.client.1.smithi138.stdout:3/27: rmdir - no directory 2024-04-25T01:35:50.957 INFO:tasks.workunit.client.1.smithi138.stdout:3/28: creat f9 x:0 0 0 2024-04-25T01:35:50.958 INFO:tasks.workunit.client.1.smithi138.stdout:3/29: write f5 [4426690,63654] 0 2024-04-25T01:35:50.958 INFO:tasks.workunit.client.1.smithi138.stdout:4/27: truncate d0/f4 1335325 0 2024-04-25T01:35:50.960 INFO:tasks.workunit.client.1.smithi138.stdout:3/30: symlink la 0 2024-04-25T01:35:50.961 INFO:tasks.workunit.client.1.smithi138.stdout:3/31: fdatasync f9 0 2024-04-25T01:35:50.961 INFO:tasks.workunit.client.1.smithi138.stdout:3/32: fdatasync f1 0 2024-04-25T01:35:50.961 INFO:tasks.workunit.client.1.smithi138.stdout:3/33: readlink la 0 2024-04-25T01:35:50.961 INFO:tasks.workunit.client.1.smithi138.stdout:3/34: rmdir - no directory 2024-04-25T01:35:50.962 INFO:tasks.workunit.client.1.smithi138.stdout:3/35: write f9 [973330,46257] 0 2024-04-25T01:35:50.965 INFO:tasks.workunit.client.0.smithi033.stdout:7/473: sync 2024-04-25T01:35:50.965 INFO:tasks.workunit.client.0.smithi033.stdout:7/474: readlink d4/d10/l4b 0 2024-04-25T01:35:50.966 INFO:tasks.workunit.client.0.smithi033.stdout:7/475: write d4/d2e/f48 [2836957,51682] 0 2024-04-25T01:35:50.967 INFO:tasks.workunit.client.0.smithi033.stdout:5/517: mkdir d1c/d2a/d50/daf 0 2024-04-25T01:35:50.974 INFO:tasks.workunit.client.0.smithi033.stdout:5/518: rmdir d1c/d2a/d50/d52/da6/d3e/d70/d89/da7 0 2024-04-25T01:35:50.976 INFO:tasks.workunit.client.1.smithi138.stdout:2/40: rmdir d0 39 2024-04-25T01:35:50.978 INFO:tasks.workunit.client.0.smithi033.stdout:0/398: rmdir d7/d7e 0 2024-04-25T01:35:50.980 INFO:tasks.workunit.client.1.smithi138.stdout:3/36: dread f9 [0,4194304] 0 2024-04-25T01:35:50.981 INFO:tasks.workunit.client.0.smithi033.stdout:5/519: rmdir d1c/d2a/d50/daf 0 2024-04-25T01:35:50.982 INFO:tasks.workunit.client.0.smithi033.stdout:0/399: mknod d7/d2d/c86 0 2024-04-25T01:35:50.984 INFO:tasks.workunit.client.0.smithi033.stdout:5/520: link ff d1c/d2a/d50/d52/da6/d4a/fb0 0 2024-04-25T01:35:50.985 INFO:tasks.workunit.client.0.smithi033.stdout:5/521: fdatasync d1c/f37 0 2024-04-25T01:35:50.988 INFO:tasks.workunit.client.0.smithi033.stdout:5/522: mkdir d1c/d2a/d50/db1 0 2024-04-25T01:35:50.989 INFO:tasks.workunit.client.0.smithi033.stdout:9/346: write d0/d2/dd/f24 [4263132,83706] 0 2024-04-25T01:35:50.991 INFO:tasks.workunit.client.0.smithi033.stdout:4/510: mknod d6/d14/c9c 0 2024-04-25T01:35:50.992 INFO:tasks.workunit.client.0.smithi033.stdout:0/400: dwrite d7/da/df/d14/f47 [0,4194304] 0 2024-04-25T01:35:50.994 INFO:tasks.workunit.client.0.smithi033.stdout:9/347: symlink d0/d2/dd/d20/l81 0 2024-04-25T01:35:50.997 INFO:tasks.workunit.client.0.smithi033.stdout:5/523: creat d1c/d2a/fb2 x:0 0 0 2024-04-25T01:35:50.999 INFO:tasks.workunit.client.0.smithi033.stdout:5/524: chown d1c/f5e 2080883 1 2024-04-25T01:35:51.001 INFO:tasks.workunit.client.0.smithi033.stdout:4/511: dwrite d6/d14/d48/f55 [0,4194304] 0 2024-04-25T01:35:51.002 INFO:tasks.workunit.client.0.smithi033.stdout:2/528: rename da/d27/f47 to da/d5b/d5e/da8/fae 0 2024-04-25T01:35:51.006 INFO:tasks.workunit.client.0.smithi033.stdout:2/529: write da/d5b/d5e/d7c/f98 [420400,40916] 0 2024-04-25T01:35:51.007 INFO:tasks.workunit.client.0.smithi033.stdout:9/348: unlink d0/d2/dd/d5f/l6b 0 2024-04-25T01:35:51.008 INFO:tasks.workunit.client.0.smithi033.stdout:1/316: creat d2/d11/f5e x:0 0 0 2024-04-25T01:35:51.009 INFO:tasks.workunit.client.0.smithi033.stdout:5/525: creat d1c/d2a/d50/d52/fb3 x:0 0 0 2024-04-25T01:35:51.010 INFO:tasks.workunit.client.0.smithi033.stdout:5/526: readlink d1c/d2a/d50/d52/da6/l4f 0 2024-04-25T01:35:51.012 INFO:tasks.workunit.client.0.smithi033.stdout:6/415: unlink d4/d1d/f52 0 2024-04-25T01:35:51.013 INFO:tasks.workunit.client.0.smithi033.stdout:6/416: write d4/d15/f48 [4124420,9233] 0 2024-04-25T01:35:51.015 INFO:tasks.workunit.client.0.smithi033.stdout:3/427: rename d8/f23 to d8/dd/d1d/f93 0 2024-04-25T01:35:51.020 INFO:tasks.workunit.client.0.smithi033.stdout:2/530: fsync da/d50/d6a/f56 0 2024-04-25T01:35:51.023 INFO:tasks.workunit.client.0.smithi033.stdout:1/317: mkdir d2/d5/d1b/d46/d5f 0 2024-04-25T01:35:51.024 INFO:tasks.workunit.client.0.smithi033.stdout:8/484: dwrite d6/f13 [0,4194304] 0 2024-04-25T01:35:51.025 INFO:tasks.workunit.client.0.smithi033.stdout:5/527: creat d1c/d2a/d50/d52/da6/d4a/fb4 x:0 0 0 2024-04-25T01:35:51.032 INFO:tasks.workunit.client.0.smithi033.stdout:6/417: dread - d4/d1d/d3e/d41/f58 zero size 2024-04-25T01:35:51.035 INFO:tasks.workunit.client.0.smithi033.stdout:4/512: rename d6/d14/d35/f8a to d6/d14/d35/d44/d45/d46/d57/d70/f9d 0 2024-04-25T01:35:51.037 INFO:tasks.workunit.client.0.smithi033.stdout:4/513: write d6/f9 [3746009,47596] 0 2024-04-25T01:35:51.038 INFO:tasks.workunit.client.0.smithi033.stdout:3/428: creat d8/dd/d3d/d63/d7c/d3c/f94 x:0 0 0 2024-04-25T01:35:51.043 INFO:tasks.workunit.client.0.smithi033.stdout:7/476: dwrite d4/f43 [0,4194304] 0 2024-04-25T01:35:51.105 INFO:tasks.workunit.client.0.smithi033.stdout:9/349: dread d0/d2/dd/df/f3a [0,4194304] 0 2024-04-25T01:35:51.110 INFO:tasks.workunit.client.0.smithi033.stdout:9/350: dwrite d0/d2/f1f [0,4194304] 0 2024-04-25T01:35:51.150 INFO:tasks.workunit.client.0.smithi033.stdout:2/531: symlink da/d27/d31/d81/laf 0 2024-04-25T01:35:51.190 INFO:tasks.workunit.client.1.smithi138.stdout:2/41: stat d0 0 2024-04-25T01:35:51.191 INFO:tasks.workunit.client.1.smithi138.stdout:3/37: mkdir db 0 2024-04-25T01:35:51.191 INFO:tasks.workunit.client.1.smithi138.stdout:2/42: mknod d0/c7 0 2024-04-25T01:35:51.192 INFO:tasks.workunit.client.1.smithi138.stdout:3/38: symlink db/lc 0 2024-04-25T01:35:51.193 INFO:tasks.workunit.client.1.smithi138.stdout:3/39: fdatasync f1 0 2024-04-25T01:35:51.193 INFO:tasks.workunit.client.1.smithi138.stdout:3/40: write f1 [1372642,50645] 0 2024-04-25T01:35:51.195 INFO:tasks.workunit.client.1.smithi138.stdout:2/43: dread d0/f4 [0,4194304] 0 2024-04-25T01:35:51.195 INFO:tasks.workunit.client.1.smithi138.stdout:2/44: fsync d0/f4 0 2024-04-25T01:35:51.197 INFO:tasks.workunit.client.1.smithi138.stdout:2/45: creat d0/f8 x:0 0 0 2024-04-25T01:35:51.199 INFO:tasks.workunit.client.1.smithi138.stdout:2/46: rename d0/f5 to d0/f9 0 2024-04-25T01:35:51.200 INFO:tasks.workunit.client.1.smithi138.stdout:2/47: symlink d0/la 0 2024-04-25T01:35:51.212 INFO:tasks.workunit.client.0.smithi033.stdout:5/528: rename d1c/d2a/d50/d52/da6/f3a to d1c/d2a/d50/d52/da6/d3e/d70/dae/fb5 0 2024-04-25T01:35:51.214 INFO:tasks.workunit.client.0.smithi033.stdout:4/514: write d6/d14/f31 [4450371,44699] 0 2024-04-25T01:35:51.220 INFO:tasks.workunit.client.0.smithi033.stdout:1/318: fsync d2/d11/d28/d33/f3e 0 2024-04-25T01:35:51.227 INFO:tasks.workunit.client.0.smithi033.stdout:7/477: write d4/d2e/d6b/f62 [1815473,112715] 0 2024-04-25T01:35:51.227 INFO:tasks.workunit.client.0.smithi033.stdout:0/401: truncate d7/da/df/d14/f24 3075793 0 2024-04-25T01:35:51.230 INFO:tasks.workunit.client.0.smithi033.stdout:1/319: write d2/d11/f30 [943804,112124] 0 2024-04-25T01:35:51.233 INFO:tasks.workunit.client.0.smithi033.stdout:9/351: creat d0/d2/dd/df/f82 x:0 0 0 2024-04-25T01:35:51.234 INFO:tasks.workunit.client.0.smithi033.stdout:7/478: chown d4/d2e/d6b/d67 13817 1 2024-04-25T01:35:51.235 INFO:tasks.workunit.client.0.smithi033.stdout:2/532: mkdir da/d8d/db0 0 2024-04-25T01:35:51.236 INFO:tasks.workunit.client.0.smithi033.stdout:0/402: rename d7/da/d23/f5a to d7/da/df/d14/d45/f87 0 2024-04-25T01:35:51.238 INFO:tasks.workunit.client.0.smithi033.stdout:5/529: link d1c/d53/c5c d1c/d2a/d50/d5b/cb6 0 2024-04-25T01:35:51.238 INFO:tasks.workunit.client.0.smithi033.stdout:0/403: read d7/da/f5d [32185,3237] 0 2024-04-25T01:35:51.239 INFO:tasks.workunit.client.0.smithi033.stdout:4/515: creat d6/d14/f9e x:0 0 0 2024-04-25T01:35:51.240 INFO:tasks.workunit.client.0.smithi033.stdout:4/516: chown d6/d12/f1a 31684 1 2024-04-25T01:35:51.242 INFO:tasks.workunit.client.0.smithi033.stdout:4/517: dread d6/d12/d38/f7e [0,4194304] 0 2024-04-25T01:35:51.243 INFO:tasks.workunit.client.0.smithi033.stdout:7/479: mkdir d4/d2e/d7f/d98 0 2024-04-25T01:35:51.244 INFO:tasks.workunit.client.0.smithi033.stdout:2/533: chown da/f22 4 1 2024-04-25T01:35:51.245 INFO:tasks.workunit.client.0.smithi033.stdout:7/480: write d4/d2e/d6b/f62 [2795679,121507] 0 2024-04-25T01:35:51.247 INFO:tasks.workunit.client.0.smithi033.stdout:0/404: write d7/d2d/d5e/d61/f66 [1658047,45296] 0 2024-04-25T01:35:51.247 INFO:tasks.workunit.client.0.smithi033.stdout:4/518: read d6/d14/d1d/d3b/f52 [3005218,130077] 0 2024-04-25T01:35:51.247 INFO:tasks.workunit.client.0.smithi033.stdout:2/534: read da/d50/d6a/d43/f66 [1269275,52700] 0 2024-04-25T01:35:51.249 INFO:tasks.workunit.client.0.smithi033.stdout:7/481: truncate d4/d2e/d65/f68 5166209 0 2024-04-25T01:35:51.249 INFO:tasks.workunit.client.0.smithi033.stdout:5/530: rename d1c/d2a/d50/d52/f54 to d1c/d2a/d50/d5b/fb7 0 2024-04-25T01:35:51.251 INFO:tasks.workunit.client.0.smithi033.stdout:7/482: write d4/d57/f5d [613274,66337] 0 2024-04-25T01:35:51.251 INFO:tasks.workunit.client.0.smithi033.stdout:2/535: write da/d27/d31/d40/f54 [1588362,58588] 0 2024-04-25T01:35:51.253 INFO:tasks.workunit.client.0.smithi033.stdout:2/536: dread - da/d27/d31/f83 zero size 2024-04-25T01:35:51.256 INFO:tasks.workunit.client.0.smithi033.stdout:1/320: mkdir d2/d2e/d34/d60 0 2024-04-25T01:35:51.262 INFO:tasks.workunit.client.0.smithi033.stdout:8/485: dwrite d6/db/f31 [4194304,4194304] 0 2024-04-25T01:35:51.263 INFO:tasks.workunit.client.0.smithi033.stdout:0/405: symlink d7/d2d/d5e/d68/d6c/l88 0 2024-04-25T01:35:51.265 INFO:tasks.workunit.client.0.smithi033.stdout:4/519: creat d6/d14/d35/d3a/d4c/f9f x:0 0 0 2024-04-25T01:35:51.268 INFO:tasks.workunit.client.0.smithi033.stdout:5/531: dread f5 [0,4194304] 0 2024-04-25T01:35:51.270 INFO:tasks.workunit.client.0.smithi033.stdout:9/352: write d0/d2/dd/df/d1b/f4d [309783,108693] 0 2024-04-25T01:35:51.271 INFO:tasks.workunit.client.0.smithi033.stdout:7/483: symlink d4/d2e/l99 0 2024-04-25T01:35:51.271 INFO:tasks.workunit.client.0.smithi033.stdout:9/353: chown d0/d2/dd/f54 37476 1 2024-04-25T01:35:51.273 INFO:tasks.workunit.client.0.smithi033.stdout:9/354: write d0/d2/dd/df/f3e [500292,44050] 0 2024-04-25T01:35:51.274 INFO:tasks.workunit.client.1.smithi138.stdout:2/48: truncate d0/f9 202996 0 2024-04-25T01:35:51.274 INFO:tasks.workunit.client.0.smithi033.stdout:2/537: rename da/d27/d31/d81 to da/d27/d31/d40/db1 0 2024-04-25T01:35:51.275 INFO:tasks.workunit.client.0.smithi033.stdout:2/538: read - da/d27/d31/f9c zero size 2024-04-25T01:35:51.279 INFO:tasks.workunit.client.0.smithi033.stdout:8/486: truncate d6/f58 523177 0 2024-04-25T01:35:51.280 INFO:tasks.workunit.client.0.smithi033.stdout:0/406: creat d7/d2d/d64/d51/f89 x:0 0 0 2024-04-25T01:35:51.282 INFO:tasks.workunit.client.0.smithi033.stdout:0/407: write d7/d2d/d64/d51/f52 [1415503,97174] 0 2024-04-25T01:35:51.283 INFO:tasks.workunit.client.0.smithi033.stdout:5/532: mknod d1c/d2a/d50/d52/da6/d3e/d70/d89/cb8 0 2024-04-25T01:35:51.286 INFO:tasks.workunit.client.0.smithi033.stdout:9/355: symlink d0/d2/dd/d20/l83 0 2024-04-25T01:35:51.288 INFO:tasks.workunit.client.1.smithi138.stdout:2/49: dwrite d0/f9 [0,4194304] 0 2024-04-25T01:35:51.289 INFO:tasks.workunit.client.0.smithi033.stdout:1/321: creat d2/d5/d1b/d46/d5f/f61 x:0 0 0 2024-04-25T01:35:51.290 INFO:tasks.workunit.client.0.smithi033.stdout:8/487: mkdir d6/db/d80/db4 0 2024-04-25T01:35:51.292 INFO:tasks.workunit.client.1.smithi138.stdout:2/50: getdents d0 0 2024-04-25T01:35:51.292 INFO:tasks.workunit.client.1.smithi138.stdout:2/51: chown d0/c3 1384423151 1 2024-04-25T01:35:51.294 INFO:tasks.workunit.client.0.smithi033.stdout:9/356: dwrite d0/d2/dd/f52 [0,4194304] 0 2024-04-25T01:35:51.296 INFO:tasks.workunit.client.0.smithi033.stdout:4/520: dread d6/f18 [0,4194304] 0 2024-04-25T01:35:51.297 INFO:tasks.workunit.client.0.smithi033.stdout:9/357: dread - d0/d2/dd/df/f82 zero size 2024-04-25T01:35:51.298 INFO:tasks.workunit.client.0.smithi033.stdout:9/358: chown d0/d2 115464 1 2024-04-25T01:35:51.299 INFO:tasks.workunit.client.1.smithi138.stdout:2/52: dwrite d0/f6 [0,4194304] 0 2024-04-25T01:35:51.302 INFO:tasks.workunit.client.1.smithi138.stdout:2/53: rename d0/c7 to d0/cb 0 2024-04-25T01:35:51.302 INFO:tasks.workunit.client.0.smithi033.stdout:2/539: rename da/d2c/f48 to da/d27/fb2 0 2024-04-25T01:35:51.303 INFO:tasks.workunit.client.1.smithi138.stdout:2/54: mkdir d0/dc 0 2024-04-25T01:35:51.308 INFO:tasks.workunit.client.0.smithi033.stdout:0/408: mkdir d7/d8a 0 2024-04-25T01:35:51.309 INFO:tasks.workunit.client.1.smithi138.stdout:2/55: rename d0/la to d0/dc/ld 0 2024-04-25T01:35:51.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/56: fdatasync d0/f8 0 2024-04-25T01:35:51.310 INFO:tasks.workunit.client.0.smithi033.stdout:7/484: creat d4/d2e/d6b/f9a x:0 0 0 2024-04-25T01:35:51.311 INFO:tasks.workunit.client.1.smithi138.stdout:2/57: write d0/f6 [4923623,16807] 0 2024-04-25T01:35:51.313 INFO:tasks.workunit.client.0.smithi033.stdout:8/488: dwrite d6/f1d [0,4194304] 0 2024-04-25T01:35:51.313 INFO:tasks.workunit.client.1.smithi138.stdout:2/58: creat d0/fe x:0 0 0 2024-04-25T01:35:51.314 INFO:tasks.workunit.client.0.smithi033.stdout:4/521: unlink d6/d14/d35/d3a/d4d/d5e/d7c/f8e 0 2024-04-25T01:35:51.315 INFO:tasks.workunit.client.1.smithi138.stdout:2/59: symlink d0/dc/lf 0 2024-04-25T01:35:51.317 INFO:tasks.workunit.client.0.smithi033.stdout:9/359: unlink d0/d2/dd/df/d1b/l2a 0 2024-04-25T01:35:51.320 INFO:tasks.workunit.client.1.smithi138.stdout:2/60: dread d0/f9 [0,4194304] 0 2024-04-25T01:35:51.321 INFO:tasks.workunit.client.1.smithi138.stdout:2/61: write d0/fe [192531,115802] 0 2024-04-25T01:35:51.322 INFO:tasks.workunit.client.0.smithi033.stdout:0/409: mknod d7/d2d/d5e/d61/c8b 0 2024-04-25T01:35:51.324 INFO:tasks.workunit.client.1.smithi138.stdout:2/62: rename d0/f6 to d0/f10 0 2024-04-25T01:35:51.325 INFO:tasks.workunit.client.0.smithi033.stdout:7/485: creat d4/d2e/d65/f9b x:0 0 0 2024-04-25T01:35:51.325 INFO:tasks.workunit.client.0.smithi033.stdout:5/533: write d1c/f51 [4742408,94478] 0 2024-04-25T01:35:51.326 INFO:tasks.workunit.client.1.smithi138.stdout:2/63: write d0/f10 [2646708,53084] 0 2024-04-25T01:35:51.329 INFO:tasks.workunit.client.0.smithi033.stdout:2/540: symlink da/d2c/lb3 0 2024-04-25T01:35:51.333 INFO:tasks.workunit.client.0.smithi033.stdout:5/534: rmdir d1c/d2a/d50 39 2024-04-25T01:35:51.333 INFO:tasks.workunit.client.1.smithi138.stdout:2/64: dwrite d0/f10 [0,4194304] 0 2024-04-25T01:35:51.336 INFO:tasks.workunit.client.0.smithi033.stdout:7/486: rename d4/d23/c64 to d4/d23/d3f/d45/c9c 0 2024-04-25T01:35:51.338 INFO:tasks.workunit.client.1.smithi138.stdout:2/65: rmdir d0 39 2024-04-25T01:35:51.338 INFO:tasks.workunit.client.0.smithi033.stdout:9/360: creat d0/d2/d43/d73/f84 x:0 0 0 2024-04-25T01:35:51.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/66: read - d0/f8 zero size 2024-04-25T01:35:51.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/67: fdatasync d0/fe 0 2024-04-25T01:35:51.340 INFO:tasks.workunit.client.0.smithi033.stdout:1/322: getdents d2/d11/d28 0 2024-04-25T01:35:51.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/68: mknod d0/c11 0 2024-04-25T01:35:51.342 INFO:tasks.workunit.client.0.smithi033.stdout:0/410: mknod d7/d8a/c8c 0 2024-04-25T01:35:51.346 INFO:tasks.workunit.client.1.smithi138.stdout:2/69: dwrite d0/f10 [4194304,4194304] 0 2024-04-25T01:35:51.349 INFO:tasks.workunit.client.0.smithi033.stdout:0/411: unlink d7/d33/l62 0 2024-04-25T01:35:51.350 INFO:tasks.workunit.client.0.smithi033.stdout:2/541: link da/d27/f7a da/d27/d31/d40/d77/fb4 0 2024-04-25T01:35:51.352 INFO:tasks.workunit.client.0.smithi033.stdout:0/412: chown d7/d2d/d64/d51/f52 2 1 2024-04-25T01:35:51.352 INFO:tasks.workunit.client.0.smithi033.stdout:4/522: getdents d6/d14/d35/d44 0 2024-04-25T01:35:51.354 INFO:tasks.workunit.client.0.smithi033.stdout:8/489: dwrite d6/db/f26 [0,4194304] 0 2024-04-25T01:35:51.355 INFO:tasks.workunit.client.0.smithi033.stdout:7/487: unlink d4/d2e/d6b/d1f/f24 0 2024-04-25T01:35:51.358 INFO:tasks.workunit.client.0.smithi033.stdout:7/488: write d4/d23/d3f/f60 [1820148,130006] 0 2024-04-25T01:35:51.360 INFO:tasks.workunit.client.0.smithi033.stdout:7/489: stat d4/d2e/d6b/d1f 0 2024-04-25T01:35:51.363 INFO:tasks.workunit.client.0.smithi033.stdout:9/361: creat d0/d2/dd/df/f85 x:0 0 0 2024-04-25T01:35:51.365 INFO:tasks.workunit.client.0.smithi033.stdout:2/542: dwrite da/d27/f69 [4194304,4194304] 0 2024-04-25T01:35:51.365 INFO:tasks.workunit.client.0.smithi033.stdout:8/490: symlink d6/d45/d51/d5d/lb5 0 2024-04-25T01:35:51.372 INFO:tasks.workunit.client.0.smithi033.stdout:0/413: getdents d7 0 2024-04-25T01:35:51.372 INFO:tasks.workunit.client.0.smithi033.stdout:2/543: fdatasync da/d27/f7a 0 2024-04-25T01:35:51.372 INFO:tasks.workunit.client.0.smithi033.stdout:5/535: link d1c/d2a/d50/d52/fa2 d1c/d2a/d50/d52/da6/d3e/d70/fb9 0 2024-04-25T01:35:51.373 INFO:tasks.workunit.client.0.smithi033.stdout:0/414: write d7/da/f48 [1285240,60567] 0 2024-04-25T01:35:51.376 INFO:tasks.workunit.client.0.smithi033.stdout:3/429: sync 2024-04-25T01:35:51.376 INFO:tasks.workunit.client.0.smithi033.stdout:6/418: sync 2024-04-25T01:35:51.376 INFO:tasks.workunit.client.0.smithi033.stdout:7/490: dread d4/d23/d3f/f77 [0,4194304] 0 2024-04-25T01:35:51.376 INFO:tasks.workunit.client.0.smithi033.stdout:9/362: link d0/f2d d0/d2/dd/d20/f86 0 2024-04-25T01:35:51.376 INFO:tasks.workunit.client.0.smithi033.stdout:5/536: rmdir d1c/d2a/d50/d52/da6 39 2024-04-25T01:35:51.379 INFO:tasks.workunit.client.0.smithi033.stdout:2/544: creat da/d27/d31/d40/d8a/fb5 x:0 0 0 2024-04-25T01:35:51.380 INFO:tasks.workunit.client.0.smithi033.stdout:0/415: creat d7/d2d/d64/f8d x:0 0 0 2024-04-25T01:35:51.380 INFO:tasks.workunit.client.0.smithi033.stdout:6/419: write d4/d1d/d3e/d4a/f4f [1134492,2292] 0 2024-04-25T01:35:51.382 INFO:tasks.workunit.client.0.smithi033.stdout:9/363: symlink d0/d2/l87 0 2024-04-25T01:35:51.382 INFO:tasks.workunit.client.0.smithi033.stdout:4/523: dread d6/d14/d6d/f83 [0,4194304] 0 2024-04-25T01:35:51.382 INFO:tasks.workunit.client.0.smithi033.stdout:5/537: fdatasync d1c/f2e 0 2024-04-25T01:35:51.385 INFO:tasks.workunit.client.0.smithi033.stdout:3/430: creat d8/dd/d3d/d63/f95 x:0 0 0 2024-04-25T01:35:51.385 INFO:tasks.workunit.client.0.smithi033.stdout:2/545: mknod da/d5b/d5e/cb6 0 2024-04-25T01:35:51.385 INFO:tasks.workunit.client.0.smithi033.stdout:9/364: fdatasync d0/f9 0 2024-04-25T01:35:51.387 INFO:tasks.workunit.client.0.smithi033.stdout:8/491: dread d6/db/f24 [0,4194304] 0 2024-04-25T01:35:51.388 INFO:tasks.workunit.client.0.smithi033.stdout:0/416: rename d7/d1b/c2c to d7/d2d/d64/d51/c8e 0 2024-04-25T01:35:51.390 INFO:tasks.workunit.client.0.smithi033.stdout:2/546: fdatasync da/d27/d31/d40/d77/f9b 0 2024-04-25T01:35:51.391 INFO:tasks.workunit.client.0.smithi033.stdout:4/524: mknod d6/d14/d35/d3a/d4c/ca0 0 2024-04-25T01:35:51.392 INFO:tasks.workunit.client.0.smithi033.stdout:7/491: dwrite d4/d2e/d65/f68 [0,4194304] 0 2024-04-25T01:35:51.393 INFO:tasks.workunit.client.0.smithi033.stdout:8/492: truncate d6/f3e 1554955 0 2024-04-25T01:35:51.397 INFO:tasks.workunit.client.0.smithi033.stdout:2/547: write da/d50/d6a/fa3 [363064,119142] 0 2024-04-25T01:35:51.397 INFO:tasks.workunit.client.0.smithi033.stdout:3/431: link d8/f5c d8/dd/d3d/d63/d7c/f96 0 2024-04-25T01:35:51.398 INFO:tasks.workunit.client.0.smithi033.stdout:5/538: dwrite d1c/d2a/d50/f7d [0,4194304] 0 2024-04-25T01:35:51.398 INFO:tasks.workunit.client.0.smithi033.stdout:7/492: truncate d4/d10/f89 23912 0 2024-04-25T01:35:51.399 INFO:tasks.workunit.client.0.smithi033.stdout:9/365: rename d0/d2/dd/d20/f5a to d0/d2/dd/f88 0 2024-04-25T01:35:51.402 INFO:tasks.workunit.client.0.smithi033.stdout:0/417: truncate d7/da/f58 3716290 0 2024-04-25T01:35:51.405 INFO:tasks.workunit.client.0.smithi033.stdout:3/432: mknod d8/dd/d3d/d4f/c97 0 2024-04-25T01:35:51.407 INFO:tasks.workunit.client.0.smithi033.stdout:9/366: creat d0/d2/dd/d5f/f89 x:0 0 0 2024-04-25T01:35:51.407 INFO:tasks.workunit.client.0.smithi033.stdout:2/548: mknod da/d27/d31/d40/d77/cb7 0 2024-04-25T01:35:51.409 INFO:tasks.workunit.client.0.smithi033.stdout:3/433: dread - d8/dd/d40/f90 zero size 2024-04-25T01:35:51.410 INFO:tasks.workunit.client.0.smithi033.stdout:4/525: dwrite d6/d14/d1d/d3b/f93 [0,4194304] 0 2024-04-25T01:35:51.411 INFO:tasks.workunit.client.0.smithi033.stdout:0/418: mknod d7/d2d/d5e/d68/d6c/c8f 0 2024-04-25T01:35:51.412 INFO:tasks.workunit.client.0.smithi033.stdout:8/493: dwrite d6/d45/d52/f54 [0,4194304] 0 2024-04-25T01:35:51.413 INFO:tasks.workunit.client.0.smithi033.stdout:9/367: creat d0/d2/f8a x:0 0 0 2024-04-25T01:35:51.418 INFO:tasks.workunit.client.0.smithi033.stdout:5/539: creat d1c/d2a/d50/d52/da6/d3e/fba x:0 0 0 2024-04-25T01:35:51.422 INFO:tasks.workunit.client.0.smithi033.stdout:4/526: read d6/d14/d35/d3a/d4d/d5e/f60 [885108,81308] 0 2024-04-25T01:35:51.424 INFO:tasks.workunit.client.0.smithi033.stdout:3/434: dread d8/dd/f34 [0,4194304] 0 2024-04-25T01:35:51.424 INFO:tasks.workunit.client.0.smithi033.stdout:9/368: mknod d0/d2/dd/df/d1b/d3d/d7e/d58/c8b 0 2024-04-25T01:35:51.425 INFO:tasks.workunit.client.0.smithi033.stdout:8/494: mknod d6/d45/d51/d5d/d9d/cb6 0 2024-04-25T01:35:51.426 INFO:tasks.workunit.client.0.smithi033.stdout:9/369: chown d0/d2/dd/d20/d4e/f68 51748197 1 2024-04-25T01:35:51.427 INFO:tasks.workunit.client.0.smithi033.stdout:2/549: rename da/d5b/d72/d87 to da/d2c/db8 0 2024-04-25T01:35:51.429 INFO:tasks.workunit.client.0.smithi033.stdout:8/495: creat d6/db/d80/fb7 x:0 0 0 2024-04-25T01:35:51.431 INFO:tasks.workunit.client.0.smithi033.stdout:8/496: chown d6/d45/d51/d55/l74 39 1 2024-04-25T01:35:51.431 INFO:tasks.workunit.client.0.smithi033.stdout:4/527: mknod d6/d12/ca1 0 2024-04-25T01:35:51.434 INFO:tasks.workunit.client.0.smithi033.stdout:3/435: truncate d8/dd/f34 2283604 0 2024-04-25T01:35:51.437 INFO:tasks.workunit.client.0.smithi033.stdout:2/550: link da/d5b/d5e/d76/l9a da/d50/d6a/d43/d6c/lb9 0 2024-04-25T01:35:51.442 INFO:tasks.workunit.client.0.smithi033.stdout:8/497: dwrite d6/d45/d51/d5d/f6a [0,4194304] 0 2024-04-25T01:35:51.444 INFO:tasks.workunit.client.0.smithi033.stdout:8/498: rename d6/d45 to d6/d45/d52/db0/db8 22 2024-04-25T01:35:51.447 INFO:tasks.workunit.client.0.smithi033.stdout:5/540: dread f7 [0,4194304] 0 2024-04-25T01:35:51.447 INFO:tasks.workunit.client.0.smithi033.stdout:2/551: dwrite da/d27/d31/d40/d8a/f75 [0,4194304] 0 2024-04-25T01:35:51.448 INFO:tasks.workunit.client.0.smithi033.stdout:8/499: mkdir d6/d45/d51/d5d/db9 0 2024-04-25T01:35:51.449 INFO:tasks.workunit.client.0.smithi033.stdout:8/500: dread - d6/d45/d51/f9a zero size 2024-04-25T01:35:51.450 INFO:tasks.workunit.client.0.smithi033.stdout:5/541: fsync d1c/d2a/d3d/f92 0 2024-04-25T01:35:51.450 INFO:tasks.workunit.client.0.smithi033.stdout:3/436: dread d8/dd/d3d/d63/d7c/d4d/f62 [0,4194304] 0 2024-04-25T01:35:51.450 INFO:tasks.workunit.client.0.smithi033.stdout:2/552: symlink da/d27/d31/lba 0 2024-04-25T01:35:51.451 INFO:tasks.workunit.client.0.smithi033.stdout:5/542: readlink d1c/d2a/d50/d5b/laa 0 2024-04-25T01:35:51.452 INFO:tasks.workunit.client.0.smithi033.stdout:2/553: dread - da/d27/d31/d40/d77/f7b zero size 2024-04-25T01:35:51.453 INFO:tasks.workunit.client.0.smithi033.stdout:5/543: chown d1c/d2a/f34 956381 1 2024-04-25T01:35:51.453 INFO:tasks.workunit.client.0.smithi033.stdout:3/437: mkdir d8/dd/d3d/d63/d7c/d3c/d98 0 2024-04-25T01:35:51.454 INFO:tasks.workunit.client.0.smithi033.stdout:5/544: chown d1c/d2a/d50/d52/c88 332320 1 2024-04-25T01:35:51.454 INFO:tasks.workunit.client.0.smithi033.stdout:8/501: truncate d6/db/f2d 1702078 0 2024-04-25T01:35:51.456 INFO:tasks.workunit.client.1.smithi138.stdout:2/70: rename d0/f4 to d0/dc/f12 0 2024-04-25T01:35:51.457 INFO:tasks.workunit.client.0.smithi033.stdout:5/545: chown d1c/d2a/d3d/c6e 77663085 1 2024-04-25T01:35:51.457 INFO:tasks.workunit.client.0.smithi033.stdout:8/502: write d6/db/f31 [5998243,21468] 0 2024-04-25T01:35:51.457 INFO:tasks.workunit.client.0.smithi033.stdout:3/438: unlink d8/dd/d11/l42 0 2024-04-25T01:35:51.458 INFO:tasks.workunit.client.1.smithi138.stdout:2/71: fsync d0/f9 0 2024-04-25T01:35:51.458 INFO:tasks.workunit.client.0.smithi033.stdout:8/503: write d6/db/f18 [841726,67559] 0 2024-04-25T01:35:51.459 INFO:tasks.workunit.client.0.smithi033.stdout:3/439: chown d8/dd/d1d/f93 90539575 1 2024-04-25T01:35:51.460 INFO:tasks.workunit.client.1.smithi138.stdout:2/72: write d0/f8 [201215,106037] 0 2024-04-25T01:35:51.462 INFO:tasks.workunit.client.0.smithi033.stdout:6/420: read d4/f12 [6558138,80407] 0 2024-04-25T01:35:51.462 INFO:tasks.workunit.client.0.smithi033.stdout:5/546: symlink d1c/lbb 0 2024-04-25T01:35:51.463 INFO:tasks.workunit.client.1.smithi138.stdout:2/73: creat d0/dc/f13 x:0 0 0 2024-04-25T01:35:51.466 INFO:tasks.workunit.client.1.smithi138.stdout:2/74: creat d0/f14 x:0 0 0 2024-04-25T01:35:51.470 INFO:tasks.workunit.client.0.smithi033.stdout:8/504: link d6/l1b d6/d45/d51/d9e/lba 0 2024-04-25T01:35:51.476 INFO:tasks.workunit.client.1.smithi138.stdout:2/75: dwrite d0/f8 [0,4194304] 0 2024-04-25T01:35:51.476 INFO:tasks.workunit.client.0.smithi033.stdout:8/505: getdents d6 0 2024-04-25T01:35:51.477 INFO:tasks.workunit.client.0.smithi033.stdout:1/323: truncate d2/d11/f30 209882 0 2024-04-25T01:35:51.482 INFO:tasks.workunit.client.0.smithi033.stdout:8/506: dwrite d6/f1d [0,4194304] 0 2024-04-25T01:35:51.483 INFO:tasks.workunit.client.0.smithi033.stdout:1/324: rename d2/d11/d28/d33/f3e to d2/f62 0 2024-04-25T01:35:51.487 INFO:tasks.workunit.client.0.smithi033.stdout:8/507: truncate d6/d45/d52/dae/fb3 326147 0 2024-04-25T01:35:51.487 INFO:tasks.workunit.client.0.smithi033.stdout:1/325: mkdir d2/d5/d1b/d46/d5f/d63 0 2024-04-25T01:35:51.489 INFO:tasks.workunit.client.0.smithi033.stdout:8/508: rmdir d6/d45/d51/d5d/d6e 39 2024-04-25T01:35:51.492 INFO:tasks.workunit.client.0.smithi033.stdout:1/326: mkdir d2/d5/d1b/d46/d5f/d63/d64 0 2024-04-25T01:35:51.492 INFO:tasks.workunit.client.0.smithi033.stdout:8/509: mknod d6/d45/d52/d2e/d95/cbb 0 2024-04-25T01:35:51.496 INFO:tasks.workunit.client.0.smithi033.stdout:0/419: getdents d7/d2d/d64/d51 0 2024-04-25T01:35:51.499 INFO:tasks.workunit.client.0.smithi033.stdout:8/510: dwrite d6/d45/d51/f4f [0,4194304] 0 2024-04-25T01:35:51.502 INFO:tasks.workunit.client.0.smithi033.stdout:8/511: dread d6/f99 [0,4194304] 0 2024-04-25T01:35:51.503 INFO:tasks.workunit.client.0.smithi033.stdout:9/370: dread d0/d2/dd/f17 [0,4194304] 0 2024-04-25T01:35:51.504 INFO:tasks.workunit.client.0.smithi033.stdout:8/512: mkdir d6/d45/d52/db0/dbc 0 2024-04-25T01:35:51.504 INFO:tasks.workunit.client.0.smithi033.stdout:9/371: write d0/d2/dd/d20/f38 [6277832,68386] 0 2024-04-25T01:35:51.508 INFO:tasks.workunit.client.0.smithi033.stdout:8/513: creat d6/d45/d51/d5d/d6e/fbd x:0 0 0 2024-04-25T01:35:51.509 INFO:tasks.workunit.client.0.smithi033.stdout:9/372: creat d0/d2/dd/df/f8c x:0 0 0 2024-04-25T01:35:51.509 INFO:tasks.workunit.client.0.smithi033.stdout:8/514: rmdir d6 39 2024-04-25T01:35:51.513 INFO:tasks.workunit.client.0.smithi033.stdout:8/515: creat d6/d45/d51/d55/d84/fbe x:0 0 0 2024-04-25T01:35:51.513 INFO:tasks.workunit.client.0.smithi033.stdout:9/373: dread d0/d2/dd/df/d1b/f2e [0,4194304] 0 2024-04-25T01:35:51.514 INFO:tasks.workunit.client.0.smithi033.stdout:8/516: fdatasync d6/db/f46 0 2024-04-25T01:35:51.515 INFO:tasks.workunit.client.0.smithi033.stdout:8/517: mkdir d6/db/d80/d89/dbf 0 2024-04-25T01:35:51.519 INFO:tasks.workunit.client.0.smithi033.stdout:9/374: dwrite d0/d2/dd/df/f8c [0,4194304] 0 2024-04-25T01:35:51.520 INFO:tasks.workunit.client.0.smithi033.stdout:8/518: link d6/c2a d6/d45/d51/d55/daa/cc0 0 2024-04-25T01:35:51.521 INFO:tasks.workunit.client.0.smithi033.stdout:9/375: stat d0/d2/dd/d20/l81 0 2024-04-25T01:35:51.523 INFO:tasks.workunit.client.0.smithi033.stdout:8/519: rename d6/d45/d51/d5d/c9c to d6/d45/d52/db0/dbc/cc1 0 2024-04-25T01:35:51.524 INFO:tasks.workunit.client.0.smithi033.stdout:8/520: chown d6/d45/d51/d55/d84 13328 1 2024-04-25T01:35:51.525 INFO:tasks.workunit.client.0.smithi033.stdout:9/376: getdents d0/d2/dd/df/d1b 0 2024-04-25T01:35:51.528 INFO:tasks.workunit.client.0.smithi033.stdout:9/377: truncate d0/d2/dd/df/d1b/d3d/d7e/f6a 882309 0 2024-04-25T01:35:51.529 INFO:tasks.workunit.client.0.smithi033.stdout:9/378: write d0/d2/dd/df/f85 [569752,71905] 0 2024-04-25T01:35:51.532 INFO:tasks.workunit.client.0.smithi033.stdout:9/379: creat d0/d2/d43/d4b/d55/f8d x:0 0 0 2024-04-25T01:35:51.583 INFO:tasks.workunit.client.1.smithi138.stdout:2/76: mkdir d0/dc/d15 0 2024-04-25T01:35:51.605 INFO:tasks.workunit.client.0.smithi033.stdout:8/521: dread d6/d45/d52/d2e/f4a [0,4194304] 0 2024-04-25T01:35:51.607 INFO:tasks.workunit.client.0.smithi033.stdout:8/522: mknod d6/d45/d52/cc2 0 2024-04-25T01:35:51.626 INFO:tasks.workunit.client.0.smithi033.stdout:3/440: rmdir d8/dd/d3d/d63 39 2024-04-25T01:35:51.629 INFO:tasks.workunit.client.0.smithi033.stdout:4/528: dwrite d6/d12/f58 [8388608,4194304] 0 2024-04-25T01:35:51.631 INFO:tasks.workunit.client.0.smithi033.stdout:3/441: creat d8/dd/d3d/d63/f99 x:0 0 0 2024-04-25T01:35:51.635 INFO:tasks.workunit.client.1.smithi138.stdout:2/77: fsync d0/f14 0 2024-04-25T01:35:51.635 INFO:tasks.workunit.client.0.smithi033.stdout:4/529: symlink d6/d12/la2 0 2024-04-25T01:35:51.636 INFO:tasks.workunit.client.1.smithi138.stdout:2/78: fsync d0/dc/f12 0 2024-04-25T01:35:51.638 INFO:tasks.workunit.client.0.smithi033.stdout:2/554: dwrite da/f15 [0,4194304] 0 2024-04-25T01:35:51.640 INFO:tasks.workunit.client.1.smithi138.stdout:2/79: link d0/f14 d0/dc/d15/f16 0 2024-04-25T01:35:51.643 INFO:tasks.workunit.client.1.smithi138.stdout:2/80: dread d0/dc/f12 [0,4194304] 0 2024-04-25T01:35:51.644 INFO:tasks.workunit.client.0.smithi033.stdout:3/442: rename d8/dd/d1d/f2d to d8/dd/d1d/f9a 0 2024-04-25T01:35:51.644 INFO:tasks.workunit.client.0.smithi033.stdout:6/421: truncate d4/d15/f48 3654617 0 2024-04-25T01:35:51.647 INFO:tasks.workunit.client.1.smithi138.stdout:2/81: dread d0/dc/f12 [0,4194304] 0 2024-04-25T01:35:51.648 INFO:tasks.workunit.client.0.smithi033.stdout:6/422: dread - d4/d1d/d3e/d41/d54/f8d zero size 2024-04-25T01:35:51.648 INFO:tasks.workunit.client.0.smithi033.stdout:4/530: rename d6/d12/l21 to d6/d14/d35/d3a/d4d/d5e/d7c/d94/la3 0 2024-04-25T01:35:51.650 INFO:tasks.workunit.client.0.smithi033.stdout:4/531: dread - d6/d14/d1d/f63 zero size 2024-04-25T01:35:51.651 INFO:tasks.workunit.client.0.smithi033.stdout:3/443: mknod d8/dd/d3d/c9b 0 2024-04-25T01:35:51.652 INFO:tasks.workunit.client.0.smithi033.stdout:4/532: write d6/d14/d35/d3a/d4c/f78 [1042018,35831] 0 2024-04-25T01:35:51.654 INFO:tasks.workunit.client.0.smithi033.stdout:6/423: dread d4/d15/f5f [0,4194304] 0 2024-04-25T01:35:51.655 INFO:tasks.workunit.client.0.smithi033.stdout:2/555: rename da/d5b/d5e/d7c to da/d27/d31/d40/d8a/dbb 0 2024-04-25T01:35:51.656 INFO:tasks.workunit.client.0.smithi033.stdout:0/420: write d7/da/df/d14/f15 [99945,58775] 0 2024-04-25T01:35:51.661 INFO:tasks.workunit.client.0.smithi033.stdout:2/556: read da/ff [523208,90689] 0 2024-04-25T01:35:51.664 INFO:tasks.workunit.client.0.smithi033.stdout:6/424: symlink d4/d24/d63/l9e 0 2024-04-25T01:35:51.666 INFO:tasks.workunit.client.0.smithi033.stdout:4/533: getdents d6/d14/d35/d3a/d4d 0 2024-04-25T01:35:51.666 INFO:tasks.workunit.client.0.smithi033.stdout:6/425: fsync d4/d24/f64 0 2024-04-25T01:35:51.670 INFO:tasks.workunit.client.0.smithi033.stdout:0/421: dwrite d7/da/df/d50/f7d [0,4194304] 0 2024-04-25T01:35:51.671 INFO:tasks.workunit.client.0.smithi033.stdout:3/444: dread d8/dd/d40/f65 [0,4194304] 0 2024-04-25T01:35:51.673 INFO:tasks.workunit.client.0.smithi033.stdout:6/426: mkdir d4/d1d/d3e/d41/d9f 0 2024-04-25T01:35:51.674 INFO:tasks.workunit.client.0.smithi033.stdout:4/534: rmdir d6/d14/d35 39 2024-04-25T01:35:51.674 INFO:tasks.workunit.client.0.smithi033.stdout:9/380: write d0/d11/f16 [749664,11752] 0 2024-04-25T01:35:51.676 INFO:tasks.workunit.client.0.smithi033.stdout:2/557: link da/d27/d31/d40/d77/f9b da/d27/d31/d40/d8a/fbc 0 2024-04-25T01:35:51.681 INFO:tasks.workunit.client.0.smithi033.stdout:4/535: write d6/d14/d35/d3a/f56 [2464014,13950] 0 2024-04-25T01:35:51.683 INFO:tasks.workunit.client.0.smithi033.stdout:0/422: dwrite d7/d2d/d64/f6d [0,4194304] 0 2024-04-25T01:35:51.689 INFO:tasks.workunit.client.0.smithi033.stdout:2/558: truncate da/d27/d31/f3d 1810190 0 2024-04-25T01:35:51.689 INFO:tasks.workunit.client.0.smithi033.stdout:9/381: dwrite d0/d2/dd/d5f/f89 [0,4194304] 0 2024-04-25T01:35:51.690 INFO:tasks.workunit.client.0.smithi033.stdout:0/423: write d7/d2d/d4d/f5c [4171138,23434] 0 2024-04-25T01:35:51.691 INFO:tasks.workunit.client.0.smithi033.stdout:9/382: fsync d0/f9 0 2024-04-25T01:35:51.692 INFO:tasks.workunit.client.0.smithi033.stdout:2/559: fsync da/d50/f58 0 2024-04-25T01:35:51.696 INFO:tasks.workunit.client.0.smithi033.stdout:0/424: mknod d7/d2d/d43/d44/c90 0 2024-04-25T01:35:51.697 INFO:tasks.workunit.client.0.smithi033.stdout:8/523: write d6/d45/d51/d5d/f63 [497574,2404] 0 2024-04-25T01:35:51.698 INFO:tasks.workunit.client.0.smithi033.stdout:0/425: write d7/f46 [831119,115138] 0 2024-04-25T01:35:51.700 INFO:tasks.workunit.client.0.smithi033.stdout:2/560: unlink da/d27/d31/f9c 0 2024-04-25T01:35:51.701 INFO:tasks.workunit.client.0.smithi033.stdout:2/561: dread - da/d27/d31/d40/d8a/fa5 zero size 2024-04-25T01:35:51.705 INFO:tasks.workunit.client.0.smithi033.stdout:3/445: dread d8/dd/d3d/d63/d7c/f64 [0,4194304] 0 2024-04-25T01:35:51.707 INFO:tasks.workunit.client.0.smithi033.stdout:2/562: creat da/d5b/d5e/fbd x:0 0 0 2024-04-25T01:35:51.710 INFO:tasks.workunit.client.0.smithi033.stdout:4/536: dread d6/d14/f33 [0,4194304] 0 2024-04-25T01:35:51.712 INFO:tasks.workunit.client.0.smithi033.stdout:3/446: symlink d8/dd/d3d/d63/l9c 0 2024-04-25T01:35:51.714 INFO:tasks.workunit.client.0.smithi033.stdout:2/563: creat da/d5b/d72/fbe x:0 0 0 2024-04-25T01:35:51.715 INFO:tasks.workunit.client.0.smithi033.stdout:4/537: truncate d6/d14/d1d/f2e 7024194 0 2024-04-25T01:35:51.719 INFO:tasks.workunit.client.0.smithi033.stdout:2/564: mkdir da/d27/d31/d40/db1/dbf 0 2024-04-25T01:35:51.721 INFO:tasks.workunit.client.0.smithi033.stdout:9/383: dread d0/d11/f27 [0,4194304] 0 2024-04-25T01:35:51.721 INFO:tasks.workunit.client.0.smithi033.stdout:3/447: mkdir d8/dd/d3d/d63/d7c/d9d 0 2024-04-25T01:35:51.722 INFO:tasks.workunit.client.0.smithi033.stdout:3/448: fdatasync d8/dd/d3d/d63/d7c/f73 0 2024-04-25T01:35:51.722 INFO:tasks.workunit.client.0.smithi033.stdout:9/384: write d0/d2/dd/df/f2b [541434,39591] 0 2024-04-25T01:35:51.723 INFO:tasks.workunit.client.0.smithi033.stdout:2/565: mknod da/d5b/d72/cc0 0 2024-04-25T01:35:51.724 INFO:tasks.workunit.client.0.smithi033.stdout:2/566: stat da/d27/d31/d40/db1/dbf 0 2024-04-25T01:35:51.725 INFO:tasks.workunit.client.0.smithi033.stdout:2/567: write da/d50/f58 [969635,39437] 0 2024-04-25T01:35:51.725 INFO:tasks.workunit.client.0.smithi033.stdout:3/449: fdatasync d8/dd/f15 0 2024-04-25T01:35:51.726 INFO:tasks.workunit.client.0.smithi033.stdout:3/450: dread - d8/dd/d11/f8e zero size 2024-04-25T01:35:51.727 INFO:tasks.workunit.client.0.smithi033.stdout:9/385: creat d0/d2/d43/f8e x:0 0 0 2024-04-25T01:35:51.728 INFO:tasks.workunit.client.0.smithi033.stdout:9/386: read - d0/d2/f48 zero size 2024-04-25T01:35:51.730 INFO:tasks.workunit.client.0.smithi033.stdout:3/451: symlink d8/dd/l9e 0 2024-04-25T01:35:51.731 INFO:tasks.workunit.client.0.smithi033.stdout:9/387: mkdir d0/d2/dd/df/d1b/d3d/d8f 0 2024-04-25T01:35:51.733 INFO:tasks.workunit.client.0.smithi033.stdout:9/388: creat d0/d2/dd/d20/d4e/f90 x:0 0 0 2024-04-25T01:35:51.734 INFO:tasks.workunit.client.0.smithi033.stdout:9/389: fsync d0/d2/dd/f52 0 2024-04-25T01:35:51.737 INFO:tasks.workunit.client.0.smithi033.stdout:9/390: dread d0/d2/dd/df/d1b/d3d/d7e/f6a [0,4194304] 0 2024-04-25T01:35:51.738 INFO:tasks.workunit.client.0.smithi033.stdout:9/391: fdatasync d0/d2/dd/d20/d4e/f4f 0 2024-04-25T01:35:51.739 INFO:tasks.workunit.client.0.smithi033.stdout:2/568: dread da/d27/fb2 [0,4194304] 0 2024-04-25T01:35:51.740 INFO:tasks.workunit.client.0.smithi033.stdout:2/569: stat da/d27/d31/f61 0 2024-04-25T01:35:51.741 INFO:tasks.workunit.client.0.smithi033.stdout:9/392: mknod d0/d2/d43/d4b/d55/c91 0 2024-04-25T01:35:51.741 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:51 smithi033 ceph-mon[26353]: pgmap v11: 65 pgs: 65 active+clean; 778 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 11 MiB/s rd, 82 MiB/s wr, 357 op/s 2024-04-25T01:35:51.743 INFO:tasks.workunit.client.0.smithi033.stdout:0/426: truncate d7/d33/f36 477718 0 2024-04-25T01:35:51.745 INFO:tasks.workunit.client.0.smithi033.stdout:8/524: dwrite d6/d45/d51/d5d/f81 [0,4194304] 0 2024-04-25T01:35:51.746 INFO:tasks.workunit.client.0.smithi033.stdout:9/393: creat d0/d2/d43/d4b/f92 x:0 0 0 2024-04-25T01:35:51.752 INFO:tasks.workunit.client.0.smithi033.stdout:8/525: stat d6/c68 0 2024-04-25T01:35:51.753 INFO:tasks.workunit.client.0.smithi033.stdout:8/526: fsync d6/d45/d51/d55/f5c 0 2024-04-25T01:35:51.753 INFO:tasks.workunit.client.0.smithi033.stdout:2/570: getdents da/d8d 0 2024-04-25T01:35:51.754 INFO:tasks.workunit.client.0.smithi033.stdout:2/571: dread - da/d27/d31/d40/db1/fa6 zero size 2024-04-25T01:35:51.755 INFO:tasks.workunit.client.0.smithi033.stdout:9/394: rmdir d0/d2/dd/df/d1b 39 2024-04-25T01:35:51.755 INFO:tasks.workunit.client.0.smithi033.stdout:2/572: read da/d2c/f33 [2295905,114040] 0 2024-04-25T01:35:51.756 INFO:tasks.workunit.client.0.smithi033.stdout:4/538: truncate d6/d14/f22 363587 0 2024-04-25T01:35:51.758 INFO:tasks.workunit.client.0.smithi033.stdout:8/527: mkdir d6/d45/d52/dae/dc3 0 2024-04-25T01:35:51.760 INFO:tasks.workunit.client.0.smithi033.stdout:9/395: rmdir d0/d11 39 2024-04-25T01:35:51.762 INFO:tasks.workunit.client.0.smithi033.stdout:2/573: mknod da/d27/cc1 0 2024-04-25T01:35:51.762 INFO:tasks.workunit.client.0.smithi033.stdout:3/452: write d8/dd/f34 [514327,16827] 0 2024-04-25T01:35:51.763 INFO:tasks.workunit.client.0.smithi033.stdout:4/539: symlink d6/d14/d35/d44/la4 0 2024-04-25T01:35:51.766 INFO:tasks.workunit.client.0.smithi033.stdout:8/528: creat d6/d45/d51/d5d/db9/fc4 x:0 0 0 2024-04-25T01:35:51.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:51 smithi138 ceph-mon[32977]: pgmap v11: 65 pgs: 65 active+clean; 778 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 11 MiB/s rd, 82 MiB/s wr, 357 op/s 2024-04-25T01:35:51.769 INFO:tasks.workunit.client.0.smithi033.stdout:3/453: creat d8/dd/d3d/d63/d7c/d3c/d6e/f9f x:0 0 0 2024-04-25T01:35:51.770 INFO:tasks.workunit.client.0.smithi033.stdout:9/396: dwrite d0/d2/d43/d4b/f92 [0,4194304] 0 2024-04-25T01:35:51.772 INFO:tasks.workunit.client.0.smithi033.stdout:8/529: dread - d6/d45/f76 zero size 2024-04-25T01:35:51.773 INFO:tasks.workunit.client.0.smithi033.stdout:0/427: dread d7/d33/f36 [0,4194304] 0 2024-04-25T01:35:51.779 INFO:tasks.workunit.client.0.smithi033.stdout:3/454: dwrite d8/dd/d3d/d63/d7c/d3c/d86/d92/f45 [0,4194304] 0 2024-04-25T01:35:51.780 INFO:tasks.workunit.client.0.smithi033.stdout:2/574: rename da/d50/d6a/d43/fa0 to da/fc2 0 2024-04-25T01:35:51.787 INFO:tasks.workunit.client.0.smithi033.stdout:9/397: symlink d0/d2/dd/df/d1b/d3d/d7e/l93 0 2024-04-25T01:35:51.789 INFO:tasks.workunit.client.0.smithi033.stdout:3/455: readlink d8/dd/d11/l5b 0 2024-04-25T01:35:51.789 INFO:tasks.workunit.client.0.smithi033.stdout:4/540: link d6/d14/d35/l36 d6/d14/d35/d44/d45/d46/d57/la5 0 2024-04-25T01:35:51.793 INFO:tasks.workunit.client.0.smithi033.stdout:9/398: creat d0/d2/dd/d20/d4e/f94 x:0 0 0 2024-04-25T01:35:51.795 INFO:tasks.workunit.client.0.smithi033.stdout:2/575: truncate da/f74 345206 0 2024-04-25T01:35:51.797 INFO:tasks.workunit.client.0.smithi033.stdout:8/530: write d6/d45/d52/d2e/f56 [684748,105355] 0 2024-04-25T01:35:51.799 INFO:tasks.workunit.client.0.smithi033.stdout:3/456: rename d8/dd/d11/f39 to d8/dd/d3d/d4f/fa0 0 2024-04-25T01:35:51.800 INFO:tasks.workunit.client.0.smithi033.stdout:4/541: unlink d6/d14/d35/d44/d45/d46/d57/d70/f9d 0 2024-04-25T01:35:51.801 INFO:tasks.workunit.client.0.smithi033.stdout:0/428: link d7/l25 d7/d2d/d5e/l91 0 2024-04-25T01:35:51.802 INFO:tasks.workunit.client.0.smithi033.stdout:9/399: rmdir d0/d2/d43/d4b/d55 39 2024-04-25T01:35:51.802 INFO:tasks.workunit.client.0.smithi033.stdout:9/400: chown d0/d2/l1a 1662051106 1 2024-04-25T01:35:51.803 INFO:tasks.workunit.client.0.smithi033.stdout:2/576: creat da/d50/d6a/d43/d6c/fc3 x:0 0 0 2024-04-25T01:35:51.806 INFO:tasks.workunit.client.0.smithi033.stdout:4/542: unlink d6/l4f 0 2024-04-25T01:35:51.807 INFO:tasks.workunit.client.0.smithi033.stdout:0/429: mkdir d7/d2d/d43/d92 0 2024-04-25T01:35:51.808 INFO:tasks.workunit.client.0.smithi033.stdout:0/430: write d7/da/f48 [281792,45933] 0 2024-04-25T01:35:51.808 INFO:tasks.workunit.client.0.smithi033.stdout:9/401: rmdir d0/d2/dd/d20 39 2024-04-25T01:35:51.812 INFO:tasks.workunit.client.0.smithi033.stdout:4/543: rmdir d6/d12/d84 39 2024-04-25T01:35:51.816 INFO:tasks.workunit.client.0.smithi033.stdout:0/431: symlink d7/d2d/d5e/d68/l93 0 2024-04-25T01:35:51.817 INFO:tasks.workunit.client.0.smithi033.stdout:0/432: readlink d7/l26 0 2024-04-25T01:35:51.820 INFO:tasks.workunit.client.0.smithi033.stdout:3/457: getdents d8/dd/d3d/d63/d7c 0 2024-04-25T01:35:51.821 INFO:tasks.workunit.client.0.smithi033.stdout:0/433: creat d7/d2d/d5e/d61/f94 x:0 0 0 2024-04-25T01:35:51.822 INFO:tasks.workunit.client.0.smithi033.stdout:4/544: symlink d6/d12/d84/la6 0 2024-04-25T01:35:51.824 INFO:tasks.workunit.client.0.smithi033.stdout:3/458: write d8/dd/d11/f8e [731285,129691] 0 2024-04-25T01:35:51.824 INFO:tasks.workunit.client.0.smithi033.stdout:2/577: dwrite da/d2c/f33 [0,4194304] 0 2024-04-25T01:35:51.826 INFO:tasks.workunit.client.0.smithi033.stdout:4/545: creat d6/d14/d1d/d5f/fa7 x:0 0 0 2024-04-25T01:35:51.827 INFO:tasks.workunit.client.0.smithi033.stdout:0/434: symlink d7/da/df/d14/l95 0 2024-04-25T01:35:51.829 INFO:tasks.workunit.client.0.smithi033.stdout:3/459: write d8/dd/d1d/f9a [1685435,125039] 0 2024-04-25T01:35:51.832 INFO:tasks.workunit.client.0.smithi033.stdout:0/435: creat d7/d1b/f96 x:0 0 0 2024-04-25T01:35:51.833 INFO:tasks.workunit.client.0.smithi033.stdout:3/460: dread - d8/dd/d11/f7f zero size 2024-04-25T01:35:51.833 INFO:tasks.workunit.client.0.smithi033.stdout:9/402: dread d0/d2/dd/df/d1b/d3d/f70 [0,4194304] 0 2024-04-25T01:35:51.836 INFO:tasks.workunit.client.0.smithi033.stdout:2/578: dwrite da/f15 [0,4194304] 0 2024-04-25T01:35:51.838 INFO:tasks.workunit.client.0.smithi033.stdout:0/436: symlink d7/d2d/d64/d51/l97 0 2024-04-25T01:35:51.841 INFO:tasks.workunit.client.0.smithi033.stdout:2/579: write da/d50/fad [733743,35929] 0 2024-04-25T01:35:51.842 INFO:tasks.workunit.client.0.smithi033.stdout:4/546: write d6/d12/f26 [2736142,117702] 0 2024-04-25T01:35:51.843 INFO:tasks.workunit.client.0.smithi033.stdout:2/580: chown da/d5b 921 1 2024-04-25T01:35:51.846 INFO:tasks.workunit.client.0.smithi033.stdout:0/437: creat d7/d2d/d5e/d68/f98 x:0 0 0 2024-04-25T01:35:51.847 INFO:tasks.workunit.client.0.smithi033.stdout:4/547: fdatasync d6/d14/d48/f55 0 2024-04-25T01:35:51.851 INFO:tasks.workunit.client.0.smithi033.stdout:3/461: dwrite d8/dd/d3d/d63/f99 [0,4194304] 0 2024-04-25T01:35:51.854 INFO:tasks.workunit.client.0.smithi033.stdout:0/438: creat d7/da/df/f99 x:0 0 0 2024-04-25T01:35:51.855 INFO:tasks.workunit.client.0.smithi033.stdout:9/403: dwrite d0/f29 [0,4194304] 0 2024-04-25T01:35:51.856 INFO:tasks.workunit.client.0.smithi033.stdout:2/581: link da/d5b/d5e/d76/f84 da/d27/d31/d40/d8a/dbb/d92/fc4 0 2024-04-25T01:35:51.859 INFO:tasks.workunit.client.0.smithi033.stdout:4/548: dwrite d6/f9 [0,4194304] 0 2024-04-25T01:35:51.860 INFO:tasks.workunit.client.0.smithi033.stdout:9/404: read d0/d2/dd/df/d1b/d3d/d7e/f66 [378227,53503] 0 2024-04-25T01:35:51.862 INFO:tasks.workunit.client.0.smithi033.stdout:7/493: sync 2024-04-25T01:35:51.862 INFO:tasks.workunit.client.0.smithi033.stdout:5/547: sync 2024-04-25T01:35:51.863 INFO:tasks.workunit.client.0.smithi033.stdout:2/582: truncate da/d27/d31/d40/d77/f7b 680545 0 2024-04-25T01:35:51.863 INFO:tasks.workunit.client.0.smithi033.stdout:7/494: stat d4/d2e/f63 0 2024-04-25T01:35:51.868 INFO:tasks.workunit.client.0.smithi033.stdout:4/549: rename d6/d14/d35/d3a/d4d/f5b to d6/d14/d6d/fa8 0 2024-04-25T01:35:51.869 INFO:tasks.workunit.client.0.smithi033.stdout:0/439: dwrite d7/da/df/d14/f69 [0,4194304] 0 2024-04-25T01:35:51.871 INFO:tasks.workunit.client.0.smithi033.stdout:0/440: read - d7/d2d/d43/f7b zero size 2024-04-25T01:35:51.872 INFO:tasks.workunit.client.0.smithi033.stdout:2/583: rename da/d27/d31/d40/d8a/fb5 to da/d2c/db8/fc5 0 2024-04-25T01:35:51.875 INFO:tasks.workunit.client.0.smithi033.stdout:9/405: rename d0/d2/d43/d4b/d55/f6d to d0/d2/dd/df/d1b/d3d/d6e/f95 0 2024-04-25T01:35:51.876 INFO:tasks.workunit.client.0.smithi033.stdout:4/550: truncate d6/d14/d1d/f2e 1393660 0 2024-04-25T01:35:51.878 INFO:tasks.workunit.client.0.smithi033.stdout:9/406: creat d0/d2/dd/d5f/f96 x:0 0 0 2024-04-25T01:35:51.878 INFO:tasks.workunit.client.0.smithi033.stdout:4/551: write d6/d14/d35/d44/d45/d46/d64/f69 [453780,77482] 0 2024-04-25T01:35:51.880 INFO:tasks.workunit.client.0.smithi033.stdout:0/441: rename d7/d1b/f96 to d7/da/df/f9a 0 2024-04-25T01:35:51.881 INFO:tasks.workunit.client.0.smithi033.stdout:9/407: write d0/d2/dd/d20/f7d [695918,80404] 0 2024-04-25T01:35:51.883 INFO:tasks.workunit.client.0.smithi033.stdout:9/408: fsync d0/d2/dd/df/f5c 0 2024-04-25T01:35:51.884 INFO:tasks.workunit.client.0.smithi033.stdout:4/552: dread d6/d14/d35/d3a/d4d/d5e/f60 [0,4194304] 0 2024-04-25T01:35:51.886 INFO:tasks.workunit.client.0.smithi033.stdout:0/442: rename d7/da/d23 to d7/d2d/d9b 0 2024-04-25T01:35:51.886 INFO:tasks.workunit.client.0.smithi033.stdout:4/553: read d6/d12/f40 [389867,89038] 0 2024-04-25T01:35:51.886 INFO:tasks.workunit.client.0.smithi033.stdout:0/443: chown d7/da/df/d14 507639 1 2024-04-25T01:35:51.891 INFO:tasks.workunit.client.0.smithi033.stdout:9/409: dwrite d0/d2/dd/df/f3b [0,4194304] 0 2024-04-25T01:35:51.893 INFO:tasks.workunit.client.0.smithi033.stdout:4/554: dwrite d6/f9 [4194304,4194304] 0 2024-04-25T01:35:51.896 INFO:tasks.workunit.client.0.smithi033.stdout:9/410: creat d0/d2/d43/d4b/f97 x:0 0 0 2024-04-25T01:35:51.898 INFO:tasks.workunit.client.0.smithi033.stdout:9/411: fsync d0/d2/dd/f88 0 2024-04-25T01:35:51.901 INFO:tasks.workunit.client.0.smithi033.stdout:9/412: dread - d0/f2d zero size 2024-04-25T01:35:51.904 INFO:tasks.workunit.client.0.smithi033.stdout:4/555: dwrite d6/d14/d35/d44/d45/d46/f9b [4194304,4194304] 0 2024-04-25T01:35:51.906 INFO:tasks.workunit.client.0.smithi033.stdout:9/413: creat d0/d2/dd/df/d1b/d3d/d8f/f98 x:0 0 0 2024-04-25T01:35:51.906 INFO:tasks.workunit.client.0.smithi033.stdout:4/556: stat d6/d12/d84/f88 0 2024-04-25T01:35:51.910 INFO:tasks.workunit.client.0.smithi033.stdout:9/414: creat d0/d2/dd/df/d1b/f99 x:0 0 0 2024-04-25T01:35:51.912 INFO:tasks.workunit.client.0.smithi033.stdout:9/415: mknod d0/d2/d43/c9a 0 2024-04-25T01:35:51.912 INFO:tasks.workunit.client.0.smithi033.stdout:9/416: chown d0/d2/d43/d73 32745 1 2024-04-25T01:35:51.915 INFO:tasks.workunit.client.0.smithi033.stdout:9/417: creat d0/d2/dd/d5f/f9b x:0 0 0 2024-04-25T01:35:51.915 INFO:tasks.workunit.client.0.smithi033.stdout:9/418: chown d0/d2/dd/df/f2b 188945 1 2024-04-25T01:35:51.916 INFO:tasks.workunit.client.0.smithi033.stdout:9/419: stat d0/d2/dd/d20/d4e/f78 0 2024-04-25T01:35:51.991 INFO:tasks.workunit.client.0.smithi033.stdout:0/444: rename d7/d2d/d5e to d7/d2d/d4d/d9c 0 2024-04-25T01:35:51.993 INFO:tasks.workunit.client.0.smithi033.stdout:4/557: rename d6/d14/d1d/d3b/l6e to d6/d12/d38/la9 0 2024-04-25T01:35:51.995 INFO:tasks.workunit.client.0.smithi033.stdout:0/445: write d7/d2d/d43/d44/f49 [1242515,94199] 0 2024-04-25T01:35:51.995 INFO:tasks.workunit.client.0.smithi033.stdout:4/558: chown d6/d14/d35/d3a/d4c/f8b 8029372 1 2024-04-25T01:35:51.997 INFO:tasks.workunit.client.0.smithi033.stdout:3/462: write d8/dd/f8f [408345,76890] 0 2024-04-25T01:35:51.998 INFO:tasks.workunit.client.0.smithi033.stdout:0/446: mkdir d7/da/df/d14/d9d 0 2024-04-25T01:35:51.999 INFO:tasks.workunit.client.0.smithi033.stdout:4/559: fsync d6/d14/f2b 0 2024-04-25T01:35:52.004 INFO:tasks.workunit.client.0.smithi033.stdout:4/560: symlink d6/d14/d6d/laa 0 2024-04-25T01:35:52.005 INFO:tasks.workunit.client.0.smithi033.stdout:6/427: sync 2024-04-25T01:35:52.005 INFO:tasks.workunit.client.0.smithi033.stdout:1/327: sync 2024-04-25T01:35:52.009 INFO:tasks.workunit.client.0.smithi033.stdout:1/328: write d2/f26 [5607963,124263] 0 2024-04-25T01:35:52.010 INFO:tasks.workunit.client.0.smithi033.stdout:5/548: dwrite d1c/f23 [0,4194304] 0 2024-04-25T01:35:52.013 INFO:tasks.workunit.client.0.smithi033.stdout:5/549: fsync d1c/d53/f99 0 2024-04-25T01:35:52.014 INFO:tasks.workunit.client.0.smithi033.stdout:7/495: dwrite d4/d2e/f4c [0,4194304] 0 2024-04-25T01:35:52.017 INFO:tasks.workunit.client.0.smithi033.stdout:7/496: write d4/d23/d3f/f79 [3267530,90273] 0 2024-04-25T01:35:52.019 INFO:tasks.workunit.client.0.smithi033.stdout:4/561: dwrite d6/d14/d1d/d3b/f3e [0,4194304] 0 2024-04-25T01:35:52.021 INFO:tasks.workunit.client.0.smithi033.stdout:7/497: stat d4/d23/d3f/c4a 0 2024-04-25T01:35:52.022 INFO:tasks.workunit.client.0.smithi033.stdout:7/498: fsync d4/d10/f89 0 2024-04-25T01:35:52.026 INFO:tasks.workunit.client.0.smithi033.stdout:3/463: getdents d8 0 2024-04-25T01:35:52.027 INFO:tasks.workunit.client.0.smithi033.stdout:3/464: write d8/dd/f83 [1239769,17982] 0 2024-04-25T01:35:52.033 INFO:tasks.workunit.client.0.smithi033.stdout:4/562: symlink d6/d14/d35/d3a/lab 0 2024-04-25T01:35:52.034 INFO:tasks.workunit.client.0.smithi033.stdout:4/563: fsync d6/d14/d6d/f77 0 2024-04-25T01:35:52.037 INFO:tasks.workunit.client.0.smithi033.stdout:7/499: mknod d4/d23/d3f/d45/d73/d8f/c9d 0 2024-04-25T01:35:52.040 INFO:tasks.workunit.client.0.smithi033.stdout:1/329: getdents d2/d5/d1b/d46/d5f/d63/d64 0 2024-04-25T01:35:52.043 INFO:tasks.workunit.client.0.smithi033.stdout:0/447: write d7/da/df/d14/f24 [4064053,87951] 0 2024-04-25T01:35:52.043 INFO:tasks.workunit.client.0.smithi033.stdout:7/500: truncate d4/d23/d3f/f77 1131586 0 2024-04-25T01:35:52.044 INFO:tasks.workunit.client.0.smithi033.stdout:7/501: chown d4/d2e/d6b 294524100 1 2024-04-25T01:35:52.046 INFO:tasks.workunit.client.0.smithi033.stdout:6/428: link d4/d15/f94 d4/d1d/d3e/fa0 0 2024-04-25T01:35:52.049 INFO:tasks.workunit.client.0.smithi033.stdout:6/429: write d4/d1d/d3e/d41/d5c/d7f/f67 [497659,117062] 0 2024-04-25T01:35:52.049 INFO:tasks.workunit.client.0.smithi033.stdout:1/330: creat d2/d5/d1b/d46/d5f/d63/f65 x:0 0 0 2024-04-25T01:35:52.050 INFO:tasks.workunit.client.0.smithi033.stdout:1/331: read - d2/d11/f5e zero size 2024-04-25T01:35:52.052 INFO:tasks.workunit.client.0.smithi033.stdout:6/430: truncate d4/d24/d63/f9a 144235 0 2024-04-25T01:35:52.054 INFO:tasks.workunit.client.0.smithi033.stdout:0/448: mknod d7/d2d/d43/c9e 0 2024-04-25T01:35:52.056 INFO:tasks.workunit.client.0.smithi033.stdout:7/502: rename d4/d57/f5d to d4/d23/d3f/d45/d6c/f9e 0 2024-04-25T01:35:52.056 INFO:tasks.workunit.client.0.smithi033.stdout:5/550: dread f15 [0,4194304] 0 2024-04-25T01:35:52.059 INFO:tasks.workunit.client.0.smithi033.stdout:5/551: fsync d1c/d2a/d50/d52/da6/d4a/f4e 0 2024-04-25T01:35:52.063 INFO:tasks.workunit.client.0.smithi033.stdout:5/552: truncate d1c/d2a/d3d/d59/f9e 142921 0 2024-04-25T01:35:52.064 INFO:tasks.workunit.client.0.smithi033.stdout:3/465: getdents d8/dd/d3d/d63/d7c/d3c/d86/d92 0 2024-04-25T01:35:52.069 INFO:tasks.workunit.client.0.smithi033.stdout:1/332: dread d2/f29 [0,4194304] 0 2024-04-25T01:35:52.069 INFO:tasks.workunit.client.0.smithi033.stdout:0/449: rename d7/d33/l77 to d7/d2d/d4d/d9c/l9f 0 2024-04-25T01:35:52.071 INFO:tasks.workunit.client.0.smithi033.stdout:6/431: dread d4/f1b [0,4194304] 0 2024-04-25T01:35:52.071 INFO:tasks.workunit.client.0.smithi033.stdout:4/564: write d6/d14/d35/d44/d45/f75 [5951194,2271] 0 2024-04-25T01:35:52.071 INFO:tasks.workunit.client.0.smithi033.stdout:1/333: rename d2/d5 to d2/d5/d1b/d46/d66 22 2024-04-25T01:35:52.075 INFO:tasks.workunit.client.0.smithi033.stdout:1/334: creat d2/d2e/f67 x:0 0 0 2024-04-25T01:35:52.076 INFO:tasks.workunit.client.0.smithi033.stdout:3/466: mkdir d8/dd/d3d/d63/d7c/d3c/d98/da1 0 2024-04-25T01:35:52.077 INFO:tasks.workunit.client.0.smithi033.stdout:7/503: dread d4/d2e/d6b/d1f/f46 [0,4194304] 0 2024-04-25T01:35:52.080 INFO:tasks.workunit.client.0.smithi033.stdout:6/432: mkdir d4/d1d/d3e/d41/d54/da1 0 2024-04-25T01:35:52.082 INFO:tasks.workunit.client.0.smithi033.stdout:6/433: write d4/d1d/d3e/d41/d5c/f6d [853743,15685] 0 2024-04-25T01:35:52.086 INFO:tasks.workunit.client.0.smithi033.stdout:1/335: mknod d2/d5/c68 0 2024-04-25T01:35:52.087 INFO:tasks.workunit.client.0.smithi033.stdout:1/336: dread - d2/d11/d28/f2c zero size 2024-04-25T01:35:52.088 INFO:tasks.workunit.client.0.smithi033.stdout:5/553: link d1c/d2a/d50/d52/l80 d1c/d2a/d50/d52/da6/lbc 0 2024-04-25T01:35:52.088 INFO:tasks.workunit.client.0.smithi033.stdout:5/554: fdatasync d1c/f32 0 2024-04-25T01:35:52.089 INFO:tasks.workunit.client.0.smithi033.stdout:7/504: mknod d4/d23/d3f/c9f 0 2024-04-25T01:35:52.089 INFO:tasks.workunit.client.0.smithi033.stdout:1/337: write d2/d5/d1b/d46/d5f/f61 [884869,31204] 0 2024-04-25T01:35:52.092 INFO:tasks.workunit.client.0.smithi033.stdout:7/505: chown d4/d2e/d6b/d1f/f42 406 1 2024-04-25T01:35:52.093 INFO:tasks.workunit.client.0.smithi033.stdout:6/434: creat d4/d24/d63/fa2 x:0 0 0 2024-04-25T01:35:52.097 INFO:tasks.workunit.client.0.smithi033.stdout:1/338: rename d2/d5/f14 to d2/d5/d1b/d46/f69 0 2024-04-25T01:35:52.102 INFO:tasks.workunit.client.0.smithi033.stdout:5/555: dwrite d1c/d2a/d50/d52/da6/d4a/fb4 [0,4194304] 0 2024-04-25T01:35:52.104 INFO:tasks.workunit.client.0.smithi033.stdout:3/467: link l7 d8/dd/d3d/d63/d7c/la2 0 2024-04-25T01:35:52.106 INFO:tasks.workunit.client.0.smithi033.stdout:7/506: dwrite d4/d23/d3f/d45/f5e [0,4194304] 0 2024-04-25T01:35:52.109 INFO:tasks.workunit.client.0.smithi033.stdout:6/435: dwrite d4/d1d/d3e/d41/d54/f57 [0,4194304] 0 2024-04-25T01:35:52.109 INFO:tasks.workunit.client.0.smithi033.stdout:1/339: rmdir d2/d5/d1b/d46/d5f 39 2024-04-25T01:35:52.111 INFO:tasks.workunit.client.0.smithi033.stdout:0/450: write d7/d2d/d43/d44/f63 [1300658,120400] 0 2024-04-25T01:35:52.118 INFO:tasks.workunit.client.0.smithi033.stdout:4/565: dwrite d6/d12/d38/f7e [0,4194304] 0 2024-04-25T01:35:52.119 INFO:tasks.workunit.client.0.smithi033.stdout:6/436: readlink d4/d1d/l2e 0 2024-04-25T01:35:52.120 INFO:tasks.workunit.client.0.smithi033.stdout:1/340: unlink d2/d5/c68 0 2024-04-25T01:35:52.125 INFO:tasks.workunit.client.0.smithi033.stdout:0/451: rename d7/d2d/d4d/d9c/d68/d6c to d7/d2d/d4c/da0 0 2024-04-25T01:35:52.129 INFO:tasks.workunit.client.0.smithi033.stdout:0/452: write d7/d2d/d4d/d9c/f84 [491955,53174] 0 2024-04-25T01:35:52.130 INFO:tasks.workunit.client.0.smithi033.stdout:1/341: dwrite d2/f20 [4194304,4194304] 0 2024-04-25T01:35:52.133 INFO:tasks.workunit.client.0.smithi033.stdout:4/566: unlink d6/l1b 0 2024-04-25T01:35:52.135 INFO:tasks.workunit.client.0.smithi033.stdout:4/567: truncate d6/d14/d35/d44/d45/d46/d57/f62 4858363 0 2024-04-25T01:35:52.138 INFO:tasks.workunit.client.0.smithi033.stdout:0/453: mkdir d7/d2d/d4c/da0/da1 0 2024-04-25T01:35:52.140 INFO:tasks.workunit.client.0.smithi033.stdout:0/454: truncate d7/da/df/f9a 862365 0 2024-04-25T01:35:52.142 INFO:tasks.workunit.client.0.smithi033.stdout:6/437: symlink d4/d24/d36/d42/d9d/la3 0 2024-04-25T01:35:52.144 INFO:tasks.workunit.client.0.smithi033.stdout:1/342: write d2/d5/d1b/d46/d5f/d63/f65 [372583,48011] 0 2024-04-25T01:35:52.145 INFO:tasks.workunit.client.0.smithi033.stdout:4/568: read d6/f16 [1267391,107166] 0 2024-04-25T01:35:52.146 INFO:tasks.workunit.client.0.smithi033.stdout:3/468: creat d8/dd/d3d/d63/d7c/d3c/fa3 x:0 0 0 2024-04-25T01:35:52.147 INFO:tasks.workunit.client.0.smithi033.stdout:0/455: mkdir d7/da/df/d14/d45/da2 0 2024-04-25T01:35:52.147 INFO:tasks.workunit.client.0.smithi033.stdout:3/469: fdatasync d8/f37 0 2024-04-25T01:35:52.148 INFO:tasks.workunit.client.0.smithi033.stdout:6/438: mkdir d4/d1d/d3e/d41/d5c/d7f/da4 0 2024-04-25T01:35:52.150 INFO:tasks.workunit.client.0.smithi033.stdout:0/456: fdatasync d7/d33/f36 0 2024-04-25T01:35:52.153 INFO:tasks.workunit.client.0.smithi033.stdout:1/343: dwrite d2/d2e/f67 [0,4194304] 0 2024-04-25T01:35:52.155 INFO:tasks.workunit.client.0.smithi033.stdout:3/470: mkdir d8/dd/d3d/d4f/da4 0 2024-04-25T01:35:52.156 INFO:tasks.workunit.client.0.smithi033.stdout:4/569: creat d6/d14/fac x:0 0 0 2024-04-25T01:35:52.156 INFO:tasks.workunit.client.0.smithi033.stdout:3/471: write d8/dd/d1d/f7e [603060,104303] 0 2024-04-25T01:35:52.160 INFO:tasks.workunit.client.0.smithi033.stdout:1/344: unlink d2/d2e/d49/c5c 0 2024-04-25T01:35:52.160 INFO:tasks.workunit.client.0.smithi033.stdout:1/345: chown d2/d2e/d49 1 1 2024-04-25T01:35:52.161 INFO:tasks.workunit.client.0.smithi033.stdout:6/439: symlink d4/d1d/d3e/d41/d93/la5 0 2024-04-25T01:35:52.162 INFO:tasks.workunit.client.0.smithi033.stdout:7/507: write d4/d23/d3f/f77 [2015533,36909] 0 2024-04-25T01:35:52.166 INFO:tasks.workunit.client.0.smithi033.stdout:7/508: write d4/d10/f89 [524286,2927] 0 2024-04-25T01:35:52.167 INFO:tasks.workunit.client.0.smithi033.stdout:0/457: unlink d7/da/df/f56 0 2024-04-25T01:35:52.168 INFO:tasks.workunit.client.0.smithi033.stdout:4/570: unlink d6/d14/d35/d44/c54 0 2024-04-25T01:35:52.169 INFO:tasks.workunit.client.0.smithi033.stdout:6/440: dread d4/d1d/d3e/d41/d5c/d7f/f67 [0,4194304] 0 2024-04-25T01:35:52.170 INFO:tasks.workunit.client.0.smithi033.stdout:7/509: write d4/d2e/d6d/f75 [2667367,60762] 0 2024-04-25T01:35:52.171 INFO:tasks.workunit.client.0.smithi033.stdout:3/472: mkdir d8/dd/d3d/d63/d7c/d4d/da5 0 2024-04-25T01:35:52.174 INFO:tasks.workunit.client.0.smithi033.stdout:4/571: unlink d6/f9 0 2024-04-25T01:35:52.175 INFO:tasks.workunit.client.0.smithi033.stdout:1/346: dwrite d2/d5/d1b/d46/d5f/d63/f65 [0,4194304] 0 2024-04-25T01:35:52.175 INFO:tasks.workunit.client.0.smithi033.stdout:3/473: chown d8/dd/d3d/d63/d7c/d3c/d86/d92/f29 5 1 2024-04-25T01:35:52.176 INFO:tasks.workunit.client.0.smithi033.stdout:3/474: write d8/f51 [1245054,88363] 0 2024-04-25T01:35:52.180 INFO:tasks.workunit.client.0.smithi033.stdout:0/458: link d7/d2d/d43/d44/f49 d7/d1b/fa3 0 2024-04-25T01:35:52.186 INFO:tasks.workunit.client.0.smithi033.stdout:7/510: symlink d4/d2e/d6b/d67/la0 0 2024-04-25T01:35:52.187 INFO:tasks.workunit.client.0.smithi033.stdout:3/475: mknod d8/dd/d11/ca6 0 2024-04-25T01:35:52.189 INFO:tasks.workunit.client.0.smithi033.stdout:1/347: creat d2/d5/d1b/d46/d5f/d63/d64/f6a x:0 0 0 2024-04-25T01:35:52.191 INFO:tasks.workunit.client.0.smithi033.stdout:3/476: chown d8/dd/d3d/d63/d7c/d3c/d86/d92/l74 31 1 2024-04-25T01:35:52.193 INFO:tasks.workunit.client.0.smithi033.stdout:7/511: dwrite d4/d2e/d65/f76 [0,4194304] 0 2024-04-25T01:35:52.205 INFO:tasks.workunit.client.0.smithi033.stdout:6/441: dread d4/d1d/d3e/d41/d5c/f8b [0,4194304] 0 2024-04-25T01:35:52.206 INFO:tasks.workunit.client.0.smithi033.stdout:7/512: dwrite d4/f32 [0,4194304] 0 2024-04-25T01:35:52.206 INFO:tasks.workunit.client.0.smithi033.stdout:3/477: dwrite d8/dd/d3d/d63/d7c/d4d/f62 [0,4194304] 0 2024-04-25T01:35:52.206 INFO:tasks.workunit.client.0.smithi033.stdout:0/459: truncate d7/da/df/d14/f69 883649 0 2024-04-25T01:35:52.210 INFO:tasks.workunit.client.0.smithi033.stdout:0/460: write d7/d2d/d64/d51/f7f [153938,74051] 0 2024-04-25T01:35:52.210 INFO:tasks.workunit.client.0.smithi033.stdout:6/442: dread d4/d1d/d3e/d70/f86 [0,4194304] 0 2024-04-25T01:35:52.210 INFO:tasks.workunit.client.0.smithi033.stdout:7/513: write d4/d2e/d65/f68 [3773694,97516] 0 2024-04-25T01:35:52.214 INFO:tasks.workunit.client.0.smithi033.stdout:6/443: write d4/d1d/f69 [141247,40638] 0 2024-04-25T01:35:52.217 INFO:tasks.workunit.client.0.smithi033.stdout:0/461: write d7/d2d/d64/d51/f89 [519744,102504] 0 2024-04-25T01:35:52.221 INFO:tasks.workunit.client.0.smithi033.stdout:3/478: dwrite d8/dd/d40/f90 [0,4194304] 0 2024-04-25T01:35:52.221 INFO:tasks.workunit.client.0.smithi033.stdout:7/514: write f3 [4746832,92116] 0 2024-04-25T01:35:52.223 INFO:tasks.workunit.client.0.smithi033.stdout:3/479: dread - d8/dd/d11/f8b zero size 2024-04-25T01:35:52.223 INFO:tasks.workunit.client.0.smithi033.stdout:7/515: readlink d4/d23/d3f/l58 0 2024-04-25T01:35:52.223 INFO:tasks.workunit.client.0.smithi033.stdout:3/480: readlink d8/dd/l9e 0 2024-04-25T01:35:52.224 INFO:tasks.workunit.client.0.smithi033.stdout:0/462: unlink d7/d2d/d64/f8d 0 2024-04-25T01:35:52.226 INFO:tasks.workunit.client.0.smithi033.stdout:7/516: unlink d4/d23/d3f/d45/f5b 0 2024-04-25T01:35:52.228 INFO:tasks.workunit.client.0.smithi033.stdout:0/463: symlink d7/d2d/d4d/d9c/d61/d70/la4 0 2024-04-25T01:35:52.229 INFO:tasks.workunit.client.0.smithi033.stdout:0/464: chown d7/l57 630585 1 2024-04-25T01:35:52.230 INFO:tasks.workunit.client.0.smithi033.stdout:3/481: creat d8/dd/d11/fa7 x:0 0 0 2024-04-25T01:35:52.231 INFO:tasks.workunit.client.0.smithi033.stdout:0/465: write d7/da/f48 [1893983,127006] 0 2024-04-25T01:35:52.233 INFO:tasks.workunit.client.0.smithi033.stdout:0/466: chown d7/d2d/d4d/d9c/f59 4021 1 2024-04-25T01:35:52.233 INFO:tasks.workunit.client.0.smithi033.stdout:7/517: dwrite d4/d2e/d65/f68 [0,4194304] 0 2024-04-25T01:35:52.235 INFO:tasks.workunit.client.0.smithi033.stdout:7/518: write d4/d23/d3f/f77 [1059635,74595] 0 2024-04-25T01:35:52.235 INFO:tasks.workunit.client.0.smithi033.stdout:3/482: creat d8/dd/d3d/d63/d7c/d3c/d86/d92/fa8 x:0 0 0 2024-04-25T01:35:52.236 INFO:tasks.workunit.client.0.smithi033.stdout:0/467: symlink d7/da/df/d14/d45/da2/la5 0 2024-04-25T01:35:52.237 INFO:tasks.workunit.client.0.smithi033.stdout:3/483: readlink d8/dd/d3d/d63/d7c/d4d/l4c 0 2024-04-25T01:35:52.237 INFO:tasks.workunit.client.0.smithi033.stdout:7/519: mknod d4/d23/d3f/ca1 0 2024-04-25T01:35:52.239 INFO:tasks.workunit.client.0.smithi033.stdout:7/520: dread - d4/d23/d3f/d45/d6c/f97 zero size 2024-04-25T01:35:52.240 INFO:tasks.workunit.client.0.smithi033.stdout:6/444: truncate f3 4108318 0 2024-04-25T01:35:52.241 INFO:tasks.workunit.client.0.smithi033.stdout:0/468: symlink d7/d2d/d43/d92/la6 0 2024-04-25T01:35:52.244 INFO:tasks.workunit.client.0.smithi033.stdout:6/445: write d4/d1d/d3e/d41/d5c/f8b [2616722,111082] 0 2024-04-25T01:35:52.245 INFO:tasks.workunit.client.0.smithi033.stdout:0/469: symlink d7/d2d/d4c/da0/da1/la7 0 2024-04-25T01:35:52.246 INFO:tasks.workunit.client.0.smithi033.stdout:6/446: write d4/d24/f64 [943347,72465] 0 2024-04-25T01:35:52.248 INFO:tasks.workunit.client.0.smithi033.stdout:7/521: dwrite d4/d23/d3f/f60 [0,4194304] 0 2024-04-25T01:35:52.249 INFO:tasks.workunit.client.0.smithi033.stdout:0/470: mkdir d7/d2d/d4c/da0/da1/da8 0 2024-04-25T01:35:52.253 INFO:tasks.workunit.client.0.smithi033.stdout:6/447: creat d4/d24/d36/d42/d97/fa6 x:0 0 0 2024-04-25T01:35:52.254 INFO:tasks.workunit.client.0.smithi033.stdout:6/448: readlink d4/l84 0 2024-04-25T01:35:52.256 INFO:tasks.workunit.client.0.smithi033.stdout:7/522: rename d4/d23/d3f/d45/d73/d8f/l96 to d4/d23/d3f/d45/d73/la2 0 2024-04-25T01:35:52.259 INFO:tasks.workunit.client.0.smithi033.stdout:6/449: chown d4/d1d/d3e/d70/f72 63980249 1 2024-04-25T01:35:52.262 INFO:tasks.workunit.client.0.smithi033.stdout:3/484: dread d8/dd/d3d/d63/d7c/d3c/d86/d92/f45 [0,4194304] 0 2024-04-25T01:35:52.264 INFO:tasks.workunit.client.0.smithi033.stdout:3/485: truncate d8/dd/f8f 1418566 0 2024-04-25T01:35:52.265 INFO:tasks.workunit.client.0.smithi033.stdout:3/486: stat d8/f1b 0 2024-04-25T01:35:52.265 INFO:tasks.workunit.client.0.smithi033.stdout:3/487: chown d8/f1b 3665382 1 2024-04-25T01:35:52.266 INFO:tasks.workunit.client.0.smithi033.stdout:7/523: rename d4/d2e/d6b/d1f/l8b to d4/d2e/la3 0 2024-04-25T01:35:52.267 INFO:tasks.workunit.client.0.smithi033.stdout:7/524: write d4/d10/f3b [1379541,121586] 0 2024-04-25T01:35:52.268 INFO:tasks.workunit.client.0.smithi033.stdout:7/525: fdatasync d4/d23/d3f/f4d 0 2024-04-25T01:35:52.269 INFO:tasks.workunit.client.0.smithi033.stdout:3/488: readlink d8/dd/d3d/d63/d7c/d4d/l8c 0 2024-04-25T01:35:52.269 INFO:tasks.workunit.client.0.smithi033.stdout:7/526: write d4/f9 [1941629,45327] 0 2024-04-25T01:35:52.273 INFO:tasks.workunit.client.0.smithi033.stdout:7/527: fdatasync d4/f39 0 2024-04-25T01:35:52.277 INFO:tasks.workunit.client.0.smithi033.stdout:3/489: dwrite d8/dd/f83 [0,4194304] 0 2024-04-25T01:35:52.282 INFO:tasks.workunit.client.0.smithi033.stdout:6/450: dread d4/d1d/d3e/d41/d5c/d7f/f44 [0,4194304] 0 2024-04-25T01:35:52.286 INFO:tasks.workunit.client.0.smithi033.stdout:7/528: link d4/d23/l27 d4/d10/la4 0 2024-04-25T01:35:52.287 INFO:tasks.workunit.client.0.smithi033.stdout:7/529: mkdir d4/d57/da5 0 2024-04-25T01:35:52.290 INFO:tasks.workunit.client.0.smithi033.stdout:7/530: creat d4/d2e/d6b/d1f/d78/d86/fa6 x:0 0 0 2024-04-25T01:35:52.291 INFO:tasks.workunit.client.0.smithi033.stdout:3/490: write d8/dd/d3d/d63/d7c/d3c/d86/d92/f21 [4447553,13417] 0 2024-04-25T01:35:52.292 INFO:tasks.workunit.client.0.smithi033.stdout:3/491: dread - d8/f5c zero size 2024-04-25T01:35:52.298 INFO:tasks.workunit.client.0.smithi033.stdout:6/451: dwrite d4/d24/f5b [0,4194304] 0 2024-04-25T01:35:52.300 INFO:tasks.workunit.client.0.smithi033.stdout:3/492: dread d8/dd/d40/f65 [0,4194304] 0 2024-04-25T01:35:52.301 INFO:tasks.workunit.client.0.smithi033.stdout:3/493: read - d8/dd/d3d/d63/f95 zero size 2024-04-25T01:35:52.302 INFO:tasks.workunit.client.0.smithi033.stdout:6/452: rename d4/d2a/l68 to d4/d15/la7 0 2024-04-25T01:35:52.305 INFO:tasks.workunit.client.0.smithi033.stdout:3/494: chown d8/dd/d3d/d63/d7c/d3c/d86/d92/l74 2 1 2024-04-25T01:35:52.307 INFO:tasks.workunit.client.0.smithi033.stdout:3/495: truncate d8/f59 732020 0 2024-04-25T01:35:52.307 INFO:tasks.workunit.client.0.smithi033.stdout:7/531: dread d4/fb [0,4194304] 0 2024-04-25T01:35:52.310 INFO:tasks.workunit.client.0.smithi033.stdout:6/453: dwrite d4/d2a/f3f [0,4194304] 0 2024-04-25T01:35:52.310 INFO:tasks.workunit.client.0.smithi033.stdout:6/454: stat d4/c1c 0 2024-04-25T01:35:52.311 INFO:tasks.workunit.client.0.smithi033.stdout:3/496: truncate d8/dd/d3d/d63/d7c/d3c/d86/f8a 1663055 0 2024-04-25T01:35:52.312 INFO:tasks.workunit.client.0.smithi033.stdout:6/455: chown d4/d1d/d3e/d4a/c7d 16147 1 2024-04-25T01:35:52.312 INFO:tasks.workunit.client.0.smithi033.stdout:7/532: getdents d4/d2e/d6b/d1f/d78/d93 0 2024-04-25T01:35:52.313 INFO:tasks.workunit.client.0.smithi033.stdout:6/456: write d4/d1d/d3e/d4a/f4f [990522,126434] 0 2024-04-25T01:35:52.314 INFO:tasks.workunit.client.0.smithi033.stdout:3/497: creat d8/dd/d3d/d63/d7c/d3c/d98/fa9 x:0 0 0 2024-04-25T01:35:52.315 INFO:tasks.workunit.client.0.smithi033.stdout:7/533: mkdir d4/d23/da7 0 2024-04-25T01:35:52.317 INFO:tasks.workunit.client.0.smithi033.stdout:6/457: chown d4/d15/f48 1163 1 2024-04-25T01:35:52.318 INFO:tasks.workunit.client.0.smithi033.stdout:6/458: write d4/d1d/d3e/d4a/f4f [1464821,14742] 0 2024-04-25T01:35:52.318 INFO:tasks.workunit.client.0.smithi033.stdout:3/498: mkdir d8/dd/d3d/d63/d7c/d3c/d6e/daa 0 2024-04-25T01:35:52.320 INFO:tasks.workunit.client.0.smithi033.stdout:7/534: symlink d4/la8 0 2024-04-25T01:35:52.322 INFO:tasks.workunit.client.0.smithi033.stdout:7/535: unlink d4/d10/f94 0 2024-04-25T01:35:52.324 INFO:tasks.workunit.client.0.smithi033.stdout:6/459: symlink d4/d1d/d3e/d41/d9f/la8 0 2024-04-25T01:35:52.327 INFO:tasks.workunit.client.0.smithi033.stdout:7/536: mkdir d4/d2e/d6b/da9 0 2024-04-25T01:35:52.327 INFO:tasks.workunit.client.0.smithi033.stdout:7/537: chown d4/f43 7564 1 2024-04-25T01:35:52.331 INFO:tasks.workunit.client.0.smithi033.stdout:6/460: getdents d4/d2a 0 2024-04-25T01:35:52.331 INFO:tasks.workunit.client.0.smithi033.stdout:7/538: dread d4/f17 [0,4194304] 0 2024-04-25T01:35:52.332 INFO:tasks.workunit.client.0.smithi033.stdout:6/461: fdatasync d4/d24/f8f 0 2024-04-25T01:35:52.332 INFO:tasks.workunit.client.0.smithi033.stdout:3/499: write d8/dd/d3d/d63/d7c/f96 [475763,124389] 0 2024-04-25T01:35:52.333 INFO:tasks.workunit.client.0.smithi033.stdout:6/462: fsync d4/d24/d63/f96 0 2024-04-25T01:35:52.333 INFO:tasks.workunit.client.0.smithi033.stdout:3/500: fsync d8/dd/d11/fa7 0 2024-04-25T01:35:52.335 INFO:tasks.workunit.client.0.smithi033.stdout:6/463: creat d4/d2a/fa9 x:0 0 0 2024-04-25T01:35:52.336 INFO:tasks.workunit.client.0.smithi033.stdout:3/501: chown d8/dd/d3d/d63/d7c/f73 502870873 1 2024-04-25T01:35:52.337 INFO:tasks.workunit.client.0.smithi033.stdout:6/464: write d4/d24/d36/d42/f83 [305501,70830] 0 2024-04-25T01:35:52.338 INFO:tasks.workunit.client.0.smithi033.stdout:7/539: write d4/f41 [1052523,1988] 0 2024-04-25T01:35:52.338 INFO:tasks.workunit.client.0.smithi033.stdout:6/465: dread - d4/d1d/d3e/d41/d54/f74 zero size 2024-04-25T01:35:52.341 INFO:tasks.workunit.client.0.smithi033.stdout:7/540: mkdir d4/d2e/daa 0 2024-04-25T01:35:52.341 INFO:tasks.workunit.client.0.smithi033.stdout:6/466: truncate d4/d1d/d3e/d70/f95 47282 0 2024-04-25T01:35:52.344 INFO:tasks.workunit.client.0.smithi033.stdout:6/467: unlink d4/d2a/l91 0 2024-04-25T01:35:52.348 INFO:tasks.workunit.client.0.smithi033.stdout:7/541: dwrite d4/d23/d3f/f4d [0,4194304] 0 2024-04-25T01:35:52.350 INFO:tasks.workunit.client.0.smithi033.stdout:3/502: truncate d8/dd/d3d/d63/d7c/d3c/d86/d92/f29 1977714 0 2024-04-25T01:35:52.352 INFO:tasks.workunit.client.0.smithi033.stdout:7/542: creat d4/d2e/d7f/fab x:0 0 0 2024-04-25T01:35:52.352 INFO:tasks.workunit.client.0.smithi033.stdout:6/468: dwrite d4/d24/d36/d42/d97/fa6 [0,4194304] 0 2024-04-25T01:35:52.356 INFO:tasks.workunit.client.0.smithi033.stdout:6/469: write d4/d1d/d3e/d41/d54/f8d [883712,58082] 0 2024-04-25T01:35:52.359 INFO:tasks.workunit.client.0.smithi033.stdout:3/503: getdents d8/dd/d40 0 2024-04-25T01:35:52.362 INFO:tasks.workunit.client.0.smithi033.stdout:3/504: rmdir d8/dd/d40 39 2024-04-25T01:35:52.362 INFO:tasks.workunit.client.0.smithi033.stdout:7/543: dwrite d4/d23/d3f/d45/f66 [0,4194304] 0 2024-04-25T01:35:52.365 INFO:tasks.workunit.client.0.smithi033.stdout:7/544: symlink d4/d2e/d7f/lac 0 2024-04-25T01:35:52.367 INFO:tasks.workunit.client.0.smithi033.stdout:6/470: rename d4/d1d/d3e/d41/d5c/d7f/f67 to d4/d1d/d3e/faa 0 2024-04-25T01:35:52.371 INFO:tasks.workunit.client.0.smithi033.stdout:3/505: mknod d8/dd/d3d/d63/d7c/d3c/d6e/daa/cab 0 2024-04-25T01:35:52.372 INFO:tasks.workunit.client.0.smithi033.stdout:6/471: creat d4/d1d/d3e/d41/d93/fab x:0 0 0 2024-04-25T01:35:52.376 INFO:tasks.workunit.client.0.smithi033.stdout:7/545: link d4/d23/d3f/c5f d4/d23/d3f/d45/d6c/cad 0 2024-04-25T01:35:52.377 INFO:tasks.workunit.client.0.smithi033.stdout:7/546: stat d4/d2e/d6b 0 2024-04-25T01:35:52.380 INFO:tasks.workunit.client.0.smithi033.stdout:3/506: getdents d8/dd/d3d 0 2024-04-25T01:35:52.381 INFO:tasks.workunit.client.0.smithi033.stdout:7/547: creat d4/d57/fae x:0 0 0 2024-04-25T01:35:52.382 INFO:tasks.workunit.client.0.smithi033.stdout:7/548: truncate d4/f41 4401637 0 2024-04-25T01:35:52.383 INFO:tasks.workunit.client.0.smithi033.stdout:7/549: write d4/d2e/f4c [4530064,64705] 0 2024-04-25T01:35:52.385 INFO:tasks.workunit.client.0.smithi033.stdout:3/507: creat d8/dd/d11/fac x:0 0 0 2024-04-25T01:35:52.390 INFO:tasks.workunit.client.0.smithi033.stdout:7/550: symlink d4/d2e/d50/laf 0 2024-04-25T01:35:52.391 INFO:tasks.workunit.client.0.smithi033.stdout:7/551: write d4/d10/f29 [1202153,36512] 0 2024-04-25T01:35:52.392 INFO:tasks.workunit.client.0.smithi033.stdout:3/508: creat d8/dd/d11/fad x:0 0 0 2024-04-25T01:35:52.395 INFO:tasks.workunit.client.0.smithi033.stdout:6/472: dwrite d4/d1d/d3e/d41/d5c/d7f/f81 [0,4194304] 0 2024-04-25T01:35:52.401 INFO:tasks.workunit.client.0.smithi033.stdout:7/552: dwrite d4/f39 [0,4194304] 0 2024-04-25T01:35:52.402 INFO:tasks.workunit.client.0.smithi033.stdout:6/473: unlink d4/d24/d63/l6a 0 2024-04-25T01:35:52.403 INFO:tasks.workunit.client.0.smithi033.stdout:3/509: dwrite d8/fa [4194304,4194304] 0 2024-04-25T01:35:52.408 INFO:tasks.workunit.client.0.smithi033.stdout:6/474: symlink d4/d24/lac 0 2024-04-25T01:35:52.408 INFO:tasks.workunit.client.0.smithi033.stdout:6/475: readlink d4/d15/l51 0 2024-04-25T01:35:52.409 INFO:tasks.workunit.client.0.smithi033.stdout:3/510: chown d8/dd/d3d/d63/d7c/c85 489135813 1 2024-04-25T01:35:52.410 INFO:tasks.workunit.client.0.smithi033.stdout:6/476: creat d4/d1d/d3e/d41/d93/fad x:0 0 0 2024-04-25T01:35:52.417 INFO:tasks.workunit.client.0.smithi033.stdout:3/511: dwrite d8/f5c [0,4194304] 0 2024-04-25T01:35:52.417 INFO:tasks.workunit.client.0.smithi033.stdout:6/477: dread d4/d24/d29/f8c [0,4194304] 0 2024-04-25T01:35:52.418 INFO:tasks.workunit.client.0.smithi033.stdout:6/478: stat d4/d24/d36/d42 0 2024-04-25T01:35:52.418 INFO:tasks.workunit.client.0.smithi033.stdout:3/512: chown d8/dd/d3d/d63/d7c/d3c/l44 3 1 2024-04-25T01:35:52.421 INFO:tasks.workunit.client.0.smithi033.stdout:3/513: write d8/dd/d3d/d63/d7c/d3c/d98/fa9 [855592,54714] 0 2024-04-25T01:35:52.422 INFO:tasks.workunit.client.0.smithi033.stdout:6/479: mkdir d4/d1d/d3e/d41/d5c/dae 0 2024-04-25T01:35:52.422 INFO:tasks.workunit.client.0.smithi033.stdout:3/514: read d8/dd/d3d/d63/f99 [3442997,127996] 0 2024-04-25T01:35:52.425 INFO:tasks.workunit.client.0.smithi033.stdout:6/480: creat d4/d1d/d3e/d41/d5c/d7f/da4/faf x:0 0 0 2024-04-25T01:35:52.427 INFO:tasks.workunit.client.0.smithi033.stdout:3/515: truncate d8/dd/f15 3125314 0 2024-04-25T01:35:52.427 INFO:tasks.workunit.client.0.smithi033.stdout:8/531: sync 2024-04-25T01:35:52.427 INFO:tasks.workunit.client.0.smithi033.stdout:9/420: sync 2024-04-25T01:35:52.427 INFO:tasks.workunit.client.0.smithi033.stdout:2/584: sync 2024-04-25T01:35:52.428 INFO:tasks.workunit.client.0.smithi033.stdout:6/481: creat d4/d24/d36/d42/d9d/fb0 x:0 0 0 2024-04-25T01:35:52.430 INFO:tasks.workunit.client.0.smithi033.stdout:9/421: creat d0/d2/f9c x:0 0 0 2024-04-25T01:35:52.436 INFO:tasks.workunit.client.0.smithi033.stdout:6/482: creat d4/d24/d36/d42/fb1 x:0 0 0 2024-04-25T01:35:52.436 INFO:tasks.workunit.client.0.smithi033.stdout:3/516: symlink d8/dd/d3d/d63/d7c/d9d/lae 0 2024-04-25T01:35:52.437 INFO:tasks.workunit.client.0.smithi033.stdout:8/532: dwrite d6/f60 [0,4194304] 0 2024-04-25T01:35:52.439 INFO:tasks.workunit.client.0.smithi033.stdout:6/483: read - d4/d15/f94 zero size 2024-04-25T01:35:52.442 INFO:tasks.workunit.client.0.smithi033.stdout:6/484: stat d4/d1d/l2e 0 2024-04-25T01:35:52.443 INFO:tasks.workunit.client.0.smithi033.stdout:8/533: mknod d6/cc5 0 2024-04-25T01:35:52.445 INFO:tasks.workunit.client.0.smithi033.stdout:9/422: dwrite d0/d2/f48 [0,4194304] 0 2024-04-25T01:35:52.448 INFO:tasks.workunit.client.0.smithi033.stdout:6/485: mkdir d4/d24/d36/d42/d97/db2 0 2024-04-25T01:35:52.450 INFO:tasks.workunit.client.0.smithi033.stdout:7/553: dread d4/d2e/d6d/f75 [0,4194304] 0 2024-04-25T01:35:52.454 INFO:tasks.workunit.client.0.smithi033.stdout:9/423: dwrite d0/d2/dd/df/f3b [0,4194304] 0 2024-04-25T01:35:52.458 INFO:tasks.workunit.client.0.smithi033.stdout:7/554: dread d4/d23/d3f/d45/f66 [0,4194304] 0 2024-04-25T01:35:52.461 INFO:tasks.workunit.client.0.smithi033.stdout:8/534: getdents d6/d45/d51/d55/d84 0 2024-04-25T01:35:52.462 INFO:tasks.workunit.client.0.smithi033.stdout:2/585: write da/d27/d31/d40/d77/f9b [56372,98453] 0 2024-04-25T01:35:52.464 INFO:tasks.workunit.client.0.smithi033.stdout:7/555: rmdir d4 39 2024-04-25T01:35:52.467 INFO:tasks.workunit.client.0.smithi033.stdout:6/486: write d4/d1d/d3e/fa0 [786086,205] 0 2024-04-25T01:35:52.469 INFO:tasks.workunit.client.0.smithi033.stdout:3/517: dread d8/dd/d3d/d63/d7c/d3c/d86/d92/f29 [0,4194304] 0 2024-04-25T01:35:52.470 INFO:tasks.workunit.client.0.smithi033.stdout:8/535: rename d6/d45/d51/c8c to d6/d45/d51/d5d/db9/cc6 0 2024-04-25T01:35:52.473 INFO:tasks.workunit.client.0.smithi033.stdout:2/586: creat da/d5b/d5e/da8/fc6 x:0 0 0 2024-04-25T01:35:52.474 INFO:tasks.workunit.client.0.smithi033.stdout:9/424: creat d0/d2/d43/d4b/f9d x:0 0 0 2024-04-25T01:35:52.475 INFO:tasks.workunit.client.0.smithi033.stdout:6/487: rmdir d4/d1d/d3e/d41/d5c 39 2024-04-25T01:35:52.477 INFO:tasks.workunit.client.0.smithi033.stdout:3/518: creat d8/dd/d3d/d4f/faf x:0 0 0 2024-04-25T01:35:52.478 INFO:tasks.workunit.client.0.smithi033.stdout:8/536: truncate d6/d45/d52/f28 6403358 0 2024-04-25T01:35:52.480 INFO:tasks.workunit.client.0.smithi033.stdout:2/587: readlink da/l6b 0 2024-04-25T01:35:52.487 INFO:tasks.workunit.client.0.smithi033.stdout:7/556: rmdir d4/d23/d3f/d45/d73/d8a 0 2024-04-25T01:35:52.488 INFO:tasks.workunit.client.0.smithi033.stdout:2/588: creat da/d27/d31/d40/d8a/dbb/d92/fc7 x:0 0 0 2024-04-25T01:35:52.489 INFO:tasks.workunit.client.0.smithi033.stdout:3/519: creat d8/dd/d11/fb0 x:0 0 0 2024-04-25T01:35:52.491 INFO:tasks.workunit.client.0.smithi033.stdout:7/557: creat d4/d23/d3f/d45/d73/d8f/fb0 x:0 0 0 2024-04-25T01:35:52.492 INFO:tasks.workunit.client.0.smithi033.stdout:8/537: getdents d6/db/d80 0 2024-04-25T01:35:52.493 INFO:tasks.workunit.client.0.smithi033.stdout:9/425: link d0/d2/dd/df/d1b/f4d d0/f9e 0 2024-04-25T01:35:52.494 INFO:tasks.workunit.client.0.smithi033.stdout:2/589: symlink da/d27/d31/lc8 0 2024-04-25T01:35:52.496 INFO:tasks.workunit.client.0.smithi033.stdout:3/520: mknod d8/dd/d3d/d63/d7c/d3c/d86/d92/cb1 0 2024-04-25T01:35:52.497 INFO:tasks.workunit.client.0.smithi033.stdout:2/590: creat da/d5b/d5e/d76/fc9 x:0 0 0 2024-04-25T01:35:52.498 INFO:tasks.workunit.client.0.smithi033.stdout:8/538: creat d6/d45/d52/fc7 x:0 0 0 2024-04-25T01:35:52.500 INFO:tasks.workunit.client.0.smithi033.stdout:3/521: chown d8/dd/d3d/d63/d7c/d3c/d86/d92/l22 9831056 1 2024-04-25T01:35:52.501 INFO:tasks.workunit.client.0.smithi033.stdout:3/522: chown d8/dd/l9e 17517249 1 2024-04-25T01:35:52.501 INFO:tasks.workunit.client.0.smithi033.stdout:8/539: mkdir d6/d45/d51/d55/d84/dc8 0 2024-04-25T01:35:52.502 INFO:tasks.workunit.client.0.smithi033.stdout:2/591: truncate da/f18 256049 0 2024-04-25T01:35:52.504 INFO:tasks.workunit.client.0.smithi033.stdout:3/523: write d8/dd/d3d/d63/d7c/d4d/f8d [345967,120876] 0 2024-04-25T01:35:52.504 INFO:tasks.workunit.client.0.smithi033.stdout:8/540: creat d6/d45/d52/d2e/fc9 x:0 0 0 2024-04-25T01:35:52.506 INFO:tasks.workunit.client.0.smithi033.stdout:8/541: write d6/d45/f8b [317776,94241] 0 2024-04-25T01:35:52.506 INFO:tasks.workunit.client.0.smithi033.stdout:2/592: mkdir da/d27/d31/dca 0 2024-04-25T01:35:52.507 INFO:tasks.workunit.client.0.smithi033.stdout:9/426: dread d0/d2/dd/f13 [0,4194304] 0 2024-04-25T01:35:52.508 INFO:tasks.workunit.client.0.smithi033.stdout:3/524: mkdir d8/dd/d3d/d4f/da4/db2 0 2024-04-25T01:35:52.509 INFO:tasks.workunit.client.0.smithi033.stdout:8/542: read d6/f1d [2174693,15662] 0 2024-04-25T01:35:52.510 INFO:tasks.workunit.client.0.smithi033.stdout:3/525: readlink d8/dd/d3d/d63/d7c/d3c/l3f 0 2024-04-25T01:35:52.511 INFO:tasks.workunit.client.0.smithi033.stdout:2/593: truncate da/d27/d31/d40/f54 2245198 0 2024-04-25T01:35:52.514 INFO:tasks.workunit.client.0.smithi033.stdout:2/594: readlink da/d5b/la4 0 2024-04-25T01:35:52.515 INFO:tasks.workunit.client.0.smithi033.stdout:3/526: symlink d8/dd/d3d/d63/d7c/d9d/lb3 0 2024-04-25T01:35:52.518 INFO:tasks.workunit.client.0.smithi033.stdout:9/427: dwrite d0/d2/dd/d20/d4e/f4f [0,4194304] 0 2024-04-25T01:35:52.518 INFO:tasks.workunit.client.0.smithi033.stdout:3/527: write d8/dd/d3d/d63/d7c/d3c/d98/fa9 [11878,130115] 0 2024-04-25T01:35:52.519 INFO:tasks.workunit.client.0.smithi033.stdout:6/488: truncate d4/d24/d29/f55 437339 0 2024-04-25T01:35:52.521 INFO:tasks.workunit.client.0.smithi033.stdout:9/428: dread - d0/d2/dd/d20/d4e/f68 zero size 2024-04-25T01:35:52.522 INFO:tasks.workunit.client.0.smithi033.stdout:3/528: chown d8/dd/d3d/d63/d7c/c85 0 1 2024-04-25T01:35:52.523 INFO:tasks.workunit.client.0.smithi033.stdout:3/529: dread - d8/dd/d11/fb0 zero size 2024-04-25T01:35:52.523 INFO:tasks.workunit.client.0.smithi033.stdout:2/595: mknod da/d27/d31/ccb 0 2024-04-25T01:35:52.525 INFO:tasks.workunit.client.0.smithi033.stdout:2/596: write da/d5b/d72/fbe [960831,15356] 0 2024-04-25T01:35:52.531 INFO:tasks.workunit.client.0.smithi033.stdout:2/597: unlink da/d5b/d5e/da8/fc6 0 2024-04-25T01:35:52.533 INFO:tasks.workunit.client.0.smithi033.stdout:3/530: rename d8/dd/c1c to d8/dd/d3d/d63/d7c/d3c/d86/cb4 0 2024-04-25T01:35:52.538 INFO:tasks.workunit.client.0.smithi033.stdout:2/598: creat da/d27/d31/d40/db1/fcc x:0 0 0 2024-04-25T01:35:52.540 INFO:tasks.workunit.client.0.smithi033.stdout:9/429: rmdir d0/d2/dd/df/d1b/d3d/d7c 0 2024-04-25T01:35:52.541 INFO:tasks.workunit.client.0.smithi033.stdout:9/430: fsync d0/d2/dd/df/f85 0 2024-04-25T01:35:52.542 INFO:tasks.workunit.client.0.smithi033.stdout:3/531: creat d8/dd/d40/fb5 x:0 0 0 2024-04-25T01:35:52.543 INFO:tasks.workunit.client.0.smithi033.stdout:3/532: dread - d8/dd/d3d/d63/d7c/d4d/f6c zero size 2024-04-25T01:35:52.544 INFO:tasks.workunit.client.0.smithi033.stdout:9/431: creat d0/d2/d43/d73/f9f x:0 0 0 2024-04-25T01:35:52.550 INFO:tasks.workunit.client.0.smithi033.stdout:3/533: write d8/dd/d11/fad [423490,119739] 0 2024-04-25T01:35:52.552 INFO:tasks.workunit.client.0.smithi033.stdout:8/543: dwrite d6/f15 [4194304,4194304] 0 2024-04-25T01:35:52.559 INFO:tasks.workunit.client.0.smithi033.stdout:6/489: dwrite d4/d1d/d3e/d70/f86 [0,4194304] 0 2024-04-25T01:35:52.566 INFO:tasks.workunit.client.0.smithi033.stdout:2/599: rename da/d27/d31/d40/d8a/dbb/d92 to da/d2c/dcd 0 2024-04-25T01:35:52.568 INFO:tasks.workunit.client.0.smithi033.stdout:3/534: fsync d8/dd/d3d/d63/d7c/d3c/f7d 0 2024-04-25T01:35:52.569 INFO:tasks.workunit.client.0.smithi033.stdout:8/544: creat d6/db/d80/d89/fca x:0 0 0 2024-04-25T01:35:52.571 INFO:tasks.workunit.client.0.smithi033.stdout:6/490: truncate d4/d1d/d3e/d70/f72 568248 0 2024-04-25T01:35:52.572 INFO:tasks.workunit.client.0.smithi033.stdout:2/600: fsync da/d2c/db8/fc5 0 2024-04-25T01:35:52.573 INFO:tasks.workunit.client.0.smithi033.stdout:6/491: write d4/d24/d7c/f80 [4390472,87683] 0 2024-04-25T01:35:52.573 INFO:tasks.workunit.client.0.smithi033.stdout:8/545: read d6/d45/d51/d5d/f6a [785515,61631] 0 2024-04-25T01:35:52.575 INFO:tasks.workunit.client.0.smithi033.stdout:3/535: rmdir d8/dd/d3d/d63/d7c 39 2024-04-25T01:35:52.575 INFO:tasks.workunit.client.0.smithi033.stdout:8/546: truncate d6/d45/f8b 982871 0 2024-04-25T01:35:52.576 INFO:tasks.workunit.client.0.smithi033.stdout:8/547: chown d6/d45/d52/d2e/d87/ca2 2244 1 2024-04-25T01:35:52.579 INFO:tasks.workunit.client.0.smithi033.stdout:9/432: getdents d0/d2/dd/df/d1b 0 2024-04-25T01:35:52.582 INFO:tasks.workunit.client.0.smithi033.stdout:8/548: mknod d6/d45/d52/d2e/ccb 0 2024-04-25T01:35:52.584 INFO:tasks.workunit.client.0.smithi033.stdout:6/492: symlink d4/d24/lb3 0 2024-04-25T01:35:52.586 INFO:tasks.workunit.client.0.smithi033.stdout:6/493: write d4/d1d/d3e/d41/d54/f57 [5127463,52185] 0 2024-04-25T01:35:52.588 INFO:tasks.workunit.client.0.smithi033.stdout:3/536: unlink d8/dd/d3d/d63/d7c/d3c/d86/d92/cb1 0 2024-04-25T01:35:52.590 INFO:tasks.workunit.client.0.smithi033.stdout:3/537: dread - d8/dd/d3d/d63/d7c/d3c/fa3 zero size 2024-04-25T01:35:52.595 INFO:tasks.workunit.client.0.smithi033.stdout:6/494: dwrite d4/d1d/d3e/d41/d54/f57 [4194304,4194304] 0 2024-04-25T01:35:52.595 INFO:tasks.workunit.client.0.smithi033.stdout:6/495: truncate d4/d15/f5f 1348197 0 2024-04-25T01:35:52.596 INFO:tasks.workunit.client.0.smithi033.stdout:2/601: rename da/c1a to da/cce 0 2024-04-25T01:35:52.597 INFO:tasks.workunit.client.0.smithi033.stdout:9/433: link d0/d2/dd/d20/d4e/c72 d0/d2/ca0 0 2024-04-25T01:35:52.597 INFO:tasks.workunit.client.0.smithi033.stdout:9/434: fdatasync d0/f23 0 2024-04-25T01:35:52.598 INFO:tasks.workunit.client.0.smithi033.stdout:9/435: fsync d0/d2/d43/d73/f84 0 2024-04-25T01:35:52.599 INFO:tasks.workunit.client.0.smithi033.stdout:9/436: stat d0/d2/dd/df/d1b 0 2024-04-25T01:35:52.601 INFO:tasks.workunit.client.0.smithi033.stdout:8/549: write d6/d45/d52/f2b [1855318,129637] 0 2024-04-25T01:35:52.601 INFO:tasks.workunit.client.0.smithi033.stdout:2/602: dread - da/f20 zero size 2024-04-25T01:35:52.607 INFO:tasks.workunit.client.0.smithi033.stdout:9/437: dread d0/d2/dd/d20/d4e/f4f [0,4194304] 0 2024-04-25T01:35:52.608 INFO:tasks.workunit.client.0.smithi033.stdout:6/496: dwrite d4/d1d/d3e/f62 [0,4194304] 0 2024-04-25T01:35:52.609 INFO:tasks.workunit.client.0.smithi033.stdout:9/438: fdatasync d0/d2/d43/d73/f9f 0 2024-04-25T01:35:52.611 INFO:tasks.workunit.client.0.smithi033.stdout:8/550: mkdir d6/d45/d52/dcc 0 2024-04-25T01:35:52.614 INFO:tasks.workunit.client.0.smithi033.stdout:3/538: truncate d8/dd/f8f 1178867 0 2024-04-25T01:35:52.615 INFO:tasks.workunit.client.0.smithi033.stdout:3/539: fsync d8/dd/d3d/d4f/faf 0 2024-04-25T01:35:52.616 INFO:tasks.workunit.client.0.smithi033.stdout:9/439: mkdir d0/d11/da1 0 2024-04-25T01:35:52.619 INFO:tasks.workunit.client.0.smithi033.stdout:8/551: mknod d6/db/d80/db4/ccd 0 2024-04-25T01:35:52.620 INFO:tasks.workunit.client.0.smithi033.stdout:6/497: getdents d4/d15 0 2024-04-25T01:35:52.622 INFO:tasks.workunit.client.0.smithi033.stdout:3/540: rename d8/dd/d3d/d63/d7c/d3c/d86/d92/fa8 to d8/dd/d3d/d63/d7c/d9d/fb6 0 2024-04-25T01:35:52.623 INFO:tasks.workunit.client.0.smithi033.stdout:6/498: chown d4/d1d/f69 1518 1 2024-04-25T01:35:52.659 INFO:tasks.workunit.client.0.smithi033.stdout:5/556: sync 2024-04-25T01:35:52.659 INFO:tasks.workunit.client.0.smithi033.stdout:2/603: dwrite da/fd [0,4194304] 0 2024-04-25T01:35:52.659 INFO:tasks.workunit.client.0.smithi033.stdout:8/552: creat d6/d45/d51/d5d/d9d/fce x:0 0 0 2024-04-25T01:35:52.660 INFO:tasks.workunit.client.0.smithi033.stdout:9/440: creat d0/fa2 x:0 0 0 2024-04-25T01:35:52.660 INFO:tasks.workunit.client.0.smithi033.stdout:6/499: dread - d4/d1d/d3e/d41/d93/fad zero size 2024-04-25T01:35:52.662 INFO:tasks.workunit.client.0.smithi033.stdout:6/500: chown d4/d1d/d3e/l65 29 1 2024-04-25T01:35:52.663 INFO:tasks.workunit.client.0.smithi033.stdout:3/541: write d8/dd/d3d/d63/d7c/d9d/fb6 [717222,105276] 0 2024-04-25T01:35:52.664 INFO:tasks.workunit.client.0.smithi033.stdout:2/604: fdatasync da/f85 0 2024-04-25T01:35:52.665 INFO:tasks.workunit.client.0.smithi033.stdout:3/542: chown d8/dd/d3d/d4f/faf 9398605 1 2024-04-25T01:35:52.665 INFO:tasks.workunit.client.0.smithi033.stdout:2/605: write da/d27/d31/d40/d8a/dbb/f68 [153126,38992] 0 2024-04-25T01:35:52.666 INFO:tasks.workunit.client.0.smithi033.stdout:9/441: mknod d0/d2/dd/df/d1b/d3d/d6e/ca3 0 2024-04-25T01:35:52.668 INFO:tasks.workunit.client.0.smithi033.stdout:6/501: symlink d4/d24/d7c/lb4 0 2024-04-25T01:35:52.669 INFO:tasks.workunit.client.0.smithi033.stdout:5/557: symlink d1c/d2a/d50/db1/lbd 0 2024-04-25T01:35:52.671 INFO:tasks.workunit.client.0.smithi033.stdout:2/606: mknod da/d5b/d72/ccf 0 2024-04-25T01:35:52.676 INFO:tasks.workunit.client.0.smithi033.stdout:9/442: link d0/d2/f8a d0/d2/dd/d20/fa4 0 2024-04-25T01:35:52.677 INFO:tasks.workunit.client.0.smithi033.stdout:9/443: write d0/d2/dd/fe [4098899,108970] 0 2024-04-25T01:35:52.677 INFO:tasks.workunit.client.0.smithi033.stdout:8/553: link d6/d45/d52/d2e/d95/lad d6/d45/d52/lcf 0 2024-04-25T01:35:52.680 INFO:tasks.workunit.client.0.smithi033.stdout:6/502: creat d4/d24/d36/fb5 x:0 0 0 2024-04-25T01:35:52.683 INFO:tasks.workunit.client.0.smithi033.stdout:5/558: rename d1c/d2a/d3d/c6e to d1c/cbe 0 2024-04-25T01:35:52.684 INFO:tasks.workunit.client.0.smithi033.stdout:8/554: mknod d6/d45/d51/cd0 0 2024-04-25T01:35:52.686 INFO:tasks.workunit.client.0.smithi033.stdout:9/444: symlink d0/d11/da1/la5 0 2024-04-25T01:35:52.686 INFO:tasks.workunit.client.0.smithi033.stdout:8/555: dread - d6/d45/d51/d55/fa9 zero size 2024-04-25T01:35:52.687 INFO:tasks.workunit.client.0.smithi033.stdout:6/503: mknod d4/d24/cb6 0 2024-04-25T01:35:52.689 INFO:tasks.workunit.client.0.smithi033.stdout:3/543: dwrite d8/dd/d1d/f2b [0,4194304] 0 2024-04-25T01:35:52.689 INFO:tasks.workunit.client.0.smithi033.stdout:8/556: chown d6/d45/d51/d5d/db9/fc4 2736 1 2024-04-25T01:35:52.694 INFO:tasks.workunit.client.0.smithi033.stdout:6/504: creat d4/d1d/d3e/d41/d5c/fb7 x:0 0 0 2024-04-25T01:35:52.696 INFO:tasks.workunit.client.0.smithi033.stdout:3/544: unlink d8/dd/d3d/d63/d7c/d4d/c87 0 2024-04-25T01:35:52.696 INFO:tasks.workunit.client.0.smithi033.stdout:6/505: write d4/d1d/d3e/d41/d54/f57 [8172677,36042] 0 2024-04-25T01:35:52.699 INFO:tasks.workunit.client.0.smithi033.stdout:8/557: dwrite d6/db/d80/fb7 [0,4194304] 0 2024-04-25T01:35:52.701 INFO:tasks.workunit.client.0.smithi033.stdout:6/506: rmdir d4/d24/d63 39 2024-04-25T01:35:52.702 INFO:tasks.workunit.client.0.smithi033.stdout:6/507: write d4/d2a/f3f [286940,40491] 0 2024-04-25T01:35:52.707 INFO:tasks.workunit.client.0.smithi033.stdout:9/445: write d0/d11/f27 [4824940,94893] 0 2024-04-25T01:35:52.709 INFO:tasks.workunit.client.0.smithi033.stdout:3/545: mknod d8/dd/d11/cb7 0 2024-04-25T01:35:52.709 INFO:tasks.workunit.client.0.smithi033.stdout:6/508: dread d4/d24/d63/f9a [0,4194304] 0 2024-04-25T01:35:52.711 INFO:tasks.workunit.client.0.smithi033.stdout:9/446: fdatasync d0/d2/dd/df/d1b/f2e 0 2024-04-25T01:35:52.714 INFO:tasks.workunit.client.0.smithi033.stdout:6/509: symlink d4/lb8 0 2024-04-25T01:35:52.715 INFO:tasks.workunit.client.0.smithi033.stdout:3/546: mknod d8/dd/d3d/d63/cb8 0 2024-04-25T01:35:52.717 INFO:tasks.workunit.client.0.smithi033.stdout:9/447: creat d0/d2/dd/df/d1b/d3d/d7e/d58/fa6 x:0 0 0 2024-04-25T01:35:52.717 INFO:tasks.workunit.client.0.smithi033.stdout:8/558: dwrite d6/d45/d51/d5d/f6a [4194304,4194304] 0 2024-04-25T01:35:52.722 INFO:tasks.workunit.client.0.smithi033.stdout:8/559: mkdir d6/db/d80/dd1 0 2024-04-25T01:35:52.723 INFO:tasks.workunit.client.0.smithi033.stdout:8/560: chown d6/db/f61 0 1 2024-04-25T01:35:52.725 INFO:tasks.workunit.client.0.smithi033.stdout:9/448: mkdir d0/d2/dd/da7 0 2024-04-25T01:35:52.728 INFO:tasks.workunit.client.0.smithi033.stdout:8/561: symlink d6/d45/d52/dcc/ld2 0 2024-04-25T01:35:52.729 INFO:tasks.workunit.client.0.smithi033.stdout:8/562: fdatasync d6/d45/f6d 0 2024-04-25T01:35:52.731 INFO:tasks.workunit.client.0.smithi033.stdout:9/449: creat d0/d2/d43/d4b/fa8 x:0 0 0 2024-04-25T01:35:52.732 INFO:tasks.workunit.client.0.smithi033.stdout:8/563: mkdir d6/dd3 0 2024-04-25T01:35:52.734 INFO:tasks.workunit.client.0.smithi033.stdout:9/450: symlink d0/d2/dd/d20/d4e/la9 0 2024-04-25T01:35:52.736 INFO:tasks.workunit.client.0.smithi033.stdout:6/510: dwrite d4/d1d/d3e/d70/f72 [0,4194304] 0 2024-04-25T01:35:52.736 INFO:tasks.workunit.client.0.smithi033.stdout:8/564: mkdir d6/d45/d52/d2e/dd4 0 2024-04-25T01:35:52.738 INFO:tasks.workunit.client.0.smithi033.stdout:9/451: unlink d0/d2/d43/d4b/c62 0 2024-04-25T01:35:52.741 INFO:tasks.workunit.client.0.smithi033.stdout:8/565: creat d6/d45/d52/d2e/d87/fd5 x:0 0 0 2024-04-25T01:35:52.745 INFO:tasks.workunit.client.0.smithi033.stdout:9/452: dread d0/d2/dd/d20/f7d [0,4194304] 0 2024-04-25T01:35:52.745 INFO:tasks.workunit.client.0.smithi033.stdout:8/566: creat d6/db/d80/db4/fd6 x:0 0 0 2024-04-25T01:35:52.747 INFO:tasks.workunit.client.0.smithi033.stdout:6/511: creat d4/fb9 x:0 0 0 2024-04-25T01:35:52.749 INFO:tasks.workunit.client.0.smithi033.stdout:9/453: rmdir d0/d2/dd/df/d1b/d3d 39 2024-04-25T01:35:52.750 INFO:tasks.workunit.client.0.smithi033.stdout:6/512: creat d4/d24/d7c/fba x:0 0 0 2024-04-25T01:35:52.753 INFO:tasks.workunit.client.0.smithi033.stdout:6/513: rename d4/d24/d63/f77 to d4/d24/d63/fbb 0 2024-04-25T01:35:52.755 INFO:tasks.workunit.client.0.smithi033.stdout:9/454: creat d0/d2/dd/df/d1b/d3d/faa x:0 0 0 2024-04-25T01:35:52.756 INFO:tasks.workunit.client.0.smithi033.stdout:6/514: fdatasync d4/d24/d63/fbb 0 2024-04-25T01:35:52.758 INFO:tasks.workunit.client.0.smithi033.stdout:8/567: truncate d6/d45/d52/dae/fb3 132659 0 2024-04-25T01:35:52.760 INFO:tasks.workunit.client.0.smithi033.stdout:6/515: mknod d4/d24/d36/d42/d9d/cbc 0 2024-04-25T01:35:52.760 INFO:tasks.workunit.client.0.smithi033.stdout:6/516: dread - d4/d1d/d3e/d41/d54/f74 zero size 2024-04-25T01:35:52.761 INFO:tasks.workunit.client.0.smithi033.stdout:9/455: truncate d0/d2/dd/df/f3a 1034538 0 2024-04-25T01:35:52.763 INFO:tasks.workunit.client.0.smithi033.stdout:9/456: mkdir d0/d2/d43/d73/dab 0 2024-04-25T01:35:52.770 INFO:tasks.workunit.client.0.smithi033.stdout:6/517: getdents d4/d1d/d3e/d41/d5c 0 2024-04-25T01:35:52.770 INFO:tasks.workunit.client.0.smithi033.stdout:9/457: dwrite d0/d2/dd/d20/f77 [0,4194304] 0 2024-04-25T01:35:52.773 INFO:tasks.workunit.client.0.smithi033.stdout:9/458: mkdir d0/d2/dac 0 2024-04-25T01:35:52.773 INFO:tasks.workunit.client.0.smithi033.stdout:8/568: write d6/d45/d52/d2e/f3a [2416969,116367] 0 2024-04-25T01:35:52.780 INFO:tasks.workunit.client.0.smithi033.stdout:6/518: dwrite d4/d1d/d3e/d41/d54/f7b [0,4194304] 0 2024-04-25T01:35:52.781 INFO:tasks.workunit.client.0.smithi033.stdout:9/459: rename d0/d2/dac to d0/d2/d43/d73/dab/dad 0 2024-04-25T01:35:52.785 INFO:tasks.workunit.client.0.smithi033.stdout:8/569: dwrite d6/db/d80/d89/fab [0,4194304] 0 2024-04-25T01:35:52.787 INFO:tasks.workunit.client.0.smithi033.stdout:6/519: mkdir d4/d24/d36/dbd 0 2024-04-25T01:35:52.792 INFO:tasks.workunit.client.0.smithi033.stdout:8/570: creat d6/d45/d51/d55/daa/fd7 x:0 0 0 2024-04-25T01:35:52.794 INFO:tasks.workunit.client.0.smithi033.stdout:9/460: write d0/d2/dd/df/d1b/d3d/f70 [2825688,112139] 0 2024-04-25T01:35:52.798 INFO:tasks.workunit.client.0.smithi033.stdout:6/520: dwrite d4/d24/d7c/f80 [0,4194304] 0 2024-04-25T01:35:52.802 INFO:tasks.workunit.client.0.smithi033.stdout:6/521: dwrite d4/d1d/d3e/d70/f72 [0,4194304] 0 2024-04-25T01:35:52.804 INFO:tasks.workunit.client.0.smithi033.stdout:6/522: write d4/d24/d63/fa2 [348094,24053] 0 2024-04-25T01:35:52.804 INFO:tasks.workunit.client.0.smithi033.stdout:6/523: write d4/d1d/f1f [1816068,47446] 0 2024-04-25T01:35:52.808 INFO:tasks.workunit.client.0.smithi033.stdout:6/524: rename d4/d15/l7e to d4/d1d/d3e/d41/d54/lbe 0 2024-04-25T01:35:52.810 INFO:tasks.workunit.client.0.smithi033.stdout:6/525: mkdir d4/d1d/dbf 0 2024-04-25T01:35:52.811 INFO:tasks.workunit.client.0.smithi033.stdout:6/526: dread - d4/d1d/d3e/d41/d93/fab zero size 2024-04-25T01:35:52.811 INFO:tasks.workunit.client.0.smithi033.stdout:6/527: readlink d4/d24/l73 0 2024-04-25T01:35:52.813 INFO:tasks.workunit.client.0.smithi033.stdout:6/528: write d4/d24/d29/f8c [1658408,74143] 0 2024-04-25T01:35:52.820 INFO:tasks.workunit.client.0.smithi033.stdout:6/529: dwrite d4/d24/d36/d42/f83 [0,4194304] 0 2024-04-25T01:35:52.827 INFO:tasks.workunit.client.0.smithi033.stdout:8/571: fsync d6/f3e 0 2024-04-25T01:35:52.831 INFO:tasks.workunit.client.0.smithi033.stdout:8/572: mknod d6/d45/d51/d55/d84/dc8/cd8 0 2024-04-25T01:35:52.832 INFO:tasks.workunit.client.0.smithi033.stdout:1/348: sync 2024-04-25T01:35:52.832 INFO:tasks.workunit.client.0.smithi033.stdout:4/572: sync 2024-04-25T01:35:52.833 INFO:tasks.workunit.client.0.smithi033.stdout:8/573: stat d6/d45 0 2024-04-25T01:35:52.834 INFO:tasks.workunit.client.0.smithi033.stdout:9/461: dwrite d0/d2/dd/df/d1b/f4d [0,4194304] 0 2024-04-25T01:35:52.838 INFO:tasks.workunit.client.0.smithi033.stdout:6/530: dwrite d4/f6 [4194304,4194304] 0 2024-04-25T01:35:52.838 INFO:tasks.workunit.client.0.smithi033.stdout:1/349: write d2/d11/f1a [966101,94568] 0 2024-04-25T01:35:52.854 INFO:tasks.workunit.client.0.smithi033.stdout:1/350: fdatasync d2/d5/d1b/d46/d5f/d63/d64/f6a 0 2024-04-25T01:35:52.855 INFO:tasks.workunit.client.0.smithi033.stdout:8/574: creat d6/d45/d52/fd9 x:0 0 0 2024-04-25T01:35:52.856 INFO:tasks.workunit.client.0.smithi033.stdout:9/462: truncate d0/d2/dd/d20/d4e/f76 5072340 0 2024-04-25T01:35:52.857 INFO:tasks.workunit.client.0.smithi033.stdout:4/573: symlink d6/d14/d1d/d80/lad 0 2024-04-25T01:35:52.860 INFO:tasks.workunit.client.0.smithi033.stdout:6/531: dwrite d4/f1b [0,4194304] 0 2024-04-25T01:35:52.861 INFO:tasks.workunit.client.0.smithi033.stdout:9/463: creat d0/d2/dd/df/d1b/d3d/d42/fae x:0 0 0 2024-04-25T01:35:52.863 INFO:tasks.workunit.client.0.smithi033.stdout:8/575: rename d6/d45/d51/cd0 to d6/db/cda 0 2024-04-25T01:35:52.867 INFO:tasks.workunit.client.0.smithi033.stdout:1/351: dwrite d2/d5/d1b/f54 [0,4194304] 0 2024-04-25T01:35:52.868 INFO:tasks.workunit.client.0.smithi033.stdout:4/574: rename d6/f1f to d6/d12/d84/fae 0 2024-04-25T01:35:52.871 INFO:tasks.workunit.client.0.smithi033.stdout:8/576: symlink d6/db/d80/db4/ldb 0 2024-04-25T01:35:52.873 INFO:tasks.workunit.client.0.smithi033.stdout:9/464: mknod d0/d2/dd/df/caf 0 2024-04-25T01:35:52.878 INFO:tasks.workunit.client.0.smithi033.stdout:8/577: creat d6/d45/d52/dcc/fdc x:0 0 0 2024-04-25T01:35:52.882 INFO:tasks.workunit.client.0.smithi033.stdout:4/575: creat d6/d14/d35/d44/d45/d46/d57/d70/d87/faf x:0 0 0 2024-04-25T01:35:52.884 INFO:tasks.workunit.client.0.smithi033.stdout:1/352: getdents d2/d2e 0 2024-04-25T01:35:52.886 INFO:tasks.workunit.client.0.smithi033.stdout:9/465: getdents d0/d2/dd/d20 0 2024-04-25T01:35:52.888 INFO:tasks.workunit.client.0.smithi033.stdout:8/578: getdents d6/d45/d52/d2e 0 2024-04-25T01:35:52.888 INFO:tasks.workunit.client.0.smithi033.stdout:6/532: dread d4/f2d [0,4194304] 0 2024-04-25T01:35:52.889 INFO:tasks.workunit.client.0.smithi033.stdout:1/353: rmdir d2/d11 39 2024-04-25T01:35:52.892 INFO:tasks.workunit.client.0.smithi033.stdout:6/533: unlink d4/d15/f5f 0 2024-04-25T01:35:52.893 INFO:tasks.workunit.client.0.smithi033.stdout:9/466: dwrite d0/f9 [0,4194304] 0 2024-04-25T01:35:52.894 INFO:tasks.workunit.client.0.smithi033.stdout:6/534: write d4/d24/d63/f96 [527864,126807] 0 2024-04-25T01:35:52.894 INFO:tasks.workunit.client.0.smithi033.stdout:1/354: rename d2/d5/f18 to d2/d5/d1b/d46/d5f/d63/d64/f6b 0 2024-04-25T01:35:52.896 INFO:tasks.workunit.client.0.smithi033.stdout:6/535: fdatasync d4/d1d/d3e/d41/d5c/d7f/f81 0 2024-04-25T01:35:52.898 INFO:tasks.workunit.client.0.smithi033.stdout:9/467: symlink d0/d2/d43/lb0 0 2024-04-25T01:35:52.900 INFO:tasks.workunit.client.0.smithi033.stdout:1/355: mkdir d2/d5/d6c 0 2024-04-25T01:35:52.903 INFO:tasks.workunit.client.0.smithi033.stdout:6/536: unlink d4/f1b 0 2024-04-25T01:35:52.905 INFO:tasks.workunit.client.0.smithi033.stdout:9/468: creat d0/d2/d43/d4b/d55/fb1 x:0 0 0 2024-04-25T01:35:52.906 INFO:tasks.workunit.client.0.smithi033.stdout:8/579: write d6/d45/d52/f27 [306684,42395] 0 2024-04-25T01:35:52.906 INFO:tasks.workunit.client.0.smithi033.stdout:1/356: rmdir d2/d5/d1b/d46 39 2024-04-25T01:35:52.906 INFO:tasks.workunit.client.0.smithi033.stdout:6/537: dread - d4/d1d/d3e/d41/d93/fab zero size 2024-04-25T01:35:52.910 INFO:tasks.workunit.client.0.smithi033.stdout:4/576: dwrite d6/f16 [0,4194304] 0 2024-04-25T01:35:52.915 INFO:tasks.workunit.client.0.smithi033.stdout:1/357: rmdir d2 39 2024-04-25T01:35:52.916 INFO:tasks.workunit.client.0.smithi033.stdout:4/577: mkdir d6/d14/d35/d44/d45/d46/db0 0 2024-04-25T01:35:52.916 INFO:tasks.workunit.client.0.smithi033.stdout:6/538: dwrite d4/d2a/f34 [0,4194304] 0 2024-04-25T01:35:52.921 INFO:tasks.workunit.client.0.smithi033.stdout:8/580: getdents d6/d45/d52/d2e/d87 0 2024-04-25T01:35:52.925 INFO:tasks.workunit.client.0.smithi033.stdout:8/581: mkdir d6/d45/d51/d55/d84/ddd 0 2024-04-25T01:35:52.928 INFO:tasks.workunit.client.0.smithi033.stdout:4/578: creat d6/d14/fb1 x:0 0 0 2024-04-25T01:35:52.930 INFO:tasks.workunit.client.0.smithi033.stdout:6/539: link d4/d24/d63/fa2 d4/d24/d36/fc0 0 2024-04-25T01:35:52.931 INFO:tasks.workunit.client.0.smithi033.stdout:8/582: mknod d6/d45/d52/d2e/d95/cde 0 2024-04-25T01:35:52.933 INFO:tasks.workunit.client.0.smithi033.stdout:6/540: symlink d4/d24/d36/d42/d97/lc1 0 2024-04-25T01:35:52.935 INFO:tasks.workunit.client.0.smithi033.stdout:4/579: symlink d6/d14/lb2 0 2024-04-25T01:35:52.937 INFO:tasks.workunit.client.0.smithi033.stdout:6/541: mknod d4/d24/d29/cc2 0 2024-04-25T01:35:52.939 INFO:tasks.workunit.client.0.smithi033.stdout:6/542: write d4/d1d/d3e/d41/d5c/d7f/da4/faf [828731,61669] 0 2024-04-25T01:35:52.940 INFO:tasks.workunit.client.0.smithi033.stdout:4/580: rename d6/d14/d35/d3a/d4c/f8b to d6/d14/d1d/d5f/fb3 0 2024-04-25T01:35:52.942 INFO:tasks.workunit.client.0.smithi033.stdout:9/469: dread d0/d2/dd/df/f8c [0,4194304] 0 2024-04-25T01:35:52.947 INFO:tasks.workunit.client.0.smithi033.stdout:1/358: dread d2/d11/f1a [4194304,4194304] 0 2024-04-25T01:35:52.948 INFO:tasks.workunit.client.0.smithi033.stdout:6/543: getdents d4/d15 0 2024-04-25T01:35:52.952 INFO:tasks.workunit.client.0.smithi033.stdout:8/583: dwrite d6/d45/d51/d5d/d6e/f85 [0,4194304] 0 2024-04-25T01:35:52.953 INFO:tasks.workunit.client.0.smithi033.stdout:4/581: truncate d6/d14/f31 2389461 0 2024-04-25T01:35:52.954 INFO:tasks.workunit.client.0.smithi033.stdout:9/470: truncate d0/f29 1075834 0 2024-04-25T01:35:52.956 INFO:tasks.workunit.client.0.smithi033.stdout:9/471: chown d0/d2/d43/d4b/fa8 6319995 1 2024-04-25T01:35:52.957 INFO:tasks.workunit.client.0.smithi033.stdout:6/544: creat d4/d24/d7c/fc3 x:0 0 0 2024-04-25T01:35:52.959 INFO:tasks.workunit.client.0.smithi033.stdout:8/584: rmdir d6/d45/d51/d55/d84 39 2024-04-25T01:35:52.959 INFO:tasks.workunit.client.0.smithi033.stdout:4/582: write d6/d14/d1d/d3b/f52 [1804989,26702] 0 2024-04-25T01:35:52.963 INFO:tasks.workunit.client.0.smithi033.stdout:6/545: unlink d4/d24/l73 0 2024-04-25T01:35:52.965 INFO:tasks.workunit.client.0.smithi033.stdout:9/472: mknod d0/d2/dd/df/d1b/d3d/d7e/cb2 0 2024-04-25T01:35:52.966 INFO:tasks.workunit.client.0.smithi033.stdout:8/585: dwrite d6/d45/d52/d2e/f3a [0,4194304] 0 2024-04-25T01:35:52.967 INFO:tasks.workunit.client.0.smithi033.stdout:9/473: stat d0/d2/dd/df/d1b/d3d/d7e/d58 0 2024-04-25T01:35:52.968 INFO:tasks.workunit.client.0.smithi033.stdout:8/586: stat d6/d45/d52/d2e/f4a 0 2024-04-25T01:35:52.971 INFO:tasks.workunit.client.0.smithi033.stdout:9/474: symlink d0/d2/lb3 0 2024-04-25T01:35:52.973 INFO:tasks.workunit.client.0.smithi033.stdout:6/546: getdents d4/d1d/d3e/d41/d5c/dae 0 2024-04-25T01:35:52.974 INFO:tasks.workunit.client.0.smithi033.stdout:8/587: write d6/d45/d52/d2e/f30 [3351783,38624] 0 2024-04-25T01:35:52.977 INFO:tasks.workunit.client.0.smithi033.stdout:9/475: mknod d0/d2/dd/df/cb4 0 2024-04-25T01:35:52.977 INFO:tasks.workunit.client.0.smithi033.stdout:1/359: dwrite d2/d11/f30 [0,4194304] 0 2024-04-25T01:35:52.979 INFO:tasks.workunit.client.0.smithi033.stdout:9/476: write d0/d2/dd/df/f82 [951656,80509] 0 2024-04-25T01:35:52.979 INFO:tasks.workunit.client.0.smithi033.stdout:6/547: creat d4/d1d/d3e/d41/d5c/d7f/da4/fc4 x:0 0 0 2024-04-25T01:35:52.981 INFO:tasks.workunit.client.0.smithi033.stdout:9/477: chown d0/d2/dd/f56 69881188 1 2024-04-25T01:35:52.983 INFO:tasks.workunit.client.0.smithi033.stdout:6/548: dread d4/d1d/d3e/d70/f95 [0,4194304] 0 2024-04-25T01:35:52.983 INFO:tasks.workunit.client.0.smithi033.stdout:4/583: write d6/d14/d1d/f6f [868202,100245] 0 2024-04-25T01:35:52.984 INFO:tasks.workunit.client.0.smithi033.stdout:4/584: write d6/d14/d48/f99 [977158,12088] 0 2024-04-25T01:35:52.987 INFO:tasks.workunit.client.0.smithi033.stdout:1/360: creat d2/d11/f6d x:0 0 0 2024-04-25T01:35:52.987 INFO:tasks.workunit.client.0.smithi033.stdout:6/549: write d4/f12 [3459965,84532] 0 2024-04-25T01:35:52.990 INFO:tasks.workunit.client.0.smithi033.stdout:9/478: truncate d0/d11/f18 3615045 0 2024-04-25T01:35:52.991 INFO:tasks.workunit.client.0.smithi033.stdout:4/585: write d6/d14/d6d/f96 [969033,62617] 0 2024-04-25T01:35:52.992 INFO:tasks.workunit.client.0.smithi033.stdout:1/361: fdatasync d2/d11/f30 0 2024-04-25T01:35:52.993 INFO:tasks.workunit.client.0.smithi033.stdout:6/550: unlink d4/d1d/d3e/d41/d5c/d7f/da4/fc4 0 2024-04-25T01:35:52.993 INFO:tasks.workunit.client.0.smithi033.stdout:6/551: stat d4/d24/f99 0 2024-04-25T01:35:52.994 INFO:tasks.workunit.client.0.smithi033.stdout:6/552: dread - d4/d2a/fa9 zero size 2024-04-25T01:35:53.000 INFO:tasks.workunit.client.0.smithi033.stdout:9/479: truncate d0/f23 424348 0 2024-04-25T01:35:53.000 INFO:tasks.workunit.client.0.smithi033.stdout:6/553: dwrite d4/d24/f64 [0,4194304] 0 2024-04-25T01:35:53.007 INFO:tasks.workunit.client.0.smithi033.stdout:1/362: unlink d2/d11/c19 0 2024-04-25T01:35:53.007 INFO:tasks.workunit.client.0.smithi033.stdout:4/586: creat d6/d14/d35/fb4 x:0 0 0 2024-04-25T01:35:53.007 INFO:tasks.workunit.client.0.smithi033.stdout:4/587: unlink d6/d14/d6d/laa 0 2024-04-25T01:35:53.009 INFO:tasks.workunit.client.0.smithi033.stdout:9/480: link d0/d2/d43/d4b/f5b d0/d2/dd/da7/fb5 0 2024-04-25T01:35:53.010 INFO:tasks.workunit.client.0.smithi033.stdout:9/481: chown d0/d2/d43/d4b/d55/f8d 2595 1 2024-04-25T01:35:53.011 INFO:tasks.workunit.client.0.smithi033.stdout:9/482: stat d0/d2/dd/df/f5c 0 2024-04-25T01:35:53.012 INFO:tasks.workunit.client.0.smithi033.stdout:4/588: creat d6/d14/d1d/d5f/fb5 x:0 0 0 2024-04-25T01:35:53.013 INFO:tasks.workunit.client.0.smithi033.stdout:6/554: write d4/d15/f48 [3999558,29224] 0 2024-04-25T01:35:53.016 INFO:tasks.workunit.client.0.smithi033.stdout:6/555: write d4/d24/d7c/fba [213298,23292] 0 2024-04-25T01:35:53.017 INFO:tasks.workunit.client.0.smithi033.stdout:6/556: dread - d4/d2a/fa9 zero size 2024-04-25T01:35:53.019 INFO:tasks.workunit.client.0.smithi033.stdout:4/589: mknod d6/d14/d35/d44/d45/d46/db0/cb6 0 2024-04-25T01:35:53.021 INFO:tasks.workunit.client.0.smithi033.stdout:9/483: dwrite d0/d2/dd/df/d1b/f4d [0,4194304] 0 2024-04-25T01:35:53.023 INFO:tasks.workunit.client.0.smithi033.stdout:4/590: mknod d6/d14/d35/d44/cb7 0 2024-04-25T01:35:53.024 INFO:tasks.workunit.client.0.smithi033.stdout:6/557: getdents d4/d1d/dbf 0 2024-04-25T01:35:53.026 INFO:tasks.workunit.client.0.smithi033.stdout:1/363: dread d2/f62 [0,4194304] 0 2024-04-25T01:35:53.027 INFO:tasks.workunit.client.0.smithi033.stdout:9/484: unlink d0/d2/d43/lb0 0 2024-04-25T01:35:53.028 INFO:tasks.workunit.client.0.smithi033.stdout:6/558: symlink d4/d1d/d3e/d41/d5c/d7f/lc5 0 2024-04-25T01:35:53.029 INFO:tasks.workunit.client.0.smithi033.stdout:4/591: mkdir d6/d14/d35/d44/d45/d46/d57/d70/d87/db8 0 2024-04-25T01:35:53.031 INFO:tasks.workunit.client.0.smithi033.stdout:9/485: truncate d0/d2/d43/d4b/d55/f8d 382196 0 2024-04-25T01:35:53.033 INFO:tasks.workunit.client.0.smithi033.stdout:1/364: dread d2/d11/f30 [0,4194304] 0 2024-04-25T01:35:53.033 INFO:tasks.workunit.client.0.smithi033.stdout:6/559: mknod d4/d24/d36/d42/d9d/cc6 0 2024-04-25T01:35:53.036 INFO:tasks.workunit.client.0.smithi033.stdout:1/365: write d2/d11/f42 [4543959,47275] 0 2024-04-25T01:35:53.037 INFO:tasks.workunit.client.0.smithi033.stdout:9/486: dread d0/d2/d43/f3c [0,4194304] 0 2024-04-25T01:35:53.038 INFO:tasks.workunit.client.0.smithi033.stdout:1/366: read d2/f26 [4133937,64857] 0 2024-04-25T01:35:53.039 INFO:tasks.workunit.client.0.smithi033.stdout:9/487: write d0/d2/dd/d5f/f89 [3663507,97927] 0 2024-04-25T01:35:53.040 INFO:tasks.workunit.client.0.smithi033.stdout:6/560: creat d4/d1d/d3e/fc7 x:0 0 0 2024-04-25T01:35:53.041 INFO:tasks.workunit.client.0.smithi033.stdout:9/488: write d0/d2/dd/d20/d4e/f90 [755795,130322] 0 2024-04-25T01:35:53.042 INFO:tasks.workunit.client.0.smithi033.stdout:6/561: write d4/d24/d7c/f80 [5289598,60164] 0 2024-04-25T01:35:53.044 INFO:tasks.workunit.client.0.smithi033.stdout:1/367: mknod d2/d2e/d49/c6e 0 2024-04-25T01:35:53.050 INFO:tasks.workunit.client.0.smithi033.stdout:6/562: unlink d4/d1d/c87 0 2024-04-25T01:35:53.052 INFO:tasks.workunit.client.0.smithi033.stdout:9/489: link d0/d2/dd/d20/d4e/f94 d0/d2/d43/d73/dab/fb6 0 2024-04-25T01:35:53.053 INFO:tasks.workunit.client.0.smithi033.stdout:4/592: dread d6/d12/d84/fae [0,4194304] 0 2024-04-25T01:35:53.054 INFO:tasks.workunit.client.0.smithi033.stdout:6/563: dread d4/d1d/d3e/d70/f86 [0,4194304] 0 2024-04-25T01:35:53.055 INFO:tasks.workunit.client.0.smithi033.stdout:6/564: readlink d4/d24/lb3 0 2024-04-25T01:35:53.055 INFO:tasks.workunit.client.0.smithi033.stdout:6/565: readlink d4/d1d/d3e/l78 0 2024-04-25T01:35:53.056 INFO:tasks.workunit.client.0.smithi033.stdout:1/368: link d2/f26 d2/d2e/d34/f6f 0 2024-04-25T01:35:53.057 INFO:tasks.workunit.client.0.smithi033.stdout:4/593: creat d6/d14/d1d/fb9 x:0 0 0 2024-04-25T01:35:53.058 INFO:tasks.workunit.client.0.smithi033.stdout:6/566: mknod d4/d24/d36/d42/d97/cc8 0 2024-04-25T01:35:53.059 INFO:tasks.workunit.client.0.smithi033.stdout:6/567: chown d4/d1d/d3e 10685746 1 2024-04-25T01:35:53.060 INFO:tasks.workunit.client.0.smithi033.stdout:6/568: chown d4/d1d/d3e/d41/d54 432386717 1 2024-04-25T01:35:53.060 INFO:tasks.workunit.client.0.smithi033.stdout:4/594: symlink d6/d14/d48/lba 0 2024-04-25T01:35:53.061 INFO:tasks.workunit.client.0.smithi033.stdout:1/369: mknod d2/d5/d1b/d46/d5f/d63/c70 0 2024-04-25T01:35:53.062 INFO:tasks.workunit.client.0.smithi033.stdout:6/569: mknod d4/d24/d29/cc9 0 2024-04-25T01:35:53.064 INFO:tasks.workunit.client.0.smithi033.stdout:4/595: creat d6/d12/d84/fbb x:0 0 0 2024-04-25T01:35:53.066 INFO:tasks.workunit.client.0.smithi033.stdout:4/596: truncate d6/d14/d48/f99 1314298 0 2024-04-25T01:35:53.067 INFO:tasks.workunit.client.0.smithi033.stdout:6/570: creat d4/d24/d36/d42/fca x:0 0 0 2024-04-25T01:35:53.069 INFO:tasks.workunit.client.0.smithi033.stdout:9/490: dread d0/d2/dd/f52 [0,4194304] 0 2024-04-25T01:35:53.069 INFO:tasks.workunit.client.0.smithi033.stdout:1/370: truncate d2/d11/f30 3889618 0 2024-04-25T01:35:53.069 INFO:tasks.workunit.client.0.smithi033.stdout:6/571: write d4/d1d/d3e/d4a/f4f [2323428,71600] 0 2024-04-25T01:35:53.070 INFO:tasks.workunit.client.0.smithi033.stdout:6/572: readlink d4/l56 0 2024-04-25T01:35:53.074 INFO:tasks.workunit.client.0.smithi033.stdout:4/597: dwrite d6/d12/d38/f7e [0,4194304] 0 2024-04-25T01:35:53.079 INFO:tasks.workunit.client.0.smithi033.stdout:9/491: rmdir d0 39 2024-04-25T01:35:53.081 INFO:tasks.workunit.client.0.smithi033.stdout:6/573: mkdir d4/d1d/d3e/d4a/dcb 0 2024-04-25T01:35:53.084 INFO:tasks.workunit.client.0.smithi033.stdout:4/598: dwrite d6/d14/d48/f99 [0,4194304] 0 2024-04-25T01:35:53.084 INFO:tasks.workunit.client.0.smithi033.stdout:1/371: rename d2/d5/d1b/d46/d5f/d63/d64/f6b to d2/d2e/f71 0 2024-04-25T01:35:53.089 INFO:tasks.workunit.client.0.smithi033.stdout:0/471: sync 2024-04-25T01:35:53.089 INFO:tasks.workunit.client.0.smithi033.stdout:6/574: mkdir d4/d1d/d3e/d41/d5c/d7f/da4/dcc 0 2024-04-25T01:35:53.090 INFO:tasks.workunit.client.0.smithi033.stdout:0/472: write d7/f46 [1036540,26700] 0 2024-04-25T01:35:53.100 INFO:tasks.workunit.client.0.smithi033.stdout:1/372: dwrite d2/d5/d1b/f40 [0,4194304] 0 2024-04-25T01:35:53.102 INFO:tasks.workunit.client.0.smithi033.stdout:0/473: dwrite d7/d2d/d43/d44/f63 [0,4194304] 0 2024-04-25T01:35:53.107 INFO:tasks.workunit.client.0.smithi033.stdout:9/492: link d0/c2c d0/d2/d43/d73/dab/cb7 0 2024-04-25T01:35:53.111 INFO:tasks.workunit.client.0.smithi033.stdout:0/474: fdatasync d7/da/df/f72 0 2024-04-25T01:35:53.111 INFO:tasks.workunit.client.0.smithi033.stdout:9/493: symlink d0/d2/dd/df/d1b/d3d/d8f/lb8 0 2024-04-25T01:35:53.113 INFO:tasks.workunit.client.0.smithi033.stdout:1/373: dwrite d2/d5/d1b/d46/f47 [0,4194304] 0 2024-04-25T01:35:53.119 INFO:tasks.workunit.client.0.smithi033.stdout:0/475: creat d7/d2d/d4d/d9c/d61/d70/fa9 x:0 0 0 2024-04-25T01:35:53.121 INFO:tasks.workunit.client.0.smithi033.stdout:1/374: dwrite d2/d2e/f67 [0,4194304] 0 2024-04-25T01:35:53.123 INFO:tasks.workunit.client.0.smithi033.stdout:4/599: dread d6/d14/d1d/d80/f7b [0,4194304] 0 2024-04-25T01:35:53.124 INFO:tasks.workunit.client.0.smithi033.stdout:9/494: rename d0/d2/dd/df/d1b/l21 to d0/lb9 0 2024-04-25T01:35:53.125 INFO:tasks.workunit.client.0.smithi033.stdout:9/495: fsync d0/d2/f6c 0 2024-04-25T01:35:53.126 INFO:tasks.workunit.client.0.smithi033.stdout:4/600: chown d6/d12/d38/c67 3468132 1 2024-04-25T01:35:53.128 INFO:tasks.workunit.client.0.smithi033.stdout:1/375: rename d2/d2e/d34/c56 to d2/d2e/d49/c72 0 2024-04-25T01:35:53.130 INFO:tasks.workunit.client.0.smithi033.stdout:9/496: unlink d0/d2/d43/d73/dab/fb6 0 2024-04-25T01:35:53.130 INFO:tasks.workunit.client.0.smithi033.stdout:9/497: readlink d0/d2/l10 0 2024-04-25T01:35:53.132 INFO:tasks.workunit.client.0.smithi033.stdout:9/498: mkdir d0/d2/dd/dba 0 2024-04-25T01:35:53.133 INFO:tasks.workunit.client.0.smithi033.stdout:9/499: truncate d0/d2/dd/d5f/f89 4233708 0 2024-04-25T01:35:53.136 INFO:tasks.workunit.client.0.smithi033.stdout:9/500: getdents d0/d2 0 2024-04-25T01:35:53.138 INFO:tasks.workunit.client.0.smithi033.stdout:9/501: write d0/d2/dd/d20/f7d [1181098,106431] 0 2024-04-25T01:35:53.169 INFO:tasks.workunit.client.0.smithi033.stdout:6/575: rmdir d4/d1d/d3e 39 2024-04-25T01:35:53.170 INFO:tasks.workunit.client.0.smithi033.stdout:6/576: chown d4/d1d/d3e/d70 22900 1 2024-04-25T01:35:53.174 INFO:tasks.workunit.client.0.smithi033.stdout:6/577: dread d4/d24/d29/f55 [0,4194304] 0 2024-04-25T01:35:53.176 INFO:tasks.workunit.client.0.smithi033.stdout:0/476: dread d7/da/df/d14/f15 [0,4194304] 0 2024-04-25T01:35:53.179 INFO:tasks.workunit.client.0.smithi033.stdout:0/477: symlink d7/da/laa 0 2024-04-25T01:35:53.180 INFO:tasks.workunit.client.0.smithi033.stdout:1/376: fsync d2/d5/d1b/d46/f47 0 2024-04-25T01:35:53.181 INFO:tasks.workunit.client.0.smithi033.stdout:0/478: write d7/d2d/d4d/d9c/d68/f98 [715005,102861] 0 2024-04-25T01:35:53.187 INFO:tasks.workunit.client.0.smithi033.stdout:1/377: dwrite d2/d11/f5e [0,4194304] 0 2024-04-25T01:35:53.188 INFO:tasks.workunit.client.0.smithi033.stdout:1/378: fdatasync d2/d5/d1b/f40 0 2024-04-25T01:35:53.189 INFO:tasks.workunit.client.0.smithi033.stdout:6/578: truncate d4/d1d/d3e/d41/d54/f7b 2920488 0 2024-04-25T01:35:53.194 INFO:tasks.workunit.client.0.smithi033.stdout:6/579: symlink d4/d1d/d3e/d4a/dcb/lcd 0 2024-04-25T01:35:53.196 INFO:tasks.workunit.client.0.smithi033.stdout:6/580: write d4/d1d/d3e/d41/d5c/d7f/da4/faf [1689196,109394] 0 2024-04-25T01:35:53.198 INFO:tasks.workunit.client.0.smithi033.stdout:1/379: truncate d2/f62 2559047 0 2024-04-25T01:35:53.199 INFO:tasks.workunit.client.0.smithi033.stdout:6/581: symlink d4/d1d/d3e/d41/d93/lce 0 2024-04-25T01:35:53.201 INFO:tasks.workunit.client.0.smithi033.stdout:1/380: creat d2/d2e/f73 x:0 0 0 2024-04-25T01:35:53.211 INFO:tasks.workunit.client.0.smithi033.stdout:6/582: truncate d4/d1d/f50 2126086 0 2024-04-25T01:35:53.214 INFO:tasks.workunit.client.0.smithi033.stdout:6/583: truncate d4/d2a/f34 4564633 0 2024-04-25T01:35:53.215 INFO:tasks.workunit.client.0.smithi033.stdout:6/584: chown d4/d15 270627 1 2024-04-25T01:35:53.216 INFO:tasks.workunit.client.0.smithi033.stdout:6/585: read - d4/d1d/d3e/d41/d5c/fb7 zero size 2024-04-25T01:35:53.220 INFO:tasks.workunit.client.0.smithi033.stdout:6/586: unlink d4/d1d/d3e/d4a/dcb/lcd 0 2024-04-25T01:35:53.220 INFO:tasks.workunit.client.0.smithi033.stdout:6/587: write d4/d1d/d3e/d41/d5c/fb7 [872166,99656] 0 2024-04-25T01:35:53.226 INFO:tasks.workunit.client.0.smithi033.stdout:6/588: dwrite d4/d24/d7c/fc3 [0,4194304] 0 2024-04-25T01:35:53.228 INFO:tasks.workunit.client.0.smithi033.stdout:6/589: dread d4/d24/d29/f55 [0,4194304] 0 2024-04-25T01:35:53.230 INFO:tasks.workunit.client.0.smithi033.stdout:6/590: chown d4/d24/d36/d42/fca 0 1 2024-04-25T01:35:53.232 INFO:tasks.workunit.client.0.smithi033.stdout:6/591: unlink d4/d1d/d3e/d41/d54/f74 0 2024-04-25T01:35:53.236 INFO:tasks.workunit.client.0.smithi033.stdout:6/592: dread d4/d24/d36/fc0 [0,4194304] 0 2024-04-25T01:35:53.241 INFO:tasks.workunit.client.0.smithi033.stdout:6/593: dwrite d4/d1d/f1f [0,4194304] 0 2024-04-25T01:35:53.243 INFO:tasks.workunit.client.0.smithi033.stdout:6/594: chown d4/d24/d36/d42/d9d/cbc 138358880 1 2024-04-25T01:35:53.245 INFO:tasks.workunit.client.0.smithi033.stdout:6/595: creat d4/d1d/d3e/d41/d5c/dae/fcf x:0 0 0 2024-04-25T01:35:53.246 INFO:tasks.workunit.client.0.smithi033.stdout:6/596: dread - d4/d24/d36/f6c zero size 2024-04-25T01:35:53.250 INFO:tasks.workunit.client.0.smithi033.stdout:6/597: dread d4/d1d/d3e/f62 [0,4194304] 0 2024-04-25T01:35:53.253 INFO:tasks.workunit.client.0.smithi033.stdout:6/598: mknod d4/d1d/cd0 0 2024-04-25T01:35:53.253 INFO:tasks.workunit.client.0.smithi033.stdout:6/599: dread - d4/d24/d36/d42/d9d/fb0 zero size 2024-04-25T01:35:53.257 INFO:tasks.workunit.client.0.smithi033.stdout:6/600: getdents d4/d1d/d3e 0 2024-04-25T01:35:53.259 INFO:tasks.workunit.client.0.smithi033.stdout:6/601: link d4/d24/d36/f6c d4/d24/d36/d42/d9d/fd1 0 2024-04-25T01:35:53.260 INFO:tasks.workunit.client.0.smithi033.stdout:6/602: rmdir d4/d1d/d3e/d41/d9f 39 2024-04-25T01:35:53.262 INFO:tasks.workunit.client.0.smithi033.stdout:6/603: symlink d4/d24/d36/d42/d97/db2/ld2 0 2024-04-25T01:35:53.264 INFO:tasks.workunit.client.0.smithi033.stdout:5/559: sync 2024-04-25T01:35:53.264 INFO:tasks.workunit.client.0.smithi033.stdout:2/607: sync 2024-04-25T01:35:53.264 INFO:tasks.workunit.client.0.smithi033.stdout:8/588: sync 2024-04-25T01:35:53.264 INFO:tasks.workunit.client.0.smithi033.stdout:7/558: sync 2024-04-25T01:35:53.264 INFO:tasks.workunit.client.0.smithi033.stdout:3/547: sync 2024-04-25T01:35:53.264 INFO:tasks.workunit.client.0.smithi033.stdout:6/604: mknod d4/d24/d36/d42/cd3 0 2024-04-25T01:35:53.266 INFO:tasks.workunit.client.0.smithi033.stdout:6/605: readlink d4/d1d/d3e/d41/d54/l75 0 2024-04-25T01:35:53.267 INFO:tasks.workunit.client.0.smithi033.stdout:8/589: symlink d6/d45/d51/ldf 0 2024-04-25T01:35:53.268 INFO:tasks.workunit.client.0.smithi033.stdout:5/560: rmdir d1c/d2a/d3d/d59 39 2024-04-25T01:35:53.271 INFO:tasks.workunit.client.0.smithi033.stdout:5/561: chown d1c/d2a/d50/d52/da6/d4a/f4e 88277 1 2024-04-25T01:35:53.272 INFO:tasks.workunit.client.0.smithi033.stdout:7/559: rename d4/d23/d3f/d45/d6c/f9e to d4/d23/d3f/d45/d90/fb1 0 2024-04-25T01:35:53.274 INFO:tasks.workunit.client.0.smithi033.stdout:2/608: dwrite da/d50/d6a/f56 [0,4194304] 0 2024-04-25T01:35:53.274 INFO:tasks.workunit.client.0.smithi033.stdout:7/560: readlink d4/d10/l4b 0 2024-04-25T01:35:53.277 INFO:tasks.workunit.client.0.smithi033.stdout:5/562: creat d1c/d2a/d50/d52/da6/d4a/fbf x:0 0 0 2024-04-25T01:35:53.280 INFO:tasks.workunit.client.0.smithi033.stdout:8/590: creat d6/db/fe0 x:0 0 0 2024-04-25T01:35:53.280 INFO:tasks.workunit.client.0.smithi033.stdout:3/548: dread d8/dd/f83 [0,4194304] 0 2024-04-25T01:35:53.280 INFO:tasks.workunit.client.0.smithi033.stdout:5/563: read d1c/f5e [2804851,42816] 0 2024-04-25T01:35:53.281 INFO:tasks.workunit.client.0.smithi033.stdout:6/606: rename d4/d2a/c9b to d4/d24/d36/d42/d9d/cd4 0 2024-04-25T01:35:53.283 INFO:tasks.workunit.client.0.smithi033.stdout:8/591: write d6/d45/d51/d55/daa/fd7 [904532,122513] 0 2024-04-25T01:35:53.286 INFO:tasks.workunit.client.0.smithi033.stdout:7/561: rename d4/d2e/f4c to d4/d2e/d7f/fb2 0 2024-04-25T01:35:53.286 INFO:tasks.workunit.client.0.smithi033.stdout:3/549: fsync d8/dd/d3d/d4f/f69 0 2024-04-25T01:35:53.287 INFO:tasks.workunit.client.0.smithi033.stdout:6/607: creat d4/d15/fd5 x:0 0 0 2024-04-25T01:35:53.384 INFO:tasks.workunit.client.0.smithi033.stdout:5/564: mkdir d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0 0 2024-04-25T01:35:53.386 INFO:tasks.workunit.client.0.smithi033.stdout:5/565: stat d1c/d2a/d50/d52/da6/l79 0 2024-04-25T01:35:53.387 INFO:tasks.workunit.client.0.smithi033.stdout:7/562: rmdir d4/d23/d3f/d45 39 2024-04-25T01:35:53.387 INFO:tasks.workunit.client.0.smithi033.stdout:5/566: stat d1c/d2a/d3d/f95 0 2024-04-25T01:35:53.389 INFO:tasks.workunit.client.0.smithi033.stdout:7/563: chown d4/f7a 3 1 2024-04-25T01:35:53.389 INFO:tasks.workunit.client.0.smithi033.stdout:6/608: dread d4/d1d/d3e/d70/f72 [0,4194304] 0 2024-04-25T01:35:53.390 INFO:tasks.workunit.client.0.smithi033.stdout:5/567: chown d1c/d2a/d50/d52/da6/d3e/d70/d89/la0 10 1 2024-04-25T01:35:53.392 INFO:tasks.workunit.client.0.smithi033.stdout:5/568: chown d1c/d2a/d50/d52/da6/f6d 103021836 1 2024-04-25T01:35:53.393 INFO:tasks.workunit.client.0.smithi033.stdout:3/550: rename d8/dd/d11/f8b to d8/dd/d3d/d63/d7c/d3c/d86/d92/fb9 0 2024-04-25T01:35:53.393 INFO:tasks.workunit.client.0.smithi033.stdout:2/609: write da/f74 [129178,17445] 0 2024-04-25T01:35:53.394 INFO:tasks.workunit.client.0.smithi033.stdout:2/610: stat da/d8d 0 2024-04-25T01:35:53.395 INFO:tasks.workunit.client.0.smithi033.stdout:2/611: write da/d5b/f79 [545461,125454] 0 2024-04-25T01:35:53.399 INFO:tasks.workunit.client.0.smithi033.stdout:8/592: getdents d6/d45/d51/d9e 0 2024-04-25T01:35:53.401 INFO:tasks.workunit.client.0.smithi033.stdout:6/609: rmdir d4 39 2024-04-25T01:35:53.402 INFO:tasks.workunit.client.0.smithi033.stdout:3/551: dwrite d8/dd/d3d/d63/d7c/d3c/d6e/f9f [0,4194304] 0 2024-04-25T01:35:53.408 INFO:tasks.workunit.client.0.smithi033.stdout:8/593: mknod d6/d45/d51/d55/daa/ce1 0 2024-04-25T01:35:53.410 INFO:tasks.workunit.client.0.smithi033.stdout:3/552: dwrite d8/dd/f34 [0,4194304] 0 2024-04-25T01:35:53.415 INFO:tasks.workunit.client.0.smithi033.stdout:2/612: fsync da/d5b/f63 0 2024-04-25T01:35:53.415 INFO:tasks.workunit.client.0.smithi033.stdout:2/613: readlink da/d2c/lb3 0 2024-04-25T01:35:53.415 INFO:tasks.workunit.client.0.smithi033.stdout:8/594: unlink d6/d45/d51/f79 0 2024-04-25T01:35:53.415 INFO:tasks.workunit.client.0.smithi033.stdout:2/614: chown da/d50/d6a/f56 24998844 1 2024-04-25T01:35:53.416 INFO:tasks.workunit.client.0.smithi033.stdout:6/610: mkdir d4/d24/d36/d42/d97/dd6 0 2024-04-25T01:35:53.416 INFO:tasks.workunit.client.0.smithi033.stdout:8/595: chown d6/db/d80/d89/fca 1409700417 1 2024-04-25T01:35:53.422 INFO:tasks.workunit.client.0.smithi033.stdout:5/569: dwrite d1c/d2a/d50/d52/da6/f63 [0,4194304] 0 2024-04-25T01:35:53.423 INFO:tasks.workunit.client.0.smithi033.stdout:2/615: unlink da/f17 0 2024-04-25T01:35:53.424 INFO:tasks.workunit.client.0.smithi033.stdout:6/611: mkdir d4/d24/d36/dd7 0 2024-04-25T01:35:53.426 INFO:tasks.workunit.client.0.smithi033.stdout:3/553: link d8/dd/d11/ca6 d8/dd/d3d/d63/d7c/d3c/d98/cba 0 2024-04-25T01:35:53.432 INFO:tasks.workunit.client.0.smithi033.stdout:3/554: chown d8/dd/d11/l35 68389498 1 2024-04-25T01:35:53.432 INFO:tasks.workunit.client.0.smithi033.stdout:5/570: dwrite d1c/f23 [0,4194304] 0 2024-04-25T01:35:53.432 INFO:tasks.workunit.client.0.smithi033.stdout:6/612: mknod d4/d24/d36/dbd/cd8 0 2024-04-25T01:35:53.432 INFO:tasks.workunit.client.0.smithi033.stdout:2/616: truncate da/f18 389439 0 2024-04-25T01:35:53.433 INFO:tasks.workunit.client.0.smithi033.stdout:5/571: fsync d1c/d2a/d50/d52/da6/d3e/d70/dae/fb5 0 2024-04-25T01:35:53.436 INFO:tasks.workunit.client.0.smithi033.stdout:3/555: creat d8/dd/d3d/d63/d7c/d3c/d6e/fbb x:0 0 0 2024-04-25T01:35:53.436 INFO:tasks.workunit.client.0.smithi033.stdout:8/596: write d6/db/f61 [12932,128467] 0 2024-04-25T01:35:53.440 INFO:tasks.workunit.client.0.smithi033.stdout:6/613: rmdir d4/d24/d36/dd7 0 2024-04-25T01:35:53.443 INFO:tasks.workunit.client.0.smithi033.stdout:8/597: fsync d6/d45/d52/d2e/f4a 0 2024-04-25T01:35:53.445 INFO:tasks.workunit.client.0.smithi033.stdout:6/614: creat d4/d24/d36/dbd/fd9 x:0 0 0 2024-04-25T01:35:53.447 INFO:tasks.workunit.client.0.smithi033.stdout:2/617: getdents da/d27 0 2024-04-25T01:35:53.448 INFO:tasks.workunit.client.0.smithi033.stdout:2/618: dread - da/d27/d31/d40/db1/fcc zero size 2024-04-25T01:35:53.449 INFO:tasks.workunit.client.0.smithi033.stdout:6/615: rmdir d4/d24/d36 39 2024-04-25T01:35:53.450 INFO:tasks.workunit.client.0.smithi033.stdout:5/572: link d1c/d2a/d50/d52/c88 d1c/d2a/d3d/cc1 0 2024-04-25T01:35:53.450 INFO:tasks.workunit.client.0.smithi033.stdout:6/616: fdatasync d4/d2a/f3f 0 2024-04-25T01:35:53.450 INFO:tasks.workunit.client.0.smithi033.stdout:2/619: write da/d27/d31/d40/d8a/f3b [669052,66817] 0 2024-04-25T01:35:53.452 INFO:tasks.workunit.client.0.smithi033.stdout:3/556: link d8/dd/d3d/d63/d7c/d3c/d86/d92/f3b d8/dd/d11/fbc 0 2024-04-25T01:35:53.453 INFO:tasks.workunit.client.0.smithi033.stdout:5/573: dread - d1c/d2a/d50/d5b/d86/f7c zero size 2024-04-25T01:35:53.453 INFO:tasks.workunit.client.0.smithi033.stdout:2/620: chown da/d27/d31/d40/db1/fa6 393474 1 2024-04-25T01:35:53.455 INFO:tasks.workunit.client.0.smithi033.stdout:3/557: readlink d8/dd/d3d/d63/d7c/d3c/l44 0 2024-04-25T01:35:53.458 INFO:tasks.workunit.client.0.smithi033.stdout:6/617: dread d4/f6 [4194304,4194304] 0 2024-04-25T01:35:53.459 INFO:tasks.workunit.client.0.smithi033.stdout:6/618: chown d4/d15/c21 10 1 2024-04-25T01:35:53.459 INFO:tasks.workunit.client.0.smithi033.stdout:6/619: readlink d4/lb8 0 2024-04-25T01:35:53.461 INFO:tasks.workunit.client.0.smithi033.stdout:2/621: getdents da/d2c/dcd 0 2024-04-25T01:35:53.463 INFO:tasks.workunit.client.0.smithi033.stdout:2/622: fdatasync da/d27/d31/f83 0 2024-04-25T01:35:53.463 INFO:tasks.workunit.client.0.smithi033.stdout:8/598: dread d6/f60 [0,4194304] 0 2024-04-25T01:35:53.464 INFO:tasks.workunit.client.0.smithi033.stdout:3/558: link d8/dd/d1d/c3a d8/dd/d1d/cbd 0 2024-04-25T01:35:53.465 INFO:tasks.workunit.client.0.smithi033.stdout:5/574: getdents d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e 0 2024-04-25T01:35:53.466 INFO:tasks.workunit.client.0.smithi033.stdout:2/623: rmdir da/d27 39 2024-04-25T01:35:53.468 INFO:tasks.workunit.client.0.smithi033.stdout:8/599: write d6/f60 [4046663,39041] 0 2024-04-25T01:35:53.470 INFO:tasks.workunit.client.0.smithi033.stdout:6/620: dwrite d4/d1d/d3e/fa0 [0,4194304] 0 2024-04-25T01:35:53.474 INFO:tasks.workunit.client.0.smithi033.stdout:3/559: creat d8/dd/d3d/d63/d7c/fbe x:0 0 0 2024-04-25T01:35:53.475 INFO:tasks.workunit.client.0.smithi033.stdout:3/560: read - d8/dd/d3d/d63/f95 zero size 2024-04-25T01:35:53.477 INFO:tasks.workunit.client.0.smithi033.stdout:6/621: fsync d4/d24/d63/fa2 0 2024-04-25T01:35:53.482 INFO:tasks.workunit.client.0.smithi033.stdout:6/622: read d4/d1d/d3e/d4a/f4f [1810549,105802] 0 2024-04-25T01:35:53.483 INFO:tasks.workunit.client.0.smithi033.stdout:8/600: creat d6/d45/d52/fe2 x:0 0 0 2024-04-25T01:35:53.484 INFO:tasks.workunit.client.0.smithi033.stdout:3/561: dwrite d8/dd/d11/fa7 [0,4194304] 0 2024-04-25T01:35:53.485 INFO:tasks.workunit.client.0.smithi033.stdout:6/623: fdatasync d4/d1d/d3e/fc7 0 2024-04-25T01:35:53.490 INFO:tasks.workunit.client.0.smithi033.stdout:5/575: link d1c/d2a/d3d/f44 d1c/d2a/d50/d52/da6/d3e/d70/fc2 0 2024-04-25T01:35:53.493 INFO:tasks.workunit.client.0.smithi033.stdout:8/601: truncate d6/f41 648503 0 2024-04-25T01:35:53.493 INFO:tasks.workunit.client.0.smithi033.stdout:2/624: write da/fe [1709619,104218] 0 2024-04-25T01:35:53.500 INFO:tasks.workunit.client.0.smithi033.stdout:2/625: rename da/d5b/d5e/da8/fae to da/d8d/db0/fd0 0 2024-04-25T01:35:53.502 INFO:tasks.workunit.client.0.smithi033.stdout:3/562: read d8/dd/d3d/d63/d7c/d4d/f8d [203468,18965] 0 2024-04-25T01:35:53.503 INFO:tasks.workunit.client.0.smithi033.stdout:8/602: creat d6/db/d80/dd1/fe3 x:0 0 0 2024-04-25T01:35:53.505 INFO:tasks.workunit.client.0.smithi033.stdout:2/626: mkdir da/d5b/d72/dd1 0 2024-04-25T01:35:53.507 INFO:tasks.workunit.client.0.smithi033.stdout:6/624: creat d4/d24/d36/fda x:0 0 0 2024-04-25T01:35:53.508 INFO:tasks.workunit.client.0.smithi033.stdout:5/576: getdents d1c/d2a/d50/d52/da6/d3e 0 2024-04-25T01:35:53.515 INFO:tasks.workunit.client.0.smithi033.stdout:6/625: dwrite d4/d24/d7c/fba [0,4194304] 0 2024-04-25T01:35:53.515 INFO:tasks.workunit.client.0.smithi033.stdout:3/563: mkdir d8/dd/d3d/d4f/da4/db2/dbf 0 2024-04-25T01:35:53.517 INFO:tasks.workunit.client.0.smithi033.stdout:8/603: write d6/d45/d52/f36 [2285255,98342] 0 2024-04-25T01:35:53.519 INFO:tasks.workunit.client.0.smithi033.stdout:8/604: rmdir d6/d49 39 2024-04-25T01:35:53.523 INFO:tasks.workunit.client.0.smithi033.stdout:8/605: mkdir d6/db/d80/db4/de4 0 2024-04-25T01:35:53.524 INFO:tasks.workunit.client.0.smithi033.stdout:5/577: dread d1c/f1e [0,4194304] 0 2024-04-25T01:35:53.524 INFO:tasks.workunit.client.0.smithi033.stdout:3/564: fdatasync d8/dd/d3d/d63/d7c/d3c/d86/d92/f45 0 2024-04-25T01:35:53.525 INFO:tasks.workunit.client.0.smithi033.stdout:8/606: write d6/d45/d51/d5d/d6e/fbd [688831,30524] 0 2024-04-25T01:35:53.527 INFO:tasks.workunit.client.0.smithi033.stdout:3/565: read d8/fa [8320643,67372] 0 2024-04-25T01:35:53.528 INFO:tasks.workunit.client.0.smithi033.stdout:8/607: creat d6/d45/d52/db0/dbc/fe5 x:0 0 0 2024-04-25T01:35:53.534 INFO:tasks.workunit.client.0.smithi033.stdout:3/566: fdatasync d8/dd/d3d/d63/d7c/f64 0 2024-04-25T01:35:53.537 INFO:tasks.workunit.client.0.smithi033.stdout:5/578: creat d1c/d2a/d50/d52/da6/d3e/d70/fc3 x:0 0 0 2024-04-25T01:35:53.540 INFO:tasks.workunit.client.0.smithi033.stdout:8/608: rename d6/d45/d52/d2e/d95/lad to d6/d45/d51/d5d/d6e/le6 0 2024-04-25T01:35:53.540 INFO:tasks.workunit.client.0.smithi033.stdout:3/567: dread d8/dd/d3d/d63/f71 [0,4194304] 0 2024-04-25T01:35:53.541 INFO:tasks.workunit.client.0.smithi033.stdout:8/609: stat d6/db/d80/d89/f91 0 2024-04-25T01:35:53.542 INFO:tasks.workunit.client.0.smithi033.stdout:8/610: fsync d6/db/d80/fb7 0 2024-04-25T01:35:53.542 INFO:tasks.workunit.client.0.smithi033.stdout:3/568: truncate d8/dd/d11/fac 896628 0 2024-04-25T01:35:53.543 INFO:tasks.workunit.client.0.smithi033.stdout:3/569: write d8/f59 [600443,16537] 0 2024-04-25T01:35:53.544 INFO:tasks.workunit.client.0.smithi033.stdout:5/579: rename d1c/d2a/f47 to d1c/d2a/d50/d52/da6/d3e/d70/d7a/dab/fc4 0 2024-04-25T01:35:53.555 INFO:tasks.workunit.client.0.smithi033.stdout:8/611: symlink d6/d45/d52/dae/dc3/le7 0 2024-04-25T01:35:53.561 INFO:tasks.workunit.client.0.smithi033.stdout:8/612: dwrite d6/d45/d51/f4f [4194304,4194304] 0 2024-04-25T01:35:53.561 INFO:tasks.workunit.client.0.smithi033.stdout:3/570: truncate d8/dd/d3d/d4f/f69 1090158 0 2024-04-25T01:35:53.563 INFO:tasks.workunit.client.0.smithi033.stdout:3/571: write d8/dd/d11/fa7 [4269846,59434] 0 2024-04-25T01:35:53.564 INFO:tasks.workunit.client.0.smithi033.stdout:8/613: creat d6/db/d80/dd1/fe8 x:0 0 0 2024-04-25T01:35:53.564 INFO:tasks.workunit.client.0.smithi033.stdout:3/572: chown d8/f59 40352894 1 2024-04-25T01:35:53.565 INFO:tasks.workunit.client.0.smithi033.stdout:3/573: fdatasync d8/dd/f34 0 2024-04-25T01:35:53.566 INFO:tasks.workunit.client.0.smithi033.stdout:8/614: symlink d6/d45/d51/d5d/d6e/le9 0 2024-04-25T01:35:53.572 INFO:tasks.workunit.client.0.smithi033.stdout:8/615: link d6/f34 d6/d45/d51/d55/daa/fea 0 2024-04-25T01:35:53.572 INFO:tasks.workunit.client.0.smithi033.stdout:3/574: dwrite d8/dd/d11/fad [0,4194304] 0 2024-04-25T01:35:53.575 INFO:tasks.workunit.client.0.smithi033.stdout:3/575: fdatasync d8/dd/d1d/f93 0 2024-04-25T01:35:53.577 INFO:tasks.workunit.client.0.smithi033.stdout:5/580: dread d1c/d2a/d50/d52/da6/f26 [0,4194304] 0 2024-04-25T01:35:53.579 INFO:tasks.workunit.client.0.smithi033.stdout:3/576: write d8/dd/d3d/d63/d7c/d3c/fa3 [658789,17541] 0 2024-04-25T01:35:53.581 INFO:tasks.workunit.client.0.smithi033.stdout:3/577: write d8/dd/d11/fa7 [4686873,65210] 0 2024-04-25T01:35:53.581 INFO:tasks.workunit.client.0.smithi033.stdout:5/581: mknod d1c/d2a/d3d/cc5 0 2024-04-25T01:35:53.584 INFO:tasks.workunit.client.0.smithi033.stdout:3/578: mkdir d8/dd/d3d/d63/d7c/d3c/dc0 0 2024-04-25T01:35:53.590 INFO:tasks.workunit.client.0.smithi033.stdout:3/579: rmdir d8/dd/d3d/d63 39 2024-04-25T01:35:53.594 INFO:tasks.workunit.client.0.smithi033.stdout:3/580: creat d8/dd/d3d/d63/d7c/d3c/d98/da1/fc1 x:0 0 0 2024-04-25T01:35:53.596 INFO:tasks.workunit.client.0.smithi033.stdout:3/581: creat d8/dd/d3d/d63/d7c/d3c/dc0/fc2 x:0 0 0 2024-04-25T01:35:53.598 INFO:tasks.workunit.client.0.smithi033.stdout:5/582: dread d1c/d2a/d50/f7d [0,4194304] 0 2024-04-25T01:35:53.599 INFO:tasks.workunit.client.0.smithi033.stdout:5/583: stat d1c/d2a/d50/d52/da6/d3e/d70/d9b/la4 0 2024-04-25T01:35:53.599 INFO:tasks.workunit.client.0.smithi033.stdout:3/582: mkdir d8/dd/d3d/d63/d7c/d3c/d98/da1/dc3 0 2024-04-25T01:35:53.600 INFO:tasks.workunit.client.0.smithi033.stdout:3/583: readlink d8/dd/l9e 0 2024-04-25T01:35:53.601 INFO:tasks.workunit.client.0.smithi033.stdout:5/584: symlink d1c/d2a/d50/db1/lc6 0 2024-04-25T01:35:53.603 INFO:tasks.workunit.client.0.smithi033.stdout:3/584: mkdir d8/dd/d3d/d4f/da4/dc4 0 2024-04-25T01:35:53.605 INFO:tasks.workunit.client.0.smithi033.stdout:5/585: link d1c/d2a/d50/d52/da6/d3e/d70/l9c d1c/d2a/d50/d5b/lc7 0 2024-04-25T01:35:53.606 INFO:tasks.workunit.client.0.smithi033.stdout:3/585: mkdir d8/dd/d3d/d4f/da4/db2/dbf/dc5 0 2024-04-25T01:35:53.606 INFO:tasks.workunit.client.0.smithi033.stdout:5/586: write d1c/d2a/d3d/f40 [650908,107352] 0 2024-04-25T01:35:53.608 INFO:tasks.workunit.client.0.smithi033.stdout:3/586: dread - d8/dd/d3d/d63/d7c/d3c/d98/da1/fc1 zero size 2024-04-25T01:35:53.610 INFO:tasks.workunit.client.0.smithi033.stdout:3/587: creat d8/dd/d3d/d4f/da4/db2/dbf/dc5/fc6 x:0 0 0 2024-04-25T01:35:53.612 INFO:tasks.workunit.client.0.smithi033.stdout:3/588: write d8/dd/d3d/d63/d7c/d4d/f89 [647079,123333] 0 2024-04-25T01:35:53.613 INFO:tasks.workunit.client.0.smithi033.stdout:5/587: rmdir d1c/d2a/d50/d5b 39 2024-04-25T01:35:53.614 INFO:tasks.workunit.client.0.smithi033.stdout:3/589: mkdir d8/dd/d3d/d4f/da4/db2/dc7 0 2024-04-25T01:35:53.619 INFO:tasks.workunit.client.0.smithi033.stdout:5/588: fsync f7 0 2024-04-25T01:35:53.619 INFO:tasks.workunit.client.0.smithi033.stdout:5/589: creat d1c/d2a/d50/d52/da6/d3e/fc8 x:0 0 0 2024-04-25T01:35:53.619 INFO:tasks.workunit.client.0.smithi033.stdout:3/590: dread d8/dd/d11/fa7 [0,4194304] 0 2024-04-25T01:35:53.626 INFO:tasks.workunit.client.0.smithi033.stdout:4/601: sync 2024-04-25T01:35:53.626 INFO:tasks.workunit.client.0.smithi033.stdout:9/502: sync 2024-04-25T01:35:53.631 INFO:tasks.workunit.client.0.smithi033.stdout:5/590: dwrite d1c/d2a/d50/d52/da6/d3e/d70/dae/fb5 [0,4194304] 0 2024-04-25T01:35:53.632 INFO:tasks.workunit.client.0.smithi033.stdout:4/602: rename d6/d14/d35/d3a/d4d/l98 to d6/d14/d35/d44/d45/d46/db0/lbc 0 2024-04-25T01:35:53.637 INFO:tasks.workunit.client.0.smithi033.stdout:9/503: dwrite d0/fa2 [0,4194304] 0 2024-04-25T01:35:53.637 INFO:tasks.workunit.client.0.smithi033.stdout:3/591: dread d8/dd/d11/f24 [0,4194304] 0 2024-04-25T01:35:53.640 INFO:tasks.workunit.client.0.smithi033.stdout:5/591: write d1c/d2a/d3d/d59/f84 [624775,88490] 0 2024-04-25T01:35:53.644 INFO:tasks.workunit.client.0.smithi033.stdout:3/592: dread d8/dd/d3d/d63/d7c/f64 [4194304,4194304] 0 2024-04-25T01:35:53.648 INFO:tasks.workunit.client.0.smithi033.stdout:5/592: unlink d1c/d2a/d50/d5b/lc7 0 2024-04-25T01:35:53.649 INFO:tasks.workunit.client.0.smithi033.stdout:4/603: getdents d6/d14/d35/d44/d45/d46/d57/d70/d87 0 2024-04-25T01:35:53.652 INFO:tasks.workunit.client.0.smithi033.stdout:9/504: dwrite d0/d2/f8a [0,4194304] 0 2024-04-25T01:35:53.652 INFO:tasks.workunit.client.0.smithi033.stdout:3/593: rename d8/dd/d11/f24 to d8/dd/d3d/fc8 0 2024-04-25T01:35:53.654 INFO:tasks.workunit.client.0.smithi033.stdout:5/593: fsync d1c/f1f 0 2024-04-25T01:35:53.655 INFO:tasks.workunit.client.0.smithi033.stdout:4/604: creat d6/d14/d35/d44/d45/d46/fbd x:0 0 0 2024-04-25T01:35:53.658 INFO:tasks.workunit.client.0.smithi033.stdout:9/505: mknod d0/d2/dd/df/d1b/d3d/d6e/cbb 0 2024-04-25T01:35:53.661 INFO:tasks.workunit.client.0.smithi033.stdout:3/594: unlink d8/dd/d11/c61 0 2024-04-25T01:35:53.663 INFO:tasks.workunit.client.0.smithi033.stdout:4/605: write d6/d12/d84/fae [3617994,67146] 0 2024-04-25T01:35:53.665 INFO:tasks.workunit.client.0.smithi033.stdout:5/594: creat d1c/d2a/d50/d52/fc9 x:0 0 0 2024-04-25T01:35:53.666 INFO:tasks.workunit.client.0.smithi033.stdout:9/506: dwrite d0/d11/f16 [0,4194304] 0 2024-04-25T01:35:53.666 INFO:tasks.workunit.client.0.smithi033.stdout:3/595: chown d8/dd/d3d/d63/d7c/f73 0 1 2024-04-25T01:35:53.669 INFO:tasks.workunit.client.0.smithi033.stdout:3/596: mknod d8/dd/d40/cc9 0 2024-04-25T01:35:53.670 INFO:tasks.workunit.client.0.smithi033.stdout:5/595: mknod d1c/d2a/d50/d52/da6/d3e/d70/dae/cca 0 2024-04-25T01:35:53.671 INFO:tasks.workunit.client.0.smithi033.stdout:9/507: creat d0/d2/d43/d4b/d55/fbc x:0 0 0 2024-04-25T01:35:53.674 INFO:tasks.workunit.client.0.smithi033.stdout:5/596: mkdir d1c/d2a/d50/dcb 0 2024-04-25T01:35:53.676 INFO:tasks.workunit.client.0.smithi033.stdout:4/606: dwrite d6/d14/d1d/f63 [0,4194304] 0 2024-04-25T01:35:53.677 INFO:tasks.workunit.client.0.smithi033.stdout:9/508: mknod d0/d2/dd/da7/cbd 0 2024-04-25T01:35:53.681 INFO:tasks.workunit.client.0.smithi033.stdout:9/509: chown d0/d2/dd/f24 120254 1 2024-04-25T01:35:53.683 INFO:tasks.workunit.client.0.smithi033.stdout:4/607: stat c1 0 2024-04-25T01:35:53.684 INFO:tasks.workunit.client.0.smithi033.stdout:4/608: stat d6/d12/d84 0 2024-04-25T01:35:53.686 INFO:tasks.workunit.client.0.smithi033.stdout:3/597: dwrite d8/dd/d3d/d63/d7c/f73 [0,4194304] 0 2024-04-25T01:35:53.686 INFO:tasks.workunit.client.0.smithi033.stdout:5/597: dread d1c/f23 [0,4194304] 0 2024-04-25T01:35:53.687 INFO:tasks.workunit.client.0.smithi033.stdout:4/609: chown d6/d12/d38/f91 460373 1 2024-04-25T01:35:53.693 INFO:tasks.workunit.client.0.smithi033.stdout:5/598: unlink d1c/d2a/d50/d52/da6/f26 0 2024-04-25T01:35:53.694 INFO:tasks.workunit.client.0.smithi033.stdout:4/610: dread d6/d14/f22 [0,4194304] 0 2024-04-25T01:35:53.695 INFO:tasks.workunit.client.0.smithi033.stdout:9/510: dwrite d0/d11/f16 [0,4194304] 0 2024-04-25T01:35:53.698 INFO:tasks.workunit.client.0.smithi033.stdout:3/598: dwrite d8/dd/d40/f65 [0,4194304] 0 2024-04-25T01:35:53.700 INFO:tasks.workunit.client.0.smithi033.stdout:9/511: mkdir d0/d2/dd/dba/dbe 0 2024-04-25T01:35:53.701 INFO:tasks.workunit.client.0.smithi033.stdout:9/512: write d0/f9e [1767734,74475] 0 2024-04-25T01:35:53.702 INFO:tasks.workunit.client.0.smithi033.stdout:4/611: rename d6/d14/d6d/f77 to d6/d14/d35/d44/d45/d46/db0/fbe 0 2024-04-25T01:35:53.707 INFO:tasks.workunit.client.0.smithi033.stdout:9/513: symlink d0/d2/dd/d20/lbf 0 2024-04-25T01:35:53.710 INFO:tasks.workunit.client.0.smithi033.stdout:3/599: dwrite d8/dd/d3d/d63/d7c/d3c/f94 [0,4194304] 0 2024-04-25T01:35:53.710 INFO:tasks.workunit.client.0.smithi033.stdout:9/514: dread - d0/d2/d43/d73/f9f zero size 2024-04-25T01:35:53.716 INFO:tasks.workunit.client.0.smithi033.stdout:9/515: mkdir d0/d2/dd/df/d1b/dc0 0 2024-04-25T01:35:53.718 INFO:tasks.workunit.client.0.smithi033.stdout:3/600: link d8/dd/d3d/d63/d7c/d3c/d6e/f9f d8/dd/d3d/d63/fca 0 2024-04-25T01:35:53.720 INFO:tasks.workunit.client.0.smithi033.stdout:3/601: truncate d8/dd/d3d/d63/d7c/d4d/f8d 1076695 0 2024-04-25T01:35:53.722 INFO:tasks.workunit.client.0.smithi033.stdout:3/602: creat d8/dd/d3d/d4f/fcb x:0 0 0 2024-04-25T01:35:53.723 INFO:tasks.workunit.client.0.smithi033.stdout:3/603: chown d8/dd/f34 33891017 1 2024-04-25T01:35:53.724 INFO:tasks.workunit.client.0.smithi033.stdout:3/604: dread - d8/dd/d3d/d63/d7c/d3c/dc0/fc2 zero size 2024-04-25T01:35:53.725 INFO:tasks.workunit.client.0.smithi033.stdout:3/605: dread - d8/dd/d3d/d4f/faf zero size 2024-04-25T01:35:53.727 INFO:tasks.workunit.client.0.smithi033.stdout:3/606: creat d8/dd/fcc x:0 0 0 2024-04-25T01:35:53.731 INFO:tasks.workunit.client.0.smithi033.stdout:4/612: dread d6/d12/d38/f41 [0,4194304] 0 2024-04-25T01:35:53.733 INFO:tasks.workunit.client.0.smithi033.stdout:3/607: creat d8/dd/d3d/d63/d7c/d3c/d86/d92/fcd x:0 0 0 2024-04-25T01:35:53.734 INFO:tasks.workunit.client.1.smithi138.stdout:5/3: sync 2024-04-25T01:35:53.736 INFO:tasks.workunit.client.1.smithi138.stdout:5/4: symlink l0 0 2024-04-25T01:35:53.736 INFO:tasks.workunit.client.1.smithi138.stdout:5/5: dwrite - no filename 2024-04-25T01:35:53.737 INFO:tasks.workunit.client.0.smithi033.stdout:4/613: link d6/d14/d1d/d3b/f52 d6/d14/d1d/d80/fbf 0 2024-04-25T01:35:53.739 INFO:tasks.workunit.client.0.smithi033.stdout:3/608: mkdir d8/dd/d3d/dce 0 2024-04-25T01:35:53.739 INFO:tasks.workunit.client.1.smithi138.stdout:5/6: getdents . 0 2024-04-25T01:35:53.739 INFO:tasks.workunit.client.1.smithi138.stdout:5/7: dwrite - no filename 2024-04-25T01:35:53.740 INFO:tasks.workunit.client.1.smithi138.stdout:5/8: dwrite - no filename 2024-04-25T01:35:53.740 INFO:tasks.workunit.client.1.smithi138.stdout:5/9: dwrite - no filename 2024-04-25T01:35:53.740 INFO:tasks.workunit.client.1.smithi138.stdout:5/10: read - no filename 2024-04-25T01:35:53.740 INFO:tasks.workunit.client.0.smithi033.stdout:4/614: mkdir d6/d12/d38/dc0 0 2024-04-25T01:35:53.741 INFO:tasks.workunit.client.1.smithi138.stdout:5/11: rename l0 to l1 0 2024-04-25T01:35:53.741 INFO:tasks.workunit.client.1.smithi138.stdout:5/12: write - no filename 2024-04-25T01:35:53.741 INFO:tasks.workunit.client.1.smithi138.stdout:5/13: dwrite - no filename 2024-04-25T01:35:53.741 INFO:tasks.workunit.client.1.smithi138.stdout:5/14: truncate - no filename 2024-04-25T01:35:53.741 INFO:tasks.workunit.client.1.smithi138.stdout:5/15: fsync - no filename 2024-04-25T01:35:53.741 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:53 smithi033 ceph-mon[26353]: pgmap v12: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 140 MiB/s wr, 534 op/s 2024-04-25T01:35:53.742 INFO:tasks.workunit.client.0.smithi033.stdout:4/615: write d6/d12/d38/f41 [3892517,59118] 0 2024-04-25T01:35:53.744 INFO:tasks.workunit.client.0.smithi033.stdout:9/516: write d0/d2/dd/f13 [8796776,76655] 0 2024-04-25T01:35:53.745 INFO:tasks.workunit.client.0.smithi033.stdout:4/616: creat d6/d14/d6d/fc1 x:0 0 0 2024-04-25T01:35:53.752 INFO:tasks.workunit.client.0.smithi033.stdout:9/517: dwrite d0/d2/dd/df/d1b/d3d/d7e/d58/fa6 [0,4194304] 0 2024-04-25T01:35:53.753 INFO:tasks.workunit.client.0.smithi033.stdout:4/617: link d6/d14/d35/l66 d6/d14/d35/lc2 0 2024-04-25T01:35:53.756 INFO:tasks.workunit.client.0.smithi033.stdout:3/609: dwrite d8/dd/d3d/d63/f99 [0,4194304] 0 2024-04-25T01:35:53.756 INFO:tasks.workunit.client.0.smithi033.stdout:4/618: write d6/d14/d35/d3a/d4d/d5e/f76 [5251808,47297] 0 2024-04-25T01:35:53.765 INFO:tasks.workunit.client.0.smithi033.stdout:9/518: dwrite d0/d2/d43/d4b/fa8 [0,4194304] 0 2024-04-25T01:35:53.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:53 smithi138 ceph-mon[32977]: pgmap v12: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 34 MiB/s rd, 140 MiB/s wr, 534 op/s 2024-04-25T01:35:53.767 INFO:tasks.workunit.client.0.smithi033.stdout:9/519: mkdir d0/d2/dd/dba/dc1 0 2024-04-25T01:35:53.770 INFO:tasks.workunit.client.0.smithi033.stdout:9/520: rename d0/d2/dd/df/f82 to d0/d2/dd/d20/fc2 0 2024-04-25T01:35:53.771 INFO:tasks.workunit.client.0.smithi033.stdout:3/610: dread d8/dd/d3d/d63/d7c/d3c/fa3 [0,4194304] 0 2024-04-25T01:35:53.775 INFO:tasks.workunit.client.0.smithi033.stdout:3/611: creat d8/dd/fcf x:0 0 0 2024-04-25T01:35:53.779 INFO:tasks.workunit.client.0.smithi033.stdout:9/521: dwrite d0/d2/dd/d5f/f89 [0,4194304] 0 2024-04-25T01:35:53.781 INFO:tasks.workunit.client.0.smithi033.stdout:3/612: creat d8/dd/d3d/d63/d7c/d3c/d98/da1/dc3/fd0 x:0 0 0 2024-04-25T01:35:53.784 INFO:tasks.workunit.client.0.smithi033.stdout:9/522: mkdir d0/d2/d43/d4b/dc3 0 2024-04-25T01:35:53.788 INFO:tasks.workunit.client.0.smithi033.stdout:9/523: mkdir d0/d2/d43/d73/dc4 0 2024-04-25T01:35:53.790 INFO:tasks.workunit.client.0.smithi033.stdout:9/524: dread - d0/d2/d43/d4b/d55/fb1 zero size 2024-04-25T01:35:53.790 INFO:tasks.workunit.client.0.smithi033.stdout:4/619: dread d6/f18 [0,4194304] 0 2024-04-25T01:35:53.791 INFO:tasks.workunit.client.0.smithi033.stdout:9/525: write d0/d2/dd/df/d1b/f99 [520882,44905] 0 2024-04-25T01:35:53.792 INFO:tasks.workunit.client.0.smithi033.stdout:4/620: chown d6/d14/d1d/f2e 2316731 1 2024-04-25T01:35:53.794 INFO:tasks.workunit.client.0.smithi033.stdout:9/526: chown d0/d11/l40 32107 1 2024-04-25T01:35:53.794 INFO:tasks.workunit.client.0.smithi033.stdout:4/621: write d6/d14/d48/f55 [1157046,42147] 0 2024-04-25T01:35:53.799 INFO:tasks.workunit.client.0.smithi033.stdout:9/527: getdents d0 0 2024-04-25T01:35:53.799 INFO:tasks.workunit.client.0.smithi033.stdout:3/613: dwrite d8/dd/f15 [0,4194304] 0 2024-04-25T01:35:53.799 INFO:tasks.workunit.client.0.smithi033.stdout:4/622: dread - d6/d14/d35/d44/d45/d46/d57/d70/d87/faf zero size 2024-04-25T01:35:53.801 INFO:tasks.workunit.client.0.smithi033.stdout:9/528: chown d0/d2/dd/d20/l83 4379473 1 2024-04-25T01:35:53.801 INFO:tasks.workunit.client.0.smithi033.stdout:3/614: fsync d8/dd/d3d/d63/d7c/d3c/f7d 0 2024-04-25T01:35:53.801 INFO:tasks.workunit.client.0.smithi033.stdout:4/623: stat d6/d14/f33 0 2024-04-25T01:35:53.803 INFO:tasks.workunit.client.0.smithi033.stdout:4/624: dread - d6/d14/d1d/fb9 zero size 2024-04-25T01:35:53.804 INFO:tasks.workunit.client.0.smithi033.stdout:0/479: sync 2024-04-25T01:35:53.804 INFO:tasks.workunit.client.0.smithi033.stdout:1/381: sync 2024-04-25T01:35:53.805 INFO:tasks.workunit.client.0.smithi033.stdout:3/615: fdatasync d8/dd/d3d/d4f/f6f 0 2024-04-25T01:35:53.805 INFO:tasks.workunit.client.0.smithi033.stdout:9/529: mkdir d0/d2/d43/d4b/dc5 0 2024-04-25T01:35:53.808 INFO:tasks.workunit.client.0.smithi033.stdout:3/616: mkdir d8/dd/d1d/dd1 0 2024-04-25T01:35:53.816 INFO:tasks.workunit.client.0.smithi033.stdout:1/382: creat d2/d2e/d34/d60/f74 x:0 0 0 2024-04-25T01:35:53.823 INFO:tasks.workunit.client.0.smithi033.stdout:0/480: dwrite d7/da/df/d50/f7d [4194304,4194304] 0 2024-04-25T01:35:53.823 INFO:tasks.workunit.client.0.smithi033.stdout:9/530: truncate d0/d2/f1f 1623113 0 2024-04-25T01:35:53.824 INFO:tasks.workunit.client.0.smithi033.stdout:3/617: rename d8/dd/d11/fb0 to d8/dd/d3d/d4f/da4/fd2 0 2024-04-25T01:35:53.827 INFO:tasks.workunit.client.0.smithi033.stdout:9/531: dread - d0/d2/dd/d20/d4e/f94 zero size 2024-04-25T01:35:53.828 INFO:tasks.workunit.client.0.smithi033.stdout:0/481: dwrite d7/d2d/d4d/d9c/d61/f66 [0,4194304] 0 2024-04-25T01:35:53.829 INFO:tasks.workunit.client.0.smithi033.stdout:0/482: readlink d7/d2d/d4c/l55 0 2024-04-25T01:35:53.833 INFO:tasks.workunit.client.0.smithi033.stdout:1/383: dread d2/d11/d28/d33/f53 [0,4194304] 0 2024-04-25T01:35:53.835 INFO:tasks.workunit.client.0.smithi033.stdout:1/384: write d2/d11/d28/f2c [456020,42574] 0 2024-04-25T01:35:53.892 INFO:tasks.workunit.client.1.smithi138.stdout:7/9: sync 2024-04-25T01:35:53.893 INFO:tasks.workunit.client.1.smithi138.stdout:7/10: chown c2 14 1 2024-04-25T01:35:53.894 INFO:tasks.workunit.client.1.smithi138.stdout:7/11: symlink l3 0 2024-04-25T01:35:53.894 INFO:tasks.workunit.client.1.smithi138.stdout:7/12: write - no filename 2024-04-25T01:35:53.895 INFO:tasks.workunit.client.1.smithi138.stdout:7/13: creat f4 x:0 0 0 2024-04-25T01:35:53.895 INFO:tasks.workunit.client.1.smithi138.stdout:7/14: chown c2 115732 1 2024-04-25T01:35:53.897 INFO:tasks.workunit.client.0.smithi033.stdout:9/532: rmdir d0/d2/d43/d4b/d55 39 2024-04-25T01:35:53.898 INFO:tasks.workunit.client.1.smithi138.stdout:7/15: rename l1 to l5 0 2024-04-25T01:35:53.899 INFO:tasks.workunit.client.1.smithi138.stdout:7/16: creat f6 x:0 0 0 2024-04-25T01:35:53.899 INFO:tasks.workunit.client.1.smithi138.stdout:7/17: write f6 [616834,2136] 0 2024-04-25T01:35:53.901 INFO:tasks.workunit.client.1.smithi138.stdout:7/18: symlink l7 0 2024-04-25T01:35:53.904 INFO:tasks.workunit.client.0.smithi033.stdout:1/385: unlink d2/d2e/d34/f4e 0 2024-04-25T01:35:53.905 INFO:tasks.workunit.client.0.smithi033.stdout:3/618: truncate d8/dd/d1d/f7e 157191 0 2024-04-25T01:35:53.906 INFO:tasks.workunit.client.0.smithi033.stdout:3/619: fdatasync d8/dd/d3d/d63/f95 0 2024-04-25T01:35:53.906 INFO:tasks.workunit.client.0.smithi033.stdout:9/533: unlink d0/d2/ca0 0 2024-04-25T01:35:53.908 INFO:tasks.workunit.client.1.smithi138.stdout:7/19: dwrite f4 [0,4194304] 0 2024-04-25T01:35:53.908 INFO:tasks.workunit.client.0.smithi033.stdout:1/386: rename d2/d11/d28/d33 to d2/d5/d75 0 2024-04-25T01:35:53.909 INFO:tasks.workunit.client.1.smithi138.stdout:7/20: mknod c8 0 2024-04-25T01:35:53.910 INFO:tasks.workunit.client.0.smithi033.stdout:0/483: getdents d7/da/df/d14 0 2024-04-25T01:35:53.910 INFO:tasks.workunit.client.1.smithi138.stdout:7/21: write f6 [1264805,106960] 0 2024-04-25T01:35:53.911 INFO:tasks.workunit.client.0.smithi033.stdout:9/534: rmdir d0/d2/dd/d20 39 2024-04-25T01:35:53.911 INFO:tasks.workunit.client.1.smithi138.stdout:7/22: read f6 [1308077,686] 0 2024-04-25T01:35:53.916 INFO:tasks.workunit.client.1.smithi138.stdout:7/23: link f6 f9 0 2024-04-25T01:35:53.916 INFO:tasks.workunit.client.0.smithi033.stdout:1/387: mknod d2/d5/d1b/d46/d5f/d63/c76 0 2024-04-25T01:35:53.916 INFO:tasks.workunit.client.1.smithi138.stdout:7/24: chown l3 11415 1 2024-04-25T01:35:53.917 INFO:tasks.workunit.client.0.smithi033.stdout:1/388: fsync d2/d11/f6d 0 2024-04-25T01:35:53.917 INFO:tasks.workunit.client.1.smithi138.stdout:7/25: mkdir da 0 2024-04-25T01:35:53.918 INFO:tasks.workunit.client.0.smithi033.stdout:1/389: chown d2/d5/cc 2 1 2024-04-25T01:35:53.919 INFO:tasks.workunit.client.0.smithi033.stdout:0/484: dwrite d7/d2d/d4d/d9c/d61/d70/fa9 [0,4194304] 0 2024-04-25T01:35:53.919 INFO:tasks.workunit.client.1.smithi138.stdout:7/26: rename l7 to da/lb 0 2024-04-25T01:35:53.920 INFO:tasks.workunit.client.0.smithi033.stdout:3/620: rename d8/dd/d3d/d63/d7c/d3c/d86/d92/l74 to d8/dd/d3d/d63/d7c/d3c/d98/ld3 0 2024-04-25T01:35:53.922 INFO:tasks.workunit.client.0.smithi033.stdout:9/535: symlink d0/d2/dd/df/d1b/d3d/d42/lc6 0 2024-04-25T01:35:53.923 INFO:tasks.workunit.client.0.smithi033.stdout:1/390: rmdir d2/d5/d75/d59 39 2024-04-25T01:35:53.925 INFO:tasks.workunit.client.0.smithi033.stdout:0/485: mknod d7/da/df/d14/cab 0 2024-04-25T01:35:53.925 INFO:tasks.workunit.client.0.smithi033.stdout:1/391: read d2/d11/f42 [1380011,25534] 0 2024-04-25T01:35:53.928 INFO:tasks.workunit.client.1.smithi138.stdout:7/27: dwrite f6 [0,4194304] 0 2024-04-25T01:35:53.928 INFO:tasks.workunit.client.0.smithi033.stdout:3/621: mknod d8/dd/d1d/dd1/cd4 0 2024-04-25T01:35:53.934 INFO:tasks.workunit.client.0.smithi033.stdout:1/392: symlink d2/d5/d1b/d46/d5f/l77 0 2024-04-25T01:35:53.936 INFO:tasks.workunit.client.1.smithi138.stdout:7/28: dwrite f6 [0,4194304] 0 2024-04-25T01:35:53.936 INFO:tasks.workunit.client.0.smithi033.stdout:9/536: dwrite d0/d2/dd/d20/fa4 [4194304,4194304] 0 2024-04-25T01:35:53.937 INFO:tasks.workunit.client.1.smithi138.stdout:7/29: chown f4 62 1 2024-04-25T01:35:53.938 INFO:tasks.workunit.client.0.smithi033.stdout:3/622: rename d8/dd/d3d/d63/d7c/d3c/f94 to d8/dd/d3d/d4f/fd5 0 2024-04-25T01:35:53.939 INFO:tasks.workunit.client.1.smithi138.stdout:7/30: creat da/fc x:0 0 0 2024-04-25T01:35:53.941 INFO:tasks.workunit.client.1.smithi138.stdout:7/31: symlink da/ld 0 2024-04-25T01:35:53.942 INFO:tasks.workunit.client.1.smithi138.stdout:7/32: dread - da/fc zero size 2024-04-25T01:35:53.945 INFO:tasks.workunit.client.1.smithi138.stdout:7/33: mkdir da/de 0 2024-04-25T01:35:53.946 INFO:tasks.workunit.client.0.smithi033.stdout:3/623: mknod d8/dd/d1d/cd6 0 2024-04-25T01:35:53.947 INFO:tasks.workunit.client.0.smithi033.stdout:9/537: symlink d0/d2/d43/d73/lc7 0 2024-04-25T01:35:53.948 INFO:tasks.workunit.client.1.smithi138.stdout:7/34: rename f6 to da/ff 0 2024-04-25T01:35:53.949 INFO:tasks.workunit.client.0.smithi033.stdout:0/486: getdents d7/d2d/d64/d51 0 2024-04-25T01:35:53.951 INFO:tasks.workunit.client.0.smithi033.stdout:3/624: unlink d8/dd/d11/l3e 0 2024-04-25T01:35:53.952 INFO:tasks.workunit.client.1.smithi138.stdout:7/35: dread da/ff [0,4194304] 0 2024-04-25T01:35:53.952 INFO:tasks.workunit.client.0.smithi033.stdout:0/487: symlink d7/da/d80/lac 0 2024-04-25T01:35:53.953 INFO:tasks.workunit.client.0.smithi033.stdout:0/488: chown d7/d33/f3e 23 1 2024-04-25T01:35:53.954 INFO:tasks.workunit.client.0.smithi033.stdout:3/625: symlink d8/dd/d1d/ld7 0 2024-04-25T01:35:53.954 INFO:tasks.workunit.client.1.smithi138.stdout:7/36: creat da/de/f10 x:0 0 0 2024-04-25T01:35:53.956 INFO:tasks.workunit.client.0.smithi033.stdout:0/489: creat d7/d2d/d64/fad x:0 0 0 2024-04-25T01:35:53.956 INFO:tasks.workunit.client.1.smithi138.stdout:7/37: symlink da/l11 0 2024-04-25T01:35:53.957 INFO:tasks.workunit.client.1.smithi138.stdout:7/38: creat da/f12 x:0 0 0 2024-04-25T01:35:53.957 INFO:tasks.workunit.client.0.smithi033.stdout:3/626: write d8/dd/d3d/d4f/fd5 [2281521,59790] 0 2024-04-25T01:35:53.957 INFO:tasks.workunit.client.1.smithi138.stdout:7/39: chown c2 0 1 2024-04-25T01:35:53.958 INFO:tasks.workunit.client.0.smithi033.stdout:9/538: dread d0/d2/dd/f56 [4194304,4194304] 0 2024-04-25T01:35:53.959 INFO:tasks.workunit.client.1.smithi138.stdout:7/40: mkdir da/de/d13 0 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:7/41: write da/de/f10 [56565,40473] 0 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:5/16: sync 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:5/17: truncate - no filename 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:0/11: sync 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:8/22: sync 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:3/41: sync 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:9/34: sync 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:1/31: sync 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:6/34: sync 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:4/28: sync 2024-04-25T01:35:53.960 INFO:tasks.workunit.client.1.smithi138.stdout:2/82: sync 2024-04-25T01:35:53.961 INFO:tasks.workunit.client.0.smithi033.stdout:0/490: creat d7/d2d/d4d/d9c/fae x:0 0 0 2024-04-25T01:35:53.962 INFO:tasks.workunit.client.1.smithi138.stdout:0/12: chown f0 1481 1 2024-04-25T01:35:53.963 INFO:tasks.workunit.client.0.smithi033.stdout:9/539: symlink d0/d2/dd/dba/dbe/lc8 0 2024-04-25T01:35:53.963 INFO:tasks.workunit.client.0.smithi033.stdout:0/491: stat d7/d2d/d43 0 2024-04-25T01:35:53.963 INFO:tasks.workunit.client.1.smithi138.stdout:6/35: unlink f7 0 2024-04-25T01:35:53.963 INFO:tasks.workunit.client.1.smithi138.stdout:4/29: rename d0 to d0/d6/d7 22 2024-04-25T01:35:53.963 INFO:tasks.workunit.client.1.smithi138.stdout:2/83: chown d0/f10 44 1 2024-04-25T01:35:53.964 INFO:tasks.workunit.client.0.smithi033.stdout:1/393: dwrite d2/f26 [4194304,4194304] 0 2024-04-25T01:35:53.964 INFO:tasks.workunit.client.0.smithi033.stdout:0/492: stat d7/d2d/d4c/c85 0 2024-04-25T01:35:53.965 INFO:tasks.workunit.client.1.smithi138.stdout:3/42: creat db/fd x:0 0 0 2024-04-25T01:35:53.966 INFO:tasks.workunit.client.0.smithi033.stdout:0/493: chown d7/da/c1a 1975 1 2024-04-25T01:35:53.967 INFO:tasks.workunit.client.1.smithi138.stdout:7/42: creat da/de/f14 x:0 0 0 2024-04-25T01:35:53.967 INFO:tasks.workunit.client.0.smithi033.stdout:1/394: truncate d2/d2e/f73 900371 0 2024-04-25T01:35:53.967 INFO:tasks.workunit.client.0.smithi033.stdout:3/627: dwrite d8/f1b [0,4194304] 0 2024-04-25T01:35:53.968 INFO:tasks.workunit.client.1.smithi138.stdout:1/32: dwrite f2 [0,4194304] 0 2024-04-25T01:35:53.969 INFO:tasks.workunit.client.1.smithi138.stdout:9/35: creat d4/d8/fd x:0 0 0 2024-04-25T01:35:53.970 INFO:tasks.workunit.client.1.smithi138.stdout:9/36: stat d4/c9 0 2024-04-25T01:35:53.971 INFO:tasks.workunit.client.0.smithi033.stdout:0/494: mknod d7/d2d/d4d/d9c/d68/caf 0 2024-04-25T01:35:53.972 INFO:tasks.workunit.client.1.smithi138.stdout:3/43: dread f6 [0,4194304] 0 2024-04-25T01:35:53.972 INFO:tasks.workunit.client.1.smithi138.stdout:6/36: rename l0 to la 0 2024-04-25T01:35:53.973 INFO:tasks.workunit.client.1.smithi138.stdout:3/44: dread - db/fd zero size 2024-04-25T01:35:53.974 INFO:tasks.workunit.client.1.smithi138.stdout:2/84: creat d0/dc/f17 x:0 0 0 2024-04-25T01:35:53.976 INFO:tasks.workunit.client.1.smithi138.stdout:8/23: mkdir d3/d8/da/db 0 2024-04-25T01:35:53.977 INFO:tasks.workunit.client.0.smithi033.stdout:0/495: creat d7/d2d/d4c/da0/fb0 x:0 0 0 2024-04-25T01:35:53.977 INFO:tasks.workunit.client.0.smithi033.stdout:9/540: dwrite d0/d2/dd/df/d1b/d3d/d8f/f98 [0,4194304] 0 2024-04-25T01:35:53.978 INFO:tasks.workunit.client.0.smithi033.stdout:3/628: dwrite d8/f1b [0,4194304] 0 2024-04-25T01:35:53.985 INFO:tasks.workunit.client.1.smithi138.stdout:9/37: unlink f2 0 2024-04-25T01:35:53.987 INFO:tasks.workunit.client.0.smithi033.stdout:9/541: creat d0/d2/fc9 x:0 0 0 2024-04-25T01:35:53.987 INFO:tasks.workunit.client.1.smithi138.stdout:9/38: dread - d4/d8/fc zero size 2024-04-25T01:35:53.987 INFO:tasks.workunit.client.1.smithi138.stdout:9/39: dread - d4/d8/fc zero size 2024-04-25T01:35:53.987 INFO:tasks.workunit.client.1.smithi138.stdout:3/45: creat db/fe x:0 0 0 2024-04-25T01:35:53.988 INFO:tasks.workunit.client.1.smithi138.stdout:8/24: symlink d3/d8/lc 0 2024-04-25T01:35:53.990 INFO:tasks.workunit.client.0.smithi033.stdout:0/496: creat d7/d2d/d4c/da0/da1/da8/fb1 x:0 0 0 2024-04-25T01:35:53.990 INFO:tasks.workunit.client.1.smithi138.stdout:5/18: link l1 l2 0 2024-04-25T01:35:53.992 INFO:tasks.workunit.client.1.smithi138.stdout:8/25: dread f0 [4194304,4194304] 0 2024-04-25T01:35:53.994 INFO:tasks.workunit.client.0.smithi033.stdout:1/395: dread d2/f29 [0,4194304] 0 2024-04-25T01:35:53.994 INFO:tasks.workunit.client.0.smithi033.stdout:0/497: mkdir d7/d2d/d4d/d9c/db2 0 2024-04-25T01:35:53.995 INFO:tasks.workunit.client.1.smithi138.stdout:9/40: chown d4/c6 3726987 1 2024-04-25T01:35:53.996 INFO:tasks.workunit.client.1.smithi138.stdout:0/13: rmdir d4 0 2024-04-25T01:35:53.998 INFO:tasks.workunit.client.1.smithi138.stdout:4/30: rmdir d0/d6 0 2024-04-25T01:35:53.998 INFO:tasks.workunit.client.0.smithi033.stdout:0/498: truncate d7/d33/f36 378229 0 2024-04-25T01:35:53.998 INFO:tasks.workunit.client.1.smithi138.stdout:3/46: mkdir db/df 0 2024-04-25T01:35:53.999 INFO:tasks.workunit.client.0.smithi033.stdout:1/396: mknod d2/d5/d1b/d46/d5f/d63/d64/c78 0 2024-04-25T01:35:53.999 INFO:tasks.workunit.client.0.smithi033.stdout:0/499: write d7/d2d/d4c/da0/fb0 [366958,26366] 0 2024-04-25T01:35:53.999 INFO:tasks.workunit.client.1.smithi138.stdout:4/31: readlink d0/l5 0 2024-04-25T01:35:53.999 INFO:tasks.workunit.client.1.smithi138.stdout:6/37: dwrite f8 [0,4194304] 0 2024-04-25T01:35:53.999 INFO:tasks.workunit.client.1.smithi138.stdout:5/19: creat f3 x:0 0 0 2024-04-25T01:35:54.000 INFO:tasks.workunit.client.0.smithi033.stdout:0/500: stat d7/d2d/d4d/d9c/f84 0 2024-04-25T01:35:54.000 INFO:tasks.workunit.client.0.smithi033.stdout:1/397: truncate d2/d5/d1b/f5a 141796 0 2024-04-25T01:35:54.001 INFO:tasks.workunit.client.1.smithi138.stdout:3/47: fsync f9 0 2024-04-25T01:35:54.001 INFO:tasks.workunit.client.0.smithi033.stdout:0/501: write d7/d2d/d4d/d9c/f59 [736597,101133] 0 2024-04-25T01:35:54.002 INFO:tasks.workunit.client.1.smithi138.stdout:6/38: creat fb x:0 0 0 2024-04-25T01:35:54.003 INFO:tasks.workunit.client.1.smithi138.stdout:5/20: mkdir d4 0 2024-04-25T01:35:54.004 INFO:tasks.workunit.client.0.smithi033.stdout:0/502: truncate d7/d2d/f67 413023 0 2024-04-25T01:35:54.007 INFO:tasks.workunit.client.0.smithi033.stdout:1/398: creat d2/d11/d28/f79 x:0 0 0 2024-04-25T01:35:54.007 INFO:tasks.workunit.client.0.smithi033.stdout:1/399: chown d2/d5/d1b/c3a 42553514 1 2024-04-25T01:35:54.007 INFO:tasks.workunit.client.1.smithi138.stdout:4/32: rename d0/c3 to d0/c8 0 2024-04-25T01:35:54.008 INFO:tasks.workunit.client.0.smithi033.stdout:0/503: creat d7/da/df/d50/fb3 x:0 0 0 2024-04-25T01:35:54.008 INFO:tasks.workunit.client.0.smithi033.stdout:1/400: dread - d2/d11/f6d zero size 2024-04-25T01:35:54.009 INFO:tasks.workunit.client.0.smithi033.stdout:0/504: write d7/f4e [1581836,94802] 0 2024-04-25T01:35:54.009 INFO:tasks.workunit.client.1.smithi138.stdout:5/21: dread - f3 zero size 2024-04-25T01:35:54.011 INFO:tasks.workunit.client.0.smithi033.stdout:0/505: mknod d7/d2d/d43/d92/cb4 0 2024-04-25T01:35:54.012 INFO:tasks.workunit.client.1.smithi138.stdout:5/22: dread - f3 zero size 2024-04-25T01:35:54.013 INFO:tasks.workunit.client.0.smithi033.stdout:0/506: mknod d7/d2d/d4d/cb5 0 2024-04-25T01:35:54.014 INFO:tasks.workunit.client.0.smithi033.stdout:0/507: write d7/d2d/d4d/d9c/d61/f66 [1214003,130343] 0 2024-04-25T01:35:54.015 INFO:tasks.workunit.client.1.smithi138.stdout:4/33: write d0/f4 [1400585,15946] 0 2024-04-25T01:35:54.015 INFO:tasks.workunit.client.1.smithi138.stdout:5/23: stat d4 0 2024-04-25T01:35:54.016 INFO:tasks.workunit.client.0.smithi033.stdout:0/508: write d7/da/df/d50/f7d [8757832,24949] 0 2024-04-25T01:35:54.017 INFO:tasks.workunit.client.1.smithi138.stdout:0/14: dwrite f1 [0,4194304] 0 2024-04-25T01:35:54.017 INFO:tasks.workunit.client.0.smithi033.stdout:0/509: fdatasync d7/d2d/d64/f6d 0 2024-04-25T01:35:54.018 INFO:tasks.workunit.client.1.smithi138.stdout:9/41: dwrite d4/d8/fd [0,4194304] 0 2024-04-25T01:35:54.018 INFO:tasks.workunit.client.1.smithi138.stdout:0/15: chown f1 48017 1 2024-04-25T01:35:54.019 INFO:tasks.workunit.client.0.smithi033.stdout:0/510: write d7/d2d/d64/d51/f52 [1989034,77244] 0 2024-04-25T01:35:54.020 INFO:tasks.workunit.client.0.smithi033.stdout:0/511: fsync d7/da/df/d50/f6b 0 2024-04-25T01:35:54.020 INFO:tasks.workunit.client.1.smithi138.stdout:0/16: write f0 [877673,51794] 0 2024-04-25T01:35:54.020 INFO:tasks.workunit.client.1.smithi138.stdout:7/43: rename da/ff to da/f15 0 2024-04-25T01:35:54.021 INFO:tasks.workunit.client.0.smithi033.stdout:3/629: write d8/dd/d3d/fc8 [6968853,60939] 0 2024-04-25T01:35:54.023 INFO:tasks.workunit.client.0.smithi033.stdout:3/630: write d8/f5c [334979,112611] 0 2024-04-25T01:35:54.023 INFO:tasks.workunit.client.1.smithi138.stdout:5/24: symlink d4/l5 0 2024-04-25T01:35:54.025 INFO:tasks.workunit.client.0.smithi033.stdout:0/512: rename d7/d2d/d4d/d9c/d68/f98 to d7/d2d/d64/d51/fb6 0 2024-04-25T01:35:54.026 INFO:tasks.workunit.client.1.smithi138.stdout:1/33: fdatasync f2 0 2024-04-25T01:35:54.026 INFO:tasks.workunit.client.1.smithi138.stdout:5/25: fdatasync f3 0 2024-04-25T01:35:54.033 INFO:tasks.workunit.client.0.smithi033.stdout:9/542: write d0/d2/dd/df/d1b/d3d/d7e/f66 [885189,42108] 0 2024-04-25T01:35:54.033 INFO:tasks.workunit.client.1.smithi138.stdout:9/42: symlink d4/le 0 2024-04-25T01:35:54.035 INFO:tasks.workunit.client.1.smithi138.stdout:0/17: fsync f0 0 2024-04-25T01:35:54.036 INFO:tasks.workunit.client.1.smithi138.stdout:4/34: fdatasync d0/f4 0 2024-04-25T01:35:54.038 INFO:tasks.workunit.client.1.smithi138.stdout:9/43: chown d4/d8 1865094780 1 2024-04-25T01:35:54.039 INFO:tasks.workunit.client.0.smithi033.stdout:3/631: dwrite d8/dd/d3d/d63/d7c/f70 [0,4194304] 0 2024-04-25T01:35:54.040 INFO:tasks.workunit.client.0.smithi033.stdout:0/513: dwrite d7/d2d/d4d/d9c/fae [0,4194304] 0 2024-04-25T01:35:54.043 INFO:tasks.workunit.client.1.smithi138.stdout:5/26: dwrite f3 [0,4194304] 0 2024-04-25T01:35:54.043 INFO:tasks.workunit.client.1.smithi138.stdout:2/85: rmdir d0 39 2024-04-25T01:35:54.043 INFO:tasks.workunit.client.1.smithi138.stdout:7/44: dwrite f4 [0,4194304] 0 2024-04-25T01:35:54.043 INFO:tasks.workunit.client.0.smithi033.stdout:1/401: write d2/d11/f2a [309036,53738] 0 2024-04-25T01:35:54.044 INFO:tasks.workunit.client.0.smithi033.stdout:1/402: chown d2 49690633 1 2024-04-25T01:35:54.044 INFO:tasks.workunit.client.1.smithi138.stdout:7/45: dread - da/fc zero size 2024-04-25T01:35:54.044 INFO:tasks.workunit.client.1.smithi138.stdout:4/35: truncate d0/f4 1738723 0 2024-04-25T01:35:54.044 INFO:tasks.workunit.client.0.smithi033.stdout:9/543: unlink d0/d2/dd/df/f3a 0 2024-04-25T01:35:54.046 INFO:tasks.workunit.client.1.smithi138.stdout:4/36: truncate d0/f4 2768585 0 2024-04-25T01:35:54.046 INFO:tasks.workunit.client.1.smithi138.stdout:1/34: creat d3/d4/f9 x:0 0 0 2024-04-25T01:35:54.048 INFO:tasks.workunit.client.0.smithi033.stdout:1/403: fdatasync d2/d2e/d34/d60/f74 0 2024-04-25T01:35:54.052 INFO:tasks.workunit.client.0.smithi033.stdout:3/632: unlink d8/dd/d3d/d63/d7c/c85 0 2024-04-25T01:35:54.053 INFO:tasks.workunit.client.0.smithi033.stdout:9/544: unlink d0/d2/dd/df/d1b/d3d/d7e/l93 0 2024-04-25T01:35:54.054 INFO:tasks.workunit.client.1.smithi138.stdout:0/18: symlink l5 0 2024-04-25T01:35:54.054 INFO:tasks.workunit.client.1.smithi138.stdout:0/19: chown f0 505 1 2024-04-25T01:35:54.060 INFO:tasks.workunit.client.0.smithi033.stdout:9/545: write d0/d2/d43/d4b/f9d [941016,30460] 0 2024-04-25T01:35:54.061 INFO:tasks.workunit.client.0.smithi033.stdout:0/514: dwrite d7/da/df/d14/f24 [4194304,4194304] 0 2024-04-25T01:35:54.062 INFO:tasks.workunit.client.1.smithi138.stdout:3/48: rmdir db 39 2024-04-25T01:35:54.064 INFO:tasks.workunit.client.1.smithi138.stdout:7/46: mknod da/c16 0 2024-04-25T01:35:54.066 INFO:tasks.workunit.client.1.smithi138.stdout:3/49: dread f5 [0,4194304] 0 2024-04-25T01:35:54.067 INFO:tasks.workunit.client.0.smithi033.stdout:1/404: dwrite d2/d5/d1b/f40 [0,4194304] 0 2024-04-25T01:35:54.067 INFO:tasks.workunit.client.1.smithi138.stdout:4/37: mknod d0/c9 0 2024-04-25T01:35:54.067 INFO:tasks.workunit.client.1.smithi138.stdout:8/26: truncate f0 1587643 0 2024-04-25T01:35:54.069 INFO:tasks.workunit.client.0.smithi033.stdout:1/405: write d2/d2e/f67 [1313162,33798] 0 2024-04-25T01:35:54.070 INFO:tasks.workunit.client.1.smithi138.stdout:6/39: getdents . 0 2024-04-25T01:35:54.074 INFO:tasks.workunit.client.1.smithi138.stdout:0/20: mkdir d6 0 2024-04-25T01:35:54.078 INFO:tasks.workunit.client.1.smithi138.stdout:6/40: dread f1 [0,4194304] 0 2024-04-25T01:35:54.079 INFO:tasks.workunit.client.0.smithi033.stdout:0/515: dwrite d7/d2d/d4d/d9c/fae [0,4194304] 0 2024-04-25T01:35:54.079 INFO:tasks.workunit.client.1.smithi138.stdout:6/41: dread - fb zero size 2024-04-25T01:35:54.080 INFO:tasks.workunit.client.1.smithi138.stdout:0/21: dread f1 [0,4194304] 0 2024-04-25T01:35:54.082 INFO:tasks.workunit.client.0.smithi033.stdout:9/546: dwrite d0/fa2 [0,4194304] 0 2024-04-25T01:35:54.086 INFO:tasks.workunit.client.0.smithi033.stdout:9/547: mknod d0/d2/dd/d5f/cca 0 2024-04-25T01:35:54.088 INFO:tasks.workunit.client.0.smithi033.stdout:9/548: readlink d0/d2/dd/dba/dbe/lc8 0 2024-04-25T01:35:54.088 INFO:tasks.workunit.client.1.smithi138.stdout:7/47: mknod da/de/c17 0 2024-04-25T01:35:54.089 INFO:tasks.workunit.client.1.smithi138.stdout:0/22: dwrite f0 [0,4194304] 0 2024-04-25T01:35:54.089 INFO:tasks.workunit.client.0.smithi033.stdout:3/633: truncate d8/dd/d3d/d63/d7c/f70 1284444 0 2024-04-25T01:35:54.090 INFO:tasks.workunit.client.0.smithi033.stdout:0/516: dwrite d7/d2d/d64/d51/f89 [0,4194304] 0 2024-04-25T01:35:54.090 INFO:tasks.workunit.client.0.smithi033.stdout:9/549: fsync d0/d2/dd/d5f/f89 0 2024-04-25T01:35:54.099 INFO:tasks.workunit.client.0.smithi033.stdout:1/406: dwrite d2/f32 [0,4194304] 0 2024-04-25T01:35:54.100 INFO:tasks.workunit.client.0.smithi033.stdout:1/407: chown d2/d5/d6c 6175190 1 2024-04-25T01:35:54.101 INFO:tasks.workunit.client.0.smithi033.stdout:3/634: mknod d8/dd/d3d/d63/d7c/d4d/da5/cd8 0 2024-04-25T01:35:54.104 INFO:tasks.workunit.client.0.smithi033.stdout:1/408: truncate d2/d11/f42 5182626 0 2024-04-25T01:35:54.104 INFO:tasks.workunit.client.1.smithi138.stdout:1/35: rmdir d3 39 2024-04-25T01:35:54.105 INFO:tasks.workunit.client.1.smithi138.stdout:0/23: dwrite f0 [0,4194304] 0 2024-04-25T01:35:54.106 INFO:tasks.workunit.client.1.smithi138.stdout:0/24: chown f1 473637954 1 2024-04-25T01:35:54.106 INFO:tasks.workunit.client.1.smithi138.stdout:0/25: stat f1 0 2024-04-25T01:35:54.108 INFO:tasks.workunit.client.0.smithi033.stdout:7/564: sync 2024-04-25T01:35:54.108 INFO:tasks.workunit.client.0.smithi033.stdout:6/626: sync 2024-04-25T01:35:54.108 INFO:tasks.workunit.client.0.smithi033.stdout:2/627: sync 2024-04-25T01:35:54.109 INFO:tasks.workunit.client.0.smithi033.stdout:3/635: getdents d8/dd/d3d/d63/d7c/d3c/d6e/daa 0 2024-04-25T01:35:54.110 INFO:tasks.workunit.client.0.smithi033.stdout:2/628: fdatasync da/fc2 0 2024-04-25T01:35:54.111 INFO:tasks.workunit.client.0.smithi033.stdout:6/627: symlink d4/d2a/ldb 0 2024-04-25T01:35:54.111 INFO:tasks.workunit.client.0.smithi033.stdout:8/616: sync 2024-04-25T01:35:54.112 INFO:tasks.workunit.client.0.smithi033.stdout:4/625: sync 2024-04-25T01:35:54.112 INFO:tasks.workunit.client.0.smithi033.stdout:5/599: sync 2024-04-25T01:35:54.114 INFO:tasks.workunit.client.0.smithi033.stdout:8/617: write d6/db/f61 [1385115,107281] 0 2024-04-25T01:35:54.115 INFO:tasks.workunit.client.1.smithi138.stdout:9/44: getdents d4/d8 0 2024-04-25T01:35:54.116 INFO:tasks.workunit.client.0.smithi033.stdout:7/565: mkdir d4/d2e/d6b/da9/db3 0 2024-04-25T01:35:54.118 INFO:tasks.workunit.client.1.smithi138.stdout:5/27: link l2 d4/l6 0 2024-04-25T01:35:54.122 INFO:tasks.workunit.client.1.smithi138.stdout:1/36: dread - d3/d4/f9 zero size 2024-04-25T01:35:54.123 INFO:tasks.workunit.client.1.smithi138.stdout:3/50: symlink db/df/l10 0 2024-04-25T01:35:54.125 INFO:tasks.workunit.client.0.smithi033.stdout:5/600: mknod d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/ccc 0 2024-04-25T01:35:54.126 INFO:tasks.workunit.client.1.smithi138.stdout:9/45: rename d4/c9 to d4/cf 0 2024-04-25T01:35:54.127 INFO:tasks.workunit.client.0.smithi033.stdout:7/566: readlink d4/d23/d3f/d45/d6c/l72 0 2024-04-25T01:35:54.127 INFO:tasks.workunit.client.1.smithi138.stdout:5/28: dwrite f3 [0,4194304] 0 2024-04-25T01:35:54.128 INFO:tasks.workunit.client.0.smithi033.stdout:3/636: dwrite d8/dd/d3d/d63/d7c/f96 [4194304,4194304] 0 2024-04-25T01:35:54.131 INFO:tasks.workunit.client.1.smithi138.stdout:0/26: getdents d6 0 2024-04-25T01:35:54.132 INFO:tasks.workunit.client.0.smithi033.stdout:7/567: fsync d4/d2e/d6b/d1f/f42 0 2024-04-25T01:35:54.132 INFO:tasks.workunit.client.0.smithi033.stdout:3/637: fdatasync d8/dd/d11/fad 0 2024-04-25T01:35:54.133 INFO:tasks.workunit.client.0.smithi033.stdout:2/629: dwrite da/f45 [0,4194304] 0 2024-04-25T01:35:54.135 INFO:tasks.workunit.client.0.smithi033.stdout:3/638: chown d8/dd/d40/f90 1 1 2024-04-25T01:35:54.136 INFO:tasks.workunit.client.1.smithi138.stdout:2/86: getdents d0 0 2024-04-25T01:35:54.137 INFO:tasks.workunit.client.0.smithi033.stdout:2/630: write da/d2c/f33 [2152306,31227] 0 2024-04-25T01:35:54.137 INFO:tasks.workunit.client.0.smithi033.stdout:7/568: readlink d4/d10/la4 0 2024-04-25T01:35:54.137 INFO:tasks.workunit.client.1.smithi138.stdout:1/37: unlink f2 0 2024-04-25T01:35:54.137 INFO:tasks.workunit.client.1.smithi138.stdout:5/29: dwrite f3 [0,4194304] 0 2024-04-25T01:35:54.139 INFO:tasks.workunit.client.0.smithi033.stdout:2/631: truncate da/fc2 69438 0 2024-04-25T01:35:54.140 INFO:tasks.workunit.client.0.smithi033.stdout:0/517: write d7/da/df/d14/f69 [1711791,86687] 0 2024-04-25T01:35:54.140 INFO:tasks.workunit.client.1.smithi138.stdout:9/46: symlink d4/d8/l10 0 2024-04-25T01:35:54.140 INFO:tasks.workunit.client.1.smithi138.stdout:5/30: write f3 [4573095,97090] 0 2024-04-25T01:35:54.141 INFO:tasks.workunit.client.0.smithi033.stdout:1/409: rename d2/d11/f42 to d2/d2e/d34/d60/f7a 0 2024-04-25T01:35:54.141 INFO:tasks.workunit.client.1.smithi138.stdout:1/38: truncate d3/d4/f9 240315 0 2024-04-25T01:35:54.143 INFO:tasks.workunit.client.0.smithi033.stdout:7/569: symlink d4/d2e/d6b/d1f/d78/lb4 0 2024-04-25T01:35:54.146 INFO:tasks.workunit.client.0.smithi033.stdout:3/639: unlink d8/dd/d3d/d4f/c58 0 2024-04-25T01:35:54.147 INFO:tasks.workunit.client.0.smithi033.stdout:7/570: truncate d4/d23/d3f/d45/d73/d8f/fb0 560842 0 2024-04-25T01:35:54.149 INFO:tasks.workunit.client.1.smithi138.stdout:5/31: dread f3 [4194304,4194304] 0 2024-04-25T01:35:54.150 INFO:tasks.workunit.client.1.smithi138.stdout:4/38: truncate d0/f4 2397276 0 2024-04-25T01:35:54.150 INFO:tasks.workunit.client.1.smithi138.stdout:4/39: chown d0 60 1 2024-04-25T01:35:54.151 INFO:tasks.workunit.client.0.smithi033.stdout:1/410: rmdir d2/d5/d1b 39 2024-04-25T01:35:54.153 INFO:tasks.workunit.client.1.smithi138.stdout:2/87: mkdir d0/d18 0 2024-04-25T01:35:54.154 INFO:tasks.workunit.client.1.smithi138.stdout:0/27: truncate f1 3910983 0 2024-04-25T01:35:54.155 INFO:tasks.workunit.client.0.smithi033.stdout:3/640: mknod d8/dd/d40/cd9 0 2024-04-25T01:35:54.156 INFO:tasks.workunit.client.0.smithi033.stdout:9/550: dread d0/d2/dd/df/d1b/f4d [0,4194304] 0 2024-04-25T01:35:54.156 INFO:tasks.workunit.client.1.smithi138.stdout:6/42: truncate f6 2481675 0 2024-04-25T01:35:54.158 INFO:tasks.workunit.client.1.smithi138.stdout:7/48: truncate da/f15 406879 0 2024-04-25T01:35:54.159 INFO:tasks.workunit.client.0.smithi033.stdout:4/626: dwrite d6/d12/f32 [0,4194304] 0 2024-04-25T01:35:54.159 INFO:tasks.workunit.client.0.smithi033.stdout:7/571: unlink d4/d2e/d6b/d1f/c80 0 2024-04-25T01:35:54.159 INFO:tasks.workunit.client.0.smithi033.stdout:8/618: dread d6/d45/d52/d2e/f56 [0,4194304] 0 2024-04-25T01:35:54.161 INFO:tasks.workunit.client.0.smithi033.stdout:2/632: creat da/d27/fd2 x:0 0 0 2024-04-25T01:35:54.163 INFO:tasks.workunit.client.1.smithi138.stdout:2/88: dwrite d0/f8 [0,4194304] 0 2024-04-25T01:35:54.163 INFO:tasks.workunit.client.0.smithi033.stdout:7/572: read d4/d2e/f1c [294231,85563] 0 2024-04-25T01:35:54.163 INFO:tasks.workunit.client.1.smithi138.stdout:1/39: mkdir d3/d4/da 0 2024-04-25T01:35:54.165 INFO:tasks.workunit.client.1.smithi138.stdout:2/89: chown d0/dc/f13 0 1 2024-04-25T01:35:54.167 INFO:tasks.workunit.client.1.smithi138.stdout:2/90: write d0/dc/f12 [2590141,23434] 0 2024-04-25T01:35:54.169 INFO:tasks.workunit.client.1.smithi138.stdout:1/40: truncate d3/d4/f9 381823 0 2024-04-25T01:35:54.170 INFO:tasks.workunit.client.0.smithi033.stdout:8/619: chown d6/d45/d51/d9e/lba 60834 1 2024-04-25T01:35:54.170 INFO:tasks.workunit.client.0.smithi033.stdout:8/620: dread - d6/d45/f76 zero size 2024-04-25T01:35:54.171 INFO:tasks.workunit.client.0.smithi033.stdout:1/411: readlink d2/d11/d28/l5b 0 2024-04-25T01:35:54.171 INFO:tasks.workunit.client.1.smithi138.stdout:6/43: dwrite f5 [0,4194304] 0 2024-04-25T01:35:54.172 INFO:tasks.workunit.client.1.smithi138.stdout:5/32: symlink d4/l7 0 2024-04-25T01:35:54.174 INFO:tasks.workunit.client.0.smithi033.stdout:0/518: creat d7/da/fb7 x:0 0 0 2024-04-25T01:35:54.175 INFO:tasks.workunit.client.1.smithi138.stdout:4/40: creat d0/fa x:0 0 0 2024-04-25T01:35:54.175 INFO:tasks.workunit.client.1.smithi138.stdout:7/49: creat da/f18 x:0 0 0 2024-04-25T01:35:54.175 INFO:tasks.workunit.client.0.smithi033.stdout:7/573: fdatasync d4/fb 0 2024-04-25T01:35:54.175 INFO:tasks.workunit.client.1.smithi138.stdout:1/41: mkdir d3/db 0 2024-04-25T01:35:54.177 INFO:tasks.workunit.client.0.smithi033.stdout:4/627: dread d6/d14/f2b [0,4194304] 0 2024-04-25T01:35:54.177 INFO:tasks.workunit.client.0.smithi033.stdout:7/574: fdatasync d4/d23/d3f/d45/f5e 0 2024-04-25T01:35:54.178 INFO:tasks.workunit.client.1.smithi138.stdout:2/91: mknod d0/dc/c19 0 2024-04-25T01:35:54.178 INFO:tasks.workunit.client.0.smithi033.stdout:9/551: mkdir d0/d2/d43/d73/dc4/dcb 0 2024-04-25T01:35:54.179 INFO:tasks.workunit.client.0.smithi033.stdout:4/628: read d6/d12/f32 [4014363,81836] 0 2024-04-25T01:35:54.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/50: write da/de/f10 [861130,66795] 0 2024-04-25T01:35:54.181 INFO:tasks.workunit.client.0.smithi033.stdout:7/575: write d4/d23/d3f/d45/d6c/f97 [782576,28113] 0 2024-04-25T01:35:54.181 INFO:tasks.workunit.client.1.smithi138.stdout:7/51: dread - da/de/f14 zero size 2024-04-25T01:35:54.182 INFO:tasks.workunit.client.0.smithi033.stdout:1/412: rmdir d2/d11/d28 39 2024-04-25T01:35:54.184 INFO:tasks.workunit.client.0.smithi033.stdout:7/576: write d4/d57/fae [1035170,100038] 0 2024-04-25T01:35:54.185 INFO:tasks.workunit.client.0.smithi033.stdout:0/519: unlink d7/da/df/d50/f7d 0 2024-04-25T01:35:54.186 INFO:tasks.workunit.client.1.smithi138.stdout:0/28: mknod d6/c7 0 2024-04-25T01:35:54.187 INFO:tasks.workunit.client.0.smithi033.stdout:7/577: write d4/d2e/d50/f55 [2767749,126895] 0 2024-04-25T01:35:54.188 INFO:tasks.workunit.client.1.smithi138.stdout:4/41: creat d0/fb x:0 0 0 2024-04-25T01:35:54.189 INFO:tasks.workunit.client.0.smithi033.stdout:9/552: dwrite d0/d2/dd/f13 [4194304,4194304] 0 2024-04-25T01:35:54.189 INFO:tasks.workunit.client.1.smithi138.stdout:4/42: truncate d0/fb 13467 0 2024-04-25T01:35:54.190 INFO:tasks.workunit.client.1.smithi138.stdout:8/27: link d3/f5 d3/d8/fd 0 2024-04-25T01:35:54.191 INFO:tasks.workunit.client.1.smithi138.stdout:2/92: dwrite d0/dc/f12 [0,4194304] 0 2024-04-25T01:35:54.192 INFO:tasks.workunit.client.0.smithi033.stdout:9/553: write d0/d2/dd/d20/d4e/f90 [1907020,58504] 0 2024-04-25T01:35:54.192 INFO:tasks.workunit.client.0.smithi033.stdout:8/621: mkdir d6/d45/d51/d55/deb 0 2024-04-25T01:35:54.192 INFO:tasks.workunit.client.1.smithi138.stdout:4/43: truncate d0/fa 207802 0 2024-04-25T01:35:54.192 INFO:tasks.workunit.client.1.smithi138.stdout:2/93: read - d0/dc/f13 zero size 2024-04-25T01:35:54.193 INFO:tasks.workunit.client.0.smithi033.stdout:5/601: dread d1c/f22 [0,4194304] 0 2024-04-25T01:35:54.195 INFO:tasks.workunit.client.0.smithi033.stdout:4/629: mknod d6/d12/d38/dc0/cc3 0 2024-04-25T01:35:54.196 INFO:tasks.workunit.client.0.smithi033.stdout:7/578: creat d4/d2e/d50/fb5 x:0 0 0 2024-04-25T01:35:54.197 INFO:tasks.workunit.client.0.smithi033.stdout:4/630: write d6/d14/d35/d3a/d4c/f9f [231017,67238] 0 2024-04-25T01:35:54.198 INFO:tasks.workunit.client.1.smithi138.stdout:2/94: dread - d0/dc/d15/f16 zero size 2024-04-25T01:35:54.198 INFO:tasks.workunit.client.1.smithi138.stdout:2/95: chown d0/d18 0 1 2024-04-25T01:35:54.201 INFO:tasks.workunit.client.0.smithi033.stdout:9/554: unlink d0/d2/dd/df/f85 0 2024-04-25T01:35:54.202 INFO:tasks.workunit.client.0.smithi033.stdout:9/555: stat d0/d2/d43/d4b/dc3 0 2024-04-25T01:35:54.204 INFO:tasks.workunit.client.0.smithi033.stdout:7/579: chown d4/d2e/d6b/l84 187177393 1 2024-04-25T01:35:54.205 INFO:tasks.workunit.client.1.smithi138.stdout:3/51: truncate f6 1492710 0 2024-04-25T01:35:54.206 INFO:tasks.workunit.client.1.smithi138.stdout:3/52: chown l2 32081854 1 2024-04-25T01:35:54.208 INFO:tasks.workunit.client.0.smithi033.stdout:1/413: getdents d2/d5 0 2024-04-25T01:35:54.209 INFO:tasks.workunit.client.1.smithi138.stdout:0/29: symlink d6/l8 0 2024-04-25T01:35:54.209 INFO:tasks.workunit.client.0.smithi033.stdout:5/602: symlink d1c/d2a/d50/d52/da6/d3e/d70/d7a/lcd 0 2024-04-25T01:35:54.211 INFO:tasks.workunit.client.0.smithi033.stdout:3/641: write d8/dd/f8f [2158586,66743] 0 2024-04-25T01:35:54.211 INFO:tasks.workunit.client.1.smithi138.stdout:8/28: mkdir d3/d8/da/de 0 2024-04-25T01:35:54.212 INFO:tasks.workunit.client.0.smithi033.stdout:9/556: mknod d0/d2/dd/dba/dc1/ccc 0 2024-04-25T01:35:54.214 INFO:tasks.workunit.client.0.smithi033.stdout:8/622: getdents d6/d45/d52/dae 0 2024-04-25T01:35:54.215 INFO:tasks.workunit.client.1.smithi138.stdout:9/47: dwrite f1 [0,4194304] 0 2024-04-25T01:35:54.216 INFO:tasks.workunit.client.0.smithi033.stdout:7/580: rename d4/ca to d4/d23/d3f/d45/d90/cb6 0 2024-04-25T01:35:54.217 INFO:tasks.workunit.client.1.smithi138.stdout:1/42: mknod d3/db/cc 0 2024-04-25T01:35:54.218 INFO:tasks.workunit.client.1.smithi138.stdout:8/29: dread f2 [0,4194304] 0 2024-04-25T01:35:54.219 INFO:tasks.workunit.client.0.smithi033.stdout:1/414: unlink d2/d11/l41 0 2024-04-25T01:35:54.220 INFO:tasks.workunit.client.1.smithi138.stdout:9/48: write d4/d8/fd [4487068,124702] 0 2024-04-25T01:35:54.220 INFO:tasks.workunit.client.1.smithi138.stdout:8/30: readlink d3/d8/lc 0 2024-04-25T01:35:54.220 INFO:tasks.workunit.client.1.smithi138.stdout:3/53: creat db/f11 x:0 0 0 2024-04-25T01:35:54.220 INFO:tasks.workunit.client.1.smithi138.stdout:8/31: read f2 [2984664,13011] 0 2024-04-25T01:35:54.222 INFO:tasks.workunit.client.0.smithi033.stdout:2/633: dwrite da/f12 [0,4194304] 0 2024-04-25T01:35:54.222 INFO:tasks.workunit.client.1.smithi138.stdout:7/52: symlink da/de/d13/l19 0 2024-04-25T01:35:54.223 INFO:tasks.workunit.client.0.smithi033.stdout:2/634: write da/d5b/d5e/f6f [1962440,60605] 0 2024-04-25T01:35:54.224 INFO:tasks.workunit.client.1.smithi138.stdout:0/30: creat d6/f9 x:0 0 0 2024-04-25T01:35:54.225 INFO:tasks.workunit.client.0.smithi033.stdout:8/623: creat d6/d45/d52/dae/fec x:0 0 0 2024-04-25T01:35:54.226 INFO:tasks.workunit.client.1.smithi138.stdout:0/31: write f0 [4739258,32861] 0 2024-04-25T01:35:54.226 INFO:tasks.workunit.client.0.smithi033.stdout:4/631: dread d6/d12/f23 [0,4194304] 0 2024-04-25T01:35:54.229 INFO:tasks.workunit.client.1.smithi138.stdout:2/96: link d0/dc/d15/f16 d0/dc/f1a 0 2024-04-25T01:35:54.230 INFO:tasks.workunit.client.0.smithi033.stdout:4/632: readlink d6/l6c 0 2024-04-25T01:35:54.230 INFO:tasks.workunit.client.1.smithi138.stdout:3/54: dwrite db/fe [0,4194304] 0 2024-04-25T01:35:54.230 INFO:tasks.workunit.client.0.smithi033.stdout:3/642: unlink d8/dd/c84 0 2024-04-25T01:35:54.233 INFO:tasks.workunit.client.1.smithi138.stdout:3/55: dread f9 [0,4194304] 0 2024-04-25T01:35:54.234 INFO:tasks.workunit.client.1.smithi138.stdout:4/44: rmdir d0 39 2024-04-25T01:35:54.236 INFO:tasks.workunit.client.0.smithi033.stdout:3/643: truncate d8/dd/d3d/d63/d7c/d3c/d86/d92/fcd 456498 0 2024-04-25T01:35:54.236 INFO:tasks.workunit.client.0.smithi033.stdout:8/624: symlink d6/d45/d52/db0/dbc/led 0 2024-04-25T01:35:54.237 INFO:tasks.workunit.client.0.smithi033.stdout:2/635: dwrite da/d27/d31/f83 [0,4194304] 0 2024-04-25T01:35:54.239 INFO:tasks.workunit.client.0.smithi033.stdout:4/633: mknod d6/d14/d35/d3a/d4d/d5e/d7c/cc4 0 2024-04-25T01:35:54.239 INFO:tasks.workunit.client.0.smithi033.stdout:8/625: write d6/f60 [4851977,77376] 0 2024-04-25T01:35:54.240 INFO:tasks.workunit.client.0.smithi033.stdout:0/520: write d7/d33/f36 [1379661,13262] 0 2024-04-25T01:35:54.240 INFO:tasks.workunit.client.1.smithi138.stdout:7/53: creat da/de/f1a x:0 0 0 2024-04-25T01:35:54.241 INFO:tasks.workunit.client.1.smithi138.stdout:6/44: truncate f5 809276 0 2024-04-25T01:35:54.244 INFO:tasks.workunit.client.0.smithi033.stdout:4/634: chown d6/d14/d35/d44/d45/d46/f9b 0 1 2024-04-25T01:35:54.245 INFO:tasks.workunit.client.1.smithi138.stdout:5/33: truncate f3 1167411 0 2024-04-25T01:35:54.248 INFO:tasks.workunit.client.0.smithi033.stdout:4/635: mkdir d6/d14/d35/d3a/d4d/d5e/dc5 0 2024-04-25T01:35:54.253 INFO:tasks.workunit.client.0.smithi033.stdout:3/644: rename d8/dd/c91 to d8/dd/d3d/d63/d7c/d3c/d98/cda 0 2024-04-25T01:35:54.254 INFO:tasks.workunit.client.1.smithi138.stdout:4/45: dread d0/fa [0,4194304] 0 2024-04-25T01:35:54.254 INFO:tasks.workunit.client.1.smithi138.stdout:3/56: unlink c8 0 2024-04-25T01:35:54.255 INFO:tasks.workunit.client.0.smithi033.stdout:4/636: dread d6/d14/d1d/f28 [0,4194304] 0 2024-04-25T01:35:54.256 INFO:tasks.workunit.client.1.smithi138.stdout:7/54: mkdir da/de/d1b 0 2024-04-25T01:35:54.257 INFO:tasks.workunit.client.0.smithi033.stdout:4/637: write d6/d14/d1d/d3b/f3e [7827729,129402] 0 2024-04-25T01:35:54.259 INFO:tasks.workunit.client.1.smithi138.stdout:6/45: mkdir dc 0 2024-04-25T01:35:54.259 INFO:tasks.workunit.client.0.smithi033.stdout:7/581: dwrite d4/f7a [0,4194304] 0 2024-04-25T01:35:54.259 INFO:tasks.workunit.client.1.smithi138.stdout:5/34: rmdir d4 39 2024-04-25T01:35:54.262 INFO:tasks.workunit.client.0.smithi033.stdout:7/582: mkdir d4/d57/db7 0 2024-04-25T01:35:54.263 INFO:tasks.workunit.client.1.smithi138.stdout:9/49: write f1 [4947108,111334] 0 2024-04-25T01:35:54.265 INFO:tasks.workunit.client.0.smithi033.stdout:9/557: truncate d0/d2/dd/df/f3e 72267 0 2024-04-25T01:35:54.267 INFO:tasks.workunit.client.0.smithi033.stdout:9/558: chown d0/d2/dd/d20/l83 262 1 2024-04-25T01:35:54.268 INFO:tasks.workunit.client.0.smithi033.stdout:9/559: chown d0/d2/d43/l7a 896374 1 2024-04-25T01:35:54.269 INFO:tasks.workunit.client.1.smithi138.stdout:7/55: rename da/de/f10 to da/de/f1c 0 2024-04-25T01:35:54.269 INFO:tasks.workunit.client.0.smithi033.stdout:4/638: mknod d6/d14/d35/d44/d45/d46/d57/d70/d87/db8/cc6 0 2024-04-25T01:35:54.269 INFO:tasks.workunit.client.0.smithi033.stdout:9/560: write d0/d2/dd/d20/d4e/f78 [929565,57759] 0 2024-04-25T01:35:54.271 INFO:tasks.workunit.client.0.smithi033.stdout:7/583: unlink d4/c5a 0 2024-04-25T01:35:54.273 INFO:tasks.workunit.client.1.smithi138.stdout:9/50: unlink d4/d8/lb 0 2024-04-25T01:35:54.273 INFO:tasks.workunit.client.0.smithi033.stdout:4/639: fsync d6/d14/d48/f99 0 2024-04-25T01:35:54.277 INFO:tasks.workunit.client.0.smithi033.stdout:7/584: dread d4/f17 [0,4194304] 0 2024-04-25T01:35:54.277 INFO:tasks.workunit.client.0.smithi033.stdout:4/640: dread d6/d14/d1d/f28 [0,4194304] 0 2024-04-25T01:35:54.278 INFO:tasks.workunit.client.1.smithi138.stdout:9/51: dread d4/d8/fd [0,4194304] 0 2024-04-25T01:35:54.278 INFO:tasks.workunit.client.1.smithi138.stdout:7/56: unlink da/de/c17 0 2024-04-25T01:35:54.279 INFO:tasks.workunit.client.1.smithi138.stdout:7/57: fdatasync da/f18 0 2024-04-25T01:35:54.280 INFO:tasks.workunit.client.0.smithi033.stdout:8/626: dwrite d6/db/d80/d89/f91 [0,4194304] 0 2024-04-25T01:35:54.284 INFO:tasks.workunit.client.1.smithi138.stdout:9/52: creat d4/f11 x:0 0 0 2024-04-25T01:35:54.285 INFO:tasks.workunit.client.1.smithi138.stdout:8/32: write d3/f5 [2188212,86624] 0 2024-04-25T01:35:54.286 INFO:tasks.workunit.client.1.smithi138.stdout:2/97: dwrite d0/dc/f1a [0,4194304] 0 2024-04-25T01:35:54.286 INFO:tasks.workunit.client.0.smithi033.stdout:8/627: creat d6/d49/fee x:0 0 0 2024-04-25T01:35:54.287 INFO:tasks.workunit.client.1.smithi138.stdout:7/58: symlink da/de/l1d 0 2024-04-25T01:35:54.287 INFO:tasks.workunit.client.1.smithi138.stdout:2/98: chown d0/dc/d15 341395019 1 2024-04-25T01:35:54.289 INFO:tasks.workunit.client.0.smithi033.stdout:7/585: symlink d4/d2e/d6b/d1f/d78/d93/lb8 0 2024-04-25T01:35:54.290 INFO:tasks.workunit.client.1.smithi138.stdout:5/35: rename l2 to d4/l8 0 2024-04-25T01:35:54.293 INFO:tasks.workunit.client.0.smithi033.stdout:4/641: rename d6/d14/d1d/d80 to d6/d14/d35/d44/d45/d46/d57/d70/d87/dc7 0 2024-04-25T01:35:54.293 INFO:tasks.workunit.client.1.smithi138.stdout:8/33: rmdir d3/d8/da 39 2024-04-25T01:35:54.297 INFO:tasks.workunit.client.0.smithi033.stdout:7/586: fsync d4/d2e/d7f/fb2 0 2024-04-25T01:35:54.302 INFO:tasks.workunit.client.1.smithi138.stdout:2/99: rename d0/dc/f13 to d0/d18/f1b 0 2024-04-25T01:35:54.303 INFO:tasks.workunit.client.0.smithi033.stdout:8/628: link d6/db/d80/l8d d6/db/d80/lef 0 2024-04-25T01:35:54.303 INFO:tasks.workunit.client.0.smithi033.stdout:4/642: dwrite d6/d12/d84/f88 [0,4194304] 0 2024-04-25T01:35:54.303 INFO:tasks.workunit.client.0.smithi033.stdout:9/561: dread d0/d2/dd/f52 [0,4194304] 0 2024-04-25T01:35:54.304 INFO:tasks.workunit.client.1.smithi138.stdout:3/57: dwrite f5 [0,4194304] 0 2024-04-25T01:35:54.306 INFO:tasks.workunit.client.0.smithi033.stdout:4/643: mknod d6/d14/d1d/cc8 0 2024-04-25T01:35:54.308 INFO:tasks.workunit.client.0.smithi033.stdout:9/562: creat d0/d2/dd/dba/dc1/fcd x:0 0 0 2024-04-25T01:35:54.309 INFO:tasks.workunit.client.0.smithi033.stdout:9/563: chown d0/d2/dd/d20/fa4 342875924 1 2024-04-25T01:35:54.309 INFO:tasks.workunit.client.0.smithi033.stdout:4/644: chown d6/d14/d35/d44/d45/d46/d57/d70/d87/db8/cc6 23 1 2024-04-25T01:35:54.311 INFO:tasks.workunit.client.0.smithi033.stdout:4/645: chown d6/d14/d35/d3a/d4d/d5e/f76 21 1 2024-04-25T01:35:54.312 INFO:tasks.workunit.client.1.smithi138.stdout:3/58: rename db/fd to db/f12 0 2024-04-25T01:35:54.313 INFO:tasks.workunit.client.1.smithi138.stdout:5/36: read f3 [60992,28601] 0 2024-04-25T01:35:54.315 INFO:tasks.workunit.client.1.smithi138.stdout:8/34: creat d3/d8/da/de/ff x:0 0 0 2024-04-25T01:35:54.316 INFO:tasks.workunit.client.1.smithi138.stdout:8/35: write d3/d8/da/de/ff [899434,48443] 0 2024-04-25T01:35:54.316 INFO:tasks.workunit.client.0.smithi033.stdout:9/564: rename d0/d2/dd/df/f3b to d0/d2/dd/d5f/fce 0 2024-04-25T01:35:54.320 INFO:tasks.workunit.client.0.smithi033.stdout:4/646: symlink d6/d14/d35/lc9 0 2024-04-25T01:35:54.322 INFO:tasks.workunit.client.0.smithi033.stdout:7/587: truncate d4/f9 1769627 0 2024-04-25T01:35:54.323 INFO:tasks.workunit.client.0.smithi033.stdout:9/565: dread d0/f23 [0,4194304] 0 2024-04-25T01:35:54.324 INFO:tasks.workunit.client.0.smithi033.stdout:7/588: symlink d4/d23/d3f/d45/d90/lb9 0 2024-04-25T01:35:54.324 INFO:tasks.workunit.client.1.smithi138.stdout:5/37: unlink l1 0 2024-04-25T01:35:54.326 INFO:tasks.workunit.client.1.smithi138.stdout:8/36: dwrite f0 [0,4194304] 0 2024-04-25T01:35:54.326 INFO:tasks.workunit.client.1.smithi138.stdout:2/100: link d0/dc/ld d0/l1c 0 2024-04-25T01:35:54.327 INFO:tasks.workunit.client.0.smithi033.stdout:7/589: creat d4/d23/d3f/d45/d73/d8f/fba x:0 0 0 2024-04-25T01:35:54.328 INFO:tasks.workunit.client.1.smithi138.stdout:5/38: write f3 [187577,112821] 0 2024-04-25T01:35:54.328 INFO:tasks.workunit.client.1.smithi138.stdout:4/46: dwrite d0/f4 [0,4194304] 0 2024-04-25T01:35:54.330 INFO:tasks.workunit.client.0.smithi033.stdout:9/566: creat d0/d2/dd/df/d1b/fcf x:0 0 0 2024-04-25T01:35:54.330 INFO:tasks.workunit.client.1.smithi138.stdout:5/39: chown f3 7153 1 2024-04-25T01:35:54.330 INFO:tasks.workunit.client.1.smithi138.stdout:2/101: mknod d0/c1d 0 2024-04-25T01:35:54.330 INFO:tasks.workunit.client.1.smithi138.stdout:5/40: stat d4/l7 0 2024-04-25T01:35:54.331 INFO:tasks.workunit.client.1.smithi138.stdout:8/37: creat d3/d8/da/de/f10 x:0 0 0 2024-04-25T01:35:54.331 INFO:tasks.workunit.client.0.smithi033.stdout:7/590: mkdir d4/d2e/d6d/dbb 0 2024-04-25T01:35:54.332 INFO:tasks.workunit.client.1.smithi138.stdout:8/38: chown d3/f5 63274246 1 2024-04-25T01:35:54.333 INFO:tasks.workunit.client.1.smithi138.stdout:5/41: dread f3 [0,4194304] 0 2024-04-25T01:35:54.337 INFO:tasks.workunit.client.1.smithi138.stdout:2/102: creat d0/d18/f1e x:0 0 0 2024-04-25T01:35:54.339 INFO:tasks.workunit.client.0.smithi033.stdout:7/591: unlink d4/d2e/d6b/l84 0 2024-04-25T01:35:54.339 INFO:tasks.workunit.client.0.smithi033.stdout:9/567: dwrite d0/d2/dd/d20/d4e/f78 [0,4194304] 0 2024-04-25T01:35:54.340 INFO:tasks.workunit.client.1.smithi138.stdout:3/59: truncate f1 950061 0 2024-04-25T01:35:54.342 INFO:tasks.workunit.client.1.smithi138.stdout:2/103: dread d0/dc/d15/f16 [0,4194304] 0 2024-04-25T01:35:54.343 INFO:tasks.workunit.client.0.smithi033.stdout:4/647: write d6/d14/d6d/fa8 [812986,5178] 0 2024-04-25T01:35:54.343 INFO:tasks.workunit.client.0.smithi033.stdout:9/568: chown d0/d11/da1 8630 1 2024-04-25T01:35:54.344 INFO:tasks.workunit.client.1.smithi138.stdout:8/39: creat d3/d8/da/db/f11 x:0 0 0 2024-04-25T01:35:54.345 INFO:tasks.workunit.client.0.smithi033.stdout:9/569: readlink d0/d2/dd/df/d1b/d3d/d8f/lb8 0 2024-04-25T01:35:54.345 INFO:tasks.workunit.client.1.smithi138.stdout:4/47: getdents d0 0 2024-04-25T01:35:54.347 INFO:tasks.workunit.client.1.smithi138.stdout:3/60: symlink db/df/l13 0 2024-04-25T01:35:54.348 INFO:tasks.workunit.client.0.smithi033.stdout:4/648: fdatasync d6/d12/f1a 0 2024-04-25T01:35:54.348 INFO:tasks.workunit.client.0.smithi033.stdout:4/649: chown c1 394098 1 2024-04-25T01:35:54.349 INFO:tasks.workunit.client.0.smithi033.stdout:7/592: dwrite d4/f41 [4194304,4194304] 0 2024-04-25T01:35:54.352 INFO:tasks.workunit.client.1.smithi138.stdout:2/104: write d0/dc/f1a [2061555,31532] 0 2024-04-25T01:35:54.354 INFO:tasks.workunit.client.1.smithi138.stdout:8/40: mknod d3/c12 0 2024-04-25T01:35:54.355 INFO:tasks.workunit.client.0.smithi033.stdout:7/593: creat d4/d2e/d6b/fbc x:0 0 0 2024-04-25T01:35:54.355 INFO:tasks.workunit.client.1.smithi138.stdout:8/41: write f0 [2095150,22071] 0 2024-04-25T01:35:54.357 INFO:tasks.workunit.client.1.smithi138.stdout:3/61: mkdir db/d14 0 2024-04-25T01:35:54.360 INFO:tasks.workunit.client.0.smithi033.stdout:7/594: rename d4/d10/l4b to d4/d10/lbd 0 2024-04-25T01:35:54.361 INFO:tasks.workunit.client.1.smithi138.stdout:8/42: creat d3/d8/da/de/f13 x:0 0 0 2024-04-25T01:35:54.361 INFO:tasks.workunit.client.1.smithi138.stdout:2/105: dread d0/f14 [0,4194304] 0 2024-04-25T01:35:54.362 INFO:tasks.workunit.client.0.smithi033.stdout:7/595: mkdir d4/d23/d3f/d45/d73/dbe 0 2024-04-25T01:35:54.363 INFO:tasks.workunit.client.1.smithi138.stdout:3/62: creat db/f15 x:0 0 0 2024-04-25T01:35:54.365 INFO:tasks.workunit.client.0.smithi033.stdout:9/570: write d0/d2/dd/f54 [853122,123363] 0 2024-04-25T01:35:54.370 INFO:tasks.workunit.client.0.smithi033.stdout:9/571: creat d0/d2/dd/d5f/fd0 x:0 0 0 2024-04-25T01:35:54.370 INFO:tasks.workunit.client.1.smithi138.stdout:8/43: dwrite f0 [0,4194304] 0 2024-04-25T01:35:54.371 INFO:tasks.workunit.client.1.smithi138.stdout:3/63: mknod db/df/c16 0 2024-04-25T01:35:54.372 INFO:tasks.workunit.client.0.smithi033.stdout:4/650: dwrite d6/d14/d1d/f28 [0,4194304] 0 2024-04-25T01:35:54.372 INFO:tasks.workunit.client.1.smithi138.stdout:2/106: dread d0/dc/f12 [0,4194304] 0 2024-04-25T01:35:54.373 INFO:tasks.workunit.client.1.smithi138.stdout:8/44: write f0 [1165013,85394] 0 2024-04-25T01:35:54.374 INFO:tasks.workunit.client.1.smithi138.stdout:8/45: readlink d3/d8/lc 0 2024-04-25T01:35:54.375 INFO:tasks.workunit.client.0.smithi033.stdout:4/651: symlink d6/d14/d35/lca 0 2024-04-25T01:35:54.376 INFO:tasks.workunit.client.0.smithi033.stdout:4/652: readlink d6/d14/d35/lc9 0 2024-04-25T01:35:54.381 INFO:tasks.workunit.client.1.smithi138.stdout:3/64: mkdir db/df/d17 0 2024-04-25T01:35:54.382 INFO:tasks.workunit.client.0.smithi033.stdout:4/653: symlink d6/d14/lcb 0 2024-04-25T01:35:54.383 INFO:tasks.workunit.client.1.smithi138.stdout:3/65: read - db/f11 zero size 2024-04-25T01:35:54.383 INFO:tasks.workunit.client.1.smithi138.stdout:2/107: chown d0/l1c 106880 1 2024-04-25T01:35:54.385 INFO:tasks.workunit.client.1.smithi138.stdout:3/66: write db/f11 [973138,43872] 0 2024-04-25T01:35:54.385 INFO:tasks.workunit.client.0.smithi033.stdout:4/654: mkdir d6/d14/d35/d44/d45/d46/db0/dcc 0 2024-04-25T01:35:54.387 INFO:tasks.workunit.client.1.smithi138.stdout:8/46: dwrite d3/d8/fd [0,4194304] 0 2024-04-25T01:35:54.389 INFO:tasks.workunit.client.1.smithi138.stdout:2/108: unlink d0/cb 0 2024-04-25T01:35:54.391 INFO:tasks.workunit.client.1.smithi138.stdout:3/67: symlink db/df/l18 0 2024-04-25T01:35:54.393 INFO:tasks.workunit.client.0.smithi033.stdout:7/596: dread d4/d2e/d65/f68 [0,4194304] 0 2024-04-25T01:35:54.394 INFO:tasks.workunit.client.0.smithi033.stdout:7/597: stat d4/d23/d3f/d45/d6c 0 2024-04-25T01:35:54.395 INFO:tasks.workunit.client.1.smithi138.stdout:8/47: dread d3/d8/da/de/ff [0,4194304] 0 2024-04-25T01:35:54.395 INFO:tasks.workunit.client.0.smithi033.stdout:7/598: write d4/d2e/d50/fb5 [525785,112849] 0 2024-04-25T01:35:54.396 INFO:tasks.workunit.client.1.smithi138.stdout:3/68: dread f9 [0,4194304] 0 2024-04-25T01:35:54.398 INFO:tasks.workunit.client.1.smithi138.stdout:3/69: dread - db/f12 zero size 2024-04-25T01:35:54.399 INFO:tasks.workunit.client.0.smithi033.stdout:7/599: mkdir d4/d2e/d7f/d98/dbf 0 2024-04-25T01:35:54.400 INFO:tasks.workunit.client.1.smithi138.stdout:4/48: dwrite d0/fa [0,4194304] 0 2024-04-25T01:35:54.403 INFO:tasks.workunit.client.1.smithi138.stdout:2/109: getdents d0/dc 0 2024-04-25T01:35:54.405 INFO:tasks.workunit.client.1.smithi138.stdout:3/70: mkdir db/df/d19 0 2024-04-25T01:35:54.409 INFO:tasks.workunit.client.1.smithi138.stdout:3/71: chown db/f15 2 1 2024-04-25T01:35:54.410 INFO:tasks.workunit.client.1.smithi138.stdout:2/110: mkdir d0/dc/d15/d1f 0 2024-04-25T01:35:54.413 INFO:tasks.workunit.client.1.smithi138.stdout:8/48: dwrite d3/d8/da/de/f10 [0,4194304] 0 2024-04-25T01:35:54.415 INFO:tasks.workunit.client.1.smithi138.stdout:2/111: symlink d0/l20 0 2024-04-25T01:35:54.415 INFO:tasks.workunit.client.1.smithi138.stdout:4/49: dwrite d0/f4 [0,4194304] 0 2024-04-25T01:35:54.416 INFO:tasks.workunit.client.1.smithi138.stdout:8/49: write d3/d8/fd [5026202,79722] 0 2024-04-25T01:35:54.416 INFO:tasks.workunit.client.1.smithi138.stdout:2/112: stat d0/dc/f12 0 2024-04-25T01:35:54.417 INFO:tasks.workunit.client.1.smithi138.stdout:3/72: mkdir db/df/d19/d1a 0 2024-04-25T01:35:54.419 INFO:tasks.workunit.client.1.smithi138.stdout:4/50: write d0/f4 [1236663,3168] 0 2024-04-25T01:35:54.419 INFO:tasks.workunit.client.1.smithi138.stdout:8/50: write d3/d8/da/de/f10 [1565432,30305] 0 2024-04-25T01:35:54.422 INFO:tasks.workunit.client.1.smithi138.stdout:8/51: fdatasync f0 0 2024-04-25T01:35:54.428 INFO:tasks.workunit.client.1.smithi138.stdout:4/51: dwrite d0/fa [0,4194304] 0 2024-04-25T01:35:54.431 INFO:tasks.workunit.client.1.smithi138.stdout:8/52: creat d3/d8/da/db/f14 x:0 0 0 2024-04-25T01:35:54.432 INFO:tasks.workunit.client.1.smithi138.stdout:8/53: readlink d3/l9 0 2024-04-25T01:35:54.433 INFO:tasks.workunit.client.1.smithi138.stdout:8/54: dread - d3/d8/da/de/f13 zero size 2024-04-25T01:35:54.434 INFO:tasks.workunit.client.0.smithi033.stdout:7/600: dread d4/f32 [0,4194304] 0 2024-04-25T01:35:54.437 INFO:tasks.workunit.client.1.smithi138.stdout:4/52: rename d0/fb to d0/fc 0 2024-04-25T01:35:54.440 INFO:tasks.workunit.client.1.smithi138.stdout:2/113: rmdir d0 39 2024-04-25T01:35:54.442 INFO:tasks.workunit.client.1.smithi138.stdout:4/53: unlink d0/c2 0 2024-04-25T01:35:54.442 INFO:tasks.workunit.client.1.smithi138.stdout:8/55: dread f0 [0,4194304] 0 2024-04-25T01:35:54.444 INFO:tasks.workunit.client.1.smithi138.stdout:4/54: chown d0/l5 8024621 1 2024-04-25T01:35:54.444 INFO:tasks.workunit.client.1.smithi138.stdout:8/56: dread f0 [4194304,4194304] 0 2024-04-25T01:35:54.444 INFO:tasks.workunit.client.1.smithi138.stdout:2/114: write d0/f14 [3147633,59648] 0 2024-04-25T01:35:54.445 INFO:tasks.workunit.client.1.smithi138.stdout:2/115: chown d0/dc/d15/f16 3272 1 2024-04-25T01:35:54.447 INFO:tasks.workunit.client.1.smithi138.stdout:2/116: write d0/f10 [7374229,125628] 0 2024-04-25T01:35:54.447 INFO:tasks.workunit.client.1.smithi138.stdout:8/57: mkdir d3/d8/da/db/d15 0 2024-04-25T01:35:54.448 INFO:tasks.workunit.client.1.smithi138.stdout:8/58: chown d3/d8 317 1 2024-04-25T01:35:54.450 INFO:tasks.workunit.client.1.smithi138.stdout:2/117: mkdir d0/d21 0 2024-04-25T01:35:54.450 INFO:tasks.workunit.client.0.smithi033.stdout:7/601: write d4/d23/d3f/f6f [736789,123242] 0 2024-04-25T01:35:54.453 INFO:tasks.workunit.client.0.smithi033.stdout:7/602: rename d4/d2e/c49 to d4/d2e/d6b/d1f/d78/cc0 0 2024-04-25T01:35:54.453 INFO:tasks.workunit.client.0.smithi033.stdout:7/603: stat d4/d2e/c4e 0 2024-04-25T01:35:54.456 INFO:tasks.workunit.client.1.smithi138.stdout:2/118: write d0/d18/f1b [253254,316] 0 2024-04-25T01:35:54.457 INFO:tasks.workunit.client.1.smithi138.stdout:3/73: write f1 [328304,68833] 0 2024-04-25T01:35:54.458 INFO:tasks.workunit.client.1.smithi138.stdout:8/59: dwrite d3/d8/da/db/f11 [0,4194304] 0 2024-04-25T01:35:54.459 INFO:tasks.workunit.client.1.smithi138.stdout:2/119: creat d0/d21/f22 x:0 0 0 2024-04-25T01:35:54.462 INFO:tasks.workunit.client.1.smithi138.stdout:8/60: truncate f0 5382144 0 2024-04-25T01:35:54.464 INFO:tasks.workunit.client.0.smithi033.stdout:7/604: dread d4/d2e/d6b/d1f/f46 [0,4194304] 0 2024-04-25T01:35:54.465 INFO:tasks.workunit.client.0.smithi033.stdout:7/605: fdatasync d4/d23/d3f/d45/f5e 0 2024-04-25T01:35:54.467 INFO:tasks.workunit.client.1.smithi138.stdout:8/61: mkdir d3/d16 0 2024-04-25T01:35:54.467 INFO:tasks.workunit.client.1.smithi138.stdout:3/74: dwrite db/f11 [0,4194304] 0 2024-04-25T01:35:54.472 INFO:tasks.workunit.client.1.smithi138.stdout:8/62: symlink d3/l17 0 2024-04-25T01:35:54.474 INFO:tasks.workunit.client.1.smithi138.stdout:3/75: truncate f9 148888 0 2024-04-25T01:35:54.474 INFO:tasks.workunit.client.1.smithi138.stdout:2/120: dread d0/f14 [0,4194304] 0 2024-04-25T01:35:54.478 INFO:tasks.workunit.client.0.smithi033.stdout:7/606: dwrite d4/d2e/d65/f69 [0,4194304] 0 2024-04-25T01:35:54.482 INFO:tasks.workunit.client.0.smithi033.stdout:7/607: creat d4/d23/d3f/d45/d73/dbe/fc1 x:0 0 0 2024-04-25T01:35:54.485 INFO:tasks.workunit.client.0.smithi033.stdout:7/608: rename d4/d2e/d6b/d1f/d78/d93 to d4/d23/da7/dc2 0 2024-04-25T01:35:54.485 INFO:tasks.workunit.client.0.smithi033.stdout:7/609: fsync d4/d2e/d7f/fb2 0 2024-04-25T01:35:54.487 INFO:tasks.workunit.client.0.smithi033.stdout:7/610: chown d4/d2e/d6b/c59 739 1 2024-04-25T01:35:54.490 INFO:tasks.workunit.client.0.smithi033.stdout:7/611: mknod d4/d2e/d65/cc3 0 2024-04-25T01:35:54.503 INFO:tasks.workunit.client.0.smithi033.stdout:7/612: dread d4/d2e/d7f/fb2 [0,4194304] 0 2024-04-25T01:35:54.560 INFO:tasks.workunit.client.1.smithi138.stdout:8/63: rename d3/l9 to d3/d8/da/de/l18 0 2024-04-25T01:35:54.560 INFO:tasks.workunit.client.1.smithi138.stdout:3/76: rmdir db 39 2024-04-25T01:35:54.560 INFO:tasks.workunit.client.1.smithi138.stdout:2/121: mknod d0/c23 0 2024-04-25T01:35:54.561 INFO:tasks.workunit.client.1.smithi138.stdout:3/77: dread - db/f15 zero size 2024-04-25T01:35:54.563 INFO:tasks.workunit.client.1.smithi138.stdout:8/64: creat d3/f19 x:0 0 0 2024-04-25T01:35:54.563 INFO:tasks.workunit.client.1.smithi138.stdout:8/65: stat f0 0 2024-04-25T01:35:54.563 INFO:tasks.workunit.client.1.smithi138.stdout:2/122: symlink d0/d18/l24 0 2024-04-25T01:35:54.565 INFO:tasks.workunit.client.1.smithi138.stdout:8/66: rmdir d3/d8 39 2024-04-25T01:35:54.565 INFO:tasks.workunit.client.1.smithi138.stdout:3/78: dread f1 [0,4194304] 0 2024-04-25T01:35:54.569 INFO:tasks.workunit.client.1.smithi138.stdout:8/67: rename d3/d8/fd to d3/d8/da/db/f1a 0 2024-04-25T01:35:54.570 INFO:tasks.workunit.client.1.smithi138.stdout:2/123: link d0/dc/ld d0/dc/d15/l25 0 2024-04-25T01:35:54.574 INFO:tasks.workunit.client.1.smithi138.stdout:3/79: link l4 db/df/d19/l1b 0 2024-04-25T01:35:54.575 INFO:tasks.workunit.client.1.smithi138.stdout:2/124: link d0/f8 d0/dc/d15/d1f/f26 0 2024-04-25T01:35:54.576 INFO:tasks.workunit.client.1.smithi138.stdout:3/80: getdents db/df/d19/d1a 0 2024-04-25T01:35:54.579 INFO:tasks.workunit.client.1.smithi138.stdout:8/68: rmdir d3 39 2024-04-25T01:35:54.582 INFO:tasks.workunit.client.1.smithi138.stdout:8/69: mkdir d3/d8/da/d1b 0 2024-04-25T01:35:54.583 INFO:tasks.workunit.client.1.smithi138.stdout:8/70: truncate d3/d8/da/de/f13 394839 0 2024-04-25T01:35:54.585 INFO:tasks.workunit.client.1.smithi138.stdout:8/71: creat d3/d16/f1c x:0 0 0 2024-04-25T01:35:54.586 INFO:tasks.workunit.client.1.smithi138.stdout:8/72: readlink d3/d8/da/de/l18 0 2024-04-25T01:35:54.587 INFO:tasks.workunit.client.1.smithi138.stdout:8/73: mknod d3/d16/c1d 0 2024-04-25T01:35:54.589 INFO:tasks.workunit.client.1.smithi138.stdout:8/74: creat d3/f1e x:0 0 0 2024-04-25T01:35:54.590 INFO:tasks.workunit.client.1.smithi138.stdout:8/75: write d3/d8/da/db/f14 [532650,5033] 0 2024-04-25T01:35:54.602 INFO:tasks.workunit.client.1.smithi138.stdout:8/76: dwrite f2 [0,4194304] 0 2024-04-25T01:35:54.608 INFO:tasks.workunit.client.1.smithi138.stdout:8/77: truncate d3/d8/da/db/f14 311340 0 2024-04-25T01:35:54.610 INFO:tasks.workunit.client.0.smithi033.stdout:6/628: sync 2024-04-25T01:35:54.610 INFO:tasks.workunit.client.1.smithi138.stdout:8/78: dread d3/d8/da/de/f13 [0,4194304] 0 2024-04-25T01:35:54.612 INFO:tasks.workunit.client.1.smithi138.stdout:8/79: mkdir d3/d16/d1f 0 2024-04-25T01:35:54.614 INFO:tasks.workunit.client.1.smithi138.stdout:8/80: mkdir d3/d8/da/db/d15/d20 0 2024-04-25T01:35:54.615 INFO:tasks.workunit.client.0.smithi033.stdout:6/629: dwrite d4/d1d/d3e/d41/d5c/dae/fcf [0,4194304] 0 2024-04-25T01:35:54.616 INFO:tasks.workunit.client.0.smithi033.stdout:6/630: readlink d4/d1d/d3e/d41/d5c/d7f/lc5 0 2024-04-25T01:35:54.616 INFO:tasks.workunit.client.0.smithi033.stdout:6/631: fsync d4/d24/d36/d42/fca 0 2024-04-25T01:35:54.618 INFO:tasks.workunit.client.0.smithi033.stdout:6/632: write d4/d1d/d3e/d41/d5c/fb7 [527415,116641] 0 2024-04-25T01:35:54.618 INFO:tasks.workunit.client.0.smithi033.stdout:6/633: stat d4/d15 0 2024-04-25T01:35:54.625 INFO:tasks.workunit.client.1.smithi138.stdout:8/81: dwrite d3/f1e [0,4194304] 0 2024-04-25T01:35:54.628 INFO:tasks.workunit.client.1.smithi138.stdout:8/82: creat d3/d8/da/db/d15/f21 x:0 0 0 2024-04-25T01:35:54.630 INFO:tasks.workunit.client.1.smithi138.stdout:8/83: dread d3/d8/da/de/ff [0,4194304] 0 2024-04-25T01:35:54.634 INFO:tasks.workunit.client.1.smithi138.stdout:8/84: mknod d3/c22 0 2024-04-25T01:35:54.638 INFO:tasks.workunit.client.1.smithi138.stdout:8/85: mknod d3/d16/d1f/c23 0 2024-04-25T01:35:54.641 INFO:tasks.workunit.client.1.smithi138.stdout:8/86: creat d3/d8/da/f24 x:0 0 0 2024-04-25T01:35:54.655 INFO:tasks.workunit.client.1.smithi138.stdout:8/87: dread d3/d8/da/db/f14 [0,4194304] 0 2024-04-25T01:35:54.656 INFO:tasks.workunit.client.1.smithi138.stdout:8/88: write d3/d8/da/de/f10 [3343587,36648] 0 2024-04-25T01:35:54.656 INFO:tasks.workunit.client.1.smithi138.stdout:8/89: stat d3/d8/da/db/f14 0 2024-04-25T01:35:54.658 INFO:tasks.workunit.client.1.smithi138.stdout:8/90: fdatasync d3/d8/da/de/f10 0 2024-04-25T01:35:54.659 INFO:tasks.workunit.client.1.smithi138.stdout:8/91: truncate d3/f19 227298 0 2024-04-25T01:35:54.663 INFO:tasks.workunit.client.0.smithi033.stdout:6/634: mkdir d4/d2a/ddc 0 2024-04-25T01:35:54.663 INFO:tasks.workunit.client.0.smithi033.stdout:6/635: chown d4/d2a/l71 258273642 1 2024-04-25T01:35:54.664 INFO:tasks.workunit.client.1.smithi138.stdout:8/92: truncate d3/f1e 1145327 0 2024-04-25T01:35:54.664 INFO:tasks.workunit.client.0.smithi033.stdout:6/636: truncate d4/d24/d29/f55 400721 0 2024-04-25T01:35:54.665 INFO:tasks.workunit.client.1.smithi138.stdout:8/93: creat d3/d8/da/db/d15/f25 x:0 0 0 2024-04-25T01:35:54.665 INFO:tasks.workunit.client.0.smithi033.stdout:6/637: creat d4/d24/d36/d42/d9d/fdd x:0 0 0 2024-04-25T01:35:54.668 INFO:tasks.workunit.client.1.smithi138.stdout:8/94: rmdir d3/d8/da/d1b 0 2024-04-25T01:35:54.671 INFO:tasks.workunit.client.1.smithi138.stdout:8/95: truncate d3/d8/da/db/f14 356487 0 2024-04-25T01:35:54.674 INFO:tasks.workunit.client.1.smithi138.stdout:8/96: dread f0 [0,4194304] 0 2024-04-25T01:35:54.678 INFO:tasks.workunit.client.1.smithi138.stdout:8/97: dread d3/d8/da/db/f1a [0,4194304] 0 2024-04-25T01:35:54.678 INFO:tasks.workunit.client.1.smithi138.stdout:8/98: fsync d3/d8/da/de/f10 0 2024-04-25T01:35:54.681 INFO:tasks.workunit.client.1.smithi138.stdout:8/99: creat d3/d8/da/db/d15/d20/f26 x:0 0 0 2024-04-25T01:35:54.682 INFO:tasks.workunit.client.1.smithi138.stdout:8/100: rename d3 to d3/d8/d27 22 2024-04-25T01:35:54.683 INFO:tasks.workunit.client.1.smithi138.stdout:8/101: symlink d3/d8/da/db/l28 0 2024-04-25T01:35:54.685 INFO:tasks.workunit.client.1.smithi138.stdout:8/102: mknod d3/d8/da/db/d15/d20/c29 0 2024-04-25T01:35:54.686 INFO:tasks.workunit.client.1.smithi138.stdout:8/103: write d3/d8/da/db/d15/f25 [27360,108121] 0 2024-04-25T01:35:54.687 INFO:tasks.workunit.client.1.smithi138.stdout:8/104: readlink d3/d8/da/de/l18 0 2024-04-25T01:35:54.689 INFO:tasks.workunit.client.1.smithi138.stdout:8/105: mkdir d3/d16/d1f/d2a 0 2024-04-25T01:35:54.693 INFO:tasks.workunit.client.1.smithi138.stdout:8/106: dread d3/f1e [0,4194304] 0 2024-04-25T01:35:54.694 INFO:tasks.workunit.client.1.smithi138.stdout:8/107: read f2 [566589,81924] 0 2024-04-25T01:35:54.698 INFO:tasks.workunit.client.0.smithi033.stdout:5/603: sync 2024-04-25T01:35:54.700 INFO:tasks.workunit.client.0.smithi033.stdout:6/638: dwrite d4/d1d/d3e/d41/f58 [0,4194304] 0 2024-04-25T01:35:54.701 INFO:tasks.workunit.client.1.smithi138.stdout:8/108: rename d3/d8/da/db/d15/f25 to d3/f2b 0 2024-04-25T01:35:54.702 INFO:tasks.workunit.client.0.smithi033.stdout:5/604: creat d1c/d2a/d50/db1/fce x:0 0 0 2024-04-25T01:35:54.706 INFO:tasks.workunit.client.1.smithi138.stdout:8/109: dread f2 [0,4194304] 0 2024-04-25T01:35:54.706 INFO:tasks.workunit.client.0.smithi033.stdout:5/605: unlink d1c/l24 0 2024-04-25T01:35:54.707 INFO:tasks.workunit.client.1.smithi138.stdout:8/110: read d3/f5 [717446,120546] 0 2024-04-25T01:35:54.707 INFO:tasks.workunit.client.0.smithi033.stdout:6/639: dwrite d4/d1d/d3e/d41/f58 [0,4194304] 0 2024-04-25T01:35:54.708 INFO:tasks.workunit.client.1.smithi138.stdout:8/111: truncate d3/f19 499552 0 2024-04-25T01:35:54.709 INFO:tasks.workunit.client.0.smithi033.stdout:5/606: rmdir d1c/d2a/d50/d52/da6/d3e 39 2024-04-25T01:35:54.710 INFO:tasks.workunit.client.1.smithi138.stdout:8/112: write d3/f2b [1172623,97320] 0 2024-04-25T01:35:54.711 INFO:tasks.workunit.client.0.smithi033.stdout:5/607: creat d1c/d53/fcf x:0 0 0 2024-04-25T01:35:54.714 INFO:tasks.workunit.client.0.smithi033.stdout:5/608: creat d1c/d53/fd0 x:0 0 0 2024-04-25T01:35:54.716 INFO:tasks.workunit.client.0.smithi033.stdout:5/609: write f5 [259399,72812] 0 2024-04-25T01:35:54.719 INFO:tasks.workunit.client.1.smithi138.stdout:8/113: dwrite d3/d16/f1c [0,4194304] 0 2024-04-25T01:35:54.720 INFO:tasks.workunit.client.0.smithi033.stdout:5/610: rmdir d1c/d2a/d50/d5b/d86 39 2024-04-25T01:35:54.720 INFO:tasks.workunit.client.1.smithi138.stdout:8/114: fdatasync d3/f19 0 2024-04-25T01:35:54.720 INFO:tasks.workunit.client.0.smithi033.stdout:6/640: dwrite d4/d24/d36/fc0 [0,4194304] 0 2024-04-25T01:35:54.723 INFO:tasks.workunit.client.0.smithi033.stdout:6/641: write d4/d1d/d3e/d41/f58 [2281533,110507] 0 2024-04-25T01:35:54.725 INFO:tasks.workunit.client.1.smithi138.stdout:8/115: rename d3/c12 to d3/d8/da/db/d15/c2c 0 2024-04-25T01:35:54.731 INFO:tasks.workunit.client.0.smithi033.stdout:5/611: rename d1c/d2a/d50/d52/l80 to d1c/d2a/d50/d52/da6/d3e/d70/dae/ld1 0 2024-04-25T01:35:54.732 INFO:tasks.workunit.client.0.smithi033.stdout:5/612: dread - d1c/d2a/fb2 zero size 2024-04-25T01:35:54.733 INFO:tasks.workunit.client.0.smithi033.stdout:5/613: write d1c/d53/f99 [342962,22081] 0 2024-04-25T01:35:54.735 INFO:tasks.workunit.client.0.smithi033.stdout:5/614: chown d1c/d2a/d50/d52/fb3 2 1 2024-04-25T01:35:54.735 INFO:tasks.workunit.client.0.smithi033.stdout:6/642: truncate d4/d1d/d3e/f62 2351607 0 2024-04-25T01:35:54.737 INFO:tasks.workunit.client.0.smithi033.stdout:5/615: symlink d1c/d2a/d50/dcb/ld2 0 2024-04-25T01:35:54.738 INFO:tasks.workunit.client.0.smithi033.stdout:6/643: mkdir d4/d24/d36/dbd/dde 0 2024-04-25T01:35:54.739 INFO:tasks.workunit.client.0.smithi033.stdout:6/644: chown d4/d1d/d3e 0 1 2024-04-25T01:35:54.741 INFO:tasks.workunit.client.0.smithi033.stdout:6/645: symlink d4/d24/d36/d42/d97/ldf 0 2024-04-25T01:35:54.743 INFO:tasks.workunit.client.0.smithi033.stdout:5/616: dwrite d1c/d2a/fb2 [0,4194304] 0 2024-04-25T01:35:54.747 INFO:tasks.workunit.client.1.smithi138.stdout:8/116: dwrite d3/d8/da/db/f14 [0,4194304] 0 2024-04-25T01:35:54.748 INFO:tasks.workunit.client.0.smithi033.stdout:5/617: dread d1c/d2a/f34 [0,4194304] 0 2024-04-25T01:35:54.750 INFO:tasks.workunit.client.1.smithi138.stdout:8/117: mknod d3/c2d 0 2024-04-25T01:35:54.751 INFO:tasks.workunit.client.0.smithi033.stdout:5/618: truncate d1c/d2a/d3d/d59/f9e 844258 0 2024-04-25T01:35:54.753 INFO:tasks.workunit.client.0.smithi033.stdout:5/619: write d1c/d2a/d50/d5b/fa8 [1724515,49800] 0 2024-04-25T01:35:54.754 INFO:tasks.workunit.client.1.smithi138.stdout:8/118: dread f2 [0,4194304] 0 2024-04-25T01:35:54.755 INFO:tasks.workunit.client.0.smithi033.stdout:5/620: mkdir d1c/d2a/d50/db1/dd3 0 2024-04-25T01:35:54.756 INFO:tasks.workunit.client.1.smithi138.stdout:8/119: chown d3/d8/da/db 441791812 1 2024-04-25T01:35:54.756 INFO:tasks.workunit.client.1.smithi138.stdout:8/120: stat d3/d8/da/de/ff 0 2024-04-25T01:35:54.757 INFO:tasks.workunit.client.0.smithi033.stdout:5/621: write d1c/d2a/d50/d52/da6/f63 [70725,34976] 0 2024-04-25T01:35:54.757 INFO:tasks.workunit.client.1.smithi138.stdout:8/121: rmdir d3/d8 39 2024-04-25T01:35:54.758 INFO:tasks.workunit.client.1.smithi138.stdout:8/122: mknod d3/d8/da/db/c2e 0 2024-04-25T01:35:54.759 INFO:tasks.workunit.client.0.smithi033.stdout:5/622: getdents d1c/d2a/d50 0 2024-04-25T01:35:54.775 INFO:tasks.workunit.client.1.smithi138.stdout:1/43: sync 2024-04-25T01:35:54.776 INFO:tasks.workunit.client.1.smithi138.stdout:8/123: write d3/f19 [775329,39902] 0 2024-04-25T01:35:54.782 INFO:tasks.workunit.client.1.smithi138.stdout:1/44: creat d3/d4/fd x:0 0 0 2024-04-25T01:35:54.784 INFO:tasks.workunit.client.0.smithi033.stdout:6/646: write d4/d24/d7c/f80 [2140873,38844] 0 2024-04-25T01:35:54.784 INFO:tasks.workunit.client.0.smithi033.stdout:0/521: sync 2024-04-25T01:35:54.784 INFO:tasks.workunit.client.0.smithi033.stdout:1/415: sync 2024-04-25T01:35:54.785 INFO:tasks.workunit.client.0.smithi033.stdout:2/636: sync 2024-04-25T01:35:54.786 INFO:tasks.workunit.client.1.smithi138.stdout:1/45: truncate d3/d4/fd 813913 0 2024-04-25T01:35:54.786 INFO:tasks.workunit.client.1.smithi138.stdout:8/124: dwrite d3/f2b [0,4194304] 0 2024-04-25T01:35:54.788 INFO:tasks.workunit.client.0.smithi033.stdout:6/647: mkdir d4/d24/d36/d42/d97/de0 0 2024-04-25T01:35:54.789 INFO:tasks.workunit.client.1.smithi138.stdout:1/46: write d3/d4/fd [1655367,78460] 0 2024-04-25T01:35:54.790 INFO:tasks.workunit.client.1.smithi138.stdout:1/47: write d3/d4/fd [1476863,6598] 0 2024-04-25T01:35:54.791 INFO:tasks.workunit.client.0.smithi033.stdout:6/648: truncate d4/d24/f99 181163 0 2024-04-25T01:35:54.792 INFO:tasks.workunit.client.0.smithi033.stdout:1/416: mknod d2/d11/d28/c7b 0 2024-04-25T01:35:54.792 INFO:tasks.workunit.client.1.smithi138.stdout:8/125: rename d3/f2b to d3/d8/da/db/d15/f2f 0 2024-04-25T01:35:54.792 INFO:tasks.workunit.client.0.smithi033.stdout:0/522: dwrite d7/da/df/f99 [0,4194304] 0 2024-04-25T01:35:54.793 INFO:tasks.workunit.client.1.smithi138.stdout:8/126: chown d3/c7 280751871 1 2024-04-25T01:35:54.794 INFO:tasks.workunit.client.0.smithi033.stdout:1/417: write d2/d2e/f73 [929596,54364] 0 2024-04-25T01:35:54.797 INFO:tasks.workunit.client.0.smithi033.stdout:6/649: mkdir d4/d1d/d3e/d41/d5c/dae/de1 0 2024-04-25T01:35:54.798 INFO:tasks.workunit.client.0.smithi033.stdout:5/623: truncate d1c/d2a/fb2 1256558 0 2024-04-25T01:35:54.800 INFO:tasks.workunit.client.1.smithi138.stdout:1/48: dwrite d3/d4/f9 [0,4194304] 0 2024-04-25T01:35:54.802 INFO:tasks.workunit.client.1.smithi138.stdout:8/127: rename d3/d8/da/db/f1a to d3/d8/da/db/d15/f30 0 2024-04-25T01:35:54.802 INFO:tasks.workunit.client.1.smithi138.stdout:8/128: stat d3/d16/f1c 0 2024-04-25T01:35:54.802 INFO:tasks.workunit.client.1.smithi138.stdout:1/49: chown d3/d4/f9 0 1 2024-04-25T01:35:54.803 INFO:tasks.workunit.client.0.smithi033.stdout:0/523: creat d7/d2d/d4d/d9c/fb8 x:0 0 0 2024-04-25T01:35:54.803 INFO:tasks.workunit.client.0.smithi033.stdout:1/418: creat d2/d5/d1b/d46/d5f/d63/f7c x:0 0 0 2024-04-25T01:35:54.804 INFO:tasks.workunit.client.1.smithi138.stdout:8/129: mkdir d3/d16/d1f/d31 0 2024-04-25T01:35:54.805 INFO:tasks.workunit.client.1.smithi138.stdout:1/50: creat d3/db/fe x:0 0 0 2024-04-25T01:35:54.809 INFO:tasks.workunit.client.1.smithi138.stdout:1/51: creat d3/d4/da/ff x:0 0 0 2024-04-25T01:35:54.810 INFO:tasks.workunit.client.1.smithi138.stdout:1/52: creat d3/f10 x:0 0 0 2024-04-25T01:35:54.811 INFO:tasks.workunit.client.1.smithi138.stdout:1/53: chown d3/d4 1101 1 2024-04-25T01:35:54.813 INFO:tasks.workunit.client.0.smithi033.stdout:2/637: dwrite f7 [4194304,4194304] 0 2024-04-25T01:35:54.815 INFO:tasks.workunit.client.0.smithi033.stdout:5/624: dwrite d1c/d2a/d50/f7d [0,4194304] 0 2024-04-25T01:35:54.816 INFO:tasks.workunit.client.0.smithi033.stdout:0/524: truncate d7/d2d/d64/f6d 3915332 0 2024-04-25T01:35:54.817 INFO:tasks.workunit.client.0.smithi033.stdout:0/525: stat d7/da/fb7 0 2024-04-25T01:35:54.818 INFO:tasks.workunit.client.0.smithi033.stdout:0/526: chown d7/da/df/d50/f6b 1 1 2024-04-25T01:35:54.824 INFO:tasks.workunit.client.0.smithi033.stdout:0/527: chown d7/da/df/c39 543 1 2024-04-25T01:35:54.825 INFO:tasks.workunit.client.0.smithi033.stdout:1/419: creat d2/d11/d28/f7d x:0 0 0 2024-04-25T01:35:54.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/54: fsync d3/d4/fd 0 2024-04-25T01:35:54.831 INFO:tasks.workunit.client.0.smithi033.stdout:5/625: dread d1c/f1f [0,4194304] 0 2024-04-25T01:35:54.831 INFO:tasks.workunit.client.0.smithi033.stdout:5/626: dread - d1c/d2a/d3d/f46 zero size 2024-04-25T01:35:54.833 INFO:tasks.workunit.client.1.smithi138.stdout:8/130: getdents d3/d8/da/db 0 2024-04-25T01:35:54.836 INFO:tasks.workunit.client.1.smithi138.stdout:1/55: rmdir d3/db 39 2024-04-25T01:35:54.837 INFO:tasks.workunit.client.0.smithi033.stdout:5/627: dwrite d1c/d2a/d50/d52/da6/d4a/fb4 [0,4194304] 0 2024-04-25T01:35:54.837 INFO:tasks.workunit.client.1.smithi138.stdout:1/56: read - d3/d4/da/ff zero size 2024-04-25T01:35:54.838 INFO:tasks.workunit.client.0.smithi033.stdout:5/628: dread - d1c/d53/fd0 zero size 2024-04-25T01:35:54.841 INFO:tasks.workunit.client.0.smithi033.stdout:5/629: symlink d1c/ld4 0 2024-04-25T01:35:54.842 INFO:tasks.workunit.client.1.smithi138.stdout:1/57: truncate d3/d4/fd 1201642 0 2024-04-25T01:35:54.843 INFO:tasks.workunit.client.0.smithi033.stdout:2/638: write da/d5b/d5e/d76/f90 [505083,15992] 0 2024-04-25T01:35:54.844 INFO:tasks.workunit.client.0.smithi033.stdout:5/630: mknod d1c/d2a/d50/dcb/cd5 0 2024-04-25T01:35:54.846 INFO:tasks.workunit.client.0.smithi033.stdout:2/639: creat da/d5b/d72/fd3 x:0 0 0 2024-04-25T01:35:54.846 INFO:tasks.workunit.client.0.smithi033.stdout:0/528: write d7/da/f5d [905203,64795] 0 2024-04-25T01:35:54.847 INFO:tasks.workunit.client.1.smithi138.stdout:1/58: mknod d3/db/c11 0 2024-04-25T01:35:54.847 INFO:tasks.workunit.client.0.smithi033.stdout:2/640: fsync da/d50/d6a/f56 0 2024-04-25T01:35:54.848 INFO:tasks.workunit.client.0.smithi033.stdout:1/420: write d2/d2e/d34/f50 [466501,60212] 0 2024-04-25T01:35:54.850 INFO:tasks.workunit.client.1.smithi138.stdout:8/131: dwrite d3/f1e [0,4194304] 0 2024-04-25T01:35:54.851 INFO:tasks.workunit.client.0.smithi033.stdout:2/641: write da/d27/d31/d40/db1/fa6 [321232,66542] 0 2024-04-25T01:35:54.852 INFO:tasks.workunit.client.1.smithi138.stdout:1/59: unlink d3/d4/da/ff 0 2024-04-25T01:35:54.852 INFO:tasks.workunit.client.0.smithi033.stdout:0/529: creat d7/d2d/d4d/d9c/d68/fb9 x:0 0 0 2024-04-25T01:35:54.853 INFO:tasks.workunit.client.0.smithi033.stdout:5/631: creat d1c/d2a/d3d/fd6 x:0 0 0 2024-04-25T01:35:54.854 INFO:tasks.workunit.client.0.smithi033.stdout:2/642: mknod da/d8d/db0/cd4 0 2024-04-25T01:35:54.856 INFO:tasks.workunit.client.0.smithi033.stdout:0/530: creat d7/da/df/d50/fba x:0 0 0 2024-04-25T01:35:54.856 INFO:tasks.workunit.client.1.smithi138.stdout:8/132: mknod d3/d8/c32 0 2024-04-25T01:35:54.857 INFO:tasks.workunit.client.1.smithi138.stdout:8/133: write d3/f19 [219909,48723] 0 2024-04-25T01:35:54.858 INFO:tasks.workunit.client.1.smithi138.stdout:1/60: mknod d3/c12 0 2024-04-25T01:35:54.858 INFO:tasks.workunit.client.1.smithi138.stdout:8/134: chown d3/d8/da/de/f13 264488210 1 2024-04-25T01:35:54.859 INFO:tasks.workunit.client.0.smithi033.stdout:1/421: dwrite d2/d5/d1b/f40 [0,4194304] 0 2024-04-25T01:35:54.862 INFO:tasks.workunit.client.1.smithi138.stdout:8/135: symlink d3/l33 0 2024-04-25T01:35:54.862 INFO:tasks.workunit.client.0.smithi033.stdout:1/422: chown d2/d11/f2a 47718540 1 2024-04-25T01:35:54.862 INFO:tasks.workunit.client.0.smithi033.stdout:5/632: creat d1c/d2a/d50/d5b/fd7 x:0 0 0 2024-04-25T01:35:54.863 INFO:tasks.workunit.client.0.smithi033.stdout:0/531: write d7/d2d/d64/d51/fb6 [1228341,113051] 0 2024-04-25T01:35:54.864 INFO:tasks.workunit.client.1.smithi138.stdout:8/136: creat d3/d8/da/db/d15/f34 x:0 0 0 2024-04-25T01:35:54.865 INFO:tasks.workunit.client.1.smithi138.stdout:8/137: readlink d3/d8/da/db/l28 0 2024-04-25T01:35:54.866 INFO:tasks.workunit.client.0.smithi033.stdout:2/643: mkdir da/d27/d31/d40/db1/dbf/dd5 0 2024-04-25T01:35:54.867 INFO:tasks.workunit.client.0.smithi033.stdout:5/633: creat d1c/d53/fd8 x:0 0 0 2024-04-25T01:35:54.867 INFO:tasks.workunit.client.1.smithi138.stdout:8/138: symlink d3/d16/d1f/l35 0 2024-04-25T01:35:54.868 INFO:tasks.workunit.client.1.smithi138.stdout:8/139: readlink d3/d16/d1f/l35 0 2024-04-25T01:35:54.868 INFO:tasks.workunit.client.0.smithi033.stdout:2/644: chown da/d27/d31/d40/d8a/dbb/f68 975795 1 2024-04-25T01:35:54.868 INFO:tasks.workunit.client.0.smithi033.stdout:0/532: mkdir d7/da/df/d50/dbb 0 2024-04-25T01:35:54.870 INFO:tasks.workunit.client.0.smithi033.stdout:0/533: write d7/d2d/d4d/d9c/f59 [1193686,81576] 0 2024-04-25T01:35:54.870 INFO:tasks.workunit.client.1.smithi138.stdout:8/140: creat d3/d8/da/de/f36 x:0 0 0 2024-04-25T01:35:54.872 INFO:tasks.workunit.client.0.smithi033.stdout:2/645: mkdir da/d27/d31/d40/d8a/dbb/dd6 0 2024-04-25T01:35:54.873 INFO:tasks.workunit.client.1.smithi138.stdout:8/141: symlink d3/d16/d1f/l37 0 2024-04-25T01:35:54.875 INFO:tasks.workunit.client.0.smithi033.stdout:0/534: creat d7/d2d/d4d/d9c/fbc x:0 0 0 2024-04-25T01:35:54.878 INFO:tasks.workunit.client.0.smithi033.stdout:5/634: dwrite d1c/d2a/d50/d52/da6/d3e/d70/d7a/dab/fc4 [0,4194304] 0 2024-04-25T01:35:54.878 INFO:tasks.workunit.client.0.smithi033.stdout:0/535: write d7/da/df/f99 [2660708,81628] 0 2024-04-25T01:35:54.879 INFO:tasks.workunit.client.1.smithi138.stdout:1/61: truncate d3/d4/f9 3097243 0 2024-04-25T01:35:54.880 INFO:tasks.workunit.client.0.smithi033.stdout:2/646: dread da/f12 [0,4194304] 0 2024-04-25T01:35:54.880 INFO:tasks.workunit.client.0.smithi033.stdout:5/635: chown d1c/d2a/d50/f7d 0 1 2024-04-25T01:35:54.881 INFO:tasks.workunit.client.1.smithi138.stdout:8/142: dwrite d3/f19 [0,4194304] 0 2024-04-25T01:35:54.883 INFO:tasks.workunit.client.1.smithi138.stdout:1/62: creat d3/f13 x:0 0 0 2024-04-25T01:35:54.885 INFO:tasks.workunit.client.0.smithi033.stdout:7/613: sync 2024-04-25T01:35:54.885 INFO:tasks.workunit.client.0.smithi033.stdout:3/645: sync 2024-04-25T01:35:54.885 INFO:tasks.workunit.client.0.smithi033.stdout:4/655: sync 2024-04-25T01:35:54.885 INFO:tasks.workunit.client.0.smithi033.stdout:8/629: sync 2024-04-25T01:35:54.885 INFO:tasks.workunit.client.0.smithi033.stdout:9/572: sync 2024-04-25T01:35:54.887 INFO:tasks.workunit.client.0.smithi033.stdout:5/636: rmdir d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e 39 2024-04-25T01:35:54.888 INFO:tasks.workunit.client.1.smithi138.stdout:8/143: creat d3/f38 x:0 0 0 2024-04-25T01:35:54.889 INFO:tasks.workunit.client.1.smithi138.stdout:8/144: chown d3/c6 237482 1 2024-04-25T01:35:54.890 INFO:tasks.workunit.client.0.smithi033.stdout:2/647: creat da/d27/d31/dca/fd7 x:0 0 0 2024-04-25T01:35:54.891 INFO:tasks.workunit.client.0.smithi033.stdout:9/573: dread - d0/d2/dd/d20/d4e/f68 zero size 2024-04-25T01:35:54.894 INFO:tasks.workunit.client.0.smithi033.stdout:8/630: rmdir d6/d45/d52/dae 39 2024-04-25T01:35:54.897 INFO:tasks.workunit.client.0.smithi033.stdout:1/423: dwrite d2/d2e/d34/d60/f7a [0,4194304] 0 2024-04-25T01:35:54.897 INFO:tasks.workunit.client.1.smithi138.stdout:8/145: dwrite d3/f5 [0,4194304] 0 2024-04-25T01:35:54.897 INFO:tasks.workunit.client.1.smithi138.stdout:1/63: rmdir d3/d4 39 2024-04-25T01:35:54.898 INFO:tasks.workunit.client.0.smithi033.stdout:0/536: getdents d7/da/df/d14/d45 0 2024-04-25T01:35:54.899 INFO:tasks.workunit.client.0.smithi033.stdout:5/637: readlink d1c/d2a/d50/d52/da6/d3e/l97 0 2024-04-25T01:35:54.901 INFO:tasks.workunit.client.1.smithi138.stdout:1/64: creat d3/db/f14 x:0 0 0 2024-04-25T01:35:54.901 INFO:tasks.workunit.client.1.smithi138.stdout:8/146: mknod d3/d8/da/de/c39 0 2024-04-25T01:35:54.901 INFO:tasks.workunit.client.0.smithi033.stdout:4/656: mknod d6/d14/d35/d3a/d4d/d5e/ccd 0 2024-04-25T01:35:54.902 INFO:tasks.workunit.client.1.smithi138.stdout:8/147: fsync d3/f1e 0 2024-04-25T01:35:54.903 INFO:tasks.workunit.client.0.smithi033.stdout:1/424: fdatasync d2/d5/d1b/f54 0 2024-04-25T01:35:54.906 INFO:tasks.workunit.client.0.smithi033.stdout:9/574: creat d0/d2/d43/d73/fd1 x:0 0 0 2024-04-25T01:35:54.907 INFO:tasks.workunit.client.0.smithi033.stdout:3/646: creat d8/dd/fdb x:0 0 0 2024-04-25T01:35:54.908 INFO:tasks.workunit.client.1.smithi138.stdout:8/148: creat d3/f3a x:0 0 0 2024-04-25T01:35:54.908 INFO:tasks.workunit.client.0.smithi033.stdout:0/537: symlink d7/d2d/d4c/lbd 0 2024-04-25T01:35:54.909 INFO:tasks.workunit.client.0.smithi033.stdout:5/638: readlink d1c/d2a/d50/d52/da6/d3e/d70/d89/la0 0 2024-04-25T01:35:54.910 INFO:tasks.workunit.client.1.smithi138.stdout:1/65: dwrite d3/f13 [0,4194304] 0 2024-04-25T01:35:54.912 INFO:tasks.workunit.client.1.smithi138.stdout:8/149: write d3/f38 [450549,90920] 0 2024-04-25T01:35:54.914 INFO:tasks.workunit.client.1.smithi138.stdout:8/150: chown d3/d8/da/de/f13 9799968 1 2024-04-25T01:35:54.916 INFO:tasks.workunit.client.0.smithi033.stdout:1/425: write d2/d2e/f71 [3674274,120064] 0 2024-04-25T01:35:54.918 INFO:tasks.workunit.client.1.smithi138.stdout:1/66: creat d3/f15 x:0 0 0 2024-04-25T01:35:54.918 INFO:tasks.workunit.client.0.smithi033.stdout:7/614: dwrite d4/d23/d3f/f6f [0,4194304] 0 2024-04-25T01:35:54.919 INFO:tasks.workunit.client.0.smithi033.stdout:2/648: dwrite da/d27/d31/d40/db1/fa6 [0,4194304] 0 2024-04-25T01:35:54.922 INFO:tasks.workunit.client.0.smithi033.stdout:2/649: chown da/d50 914971 1 2024-04-25T01:35:54.924 INFO:tasks.workunit.client.0.smithi033.stdout:9/575: fsync d0/d2/dd/f54 0 2024-04-25T01:35:54.924 INFO:tasks.workunit.client.0.smithi033.stdout:7/615: readlink d4/d10/l40 0 2024-04-25T01:35:54.925 INFO:tasks.workunit.client.0.smithi033.stdout:0/538: write d7/da/df/d50/f6b [19130,127710] 0 2024-04-25T01:35:54.926 INFO:tasks.workunit.client.0.smithi033.stdout:4/657: read d6/d14/d35/d3a/d4d/d5e/f76 [10704388,53922] 0 2024-04-25T01:35:54.926 INFO:tasks.workunit.client.1.smithi138.stdout:8/151: dwrite d3/f3a [0,4194304] 0 2024-04-25T01:35:54.927 INFO:tasks.workunit.client.0.smithi033.stdout:8/631: dwrite d6/d45/d52/dcc/fdc [0,4194304] 0 2024-04-25T01:35:54.931 INFO:tasks.workunit.client.0.smithi033.stdout:2/650: fsync da/d27/f5c 0 2024-04-25T01:35:54.933 INFO:tasks.workunit.client.1.smithi138.stdout:1/67: fsync d3/f13 0 2024-04-25T01:35:54.935 INFO:tasks.workunit.client.0.smithi033.stdout:4/658: fdatasync d6/d12/f40 0 2024-04-25T01:35:54.938 INFO:tasks.workunit.client.1.smithi138.stdout:1/68: mkdir d3/d4/da/d16 0 2024-04-25T01:35:54.941 INFO:tasks.workunit.client.1.smithi138.stdout:8/152: getdents d3 0 2024-04-25T01:35:54.942 INFO:tasks.workunit.client.0.smithi033.stdout:1/426: rename d2/d5/d75/c58 to d2/d2e/d34/d60/c7e 0 2024-04-25T01:35:54.943 INFO:tasks.workunit.client.0.smithi033.stdout:5/639: dread d1c/d2a/d3d/d59/f84 [0,4194304] 0 2024-04-25T01:35:54.943 INFO:tasks.workunit.client.0.smithi033.stdout:3/647: dwrite d8/dd/d3d/d63/d7c/d3c/d86/d92/f21 [0,4194304] 0 2024-04-25T01:35:54.944 INFO:tasks.workunit.client.1.smithi138.stdout:1/69: link d3/db/f14 d3/db/f17 0 2024-04-25T01:35:54.944 INFO:tasks.workunit.client.0.smithi033.stdout:6/650: sync 2024-04-25T01:35:54.945 INFO:tasks.workunit.client.1.smithi138.stdout:1/70: chown d3/db/f17 15 1 2024-04-25T01:35:54.947 INFO:tasks.workunit.client.0.smithi033.stdout:8/632: symlink d6/d45/d51/d5d/lf0 0 2024-04-25T01:35:54.950 INFO:tasks.workunit.client.1.smithi138.stdout:1/71: creat d3/d4/da/d16/f18 x:0 0 0 2024-04-25T01:35:54.951 INFO:tasks.workunit.client.1.smithi138.stdout:8/153: dwrite d3/d8/da/f24 [0,4194304] 0 2024-04-25T01:35:54.951 INFO:tasks.workunit.client.0.smithi033.stdout:1/427: mkdir d2/d7f 0 2024-04-25T01:35:54.953 INFO:tasks.workunit.client.0.smithi033.stdout:1/428: chown d2/d2e 1513934 1 2024-04-25T01:35:54.953 INFO:tasks.workunit.client.0.smithi033.stdout:8/633: write d6/d45/d51/f4f [7077893,34469] 0 2024-04-25T01:35:54.953 INFO:tasks.workunit.client.0.smithi033.stdout:2/651: getdents da/d5b 0 2024-04-25T01:35:54.957 INFO:tasks.workunit.client.0.smithi033.stdout:0/539: dwrite d7/d2d/d4d/d9c/f83 [0,4194304] 0 2024-04-25T01:35:54.958 INFO:tasks.workunit.client.0.smithi033.stdout:3/648: symlink d8/dd/d3d/d63/d7c/d3c/d6e/ldc 0 2024-04-25T01:35:54.961 INFO:tasks.workunit.client.0.smithi033.stdout:5/640: unlink d1c/d2a/d50/d52/da6/d3e/d70/c8a 0 2024-04-25T01:35:54.964 INFO:tasks.workunit.client.0.smithi033.stdout:4/659: dread d6/d14/d1d/d3b/f52 [0,4194304] 0 2024-04-25T01:35:54.966 INFO:tasks.workunit.client.0.smithi033.stdout:2/652: dread da/d27/d31/d40/d8a/fbc [0,4194304] 0 2024-04-25T01:35:54.966 INFO:tasks.workunit.client.1.smithi138.stdout:8/154: dwrite d3/d8/da/db/d15/d20/f26 [0,4194304] 0 2024-04-25T01:35:54.970 INFO:tasks.workunit.client.0.smithi033.stdout:1/429: symlink d2/d2e/l80 0 2024-04-25T01:35:54.971 INFO:tasks.workunit.client.1.smithi138.stdout:8/155: rmdir d3/d8/da/db/d15 39 2024-04-25T01:35:54.971 INFO:tasks.workunit.client.1.smithi138.stdout:8/156: chown d3/f38 129634 1 2024-04-25T01:35:54.972 INFO:tasks.workunit.client.0.smithi033.stdout:8/634: fsync d6/db/f24 0 2024-04-25T01:35:54.973 INFO:tasks.workunit.client.0.smithi033.stdout:0/540: unlink d7/d2d/d9b/l40 0 2024-04-25T01:35:54.976 INFO:tasks.workunit.client.1.smithi138.stdout:8/157: write d3/d8/da/db/d15/f34 [667654,86615] 0 2024-04-25T01:35:54.978 INFO:tasks.workunit.client.1.smithi138.stdout:8/158: creat d3/d16/f3b x:0 0 0 2024-04-25T01:35:54.980 INFO:tasks.workunit.client.0.smithi033.stdout:7/616: dwrite d4/d2e/d50/f6e [0,4194304] 0 2024-04-25T01:35:54.981 INFO:tasks.workunit.client.1.smithi138.stdout:8/159: creat d3/f3c x:0 0 0 2024-04-25T01:35:54.982 INFO:tasks.workunit.client.0.smithi033.stdout:2/653: write da/d27/fb2 [3054205,103396] 0 2024-04-25T01:35:54.982 INFO:tasks.workunit.client.0.smithi033.stdout:3/649: link d8/dd/d3d/d63/d7c/d3c/fa3 d8/dd/d1d/fdd 0 2024-04-25T01:35:54.985 INFO:tasks.workunit.client.0.smithi033.stdout:4/660: dwrite d6/d12/d38/f7e [0,4194304] 0 2024-04-25T01:35:54.986 INFO:tasks.workunit.client.0.smithi033.stdout:1/430: creat d2/f81 x:0 0 0 2024-04-25T01:35:54.989 INFO:tasks.workunit.client.0.smithi033.stdout:5/641: creat d1c/d2a/d50/d52/fd9 x:0 0 0 2024-04-25T01:35:54.990 INFO:tasks.workunit.client.1.smithi138.stdout:8/160: dwrite d3/d8/da/db/d15/f34 [0,4194304] 0 2024-04-25T01:35:54.994 INFO:tasks.workunit.client.0.smithi033.stdout:6/651: dwrite d4/d24/d63/f6f [4194304,4194304] 0 2024-04-25T01:35:54.994 INFO:tasks.workunit.client.0.smithi033.stdout:5/642: fsync d1c/d53/fd0 0 2024-04-25T01:35:54.994 INFO:tasks.workunit.client.1.smithi138.stdout:8/161: creat d3/d16/d1f/d31/f3d x:0 0 0 2024-04-25T01:35:54.995 INFO:tasks.workunit.client.0.smithi033.stdout:3/650: chown d8/dd/d3d/d63/d7c/d3c/d98/da1/dc3/fd0 0 1 2024-04-25T01:35:54.995 INFO:tasks.workunit.client.0.smithi033.stdout:9/576: dread d0/d11/f16 [0,4194304] 0 2024-04-25T01:35:54.995 INFO:tasks.workunit.client.1.smithi138.stdout:8/162: stat d3/d8/da/db 0 2024-04-25T01:35:54.999 INFO:tasks.workunit.client.0.smithi033.stdout:0/541: dread d7/d2d/d4d/d9c/f59 [0,4194304] 0 2024-04-25T01:35:54.999 INFO:tasks.workunit.client.1.smithi138.stdout:0/32: sync 2024-04-25T01:35:55.003 INFO:tasks.workunit.client.1.smithi138.stdout:0/33: stat l5 0 2024-04-25T01:35:55.003 INFO:tasks.workunit.client.0.smithi033.stdout:8/635: dwrite d6/f13 [0,4194304] 0 2024-04-25T01:35:55.005 INFO:tasks.workunit.client.0.smithi033.stdout:6/652: mknod d4/d1d/dbf/ce2 0 2024-04-25T01:35:55.006 INFO:tasks.workunit.client.1.smithi138.stdout:0/34: unlink l5 0 2024-04-25T01:35:55.011 INFO:tasks.workunit.client.1.smithi138.stdout:0/35: symlink d6/la 0 2024-04-25T01:35:55.013 INFO:tasks.workunit.client.0.smithi033.stdout:5/643: unlink d1c/d2a/d50/d52/da6/d4a/f4e 0 2024-04-25T01:35:55.013 INFO:tasks.workunit.client.1.smithi138.stdout:8/163: dwrite d3/d8/da/f24 [0,4194304] 0 2024-04-25T01:35:55.015 INFO:tasks.workunit.client.0.smithi033.stdout:9/577: write d0/d2/dd/df/d1b/d3d/d7e/d58/fa6 [2581839,68166] 0 2024-04-25T01:35:55.017 INFO:tasks.workunit.client.1.smithi138.stdout:8/164: chown d3/f38 287821 1 2024-04-25T01:35:55.017 INFO:tasks.workunit.client.0.smithi033.stdout:2/654: dwrite da/d27/d31/d40/db1/fcc [0,4194304] 0 2024-04-25T01:35:55.017 INFO:tasks.workunit.client.0.smithi033.stdout:3/651: readlink d8/dd/d3d/d63/d7c/d3c/d86/d92/l56 0 2024-04-25T01:35:55.018 INFO:tasks.workunit.client.0.smithi033.stdout:6/653: chown d4/d24/d63/c6e 45 1 2024-04-25T01:35:55.018 INFO:tasks.workunit.client.0.smithi033.stdout:8/636: symlink d6/d45/d51/d55/daa/lf1 0 2024-04-25T01:35:55.020 INFO:tasks.workunit.client.1.smithi138.stdout:8/165: mkdir d3/d16/d1f/d2a/d3e 0 2024-04-25T01:35:55.023 INFO:tasks.workunit.client.0.smithi033.stdout:3/652: stat d8/c2a 0 2024-04-25T01:35:55.025 INFO:tasks.workunit.client.1.smithi138.stdout:8/166: dread d3/d8/da/db/d15/d20/f26 [0,4194304] 0 2024-04-25T01:35:55.026 INFO:tasks.workunit.client.0.smithi033.stdout:4/661: dwrite d6/d14/d35/d44/d45/d46/fbd [0,4194304] 0 2024-04-25T01:35:55.027 INFO:tasks.workunit.client.0.smithi033.stdout:8/637: chown d6/d45/d51/d5d/d6e/cb2 25729054 1 2024-04-25T01:35:55.027 INFO:tasks.workunit.client.0.smithi033.stdout:0/542: rename d7/d2d/d4d/d9c/fbc to d7/da/df/d14/fbe 0 2024-04-25T01:35:55.027 INFO:tasks.workunit.client.1.smithi138.stdout:8/167: mknod d3/d8/da/de/c3f 0 2024-04-25T01:35:55.030 INFO:tasks.workunit.client.0.smithi033.stdout:1/431: dwrite d2/d5/d1b/d46/f69 [0,4194304] 0 2024-04-25T01:35:55.030 INFO:tasks.workunit.client.0.smithi033.stdout:8/638: chown d6/db/d80/la5 40792 1 2024-04-25T01:35:55.030 INFO:tasks.workunit.client.0.smithi033.stdout:5/644: fdatasync f1b 0 2024-04-25T01:35:55.032 INFO:tasks.workunit.client.1.smithi138.stdout:8/168: link d3/d8/da/db/d15/f21 d3/d8/f40 0 2024-04-25T01:35:55.034 INFO:tasks.workunit.client.0.smithi033.stdout:2/655: mkdir da/d5b/d5e/dd8 0 2024-04-25T01:35:55.034 INFO:tasks.workunit.client.1.smithi138.stdout:8/169: mknod d3/d16/d1f/d2a/c41 0 2024-04-25T01:35:55.036 INFO:tasks.workunit.client.0.smithi033.stdout:9/578: stat d0/d2/d43/d73/dab/cb7 0 2024-04-25T01:35:55.037 INFO:tasks.workunit.client.0.smithi033.stdout:6/654: mknod d4/d24/d36/dbd/ce3 0 2024-04-25T01:35:55.039 INFO:tasks.workunit.client.1.smithi138.stdout:8/170: rmdir d3/d16/d1f/d2a/d3e 0 2024-04-25T01:35:55.041 INFO:tasks.workunit.client.0.smithi033.stdout:0/543: symlink d7/d2d/d4d/d9c/d68/lbf 0 2024-04-25T01:35:55.042 INFO:tasks.workunit.client.1.smithi138.stdout:8/171: symlink d3/d8/da/db/d15/l42 0 2024-04-25T01:35:55.043 INFO:tasks.workunit.client.0.smithi033.stdout:3/653: dwrite d8/dd/d3d/d63/d7c/d3c/d98/da1/fc1 [0,4194304] 0 2024-04-25T01:35:55.047 INFO:tasks.workunit.client.0.smithi033.stdout:4/662: creat d6/d14/d35/d3a/d4d/fce x:0 0 0 2024-04-25T01:35:55.050 INFO:tasks.workunit.client.1.smithi138.stdout:8/172: dwrite f0 [0,4194304] 0 2024-04-25T01:35:55.051 INFO:tasks.workunit.client.1.smithi138.stdout:0/36: truncate f0 2808645 0 2024-04-25T01:35:55.051 INFO:tasks.workunit.client.1.smithi138.stdout:0/37: chown d6 20 1 2024-04-25T01:35:55.052 INFO:tasks.workunit.client.1.smithi138.stdout:0/38: chown c2 0 1 2024-04-25T01:35:55.052 INFO:tasks.workunit.client.0.smithi033.stdout:8/639: mkdir d6/d45/d52/df2 0 2024-04-25T01:35:55.053 INFO:tasks.workunit.client.1.smithi138.stdout:8/173: chown d3/f3a 878449 1 2024-04-25T01:35:55.054 INFO:tasks.workunit.client.0.smithi033.stdout:7/617: dwrite d4/d2e/d6b/f47 [0,4194304] 0 2024-04-25T01:35:55.055 INFO:tasks.workunit.client.1.smithi138.stdout:8/174: creat d3/d8/da/f43 x:0 0 0 2024-04-25T01:35:55.057 INFO:tasks.workunit.client.0.smithi033.stdout:4/663: chown d6/d14/d35/d3a/f56 4 1 2024-04-25T01:35:55.058 INFO:tasks.workunit.client.1.smithi138.stdout:8/175: mkdir d3/d44 0 2024-04-25T01:35:55.060 INFO:tasks.workunit.client.1.smithi138.stdout:8/176: write d3/d8/da/db/d15/f30 [4120468,71430] 0 2024-04-25T01:35:55.061 INFO:tasks.workunit.client.0.smithi033.stdout:0/544: rename d7/d1b to d7/d2d/d43/dc0 0 2024-04-25T01:35:55.061 INFO:tasks.workunit.client.0.smithi033.stdout:7/618: read d4/d2e/d6b/d1f/f42 [246344,49428] 0 2024-04-25T01:35:55.061 INFO:tasks.workunit.client.0.smithi033.stdout:6/655: dwrite d4/d24/d36/d42/fb1 [0,4194304] 0 2024-04-25T01:35:55.062 INFO:tasks.workunit.client.1.smithi138.stdout:8/177: write d3/d16/f3b [332417,104829] 0 2024-04-25T01:35:55.067 INFO:tasks.workunit.client.0.smithi033.stdout:9/579: dwrite d0/d2/d43/d4b/d55/fbc [0,4194304] 0 2024-04-25T01:35:55.073 INFO:tasks.workunit.client.1.smithi138.stdout:8/178: dwrite d3/d8/da/db/d15/f30 [4194304,4194304] 0 2024-04-25T01:35:55.076 INFO:tasks.workunit.client.1.smithi138.stdout:8/179: symlink d3/d8/da/db/d15/l45 0 2024-04-25T01:35:55.078 INFO:tasks.workunit.client.1.smithi138.stdout:0/39: dread f0 [0,4194304] 0 2024-04-25T01:35:55.079 INFO:tasks.workunit.client.0.smithi033.stdout:1/432: creat d2/d2e/d34/f82 x:0 0 0 2024-04-25T01:35:55.079 INFO:tasks.workunit.client.0.smithi033.stdout:9/580: dwrite d0/d2/d43/d73/f84 [0,4194304] 0 2024-04-25T01:35:55.084 INFO:tasks.workunit.client.0.smithi033.stdout:6/656: creat d4/d24/d36/d42/fe4 x:0 0 0 2024-04-25T01:35:55.084 INFO:tasks.workunit.client.0.smithi033.stdout:6/657: stat d4/ld 0 2024-04-25T01:35:55.084 INFO:tasks.workunit.client.0.smithi033.stdout:6/658: chown d4/d1d/d3e/d70 4371 1 2024-04-25T01:35:55.085 INFO:tasks.workunit.client.0.smithi033.stdout:8/640: rename d6/d49/c6c to d6/d45/d51/d55/d84/dc8/cf3 0 2024-04-25T01:35:55.086 INFO:tasks.workunit.client.1.smithi138.stdout:0/40: fdatasync f1 0 2024-04-25T01:35:55.086 INFO:tasks.workunit.client.1.smithi138.stdout:0/41: rename d6 to d6/db 22 2024-04-25T01:35:55.088 INFO:tasks.workunit.client.0.smithi033.stdout:9/581: mknod d0/d2/dd/dba/dc1/cd2 0 2024-04-25T01:35:55.089 INFO:tasks.workunit.client.0.smithi033.stdout:9/582: fsync d0/d2/dd/df/d1b/d3d/d42/fae 0 2024-04-25T01:35:55.090 INFO:tasks.workunit.client.0.smithi033.stdout:8/641: fdatasync d6/d45/d52/d2e/f98 0 2024-04-25T01:35:55.091 INFO:tasks.workunit.client.1.smithi138.stdout:0/42: dread f0 [0,4194304] 0 2024-04-25T01:35:55.091 INFO:tasks.workunit.client.0.smithi033.stdout:9/583: chown d0/d2/d43/d4b/f97 107288281 1 2024-04-25T01:35:55.093 INFO:tasks.workunit.client.0.smithi033.stdout:1/433: mkdir d2/d5/d75/d59/d83 0 2024-04-25T01:35:55.094 INFO:tasks.workunit.client.1.smithi138.stdout:0/43: creat d6/fc x:0 0 0 2024-04-25T01:35:55.097 INFO:tasks.workunit.client.0.smithi033.stdout:7/619: link d4/d2e/d6b/d1f/f36 d4/d2e/d6d/dbb/fc4 0 2024-04-25T01:35:55.097 INFO:tasks.workunit.client.1.smithi138.stdout:0/44: link f0 d6/fd 0 2024-04-25T01:35:55.099 INFO:tasks.workunit.client.1.smithi138.stdout:0/45: chown d6/la 714357744 1 2024-04-25T01:35:55.099 INFO:tasks.workunit.client.1.smithi138.stdout:0/46: chown d6/la 133838310 1 2024-04-25T01:35:55.100 INFO:tasks.workunit.client.0.smithi033.stdout:2/656: truncate f7 319050 0 2024-04-25T01:35:55.102 INFO:tasks.workunit.client.1.smithi138.stdout:0/47: rename d6/fd to d6/fe 0 2024-04-25T01:35:55.103 INFO:tasks.workunit.client.0.smithi033.stdout:6/659: creat d4/d1d/d3e/d41/d5c/d7f/da4/dcc/fe5 x:0 0 0 2024-04-25T01:35:55.103 INFO:tasks.workunit.client.1.smithi138.stdout:0/48: read - d6/f9 zero size 2024-04-25T01:35:55.104 INFO:tasks.workunit.client.1.smithi138.stdout:8/180: truncate d3/d8/da/db/d15/f30 3990344 0 2024-04-25T01:35:55.105 INFO:tasks.workunit.client.1.smithi138.stdout:8/181: fdatasync d3/d16/f1c 0 2024-04-25T01:35:55.106 INFO:tasks.workunit.client.1.smithi138.stdout:0/49: unlink d6/c7 0 2024-04-25T01:35:55.108 INFO:tasks.workunit.client.0.smithi033.stdout:7/620: rename d4/d2e/d6b/d1f/f42 to d4/d23/fc5 0 2024-04-25T01:35:55.109 INFO:tasks.workunit.client.0.smithi033.stdout:2/657: mknod da/d27/d31/dca/cd9 0 2024-04-25T01:35:55.109 INFO:tasks.workunit.client.0.smithi033.stdout:0/545: truncate d7/da/df/d14/f15 1756238 0 2024-04-25T01:35:55.111 INFO:tasks.workunit.client.0.smithi033.stdout:6/660: mknod d4/d1d/d3e/d70/ce6 0 2024-04-25T01:35:55.112 INFO:tasks.workunit.client.0.smithi033.stdout:5/645: truncate d1c/d2a/d50/d52/da6/d4a/f8c 1083271 0 2024-04-25T01:35:55.113 INFO:tasks.workunit.client.1.smithi138.stdout:0/50: dwrite d6/fc [0,4194304] 0 2024-04-25T01:35:55.114 INFO:tasks.workunit.client.0.smithi033.stdout:8/642: truncate d6/d45/d51/d55/daa/fea 486089 0 2024-04-25T01:35:55.115 INFO:tasks.workunit.client.1.smithi138.stdout:0/51: write d6/fc [283402,1250] 0 2024-04-25T01:35:55.115 INFO:tasks.workunit.client.0.smithi033.stdout:3/654: dwrite d8/dd/d3d/d63/d7c/d3c/fa3 [0,4194304] 0 2024-04-25T01:35:55.118 INFO:tasks.workunit.client.1.smithi138.stdout:0/52: dread d6/fe [0,4194304] 0 2024-04-25T01:35:55.118 INFO:tasks.workunit.client.0.smithi033.stdout:6/661: dread d4/d24/f99 [0,4194304] 0 2024-04-25T01:35:55.119 INFO:tasks.workunit.client.1.smithi138.stdout:0/53: readlink d6/l8 0 2024-04-25T01:35:55.119 INFO:tasks.workunit.client.1.smithi138.stdout:8/182: write d3/f5 [1073460,112229] 0 2024-04-25T01:35:55.120 INFO:tasks.workunit.client.0.smithi033.stdout:2/658: chown da/d27/d31/f3d 0 1 2024-04-25T01:35:55.121 INFO:tasks.workunit.client.0.smithi033.stdout:0/546: write d7/d2d/d64/fad [176104,108123] 0 2024-04-25T01:35:55.124 INFO:tasks.workunit.client.1.smithi138.stdout:0/54: mkdir d6/df 0 2024-04-25T01:35:55.126 INFO:tasks.workunit.client.1.smithi138.stdout:8/183: fdatasync d3/d8/da/de/f13 0 2024-04-25T01:35:55.127 INFO:tasks.workunit.client.0.smithi033.stdout:7/621: dwrite d4/d23/d3f/f77 [0,4194304] 0 2024-04-25T01:35:55.127 INFO:tasks.workunit.client.0.smithi033.stdout:6/662: write d4/d1d/d3e/d41/d5c/d7f/da4/faf [1859505,10548] 0 2024-04-25T01:35:55.131 INFO:tasks.workunit.client.0.smithi033.stdout:6/663: chown d4/d1d/d3e/d41/d5c/d7f/da4/dcc 10 1 2024-04-25T01:35:55.135 INFO:tasks.workunit.client.1.smithi138.stdout:0/55: creat d6/df/f10 x:0 0 0 2024-04-25T01:35:55.135 INFO:tasks.workunit.client.0.smithi033.stdout:0/547: dread d7/da/f5d [0,4194304] 0 2024-04-25T01:35:55.136 INFO:tasks.workunit.client.1.smithi138.stdout:0/56: chown d6/l8 1252 1 2024-04-25T01:35:55.136 INFO:tasks.workunit.client.0.smithi033.stdout:2/659: creat da/d8d/fda x:0 0 0 2024-04-25T01:35:55.137 INFO:tasks.workunit.client.1.smithi138.stdout:8/184: creat d3/d44/f46 x:0 0 0 2024-04-25T01:35:55.138 INFO:tasks.workunit.client.1.smithi138.stdout:8/185: chown d3/d8/da/db/d15/l42 0 1 2024-04-25T01:35:55.138 INFO:tasks.workunit.client.1.smithi138.stdout:0/57: creat d6/f11 x:0 0 0 2024-04-25T01:35:55.140 INFO:tasks.workunit.client.1.smithi138.stdout:0/58: write d6/fc [1703068,75159] 0 2024-04-25T01:35:55.140 INFO:tasks.workunit.client.0.smithi033.stdout:0/548: truncate d7/da/fb7 242454 0 2024-04-25T01:35:55.140 INFO:tasks.workunit.client.0.smithi033.stdout:8/643: symlink d6/db/lf4 0 2024-04-25T01:35:55.141 INFO:tasks.workunit.client.1.smithi138.stdout:0/59: stat d6 0 2024-04-25T01:35:55.141 INFO:tasks.workunit.client.1.smithi138.stdout:8/186: write d3/f1e [4472483,103557] 0 2024-04-25T01:35:55.142 INFO:tasks.workunit.client.0.smithi033.stdout:4/664: dwrite d6/d14/ff [0,4194304] 0 2024-04-25T01:35:55.143 INFO:tasks.workunit.client.1.smithi138.stdout:0/60: truncate f1 173798 0 2024-04-25T01:35:55.143 INFO:tasks.workunit.client.0.smithi033.stdout:3/655: mknod d8/dd/d3d/d63/d7c/d3c/d86/d92/cde 0 2024-04-25T01:35:55.144 INFO:tasks.workunit.client.1.smithi138.stdout:0/61: read - d6/f9 zero size 2024-04-25T01:35:55.144 INFO:tasks.workunit.client.1.smithi138.stdout:0/62: dread - d6/f11 zero size 2024-04-25T01:35:55.145 INFO:tasks.workunit.client.0.smithi033.stdout:6/664: dread d4/d24/f99 [0,4194304] 0 2024-04-25T01:35:55.145 INFO:tasks.workunit.client.1.smithi138.stdout:8/187: creat d3/d8/f47 x:0 0 0 2024-04-25T01:35:55.147 INFO:tasks.workunit.client.0.smithi033.stdout:2/660: creat da/d27/d31/dca/fdb x:0 0 0 2024-04-25T01:35:55.148 INFO:tasks.workunit.client.0.smithi033.stdout:8/644: mknod d6/d45/d51/d5d/cf5 0 2024-04-25T01:35:55.150 INFO:tasks.workunit.client.0.smithi033.stdout:5/646: dread d1c/d2a/d50/d52/da6/d3e/d70/dae/fb5 [0,4194304] 0 2024-04-25T01:35:55.150 INFO:tasks.workunit.client.0.smithi033.stdout:7/622: dwrite d4/d23/d3f/f6f [0,4194304] 0 2024-04-25T01:35:55.150 INFO:tasks.workunit.client.0.smithi033.stdout:8/645: chown d6/dd3 138 1 2024-04-25T01:35:55.150 INFO:tasks.workunit.client.1.smithi138.stdout:0/63: rename f0 to d6/df/f12 0 2024-04-25T01:35:55.152 INFO:tasks.workunit.client.1.smithi138.stdout:0/64: write d6/f11 [360256,17793] 0 2024-04-25T01:35:55.153 INFO:tasks.workunit.client.0.smithi033.stdout:4/665: fsync d6/d14/f53 0 2024-04-25T01:35:55.154 INFO:tasks.workunit.client.0.smithi033.stdout:8/646: write d6/db/d80/d89/f91 [3095689,57191] 0 2024-04-25T01:35:55.155 INFO:tasks.workunit.client.1.smithi138.stdout:9/53: sync 2024-04-25T01:35:55.155 INFO:tasks.workunit.client.1.smithi138.stdout:6/46: sync 2024-04-25T01:35:55.155 INFO:tasks.workunit.client.1.smithi138.stdout:5/42: sync 2024-04-25T01:35:55.155 INFO:tasks.workunit.client.1.smithi138.stdout:7/59: sync 2024-04-25T01:35:55.156 INFO:tasks.workunit.client.1.smithi138.stdout:8/188: dwrite d3/d8/da/db/d15/f34 [0,4194304] 0 2024-04-25T01:35:55.157 INFO:tasks.workunit.client.1.smithi138.stdout:5/43: read f3 [937157,40785] 0 2024-04-25T01:35:55.157 INFO:tasks.workunit.client.0.smithi033.stdout:2/661: mknod da/d2c/dcd/cdc 0 2024-04-25T01:35:55.157 INFO:tasks.workunit.client.0.smithi033.stdout:0/549: creat d7/d2d/fc1 x:0 0 0 2024-04-25T01:35:55.160 INFO:tasks.workunit.client.0.smithi033.stdout:6/665: creat d4/d1d/d3e/d41/d9f/fe7 x:0 0 0 2024-04-25T01:35:55.166 INFO:tasks.workunit.client.1.smithi138.stdout:6/47: truncate f1 963989 0 2024-04-25T01:35:55.166 INFO:tasks.workunit.client.0.smithi033.stdout:3/656: dread d8/dd/d3d/d63/d7c/f70 [0,4194304] 0 2024-04-25T01:35:55.169 INFO:tasks.workunit.client.1.smithi138.stdout:8/189: rmdir d3/d8/da/db 39 2024-04-25T01:35:55.170 INFO:tasks.workunit.client.0.smithi033.stdout:2/662: dwrite da/d27/f7a [0,4194304] 0 2024-04-25T01:35:55.172 INFO:tasks.workunit.client.1.smithi138.stdout:0/65: dread d6/f11 [0,4194304] 0 2024-04-25T01:35:55.172 INFO:tasks.workunit.client.1.smithi138.stdout:6/48: chown la 268786 1 2024-04-25T01:35:55.173 INFO:tasks.workunit.client.0.smithi033.stdout:7/623: symlink d4/d2e/d6b/d1f/d78/d86/lc6 0 2024-04-25T01:35:55.173 INFO:tasks.workunit.client.0.smithi033.stdout:4/666: symlink d6/d14/d35/d3a/d4d/d5e/lcf 0 2024-04-25T01:35:55.174 INFO:tasks.workunit.client.1.smithi138.stdout:7/60: dwrite da/de/f1c [0,4194304] 0 2024-04-25T01:35:55.175 INFO:tasks.workunit.client.1.smithi138.stdout:9/54: dread d4/d8/fd [0,4194304] 0 2024-04-25T01:35:55.178 INFO:tasks.workunit.client.1.smithi138.stdout:0/66: chown c2 84160878 1 2024-04-25T01:35:55.178 INFO:tasks.workunit.client.0.smithi033.stdout:6/666: mknod d4/d24/d63/ce8 0 2024-04-25T01:35:55.179 INFO:tasks.workunit.client.0.smithi033.stdout:1/434: truncate d2/d11/d28/f2c 256944 0 2024-04-25T01:35:55.180 INFO:tasks.workunit.client.0.smithi033.stdout:6/667: read - d4/d24/d36/f6c zero size 2024-04-25T01:35:55.182 INFO:tasks.workunit.client.0.smithi033.stdout:6/668: write d4/d24/d29/f33 [1709578,79729] 0 2024-04-25T01:35:55.183 INFO:tasks.workunit.client.1.smithi138.stdout:7/61: symlink da/l1e 0 2024-04-25T01:35:55.185 INFO:tasks.workunit.client.1.smithi138.stdout:8/190: rmdir d3/d16/d1f 39 2024-04-25T01:35:55.187 INFO:tasks.workunit.client.0.smithi033.stdout:2/663: mknod da/d8d/db0/cdd 0 2024-04-25T01:35:55.188 INFO:tasks.workunit.client.1.smithi138.stdout:9/55: rename d4/c6 to d4/d8/c12 0 2024-04-25T01:35:55.189 INFO:tasks.workunit.client.1.smithi138.stdout:8/191: write d3/f38 [514171,63668] 0 2024-04-25T01:35:55.189 INFO:tasks.workunit.client.0.smithi033.stdout:2/664: readlink da/d27/d31/d40/db1/laf 0 2024-04-25T01:35:55.191 INFO:tasks.workunit.client.0.smithi033.stdout:4/667: rename d6/d14/d6d/fa8 to d6/d14/d35/d44/d45/d46/d57/d70/d87/fd0 0 2024-04-25T01:35:55.192 INFO:tasks.workunit.client.1.smithi138.stdout:0/67: link d6/fc d6/f13 0 2024-04-25T01:35:55.193 INFO:tasks.workunit.client.0.smithi033.stdout:5/647: link f17 d1c/d2a/d50/db1/dd3/fda 0 2024-04-25T01:35:55.193 INFO:tasks.workunit.client.0.smithi033.stdout:1/435: dwrite d2/f81 [0,4194304] 0 2024-04-25T01:35:55.194 INFO:tasks.workunit.client.0.smithi033.stdout:0/550: dread d7/d2d/d43/dc0/f32 [0,4194304] 0 2024-04-25T01:35:55.195 INFO:tasks.workunit.client.1.smithi138.stdout:8/192: mkdir d3/d44/d48 0 2024-04-25T01:35:55.196 INFO:tasks.workunit.client.0.smithi033.stdout:1/436: chown d2/c31 413 1 2024-04-25T01:35:55.197 INFO:tasks.workunit.client.0.smithi033.stdout:4/668: write d6/d14/d1d/f63 [2527864,6609] 0 2024-04-25T01:35:55.198 INFO:tasks.workunit.client.0.smithi033.stdout:0/551: chown d7/d2d/d4c/c85 501006 1 2024-04-25T01:35:55.202 INFO:tasks.workunit.client.1.smithi138.stdout:0/68: dwrite d6/df/f10 [0,4194304] 0 2024-04-25T01:35:55.203 INFO:tasks.workunit.client.0.smithi033.stdout:6/669: rename d4/d24/lb3 to d4/d24/d63/le9 0 2024-04-25T01:35:55.205 INFO:tasks.workunit.client.0.smithi033.stdout:5/648: creat d1c/d2a/d50/dcb/fdb x:0 0 0 2024-04-25T01:35:55.207 INFO:tasks.workunit.client.0.smithi033.stdout:7/624: link d4/d2e/d7f/fab d4/d2e/d6b/d1f/fc7 0 2024-04-25T01:35:55.211 INFO:tasks.workunit.client.0.smithi033.stdout:8/647: dread d6/d45/d51/f4f [0,4194304] 0 2024-04-25T01:35:55.213 INFO:tasks.workunit.client.0.smithi033.stdout:4/669: unlink d6/d14/d1d/d5f/c73 0 2024-04-25T01:35:55.213 INFO:tasks.workunit.client.0.smithi033.stdout:8/648: dread - d6/d45/d51/d5d/d9d/fce zero size 2024-04-25T01:35:55.213 INFO:tasks.workunit.client.1.smithi138.stdout:0/69: truncate d6/df/f12 1108115 0 2024-04-25T01:35:55.213 INFO:tasks.workunit.client.1.smithi138.stdout:8/193: dwrite d3/d8/da/db/d15/f34 [0,4194304] 0 2024-04-25T01:35:55.213 INFO:tasks.workunit.client.1.smithi138.stdout:0/70: readlink d6/la 0 2024-04-25T01:35:55.215 INFO:tasks.workunit.client.1.smithi138.stdout:0/71: write d6/f9 [468044,112765] 0 2024-04-25T01:35:55.215 INFO:tasks.workunit.client.1.smithi138.stdout:8/194: chown d3/d8/da/db/l28 2422 1 2024-04-25T01:35:55.215 INFO:tasks.workunit.client.0.smithi033.stdout:1/437: mkdir d2/d84 0 2024-04-25T01:35:55.216 INFO:tasks.workunit.client.1.smithi138.stdout:8/195: write d3/d8/da/f43 [872480,106889] 0 2024-04-25T01:35:55.218 INFO:tasks.workunit.client.1.smithi138.stdout:8/196: write d3/f3c [368099,103980] 0 2024-04-25T01:35:55.220 INFO:tasks.workunit.client.1.smithi138.stdout:8/197: dread - d3/d8/f40 zero size 2024-04-25T01:35:55.222 INFO:tasks.workunit.client.1.smithi138.stdout:8/198: write d3/f5 [1002943,9433] 0 2024-04-25T01:35:55.225 INFO:tasks.workunit.client.0.smithi033.stdout:5/649: dread d1c/f22 [0,4194304] 0 2024-04-25T01:35:55.226 INFO:tasks.workunit.client.1.smithi138.stdout:9/56: write d4/d8/fd [960968,94641] 0 2024-04-25T01:35:55.229 INFO:tasks.workunit.client.0.smithi033.stdout:7/625: mknod d4/d2e/d7f/d98/dbf/cc8 0 2024-04-25T01:35:55.230 INFO:tasks.workunit.client.1.smithi138.stdout:7/62: dwrite da/f15 [0,4194304] 0 2024-04-25T01:35:55.232 INFO:tasks.workunit.client.1.smithi138.stdout:7/63: write f4 [2900515,115294] 0 2024-04-25T01:35:55.233 INFO:tasks.workunit.client.1.smithi138.stdout:8/199: unlink d3/d16/d1f/l35 0 2024-04-25T01:35:55.235 INFO:tasks.workunit.client.0.smithi033.stdout:4/670: creat d6/d12/fd1 x:0 0 0 2024-04-25T01:35:55.235 INFO:tasks.workunit.client.0.smithi033.stdout:3/657: dwrite d8/dd/d3d/d63/fca [4194304,4194304] 0 2024-04-25T01:35:55.236 INFO:tasks.workunit.client.1.smithi138.stdout:8/200: fdatasync d3/f38 0 2024-04-25T01:35:55.236 INFO:tasks.workunit.client.0.smithi033.stdout:8/649: creat d6/d45/d51/d55/ff6 x:0 0 0 2024-04-25T01:35:55.237 INFO:tasks.workunit.client.1.smithi138.stdout:0/72: fsync d6/fe 0 2024-04-25T01:35:55.239 INFO:tasks.workunit.client.0.smithi033.stdout:8/650: write d6/d45/d51/d5d/d9d/fce [531469,97049] 0 2024-04-25T01:35:55.240 INFO:tasks.workunit.client.0.smithi033.stdout:5/650: mkdir d1c/d2a/d50/d52/da6/ddc 0 2024-04-25T01:35:55.241 INFO:tasks.workunit.client.1.smithi138.stdout:9/57: dwrite d4/f11 [0,4194304] 0 2024-04-25T01:35:55.243 INFO:tasks.workunit.client.0.smithi033.stdout:8/651: truncate d6/db/d80/db4/fd6 227371 0 2024-04-25T01:35:55.243 INFO:tasks.workunit.client.1.smithi138.stdout:0/73: creat d6/df/f14 x:0 0 0 2024-04-25T01:35:55.244 INFO:tasks.workunit.client.0.smithi033.stdout:8/652: write d6/d45/d51/d55/fa9 [696943,51675] 0 2024-04-25T01:35:55.245 INFO:tasks.workunit.client.1.smithi138.stdout:8/201: dread - d3/d8/da/db/d15/f21 zero size 2024-04-25T01:35:55.246 INFO:tasks.workunit.client.0.smithi033.stdout:8/653: stat d6/d45/d51/d55/f5c 0 2024-04-25T01:35:55.247 INFO:tasks.workunit.client.1.smithi138.stdout:7/64: dread da/de/f1c [0,4194304] 0 2024-04-25T01:35:55.247 INFO:tasks.workunit.client.0.smithi033.stdout:3/658: read d8/dd/d3d/fc8 [84788,6658] 0 2024-04-25T01:35:55.248 INFO:tasks.workunit.client.0.smithi033.stdout:5/651: chown d1c/d2a/c43 16893467 1 2024-04-25T01:35:55.249 INFO:tasks.workunit.client.1.smithi138.stdout:0/74: dread - d6/df/f14 zero size 2024-04-25T01:35:55.255 INFO:tasks.workunit.client.1.smithi138.stdout:9/58: dwrite d4/d8/fd [0,4194304] 0 2024-04-25T01:35:55.256 INFO:tasks.workunit.client.1.smithi138.stdout:0/75: creat d6/df/f15 x:0 0 0 2024-04-25T01:35:55.256 INFO:tasks.workunit.client.0.smithi033.stdout:8/654: rename d6/d45/d51/d5d/d6e/cb2 to d6/d45/d52/db0/dbc/cf7 0 2024-04-25T01:35:55.257 INFO:tasks.workunit.client.1.smithi138.stdout:0/76: truncate d6/f13 4896218 0 2024-04-25T01:35:55.258 INFO:tasks.workunit.client.0.smithi033.stdout:0/552: truncate d7/da/df/f17 132334 0 2024-04-25T01:35:55.260 INFO:tasks.workunit.client.0.smithi033.stdout:3/659: dwrite d8/dd/d3d/d63/d7c/d3c/d98/da1/dc3/fd0 [0,4194304] 0 2024-04-25T01:35:55.263 INFO:tasks.workunit.client.1.smithi138.stdout:7/65: link da/de/d13/l19 da/l1f 0 2024-04-25T01:35:55.265 INFO:tasks.workunit.client.1.smithi138.stdout:8/202: getdents d3/d44 0 2024-04-25T01:35:55.265 INFO:tasks.workunit.client.1.smithi138.stdout:7/66: write da/fc [1023324,73178] 0 2024-04-25T01:35:55.265 INFO:tasks.workunit.client.1.smithi138.stdout:9/59: dwrite f1 [0,4194304] 0 2024-04-25T01:35:55.267 INFO:tasks.workunit.client.1.smithi138.stdout:0/77: dread d6/df/f10 [0,4194304] 0 2024-04-25T01:35:55.268 INFO:tasks.workunit.client.0.smithi033.stdout:8/655: rename d6/d45/d52/c92 to d6/d45/d52/dcc/cf8 0 2024-04-25T01:35:55.269 INFO:tasks.workunit.client.0.smithi033.stdout:0/553: mknod d7/d2d/d4d/d9c/cc2 0 2024-04-25T01:35:55.272 INFO:tasks.workunit.client.1.smithi138.stdout:0/78: unlink c3 0 2024-04-25T01:35:55.272 INFO:tasks.workunit.client.0.smithi033.stdout:7/626: truncate d4/f43 1565185 0 2024-04-25T01:35:55.275 INFO:tasks.workunit.client.0.smithi033.stdout:8/656: unlink d6/d45/d52/d2e/d95/cbb 0 2024-04-25T01:35:55.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/60: rename d4/d8/fd to d4/f13 0 2024-04-25T01:35:55.276 INFO:tasks.workunit.client.0.smithi033.stdout:8/657: fsync d6/db/d80/d89/fca 0 2024-04-25T01:35:55.277 INFO:tasks.workunit.client.1.smithi138.stdout:8/203: dwrite d3/d8/da/de/f10 [0,4194304] 0 2024-04-25T01:35:55.277 INFO:tasks.workunit.client.1.smithi138.stdout:9/61: read f1 [3957413,15056] 0 2024-04-25T01:35:55.278 INFO:tasks.workunit.client.0.smithi033.stdout:3/660: mkdir d8/dd/d3d/d4f/da4/db2/dc7/ddf 0 2024-04-25T01:35:55.279 INFO:tasks.workunit.client.0.smithi033.stdout:4/671: truncate d6/d14/ff 7446380 0 2024-04-25T01:35:55.279 INFO:tasks.workunit.client.1.smithi138.stdout:7/67: link da/l11 da/l20 0 2024-04-25T01:35:55.279 INFO:tasks.workunit.client.1.smithi138.stdout:0/79: write d6/df/f12 [1228637,95505] 0 2024-04-25T01:35:55.279 INFO:tasks.workunit.client.0.smithi033.stdout:4/672: dread - d6/d12/d38/f91 zero size 2024-04-25T01:35:55.280 INFO:tasks.workunit.client.1.smithi138.stdout:7/68: write da/fc [1962163,28935] 0 2024-04-25T01:35:55.281 INFO:tasks.workunit.client.1.smithi138.stdout:8/204: write d3/d16/f1c [3369012,117026] 0 2024-04-25T01:35:55.281 INFO:tasks.workunit.client.0.smithi033.stdout:1/438: rename d2/d11/d28/f2c to d2/d11/d28/f85 0 2024-04-25T01:35:55.282 INFO:tasks.workunit.client.1.smithi138.stdout:7/69: write da/de/f14 [844452,69653] 0 2024-04-25T01:35:55.290 INFO:tasks.workunit.client.1.smithi138.stdout:0/80: link d6/f9 d6/f16 0 2024-04-25T01:35:55.291 INFO:tasks.workunit.client.1.smithi138.stdout:7/70: dwrite f9 [0,4194304] 0 2024-04-25T01:35:55.294 INFO:tasks.workunit.client.1.smithi138.stdout:0/81: mkdir d6/d17 0 2024-04-25T01:35:55.295 INFO:tasks.workunit.client.1.smithi138.stdout:7/71: mkdir da/de/d13/d21 0 2024-04-25T01:35:55.296 INFO:tasks.workunit.client.1.smithi138.stdout:0/82: chown f1 531329 1 2024-04-25T01:35:55.297 INFO:tasks.workunit.client.1.smithi138.stdout:0/83: chown d6/df/f15 3600 1 2024-04-25T01:35:55.299 INFO:tasks.workunit.client.1.smithi138.stdout:0/84: rmdir d6/d17 0 2024-04-25T01:35:55.306 INFO:tasks.workunit.client.0.smithi033.stdout:1/439: write d2/f20 [5896037,89818] 0 2024-04-25T01:35:55.308 INFO:tasks.workunit.client.0.smithi033.stdout:4/673: fdatasync d6/d12/f40 0 2024-04-25T01:35:55.312 INFO:tasks.workunit.client.0.smithi033.stdout:4/674: write d6/d14/d35/d44/d45/d46/d57/f62 [4280612,37441] 0 2024-04-25T01:35:55.313 INFO:tasks.workunit.client.1.smithi138.stdout:7/72: fsync da/fc 0 2024-04-25T01:35:55.314 INFO:tasks.workunit.client.0.smithi033.stdout:4/675: write d6/d14/d48/f99 [5139845,112690] 0 2024-04-25T01:35:55.317 INFO:tasks.workunit.client.0.smithi033.stdout:3/661: mknod d8/dd/d3d/d4f/da4/dc4/ce0 0 2024-04-25T01:35:55.317 INFO:tasks.workunit.client.1.smithi138.stdout:7/73: unlink da/lb 0 2024-04-25T01:35:55.317 INFO:tasks.workunit.client.0.smithi033.stdout:4/676: dread - d6/d14/d35/d44/d45/d46/d57/d70/d87/faf zero size 2024-04-25T01:35:55.318 INFO:tasks.workunit.client.0.smithi033.stdout:5/652: dread d1c/d2a/d50/d52/da6/f63 [0,4194304] 0 2024-04-25T01:35:55.321 INFO:tasks.workunit.client.0.smithi033.stdout:1/440: stat d2/d11/l23 0 2024-04-25T01:35:55.322 INFO:tasks.workunit.client.0.smithi033.stdout:7/627: write d4/d2e/d6b/d1f/fc7 [438218,66344] 0 2024-04-25T01:35:55.322 INFO:tasks.workunit.client.1.smithi138.stdout:9/62: getdents d4/d8 0 2024-04-25T01:35:55.324 INFO:tasks.workunit.client.1.smithi138.stdout:9/63: write f1 [5335054,115299] 0 2024-04-25T01:35:55.325 INFO:tasks.workunit.client.1.smithi138.stdout:8/205: write d3/d8/da/db/d15/f21 [188600,4287] 0 2024-04-25T01:35:55.327 INFO:tasks.workunit.client.1.smithi138.stdout:7/74: dwrite f4 [0,4194304] 0 2024-04-25T01:35:55.327 INFO:tasks.workunit.client.0.smithi033.stdout:1/441: dread d2/d5/d75/f53 [0,4194304] 0 2024-04-25T01:35:55.328 INFO:tasks.workunit.client.1.smithi138.stdout:8/206: write d3/d8/da/de/f10 [1676708,89431] 0 2024-04-25T01:35:55.328 INFO:tasks.workunit.client.1.smithi138.stdout:1/72: sync 2024-04-25T01:35:55.328 INFO:tasks.workunit.client.1.smithi138.stdout:3/81: sync 2024-04-25T01:35:55.329 INFO:tasks.workunit.client.1.smithi138.stdout:2/125: sync 2024-04-25T01:35:55.329 INFO:tasks.workunit.client.1.smithi138.stdout:4/55: sync 2024-04-25T01:35:55.329 INFO:tasks.workunit.client.1.smithi138.stdout:9/64: rmdir d4/d8 39 2024-04-25T01:35:55.330 INFO:tasks.workunit.client.0.smithi033.stdout:4/677: creat d6/d14/d35/d3a/d4d/d5e/d7c/d94/fd2 x:0 0 0 2024-04-25T01:35:55.333 INFO:tasks.workunit.client.1.smithi138.stdout:2/126: fsync d0/d18/f1e 0 2024-04-25T01:35:55.335 INFO:tasks.workunit.client.0.smithi033.stdout:0/554: dwrite d7/d2d/f67 [0,4194304] 0 2024-04-25T01:35:55.336 INFO:tasks.workunit.client.0.smithi033.stdout:5/653: mkdir d1c/d2a/d50/db1/dd3/ddd 0 2024-04-25T01:35:55.337 INFO:tasks.workunit.client.1.smithi138.stdout:4/56: read d0/f4 [3802791,120015] 0 2024-04-25T01:35:55.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/127: read d0/d18/f1b [38933,8317] 0 2024-04-25T01:35:55.340 INFO:tasks.workunit.client.0.smithi033.stdout:1/442: unlink d2/d11/l23 0 2024-04-25T01:35:55.341 INFO:tasks.workunit.client.1.smithi138.stdout:3/82: creat db/df/d17/f1c x:0 0 0 2024-04-25T01:35:55.344 INFO:tasks.workunit.client.1.smithi138.stdout:3/83: chown db 4 1 2024-04-25T01:35:55.346 INFO:tasks.workunit.client.0.smithi033.stdout:4/678: symlink d6/d14/d35/d3a/d4d/d5e/d7c/d94/ld3 0 2024-04-25T01:35:55.346 INFO:tasks.workunit.client.1.smithi138.stdout:3/84: readlink l2 0 2024-04-25T01:35:55.347 INFO:tasks.workunit.client.1.smithi138.stdout:9/65: mknod d4/c14 0 2024-04-25T01:35:55.349 INFO:tasks.workunit.client.0.smithi033.stdout:4/679: truncate d6/d12/d38/f7e 5240212 0 2024-04-25T01:35:55.349 INFO:tasks.workunit.client.0.smithi033.stdout:3/662: link c2 d8/dd/d3d/d63/d7c/d3c/d98/da1/ce1 0 2024-04-25T01:35:55.349 INFO:tasks.workunit.client.1.smithi138.stdout:4/57: mknod d0/cd 0 2024-04-25T01:35:55.349 INFO:tasks.workunit.client.1.smithi138.stdout:1/73: dread d3/d4/f9 [0,4194304] 0 2024-04-25T01:35:55.350 INFO:tasks.workunit.client.1.smithi138.stdout:0/85: dwrite f1 [0,4194304] 0 2024-04-25T01:35:55.351 INFO:tasks.workunit.client.0.smithi033.stdout:1/443: symlink d2/d5/l86 0 2024-04-25T01:35:55.353 INFO:tasks.workunit.client.1.smithi138.stdout:3/85: symlink db/df/l1d 0 2024-04-25T01:35:55.353 INFO:tasks.workunit.client.1.smithi138.stdout:8/207: link d3/l17 d3/d8/da/de/l49 0 2024-04-25T01:35:55.354 INFO:tasks.workunit.client.0.smithi033.stdout:5/654: creat d1c/d2a/d3d/d83/fde x:0 0 0 2024-04-25T01:35:55.354 INFO:tasks.workunit.client.0.smithi033.stdout:5/655: chown d1c/f5e 2468708 1 2024-04-25T01:35:55.354 INFO:tasks.workunit.client.0.smithi033.stdout:4/680: rename d6/d14/d35/d44/d45/d46/d64/f69 to d6/d14/d1d/d5f/fd4 0 2024-04-25T01:35:55.356 INFO:tasks.workunit.client.1.smithi138.stdout:7/75: dwrite da/de/f1a [0,4194304] 0 2024-04-25T01:35:55.357 INFO:tasks.workunit.client.0.smithi033.stdout:3/663: dwrite d8/dd/fdb [0,4194304] 0 2024-04-25T01:35:55.357 INFO:tasks.workunit.client.1.smithi138.stdout:8/208: chown d3/d16/c1d 30449593 1 2024-04-25T01:35:55.358 INFO:tasks.workunit.client.1.smithi138.stdout:8/209: chown d3/d8/da/db/f14 31951527 1 2024-04-25T01:35:55.360 INFO:tasks.workunit.client.0.smithi033.stdout:4/681: mkdir d6/d14/d1d/d3b/dd5 0 2024-04-25T01:35:55.361 INFO:tasks.workunit.client.0.smithi033.stdout:3/664: truncate d8/dd/d3d/d63/d7c/fbe 646656 0 2024-04-25T01:35:55.363 INFO:tasks.workunit.client.0.smithi033.stdout:5/656: mknod d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/cdf 0 2024-04-25T01:35:55.364 INFO:tasks.workunit.client.1.smithi138.stdout:4/58: dread d0/fa [0,4194304] 0 2024-04-25T01:35:55.366 INFO:tasks.workunit.client.1.smithi138.stdout:7/76: read da/de/f1a [1755847,70251] 0 2024-04-25T01:35:55.367 INFO:tasks.workunit.client.1.smithi138.stdout:2/128: write d0/f8 [584104,25258] 0 2024-04-25T01:35:55.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/74: link d3/f15 d3/d4/f19 0 2024-04-25T01:35:55.369 INFO:tasks.workunit.client.1.smithi138.stdout:7/77: fdatasync da/f15 0 2024-04-25T01:35:55.375 INFO:tasks.workunit.client.1.smithi138.stdout:0/86: link d6/f13 d6/df/f18 0 2024-04-25T01:35:55.376 INFO:tasks.workunit.client.1.smithi138.stdout:8/210: dwrite d3/d16/f3b [0,4194304] 0 2024-04-25T01:35:55.377 INFO:tasks.workunit.client.1.smithi138.stdout:4/59: unlink d0/c9 0 2024-04-25T01:35:55.377 INFO:tasks.workunit.client.1.smithi138.stdout:9/66: truncate f1 604984 0 2024-04-25T01:35:55.378 INFO:tasks.workunit.client.0.smithi033.stdout:7/628: truncate d4/f7a 3664488 0 2024-04-25T01:35:55.379 INFO:tasks.workunit.client.1.smithi138.stdout:2/129: creat d0/d21/f27 x:0 0 0 2024-04-25T01:35:55.379 INFO:tasks.workunit.client.1.smithi138.stdout:7/78: symlink da/l22 0 2024-04-25T01:35:55.381 INFO:tasks.workunit.client.0.smithi033.stdout:5/657: dwrite d1c/d2a/fb2 [0,4194304] 0 2024-04-25T01:35:55.382 INFO:tasks.workunit.client.1.smithi138.stdout:3/86: dwrite db/df/d17/f1c [0,4194304] 0 2024-04-25T01:35:55.383 INFO:tasks.workunit.client.1.smithi138.stdout:1/75: write d3/d4/f19 [13044,7957] 0 2024-04-25T01:35:55.385 INFO:tasks.workunit.client.1.smithi138.stdout:0/87: mkdir d6/d19 0 2024-04-25T01:35:55.389 INFO:tasks.workunit.client.0.smithi033.stdout:5/658: creat d1c/d2a/d50/dcb/fe0 x:0 0 0 2024-04-25T01:35:55.390 INFO:tasks.workunit.client.1.smithi138.stdout:0/88: dread f1 [0,4194304] 0 2024-04-25T01:35:55.390 INFO:tasks.workunit.client.0.smithi033.stdout:1/444: creat d2/d5/d6c/f87 x:0 0 0 2024-04-25T01:35:55.392 INFO:tasks.workunit.client.0.smithi033.stdout:4/682: truncate d6/d12/d38/f41 3438284 0 2024-04-25T01:35:55.396 INFO:tasks.workunit.client.1.smithi138.stdout:1/76: mknod d3/d4/da/c1a 0 2024-04-25T01:35:55.399 INFO:tasks.workunit.client.1.smithi138.stdout:8/211: dwrite d3/d8/da/de/ff [0,4194304] 0 2024-04-25T01:35:55.400 INFO:tasks.workunit.client.0.smithi033.stdout:3/665: dread d8/dd/d3d/d63/fca [0,4194304] 0 2024-04-25T01:35:55.402 INFO:tasks.workunit.client.0.smithi033.stdout:1/445: mknod d2/d2e/d34/c88 0 2024-04-25T01:35:55.404 INFO:tasks.workunit.client.0.smithi033.stdout:5/659: rename d1c/d2a/d50/dcb to d1c/d2a/d50/d8d/de1 0 2024-04-25T01:35:55.404 INFO:tasks.workunit.client.1.smithi138.stdout:9/67: dread f1 [0,4194304] 0 2024-04-25T01:35:55.406 INFO:tasks.workunit.client.1.smithi138.stdout:3/87: rename f9 to db/d14/f1e 0 2024-04-25T01:35:55.408 INFO:tasks.workunit.client.1.smithi138.stdout:3/88: rename db/df/d17 to db/df/d17/d1f 22 2024-04-25T01:35:55.410 INFO:tasks.workunit.client.0.smithi033.stdout:7/629: truncate d4/d2e/d50/f55 407666 0 2024-04-25T01:35:55.411 INFO:tasks.workunit.client.0.smithi033.stdout:4/683: dread d6/d14/d35/d44/d45/d46/d57/f62 [0,4194304] 0 2024-04-25T01:35:55.411 INFO:tasks.workunit.client.1.smithi138.stdout:0/89: mkdir d6/d19/d1a 0 2024-04-25T01:35:55.411 INFO:tasks.workunit.client.0.smithi033.stdout:3/666: dread d8/dd/d3d/d63/d7c/d3c/d98/da1/dc3/fd0 [0,4194304] 0 2024-04-25T01:35:55.412 INFO:tasks.workunit.client.0.smithi033.stdout:5/660: unlink d1c/f1d 0 2024-04-25T01:35:55.414 INFO:tasks.workunit.client.0.smithi033.stdout:3/667: read d8/dd/d3d/d63/fca [3986392,50876] 0 2024-04-25T01:35:55.415 INFO:tasks.workunit.client.0.smithi033.stdout:4/684: creat d6/d14/d35/d44/d45/d46/d64/fd6 x:0 0 0 2024-04-25T01:35:55.416 INFO:tasks.workunit.client.0.smithi033.stdout:1/446: creat d2/d11/d28/f89 x:0 0 0 2024-04-25T01:35:55.418 INFO:tasks.workunit.client.1.smithi138.stdout:4/60: write d0/f4 [4845486,94503] 0 2024-04-25T01:35:55.421 INFO:tasks.workunit.client.0.smithi033.stdout:7/630: truncate d4/fb 1593489 0 2024-04-25T01:35:55.422 INFO:tasks.workunit.client.1.smithi138.stdout:9/68: mkdir d4/d8/d15 0 2024-04-25T01:35:55.423 INFO:tasks.workunit.client.1.smithi138.stdout:4/61: stat d0/fa 0 2024-04-25T01:35:55.475 INFO:tasks.workunit.client.0.smithi033.stdout:7/631: read d4/d23/d3f/d45/f5e [3142432,117143] 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:1/447: rmdir d2/d2e/d49 39 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:3/668: rename d8/dd/d11/c14 to d8/dd/d3d/d4f/da4/db2/ce2 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:5/661: dwrite d1c/d2a/d50/d52/da6/d4a/fb4 [0,4194304] 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:5/662: fdatasync d1c/d2a/d3d/d83/fde 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:7/632: mknod d4/d23/d3f/cc9 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:3/669: symlink d8/dd/d3d/d4f/da4/db2/dbf/dc5/le3 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:1/448: symlink d2/d7f/l8a 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:7/633: dread d4/d2e/d6b/f47 [0,4194304] 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:5/663: dread - d1c/d2a/d50/d5b/fd7 zero size 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:4/685: truncate d6/d12/f26 122573 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:7/634: mkdir d4/d2e/d7f/d98/dca 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:3/670: dwrite d8/dd/d3d/d63/d7c/d3c/d98/da1/fc1 [0,4194304] 0 2024-04-25T01:35:55.476 INFO:tasks.workunit.client.0.smithi033.stdout:7/635: mkdir d4/d2e/d65/dcb 0 2024-04-25T01:35:55.477 INFO:tasks.workunit.client.1.smithi138.stdout:2/130: truncate d0/f14 1650253 0 2024-04-25T01:35:55.477 INFO:tasks.workunit.client.1.smithi138.stdout:0/90: dread d6/fc [0,4194304] 0 2024-04-25T01:35:55.477 INFO:tasks.workunit.client.1.smithi138.stdout:4/62: symlink d0/le 0 2024-04-25T01:35:55.477 INFO:tasks.workunit.client.1.smithi138.stdout:8/212: truncate d3/d8/da/db/d15/f34 2775118 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:3/89: truncate f1 726007 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:1/77: dwrite d3/db/f17 [0,4194304] 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:9/69: rename d4/d8/fc to d4/d8/d15/f16 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/63: rename d0/f4 to d0/ff 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:8/213: dread d3/d8/da/f43 [0,4194304] 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:0/91: symlink d6/d19/d1a/l1b 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:8/214: write d3/f5 [3429996,99148] 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:8/215: dread d3/f1e [0,4194304] 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:3/90: mkdir db/df/d17/d20 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:8/216: read d3/d8/da/de/f10 [239607,114155] 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:0/92: mkdir d6/d19/d1c 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:0/93: read d6/f11 [373783,57701] 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/64: dwrite d0/ff [0,4194304] 0 2024-04-25T01:35:55.478 INFO:tasks.workunit.client.1.smithi138.stdout:3/91: write db/f12 [105158,10628] 0 2024-04-25T01:35:55.479 INFO:tasks.workunit.client.1.smithi138.stdout:8/217: symlink d3/d16/d1f/d2a/l4a 0 2024-04-25T01:35:55.479 INFO:tasks.workunit.client.1.smithi138.stdout:1/78: fsync d3/d4/fd 0 2024-04-25T01:35:55.479 INFO:tasks.workunit.client.1.smithi138.stdout:2/131: dwrite d0/dc/d15/f16 [0,4194304] 0 2024-04-25T01:35:55.479 INFO:tasks.workunit.client.1.smithi138.stdout:4/65: rmdir d0 39 2024-04-25T01:35:55.481 INFO:tasks.workunit.client.1.smithi138.stdout:8/218: fdatasync d3/d8/da/f43 0 2024-04-25T01:35:55.484 INFO:tasks.workunit.client.1.smithi138.stdout:9/70: creat d4/f17 x:0 0 0 2024-04-25T01:35:55.486 INFO:tasks.workunit.client.1.smithi138.stdout:1/79: creat d3/db/f1b x:0 0 0 2024-04-25T01:35:55.490 INFO:tasks.workunit.client.1.smithi138.stdout:8/219: dwrite d3/d16/f3b [0,4194304] 0 2024-04-25T01:35:55.491 INFO:tasks.workunit.client.1.smithi138.stdout:2/132: creat d0/dc/d15/f28 x:0 0 0 2024-04-25T01:35:55.492 INFO:tasks.workunit.client.1.smithi138.stdout:2/133: chown d0/dc/f1a 1777 1 2024-04-25T01:35:55.494 INFO:tasks.workunit.client.1.smithi138.stdout:2/134: truncate d0/dc/d15/f28 958946 0 2024-04-25T01:35:55.498 INFO:tasks.workunit.client.1.smithi138.stdout:1/80: mknod d3/d4/da/c1c 0 2024-04-25T01:35:55.500 INFO:tasks.workunit.client.1.smithi138.stdout:2/135: dread d0/fe [0,4194304] 0 2024-04-25T01:35:55.501 INFO:tasks.workunit.client.1.smithi138.stdout:9/71: mknod d4/c18 0 2024-04-25T01:35:55.501 INFO:tasks.workunit.client.1.smithi138.stdout:8/220: dread d3/d8/da/db/f14 [0,4194304] 0 2024-04-25T01:35:55.503 INFO:tasks.workunit.client.1.smithi138.stdout:0/94: getdents d6/df 0 2024-04-25T01:35:55.503 INFO:tasks.workunit.client.1.smithi138.stdout:9/72: write d4/f17 [295274,74154] 0 2024-04-25T01:35:55.504 INFO:tasks.workunit.client.1.smithi138.stdout:8/221: write d3/d8/da/db/f11 [5097640,105245] 0 2024-04-25T01:35:55.505 INFO:tasks.workunit.client.1.smithi138.stdout:1/81: unlink d3/d4/f19 0 2024-04-25T01:35:55.506 INFO:tasks.workunit.client.1.smithi138.stdout:8/222: truncate d3/d8/f47 585771 0 2024-04-25T01:35:55.506 INFO:tasks.workunit.client.1.smithi138.stdout:8/223: fdatasync d3/f5 0 2024-04-25T01:35:55.508 INFO:tasks.workunit.client.1.smithi138.stdout:2/136: mknod d0/d21/c29 0 2024-04-25T01:35:55.509 INFO:tasks.workunit.client.0.smithi033.stdout:9/584: sync 2024-04-25T01:35:55.511 INFO:tasks.workunit.client.1.smithi138.stdout:3/92: truncate db/df/d17/f1c 16871 0 2024-04-25T01:35:55.511 INFO:tasks.workunit.client.1.smithi138.stdout:1/82: dread d3/db/f17 [0,4194304] 0 2024-04-25T01:35:55.511 INFO:tasks.workunit.client.0.smithi033.stdout:4/686: rename d6/d14/f53 to d6/d14/fd7 0 2024-04-25T01:35:55.513 INFO:tasks.workunit.client.0.smithi033.stdout:1/449: getdents d2/d5/d75 0 2024-04-25T01:35:55.513 INFO:tasks.workunit.client.1.smithi138.stdout:0/95: rename d6/d19/d1a to d6/d1d 0 2024-04-25T01:35:55.515 INFO:tasks.workunit.client.0.smithi033.stdout:3/671: truncate d8/dd/d1d/f93 3468359 0 2024-04-25T01:35:55.516 INFO:tasks.workunit.client.0.smithi033.stdout:9/585: creat d0/d11/da1/fd3 x:0 0 0 2024-04-25T01:35:55.517 INFO:tasks.workunit.client.0.smithi033.stdout:7/636: dwrite d4/d57/fae [0,4194304] 0 2024-04-25T01:35:55.517 INFO:tasks.workunit.client.0.smithi033.stdout:9/586: read - d0/f2d zero size 2024-04-25T01:35:55.520 INFO:tasks.workunit.client.1.smithi138.stdout:2/137: creat d0/d18/f2a x:0 0 0 2024-04-25T01:35:55.521 INFO:tasks.workunit.client.1.smithi138.stdout:1/83: creat d3/db/f1d x:0 0 0 2024-04-25T01:35:55.523 INFO:tasks.workunit.client.0.smithi033.stdout:1/450: creat d2/d2e/d34/d60/f8b x:0 0 0 2024-04-25T01:35:55.525 INFO:tasks.workunit.client.1.smithi138.stdout:0/96: creat d6/f1e x:0 0 0 2024-04-25T01:35:55.525 INFO:tasks.workunit.client.0.smithi033.stdout:3/672: mknod d8/ce4 0 2024-04-25T01:35:55.527 INFO:tasks.workunit.client.0.smithi033.stdout:5/664: dread ff [0,4194304] 0 2024-04-25T01:35:55.527 INFO:tasks.workunit.client.0.smithi033.stdout:7/637: dwrite d4/d2e/d6b/d1f/f71 [0,4194304] 0 2024-04-25T01:35:55.528 INFO:tasks.workunit.client.0.smithi033.stdout:4/687: mknod d6/d14/cd8 0 2024-04-25T01:35:55.528 INFO:tasks.workunit.client.0.smithi033.stdout:9/587: creat d0/d2/dd/dba/fd4 x:0 0 0 2024-04-25T01:35:55.530 INFO:tasks.workunit.client.1.smithi138.stdout:2/138: write d0/d18/f1b [1043431,47652] 0 2024-04-25T01:35:55.531 INFO:tasks.workunit.client.1.smithi138.stdout:1/84: creat d3/d4/da/f1e x:0 0 0 2024-04-25T01:35:55.533 INFO:tasks.workunit.client.0.smithi033.stdout:9/588: rmdir d0/d2/dd/df/d1b/d3d 39 2024-04-25T01:35:55.534 INFO:tasks.workunit.client.0.smithi033.stdout:1/451: dwrite d2/d2e/f73 [0,4194304] 0 2024-04-25T01:35:55.535 INFO:tasks.workunit.client.0.smithi033.stdout:9/589: stat d0/d2/d43/d4b/d55/fbc 0 2024-04-25T01:35:55.536 INFO:tasks.workunit.client.1.smithi138.stdout:2/139: dread d0/dc/f12 [0,4194304] 0 2024-04-25T01:35:55.536 INFO:tasks.workunit.client.1.smithi138.stdout:3/93: rename db/d14/f1e to db/df/d19/d1a/f21 0 2024-04-25T01:35:55.538 INFO:tasks.workunit.client.1.smithi138.stdout:3/94: write db/f11 [3932366,122842] 0 2024-04-25T01:35:55.538 INFO:tasks.workunit.client.0.smithi033.stdout:4/688: rename d6/d14/fac to d6/d14/d35/d3a/d4d/fd9 0 2024-04-25T01:35:55.539 INFO:tasks.workunit.client.0.smithi033.stdout:5/665: rename d1c/d2a/d50/d52/da6 to d1c/d2a/d50/d52/da6/ddc/de2 22 2024-04-25T01:35:55.539 INFO:tasks.workunit.client.0.smithi033.stdout:7/638: dread d4/d2e/d6b/d67/f7d [0,4194304] 0 2024-04-25T01:35:55.540 INFO:tasks.workunit.client.1.smithi138.stdout:0/97: creat d6/f1f x:0 0 0 2024-04-25T01:35:55.540 INFO:tasks.workunit.client.1.smithi138.stdout:9/73: link d4/ca d4/c19 0 2024-04-25T01:35:55.540 INFO:tasks.workunit.client.1.smithi138.stdout:3/95: write db/f12 [287067,28375] 0 2024-04-25T01:35:55.540 INFO:tasks.workunit.client.1.smithi138.stdout:1/85: mknod d3/d4/c1f 0 2024-04-25T01:35:55.542 INFO:tasks.workunit.client.0.smithi033.stdout:9/590: creat d0/d2/d43/d73/dc4/fd5 x:0 0 0 2024-04-25T01:35:55.542 INFO:tasks.workunit.client.1.smithi138.stdout:9/74: dread d4/f13 [4194304,4194304] 0 2024-04-25T01:35:55.543 INFO:tasks.workunit.client.0.smithi033.stdout:5/666: dread - d1c/d2a/d50/d52/da6/d3e/d70/d7a/fa9 zero size 2024-04-25T01:35:55.546 INFO:tasks.workunit.client.0.smithi033.stdout:4/689: creat d6/d14/d35/d44/d45/d46/d64/fda x:0 0 0 2024-04-25T01:35:55.548 INFO:tasks.workunit.client.1.smithi138.stdout:0/98: fdatasync d6/df/f10 0 2024-04-25T01:35:55.548 INFO:tasks.workunit.client.1.smithi138.stdout:1/86: dwrite d3/db/f1b [0,4194304] 0 2024-04-25T01:35:55.550 INFO:tasks.workunit.client.0.smithi033.stdout:9/591: unlink d0/d2/dd/d5f/f96 0 2024-04-25T01:35:55.551 INFO:tasks.workunit.client.1.smithi138.stdout:2/140: mknod d0/dc/d15/d1f/c2b 0 2024-04-25T01:35:55.551 INFO:tasks.workunit.client.1.smithi138.stdout:8/224: link d3/c22 d3/d8/c4b 0 2024-04-25T01:35:55.552 INFO:tasks.workunit.client.1.smithi138.stdout:0/99: dread d6/f11 [0,4194304] 0 2024-04-25T01:35:55.552 INFO:tasks.workunit.client.0.smithi033.stdout:7/639: creat d4/d23/d3f/d45/d6c/fcc x:0 0 0 2024-04-25T01:35:55.552 INFO:tasks.workunit.client.1.smithi138.stdout:8/225: read f0 [462484,94365] 0 2024-04-25T01:35:55.553 INFO:tasks.workunit.client.1.smithi138.stdout:9/75: mkdir d4/d1a 0 2024-04-25T01:35:55.553 INFO:tasks.workunit.client.1.smithi138.stdout:2/141: write d0/f9 [4835959,86133] 0 2024-04-25T01:35:55.555 INFO:tasks.workunit.client.0.smithi033.stdout:4/690: fdatasync d6/d14/d1d/d3b/f50 0 2024-04-25T01:35:55.557 INFO:tasks.workunit.client.0.smithi033.stdout:5/667: creat d1c/d2a/d50/db1/fe3 x:0 0 0 2024-04-25T01:35:55.561 INFO:tasks.workunit.client.1.smithi138.stdout:8/226: dread f2 [0,4194304] 0 2024-04-25T01:35:55.561 INFO:tasks.workunit.client.1.smithi138.stdout:2/142: link d0/dc/c19 d0/dc/d15/c2c 0 2024-04-25T01:35:55.563 INFO:tasks.workunit.client.1.smithi138.stdout:5/44: sync 2024-04-25T01:35:55.563 INFO:tasks.workunit.client.1.smithi138.stdout:0/100: dread d6/f13 [0,4194304] 0 2024-04-25T01:35:55.563 INFO:tasks.workunit.client.1.smithi138.stdout:2/143: truncate d0/d21/f27 243855 0 2024-04-25T01:35:55.563 INFO:tasks.workunit.client.1.smithi138.stdout:8/227: creat d3/d16/d1f/d31/f4c x:0 0 0 2024-04-25T01:35:55.564 INFO:tasks.workunit.client.1.smithi138.stdout:5/45: fdatasync f3 0 2024-04-25T01:35:55.566 INFO:tasks.workunit.client.1.smithi138.stdout:8/228: unlink f2 0 2024-04-25T01:35:55.566 INFO:tasks.workunit.client.0.smithi033.stdout:1/452: dread d2/d11/f5e [0,4194304] 0 2024-04-25T01:35:55.567 INFO:tasks.workunit.client.1.smithi138.stdout:0/101: symlink d6/d1d/l20 0 2024-04-25T01:35:55.567 INFO:tasks.workunit.client.0.smithi033.stdout:7/640: dread d4/d2e/d50/fb5 [0,4194304] 0 2024-04-25T01:35:55.568 INFO:tasks.workunit.client.0.smithi033.stdout:5/668: stat d1c/cbe 0 2024-04-25T01:35:55.570 INFO:tasks.workunit.client.1.smithi138.stdout:2/144: dwrite d0/f14 [0,4194304] 0 2024-04-25T01:35:55.570 INFO:tasks.workunit.client.0.smithi033.stdout:4/691: rename d6/d14/fb1 to d6/d14/d48/fdb 0 2024-04-25T01:35:55.578 INFO:tasks.workunit.client.0.smithi033.stdout:9/592: dread d0/d2/dd/f41 [0,4194304] 0 2024-04-25T01:35:55.585 INFO:tasks.workunit.client.0.smithi033.stdout:7/641: creat d4/d2e/d50/fcd x:0 0 0 2024-04-25T01:35:55.595 INFO:tasks.workunit.client.1.smithi138.stdout:2/145: read - d0/d18/f1e zero size 2024-04-25T01:35:55.596 INFO:tasks.workunit.client.1.smithi138.stdout:0/102: write d6/f13 [796233,61427] 0 2024-04-25T01:35:55.597 INFO:tasks.workunit.client.1.smithi138.stdout:2/146: write d0/fe [260818,45335] 0 2024-04-25T01:35:55.602 INFO:tasks.workunit.client.1.smithi138.stdout:2/147: unlink d0/d18/f1e 0 2024-04-25T01:35:55.605 INFO:tasks.workunit.client.1.smithi138.stdout:8/229: rename d3/d16/d1f/d31/f4c to d3/d8/f4d 0 2024-04-25T01:35:55.607 INFO:tasks.workunit.client.1.smithi138.stdout:5/46: getdents d4 0 2024-04-25T01:35:55.609 INFO:tasks.workunit.client.1.smithi138.stdout:2/148: dwrite d0/dc/d15/d1f/f26 [0,4194304] 0 2024-04-25T01:35:55.610 INFO:tasks.workunit.client.1.smithi138.stdout:9/76: write f1 [1018694,105946] 0 2024-04-25T01:35:55.611 INFO:tasks.workunit.client.1.smithi138.stdout:8/230: mknod d3/d8/da/db/d15/c4e 0 2024-04-25T01:35:55.618 INFO:tasks.workunit.client.1.smithi138.stdout:0/103: write d6/f16 [375369,98372] 0 2024-04-25T01:35:55.621 INFO:tasks.workunit.client.1.smithi138.stdout:8/231: mkdir d3/d16/d1f/d2a/d4f 0 2024-04-25T01:35:55.622 INFO:tasks.workunit.client.1.smithi138.stdout:9/77: symlink d4/d1a/l1b 0 2024-04-25T01:35:55.623 INFO:tasks.workunit.client.1.smithi138.stdout:5/47: dwrite f3 [0,4194304] 0 2024-04-25T01:35:55.623 INFO:tasks.workunit.client.1.smithi138.stdout:0/104: stat d6/d1d/l20 0 2024-04-25T01:35:55.624 INFO:tasks.workunit.client.1.smithi138.stdout:5/48: stat f3 0 2024-04-25T01:35:55.626 INFO:tasks.workunit.client.1.smithi138.stdout:1/87: dwrite d3/f15 [0,4194304] 0 2024-04-25T01:35:55.626 INFO:tasks.workunit.client.1.smithi138.stdout:1/88: readlink - no filename 2024-04-25T01:35:55.631 INFO:tasks.workunit.client.1.smithi138.stdout:2/149: dread d0/f9 [0,4194304] 0 2024-04-25T01:35:55.633 INFO:tasks.workunit.client.1.smithi138.stdout:1/89: symlink d3/d4/da/l20 0 2024-04-25T01:35:55.634 INFO:tasks.workunit.client.1.smithi138.stdout:9/78: write d4/d8/d15/f16 [678472,21695] 0 2024-04-25T01:35:55.635 INFO:tasks.workunit.client.1.smithi138.stdout:5/49: chown d4/l8 3984 1 2024-04-25T01:35:55.639 INFO:tasks.workunit.client.1.smithi138.stdout:9/79: truncate f1 1256198 0 2024-04-25T01:35:55.639 INFO:tasks.workunit.client.1.smithi138.stdout:8/232: dwrite d3/f3c [0,4194304] 0 2024-04-25T01:35:55.641 INFO:tasks.workunit.client.1.smithi138.stdout:1/90: write d3/d4/da/f1e [619831,292] 0 2024-04-25T01:35:55.642 INFO:tasks.workunit.client.1.smithi138.stdout:0/105: dread d6/fc [0,4194304] 0 2024-04-25T01:35:55.643 INFO:tasks.workunit.client.1.smithi138.stdout:5/50: rename f3 to d4/f9 0 2024-04-25T01:35:55.649 INFO:tasks.workunit.client.1.smithi138.stdout:9/80: mknod d4/d8/d15/c1c 0 2024-04-25T01:35:55.649 INFO:tasks.workunit.client.1.smithi138.stdout:2/150: dwrite d0/fe [0,4194304] 0 2024-04-25T01:35:55.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/106: dread - d6/df/f15 zero size 2024-04-25T01:35:55.650 INFO:tasks.workunit.client.1.smithi138.stdout:1/91: write d3/f13 [2420065,93204] 0 2024-04-25T01:35:55.651 INFO:tasks.workunit.client.1.smithi138.stdout:8/233: mkdir d3/d16/d1f/d50 0 2024-04-25T01:35:55.652 INFO:tasks.workunit.client.1.smithi138.stdout:9/81: write d4/f11 [2956742,108138] 0 2024-04-25T01:35:55.653 INFO:tasks.workunit.client.1.smithi138.stdout:2/151: chown d0/dc/d15/f16 3 1 2024-04-25T01:35:55.653 INFO:tasks.workunit.client.1.smithi138.stdout:8/234: fsync d3/d8/da/f24 0 2024-04-25T01:35:55.655 INFO:tasks.workunit.client.1.smithi138.stdout:0/107: creat d6/d19/d1c/f21 x:0 0 0 2024-04-25T01:35:55.657 INFO:tasks.workunit.client.1.smithi138.stdout:9/82: mkdir d4/d8/d15/d1d 0 2024-04-25T01:35:55.657 INFO:tasks.workunit.client.1.smithi138.stdout:9/83: stat c3 0 2024-04-25T01:35:55.658 INFO:tasks.workunit.client.1.smithi138.stdout:2/152: unlink d0/d18/l24 0 2024-04-25T01:35:55.661 INFO:tasks.workunit.client.1.smithi138.stdout:0/108: symlink d6/d1d/l22 0 2024-04-25T01:35:55.666 INFO:tasks.workunit.client.1.smithi138.stdout:2/153: creat d0/dc/d15/d1f/f2d x:0 0 0 2024-04-25T01:35:55.671 INFO:tasks.workunit.client.1.smithi138.stdout:0/109: truncate d6/f9 1451703 0 2024-04-25T01:35:55.671 INFO:tasks.workunit.client.1.smithi138.stdout:2/154: fdatasync d0/dc/f12 0 2024-04-25T01:35:55.671 INFO:tasks.workunit.client.1.smithi138.stdout:5/51: truncate d4/f9 2876277 0 2024-04-25T01:35:55.672 INFO:tasks.workunit.client.1.smithi138.stdout:1/92: truncate d3/f13 654321 0 2024-04-25T01:35:55.674 INFO:tasks.workunit.client.1.smithi138.stdout:9/84: dwrite d4/d8/d15/f16 [0,4194304] 0 2024-04-25T01:35:55.675 INFO:tasks.workunit.client.1.smithi138.stdout:0/110: creat d6/d19/f23 x:0 0 0 2024-04-25T01:35:55.676 INFO:tasks.workunit.client.1.smithi138.stdout:1/93: symlink d3/d4/da/d16/l21 0 2024-04-25T01:35:55.677 INFO:tasks.workunit.client.1.smithi138.stdout:8/235: truncate d3/f3a 1746569 0 2024-04-25T01:35:55.678 INFO:tasks.workunit.client.1.smithi138.stdout:0/111: dread d6/fe [0,4194304] 0 2024-04-25T01:35:55.688 INFO:tasks.workunit.client.1.smithi138.stdout:5/52: write d4/f9 [2747941,47249] 0 2024-04-25T01:35:55.690 INFO:tasks.workunit.client.1.smithi138.stdout:9/85: getdents d4/d8/d15/d1d 0 2024-04-25T01:35:55.693 INFO:tasks.workunit.client.1.smithi138.stdout:5/53: mkdir d4/da 0 2024-04-25T01:35:55.694 INFO:tasks.workunit.client.0.smithi033.stdout:7/642: write d4/d23/d3f/d45/d73/dbe/fc1 [840760,38114] 0 2024-04-25T01:35:55.696 INFO:tasks.workunit.client.1.smithi138.stdout:0/112: rename d6/df/f12 to d6/d1d/f24 0 2024-04-25T01:35:55.696 INFO:tasks.workunit.client.0.smithi033.stdout:4/692: dread d6/d12/d84/f88 [0,4194304] 0 2024-04-25T01:35:55.697 INFO:tasks.workunit.client.1.smithi138.stdout:2/155: link d0/dc/d15/d1f/c2b d0/dc/d15/d1f/c2e 0 2024-04-25T01:35:55.698 INFO:tasks.workunit.client.0.smithi033.stdout:5/669: rename d1c/d2a/d50/d52/da6/l2f to d1c/d2a/d50/d8d/de1/le4 0 2024-04-25T01:35:55.698 INFO:tasks.workunit.client.1.smithi138.stdout:1/94: link d3/db/f14 d3/d4/f22 0 2024-04-25T01:35:55.699 INFO:tasks.workunit.client.0.smithi033.stdout:1/453: truncate d2/d5/d1b/d46/d5f/d63/f65 2600805 0 2024-04-25T01:35:55.701 INFO:tasks.workunit.client.0.smithi033.stdout:4/693: mkdir d6/d14/d35/d3a/d4d/d5e/ddc 0 2024-04-25T01:35:55.701 INFO:tasks.workunit.client.0.smithi033.stdout:1/454: chown d2/d2e/l80 88919418 1 2024-04-25T01:35:55.702 INFO:tasks.workunit.client.0.smithi033.stdout:7/643: truncate d4/fb 802560 0 2024-04-25T01:35:55.703 INFO:tasks.workunit.client.0.smithi033.stdout:9/593: dwrite d0/f33 [0,4194304] 0 2024-04-25T01:35:55.703 INFO:tasks.workunit.client.1.smithi138.stdout:0/113: symlink d6/d19/l25 0 2024-04-25T01:35:55.704 INFO:tasks.workunit.client.0.smithi033.stdout:4/694: fdatasync d6/d14/d35/d3a/d4c/f78 0 2024-04-25T01:35:55.704 INFO:tasks.workunit.client.1.smithi138.stdout:2/156: symlink d0/dc/d15/l2f 0 2024-04-25T01:35:55.705 INFO:tasks.workunit.client.0.smithi033.stdout:5/670: chown d1c/d2a/d50/d52/da6/d3e/d70/dae/ld1 5 1 2024-04-25T01:35:55.707 INFO:tasks.workunit.client.0.smithi033.stdout:1/455: symlink d2/d2e/d49/l8c 0 2024-04-25T01:35:55.708 INFO:tasks.workunit.client.1.smithi138.stdout:5/54: getdents d4/da 0 2024-04-25T01:35:55.708 INFO:tasks.workunit.client.0.smithi033.stdout:7/644: creat d4/d2e/d6d/fce x:0 0 0 2024-04-25T01:35:55.709 INFO:tasks.workunit.client.0.smithi033.stdout:9/594: creat d0/d2/fd6 x:0 0 0 2024-04-25T01:35:55.710 INFO:tasks.workunit.client.0.smithi033.stdout:4/695: fdatasync d6/d14/d1d/d3b/f50 0 2024-04-25T01:35:55.711 INFO:tasks.workunit.client.0.smithi033.stdout:1/456: chown d2/d11/d28/l44 86896 1 2024-04-25T01:35:55.711 INFO:tasks.workunit.client.0.smithi033.stdout:3/673: dread d8/dd/d40/f53 [4194304,4194304] 0 2024-04-25T01:35:55.712 INFO:tasks.workunit.client.0.smithi033.stdout:9/595: truncate d0/d2/dd/d20/d4e/f90 2777005 0 2024-04-25T01:35:55.712 INFO:tasks.workunit.client.1.smithi138.stdout:8/236: dwrite d3/f3a [0,4194304] 0 2024-04-25T01:35:55.713 INFO:tasks.workunit.client.0.smithi033.stdout:1/457: fdatasync d2/f81 0 2024-04-25T01:35:55.713 INFO:tasks.workunit.client.1.smithi138.stdout:5/55: write d4/f9 [2075183,3874] 0 2024-04-25T01:35:55.713 INFO:tasks.workunit.client.1.smithi138.stdout:1/95: dwrite d3/f10 [0,4194304] 0 2024-04-25T01:35:55.714 INFO:tasks.workunit.client.1.smithi138.stdout:0/114: link f1 d6/f26 0 2024-04-25T01:35:55.715 INFO:tasks.workunit.client.0.smithi033.stdout:1/458: dread - d2/d5/d1b/d46/d5f/d63/d64/f6a zero size 2024-04-25T01:35:55.715 INFO:tasks.workunit.client.1.smithi138.stdout:2/157: symlink d0/d21/l30 0 2024-04-25T01:35:55.716 INFO:tasks.workunit.client.0.smithi033.stdout:4/696: creat d6/d14/d35/d3a/d4d/d5e/fdd x:0 0 0 2024-04-25T01:35:55.718 INFO:tasks.workunit.client.0.smithi033.stdout:1/459: rmdir d2/d2e/d34/d60 39 2024-04-25T01:35:55.721 INFO:tasks.workunit.client.1.smithi138.stdout:9/86: truncate d4/d8/d15/f16 2328351 0 2024-04-25T01:35:55.722 INFO:tasks.workunit.client.0.smithi033.stdout:3/674: rename d8/dd/d3d/d63/d7c/d3c to d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5 0 2024-04-25T01:35:55.723 INFO:tasks.workunit.client.1.smithi138.stdout:8/237: stat d3/l17 0 2024-04-25T01:35:55.725 INFO:tasks.workunit.client.0.smithi033.stdout:1/460: creat d2/d5/d75/f8d x:0 0 0 2024-04-25T01:35:55.727 INFO:tasks.workunit.client.0.smithi033.stdout:3/675: unlink d8/dd/d40/fb5 0 2024-04-25T01:35:55.729 INFO:tasks.workunit.client.0.smithi033.stdout:4/697: rename d6/d14/d1d/c2d to d6/d14/d1d/d3b/cde 0 2024-04-25T01:35:55.729 INFO:tasks.workunit.client.0.smithi033.stdout:3/676: readlink d8/dd/d3d/d4f/da4/db2/dbf/dc5/le3 0 2024-04-25T01:35:55.731 INFO:tasks.workunit.client.1.smithi138.stdout:1/96: link d3/db/cc d3/d4/da/c23 0 2024-04-25T01:35:55.731 INFO:tasks.workunit.client.1.smithi138.stdout:5/56: dwrite d4/f9 [0,4194304] 0 2024-04-25T01:35:55.731 INFO:tasks.workunit.client.0.smithi033.stdout:3/677: fdatasync d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5/d86/d92/f21 0 2024-04-25T01:35:55.732 INFO:tasks.workunit.client.1.smithi138.stdout:8/238: chown d3/d8/da/de 6924585 1 2024-04-25T01:35:55.736 INFO:tasks.workunit.client.1.smithi138.stdout:2/158: dwrite d0/d21/f27 [0,4194304] 0 2024-04-25T01:35:55.737 INFO:tasks.workunit.client.1.smithi138.stdout:8/239: fdatasync d3/d8/da/f43 0 2024-04-25T01:35:55.737 INFO:tasks.workunit.client.1.smithi138.stdout:8/240: stat d3/d8/da/de 0 2024-04-25T01:35:55.738 INFO:tasks.workunit.client.0.smithi033.stdout:2/665: sync 2024-04-25T01:35:55.738 INFO:tasks.workunit.client.0.smithi033.stdout:6/670: sync 2024-04-25T01:35:55.738 INFO:tasks.workunit.client.0.smithi033.stdout:4/698: dwrite d6/d14/d35/fb4 [0,4194304] 0 2024-04-25T01:35:55.739 INFO:tasks.workunit.client.1.smithi138.stdout:8/241: read - d3/d44/f46 zero size 2024-04-25T01:35:55.741 INFO:tasks.workunit.client.1.smithi138.stdout:2/159: creat d0/f31 x:0 0 0 2024-04-25T01:35:55.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:55 smithi033 ceph-mon[26353]: pgmap v13: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 129 MiB/s wr, 373 op/s 2024-04-25T01:35:55.742 INFO:tasks.workunit.client.0.smithi033.stdout:6/671: fdatasync d4/d1d/d3e/d41/d93/fab 0 2024-04-25T01:35:55.743 INFO:tasks.workunit.client.1.smithi138.stdout:5/57: rename d4/l7 to d4/da/lb 0 2024-04-25T01:35:55.744 INFO:tasks.workunit.client.1.smithi138.stdout:8/242: write d3/d8/da/db/d15/f30 [2971597,110536] 0 2024-04-25T01:35:55.744 INFO:tasks.workunit.client.1.smithi138.stdout:2/160: chown d0/dc/f1a 9 1 2024-04-25T01:35:55.747 INFO:tasks.workunit.client.1.smithi138.stdout:8/243: write d3/d8/f47 [1087225,123269] 0 2024-04-25T01:35:55.748 INFO:tasks.workunit.client.1.smithi138.stdout:0/115: dwrite d6/f11 [0,4194304] 0 2024-04-25T01:35:55.751 INFO:tasks.workunit.client.1.smithi138.stdout:5/58: mknod d4/cc 0 2024-04-25T01:35:55.751 INFO:tasks.workunit.client.1.smithi138.stdout:7/79: sync 2024-04-25T01:35:55.751 INFO:tasks.workunit.client.1.smithi138.stdout:6/49: sync 2024-04-25T01:35:55.753 INFO:tasks.workunit.client.1.smithi138.stdout:7/80: chown da/ld 9901 1 2024-04-25T01:35:55.755 INFO:tasks.workunit.client.1.smithi138.stdout:8/244: getdents d3/d16/d1f/d2a/d4f 0 2024-04-25T01:35:55.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/59: dread d4/f9 [0,4194304] 0 2024-04-25T01:35:55.764 INFO:tasks.workunit.client.1.smithi138.stdout:6/50: dwrite f5 [0,4194304] 0 2024-04-25T01:35:55.766 INFO:tasks.workunit.client.1.smithi138.stdout:6/51: chown c3 401 1 2024-04-25T01:35:55.766 INFO:tasks.workunit.client.1.smithi138.stdout:6/52: readlink la 0 2024-04-25T01:35:55.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:55 smithi138 ceph-mon[32977]: pgmap v13: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 33 MiB/s rd, 129 MiB/s wr, 373 op/s 2024-04-25T01:35:55.769 INFO:tasks.workunit.client.0.smithi033.stdout:5/671: chown d1c/d2a/d50/d8d/de1/le4 2016 1 2024-04-25T01:35:55.770 INFO:tasks.workunit.client.1.smithi138.stdout:7/81: creat da/de/d13/f23 x:0 0 0 2024-04-25T01:35:55.773 INFO:tasks.workunit.client.1.smithi138.stdout:0/116: link d6/la d6/df/l27 0 2024-04-25T01:35:55.773 INFO:tasks.workunit.client.0.smithi033.stdout:3/678: fdatasync d8/dd/d3d/d63/d7c/f73 0 2024-04-25T01:35:55.775 INFO:tasks.workunit.client.1.smithi138.stdout:0/117: dread - d6/f1e zero size 2024-04-25T01:35:55.775 INFO:tasks.workunit.client.1.smithi138.stdout:5/60: creat d4/da/fd x:0 0 0 2024-04-25T01:35:55.778 INFO:tasks.workunit.client.1.smithi138.stdout:1/97: write d3/d4/f22 [1608324,75161] 0 2024-04-25T01:35:55.779 INFO:tasks.workunit.client.1.smithi138.stdout:7/82: creat da/de/d13/f24 x:0 0 0 2024-04-25T01:35:55.781 INFO:tasks.workunit.client.0.smithi033.stdout:3/679: dwrite d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5/f7d [0,4194304] 0 2024-04-25T01:35:55.782 INFO:tasks.workunit.client.1.smithi138.stdout:6/53: creat dc/fd x:0 0 0 2024-04-25T01:35:55.784 INFO:tasks.workunit.client.1.smithi138.stdout:0/118: rename d6/f16 to d6/f28 0 2024-04-25T01:35:55.785 INFO:tasks.workunit.client.0.smithi033.stdout:7/645: write f3 [1719553,42207] 0 2024-04-25T01:35:55.786 INFO:tasks.workunit.client.1.smithi138.stdout:8/245: creat d3/d16/f51 x:0 0 0 2024-04-25T01:35:55.787 INFO:tasks.workunit.client.0.smithi033.stdout:7/646: chown d4/c18 63157 1 2024-04-25T01:35:55.788 INFO:tasks.workunit.client.1.smithi138.stdout:8/246: fsync f0 0 2024-04-25T01:35:55.788 INFO:tasks.workunit.client.1.smithi138.stdout:7/83: chown da/de/d13/l19 18600466 1 2024-04-25T01:35:55.790 INFO:tasks.workunit.client.0.smithi033.stdout:5/672: dwrite d1c/d2a/d50/d52/da6/d4a/f72 [0,4194304] 0 2024-04-25T01:35:55.792 INFO:tasks.workunit.client.0.smithi033.stdout:5/673: readlink d1c/d2a/d50/d52/da6/l79 0 2024-04-25T01:35:55.793 INFO:tasks.workunit.client.1.smithi138.stdout:6/54: creat dc/fe x:0 0 0 2024-04-25T01:35:55.796 INFO:tasks.workunit.client.1.smithi138.stdout:7/84: dread da/de/f1a [0,4194304] 0 2024-04-25T01:35:55.797 INFO:tasks.workunit.client.1.smithi138.stdout:9/87: write d4/d8/d15/f16 [3107660,40859] 0 2024-04-25T01:35:55.797 INFO:tasks.workunit.client.1.smithi138.stdout:7/85: write da/f15 [85881,36125] 0 2024-04-25T01:35:55.800 INFO:tasks.workunit.client.0.smithi033.stdout:7/647: dread d4/d23/d3f/d45/d90/fb1 [0,4194304] 0 2024-04-25T01:35:55.806 INFO:tasks.workunit.client.1.smithi138.stdout:0/119: link d6/df/f15 d6/f29 0 2024-04-25T01:35:55.806 INFO:tasks.workunit.client.0.smithi033.stdout:3/680: dread d8/f51 [0,4194304] 0 2024-04-25T01:35:55.811 INFO:tasks.workunit.client.1.smithi138.stdout:7/86: dwrite f4 [0,4194304] 0 2024-04-25T01:35:55.811 INFO:tasks.workunit.client.1.smithi138.stdout:6/55: dread f1 [0,4194304] 0 2024-04-25T01:35:55.812 INFO:tasks.workunit.client.1.smithi138.stdout:5/61: getdents d4/da 0 2024-04-25T01:35:55.817 INFO:tasks.workunit.client.1.smithi138.stdout:5/62: readlink d4/da/lb 0 2024-04-25T01:35:55.818 INFO:tasks.workunit.client.1.smithi138.stdout:6/56: write f5 [3375892,83010] 0 2024-04-25T01:35:55.818 INFO:tasks.workunit.client.1.smithi138.stdout:9/88: dwrite d4/d8/d15/f16 [0,4194304] 0 2024-04-25T01:35:55.822 INFO:tasks.workunit.client.0.smithi033.stdout:7/648: dread d4/d2e/d6d/dbb/fc4 [0,4194304] 0 2024-04-25T01:35:55.823 INFO:tasks.workunit.client.1.smithi138.stdout:6/57: read f5 [962326,87986] 0 2024-04-25T01:35:55.823 INFO:tasks.workunit.client.1.smithi138.stdout:8/247: link d3/l17 d3/d16/d1f/d2a/d4f/l52 0 2024-04-25T01:35:55.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/248: write d3/d8/da/f43 [1539258,72018] 0 2024-04-25T01:35:55.831 INFO:tasks.workunit.client.0.smithi033.stdout:7/649: rmdir d4/d57 39 2024-04-25T01:35:55.832 INFO:tasks.workunit.client.0.smithi033.stdout:7/650: fdatasync d4/d23/d3f/d45/f5e 0 2024-04-25T01:35:55.835 INFO:tasks.workunit.client.1.smithi138.stdout:7/87: mknod da/de/c25 0 2024-04-25T01:35:55.836 INFO:tasks.workunit.client.1.smithi138.stdout:6/58: creat dc/ff x:0 0 0 2024-04-25T01:35:55.837 INFO:tasks.workunit.client.1.smithi138.stdout:6/59: stat f8 0 2024-04-25T01:35:55.840 INFO:tasks.workunit.client.1.smithi138.stdout:0/120: getdents d6/df 0 2024-04-25T01:35:55.842 INFO:tasks.workunit.client.0.smithi033.stdout:2/666: mknod da/d27/d31/d40/d8a/cde 0 2024-04-25T01:35:55.842 INFO:tasks.workunit.client.1.smithi138.stdout:8/249: write d3/f3c [2675730,118242] 0 2024-04-25T01:35:55.843 INFO:tasks.workunit.client.1.smithi138.stdout:6/60: mkdir dc/d10 0 2024-04-25T01:35:55.843 INFO:tasks.workunit.client.1.smithi138.stdout:9/89: dwrite d4/f17 [0,4194304] 0 2024-04-25T01:35:55.843 INFO:tasks.workunit.client.0.smithi033.stdout:5/674: creat d1c/d2a/d50/d52/da6/d3e/d70/d7a/fe5 x:0 0 0 2024-04-25T01:35:55.844 INFO:tasks.workunit.client.1.smithi138.stdout:7/88: fdatasync da/fc 0 2024-04-25T01:35:55.846 INFO:tasks.workunit.client.1.smithi138.stdout:0/121: dread - d6/f29 zero size 2024-04-25T01:35:55.847 INFO:tasks.workunit.client.0.smithi033.stdout:2/667: rename da/d2c/dcd/fc7 to da/d27/fdf 0 2024-04-25T01:35:55.847 INFO:tasks.workunit.client.1.smithi138.stdout:9/90: write d4/d8/d15/f16 [2813347,35378] 0 2024-04-25T01:35:55.848 INFO:tasks.workunit.client.1.smithi138.stdout:5/63: dwrite d4/f9 [0,4194304] 0 2024-04-25T01:35:55.850 INFO:tasks.workunit.client.1.smithi138.stdout:9/91: write d4/d8/d15/f16 [3116298,29248] 0 2024-04-25T01:35:55.854 INFO:tasks.workunit.client.1.smithi138.stdout:6/61: creat dc/f11 x:0 0 0 2024-04-25T01:35:55.855 INFO:tasks.workunit.client.1.smithi138.stdout:5/64: truncate d4/da/fd 720351 0 2024-04-25T01:35:55.857 INFO:tasks.workunit.client.1.smithi138.stdout:7/89: dread da/de/f1a [0,4194304] 0 2024-04-25T01:35:55.859 INFO:tasks.workunit.client.1.smithi138.stdout:8/250: creat d3/d8/f53 x:0 0 0 2024-04-25T01:35:55.861 INFO:tasks.workunit.client.0.smithi033.stdout:6/672: truncate d4/d1d/d3e/d41/d5c/f8b 6025203 0 2024-04-25T01:35:55.862 INFO:tasks.workunit.client.1.smithi138.stdout:0/122: truncate d6/fe 1147667 0 2024-04-25T01:35:55.863 INFO:tasks.workunit.client.0.smithi033.stdout:7/651: dread d4/f41 [0,4194304] 0 2024-04-25T01:35:55.863 INFO:tasks.workunit.client.0.smithi033.stdout:4/699: dwrite d6/d14/d35/d3a/f49 [0,4194304] 0 2024-04-25T01:35:55.869 INFO:tasks.workunit.client.0.smithi033.stdout:5/675: dread d1c/f2b [0,4194304] 0 2024-04-25T01:35:55.869 INFO:tasks.workunit.client.1.smithi138.stdout:9/92: dwrite f1 [0,4194304] 0 2024-04-25T01:35:55.870 INFO:tasks.workunit.client.0.smithi033.stdout:5/676: stat d1c/d2a/d3d/fd6 0 2024-04-25T01:35:55.873 INFO:tasks.workunit.client.1.smithi138.stdout:5/65: creat d4/da/fe x:0 0 0 2024-04-25T01:35:55.874 INFO:tasks.workunit.client.0.smithi033.stdout:4/700: creat d6/d14/d35/d44/d45/d46/d57/d70/d87/fdf x:0 0 0 2024-04-25T01:35:55.874 INFO:tasks.workunit.client.0.smithi033.stdout:6/673: truncate d4/d1d/d3e/d70/f95 137946 0 2024-04-25T01:35:55.876 INFO:tasks.workunit.client.0.smithi033.stdout:4/701: fdatasync d6/d12/d38/f91 0 2024-04-25T01:35:55.878 INFO:tasks.workunit.client.1.smithi138.stdout:6/62: creat dc/d10/f12 x:0 0 0 2024-04-25T01:35:55.879 INFO:tasks.workunit.client.1.smithi138.stdout:0/123: mknod d6/d19/d1c/c2a 0 2024-04-25T01:35:55.880 INFO:tasks.workunit.client.0.smithi033.stdout:5/677: dwrite d1c/d2a/d50/d52/da6/d3e/d70/fc3 [0,4194304] 0 2024-04-25T01:35:55.881 INFO:tasks.workunit.client.0.smithi033.stdout:6/674: creat d4/d1d/d3e/d41/d5c/fea x:0 0 0 2024-04-25T01:35:55.882 INFO:tasks.workunit.client.1.smithi138.stdout:8/251: dwrite d3/f3a [0,4194304] 0 2024-04-25T01:35:55.883 INFO:tasks.workunit.client.1.smithi138.stdout:5/66: truncate d4/da/fd 1158825 0 2024-04-25T01:35:55.885 INFO:tasks.workunit.client.1.smithi138.stdout:7/90: dread da/de/f1c [0,4194304] 0 2024-04-25T01:35:55.885 INFO:tasks.workunit.client.0.smithi033.stdout:2/668: dread da/d27/d31/d40/d8a/f75 [0,4194304] 0 2024-04-25T01:35:55.889 INFO:tasks.workunit.client.1.smithi138.stdout:5/67: chown d4 0 1 2024-04-25T01:35:55.889 INFO:tasks.workunit.client.0.smithi033.stdout:7/652: dwrite d4/d2e/d6b/d1f/f46 [4194304,4194304] 0 2024-04-25T01:35:55.890 INFO:tasks.workunit.client.1.smithi138.stdout:7/91: write da/fc [1011076,10520] 0 2024-04-25T01:35:55.890 INFO:tasks.workunit.client.0.smithi033.stdout:7/653: chown d4/d2e/f1c 12 1 2024-04-25T01:35:55.891 INFO:tasks.workunit.client.0.smithi033.stdout:7/654: dread - d4/d23/d3f/d45/d6c/f92 zero size 2024-04-25T01:35:55.893 INFO:tasks.workunit.client.0.smithi033.stdout:2/669: unlink da/d27/d31/d40/d8a/fa5 0 2024-04-25T01:35:55.894 INFO:tasks.workunit.client.1.smithi138.stdout:6/63: write dc/fd [457413,8416] 0 2024-04-25T01:35:55.895 INFO:tasks.workunit.client.0.smithi033.stdout:2/670: chown da/d5b/d5e/f9f 2080 1 2024-04-25T01:35:55.897 INFO:tasks.workunit.client.0.smithi033.stdout:6/675: link d4/d1d/d3e/d41/d54/f8d d4/d24/feb 0 2024-04-25T01:35:55.899 INFO:tasks.workunit.client.0.smithi033.stdout:8/658: sync 2024-04-25T01:35:55.901 INFO:tasks.workunit.client.1.smithi138.stdout:6/64: write dc/fd [216081,78236] 0 2024-04-25T01:35:55.902 INFO:tasks.workunit.client.0.smithi033.stdout:6/676: mknod d4/d24/d63/cec 0 2024-04-25T01:35:55.902 INFO:tasks.workunit.client.1.smithi138.stdout:0/124: write d6/f29 [105341,45420] 0 2024-04-25T01:35:55.903 INFO:tasks.workunit.client.1.smithi138.stdout:5/68: creat d4/ff x:0 0 0 2024-04-25T01:35:55.903 INFO:tasks.workunit.client.1.smithi138.stdout:8/252: dwrite d3/d8/f53 [0,4194304] 0 2024-04-25T01:35:55.904 INFO:tasks.workunit.client.0.smithi033.stdout:8/659: creat d6/d45/d51/d55/d84/dc8/ff9 x:0 0 0 2024-04-25T01:35:55.906 INFO:tasks.workunit.client.0.smithi033.stdout:6/677: readlink d4/d1d/d3e/d41/d54/lbe 0 2024-04-25T01:35:55.907 INFO:tasks.workunit.client.0.smithi033.stdout:2/671: dwrite da/f12 [4194304,4194304] 0 2024-04-25T01:35:55.908 INFO:tasks.workunit.client.0.smithi033.stdout:6/678: write d4/d24/f8f [702286,28571] 0 2024-04-25T01:35:55.910 INFO:tasks.workunit.client.0.smithi033.stdout:6/679: truncate d4/d1d/d3e/fc7 901580 0 2024-04-25T01:35:55.910 INFO:tasks.workunit.client.1.smithi138.stdout:6/65: creat dc/d10/f13 x:0 0 0 2024-04-25T01:35:55.910 INFO:tasks.workunit.client.0.smithi033.stdout:2/672: readlink da/l1d 0 2024-04-25T01:35:55.915 INFO:tasks.workunit.client.1.smithi138.stdout:0/125: chown d6/df/f10 46373911 1 2024-04-25T01:35:55.916 INFO:tasks.workunit.client.1.smithi138.stdout:5/69: creat d4/da/f10 x:0 0 0 2024-04-25T01:35:55.918 INFO:tasks.workunit.client.0.smithi033.stdout:6/680: dwrite d4/d1d/d3e/d41/f58 [0,4194304] 0 2024-04-25T01:35:55.919 INFO:tasks.workunit.client.1.smithi138.stdout:7/92: dwrite da/de/d13/f23 [0,4194304] 0 2024-04-25T01:35:55.920 INFO:tasks.workunit.client.1.smithi138.stdout:9/93: dwrite d4/f13 [4194304,4194304] 0 2024-04-25T01:35:55.920 INFO:tasks.workunit.client.1.smithi138.stdout:6/66: creat dc/f14 x:0 0 0 2024-04-25T01:35:55.921 INFO:tasks.workunit.client.0.smithi033.stdout:6/681: symlink d4/d1d/d3e/d4a/led 0 2024-04-25T01:35:55.921 INFO:tasks.workunit.client.1.smithi138.stdout:6/67: dread - fb zero size 2024-04-25T01:35:55.922 INFO:tasks.workunit.client.1.smithi138.stdout:0/126: truncate d6/f1f 625364 0 2024-04-25T01:35:55.923 INFO:tasks.workunit.client.0.smithi033.stdout:6/682: mknod d4/d24/d36/cee 0 2024-04-25T01:35:55.926 INFO:tasks.workunit.client.1.smithi138.stdout:7/93: rmdir da/de 39 2024-04-25T01:35:55.926 INFO:tasks.workunit.client.0.smithi033.stdout:6/683: mknod d4/d24/d36/dbd/dde/cef 0 2024-04-25T01:35:55.932 INFO:tasks.workunit.client.0.smithi033.stdout:6/684: link d4/d24/d36/dbd/dde/cef d4/d24/d36/d42/d9d/cf0 0 2024-04-25T01:35:55.934 INFO:tasks.workunit.client.1.smithi138.stdout:5/70: rename d4/da/lb to d4/l11 0 2024-04-25T01:35:55.937 INFO:tasks.workunit.client.1.smithi138.stdout:5/71: chown d4/l5 1 1 2024-04-25T01:35:55.938 INFO:tasks.workunit.client.1.smithi138.stdout:5/72: rename d4 to d4/da/d12 22 2024-04-25T01:35:55.941 INFO:tasks.workunit.client.1.smithi138.stdout:6/68: dwrite dc/ff [0,4194304] 0 2024-04-25T01:35:55.941 INFO:tasks.workunit.client.0.smithi033.stdout:8/660: dread d6/d45/d51/d5d/f6a [4194304,4194304] 0 2024-04-25T01:35:55.943 INFO:tasks.workunit.client.1.smithi138.stdout:8/253: write d3/d8/da/db/d15/f34 [3010128,46514] 0 2024-04-25T01:35:55.944 INFO:tasks.workunit.client.0.smithi033.stdout:4/702: truncate d6/d14/d1d/f28 2925163 0 2024-04-25T01:35:55.948 INFO:tasks.workunit.client.0.smithi033.stdout:4/703: creat d6/d14/d35/d44/d45/d46/d64/fe0 x:0 0 0 2024-04-25T01:35:55.948 INFO:tasks.workunit.client.1.smithi138.stdout:0/127: dwrite d6/f1f [0,4194304] 0 2024-04-25T01:35:55.949 INFO:tasks.workunit.client.0.smithi033.stdout:4/704: chown d6/d14 70487 1 2024-04-25T01:35:55.953 INFO:tasks.workunit.client.1.smithi138.stdout:5/73: dwrite d4/f9 [4194304,4194304] 0 2024-04-25T01:35:55.953 INFO:tasks.workunit.client.1.smithi138.stdout:6/69: creat dc/d10/f15 x:0 0 0 2024-04-25T01:35:55.954 INFO:tasks.workunit.client.0.smithi033.stdout:5/678: dread d1c/d2a/d3d/f92 [0,4194304] 0 2024-04-25T01:35:55.955 INFO:tasks.workunit.client.0.smithi033.stdout:4/705: link d6/d14/d35/d44/d45/d46/f9b d6/d14/d35/d44/fe1 0 2024-04-25T01:35:55.956 INFO:tasks.workunit.client.1.smithi138.stdout:8/254: mkdir d3/d8/da/db/d15/d54 0 2024-04-25T01:35:55.957 INFO:tasks.workunit.client.1.smithi138.stdout:9/94: link d4/d8/l10 d4/d8/d15/l1e 0 2024-04-25T01:35:55.957 INFO:tasks.workunit.client.0.smithi033.stdout:4/706: fsync d6/d14/d35/d3a/d4c/f9f 0 2024-04-25T01:35:55.958 INFO:tasks.workunit.client.0.smithi033.stdout:5/679: creat d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/fe6 x:0 0 0 2024-04-25T01:35:55.961 INFO:tasks.workunit.client.0.smithi033.stdout:5/680: rmdir d1c/d2a/d50/d52/da6/d4a 39 2024-04-25T01:35:55.963 INFO:tasks.workunit.client.0.smithi033.stdout:5/681: chown d1c/d2a/d3d/d59/l74 211 1 2024-04-25T01:35:55.964 INFO:tasks.workunit.client.0.smithi033.stdout:5/682: truncate d1c/d53/fd8 201979 0 2024-04-25T01:35:55.964 INFO:tasks.workunit.client.0.smithi033.stdout:5/683: stat d1c/d2a/d3d/fd6 0 2024-04-25T01:35:55.965 INFO:tasks.workunit.client.1.smithi138.stdout:0/128: dread f1 [0,4194304] 0 2024-04-25T01:35:55.965 INFO:tasks.workunit.client.1.smithi138.stdout:7/94: rename da/l20 to da/de/d13/l26 0 2024-04-25T01:35:55.966 INFO:tasks.workunit.client.1.smithi138.stdout:8/255: symlink d3/d8/da/db/d15/d20/l55 0 2024-04-25T01:35:55.968 INFO:tasks.workunit.client.0.smithi033.stdout:5/684: creat d1c/d2a/d3d/fe7 x:0 0 0 2024-04-25T01:35:55.970 INFO:tasks.workunit.client.1.smithi138.stdout:0/129: dread d6/f9 [0,4194304] 0 2024-04-25T01:35:55.971 INFO:tasks.workunit.client.1.smithi138.stdout:9/95: rename d4/f11 to d4/d8/f1f 0 2024-04-25T01:35:55.972 INFO:tasks.workunit.client.1.smithi138.stdout:0/130: write d6/df/f15 [1163168,12239] 0 2024-04-25T01:35:55.973 INFO:tasks.workunit.client.1.smithi138.stdout:9/96: write d4/f13 [9192716,57332] 0 2024-04-25T01:35:55.977 INFO:tasks.workunit.client.1.smithi138.stdout:0/131: write d6/f9 [1250822,90397] 0 2024-04-25T01:35:55.978 INFO:tasks.workunit.client.1.smithi138.stdout:5/74: getdents d4/da 0 2024-04-25T01:35:55.980 INFO:tasks.workunit.client.1.smithi138.stdout:0/132: symlink d6/l2b 0 2024-04-25T01:35:55.982 INFO:tasks.workunit.client.1.smithi138.stdout:5/75: creat d4/f13 x:0 0 0 2024-04-25T01:35:55.983 INFO:tasks.workunit.client.1.smithi138.stdout:5/76: truncate d4/da/fd 1954134 0 2024-04-25T01:35:55.984 INFO:tasks.workunit.client.1.smithi138.stdout:5/77: mkdir d4/da/d14 0 2024-04-25T01:35:55.990 INFO:tasks.workunit.client.1.smithi138.stdout:4/66: sync 2024-04-25T01:35:55.990 INFO:tasks.workunit.client.1.smithi138.stdout:3/96: sync 2024-04-25T01:35:55.992 INFO:tasks.workunit.client.1.smithi138.stdout:5/78: dwrite d4/da/fd [0,4194304] 0 2024-04-25T01:35:55.992 INFO:tasks.workunit.client.1.smithi138.stdout:8/256: fsync d3/d8/f53 0 2024-04-25T01:35:55.993 INFO:tasks.workunit.client.1.smithi138.stdout:6/70: fsync dc/ff 0 2024-04-25T01:35:55.995 INFO:tasks.workunit.client.0.smithi033.stdout:6/685: write d4/d1d/f88 [762596,76793] 0 2024-04-25T01:35:55.996 INFO:tasks.workunit.client.0.smithi033.stdout:8/661: write d6/db/d80/fa3 [313460,104573] 0 2024-04-25T01:35:55.996 INFO:tasks.workunit.client.1.smithi138.stdout:5/79: write d4/da/f10 [986130,16143] 0 2024-04-25T01:35:55.999 INFO:tasks.workunit.client.1.smithi138.stdout:8/257: mkdir d3/d8/da/db/d15/d56 0 2024-04-25T01:35:55.999 INFO:tasks.workunit.client.0.smithi033.stdout:2/673: dwrite da/d27/f86 [0,4194304] 0 2024-04-25T01:35:56.002 INFO:tasks.workunit.client.1.smithi138.stdout:6/71: rename dc/d10/f13 to dc/f16 0 2024-04-25T01:35:56.003 INFO:tasks.workunit.client.0.smithi033.stdout:6/686: mkdir d4/df1 0 2024-04-25T01:35:56.003 INFO:tasks.workunit.client.1.smithi138.stdout:7/95: rmdir da 39 2024-04-25T01:35:56.005 INFO:tasks.workunit.client.0.smithi033.stdout:0/555: sync 2024-04-25T01:35:56.006 INFO:tasks.workunit.client.0.smithi033.stdout:2/674: chown da/d50/d6a/f4b 985991 1 2024-04-25T01:35:56.007 INFO:tasks.workunit.client.1.smithi138.stdout:5/80: creat d4/f15 x:0 0 0 2024-04-25T01:35:56.007 INFO:tasks.workunit.client.1.smithi138.stdout:4/67: dwrite d0/fa [0,4194304] 0 2024-04-25T01:35:56.007 INFO:tasks.workunit.client.0.smithi033.stdout:2/675: fsync da/d27/f69 0 2024-04-25T01:35:56.008 INFO:tasks.workunit.client.1.smithi138.stdout:4/68: chown d0/ff 57574056 1 2024-04-25T01:35:56.008 INFO:tasks.workunit.client.1.smithi138.stdout:8/258: stat d3/c6 0 2024-04-25T01:35:56.008 INFO:tasks.workunit.client.0.smithi033.stdout:0/556: readlink d7/l57 0 2024-04-25T01:35:56.009 INFO:tasks.workunit.client.0.smithi033.stdout:6/687: dread d4/d1d/d3e/d41/f58 [0,4194304] 0 2024-04-25T01:35:56.009 INFO:tasks.workunit.client.0.smithi033.stdout:5/685: write d1c/d2a/d50/d52/da6/d4a/f7f [852611,62092] 0 2024-04-25T01:35:56.009 INFO:tasks.workunit.client.0.smithi033.stdout:2/676: write da/d50/f58 [30807,72408] 0 2024-04-25T01:35:56.012 INFO:tasks.workunit.client.0.smithi033.stdout:2/677: write da/fd [335276,51298] 0 2024-04-25T01:35:56.013 INFO:tasks.workunit.client.0.smithi033.stdout:8/662: write d6/d49/f5a [120951,26240] 0 2024-04-25T01:35:56.013 INFO:tasks.workunit.client.1.smithi138.stdout:6/72: symlink dc/d10/l17 0 2024-04-25T01:35:56.015 INFO:tasks.workunit.client.0.smithi033.stdout:5/686: fsync d1c/d2a/f73 0 2024-04-25T01:35:56.018 INFO:tasks.workunit.client.0.smithi033.stdout:8/663: mkdir d6/d45/d52/df2/dfa 0 2024-04-25T01:35:56.018 INFO:tasks.workunit.client.0.smithi033.stdout:2/678: readlink da/d27/d31/d40/l52 0 2024-04-25T01:35:56.018 INFO:tasks.workunit.client.0.smithi033.stdout:4/707: dwrite d6/d12/d38/f41 [0,4194304] 0 2024-04-25T01:35:56.022 INFO:tasks.workunit.client.1.smithi138.stdout:9/97: truncate f1 197287 0 2024-04-25T01:35:56.022 INFO:tasks.workunit.client.0.smithi033.stdout:2/679: readlink da/l21 0 2024-04-25T01:35:56.022 INFO:tasks.workunit.client.0.smithi033.stdout:5/687: chown d1c/d2a/d50/d52/da6/d4a/fb4 26137 1 2024-04-25T01:35:56.023 INFO:tasks.workunit.client.1.smithi138.stdout:4/69: fsync d0/fc 0 2024-04-25T01:35:56.023 INFO:tasks.workunit.client.1.smithi138.stdout:4/70: fsync d0/fc 0 2024-04-25T01:35:56.024 INFO:tasks.workunit.client.1.smithi138.stdout:0/133: truncate d6/df/f18 4724836 0 2024-04-25T01:35:56.025 INFO:tasks.workunit.client.1.smithi138.stdout:1/98: sync 2024-04-25T01:35:56.025 INFO:tasks.workunit.client.1.smithi138.stdout:2/161: sync 2024-04-25T01:35:56.025 INFO:tasks.workunit.client.1.smithi138.stdout:4/71: chown d0/le 6337 1 2024-04-25T01:35:56.025 INFO:tasks.workunit.client.0.smithi033.stdout:6/688: dwrite d4/d24/d36/d42/d9d/fdd [0,4194304] 0 2024-04-25T01:35:56.030 INFO:tasks.workunit.client.1.smithi138.stdout:7/96: creat da/de/f27 x:0 0 0 2024-04-25T01:35:56.032 INFO:tasks.workunit.client.0.smithi033.stdout:2/680: readlink da/l4c 0 2024-04-25T01:35:56.032 INFO:tasks.workunit.client.1.smithi138.stdout:9/98: write d4/d8/f1f [2526505,122094] 0 2024-04-25T01:35:56.033 INFO:tasks.workunit.client.1.smithi138.stdout:4/72: rename d0/c8 to d0/c10 0 2024-04-25T01:35:56.035 INFO:tasks.workunit.client.0.smithi033.stdout:6/689: chown d4/d1d/d3e/d70/f95 386 1 2024-04-25T01:35:56.035 INFO:tasks.workunit.client.0.smithi033.stdout:0/557: dread d7/da/df/f72 [0,4194304] 0 2024-04-25T01:35:56.036 INFO:tasks.workunit.client.1.smithi138.stdout:7/97: dread - da/f18 zero size 2024-04-25T01:35:56.036 INFO:tasks.workunit.client.1.smithi138.stdout:2/162: chown d0/l1c 14308199 1 2024-04-25T01:35:56.038 INFO:tasks.workunit.client.1.smithi138.stdout:0/134: dread d6/d1d/f24 [0,4194304] 0 2024-04-25T01:35:56.038 INFO:tasks.workunit.client.0.smithi033.stdout:2/681: mkdir da/d27/d31/d40/db1/dbf/dd5/de0 0 2024-04-25T01:35:56.038 INFO:tasks.workunit.client.1.smithi138.stdout:9/99: chown d4/d8/d15/d1d 645616095 1 2024-04-25T01:35:56.039 INFO:tasks.workunit.client.1.smithi138.stdout:0/135: readlink d6/d19/l25 0 2024-04-25T01:35:56.039 INFO:tasks.workunit.client.1.smithi138.stdout:8/259: dwrite d3/d8/f4d [0,4194304] 0 2024-04-25T01:35:56.040 INFO:tasks.workunit.client.1.smithi138.stdout:2/163: chown d0 1245 1 2024-04-25T01:35:56.040 INFO:tasks.workunit.client.0.smithi033.stdout:0/558: rename d7/da/df/d50/fba to d7/d2d/d43/dc0/fc3 0 2024-04-25T01:35:56.041 INFO:tasks.workunit.client.0.smithi033.stdout:6/690: creat d4/d24/d36/d42/d97/de0/ff2 x:0 0 0 2024-04-25T01:35:56.042 INFO:tasks.workunit.client.1.smithi138.stdout:4/73: write d0/ff [5090385,27171] 0 2024-04-25T01:35:56.044 INFO:tasks.workunit.client.1.smithi138.stdout:1/99: dwrite d3/f10 [0,4194304] 0 2024-04-25T01:35:56.045 INFO:tasks.workunit.client.0.smithi033.stdout:2/682: mkdir da/d27/d31/d40/d8a/dbb/dd6/de1 0 2024-04-25T01:35:56.045 INFO:tasks.workunit.client.1.smithi138.stdout:6/73: getdents dc 0 2024-04-25T01:35:56.047 INFO:tasks.workunit.client.1.smithi138.stdout:2/164: mkdir d0/d32 0 2024-04-25T01:35:56.049 INFO:tasks.workunit.client.1.smithi138.stdout:3/97: write db/df/d17/f1c [147082,83902] 0 2024-04-25T01:35:56.049 INFO:tasks.workunit.client.0.smithi033.stdout:2/683: write da/d8d/fda [196748,66812] 0 2024-04-25T01:35:56.051 INFO:tasks.workunit.client.1.smithi138.stdout:7/98: dwrite da/f12 [0,4194304] 0 2024-04-25T01:35:56.053 INFO:tasks.workunit.client.0.smithi033.stdout:0/559: dwrite d7/d2d/d4d/d9c/d61/f94 [0,4194304] 0 2024-04-25T01:35:56.054 INFO:tasks.workunit.client.1.smithi138.stdout:9/100: chown d4/d8/d15 14584 1 2024-04-25T01:35:56.055 INFO:tasks.workunit.client.0.smithi033.stdout:9/596: sync 2024-04-25T01:35:56.055 INFO:tasks.workunit.client.0.smithi033.stdout:1/461: sync 2024-04-25T01:35:56.055 INFO:tasks.workunit.client.0.smithi033.stdout:3/681: sync 2024-04-25T01:35:56.055 INFO:tasks.workunit.client.0.smithi033.stdout:7/655: sync 2024-04-25T01:35:56.055 INFO:tasks.workunit.client.1.smithi138.stdout:3/98: chown la 6044596 1 2024-04-25T01:35:56.057 INFO:tasks.workunit.client.1.smithi138.stdout:0/136: chown d6/df/f14 1392516 1 2024-04-25T01:35:56.058 INFO:tasks.workunit.client.0.smithi033.stdout:2/684: write da/d27/d31/d40/d77/fb4 [4624327,109728] 0 2024-04-25T01:35:56.059 INFO:tasks.workunit.client.1.smithi138.stdout:4/74: mknod d0/c11 0 2024-04-25T01:35:56.060 INFO:tasks.workunit.client.1.smithi138.stdout:1/100: mkdir d3/d4/da/d24 0 2024-04-25T01:35:56.060 INFO:tasks.workunit.client.0.smithi033.stdout:2/685: readlink da/d2c/l4e 0 2024-04-25T01:35:56.061 INFO:tasks.workunit.client.1.smithi138.stdout:9/101: write d4/f17 [3037670,123876] 0 2024-04-25T01:35:56.063 INFO:tasks.workunit.client.1.smithi138.stdout:5/81: rmdir d4 39 2024-04-25T01:35:56.068 INFO:tasks.workunit.client.0.smithi033.stdout:6/691: dwrite d4/d1d/d3e/d41/d5c/d7f/f81 [0,4194304] 0 2024-04-25T01:35:56.068 INFO:tasks.workunit.client.0.smithi033.stdout:6/692: write d4/d1d/f88 [683965,55251] 0 2024-04-25T01:35:56.068 INFO:tasks.workunit.client.0.smithi033.stdout:7/656: stat d4/d23/d3f/d45/d90/fb1 0 2024-04-25T01:35:56.069 INFO:tasks.workunit.client.1.smithi138.stdout:7/99: dread da/de/d13/f23 [0,4194304] 0 2024-04-25T01:35:56.072 INFO:tasks.workunit.client.1.smithi138.stdout:6/74: creat dc/f18 x:0 0 0 2024-04-25T01:35:56.073 INFO:tasks.workunit.client.0.smithi033.stdout:8/664: write d6/d45/f5b [677490,95610] 0 2024-04-25T01:35:56.074 INFO:tasks.workunit.client.0.smithi033.stdout:0/560: creat d7/d2d/fc4 x:0 0 0 2024-04-25T01:35:56.075 INFO:tasks.workunit.client.1.smithi138.stdout:0/137: rename d6/d19/f23 to d6/df/f2c 0 2024-04-25T01:35:56.076 INFO:tasks.workunit.client.0.smithi033.stdout:8/665: dread d6/f99 [0,4194304] 0 2024-04-25T01:35:56.076 INFO:tasks.workunit.client.0.smithi033.stdout:3/682: mkdir d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5/de6 0 2024-04-25T01:35:56.077 INFO:tasks.workunit.client.0.smithi033.stdout:4/708: truncate d6/d14/d35/d3a/d4d/d5e/f60 62221 0 2024-04-25T01:35:56.078 INFO:tasks.workunit.client.1.smithi138.stdout:9/102: chown d4/d8/l10 0 1 2024-04-25T01:35:56.079 INFO:tasks.workunit.client.1.smithi138.stdout:9/103: stat d4/le 0 2024-04-25T01:35:56.081 INFO:tasks.workunit.client.1.smithi138.stdout:4/75: creat d0/f12 x:0 0 0 2024-04-25T01:35:56.082 INFO:tasks.workunit.client.0.smithi033.stdout:5/688: dwrite d1c/d2a/d50/db1/dd3/fda [0,4194304] 0 2024-04-25T01:35:56.086 INFO:tasks.workunit.client.0.smithi033.stdout:9/597: creat d0/d2/dd/df/d1b/d3d/d7e/d58/fd7 x:0 0 0 2024-04-25T01:35:56.089 INFO:tasks.workunit.client.1.smithi138.stdout:0/138: dwrite d6/f29 [0,4194304] 0 2024-04-25T01:35:56.090 INFO:tasks.workunit.client.0.smithi033.stdout:0/561: symlink d7/d2d/d4d/d9c/d68/lc5 0 2024-04-25T01:35:56.092 INFO:tasks.workunit.client.0.smithi033.stdout:4/709: unlink d6/d12/d38/f41 0 2024-04-25T01:35:56.093 INFO:tasks.workunit.client.0.smithi033.stdout:7/657: symlink d4/d2e/d65/dcb/lcf 0 2024-04-25T01:35:56.095 INFO:tasks.workunit.client.0.smithi033.stdout:0/562: chown d7/da/df/d14/l95 7107160 1 2024-04-25T01:35:56.095 INFO:tasks.workunit.client.0.smithi033.stdout:3/683: mknod d8/dd/d3d/d63/d7c/d4d/da5/ce7 0 2024-04-25T01:35:56.095 INFO:tasks.workunit.client.1.smithi138.stdout:0/139: chown d6/df/f2c 15982 1 2024-04-25T01:35:56.096 INFO:tasks.workunit.client.1.smithi138.stdout:9/104: creat d4/d1a/f20 x:0 0 0 2024-04-25T01:35:56.097 INFO:tasks.workunit.client.0.smithi033.stdout:6/693: creat d4/d1d/d3e/d41/d54/da1/ff3 x:0 0 0 2024-04-25T01:35:56.098 INFO:tasks.workunit.client.1.smithi138.stdout:5/82: dwrite d4/f13 [0,4194304] 0 2024-04-25T01:35:56.101 INFO:tasks.workunit.client.0.smithi033.stdout:5/689: dwrite d1c/d2a/d50/d52/da6/d3e/fc8 [0,4194304] 0 2024-04-25T01:35:56.102 INFO:tasks.workunit.client.0.smithi033.stdout:2/686: dread da/f42 [0,4194304] 0 2024-04-25T01:35:56.103 INFO:tasks.workunit.client.1.smithi138.stdout:6/75: dread f5 [0,4194304] 0 2024-04-25T01:35:56.105 INFO:tasks.workunit.client.0.smithi033.stdout:8/666: dwrite d6/d45/d51/f4f [0,4194304] 0 2024-04-25T01:35:56.107 INFO:tasks.workunit.client.1.smithi138.stdout:1/101: rename d3/d4/da/c23 to d3/d4/da/c25 0 2024-04-25T01:35:56.109 INFO:tasks.workunit.client.0.smithi033.stdout:9/598: mkdir d0/d2/dd/df/dd8 0 2024-04-25T01:35:56.109 INFO:tasks.workunit.client.0.smithi033.stdout:4/710: symlink d6/d14/d35/d44/d45/d46/le2 0 2024-04-25T01:35:56.109 INFO:tasks.workunit.client.1.smithi138.stdout:7/100: creat da/de/d1b/f28 x:0 0 0 2024-04-25T01:35:56.111 INFO:tasks.workunit.client.1.smithi138.stdout:7/101: stat da 0 2024-04-25T01:35:56.111 INFO:tasks.workunit.client.1.smithi138.stdout:2/165: dread d0/f10 [0,4194304] 0 2024-04-25T01:35:56.112 INFO:tasks.workunit.client.1.smithi138.stdout:0/140: mknod d6/df/c2d 0 2024-04-25T01:35:56.113 INFO:tasks.workunit.client.0.smithi033.stdout:1/462: getdents d2/d5/d75/d59 0 2024-04-25T01:35:56.113 INFO:tasks.workunit.client.1.smithi138.stdout:7/102: write da/f12 [1338318,46186] 0 2024-04-25T01:35:56.115 INFO:tasks.workunit.client.1.smithi138.stdout:8/260: dread d3/d16/f1c [0,4194304] 0 2024-04-25T01:35:56.117 INFO:tasks.workunit.client.1.smithi138.stdout:7/103: dread - da/de/f27 zero size 2024-04-25T01:35:56.117 INFO:tasks.workunit.client.0.smithi033.stdout:3/684: mkdir d8/dd/d3d/d63/d7c/d9d/de8 0 2024-04-25T01:35:56.118 INFO:tasks.workunit.client.0.smithi033.stdout:6/694: creat d4/d24/d36/d42/d9d/ff4 x:0 0 0 2024-04-25T01:35:56.119 INFO:tasks.workunit.client.1.smithi138.stdout:5/83: rmdir d4 39 2024-04-25T01:35:56.120 INFO:tasks.workunit.client.1.smithi138.stdout:7/104: write da/de/d13/f24 [367577,131070] 0 2024-04-25T01:35:56.120 INFO:tasks.workunit.client.0.smithi033.stdout:7/658: dread d4/d2e/f1c [0,4194304] 0 2024-04-25T01:35:56.121 INFO:tasks.workunit.client.1.smithi138.stdout:9/105: creat d4/d1a/f21 x:0 0 0 2024-04-25T01:35:56.122 INFO:tasks.workunit.client.0.smithi033.stdout:4/711: unlink d6/d14/d35/d3a/d4d/d5e/f76 0 2024-04-25T01:35:56.123 INFO:tasks.workunit.client.1.smithi138.stdout:3/99: link l4 db/df/d19/d1a/l22 0 2024-04-25T01:35:56.123 INFO:tasks.workunit.client.1.smithi138.stdout:6/76: creat dc/f19 x:0 0 0 2024-04-25T01:35:56.124 INFO:tasks.workunit.client.0.smithi033.stdout:7/659: chown d4/d2e/d6b/d1f/d78/d86/fa6 4 1 2024-04-25T01:35:56.124 INFO:tasks.workunit.client.1.smithi138.stdout:0/141: unlink d6/f11 0 2024-04-25T01:35:56.126 INFO:tasks.workunit.client.0.smithi033.stdout:8/667: mkdir d6/dd3/dfb 0 2024-04-25T01:35:56.126 INFO:tasks.workunit.client.1.smithi138.stdout:6/77: chown dc/f14 17 1 2024-04-25T01:35:56.128 INFO:tasks.workunit.client.0.smithi033.stdout:9/599: dwrite d0/d2/dd/df/d1b/d3d/d6e/f95 [0,4194304] 0 2024-04-25T01:35:56.128 INFO:tasks.workunit.client.1.smithi138.stdout:1/102: getdents d3/d4/da/d24 0 2024-04-25T01:35:56.129 INFO:tasks.workunit.client.0.smithi033.stdout:6/695: unlink d4/d24/d63/l9e 0 2024-04-25T01:35:56.131 INFO:tasks.workunit.client.0.smithi033.stdout:9/600: chown d0/d2/dd/d20/f7d 14306 1 2024-04-25T01:35:56.131 INFO:tasks.workunit.client.0.smithi033.stdout:4/712: rmdir d6/d14/d48 39 2024-04-25T01:35:56.133 INFO:tasks.workunit.client.1.smithi138.stdout:2/166: mknod d0/d32/c33 0 2024-04-25T01:35:56.134 INFO:tasks.workunit.client.0.smithi033.stdout:7/660: mkdir d4/d23/da7/dc2/dd0 0 2024-04-25T01:35:56.135 INFO:tasks.workunit.client.0.smithi033.stdout:8/668: chown d6/db/l2c 13127869 1 2024-04-25T01:35:56.137 INFO:tasks.workunit.client.0.smithi033.stdout:6/696: chown d4/d24/d29/f55 15 1 2024-04-25T01:35:56.137 INFO:tasks.workunit.client.0.smithi033.stdout:8/669: readlink d6/d45/d51/d5d/d6e/le9 0 2024-04-25T01:35:56.138 INFO:tasks.workunit.client.1.smithi138.stdout:3/100: dwrite db/df/d17/f1c [0,4194304] 0 2024-04-25T01:35:56.138 INFO:tasks.workunit.client.0.smithi033.stdout:1/463: link d2/c31 d2/d7f/c8e 0 2024-04-25T01:35:56.141 INFO:tasks.workunit.client.1.smithi138.stdout:3/101: fdatasync db/f11 0 2024-04-25T01:35:56.141 INFO:tasks.workunit.client.1.smithi138.stdout:0/142: rename d6/l8 to d6/l2e 0 2024-04-25T01:35:56.141 INFO:tasks.workunit.client.1.smithi138.stdout:5/84: unlink d4/l6 0 2024-04-25T01:35:56.144 INFO:tasks.workunit.client.1.smithi138.stdout:6/78: unlink dc/fe 0 2024-04-25T01:35:56.144 INFO:tasks.workunit.client.1.smithi138.stdout:5/85: truncate d4/da/fe 205756 0 2024-04-25T01:35:56.146 INFO:tasks.workunit.client.1.smithi138.stdout:0/143: read - d6/df/f2c zero size 2024-04-25T01:35:56.146 INFO:tasks.workunit.client.1.smithi138.stdout:1/103: dwrite d3/d4/da/f1e [0,4194304] 0 2024-04-25T01:35:56.146 INFO:tasks.workunit.client.1.smithi138.stdout:4/76: rmdir d0 39 2024-04-25T01:35:56.148 INFO:tasks.workunit.client.1.smithi138.stdout:1/104: write d3/f10 [241262,12248] 0 2024-04-25T01:35:56.149 INFO:tasks.workunit.client.0.smithi033.stdout:8/670: symlink d6/d45/d51/d55/deb/lfc 0 2024-04-25T01:35:56.153 INFO:tasks.workunit.client.0.smithi033.stdout:7/661: dwrite d4/f41 [8388608,4194304] 0 2024-04-25T01:35:56.153 INFO:tasks.workunit.client.0.smithi033.stdout:9/601: creat d0/d2/d43/d4b/fd9 x:0 0 0 2024-04-25T01:35:56.154 INFO:tasks.workunit.client.0.smithi033.stdout:1/464: dwrite d2/f29 [0,4194304] 0 2024-04-25T01:35:56.156 INFO:tasks.workunit.client.1.smithi138.stdout:1/105: dread d3/db/f14 [0,4194304] 0 2024-04-25T01:35:56.156 INFO:tasks.workunit.client.1.smithi138.stdout:5/86: unlink d4/ff 0 2024-04-25T01:35:56.157 INFO:tasks.workunit.client.1.smithi138.stdout:6/79: mknod dc/c1a 0 2024-04-25T01:35:56.159 INFO:tasks.workunit.client.1.smithi138.stdout:7/105: rmdir da/de 39 2024-04-25T01:35:56.159 INFO:tasks.workunit.client.0.smithi033.stdout:6/697: getdents d4/d24 0 2024-04-25T01:35:56.160 INFO:tasks.workunit.client.0.smithi033.stdout:1/465: unlink d2/f20 0 2024-04-25T01:35:56.161 INFO:tasks.workunit.client.1.smithi138.stdout:1/106: dread d3/d4/fd [0,4194304] 0 2024-04-25T01:35:56.162 INFO:tasks.workunit.client.1.smithi138.stdout:0/144: mkdir d6/d2f 0 2024-04-25T01:35:56.162 INFO:tasks.workunit.client.0.smithi033.stdout:9/602: mknod d0/d2/d43/d73/dab/cda 0 2024-04-25T01:35:56.163 INFO:tasks.workunit.client.1.smithi138.stdout:0/145: write d6/d19/d1c/f21 [642353,96180] 0 2024-04-25T01:35:56.164 INFO:tasks.workunit.client.0.smithi033.stdout:7/662: mknod d4/d23/da7/dc2/cd1 0 2024-04-25T01:35:56.166 INFO:tasks.workunit.client.0.smithi033.stdout:8/671: mknod d6/d45/d52/dae/dc3/cfd 0 2024-04-25T01:35:56.167 INFO:tasks.workunit.client.0.smithi033.stdout:8/672: readlink d6/d45/d51/ldf 0 2024-04-25T01:35:56.167 INFO:tasks.workunit.client.0.smithi033.stdout:8/673: chown d6/d45/d52/f3f 4796953 1 2024-04-25T01:35:56.169 INFO:tasks.workunit.client.0.smithi033.stdout:9/603: dread d0/d2/d43/f3c [0,4194304] 0 2024-04-25T01:35:56.170 INFO:tasks.workunit.client.0.smithi033.stdout:9/604: chown d0/d11/f16 2701 1 2024-04-25T01:35:56.172 INFO:tasks.workunit.client.0.smithi033.stdout:1/466: mkdir d2/d5/d1b/d46/d5f/d63/d8f 0 2024-04-25T01:35:56.173 INFO:tasks.workunit.client.1.smithi138.stdout:0/146: dwrite d6/f29 [0,4194304] 0 2024-04-25T01:35:56.173 INFO:tasks.workunit.client.0.smithi033.stdout:7/663: chown d4/f9 32 1 2024-04-25T01:35:56.174 INFO:tasks.workunit.client.0.smithi033.stdout:8/674: mknod d6/d45/d52/db0/cfe 0 2024-04-25T01:35:56.174 INFO:tasks.workunit.client.0.smithi033.stdout:7/664: chown d4/d2e/d6b/d1f/c70 3099 1 2024-04-25T01:35:56.175 INFO:tasks.workunit.client.0.smithi033.stdout:6/698: dread d4/d1d/d3e/d41/d54/f8d [0,4194304] 0 2024-04-25T01:35:56.175 INFO:tasks.workunit.client.1.smithi138.stdout:5/87: mknod d4/c16 0 2024-04-25T01:35:56.176 INFO:tasks.workunit.client.0.smithi033.stdout:9/605: creat d0/d2/dd/dba/fdb x:0 0 0 2024-04-25T01:35:56.178 INFO:tasks.workunit.client.0.smithi033.stdout:7/665: truncate d4/d2e/d6b/fbc 956454 0 2024-04-25T01:35:56.178 INFO:tasks.workunit.client.0.smithi033.stdout:1/467: readlink d2/d5/l35 0 2024-04-25T01:35:56.179 INFO:tasks.workunit.client.0.smithi033.stdout:7/666: chown d4/d2e 49 1 2024-04-25T01:35:56.180 INFO:tasks.workunit.client.1.smithi138.stdout:8/261: truncate d3/d8/f47 1105065 0 2024-04-25T01:35:56.180 INFO:tasks.workunit.client.1.smithi138.stdout:8/262: fdatasync f0 0 2024-04-25T01:35:56.181 INFO:tasks.workunit.client.0.smithi033.stdout:6/699: creat d4/d24/ff5 x:0 0 0 2024-04-25T01:35:56.181 INFO:tasks.workunit.client.1.smithi138.stdout:1/107: creat d3/d4/f26 x:0 0 0 2024-04-25T01:35:56.182 INFO:tasks.workunit.client.1.smithi138.stdout:1/108: truncate d3/db/f1d 406764 0 2024-04-25T01:35:56.184 INFO:tasks.workunit.client.1.smithi138.stdout:0/147: symlink d6/d19/l30 0 2024-04-25T01:35:56.185 INFO:tasks.workunit.client.1.smithi138.stdout:5/88: readlink d4/l8 0 2024-04-25T01:35:56.185 INFO:tasks.workunit.client.1.smithi138.stdout:0/148: truncate d6/df/f2c 275621 0 2024-04-25T01:35:56.186 INFO:tasks.workunit.client.1.smithi138.stdout:5/89: write d4/da/fd [1575679,86196] 0 2024-04-25T01:35:56.186 INFO:tasks.workunit.client.0.smithi033.stdout:9/606: dwrite d0/f33 [0,4194304] 0 2024-04-25T01:35:56.187 INFO:tasks.workunit.client.1.smithi138.stdout:4/77: link d0/fc d0/f13 0 2024-04-25T01:35:56.188 INFO:tasks.workunit.client.1.smithi138.stdout:4/78: read d0/fc [4451,41832] 0 2024-04-25T01:35:56.188 INFO:tasks.workunit.client.0.smithi033.stdout:9/607: fdatasync d0/fa2 0 2024-04-25T01:35:56.189 INFO:tasks.workunit.client.1.smithi138.stdout:4/79: write d0/f12 [297428,95800] 0 2024-04-25T01:35:56.191 INFO:tasks.workunit.client.0.smithi033.stdout:6/700: mkdir d4/d1d/d3e/d41/d5c/df6 0 2024-04-25T01:35:56.196 INFO:tasks.workunit.client.0.smithi033.stdout:2/687: write f7 [391440,23036] 0 2024-04-25T01:35:56.198 INFO:tasks.workunit.client.0.smithi033.stdout:1/468: dwrite d2/d5/d1b/f5a [0,4194304] 0 2024-04-25T01:35:56.203 INFO:tasks.workunit.client.0.smithi033.stdout:3/685: write d8/dd/d1d/f7e [1089186,52059] 0 2024-04-25T01:35:56.203 INFO:tasks.workunit.client.0.smithi033.stdout:7/667: dwrite d4/d23/d3f/f6f [0,4194304] 0 2024-04-25T01:35:56.203 INFO:tasks.workunit.client.1.smithi138.stdout:9/106: truncate d4/d8/d15/f16 3300731 0 2024-04-25T01:35:56.204 INFO:tasks.workunit.client.1.smithi138.stdout:4/80: dwrite d0/ff [0,4194304] 0 2024-04-25T01:35:56.204 INFO:tasks.workunit.client.0.smithi033.stdout:4/713: write d6/d14/d35/d44/fe1 [6176242,62035] 0 2024-04-25T01:35:56.205 INFO:tasks.workunit.client.1.smithi138.stdout:5/90: link d4/da/fe d4/f17 0 2024-04-25T01:35:56.205 INFO:tasks.workunit.client.1.smithi138.stdout:8/263: dwrite d3/f19 [0,4194304] 0 2024-04-25T01:35:56.208 INFO:tasks.workunit.client.1.smithi138.stdout:7/106: getdents da 0 2024-04-25T01:35:56.208 INFO:tasks.workunit.client.1.smithi138.stdout:4/81: write d0/fa [911052,119359] 0 2024-04-25T01:35:56.211 INFO:tasks.workunit.client.0.smithi033.stdout:2/688: chown da/d50/d6a/d43/d6c/c7f 13189630 1 2024-04-25T01:35:56.214 INFO:tasks.workunit.client.1.smithi138.stdout:1/109: rename d3/d4/da to d3/d27 0 2024-04-25T01:35:56.214 INFO:tasks.workunit.client.0.smithi033.stdout:0/563: dwrite d7/d2d/d43/dc0/f4b [0,4194304] 0 2024-04-25T01:35:56.217 INFO:tasks.workunit.client.0.smithi033.stdout:5/690: dwrite f1b [0,4194304] 0 2024-04-25T01:35:56.219 INFO:tasks.workunit.client.1.smithi138.stdout:2/167: truncate d0/f10 3307669 0 2024-04-25T01:35:56.222 INFO:tasks.workunit.client.1.smithi138.stdout:2/168: chown d0/dc/d15/d1f/f26 3 1 2024-04-25T01:35:56.223 INFO:tasks.workunit.client.0.smithi033.stdout:4/714: fsync d6/d14/d1d/d5f/fb3 0 2024-04-25T01:35:56.224 INFO:tasks.workunit.client.1.smithi138.stdout:3/102: write db/df/d19/d1a/f21 [1123336,60162] 0 2024-04-25T01:35:56.225 INFO:tasks.workunit.client.1.smithi138.stdout:4/82: fsync d0/fa 0 2024-04-25T01:35:56.225 INFO:tasks.workunit.client.0.smithi033.stdout:7/668: dwrite d4/d23/d3f/f79 [0,4194304] 0 2024-04-25T01:35:56.226 INFO:tasks.workunit.client.0.smithi033.stdout:8/675: truncate d6/d45/d52/f54 1424832 0 2024-04-25T01:35:56.227 INFO:tasks.workunit.client.0.smithi033.stdout:8/676: chown d6/d45/d52/db0/dbc 4 1 2024-04-25T01:35:56.228 INFO:tasks.workunit.client.1.smithi138.stdout:4/83: read d0/f12 [346522,80572] 0 2024-04-25T01:35:56.228 INFO:tasks.workunit.client.1.smithi138.stdout:6/80: write f5 [3710864,60216] 0 2024-04-25T01:35:56.230 INFO:tasks.workunit.client.1.smithi138.stdout:8/264: dwrite d3/d8/da/db/d15/f34 [0,4194304] 0 2024-04-25T01:35:56.233 INFO:tasks.workunit.client.1.smithi138.stdout:3/103: chown db/f11 139146632 1 2024-04-25T01:35:56.234 INFO:tasks.workunit.client.1.smithi138.stdout:6/81: write dc/f19 [927082,44725] 0 2024-04-25T01:35:56.234 INFO:tasks.workunit.client.1.smithi138.stdout:7/107: unlink da/f12 0 2024-04-25T01:35:56.236 INFO:tasks.workunit.client.0.smithi033.stdout:6/701: creat d4/d1d/d3e/d41/d54/ff7 x:0 0 0 2024-04-25T01:35:56.238 INFO:tasks.workunit.client.0.smithi033.stdout:8/677: dwrite d6/db/d80/dd1/fe3 [0,4194304] 0 2024-04-25T01:35:56.241 INFO:tasks.workunit.client.0.smithi033.stdout:1/469: symlink d2/d11/d28/l90 0 2024-04-25T01:35:56.241 INFO:tasks.workunit.client.1.smithi138.stdout:1/110: creat d3/d27/d16/f28 x:0 0 0 2024-04-25T01:35:56.243 INFO:tasks.workunit.client.0.smithi033.stdout:0/564: rename d7/da/df/d14/f24 to d7/d2d/d64/d51/fc6 0 2024-04-25T01:35:56.244 INFO:tasks.workunit.client.0.smithi033.stdout:4/715: creat d6/d14/d35/d44/d45/fe3 x:0 0 0 2024-04-25T01:35:56.244 INFO:tasks.workunit.client.0.smithi033.stdout:7/669: symlink d4/d23/d3f/d45/d6c/ld2 0 2024-04-25T01:35:56.245 INFO:tasks.workunit.client.1.smithi138.stdout:3/104: chown db/f11 75 1 2024-04-25T01:35:56.246 INFO:tasks.workunit.client.1.smithi138.stdout:8/265: mkdir d3/d8/da/de/d57 0 2024-04-25T01:35:56.247 INFO:tasks.workunit.client.0.smithi033.stdout:5/691: mknod d1c/d2a/d3d/ce8 0 2024-04-25T01:35:56.249 INFO:tasks.workunit.client.0.smithi033.stdout:7/670: chown d4/d2e/d6b/c87 82524631 1 2024-04-25T01:35:56.251 INFO:tasks.workunit.client.1.smithi138.stdout:4/84: dwrite d0/f12 [0,4194304] 0 2024-04-25T01:35:56.251 INFO:tasks.workunit.client.1.smithi138.stdout:2/169: dread d0/d21/f27 [0,4194304] 0 2024-04-25T01:35:56.252 INFO:tasks.workunit.client.0.smithi033.stdout:1/470: read d2/d2e/d34/f6f [7191431,115534] 0 2024-04-25T01:35:56.254 INFO:tasks.workunit.client.0.smithi033.stdout:8/678: truncate d6/db/d80/fb7 3631049 0 2024-04-25T01:35:56.255 INFO:tasks.workunit.client.1.smithi138.stdout:7/108: unlink l3 0 2024-04-25T01:35:56.256 INFO:tasks.workunit.client.1.smithi138.stdout:9/107: creat d4/f22 x:0 0 0 2024-04-25T01:35:56.256 INFO:tasks.workunit.client.0.smithi033.stdout:4/716: mknod d6/d14/d35/d3a/d4d/d5e/d7c/ce4 0 2024-04-25T01:35:56.256 INFO:tasks.workunit.client.0.smithi033.stdout:8/679: write d6/d45/d51/d5d/f81 [3144058,73922] 0 2024-04-25T01:35:56.258 INFO:tasks.workunit.client.1.smithi138.stdout:6/82: unlink dc/fd 0 2024-04-25T01:35:56.258 INFO:tasks.workunit.client.1.smithi138.stdout:0/149: write d6/fc [1349705,119430] 0 2024-04-25T01:35:56.259 INFO:tasks.workunit.client.1.smithi138.stdout:3/105: dread db/df/d17/f1c [0,4194304] 0 2024-04-25T01:35:56.260 INFO:tasks.workunit.client.0.smithi033.stdout:0/565: mkdir d7/da/df/dc7 0 2024-04-25T01:35:56.262 INFO:tasks.workunit.client.0.smithi033.stdout:3/686: getdents d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5/d98/da1 0 2024-04-25T01:35:56.263 INFO:tasks.workunit.client.1.smithi138.stdout:8/266: fdatasync d3/d8/da/db/f14 0 2024-04-25T01:35:56.264 INFO:tasks.workunit.client.1.smithi138.stdout:2/170: symlink d0/d18/l34 0 2024-04-25T01:35:56.264 INFO:tasks.workunit.client.1.smithi138.stdout:4/85: rename d0/c11 to d0/c14 0 2024-04-25T01:35:56.265 INFO:tasks.workunit.client.0.smithi033.stdout:2/689: getdents da/d5b/d5e 0 2024-04-25T01:35:56.266 INFO:tasks.workunit.client.1.smithi138.stdout:2/171: read d0/d21/f27 [1130776,23719] 0 2024-04-25T01:35:56.267 INFO:tasks.workunit.client.1.smithi138.stdout:6/83: rename dc/d10/l17 to dc/d10/l1b 0 2024-04-25T01:35:56.269 INFO:tasks.workunit.client.0.smithi033.stdout:4/717: chown d6/d14/d48/f85 2087348878 1 2024-04-25T01:35:56.269 INFO:tasks.workunit.client.0.smithi033.stdout:8/680: creat d6/d49/fff x:0 0 0 2024-04-25T01:35:56.270 INFO:tasks.workunit.client.1.smithi138.stdout:3/106: dread db/df/d19/d1a/f21 [0,4194304] 0 2024-04-25T01:35:56.270 INFO:tasks.workunit.client.0.smithi033.stdout:6/702: dwrite d4/d24/d63/fa2 [0,4194304] 0 2024-04-25T01:35:56.271 INFO:tasks.workunit.client.1.smithi138.stdout:4/86: rmdir d0 39 2024-04-25T01:35:56.271 INFO:tasks.workunit.client.0.smithi033.stdout:5/692: link cd d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/ce9 0 2024-04-25T01:35:56.272 INFO:tasks.workunit.client.1.smithi138.stdout:2/172: mkdir d0/d32/d35 0 2024-04-25T01:35:56.273 INFO:tasks.workunit.client.1.smithi138.stdout:0/150: mkdir d6/d2f/d31 0 2024-04-25T01:35:56.274 INFO:tasks.workunit.client.0.smithi033.stdout:2/690: mknod da/d27/ce2 0 2024-04-25T01:35:56.274 INFO:tasks.workunit.client.1.smithi138.stdout:6/84: rename dc/d10/f12 to dc/d10/f1c 0 2024-04-25T01:35:56.275 INFO:tasks.workunit.client.0.smithi033.stdout:5/693: fdatasync d1c/d2a/d3d/fd6 0 2024-04-25T01:35:56.276 INFO:tasks.workunit.client.1.smithi138.stdout:8/267: symlink d3/d44/d48/l58 0 2024-04-25T01:35:56.277 INFO:tasks.workunit.client.0.smithi033.stdout:0/566: getdents d7/da/df/d14/d45 0 2024-04-25T01:35:56.278 INFO:tasks.workunit.client.0.smithi033.stdout:4/718: truncate d6/d12/f26 426866 0 2024-04-25T01:35:56.279 INFO:tasks.workunit.client.0.smithi033.stdout:1/471: dwrite d2/f26 [8388608,4194304] 0 2024-04-25T01:35:56.279 INFO:tasks.workunit.client.1.smithi138.stdout:3/107: mkdir db/df/d17/d23 0 2024-04-25T01:35:56.279 INFO:tasks.workunit.client.1.smithi138.stdout:3/108: readlink db/df/l1d 0 2024-04-25T01:35:56.281 INFO:tasks.workunit.client.0.smithi033.stdout:5/694: dread - d1c/d2a/d3d/fe7 zero size 2024-04-25T01:35:56.284 INFO:tasks.workunit.client.1.smithi138.stdout:2/173: rename d0/dc to d0/d21/d36 0 2024-04-25T01:35:56.284 INFO:tasks.workunit.client.0.smithi033.stdout:2/691: mknod da/d27/d31/d40/d8a/dbb/dd6/ce3 0 2024-04-25T01:35:56.285 INFO:tasks.workunit.client.0.smithi033.stdout:5/695: write d1c/d53/f57 [5208793,122245] 0 2024-04-25T01:35:56.286 INFO:tasks.workunit.client.0.smithi033.stdout:2/692: chown da/d27/d31/d40/db1/fcc 51988430 1 2024-04-25T01:35:56.287 INFO:tasks.workunit.client.1.smithi138.stdout:0/151: symlink d6/d2f/l32 0 2024-04-25T01:35:56.287 INFO:tasks.workunit.client.1.smithi138.stdout:2/174: fsync d0/f14 0 2024-04-25T01:35:56.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/152: chown d6/fe 3660775 1 2024-04-25T01:35:56.291 INFO:tasks.workunit.client.0.smithi033.stdout:1/472: truncate d2/d2e/d34/d60/f74 742186 0 2024-04-25T01:35:56.291 INFO:tasks.workunit.client.1.smithi138.stdout:4/87: dwrite d0/ff [4194304,4194304] 0 2024-04-25T01:35:56.292 INFO:tasks.workunit.client.1.smithi138.stdout:0/153: stat d6/d2f/d31 0 2024-04-25T01:35:56.293 INFO:tasks.workunit.client.1.smithi138.stdout:1/111: link c0 d3/d4/c29 0 2024-04-25T01:35:56.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/88: read d0/f12 [3403729,32932] 0 2024-04-25T01:35:56.294 INFO:tasks.workunit.client.0.smithi033.stdout:1/473: fsync d2/d2e/d34/d60/f74 0 2024-04-25T01:35:56.294 INFO:tasks.workunit.client.0.smithi033.stdout:5/696: dwrite d1c/d53/fad [0,4194304] 0 2024-04-25T01:35:56.296 INFO:tasks.workunit.client.0.smithi033.stdout:2/693: creat da/d8d/fe4 x:0 0 0 2024-04-25T01:35:56.296 INFO:tasks.workunit.client.0.smithi033.stdout:5/697: readlink l13 0 2024-04-25T01:35:56.301 INFO:tasks.workunit.client.1.smithi138.stdout:2/175: creat d0/d21/d36/d15/d1f/f37 x:0 0 0 2024-04-25T01:35:56.302 INFO:tasks.workunit.client.0.smithi033.stdout:8/681: link d6/db/d80/db4/ldb d6/d45/d52/df2/dfa/l100 0 2024-04-25T01:35:56.303 INFO:tasks.workunit.client.0.smithi033.stdout:1/474: read d2/d2e/d34/d60/f7a [3697239,13956] 0 2024-04-25T01:35:56.304 INFO:tasks.workunit.client.1.smithi138.stdout:4/89: dwrite d0/ff [0,4194304] 0 2024-04-25T01:35:56.307 INFO:tasks.workunit.client.1.smithi138.stdout:4/90: chown d0/l5 98 1 2024-04-25T01:35:56.307 INFO:tasks.workunit.client.1.smithi138.stdout:0/154: creat d6/d19/f33 x:0 0 0 2024-04-25T01:35:56.307 INFO:tasks.workunit.client.0.smithi033.stdout:9/608: truncate d0/d2/d43/d4b/f9d 710671 0 2024-04-25T01:35:56.308 INFO:tasks.workunit.client.0.smithi033.stdout:9/609: readlink d0/d11/l7b 0 2024-04-25T01:35:56.309 INFO:tasks.workunit.client.1.smithi138.stdout:3/109: mknod db/df/d17/d20/c24 0 2024-04-25T01:35:56.313 INFO:tasks.workunit.client.0.smithi033.stdout:2/694: symlink da/d27/d31/le5 0 2024-04-25T01:35:56.314 INFO:tasks.workunit.client.0.smithi033.stdout:8/682: dread d6/d45/d51/d5d/d9d/fce [0,4194304] 0 2024-04-25T01:35:56.314 INFO:tasks.workunit.client.1.smithi138.stdout:8/268: rmdir d3/d8/da/db/d15/d54 0 2024-04-25T01:35:56.314 INFO:tasks.workunit.client.1.smithi138.stdout:2/176: dwrite d0/d21/d36/f12 [0,4194304] 0 2024-04-25T01:35:56.315 INFO:tasks.workunit.client.0.smithi033.stdout:3/687: dread d8/dd/d3d/fc8 [0,4194304] 0 2024-04-25T01:35:56.318 INFO:tasks.workunit.client.0.smithi033.stdout:9/610: mkdir d0/d2/dd/df/d1b/d3d/d7e/ddc 0 2024-04-25T01:35:56.319 INFO:tasks.workunit.client.0.smithi033.stdout:1/475: dwrite d2/d5/d1b/f54 [0,4194304] 0 2024-04-25T01:35:56.320 INFO:tasks.workunit.client.1.smithi138.stdout:4/91: dwrite d0/ff [0,4194304] 0 2024-04-25T01:35:56.323 INFO:tasks.workunit.client.1.smithi138.stdout:1/112: link d3/d27/d16/f18 d3/d27/d16/f2a 0 2024-04-25T01:35:56.323 INFO:tasks.workunit.client.0.smithi033.stdout:2/695: creat da/d27/d31/d40/fe6 x:0 0 0 2024-04-25T01:35:56.323 INFO:tasks.workunit.client.0.smithi033.stdout:9/611: chown d0/c30 15014685 1 2024-04-25T01:35:56.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/92: dread d0/f13 [0,4194304] 0 2024-04-25T01:35:56.326 INFO:tasks.workunit.client.0.smithi033.stdout:5/698: getdents d1c/d2a/d50/d8d 0 2024-04-25T01:35:56.328 INFO:tasks.workunit.client.1.smithi138.stdout:2/177: dwrite d0/d18/f1b [0,4194304] 0 2024-04-25T01:35:56.328 INFO:tasks.workunit.client.1.smithi138.stdout:4/93: chown d0/ff 16865635 1 2024-04-25T01:35:56.330 INFO:tasks.workunit.client.1.smithi138.stdout:4/94: readlink d0/l5 0 2024-04-25T01:35:56.330 INFO:tasks.workunit.client.1.smithi138.stdout:0/155: link d6/d19/f33 d6/d2f/f34 0 2024-04-25T01:35:56.332 INFO:tasks.workunit.client.1.smithi138.stdout:8/269: mkdir d3/d8/da/db/d15/d59 0 2024-04-25T01:35:56.332 INFO:tasks.workunit.client.0.smithi033.stdout:5/699: chown d1c/d2a/d50/d8d/de1/cd5 6310213 1 2024-04-25T01:35:56.333 INFO:tasks.workunit.client.1.smithi138.stdout:1/113: readlink d3/d27/d16/l21 0 2024-04-25T01:35:56.333 INFO:tasks.workunit.client.1.smithi138.stdout:2/178: read d0/d18/f1b [2428336,130856] 0 2024-04-25T01:35:56.335 INFO:tasks.workunit.client.0.smithi033.stdout:3/688: dwrite d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5/dc0/fc2 [0,4194304] 0 2024-04-25T01:35:56.338 INFO:tasks.workunit.client.0.smithi033.stdout:2/696: unlink da/d27/d31/d40/db1/fa6 0 2024-04-25T01:35:56.339 INFO:tasks.workunit.client.0.smithi033.stdout:6/703: write d4/d24/d36/f6c [489573,31508] 0 2024-04-25T01:35:56.341 INFO:tasks.workunit.client.1.smithi138.stdout:2/179: symlink d0/l38 0 2024-04-25T01:35:56.342 INFO:tasks.workunit.client.0.smithi033.stdout:5/700: unlink d1c/d2a/d50/d8d/de1/ld2 0 2024-04-25T01:35:56.345 INFO:tasks.workunit.client.0.smithi033.stdout:1/476: creat d2/d11/f91 x:0 0 0 2024-04-25T01:35:56.348 INFO:tasks.workunit.client.0.smithi033.stdout:6/704: creat d4/df1/ff8 x:0 0 0 2024-04-25T01:35:56.351 INFO:tasks.workunit.client.1.smithi138.stdout:4/95: dread d0/ff [4194304,4194304] 0 2024-04-25T01:35:56.351 INFO:tasks.workunit.client.1.smithi138.stdout:0/156: dread d6/fe [0,4194304] 0 2024-04-25T01:35:56.351 INFO:tasks.workunit.client.0.smithi033.stdout:3/689: dwrite d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5/f7d [0,4194304] 0 2024-04-25T01:35:56.352 INFO:tasks.workunit.client.1.smithi138.stdout:2/180: dread d0/d21/d36/d15/d1f/f26 [0,4194304] 0 2024-04-25T01:35:56.354 INFO:tasks.workunit.client.1.smithi138.stdout:3/110: getdents db/df/d19 0 2024-04-25T01:35:56.355 INFO:tasks.workunit.client.0.smithi033.stdout:9/612: dwrite d0/d2/dd/d20/d4e/f4f [0,4194304] 0 2024-04-25T01:35:56.355 INFO:tasks.workunit.client.0.smithi033.stdout:6/705: write d4/d2a/f3f [1753347,65399] 0 2024-04-25T01:35:56.355 INFO:tasks.workunit.client.1.smithi138.stdout:8/270: unlink d3/d8/da/de/c3f 0 2024-04-25T01:35:56.355 INFO:tasks.workunit.client.1.smithi138.stdout:3/111: fdatasync db/f12 0 2024-04-25T01:35:56.357 INFO:tasks.workunit.client.1.smithi138.stdout:8/271: write f0 [4713142,89992] 0 2024-04-25T01:35:56.359 INFO:tasks.workunit.client.0.smithi033.stdout:2/697: dwrite da/d50/d6a/d43/d6c/fc3 [0,4194304] 0 2024-04-25T01:35:56.359 INFO:tasks.workunit.client.1.smithi138.stdout:2/181: fdatasync d0/f9 0 2024-04-25T01:35:56.361 INFO:tasks.workunit.client.0.smithi033.stdout:5/701: truncate d1c/d2a/d50/d52/da6/f8e 349516 0 2024-04-25T01:35:56.363 INFO:tasks.workunit.client.0.smithi033.stdout:1/477: dread d2/d2e/f73 [0,4194304] 0 2024-04-25T01:35:56.365 INFO:tasks.workunit.client.0.smithi033.stdout:5/702: fsync d1c/d53/f57 0 2024-04-25T01:35:56.365 INFO:tasks.workunit.client.0.smithi033.stdout:6/706: creat d4/d1d/d3e/d41/d93/ff9 x:0 0 0 2024-04-25T01:35:56.366 INFO:tasks.workunit.client.1.smithi138.stdout:0/157: creat d6/f35 x:0 0 0 2024-04-25T01:35:56.366 INFO:tasks.workunit.client.1.smithi138.stdout:0/158: chown d6/d1d/l22 1769 1 2024-04-25T01:35:56.366 INFO:tasks.workunit.client.1.smithi138.stdout:4/96: truncate d0/fc 249493 0 2024-04-25T01:35:56.370 INFO:tasks.workunit.client.0.smithi033.stdout:1/478: mknod d2/d5/d6c/c92 0 2024-04-25T01:35:56.370 INFO:tasks.workunit.client.1.smithi138.stdout:3/112: creat db/df/f25 x:0 0 0 2024-04-25T01:35:56.371 INFO:tasks.workunit.client.1.smithi138.stdout:3/113: readlink la 0 2024-04-25T01:35:56.372 INFO:tasks.workunit.client.1.smithi138.stdout:7/109: dread da/de/f1a [0,4194304] 0 2024-04-25T01:35:56.373 INFO:tasks.workunit.client.0.smithi033.stdout:1/479: chown d2/d5/d75/f8d 46 1 2024-04-25T01:35:56.373 INFO:tasks.workunit.client.0.smithi033.stdout:9/613: dwrite d0/d2/dd/df/d1b/f4d [0,4194304] 0 2024-04-25T01:35:56.375 INFO:tasks.workunit.client.1.smithi138.stdout:7/110: creat da/de/f29 x:0 0 0 2024-04-25T01:35:56.375 INFO:tasks.workunit.client.0.smithi033.stdout:3/690: link d8/cc d8/dd/d3d/d63/d7c/d9d/de8/ce9 0 2024-04-25T01:35:56.377 INFO:tasks.workunit.client.0.smithi033.stdout:1/480: read - d2/d11/d28/f89 zero size 2024-04-25T01:35:56.379 INFO:tasks.workunit.client.1.smithi138.stdout:2/182: rename d0/d21/d36/d15/c2c to d0/d21/d36/d15/c39 0 2024-04-25T01:35:56.382 INFO:tasks.workunit.client.0.smithi033.stdout:5/703: getdents d1c/d2a/d50/d52/da6/d4a 0 2024-04-25T01:35:56.383 INFO:tasks.workunit.client.1.smithi138.stdout:3/114: truncate db/f11 3000382 0 2024-04-25T01:35:56.383 INFO:tasks.workunit.client.0.smithi033.stdout:3/691: write d8/fa [2837404,75977] 0 2024-04-25T01:35:56.384 INFO:tasks.workunit.client.1.smithi138.stdout:3/115: write db/f15 [837728,78931] 0 2024-04-25T01:35:56.386 INFO:tasks.workunit.client.0.smithi033.stdout:6/707: dwrite d4/d1d/d3e/d70/f72 [0,4194304] 0 2024-04-25T01:35:56.387 INFO:tasks.workunit.client.1.smithi138.stdout:6/85: getdents dc/d10 0 2024-04-25T01:35:56.392 INFO:tasks.workunit.client.0.smithi033.stdout:6/708: fsync d4/d1d/d3e/d41/f58 0 2024-04-25T01:35:56.393 INFO:tasks.workunit.client.1.smithi138.stdout:2/183: unlink d0/d21/d36/d15/f16 0 2024-04-25T01:35:56.393 INFO:tasks.workunit.client.0.smithi033.stdout:2/698: dread da/d27/d31/d40/d77/fb4 [0,4194304] 0 2024-04-25T01:35:56.394 INFO:tasks.workunit.client.1.smithi138.stdout:2/184: stat d0/d18/l34 0 2024-04-25T01:35:56.394 INFO:tasks.workunit.client.0.smithi033.stdout:2/699: write da/d8d/fda [15073,115009] 0 2024-04-25T01:35:56.395 INFO:tasks.workunit.client.1.smithi138.stdout:9/108: write d4/d8/d15/f16 [217573,88356] 0 2024-04-25T01:35:56.395 INFO:tasks.workunit.client.1.smithi138.stdout:7/111: mkdir da/de/d13/d21/d2a 0 2024-04-25T01:35:56.395 INFO:tasks.workunit.client.0.smithi033.stdout:6/709: write d4/d15/f94 [2657252,46775] 0 2024-04-25T01:35:56.396 INFO:tasks.workunit.client.1.smithi138.stdout:9/109: fsync d4/f17 0 2024-04-25T01:35:56.397 INFO:tasks.workunit.client.0.smithi033.stdout:3/692: mknod d8/dd/d3d/d4f/da4/db2/dc7/ddf/cea 0 2024-04-25T01:35:56.398 INFO:tasks.workunit.client.0.smithi033.stdout:2/700: write da/d27/d31/d40/d8a/dbb/f98 [1217198,54715] 0 2024-04-25T01:35:56.400 INFO:tasks.workunit.client.1.smithi138.stdout:3/116: creat db/df/d17/d20/f26 x:0 0 0 2024-04-25T01:35:56.401 INFO:tasks.workunit.client.1.smithi138.stdout:9/110: dread d4/f17 [0,4194304] 0 2024-04-25T01:35:56.402 INFO:tasks.workunit.client.0.smithi033.stdout:6/710: symlink d4/d24/d36/d42/lfa 0 2024-04-25T01:35:56.402 INFO:tasks.workunit.client.0.smithi033.stdout:6/711: stat d4/l13 0 2024-04-25T01:35:56.403 INFO:tasks.workunit.client.1.smithi138.stdout:2/185: creat d0/d21/f3a x:0 0 0 2024-04-25T01:35:56.404 INFO:tasks.workunit.client.1.smithi138.stdout:2/186: stat d0/d21/f22 0 2024-04-25T01:35:56.404 INFO:tasks.workunit.client.1.smithi138.stdout:2/187: chown d0/d21/c29 621894 1 2024-04-25T01:35:56.405 INFO:tasks.workunit.client.0.smithi033.stdout:2/701: symlink da/d50/d6a/d43/le7 0 2024-04-25T01:35:56.406 INFO:tasks.workunit.client.0.smithi033.stdout:3/693: readlink d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5/d98/ld3 0 2024-04-25T01:35:56.408 INFO:tasks.workunit.client.0.smithi033.stdout:8/683: dwrite d6/d45/d51/d5d/d6e/f77 [0,4194304] 0 2024-04-25T01:35:56.410 INFO:tasks.workunit.client.0.smithi033.stdout:2/702: read da/d27/d31/d40/f82 [1630327,124544] 0 2024-04-25T01:35:56.410 INFO:tasks.workunit.client.0.smithi033.stdout:6/712: unlink d4/d1d/cd0 0 2024-04-25T01:35:56.411 INFO:tasks.workunit.client.0.smithi033.stdout:8/684: dread - d6/d45/d51/d55/ff6 zero size 2024-04-25T01:35:56.411 INFO:tasks.workunit.client.0.smithi033.stdout:6/713: dread - d4/d24/d36/fb5 zero size 2024-04-25T01:35:56.412 INFO:tasks.workunit.client.0.smithi033.stdout:2/703: chown da/d27/d31 32271449 1 2024-04-25T01:35:56.413 INFO:tasks.workunit.client.0.smithi033.stdout:3/694: mknod d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5/d98/da1/ceb 0 2024-04-25T01:35:56.413 INFO:tasks.workunit.client.1.smithi138.stdout:7/112: symlink da/de/d13/d21/d2a/l2b 0 2024-04-25T01:35:56.413 INFO:tasks.workunit.client.1.smithi138.stdout:5/91: dwrite d4/da/fd [0,4194304] 0 2024-04-25T01:35:56.415 INFO:tasks.workunit.client.0.smithi033.stdout:2/704: write da/d27/d31/d40/d8a/f3b [4780648,79534] 0 2024-04-25T01:35:56.416 INFO:tasks.workunit.client.1.smithi138.stdout:2/188: creat d0/d21/d36/d15/f3b x:0 0 0 2024-04-25T01:35:56.418 INFO:tasks.workunit.client.1.smithi138.stdout:2/189: read - d0/d18/f2a zero size 2024-04-25T01:35:56.423 INFO:tasks.workunit.client.1.smithi138.stdout:2/190: write d0/d21/d36/d15/d1f/f2d [161410,98579] 0 2024-04-25T01:35:56.425 INFO:tasks.workunit.client.1.smithi138.stdout:7/113: dwrite da/de/d13/f24 [0,4194304] 0 2024-04-25T01:35:56.425 INFO:tasks.workunit.client.0.smithi033.stdout:8/685: rename d6/f99 to d6/dd3/dfb/f101 0 2024-04-25T01:35:56.428 INFO:tasks.workunit.client.1.smithi138.stdout:2/191: write d0/f8 [1714656,38506] 0 2024-04-25T01:35:56.430 INFO:tasks.workunit.client.0.smithi033.stdout:8/686: write d6/d45/f6d [77564,122589] 0 2024-04-25T01:35:56.434 INFO:tasks.workunit.client.1.smithi138.stdout:2/192: creat d0/d21/d36/f3c x:0 0 0 2024-04-25T01:35:56.434 INFO:tasks.workunit.client.1.smithi138.stdout:4/97: dread d0/f13 [0,4194304] 0 2024-04-25T01:35:56.435 INFO:tasks.workunit.client.0.smithi033.stdout:9/614: truncate d0/d2/dd/d20/d4e/f78 717839 0 2024-04-25T01:35:56.437 INFO:tasks.workunit.client.1.smithi138.stdout:5/92: link d4/f15 d4/f18 0 2024-04-25T01:35:56.438 INFO:tasks.workunit.client.1.smithi138.stdout:4/98: stat d0/c10 0 2024-04-25T01:35:56.440 INFO:tasks.workunit.client.0.smithi033.stdout:1/481: truncate d2/d2e/f73 2109055 0 2024-04-25T01:35:56.441 INFO:tasks.workunit.client.0.smithi033.stdout:3/695: dwrite d8/f1b [4194304,4194304] 0 2024-04-25T01:35:56.442 INFO:tasks.workunit.client.1.smithi138.stdout:8/272: rename d3/d8/f47 to d3/d8/da/db/d15/d20/f5a 0 2024-04-25T01:35:56.444 INFO:tasks.workunit.client.1.smithi138.stdout:5/93: readlink d4/l5 0 2024-04-25T01:35:56.445 INFO:tasks.workunit.client.0.smithi033.stdout:1/482: chown d2/d2e/d34/f82 3306 1 2024-04-25T01:35:56.445 INFO:tasks.workunit.client.1.smithi138.stdout:7/114: fsync da/de/f29 0 2024-04-25T01:35:56.445 INFO:tasks.workunit.client.1.smithi138.stdout:2/193: dwrite d0/d21/d36/d15/d1f/f37 [0,4194304] 0 2024-04-25T01:35:56.445 INFO:tasks.workunit.client.0.smithi033.stdout:5/704: truncate f17 5054691 0 2024-04-25T01:35:56.446 INFO:tasks.workunit.client.1.smithi138.stdout:8/273: write d3/d8/da/db/d15/f34 [2461842,89409] 0 2024-04-25T01:35:56.447 INFO:tasks.workunit.client.1.smithi138.stdout:4/99: creat d0/f15 x:0 0 0 2024-04-25T01:35:56.450 INFO:tasks.workunit.client.0.smithi033.stdout:5/705: chown d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/ccc 0 1 2024-04-25T01:35:56.451 INFO:tasks.workunit.client.1.smithi138.stdout:3/117: unlink db/f11 0 2024-04-25T01:35:56.451 INFO:tasks.workunit.client.1.smithi138.stdout:0/159: dwrite d6/f26 [0,4194304] 0 2024-04-25T01:35:56.452 INFO:tasks.workunit.client.0.smithi033.stdout:5/706: dread - d1c/d2a/f73 zero size 2024-04-25T01:35:56.453 INFO:tasks.workunit.client.1.smithi138.stdout:2/194: fdatasync d0/f8 0 2024-04-25T01:35:56.454 INFO:tasks.workunit.client.0.smithi033.stdout:9/615: dwrite d0/d2/dd/df/d1b/d3d/d8f/f98 [0,4194304] 0 2024-04-25T01:35:56.454 INFO:tasks.workunit.client.1.smithi138.stdout:5/94: write d4/f13 [5060408,58177] 0 2024-04-25T01:35:56.455 INFO:tasks.workunit.client.0.smithi033.stdout:6/714: dread d4/d1d/d3e/fa0 [0,4194304] 0 2024-04-25T01:35:56.455 INFO:tasks.workunit.client.1.smithi138.stdout:2/195: fdatasync d0/d21/f3a 0 2024-04-25T01:35:56.455 INFO:tasks.workunit.client.1.smithi138.stdout:5/95: chown d4/da/fd 194 1 2024-04-25T01:35:56.456 INFO:tasks.workunit.client.1.smithi138.stdout:7/115: creat da/de/f2c x:0 0 0 2024-04-25T01:35:56.458 INFO:tasks.workunit.client.0.smithi033.stdout:5/707: chown d1c/f23 0 1 2024-04-25T01:35:56.458 INFO:tasks.workunit.client.1.smithi138.stdout:4/100: mkdir d0/d16 0 2024-04-25T01:35:56.460 INFO:tasks.workunit.client.0.smithi033.stdout:1/483: truncate d2/d5/fd 2126267 0 2024-04-25T01:35:56.462 INFO:tasks.workunit.client.1.smithi138.stdout:8/274: unlink d3/d8/da/db/d15/d20/l55 0 2024-04-25T01:35:56.467 INFO:tasks.workunit.client.0.smithi033.stdout:2/705: dread da/f13 [0,4194304] 0 2024-04-25T01:35:56.469 INFO:tasks.workunit.client.1.smithi138.stdout:0/160: rename d6/d1d/l20 to d6/d1d/l36 0 2024-04-25T01:35:56.469 INFO:tasks.workunit.client.1.smithi138.stdout:8/275: dwrite d3/d8/da/db/f11 [0,4194304] 0 2024-04-25T01:35:56.470 INFO:tasks.workunit.client.1.smithi138.stdout:2/196: mknod d0/d32/c3d 0 2024-04-25T01:35:56.471 INFO:tasks.workunit.client.1.smithi138.stdout:9/111: write f1 [1054888,45059] 0 2024-04-25T01:35:56.472 INFO:tasks.workunit.client.1.smithi138.stdout:8/276: chown d3/d8/f40 95025 1 2024-04-25T01:35:56.472 INFO:tasks.workunit.client.0.smithi033.stdout:9/616: dwrite d0/d2/dd/df/d1b/fcf [0,4194304] 0 2024-04-25T01:35:56.473 INFO:tasks.workunit.client.0.smithi033.stdout:5/708: dread d1c/f5e [0,4194304] 0 2024-04-25T01:35:56.473 INFO:tasks.workunit.client.1.smithi138.stdout:8/277: read - d3/d16/f51 zero size 2024-04-25T01:35:56.473 INFO:tasks.workunit.client.1.smithi138.stdout:2/197: write d0/f31 [727429,111010] 0 2024-04-25T01:35:56.474 INFO:tasks.workunit.client.0.smithi033.stdout:6/715: rename d4/d24/d7c/lb4 to d4/d1d/d3e/d41/d5c/d7f/da4/lfb 0 2024-04-25T01:35:56.475 INFO:tasks.workunit.client.1.smithi138.stdout:7/116: creat da/de/d13/f2d x:0 0 0 2024-04-25T01:35:56.476 INFO:tasks.workunit.client.1.smithi138.stdout:8/278: write d3/f38 [449657,22139] 0 2024-04-25T01:35:56.477 INFO:tasks.workunit.client.0.smithi033.stdout:1/484: dwrite d2/d11/f5e [4194304,4194304] 0 2024-04-25T01:35:56.477 INFO:tasks.workunit.client.0.smithi033.stdout:6/716: mkdir d4/dfc 0 2024-04-25T01:35:56.478 INFO:tasks.workunit.client.1.smithi138.stdout:4/101: fdatasync d0/fc 0 2024-04-25T01:35:56.478 INFO:tasks.workunit.client.0.smithi033.stdout:6/717: fsync d4/d2a/f39 0 2024-04-25T01:35:56.479 INFO:tasks.workunit.client.0.smithi033.stdout:1/485: fdatasync d2/d5/d1b/d46/f47 0 2024-04-25T01:35:56.481 INFO:tasks.workunit.client.0.smithi033.stdout:6/718: chown d4/d1d/d3e/d4a/c7d 408 1 2024-04-25T01:35:56.486 INFO:tasks.workunit.client.1.smithi138.stdout:0/161: mknod d6/d19/c37 0 2024-04-25T01:35:56.490 INFO:tasks.workunit.client.1.smithi138.stdout:2/198: rmdir d0/d21/d36 39 2024-04-25T01:35:56.491 INFO:tasks.workunit.client.1.smithi138.stdout:7/117: rmdir da 39 2024-04-25T01:35:56.496 INFO:tasks.workunit.client.0.smithi033.stdout:3/696: dwrite d8/dd/d3d/d4f/da4/db2/dbf/dc5/de5/d86/d92/f45 [0,4194304] 0 2024-04-25T01:35:56.496 INFO:tasks.workunit.client.1.smithi138.stdout:0/162: symlink d6/df/l38 0 2024-04-25T01:35:56.498 INFO:tasks.workunit.client.0.smithi033.stdout:3/697: mknod d8/dd/d1d/cec 0 2024-04-25T01:35:56.498 INFO:tasks.workunit.client.1.smithi138.stdout:5/96: creat d4/f19 x:0 0 0 2024-04-25T01:35:56.499 INFO:tasks.workunit.client.1.smithi138.stdout:2/199: dwrite d0/d18/f2a [0,4194304] 0 2024-04-25T01:35:56.499 INFO:tasks.workunit.client.1.smithi138.stdout:5/97: dread - d4/f19 zero size 2024-04-25T01:35:56.500 INFO:tasks.workunit.client.1.smithi138.stdout:7/118: read da/de/f1c [1750398,77171] 0 2024-04-25T01:35:56.502 INFO:tasks.workunit.client.0.smithi033.stdout:1/486: dread d2/d5/d1b/d46/f69 [0,4194304] 0 2024-04-25T01:35:56.503 INFO:tasks.workunit.client.0.smithi033.stdout:3/698: rename d8/dd/d3d/d4f/da4/db2 to d8/dd/d3d/d63/d7c/ded 0 2024-04-25T01:35:56.503 INFO:tasks.workunit.client.1.smithi138.stdout:4/102: rename d0/f15 to d0/f17 0 2024-04-25T01:35:56.504 INFO:tasks.workunit.client.0.smithi033.stdout:3/699: readlink d8/dd/l9e 0 2024-04-25T01:35:56.505 INFO:tasks.workunit.client.0.smithi033.stdout:1/487: mknod d2/d5/d75/c93 0 2024-04-25T01:35:56.506 INFO:tasks.workunit.client.1.smithi138.stdout:0/163: mkdir d6/d39 0 2024-04-25T01:35:56.506 INFO:tasks.workunit.client.0.smithi033.stdout:3/700: fsync d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/f21 0 2024-04-25T01:35:56.509 INFO:tasks.workunit.client.1.smithi138.stdout:0/164: dread d6/df/f18 [0,4194304] 0 2024-04-25T01:35:56.509 INFO:tasks.workunit.client.0.smithi033.stdout:3/701: creat d8/dd/d3d/dce/fee x:0 0 0 2024-04-25T01:35:56.510 INFO:tasks.workunit.client.0.smithi033.stdout:3/702: write d8/fa [4292441,28721] 0 2024-04-25T01:35:56.512 INFO:tasks.workunit.client.0.smithi033.stdout:3/703: truncate d8/dd/d3d/d4f/faf 624596 0 2024-04-25T01:35:56.514 INFO:tasks.workunit.client.0.smithi033.stdout:3/704: mknod d8/dd/d3d/d63/d7c/d4d/cef 0 2024-04-25T01:35:56.516 INFO:tasks.workunit.client.0.smithi033.stdout:3/705: dread d8/dd/d3d/d63/d7c/fbe [0,4194304] 0 2024-04-25T01:35:56.520 INFO:tasks.workunit.client.0.smithi033.stdout:1/488: dread d2/d5/fd [0,4194304] 0 2024-04-25T01:35:56.524 INFO:tasks.workunit.client.0.smithi033.stdout:6/719: rmdir d4/d1d/d3e/d41/d5c 39 2024-04-25T01:35:56.524 INFO:tasks.workunit.client.0.smithi033.stdout:3/706: dwrite d8/dd/d3d/d63/d7c/ded/dbf/dc5/fc6 [0,4194304] 0 2024-04-25T01:35:56.525 INFO:tasks.workunit.client.0.smithi033.stdout:6/720: fsync d4/d1d/d3e/d41/d54/f57 0 2024-04-25T01:35:56.527 INFO:tasks.workunit.client.0.smithi033.stdout:6/721: fdatasync d4/d2a/f3f 0 2024-04-25T01:35:56.529 INFO:tasks.workunit.client.0.smithi033.stdout:5/709: write d1c/f98 [926359,34369] 0 2024-04-25T01:35:56.530 INFO:tasks.workunit.client.0.smithi033.stdout:1/489: unlink d2/d2e/d34/d60/f8b 0 2024-04-25T01:35:56.534 INFO:tasks.workunit.client.0.smithi033.stdout:3/707: mkdir d8/dd/d3d/d4f/df0 0 2024-04-25T01:35:56.538 INFO:tasks.workunit.client.0.smithi033.stdout:6/722: rmdir d4/d1d/d3e/d41/d54 39 2024-04-25T01:35:56.539 INFO:tasks.workunit.client.0.smithi033.stdout:1/490: mkdir d2/d5/d6c/d94 0 2024-04-25T01:35:56.540 INFO:tasks.workunit.client.0.smithi033.stdout:1/491: fsync d2/d2e/d34/f50 0 2024-04-25T01:35:56.540 INFO:tasks.workunit.client.0.smithi033.stdout:6/723: write d4/d24/d63/fbb [275060,62629] 0 2024-04-25T01:35:56.541 INFO:tasks.workunit.client.0.smithi033.stdout:3/708: symlink d8/dd/d3d/d63/d7c/d9d/lf1 0 2024-04-25T01:35:56.543 INFO:tasks.workunit.client.0.smithi033.stdout:6/724: chown d4/d1d/d3e/d41/d93/lce 48560583 1 2024-04-25T01:35:56.544 INFO:tasks.workunit.client.0.smithi033.stdout:5/710: mknod d1c/d2a/d50/db1/dd3/ddd/cea 0 2024-04-25T01:35:56.546 INFO:tasks.workunit.client.0.smithi033.stdout:3/709: mkdir d8/dd/d3d/d63/d7c/ded/dc7/ddf/df2 0 2024-04-25T01:35:56.547 INFO:tasks.workunit.client.0.smithi033.stdout:5/711: truncate d1c/d2a/d50/d52/da6/d4a/fbf 203065 0 2024-04-25T01:35:56.548 INFO:tasks.workunit.client.0.smithi033.stdout:6/725: rename d4/d15/c21 to d4/d24/d36/d42/cfd 0 2024-04-25T01:35:56.549 INFO:tasks.workunit.client.0.smithi033.stdout:5/712: write f1b [65898,21537] 0 2024-04-25T01:35:56.553 INFO:tasks.workunit.client.0.smithi033.stdout:3/710: fsync d8/dd/d1d/f93 0 2024-04-25T01:35:56.553 INFO:tasks.workunit.client.0.smithi033.stdout:1/492: link d2/d5/d1b/d46/d5f/l77 d2/d11/d28/l95 0 2024-04-25T01:35:56.555 INFO:tasks.workunit.client.0.smithi033.stdout:3/711: stat d8/dd/d3d/d63/d7c 0 2024-04-25T01:35:56.557 INFO:tasks.workunit.client.1.smithi138.stdout:8/279: write d3/f1e [2632569,45131] 0 2024-04-25T01:35:56.558 INFO:tasks.workunit.client.1.smithi138.stdout:2/200: dwrite d0/d21/d36/f17 [0,4194304] 0 2024-04-25T01:35:56.559 INFO:tasks.workunit.client.0.smithi033.stdout:6/726: dwrite d4/d15/f94 [4194304,4194304] 0 2024-04-25T01:35:56.561 INFO:tasks.workunit.client.0.smithi033.stdout:5/713: link d1c/d53/f57 d1c/d2a/d50/d5b/feb 0 2024-04-25T01:35:56.561 INFO:tasks.workunit.client.1.smithi138.stdout:5/98: creat d4/f1a x:0 0 0 2024-04-25T01:35:56.562 INFO:tasks.workunit.client.1.smithi138.stdout:7/119: dwrite da/fc [0,4194304] 0 2024-04-25T01:35:56.562 INFO:tasks.workunit.client.0.smithi033.stdout:1/493: creat d2/d5/d1b/d46/d5f/d63/d8f/f96 x:0 0 0 2024-04-25T01:35:56.565 INFO:tasks.workunit.client.1.smithi138.stdout:3/118: dwrite f6 [0,4194304] 0 2024-04-25T01:35:56.566 INFO:tasks.workunit.client.1.smithi138.stdout:4/103: rename d0/f17 to d0/d16/f18 0 2024-04-25T01:35:56.572 INFO:tasks.workunit.client.0.smithi033.stdout:5/714: truncate d1c/d2a/d50/d52/fc9 9435 0 2024-04-25T01:35:56.573 INFO:tasks.workunit.client.1.smithi138.stdout:3/119: read db/f15 [291897,117198] 0 2024-04-25T01:35:56.574 INFO:tasks.workunit.client.1.smithi138.stdout:5/99: unlink d4/f15 0 2024-04-25T01:35:56.574 INFO:tasks.workunit.client.0.smithi033.stdout:5/715: write d1c/d2a/d3d/f40 [3486043,7627] 0 2024-04-25T01:35:56.575 INFO:tasks.workunit.client.1.smithi138.stdout:2/201: dwrite d0/d18/f2a [0,4194304] 0 2024-04-25T01:35:56.578 INFO:tasks.workunit.client.1.smithi138.stdout:8/280: unlink d3/d8/da/db/d15/l45 0 2024-04-25T01:35:56.580 INFO:tasks.workunit.client.1.smithi138.stdout:7/120: mkdir da/de/d2e 0 2024-04-25T01:35:56.583 INFO:tasks.workunit.client.1.smithi138.stdout:0/165: truncate d6/f28 554200 0 2024-04-25T01:35:56.586 INFO:tasks.workunit.client.1.smithi138.stdout:5/100: rename d4/da/d14 to d4/da/d1b 0 2024-04-25T01:35:56.587 INFO:tasks.workunit.client.1.smithi138.stdout:2/202: symlink d0/d18/l3e 0 2024-04-25T01:35:56.590 INFO:tasks.workunit.client.1.smithi138.stdout:3/120: symlink db/df/d17/d23/l27 0 2024-04-25T01:35:56.590 INFO:tasks.workunit.client.1.smithi138.stdout:0/166: mkdir d6/df/d3a 0 2024-04-25T01:35:56.592 INFO:tasks.workunit.client.1.smithi138.stdout:5/101: mkdir d4/da/d1b/d1c 0 2024-04-25T01:35:56.592 INFO:tasks.workunit.client.1.smithi138.stdout:8/281: symlink d3/d8/da/l5b 0 2024-04-25T01:35:56.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/121: symlink da/de/d2e/l2f 0 2024-04-25T01:35:56.595 INFO:tasks.workunit.client.1.smithi138.stdout:3/121: symlink db/df/d17/d20/l28 0 2024-04-25T01:35:56.597 INFO:tasks.workunit.client.1.smithi138.stdout:0/167: creat d6/f3b x:0 0 0 2024-04-25T01:35:56.599 INFO:tasks.workunit.client.0.smithi033.stdout:1/494: rmdir d2/d5/d75/d59 39 2024-04-25T01:35:56.601 INFO:tasks.workunit.client.0.smithi033.stdout:6/727: creat d4/d1d/d3e/d41/d5c/d7f/da4/ffe x:0 0 0 2024-04-25T01:35:56.602 INFO:tasks.workunit.client.1.smithi138.stdout:7/122: dread da/fc [0,4194304] 0 2024-04-25T01:35:56.602 INFO:tasks.workunit.client.1.smithi138.stdout:8/282: dwrite d3/d8/da/db/d15/f30 [0,4194304] 0 2024-04-25T01:35:56.602 INFO:tasks.workunit.client.0.smithi033.stdout:5/716: chown d1c/d2a/d50/f90 227383645 1 2024-04-25T01:35:56.604 INFO:tasks.workunit.client.1.smithi138.stdout:7/123: truncate da/de/d13/f2d 500135 0 2024-04-25T01:35:56.604 INFO:tasks.workunit.client.1.smithi138.stdout:8/283: chown d3/d8/da/db/d15/d20/c29 29588 1 2024-04-25T01:35:56.604 INFO:tasks.workunit.client.0.smithi033.stdout:5/717: dread - d1c/d2a/d50/d52/fd9 zero size 2024-04-25T01:35:56.605 INFO:tasks.workunit.client.0.smithi033.stdout:1/495: mknod d2/d5/d75/c97 0 2024-04-25T01:35:56.606 INFO:tasks.workunit.client.1.smithi138.stdout:4/104: write d0/fc [969918,90000] 0 2024-04-25T01:35:56.606 INFO:tasks.workunit.client.0.smithi033.stdout:6/728: symlink d4/d1d/d3e/d41/d5c/df6/lff 0 2024-04-25T01:35:56.607 INFO:tasks.workunit.client.1.smithi138.stdout:3/122: creat db/df/d17/d20/f29 x:0 0 0 2024-04-25T01:35:56.607 INFO:tasks.workunit.client.1.smithi138.stdout:4/105: write d0/f13 [762417,102356] 0 2024-04-25T01:35:56.608 INFO:tasks.workunit.client.1.smithi138.stdout:0/168: creat d6/d1d/f3c x:0 0 0 2024-04-25T01:35:56.609 INFO:tasks.workunit.client.1.smithi138.stdout:3/123: truncate db/df/d17/d20/f26 439474 0 2024-04-25T01:35:56.609 INFO:tasks.workunit.client.0.smithi033.stdout:5/718: mknod d1c/d2a/d50/d52/da6/d3e/d70/d7a/dab/cec 0 2024-04-25T01:35:56.610 INFO:tasks.workunit.client.0.smithi033.stdout:1/496: stat d2/d5/d75/d59 0 2024-04-25T01:35:56.613 INFO:tasks.workunit.client.1.smithi138.stdout:7/124: write da/f15 [3620504,1601] 0 2024-04-25T01:35:56.613 INFO:tasks.workunit.client.0.smithi033.stdout:5/719: fsync f14 0 2024-04-25T01:35:56.613 INFO:tasks.workunit.client.1.smithi138.stdout:8/284: readlink d3/d8/da/de/l49 0 2024-04-25T01:35:56.621 INFO:tasks.workunit.client.1.smithi138.stdout:3/124: creat db/d14/f2a x:0 0 0 2024-04-25T01:35:56.622 INFO:tasks.workunit.client.1.smithi138.stdout:3/125: chown f6 28612 1 2024-04-25T01:35:56.624 INFO:tasks.workunit.client.1.smithi138.stdout:7/125: write da/fc [1052602,114111] 0 2024-04-25T01:35:56.624 INFO:tasks.workunit.client.0.smithi033.stdout:3/712: truncate d8/dd/d11/fac 156890 0 2024-04-25T01:35:56.625 INFO:tasks.workunit.client.0.smithi033.stdout:1/497: dwrite d2/d5/d1b/d46/d5f/d63/d8f/f96 [0,4194304] 0 2024-04-25T01:35:56.626 INFO:tasks.workunit.client.0.smithi033.stdout:6/729: write d4/d1d/f50 [267449,35641] 0 2024-04-25T01:35:56.627 INFO:tasks.workunit.client.1.smithi138.stdout:7/126: write da/f15 [3463649,15] 0 2024-04-25T01:35:56.627 INFO:tasks.workunit.client.1.smithi138.stdout:2/203: truncate d0/d21/f27 511759 0 2024-04-25T01:35:56.628 INFO:tasks.workunit.client.0.smithi033.stdout:6/730: chown d4/d1d/d3e/fa0 9 1 2024-04-25T01:35:56.629 INFO:tasks.workunit.client.1.smithi138.stdout:5/102: dwrite d4/da/fe [0,4194304] 0 2024-04-25T01:35:56.629 INFO:tasks.workunit.client.0.smithi033.stdout:5/720: rename d1c/c56 to d1c/ced 0 2024-04-25T01:35:56.630 INFO:tasks.workunit.client.1.smithi138.stdout:2/204: write d0/d21/d36/d15/d1f/f26 [5047155,66242] 0 2024-04-25T01:35:56.632 INFO:tasks.workunit.client.1.smithi138.stdout:7/127: mkdir da/de/d13/d30 0 2024-04-25T01:35:56.632 INFO:tasks.workunit.client.1.smithi138.stdout:2/205: write d0/f31 [1608362,25677] 0 2024-04-25T01:35:56.632 INFO:tasks.workunit.client.0.smithi033.stdout:3/713: symlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/dc0/lf3 0 2024-04-25T01:35:56.633 INFO:tasks.workunit.client.0.smithi033.stdout:1/498: mkdir d2/d5/d75/d59/d83/d98 0 2024-04-25T01:35:56.634 INFO:tasks.workunit.client.0.smithi033.stdout:1/499: chown d2/d2e/f67 0 1 2024-04-25T01:35:56.636 INFO:tasks.workunit.client.0.smithi033.stdout:5/721: rmdir d1c/d2a/d50/d52/da6 39 2024-04-25T01:35:56.637 INFO:tasks.workunit.client.1.smithi138.stdout:3/126: getdents db/df/d17/d20 0 2024-04-25T01:35:56.637 INFO:tasks.workunit.client.0.smithi033.stdout:6/731: creat d4/dfc/f100 x:0 0 0 2024-04-25T01:35:56.639 INFO:tasks.workunit.client.0.smithi033.stdout:6/732: chown d4/dfc/f100 0 1 2024-04-25T01:35:56.639 INFO:tasks.workunit.client.0.smithi033.stdout:3/714: unlink d8/dd/d3d/d63/d7c/d9d/lf1 0 2024-04-25T01:35:56.639 INFO:tasks.workunit.client.1.smithi138.stdout:2/206: creat d0/d32/f3f x:0 0 0 2024-04-25T01:35:56.641 INFO:tasks.workunit.client.1.smithi138.stdout:7/128: unlink da/de/d1b/f28 0 2024-04-25T01:35:56.642 INFO:tasks.workunit.client.0.smithi033.stdout:1/500: symlink d2/d5/d1b/d46/d5f/l99 0 2024-04-25T01:35:56.643 INFO:tasks.workunit.client.1.smithi138.stdout:7/129: truncate da/fc 4628032 0 2024-04-25T01:35:56.644 INFO:tasks.workunit.client.0.smithi033.stdout:5/722: chown d1c/ced 3382152 1 2024-04-25T01:35:56.646 INFO:tasks.workunit.client.0.smithi033.stdout:6/733: unlink d4/d1d/f1f 0 2024-04-25T01:35:56.647 INFO:tasks.workunit.client.1.smithi138.stdout:0/169: write d6/fc [2970204,2727] 0 2024-04-25T01:35:56.647 INFO:tasks.workunit.client.0.smithi033.stdout:6/734: write d4/d15/fd5 [922469,120747] 0 2024-04-25T01:35:56.648 INFO:tasks.workunit.client.0.smithi033.stdout:3/715: dwrite d8/dd/d1d/f7e [0,4194304] 0 2024-04-25T01:35:56.650 INFO:tasks.workunit.client.1.smithi138.stdout:3/127: dwrite db/df/d19/d1a/f21 [0,4194304] 0 2024-04-25T01:35:56.651 INFO:tasks.workunit.client.1.smithi138.stdout:4/106: truncate d0/ff 6639829 0 2024-04-25T01:35:56.653 INFO:tasks.workunit.client.1.smithi138.stdout:5/103: rename d4/f17 to d4/da/d1b/d1c/f1d 0 2024-04-25T01:35:56.655 INFO:tasks.workunit.client.1.smithi138.stdout:8/285: truncate d3/f5 3406603 0 2024-04-25T01:35:56.656 INFO:tasks.workunit.client.0.smithi033.stdout:3/716: creat d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/ff4 x:0 0 0 2024-04-25T01:35:56.656 INFO:tasks.workunit.client.1.smithi138.stdout:2/207: link d0/d21/f3a d0/d32/d35/f40 0 2024-04-25T01:35:56.658 INFO:tasks.workunit.client.1.smithi138.stdout:7/130: getdents da/de/d13/d30 0 2024-04-25T01:35:56.660 INFO:tasks.workunit.client.0.smithi033.stdout:1/501: dwrite d2/d5/d1b/d46/f69 [0,4194304] 0 2024-04-25T01:35:56.661 INFO:tasks.workunit.client.1.smithi138.stdout:3/128: truncate db/df/d17/f1c 3711582 0 2024-04-25T01:35:56.663 INFO:tasks.workunit.client.0.smithi033.stdout:3/717: chown d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e 0 1 2024-04-25T01:35:56.663 INFO:tasks.workunit.client.0.smithi033.stdout:5/723: mknod d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e/cee 0 2024-04-25T01:35:56.663 INFO:tasks.workunit.client.1.smithi138.stdout:0/170: creat d6/df/d3a/f3d x:0 0 0 2024-04-25T01:35:56.664 INFO:tasks.workunit.client.0.smithi033.stdout:6/735: dwrite d4/d24/f64 [0,4194304] 0 2024-04-25T01:35:56.666 INFO:tasks.workunit.client.0.smithi033.stdout:1/502: chown d2/d5/d75/d59/l5d 906531 1 2024-04-25T01:35:56.666 INFO:tasks.workunit.client.1.smithi138.stdout:7/131: rename f4 to da/de/d1b/f31 0 2024-04-25T01:35:56.669 INFO:tasks.workunit.client.1.smithi138.stdout:8/286: truncate d3/d8/da/db/d15/d20/f5a 296609 0 2024-04-25T01:35:56.671 INFO:tasks.workunit.client.0.smithi033.stdout:3/718: mknod d8/dd/d3d/cf5 0 2024-04-25T01:35:56.671 INFO:tasks.workunit.client.1.smithi138.stdout:3/129: dwrite f6 [0,4194304] 0 2024-04-25T01:35:56.672 INFO:tasks.workunit.client.0.smithi033.stdout:6/736: creat d4/dfc/f101 x:0 0 0 2024-04-25T01:35:56.673 INFO:tasks.workunit.client.1.smithi138.stdout:0/171: fdatasync d6/df/f2c 0 2024-04-25T01:35:56.673 INFO:tasks.workunit.client.0.smithi033.stdout:1/503: symlink d2/d5/d1b/l9a 0 2024-04-25T01:35:56.675 INFO:tasks.workunit.client.0.smithi033.stdout:1/504: write d2/d5/d75/f8d [200397,36564] 0 2024-04-25T01:35:56.678 INFO:tasks.workunit.client.0.smithi033.stdout:3/719: getdents d8/dd/d3d/d63/d7c/ded/dbf 0 2024-04-25T01:35:56.679 INFO:tasks.workunit.client.1.smithi138.stdout:7/132: write da/de/d13/f23 [5021021,21776] 0 2024-04-25T01:35:56.680 INFO:tasks.workunit.client.1.smithi138.stdout:7/133: chown da/de/d13/d21 79 1 2024-04-25T01:35:56.680 INFO:tasks.workunit.client.0.smithi033.stdout:1/505: read d2/d5/d1b/d46/f47 [3531914,19075] 0 2024-04-25T01:35:56.684 INFO:tasks.workunit.client.1.smithi138.stdout:2/208: creat d0/d21/f41 x:0 0 0 2024-04-25T01:35:56.689 INFO:tasks.workunit.client.1.smithi138.stdout:0/172: stat d6/d1d/l36 0 2024-04-25T01:35:56.689 INFO:tasks.workunit.client.0.smithi033.stdout:6/737: dwrite d4/d24/f8f [0,4194304] 0 2024-04-25T01:35:56.689 INFO:tasks.workunit.client.1.smithi138.stdout:7/134: write da/de/f1c [2568780,34545] 0 2024-04-25T01:35:56.691 INFO:tasks.workunit.client.0.smithi033.stdout:1/506: mkdir d2/d5/d75/d9b 0 2024-04-25T01:35:56.692 INFO:tasks.workunit.client.1.smithi138.stdout:8/287: symlink d3/d16/d1f/d2a/d4f/l5c 0 2024-04-25T01:35:56.692 INFO:tasks.workunit.client.0.smithi033.stdout:1/507: stat d2/d5/d6c 0 2024-04-25T01:35:56.693 INFO:tasks.workunit.client.0.smithi033.stdout:6/738: rename d4/d1d/d3e/d4a/f4f to d4/d1d/d3e/d4a/dcb/f102 0 2024-04-25T01:35:56.696 INFO:tasks.workunit.client.0.smithi033.stdout:1/508: dread d2/d2e/d34/d60/f74 [0,4194304] 0 2024-04-25T01:35:56.696 INFO:tasks.workunit.client.0.smithi033.stdout:6/739: symlink d4/dfc/l103 0 2024-04-25T01:35:56.697 INFO:tasks.workunit.client.0.smithi033.stdout:3/720: dread d8/dd/d3d/d4f/f69 [0,4194304] 0 2024-04-25T01:35:56.698 INFO:tasks.workunit.client.1.smithi138.stdout:0/173: read d6/f28 [475545,125406] 0 2024-04-25T01:35:56.699 INFO:tasks.workunit.client.0.smithi033.stdout:5/724: dread d1c/d2a/d3d/f4d [0,4194304] 0 2024-04-25T01:35:56.699 INFO:tasks.workunit.client.0.smithi033.stdout:1/509: truncate d2/d11/f91 741603 0 2024-04-25T01:35:56.699 INFO:tasks.workunit.client.1.smithi138.stdout:0/174: dread - d6/df/d3a/f3d zero size 2024-04-25T01:35:56.700 INFO:tasks.workunit.client.1.smithi138.stdout:5/104: symlink d4/l1e 0 2024-04-25T01:35:56.701 INFO:tasks.workunit.client.0.smithi033.stdout:6/740: truncate d4/d1d/d3e/d41/d54/ff7 528598 0 2024-04-25T01:35:56.703 INFO:tasks.workunit.client.1.smithi138.stdout:7/135: unlink da/de/d2e/l2f 0 2024-04-25T01:35:56.703 INFO:tasks.workunit.client.0.smithi033.stdout:1/510: symlink d2/d5/d1b/d46/l9c 0 2024-04-25T01:35:56.703 INFO:tasks.workunit.client.1.smithi138.stdout:7/136: chown c8 10879612 1 2024-04-25T01:35:56.704 INFO:tasks.workunit.client.0.smithi033.stdout:6/741: write d4/d24/d36/d42/d9d/fdd [3728672,109613] 0 2024-04-25T01:35:56.704 INFO:tasks.workunit.client.1.smithi138.stdout:8/288: mkdir d3/d8/da/db/d15/d5d 0 2024-04-25T01:35:56.704 INFO:tasks.workunit.client.1.smithi138.stdout:0/175: dread d6/fc [0,4194304] 0 2024-04-25T01:35:56.705 INFO:tasks.workunit.client.0.smithi033.stdout:3/721: creat d8/dd/d3d/d63/d7c/ff6 x:0 0 0 2024-04-25T01:35:56.705 INFO:tasks.workunit.client.0.smithi033.stdout:6/742: fsync d4/d24/d36/f6c 0 2024-04-25T01:35:56.707 INFO:tasks.workunit.client.0.smithi033.stdout:1/511: dread - d2/d5/d1b/d46/d5f/d63/d64/f6a zero size 2024-04-25T01:35:56.708 INFO:tasks.workunit.client.0.smithi033.stdout:3/722: mknod d8/dd/d1d/cf7 0 2024-04-25T01:35:56.708 INFO:tasks.workunit.client.1.smithi138.stdout:5/105: read d4/f9 [4948511,111454] 0 2024-04-25T01:35:56.709 INFO:tasks.workunit.client.0.smithi033.stdout:1/512: write d2/d11/f91 [1128301,33683] 0 2024-04-25T01:35:56.709 INFO:tasks.workunit.client.0.smithi033.stdout:6/743: creat d4/d1d/d3e/d41/d93/f104 x:0 0 0 2024-04-25T01:35:56.710 INFO:tasks.workunit.client.0.smithi033.stdout:3/723: chown d8/dd/c4a 31657054 1 2024-04-25T01:35:56.711 INFO:tasks.workunit.client.1.smithi138.stdout:0/176: mknod d6/d19/d1c/c3e 0 2024-04-25T01:35:56.712 INFO:tasks.workunit.client.0.smithi033.stdout:6/744: write d4/d1d/d3e/d41/f58 [976361,27238] 0 2024-04-25T01:35:56.713 INFO:tasks.workunit.client.0.smithi033.stdout:1/513: mkdir d2/d5/d1b/d46/d5f/d63/d9d 0 2024-04-25T01:35:56.715 INFO:tasks.workunit.client.1.smithi138.stdout:8/289: symlink d3/l5e 0 2024-04-25T01:35:56.716 INFO:tasks.workunit.client.0.smithi033.stdout:6/745: rmdir d4/d2a 39 2024-04-25T01:35:56.716 INFO:tasks.workunit.client.1.smithi138.stdout:8/290: read - d3/d16/d1f/d31/f3d zero size 2024-04-25T01:35:56.717 INFO:tasks.workunit.client.0.smithi033.stdout:3/724: dread d8/f59 [0,4194304] 0 2024-04-25T01:35:56.717 INFO:tasks.workunit.client.1.smithi138.stdout:2/209: getdents d0/d32/d35 0 2024-04-25T01:35:56.719 INFO:tasks.workunit.client.1.smithi138.stdout:2/210: dread - d0/d21/f41 zero size 2024-04-25T01:35:56.720 INFO:tasks.workunit.client.1.smithi138.stdout:5/106: write d4/f19 [640768,56355] 0 2024-04-25T01:35:56.720 INFO:tasks.workunit.client.0.smithi033.stdout:6/746: write d4/d1d/d3e/d41/d93/f104 [815119,87666] 0 2024-04-25T01:35:56.721 INFO:tasks.workunit.client.1.smithi138.stdout:2/211: readlink d0/l1c 0 2024-04-25T01:35:56.722 INFO:tasks.workunit.client.0.smithi033.stdout:1/514: dwrite d2/d5/d1b/d46/d5f/d63/d8f/f96 [0,4194304] 0 2024-04-25T01:35:56.723 INFO:tasks.workunit.client.1.smithi138.stdout:3/130: dwrite db/df/d17/f1c [0,4194304] 0 2024-04-25T01:35:56.724 INFO:tasks.workunit.client.0.smithi033.stdout:1/515: readlink d2/d5/d1b/l9a 0 2024-04-25T01:35:56.724 INFO:tasks.workunit.client.1.smithi138.stdout:0/177: mknod d6/d19/d1c/c3f 0 2024-04-25T01:35:56.727 INFO:tasks.workunit.client.0.smithi033.stdout:5/725: dread d1c/f22 [0,4194304] 0 2024-04-25T01:35:56.728 INFO:tasks.workunit.client.1.smithi138.stdout:5/107: fsync d4/f13 0 2024-04-25T01:35:56.729 INFO:tasks.workunit.client.0.smithi033.stdout:5/726: readlink d1c/d2a/d50/d52/da6/l27 0 2024-04-25T01:35:56.732 INFO:tasks.workunit.client.1.smithi138.stdout:2/212: dread d0/f10 [0,4194304] 0 2024-04-25T01:35:56.732 INFO:tasks.workunit.client.1.smithi138.stdout:8/291: mknod d3/d8/da/db/d15/d5d/c5f 0 2024-04-25T01:35:56.734 INFO:tasks.workunit.client.0.smithi033.stdout:3/725: link d8/dd/c4a d8/dd/d3d/d63/d7c/ded/dbf/dc5/cf8 0 2024-04-25T01:35:56.734 INFO:tasks.workunit.client.0.smithi033.stdout:1/516: fsync d2/d5/d75/f53 0 2024-04-25T01:35:56.735 INFO:tasks.workunit.client.1.smithi138.stdout:5/108: rename d4/da/d1b/d1c/f1d to d4/da/d1b/f1f 0 2024-04-25T01:35:56.736 INFO:tasks.workunit.client.0.smithi033.stdout:1/517: dread - d2/d5/d6c/f87 zero size 2024-04-25T01:35:56.738 INFO:tasks.workunit.client.0.smithi033.stdout:3/726: readlink d8/dd/d3d/d63/l67 0 2024-04-25T01:35:56.739 INFO:tasks.workunit.client.1.smithi138.stdout:2/213: dread d0/f10 [0,4194304] 0 2024-04-25T01:35:56.739 INFO:tasks.workunit.client.0.smithi033.stdout:6/747: getdents d4/d1d/d3e/d70 0 2024-04-25T01:35:56.740 INFO:tasks.workunit.client.1.smithi138.stdout:7/137: truncate da/de/f14 830975 0 2024-04-25T01:35:56.740 INFO:tasks.workunit.client.1.smithi138.stdout:0/178: link d6/d1d/l1b d6/d39/l40 0 2024-04-25T01:35:56.741 INFO:tasks.workunit.client.0.smithi033.stdout:1/518: creat d2/d5/d1b/d46/d5f/d63/d8f/f9e x:0 0 0 2024-04-25T01:35:56.742 INFO:tasks.workunit.client.0.smithi033.stdout:3/727: mkdir d8/dd/d3d/d63/d7c/d9d/de8/df9 0 2024-04-25T01:35:56.743 INFO:tasks.workunit.client.0.smithi033.stdout:1/519: readlink d2/d5/d1b/d46/d5f/l77 0 2024-04-25T01:35:56.744 INFO:tasks.workunit.client.0.smithi033.stdout:1/520: dread - d2/d11/d28/f7d zero size 2024-04-25T01:35:56.746 INFO:tasks.workunit.client.0.smithi033.stdout:1/521: write d2/d2e/d34/f50 [1135496,116255] 0 2024-04-25T01:35:56.747 INFO:tasks.workunit.client.1.smithi138.stdout:8/292: dwrite d3/d16/f51 [0,4194304] 0 2024-04-25T01:35:56.748 INFO:tasks.workunit.client.0.smithi033.stdout:1/522: truncate d2/d2e/d34/f50 1489957 0 2024-04-25T01:35:56.748 INFO:tasks.workunit.client.0.smithi033.stdout:3/728: mkdir d8/dd/dfa 0 2024-04-25T01:35:56.749 INFO:tasks.workunit.client.0.smithi033.stdout:6/748: link d4/d24/d36/d42/d97/ldf d4/d1d/d3e/d4a/l105 0 2024-04-25T01:35:56.750 INFO:tasks.workunit.client.0.smithi033.stdout:1/523: chown d2/d5/d1b/d46/d5f/d63/d64/c78 63276 1 2024-04-25T01:35:56.753 INFO:tasks.workunit.client.0.smithi033.stdout:6/749: creat d4/d1d/d3e/d41/d5c/dae/f106 x:0 0 0 2024-04-25T01:35:56.755 INFO:tasks.workunit.client.0.smithi033.stdout:6/750: symlink d4/d1d/d3e/d41/d5c/d7f/da4/l107 0 2024-04-25T01:35:56.757 INFO:tasks.workunit.client.0.smithi033.stdout:5/727: dread d1c/d2a/d50/d52/da6/d3e/d70/fc3 [0,4194304] 0 2024-04-25T01:35:56.759 INFO:tasks.workunit.client.0.smithi033.stdout:5/728: write d1c/d2a/d50/d52/fc9 [483416,128647] 0 2024-04-25T01:35:56.760 INFO:tasks.workunit.client.0.smithi033.stdout:6/751: rename d4/d1d/f50 to d4/d1d/d3e/d70/f108 0 2024-04-25T01:35:56.766 INFO:tasks.workunit.client.0.smithi033.stdout:5/729: creat d1c/d2a/d50/d52/da6/d3e/d70/dae/fef x:0 0 0 2024-04-25T01:35:56.767 INFO:tasks.workunit.client.0.smithi033.stdout:6/752: chown d4/d24/d36/d42/d97/ldf 3210 1 2024-04-25T01:35:56.769 INFO:tasks.workunit.client.1.smithi138.stdout:3/131: dwrite db/f15 [0,4194304] 0 2024-04-25T01:35:56.771 INFO:tasks.workunit.client.0.smithi033.stdout:6/753: truncate d4/d15/f94 9403385 0 2024-04-25T01:35:56.776 INFO:tasks.workunit.client.0.smithi033.stdout:1/524: dwrite d2/f62 [0,4194304] 0 2024-04-25T01:35:56.777 INFO:tasks.workunit.client.0.smithi033.stdout:6/754: rename d4/d1d/f69 to d4/d1d/d3e/d41/d54/da1/f109 0 2024-04-25T01:35:56.780 INFO:tasks.workunit.client.0.smithi033.stdout:6/755: creat d4/d24/d36/d42/d97/db2/f10a x:0 0 0 2024-04-25T01:35:56.781 INFO:tasks.workunit.client.0.smithi033.stdout:1/525: symlink d2/d5/d6c/d94/l9f 0 2024-04-25T01:35:56.782 INFO:tasks.workunit.client.0.smithi033.stdout:6/756: chown d4/d1d/d3e/d4a/l105 190 1 2024-04-25T01:35:56.783 INFO:tasks.workunit.client.0.smithi033.stdout:5/730: truncate d1c/d2a/d3d/d59/f9e 304184 0 2024-04-25T01:35:56.784 INFO:tasks.workunit.client.0.smithi033.stdout:6/757: fdatasync d4/f6 0 2024-04-25T01:35:56.787 INFO:tasks.workunit.client.0.smithi033.stdout:6/758: write d4/d24/d36/d42/d9d/fb0 [378964,34329] 0 2024-04-25T01:35:56.788 INFO:tasks.workunit.client.0.smithi033.stdout:5/731: creat d1c/d2a/d50/d52/ff0 x:0 0 0 2024-04-25T01:35:56.789 INFO:tasks.workunit.client.0.smithi033.stdout:1/526: dread d2/d5/d1b/d46/d5f/d63/d8f/f96 [0,4194304] 0 2024-04-25T01:35:56.789 INFO:tasks.workunit.client.0.smithi033.stdout:6/759: dread - d4/d1d/d3e/d41/d93/fad zero size 2024-04-25T01:35:56.791 INFO:tasks.workunit.client.0.smithi033.stdout:5/732: mkdir d1c/d2a/d50/d52/da6/d3e/d70/dae/df1 0 2024-04-25T01:35:56.791 INFO:tasks.workunit.client.0.smithi033.stdout:3/729: dread d8/fa [0,4194304] 0 2024-04-25T01:35:56.792 INFO:tasks.workunit.client.0.smithi033.stdout:1/527: write d2/d11/f1a [324454,130071] 0 2024-04-25T01:35:56.793 INFO:tasks.workunit.client.0.smithi033.stdout:6/760: mknod d4/d1d/d3e/d41/d5c/d7f/da4/dcc/c10b 0 2024-04-25T01:35:56.794 INFO:tasks.workunit.client.0.smithi033.stdout:1/528: dread - d2/d11/f6d zero size 2024-04-25T01:35:56.794 INFO:tasks.workunit.client.0.smithi033.stdout:5/733: dread - d1c/d2a/d50/d5b/d86/f7c zero size 2024-04-25T01:35:56.795 INFO:tasks.workunit.client.0.smithi033.stdout:7/671: sync 2024-04-25T01:35:56.795 INFO:tasks.workunit.client.0.smithi033.stdout:3/730: chown d8/dd/d3d/d63/d7c/ded/dbf/dc5/le3 13673 1 2024-04-25T01:35:56.797 INFO:tasks.workunit.client.0.smithi033.stdout:1/529: mknod d2/d2e/ca0 0 2024-04-25T01:35:56.800 INFO:tasks.workunit.client.1.smithi138.stdout:8/293: getdents d3/d16/d1f 0 2024-04-25T01:35:56.801 INFO:tasks.workunit.client.0.smithi033.stdout:3/731: creat d8/dd/d3d/d4f/da4/dc4/ffb x:0 0 0 2024-04-25T01:35:56.805 INFO:tasks.workunit.client.1.smithi138.stdout:2/214: dwrite d0/d32/f3f [0,4194304] 0 2024-04-25T01:35:56.805 INFO:tasks.workunit.client.1.smithi138.stdout:7/138: dwrite da/de/d13/f24 [0,4194304] 0 2024-04-25T01:35:56.806 INFO:tasks.workunit.client.0.smithi033.stdout:7/672: mkdir d4/d2e/d7f/d98/dca/dd3 0 2024-04-25T01:35:56.806 INFO:tasks.workunit.client.0.smithi033.stdout:6/761: dwrite d4/d24/d36/d42/d97/fa6 [0,4194304] 0 2024-04-25T01:35:56.806 INFO:tasks.workunit.client.1.smithi138.stdout:0/179: dwrite d6/f35 [0,4194304] 0 2024-04-25T01:35:56.809 INFO:tasks.workunit.client.0.smithi033.stdout:6/762: fsync d4/d1d/f88 0 2024-04-25T01:35:56.809 INFO:tasks.workunit.client.0.smithi033.stdout:5/734: dwrite d1c/d2a/d50/d5b/d86/f7c [0,4194304] 0 2024-04-25T01:35:56.810 INFO:tasks.workunit.client.1.smithi138.stdout:5/109: dwrite d4/f18 [0,4194304] 0 2024-04-25T01:35:56.810 INFO:tasks.workunit.client.0.smithi033.stdout:7/673: fsync d4/f11 0 2024-04-25T01:35:56.814 INFO:tasks.workunit.client.1.smithi138.stdout:3/132: rename db/df/l10 to db/df/l2b 0 2024-04-25T01:35:56.814 INFO:tasks.workunit.client.0.smithi033.stdout:5/735: fsync d1c/d2a/d3d/fe7 0 2024-04-25T01:35:56.815 INFO:tasks.workunit.client.1.smithi138.stdout:8/294: creat d3/d8/f60 x:0 0 0 2024-04-25T01:35:56.815 INFO:tasks.workunit.client.1.smithi138.stdout:3/133: readlink db/lc 0 2024-04-25T01:35:56.815 INFO:tasks.workunit.client.1.smithi138.stdout:2/215: symlink d0/d32/l42 0 2024-04-25T01:35:56.815 INFO:tasks.workunit.client.1.smithi138.stdout:0/180: mkdir d6/df/d41 0 2024-04-25T01:35:56.820 INFO:tasks.workunit.client.0.smithi033.stdout:6/763: chown d4/d24/d63/ce8 24995 1 2024-04-25T01:35:56.821 INFO:tasks.workunit.client.0.smithi033.stdout:3/732: dwrite d8/dd/d3d/d4f/faf [0,4194304] 0 2024-04-25T01:35:56.826 INFO:tasks.workunit.client.0.smithi033.stdout:3/733: mknod d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/cfc 0 2024-04-25T01:35:56.828 INFO:tasks.workunit.client.0.smithi033.stdout:6/764: unlink d4/d24/d36/d42/d9d/cd4 0 2024-04-25T01:35:56.829 INFO:tasks.workunit.client.0.smithi033.stdout:7/674: link d4/d57/l61 d4/d2e/d6d/dbb/ld4 0 2024-04-25T01:35:56.831 INFO:tasks.workunit.client.0.smithi033.stdout:7/675: truncate d4/f17 1096676 0 2024-04-25T01:35:56.833 INFO:tasks.workunit.client.0.smithi033.stdout:6/765: creat d4/d1d/d3e/d41/f10c x:0 0 0 2024-04-25T01:35:56.834 INFO:tasks.workunit.client.0.smithi033.stdout:7/676: symlink d4/d23/da7/dc2/dd0/ld5 0 2024-04-25T01:35:56.837 INFO:tasks.workunit.client.0.smithi033.stdout:6/766: mknod d4/d24/d36/dbd/dde/c10d 0 2024-04-25T01:35:56.837 INFO:tasks.workunit.client.0.smithi033.stdout:3/734: dwrite d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/f9f [8388608,4194304] 0 2024-04-25T01:35:56.838 INFO:tasks.workunit.client.0.smithi033.stdout:7/677: creat d4/d23/da7/dc2/fd6 x:0 0 0 2024-04-25T01:35:56.838 INFO:tasks.workunit.client.0.smithi033.stdout:3/735: truncate d8/dd/f15 5025260 0 2024-04-25T01:35:56.844 INFO:tasks.workunit.client.0.smithi033.stdout:3/736: fsync d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/f29 0 2024-04-25T01:35:56.846 INFO:tasks.workunit.client.0.smithi033.stdout:7/678: dwrite d4/d23/d3f/d45/d73/dbe/fc1 [0,4194304] 0 2024-04-25T01:35:56.847 INFO:tasks.workunit.client.0.smithi033.stdout:7/679: chown d4/d2e/d7f/d98/dbf 68 1 2024-04-25T01:35:56.848 INFO:tasks.workunit.client.0.smithi033.stdout:7/680: chown d4/d2e/d6b/d67/f7d 159 1 2024-04-25T01:35:56.850 INFO:tasks.workunit.client.0.smithi033.stdout:3/737: rename d8/dd/d3d/d4f/fd5 to d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/ffd 0 2024-04-25T01:35:56.851 INFO:tasks.workunit.client.0.smithi033.stdout:3/738: chown d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/ff4 5 1 2024-04-25T01:35:56.853 INFO:tasks.workunit.client.0.smithi033.stdout:3/739: symlink d8/dd/lfe 0 2024-04-25T01:35:56.854 INFO:tasks.workunit.client.0.smithi033.stdout:3/740: mkdir d8/dff 0 2024-04-25T01:35:56.857 INFO:tasks.workunit.client.0.smithi033.stdout:3/741: unlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/ldc 0 2024-04-25T01:35:56.858 INFO:tasks.workunit.client.0.smithi033.stdout:3/742: stat d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/l3f 0 2024-04-25T01:35:56.859 INFO:tasks.workunit.client.1.smithi138.stdout:2/216: readlink d0/d21/d36/d15/l2f 0 2024-04-25T01:35:56.859 INFO:tasks.workunit.client.1.smithi138.stdout:8/295: chown d3/d8/da/db/d15/f2f 44739 1 2024-04-25T01:35:56.859 INFO:tasks.workunit.client.0.smithi033.stdout:9/617: sync 2024-04-25T01:35:56.859 INFO:tasks.workunit.client.0.smithi033.stdout:2/706: sync 2024-04-25T01:35:56.859 INFO:tasks.workunit.client.0.smithi033.stdout:0/567: sync 2024-04-25T01:35:56.859 INFO:tasks.workunit.client.0.smithi033.stdout:4/719: sync 2024-04-25T01:35:56.859 INFO:tasks.workunit.client.0.smithi033.stdout:8/687: sync 2024-04-25T01:35:56.860 INFO:tasks.workunit.client.1.smithi138.stdout:7/139: dread da/de/d1b/f31 [0,4194304] 0 2024-04-25T01:35:56.862 INFO:tasks.workunit.client.0.smithi033.stdout:3/743: truncate d8/dd/d3d/d4f/f6f 1277304 0 2024-04-25T01:35:56.862 INFO:tasks.workunit.client.1.smithi138.stdout:5/110: creat d4/da/f20 x:0 0 0 2024-04-25T01:35:56.863 INFO:tasks.workunit.client.0.smithi033.stdout:8/688: unlink d6/d45/d52/dae/fb3 0 2024-04-25T01:35:56.863 INFO:tasks.workunit.client.1.smithi138.stdout:3/134: mknod db/df/d19/c2c 0 2024-04-25T01:35:56.865 INFO:tasks.workunit.client.0.smithi033.stdout:9/618: mknod d0/d2/dd/cdd 0 2024-04-25T01:35:56.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/217: truncate d0/d21/d36/d15/f28 1173497 0 2024-04-25T01:35:56.866 INFO:tasks.workunit.client.0.smithi033.stdout:2/707: creat da/d5b/d5e/da8/fe8 x:0 0 0 2024-04-25T01:35:56.869 INFO:tasks.workunit.client.0.smithi033.stdout:6/767: read d4/d2a/f3f [1800027,122349] 0 2024-04-25T01:35:56.869 INFO:tasks.workunit.client.0.smithi033.stdout:8/689: read d6/f1d [1067297,16925] 0 2024-04-25T01:35:56.869 INFO:tasks.workunit.client.1.smithi138.stdout:7/140: dread - da/f18 zero size 2024-04-25T01:35:56.870 INFO:tasks.workunit.client.1.smithi138.stdout:3/135: write db/df/f25 [314533,65748] 0 2024-04-25T01:35:56.870 INFO:tasks.workunit.client.0.smithi033.stdout:6/768: chown d4/d1d/d3e/d41/d5c/dae 1966 1 2024-04-25T01:35:56.871 INFO:tasks.workunit.client.0.smithi033.stdout:3/744: rmdir d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3 39 2024-04-25T01:35:56.875 INFO:tasks.workunit.client.1.smithi138.stdout:8/296: link d3/f3c d3/d16/d1f/f61 0 2024-04-25T01:35:56.876 INFO:tasks.workunit.client.1.smithi138.stdout:0/181: dwrite d6/f35 [4194304,4194304] 0 2024-04-25T01:35:56.876 INFO:tasks.workunit.client.0.smithi033.stdout:8/690: creat d6/dd3/dfb/f102 x:0 0 0 2024-04-25T01:35:56.879 INFO:tasks.workunit.client.1.smithi138.stdout:2/218: creat d0/d32/f43 x:0 0 0 2024-04-25T01:35:56.879 INFO:tasks.workunit.client.0.smithi033.stdout:0/568: dread d7/d2d/d4d/d9c/f83 [0,4194304] 0 2024-04-25T01:35:56.880 INFO:tasks.workunit.client.0.smithi033.stdout:8/691: write d6/d45/d52/d2e/f4a [338037,37875] 0 2024-04-25T01:35:56.881 INFO:tasks.workunit.client.0.smithi033.stdout:3/745: creat d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/daa/f100 x:0 0 0 2024-04-25T01:35:56.882 INFO:tasks.workunit.client.1.smithi138.stdout:8/297: fsync d3/d8/da/db/f11 0 2024-04-25T01:35:56.883 INFO:tasks.workunit.client.0.smithi033.stdout:6/769: dwrite d4/d24/d36/d42/fb1 [0,4194304] 0 2024-04-25T01:35:56.884 INFO:tasks.workunit.client.0.smithi033.stdout:9/619: creat d0/d2/dd/fde x:0 0 0 2024-04-25T01:35:56.885 INFO:tasks.workunit.client.0.smithi033.stdout:4/720: getdents d6/d14/d35/d44/d45/d46/d57/d70/d87/dc7 0 2024-04-25T01:35:56.887 INFO:tasks.workunit.client.1.smithi138.stdout:8/298: write d3/d16/f3b [26313,32070] 0 2024-04-25T01:35:56.888 INFO:tasks.workunit.client.0.smithi033.stdout:0/569: creat d7/da/df/d14/d45/da2/fc8 x:0 0 0 2024-04-25T01:35:56.888 INFO:tasks.workunit.client.0.smithi033.stdout:0/570: dread - d7/d2d/d43/dc0/fc3 zero size 2024-04-25T01:35:56.889 INFO:tasks.workunit.client.1.smithi138.stdout:7/141: dwrite da/f15 [0,4194304] 0 2024-04-25T01:35:56.890 INFO:tasks.workunit.client.1.smithi138.stdout:5/111: dwrite d4/f18 [0,4194304] 0 2024-04-25T01:35:56.891 INFO:tasks.workunit.client.0.smithi033.stdout:0/571: write d7/d2d/d4d/d9c/fb8 [377635,62114] 0 2024-04-25T01:35:56.891 INFO:tasks.workunit.client.1.smithi138.stdout:2/219: mkdir d0/d21/d36/d15/d44 0 2024-04-25T01:35:56.892 INFO:tasks.workunit.client.0.smithi033.stdout:8/692: mknod d6/d45/d51/d55/c103 0 2024-04-25T01:35:56.892 INFO:tasks.workunit.client.0.smithi033.stdout:1/530: dwrite d2/d5/fd [0,4194304] 0 2024-04-25T01:35:56.893 INFO:tasks.workunit.client.1.smithi138.stdout:0/182: mknod d6/d1d/c42 0 2024-04-25T01:35:56.893 INFO:tasks.workunit.client.1.smithi138.stdout:2/220: write d0/d21/d36/d15/f28 [1303479,49442] 0 2024-04-25T01:35:56.895 INFO:tasks.workunit.client.1.smithi138.stdout:8/299: creat d3/d8/da/db/d15/d5d/f62 x:0 0 0 2024-04-25T01:35:56.895 INFO:tasks.workunit.client.1.smithi138.stdout:5/112: symlink d4/da/l21 0 2024-04-25T01:35:56.896 INFO:tasks.workunit.client.1.smithi138.stdout:8/300: stat d3/d8/da/db/f14 0 2024-04-25T01:35:56.897 INFO:tasks.workunit.client.0.smithi033.stdout:3/746: mkdir d8/dd/d3d/d63/d7c/d9d/de8/d101 0 2024-04-25T01:35:56.898 INFO:tasks.workunit.client.0.smithi033.stdout:9/620: creat d0/d11/da1/fdf x:0 0 0 2024-04-25T01:35:56.900 INFO:tasks.workunit.client.0.smithi033.stdout:4/721: symlink d6/d14/d6d/le5 0 2024-04-25T01:35:56.901 INFO:tasks.workunit.client.1.smithi138.stdout:0/183: rename d6/d19/d1c to d6/d39/d43 0 2024-04-25T01:35:56.901 INFO:tasks.workunit.client.1.smithi138.stdout:8/301: rename d3/d8 to d3/d8/da/db/d15/d20/d63 22 2024-04-25T01:35:56.902 INFO:tasks.workunit.client.1.smithi138.stdout:5/113: mkdir d4/da/d1b/d1c/d22 0 2024-04-25T01:35:56.902 INFO:tasks.workunit.client.0.smithi033.stdout:8/693: mknod d6/d45/d52/d2e/d95/c104 0 2024-04-25T01:35:56.904 INFO:tasks.workunit.client.0.smithi033.stdout:0/572: symlink d7/da/df/lc9 0 2024-04-25T01:35:56.905 INFO:tasks.workunit.client.1.smithi138.stdout:2/221: mkdir d0/d21/d45 0 2024-04-25T01:35:56.906 INFO:tasks.workunit.client.0.smithi033.stdout:6/770: rename d4/d1d/d3e/d41/d54/ff7 to d4/d1d/d3e/d41/d54/f10e 0 2024-04-25T01:35:56.908 INFO:tasks.workunit.client.1.smithi138.stdout:0/184: rename d6/df/c2d to d6/d39/c44 0 2024-04-25T01:35:56.908 INFO:tasks.workunit.client.0.smithi033.stdout:6/771: stat d4/l13 0 2024-04-25T01:35:56.909 INFO:tasks.workunit.client.0.smithi033.stdout:9/621: dwrite d0/d2/f48 [0,4194304] 0 2024-04-25T01:35:56.909 INFO:tasks.workunit.client.1.smithi138.stdout:5/114: chown d4/c16 1132088235 1 2024-04-25T01:35:56.911 INFO:tasks.workunit.client.0.smithi033.stdout:3/747: chown d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/cba 237 1 2024-04-25T01:35:56.914 INFO:tasks.workunit.client.0.smithi033.stdout:0/573: creat d7/d2d/d4d/d9c/d61/fca x:0 0 0 2024-04-25T01:35:56.915 INFO:tasks.workunit.client.1.smithi138.stdout:8/302: dwrite d3/d8/da/db/d15/f21 [0,4194304] 0 2024-04-25T01:35:56.916 INFO:tasks.workunit.client.0.smithi033.stdout:4/722: rename d6/d14/cb to d6/d14/d35/d44/d45/d46/d64/ce6 0 2024-04-25T01:35:56.917 INFO:tasks.workunit.client.1.smithi138.stdout:8/303: stat d3/d44/d48/l58 0 2024-04-25T01:35:56.917 INFO:tasks.workunit.client.0.smithi033.stdout:6/772: mknod d4/dfc/c10f 0 2024-04-25T01:35:56.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/304: mknod d3/d16/d1f/d2a/c64 0 2024-04-25T01:35:56.920 INFO:tasks.workunit.client.0.smithi033.stdout:8/694: link d6/d45/d52/dcc/fdc d6/dd3/dfb/f105 0 2024-04-25T01:35:56.922 INFO:tasks.workunit.client.0.smithi033.stdout:9/622: rmdir d0/d2/dd/d20/d4e 39 2024-04-25T01:35:56.922 INFO:tasks.workunit.client.0.smithi033.stdout:8/695: truncate d6/db/d80/d89/f91 4209578 0 2024-04-25T01:35:56.928 INFO:tasks.workunit.client.0.smithi033.stdout:4/723: creat d6/d14/d35/d3a/d4d/d5e/d7c/d94/fe7 x:0 0 0 2024-04-25T01:35:56.928 INFO:tasks.workunit.client.0.smithi033.stdout:8/696: dread d6/db/d80/db4/fd6 [0,4194304] 0 2024-04-25T01:35:56.928 INFO:tasks.workunit.client.0.smithi033.stdout:1/531: link d2/c22 d2/d7f/ca1 0 2024-04-25T01:35:56.928 INFO:tasks.workunit.client.0.smithi033.stdout:0/574: mknod d7/d2d/d4d/d9c/db2/ccb 0 2024-04-25T01:35:56.929 INFO:tasks.workunit.client.0.smithi033.stdout:0/575: fsync d7/d2d/d4d/f5c 0 2024-04-25T01:35:56.930 INFO:tasks.workunit.client.0.smithi033.stdout:6/773: symlink d4/l110 0 2024-04-25T01:35:56.932 INFO:tasks.workunit.client.0.smithi033.stdout:6/774: write d4/d24/d36/d42/fca [25150,62711] 0 2024-04-25T01:35:56.933 INFO:tasks.workunit.client.0.smithi033.stdout:9/623: rename d0/d2/dd/df/d1b/d3d to d0/d2/d43/d4b/dc5/de0 0 2024-04-25T01:35:56.934 INFO:tasks.workunit.client.0.smithi033.stdout:9/624: chown d0/d2/d43/d4b/fd9 16312202 1 2024-04-25T01:35:56.935 INFO:tasks.workunit.client.0.smithi033.stdout:4/724: mkdir d6/d14/d35/d3a/de8 0 2024-04-25T01:35:56.937 INFO:tasks.workunit.client.0.smithi033.stdout:6/775: creat d4/dfc/f111 x:0 0 0 2024-04-25T01:35:56.939 INFO:tasks.workunit.client.0.smithi033.stdout:9/625: truncate d0/d2/d43/d4b/d55/f8d 568265 0 2024-04-25T01:35:56.939 INFO:tasks.workunit.client.0.smithi033.stdout:8/697: rename d6/d45/d51/d5d/lb5 to d6/db/d80/dd1/l106 0 2024-04-25T01:35:56.940 INFO:tasks.workunit.client.0.smithi033.stdout:0/576: dread d7/da/df/f72 [0,4194304] 0 2024-04-25T01:35:56.940 INFO:tasks.workunit.client.0.smithi033.stdout:6/776: write d4/d24/d36/f6c [854013,64681] 0 2024-04-25T01:35:56.940 INFO:tasks.workunit.client.0.smithi033.stdout:1/532: symlink d2/d2e/la2 0 2024-04-25T01:35:56.942 INFO:tasks.workunit.client.0.smithi033.stdout:4/725: chown d6/d14/d35/d44/d45/d46/db0/lbc 9 1 2024-04-25T01:35:56.943 INFO:tasks.workunit.client.0.smithi033.stdout:8/698: write d6/d45/d51/d5d/d6e/f85 [3074085,75087] 0 2024-04-25T01:35:56.944 INFO:tasks.workunit.client.0.smithi033.stdout:0/577: dread - d7/d2d/d4d/d9c/d68/fb9 zero size 2024-04-25T01:35:56.945 INFO:tasks.workunit.client.0.smithi033.stdout:1/533: unlink d2/d5/d1b/f40 0 2024-04-25T01:35:56.946 INFO:tasks.workunit.client.0.smithi033.stdout:9/626: mknod d0/d2/d43/d4b/dc5/de0/d7e/d58/ce1 0 2024-04-25T01:35:56.947 INFO:tasks.workunit.client.0.smithi033.stdout:0/578: write d7/da/fb7 [861289,91367] 0 2024-04-25T01:35:56.948 INFO:tasks.workunit.client.0.smithi033.stdout:1/534: rename d2/d5/d1b to d2/d5/d1b/d46/d5f/d63/d8f/da3 22 2024-04-25T01:35:56.949 INFO:tasks.workunit.client.0.smithi033.stdout:4/726: creat d6/fe9 x:0 0 0 2024-04-25T01:35:56.951 INFO:tasks.workunit.client.0.smithi033.stdout:4/727: chown d6/d14/d35/d3a/d4d/fce 3636 1 2024-04-25T01:35:56.952 INFO:tasks.workunit.client.0.smithi033.stdout:9/627: mkdir d0/d2/dd/dba/dc1/de2 0 2024-04-25T01:35:56.956 INFO:tasks.workunit.client.1.smithi138.stdout:8/305: fdatasync d3/d16/f3b 0 2024-04-25T01:35:56.957 INFO:tasks.workunit.client.0.smithi033.stdout:5/736: write d1c/d2a/d50/d52/da6/d3e/fba [197928,118436] 0 2024-04-25T01:35:56.958 INFO:tasks.workunit.client.1.smithi138.stdout:8/306: write d3/f38 [724914,9700] 0 2024-04-25T01:35:56.959 INFO:tasks.workunit.client.1.smithi138.stdout:8/307: chown d3/d8/da/db/d15 468393165 1 2024-04-25T01:35:56.966 INFO:tasks.workunit.client.0.smithi033.stdout:0/579: dread d7/da/df/d14/d45/f87 [0,4194304] 0 2024-04-25T01:35:56.966 INFO:tasks.workunit.client.1.smithi138.stdout:3/136: dwrite f1 [0,4194304] 0 2024-04-25T01:35:56.968 INFO:tasks.workunit.client.0.smithi033.stdout:0/580: fdatasync d7/da/df/d50/f6b 0 2024-04-25T01:35:56.968 INFO:tasks.workunit.client.1.smithi138.stdout:5/115: rmdir d4/da 39 2024-04-25T01:35:56.970 INFO:tasks.workunit.client.0.smithi033.stdout:0/581: truncate d7/f46 1691098 0 2024-04-25T01:35:56.972 INFO:tasks.workunit.client.1.smithi138.stdout:0/185: write d6/f9 [929281,62349] 0 2024-04-25T01:35:56.973 INFO:tasks.workunit.client.1.smithi138.stdout:2/222: write d0/d21/f27 [59505,97356] 0 2024-04-25T01:35:56.973 INFO:tasks.workunit.client.0.smithi033.stdout:9/628: dread d0/d2/dd/d20/d4e/f90 [0,4194304] 0 2024-04-25T01:35:56.976 INFO:tasks.workunit.client.1.smithi138.stdout:5/116: rename d4/f19 to d4/f23 0 2024-04-25T01:35:56.976 INFO:tasks.workunit.client.1.smithi138.stdout:3/137: dwrite db/f15 [4194304,4194304] 0 2024-04-25T01:35:56.976 INFO:tasks.workunit.client.0.smithi033.stdout:0/582: symlink d7/da/df/d14/lcc 0 2024-04-25T01:35:56.978 INFO:tasks.workunit.client.1.smithi138.stdout:0/186: readlink d6/l2e 0 2024-04-25T01:35:56.978 INFO:tasks.workunit.client.0.smithi033.stdout:9/629: write d0/d2/dd/f52 [2212154,41501] 0 2024-04-25T01:35:56.978 INFO:tasks.workunit.client.0.smithi033.stdout:7/681: dwrite d4/f9 [0,4194304] 0 2024-04-25T01:35:56.979 INFO:tasks.workunit.client.0.smithi033.stdout:1/535: dread d2/d5/d1b/d46/d5f/d63/f65 [0,4194304] 0 2024-04-25T01:35:56.981 INFO:tasks.workunit.client.1.smithi138.stdout:2/223: fsync d0/d32/d35/f40 0 2024-04-25T01:35:56.981 INFO:tasks.workunit.client.0.smithi033.stdout:9/630: readlink d0/d2/d43/d4b/dc5/de0/d42/lc6 0 2024-04-25T01:35:56.984 INFO:tasks.workunit.client.1.smithi138.stdout:3/138: unlink f5 0 2024-04-25T01:35:56.986 INFO:tasks.workunit.client.0.smithi033.stdout:0/583: mkdir d7/d33/dcd 0 2024-04-25T01:35:56.986 INFO:tasks.workunit.client.0.smithi033.stdout:9/631: write d0/d2/dd/d20/d4e/f4f [1855131,38029] 0 2024-04-25T01:35:56.989 INFO:tasks.workunit.client.1.smithi138.stdout:2/224: rename d0/d18/l3e to d0/d32/l46 0 2024-04-25T01:35:56.990 INFO:tasks.workunit.client.0.smithi033.stdout:9/632: chown d0/d2/d43/d4b 56380045 1 2024-04-25T01:35:56.990 INFO:tasks.workunit.client.1.smithi138.stdout:3/139: dread f1 [0,4194304] 0 2024-04-25T01:35:56.990 INFO:tasks.workunit.client.0.smithi033.stdout:7/682: dread d4/d23/d3f/d45/d90/fb1 [0,4194304] 0 2024-04-25T01:35:56.991 INFO:tasks.workunit.client.1.smithi138.stdout:8/308: dread d3/f1e [0,4194304] 0 2024-04-25T01:35:56.992 INFO:tasks.workunit.client.1.smithi138.stdout:3/140: dread - db/d14/f2a zero size 2024-04-25T01:35:56.992 INFO:tasks.workunit.client.0.smithi033.stdout:0/584: readlink d7/d2d/d4d/d9c/l53 0 2024-04-25T01:35:56.992 INFO:tasks.workunit.client.0.smithi033.stdout:2/708: dwrite da/f42 [0,4194304] 0 2024-04-25T01:35:56.995 INFO:tasks.workunit.client.0.smithi033.stdout:0/585: write d7/da/df/f9a [899744,62287] 0 2024-04-25T01:35:56.997 INFO:tasks.workunit.client.1.smithi138.stdout:0/187: link d6/fe d6/f45 0 2024-04-25T01:35:56.999 INFO:tasks.workunit.client.1.smithi138.stdout:2/225: mknod d0/d21/d45/c47 0 2024-04-25T01:35:57.003 INFO:tasks.workunit.client.0.smithi033.stdout:0/586: fsync d7/d2d/d43/dc0/fc3 0 2024-04-25T01:35:57.004 INFO:tasks.workunit.client.0.smithi033.stdout:9/633: getdents d0/d2/d43/d4b/dc5/de0/d7e/ddc 0 2024-04-25T01:35:57.007 INFO:tasks.workunit.client.0.smithi033.stdout:7/683: link d4/d57/fae d4/d2e/d7f/d98/fd7 0 2024-04-25T01:35:57.007 INFO:tasks.workunit.client.1.smithi138.stdout:3/141: dwrite db/df/d17/d20/f29 [0,4194304] 0 2024-04-25T01:35:57.008 INFO:tasks.workunit.client.0.smithi033.stdout:9/634: mknod d0/d2/dd/df/d1b/dc0/ce3 0 2024-04-25T01:35:57.010 INFO:tasks.workunit.client.0.smithi033.stdout:3/748: write d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/f75 [4687217,42490] 0 2024-04-25T01:35:57.011 INFO:tasks.workunit.client.1.smithi138.stdout:8/309: dwrite f0 [0,4194304] 0 2024-04-25T01:35:57.011 INFO:tasks.workunit.client.1.smithi138.stdout:0/188: dwrite d6/f28 [0,4194304] 0 2024-04-25T01:35:57.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/142: unlink db/df/l1d 0 2024-04-25T01:35:57.015 INFO:tasks.workunit.client.1.smithi138.stdout:2/226: link d0/d21/d36/f1a d0/d21/d36/f48 0 2024-04-25T01:35:57.016 INFO:tasks.workunit.client.1.smithi138.stdout:8/310: fsync d3/d8/da/db/d15/f21 0 2024-04-25T01:35:57.017 INFO:tasks.workunit.client.0.smithi033.stdout:3/749: mkdir d8/dd/d11/d102 0 2024-04-25T01:35:57.021 INFO:tasks.workunit.client.0.smithi033.stdout:9/635: read d0/d2/dd/df/d1b/f2e [68432,64724] 0 2024-04-25T01:35:57.022 INFO:tasks.workunit.client.1.smithi138.stdout:3/143: mkdir db/df/d19/d2d 0 2024-04-25T01:35:57.023 INFO:tasks.workunit.client.1.smithi138.stdout:3/144: fsync f6 0 2024-04-25T01:35:57.023 INFO:tasks.workunit.client.1.smithi138.stdout:0/189: rmdir d6 39 2024-04-25T01:35:57.024 INFO:tasks.workunit.client.0.smithi033.stdout:3/750: mknod d8/dd/d3d/d63/d7c/d4d/c103 0 2024-04-25T01:35:57.025 INFO:tasks.workunit.client.1.smithi138.stdout:8/311: symlink d3/d8/da/db/d15/l65 0 2024-04-25T01:35:57.026 INFO:tasks.workunit.client.1.smithi138.stdout:8/312: readlink d3/d44/d48/l58 0 2024-04-25T01:35:57.026 INFO:tasks.workunit.client.0.smithi033.stdout:3/751: chown d8/dd/d11/c19 24593 1 2024-04-25T01:35:57.030 INFO:tasks.workunit.client.0.smithi033.stdout:9/636: unlink d0/d2/dd/df/d1b/fcf 0 2024-04-25T01:35:57.031 INFO:tasks.workunit.client.1.smithi138.stdout:2/227: dread d0/f31 [0,4194304] 0 2024-04-25T01:35:57.032 INFO:tasks.workunit.client.0.smithi033.stdout:9/637: stat d0/d2/d43/d4b/dc5/de0/d6e/ca3 0 2024-04-25T01:35:57.033 INFO:tasks.workunit.client.1.smithi138.stdout:8/313: dread d3/d8/da/db/d15/f21 [0,4194304] 0 2024-04-25T01:35:57.033 INFO:tasks.workunit.client.0.smithi033.stdout:3/752: mknod d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/c104 0 2024-04-25T01:35:57.034 INFO:tasks.workunit.client.1.smithi138.stdout:2/228: creat d0/d18/f49 x:0 0 0 2024-04-25T01:35:57.034 INFO:tasks.workunit.client.0.smithi033.stdout:8/699: write d6/d45/d52/f3f [4367595,12082] 0 2024-04-25T01:35:57.034 INFO:tasks.workunit.client.0.smithi033.stdout:2/709: dread da/d5b/f79 [0,4194304] 0 2024-04-25T01:35:57.035 INFO:tasks.workunit.client.0.smithi033.stdout:6/777: truncate d4/d15/f94 8406220 0 2024-04-25T01:35:57.035 INFO:tasks.workunit.client.0.smithi033.stdout:3/753: chown d8/dd/d1d/f2b 3153 1 2024-04-25T01:35:57.036 INFO:tasks.workunit.client.0.smithi033.stdout:8/700: fdatasync d6/d45/d52/db0/dbc/fe5 0 2024-04-25T01:35:57.036 INFO:tasks.workunit.client.1.smithi138.stdout:8/314: mknod d3/d8/da/db/d15/d59/c66 0 2024-04-25T01:35:57.037 INFO:tasks.workunit.client.1.smithi138.stdout:8/315: write f0 [4857041,7718] 0 2024-04-25T01:35:57.038 INFO:tasks.workunit.client.0.smithi033.stdout:9/638: rename d0/d2/dd/f13 to d0/d2/d43/d4b/dc5/de0/d7e/fe4 0 2024-04-25T01:35:57.038 INFO:tasks.workunit.client.0.smithi033.stdout:8/701: dread - d6/db/fe0 zero size 2024-04-25T01:35:57.039 INFO:tasks.workunit.client.0.smithi033.stdout:4/728: write d6/d12/f1a [346904,64024] 0 2024-04-25T01:35:57.039 INFO:tasks.workunit.client.1.smithi138.stdout:2/229: getdents d0 0 2024-04-25T01:35:57.040 INFO:tasks.workunit.client.1.smithi138.stdout:8/316: write d3/d8/da/db/d15/d20/f26 [5155284,37397] 0 2024-04-25T01:35:57.041 INFO:tasks.workunit.client.0.smithi033.stdout:9/639: write d0/d2/dd/da7/fb5 [1112075,98805] 0 2024-04-25T01:35:57.041 INFO:tasks.workunit.client.1.smithi138.stdout:8/317: read d3/f5 [1230317,2710] 0 2024-04-25T01:35:57.042 INFO:tasks.workunit.client.0.smithi033.stdout:8/702: chown d6/d45/d51/d5d/f81 1400 1 2024-04-25T01:35:57.043 INFO:tasks.workunit.client.0.smithi033.stdout:4/729: fdatasync d6/d14/d35/d3a/d4d/fce 0 2024-04-25T01:35:57.046 INFO:tasks.workunit.client.1.smithi138.stdout:8/318: getdents d3/d44 0 2024-04-25T01:35:57.048 INFO:tasks.workunit.client.1.smithi138.stdout:8/319: chown d3/d8/da/db/d15/d20 949828082 1 2024-04-25T01:35:57.049 INFO:tasks.workunit.client.0.smithi033.stdout:5/737: dwrite f15 [0,4194304] 0 2024-04-25T01:35:57.049 INFO:tasks.workunit.client.0.smithi033.stdout:7/684: dread d4/d2e/d65/f68 [0,4194304] 0 2024-04-25T01:35:57.050 INFO:tasks.workunit.client.1.smithi138.stdout:2/230: dwrite d0/d18/f1b [0,4194304] 0 2024-04-25T01:35:57.050 INFO:tasks.workunit.client.0.smithi033.stdout:9/640: symlink d0/d2/d43/d4b/dc5/de0/d42/le5 0 2024-04-25T01:35:57.051 INFO:tasks.workunit.client.1.smithi138.stdout:8/320: read - d3/d8/f60 zero size 2024-04-25T01:35:57.052 INFO:tasks.workunit.client.0.smithi033.stdout:4/730: creat d6/d14/d35/d44/fea x:0 0 0 2024-04-25T01:35:57.052 INFO:tasks.workunit.client.0.smithi033.stdout:9/641: fdatasync d0/d2/dd/fe 0 2024-04-25T01:35:57.054 INFO:tasks.workunit.client.1.smithi138.stdout:8/321: chown d3/d16/d1f/c23 0 1 2024-04-25T01:35:57.054 INFO:tasks.workunit.client.0.smithi033.stdout:2/710: rename da/d27/d31/d40/d8a/d80/f8f to da/d27/d31/fe9 0 2024-04-25T01:35:57.054 INFO:tasks.workunit.client.1.smithi138.stdout:0/190: dwrite d6/d2f/f34 [0,4194304] 0 2024-04-25T01:35:57.056 INFO:tasks.workunit.client.0.smithi033.stdout:7/685: chown d4/d2e/f48 216 1 2024-04-25T01:35:57.057 INFO:tasks.workunit.client.1.smithi138.stdout:2/231: creat d0/d21/d36/d15/d1f/f4a x:0 0 0 2024-04-25T01:35:57.058 INFO:tasks.workunit.client.1.smithi138.stdout:8/322: write d3/d16/f1c [821310,115761] 0 2024-04-25T01:35:57.059 INFO:tasks.workunit.client.0.smithi033.stdout:7/686: write d4/d23/d3f/d45/d6c/f92 [411969,36466] 0 2024-04-25T01:35:57.060 INFO:tasks.workunit.client.1.smithi138.stdout:8/323: fdatasync d3/d44/f46 0 2024-04-25T01:35:57.061 INFO:tasks.workunit.client.1.smithi138.stdout:8/324: readlink d3/d8/da/l5b 0 2024-04-25T01:35:57.062 INFO:tasks.workunit.client.0.smithi033.stdout:5/738: symlink d1c/d2a/d50/db1/lf2 0 2024-04-25T01:35:57.063 INFO:tasks.workunit.client.0.smithi033.stdout:9/642: mknod d0/d2/dd/d5f/ce6 0 2024-04-25T01:35:57.063 INFO:tasks.workunit.client.1.smithi138.stdout:3/145: dread db/fe [0,4194304] 0 2024-04-25T01:35:57.063 INFO:tasks.workunit.client.0.smithi033.stdout:7/687: chown d4/d2e/d50/f6e 39575824 1 2024-04-25T01:35:57.064 INFO:tasks.workunit.client.0.smithi033.stdout:4/731: dwrite d6/d14/d35/d44/fea [0,4194304] 0 2024-04-25T01:35:57.067 INFO:tasks.workunit.client.0.smithi033.stdout:6/778: rename d4/d1d/d3e/d41/d54/lbe to d4/d1d/d3e/d41/d5c/df6/l112 0 2024-04-25T01:35:57.067 INFO:tasks.workunit.client.0.smithi033.stdout:7/688: chown d4/d23/fc5 29 1 2024-04-25T01:35:57.068 INFO:tasks.workunit.client.0.smithi033.stdout:4/732: write d6/d14/d35/d3a/d4d/fce [13346,93431] 0 2024-04-25T01:35:57.068 INFO:tasks.workunit.client.0.smithi033.stdout:2/711: fsync da/d8d/db0/fd0 0 2024-04-25T01:35:57.069 INFO:tasks.workunit.client.1.smithi138.stdout:8/325: write d3/d8/da/db/d15/f30 [2908260,104228] 0 2024-04-25T01:35:57.071 INFO:tasks.workunit.client.0.smithi033.stdout:4/733: fdatasync d6/d14/d35/d3a/f56 0 2024-04-25T01:35:57.072 INFO:tasks.workunit.client.1.smithi138.stdout:2/232: link d0/f31 d0/d32/f4b 0 2024-04-25T01:35:57.073 INFO:tasks.workunit.client.1.smithi138.stdout:3/146: link db/fe db/df/d17/f2e 0 2024-04-25T01:35:57.074 INFO:tasks.workunit.client.0.smithi033.stdout:5/739: mkdir d1c/d2a/d50/d52/da6/d3e/d70/d9b/df3 0 2024-04-25T01:35:57.076 INFO:tasks.workunit.client.1.smithi138.stdout:2/233: chown d0/d21/d36/d15/d1f/c2b 1 1 2024-04-25T01:35:57.077 INFO:tasks.workunit.client.0.smithi033.stdout:4/734: stat d6/d14/d35/d44/d45/d46/d57/d70/d87/dc7/lad 0 2024-04-25T01:35:57.077 INFO:tasks.workunit.client.0.smithi033.stdout:7/689: creat d4/d2e/d6b/d67/fd8 x:0 0 0 2024-04-25T01:35:57.079 INFO:tasks.workunit.client.1.smithi138.stdout:8/326: dwrite d3/d8/da/f43 [0,4194304] 0 2024-04-25T01:35:57.080 INFO:tasks.workunit.client.0.smithi033.stdout:5/740: chown d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e/l94 365761 1 2024-04-25T01:35:57.080 INFO:tasks.workunit.client.1.smithi138.stdout:8/327: stat d3/d16 0 2024-04-25T01:35:57.081 INFO:tasks.workunit.client.1.smithi138.stdout:8/328: write d3/f3a [295565,89949] 0 2024-04-25T01:35:57.082 INFO:tasks.workunit.client.0.smithi033.stdout:2/712: dwrite da/d2c/f33 [0,4194304] 0 2024-04-25T01:35:57.084 INFO:tasks.workunit.client.1.smithi138.stdout:3/147: dread db/f15 [0,4194304] 0 2024-04-25T01:35:57.086 INFO:tasks.workunit.client.0.smithi033.stdout:9/643: rename d0/d2/d43/d4b/d55/fb1 to d0/d2/d43/d4b/dc5/de0/d8f/fe7 0 2024-04-25T01:35:57.086 INFO:tasks.workunit.client.1.smithi138.stdout:3/148: write db/df/d17/f1c [3131694,36736] 0 2024-04-25T01:35:57.088 INFO:tasks.workunit.client.1.smithi138.stdout:3/149: creat db/d14/f2f x:0 0 0 2024-04-25T01:35:57.089 INFO:tasks.workunit.client.1.smithi138.stdout:8/329: read d3/d8/f53 [1602878,80323] 0 2024-04-25T01:35:57.089 INFO:tasks.workunit.client.0.smithi033.stdout:4/735: readlink d6/d14/d35/l66 0 2024-04-25T01:35:57.089 INFO:tasks.workunit.client.0.smithi033.stdout:7/690: creat d4/d2e/d7f/d98/fd9 x:0 0 0 2024-04-25T01:35:57.092 INFO:tasks.workunit.client.0.smithi033.stdout:2/713: symlink da/d5b/d5e/da8/lea 0 2024-04-25T01:35:57.093 INFO:tasks.workunit.client.1.smithi138.stdout:8/330: mkdir d3/d16/d1f/d50/d67 0 2024-04-25T01:35:57.093 INFO:tasks.workunit.client.1.smithi138.stdout:8/331: fdatasync d3/d8/f53 0 2024-04-25T01:35:57.097 INFO:tasks.workunit.client.1.smithi138.stdout:8/332: rename d3/d16/d1f/d2a/d4f/l52 to d3/d8/da/db/d15/d20/l68 0 2024-04-25T01:35:57.097 INFO:tasks.workunit.client.1.smithi138.stdout:8/333: chown d3/d16/d1f/d2a/c41 61 1 2024-04-25T01:35:57.098 INFO:tasks.workunit.client.0.smithi033.stdout:2/714: fdatasync da/f8e 0 2024-04-25T01:35:57.101 INFO:tasks.workunit.client.0.smithi033.stdout:4/736: rename d6/d12/f40 to d6/d12/feb 0 2024-04-25T01:35:57.101 INFO:tasks.workunit.client.0.smithi033.stdout:9/644: truncate d0/d2/dd/d20/d4e/f76 707042 0 2024-04-25T01:35:57.103 INFO:tasks.workunit.client.0.smithi033.stdout:8/703: dwrite d6/d45/f76 [0,4194304] 0 2024-04-25T01:35:57.103 INFO:tasks.workunit.client.0.smithi033.stdout:2/715: write da/d5b/d5e/da8/fe8 [645872,42392] 0 2024-04-25T01:35:57.103 INFO:tasks.workunit.client.1.smithi138.stdout:8/334: dwrite d3/f3a [0,4194304] 0 2024-04-25T01:35:57.104 INFO:tasks.workunit.client.0.smithi033.stdout:6/779: dread d4/d24/d36/d42/d9d/fdd [0,4194304] 0 2024-04-25T01:35:57.106 INFO:tasks.workunit.client.0.smithi033.stdout:9/645: stat d0/d2/d43/d4b/dc5/de0/d7e/cb2 0 2024-04-25T01:35:57.107 INFO:tasks.workunit.client.0.smithi033.stdout:2/716: write da/d5b/d72/fbe [469173,5323] 0 2024-04-25T01:35:57.107 INFO:tasks.workunit.client.0.smithi033.stdout:7/691: dwrite d4/d2e/d6b/d67/fd8 [0,4194304] 0 2024-04-25T01:35:57.108 INFO:tasks.workunit.client.1.smithi138.stdout:8/335: unlink d3/f19 0 2024-04-25T01:35:57.109 INFO:tasks.workunit.client.0.smithi033.stdout:8/704: creat d6/dd3/dfb/f107 x:0 0 0 2024-04-25T01:35:57.109 INFO:tasks.workunit.client.0.smithi033.stdout:6/780: dread - d4/d1d/d3e/d41/d93/fab zero size 2024-04-25T01:35:57.110 INFO:tasks.workunit.client.0.smithi033.stdout:6/781: chown d4/dfc/f101 1 1 2024-04-25T01:35:57.111 INFO:tasks.workunit.client.0.smithi033.stdout:4/737: creat d6/d14/d35/d44/d45/d46/d64/fec x:0 0 0 2024-04-25T01:35:57.112 INFO:tasks.workunit.client.0.smithi033.stdout:9/646: dread - d0/d2/d43/d4b/dc5/de0/d42/fae zero size 2024-04-25T01:35:57.113 INFO:tasks.workunit.client.0.smithi033.stdout:4/738: readlink d6/l3c 0 2024-04-25T01:35:57.113 INFO:tasks.workunit.client.1.smithi138.stdout:8/336: creat d3/d8/da/db/d15/d56/f69 x:0 0 0 2024-04-25T01:35:57.115 INFO:tasks.workunit.client.1.smithi138.stdout:0/191: write d6/d1d/f24 [470961,2631] 0 2024-04-25T01:35:57.116 INFO:tasks.workunit.client.0.smithi033.stdout:4/739: dread - d6/d14/d1d/d5f/fb5 zero size 2024-04-25T01:35:57.119 INFO:tasks.workunit.client.0.smithi033.stdout:6/782: creat d4/d24/d36/dbd/dde/f113 x:0 0 0 2024-04-25T01:35:57.119 INFO:tasks.workunit.client.1.smithi138.stdout:8/337: read d3/d8/da/db/f14 [2134264,76790] 0 2024-04-25T01:35:57.121 INFO:tasks.workunit.client.0.smithi033.stdout:8/705: getdents d6/d45/d52/df2/dfa 0 2024-04-25T01:35:57.121 INFO:tasks.workunit.client.0.smithi033.stdout:2/717: dwrite da/fe [0,4194304] 0 2024-04-25T01:35:57.122 INFO:tasks.workunit.client.1.smithi138.stdout:0/192: link d6/fc d6/d39/d43/f46 0 2024-04-25T01:35:57.122 INFO:tasks.workunit.client.0.smithi033.stdout:6/783: write d4/d24/d36/dbd/fd9 [455941,62208] 0 2024-04-25T01:35:57.124 INFO:tasks.workunit.client.1.smithi138.stdout:8/338: creat d3/d16/d1f/d2a/f6a x:0 0 0 2024-04-25T01:35:57.124 INFO:tasks.workunit.client.0.smithi033.stdout:6/784: readlink d4/d1d/d3e/l8e 0 2024-04-25T01:35:57.125 INFO:tasks.workunit.client.1.smithi138.stdout:3/150: truncate db/df/d17/d20/f29 2136235 0 2024-04-25T01:35:57.125 INFO:tasks.workunit.client.1.smithi138.stdout:8/339: chown d3/d8/f53 18663052 1 2024-04-25T01:35:57.125 INFO:tasks.workunit.client.0.smithi033.stdout:9/647: creat d0/d2/d43/d4b/dc5/de0/d7e/fe8 x:0 0 0 2024-04-25T01:35:57.127 INFO:tasks.workunit.client.1.smithi138.stdout:0/193: unlink d6/f35 0 2024-04-25T01:35:57.130 INFO:tasks.workunit.client.1.smithi138.stdout:0/194: rmdir d6/d1d 39 2024-04-25T01:35:57.130 INFO:tasks.workunit.client.0.smithi033.stdout:8/706: mkdir d6/d45/d108 0 2024-04-25T01:35:57.132 INFO:tasks.workunit.client.0.smithi033.stdout:8/707: fdatasync d6/db/fe0 0 2024-04-25T01:35:57.132 INFO:tasks.workunit.client.1.smithi138.stdout:3/151: link db/d14/f2f db/df/d17/d23/f30 0 2024-04-25T01:35:57.135 INFO:tasks.workunit.client.1.smithi138.stdout:8/340: truncate d3/d8/f40 3189360 0 2024-04-25T01:35:57.136 INFO:tasks.workunit.client.0.smithi033.stdout:9/648: creat d0/d2/d43/d4b/fe9 x:0 0 0 2024-04-25T01:35:57.137 INFO:tasks.workunit.client.0.smithi033.stdout:8/708: symlink d6/db/d80/d89/dbf/l109 0 2024-04-25T01:35:57.138 INFO:tasks.workunit.client.1.smithi138.stdout:3/152: write db/d14/f2f [377148,100033] 0 2024-04-25T01:35:57.139 INFO:tasks.workunit.client.1.smithi138.stdout:0/195: rename d6/f13 to d6/d19/f47 0 2024-04-25T01:35:57.141 INFO:tasks.workunit.client.1.smithi138.stdout:0/196: write d6/f28 [203515,70636] 0 2024-04-25T01:35:57.141 INFO:tasks.workunit.client.1.smithi138.stdout:3/153: chown db/df/d17/d23/f30 9349 1 2024-04-25T01:35:57.141 INFO:tasks.workunit.client.0.smithi033.stdout:9/649: mkdir d0/d2/d43/d4b/dc5/de0/d42/dea 0 2024-04-25T01:35:57.143 INFO:tasks.workunit.client.0.smithi033.stdout:8/709: rename d6/d45/d52/l14 to d6/d45/d51/d55/d84/l10a 0 2024-04-25T01:35:57.144 INFO:tasks.workunit.client.1.smithi138.stdout:8/341: getdents d3/d16 0 2024-04-25T01:35:57.146 INFO:tasks.workunit.client.0.smithi033.stdout:6/785: dread d4/d2a/f39 [0,4194304] 0 2024-04-25T01:35:57.147 INFO:tasks.workunit.client.0.smithi033.stdout:9/650: truncate d0/d2/d43/d4b/f63 675950 0 2024-04-25T01:35:57.149 INFO:tasks.workunit.client.1.smithi138.stdout:8/342: rename d3/d8/da/db/d15/f34 to d3/d8/da/db/d15/d5d/f6b 0 2024-04-25T01:35:57.150 INFO:tasks.workunit.client.0.smithi033.stdout:2/718: dread da/d27/d31/d40/db1/fcc [0,4194304] 0 2024-04-25T01:35:57.150 INFO:tasks.workunit.client.1.smithi138.stdout:8/343: write f0 [3216809,82268] 0 2024-04-25T01:35:57.153 INFO:tasks.workunit.client.1.smithi138.stdout:0/197: getdents d6/d39/d43 0 2024-04-25T01:35:57.154 INFO:tasks.workunit.client.0.smithi033.stdout:9/651: rename d0/c30 to d0/d2/ceb 0 2024-04-25T01:35:57.155 INFO:tasks.workunit.client.1.smithi138.stdout:8/344: rename d3/d8/da/db/d15/l65 to d3/d8/da/db/d15/d20/l6c 0 2024-04-25T01:35:57.155 INFO:tasks.workunit.client.0.smithi033.stdout:6/786: creat d4/d24/d36/d42/d97/dd6/f114 x:0 0 0 2024-04-25T01:35:57.156 INFO:tasks.workunit.client.0.smithi033.stdout:2/719: unlink da/d8d/db0/cdd 0 2024-04-25T01:35:57.157 INFO:tasks.workunit.client.0.smithi033.stdout:2/720: chown da/d50/d6a/d43/d6c 499 1 2024-04-25T01:35:57.160 INFO:tasks.workunit.client.0.smithi033.stdout:6/787: creat d4/d24/d36/d42/f115 x:0 0 0 2024-04-25T01:35:57.161 INFO:tasks.workunit.client.0.smithi033.stdout:6/788: fdatasync d4/d24/ff5 0 2024-04-25T01:35:57.162 INFO:tasks.workunit.client.1.smithi138.stdout:8/345: getdents d3/d16/d1f 0 2024-04-25T01:35:57.164 INFO:tasks.workunit.client.0.smithi033.stdout:7/692: truncate d4/d23/d3f/d45/f66 3055157 0 2024-04-25T01:35:57.167 INFO:tasks.workunit.client.1.smithi138.stdout:8/346: rename d3/d16/d1f/d31/f3d to d3/f6d 0 2024-04-25T01:35:57.167 INFO:tasks.workunit.client.0.smithi033.stdout:6/789: rename d4/d24/d29 to d4/d1d/d3e/d41/d5c/d7f/d116 0 2024-04-25T01:35:57.167 INFO:tasks.workunit.client.1.smithi138.stdout:8/347: stat d3/d8/da/f24 0 2024-04-25T01:35:57.168 INFO:tasks.workunit.client.1.smithi138.stdout:8/348: stat d3/d16/f51 0 2024-04-25T01:35:57.169 INFO:tasks.workunit.client.0.smithi033.stdout:4/740: dwrite d6/d12/d38/f5d [0,4194304] 0 2024-04-25T01:35:57.170 INFO:tasks.workunit.client.0.smithi033.stdout:6/790: fsync d4/d1d/d3e/d41/d5c/d7f/d116/f33 0 2024-04-25T01:35:57.172 INFO:tasks.workunit.client.1.smithi138.stdout:0/198: write d6/df/f18 [2380589,70657] 0 2024-04-25T01:35:57.175 INFO:tasks.workunit.client.0.smithi033.stdout:4/741: creat d6/d14/d1d/d5f/fed x:0 0 0 2024-04-25T01:35:57.176 INFO:tasks.workunit.client.0.smithi033.stdout:4/742: dread - d6/d12/d84/fbb zero size 2024-04-25T01:35:57.177 INFO:tasks.workunit.client.1.smithi138.stdout:0/199: rename d6/f29 to d6/d19/f48 0 2024-04-25T01:35:57.177 INFO:tasks.workunit.client.1.smithi138.stdout:3/154: dwrite db/df/d17/d20/f29 [0,4194304] 0 2024-04-25T01:35:57.179 INFO:tasks.workunit.client.1.smithi138.stdout:3/155: truncate db/df/d17/d20/f29 4585355 0 2024-04-25T01:35:57.179 INFO:tasks.workunit.client.0.smithi033.stdout:4/743: mknod d6/d14/d1d/cee 0 2024-04-25T01:35:57.181 INFO:tasks.workunit.client.0.smithi033.stdout:4/744: fdatasync d6/f8 0 2024-04-25T01:35:57.181 INFO:tasks.workunit.client.0.smithi033.stdout:8/710: dwrite d6/db/f18 [0,4194304] 0 2024-04-25T01:35:57.182 INFO:tasks.workunit.client.1.smithi138.stdout:3/156: dread db/f15 [4194304,4194304] 0 2024-04-25T01:35:57.185 INFO:tasks.workunit.client.0.smithi033.stdout:4/745: fsync d6/d14/d35/d3a/d4d/fce 0 2024-04-25T01:35:57.186 INFO:tasks.workunit.client.0.smithi033.stdout:9/652: creat d0/d2/d43/d4b/dc5/de0/fec x:0 0 0 2024-04-25T01:35:57.187 INFO:tasks.workunit.client.0.smithi033.stdout:4/746: chown d6/d14/d35/d44/d45/d46/fbd 309776914 1 2024-04-25T01:35:57.187 INFO:tasks.workunit.client.0.smithi033.stdout:9/653: chown d0/d2/dd/f51 155978448 1 2024-04-25T01:35:57.188 INFO:tasks.workunit.client.0.smithi033.stdout:4/747: chown c4 1785 1 2024-04-25T01:35:57.189 INFO:tasks.workunit.client.1.smithi138.stdout:3/157: dwrite f1 [0,4194304] 0 2024-04-25T01:35:57.190 INFO:tasks.workunit.client.1.smithi138.stdout:3/158: write db/df/f25 [344286,26363] 0 2024-04-25T01:35:57.193 INFO:tasks.workunit.client.0.smithi033.stdout:4/748: mkdir d6/d14/d35/d3a/d4d/d5e/def 0 2024-04-25T01:35:57.195 INFO:tasks.workunit.client.0.smithi033.stdout:9/654: rename d0/d2/d43/d4b/dc5/de0/d8f/fe7 to d0/d2/dd/df/d1b/dc0/fed 0 2024-04-25T01:35:57.195 INFO:tasks.workunit.client.1.smithi138.stdout:3/159: dread db/f15 [4194304,4194304] 0 2024-04-25T01:35:57.196 INFO:tasks.workunit.client.0.smithi033.stdout:6/791: stat d4/d24/lac 0 2024-04-25T01:35:57.196 INFO:tasks.workunit.client.1.smithi138.stdout:3/160: write db/df/d17/d20/f29 [4535367,14057] 0 2024-04-25T01:35:57.197 INFO:tasks.workunit.client.1.smithi138.stdout:3/161: write db/d14/f2f [1180995,13985] 0 2024-04-25T01:35:57.197 INFO:tasks.workunit.client.0.smithi033.stdout:6/792: chown d4/d1d/d3e/d41/d5c/d7f/da4/l107 8682011 1 2024-04-25T01:35:57.198 INFO:tasks.workunit.client.0.smithi033.stdout:6/793: write d4/d2a/f34 [3061655,66133] 0 2024-04-25T01:35:57.198 INFO:tasks.workunit.client.1.smithi138.stdout:3/162: chown db/df/d17/d20/c24 36204 1 2024-04-25T01:35:57.199 INFO:tasks.workunit.client.0.smithi033.stdout:6/794: fsync d4/d2a/fa9 0 2024-04-25T01:35:57.204 INFO:tasks.workunit.client.1.smithi138.stdout:3/163: dwrite db/f15 [0,4194304] 0 2024-04-25T01:35:57.206 INFO:tasks.workunit.client.1.smithi138.stdout:3/164: rmdir db 39 2024-04-25T01:35:57.227 INFO:tasks.workunit.client.0.smithi033.stdout:9/655: mkdir d0/d2/d43/d73/dab/dad/dee 0 2024-04-25T01:35:57.227 INFO:tasks.workunit.client.0.smithi033.stdout:8/711: write d6/db/f26 [4751661,1172] 0 2024-04-25T01:35:57.234 INFO:tasks.workunit.client.0.smithi033.stdout:9/656: symlink d0/d2/d43/d73/dab/lef 0 2024-04-25T01:35:57.236 INFO:tasks.workunit.client.0.smithi033.stdout:6/795: mkdir d4/d24/d36/dbd/d117 0 2024-04-25T01:35:57.239 INFO:tasks.workunit.client.0.smithi033.stdout:6/796: truncate d4/d1d/d3e/d41/d9f/fe7 140878 0 2024-04-25T01:35:57.241 INFO:tasks.workunit.client.0.smithi033.stdout:9/657: mkdir d0/d2/d43/d73/dc4/dcb/df0 0 2024-04-25T01:35:57.245 INFO:tasks.workunit.client.0.smithi033.stdout:4/749: dwrite d6/d14/f2b [0,4194304] 0 2024-04-25T01:35:57.246 INFO:tasks.workunit.client.0.smithi033.stdout:9/658: truncate d0/d2/d43/d4b/dc5/de0/d7e/f66 1469593 0 2024-04-25T01:35:57.248 INFO:tasks.workunit.client.0.smithi033.stdout:4/750: mknod d6/d14/d1d/cf0 0 2024-04-25T01:35:57.248 INFO:tasks.workunit.client.0.smithi033.stdout:4/751: readlink d6/d14/d35/l66 0 2024-04-25T01:35:57.249 INFO:tasks.workunit.client.0.smithi033.stdout:9/659: truncate d0/d2/d43/d4b/dc5/de0/d7e/d58/fa6 4503739 0 2024-04-25T01:35:57.249 INFO:tasks.workunit.client.0.smithi033.stdout:6/797: dread d4/d1d/d3e/faa [0,4194304] 0 2024-04-25T01:35:57.255 INFO:tasks.workunit.client.1.smithi138.stdout:1/114: sync 2024-04-25T01:35:57.257 INFO:tasks.workunit.client.1.smithi138.stdout:1/115: fsync d3/db/f14 0 2024-04-25T01:35:57.258 INFO:tasks.workunit.client.0.smithi033.stdout:8/712: write d6/db/d80/fb7 [56529,20121] 0 2024-04-25T01:35:57.258 INFO:tasks.workunit.client.0.smithi033.stdout:4/752: dwrite d6/d14/d35/d3a/f49 [0,4194304] 0 2024-04-25T01:35:57.259 INFO:tasks.workunit.client.1.smithi138.stdout:1/116: mkdir d3/db/d2b 0 2024-04-25T01:35:57.260 INFO:tasks.workunit.client.0.smithi033.stdout:6/798: dwrite d4/d1d/d3e/d41/d5c/d7f/f81 [0,4194304] 0 2024-04-25T01:35:57.260 INFO:tasks.workunit.client.0.smithi033.stdout:4/753: chown d6/d14/lcb 16453 1 2024-04-25T01:35:57.261 INFO:tasks.workunit.client.1.smithi138.stdout:1/117: truncate d3/d27/d16/f2a 757500 0 2024-04-25T01:35:57.263 INFO:tasks.workunit.client.0.smithi033.stdout:8/713: mkdir d6/d45/d51/d5d/d9d/d10b 0 2024-04-25T01:35:57.263 INFO:tasks.workunit.client.0.smithi033.stdout:4/754: write d6/d14/d1d/f63 [4758364,75312] 0 2024-04-25T01:35:57.266 INFO:tasks.workunit.client.1.smithi138.stdout:1/118: rename d3/d27/d16/f2a to d3/f2c 0 2024-04-25T01:35:57.266 INFO:tasks.workunit.client.0.smithi033.stdout:6/799: unlink d4/d1d/d3e/d41/d5c/d7f/d116/l76 0 2024-04-25T01:35:57.268 INFO:tasks.workunit.client.0.smithi033.stdout:6/800: write d4/d1d/d3e/d41/d93/f104 [1839262,84104] 0 2024-04-25T01:35:57.269 INFO:tasks.workunit.client.0.smithi033.stdout:8/714: rename d6/db/l88 to d6/d45/d51/d5d/d6e/l10c 0 2024-04-25T01:35:57.270 INFO:tasks.workunit.client.0.smithi033.stdout:6/801: chown d4/d24/d36/d42/d97/ldf 54688 1 2024-04-25T01:35:57.271 INFO:tasks.workunit.client.0.smithi033.stdout:6/802: write d4/dfc/f101 [356511,105616] 0 2024-04-25T01:35:57.274 INFO:tasks.workunit.client.0.smithi033.stdout:4/755: rename d6/d14/d35/d44/d45/fe3 to d6/d14/d35/d3a/d4d/d5e/d7c/d94/ff1 0 2024-04-25T01:35:57.274 INFO:tasks.workunit.client.1.smithi138.stdout:1/119: rmdir d3/db 39 2024-04-25T01:35:57.275 INFO:tasks.workunit.client.0.smithi033.stdout:8/715: mkdir d6/db/d80/dd1/d10d 0 2024-04-25T01:35:57.275 INFO:tasks.workunit.client.1.smithi138.stdout:1/120: rmdir d3/d27/d16 39 2024-04-25T01:35:57.277 INFO:tasks.workunit.client.0.smithi033.stdout:4/756: creat d6/d14/d35/d3a/d4d/d5e/ff2 x:0 0 0 2024-04-25T01:35:57.277 INFO:tasks.workunit.client.1.smithi138.stdout:1/121: mkdir d3/d27/d2d 0 2024-04-25T01:35:57.279 INFO:tasks.workunit.client.1.smithi138.stdout:1/122: dread - d3/d27/d16/f28 zero size 2024-04-25T01:35:57.280 INFO:tasks.workunit.client.0.smithi033.stdout:4/757: rmdir d6/d12/d38/dc0 39 2024-04-25T01:35:57.282 INFO:tasks.workunit.client.0.smithi033.stdout:8/716: getdents d6/db/d80/d89/dbf 0 2024-04-25T01:35:57.285 INFO:tasks.workunit.client.0.smithi033.stdout:4/758: link d6/d12/f26 d6/d14/d35/d3a/de8/ff3 0 2024-04-25T01:35:57.288 INFO:tasks.workunit.client.0.smithi033.stdout:4/759: symlink d6/lf4 0 2024-04-25T01:35:57.288 INFO:tasks.workunit.client.0.smithi033.stdout:4/760: chown d6/d14/d1d/d5f/fa7 114 1 2024-04-25T01:35:57.289 INFO:tasks.workunit.client.0.smithi033.stdout:4/761: readlink d6/d14/lb2 0 2024-04-25T01:35:57.289 INFO:tasks.workunit.client.0.smithi033.stdout:4/762: dread - d6/d14/d6d/fc1 zero size 2024-04-25T01:35:57.290 INFO:tasks.workunit.client.0.smithi033.stdout:4/763: fsync d6/d14/d35/d3a/d4c/f9f 0 2024-04-25T01:35:57.293 INFO:tasks.workunit.client.0.smithi033.stdout:4/764: rename d6/lf4 to d6/d14/d48/lf5 0 2024-04-25T01:35:57.294 INFO:tasks.workunit.client.0.smithi033.stdout:4/765: readlink d6/d12/d38/l86 0 2024-04-25T01:35:57.297 INFO:tasks.workunit.client.1.smithi138.stdout:1/123: dread d3/d4/f9 [0,4194304] 0 2024-04-25T01:35:57.298 INFO:tasks.workunit.client.1.smithi138.stdout:1/124: readlink d3/d27/d16/l21 0 2024-04-25T01:35:57.299 INFO:tasks.workunit.client.1.smithi138.stdout:1/125: mknod d3/d27/d2d/c2e 0 2024-04-25T01:35:57.301 INFO:tasks.workunit.client.0.smithi033.stdout:8/717: truncate d6/d45/f76 3418124 0 2024-04-25T01:35:57.301 INFO:tasks.workunit.client.0.smithi033.stdout:9/660: dread d0/f9e [0,4194304] 0 2024-04-25T01:35:57.302 INFO:tasks.workunit.client.0.smithi033.stdout:4/766: dwrite d6/d14/f2b [0,4194304] 0 2024-04-25T01:35:57.302 INFO:tasks.workunit.client.1.smithi138.stdout:1/126: rename d3/db/f1b to d3/d27/d2d/f2f 0 2024-04-25T01:35:57.304 INFO:tasks.workunit.client.1.smithi138.stdout:1/127: creat d3/d27/d2d/f30 x:0 0 0 2024-04-25T01:35:57.305 INFO:tasks.workunit.client.0.smithi033.stdout:9/661: readlink d0/d2/d43/d4b/dc5/de0/d42/lc6 0 2024-04-25T01:35:57.305 INFO:tasks.workunit.client.0.smithi033.stdout:4/767: chown d6/d14/d35/d3a/d4d/d5e/fdd 3 1 2024-04-25T01:35:57.305 INFO:tasks.workunit.client.1.smithi138.stdout:1/128: fsync d3/d4/f9 0 2024-04-25T01:35:57.306 INFO:tasks.workunit.client.0.smithi033.stdout:6/803: dwrite d4/d1d/d3e/fa0 [4194304,4194304] 0 2024-04-25T01:35:57.306 INFO:tasks.workunit.client.0.smithi033.stdout:8/718: mknod d6/d45/d51/d5d/db9/c10e 0 2024-04-25T01:35:57.309 INFO:tasks.workunit.client.1.smithi138.stdout:1/129: read d3/d27/d16/f18 [689139,14346] 0 2024-04-25T01:35:57.310 INFO:tasks.workunit.client.1.smithi138.stdout:1/130: symlink d3/d27/d2d/l31 0 2024-04-25T01:35:57.312 INFO:tasks.workunit.client.1.smithi138.stdout:1/131: truncate d3/d27/d16/f28 797127 0 2024-04-25T01:35:57.312 INFO:tasks.workunit.client.0.smithi033.stdout:4/768: mkdir d6/d14/df6 0 2024-04-25T01:35:57.314 INFO:tasks.workunit.client.0.smithi033.stdout:8/719: unlink d6/db/d80/d89/dbf/l109 0 2024-04-25T01:35:57.316 INFO:tasks.workunit.client.0.smithi033.stdout:6/804: creat d4/d24/f118 x:0 0 0 2024-04-25T01:35:57.317 INFO:tasks.workunit.client.0.smithi033.stdout:4/769: fsync d6/d14/d35/d44/d45/d46/d57/d70/d87/fd0 0 2024-04-25T01:35:57.318 INFO:tasks.workunit.client.0.smithi033.stdout:9/662: write d0/d2/dd/d20/d4e/f90 [1242847,2669] 0 2024-04-25T01:35:57.320 INFO:tasks.workunit.client.1.smithi138.stdout:1/132: dwrite d3/d27/d2d/f2f [0,4194304] 0 2024-04-25T01:35:57.321 INFO:tasks.workunit.client.0.smithi033.stdout:8/720: creat d6/db/f10f x:0 0 0 2024-04-25T01:35:57.323 INFO:tasks.workunit.client.1.smithi138.stdout:1/133: unlink d3/d4/c1f 0 2024-04-25T01:35:57.323 INFO:tasks.workunit.client.0.smithi033.stdout:9/663: dread d0/d2/dd/d20/d4e/f90 [0,4194304] 0 2024-04-25T01:35:57.324 INFO:tasks.workunit.client.1.smithi138.stdout:1/134: write d3/d27/d2d/f2f [1421205,17278] 0 2024-04-25T01:35:57.325 INFO:tasks.workunit.client.0.smithi033.stdout:9/664: stat d0/d2/dd/df/d1b/f99 0 2024-04-25T01:35:57.326 INFO:tasks.workunit.client.1.smithi138.stdout:1/135: getdents d3/db/d2b 0 2024-04-25T01:35:57.327 INFO:tasks.workunit.client.0.smithi033.stdout:8/721: symlink d6/dd3/l110 0 2024-04-25T01:35:57.331 INFO:tasks.workunit.client.0.smithi033.stdout:9/665: chown d0/d11/f4c 5 1 2024-04-25T01:35:57.332 INFO:tasks.workunit.client.1.smithi138.stdout:1/136: dwrite d3/f15 [4194304,4194304] 0 2024-04-25T01:35:57.333 INFO:tasks.workunit.client.0.smithi033.stdout:9/666: write d0/d2/dd/df/d1b/f2e [702270,57071] 0 2024-04-25T01:35:57.334 INFO:tasks.workunit.client.0.smithi033.stdout:6/805: truncate d4/d1d/d3e/fa0 2860655 0 2024-04-25T01:35:57.335 INFO:tasks.workunit.client.0.smithi033.stdout:8/722: truncate d6/dd3/dfb/f101 197412 0 2024-04-25T01:35:57.338 INFO:tasks.workunit.client.0.smithi033.stdout:9/667: creat d0/d2/d43/d73/dc4/dcb/ff1 x:0 0 0 2024-04-25T01:35:57.339 INFO:tasks.workunit.client.0.smithi033.stdout:6/806: creat d4/d1d/d3e/d41/d5c/d7f/da4/f119 x:0 0 0 2024-04-25T01:35:57.340 INFO:tasks.workunit.client.1.smithi138.stdout:1/137: write d3/db/f17 [4137224,42422] 0 2024-04-25T01:35:57.341 INFO:tasks.workunit.client.0.smithi033.stdout:6/807: chown d4/dfc/l103 1445567 1 2024-04-25T01:35:57.345 INFO:tasks.workunit.client.0.smithi033.stdout:8/723: dwrite d6/d45/d51/d55/ff6 [0,4194304] 0 2024-04-25T01:35:57.353 INFO:tasks.workunit.client.0.smithi033.stdout:6/808: dwrite d4/d24/d36/d42/d9d/fb0 [0,4194304] 0 2024-04-25T01:35:57.355 INFO:tasks.workunit.client.0.smithi033.stdout:4/770: dread d6/d14/d35/d44/fe1 [4194304,4194304] 0 2024-04-25T01:35:57.355 INFO:tasks.workunit.client.1.smithi138.stdout:1/138: dwrite d3/d4/fd [0,4194304] 0 2024-04-25T01:35:57.356 INFO:tasks.workunit.client.0.smithi033.stdout:9/668: write d0/d2/d43/d4b/f63 [1371510,38954] 0 2024-04-25T01:35:57.356 INFO:tasks.workunit.client.0.smithi033.stdout:8/724: write d6/db/f2d [970606,112874] 0 2024-04-25T01:35:57.358 INFO:tasks.workunit.client.0.smithi033.stdout:6/809: dread - d4/d24/d36/d42/d97/dd6/f114 zero size 2024-04-25T01:35:57.364 INFO:tasks.workunit.client.0.smithi033.stdout:6/810: creat d4/d24/d36/d42/d97/db2/f11a x:0 0 0 2024-04-25T01:35:57.368 INFO:tasks.workunit.client.0.smithi033.stdout:8/725: dwrite d6/d45/d51/d5d/d6e/f77 [0,4194304] 0 2024-04-25T01:35:57.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/139: truncate d3/d27/f1e 1416907 0 2024-04-25T01:35:57.370 INFO:tasks.workunit.client.0.smithi033.stdout:9/669: dwrite d0/d2/d43/f3c [0,4194304] 0 2024-04-25T01:35:57.371 INFO:tasks.workunit.client.1.smithi138.stdout:1/140: dread d3/f2c [0,4194304] 0 2024-04-25T01:35:57.372 INFO:tasks.workunit.client.1.smithi138.stdout:1/141: truncate d3/db/fe 302265 0 2024-04-25T01:35:57.373 INFO:tasks.workunit.client.0.smithi033.stdout:9/670: fdatasync d0/d2/fd6 0 2024-04-25T01:35:57.374 INFO:tasks.workunit.client.0.smithi033.stdout:8/726: mkdir d6/db/d111 0 2024-04-25T01:35:57.374 INFO:tasks.workunit.client.1.smithi138.stdout:1/142: creat d3/d27/d2d/f32 x:0 0 0 2024-04-25T01:35:57.377 INFO:tasks.workunit.client.0.smithi033.stdout:9/671: creat d0/d2/d43/d4b/dc5/de0/ff2 x:0 0 0 2024-04-25T01:35:57.379 INFO:tasks.workunit.client.0.smithi033.stdout:6/811: dwrite d4/d24/d36/d42/f115 [0,4194304] 0 2024-04-25T01:35:57.380 INFO:tasks.workunit.client.0.smithi033.stdout:8/727: mkdir d6/d45/d52/dcc/d112 0 2024-04-25T01:35:57.390 INFO:tasks.workunit.client.0.smithi033.stdout:9/672: dwrite d0/d2/dd/df/d1b/f2e [0,4194304] 0 2024-04-25T01:35:57.395 INFO:tasks.workunit.client.0.smithi033.stdout:6/812: dwrite d4/d24/d7c/f80 [4194304,4194304] 0 2024-04-25T01:35:57.397 INFO:tasks.workunit.client.1.smithi138.stdout:1/143: dwrite d3/d27/d16/f18 [0,4194304] 0 2024-04-25T01:35:57.403 INFO:tasks.workunit.client.0.smithi033.stdout:9/673: dwrite d0/d2/d43/d73/f9f [0,4194304] 0 2024-04-25T01:35:57.405 INFO:tasks.workunit.client.0.smithi033.stdout:9/674: write d0/d2/d43/d73/dc4/fd5 [301987,116955] 0 2024-04-25T01:35:57.413 INFO:tasks.workunit.client.1.smithi138.stdout:1/144: truncate d3/db/f17 2206204 0 2024-04-25T01:35:57.415 INFO:tasks.workunit.client.0.smithi033.stdout:8/728: dwrite d6/db/fc [0,4194304] 0 2024-04-25T01:35:57.417 INFO:tasks.workunit.client.0.smithi033.stdout:8/729: fdatasync d6/d45/d51/d5d/d9d/fce 0 2024-04-25T01:35:57.419 INFO:tasks.workunit.client.1.smithi138.stdout:1/145: dwrite d3/d27/d16/f18 [0,4194304] 0 2024-04-25T01:35:57.422 INFO:tasks.workunit.client.1.smithi138.stdout:1/146: mknod d3/d27/d16/c33 0 2024-04-25T01:35:57.423 INFO:tasks.workunit.client.1.smithi138.stdout:1/147: stat d3/db/c11 0 2024-04-25T01:35:57.447 INFO:tasks.workunit.client.0.smithi033.stdout:9/675: read d0/f23 [200601,24939] 0 2024-04-25T01:35:57.449 INFO:tasks.workunit.client.0.smithi033.stdout:9/676: symlink d0/d2/d43/d4b/dc5/de0/lf3 0 2024-04-25T01:35:57.450 INFO:tasks.workunit.client.0.smithi033.stdout:9/677: fsync d0/d2/dd/d5f/fce 0 2024-04-25T01:35:57.457 INFO:tasks.workunit.client.0.smithi033.stdout:6/813: truncate d4/d24/d7c/fc3 3177976 0 2024-04-25T01:35:57.457 INFO:tasks.workunit.client.0.smithi033.stdout:9/678: dwrite d0/d2/d43/d73/dc4/dcb/ff1 [0,4194304] 0 2024-04-25T01:35:57.458 INFO:tasks.workunit.client.0.smithi033.stdout:8/730: truncate d6/d45/d52/d2e/f56 45750 0 2024-04-25T01:35:57.458 INFO:tasks.workunit.client.0.smithi033.stdout:9/679: write d0/f33 [3046703,76315] 0 2024-04-25T01:35:57.460 INFO:tasks.workunit.client.0.smithi033.stdout:6/814: fdatasync d4/d24/d63/f6f 0 2024-04-25T01:35:57.464 INFO:tasks.workunit.client.0.smithi033.stdout:8/731: rmdir d6/db/d80/db4 39 2024-04-25T01:35:57.465 INFO:tasks.workunit.client.0.smithi033.stdout:9/680: mknod d0/d2/d43/d73/cf4 0 2024-04-25T01:35:57.466 INFO:tasks.workunit.client.0.smithi033.stdout:6/815: mknod d4/d1d/d3e/d41/d54/da1/c11b 0 2024-04-25T01:35:57.468 INFO:tasks.workunit.client.0.smithi033.stdout:6/816: write d4/d24/d36/d42/d9d/fdd [2807759,147] 0 2024-04-25T01:35:57.469 INFO:tasks.workunit.client.0.smithi033.stdout:9/681: link d0/d2/dd/d5f/fce d0/d2/dd/dba/ff5 0 2024-04-25T01:35:57.483 INFO:tasks.workunit.client.0.smithi033.stdout:9/682: truncate d0/f9e 1393487 0 2024-04-25T01:35:57.484 INFO:tasks.workunit.client.0.smithi033.stdout:9/683: chown d0/d2/d43/d4b/fe9 188653299 1 2024-04-25T01:35:57.485 INFO:tasks.workunit.client.0.smithi033.stdout:9/684: stat d0/d2/d43/d4b/fa8 0 2024-04-25T01:35:57.488 INFO:tasks.workunit.client.0.smithi033.stdout:9/685: symlink d0/d2/dd/dba/lf6 0 2024-04-25T01:35:57.489 INFO:tasks.workunit.client.0.smithi033.stdout:6/817: dread d4/d24/d7c/fba [0,4194304] 0 2024-04-25T01:35:57.490 INFO:tasks.workunit.client.0.smithi033.stdout:9/686: truncate d0/d2/dd/f41 1509271 0 2024-04-25T01:35:57.491 INFO:tasks.workunit.client.0.smithi033.stdout:6/818: dread - d4/d1d/d3e/d41/d93/fab zero size 2024-04-25T01:35:57.493 INFO:tasks.workunit.client.0.smithi033.stdout:9/687: unlink d0/d2/dd/f52 0 2024-04-25T01:35:57.494 INFO:tasks.workunit.client.0.smithi033.stdout:6/819: creat d4/d24/d36/dbd/f11c x:0 0 0 2024-04-25T01:35:57.497 INFO:tasks.workunit.client.0.smithi033.stdout:6/820: fdatasync f3 0 2024-04-25T01:35:57.498 INFO:tasks.workunit.client.0.smithi033.stdout:6/821: chown d4/d24/d36/d42/d97 58178781 1 2024-04-25T01:35:57.500 INFO:tasks.workunit.client.0.smithi033.stdout:6/822: symlink d4/d24/d36/d42/d97/db2/l11d 0 2024-04-25T01:35:57.501 INFO:tasks.workunit.client.0.smithi033.stdout:1/536: sync 2024-04-25T01:35:57.526 INFO:tasks.workunit.client.0.smithi033.stdout:9/688: write d0/d2/f1f [514503,73608] 0 2024-04-25T01:35:57.526 INFO:tasks.workunit.client.0.smithi033.stdout:6/823: dwrite d4/d24/d36/d42/d97/db2/f11a [0,4194304] 0 2024-04-25T01:35:57.528 INFO:tasks.workunit.client.0.smithi033.stdout:1/537: dwrite d2/d11/d28/f79 [0,4194304] 0 2024-04-25T01:35:57.530 INFO:tasks.workunit.client.0.smithi033.stdout:1/538: dread - d2/d5/d1b/d46/d5f/d63/d64/f6a zero size 2024-04-25T01:35:57.537 INFO:tasks.workunit.client.0.smithi033.stdout:9/689: truncate d0/f2d 610777 0 2024-04-25T01:35:57.539 INFO:tasks.workunit.client.0.smithi033.stdout:1/539: mknod d2/d5/ca4 0 2024-04-25T01:35:57.539 INFO:tasks.workunit.client.0.smithi033.stdout:6/824: dwrite d4/d1d/d3e/d41/d93/fab [0,4194304] 0 2024-04-25T01:35:57.540 INFO:tasks.workunit.client.0.smithi033.stdout:9/690: chown d0/d2/d43/d4b/d55/c91 144062512 1 2024-04-25T01:35:57.541 INFO:tasks.workunit.client.0.smithi033.stdout:1/540: readlink d2/d11/l4b 0 2024-04-25T01:35:57.544 INFO:tasks.workunit.client.0.smithi033.stdout:6/825: rmdir d4/d15 39 2024-04-25T01:35:57.545 INFO:tasks.workunit.client.0.smithi033.stdout:9/691: truncate d0/d2/dd/df/f5c 226377 0 2024-04-25T01:35:57.548 INFO:tasks.workunit.client.0.smithi033.stdout:1/541: creat d2/d5/d1b/d46/d5f/d63/d64/fa5 x:0 0 0 2024-04-25T01:35:57.553 INFO:tasks.workunit.client.0.smithi033.stdout:1/542: creat d2/d11/fa6 x:0 0 0 2024-04-25T01:35:57.555 INFO:tasks.workunit.client.0.smithi033.stdout:1/543: creat d2/d2e/d34/d60/fa7 x:0 0 0 2024-04-25T01:35:57.557 INFO:tasks.workunit.client.0.smithi033.stdout:6/826: dread d4/d1d/d3e/d41/d5c/f8b [0,4194304] 0 2024-04-25T01:35:57.562 INFO:tasks.workunit.client.0.smithi033.stdout:9/692: write d0/d2/f32 [170801,69152] 0 2024-04-25T01:35:57.564 INFO:tasks.workunit.client.0.smithi033.stdout:6/827: truncate d4/d2a/f39 3204073 0 2024-04-25T01:35:57.564 INFO:tasks.workunit.client.0.smithi033.stdout:9/693: chown d0/d2/dd/f51 150 1 2024-04-25T01:35:57.565 INFO:tasks.workunit.client.0.smithi033.stdout:6/828: dread - d4/d1d/d3e/d41/d93/ff9 zero size 2024-04-25T01:35:57.568 INFO:tasks.workunit.client.0.smithi033.stdout:6/829: rmdir d4/d1d/d3e/d41/d5c/dae 39 2024-04-25T01:35:57.568 INFO:tasks.workunit.client.1.smithi138.stdout:6/86: sync 2024-04-25T01:35:57.568 INFO:tasks.workunit.client.0.smithi033.stdout:6/830: chown d4/d1d/d3e/d41/f10c 422 1 2024-04-25T01:35:57.569 INFO:tasks.workunit.client.0.smithi033.stdout:6/831: readlink d4/d1d/d3e/l65 0 2024-04-25T01:35:57.570 INFO:tasks.workunit.client.1.smithi138.stdout:6/87: creat dc/f1d x:0 0 0 2024-04-25T01:35:57.570 INFO:tasks.workunit.client.1.smithi138.stdout:6/88: dread - dc/f16 zero size 2024-04-25T01:35:57.572 INFO:tasks.workunit.client.1.smithi138.stdout:6/89: truncate dc/d10/f15 350558 0 2024-04-25T01:35:57.576 INFO:tasks.workunit.client.0.smithi033.stdout:9/694: link d0/d2/d43/d4b/dc5/de0/f70 d0/d2/d43/d73/dab/dad/dee/ff7 0 2024-04-25T01:35:57.577 INFO:tasks.workunit.client.1.smithi138.stdout:6/90: rmdir dc 39 2024-04-25T01:35:57.578 INFO:tasks.workunit.client.0.smithi033.stdout:1/544: dread d2/d2e/d34/d60/f7a [0,4194304] 0 2024-04-25T01:35:57.578 INFO:tasks.workunit.client.0.smithi033.stdout:6/832: truncate d4/d24/d36/dbd/f11c 307990 0 2024-04-25T01:35:57.579 INFO:tasks.workunit.client.1.smithi138.stdout:6/91: rmdir dc 39 2024-04-25T01:35:57.580 INFO:tasks.workunit.client.0.smithi033.stdout:1/545: write d2/d2e/d34/d60/f74 [153094,2973] 0 2024-04-25T01:35:57.581 INFO:tasks.workunit.client.1.smithi138.stdout:6/92: mknod dc/d10/c1e 0 2024-04-25T01:35:57.583 INFO:tasks.workunit.client.1.smithi138.stdout:6/93: creat dc/f1f x:0 0 0 2024-04-25T01:35:57.585 INFO:tasks.workunit.client.0.smithi033.stdout:9/695: dwrite d0/d2/dd/dba/fd4 [0,4194304] 0 2024-04-25T01:35:57.587 INFO:tasks.workunit.client.0.smithi033.stdout:9/696: dread - d0/d2/dd/d20/d4e/f68 zero size 2024-04-25T01:35:57.588 INFO:tasks.workunit.client.1.smithi138.stdout:6/94: truncate f8 2233450 0 2024-04-25T01:35:57.589 INFO:tasks.workunit.client.0.smithi033.stdout:9/697: chown d0/d2/dd/f54 54 1 2024-04-25T01:35:57.589 INFO:tasks.workunit.client.1.smithi138.stdout:6/95: symlink dc/d10/l20 0 2024-04-25T01:35:57.590 INFO:tasks.workunit.client.0.smithi033.stdout:1/546: dwrite d2/d11/d28/f79 [0,4194304] 0 2024-04-25T01:35:57.590 INFO:tasks.workunit.client.1.smithi138.stdout:6/96: dread - dc/f11 zero size 2024-04-25T01:35:57.591 INFO:tasks.workunit.client.0.smithi033.stdout:9/698: truncate d0/d11/da1/fd3 688764 0 2024-04-25T01:35:57.592 INFO:tasks.workunit.client.1.smithi138.stdout:6/97: mknod dc/c21 0 2024-04-25T01:35:57.594 INFO:tasks.workunit.client.1.smithi138.stdout:6/98: rename dc/d10/f1c to dc/d10/f22 0 2024-04-25T01:35:57.594 INFO:tasks.workunit.client.0.smithi033.stdout:9/699: read d0/d2/dd/fe [1174032,9436] 0 2024-04-25T01:35:57.596 INFO:tasks.workunit.client.1.smithi138.stdout:6/99: write dc/d10/f22 [222846,52668] 0 2024-04-25T01:35:57.596 INFO:tasks.workunit.client.1.smithi138.stdout:6/100: readlink dc/d10/l1b 0 2024-04-25T01:35:57.602 INFO:tasks.workunit.client.0.smithi033.stdout:6/833: symlink d4/d24/d36/d42/d97/de0/l11e 0 2024-04-25T01:35:57.604 INFO:tasks.workunit.client.0.smithi033.stdout:1/547: chown d2/d5/d6c/d94 949544 1 2024-04-25T01:35:57.605 INFO:tasks.workunit.client.0.smithi033.stdout:9/700: creat d0/d2/d43/d73/dab/dad/ff8 x:0 0 0 2024-04-25T01:35:57.609 INFO:tasks.workunit.client.0.smithi033.stdout:6/834: dwrite d4/df1/ff8 [0,4194304] 0 2024-04-25T01:35:57.611 INFO:tasks.workunit.client.0.smithi033.stdout:1/548: rename d2/d2e/l43 to d2/d5/d1b/d46/d5f/d63/la8 0 2024-04-25T01:35:57.612 INFO:tasks.workunit.client.0.smithi033.stdout:1/549: stat d2/d2e/ca0 0 2024-04-25T01:35:57.613 INFO:tasks.workunit.client.0.smithi033.stdout:9/701: rmdir d0/d2/dd/dba/dc1/de2 0 2024-04-25T01:35:57.652 INFO:tasks.workunit.client.1.smithi138.stdout:9/112: sync 2024-04-25T01:35:57.653 INFO:tasks.workunit.client.1.smithi138.stdout:9/113: creat d4/d8/f23 x:0 0 0 2024-04-25T01:35:57.654 INFO:tasks.workunit.client.1.smithi138.stdout:9/114: truncate d4/f22 372091 0 2024-04-25T01:35:57.656 INFO:tasks.workunit.client.1.smithi138.stdout:9/115: symlink d4/d8/d15/d1d/l24 0 2024-04-25T01:35:57.656 INFO:tasks.workunit.client.1.smithi138.stdout:9/116: dread - d4/d1a/f20 zero size 2024-04-25T01:35:57.658 INFO:tasks.workunit.client.1.smithi138.stdout:9/117: read d4/d8/d15/f16 [2973719,9192] 0 2024-04-25T01:35:57.659 INFO:tasks.workunit.client.1.smithi138.stdout:9/118: fsync d4/d8/d15/f16 0 2024-04-25T01:35:57.661 INFO:tasks.workunit.client.1.smithi138.stdout:9/119: creat d4/f25 x:0 0 0 2024-04-25T01:35:57.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/120: dwrite d4/d8/f1f [0,4194304] 0 2024-04-25T01:35:57.674 INFO:tasks.workunit.client.1.smithi138.stdout:9/121: getdents d4/d8 0 2024-04-25T01:35:57.677 INFO:tasks.workunit.client.1.smithi138.stdout:9/122: creat d4/f26 x:0 0 0 2024-04-25T01:35:57.678 INFO:tasks.workunit.client.1.smithi138.stdout:9/123: write f1 [1250731,44488] 0 2024-04-25T01:35:57.678 INFO:tasks.workunit.client.1.smithi138.stdout:9/124: dread - d4/d1a/f20 zero size 2024-04-25T01:35:57.679 INFO:tasks.workunit.client.1.smithi138.stdout:9/125: dread - d4/d8/f23 zero size 2024-04-25T01:35:57.687 INFO:tasks.workunit.client.1.smithi138.stdout:9/126: getdents d4 0 2024-04-25T01:35:57.687 INFO:tasks.workunit.client.1.smithi138.stdout:9/127: dread - d4/d1a/f21 zero size 2024-04-25T01:35:57.688 INFO:tasks.workunit.client.1.smithi138.stdout:9/128: symlink d4/d8/d15/d1d/l27 0 2024-04-25T01:35:57.689 INFO:tasks.workunit.client.0.smithi033.stdout:6/835: stat d4/d24/d36/d42/c4d 0 2024-04-25T01:35:57.690 INFO:tasks.workunit.client.0.smithi033.stdout:6/836: chown d4/f2d 5 1 2024-04-25T01:35:57.690 INFO:tasks.workunit.client.1.smithi138.stdout:9/129: link d4/c18 d4/c28 0 2024-04-25T01:35:57.691 INFO:tasks.workunit.client.0.smithi033.stdout:0/587: sync 2024-04-25T01:35:57.691 INFO:tasks.workunit.client.0.smithi033.stdout:9/702: mkdir d0/d2/dd/df/d1b/dc0/df9 0 2024-04-25T01:35:57.692 INFO:tasks.workunit.client.1.smithi138.stdout:9/130: chown d4/d8/d15/f16 13780543 1 2024-04-25T01:35:57.693 INFO:tasks.workunit.client.1.smithi138.stdout:9/131: mknod d4/d8/c29 0 2024-04-25T01:35:57.695 INFO:tasks.workunit.client.0.smithi033.stdout:9/703: dread d0/f2d [0,4194304] 0 2024-04-25T01:35:57.695 INFO:tasks.workunit.client.1.smithi138.stdout:9/132: creat d4/f2a x:0 0 0 2024-04-25T01:35:57.695 INFO:tasks.workunit.client.0.smithi033.stdout:0/588: mknod d7/d2d/d9b/cce 0 2024-04-25T01:35:57.696 INFO:tasks.workunit.client.1.smithi138.stdout:9/133: write d4/f2a [760093,27627] 0 2024-04-25T01:35:57.696 INFO:tasks.workunit.client.0.smithi033.stdout:1/550: dwrite d2/d5/fd [4194304,4194304] 0 2024-04-25T01:35:57.697 INFO:tasks.workunit.client.1.smithi138.stdout:9/134: rmdir d4/d8 39 2024-04-25T01:35:57.698 INFO:tasks.workunit.client.1.smithi138.stdout:9/135: stat d4/d1a 0 2024-04-25T01:35:57.698 INFO:tasks.workunit.client.0.smithi033.stdout:6/837: mknod d4/d1d/d3e/d41/d5c/dae/de1/c11f 0 2024-04-25T01:35:57.699 INFO:tasks.workunit.client.1.smithi138.stdout:9/136: dread - d4/d1a/f21 zero size 2024-04-25T01:35:57.701 INFO:tasks.workunit.client.1.smithi138.stdout:9/137: write d4/d8/f1f [3644454,81661] 0 2024-04-25T01:35:57.708 INFO:tasks.workunit.client.1.smithi138.stdout:9/138: dwrite d4/d8/f1f [4194304,4194304] 0 2024-04-25T01:35:57.711 INFO:tasks.workunit.client.1.smithi138.stdout:9/139: symlink d4/l2b 0 2024-04-25T01:35:57.717 INFO:tasks.workunit.client.1.smithi138.stdout:9/140: dread f1 [0,4194304] 0 2024-04-25T01:35:57.718 INFO:tasks.workunit.client.1.smithi138.stdout:9/141: chown d4/d8/d15/d1d/l27 2037271013 1 2024-04-25T01:35:57.720 INFO:tasks.workunit.client.1.smithi138.stdout:9/142: fsync d4/f13 0 2024-04-25T01:35:57.720 INFO:tasks.workunit.client.1.smithi138.stdout:9/143: truncate d4/f2a 910437 0 2024-04-25T01:35:57.721 INFO:tasks.workunit.client.1.smithi138.stdout:9/144: read - d4/d1a/f21 zero size 2024-04-25T01:35:57.723 INFO:tasks.workunit.client.1.smithi138.stdout:9/145: mkdir d4/d8/d2c 0 2024-04-25T01:35:57.723 INFO:tasks.workunit.client.0.smithi033.stdout:9/704: truncate d0/d2/d43/d4b/dc5/de0/d7e/f6a 15855 0 2024-04-25T01:35:57.725 INFO:tasks.workunit.client.0.smithi033.stdout:6/838: mknod d4/d1d/d3e/d41/d5c/c120 0 2024-04-25T01:35:57.726 INFO:tasks.workunit.client.1.smithi138.stdout:9/146: unlink d4/c19 0 2024-04-25T01:35:57.727 INFO:tasks.workunit.client.0.smithi033.stdout:6/839: chown d4/d1d/d3e/d41/d5c/d7f/f44 0 1 2024-04-25T01:35:57.728 INFO:tasks.workunit.client.0.smithi033.stdout:0/589: getdents d7/da 0 2024-04-25T01:35:57.729 INFO:tasks.workunit.client.1.smithi138.stdout:9/147: rename d4/f2a to d4/d1a/f2d 0 2024-04-25T01:35:57.731 INFO:tasks.workunit.client.1.smithi138.stdout:9/148: dread d4/f22 [0,4194304] 0 2024-04-25T01:35:57.733 INFO:tasks.workunit.client.1.smithi138.stdout:9/149: symlink d4/d8/l2e 0 2024-04-25T01:35:57.740 INFO:tasks.workunit.client.1.smithi138.stdout:9/150: dwrite d4/f22 [0,4194304] 0 2024-04-25T01:35:57.741 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:57 smithi033 ceph-mon[26353]: pgmap v14: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 57 MiB/s rd, 226 MiB/s wr, 546 op/s 2024-04-25T01:35:57.748 INFO:tasks.workunit.client.1.smithi138.stdout:9/151: dwrite d4/d8/f1f [0,4194304] 0 2024-04-25T01:35:57.754 INFO:tasks.workunit.client.1.smithi138.stdout:9/152: dwrite d4/d8/d15/f16 [0,4194304] 0 2024-04-25T01:35:57.759 INFO:tasks.workunit.client.1.smithi138.stdout:9/153: dread d4/d1a/f2d [0,4194304] 0 2024-04-25T01:35:57.760 INFO:tasks.workunit.client.1.smithi138.stdout:9/154: stat d4/d1a/f21 0 2024-04-25T01:35:57.760 INFO:tasks.workunit.client.1.smithi138.stdout:9/155: fdatasync d4/d1a/f20 0 2024-04-25T01:35:57.761 INFO:tasks.workunit.client.1.smithi138.stdout:9/156: chown d4/l2b 6056271 1 2024-04-25T01:35:57.762 INFO:tasks.workunit.client.1.smithi138.stdout:9/157: chown d4/d8/d15/d1d/l24 3 1 2024-04-25T01:35:57.763 INFO:tasks.workunit.client.1.smithi138.stdout:9/158: mknod d4/d8/d15/c2f 0 2024-04-25T01:35:57.765 INFO:tasks.workunit.client.1.smithi138.stdout:9/159: stat d4/f26 0 2024-04-25T01:35:57.766 INFO:tasks.workunit.client.1.smithi138.stdout:9/160: symlink d4/d8/d2c/l30 0 2024-04-25T01:35:57.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:57 smithi138 ceph-mon[32977]: pgmap v14: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 57 MiB/s rd, 226 MiB/s wr, 546 op/s 2024-04-25T01:35:57.768 INFO:tasks.workunit.client.1.smithi138.stdout:9/161: creat d4/d8/d15/d1d/f31 x:0 0 0 2024-04-25T01:35:57.769 INFO:tasks.workunit.client.1.smithi138.stdout:9/162: rmdir d4/d8/d15 39 2024-04-25T01:35:57.770 INFO:tasks.workunit.client.1.smithi138.stdout:9/163: read d4/d1a/f2d [862857,66038] 0 2024-04-25T01:35:57.772 INFO:tasks.workunit.client.1.smithi138.stdout:9/164: symlink d4/d8/d15/d1d/l32 0 2024-04-25T01:35:57.773 INFO:tasks.workunit.client.1.smithi138.stdout:9/165: dread - d4/f25 zero size 2024-04-25T01:35:57.778 INFO:tasks.workunit.client.1.smithi138.stdout:9/166: rename d4/d8/d15 to d4/d1a/d33 0 2024-04-25T01:35:57.779 INFO:tasks.workunit.client.1.smithi138.stdout:9/167: write d4/d1a/d33/d1d/f31 [921457,33353] 0 2024-04-25T01:35:57.780 INFO:tasks.workunit.client.1.smithi138.stdout:9/168: symlink d4/d8/d2c/l34 0 2024-04-25T01:35:57.784 INFO:tasks.workunit.client.1.smithi138.stdout:9/169: dread d4/d8/f1f [0,4194304] 0 2024-04-25T01:35:57.787 INFO:tasks.workunit.client.1.smithi138.stdout:9/170: rmdir d4 39 2024-04-25T01:35:57.788 INFO:tasks.workunit.client.1.smithi138.stdout:4/107: sync 2024-04-25T01:35:57.789 INFO:tasks.workunit.client.1.smithi138.stdout:9/171: readlink d4/d1a/d33/l1e 0 2024-04-25T01:35:57.790 INFO:tasks.workunit.client.1.smithi138.stdout:4/108: mknod d0/c19 0 2024-04-25T01:35:57.791 INFO:tasks.workunit.client.1.smithi138.stdout:4/109: dread - d0/d16/f18 zero size 2024-04-25T01:35:57.791 INFO:tasks.workunit.client.1.smithi138.stdout:4/110: fdatasync d0/fa 0 2024-04-25T01:35:57.792 INFO:tasks.workunit.client.1.smithi138.stdout:4/111: fsync d0/fc 0 2024-04-25T01:35:57.794 INFO:tasks.workunit.client.1.smithi138.stdout:9/172: rename d4/d1a/d33/d1d/l27 to d4/l35 0 2024-04-25T01:35:57.796 INFO:tasks.workunit.client.1.smithi138.stdout:9/173: creat d4/d1a/d33/f36 x:0 0 0 2024-04-25T01:35:57.797 INFO:tasks.workunit.client.1.smithi138.stdout:4/112: rename d0/ff to d0/f1a 0 2024-04-25T01:35:57.798 INFO:tasks.workunit.client.1.smithi138.stdout:4/113: write d0/fa [4330818,47550] 0 2024-04-25T01:35:57.798 INFO:tasks.workunit.client.1.smithi138.stdout:4/114: chown d0/d16 28900329 1 2024-04-25T01:35:57.802 INFO:tasks.workunit.client.1.smithi138.stdout:4/115: getdents d0/d16 0 2024-04-25T01:35:57.807 INFO:tasks.workunit.client.1.smithi138.stdout:4/116: dwrite d0/fa [0,4194304] 0 2024-04-25T01:35:57.809 INFO:tasks.workunit.client.1.smithi138.stdout:4/117: symlink d0/d16/l1b 0 2024-04-25T01:35:57.810 INFO:tasks.workunit.client.1.smithi138.stdout:4/118: write d0/f13 [51533,60130] 0 2024-04-25T01:35:57.814 INFO:tasks.workunit.client.1.smithi138.stdout:4/119: mkdir d0/d1c 0 2024-04-25T01:35:57.814 INFO:tasks.workunit.client.1.smithi138.stdout:9/174: dread d4/f13 [4194304,4194304] 0 2024-04-25T01:35:57.816 INFO:tasks.workunit.client.1.smithi138.stdout:9/175: creat d4/d1a/d33/f37 x:0 0 0 2024-04-25T01:35:57.819 INFO:tasks.workunit.client.0.smithi033.stdout:0/590: chown d7/d2d/d4d/d9c/l53 115907 1 2024-04-25T01:35:57.819 INFO:tasks.workunit.client.0.smithi033.stdout:1/551: write d2/d2e/d34/d60/f7a [1796765,4482] 0 2024-04-25T01:35:57.820 INFO:tasks.workunit.client.1.smithi138.stdout:9/176: creat d4/f38 x:0 0 0 2024-04-25T01:35:57.821 INFO:tasks.workunit.client.0.smithi033.stdout:9/705: dwrite d0/d2/d43/d4b/dc5/de0/d7e/d58/fa6 [0,4194304] 0 2024-04-25T01:35:57.822 INFO:tasks.workunit.client.1.smithi138.stdout:4/120: dwrite d0/d16/f18 [0,4194304] 0 2024-04-25T01:35:57.822 INFO:tasks.workunit.client.0.smithi033.stdout:6/840: rename d4/d2a/f34 to d4/dfc/f121 0 2024-04-25T01:35:57.824 INFO:tasks.workunit.client.1.smithi138.stdout:9/177: symlink d4/d1a/d33/d1d/l39 0 2024-04-25T01:35:57.824 INFO:tasks.workunit.client.0.smithi033.stdout:6/841: chown d4/d1d/d3e/d41/d54/f57 2417 1 2024-04-25T01:35:57.824 INFO:tasks.workunit.client.1.smithi138.stdout:9/178: stat d4/d1a/d33/l1e 0 2024-04-25T01:35:57.827 INFO:tasks.workunit.client.1.smithi138.stdout:4/121: symlink d0/d1c/l1d 0 2024-04-25T01:35:57.827 INFO:tasks.workunit.client.0.smithi033.stdout:0/591: rename d7/d2d/d4c/da0/da1/da8 to d7/d2d/d4d/d9c/dcf 0 2024-04-25T01:35:57.829 INFO:tasks.workunit.client.0.smithi033.stdout:9/706: dwrite d0/d2/dd/d20/f7d [0,4194304] 0 2024-04-25T01:35:57.830 INFO:tasks.workunit.client.1.smithi138.stdout:9/179: rmdir d4/d8 39 2024-04-25T01:35:57.832 INFO:tasks.workunit.client.1.smithi138.stdout:4/122: mkdir d0/d1c/d1e 0 2024-04-25T01:35:57.833 INFO:tasks.workunit.client.1.smithi138.stdout:2/234: sync 2024-04-25T01:35:57.833 INFO:tasks.workunit.client.1.smithi138.stdout:0/200: sync 2024-04-25T01:35:57.833 INFO:tasks.workunit.client.1.smithi138.stdout:6/101: sync 2024-04-25T01:35:57.833 INFO:tasks.workunit.client.1.smithi138.stdout:5/117: sync 2024-04-25T01:35:57.833 INFO:tasks.workunit.client.1.smithi138.stdout:7/142: sync 2024-04-25T01:35:57.833 INFO:tasks.workunit.client.1.smithi138.stdout:8/349: sync 2024-04-25T01:35:57.833 INFO:tasks.workunit.client.1.smithi138.stdout:1/148: sync 2024-04-25T01:35:57.833 INFO:tasks.workunit.client.1.smithi138.stdout:3/165: sync 2024-04-25T01:35:57.834 INFO:tasks.workunit.client.0.smithi033.stdout:1/552: creat d2/d5/d75/d59/d83/fa9 x:0 0 0 2024-04-25T01:35:57.836 INFO:tasks.workunit.client.0.smithi033.stdout:1/553: write d2/d2e/f67 [1036720,94839] 0 2024-04-25T01:35:57.836 INFO:tasks.workunit.client.0.smithi033.stdout:9/707: creat d0/d2/ffa x:0 0 0 2024-04-25T01:35:57.840 INFO:tasks.workunit.client.0.smithi033.stdout:6/842: dwrite d4/dfc/f121 [4194304,4194304] 0 2024-04-25T01:35:57.840 INFO:tasks.workunit.client.1.smithi138.stdout:0/201: chown d6/la 0 1 2024-04-25T01:35:57.840 INFO:tasks.workunit.client.1.smithi138.stdout:2/235: dread d0/d32/f4b [0,4194304] 0 2024-04-25T01:35:57.842 INFO:tasks.workunit.client.1.smithi138.stdout:9/180: symlink d4/l3a 0 2024-04-25T01:35:57.844 INFO:tasks.workunit.client.0.smithi033.stdout:6/843: creat d4/d1d/d3e/d41/d5c/dae/de1/f122 x:0 0 0 2024-04-25T01:35:57.845 INFO:tasks.workunit.client.0.smithi033.stdout:0/592: link d7/d2d/d43/d44/l78 d7/d2d/d43/dc0/ld0 0 2024-04-25T01:35:57.845 INFO:tasks.workunit.client.1.smithi138.stdout:8/350: dread d3/f3a [0,4194304] 0 2024-04-25T01:35:57.846 INFO:tasks.workunit.client.1.smithi138.stdout:6/102: dwrite dc/f14 [0,4194304] 0 2024-04-25T01:35:57.847 INFO:tasks.workunit.client.1.smithi138.stdout:7/143: mknod da/de/c32 0 2024-04-25T01:35:57.851 INFO:tasks.workunit.client.0.smithi033.stdout:1/554: dwrite d2/d5/d75/d59/d83/fa9 [0,4194304] 0 2024-04-25T01:35:57.851 INFO:tasks.workunit.client.0.smithi033.stdout:0/593: creat d7/d2d/d64/fd1 x:0 0 0 2024-04-25T01:35:57.854 INFO:tasks.workunit.client.0.smithi033.stdout:0/594: chown d7/d2d/d64/d51/f52 238823 1 2024-04-25T01:35:57.854 INFO:tasks.workunit.client.0.smithi033.stdout:1/555: dread - d2/d5/d1b/d46/d5f/d63/d64/fa5 zero size 2024-04-25T01:35:57.857 INFO:tasks.workunit.client.1.smithi138.stdout:8/351: creat d3/d8/da/db/d15/d20/f6e x:0 0 0 2024-04-25T01:35:57.857 INFO:tasks.workunit.client.0.smithi033.stdout:0/595: truncate d7/d2d/d64/d51/f89 5000465 0 2024-04-25T01:35:57.859 INFO:tasks.workunit.client.0.smithi033.stdout:0/596: chown d7/c7c 34422879 1 2024-04-25T01:35:57.859 INFO:tasks.workunit.client.0.smithi033.stdout:0/597: stat d7/f46 0 2024-04-25T01:35:57.860 INFO:tasks.workunit.client.1.smithi138.stdout:3/166: rmdir db/df/d19/d2d 0 2024-04-25T01:35:57.860 INFO:tasks.workunit.client.1.smithi138.stdout:7/144: write da/de/f1a [3114110,18270] 0 2024-04-25T01:35:57.860 INFO:tasks.workunit.client.0.smithi033.stdout:1/556: dread d2/d11/f2a [0,4194304] 0 2024-04-25T01:35:57.863 INFO:tasks.workunit.client.0.smithi033.stdout:0/598: mkdir d7/d2d/d4d/d9c/d68/dd2 0 2024-04-25T01:35:57.863 INFO:tasks.workunit.client.1.smithi138.stdout:1/149: rename d3/d4/f22 to d3/f34 0 2024-04-25T01:35:57.864 INFO:tasks.workunit.client.0.smithi033.stdout:0/599: chown d7/d2d/d4c/l76 5971 1 2024-04-25T01:35:57.864 INFO:tasks.workunit.client.0.smithi033.stdout:1/557: unlink d2/d11/d28/f89 0 2024-04-25T01:35:57.868 INFO:tasks.workunit.client.1.smithi138.stdout:6/103: dwrite dc/d10/f22 [0,4194304] 0 2024-04-25T01:35:57.869 INFO:tasks.workunit.client.1.smithi138.stdout:7/145: write da/de/d13/f24 [3021052,128485] 0 2024-04-25T01:35:57.871 INFO:tasks.workunit.client.1.smithi138.stdout:2/236: dwrite d0/d21/d36/d15/d1f/f37 [0,4194304] 0 2024-04-25T01:35:57.872 INFO:tasks.workunit.client.1.smithi138.stdout:5/118: rename d4/f13 to d4/da/d1b/d1c/d22/f24 0 2024-04-25T01:35:57.873 INFO:tasks.workunit.client.1.smithi138.stdout:0/202: dwrite d6/d19/f48 [0,4194304] 0 2024-04-25T01:35:57.873 INFO:tasks.workunit.client.1.smithi138.stdout:3/167: rename db/df/d17 to db/df/d17/d20/d31 22 2024-04-25T01:35:57.874 INFO:tasks.workunit.client.0.smithi033.stdout:1/558: dwrite d2/d2e/f71 [4194304,4194304] 0 2024-04-25T01:35:57.874 INFO:tasks.workunit.client.1.smithi138.stdout:7/146: write da/de/d13/f23 [1192249,24462] 0 2024-04-25T01:35:57.877 INFO:tasks.workunit.client.0.smithi033.stdout:9/708: dread d0/d2/dd/d20/fa4 [4194304,4194304] 0 2024-04-25T01:35:57.877 INFO:tasks.workunit.client.1.smithi138.stdout:2/237: dread - d0/d32/f43 zero size 2024-04-25T01:35:57.878 INFO:tasks.workunit.client.1.smithi138.stdout:6/104: truncate dc/d10/f22 4794899 0 2024-04-25T01:35:57.880 INFO:tasks.workunit.client.1.smithi138.stdout:2/238: chown d0/d21/f3a 198628995 1 2024-04-25T01:35:57.882 INFO:tasks.workunit.client.1.smithi138.stdout:2/239: dread - d0/d21/f22 zero size 2024-04-25T01:35:57.882 INFO:tasks.workunit.client.1.smithi138.stdout:7/147: readlink da/ld 0 2024-04-25T01:35:57.884 INFO:tasks.workunit.client.0.smithi033.stdout:1/559: symlink d2/d5/d75/laa 0 2024-04-25T01:35:57.885 INFO:tasks.workunit.client.1.smithi138.stdout:2/240: chown d0/d21/d36/d15/d1f/f2d 14 1 2024-04-25T01:35:57.885 INFO:tasks.workunit.client.1.smithi138.stdout:5/119: write d4/f23 [582752,776] 0 2024-04-25T01:35:57.886 INFO:tasks.workunit.client.0.smithi033.stdout:6/844: write d4/d1d/d3e/d70/f95 [67598,72819] 0 2024-04-25T01:35:57.886 INFO:tasks.workunit.client.1.smithi138.stdout:6/105: mknod dc/d10/c23 0 2024-04-25T01:35:57.887 INFO:tasks.workunit.client.1.smithi138.stdout:1/150: link d3/c12 d3/d27/d16/c35 0 2024-04-25T01:35:57.887 INFO:tasks.workunit.client.0.smithi033.stdout:2/721: sync 2024-04-25T01:35:57.887 INFO:tasks.workunit.client.0.smithi033.stdout:3/754: sync 2024-04-25T01:35:57.887 INFO:tasks.workunit.client.0.smithi033.stdout:5/741: sync 2024-04-25T01:35:57.888 INFO:tasks.workunit.client.1.smithi138.stdout:2/241: write d0/f9 [5042155,38435] 0 2024-04-25T01:35:57.889 INFO:tasks.workunit.client.1.smithi138.stdout:8/352: getdents d3/d16/d1f/d31 0 2024-04-25T01:35:57.891 INFO:tasks.workunit.client.1.smithi138.stdout:2/242: write d0/d32/f3f [4667601,35241] 0 2024-04-25T01:35:57.892 INFO:tasks.workunit.client.1.smithi138.stdout:8/353: write f0 [1799653,112836] 0 2024-04-25T01:35:57.895 INFO:tasks.workunit.client.1.smithi138.stdout:5/120: rmdir d4 39 2024-04-25T01:35:57.895 INFO:tasks.workunit.client.0.smithi033.stdout:1/560: dwrite d2/d11/d28/f79 [4194304,4194304] 0 2024-04-25T01:35:57.897 INFO:tasks.workunit.client.1.smithi138.stdout:1/151: dread d3/db/fe [0,4194304] 0 2024-04-25T01:35:57.899 INFO:tasks.workunit.client.1.smithi138.stdout:6/106: rename c3 to dc/d10/c24 0 2024-04-25T01:35:57.900 INFO:tasks.workunit.client.0.smithi033.stdout:0/600: write d7/da/f5d [85749,44073] 0 2024-04-25T01:35:57.902 INFO:tasks.workunit.client.0.smithi033.stdout:0/601: fdatasync d7/d2d/d64/d51/fb6 0 2024-04-25T01:35:57.904 INFO:tasks.workunit.client.0.smithi033.stdout:3/755: creat d8/dd/d3d/d63/d7c/d9d/f105 x:0 0 0 2024-04-25T01:35:57.904 INFO:tasks.workunit.client.1.smithi138.stdout:7/148: dwrite f9 [0,4194304] 0 2024-04-25T01:35:57.905 INFO:tasks.workunit.client.0.smithi033.stdout:5/742: mkdir d1c/d2a/d50/d52/da6/d3e/d70/dae/df1/df4 0 2024-04-25T01:35:57.905 INFO:tasks.workunit.client.1.smithi138.stdout:4/123: dwrite d0/f1a [0,4194304] 0 2024-04-25T01:35:57.906 INFO:tasks.workunit.client.0.smithi033.stdout:9/709: dwrite d0/d2/dd/dba/ff5 [0,4194304] 0 2024-04-25T01:35:57.907 INFO:tasks.workunit.client.0.smithi033.stdout:1/561: symlink d2/d5/d1b/d46/d5f/lab 0 2024-04-25T01:35:57.908 INFO:tasks.workunit.client.0.smithi033.stdout:0/602: fdatasync d7/d2d/d43/dc0/f4b 0 2024-04-25T01:35:57.908 INFO:tasks.workunit.client.1.smithi138.stdout:8/354: mkdir d3/d8/da/db/d15/d6f 0 2024-04-25T01:35:57.909 INFO:tasks.workunit.client.0.smithi033.stdout:2/722: dread da/f74 [0,4194304] 0 2024-04-25T01:35:57.913 INFO:tasks.workunit.client.1.smithi138.stdout:2/243: dwrite d0/d21/d36/f3c [0,4194304] 0 2024-04-25T01:35:57.916 INFO:tasks.workunit.client.0.smithi033.stdout:0/603: readlink d7/da/l21 0 2024-04-25T01:35:57.917 INFO:tasks.workunit.client.1.smithi138.stdout:7/149: creat da/de/d1b/f33 x:0 0 0 2024-04-25T01:35:57.918 INFO:tasks.workunit.client.0.smithi033.stdout:6/845: link d4/d1d/d3e/d41/d5c/d7f/d116/f55 d4/d24/f123 0 2024-04-25T01:35:57.919 INFO:tasks.workunit.client.1.smithi138.stdout:4/124: rename d0/le to d0/l1f 0 2024-04-25T01:35:57.919 INFO:tasks.workunit.client.1.smithi138.stdout:1/152: creat d3/d27/f36 x:0 0 0 2024-04-25T01:35:57.920 INFO:tasks.workunit.client.0.smithi033.stdout:1/562: read d2/d5/d1b/d46/d5f/f61 [747752,107692] 0 2024-04-25T01:35:57.921 INFO:tasks.workunit.client.1.smithi138.stdout:1/153: fsync d3/f15 0 2024-04-25T01:35:57.922 INFO:tasks.workunit.client.1.smithi138.stdout:6/107: dwrite dc/d10/f22 [0,4194304] 0 2024-04-25T01:35:57.924 INFO:tasks.workunit.client.0.smithi033.stdout:3/756: getdents d8/dd/d3d/d63/d7c/d9d/de8/d101 0 2024-04-25T01:35:57.925 INFO:tasks.workunit.client.1.smithi138.stdout:8/355: mkdir d3/d8/da/d70 0 2024-04-25T01:35:57.926 INFO:tasks.workunit.client.0.smithi033.stdout:2/723: creat da/d27/d31/d40/d8a/dbb/dd6/de1/feb x:0 0 0 2024-04-25T01:35:57.927 INFO:tasks.workunit.client.1.smithi138.stdout:4/125: truncate d0/f12 1133937 0 2024-04-25T01:35:57.929 INFO:tasks.workunit.client.1.smithi138.stdout:8/356: fdatasync d3/d16/f3b 0 2024-04-25T01:35:57.930 INFO:tasks.workunit.client.1.smithi138.stdout:5/121: rename d4/cc to d4/c25 0 2024-04-25T01:35:57.933 INFO:tasks.workunit.client.0.smithi033.stdout:1/563: chown d2/d5/d75/c39 5894757 1 2024-04-25T01:35:57.933 INFO:tasks.workunit.client.1.smithi138.stdout:6/108: fsync dc/f16 0 2024-04-25T01:35:57.934 INFO:tasks.workunit.client.0.smithi033.stdout:6/846: dwrite d4/d1d/d3e/d41/d5c/fea [0,4194304] 0 2024-04-25T01:35:57.934 INFO:tasks.workunit.client.0.smithi033.stdout:1/564: dread - d2/d11/fa6 zero size 2024-04-25T01:35:57.935 INFO:tasks.workunit.client.1.smithi138.stdout:3/168: dwrite db/df/d17/f2e [0,4194304] 0 2024-04-25T01:35:57.935 INFO:tasks.workunit.client.0.smithi033.stdout:2/724: fsync da/d27/d31/f5a 0 2024-04-25T01:35:57.935 INFO:tasks.workunit.client.0.smithi033.stdout:1/565: chown d2/d5/d75/c97 3 1 2024-04-25T01:35:57.936 INFO:tasks.workunit.client.0.smithi033.stdout:3/757: unlink d8/dd/d3d/d63/d7c/d4d/da5/cd8 0 2024-04-25T01:35:57.937 INFO:tasks.workunit.client.0.smithi033.stdout:9/710: getdents d0/d11 0 2024-04-25T01:35:57.938 INFO:tasks.workunit.client.1.smithi138.stdout:7/150: write da/de/d1b/f31 [1268323,112955] 0 2024-04-25T01:35:57.938 INFO:tasks.workunit.client.1.smithi138.stdout:4/126: symlink d0/d1c/l20 0 2024-04-25T01:35:57.939 INFO:tasks.workunit.client.0.smithi033.stdout:2/725: chown da/d50/d6a/f4b 783353 1 2024-04-25T01:35:57.941 INFO:tasks.workunit.client.1.smithi138.stdout:6/109: creat dc/d10/f25 x:0 0 0 2024-04-25T01:35:57.942 INFO:tasks.workunit.client.0.smithi033.stdout:3/758: rmdir d8/dd/d3d/d4f/da4/dc4 39 2024-04-25T01:35:57.944 INFO:tasks.workunit.client.0.smithi033.stdout:6/847: dread d4/d24/d36/d42/d97/db2/f11a [0,4194304] 0 2024-04-25T01:35:57.944 INFO:tasks.workunit.client.0.smithi033.stdout:9/711: symlink d0/d2/d43/d4b/dc5/de0/d42/lfb 0 2024-04-25T01:35:57.947 INFO:tasks.workunit.client.0.smithi033.stdout:2/726: dread - da/d27/d31/fe9 zero size 2024-04-25T01:35:57.948 INFO:tasks.workunit.client.0.smithi033.stdout:9/712: read - d0/d2/d43/d4b/dc5/de0/d7e/fe8 zero size 2024-04-25T01:35:57.949 INFO:tasks.workunit.client.1.smithi138.stdout:5/122: rename d4/da/f10 to d4/da/d1b/d1c/f26 0 2024-04-25T01:35:57.949 INFO:tasks.workunit.client.1.smithi138.stdout:8/357: dwrite d3/d16/f3b [0,4194304] 0 2024-04-25T01:35:57.949 INFO:tasks.workunit.client.1.smithi138.stdout:6/110: write dc/d10/f22 [1207129,43910] 0 2024-04-25T01:35:57.949 INFO:tasks.workunit.client.0.smithi033.stdout:3/759: symlink d8/dd/d3d/d63/d7c/d9d/de8/df9/l106 0 2024-04-25T01:35:57.951 INFO:tasks.workunit.client.0.smithi033.stdout:1/566: dread d2/d5/d75/f8d [0,4194304] 0 2024-04-25T01:35:57.951 INFO:tasks.workunit.client.0.smithi033.stdout:3/760: dread - d8/dd/d3d/dce/fee zero size 2024-04-25T01:35:57.953 INFO:tasks.workunit.client.0.smithi033.stdout:2/727: dread da/d5b/d5e/d76/f90 [0,4194304] 0 2024-04-25T01:35:57.955 INFO:tasks.workunit.client.0.smithi033.stdout:1/567: mknod d2/d11/d28/cac 0 2024-04-25T01:35:57.956 INFO:tasks.workunit.client.0.smithi033.stdout:9/713: truncate d0/d2/d43/d4b/dc5/de0/d7e/fe8 214236 0 2024-04-25T01:35:57.956 INFO:tasks.workunit.client.0.smithi033.stdout:2/728: chown da/d50/d6a/f56 0 1 2024-04-25T01:35:57.957 INFO:tasks.workunit.client.1.smithi138.stdout:7/151: dwrite da/de/d13/f23 [0,4194304] 0 2024-04-25T01:35:57.957 INFO:tasks.workunit.client.0.smithi033.stdout:6/848: dread d4/d1d/d3e/d41/d93/f104 [0,4194304] 0 2024-04-25T01:35:57.958 INFO:tasks.workunit.client.0.smithi033.stdout:9/714: stat d0/d2/dd/da7/fb5 0 2024-04-25T01:35:57.960 INFO:tasks.workunit.client.0.smithi033.stdout:2/729: chown da/d27/d31/d40/d8a/d80 47364 1 2024-04-25T01:35:57.962 INFO:tasks.workunit.client.1.smithi138.stdout:4/127: dwrite d0/d16/f18 [0,4194304] 0 2024-04-25T01:35:57.962 INFO:tasks.workunit.client.1.smithi138.stdout:8/358: write d3/f6d [1015665,59807] 0 2024-04-25T01:35:57.968 INFO:tasks.workunit.client.1.smithi138.stdout:8/359: read - d3/d16/d1f/d2a/f6a zero size 2024-04-25T01:35:57.969 INFO:tasks.workunit.client.0.smithi033.stdout:2/730: unlink da/d27/d31/d40/d77/fb4 0 2024-04-25T01:35:57.969 INFO:tasks.workunit.client.1.smithi138.stdout:5/123: dread d4/da/fd [0,4194304] 0 2024-04-25T01:35:57.970 INFO:tasks.workunit.client.1.smithi138.stdout:7/152: dwrite f9 [0,4194304] 0 2024-04-25T01:35:57.973 INFO:tasks.workunit.client.1.smithi138.stdout:7/153: write da/de/d1b/f31 [3536302,105224] 0 2024-04-25T01:35:57.974 INFO:tasks.workunit.client.0.smithi033.stdout:9/715: creat d0/d2/d43/d73/ffc x:0 0 0 2024-04-25T01:35:57.979 INFO:tasks.workunit.client.1.smithi138.stdout:8/360: dwrite d3/d8/da/db/d15/d20/f6e [0,4194304] 0 2024-04-25T01:35:57.979 INFO:tasks.workunit.client.0.smithi033.stdout:1/568: dwrite d2/d2e/d34/d60/f7a [4194304,4194304] 0 2024-04-25T01:35:57.980 INFO:tasks.workunit.client.0.smithi033.stdout:6/849: dwrite d4/d24/d36/d42/d9d/ff4 [0,4194304] 0 2024-04-25T01:35:57.984 INFO:tasks.workunit.client.0.smithi033.stdout:0/604: dwrite d7/d2d/d64/d51/f89 [4194304,4194304] 0 2024-04-25T01:35:57.988 INFO:tasks.workunit.client.0.smithi033.stdout:6/850: write d4/d1d/d3e/d41/f10c [614214,124500] 0 2024-04-25T01:35:57.990 INFO:tasks.workunit.client.0.smithi033.stdout:6/851: readlink d4/d1d/d3e/d4a/led 0 2024-04-25T01:35:57.993 INFO:tasks.workunit.client.0.smithi033.stdout:9/716: creat d0/d11/ffd x:0 0 0 2024-04-25T01:35:57.994 INFO:tasks.workunit.client.0.smithi033.stdout:1/569: mknod d2/d5/d1b/d46/d5f/d63/d64/cad 0 2024-04-25T01:35:58.000 INFO:tasks.workunit.client.0.smithi033.stdout:2/731: symlink da/d50/d6a/d43/lec 0 2024-04-25T01:35:58.002 INFO:tasks.workunit.client.0.smithi033.stdout:3/761: truncate d8/dd/d3d/d63/d7c/f96 5738200 0 2024-04-25T01:35:58.002 INFO:tasks.workunit.client.0.smithi033.stdout:6/852: dwrite d4/d24/d36/f6c [0,4194304] 0 2024-04-25T01:35:58.008 INFO:tasks.workunit.client.0.smithi033.stdout:2/732: dread da/f15 [0,4194304] 0 2024-04-25T01:35:58.010 INFO:tasks.workunit.client.0.smithi033.stdout:1/570: creat d2/d5/d75/d59/fae x:0 0 0 2024-04-25T01:35:58.012 INFO:tasks.workunit.client.0.smithi033.stdout:1/571: dread - d2/d11/d28/f7d zero size 2024-04-25T01:35:58.013 INFO:tasks.workunit.client.0.smithi033.stdout:9/717: read d0/d2/dd/d20/f38 [1004898,41139] 0 2024-04-25T01:35:58.014 INFO:tasks.workunit.client.0.smithi033.stdout:3/762: creat d8/dd/d1d/f107 x:0 0 0 2024-04-25T01:35:58.017 INFO:tasks.workunit.client.0.smithi033.stdout:1/572: symlink d2/d5/d6c/laf 0 2024-04-25T01:35:58.017 INFO:tasks.workunit.client.0.smithi033.stdout:9/718: dread d0/d11/da1/fd3 [0,4194304] 0 2024-04-25T01:35:58.018 INFO:tasks.workunit.client.1.smithi138.stdout:4/128: unlink d0/d16/l1b 0 2024-04-25T01:35:58.018 INFO:tasks.workunit.client.0.smithi033.stdout:2/733: fdatasync f8 0 2024-04-25T01:35:58.018 INFO:tasks.workunit.client.0.smithi033.stdout:9/719: dread - d0/d2/d43/d73/ffc zero size 2024-04-25T01:35:58.020 INFO:tasks.workunit.client.0.smithi033.stdout:9/720: dread - d0/d2/d43/d73/fd1 zero size 2024-04-25T01:35:58.021 INFO:tasks.workunit.client.0.smithi033.stdout:1/573: mkdir d2/d7f/db0 0 2024-04-25T01:35:58.022 INFO:tasks.workunit.client.0.smithi033.stdout:3/763: write d8/dd/d3d/fc8 [9385912,83956] 0 2024-04-25T01:35:58.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/154: mkdir da/de/d1b/d34 0 2024-04-25T01:35:58.026 INFO:tasks.workunit.client.0.smithi033.stdout:9/721: rmdir d0/d2/dd 39 2024-04-25T01:35:58.027 INFO:tasks.workunit.client.1.smithi138.stdout:4/129: dwrite d0/f1a [0,4194304] 0 2024-04-25T01:35:58.029 INFO:tasks.workunit.client.1.smithi138.stdout:5/124: dwrite d4/da/fd [4194304,4194304] 0 2024-04-25T01:35:58.032 INFO:tasks.workunit.client.1.smithi138.stdout:7/155: mknod da/de/d13/d30/c35 0 2024-04-25T01:35:58.032 INFO:tasks.workunit.client.1.smithi138.stdout:5/125: chown d4/da/d1b/d1c/d22 821614065 1 2024-04-25T01:35:58.033 INFO:tasks.workunit.client.0.smithi033.stdout:3/764: unlink d8/dd/d1d/f2b 0 2024-04-25T01:35:58.034 INFO:tasks.workunit.client.1.smithi138.stdout:1/154: write d3/f13 [1204390,110002] 0 2024-04-25T01:35:58.034 INFO:tasks.workunit.client.0.smithi033.stdout:3/765: stat c6 0 2024-04-25T01:35:58.035 INFO:tasks.workunit.client.1.smithi138.stdout:6/111: rmdir dc 39 2024-04-25T01:35:58.036 INFO:tasks.workunit.client.1.smithi138.stdout:2/244: dwrite d0/f31 [0,4194304] 0 2024-04-25T01:35:58.037 INFO:tasks.workunit.client.0.smithi033.stdout:9/722: symlink d0/d2/d43/d73/dc4/dcb/df0/lfe 0 2024-04-25T01:35:58.038 INFO:tasks.workunit.client.0.smithi033.stdout:1/574: dwrite d2/d11/f1e [4194304,4194304] 0 2024-04-25T01:35:58.039 INFO:tasks.workunit.client.1.smithi138.stdout:1/155: creat d3/d4/f37 x:0 0 0 2024-04-25T01:35:58.041 INFO:tasks.workunit.client.0.smithi033.stdout:3/766: stat d8/dd/d40/f65 0 2024-04-25T01:35:58.042 INFO:tasks.workunit.client.0.smithi033.stdout:3/767: chown d8/dd/d40/f65 2019 1 2024-04-25T01:35:58.042 INFO:tasks.workunit.client.1.smithi138.stdout:1/156: chown d3/db/cc 2672577 1 2024-04-25T01:35:58.044 INFO:tasks.workunit.client.1.smithi138.stdout:1/157: write d3/f13 [152348,67076] 0 2024-04-25T01:35:58.044 INFO:tasks.workunit.client.0.smithi033.stdout:0/605: dread d7/d2d/d4d/d9c/d61/f66 [0,4194304] 0 2024-04-25T01:35:58.044 INFO:tasks.workunit.client.1.smithi138.stdout:2/245: mkdir d0/d21/d36/d15/d44/d4c 0 2024-04-25T01:35:58.047 INFO:tasks.workunit.client.1.smithi138.stdout:5/126: getdents d4 0 2024-04-25T01:35:58.051 INFO:tasks.workunit.client.0.smithi033.stdout:9/723: creat d0/d2/d43/fff x:0 0 0 2024-04-25T01:35:58.051 INFO:tasks.workunit.client.1.smithi138.stdout:6/112: link dc/f1f dc/d10/f26 0 2024-04-25T01:35:58.054 INFO:tasks.workunit.client.1.smithi138.stdout:5/127: rename d4/da/d1b to d4/da/d27 0 2024-04-25T01:35:58.055 INFO:tasks.workunit.client.0.smithi033.stdout:2/734: write da/d27/d31/f46 [2990452,72689] 0 2024-04-25T01:35:58.055 INFO:tasks.workunit.client.1.smithi138.stdout:1/158: mkdir d3/d38 0 2024-04-25T01:35:58.057 INFO:tasks.workunit.client.1.smithi138.stdout:8/361: write d3/d8/da/db/d15/d20/f5a [891585,71023] 0 2024-04-25T01:35:58.059 INFO:tasks.workunit.client.1.smithi138.stdout:6/113: creat dc/d10/f27 x:0 0 0 2024-04-25T01:35:58.059 INFO:tasks.workunit.client.1.smithi138.stdout:8/362: chown d3/d8/da/db/d15/d20/f6e 562 1 2024-04-25T01:35:58.060 INFO:tasks.workunit.client.0.smithi033.stdout:3/768: mkdir d8/dd/d3d/d63/d7c/d9d/de8/d108 0 2024-04-25T01:35:58.061 INFO:tasks.workunit.client.0.smithi033.stdout:0/606: dwrite d7/d2d/d64/d51/f52 [0,4194304] 0 2024-04-25T01:35:58.062 INFO:tasks.workunit.client.1.smithi138.stdout:6/114: rmdir dc 39 2024-04-25T01:35:58.062 INFO:tasks.workunit.client.0.smithi033.stdout:1/575: mkdir d2/d5/d1b/d46/db1 0 2024-04-25T01:35:58.065 INFO:tasks.workunit.client.0.smithi033.stdout:0/607: write d7/da/f5d [28337,7101] 0 2024-04-25T01:35:58.065 INFO:tasks.workunit.client.1.smithi138.stdout:1/159: symlink d3/db/d2b/l39 0 2024-04-25T01:35:58.067 INFO:tasks.workunit.client.0.smithi033.stdout:2/735: mkdir da/d5b/d72/dd1/ded 0 2024-04-25T01:35:58.067 INFO:tasks.workunit.client.1.smithi138.stdout:8/363: dwrite f0 [0,4194304] 0 2024-04-25T01:35:58.068 INFO:tasks.workunit.client.0.smithi033.stdout:6/853: dwrite d4/d15/f48 [0,4194304] 0 2024-04-25T01:35:58.069 INFO:tasks.workunit.client.1.smithi138.stdout:6/115: truncate dc/f11 54000 0 2024-04-25T01:35:58.070 INFO:tasks.workunit.client.0.smithi033.stdout:3/769: creat d8/dd/d1d/dd1/f109 x:0 0 0 2024-04-25T01:35:58.070 INFO:tasks.workunit.client.0.smithi033.stdout:0/608: chown d7/d2d/c4a 27 1 2024-04-25T01:35:58.074 INFO:tasks.workunit.client.0.smithi033.stdout:1/576: symlink d2/d5/d1b/d46/d5f/d63/d64/lb2 0 2024-04-25T01:35:58.074 INFO:tasks.workunit.client.0.smithi033.stdout:0/609: truncate d7/d2d/d64/fad 1205030 0 2024-04-25T01:35:58.075 INFO:tasks.workunit.client.1.smithi138.stdout:1/160: symlink d3/d27/d16/l3a 0 2024-04-25T01:35:58.076 INFO:tasks.workunit.client.0.smithi033.stdout:0/610: write d7/d2d/d4d/d9c/fae [286804,58961] 0 2024-04-25T01:35:58.077 INFO:tasks.workunit.client.0.smithi033.stdout:8/732: sync 2024-04-25T01:35:58.077 INFO:tasks.workunit.client.0.smithi033.stdout:4/771: sync 2024-04-25T01:35:58.077 INFO:tasks.workunit.client.0.smithi033.stdout:7/693: sync 2024-04-25T01:35:58.077 INFO:tasks.workunit.client.1.smithi138.stdout:6/116: dread dc/d10/f15 [0,4194304] 0 2024-04-25T01:35:58.081 INFO:tasks.workunit.client.0.smithi033.stdout:3/770: truncate d8/dd/d40/f53 1847435 0 2024-04-25T01:35:58.081 INFO:tasks.workunit.client.1.smithi138.stdout:2/246: truncate d0/d21/d36/d15/d1f/f37 1861140 0 2024-04-25T01:35:58.082 INFO:tasks.workunit.client.1.smithi138.stdout:8/364: dread d3/d8/da/db/d15/f21 [0,4194304] 0 2024-04-25T01:35:58.082 INFO:tasks.workunit.client.0.smithi033.stdout:7/694: mknod d4/d23/cda 0 2024-04-25T01:35:58.082 INFO:tasks.workunit.client.0.smithi033.stdout:6/854: read d4/d24/d63/f9a [102192,77183] 0 2024-04-25T01:35:58.082 INFO:tasks.workunit.client.1.smithi138.stdout:6/117: mkdir dc/d10/d28 0 2024-04-25T01:35:58.083 INFO:tasks.workunit.client.1.smithi138.stdout:6/118: chown dc/f16 543 1 2024-04-25T01:35:58.084 INFO:tasks.workunit.client.1.smithi138.stdout:6/119: stat dc/f16 0 2024-04-25T01:35:58.085 INFO:tasks.workunit.client.1.smithi138.stdout:2/247: mknod d0/d21/d45/c4d 0 2024-04-25T01:35:58.086 INFO:tasks.workunit.client.0.smithi033.stdout:1/577: mknod d2/d7f/cb3 0 2024-04-25T01:35:58.089 INFO:tasks.workunit.client.0.smithi033.stdout:4/772: mknod d6/d14/d35/d44/d45/d46/d57/d70/d87/cf7 0 2024-04-25T01:35:58.090 INFO:tasks.workunit.client.1.smithi138.stdout:8/365: rename d3/d8/da/db/d15/d5d/c5f to d3/d16/d1f/d2a/d4f/c71 0 2024-04-25T01:35:58.091 INFO:tasks.workunit.client.1.smithi138.stdout:5/128: truncate d4/da/fd 4894895 0 2024-04-25T01:35:58.092 INFO:tasks.workunit.client.0.smithi033.stdout:0/611: getdents d7/d2d/d4d/d9c/d68/dd2 0 2024-04-25T01:35:58.092 INFO:tasks.workunit.client.1.smithi138.stdout:6/120: creat dc/d10/d28/f29 x:0 0 0 2024-04-25T01:35:58.092 INFO:tasks.workunit.client.1.smithi138.stdout:5/129: truncate d4/f1a 932652 0 2024-04-25T01:35:58.094 INFO:tasks.workunit.client.1.smithi138.stdout:8/366: creat d3/d8/da/db/d15/f72 x:0 0 0 2024-04-25T01:35:58.095 INFO:tasks.workunit.client.0.smithi033.stdout:6/855: symlink d4/df1/l124 0 2024-04-25T01:35:58.095 INFO:tasks.workunit.client.1.smithi138.stdout:6/121: rename dc/ff to dc/d10/d28/f2a 0 2024-04-25T01:35:58.096 INFO:tasks.workunit.client.0.smithi033.stdout:4/773: symlink d6/d14/d1d/lf8 0 2024-04-25T01:35:58.097 INFO:tasks.workunit.client.1.smithi138.stdout:8/367: symlink d3/l73 0 2024-04-25T01:35:58.098 INFO:tasks.workunit.client.0.smithi033.stdout:1/578: dwrite d2/d5/d1b/d46/d5f/d63/d64/f6a [0,4194304] 0 2024-04-25T01:35:58.099 INFO:tasks.workunit.client.1.smithi138.stdout:6/122: creat dc/d10/d28/f2b x:0 0 0 2024-04-25T01:35:58.100 INFO:tasks.workunit.client.0.smithi033.stdout:4/774: stat d6/d14/d35/d44/d45/d46/d57/d70 0 2024-04-25T01:35:58.100 INFO:tasks.workunit.client.0.smithi033.stdout:0/612: symlink d7/d2d/d43/dc0/ld3 0 2024-04-25T01:35:58.101 INFO:tasks.workunit.client.0.smithi033.stdout:4/775: stat d6/d14 0 2024-04-25T01:35:58.103 INFO:tasks.workunit.client.0.smithi033.stdout:3/771: creat d8/dd/d3d/d63/f10a x:0 0 0 2024-04-25T01:35:58.103 INFO:tasks.workunit.client.1.smithi138.stdout:2/248: dread d0/d21/f27 [0,4194304] 0 2024-04-25T01:35:58.104 INFO:tasks.workunit.client.1.smithi138.stdout:2/249: chown d0/d21/d45/c4d 13835 1 2024-04-25T01:35:58.106 INFO:tasks.workunit.client.0.smithi033.stdout:0/613: readlink d7/d2d/d4d/d9c/l53 0 2024-04-25T01:35:58.108 INFO:tasks.workunit.client.1.smithi138.stdout:2/250: dread d0/f31 [0,4194304] 0 2024-04-25T01:35:58.112 INFO:tasks.workunit.client.0.smithi033.stdout:3/772: write d8/dd/d3d/d4f/f6f [390045,20981] 0 2024-04-25T01:35:58.113 INFO:tasks.workunit.client.0.smithi033.stdout:0/614: dread - d7/d2d/d4d/d9c/dcf/fb1 zero size 2024-04-25T01:35:58.116 INFO:tasks.workunit.client.0.smithi033.stdout:4/776: mkdir d6/d14/d35/d3a/d4d/d5e/d7c/df9 0 2024-04-25T01:35:58.118 INFO:tasks.workunit.client.0.smithi033.stdout:1/579: dwrite d2/d5/d1b/d46/f69 [0,4194304] 0 2024-04-25T01:35:58.119 INFO:tasks.workunit.client.0.smithi033.stdout:7/695: dread d4/d23/fc5 [0,4194304] 0 2024-04-25T01:35:58.123 INFO:tasks.workunit.client.0.smithi033.stdout:4/777: creat d6/ffa x:0 0 0 2024-04-25T01:35:58.123 INFO:tasks.workunit.client.1.smithi138.stdout:6/123: truncate dc/d10/f22 978712 0 2024-04-25T01:35:58.125 INFO:tasks.workunit.client.0.smithi033.stdout:9/724: dread d0/d2/d43/d4b/f92 [0,4194304] 0 2024-04-25T01:35:58.125 INFO:tasks.workunit.client.1.smithi138.stdout:6/124: read - dc/d10/d28/f29 zero size 2024-04-25T01:35:58.125 INFO:tasks.workunit.client.1.smithi138.stdout:8/368: dwrite d3/f3c [0,4194304] 0 2024-04-25T01:35:58.127 INFO:tasks.workunit.client.0.smithi033.stdout:3/773: fdatasync d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/fc1 0 2024-04-25T01:35:58.128 INFO:tasks.workunit.client.1.smithi138.stdout:6/125: mknod dc/d10/c2c 0 2024-04-25T01:35:58.129 INFO:tasks.workunit.client.0.smithi033.stdout:2/736: dwrite da/f20 [0,4194304] 0 2024-04-25T01:35:58.132 INFO:tasks.workunit.client.1.smithi138.stdout:6/126: mknod dc/d10/c2d 0 2024-04-25T01:35:58.135 INFO:tasks.workunit.client.1.smithi138.stdout:6/127: mkdir dc/d2e 0 2024-04-25T01:35:58.135 INFO:tasks.workunit.client.1.smithi138.stdout:6/128: chown dc/d10 118146 1 2024-04-25T01:35:58.136 INFO:tasks.workunit.client.0.smithi033.stdout:6/856: dread d4/d1d/d3e/d41/d5c/fb7 [0,4194304] 0 2024-04-25T01:35:58.137 INFO:tasks.workunit.client.0.smithi033.stdout:4/778: symlink d6/d14/d35/d44/d45/d46/d64/lfb 0 2024-04-25T01:35:58.137 INFO:tasks.workunit.client.1.smithi138.stdout:2/251: truncate d0/d18/f1b 2615887 0 2024-04-25T01:35:58.138 INFO:tasks.workunit.client.0.smithi033.stdout:0/615: read d7/d2d/d4d/d9c/f59 [15515,78545] 0 2024-04-25T01:35:58.140 INFO:tasks.workunit.client.1.smithi138.stdout:6/129: rename dc/f14 to dc/d2e/f2f 0 2024-04-25T01:35:58.140 INFO:tasks.workunit.client.1.smithi138.stdout:1/161: dread d3/f15 [0,4194304] 0 2024-04-25T01:35:58.141 INFO:tasks.workunit.client.1.smithi138.stdout:1/162: fsync d3/d27/d2d/f32 0 2024-04-25T01:35:58.142 INFO:tasks.workunit.client.0.smithi033.stdout:1/580: rmdir d2/d84 0 2024-04-25T01:35:58.143 INFO:tasks.workunit.client.1.smithi138.stdout:2/252: read d0/d21/d36/f12 [1192816,83156] 0 2024-04-25T01:35:58.145 INFO:tasks.workunit.client.0.smithi033.stdout:0/616: mkdir d7/d2d/d43/d44/dd4 0 2024-04-25T01:35:58.147 INFO:tasks.workunit.client.1.smithi138.stdout:2/253: creat d0/d21/d36/f4e x:0 0 0 2024-04-25T01:35:58.148 INFO:tasks.workunit.client.1.smithi138.stdout:2/254: chown d0/d32/l42 0 1 2024-04-25T01:35:58.148 INFO:tasks.workunit.client.1.smithi138.stdout:1/163: creat d3/d27/d24/f3b x:0 0 0 2024-04-25T01:35:58.149 INFO:tasks.workunit.client.1.smithi138.stdout:5/130: truncate d4/da/fd 1540877 0 2024-04-25T01:35:58.150 INFO:tasks.workunit.client.1.smithi138.stdout:2/255: write d0/d21/d36/f4e [648525,69754] 0 2024-04-25T01:35:58.150 INFO:tasks.workunit.client.0.smithi033.stdout:7/696: rename d4/f43 to d4/d10/fdb 0 2024-04-25T01:35:58.150 INFO:tasks.workunit.client.1.smithi138.stdout:1/164: dread - d3/d27/d2d/f30 zero size 2024-04-25T01:35:58.151 INFO:tasks.workunit.client.0.smithi033.stdout:9/725: dwrite d0/d2/dd/fde [0,4194304] 0 2024-04-25T01:35:58.151 INFO:tasks.workunit.client.1.smithi138.stdout:1/165: chown d3/d27/d16 0 1 2024-04-25T01:35:58.153 INFO:tasks.workunit.client.1.smithi138.stdout:5/131: creat d4/da/d27/d1c/f28 x:0 0 0 2024-04-25T01:35:58.153 INFO:tasks.workunit.client.0.smithi033.stdout:7/697: readlink d4/d23/l27 0 2024-04-25T01:35:58.154 INFO:tasks.workunit.client.0.smithi033.stdout:4/779: symlink d6/d14/d35/d3a/d4c/lfc 0 2024-04-25T01:35:58.156 INFO:tasks.workunit.client.0.smithi033.stdout:1/581: creat d2/d5/d75/d59/d83/fb4 x:0 0 0 2024-04-25T01:35:58.160 INFO:tasks.workunit.client.0.smithi033.stdout:3/774: dread d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/f8a [0,4194304] 0 2024-04-25T01:35:58.161 INFO:tasks.workunit.client.0.smithi033.stdout:2/737: write da/d27/d31/d40/d8a/d80/fab [385165,61501] 0 2024-04-25T01:35:58.163 INFO:tasks.workunit.client.0.smithi033.stdout:6/857: dwrite d4/d24/d36/d42/d97/de0/ff2 [0,4194304] 0 2024-04-25T01:35:58.164 INFO:tasks.workunit.client.0.smithi033.stdout:9/726: mknod d0/d2/d43/d4b/dc5/c100 0 2024-04-25T01:35:58.165 INFO:tasks.workunit.client.0.smithi033.stdout:1/582: write d2/d2e/f71 [4899038,91959] 0 2024-04-25T01:35:58.167 INFO:tasks.workunit.client.1.smithi138.stdout:9/181: sync 2024-04-25T01:35:58.169 INFO:tasks.workunit.client.1.smithi138.stdout:8/369: dwrite d3/d8/da/de/f10 [0,4194304] 0 2024-04-25T01:35:58.169 INFO:tasks.workunit.client.0.smithi033.stdout:0/617: dwrite d7/d2d/d43/d44/f63 [0,4194304] 0 2024-04-25T01:35:58.170 INFO:tasks.workunit.client.0.smithi033.stdout:6/858: mknod d4/dfc/c125 0 2024-04-25T01:35:58.173 INFO:tasks.workunit.client.0.smithi033.stdout:3/775: mkdir d8/dd/d3d/d63/d7c/ded/dc7/ddf/d10b 0 2024-04-25T01:35:58.175 INFO:tasks.workunit.client.1.smithi138.stdout:9/182: creat d4/d1a/f3b x:0 0 0 2024-04-25T01:35:58.176 INFO:tasks.workunit.client.0.smithi033.stdout:0/618: chown d7/da/l21 11231102 1 2024-04-25T01:35:58.179 INFO:tasks.workunit.client.1.smithi138.stdout:9/183: truncate d4/f38 1031783 0 2024-04-25T01:35:58.179 INFO:tasks.workunit.client.1.smithi138.stdout:5/132: mknod d4/da/d27/d1c/c29 0 2024-04-25T01:35:58.181 INFO:tasks.workunit.client.1.smithi138.stdout:5/133: chown d4/c16 0 1 2024-04-25T01:35:58.182 INFO:tasks.workunit.client.0.smithi033.stdout:7/698: dwrite d4/d23/d3f/f6f [4194304,4194304] 0 2024-04-25T01:35:58.182 INFO:tasks.workunit.client.1.smithi138.stdout:9/184: chown d4/d1a/d33/d1d/l32 1133 1 2024-04-25T01:35:58.184 INFO:tasks.workunit.client.0.smithi033.stdout:2/738: mknod da/d27/d31/d40/d8a/dbb/dd6/de1/cee 0 2024-04-25T01:35:58.185 INFO:tasks.workunit.client.1.smithi138.stdout:8/370: dwrite d3/d8/f60 [0,4194304] 0 2024-04-25T01:35:58.186 INFO:tasks.workunit.client.0.smithi033.stdout:7/699: fdatasync d4/d2e/ff 0 2024-04-25T01:35:58.186 INFO:tasks.workunit.client.1.smithi138.stdout:9/185: write d4/f25 [277171,78872] 0 2024-04-25T01:35:58.187 INFO:tasks.workunit.client.1.smithi138.stdout:2/256: dread d0/d21/d36/d15/f28 [0,4194304] 0 2024-04-25T01:35:58.192 INFO:tasks.workunit.client.0.smithi033.stdout:9/727: dwrite d0/d2/dd/df/d1b/f99 [0,4194304] 0 2024-04-25T01:35:58.193 INFO:tasks.workunit.client.0.smithi033.stdout:1/583: dwrite d2/d2e/f67 [0,4194304] 0 2024-04-25T01:35:58.196 INFO:tasks.workunit.client.0.smithi033.stdout:7/700: write d4/d2e/f1c [1922053,22233] 0 2024-04-25T01:35:58.196 INFO:tasks.workunit.client.1.smithi138.stdout:6/130: dwrite dc/d10/f15 [0,4194304] 0 2024-04-25T01:35:58.196 INFO:tasks.workunit.client.0.smithi033.stdout:2/739: mknod da/d5b/d72/dd1/cef 0 2024-04-25T01:35:58.197 INFO:tasks.workunit.client.0.smithi033.stdout:6/859: link d4/lb8 d4/d1d/d3e/d41/d5c/df6/l126 0 2024-04-25T01:35:58.198 INFO:tasks.workunit.client.1.smithi138.stdout:6/131: truncate dc/d10/f25 91333 0 2024-04-25T01:35:58.202 INFO:tasks.workunit.client.0.smithi033.stdout:1/584: dread d2/d5/d75/f53 [0,4194304] 0 2024-04-25T01:35:58.204 INFO:tasks.workunit.client.0.smithi033.stdout:2/740: creat da/d27/d31/d40/d8a/d80/ff0 x:0 0 0 2024-04-25T01:35:58.204 INFO:tasks.workunit.client.0.smithi033.stdout:9/728: getdents d0/d2/dd/dba/dbe 0 2024-04-25T01:35:58.205 INFO:tasks.workunit.client.0.smithi033.stdout:2/741: fdatasync da/f12 0 2024-04-25T01:35:58.206 INFO:tasks.workunit.client.1.smithi138.stdout:5/134: rename d4/da/l21 to d4/da/d27/d1c/l2a 0 2024-04-25T01:35:58.207 INFO:tasks.workunit.client.0.smithi033.stdout:3/776: read d8/dd/d3d/d4f/fa0 [699932,70758] 0 2024-04-25T01:35:58.207 INFO:tasks.workunit.client.0.smithi033.stdout:7/701: link d4/d2e/d6b/f62 d4/d57/db7/fdc 0 2024-04-25T01:35:58.211 INFO:tasks.workunit.client.0.smithi033.stdout:1/585: creat d2/d5/d6c/d94/fb5 x:0 0 0 2024-04-25T01:35:58.212 INFO:tasks.workunit.client.0.smithi033.stdout:9/729: write d0/d11/da1/fdf [634797,54380] 0 2024-04-25T01:35:58.215 INFO:tasks.workunit.client.1.smithi138.stdout:9/186: dread d4/f25 [0,4194304] 0 2024-04-25T01:35:58.215 INFO:tasks.workunit.client.1.smithi138.stdout:8/371: dwrite d3/d8/da/db/f14 [4194304,4194304] 0 2024-04-25T01:35:58.215 INFO:tasks.workunit.client.1.smithi138.stdout:1/166: dread d3/f34 [0,4194304] 0 2024-04-25T01:35:58.219 INFO:tasks.workunit.client.0.smithi033.stdout:7/702: creat d4/d2e/d7f/d98/dca/fdd x:0 0 0 2024-04-25T01:35:58.219 INFO:tasks.workunit.client.0.smithi033.stdout:5/743: sync 2024-04-25T01:35:58.220 INFO:tasks.workunit.client.1.smithi138.stdout:1/167: write d3/d27/d16/f18 [1385551,107969] 0 2024-04-25T01:35:58.220 INFO:tasks.workunit.client.1.smithi138.stdout:9/187: stat d4/d8/f23 0 2024-04-25T01:35:58.221 INFO:tasks.workunit.client.0.smithi033.stdout:0/619: truncate d7/da/df/d14/f15 648597 0 2024-04-25T01:35:58.222 INFO:tasks.workunit.client.0.smithi033.stdout:9/730: read d0/d2/d43/d73/f9f [1645991,3279] 0 2024-04-25T01:35:58.222 INFO:tasks.workunit.client.0.smithi033.stdout:6/860: dwrite d4/d24/d36/d42/d97/db2/f10a [0,4194304] 0 2024-04-25T01:35:58.223 INFO:tasks.workunit.client.1.smithi138.stdout:1/168: chown d3/d27/d2d/f2f 64577561 1 2024-04-25T01:35:58.223 INFO:tasks.workunit.client.0.smithi033.stdout:2/742: dwrite da/d5b/d72/fbe [0,4194304] 0 2024-04-25T01:35:58.228 INFO:tasks.workunit.client.1.smithi138.stdout:9/188: mkdir d4/d1a/d3c 0 2024-04-25T01:35:58.230 INFO:tasks.workunit.client.0.smithi033.stdout:2/743: mkdir da/d50/d6a/d43/df1 0 2024-04-25T01:35:58.231 INFO:tasks.workunit.client.0.smithi033.stdout:0/620: creat d7/d2d/d43/d44/fd5 x:0 0 0 2024-04-25T01:35:58.232 INFO:tasks.workunit.client.1.smithi138.stdout:1/169: link c0 d3/d4/c3c 0 2024-04-25T01:35:58.232 INFO:tasks.workunit.client.1.smithi138.stdout:1/170: fsync d3/d27/f36 0 2024-04-25T01:35:58.233 INFO:tasks.workunit.client.0.smithi033.stdout:2/744: stat da/d27/d31/dca 0 2024-04-25T01:35:58.233 INFO:tasks.workunit.client.1.smithi138.stdout:9/189: symlink d4/d1a/d3c/l3d 0 2024-04-25T01:35:58.234 INFO:tasks.workunit.client.1.smithi138.stdout:1/171: write d3/db/f1d [1311406,64706] 0 2024-04-25T01:35:58.235 INFO:tasks.workunit.client.0.smithi033.stdout:7/703: fdatasync d4/f11 0 2024-04-25T01:35:58.235 INFO:tasks.workunit.client.0.smithi033.stdout:0/621: creat d7/d2d/d4d/fd6 x:0 0 0 2024-04-25T01:35:58.235 INFO:tasks.workunit.client.0.smithi033.stdout:2/745: chown da/d5b/d5e/da8/cac 64 1 2024-04-25T01:35:58.237 INFO:tasks.workunit.client.1.smithi138.stdout:1/172: truncate d3/d27/d16/f28 988550 0 2024-04-25T01:35:58.237 INFO:tasks.workunit.client.0.smithi033.stdout:5/744: fsync d1c/d2a/d50/d52/da6/f6d 0 2024-04-25T01:35:58.237 INFO:tasks.workunit.client.0.smithi033.stdout:1/586: dwrite d2/d11/fa6 [0,4194304] 0 2024-04-25T01:35:58.239 INFO:tasks.workunit.client.0.smithi033.stdout:3/777: unlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/ff4 0 2024-04-25T01:35:58.243 INFO:tasks.workunit.client.0.smithi033.stdout:1/587: read - d2/d5/d75/d59/d83/fb4 zero size 2024-04-25T01:35:58.244 INFO:tasks.workunit.client.1.smithi138.stdout:6/132: dread f8 [0,4194304] 0 2024-04-25T01:35:58.245 INFO:tasks.workunit.client.0.smithi033.stdout:0/622: write d7/da/df/f9a [1488257,6813] 0 2024-04-25T01:35:58.245 INFO:tasks.workunit.client.1.smithi138.stdout:9/190: creat d4/f3e x:0 0 0 2024-04-25T01:35:58.245 INFO:tasks.workunit.client.1.smithi138.stdout:6/133: dread - dc/f1f zero size 2024-04-25T01:35:58.246 INFO:tasks.workunit.client.1.smithi138.stdout:9/191: fdatasync d4/d1a/f3b 0 2024-04-25T01:35:58.246 INFO:tasks.workunit.client.0.smithi033.stdout:2/746: dread - da/d27/d31/dca/fdb zero size 2024-04-25T01:35:58.247 INFO:tasks.workunit.client.1.smithi138.stdout:9/192: write d4/d1a/f21 [637074,36928] 0 2024-04-25T01:35:58.249 INFO:tasks.workunit.client.1.smithi138.stdout:9/193: write d4/d1a/f3b [211604,129359] 0 2024-04-25T01:35:58.249 INFO:tasks.workunit.client.0.smithi033.stdout:7/704: mknod d4/d2e/daa/cde 0 2024-04-25T01:35:58.250 INFO:tasks.workunit.client.1.smithi138.stdout:6/134: rename dc/d10/c24 to dc/c30 0 2024-04-25T01:35:58.251 INFO:tasks.workunit.client.1.smithi138.stdout:9/194: fdatasync d4/d8/f23 0 2024-04-25T01:35:58.252 INFO:tasks.workunit.client.1.smithi138.stdout:9/195: truncate f1 2080475 0 2024-04-25T01:35:58.253 INFO:tasks.workunit.client.0.smithi033.stdout:3/778: rmdir d8/dd/d3d/d63/d7c/d4d/da5 39 2024-04-25T01:35:58.253 INFO:tasks.workunit.client.1.smithi138.stdout:6/135: creat dc/d2e/f31 x:0 0 0 2024-04-25T01:35:58.255 INFO:tasks.workunit.client.0.smithi033.stdout:8/733: sync 2024-04-25T01:35:58.256 INFO:tasks.workunit.client.0.smithi033.stdout:0/623: mkdir d7/d2d/d4c/da0/da1/dd7 0 2024-04-25T01:35:58.257 INFO:tasks.workunit.client.0.smithi033.stdout:7/705: creat d4/d2e/d6d/fdf x:0 0 0 2024-04-25T01:35:58.260 INFO:tasks.workunit.client.1.smithi138.stdout:6/136: dwrite dc/d10/d28/f2b [0,4194304] 0 2024-04-25T01:35:58.260 INFO:tasks.workunit.client.0.smithi033.stdout:6/861: rename d4/d1d/d3e/d41/d5c/df6/l126 to d4/d1d/d3e/d41/d54/da1/l127 0 2024-04-25T01:35:58.265 INFO:tasks.workunit.client.0.smithi033.stdout:6/862: dread d4/d2a/f39 [0,4194304] 0 2024-04-25T01:35:58.267 INFO:tasks.workunit.client.0.smithi033.stdout:2/747: rename da/d27/d31/d40/f82 to da/d50/ff2 0 2024-04-25T01:35:58.271 INFO:tasks.workunit.client.0.smithi033.stdout:5/745: rmdir d1c/d2a/d50/d52/da6/d3e/d70/dae/df1/df4 0 2024-04-25T01:35:58.272 INFO:tasks.workunit.client.0.smithi033.stdout:4/780: dread d6/f16 [0,4194304] 0 2024-04-25T01:35:58.273 INFO:tasks.workunit.client.0.smithi033.stdout:0/624: mknod d7/d2d/d43/d44/dd4/cd8 0 2024-04-25T01:35:58.278 INFO:tasks.workunit.client.0.smithi033.stdout:6/863: chown d4/d24/l9c 10735249 1 2024-04-25T01:35:58.279 INFO:tasks.workunit.client.0.smithi033.stdout:2/748: dwrite da/d27/fb2 [0,4194304] 0 2024-04-25T01:35:58.279 INFO:tasks.workunit.client.0.smithi033.stdout:4/781: truncate d6/d14/d35/d3a/d4d/d5e/d7c/d94/fd2 136328 0 2024-04-25T01:35:58.282 INFO:tasks.workunit.client.1.smithi138.stdout:9/196: truncate d4/d8/f1f 5181138 0 2024-04-25T01:35:58.283 INFO:tasks.workunit.client.0.smithi033.stdout:9/731: dread d0/d2/d43/d4b/dc5/de0/d8f/f98 [0,4194304] 0 2024-04-25T01:35:58.283 INFO:tasks.workunit.client.1.smithi138.stdout:6/137: chown dc/c30 301 1 2024-04-25T01:35:58.284 INFO:tasks.workunit.client.0.smithi033.stdout:5/746: mknod d1c/d2a/d50/d8d/cf5 0 2024-04-25T01:35:58.285 INFO:tasks.workunit.client.1.smithi138.stdout:9/197: chown d4/d1a/f21 0 1 2024-04-25T01:35:58.285 INFO:tasks.workunit.client.1.smithi138.stdout:2/257: dwrite d0/f10 [0,4194304] 0 2024-04-25T01:35:58.287 INFO:tasks.workunit.client.0.smithi033.stdout:2/749: creat da/d5b/ff3 x:0 0 0 2024-04-25T01:35:58.287 INFO:tasks.workunit.client.0.smithi033.stdout:2/750: dread - da/d5b/ff3 zero size 2024-04-25T01:35:58.288 INFO:tasks.workunit.client.1.smithi138.stdout:2/258: readlink d0/d21/d36/lf 0 2024-04-25T01:35:58.288 INFO:tasks.workunit.client.0.smithi033.stdout:4/782: mkdir d6/dfd 0 2024-04-25T01:35:58.288 INFO:tasks.workunit.client.1.smithi138.stdout:1/173: dwrite d3/d4/f9 [0,4194304] 0 2024-04-25T01:35:58.290 INFO:tasks.workunit.client.1.smithi138.stdout:5/135: dread d4/da/fd [0,4194304] 0 2024-04-25T01:35:58.292 INFO:tasks.workunit.client.0.smithi033.stdout:6/864: creat d4/d1d/d3e/d41/d5c/d7f/da4/dcc/f128 x:0 0 0 2024-04-25T01:35:58.292 INFO:tasks.workunit.client.1.smithi138.stdout:9/198: mkdir d4/d1a/d33/d1d/d3f 0 2024-04-25T01:35:58.292 INFO:tasks.workunit.client.1.smithi138.stdout:5/136: stat d4/da/fe 0 2024-04-25T01:35:58.293 INFO:tasks.workunit.client.1.smithi138.stdout:2/259: rmdir d0/d32 39 2024-04-25T01:35:58.293 INFO:tasks.workunit.client.1.smithi138.stdout:5/137: write d4/f1a [394078,110982] 0 2024-04-25T01:35:58.295 INFO:tasks.workunit.client.0.smithi033.stdout:4/783: dwrite d6/d14/f2b [0,4194304] 0 2024-04-25T01:35:58.295 INFO:tasks.workunit.client.0.smithi033.stdout:3/779: rename d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/f21 to d8/dd/d3d/d4f/da4/f10c 0 2024-04-25T01:35:58.296 INFO:tasks.workunit.client.1.smithi138.stdout:1/174: mkdir d3/db/d2b/d3d 0 2024-04-25T01:35:58.297 INFO:tasks.workunit.client.1.smithi138.stdout:2/260: fdatasync d0/d21/d36/d15/f28 0 2024-04-25T01:35:58.302 INFO:tasks.workunit.client.1.smithi138.stdout:5/138: dread d4/f23 [0,4194304] 0 2024-04-25T01:35:58.304 INFO:tasks.workunit.client.1.smithi138.stdout:5/139: stat d4/da 0 2024-04-25T01:35:58.305 INFO:tasks.workunit.client.0.smithi033.stdout:6/865: unlink d4/d15/f48 0 2024-04-25T01:35:58.306 INFO:tasks.workunit.client.0.smithi033.stdout:1/588: write d2/d2e/f73 [379745,17841] 0 2024-04-25T01:35:58.307 INFO:tasks.workunit.client.1.smithi138.stdout:9/199: getdents d4/d8/d2c 0 2024-04-25T01:35:58.307 INFO:tasks.workunit.client.0.smithi033.stdout:3/780: mknod d8/dd/d3d/d63/d7c/d9d/c10d 0 2024-04-25T01:35:58.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/261: dwrite d0/d21/f22 [0,4194304] 0 2024-04-25T01:35:58.310 INFO:tasks.workunit.client.0.smithi033.stdout:5/747: dwrite d1c/d2a/d50/d52/da6/d3e/d70/dae/fb5 [0,4194304] 0 2024-04-25T01:35:58.310 INFO:tasks.workunit.client.0.smithi033.stdout:1/589: write d2/d2e/f71 [8631660,116623] 0 2024-04-25T01:35:58.312 INFO:tasks.workunit.client.0.smithi033.stdout:4/784: mkdir d6/d14/d35/d44/dfe 0 2024-04-25T01:35:58.316 INFO:tasks.workunit.client.0.smithi033.stdout:7/706: rename d4/d2e/d6b/d1f/f36 to d4/d23/d3f/d45/d73/dbe/fe0 0 2024-04-25T01:35:58.317 INFO:tasks.workunit.client.0.smithi033.stdout:8/734: truncate d6/d45/d51/d5d/d6e/f77 4126693 0 2024-04-25T01:35:58.324 INFO:tasks.workunit.client.1.smithi138.stdout:2/262: mkdir d0/d32/d35/d4f 0 2024-04-25T01:35:58.325 INFO:tasks.workunit.client.0.smithi033.stdout:1/590: dwrite d2/d2e/f67 [0,4194304] 0 2024-04-25T01:35:58.325 INFO:tasks.workunit.client.1.smithi138.stdout:5/140: dread d4/f1a [0,4194304] 0 2024-04-25T01:35:58.326 INFO:tasks.workunit.client.1.smithi138.stdout:9/200: rename d4/d1a/d33/d1d/l24 to d4/d8/l40 0 2024-04-25T01:35:58.327 INFO:tasks.workunit.client.0.smithi033.stdout:0/625: rename d7/d2d/c2e to d7/d2d/d43/cd9 0 2024-04-25T01:35:58.329 INFO:tasks.workunit.client.0.smithi033.stdout:3/781: creat d8/dd/dfa/f10e x:0 0 0 2024-04-25T01:35:58.330 INFO:tasks.workunit.client.0.smithi033.stdout:5/748: truncate d1c/d2a/d3d/d59/f9e 998330 0 2024-04-25T01:35:58.330 INFO:tasks.workunit.client.1.smithi138.stdout:5/141: rmdir d4/da/d27/d1c/d22 39 2024-04-25T01:35:58.331 INFO:tasks.workunit.client.0.smithi033.stdout:7/707: dwrite d4/d23/d3f/f77 [0,4194304] 0 2024-04-25T01:35:58.331 INFO:tasks.workunit.client.0.smithi033.stdout:1/591: fdatasync d2/d5/d75/d59/d83/fa9 0 2024-04-25T01:35:58.331 INFO:tasks.workunit.client.1.smithi138.stdout:5/142: write d4/f1a [1027002,109854] 0 2024-04-25T01:35:58.332 INFO:tasks.workunit.client.0.smithi033.stdout:0/626: dread - d7/d2d/d4d/d9c/dcf/fb1 zero size 2024-04-25T01:35:58.332 INFO:tasks.workunit.client.0.smithi033.stdout:8/735: symlink d6/db/d80/l113 0 2024-04-25T01:35:58.335 INFO:tasks.workunit.client.0.smithi033.stdout:5/749: mkdir d1c/d53/df6 0 2024-04-25T01:35:58.335 INFO:tasks.workunit.client.0.smithi033.stdout:8/736: chown d6/db/f24 0 1 2024-04-25T01:35:58.336 INFO:tasks.workunit.client.0.smithi033.stdout:7/708: stat d4/d23/da7/dc2/dd0 0 2024-04-25T01:35:58.336 INFO:tasks.workunit.client.0.smithi033.stdout:0/627: dread - d7/da/df/d50/fb3 zero size 2024-04-25T01:35:58.336 INFO:tasks.workunit.client.0.smithi033.stdout:1/592: read - d2/d5/d75/d59/d83/fb4 zero size 2024-04-25T01:35:58.338 INFO:tasks.workunit.client.0.smithi033.stdout:5/750: fdatasync d1c/d2a/fb2 0 2024-04-25T01:35:58.338 INFO:tasks.workunit.client.1.smithi138.stdout:9/201: dwrite d4/f26 [0,4194304] 0 2024-04-25T01:35:58.339 INFO:tasks.workunit.client.1.smithi138.stdout:3/169: sync 2024-04-25T01:35:58.340 INFO:tasks.workunit.client.0.smithi033.stdout:9/732: rename d0/d11/l7b to d0/d2/dd/l101 0 2024-04-25T01:35:58.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/175: dwrite d3/d27/f1e [0,4194304] 0 2024-04-25T01:35:58.341 INFO:tasks.workunit.client.0.smithi033.stdout:8/737: symlink d6/d45/d52/df2/l114 0 2024-04-25T01:35:58.343 INFO:tasks.workunit.client.1.smithi138.stdout:9/202: getdents d4/d1a/d33/d1d/d3f 0 2024-04-25T01:35:58.344 INFO:tasks.workunit.client.1.smithi138.stdout:1/176: mkdir d3/d27/d3e 0 2024-04-25T01:35:58.344 INFO:tasks.workunit.client.0.smithi033.stdout:8/738: chown d6/db/d80/c96 827 1 2024-04-25T01:35:58.345 INFO:tasks.workunit.client.0.smithi033.stdout:9/733: unlink d0/d2/d43/c69 0 2024-04-25T01:35:58.347 INFO:tasks.workunit.client.1.smithi138.stdout:9/203: dread - d4/d1a/d33/f37 zero size 2024-04-25T01:35:58.347 INFO:tasks.workunit.client.0.smithi033.stdout:4/785: rename d6/dfd to d6/d12/d38/dc0/dff 0 2024-04-25T01:35:58.348 INFO:tasks.workunit.client.1.smithi138.stdout:0/203: sync 2024-04-25T01:35:58.349 INFO:tasks.workunit.client.1.smithi138.stdout:1/177: dread - d3/d27/d24/f3b zero size 2024-04-25T01:35:58.350 INFO:tasks.workunit.client.0.smithi033.stdout:4/786: write d6/d14/d35/d3a/d4d/d5e/fdd [887332,40986] 0 2024-04-25T01:35:58.351 INFO:tasks.workunit.client.1.smithi138.stdout:1/178: chown d3/d27/d2d/f30 65 1 2024-04-25T01:35:58.352 INFO:tasks.workunit.client.1.smithi138.stdout:5/143: getdents d4/da/d27/d1c/d22 0 2024-04-25T01:35:58.352 INFO:tasks.workunit.client.0.smithi033.stdout:5/751: creat d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/ff7 x:0 0 0 2024-04-25T01:35:58.352 INFO:tasks.workunit.client.1.smithi138.stdout:9/204: symlink d4/d8/d2c/l41 0 2024-04-25T01:35:58.353 INFO:tasks.workunit.client.0.smithi033.stdout:8/739: dread - d6/d45/d51/f9a zero size 2024-04-25T01:35:58.353 INFO:tasks.workunit.client.0.smithi033.stdout:4/787: chown d6/d14/d6d/l97 50 1 2024-04-25T01:35:58.354 INFO:tasks.workunit.client.0.smithi033.stdout:5/752: fdatasync d1c/d2a/d50/db1/fe3 0 2024-04-25T01:35:58.354 INFO:tasks.workunit.client.0.smithi033.stdout:9/734: creat d0/d2/d43/d4b/dc5/de0/d8f/f102 x:0 0 0 2024-04-25T01:35:58.354 INFO:tasks.workunit.client.1.smithi138.stdout:0/204: unlink d6/d39/d43/f46 0 2024-04-25T01:35:58.355 INFO:tasks.workunit.client.1.smithi138.stdout:9/205: read - d4/d8/f23 zero size 2024-04-25T01:35:58.357 INFO:tasks.workunit.client.0.smithi033.stdout:9/735: write d0/d2/f48 [2577508,56041] 0 2024-04-25T01:35:58.359 INFO:tasks.workunit.client.0.smithi033.stdout:2/751: dread da/d27/f7a [0,4194304] 0 2024-04-25T01:35:58.360 INFO:tasks.workunit.client.1.smithi138.stdout:0/205: dread - d6/d1d/f3c zero size 2024-04-25T01:35:58.360 INFO:tasks.workunit.client.1.smithi138.stdout:9/206: creat d4/d1a/f42 x:0 0 0 2024-04-25T01:35:58.361 INFO:tasks.workunit.client.1.smithi138.stdout:0/206: stat d6/df/l38 0 2024-04-25T01:35:58.361 INFO:tasks.workunit.client.0.smithi033.stdout:6/866: dread d4/f6 [0,4194304] 0 2024-04-25T01:35:58.362 INFO:tasks.workunit.client.0.smithi033.stdout:5/753: fsync d1c/d2a/d50/f91 0 2024-04-25T01:35:58.363 INFO:tasks.workunit.client.0.smithi033.stdout:9/736: write d0/d2/d43/d73/ffc [939060,2852] 0 2024-04-25T01:35:58.363 INFO:tasks.workunit.client.1.smithi138.stdout:5/144: dwrite d4/da/d27/d1c/f26 [0,4194304] 0 2024-04-25T01:35:58.364 INFO:tasks.workunit.client.1.smithi138.stdout:9/207: mkdir d4/d1a/d43 0 2024-04-25T01:35:58.365 INFO:tasks.workunit.client.0.smithi033.stdout:4/788: write d6/d14/d35/d44/d45/d46/d57/f62 [713452,14431] 0 2024-04-25T01:35:58.365 INFO:tasks.workunit.client.0.smithi033.stdout:3/782: unlink d8/dd/d3d/d4f/da4/f10c 0 2024-04-25T01:35:58.366 INFO:tasks.workunit.client.0.smithi033.stdout:9/737: chown d0/d2/d43/d73 720 1 2024-04-25T01:35:58.368 INFO:tasks.workunit.client.0.smithi033.stdout:6/867: chown d4/d1d/d3e/d41/d5c/c120 703 1 2024-04-25T01:35:58.370 INFO:tasks.workunit.client.0.smithi033.stdout:8/740: dwrite d6/d45/f5b [0,4194304] 0 2024-04-25T01:35:58.372 INFO:tasks.workunit.client.0.smithi033.stdout:5/754: chown d1c/d2a/d50/d52/da6/d3e/d70/d7a/dab/cec 786875211 1 2024-04-25T01:35:58.375 INFO:tasks.workunit.client.0.smithi033.stdout:3/783: symlink d8/dd/d3d/d4f/da4/dc4/l10f 0 2024-04-25T01:35:58.379 INFO:tasks.workunit.client.1.smithi138.stdout:9/208: dwrite d4/d1a/d33/f37 [0,4194304] 0 2024-04-25T01:35:58.380 INFO:tasks.workunit.client.1.smithi138.stdout:5/145: dwrite d4/da/fe [0,4194304] 0 2024-04-25T01:35:58.381 INFO:tasks.workunit.client.0.smithi033.stdout:8/741: creat d6/d45/d51/d5d/d6e/f115 x:0 0 0 2024-04-25T01:35:58.385 INFO:tasks.workunit.client.0.smithi033.stdout:3/784: write d8/dd/d3d/d63/d7c/f64 [1649530,22812] 0 2024-04-25T01:35:58.386 INFO:tasks.workunit.client.0.smithi033.stdout:9/738: creat d0/d2/d43/d4b/dc5/de0/d7e/ddc/f103 x:0 0 0 2024-04-25T01:35:58.391 INFO:tasks.workunit.client.1.smithi138.stdout:9/209: creat d4/d1a/f44 x:0 0 0 2024-04-25T01:35:58.392 INFO:tasks.workunit.client.0.smithi033.stdout:7/709: write d4/d2e/d6d/f75 [2097736,89017] 0 2024-04-25T01:35:58.394 INFO:tasks.workunit.client.1.smithi138.stdout:2/263: write d0/d21/f3a [757909,14985] 0 2024-04-25T01:35:58.395 INFO:tasks.workunit.client.0.smithi033.stdout:3/785: rename d8/dd/d3d/d63/d7c/ded/dc7/ddf/df2 to d8/dd/d3d/d4f/d110 0 2024-04-25T01:35:58.395 INFO:tasks.workunit.client.1.smithi138.stdout:9/210: write d4/f25 [244363,11996] 0 2024-04-25T01:35:58.396 INFO:tasks.workunit.client.1.smithi138.stdout:2/264: fdatasync d0/d21/d36/d15/d1f/f2d 0 2024-04-25T01:35:58.397 INFO:tasks.workunit.client.1.smithi138.stdout:3/170: dread f6 [0,4194304] 0 2024-04-25T01:35:58.397 INFO:tasks.workunit.client.1.smithi138.stdout:2/265: chown d0/l38 793 1 2024-04-25T01:35:58.399 INFO:tasks.workunit.client.0.smithi033.stdout:0/628: write d7/d2d/d64/f6d [910731,105423] 0 2024-04-25T01:35:58.403 INFO:tasks.workunit.client.0.smithi033.stdout:4/789: dread d6/d14/d1d/d3b/f52 [0,4194304] 0 2024-04-25T01:35:58.404 INFO:tasks.workunit.client.1.smithi138.stdout:5/146: link d4/c16 d4/da/d27/c2b 0 2024-04-25T01:35:58.404 INFO:tasks.workunit.client.1.smithi138.stdout:1/179: truncate d3/f10 2608775 0 2024-04-25T01:35:58.405 INFO:tasks.workunit.client.0.smithi033.stdout:8/742: getdents d6/db/d80/d89/dbf 0 2024-04-25T01:35:58.405 INFO:tasks.workunit.client.1.smithi138.stdout:0/207: read d6/d19/f47 [1044053,31843] 0 2024-04-25T01:35:58.406 INFO:tasks.workunit.client.0.smithi033.stdout:8/743: dread - d6/d45/d52/fe2 zero size 2024-04-25T01:35:58.409 INFO:tasks.workunit.client.1.smithi138.stdout:5/147: dread d4/f23 [0,4194304] 0 2024-04-25T01:35:58.410 INFO:tasks.workunit.client.0.smithi033.stdout:4/790: readlink d6/d14/l37 0 2024-04-25T01:35:58.410 INFO:tasks.workunit.client.1.smithi138.stdout:3/171: dwrite db/df/d17/f1c [0,4194304] 0 2024-04-25T01:35:58.412 INFO:tasks.workunit.client.0.smithi033.stdout:0/629: symlink d7/da/df/dc7/lda 0 2024-04-25T01:35:58.412 INFO:tasks.workunit.client.1.smithi138.stdout:9/211: creat d4/d8/f45 x:0 0 0 2024-04-25T01:35:58.413 INFO:tasks.workunit.client.0.smithi033.stdout:4/791: chown d6/d12/l42 8 1 2024-04-25T01:35:58.413 INFO:tasks.workunit.client.1.smithi138.stdout:1/180: truncate d3/db/f14 3119593 0 2024-04-25T01:35:58.413 INFO:tasks.workunit.client.0.smithi033.stdout:3/786: creat d8/dd/d11/f111 x:0 0 0 2024-04-25T01:35:58.414 INFO:tasks.workunit.client.1.smithi138.stdout:2/266: dwrite d0/d21/d36/d15/f3b [0,4194304] 0 2024-04-25T01:35:58.415 INFO:tasks.workunit.client.0.smithi033.stdout:8/744: truncate d6/d45/d52/fc7 201905 0 2024-04-25T01:35:58.415 INFO:tasks.workunit.client.1.smithi138.stdout:1/181: stat d3/db/f1d 0 2024-04-25T01:35:58.415 INFO:tasks.workunit.client.1.smithi138.stdout:5/148: symlink d4/da/l2c 0 2024-04-25T01:35:58.417 INFO:tasks.workunit.client.0.smithi033.stdout:2/752: dwrite da/d27/d31/d40/d8a/f94 [0,4194304] 0 2024-04-25T01:35:58.417 INFO:tasks.workunit.client.1.smithi138.stdout:5/149: fsync d4/da/fe 0 2024-04-25T01:35:58.418 INFO:tasks.workunit.client.1.smithi138.stdout:5/150: read d4/da/fe [3242753,36585] 0 2024-04-25T01:35:58.419 INFO:tasks.workunit.client.1.smithi138.stdout:0/208: mkdir d6/df/d41/d49 0 2024-04-25T01:35:58.421 INFO:tasks.workunit.client.0.smithi033.stdout:7/710: write d4/d2e/d6b/f9a [255411,58974] 0 2024-04-25T01:35:58.424 INFO:tasks.workunit.client.1.smithi138.stdout:1/182: dread d3/f15 [0,4194304] 0 2024-04-25T01:35:58.425 INFO:tasks.workunit.client.0.smithi033.stdout:0/630: truncate d7/da/df/d50/f6b 628498 0 2024-04-25T01:35:58.426 INFO:tasks.workunit.client.1.smithi138.stdout:4/130: sync 2024-04-25T01:35:58.426 INFO:tasks.workunit.client.1.smithi138.stdout:7/156: sync 2024-04-25T01:35:58.426 INFO:tasks.workunit.client.1.smithi138.stdout:8/372: sync 2024-04-25T01:35:58.427 INFO:tasks.workunit.client.1.smithi138.stdout:9/212: rename d4/d1a/d33/c2f to d4/d1a/c46 0 2024-04-25T01:35:58.428 INFO:tasks.workunit.client.1.smithi138.stdout:8/373: fdatasync d3/d16/f1c 0 2024-04-25T01:35:58.428 INFO:tasks.workunit.client.0.smithi033.stdout:5/755: dwrite d1c/d2a/d50/d5b/fa1 [0,4194304] 0 2024-04-25T01:35:58.428 INFO:tasks.workunit.client.0.smithi033.stdout:3/787: creat d8/dd/d3d/d63/d7c/ded/dc7/ddf/f112 x:0 0 0 2024-04-25T01:35:58.429 INFO:tasks.workunit.client.1.smithi138.stdout:3/172: getdents db/df/d17/d20 0 2024-04-25T01:35:58.429 INFO:tasks.workunit.client.1.smithi138.stdout:5/151: creat d4/da/d27/f2d x:0 0 0 2024-04-25T01:35:58.431 INFO:tasks.workunit.client.0.smithi033.stdout:4/792: truncate d6/d14/d35/d3a/d4c/f78 2032205 0 2024-04-25T01:35:58.432 INFO:tasks.workunit.client.1.smithi138.stdout:0/209: rmdir d6/d1d 39 2024-04-25T01:35:58.434 INFO:tasks.workunit.client.1.smithi138.stdout:1/183: mknod d3/db/c3f 0 2024-04-25T01:35:58.435 INFO:tasks.workunit.client.0.smithi033.stdout:7/711: write d4/f32 [3353942,25961] 0 2024-04-25T01:35:58.435 INFO:tasks.workunit.client.0.smithi033.stdout:0/631: creat d7/d2d/d4d/d9c/d61/fdb x:0 0 0 2024-04-25T01:35:58.436 INFO:tasks.workunit.client.0.smithi033.stdout:2/753: rmdir da/d27/d31/d40/d8a/dbb/dd6/de1 39 2024-04-25T01:35:58.437 INFO:tasks.workunit.client.1.smithi138.stdout:9/213: dread d4/d1a/d33/f37 [0,4194304] 0 2024-04-25T01:35:58.437 INFO:tasks.workunit.client.0.smithi033.stdout:9/739: dwrite d0/d2/dd/df/d1b/f4d [0,4194304] 0 2024-04-25T01:35:58.440 INFO:tasks.workunit.client.0.smithi033.stdout:4/793: fsync d6/d14/d48/f99 0 2024-04-25T01:35:58.442 INFO:tasks.workunit.client.1.smithi138.stdout:8/374: dwrite d3/d8/da/db/d15/d56/f69 [0,4194304] 0 2024-04-25T01:35:58.442 INFO:tasks.workunit.client.1.smithi138.stdout:9/214: write d4/f13 [2196619,77641] 0 2024-04-25T01:35:58.442 INFO:tasks.workunit.client.1.smithi138.stdout:3/173: truncate db/df/d17/d20/f29 4917197 0 2024-04-25T01:35:58.442 INFO:tasks.workunit.client.1.smithi138.stdout:7/157: symlink da/de/d2e/l36 0 2024-04-25T01:35:58.444 INFO:tasks.workunit.client.0.smithi033.stdout:2/754: creat da/d8d/ff4 x:0 0 0 2024-04-25T01:35:58.444 INFO:tasks.workunit.client.0.smithi033.stdout:3/788: dwrite d8/dd/d1d/f7e [0,4194304] 0 2024-04-25T01:35:58.455 INFO:tasks.workunit.client.0.smithi033.stdout:9/740: creat d0/d2/dd/d20/d4e/f104 x:0 0 0 2024-04-25T01:35:58.455 INFO:tasks.workunit.client.0.smithi033.stdout:7/712: dwrite d4/d2e/d50/fb5 [0,4194304] 0 2024-04-25T01:35:58.455 INFO:tasks.workunit.client.1.smithi138.stdout:7/158: unlink c8 0 2024-04-25T01:35:58.456 INFO:tasks.workunit.client.1.smithi138.stdout:3/174: readlink db/df/d19/l1b 0 2024-04-25T01:35:58.456 INFO:tasks.workunit.client.0.smithi033.stdout:0/632: truncate d7/da/df/d14/f15 779873 0 2024-04-25T01:35:58.457 INFO:tasks.workunit.client.0.smithi033.stdout:7/713: readlink d4/lc 0 2024-04-25T01:35:58.457 INFO:tasks.workunit.client.1.smithi138.stdout:3/175: fdatasync db/df/f25 0 2024-04-25T01:35:58.458 INFO:tasks.workunit.client.0.smithi033.stdout:5/756: read - d1c/d2a/d50/d52/da6/d3e/d70/d89/fa5 zero size 2024-04-25T01:35:58.459 INFO:tasks.workunit.client.0.smithi033.stdout:2/755: fsync da/d50/d6a/d43/f99 0 2024-04-25T01:35:58.460 INFO:tasks.workunit.client.1.smithi138.stdout:8/375: truncate d3/d8/da/db/d15/f21 3001313 0 2024-04-25T01:35:58.460 INFO:tasks.workunit.client.0.smithi033.stdout:4/794: creat d6/d14/d35/d44/d45/d46/d57/d70/d87/f100 x:0 0 0 2024-04-25T01:35:58.461 INFO:tasks.workunit.client.0.smithi033.stdout:3/789: stat d8/f51 0 2024-04-25T01:35:58.462 INFO:tasks.workunit.client.0.smithi033.stdout:4/795: chown d6/d14/d1d/d5f 86954422 1 2024-04-25T01:35:58.462 INFO:tasks.workunit.client.1.smithi138.stdout:8/376: fdatasync d3/d16/d1f/d2a/f6a 0 2024-04-25T01:35:58.463 INFO:tasks.workunit.client.1.smithi138.stdout:5/152: link d4/l1e d4/l2e 0 2024-04-25T01:35:58.464 INFO:tasks.workunit.client.0.smithi033.stdout:9/741: write d0/d2/dd/da7/fb5 [625676,127298] 0 2024-04-25T01:35:58.465 INFO:tasks.workunit.client.0.smithi033.stdout:0/633: mknod d7/d2d/d64/cdc 0 2024-04-25T01:35:58.466 INFO:tasks.workunit.client.1.smithi138.stdout:3/176: dwrite f1 [0,4194304] 0 2024-04-25T01:35:58.467 INFO:tasks.workunit.client.0.smithi033.stdout:7/714: creat d4/d2e/d6b/d1f/d78/d86/fe1 x:0 0 0 2024-04-25T01:35:58.468 INFO:tasks.workunit.client.1.smithi138.stdout:3/177: dread - db/d14/f2a zero size 2024-04-25T01:35:58.468 INFO:tasks.workunit.client.0.smithi033.stdout:5/757: rmdir d1c/d2a/d50/d52/da6/d4a 39 2024-04-25T01:35:58.469 INFO:tasks.workunit.client.1.smithi138.stdout:3/178: fdatasync db/df/d17/f1c 0 2024-04-25T01:35:58.469 INFO:tasks.workunit.client.1.smithi138.stdout:3/179: readlink la 0 2024-04-25T01:35:58.469 INFO:tasks.workunit.client.1.smithi138.stdout:7/159: unlink da/l22 0 2024-04-25T01:35:58.470 INFO:tasks.workunit.client.1.smithi138.stdout:7/160: chown c2 4 1 2024-04-25T01:35:58.470 INFO:tasks.workunit.client.0.smithi033.stdout:2/756: unlink da/d50/d6a/d43/d6c/c7f 0 2024-04-25T01:35:58.471 INFO:tasks.workunit.client.1.smithi138.stdout:7/161: stat da/f18 0 2024-04-25T01:35:58.471 INFO:tasks.workunit.client.0.smithi033.stdout:8/745: dwrite d6/d45/d52/d2e/f5e [0,4194304] 0 2024-04-25T01:35:58.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/180: truncate db/df/d17/d20/f26 751774 0 2024-04-25T01:35:58.473 INFO:tasks.workunit.client.1.smithi138.stdout:2/267: rename d0/c23 to d0/d32/c50 0 2024-04-25T01:35:58.475 INFO:tasks.workunit.client.1.smithi138.stdout:3/181: truncate db/df/d17/d20/f26 1175598 0 2024-04-25T01:35:58.478 INFO:tasks.workunit.client.1.smithi138.stdout:8/377: symlink d3/d16/d1f/d50/d67/l74 0 2024-04-25T01:35:58.478 INFO:tasks.workunit.client.0.smithi033.stdout:9/742: write d0/d11/f27 [2225984,103887] 0 2024-04-25T01:35:58.480 INFO:tasks.workunit.client.0.smithi033.stdout:7/715: mknod d4/d2e/d6b/da9/ce2 0 2024-04-25T01:35:58.481 INFO:tasks.workunit.client.1.smithi138.stdout:4/131: rename d0/d1c/l20 to d0/l21 0 2024-04-25T01:35:58.481 INFO:tasks.workunit.client.0.smithi033.stdout:2/757: mkdir da/d50/d6a/d43/d6c/df5 0 2024-04-25T01:35:58.482 INFO:tasks.workunit.client.1.smithi138.stdout:4/132: fdatasync d0/f1a 0 2024-04-25T01:35:58.483 INFO:tasks.workunit.client.1.smithi138.stdout:2/268: mkdir d0/d21/d36/d15/d51 0 2024-04-25T01:35:58.483 INFO:tasks.workunit.client.0.smithi033.stdout:3/790: fsync d8/dd/d40/f53 0 2024-04-25T01:35:58.486 INFO:tasks.workunit.client.0.smithi033.stdout:5/758: rename d1c/cbe to d1c/d2a/d50/d52/da6/d3e/d70/dae/df1/cf8 0 2024-04-25T01:35:58.486 INFO:tasks.workunit.client.1.smithi138.stdout:3/182: creat db/d14/f32 x:0 0 0 2024-04-25T01:35:58.488 INFO:tasks.workunit.client.0.smithi033.stdout:8/746: mknod d6/d45/d52/db0/dbc/c116 0 2024-04-25T01:35:58.488 INFO:tasks.workunit.client.0.smithi033.stdout:7/716: dwrite d4/d23/d3f/d45/d73/d8f/fb0 [0,4194304] 0 2024-04-25T01:35:58.490 INFO:tasks.workunit.client.0.smithi033.stdout:4/796: creat d6/d14/f101 x:0 0 0 2024-04-25T01:35:58.493 INFO:tasks.workunit.client.0.smithi033.stdout:9/743: fsync d0/d2/d43/d4b/dc5/de0/f70 0 2024-04-25T01:35:58.494 INFO:tasks.workunit.client.1.smithi138.stdout:2/269: rename d0/d21/d36/d15/d1f/c2b to d0/d21/d45/c52 0 2024-04-25T01:35:58.495 INFO:tasks.workunit.client.1.smithi138.stdout:8/378: dwrite d3/f3a [0,4194304] 0 2024-04-25T01:35:58.496 INFO:tasks.workunit.client.1.smithi138.stdout:3/183: readlink db/df/l2b 0 2024-04-25T01:35:58.496 INFO:tasks.workunit.client.0.smithi033.stdout:4/797: dread d6/d12/feb [0,4194304] 0 2024-04-25T01:35:58.496 INFO:tasks.workunit.client.0.smithi033.stdout:2/758: mkdir da/d50/d6a/df6 0 2024-04-25T01:35:58.497 INFO:tasks.workunit.client.0.smithi033.stdout:4/798: chown d6/d14/d1d/d5f/fd4 241 1 2024-04-25T01:35:58.499 INFO:tasks.workunit.client.1.smithi138.stdout:3/184: write f1 [2510914,93780] 0 2024-04-25T01:35:58.499 INFO:tasks.workunit.client.1.smithi138.stdout:8/379: write f0 [296287,130583] 0 2024-04-25T01:35:58.500 INFO:tasks.workunit.client.1.smithi138.stdout:7/162: dread da/de/f1a [0,4194304] 0 2024-04-25T01:35:58.501 INFO:tasks.workunit.client.0.smithi033.stdout:8/747: chown d6/d49/c75 141260183 1 2024-04-25T01:35:58.504 INFO:tasks.workunit.client.0.smithi033.stdout:5/759: creat d1c/d2a/d50/d52/da6/d3e/d70/d89/ff9 x:0 0 0 2024-04-25T01:35:58.507 INFO:tasks.workunit.client.0.smithi033.stdout:9/744: dread - d0/d2/dd/d5f/f9b zero size 2024-04-25T01:35:58.507 INFO:tasks.workunit.client.0.smithi033.stdout:7/717: write d4/d2e/d6b/d67/f7d [6514403,86398] 0 2024-04-25T01:35:58.508 INFO:tasks.workunit.client.0.smithi033.stdout:2/759: mknod da/d27/cf7 0 2024-04-25T01:35:58.510 INFO:tasks.workunit.client.1.smithi138.stdout:3/185: rename db/df/d19/c2c to db/d14/c33 0 2024-04-25T01:35:58.510 INFO:tasks.workunit.client.1.smithi138.stdout:3/186: stat db/df/d17 0 2024-04-25T01:35:58.511 INFO:tasks.workunit.client.0.smithi033.stdout:4/799: dwrite d6/d14/f2b [4194304,4194304] 0 2024-04-25T01:35:58.512 INFO:tasks.workunit.client.1.smithi138.stdout:7/163: symlink da/de/d13/d21/l37 0 2024-04-25T01:35:58.512 INFO:tasks.workunit.client.1.smithi138.stdout:3/187: truncate db/d14/f32 660535 0 2024-04-25T01:35:58.513 INFO:tasks.workunit.client.1.smithi138.stdout:7/164: chown da/f18 59 1 2024-04-25T01:35:58.514 INFO:tasks.workunit.client.1.smithi138.stdout:2/270: symlink d0/d32/d35/d4f/l53 0 2024-04-25T01:35:58.515 INFO:tasks.workunit.client.0.smithi033.stdout:2/760: dread da/f74 [0,4194304] 0 2024-04-25T01:35:58.515 INFO:tasks.workunit.client.1.smithi138.stdout:1/184: dread d3/f10 [0,4194304] 0 2024-04-25T01:35:58.517 INFO:tasks.workunit.client.0.smithi033.stdout:5/760: creat d1c/d53/df6/ffa x:0 0 0 2024-04-25T01:35:58.517 INFO:tasks.workunit.client.0.smithi033.stdout:0/634: dwrite d7/da/f48 [0,4194304] 0 2024-04-25T01:35:58.517 INFO:tasks.workunit.client.0.smithi033.stdout:3/791: dread d8/dd/d1d/f93 [0,4194304] 0 2024-04-25T01:35:58.518 INFO:tasks.workunit.client.1.smithi138.stdout:3/188: mkdir db/df/d17/d23/d34 0 2024-04-25T01:35:58.520 INFO:tasks.workunit.client.1.smithi138.stdout:8/380: symlink d3/d8/l75 0 2024-04-25T01:35:58.521 INFO:tasks.workunit.client.1.smithi138.stdout:3/189: fdatasync db/df/d19/d1a/f21 0 2024-04-25T01:35:58.521 INFO:tasks.workunit.client.0.smithi033.stdout:7/718: fsync d4/d23/d3f/d45/f7c 0 2024-04-25T01:35:58.523 INFO:tasks.workunit.client.0.smithi033.stdout:5/761: unlink d1c/lbb 0 2024-04-25T01:35:58.525 INFO:tasks.workunit.client.0.smithi033.stdout:4/800: read d6/d14/ff [4349675,119852] 0 2024-04-25T01:35:58.525 INFO:tasks.workunit.client.1.smithi138.stdout:7/165: mknod da/de/d2e/c38 0 2024-04-25T01:35:58.527 INFO:tasks.workunit.client.1.smithi138.stdout:1/185: dwrite d3/d4/fd [0,4194304] 0 2024-04-25T01:35:58.527 INFO:tasks.workunit.client.0.smithi033.stdout:2/761: rename da/d8d/fe4 to da/d5b/d72/dd1/ded/ff8 0 2024-04-25T01:35:58.528 INFO:tasks.workunit.client.0.smithi033.stdout:9/745: mkdir d0/d2/d43/d4b/dc5/de0/d42/dea/d105 0 2024-04-25T01:35:58.528 INFO:tasks.workunit.client.1.smithi138.stdout:5/153: unlink d4/da/d27/d1c/l2a 0 2024-04-25T01:35:58.529 INFO:tasks.workunit.client.0.smithi033.stdout:2/762: chown da/d27/d31/d40/d77/f7b 38169181 1 2024-04-25T01:35:58.529 INFO:tasks.workunit.client.0.smithi033.stdout:9/746: chown d0/d2/dd/df/cb4 4 1 2024-04-25T01:35:58.529 INFO:tasks.workunit.client.0.smithi033.stdout:0/635: dread d7/da/df/f17 [0,4194304] 0 2024-04-25T01:35:58.534 INFO:tasks.workunit.client.0.smithi033.stdout:2/763: dread - da/d27/fdf zero size 2024-04-25T01:35:58.535 INFO:tasks.workunit.client.0.smithi033.stdout:7/719: creat d4/d2e/d7f/d98/fe3 x:0 0 0 2024-04-25T01:35:58.535 INFO:tasks.workunit.client.0.smithi033.stdout:5/762: mkdir d1c/d2a/d50/d52/da6/d3e/d70/dae/dfb 0 2024-04-25T01:35:58.535 INFO:tasks.workunit.client.1.smithi138.stdout:5/154: dread - d4/da/d27/d1c/f28 zero size 2024-04-25T01:35:58.535 INFO:tasks.workunit.client.1.smithi138.stdout:4/133: truncate d0/d16/f18 3575235 0 2024-04-25T01:35:58.535 INFO:tasks.workunit.client.1.smithi138.stdout:3/190: dwrite db/f15 [0,4194304] 0 2024-04-25T01:35:58.536 INFO:tasks.workunit.client.1.smithi138.stdout:8/381: mknod d3/d16/d1f/d2a/c76 0 2024-04-25T01:35:58.536 INFO:tasks.workunit.client.1.smithi138.stdout:4/134: chown d0 2 1 2024-04-25T01:35:58.536 INFO:tasks.workunit.client.1.smithi138.stdout:1/186: read d3/db/f1d [1056970,25494] 0 2024-04-25T01:35:58.536 INFO:tasks.workunit.client.1.smithi138.stdout:5/155: read d4/da/d27/d1c/f26 [575988,5591] 0 2024-04-25T01:35:58.536 INFO:tasks.workunit.client.1.smithi138.stdout:2/271: dread d0/d21/d36/d15/d1f/f37 [0,4194304] 0 2024-04-25T01:35:58.536 INFO:tasks.workunit.client.1.smithi138.stdout:3/191: fsync db/df/f25 0 2024-04-25T01:35:58.536 INFO:tasks.workunit.client.0.smithi033.stdout:7/720: chown d4/d23/d3f/d45/f5e 15150520 1 2024-04-25T01:35:58.537 INFO:tasks.workunit.client.1.smithi138.stdout:7/166: chown l5 25134 1 2024-04-25T01:35:58.538 INFO:tasks.workunit.client.0.smithi033.stdout:0/636: mkdir d7/d2d/d4d/d9c/dcf/ddd 0 2024-04-25T01:35:58.539 INFO:tasks.workunit.client.1.smithi138.stdout:7/167: write da/de/f1c [2644144,76413] 0 2024-04-25T01:35:58.540 INFO:tasks.workunit.client.0.smithi033.stdout:2/764: creat da/d27/d31/d40/db1/dbf/dd5/ff9 x:0 0 0 2024-04-25T01:35:58.541 INFO:tasks.workunit.client.0.smithi033.stdout:0/637: chown d7/d2d/d64/d51/f52 13 1 2024-04-25T01:35:58.542 INFO:tasks.workunit.client.1.smithi138.stdout:2/272: chown d0/d32/d35/d4f/l53 2 1 2024-04-25T01:35:58.542 INFO:tasks.workunit.client.0.smithi033.stdout:2/765: write da/d27/d31/dca/fd7 [599948,2393] 0 2024-04-25T01:35:58.543 INFO:tasks.workunit.client.1.smithi138.stdout:5/156: mknod d4/da/d27/d1c/d22/c2f 0 2024-04-25T01:35:58.544 INFO:tasks.workunit.client.1.smithi138.stdout:2/273: write d0/d21/d36/d15/d1f/f26 [2182989,70111] 0 2024-04-25T01:35:58.544 INFO:tasks.workunit.client.0.smithi033.stdout:0/638: fsync d7/d2d/d4d/d9c/d61/d70/fa9 0 2024-04-25T01:35:58.544 INFO:tasks.workunit.client.0.smithi033.stdout:7/721: rmdir d4/d2e/d6b/d67 39 2024-04-25T01:35:58.544 INFO:tasks.workunit.client.0.smithi033.stdout:4/801: dread d6/d14/d35/d3a/de8/ff3 [0,4194304] 0 2024-04-25T01:35:58.545 INFO:tasks.workunit.client.1.smithi138.stdout:3/192: creat db/f35 x:0 0 0 2024-04-25T01:35:58.545 INFO:tasks.workunit.client.1.smithi138.stdout:8/382: dwrite d3/d8/da/f24 [0,4194304] 0 2024-04-25T01:35:58.547 INFO:tasks.workunit.client.0.smithi033.stdout:3/792: rename d8/dd/d3d/d63/cb8 to d8/dd/c113 0 2024-04-25T01:35:58.547 INFO:tasks.workunit.client.1.smithi138.stdout:4/135: rename d0/c10 to d0/d16/c22 0 2024-04-25T01:35:58.548 INFO:tasks.workunit.client.0.smithi033.stdout:9/747: creat d0/d2/dd/d20/f106 x:0 0 0 2024-04-25T01:35:58.550 INFO:tasks.workunit.client.0.smithi033.stdout:9/748: fsync d0/d2/d43/d4b/d55/fbc 0 2024-04-25T01:35:58.553 INFO:tasks.workunit.client.0.smithi033.stdout:4/802: symlink d6/l102 0 2024-04-25T01:35:58.554 INFO:tasks.workunit.client.0.smithi033.stdout:2/766: symlink da/d27/d31/d40/lfa 0 2024-04-25T01:35:58.555 INFO:tasks.workunit.client.0.smithi033.stdout:5/763: rename d1c/d2a/l67 to d1c/d2a/d3d/d59/lfc 0 2024-04-25T01:35:58.557 INFO:tasks.workunit.client.0.smithi033.stdout:3/793: mknod d8/dd/d3d/d63/d7c/d9d/c114 0 2024-04-25T01:35:58.558 INFO:tasks.workunit.client.1.smithi138.stdout:2/274: dwrite d0/d21/d36/d15/d1f/f4a [0,4194304] 0 2024-04-25T01:35:58.559 INFO:tasks.workunit.client.0.smithi033.stdout:9/749: unlink d0/f23 0 2024-04-25T01:35:58.559 INFO:tasks.workunit.client.1.smithi138.stdout:2/275: chown d0/d32/c33 551 1 2024-04-25T01:35:58.561 INFO:tasks.workunit.client.1.smithi138.stdout:2/276: truncate d0/d32/f4b 4460201 0 2024-04-25T01:35:58.565 INFO:tasks.workunit.client.1.smithi138.stdout:8/383: symlink d3/l77 0 2024-04-25T01:35:58.567 INFO:tasks.workunit.client.1.smithi138.stdout:5/157: truncate d4/f9 3287321 0 2024-04-25T01:35:58.568 INFO:tasks.workunit.client.1.smithi138.stdout:8/384: read d3/d8/f4d [1276734,103850] 0 2024-04-25T01:35:58.572 INFO:tasks.workunit.client.1.smithi138.stdout:8/385: creat d3/d16/d1f/d50/f78 x:0 0 0 2024-04-25T01:35:58.574 INFO:tasks.workunit.client.1.smithi138.stdout:8/386: chown d3/d8/da/db/d15/d20/l68 1687307406 1 2024-04-25T01:35:58.578 INFO:tasks.workunit.client.1.smithi138.stdout:7/168: dread da/fc [0,4194304] 0 2024-04-25T01:35:58.578 INFO:tasks.workunit.client.1.smithi138.stdout:7/169: read - da/de/f2c zero size 2024-04-25T01:35:58.580 INFO:tasks.workunit.client.1.smithi138.stdout:7/170: mkdir da/de/d39 0 2024-04-25T01:35:58.613 INFO:tasks.workunit.client.0.smithi033.stdout:2/767: creat da/d5b/d5e/ffb x:0 0 0 2024-04-25T01:35:58.615 INFO:tasks.workunit.client.0.smithi033.stdout:5/764: symlink d1c/d2a/d50/db1/dd3/ddd/lfd 0 2024-04-25T01:35:58.616 INFO:tasks.workunit.client.1.smithi138.stdout:5/158: truncate d4/da/d27/d1c/f26 1124419 0 2024-04-25T01:35:58.618 INFO:tasks.workunit.client.1.smithi138.stdout:5/159: write d4/da/fe [5179734,126784] 0 2024-04-25T01:35:58.618 INFO:tasks.workunit.client.0.smithi033.stdout:9/750: stat d0/d2/d43/d4b/fe9 0 2024-04-25T01:35:58.619 INFO:tasks.workunit.client.1.smithi138.stdout:4/136: write d0/d16/f18 [1432873,83874] 0 2024-04-25T01:35:58.619 INFO:tasks.workunit.client.1.smithi138.stdout:7/171: dread da/de/d13/f24 [0,4194304] 0 2024-04-25T01:35:58.619 INFO:tasks.workunit.client.0.smithi033.stdout:9/751: readlink d0/d2/d43/d4b/l5e 0 2024-04-25T01:35:58.620 INFO:tasks.workunit.client.1.smithi138.stdout:7/172: fsync da/de/f29 0 2024-04-25T01:35:58.622 INFO:tasks.workunit.client.1.smithi138.stdout:5/160: symlink d4/da/l30 0 2024-04-25T01:35:58.624 INFO:tasks.workunit.client.1.smithi138.stdout:3/193: truncate db/df/d17/d20/f29 1242658 0 2024-04-25T01:35:58.625 INFO:tasks.workunit.client.1.smithi138.stdout:4/137: mkdir d0/d1c/d23 0 2024-04-25T01:35:58.628 INFO:tasks.workunit.client.1.smithi138.stdout:7/173: dwrite da/f15 [0,4194304] 0 2024-04-25T01:35:58.629 INFO:tasks.workunit.client.1.smithi138.stdout:3/194: creat db/df/d19/f36 x:0 0 0 2024-04-25T01:35:58.629 INFO:tasks.workunit.client.1.smithi138.stdout:8/387: getdents d3/d16/d1f/d50 0 2024-04-25T01:35:58.630 INFO:tasks.workunit.client.1.smithi138.stdout:7/174: read da/f15 [1645128,77096] 0 2024-04-25T01:35:58.631 INFO:tasks.workunit.client.1.smithi138.stdout:3/195: chown db/df/d17/f1c 0 1 2024-04-25T01:35:58.634 INFO:tasks.workunit.client.1.smithi138.stdout:4/138: dwrite d0/f1a [4194304,4194304] 0 2024-04-25T01:35:58.635 INFO:tasks.workunit.client.1.smithi138.stdout:3/196: symlink db/l37 0 2024-04-25T01:35:58.636 INFO:tasks.workunit.client.0.smithi033.stdout:7/722: dwrite d4/f11 [0,4194304] 0 2024-04-25T01:35:58.638 INFO:tasks.workunit.client.1.smithi138.stdout:3/197: truncate db/d14/f32 1309863 0 2024-04-25T01:35:58.638 INFO:tasks.workunit.client.1.smithi138.stdout:3/198: stat db/l37 0 2024-04-25T01:35:58.640 INFO:tasks.workunit.client.1.smithi138.stdout:7/175: unlink da/l1e 0 2024-04-25T01:35:58.640 INFO:tasks.workunit.client.1.smithi138.stdout:3/199: chown db/df/d19/d1a 28194124 1 2024-04-25T01:35:58.642 INFO:tasks.workunit.client.1.smithi138.stdout:7/176: write da/de/d1b/f31 [766874,47855] 0 2024-04-25T01:35:58.643 INFO:tasks.workunit.client.1.smithi138.stdout:7/177: chown da/de 0 1 2024-04-25T01:35:58.644 INFO:tasks.workunit.client.1.smithi138.stdout:4/139: symlink d0/d1c/d1e/l24 0 2024-04-25T01:35:58.645 INFO:tasks.workunit.client.1.smithi138.stdout:3/200: symlink db/d14/l38 0 2024-04-25T01:35:58.648 INFO:tasks.workunit.client.1.smithi138.stdout:3/201: symlink db/df/d17/d23/l39 0 2024-04-25T01:35:58.649 INFO:tasks.workunit.client.1.smithi138.stdout:3/202: write db/fe [511176,57406] 0 2024-04-25T01:35:58.649 INFO:tasks.workunit.client.0.smithi033.stdout:4/803: dwrite d6/d14/d1d/d5f/f8f [0,4194304] 0 2024-04-25T01:35:58.650 INFO:tasks.workunit.client.1.smithi138.stdout:7/178: rename da/de/d13/d21/d2a/l2b to da/l3a 0 2024-04-25T01:35:58.650 INFO:tasks.workunit.client.1.smithi138.stdout:3/203: chown db/df/d17/d20/l28 459264 1 2024-04-25T01:35:58.651 INFO:tasks.workunit.client.0.smithi033.stdout:4/804: read d6/d14/d35/d3a/d4d/d5e/d7c/d94/fd2 [16687,106207] 0 2024-04-25T01:35:58.658 INFO:tasks.workunit.client.1.smithi138.stdout:7/179: mknod da/de/d1b/c3b 0 2024-04-25T01:35:58.663 INFO:tasks.workunit.client.1.smithi138.stdout:3/204: rename db/d14/c33 to db/df/d17/d23/c3a 0 2024-04-25T01:35:58.663 INFO:tasks.workunit.client.0.smithi033.stdout:5/765: chown d1c/d2a/d50/d52/da6/d4a/f72 158 1 2024-04-25T01:35:58.664 INFO:tasks.workunit.client.1.smithi138.stdout:7/180: dwrite da/de/d1b/f31 [0,4194304] 0 2024-04-25T01:35:58.664 INFO:tasks.workunit.client.1.smithi138.stdout:6/138: sync 2024-04-25T01:35:58.666 INFO:tasks.workunit.client.1.smithi138.stdout:4/140: fsync d0/f1a 0 2024-04-25T01:35:58.667 INFO:tasks.workunit.client.1.smithi138.stdout:6/139: chown dc/d2e/f31 15668 1 2024-04-25T01:35:58.667 INFO:tasks.workunit.client.1.smithi138.stdout:7/181: creat da/de/d13/f3c x:0 0 0 2024-04-25T01:35:58.668 INFO:tasks.workunit.client.1.smithi138.stdout:6/140: fsync fb 0 2024-04-25T01:35:58.668 INFO:tasks.workunit.client.1.smithi138.stdout:7/182: stat l0 0 2024-04-25T01:35:58.669 INFO:tasks.workunit.client.1.smithi138.stdout:7/183: readlink da/de/l1d 0 2024-04-25T01:35:58.671 INFO:tasks.workunit.client.1.smithi138.stdout:7/184: read da/f15 [3217237,128866] 0 2024-04-25T01:35:58.672 INFO:tasks.workunit.client.0.smithi033.stdout:4/805: rename d6/d14/d35/d44/d45/d46/d64 to d6/d14/d35/d44/dfe/d103 0 2024-04-25T01:35:58.672 INFO:tasks.workunit.client.0.smithi033.stdout:9/752: dwrite d0/d2/dd/df/d1b/dc0/fed [0,4194304] 0 2024-04-25T01:35:58.674 INFO:tasks.workunit.client.0.smithi033.stdout:4/806: write d6/d12/d84/fbb [163274,74435] 0 2024-04-25T01:35:58.675 INFO:tasks.workunit.client.1.smithi138.stdout:5/161: dwrite d4/da/d27/d1c/f26 [0,4194304] 0 2024-04-25T01:35:58.676 INFO:tasks.workunit.client.0.smithi033.stdout:3/794: creat d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/f115 x:0 0 0 2024-04-25T01:35:58.676 INFO:tasks.workunit.client.0.smithi033.stdout:5/766: creat d1c/d2a/d50/d52/da6/d3e/d70/d7a/dab/ffe x:0 0 0 2024-04-25T01:35:58.679 INFO:tasks.workunit.client.0.smithi033.stdout:3/795: write d8/dd/d3d/d63/d7c/ded/dc7/ddf/f112 [684277,35524] 0 2024-04-25T01:35:58.679 INFO:tasks.workunit.client.0.smithi033.stdout:9/753: rmdir d0/d2/dd/df/d1b/dc0 39 2024-04-25T01:35:58.682 INFO:tasks.workunit.client.0.smithi033.stdout:4/807: mknod d6/d12/d38/dc0/c104 0 2024-04-25T01:35:58.683 INFO:tasks.workunit.client.0.smithi033.stdout:5/767: unlink f5 0 2024-04-25T01:35:58.684 INFO:tasks.workunit.client.0.smithi033.stdout:5/768: readlink d1c/ld4 0 2024-04-25T01:35:58.689 INFO:tasks.workunit.client.0.smithi033.stdout:3/796: dwrite d8/dd/d3d/fc8 [0,4194304] 0 2024-04-25T01:35:58.691 INFO:tasks.workunit.client.0.smithi033.stdout:3/797: fsync d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/f75 0 2024-04-25T01:35:58.696 INFO:tasks.workunit.client.0.smithi033.stdout:4/808: rename d6/d14/d35/d44/d45/d46/d57/la5 to d6/l105 0 2024-04-25T01:35:58.696 INFO:tasks.workunit.client.0.smithi033.stdout:5/769: read d1c/d2a/d50/db1/dd3/fda [2513496,113086] 0 2024-04-25T01:35:58.697 INFO:tasks.workunit.client.0.smithi033.stdout:3/798: creat d8/dd/d3d/d4f/da4/dc4/f116 x:0 0 0 2024-04-25T01:35:58.698 INFO:tasks.workunit.client.0.smithi033.stdout:5/770: readlink d1c/d2a/d50/d52/da6/d3e/d70/d9b/la4 0 2024-04-25T01:35:58.702 INFO:tasks.workunit.client.0.smithi033.stdout:4/809: unlink d6/d14/d48/lf5 0 2024-04-25T01:35:58.705 INFO:tasks.workunit.client.0.smithi033.stdout:3/799: dwrite d8/dd/d3d/dce/fee [0,4194304] 0 2024-04-25T01:35:58.707 INFO:tasks.workunit.client.0.smithi033.stdout:3/800: write d8/dd/d1d/fdd [3549858,70423] 0 2024-04-25T01:35:58.716 INFO:tasks.workunit.client.0.smithi033.stdout:3/801: creat d8/f117 x:0 0 0 2024-04-25T01:35:58.718 INFO:tasks.workunit.client.0.smithi033.stdout:3/802: creat d8/f118 x:0 0 0 2024-04-25T01:35:58.731 INFO:tasks.workunit.client.0.smithi033.stdout:9/754: truncate d0/d2/dd/fde 1893011 0 2024-04-25T01:35:58.732 INFO:tasks.workunit.client.0.smithi033.stdout:7/723: dwrite d4/f17 [0,4194304] 0 2024-04-25T01:35:58.733 INFO:tasks.workunit.client.0.smithi033.stdout:7/724: stat d4/d2e/d6b/f47 0 2024-04-25T01:35:58.736 INFO:tasks.workunit.client.0.smithi033.stdout:5/771: truncate f1b 42514 0 2024-04-25T01:35:58.737 INFO:tasks.workunit.client.0.smithi033.stdout:5/772: chown d1c/f22 3 1 2024-04-25T01:35:58.737 INFO:tasks.workunit.client.0.smithi033.stdout:9/755: mknod d0/d2/dd/d20/c107 0 2024-04-25T01:35:58.743 INFO:tasks.workunit.client.0.smithi033.stdout:7/725: getdents d4/d23/da7/dc2/dd0 0 2024-04-25T01:35:58.745 INFO:tasks.workunit.client.0.smithi033.stdout:4/810: dwrite d6/d14/fd7 [0,4194304] 0 2024-04-25T01:35:58.745 INFO:tasks.workunit.client.0.smithi033.stdout:9/756: creat d0/d2/d43/d73/f108 x:0 0 0 2024-04-25T01:35:58.747 INFO:tasks.workunit.client.0.smithi033.stdout:7/726: write d4/d23/d3f/d45/d6c/f97 [818171,51803] 0 2024-04-25T01:35:58.749 INFO:tasks.workunit.client.0.smithi033.stdout:9/757: stat d0/d2/d43/d4b/dc5/de0/d7e/cb2 0 2024-04-25T01:35:58.753 INFO:tasks.workunit.client.0.smithi033.stdout:3/803: dwrite d8/dd/d1d/f9a [0,4194304] 0 2024-04-25T01:35:58.754 INFO:tasks.workunit.client.0.smithi033.stdout:7/727: mkdir d4/d2e/d6b/d1f/d78/d86/de4 0 2024-04-25T01:35:58.755 INFO:tasks.workunit.client.0.smithi033.stdout:3/804: chown d8/dd/d1d/l49 31 1 2024-04-25T01:35:58.757 INFO:tasks.workunit.client.0.smithi033.stdout:3/805: write d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/fc1 [2860876,32171] 0 2024-04-25T01:35:58.761 INFO:tasks.workunit.client.0.smithi033.stdout:4/811: rename d6/d14/d1d/d5f/fb5 to d6/d14/d35/d44/d45/d46/db0/dcc/f106 0 2024-04-25T01:35:58.762 INFO:tasks.workunit.client.0.smithi033.stdout:5/773: write d1c/d2a/d50/d52/fb3 [470622,114387] 0 2024-04-25T01:35:58.764 INFO:tasks.workunit.client.0.smithi033.stdout:7/728: creat d4/d2e/d6d/fe5 x:0 0 0 2024-04-25T01:35:58.768 INFO:tasks.workunit.client.0.smithi033.stdout:3/806: write d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/ffd [2299,1470] 0 2024-04-25T01:35:58.774 INFO:tasks.workunit.client.0.smithi033.stdout:4/812: mkdir d6/d14/d35/d3a/de8/d107 0 2024-04-25T01:35:58.775 INFO:tasks.workunit.client.0.smithi033.stdout:9/758: getdents d0/d2/dd/d5f 0 2024-04-25T01:35:58.783 INFO:tasks.workunit.client.0.smithi033.stdout:9/759: dwrite d0/d2/dd/f24 [4194304,4194304] 0 2024-04-25T01:35:58.784 INFO:tasks.workunit.client.0.smithi033.stdout:9/760: fdatasync d0/d2/dd/d20/d4e/f4f 0 2024-04-25T01:35:58.785 INFO:tasks.workunit.client.0.smithi033.stdout:7/729: link d4/d23/d3f/f4d d4/d2e/d7f/d98/dca/fe6 0 2024-04-25T01:35:58.786 INFO:tasks.workunit.client.0.smithi033.stdout:3/807: link d8/dd/d40/cc9 d8/dd/d1d/dd1/c119 0 2024-04-25T01:35:58.788 INFO:tasks.workunit.client.0.smithi033.stdout:9/761: symlink d0/d2/d43/l109 0 2024-04-25T01:35:58.789 INFO:tasks.workunit.client.0.smithi033.stdout:5/774: truncate d1c/d2a/d50/d52/da6/d3e/d70/dae/fb5 456475 0 2024-04-25T01:35:58.791 INFO:tasks.workunit.client.0.smithi033.stdout:3/808: creat d8/dd/d3d/d63/d7c/d9d/f11a x:0 0 0 2024-04-25T01:35:58.791 INFO:tasks.workunit.client.0.smithi033.stdout:4/813: write d6/d14/d1d/f2c [1924788,116564] 0 2024-04-25T01:35:58.794 INFO:tasks.workunit.client.0.smithi033.stdout:5/775: write d1c/d2a/d50/d52/da6/d4a/fb4 [4626731,41514] 0 2024-04-25T01:35:58.797 INFO:tasks.workunit.client.0.smithi033.stdout:4/814: chown d6/d14/d35/d3a/d4d/fce 147869660 1 2024-04-25T01:35:58.801 INFO:tasks.workunit.client.0.smithi033.stdout:7/730: dwrite d4/d23/d3f/d45/d90/fb1 [0,4194304] 0 2024-04-25T01:35:58.808 INFO:tasks.workunit.client.1.smithi138.stdout:4/141: getdents d0/d1c 0 2024-04-25T01:35:58.808 INFO:tasks.workunit.client.0.smithi033.stdout:5/776: dread d1c/d2a/d50/d52/da6/d3e/fba [0,4194304] 0 2024-04-25T01:35:58.809 INFO:tasks.workunit.client.0.smithi033.stdout:3/809: dwrite d8/dd/d1d/f107 [0,4194304] 0 2024-04-25T01:35:58.809 INFO:tasks.workunit.client.0.smithi033.stdout:9/762: dwrite d0/fa2 [0,4194304] 0 2024-04-25T01:35:58.810 INFO:tasks.workunit.client.1.smithi138.stdout:7/185: creat da/de/d1b/d34/f3d x:0 0 0 2024-04-25T01:35:58.813 INFO:tasks.workunit.client.0.smithi033.stdout:4/815: symlink d6/d14/d35/d3a/d4d/d5e/ddc/l108 0 2024-04-25T01:35:58.815 INFO:tasks.workunit.client.0.smithi033.stdout:7/731: link d4/d2e/d65/f68 d4/d2e/d7f/d98/dca/fe7 0 2024-04-25T01:35:58.818 INFO:tasks.workunit.client.0.smithi033.stdout:7/732: fdatasync d4/d23/d3f/d45/d6c/fcc 0 2024-04-25T01:35:58.819 INFO:tasks.workunit.client.0.smithi033.stdout:3/810: creat d8/dd/d3d/d4f/d110/f11b x:0 0 0 2024-04-25T01:35:58.822 INFO:tasks.workunit.client.1.smithi138.stdout:4/142: dread d0/fc [0,4194304] 0 2024-04-25T01:35:58.824 INFO:tasks.workunit.client.0.smithi033.stdout:3/811: read d8/dd/f83 [1861888,87449] 0 2024-04-25T01:35:58.825 INFO:tasks.workunit.client.0.smithi033.stdout:4/816: rename d6/d14/d1d/d3b/cde to d6/d14/d1d/c109 0 2024-04-25T01:35:58.826 INFO:tasks.workunit.client.0.smithi033.stdout:3/812: write d8/dd/d11/f7f [909739,6197] 0 2024-04-25T01:35:58.828 INFO:tasks.workunit.client.0.smithi033.stdout:5/777: getdents d1c/d2a/d50/db1/dd3/ddd 0 2024-04-25T01:35:58.829 INFO:tasks.workunit.client.0.smithi033.stdout:4/817: chown d6/d14/d35/d3a/d4d/d5e/d7c/d94/fe7 1 1 2024-04-25T01:35:58.832 INFO:tasks.workunit.client.0.smithi033.stdout:4/818: getdents d6/d14/df6 0 2024-04-25T01:35:58.833 INFO:tasks.workunit.client.0.smithi033.stdout:3/813: rename d8/dd/d3d/d63/d7c/d9d/c10d to d8/dd/c11c 0 2024-04-25T01:35:58.836 INFO:tasks.workunit.client.0.smithi033.stdout:5/778: getdents d1c/d2a/d50/d52/da6 0 2024-04-25T01:35:58.837 INFO:tasks.workunit.client.0.smithi033.stdout:4/819: write d6/d12/f58 [5397428,20306] 0 2024-04-25T01:35:58.840 INFO:tasks.workunit.client.0.smithi033.stdout:3/814: unlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/f29 0 2024-04-25T01:35:58.841 INFO:tasks.workunit.client.0.smithi033.stdout:3/815: stat d8/dd/d1d/c54 0 2024-04-25T01:35:58.842 INFO:tasks.workunit.client.0.smithi033.stdout:3/816: fdatasync d8/dd/d11/f111 0 2024-04-25T01:35:58.847 INFO:tasks.workunit.client.0.smithi033.stdout:5/779: rename d1c/d2a/d50/d8d/de1/fdb to d1c/d2a/fff 0 2024-04-25T01:35:58.848 INFO:tasks.workunit.client.0.smithi033.stdout:9/763: truncate d0/d2/dd/da7/fb5 1869765 0 2024-04-25T01:35:58.850 INFO:tasks.workunit.client.0.smithi033.stdout:3/817: rename d8/dd/d11/l5b to d8/dd/d3d/d63/d7c/ded/dbf/l11d 0 2024-04-25T01:35:58.851 INFO:tasks.workunit.client.0.smithi033.stdout:7/733: dwrite d4/f7a [0,4194304] 0 2024-04-25T01:35:58.856 INFO:tasks.workunit.client.0.smithi033.stdout:4/820: truncate d6/d12/d84/fae 3633693 0 2024-04-25T01:35:58.857 INFO:tasks.workunit.client.0.smithi033.stdout:3/818: stat d8/dd/c11c 0 2024-04-25T01:35:58.858 INFO:tasks.workunit.client.0.smithi033.stdout:9/764: mknod d0/d11/c10a 0 2024-04-25T01:35:58.860 INFO:tasks.workunit.client.0.smithi033.stdout:9/765: fsync d0/d2/d43/d4b/fa8 0 2024-04-25T01:35:58.861 INFO:tasks.workunit.client.0.smithi033.stdout:7/734: creat d4/d2e/d7f/d98/dca/dd3/fe8 x:0 0 0 2024-04-25T01:35:58.862 INFO:tasks.workunit.client.0.smithi033.stdout:4/821: rename d6/d14/d35/d3a/d4c/lfc to d6/d14/d35/d3a/l10a 0 2024-04-25T01:35:58.865 INFO:tasks.workunit.client.0.smithi033.stdout:4/822: truncate d6/d14/d6d/fc1 622818 0 2024-04-25T01:35:58.865 INFO:tasks.workunit.client.0.smithi033.stdout:3/819: creat d8/dff/f11e x:0 0 0 2024-04-25T01:35:58.867 INFO:tasks.workunit.client.0.smithi033.stdout:9/766: fsync d0/d11/f47 0 2024-04-25T01:35:58.869 INFO:tasks.workunit.client.0.smithi033.stdout:5/780: getdents d1c/d2a/d50/d52 0 2024-04-25T01:35:58.872 INFO:tasks.workunit.client.0.smithi033.stdout:7/735: dread d4/d2e/d7f/d98/dca/fe7 [0,4194304] 0 2024-04-25T01:35:58.877 INFO:tasks.workunit.client.0.smithi033.stdout:3/820: mkdir d8/dd/d3d/d4f/da4/dc4/d11f 0 2024-04-25T01:35:58.881 INFO:tasks.workunit.client.0.smithi033.stdout:4/823: dwrite d6/d14/d35/d44/d45/d46/db0/dcc/f106 [0,4194304] 0 2024-04-25T01:35:58.884 INFO:tasks.workunit.client.0.smithi033.stdout:4/824: rename d6/d14/d35/d44/d45 to d6/d14/d35/d44/d45/d46/d57/d70/d87/dc7/d10b 22 2024-04-25T01:35:58.886 INFO:tasks.workunit.client.0.smithi033.stdout:5/781: creat d1c/d2a/d50/f100 x:0 0 0 2024-04-25T01:35:58.891 INFO:tasks.workunit.client.0.smithi033.stdout:5/782: mknod d1c/d53/c101 0 2024-04-25T01:35:58.893 INFO:tasks.workunit.client.0.smithi033.stdout:7/736: dread d4/d23/d3f/d45/f5e [0,4194304] 0 2024-04-25T01:35:58.898 INFO:tasks.workunit.client.0.smithi033.stdout:4/825: getdents d6/d14/d35/d3a 0 2024-04-25T01:35:58.900 INFO:tasks.workunit.client.0.smithi033.stdout:3/821: truncate d8/fa 3456832 0 2024-04-25T01:35:58.902 INFO:tasks.workunit.client.0.smithi033.stdout:5/783: creat d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e/f102 x:0 0 0 2024-04-25T01:35:58.904 INFO:tasks.workunit.client.0.smithi033.stdout:9/767: dwrite d0/f29 [0,4194304] 0 2024-04-25T01:35:58.906 INFO:tasks.workunit.client.0.smithi033.stdout:7/737: dwrite d4/d2e/d50/fcd [0,4194304] 0 2024-04-25T01:35:58.908 INFO:tasks.workunit.client.1.smithi138.stdout:5/162: rename d4/l1e to d4/da/d27/d1c/d22/l31 0 2024-04-25T01:35:58.909 INFO:tasks.workunit.client.0.smithi033.stdout:4/826: mknod d6/d14/d35/d44/dfe/d103/c10c 0 2024-04-25T01:35:58.911 INFO:tasks.workunit.client.0.smithi033.stdout:7/738: fsync d4/d2e/f4f 0 2024-04-25T01:35:58.911 INFO:tasks.workunit.client.0.smithi033.stdout:9/768: mkdir d0/d2/d43/d73/d10b 0 2024-04-25T01:35:58.912 INFO:tasks.workunit.client.0.smithi033.stdout:5/784: fsync d1c/d2a/d50/d52/da6/d4a/f72 0 2024-04-25T01:35:58.914 INFO:tasks.workunit.client.1.smithi138.stdout:3/205: getdents db/d14 0 2024-04-25T01:35:58.914 INFO:tasks.workunit.client.0.smithi033.stdout:3/822: creat d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/daa/f120 x:0 0 0 2024-04-25T01:35:58.917 INFO:tasks.workunit.client.0.smithi033.stdout:1/593: sync 2024-04-25T01:35:58.917 INFO:tasks.workunit.client.1.smithi138.stdout:6/141: truncate dc/d2e/f2f 3000673 0 2024-04-25T01:35:58.918 INFO:tasks.workunit.client.1.smithi138.stdout:6/142: write dc/f16 [972332,43803] 0 2024-04-25T01:35:58.919 INFO:tasks.workunit.client.1.smithi138.stdout:6/143: readlink la 0 2024-04-25T01:35:58.922 INFO:tasks.workunit.client.0.smithi033.stdout:3/823: rmdir d8/dd/d3d 39 2024-04-25T01:35:58.922 INFO:tasks.workunit.client.1.smithi138.stdout:4/143: unlink d0/fc 0 2024-04-25T01:35:58.923 INFO:tasks.workunit.client.0.smithi033.stdout:7/739: rmdir d4/d2e/d6b/d67 39 2024-04-25T01:35:58.924 INFO:tasks.workunit.client.0.smithi033.stdout:9/769: dwrite d0/d2/dd/f41 [0,4194304] 0 2024-04-25T01:35:58.925 INFO:tasks.workunit.client.1.smithi138.stdout:6/144: dwrite fb [0,4194304] 0 2024-04-25T01:35:58.928 INFO:tasks.workunit.client.1.smithi138.stdout:6/145: dread f8 [0,4194304] 0 2024-04-25T01:35:58.929 INFO:tasks.workunit.client.1.smithi138.stdout:6/146: dread - dc/f18 zero size 2024-04-25T01:35:58.929 INFO:tasks.workunit.client.0.smithi033.stdout:9/770: chown d0/d2/f9c 0 1 2024-04-25T01:35:58.930 INFO:tasks.workunit.client.0.smithi033.stdout:1/594: dwrite d2/d11/f91 [0,4194304] 0 2024-04-25T01:35:58.930 INFO:tasks.workunit.client.1.smithi138.stdout:6/147: readlink dc/d10/l1b 0 2024-04-25T01:35:58.930 INFO:tasks.workunit.client.1.smithi138.stdout:6/148: chown dc/c1a 6 1 2024-04-25T01:35:58.936 INFO:tasks.workunit.client.0.smithi033.stdout:5/785: dwrite d1c/d2a/d50/d52/da6/f63 [0,4194304] 0 2024-04-25T01:35:58.938 INFO:tasks.workunit.client.1.smithi138.stdout:5/163: chown d4/f9 43 1 2024-04-25T01:35:58.940 INFO:tasks.workunit.client.1.smithi138.stdout:5/164: chown d4/da/fd 947 1 2024-04-25T01:35:58.942 INFO:tasks.workunit.client.1.smithi138.stdout:7/186: truncate da/de/d13/f2d 12761 0 2024-04-25T01:35:58.942 INFO:tasks.workunit.client.1.smithi138.stdout:5/165: truncate d4/da/d27/d1c/f28 201492 0 2024-04-25T01:35:58.944 INFO:tasks.workunit.client.1.smithi138.stdout:7/187: stat da/l11 0 2024-04-25T01:35:58.945 INFO:tasks.workunit.client.1.smithi138.stdout:7/188: read da/de/d1b/f31 [658954,106306] 0 2024-04-25T01:35:58.947 INFO:tasks.workunit.client.0.smithi033.stdout:1/595: dwrite d2/d5/d75/d59/d83/fb4 [0,4194304] 0 2024-04-25T01:35:58.947 INFO:tasks.workunit.client.1.smithi138.stdout:5/166: mknod d4/da/d27/d1c/d22/c32 0 2024-04-25T01:35:58.947 INFO:tasks.workunit.client.1.smithi138.stdout:5/167: chown d4 27696219 1 2024-04-25T01:35:58.948 INFO:tasks.workunit.client.1.smithi138.stdout:4/144: truncate d0/d16/f18 779989 0 2024-04-25T01:35:58.951 INFO:tasks.workunit.client.0.smithi033.stdout:7/740: dwrite d4/d2e/d7f/fab [0,4194304] 0 2024-04-25T01:35:58.953 INFO:tasks.workunit.client.0.smithi033.stdout:1/596: write d2/d5/d1b/d46/d5f/d63/d8f/f9e [452730,111437] 0 2024-04-25T01:35:58.953 INFO:tasks.workunit.client.0.smithi033.stdout:7/741: dread - d4/d2e/d7f/d98/fe3 zero size 2024-04-25T01:35:58.954 INFO:tasks.workunit.client.1.smithi138.stdout:7/189: rename da/de/d39 to da/de/d13/d3e 0 2024-04-25T01:35:58.956 INFO:tasks.workunit.client.1.smithi138.stdout:5/168: creat d4/da/d27/f33 x:0 0 0 2024-04-25T01:35:58.958 INFO:tasks.workunit.client.1.smithi138.stdout:7/190: mknod da/de/d2e/c3f 0 2024-04-25T01:35:58.961 INFO:tasks.workunit.client.1.smithi138.stdout:7/191: write da/de/f2c [811600,83325] 0 2024-04-25T01:35:58.961 INFO:tasks.workunit.client.0.smithi033.stdout:7/742: dread d4/d2e/d6b/f9a [0,4194304] 0 2024-04-25T01:35:58.963 INFO:tasks.workunit.client.1.smithi138.stdout:5/169: getdents d4 0 2024-04-25T01:35:58.965 INFO:tasks.workunit.client.1.smithi138.stdout:7/192: symlink da/de/d1b/l40 0 2024-04-25T01:35:58.967 INFO:tasks.workunit.client.1.smithi138.stdout:5/170: creat d4/f34 x:0 0 0 2024-04-25T01:35:58.968 INFO:tasks.workunit.client.0.smithi033.stdout:7/743: getdents d4/d2e/daa 0 2024-04-25T01:35:58.970 INFO:tasks.workunit.client.0.smithi033.stdout:9/771: dread d0/d2/dd/dba/ff5 [0,4194304] 0 2024-04-25T01:35:58.970 INFO:tasks.workunit.client.1.smithi138.stdout:3/206: dread db/df/d17/d20/f29 [0,4194304] 0 2024-04-25T01:35:58.971 INFO:tasks.workunit.client.0.smithi033.stdout:4/827: truncate d6/d14/d35/d44/fea 3494852 0 2024-04-25T01:35:58.972 INFO:tasks.workunit.client.0.smithi033.stdout:7/744: mknod d4/d2e/d50/ce9 0 2024-04-25T01:35:58.974 INFO:tasks.workunit.client.1.smithi138.stdout:6/149: dwrite f8 [0,4194304] 0 2024-04-25T01:35:58.975 INFO:tasks.workunit.client.1.smithi138.stdout:6/150: fdatasync dc/f18 0 2024-04-25T01:35:58.976 INFO:tasks.workunit.client.1.smithi138.stdout:6/151: write fb [572403,35835] 0 2024-04-25T01:35:58.976 INFO:tasks.workunit.client.0.smithi033.stdout:4/828: truncate d6/d14/d35/d3a/d4d/d5e/d7c/d94/ff1 579955 0 2024-04-25T01:35:58.977 INFO:tasks.workunit.client.1.smithi138.stdout:3/207: symlink db/l3b 0 2024-04-25T01:35:58.977 INFO:tasks.workunit.client.1.smithi138.stdout:6/152: truncate dc/f18 274354 0 2024-04-25T01:35:58.978 INFO:tasks.workunit.client.1.smithi138.stdout:3/208: read db/df/d17/f2e [4035030,101633] 0 2024-04-25T01:35:58.979 INFO:tasks.workunit.client.0.smithi033.stdout:5/786: write d1c/d2a/d3d/f95 [1043247,114743] 0 2024-04-25T01:35:58.980 INFO:tasks.workunit.client.1.smithi138.stdout:3/209: chown db/df/d17/f1c 13 1 2024-04-25T01:35:58.981 INFO:tasks.workunit.client.0.smithi033.stdout:7/745: dwrite d4/f7a [0,4194304] 0 2024-04-25T01:35:58.987 INFO:tasks.workunit.client.0.smithi033.stdout:5/787: fsync f1b 0 2024-04-25T01:35:58.988 INFO:tasks.workunit.client.0.smithi033.stdout:7/746: creat d4/d23/d3f/d45/d73/fea x:0 0 0 2024-04-25T01:35:58.989 INFO:tasks.workunit.client.0.smithi033.stdout:9/772: write d0/d11/f47 [1546830,69384] 0 2024-04-25T01:35:58.990 INFO:tasks.workunit.client.0.smithi033.stdout:5/788: truncate d1c/d53/df6/ffa 267322 0 2024-04-25T01:35:58.990 INFO:tasks.workunit.client.0.smithi033.stdout:4/829: truncate d6/d14/d35/d44/d45/d46/d57/d70/d87/dc7/fbf 24755 0 2024-04-25T01:35:58.991 INFO:tasks.workunit.client.0.smithi033.stdout:7/747: dread - d4/d2e/d6d/fdf zero size 2024-04-25T01:35:58.993 INFO:tasks.workunit.client.0.smithi033.stdout:9/773: dread - d0/d2/f6c zero size 2024-04-25T01:35:58.994 INFO:tasks.workunit.client.1.smithi138.stdout:3/210: link db/df/f25 db/f3c 0 2024-04-25T01:35:58.994 INFO:tasks.workunit.client.1.smithi138.stdout:6/153: link dc/d10/c2c dc/d10/d28/c32 0 2024-04-25T01:35:58.995 INFO:tasks.workunit.client.0.smithi033.stdout:5/789: fdatasync d1c/f22 0 2024-04-25T01:35:58.995 INFO:tasks.workunit.client.0.smithi033.stdout:1/597: dread d2/d5/d1b/d46/d5f/d63/d64/f6a [0,4194304] 0 2024-04-25T01:35:58.998 INFO:tasks.workunit.client.1.smithi138.stdout:7/193: dwrite da/fc [0,4194304] 0 2024-04-25T01:35:58.998 INFO:tasks.workunit.client.1.smithi138.stdout:4/145: unlink d0/d16/c22 0 2024-04-25T01:35:58.998 INFO:tasks.workunit.client.1.smithi138.stdout:5/171: write d4/da/d27/f2d [42430,128959] 0 2024-04-25T01:35:58.999 INFO:tasks.workunit.client.1.smithi138.stdout:7/194: fdatasync da/de/f1c 0 2024-04-25T01:35:59.000 INFO:tasks.workunit.client.1.smithi138.stdout:3/211: mknod db/df/c3d 0 2024-04-25T01:35:59.001 INFO:tasks.workunit.client.0.smithi033.stdout:9/774: read d0/d2/dd/d20/d4e/f90 [1250523,44063] 0 2024-04-25T01:35:59.001 INFO:tasks.workunit.client.1.smithi138.stdout:5/172: write d4/da/d27/d1c/f26 [3372987,27057] 0 2024-04-25T01:35:59.001 INFO:tasks.workunit.client.1.smithi138.stdout:7/195: write da/de/f29 [255693,2276] 0 2024-04-25T01:35:59.002 INFO:tasks.workunit.client.1.smithi138.stdout:3/212: chown db/df/d19/f36 46110 1 2024-04-25T01:35:59.003 INFO:tasks.workunit.client.1.smithi138.stdout:3/213: readlink db/df/l13 0 2024-04-25T01:35:59.004 INFO:tasks.workunit.client.0.smithi033.stdout:1/598: creat d2/fb6 x:0 0 0 2024-04-25T01:35:59.004 INFO:tasks.workunit.client.0.smithi033.stdout:4/830: dwrite d6/d14/d35/d44/dfe/d103/fd6 [0,4194304] 0 2024-04-25T01:35:59.008 INFO:tasks.workunit.client.1.smithi138.stdout:6/154: rename f1 to dc/d10/f33 0 2024-04-25T01:35:59.010 INFO:tasks.workunit.client.1.smithi138.stdout:5/173: rmdir d4/da/d27/d1c 39 2024-04-25T01:35:59.010 INFO:tasks.workunit.client.0.smithi033.stdout:9/775: mknod d0/d2/d43/d4b/dc5/c10c 0 2024-04-25T01:35:59.011 INFO:tasks.workunit.client.1.smithi138.stdout:5/174: chown d4/da/f20 148917672 1 2024-04-25T01:35:59.011 INFO:tasks.workunit.client.0.smithi033.stdout:1/599: symlink d2/d5/d75/d59/d83/d98/lb7 0 2024-04-25T01:35:59.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/214: truncate db/f12 5715 0 2024-04-25T01:35:59.015 INFO:tasks.workunit.client.0.smithi033.stdout:1/600: unlink d2/d2e/f73 0 2024-04-25T01:35:59.016 INFO:tasks.workunit.client.0.smithi033.stdout:7/748: write d4/d2e/d50/f55 [1234169,95272] 0 2024-04-25T01:35:59.019 INFO:tasks.workunit.client.1.smithi138.stdout:3/215: symlink db/df/d17/d20/l3e 0 2024-04-25T01:35:59.020 INFO:tasks.workunit.client.1.smithi138.stdout:5/175: mkdir d4/da/d27/d1c/d22/d35 0 2024-04-25T01:35:59.020 INFO:tasks.workunit.client.1.smithi138.stdout:4/146: getdents d0/d1c/d23 0 2024-04-25T01:35:59.020 INFO:tasks.workunit.client.0.smithi033.stdout:9/776: creat d0/d2/dd/f10d x:0 0 0 2024-04-25T01:35:59.022 INFO:tasks.workunit.client.0.smithi033.stdout:7/749: read - d4/d2e/f63 zero size 2024-04-25T01:35:59.022 INFO:tasks.workunit.client.1.smithi138.stdout:6/155: link f6 dc/d10/d28/f34 0 2024-04-25T01:35:59.023 INFO:tasks.workunit.client.0.smithi033.stdout:5/790: dwrite d1c/d2a/d3d/f46 [0,4194304] 0 2024-04-25T01:35:59.024 INFO:tasks.workunit.client.1.smithi138.stdout:3/216: creat db/f3f x:0 0 0 2024-04-25T01:35:59.025 INFO:tasks.workunit.client.1.smithi138.stdout:5/176: chown d4/c25 69 1 2024-04-25T01:35:59.025 INFO:tasks.workunit.client.0.smithi033.stdout:1/601: link d2/d5/d75/d59/fae d2/d2e/d34/d60/fb8 0 2024-04-25T01:35:59.026 INFO:tasks.workunit.client.1.smithi138.stdout:4/147: readlink d0/l1f 0 2024-04-25T01:35:59.026 INFO:tasks.workunit.client.1.smithi138.stdout:3/217: dread db/f3c [0,4194304] 0 2024-04-25T01:35:59.027 INFO:tasks.workunit.client.1.smithi138.stdout:6/156: creat dc/f35 x:0 0 0 2024-04-25T01:35:59.030 INFO:tasks.workunit.client.0.smithi033.stdout:7/750: symlink d4/d23/d3f/d45/d90/leb 0 2024-04-25T01:35:59.030 INFO:tasks.workunit.client.0.smithi033.stdout:5/791: truncate d1c/d2a/d50/d52/da6/d4a/f72 4586842 0 2024-04-25T01:35:59.030 INFO:tasks.workunit.client.0.smithi033.stdout:7/751: read d4/d23/d3f/d45/d90/fb1 [541512,36770] 0 2024-04-25T01:35:59.031 INFO:tasks.workunit.client.0.smithi033.stdout:7/752: chown d4/d2e/d6d/fe5 820526 1 2024-04-25T01:35:59.033 INFO:tasks.workunit.client.1.smithi138.stdout:3/218: mkdir db/d14/d40 0 2024-04-25T01:35:59.033 INFO:tasks.workunit.client.0.smithi033.stdout:9/777: mknod d0/d2/dd/df/c10e 0 2024-04-25T01:35:59.034 INFO:tasks.workunit.client.1.smithi138.stdout:3/219: dread - db/df/d19/f36 zero size 2024-04-25T01:35:59.035 INFO:tasks.workunit.client.1.smithi138.stdout:6/157: dwrite dc/d10/f25 [0,4194304] 0 2024-04-25T01:35:59.036 INFO:tasks.workunit.client.1.smithi138.stdout:6/158: readlink la 0 2024-04-25T01:35:59.038 INFO:tasks.workunit.client.1.smithi138.stdout:6/159: truncate dc/d2e/f31 527274 0 2024-04-25T01:35:59.039 INFO:tasks.workunit.client.0.smithi033.stdout:1/602: symlink d2/lb9 0 2024-04-25T01:35:59.041 INFO:tasks.workunit.client.0.smithi033.stdout:7/753: creat d4/d2e/d6b/d1f/fec x:0 0 0 2024-04-25T01:35:59.041 INFO:tasks.workunit.client.1.smithi138.stdout:3/220: stat db/df/d17/d23/c3a 0 2024-04-25T01:35:59.044 INFO:tasks.workunit.client.1.smithi138.stdout:6/160: rename dc/d10/c23 to dc/d2e/c36 0 2024-04-25T01:35:59.049 INFO:tasks.workunit.client.0.smithi033.stdout:1/603: mkdir d2/d2e/d49/dba 0 2024-04-25T01:35:59.049 INFO:tasks.workunit.client.1.smithi138.stdout:5/177: dread d4/f18 [0,4194304] 0 2024-04-25T01:35:59.049 INFO:tasks.workunit.client.1.smithi138.stdout:3/221: mkdir db/d14/d40/d41 0 2024-04-25T01:35:59.050 INFO:tasks.workunit.client.0.smithi033.stdout:1/604: chown d2/d11/f1e 359943 1 2024-04-25T01:35:59.050 INFO:tasks.workunit.client.0.smithi033.stdout:7/754: mkdir d4/d23/d3f/d45/d90/ded 0 2024-04-25T01:35:59.050 INFO:tasks.workunit.client.1.smithi138.stdout:5/178: write d4/f34 [947848,25368] 0 2024-04-25T01:35:59.051 INFO:tasks.workunit.client.0.smithi033.stdout:9/778: mkdir d0/d2/d43/d4b/dc5/de0/d42/dea/d105/d10f 0 2024-04-25T01:35:59.053 INFO:tasks.workunit.client.0.smithi033.stdout:5/792: link d1c/d2a/d50/d5b/fd7 d1c/d2a/d50/d52/da6/d3e/d70/dae/f103 0 2024-04-25T01:35:59.055 INFO:tasks.workunit.client.1.smithi138.stdout:6/161: rmdir dc/d10 39 2024-04-25T01:35:59.056 INFO:tasks.workunit.client.1.smithi138.stdout:4/148: dwrite d0/f12 [0,4194304] 0 2024-04-25T01:35:59.056 INFO:tasks.workunit.client.0.smithi033.stdout:5/793: fdatasync d1c/d2a/d50/d52/da6/d4a/f7f 0 2024-04-25T01:35:59.057 INFO:tasks.workunit.client.0.smithi033.stdout:7/755: mkdir d4/d2e/d6b/da9/dee 0 2024-04-25T01:35:59.057 INFO:tasks.workunit.client.1.smithi138.stdout:3/222: dread db/df/d17/f1c [0,4194304] 0 2024-04-25T01:35:59.061 INFO:tasks.workunit.client.0.smithi033.stdout:5/794: mkdir d1c/d2a/d50/d52/d104 0 2024-04-25T01:35:59.062 INFO:tasks.workunit.client.1.smithi138.stdout:3/223: read db/fe [1377213,125291] 0 2024-04-25T01:35:59.062 INFO:tasks.workunit.client.1.smithi138.stdout:5/179: dread d4/da/d27/d1c/f26 [0,4194304] 0 2024-04-25T01:35:59.062 INFO:tasks.workunit.client.0.smithi033.stdout:9/779: mkdir d0/d110 0 2024-04-25T01:35:59.064 INFO:tasks.workunit.client.1.smithi138.stdout:3/224: fdatasync db/f3c 0 2024-04-25T01:35:59.066 INFO:tasks.workunit.client.1.smithi138.stdout:5/180: mknod d4/da/d27/d1c/d22/d35/c36 0 2024-04-25T01:35:59.066 INFO:tasks.workunit.client.0.smithi033.stdout:9/780: stat d0/d2/dd/ca 0 2024-04-25T01:35:59.068 INFO:tasks.workunit.client.0.smithi033.stdout:7/756: dwrite d4/d2e/d7f/d98/dca/dd3/fe8 [0,4194304] 0 2024-04-25T01:35:59.072 INFO:tasks.workunit.client.1.smithi138.stdout:3/225: dwrite db/d14/f32 [0,4194304] 0 2024-04-25T01:35:59.075 INFO:tasks.workunit.client.1.smithi138.stdout:5/181: link d4/l5 d4/da/d27/d1c/d22/d35/l37 0 2024-04-25T01:35:59.076 INFO:tasks.workunit.client.0.smithi033.stdout:7/757: dwrite d4/f7a [0,4194304] 0 2024-04-25T01:35:59.077 INFO:tasks.workunit.client.0.smithi033.stdout:9/781: creat d0/d2/d43/d4b/dc5/de0/d42/dea/d105/f111 x:0 0 0 2024-04-25T01:35:59.078 INFO:tasks.workunit.client.1.smithi138.stdout:5/182: read - d4/da/d27/f33 zero size 2024-04-25T01:35:59.079 INFO:tasks.workunit.client.0.smithi033.stdout:7/758: write d4/d2e/d50/f55 [615198,72867] 0 2024-04-25T01:35:59.081 INFO:tasks.workunit.client.1.smithi138.stdout:5/183: dread d4/da/d27/d1c/f28 [0,4194304] 0 2024-04-25T01:35:59.082 INFO:tasks.workunit.client.1.smithi138.stdout:3/226: dwrite db/df/d19/d1a/f21 [0,4194304] 0 2024-04-25T01:35:59.087 INFO:tasks.workunit.client.1.smithi138.stdout:5/184: symlink d4/da/d27/d1c/d22/d35/l38 0 2024-04-25T01:35:59.088 INFO:tasks.workunit.client.1.smithi138.stdout:3/227: dread db/d14/f32 [0,4194304] 0 2024-04-25T01:35:59.090 INFO:tasks.workunit.client.1.smithi138.stdout:6/162: dread dc/d10/d28/f2a [0,4194304] 0 2024-04-25T01:35:59.090 INFO:tasks.workunit.client.1.smithi138.stdout:5/185: mknod d4/da/c39 0 2024-04-25T01:35:59.091 INFO:tasks.workunit.client.0.smithi033.stdout:1/605: dwrite d2/d11/f30 [0,4194304] 0 2024-04-25T01:35:59.092 INFO:tasks.workunit.client.1.smithi138.stdout:6/163: chown dc/d10/f27 40 1 2024-04-25T01:35:59.093 INFO:tasks.workunit.client.1.smithi138.stdout:5/186: rmdir d4/da/d27/d1c 39 2024-04-25T01:35:59.095 INFO:tasks.workunit.client.0.smithi033.stdout:7/759: fdatasync d4/d2e/d6b/d1f/f82 0 2024-04-25T01:35:59.098 INFO:tasks.workunit.client.1.smithi138.stdout:5/187: dread d4/da/d27/d1c/f28 [0,4194304] 0 2024-04-25T01:35:59.098 INFO:tasks.workunit.client.1.smithi138.stdout:3/228: dwrite db/df/d19/f36 [0,4194304] 0 2024-04-25T01:35:59.099 INFO:tasks.workunit.client.0.smithi033.stdout:7/760: write d4/d23/d3f/f77 [1282891,21436] 0 2024-04-25T01:35:59.099 INFO:tasks.workunit.client.1.smithi138.stdout:5/188: stat d4/da/d27/f2d 0 2024-04-25T01:35:59.100 INFO:tasks.workunit.client.1.smithi138.stdout:3/229: write db/f3f [46351,11169] 0 2024-04-25T01:35:59.101 INFO:tasks.workunit.client.1.smithi138.stdout:5/189: truncate d4/f34 1684744 0 2024-04-25T01:35:59.101 INFO:tasks.workunit.client.1.smithi138.stdout:3/230: stat db/df/d19/d1a/f21 0 2024-04-25T01:35:59.102 INFO:tasks.workunit.client.0.smithi033.stdout:6/868: sync 2024-04-25T01:35:59.102 INFO:tasks.workunit.client.0.smithi033.stdout:5/795: mknod d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/c105 0 2024-04-25T01:35:59.103 INFO:tasks.workunit.client.0.smithi033.stdout:0/639: sync 2024-04-25T01:35:59.103 INFO:tasks.workunit.client.0.smithi033.stdout:2/768: sync 2024-04-25T01:35:59.104 INFO:tasks.workunit.client.1.smithi138.stdout:3/231: fdatasync db/df/f25 0 2024-04-25T01:35:59.104 INFO:tasks.workunit.client.0.smithi033.stdout:8/748: sync 2024-04-25T01:35:59.108 INFO:tasks.workunit.client.1.smithi138.stdout:3/232: link db/df/d17/d20/l28 db/df/d17/d23/d34/l42 0 2024-04-25T01:35:59.109 INFO:tasks.workunit.client.1.smithi138.stdout:3/233: chown db/df/d17/d23/l39 66268816 1 2024-04-25T01:35:59.111 INFO:tasks.workunit.client.1.smithi138.stdout:3/234: symlink db/l43 0 2024-04-25T01:35:59.112 INFO:tasks.workunit.client.0.smithi033.stdout:0/640: getdents d7/d33/dcd 0 2024-04-25T01:35:59.112 INFO:tasks.workunit.client.1.smithi138.stdout:3/235: write f1 [5057032,61899] 0 2024-04-25T01:35:59.113 INFO:tasks.workunit.client.1.smithi138.stdout:3/236: chown db/df/d19/d1a/l22 1430716206 1 2024-04-25T01:35:59.115 INFO:tasks.workunit.client.1.smithi138.stdout:3/237: creat db/df/d19/d1a/f44 x:0 0 0 2024-04-25T01:35:59.116 INFO:tasks.workunit.client.0.smithi033.stdout:8/749: rename d6/d45/d51/d55/d84/dc8 to d6/db/d80/d117 0 2024-04-25T01:35:59.117 INFO:tasks.workunit.client.1.smithi138.stdout:3/238: mknod db/df/d17/d20/c45 0 2024-04-25T01:35:59.119 INFO:tasks.workunit.client.1.smithi138.stdout:6/164: dread f5 [0,4194304] 0 2024-04-25T01:35:59.119 INFO:tasks.workunit.client.0.smithi033.stdout:7/761: dread d4/d23/fc5 [4194304,4194304] 0 2024-04-25T01:35:59.121 INFO:tasks.workunit.client.0.smithi033.stdout:7/762: fdatasync d4/d10/fdb 0 2024-04-25T01:35:59.121 INFO:tasks.workunit.client.1.smithi138.stdout:3/239: link db/f3f db/df/d19/d1a/f46 0 2024-04-25T01:35:59.121 INFO:tasks.workunit.client.1.smithi138.stdout:3/240: chown f6 424231 1 2024-04-25T01:35:59.122 INFO:tasks.workunit.client.1.smithi138.stdout:6/165: mkdir dc/d2e/d37 0 2024-04-25T01:35:59.122 INFO:tasks.workunit.client.0.smithi033.stdout:6/869: rename d4/d24/d36/dbd/dde/cef to d4/d1d/d3e/d41/d5c/d7f/da4/dcc/c129 0 2024-04-25T01:35:59.122 INFO:tasks.workunit.client.0.smithi033.stdout:7/763: chown d4/d2e/d6b/d1f/fc7 8809 1 2024-04-25T01:35:59.123 INFO:tasks.workunit.client.1.smithi138.stdout:3/241: chown db/df/d19/d1a/f46 11 1 2024-04-25T01:35:59.124 INFO:tasks.workunit.client.1.smithi138.stdout:6/166: mknod dc/d2e/d37/c38 0 2024-04-25T01:35:59.125 INFO:tasks.workunit.client.1.smithi138.stdout:3/242: truncate db/f12 667593 0 2024-04-25T01:35:59.125 INFO:tasks.workunit.client.1.smithi138.stdout:6/167: fsync dc/d10/d28/f2b 0 2024-04-25T01:35:59.126 INFO:tasks.workunit.client.1.smithi138.stdout:3/243: write db/df/d19/f36 [1295230,73014] 0 2024-04-25T01:35:59.127 INFO:tasks.workunit.client.0.smithi033.stdout:6/870: dread d4/d24/d36/d42/fca [0,4194304] 0 2024-04-25T01:35:59.128 INFO:tasks.workunit.client.1.smithi138.stdout:3/244: chown db/df/l18 195956479 1 2024-04-25T01:35:59.128 INFO:tasks.workunit.client.0.smithi033.stdout:9/782: rmdir d0/d2/d43/d4b/dc5/de0 39 2024-04-25T01:35:59.129 INFO:tasks.workunit.client.1.smithi138.stdout:3/245: chown db/df/l18 215199452 1 2024-04-25T01:35:59.131 INFO:tasks.workunit.client.1.smithi138.stdout:3/246: creat db/df/d19/d1a/f47 x:0 0 0 2024-04-25T01:35:59.133 INFO:tasks.workunit.client.0.smithi033.stdout:6/871: mknod d4/d24/d36/dbd/c12a 0 2024-04-25T01:35:59.133 INFO:tasks.workunit.client.1.smithi138.stdout:3/247: symlink db/l48 0 2024-04-25T01:35:59.135 INFO:tasks.workunit.client.0.smithi033.stdout:9/783: mknod d0/d2/dd/da7/c112 0 2024-04-25T01:35:59.137 INFO:tasks.workunit.client.0.smithi033.stdout:6/872: truncate d4/d24/d36/dbd/fd9 545439 0 2024-04-25T01:35:59.138 INFO:tasks.workunit.client.0.smithi033.stdout:8/750: write d6/d45/d52/f2b [1197086,52247] 0 2024-04-25T01:35:59.140 INFO:tasks.workunit.client.0.smithi033.stdout:9/784: unlink d0/d2/dd/f88 0 2024-04-25T01:35:59.144 INFO:tasks.workunit.client.0.smithi033.stdout:2/769: dwrite da/d27/d31/d40/d8a/f53 [0,4194304] 0 2024-04-25T01:35:59.146 INFO:tasks.workunit.client.0.smithi033.stdout:7/764: write d4/d2e/d6b/f56 [3837747,10734] 0 2024-04-25T01:35:59.149 INFO:tasks.workunit.client.0.smithi033.stdout:8/751: unlink d6/db/cda 0 2024-04-25T01:35:59.150 INFO:tasks.workunit.client.0.smithi033.stdout:9/785: dread - d0/d2/d43/d4b/dc5/de0/d42/dea/d105/f111 zero size 2024-04-25T01:35:59.153 INFO:tasks.workunit.client.0.smithi033.stdout:2/770: dread - da/d27/d31/fe9 zero size 2024-04-25T01:35:59.153 INFO:tasks.workunit.client.0.smithi033.stdout:9/786: read - d0/d11/f4c zero size 2024-04-25T01:35:59.154 INFO:tasks.workunit.client.1.smithi138.stdout:6/168: dwrite dc/f1f [0,4194304] 0 2024-04-25T01:35:59.154 INFO:tasks.workunit.client.1.smithi138.stdout:6/169: dread - dc/d10/d28/f29 zero size 2024-04-25T01:35:59.154 INFO:tasks.workunit.client.1.smithi138.stdout:3/248: dwrite db/df/f25 [0,4194304] 0 2024-04-25T01:35:59.156 INFO:tasks.workunit.client.0.smithi033.stdout:2/771: stat da/d50/d6a/d43/d6c/df5 0 2024-04-25T01:35:59.156 INFO:tasks.workunit.client.0.smithi033.stdout:0/641: dread d7/d2d/d43/d44/f63 [0,4194304] 0 2024-04-25T01:35:59.157 INFO:tasks.workunit.client.0.smithi033.stdout:7/765: creat d4/d2e/d6b/da9/db3/fef x:0 0 0 2024-04-25T01:35:59.157 INFO:tasks.workunit.client.0.smithi033.stdout:6/873: dwrite d4/d1d/d3e/d41/d5c/d7f/da4/faf [0,4194304] 0 2024-04-25T01:35:59.159 INFO:tasks.workunit.client.0.smithi033.stdout:7/766: chown d4/d2e/d6b/d1f/fc7 9785723 1 2024-04-25T01:35:59.160 INFO:tasks.workunit.client.0.smithi033.stdout:2/772: mkdir da/d5b/d5e/d76/dfc 0 2024-04-25T01:35:59.161 INFO:tasks.workunit.client.1.smithi138.stdout:3/249: rename db/d14/f2f to db/df/d19/d1a/f49 0 2024-04-25T01:35:59.162 INFO:tasks.workunit.client.1.smithi138.stdout:3/250: rmdir db/df/d17/d20 39 2024-04-25T01:35:59.163 INFO:tasks.workunit.client.0.smithi033.stdout:6/874: write d4/d24/d36/d42/d97/fa6 [1251733,61149] 0 2024-04-25T01:35:59.170 INFO:tasks.workunit.client.0.smithi033.stdout:6/875: chown d4/d1d/d3e/d41/d5c/df6/lff 25302426 1 2024-04-25T01:35:59.170 INFO:tasks.workunit.client.0.smithi033.stdout:0/642: symlink d7/d33/dcd/lde 0 2024-04-25T01:35:59.173 INFO:tasks.workunit.client.0.smithi033.stdout:0/643: creat d7/d33/fdf x:0 0 0 2024-04-25T01:35:59.174 INFO:tasks.workunit.client.0.smithi033.stdout:7/767: link d4/d2e/daa/cde d4/d23/da7/dc2/dd0/cf0 0 2024-04-25T01:35:59.177 INFO:tasks.workunit.client.0.smithi033.stdout:6/876: truncate d4/f6 652530 0 2024-04-25T01:35:59.178 INFO:tasks.workunit.client.0.smithi033.stdout:0/644: mknod d7/d2d/d4d/d9c/db2/ce0 0 2024-04-25T01:35:59.188 INFO:tasks.workunit.client.0.smithi033.stdout:9/787: write d0/d2/dd/f15 [910675,98554] 0 2024-04-25T01:35:59.188 INFO:tasks.workunit.client.1.smithi138.stdout:3/251: dwrite db/fe [0,4194304] 0 2024-04-25T01:35:59.189 INFO:tasks.workunit.client.0.smithi033.stdout:2/773: write da/d27/d31/d40/fa1 [1301150,3275] 0 2024-04-25T01:35:59.192 INFO:tasks.workunit.client.1.smithi138.stdout:3/252: creat db/df/d17/f4a x:0 0 0 2024-04-25T01:35:59.197 INFO:tasks.workunit.client.1.smithi138.stdout:6/170: dread f6 [0,4194304] 0 2024-04-25T01:35:59.199 INFO:tasks.workunit.client.1.smithi138.stdout:6/171: mknod dc/c39 0 2024-04-25T01:35:59.199 INFO:tasks.workunit.client.0.smithi033.stdout:6/877: write d4/d1d/d3e/d41/d54/f7b [874249,82308] 0 2024-04-25T01:35:59.200 INFO:tasks.workunit.client.1.smithi138.stdout:6/172: creat dc/d2e/f3a x:0 0 0 2024-04-25T01:35:59.201 INFO:tasks.workunit.client.1.smithi138.stdout:6/173: chown dc/d10/f27 3238 1 2024-04-25T01:35:59.201 INFO:tasks.workunit.client.1.smithi138.stdout:6/174: chown dc/c30 14496 1 2024-04-25T01:35:59.201 INFO:tasks.workunit.client.0.smithi033.stdout:9/788: creat d0/d2/dd/df/d1b/f113 x:0 0 0 2024-04-25T01:35:59.202 INFO:tasks.workunit.client.0.smithi033.stdout:2/774: write da/d2c/dcd/fc4 [903261,56492] 0 2024-04-25T01:35:59.202 INFO:tasks.workunit.client.1.smithi138.stdout:6/175: symlink dc/d10/l3b 0 2024-04-25T01:35:59.204 INFO:tasks.workunit.client.0.smithi033.stdout:6/878: truncate d4/d1d/d3e/d70/f108 2415391 0 2024-04-25T01:35:59.204 INFO:tasks.workunit.client.1.smithi138.stdout:6/176: creat dc/d10/d28/f3c x:0 0 0 2024-04-25T01:35:59.204 INFO:tasks.workunit.client.0.smithi033.stdout:7/768: dread d4/f11 [4194304,4194304] 0 2024-04-25T01:35:59.205 INFO:tasks.workunit.client.0.smithi033.stdout:0/645: dread d7/d2d/f3d [0,4194304] 0 2024-04-25T01:35:59.206 INFO:tasks.workunit.client.1.smithi138.stdout:6/177: creat dc/f3d x:0 0 0 2024-04-25T01:35:59.207 INFO:tasks.workunit.client.1.smithi138.stdout:6/178: chown dc/d10 2 1 2024-04-25T01:35:59.207 INFO:tasks.workunit.client.0.smithi033.stdout:7/769: chown d4/d2e/d6d/fe5 7564 1 2024-04-25T01:35:59.207 INFO:tasks.workunit.client.0.smithi033.stdout:9/789: mknod d0/d2/d43/d4b/dc5/de0/d7e/c114 0 2024-04-25T01:35:59.208 INFO:tasks.workunit.client.1.smithi138.stdout:6/179: write f5 [2151659,1090] 0 2024-04-25T01:35:59.208 INFO:tasks.workunit.client.0.smithi033.stdout:6/879: mkdir d4/d1d/d3e/d41/d5c/df6/d12b 0 2024-04-25T01:35:59.209 INFO:tasks.workunit.client.1.smithi138.stdout:6/180: write dc/d10/d28/f2b [2868628,30669] 0 2024-04-25T01:35:59.209 INFO:tasks.workunit.client.0.smithi033.stdout:6/880: fdatasync d4/dfc/f121 0 2024-04-25T01:35:59.210 INFO:tasks.workunit.client.0.smithi033.stdout:0/646: creat d7/d2d/d4d/d9c/d61/fe1 x:0 0 0 2024-04-25T01:35:59.210 INFO:tasks.workunit.client.1.smithi138.stdout:6/181: creat dc/d2e/f3e x:0 0 0 2024-04-25T01:35:59.211 INFO:tasks.workunit.client.1.smithi138.stdout:6/182: chown dc/f3d 1264783927 1 2024-04-25T01:35:59.212 INFO:tasks.workunit.client.0.smithi033.stdout:7/770: mkdir d4/d23/d3f/d45/d73/df1 0 2024-04-25T01:35:59.216 INFO:tasks.workunit.client.0.smithi033.stdout:9/790: rename d0/d2/dd/d5f/fce to d0/d2/d43/d4b/dc5/de0/d7e/d58/f115 0 2024-04-25T01:35:59.216 INFO:tasks.workunit.client.1.smithi138.stdout:6/183: rmdir dc/d2e 39 2024-04-25T01:35:59.220 INFO:tasks.workunit.client.0.smithi033.stdout:0/647: creat d7/d33/dcd/fe2 x:0 0 0 2024-04-25T01:35:59.221 INFO:tasks.workunit.client.0.smithi033.stdout:9/791: symlink d0/d2/d43/d4b/dc5/de0/l116 0 2024-04-25T01:35:59.222 INFO:tasks.workunit.client.1.smithi138.stdout:6/184: dwrite dc/d10/f25 [0,4194304] 0 2024-04-25T01:35:59.222 INFO:tasks.workunit.client.0.smithi033.stdout:9/792: chown d0/d2/dd/df/f5c 392 1 2024-04-25T01:35:59.224 INFO:tasks.workunit.client.0.smithi033.stdout:2/775: truncate da/d5b/d5e/f6f 3204822 0 2024-04-25T01:35:59.226 INFO:tasks.workunit.client.1.smithi138.stdout:6/185: unlink dc/d2e/c36 0 2024-04-25T01:35:59.227 INFO:tasks.workunit.client.1.smithi138.stdout:6/186: mknod dc/d2e/d37/c3f 0 2024-04-25T01:35:59.228 INFO:tasks.workunit.client.1.smithi138.stdout:6/187: chown dc/d10/d28/f34 1 1 2024-04-25T01:35:59.229 INFO:tasks.workunit.client.0.smithi033.stdout:0/648: creat d7/d2d/d4d/d9c/d61/fe3 x:0 0 0 2024-04-25T01:35:59.231 INFO:tasks.workunit.client.1.smithi138.stdout:6/188: dread dc/d10/d28/f2a [0,4194304] 0 2024-04-25T01:35:59.231 INFO:tasks.workunit.client.0.smithi033.stdout:2/776: dread da/d50/f58 [0,4194304] 0 2024-04-25T01:35:59.232 INFO:tasks.workunit.client.0.smithi033.stdout:6/881: truncate d4/d24/d36/f6c 2472100 0 2024-04-25T01:35:59.232 INFO:tasks.workunit.client.0.smithi033.stdout:7/771: write d4/f3d [1335963,98819] 0 2024-04-25T01:35:59.233 INFO:tasks.workunit.client.0.smithi033.stdout:0/649: creat d7/da/d80/fe4 x:0 0 0 2024-04-25T01:35:59.234 INFO:tasks.workunit.client.1.smithi138.stdout:6/189: dread f6 [0,4194304] 0 2024-04-25T01:35:59.235 INFO:tasks.workunit.client.0.smithi033.stdout:7/772: chown d4/d2e/d6b/d1f/fec 0 1 2024-04-25T01:35:59.235 INFO:tasks.workunit.client.1.smithi138.stdout:6/190: creat dc/d10/d28/f40 x:0 0 0 2024-04-25T01:35:59.236 INFO:tasks.workunit.client.0.smithi033.stdout:2/777: rmdir da/d5b/d5e/d76 39 2024-04-25T01:35:59.238 INFO:tasks.workunit.client.1.smithi138.stdout:0/210: sync 2024-04-25T01:35:59.238 INFO:tasks.workunit.client.1.smithi138.stdout:9/215: sync 2024-04-25T01:35:59.238 INFO:tasks.workunit.client.1.smithi138.stdout:6/191: creat dc/d2e/d37/f41 x:0 0 0 2024-04-25T01:35:59.238 INFO:tasks.workunit.client.1.smithi138.stdout:6/192: dread - dc/d2e/f3e zero size 2024-04-25T01:35:59.239 INFO:tasks.workunit.client.1.smithi138.stdout:0/211: write d6/df/f10 [179138,55204] 0 2024-04-25T01:35:59.239 INFO:tasks.workunit.client.0.smithi033.stdout:7/773: symlink d4/d23/da7/lf2 0 2024-04-25T01:35:59.240 INFO:tasks.workunit.client.1.smithi138.stdout:0/212: stat d6/df/f18 0 2024-04-25T01:35:59.241 INFO:tasks.workunit.client.1.smithi138.stdout:9/216: rename d4/d8/d2c/l30 to d4/d1a/d33/l47 0 2024-04-25T01:35:59.244 INFO:tasks.workunit.client.1.smithi138.stdout:6/193: dread dc/d10/f25 [0,4194304] 0 2024-04-25T01:35:59.244 INFO:tasks.workunit.client.0.smithi033.stdout:7/774: creat d4/d2e/d6b/da9/ff3 x:0 0 0 2024-04-25T01:35:59.245 INFO:tasks.workunit.client.1.smithi138.stdout:9/217: mknod d4/d1a/d33/c48 0 2024-04-25T01:35:59.245 INFO:tasks.workunit.client.1.smithi138.stdout:6/194: write dc/f35 [859126,56280] 0 2024-04-25T01:35:59.246 INFO:tasks.workunit.client.0.smithi033.stdout:7/775: write d4/d2e/d6b/d1f/d78/d86/fe1 [525378,31689] 0 2024-04-25T01:35:59.246 INFO:tasks.workunit.client.1.smithi138.stdout:0/213: link d6/f1f d6/df/d3a/f4a 0 2024-04-25T01:35:59.247 INFO:tasks.workunit.client.1.smithi138.stdout:9/218: unlink c0 0 2024-04-25T01:35:59.249 INFO:tasks.workunit.client.1.smithi138.stdout:9/219: truncate d4/d1a/f20 153980 0 2024-04-25T01:35:59.250 INFO:tasks.workunit.client.0.smithi033.stdout:7/776: chown d4/d2e/d6b/d67 126351 1 2024-04-25T01:35:59.250 INFO:tasks.workunit.client.1.smithi138.stdout:0/214: creat d6/d19/f4b x:0 0 0 2024-04-25T01:35:59.252 INFO:tasks.workunit.client.0.smithi033.stdout:2/778: truncate da/d5b/d5e/d76/f90 268308 0 2024-04-25T01:35:59.253 INFO:tasks.workunit.client.1.smithi138.stdout:6/195: dwrite dc/d10/f25 [4194304,4194304] 0 2024-04-25T01:35:59.254 INFO:tasks.workunit.client.1.smithi138.stdout:6/196: fdatasync dc/d10/d28/f40 0 2024-04-25T01:35:59.254 INFO:tasks.workunit.client.0.smithi033.stdout:6/882: dwrite d4/d1d/d3e/d41/d5c/d7f/d116/f8c [0,4194304] 0 2024-04-25T01:35:59.262 INFO:tasks.workunit.client.0.smithi033.stdout:2/779: getdents da/d2c/db8 0 2024-04-25T01:35:59.262 INFO:tasks.workunit.client.1.smithi138.stdout:6/197: dwrite dc/d2e/f31 [0,4194304] 0 2024-04-25T01:35:59.263 INFO:tasks.workunit.client.0.smithi033.stdout:0/650: dread d7/d2d/d4d/f5c [0,4194304] 0 2024-04-25T01:35:59.264 INFO:tasks.workunit.client.0.smithi033.stdout:7/777: dwrite d4/d2e/d50/fcd [0,4194304] 0 2024-04-25T01:35:59.265 INFO:tasks.workunit.client.1.smithi138.stdout:6/198: creat dc/d2e/d37/f42 x:0 0 0 2024-04-25T01:35:59.267 INFO:tasks.workunit.client.0.smithi033.stdout:2/780: fsync da/d5b/d72/fbe 0 2024-04-25T01:35:59.267 INFO:tasks.workunit.client.0.smithi033.stdout:7/778: write d4/d2e/d6b/fbc [1665920,90957] 0 2024-04-25T01:35:59.268 INFO:tasks.workunit.client.0.smithi033.stdout:0/651: readlink d7/d2d/d4d/d9c/l9f 0 2024-04-25T01:35:59.271 INFO:tasks.workunit.client.1.smithi138.stdout:6/199: creat dc/d10/f43 x:0 0 0 2024-04-25T01:35:59.271 INFO:tasks.workunit.client.1.smithi138.stdout:6/200: dread - dc/d10/f27 zero size 2024-04-25T01:35:59.274 INFO:tasks.workunit.client.0.smithi033.stdout:2/781: rename da/d27/d31/d40/f88 to da/d5b/d5e/d76/ffd 0 2024-04-25T01:35:59.275 INFO:tasks.workunit.client.1.smithi138.stdout:6/201: mknod dc/d10/c44 0 2024-04-25T01:35:59.276 INFO:tasks.workunit.client.1.smithi138.stdout:6/202: chown dc/d2e/f3e 0 1 2024-04-25T01:35:59.277 INFO:tasks.workunit.client.0.smithi033.stdout:7/779: link d4/d2e/d7f/d98/dca/fdd d4/d2e/d6d/dbb/ff4 0 2024-04-25T01:35:59.279 INFO:tasks.workunit.client.0.smithi033.stdout:0/652: dwrite d7/d2d/d4d/d9c/d61/fdb [0,4194304] 0 2024-04-25T01:35:59.279 INFO:tasks.workunit.client.0.smithi033.stdout:7/780: mkdir d4/d2e/d65/df5 0 2024-04-25T01:35:59.281 INFO:tasks.workunit.client.0.smithi033.stdout:0/653: fsync d7/d2d/fc1 0 2024-04-25T01:35:59.282 INFO:tasks.workunit.client.0.smithi033.stdout:0/654: dread - d7/d2d/d43/dc0/fc3 zero size 2024-04-25T01:35:59.283 INFO:tasks.workunit.client.0.smithi033.stdout:7/781: unlink d4/d10/c6a 0 2024-04-25T01:35:59.284 INFO:tasks.workunit.client.0.smithi033.stdout:7/782: chown d4/d2e/d7f/d98/dbf 7412 1 2024-04-25T01:35:59.284 INFO:tasks.workunit.client.0.smithi033.stdout:0/655: symlink d7/d2d/d4d/d9c/dcf/le5 0 2024-04-25T01:35:59.289 INFO:tasks.workunit.client.0.smithi033.stdout:0/656: creat d7/d2d/d4c/da0/da1/dd7/fe6 x:0 0 0 2024-04-25T01:35:59.292 INFO:tasks.workunit.client.0.smithi033.stdout:0/657: fsync d7/d2d/d43/d44/f63 0 2024-04-25T01:35:59.294 INFO:tasks.workunit.client.0.smithi033.stdout:0/658: mkdir d7/de7 0 2024-04-25T01:35:59.295 INFO:tasks.workunit.client.0.smithi033.stdout:6/883: dwrite d4/d1d/d3e/d4a/dcb/f102 [0,4194304] 0 2024-04-25T01:35:59.298 INFO:tasks.workunit.client.0.smithi033.stdout:6/884: mkdir d4/d1d/d3e/d41/d54/da1/d12c 0 2024-04-25T01:35:59.299 INFO:tasks.workunit.client.0.smithi033.stdout:0/659: fdatasync d7/d2d/d64/d51/f7f 0 2024-04-25T01:35:59.302 INFO:tasks.workunit.client.0.smithi033.stdout:0/660: mknod d7/de7/ce8 0 2024-04-25T01:35:59.303 INFO:tasks.workunit.client.1.smithi138.stdout:6/203: getdents dc/d10/d28 0 2024-04-25T01:35:59.304 INFO:tasks.workunit.client.1.smithi138.stdout:6/204: fsync dc/f11 0 2024-04-25T01:35:59.304 INFO:tasks.workunit.client.1.smithi138.stdout:6/205: stat c9 0 2024-04-25T01:35:59.306 INFO:tasks.workunit.client.0.smithi033.stdout:6/885: link d4/d24/f5b d4/d1d/d3e/d41/d9f/f12d 0 2024-04-25T01:35:59.307 INFO:tasks.workunit.client.1.smithi138.stdout:6/206: symlink dc/d10/l45 0 2024-04-25T01:35:59.308 INFO:tasks.workunit.client.0.smithi033.stdout:6/886: dread - d4/d1d/d3e/d41/d5c/dae/f106 zero size 2024-04-25T01:35:59.309 INFO:tasks.workunit.client.1.smithi138.stdout:6/207: rename dc/d10/d28/f2a to dc/d10/d28/f46 0 2024-04-25T01:35:59.310 INFO:tasks.workunit.client.0.smithi033.stdout:0/661: dwrite d7/d2d/d4d/d9c/dcf/fb1 [0,4194304] 0 2024-04-25T01:35:59.311 INFO:tasks.workunit.client.1.smithi138.stdout:6/208: symlink dc/d10/l47 0 2024-04-25T01:35:59.313 INFO:tasks.workunit.client.1.smithi138.stdout:6/209: chown dc/d10/l45 26966576 1 2024-04-25T01:35:59.313 INFO:tasks.workunit.client.0.smithi033.stdout:6/887: mknod d4/d1d/d3e/d41/d93/c12e 0 2024-04-25T01:35:59.313 INFO:tasks.workunit.client.0.smithi033.stdout:7/783: truncate d4/d2e/d7f/fab 192694 0 2024-04-25T01:35:59.316 INFO:tasks.workunit.client.1.smithi138.stdout:6/210: rename dc/d2e/d37/f41 to dc/d10/d28/f48 0 2024-04-25T01:35:59.320 INFO:tasks.workunit.client.1.smithi138.stdout:6/211: rename dc/d10/f27 to dc/d2e/f49 0 2024-04-25T01:35:59.321 INFO:tasks.workunit.client.1.smithi138.stdout:6/212: write dc/d2e/f3e [1021565,37659] 0 2024-04-25T01:35:59.323 INFO:tasks.workunit.client.1.smithi138.stdout:6/213: creat dc/d2e/f4a x:0 0 0 2024-04-25T01:35:59.324 INFO:tasks.workunit.client.1.smithi138.stdout:6/214: creat dc/d10/f4b x:0 0 0 2024-04-25T01:35:59.335 INFO:tasks.workunit.client.1.smithi138.stdout:6/215: rmdir dc/d2e/d37 39 2024-04-25T01:35:59.336 INFO:tasks.workunit.client.0.smithi033.stdout:7/784: stat d4/d23/d3f/d45/d73/dbe/fe0 0 2024-04-25T01:35:59.337 INFO:tasks.workunit.client.0.smithi033.stdout:7/785: dread - d4/d2e/f63 zero size 2024-04-25T01:35:59.337 INFO:tasks.workunit.client.1.smithi138.stdout:6/216: creat dc/d2e/f4c x:0 0 0 2024-04-25T01:35:59.340 INFO:tasks.workunit.client.1.smithi138.stdout:6/217: creat dc/d2e/f4d x:0 0 0 2024-04-25T01:35:59.342 INFO:tasks.workunit.client.0.smithi033.stdout:6/888: dwrite d4/d1d/d3e/d41/d5c/d7f/da4/dcc/f128 [0,4194304] 0 2024-04-25T01:35:59.343 INFO:tasks.workunit.client.0.smithi033.stdout:0/662: dread d7/d33/f3e [0,4194304] 0 2024-04-25T01:35:59.345 INFO:tasks.workunit.client.0.smithi033.stdout:7/786: write d4/d2e/d7f/d98/dca/fe7 [4721192,33255] 0 2024-04-25T01:35:59.350 INFO:tasks.workunit.client.0.smithi033.stdout:0/663: write d7/d2d/d4d/d9c/f83 [1379469,88027] 0 2024-04-25T01:35:59.350 INFO:tasks.workunit.client.0.smithi033.stdout:0/664: dread - d7/d2d/d4c/da0/da1/dd7/fe6 zero size 2024-04-25T01:35:59.352 INFO:tasks.workunit.client.0.smithi033.stdout:7/787: rename d4/d2e/d50/ce9 to d4/d23/d3f/d45/d90/ded/cf6 0 2024-04-25T01:35:59.358 INFO:tasks.workunit.client.0.smithi033.stdout:0/665: dwrite d7/d2d/d64/d51/f52 [0,4194304] 0 2024-04-25T01:35:59.363 INFO:tasks.workunit.client.0.smithi033.stdout:7/788: write d4/d2e/d65/f9b [215657,11105] 0 2024-04-25T01:35:59.369 INFO:tasks.workunit.client.0.smithi033.stdout:0/666: dwrite d7/da/df/f99 [0,4194304] 0 2024-04-25T01:35:59.374 INFO:tasks.workunit.client.0.smithi033.stdout:7/789: dwrite d4/d2e/d6d/f75 [0,4194304] 0 2024-04-25T01:35:59.378 INFO:tasks.workunit.client.0.smithi033.stdout:7/790: chown d4/d23/d3f/d45/d6c/cad 957 1 2024-04-25T01:35:59.378 INFO:tasks.workunit.client.0.smithi033.stdout:0/667: dwrite d7/d2d/d64/fad [0,4194304] 0 2024-04-25T01:35:59.381 INFO:tasks.workunit.client.0.smithi033.stdout:7/791: creat d4/d23/da7/ff7 x:0 0 0 2024-04-25T01:35:59.385 INFO:tasks.workunit.client.1.smithi138.stdout:1/187: sync 2024-04-25T01:35:59.385 INFO:tasks.workunit.client.1.smithi138.stdout:2/277: sync 2024-04-25T01:35:59.391 INFO:tasks.workunit.client.1.smithi138.stdout:1/188: rename d3/db/d2b/l39 to d3/d38/l40 0 2024-04-25T01:35:59.392 INFO:tasks.workunit.client.0.smithi033.stdout:6/889: dread d4/d1d/d3e/d70/f72 [0,4194304] 0 2024-04-25T01:35:59.393 INFO:tasks.workunit.client.1.smithi138.stdout:1/189: fdatasync d3/d27/f1e 0 2024-04-25T01:35:59.393 INFO:tasks.workunit.client.0.smithi033.stdout:6/890: fsync d4/d2a/f3f 0 2024-04-25T01:35:59.395 INFO:tasks.workunit.client.1.smithi138.stdout:2/278: dread d0/d21/d36/d15/f3b [0,4194304] 0 2024-04-25T01:35:59.395 INFO:tasks.workunit.client.1.smithi138.stdout:2/279: dread - d0/d18/f49 zero size 2024-04-25T01:35:59.398 INFO:tasks.workunit.client.0.smithi033.stdout:6/891: creat d4/d1d/d3e/d41/d5c/f12f x:0 0 0 2024-04-25T01:35:59.398 INFO:tasks.workunit.client.1.smithi138.stdout:2/280: chown d0/d21/d36/d15/d1f/c2e 1766761 1 2024-04-25T01:35:59.399 INFO:tasks.workunit.client.0.smithi033.stdout:0/668: write d7/da/f58 [1484591,78237] 0 2024-04-25T01:35:59.400 INFO:tasks.workunit.client.0.smithi033.stdout:6/892: mkdir d4/d1d/d3e/d41/d5c/d7f/da4/dcc/d130 0 2024-04-25T01:35:59.401 INFO:tasks.workunit.client.0.smithi033.stdout:0/669: fsync d7/d2d/d4d/f82 0 2024-04-25T01:35:59.403 INFO:tasks.workunit.client.1.smithi138.stdout:2/281: link d0/d21/d36/d15/c39 d0/d21/d36/d15/d51/c54 0 2024-04-25T01:35:59.403 INFO:tasks.workunit.client.1.smithi138.stdout:2/282: stat d0/d32/f43 0 2024-04-25T01:35:59.406 INFO:tasks.workunit.client.0.smithi033.stdout:0/670: getdents d7 0 2024-04-25T01:35:59.408 INFO:tasks.workunit.client.0.smithi033.stdout:0/671: creat d7/da/df/d14/d9d/fe9 x:0 0 0 2024-04-25T01:35:59.409 INFO:tasks.workunit.client.0.smithi033.stdout:0/672: readlink d7/da/df/d14/l30 0 2024-04-25T01:35:59.416 INFO:tasks.workunit.client.1.smithi138.stdout:1/190: write d3/db/fe [836873,6025] 0 2024-04-25T01:35:59.417 INFO:tasks.workunit.client.1.smithi138.stdout:1/191: read - d3/d27/f36 zero size 2024-04-25T01:35:59.418 INFO:tasks.workunit.client.1.smithi138.stdout:2/283: write d0/d21/d36/f1a [1970149,27654] 0 2024-04-25T01:35:59.419 INFO:tasks.workunit.client.1.smithi138.stdout:1/192: write d3/f10 [1059339,73075] 0 2024-04-25T01:35:59.421 INFO:tasks.workunit.client.1.smithi138.stdout:1/193: stat d3/d38/l40 0 2024-04-25T01:35:59.422 INFO:tasks.workunit.client.1.smithi138.stdout:2/284: dread d0/f10 [0,4194304] 0 2024-04-25T01:35:59.423 INFO:tasks.workunit.client.0.smithi033.stdout:6/893: dwrite d4/d1d/d3e/d41/d5c/f8b [4194304,4194304] 0 2024-04-25T01:35:59.423 INFO:tasks.workunit.client.0.smithi033.stdout:0/673: dwrite d7/da/df/d14/d45/f87 [0,4194304] 0 2024-04-25T01:35:59.424 INFO:tasks.workunit.client.0.smithi033.stdout:6/894: readlink d4/d1d/d3e/d70/l85 0 2024-04-25T01:35:59.427 INFO:tasks.workunit.client.0.smithi033.stdout:6/895: truncate d4/d1d/d3e/d41/d5c/dae/f106 353032 0 2024-04-25T01:35:59.429 INFO:tasks.workunit.client.1.smithi138.stdout:1/194: dwrite d3/d27/d16/f18 [0,4194304] 0 2024-04-25T01:35:59.430 INFO:tasks.workunit.client.0.smithi033.stdout:0/674: creat d7/d2d/d43/d92/fea x:0 0 0 2024-04-25T01:35:59.433 INFO:tasks.workunit.client.0.smithi033.stdout:0/675: rmdir d7/da/d80 39 2024-04-25T01:35:59.434 INFO:tasks.workunit.client.0.smithi033.stdout:6/896: dwrite d4/d1d/d3e/d41/d9f/fe7 [0,4194304] 0 2024-04-25T01:35:59.439 INFO:tasks.workunit.client.1.smithi138.stdout:1/195: dwrite d3/d27/d16/f18 [0,4194304] 0 2024-04-25T01:35:59.445 INFO:tasks.workunit.client.0.smithi033.stdout:6/897: dwrite d4/d1d/d3e/d41/d93/ff9 [0,4194304] 0 2024-04-25T01:35:59.445 INFO:tasks.workunit.client.0.smithi033.stdout:0/676: dwrite d7/d33/dcd/fe2 [0,4194304] 0 2024-04-25T01:35:59.446 INFO:tasks.workunit.client.1.smithi138.stdout:2/285: dread d0/f8 [0,4194304] 0 2024-04-25T01:35:59.451 INFO:tasks.workunit.client.0.smithi033.stdout:6/898: unlink d4/d1d/d3e/d41/d93/c12e 0 2024-04-25T01:35:59.451 INFO:tasks.workunit.client.1.smithi138.stdout:2/286: mknod d0/d21/d36/d15/d1f/c55 0 2024-04-25T01:35:59.453 INFO:tasks.workunit.client.1.smithi138.stdout:1/196: dread d3/db/f1d [0,4194304] 0 2024-04-25T01:35:59.457 INFO:tasks.workunit.client.1.smithi138.stdout:2/287: mkdir d0/d21/d36/d15/d44/d4c/d56 0 2024-04-25T01:35:59.459 INFO:tasks.workunit.client.1.smithi138.stdout:1/197: unlink d3/d27/f36 0 2024-04-25T01:35:59.460 INFO:tasks.workunit.client.0.smithi033.stdout:6/899: dwrite d4/d24/f118 [0,4194304] 0 2024-04-25T01:35:59.461 INFO:tasks.workunit.client.1.smithi138.stdout:2/288: symlink d0/d21/d36/d15/d44/l57 0 2024-04-25T01:35:59.465 INFO:tasks.workunit.client.1.smithi138.stdout:1/198: rename d3/d27/l20 to d3/d27/d2d/l41 0 2024-04-25T01:35:59.465 INFO:tasks.workunit.client.1.smithi138.stdout:2/289: mknod d0/d32/d35/c58 0 2024-04-25T01:35:59.465 INFO:tasks.workunit.client.0.smithi033.stdout:6/900: symlink d4/d24/d36/d42/d9d/l131 0 2024-04-25T01:35:59.466 INFO:tasks.workunit.client.1.smithi138.stdout:1/199: fdatasync d3/f10 0 2024-04-25T01:35:59.468 INFO:tasks.workunit.client.1.smithi138.stdout:2/290: write d0/d21/d36/f48 [404152,113194] 0 2024-04-25T01:35:59.468 INFO:tasks.workunit.client.0.smithi033.stdout:6/901: truncate d4/d24/d36/d42/f83 4039307 0 2024-04-25T01:35:59.469 INFO:tasks.workunit.client.0.smithi033.stdout:6/902: chown d4/d24/d36/dbd/dde/f113 204 1 2024-04-25T01:35:59.470 INFO:tasks.workunit.client.0.smithi033.stdout:6/903: fsync d4/d2a/f3f 0 2024-04-25T01:35:59.470 INFO:tasks.workunit.client.1.smithi138.stdout:1/200: write d3/f13 [291401,98349] 0 2024-04-25T01:35:59.472 INFO:tasks.workunit.client.1.smithi138.stdout:1/201: truncate d3/d27/d2d/f32 291928 0 2024-04-25T01:35:59.475 INFO:tasks.workunit.client.1.smithi138.stdout:1/202: creat d3/db/d2b/f42 x:0 0 0 2024-04-25T01:35:59.476 INFO:tasks.workunit.client.1.smithi138.stdout:1/203: rmdir d3 39 2024-04-25T01:35:59.476 INFO:tasks.workunit.client.0.smithi033.stdout:0/677: dread d7/d2d/d4d/d9c/d61/f66 [0,4194304] 0 2024-04-25T01:35:59.477 INFO:tasks.workunit.client.1.smithi138.stdout:1/204: write d3/f13 [62701,125627] 0 2024-04-25T01:35:59.480 INFO:tasks.workunit.client.1.smithi138.stdout:1/205: symlink d3/d27/d2d/l43 0 2024-04-25T01:35:59.480 INFO:tasks.workunit.client.1.smithi138.stdout:1/206: dread - d3/d4/f37 zero size 2024-04-25T01:35:59.480 INFO:tasks.workunit.client.0.smithi033.stdout:6/904: dwrite d4/d1d/d3e/d41/d5c/d7f/da4/f119 [0,4194304] 0 2024-04-25T01:35:59.484 INFO:tasks.workunit.client.1.smithi138.stdout:1/207: creat d3/f44 x:0 0 0 2024-04-25T01:35:59.484 INFO:tasks.workunit.client.0.smithi033.stdout:6/905: stat d4/d24/d36/d42/d9d/cf0 0 2024-04-25T01:35:59.485 INFO:tasks.workunit.client.1.smithi138.stdout:1/208: dread - d3/d27/d24/f3b zero size 2024-04-25T01:35:59.485 INFO:tasks.workunit.client.0.smithi033.stdout:0/678: symlink d7/d2d/d4d/d9c/d61/d70/leb 0 2024-04-25T01:35:59.486 INFO:tasks.workunit.client.1.smithi138.stdout:1/209: rmdir d3/d38 39 2024-04-25T01:35:59.487 INFO:tasks.workunit.client.0.smithi033.stdout:6/906: rename d4/d1d/d3e/d41/d54/f8d to d4/d2a/f132 0 2024-04-25T01:35:59.491 INFO:tasks.workunit.client.0.smithi033.stdout:0/679: rename d7/d2d/d4d/d9c/db2/ccb to d7/da/d80/cec 0 2024-04-25T01:35:59.493 INFO:tasks.workunit.client.0.smithi033.stdout:0/680: creat d7/da/df/d14/fed x:0 0 0 2024-04-25T01:35:59.495 INFO:tasks.workunit.client.0.smithi033.stdout:0/681: chown d7/da/df/f19 74813724 1 2024-04-25T01:35:59.498 INFO:tasks.workunit.client.0.smithi033.stdout:0/682: mkdir d7/d2d/d4d/d9c/d61/d70/dee 0 2024-04-25T01:35:59.499 INFO:tasks.workunit.client.0.smithi033.stdout:4/831: sync 2024-04-25T01:35:59.499 INFO:tasks.workunit.client.0.smithi033.stdout:3/824: sync 2024-04-25T01:35:59.500 INFO:tasks.workunit.client.0.smithi033.stdout:0/683: creat d7/d2d/d9b/fef x:0 0 0 2024-04-25T01:35:59.504 INFO:tasks.workunit.client.0.smithi033.stdout:3/825: symlink d8/dd/d11/d102/l121 0 2024-04-25T01:35:59.505 INFO:tasks.workunit.client.1.smithi138.stdout:2/291: dread d0/f9 [0,4194304] 0 2024-04-25T01:35:59.506 INFO:tasks.workunit.client.0.smithi033.stdout:4/832: readlink d6/d14/d35/d3a/d4d/d5e/d7c/d94/la3 0 2024-04-25T01:35:59.510 INFO:tasks.workunit.client.0.smithi033.stdout:0/684: dwrite d7/da/df/d14/d45/f87 [0,4194304] 0 2024-04-25T01:35:59.511 INFO:tasks.workunit.client.1.smithi138.stdout:2/292: creat d0/d21/f59 x:0 0 0 2024-04-25T01:35:59.511 INFO:tasks.workunit.client.1.smithi138.stdout:2/293: chown d0/d21/l30 357358 1 2024-04-25T01:35:59.511 INFO:tasks.workunit.client.1.smithi138.stdout:2/294: chown d0/f8 10216 1 2024-04-25T01:35:59.512 INFO:tasks.workunit.client.1.smithi138.stdout:2/295: write d0/d32/f3f [1218883,117825] 0 2024-04-25T01:35:59.512 INFO:tasks.workunit.client.1.smithi138.stdout:2/296: stat d0/c1d 0 2024-04-25T01:35:59.513 INFO:tasks.workunit.client.0.smithi033.stdout:3/826: creat d8/dd/d11/f122 x:0 0 0 2024-04-25T01:35:59.513 INFO:tasks.workunit.client.1.smithi138.stdout:8/388: sync 2024-04-25T01:35:59.515 INFO:tasks.workunit.client.0.smithi033.stdout:4/833: dwrite d6/d12/f58 [4194304,4194304] 0 2024-04-25T01:35:59.516 INFO:tasks.workunit.client.0.smithi033.stdout:3/827: truncate d8/dff/f11e 610144 0 2024-04-25T01:35:59.516 INFO:tasks.workunit.client.0.smithi033.stdout:0/685: symlink d7/da/df/lf0 0 2024-04-25T01:35:59.517 INFO:tasks.workunit.client.1.smithi138.stdout:8/389: truncate d3/d8/da/db/d15/d5d/f62 418209 0 2024-04-25T01:35:59.518 INFO:tasks.workunit.client.0.smithi033.stdout:0/686: write d7/da/f5d [1340753,56332] 0 2024-04-25T01:35:59.519 INFO:tasks.workunit.client.1.smithi138.stdout:2/297: rename d0/d21/d36 to d0/d5a 0 2024-04-25T01:35:59.522 INFO:tasks.workunit.client.0.smithi033.stdout:3/828: creat d8/dd/d40/f123 x:0 0 0 2024-04-25T01:35:59.524 INFO:tasks.workunit.client.0.smithi033.stdout:6/907: dread d4/d24/f8f [0,4194304] 0 2024-04-25T01:35:59.525 INFO:tasks.workunit.client.1.smithi138.stdout:2/298: rename d0/d18/f49 to d0/d21/f5b 0 2024-04-25T01:35:59.525 INFO:tasks.workunit.client.0.smithi033.stdout:4/834: mknod d6/d14/d1d/c10d 0 2024-04-25T01:35:59.528 INFO:tasks.workunit.client.0.smithi033.stdout:0/687: symlink d7/d8a/lf1 0 2024-04-25T01:35:59.528 INFO:tasks.workunit.client.1.smithi138.stdout:2/299: unlink d0/d32/f3f 0 2024-04-25T01:35:59.529 INFO:tasks.workunit.client.1.smithi138.stdout:1/210: write d3/db/f1d [2024831,21447] 0 2024-04-25T01:35:59.529 INFO:tasks.workunit.client.1.smithi138.stdout:2/300: rename d0/d5a to d0/d5a/d15/d44/d5c 22 2024-04-25T01:35:59.531 INFO:tasks.workunit.client.0.smithi033.stdout:6/908: creat d4/d24/d7c/f133 x:0 0 0 2024-04-25T01:35:59.532 INFO:tasks.workunit.client.0.smithi033.stdout:6/909: readlink d4/d24/d36/d42/d9d/l131 0 2024-04-25T01:35:59.534 INFO:tasks.workunit.client.0.smithi033.stdout:0/688: truncate d7/d2d/d4d/f82 363608 0 2024-04-25T01:35:59.534 INFO:tasks.workunit.client.1.smithi138.stdout:1/211: rename d3/d38 to d3/d45 0 2024-04-25T01:35:59.537 INFO:tasks.workunit.client.1.smithi138.stdout:1/212: mkdir d3/d27/d2d/d46 0 2024-04-25T01:35:59.537 INFO:tasks.workunit.client.0.smithi033.stdout:4/835: mkdir d6/d14/df6/d10e 0 2024-04-25T01:35:59.537 INFO:tasks.workunit.client.1.smithi138.stdout:1/213: read d3/f2c [3732734,39473] 0 2024-04-25T01:35:59.538 INFO:tasks.workunit.client.0.smithi033.stdout:3/829: getdents d8/dd 0 2024-04-25T01:35:59.539 INFO:tasks.workunit.client.1.smithi138.stdout:1/214: mkdir d3/d4/d47 0 2024-04-25T01:35:59.542 INFO:tasks.workunit.client.0.smithi033.stdout:4/836: chown d6/d14/f31 7958 1 2024-04-25T01:35:59.544 INFO:tasks.workunit.client.0.smithi033.stdout:6/910: dwrite d4/d1d/d3e/d41/d5c/f12f [0,4194304] 0 2024-04-25T01:35:59.544 INFO:tasks.workunit.client.1.smithi138.stdout:1/215: creat d3/d45/f48 x:0 0 0 2024-04-25T01:35:59.546 INFO:tasks.workunit.client.1.smithi138.stdout:8/390: dread d3/d8/da/db/d15/d20/f6e [0,4194304] 0 2024-04-25T01:35:59.548 INFO:tasks.workunit.client.1.smithi138.stdout:2/301: dread d0/d5a/f3c [0,4194304] 0 2024-04-25T01:35:59.549 INFO:tasks.workunit.client.0.smithi033.stdout:4/837: mknod d6/d14/df6/d10e/c10f 0 2024-04-25T01:35:59.549 INFO:tasks.workunit.client.0.smithi033.stdout:0/689: dread d7/d2d/d4d/d9c/dcf/fb1 [0,4194304] 0 2024-04-25T01:35:59.549 INFO:tasks.workunit.client.0.smithi033.stdout:6/911: write d4/d24/d36/d42/d9d/ff4 [4052322,40754] 0 2024-04-25T01:35:59.550 INFO:tasks.workunit.client.1.smithi138.stdout:8/391: mknod d3/d8/da/db/d15/d5d/c79 0 2024-04-25T01:35:59.550 INFO:tasks.workunit.client.0.smithi033.stdout:3/830: link d8/dd/fcc d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/daa/f124 0 2024-04-25T01:35:59.553 INFO:tasks.workunit.client.0.smithi033.stdout:6/912: creat d4/df1/f134 x:0 0 0 2024-04-25T01:35:59.554 INFO:tasks.workunit.client.0.smithi033.stdout:0/690: symlink d7/da/df/d14/lf2 0 2024-04-25T01:35:59.557 INFO:tasks.workunit.client.1.smithi138.stdout:2/302: dwrite d0/f8 [4194304,4194304] 0 2024-04-25T01:35:59.558 INFO:tasks.workunit.client.0.smithi033.stdout:3/831: symlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/l125 0 2024-04-25T01:35:59.558 INFO:tasks.workunit.client.0.smithi033.stdout:4/838: dwrite d6/d14/d35/d44/dfe/d103/fec [0,4194304] 0 2024-04-25T01:35:59.560 INFO:tasks.workunit.client.1.smithi138.stdout:2/303: write d0/d5a/d15/f3b [2006621,52647] 0 2024-04-25T01:35:59.562 INFO:tasks.workunit.client.0.smithi033.stdout:0/691: symlink d7/d2d/d4d/d9c/d61/d70/dee/lf3 0 2024-04-25T01:35:59.571 INFO:tasks.workunit.client.0.smithi033.stdout:6/913: truncate d4/d24/d63/f6f 1879370 0 2024-04-25T01:35:59.572 INFO:tasks.workunit.client.1.smithi138.stdout:8/392: dread d3/d8/da/de/ff [0,4194304] 0 2024-04-25T01:35:59.572 INFO:tasks.workunit.client.0.smithi033.stdout:0/692: symlink d7/d2d/d4d/d9c/lf4 0 2024-04-25T01:35:59.573 INFO:tasks.workunit.client.1.smithi138.stdout:8/393: fdatasync d3/d16/d1f/d2a/f6a 0 2024-04-25T01:35:59.576 INFO:tasks.workunit.client.0.smithi033.stdout:0/693: mkdir d7/d2d/d4d/d9c/db2/df5 0 2024-04-25T01:35:59.579 INFO:tasks.workunit.client.1.smithi138.stdout:8/394: dwrite d3/d8/da/de/f10 [4194304,4194304] 0 2024-04-25T01:35:59.589 INFO:tasks.workunit.client.1.smithi138.stdout:8/395: write d3/d8/da/db/d15/f21 [494194,103401] 0 2024-04-25T01:35:59.590 INFO:tasks.workunit.client.1.smithi138.stdout:8/396: readlink d3/d16/d1f/d2a/d4f/l5c 0 2024-04-25T01:35:59.590 INFO:tasks.workunit.client.1.smithi138.stdout:2/304: dread d0/f14 [0,4194304] 0 2024-04-25T01:35:59.592 INFO:tasks.workunit.client.0.smithi033.stdout:4/839: dwrite d6/d14/d1d/f7a [0,4194304] 0 2024-04-25T01:35:59.593 INFO:tasks.workunit.client.1.smithi138.stdout:2/305: creat d0/d5a/d15/f5d x:0 0 0 2024-04-25T01:35:59.594 INFO:tasks.workunit.client.1.smithi138.stdout:2/306: write d0/f8 [8019360,2639] 0 2024-04-25T01:35:59.598 INFO:tasks.workunit.client.0.smithi033.stdout:3/832: dwrite d8/fa [0,4194304] 0 2024-04-25T01:35:59.604 INFO:tasks.workunit.client.1.smithi138.stdout:2/307: dwrite d0/d5a/d15/d1f/f26 [4194304,4194304] 0 2024-04-25T01:35:59.605 INFO:tasks.workunit.client.1.smithi138.stdout:2/308: write d0/d5a/f4e [719955,27195] 0 2024-04-25T01:35:59.606 INFO:tasks.workunit.client.0.smithi033.stdout:0/694: dread d7/d2d/d43/dc0/f32 [0,4194304] 0 2024-04-25T01:35:59.607 INFO:tasks.workunit.client.1.smithi138.stdout:2/309: mknod d0/d32/d35/c5e 0 2024-04-25T01:35:59.609 INFO:tasks.workunit.client.0.smithi033.stdout:0/695: read d7/d2d/d64/d51/f89 [1277445,14800] 0 2024-04-25T01:35:59.610 INFO:tasks.workunit.client.1.smithi138.stdout:2/310: dread d0/d5a/d15/d1f/f37 [0,4194304] 0 2024-04-25T01:35:59.613 INFO:tasks.workunit.client.1.smithi138.stdout:2/311: write d0/f31 [3425465,11438] 0 2024-04-25T01:35:59.614 INFO:tasks.workunit.client.0.smithi033.stdout:4/840: mknod d6/d14/d35/d44/d45/c110 0 2024-04-25T01:35:59.616 INFO:tasks.workunit.client.1.smithi138.stdout:2/312: write d0/d5a/f12 [1195298,69400] 0 2024-04-25T01:35:59.616 INFO:tasks.workunit.client.0.smithi033.stdout:4/841: fdatasync d6/d14/d35/d44/d45/d46/d57/d70/d87/faf 0 2024-04-25T01:35:59.621 INFO:tasks.workunit.client.0.smithi033.stdout:0/696: mknod d7/d2d/d4d/d9c/d61/d70/dee/cf6 0 2024-04-25T01:35:59.623 INFO:tasks.workunit.client.0.smithi033.stdout:3/833: getdents d8/dd/d3d/d4f/df0 0 2024-04-25T01:35:59.623 INFO:tasks.workunit.client.0.smithi033.stdout:0/697: write d7/d2d/d43/dc0/f4b [1315598,124142] 0 2024-04-25T01:35:59.626 INFO:tasks.workunit.client.1.smithi138.stdout:8/397: dread d3/d16/f51 [0,4194304] 0 2024-04-25T01:35:59.627 INFO:tasks.workunit.client.0.smithi033.stdout:3/834: rmdir d8/dd/d3d/d63/d7c/d4d 39 2024-04-25T01:35:59.627 INFO:tasks.workunit.client.1.smithi138.stdout:8/398: chown d3/f6d 12194 1 2024-04-25T01:35:59.628 INFO:tasks.workunit.client.1.smithi138.stdout:8/399: symlink d3/d44/l7a 0 2024-04-25T01:35:59.629 INFO:tasks.workunit.client.0.smithi033.stdout:0/698: creat d7/da/df/d50/dbb/ff7 x:0 0 0 2024-04-25T01:35:59.630 INFO:tasks.workunit.client.1.smithi138.stdout:8/400: rmdir d3/d8/da/db/d15/d5d 39 2024-04-25T01:35:59.631 INFO:tasks.workunit.client.1.smithi138.stdout:8/401: stat d3/d8/da/db/d15/d6f 0 2024-04-25T01:35:59.632 INFO:tasks.workunit.client.0.smithi033.stdout:4/842: dwrite d6/d14/d35/d44/d45/d46/f9b [0,4194304] 0 2024-04-25T01:35:59.632 INFO:tasks.workunit.client.0.smithi033.stdout:0/699: rmdir d7/d2d/d4d/d9c 39 2024-04-25T01:35:59.636 INFO:tasks.workunit.client.0.smithi033.stdout:0/700: read d7/da/df/f9a [934932,111103] 0 2024-04-25T01:35:59.638 INFO:tasks.workunit.client.1.smithi138.stdout:8/402: dwrite d3/d8/da/db/d15/d20/f6e [0,4194304] 0 2024-04-25T01:35:59.641 INFO:tasks.workunit.client.1.smithi138.stdout:8/403: creat d3/d44/f7b x:0 0 0 2024-04-25T01:35:59.642 INFO:tasks.workunit.client.1.smithi138.stdout:8/404: readlink d3/d44/d48/l58 0 2024-04-25T01:35:59.644 INFO:tasks.workunit.client.1.smithi138.stdout:8/405: creat d3/d16/d1f/d50/f7c x:0 0 0 2024-04-25T01:35:59.645 INFO:tasks.workunit.client.0.smithi033.stdout:3/835: link d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/fa9 d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/f126 0 2024-04-25T01:35:59.646 INFO:tasks.workunit.client.1.smithi138.stdout:8/406: symlink d3/d8/da/d70/l7d 0 2024-04-25T01:35:59.647 INFO:tasks.workunit.client.1.smithi138.stdout:8/407: chown d3/d8/da/db/d15/f72 343655 1 2024-04-25T01:35:59.651 INFO:tasks.workunit.client.1.smithi138.stdout:8/408: rename d3/d8/da/db/d15/d20/c29 to d3/d8/da/db/d15/c7e 0 2024-04-25T01:35:59.651 INFO:tasks.workunit.client.0.smithi033.stdout:3/836: write d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/fb9 [732318,70207] 0 2024-04-25T01:35:59.652 INFO:tasks.workunit.client.1.smithi138.stdout:8/409: chown d3/l4 185139522 1 2024-04-25T01:35:59.654 INFO:tasks.workunit.client.1.smithi138.stdout:8/410: mknod d3/d8/da/db/d15/d20/c7f 0 2024-04-25T01:35:59.655 INFO:tasks.workunit.client.1.smithi138.stdout:8/411: dread - d3/d16/d1f/d50/f7c zero size 2024-04-25T01:35:59.655 INFO:tasks.workunit.client.1.smithi138.stdout:8/412: read - d3/d8/da/db/d15/f72 zero size 2024-04-25T01:35:59.658 INFO:tasks.workunit.client.1.smithi138.stdout:8/413: rename d3/l5e to d3/d16/d1f/d31/l80 0 2024-04-25T01:35:59.658 INFO:tasks.workunit.client.1.smithi138.stdout:8/414: write d3/d8/da/db/f11 [3959292,46726] 0 2024-04-25T01:35:59.660 INFO:tasks.workunit.client.0.smithi033.stdout:4/843: fsync d6/f18 0 2024-04-25T01:35:59.660 INFO:tasks.workunit.client.1.smithi138.stdout:8/415: creat d3/d16/d1f/d50/f81 x:0 0 0 2024-04-25T01:35:59.662 INFO:tasks.workunit.client.0.smithi033.stdout:3/837: dwrite d8/dd/d3d/d4f/da4/fd2 [0,4194304] 0 2024-04-25T01:35:59.663 INFO:tasks.workunit.client.0.smithi033.stdout:0/701: truncate d7/d2d/d64/d51/fc6 4340816 0 2024-04-25T01:35:59.663 INFO:tasks.workunit.client.1.smithi138.stdout:8/416: write d3/d8/da/db/d15/d5d/f6b [4725323,113936] 0 2024-04-25T01:35:59.666 INFO:tasks.workunit.client.1.smithi138.stdout:8/417: mkdir d3/d16/d1f/d31/d82 0 2024-04-25T01:35:59.666 INFO:tasks.workunit.client.0.smithi033.stdout:3/838: creat d8/dd/d3d/d63/d7c/d9d/f127 x:0 0 0 2024-04-25T01:35:59.667 INFO:tasks.workunit.client.0.smithi033.stdout:0/702: creat d7/da/df/dc7/ff8 x:0 0 0 2024-04-25T01:35:59.669 INFO:tasks.workunit.client.1.smithi138.stdout:8/418: dread d3/d8/da/de/f13 [0,4194304] 0 2024-04-25T01:35:59.672 INFO:tasks.workunit.client.0.smithi033.stdout:0/703: rename d7/d2d/d64/c73 to d7/da/df/d14/d45/da2/cf9 0 2024-04-25T01:35:59.672 INFO:tasks.workunit.client.0.smithi033.stdout:0/704: dread - d7/da/df/d50/fb3 zero size 2024-04-25T01:35:59.677 INFO:tasks.workunit.client.0.smithi033.stdout:0/705: rmdir d7/d2d/d4d/d9c/d61/d70 39 2024-04-25T01:35:59.678 INFO:tasks.workunit.client.1.smithi138.stdout:3/253: sync 2024-04-25T01:35:59.678 INFO:tasks.workunit.client.1.smithi138.stdout:7/196: sync 2024-04-25T01:35:59.679 INFO:tasks.workunit.client.1.smithi138.stdout:7/197: read - da/de/d1b/f33 zero size 2024-04-25T01:35:59.684 INFO:tasks.workunit.client.1.smithi138.stdout:7/198: rename da/de/f14 to da/de/d13/d21/f41 0 2024-04-25T01:35:59.685 INFO:tasks.workunit.client.1.smithi138.stdout:3/254: getdents db/df/d19/d1a 0 2024-04-25T01:35:59.686 INFO:tasks.workunit.client.1.smithi138.stdout:3/255: write db/fe [4734655,127794] 0 2024-04-25T01:35:59.691 INFO:tasks.workunit.client.0.smithi033.stdout:1/606: sync 2024-04-25T01:35:59.692 INFO:tasks.workunit.client.1.smithi138.stdout:7/199: getdents da 0 2024-04-25T01:35:59.692 INFO:tasks.workunit.client.0.smithi033.stdout:3/839: dread d8/dd/d11/fa7 [0,4194304] 0 2024-04-25T01:35:59.693 INFO:tasks.workunit.client.0.smithi033.stdout:0/706: write d7/da/df/dc7/ff8 [42279,130011] 0 2024-04-25T01:35:59.693 INFO:tasks.workunit.client.1.smithi138.stdout:8/419: dread d3/d8/f53 [0,4194304] 0 2024-04-25T01:35:59.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/256: dwrite db/df/d19/d1a/f47 [0,4194304] 0 2024-04-25T01:35:59.698 INFO:tasks.workunit.client.0.smithi033.stdout:1/607: creat d2/d5/d1b/d46/fbb x:0 0 0 2024-04-25T01:35:59.698 INFO:tasks.workunit.client.1.smithi138.stdout:7/200: dread f9 [0,4194304] 0 2024-04-25T01:35:59.700 INFO:tasks.workunit.client.1.smithi138.stdout:3/257: dread db/f15 [0,4194304] 0 2024-04-25T01:35:59.701 INFO:tasks.workunit.client.0.smithi033.stdout:3/840: rename d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/fa3 to d8/dd/d11/d102/f128 0 2024-04-25T01:35:59.702 INFO:tasks.workunit.client.1.smithi138.stdout:8/420: link d3/d8/da/db/d15/d20/f26 d3/d8/da/db/d15/f83 0 2024-04-25T01:35:59.703 INFO:tasks.workunit.client.1.smithi138.stdout:8/421: dread - d3/d16/d1f/d50/f81 zero size 2024-04-25T01:35:59.704 INFO:tasks.workunit.client.1.smithi138.stdout:8/422: write d3/f5 [4283065,63647] 0 2024-04-25T01:35:59.705 INFO:tasks.workunit.client.1.smithi138.stdout:3/258: rmdir db/df/d17/d20 39 2024-04-25T01:35:59.705 INFO:tasks.workunit.client.0.smithi033.stdout:1/608: creat d2/d5/d1b/d46/d5f/d63/d8f/fbc x:0 0 0 2024-04-25T01:35:59.707 INFO:tasks.workunit.client.1.smithi138.stdout:7/201: getdents da/de/d13/d21 0 2024-04-25T01:35:59.708 INFO:tasks.workunit.client.1.smithi138.stdout:7/202: chown da/l1f 36 1 2024-04-25T01:35:59.708 INFO:tasks.workunit.client.1.smithi138.stdout:3/259: mknod db/df/d17/d23/c4b 0 2024-04-25T01:35:59.709 INFO:tasks.workunit.client.0.smithi033.stdout:3/841: mknod d8/dd/d11/d102/c129 0 2024-04-25T01:35:59.709 INFO:tasks.workunit.client.1.smithi138.stdout:7/203: chown da/de/d13/d30/c35 179 1 2024-04-25T01:35:59.712 INFO:tasks.workunit.client.0.smithi033.stdout:1/609: dwrite d2/d11/d28/f7d [0,4194304] 0 2024-04-25T01:35:59.713 INFO:tasks.workunit.client.1.smithi138.stdout:8/423: link d3/d8/da/de/l49 d3/d8/da/db/d15/d56/l84 0 2024-04-25T01:35:59.714 INFO:tasks.workunit.client.0.smithi033.stdout:1/610: write d2/d5/d1b/f5a [885184,71221] 0 2024-04-25T01:35:59.715 INFO:tasks.workunit.client.0.smithi033.stdout:1/611: read - d2/d5/d75/d59/fae zero size 2024-04-25T01:35:59.717 INFO:tasks.workunit.client.0.smithi033.stdout:1/612: write d2/fb6 [587984,990] 0 2024-04-25T01:35:59.717 INFO:tasks.workunit.client.1.smithi138.stdout:7/204: rename da/de/f1c to da/de/d13/d30/f42 0 2024-04-25T01:35:59.717 INFO:tasks.workunit.client.1.smithi138.stdout:3/260: dwrite db/df/d19/f36 [0,4194304] 0 2024-04-25T01:35:59.718 INFO:tasks.workunit.client.1.smithi138.stdout:7/205: stat c2 0 2024-04-25T01:35:59.718 INFO:tasks.workunit.client.1.smithi138.stdout:8/424: symlink d3/d8/da/d70/l85 0 2024-04-25T01:35:59.721 INFO:tasks.workunit.client.0.smithi033.stdout:1/613: rename d2/d11/f1e to d2/d2e/d49/fbd 0 2024-04-25T01:35:59.721 INFO:tasks.workunit.client.1.smithi138.stdout:7/206: creat da/de/d2e/f43 x:0 0 0 2024-04-25T01:35:59.723 INFO:tasks.workunit.client.0.smithi033.stdout:0/707: dread d7/d2d/d64/d51/fc6 [0,4194304] 0 2024-04-25T01:35:59.724 INFO:tasks.workunit.client.0.smithi033.stdout:0/708: write d7/da/fb7 [553765,89318] 0 2024-04-25T01:35:59.726 INFO:tasks.workunit.client.1.smithi138.stdout:8/425: getdents d3/d8/da 0 2024-04-25T01:35:59.727 INFO:tasks.workunit.client.1.smithi138.stdout:3/261: dwrite f6 [0,4194304] 0 2024-04-25T01:35:59.728 INFO:tasks.workunit.client.1.smithi138.stdout:8/426: chown d3/d8/da/db 1828508554 1 2024-04-25T01:35:59.730 INFO:tasks.workunit.client.1.smithi138.stdout:7/207: dread da/de/d13/f24 [0,4194304] 0 2024-04-25T01:35:59.733 INFO:tasks.workunit.client.0.smithi033.stdout:0/709: creat d7/d2d/d4d/d9c/dcf/ddd/ffa x:0 0 0 2024-04-25T01:35:59.733 INFO:tasks.workunit.client.1.smithi138.stdout:8/427: unlink d3/d8/da/de/l18 0 2024-04-25T01:35:59.735 INFO:tasks.workunit.client.0.smithi033.stdout:0/710: read d7/f46 [160763,20642] 0 2024-04-25T01:35:59.735 INFO:tasks.workunit.client.1.smithi138.stdout:3/262: rename db/df/c3d to db/c4c 0 2024-04-25T01:35:59.736 INFO:tasks.workunit.client.0.smithi033.stdout:0/711: chown d7/d2d/d4d/d9c/db2/df5 0 1 2024-04-25T01:35:59.738 INFO:tasks.workunit.client.1.smithi138.stdout:8/428: chown d3/d16/d1f/d31/l80 3670 1 2024-04-25T01:35:59.738 INFO:tasks.workunit.client.0.smithi033.stdout:1/614: write d2/f32 [487776,60700] 0 2024-04-25T01:35:59.739 INFO:tasks.workunit.client.1.smithi138.stdout:7/208: dwrite da/fc [0,4194304] 0 2024-04-25T01:35:59.741 INFO:tasks.workunit.client.1.smithi138.stdout:7/209: write da/de/d13/f3c [611525,60000] 0 2024-04-25T01:35:59.742 INFO:tasks.workunit.client.1.smithi138.stdout:3/263: unlink c7 0 2024-04-25T01:35:59.743 INFO:tasks.workunit.client.0.smithi033.stdout:1/615: truncate d2/d5/d1b/d46/d5f/d63/d64/fa5 219726 0 2024-04-25T01:35:59.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:35:59 smithi033 ceph-mon[26353]: pgmap v15: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 47 MiB/s rd, 156 MiB/s wr, 350 op/s 2024-04-25T01:35:59.745 INFO:tasks.workunit.client.0.smithi033.stdout:0/712: dwrite d7/d33/fdf [0,4194304] 0 2024-04-25T01:35:59.745 INFO:tasks.workunit.client.1.smithi138.stdout:8/429: creat d3/d8/da/db/d15/d56/f86 x:0 0 0 2024-04-25T01:35:59.745 INFO:tasks.workunit.client.0.smithi033.stdout:1/616: write d2/d11/d28/f79 [8770863,80281] 0 2024-04-25T01:35:59.746 INFO:tasks.workunit.client.1.smithi138.stdout:8/430: write d3/f3a [4383507,54918] 0 2024-04-25T01:35:59.748 INFO:tasks.workunit.client.1.smithi138.stdout:7/210: creat da/de/d1b/f44 x:0 0 0 2024-04-25T01:35:59.748 INFO:tasks.workunit.client.1.smithi138.stdout:8/431: write d3/f38 [1213140,119063] 0 2024-04-25T01:35:59.748 INFO:tasks.workunit.client.1.smithi138.stdout:7/211: stat da/de/d1b/f31 0 2024-04-25T01:35:59.748 INFO:tasks.workunit.client.1.smithi138.stdout:3/264: symlink db/df/d19/l4d 0 2024-04-25T01:35:59.749 INFO:tasks.workunit.client.0.smithi033.stdout:1/617: write d2/d5/d1b/d46/d5f/d63/d8f/f96 [66514,88604] 0 2024-04-25T01:35:59.751 INFO:tasks.workunit.client.1.smithi138.stdout:3/265: stat db/df/d17/f4a 0 2024-04-25T01:35:59.751 INFO:tasks.workunit.client.0.smithi033.stdout:0/713: symlink d7/d2d/d43/dc0/lfb 0 2024-04-25T01:35:59.753 INFO:tasks.workunit.client.1.smithi138.stdout:7/212: symlink da/de/d1b/l45 0 2024-04-25T01:35:59.754 INFO:tasks.workunit.client.0.smithi033.stdout:1/618: symlink d2/d11/lbe 0 2024-04-25T01:35:59.755 INFO:tasks.workunit.client.0.smithi033.stdout:1/619: read d2/d11/f30 [4178579,115898] 0 2024-04-25T01:35:59.756 INFO:tasks.workunit.client.0.smithi033.stdout:0/714: rename d7/da/df/d14/l95 to d7/d2d/d4d/d9c/d68/dd2/lfc 0 2024-04-25T01:35:59.762 INFO:tasks.workunit.client.1.smithi138.stdout:3/266: dwrite f1 [4194304,4194304] 0 2024-04-25T01:35:59.763 INFO:tasks.workunit.client.1.smithi138.stdout:7/213: dwrite da/f18 [0,4194304] 0 2024-04-25T01:35:59.766 INFO:tasks.workunit.client.0.smithi033.stdout:0/715: dwrite d7/d33/fdf [0,4194304] 0 2024-04-25T01:35:59.766 INFO:tasks.workunit.client.1.smithi138.stdout:3/267: write f1 [5892206,88750] 0 2024-04-25T01:35:59.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:35:59 smithi138 ceph-mon[32977]: pgmap v15: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 47 MiB/s rd, 156 MiB/s wr, 350 op/s 2024-04-25T01:35:59.770 INFO:tasks.workunit.client.1.smithi138.stdout:7/214: creat da/de/d2e/f46 x:0 0 0 2024-04-25T01:35:59.772 INFO:tasks.workunit.client.1.smithi138.stdout:8/432: truncate d3/f3a 3133396 0 2024-04-25T01:35:59.776 INFO:tasks.workunit.client.0.smithi033.stdout:0/716: dwrite d7/da/f5d [0,4194304] 0 2024-04-25T01:35:59.777 INFO:tasks.workunit.client.1.smithi138.stdout:8/433: write d3/d8/da/db/d15/d5d/f62 [799557,34966] 0 2024-04-25T01:35:59.777 INFO:tasks.workunit.client.1.smithi138.stdout:8/434: readlink d3/d44/l7a 0 2024-04-25T01:35:59.778 INFO:tasks.workunit.client.1.smithi138.stdout:7/215: link da/de/d13/f23 da/de/d1b/d34/f47 0 2024-04-25T01:35:59.779 INFO:tasks.workunit.client.1.smithi138.stdout:3/268: truncate db/f12 1270376 0 2024-04-25T01:35:59.783 INFO:tasks.workunit.client.1.smithi138.stdout:8/435: rename d3/d16/d1f/d50/d67 to d3/d16/d1f/d87 0 2024-04-25T01:35:59.784 INFO:tasks.workunit.client.0.smithi033.stdout:0/717: dread d7/d2d/d4d/d9c/f84 [0,4194304] 0 2024-04-25T01:35:59.787 INFO:tasks.workunit.client.1.smithi138.stdout:3/269: rename l2 to db/df/d19/l4e 0 2024-04-25T01:35:59.788 INFO:tasks.workunit.client.0.smithi033.stdout:1/620: dread d2/d5/d1b/f54 [0,4194304] 0 2024-04-25T01:35:59.789 INFO:tasks.workunit.client.0.smithi033.stdout:0/718: getdents d7/d2d/d4d/d9c/d68/dd2 0 2024-04-25T01:35:59.789 INFO:tasks.workunit.client.1.smithi138.stdout:3/270: fsync db/df/d17/f1c 0 2024-04-25T01:35:59.790 INFO:tasks.workunit.client.0.smithi033.stdout:1/621: dread - d2/d5/d1b/d46/fbb zero size 2024-04-25T01:35:59.791 INFO:tasks.workunit.client.0.smithi033.stdout:1/622: write d2/f32 [4926417,5086] 0 2024-04-25T01:35:59.793 INFO:tasks.workunit.client.1.smithi138.stdout:8/436: rename d3/d8 to d3/d16/d88 0 2024-04-25T01:35:59.794 INFO:tasks.workunit.client.1.smithi138.stdout:8/437: write d3/d16/d88/da/f24 [2588641,26769] 0 2024-04-25T01:35:59.795 INFO:tasks.workunit.client.0.smithi033.stdout:1/623: unlink d2/d11/c57 0 2024-04-25T01:35:59.795 INFO:tasks.workunit.client.1.smithi138.stdout:3/271: creat db/df/f4f x:0 0 0 2024-04-25T01:35:59.795 INFO:tasks.workunit.client.0.smithi033.stdout:1/624: chown d2/d11/d28/l44 115906121 1 2024-04-25T01:35:59.796 INFO:tasks.workunit.client.1.smithi138.stdout:8/438: mknod d3/d16/d88/da/d70/c89 0 2024-04-25T01:35:59.797 INFO:tasks.workunit.client.1.smithi138.stdout:8/439: fsync d3/d16/d1f/d2a/f6a 0 2024-04-25T01:35:59.799 INFO:tasks.workunit.client.1.smithi138.stdout:8/440: write d3/d16/d1f/d50/f81 [865067,116797] 0 2024-04-25T01:35:59.800 INFO:tasks.workunit.client.1.smithi138.stdout:8/441: truncate d3/d16/d1f/d2a/f6a 749859 0 2024-04-25T01:35:59.801 INFO:tasks.workunit.client.0.smithi033.stdout:1/625: rename d2/d5/d1b/d46/fbb to d2/d11/d28/fbf 0 2024-04-25T01:35:59.801 INFO:tasks.workunit.client.0.smithi033.stdout:1/626: chown d2/d11/l4b 3874 1 2024-04-25T01:35:59.802 INFO:tasks.workunit.client.0.smithi033.stdout:1/627: chown d2/d5/d75/f53 24 1 2024-04-25T01:35:59.803 INFO:tasks.workunit.client.1.smithi138.stdout:8/442: write d3/d16/d88/da/db/d15/f2f [3744493,68543] 0 2024-04-25T01:35:59.805 INFO:tasks.workunit.client.1.smithi138.stdout:8/443: mkdir d3/d16/d1f/d31/d82/d8a 0 2024-04-25T01:35:59.805 INFO:tasks.workunit.client.0.smithi033.stdout:0/719: write d7/d2d/d43/dc0/fa3 [3300431,110729] 0 2024-04-25T01:35:59.807 INFO:tasks.workunit.client.0.smithi033.stdout:0/720: readlink d7/da/df/d14/l30 0 2024-04-25T01:35:59.809 INFO:tasks.workunit.client.1.smithi138.stdout:7/216: dread da/de/d1b/d34/f47 [0,4194304] 0 2024-04-25T01:35:59.809 INFO:tasks.workunit.client.1.smithi138.stdout:3/272: read db/f12 [729890,54204] 0 2024-04-25T01:35:59.809 INFO:tasks.workunit.client.1.smithi138.stdout:7/217: chown da/de/d13 100 1 2024-04-25T01:35:59.809 INFO:tasks.workunit.client.0.smithi033.stdout:1/628: dwrite d2/f81 [0,4194304] 0 2024-04-25T01:35:59.810 INFO:tasks.workunit.client.0.smithi033.stdout:0/721: write d7/d2d/d4d/d9c/dcf/fb1 [2829735,60151] 0 2024-04-25T01:35:59.810 INFO:tasks.workunit.client.0.smithi033.stdout:1/629: chown d2/f62 57816937 1 2024-04-25T01:35:59.812 INFO:tasks.workunit.client.0.smithi033.stdout:0/722: readlink d7/d2d/d43/dc0/lfb 0 2024-04-25T01:35:59.815 INFO:tasks.workunit.client.0.smithi033.stdout:0/723: chown d7/d2d/d43/d44/c4f 23810 1 2024-04-25T01:35:59.815 INFO:tasks.workunit.client.1.smithi138.stdout:4/149: sync 2024-04-25T01:35:59.816 INFO:tasks.workunit.client.1.smithi138.stdout:0/215: sync 2024-04-25T01:35:59.816 INFO:tasks.workunit.client.1.smithi138.stdout:9/220: sync 2024-04-25T01:35:59.816 INFO:tasks.workunit.client.1.smithi138.stdout:1/216: sync 2024-04-25T01:35:59.816 INFO:tasks.workunit.client.1.smithi138.stdout:5/190: sync 2024-04-25T01:35:59.816 INFO:tasks.workunit.client.1.smithi138.stdout:6/218: sync 2024-04-25T01:35:59.817 INFO:tasks.workunit.client.1.smithi138.stdout:3/273: dread db/df/d17/d23/f30 [0,4194304] 0 2024-04-25T01:35:59.817 INFO:tasks.workunit.client.1.smithi138.stdout:6/219: fdatasync dc/f1d 0 2024-04-25T01:35:59.818 INFO:tasks.workunit.client.1.smithi138.stdout:9/221: readlink d4/d8/l10 0 2024-04-25T01:35:59.820 INFO:tasks.workunit.client.0.smithi033.stdout:1/630: dwrite d2/d5/d6c/f87 [0,4194304] 0 2024-04-25T01:35:59.828 INFO:tasks.workunit.client.1.smithi138.stdout:0/216: mknod d6/d39/d43/c4c 0 2024-04-25T01:35:59.828 INFO:tasks.workunit.client.0.smithi033.stdout:1/631: dwrite d2/d5/d1b/d46/d5f/d63/d64/fa5 [0,4194304] 0 2024-04-25T01:35:59.828 INFO:tasks.workunit.client.1.smithi138.stdout:6/220: chown dc/f35 1 1 2024-04-25T01:35:59.829 INFO:tasks.workunit.client.1.smithi138.stdout:0/217: chown d6/f3b 28872168 1 2024-04-25T01:35:59.830 INFO:tasks.workunit.client.0.smithi033.stdout:0/724: write d7/d2d/f3d [1615544,92913] 0 2024-04-25T01:35:59.830 INFO:tasks.workunit.client.1.smithi138.stdout:1/217: unlink d3/d27/d2d/f2f 0 2024-04-25T01:35:59.831 INFO:tasks.workunit.client.1.smithi138.stdout:0/218: readlink d6/l2e 0 2024-04-25T01:35:59.831 INFO:tasks.workunit.client.0.smithi033.stdout:0/725: readlink d7/d2d/d9b/l5b 0 2024-04-25T01:35:59.832 INFO:tasks.workunit.client.0.smithi033.stdout:1/632: mknod d2/d11/cc0 0 2024-04-25T01:35:59.834 INFO:tasks.workunit.client.1.smithi138.stdout:5/191: mknod d4/da/d27/d1c/d22/d35/c3a 0 2024-04-25T01:35:59.835 INFO:tasks.workunit.client.0.smithi033.stdout:1/633: fdatasync d2/d5/d1b/d46/d5f/d63/d64/f6a 0 2024-04-25T01:35:59.837 INFO:tasks.workunit.client.0.smithi033.stdout:0/726: getdents d7/d2d/d4d 0 2024-04-25T01:35:59.837 INFO:tasks.workunit.client.0.smithi033.stdout:1/634: creat d2/d5/d75/d59/d83/fc1 x:0 0 0 2024-04-25T01:35:59.838 INFO:tasks.workunit.client.1.smithi138.stdout:7/218: dwrite da/de/d13/f2d [0,4194304] 0 2024-04-25T01:35:59.839 INFO:tasks.workunit.client.1.smithi138.stdout:9/222: mkdir d4/d49 0 2024-04-25T01:35:59.839 INFO:tasks.workunit.client.1.smithi138.stdout:3/274: write db/f15 [5204727,119247] 0 2024-04-25T01:35:59.840 INFO:tasks.workunit.client.0.smithi033.stdout:0/727: mknod d7/da/df/dc7/cfd 0 2024-04-25T01:35:59.840 INFO:tasks.workunit.client.1.smithi138.stdout:1/218: read d3/f2c [3518574,62968] 0 2024-04-25T01:35:59.841 INFO:tasks.workunit.client.1.smithi138.stdout:0/219: creat d6/df/f4d x:0 0 0 2024-04-25T01:35:59.842 INFO:tasks.workunit.client.1.smithi138.stdout:9/223: write d4/f3e [837477,40302] 0 2024-04-25T01:35:59.842 INFO:tasks.workunit.client.1.smithi138.stdout:3/275: stat db/df/l18 0 2024-04-25T01:35:59.843 INFO:tasks.workunit.client.0.smithi033.stdout:1/635: write d2/d5/d75/f8d [646267,99187] 0 2024-04-25T01:35:59.843 INFO:tasks.workunit.client.0.smithi033.stdout:0/728: mkdir d7/da/df/d50/dfe 0 2024-04-25T01:35:59.843 INFO:tasks.workunit.client.1.smithi138.stdout:0/220: chown d6/f3b 21588 1 2024-04-25T01:35:59.844 INFO:tasks.workunit.client.1.smithi138.stdout:5/192: write d4/da/d27/d1c/f26 [2303465,11367] 0 2024-04-25T01:35:59.844 INFO:tasks.workunit.client.1.smithi138.stdout:0/221: fdatasync d6/f3b 0 2024-04-25T01:35:59.847 INFO:tasks.workunit.client.0.smithi033.stdout:5/796: sync 2024-04-25T01:35:59.848 INFO:tasks.workunit.client.1.smithi138.stdout:1/219: write d3/d4/f37 [73594,96652] 0 2024-04-25T01:35:59.849 INFO:tasks.workunit.client.1.smithi138.stdout:4/150: getdents d0 0 2024-04-25T01:35:59.850 INFO:tasks.workunit.client.0.smithi033.stdout:0/729: link d7/d2d/d64/d51/l97 d7/da/df/d50/dbb/lff 0 2024-04-25T01:35:59.851 INFO:tasks.workunit.client.0.smithi033.stdout:5/797: write d1c/d2a/d50/f90 [156383,8001] 0 2024-04-25T01:35:59.852 INFO:tasks.workunit.client.1.smithi138.stdout:3/276: unlink db/df/l18 0 2024-04-25T01:35:59.853 INFO:tasks.workunit.client.1.smithi138.stdout:1/220: dread - d3/d4/f26 zero size 2024-04-25T01:35:59.853 INFO:tasks.workunit.client.1.smithi138.stdout:5/193: read d4/da/d27/d1c/f26 [3541183,128122] 0 2024-04-25T01:35:59.853 INFO:tasks.workunit.client.0.smithi033.stdout:5/798: dread - d1c/d2a/d50/d52/da6/d3e/d70/dae/f103 zero size 2024-04-25T01:35:59.854 INFO:tasks.workunit.client.1.smithi138.stdout:0/222: write d6/d19/f4b [262332,42683] 0 2024-04-25T01:35:59.855 INFO:tasks.workunit.client.0.smithi033.stdout:5/799: readlink d1c/d2a/d50/d52/da6/d3e/l97 0 2024-04-25T01:35:59.856 INFO:tasks.workunit.client.1.smithi138.stdout:0/223: chown d6/d2f 6268 1 2024-04-25T01:35:59.856 INFO:tasks.workunit.client.1.smithi138.stdout:6/221: dwrite f6 [0,4194304] 0 2024-04-25T01:35:59.856 INFO:tasks.workunit.client.0.smithi033.stdout:0/730: link d7/da/fb7 d7/d33/f100 0 2024-04-25T01:35:59.858 INFO:tasks.workunit.client.1.smithi138.stdout:3/277: truncate db/f3f 449582 0 2024-04-25T01:35:59.858 INFO:tasks.workunit.client.0.smithi033.stdout:0/731: read d7/d2d/d4d/f82 [135894,18352] 0 2024-04-25T01:35:59.859 INFO:tasks.workunit.client.1.smithi138.stdout:1/221: mknod d3/d45/c49 0 2024-04-25T01:35:59.860 INFO:tasks.workunit.client.1.smithi138.stdout:6/222: write f5 [37188,47694] 0 2024-04-25T01:35:59.860 INFO:tasks.workunit.client.0.smithi033.stdout:5/800: rmdir d1c/d2a/d50/d5b 39 2024-04-25T01:35:59.861 INFO:tasks.workunit.client.1.smithi138.stdout:9/224: creat d4/f4a x:0 0 0 2024-04-25T01:35:59.862 INFO:tasks.workunit.client.1.smithi138.stdout:0/224: dread - d6/df/f14 zero size 2024-04-25T01:35:59.862 INFO:tasks.workunit.client.1.smithi138.stdout:7/219: dwrite da/de/f1a [0,4194304] 0 2024-04-25T01:35:59.864 INFO:tasks.workunit.client.0.smithi033.stdout:5/801: chown d1c/d2a/d3d/cc1 142572702 1 2024-04-25T01:35:59.867 INFO:tasks.workunit.client.1.smithi138.stdout:5/194: getdents d4/da/d27 0 2024-04-25T01:35:59.867 INFO:tasks.workunit.client.0.smithi033.stdout:0/732: getdents d7/d2d 0 2024-04-25T01:35:59.869 INFO:tasks.workunit.client.1.smithi138.stdout:9/225: symlink d4/l4b 0 2024-04-25T01:35:59.870 INFO:tasks.workunit.client.1.smithi138.stdout:5/195: read d4/f34 [1615263,69175] 0 2024-04-25T01:35:59.871 INFO:tasks.workunit.client.0.smithi033.stdout:5/802: unlink d1c/d2a/d50/d52/da6/d3e/d70/dae/cca 0 2024-04-25T01:35:59.872 INFO:tasks.workunit.client.1.smithi138.stdout:6/223: readlink dc/d10/l47 0 2024-04-25T01:35:59.874 INFO:tasks.workunit.client.1.smithi138.stdout:1/222: rename d3/d27/d2d/d46 to d3/db/d4a 0 2024-04-25T01:35:59.874 INFO:tasks.workunit.client.0.smithi033.stdout:0/733: symlink d7/d2d/d4c/da0/da1/dd7/l101 0 2024-04-25T01:35:59.874 INFO:tasks.workunit.client.1.smithi138.stdout:9/226: chown d4/f3e 0 1 2024-04-25T01:35:59.876 INFO:tasks.workunit.client.0.smithi033.stdout:5/803: read d1c/d2a/d50/d52/da6/d3e/d70/fb9 [404052,43152] 0 2024-04-25T01:35:59.877 INFO:tasks.workunit.client.0.smithi033.stdout:1/636: dread d2/f62 [0,4194304] 0 2024-04-25T01:35:59.877 INFO:tasks.workunit.client.0.smithi033.stdout:0/734: chown d7/da/df/d14/d45/da2/la5 298883 1 2024-04-25T01:35:59.878 INFO:tasks.workunit.client.1.smithi138.stdout:7/220: dread da/de/d13/d21/f41 [0,4194304] 0 2024-04-25T01:35:59.878 INFO:tasks.workunit.client.1.smithi138.stdout:1/223: write d3/d4/f37 [165817,87149] 0 2024-04-25T01:35:59.879 INFO:tasks.workunit.client.1.smithi138.stdout:6/224: unlink dc/f11 0 2024-04-25T01:35:59.880 INFO:tasks.workunit.client.1.smithi138.stdout:3/278: dwrite f1 [0,4194304] 0 2024-04-25T01:35:59.881 INFO:tasks.workunit.client.1.smithi138.stdout:1/224: dread - d3/d4/f26 zero size 2024-04-25T01:35:59.882 INFO:tasks.workunit.client.0.smithi033.stdout:5/804: unlink d1c/d2a/d50/d52/da6/f2c 0 2024-04-25T01:35:59.883 INFO:tasks.workunit.client.1.smithi138.stdout:0/225: dwrite d6/df/d3a/f3d [0,4194304] 0 2024-04-25T01:35:59.884 INFO:tasks.workunit.client.0.smithi033.stdout:5/805: rmdir d1c/d2a/d50/d5b 39 2024-04-25T01:35:59.885 INFO:tasks.workunit.client.1.smithi138.stdout:9/227: symlink d4/d8/d2c/l4c 0 2024-04-25T01:35:59.885 INFO:tasks.workunit.client.1.smithi138.stdout:6/225: truncate dc/f3d 591142 0 2024-04-25T01:35:59.887 INFO:tasks.workunit.client.1.smithi138.stdout:0/226: read d6/d19/f4b [279560,2926] 0 2024-04-25T01:35:59.887 INFO:tasks.workunit.client.1.smithi138.stdout:3/279: rmdir db/d14 39 2024-04-25T01:35:59.890 INFO:tasks.workunit.client.1.smithi138.stdout:5/196: rename d4/f34 to d4/f3b 0 2024-04-25T01:35:59.891 INFO:tasks.workunit.client.1.smithi138.stdout:1/225: symlink d3/d27/d16/l4b 0 2024-04-25T01:35:59.893 INFO:tasks.workunit.client.1.smithi138.stdout:7/221: dwrite da/f18 [0,4194304] 0 2024-04-25T01:35:59.895 INFO:tasks.workunit.client.0.smithi033.stdout:5/806: dwrite d1c/d2a/d50/d52/fb3 [0,4194304] 0 2024-04-25T01:35:59.898 INFO:tasks.workunit.client.0.smithi033.stdout:5/807: chown d1c/d2a/d50/d52/da6/d3e/d70/d7a/dab/fc4 105 1 2024-04-25T01:35:59.899 INFO:tasks.workunit.client.0.smithi033.stdout:5/808: readlink d1c/d2a/d3d/d59/l74 0 2024-04-25T01:35:59.899 INFO:tasks.workunit.client.1.smithi138.stdout:0/227: mknod d6/df/c4e 0 2024-04-25T01:35:59.900 INFO:tasks.workunit.client.0.smithi033.stdout:5/809: stat d1c/d2a/d50/f91 0 2024-04-25T01:35:59.904 INFO:tasks.workunit.client.0.smithi033.stdout:5/810: symlink d1c/d2a/d50/d52/da6/d3e/d70/l106 0 2024-04-25T01:35:59.905 INFO:tasks.workunit.client.1.smithi138.stdout:4/151: dread d0/f1a [0,4194304] 0 2024-04-25T01:35:59.907 INFO:tasks.workunit.client.1.smithi138.stdout:9/228: dwrite d4/d1a/f2d [0,4194304] 0 2024-04-25T01:35:59.907 INFO:tasks.workunit.client.1.smithi138.stdout:4/152: chown d0 735458876 1 2024-04-25T01:35:59.907 INFO:tasks.workunit.client.0.smithi033.stdout:5/811: symlink d1c/d2a/d50/d52/l107 0 2024-04-25T01:35:59.908 INFO:tasks.workunit.client.0.smithi033.stdout:5/812: chown d1c/l25 1211 1 2024-04-25T01:35:59.908 INFO:tasks.workunit.client.0.smithi033.stdout:5/813: chown f14 1659116 1 2024-04-25T01:35:59.909 INFO:tasks.workunit.client.1.smithi138.stdout:9/229: dread - d4/d8/f23 zero size 2024-04-25T01:35:59.910 INFO:tasks.workunit.client.0.smithi033.stdout:5/814: chown d1c/d2a/d3d/cc1 2 1 2024-04-25T01:35:59.911 INFO:tasks.workunit.client.1.smithi138.stdout:9/230: truncate d4/f4a 665960 0 2024-04-25T01:35:59.911 INFO:tasks.workunit.client.1.smithi138.stdout:6/226: creat dc/f4e x:0 0 0 2024-04-25T01:35:59.911 INFO:tasks.workunit.client.1.smithi138.stdout:1/226: mknod d3/d27/c4c 0 2024-04-25T01:35:59.912 INFO:tasks.workunit.client.1.smithi138.stdout:4/153: mknod d0/c25 0 2024-04-25T01:35:59.913 INFO:tasks.workunit.client.0.smithi033.stdout:5/815: write d1c/d2a/d50/d52/fb3 [251153,26487] 0 2024-04-25T01:35:59.913 INFO:tasks.workunit.client.1.smithi138.stdout:6/227: fsync dc/d10/d28/f48 0 2024-04-25T01:35:59.914 INFO:tasks.workunit.client.0.smithi033.stdout:1/637: dread d2/f26 [8388608,4194304] 0 2024-04-25T01:35:59.915 INFO:tasks.workunit.client.1.smithi138.stdout:6/228: write dc/d10/d28/f3c [488195,68521] 0 2024-04-25T01:35:59.916 INFO:tasks.workunit.client.1.smithi138.stdout:9/231: creat d4/d1a/d33/f4d x:0 0 0 2024-04-25T01:35:59.917 INFO:tasks.workunit.client.0.smithi033.stdout:1/638: symlink d2/d7f/lc2 0 2024-04-25T01:35:59.918 INFO:tasks.workunit.client.1.smithi138.stdout:3/280: getdents db 0 2024-04-25T01:35:59.919 INFO:tasks.workunit.client.0.smithi033.stdout:5/816: read d1c/d2a/d50/d52/da6/f6d [37751,30049] 0 2024-04-25T01:35:59.920 INFO:tasks.workunit.client.0.smithi033.stdout:1/639: read d2/d2e/f67 [1963525,74573] 0 2024-04-25T01:35:59.923 INFO:tasks.workunit.client.0.smithi033.stdout:1/640: write d2/d5/d1b/d46/d5f/d63/d8f/f9e [973446,33535] 0 2024-04-25T01:35:59.923 INFO:tasks.workunit.client.1.smithi138.stdout:3/281: truncate db/df/f4f 561098 0 2024-04-25T01:35:59.924 INFO:tasks.workunit.client.0.smithi033.stdout:5/817: dread d1c/d2a/d3d/f44 [0,4194304] 0 2024-04-25T01:35:59.925 INFO:tasks.workunit.client.1.smithi138.stdout:3/282: stat db/df/d17/f1c 0 2024-04-25T01:35:59.925 INFO:tasks.workunit.client.1.smithi138.stdout:6/229: dwrite dc/f19 [0,4194304] 0 2024-04-25T01:35:59.927 INFO:tasks.workunit.client.0.smithi033.stdout:1/641: mknod d2/d5/d75/d59/d83/cc3 0 2024-04-25T01:35:59.928 INFO:tasks.workunit.client.1.smithi138.stdout:9/232: link d4/d8/d2c/l4c d4/d8/d2c/l4e 0 2024-04-25T01:35:59.929 INFO:tasks.workunit.client.1.smithi138.stdout:3/283: read db/df/d19/d1a/f49 [1113914,36997] 0 2024-04-25T01:35:59.930 INFO:tasks.workunit.client.1.smithi138.stdout:3/284: chown la 17457747 1 2024-04-25T01:35:59.930 INFO:tasks.workunit.client.1.smithi138.stdout:9/233: fsync d4/d1a/f3b 0 2024-04-25T01:35:59.931 INFO:tasks.workunit.client.1.smithi138.stdout:6/230: mknod dc/d10/d28/c4f 0 2024-04-25T01:35:59.931 INFO:tasks.workunit.client.1.smithi138.stdout:5/197: dread d4/da/d27/f2d [0,4194304] 0 2024-04-25T01:35:59.932 INFO:tasks.workunit.client.0.smithi033.stdout:1/642: symlink d2/d5/d1b/d46/lc4 0 2024-04-25T01:35:59.932 INFO:tasks.workunit.client.0.smithi033.stdout:5/818: link d1c/ced d1c/d2a/d50/d52/da6/d3e/d70/dae/dfb/c108 0 2024-04-25T01:35:59.934 INFO:tasks.workunit.client.1.smithi138.stdout:0/228: dread d6/fc [0,4194304] 0 2024-04-25T01:35:59.937 INFO:tasks.workunit.client.1.smithi138.stdout:3/285: unlink db/f3c 0 2024-04-25T01:35:59.937 INFO:tasks.workunit.client.1.smithi138.stdout:1/227: write d3/db/f14 [1388926,83199] 0 2024-04-25T01:35:59.938 INFO:tasks.workunit.client.0.smithi033.stdout:5/819: fsync d1c/d2a/d50/d5b/fd7 0 2024-04-25T01:35:59.939 INFO:tasks.workunit.client.0.smithi033.stdout:1/643: mknod d2/d5/d1b/d46/db1/cc5 0 2024-04-25T01:35:59.939 INFO:tasks.workunit.client.0.smithi033.stdout:5/820: fsync d1c/d2a/d50/d52/da6/f63 0 2024-04-25T01:35:59.941 INFO:tasks.workunit.client.0.smithi033.stdout:1/644: mkdir d2/d5/d6c/dc6 0 2024-04-25T01:35:59.942 INFO:tasks.workunit.client.0.smithi033.stdout:1/645: chown d2/d5/d6c/c92 830586 1 2024-04-25T01:35:59.943 INFO:tasks.workunit.client.1.smithi138.stdout:6/231: mknod dc/d2e/d37/c50 0 2024-04-25T01:35:59.944 INFO:tasks.workunit.client.0.smithi033.stdout:5/821: truncate d1c/d2a/d50/f91 694048 0 2024-04-25T01:35:59.944 INFO:tasks.workunit.client.1.smithi138.stdout:9/234: symlink d4/d1a/d33/d1d/d3f/l4f 0 2024-04-25T01:35:59.945 INFO:tasks.workunit.client.1.smithi138.stdout:4/154: dread d0/f1a [0,4194304] 0 2024-04-25T01:35:59.945 INFO:tasks.workunit.client.1.smithi138.stdout:5/198: write d4/f23 [27510,60168] 0 2024-04-25T01:35:59.946 INFO:tasks.workunit.client.0.smithi033.stdout:5/822: chown d1c/d2a/d50/d52/fd9 2073590 1 2024-04-25T01:35:59.947 INFO:tasks.workunit.client.0.smithi033.stdout:5/823: fdatasync f1b 0 2024-04-25T01:35:59.948 INFO:tasks.workunit.client.1.smithi138.stdout:6/232: creat dc/d10/f51 x:0 0 0 2024-04-25T01:35:59.951 INFO:tasks.workunit.client.1.smithi138.stdout:5/199: chown d4/l11 20987550 1 2024-04-25T01:35:59.956 INFO:tasks.workunit.client.1.smithi138.stdout:9/235: dwrite d4/d1a/f44 [0,4194304] 0 2024-04-25T01:35:59.957 INFO:tasks.workunit.client.1.smithi138.stdout:3/286: truncate f6 1353919 0 2024-04-25T01:35:59.957 INFO:tasks.workunit.client.1.smithi138.stdout:5/200: chown d4/da/d27/d1c/d22/d35/l38 13036056 1 2024-04-25T01:35:59.960 INFO:tasks.workunit.client.1.smithi138.stdout:0/229: dread d6/f26 [0,4194304] 0 2024-04-25T01:35:59.961 INFO:tasks.workunit.client.0.smithi033.stdout:5/824: truncate d1c/d2a/d50/f7d 116999 0 2024-04-25T01:35:59.961 INFO:tasks.workunit.client.1.smithi138.stdout:4/155: dwrite d0/f1a [4194304,4194304] 0 2024-04-25T01:35:59.963 INFO:tasks.workunit.client.1.smithi138.stdout:4/156: read d0/f1a [2267424,102692] 0 2024-04-25T01:35:59.964 INFO:tasks.workunit.client.1.smithi138.stdout:6/233: dread dc/d10/f26 [0,4194304] 0 2024-04-25T01:35:59.965 INFO:tasks.workunit.client.1.smithi138.stdout:5/201: unlink d4/c16 0 2024-04-25T01:35:59.968 INFO:tasks.workunit.client.1.smithi138.stdout:9/236: rename d4/d1a/d43 to d4/d1a/d33/d1d/d50 0 2024-04-25T01:35:59.973 INFO:tasks.workunit.client.1.smithi138.stdout:6/234: creat dc/d2e/d37/f52 x:0 0 0 2024-04-25T01:35:59.974 INFO:tasks.workunit.client.1.smithi138.stdout:6/235: write dc/d10/f51 [22447,97723] 0 2024-04-25T01:35:59.974 INFO:tasks.workunit.client.1.smithi138.stdout:6/236: dread - dc/d2e/f4a zero size 2024-04-25T01:35:59.976 INFO:tasks.workunit.client.1.smithi138.stdout:3/287: link db/df/d17/f1c db/df/d17/d23/d34/f50 0 2024-04-25T01:35:59.976 INFO:tasks.workunit.client.0.smithi033.stdout:5/825: dread d1c/d2a/d3d/d59/f84 [0,4194304] 0 2024-04-25T01:35:59.976 INFO:tasks.workunit.client.1.smithi138.stdout:6/237: chown dc/d10/d28/c4f 233 1 2024-04-25T01:35:59.978 INFO:tasks.workunit.client.1.smithi138.stdout:4/157: symlink d0/d1c/d23/l26 0 2024-04-25T01:35:59.978 INFO:tasks.workunit.client.0.smithi033.stdout:5/826: read d1c/f1e [1641374,68247] 0 2024-04-25T01:35:59.980 INFO:tasks.workunit.client.0.smithi033.stdout:5/827: dread d1c/d53/df6/ffa [0,4194304] 0 2024-04-25T01:35:59.981 INFO:tasks.workunit.client.1.smithi138.stdout:6/238: mkdir dc/d2e/d53 0 2024-04-25T01:35:59.983 INFO:tasks.workunit.client.0.smithi033.stdout:5/828: creat d1c/d2a/d50/d52/da6/d4a/f109 x:0 0 0 2024-04-25T01:35:59.990 INFO:tasks.workunit.client.1.smithi138.stdout:6/239: dwrite dc/d10/f43 [0,4194304] 0 2024-04-25T01:35:59.992 INFO:tasks.workunit.client.1.smithi138.stdout:6/240: write f5 [2948927,37700] 0 2024-04-25T01:35:59.994 INFO:tasks.workunit.client.0.smithi033.stdout:5/829: truncate d1c/d2a/d50/d52/fb3 848068 0 2024-04-25T01:35:59.996 INFO:tasks.workunit.client.1.smithi138.stdout:5/202: write d4/da/d27/d1c/f26 [2237060,38561] 0 2024-04-25T01:35:59.997 INFO:tasks.workunit.client.1.smithi138.stdout:9/237: truncate d4/d1a/d33/f16 89377 0 2024-04-25T01:35:59.999 INFO:tasks.workunit.client.1.smithi138.stdout:3/288: write db/f12 [427838,53016] 0 2024-04-25T01:35:59.999 INFO:tasks.workunit.client.1.smithi138.stdout:0/230: dread d6/df/d3a/f4a [0,4194304] 0 2024-04-25T01:36:00.002 INFO:tasks.workunit.client.1.smithi138.stdout:6/241: dwrite dc/d10/d28/f40 [0,4194304] 0 2024-04-25T01:36:00.003 INFO:tasks.workunit.client.1.smithi138.stdout:4/158: write d0/f13 [1966655,77555] 0 2024-04-25T01:36:00.004 INFO:tasks.workunit.client.1.smithi138.stdout:3/289: dread db/df/d17/d23/f30 [0,4194304] 0 2024-04-25T01:36:00.005 INFO:tasks.workunit.client.1.smithi138.stdout:0/231: symlink d6/d2f/d31/l4f 0 2024-04-25T01:36:00.013 INFO:tasks.workunit.client.0.smithi033.stdout:5/830: dread f17 [0,4194304] 0 2024-04-25T01:36:00.014 INFO:tasks.workunit.client.0.smithi033.stdout:5/831: write d1c/f32 [783054,113024] 0 2024-04-25T01:36:00.015 INFO:tasks.workunit.client.1.smithi138.stdout:6/242: mkdir dc/d54 0 2024-04-25T01:36:00.017 INFO:tasks.workunit.client.0.smithi033.stdout:5/832: creat d1c/d2a/d50/d52/da6/ddc/f10a x:0 0 0 2024-04-25T01:36:00.018 INFO:tasks.workunit.client.1.smithi138.stdout:3/290: symlink db/d14/l51 0 2024-04-25T01:36:00.021 INFO:tasks.workunit.client.0.smithi033.stdout:5/833: unlink d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/ce9 0 2024-04-25T01:36:00.022 INFO:tasks.workunit.client.0.smithi033.stdout:5/834: chown d1c/d2a/d50/d52/da6/d3e/d70/d7a/fe5 263832 1 2024-04-25T01:36:00.022 INFO:tasks.workunit.client.1.smithi138.stdout:0/232: dwrite d6/f26 [4194304,4194304] 0 2024-04-25T01:36:00.024 INFO:tasks.workunit.client.1.smithi138.stdout:5/203: mkdir d4/da/d27/d1c/d3c 0 2024-04-25T01:36:00.024 INFO:tasks.workunit.client.0.smithi033.stdout:5/835: mkdir d1c/d2a/d50/d5b/d86/d10b 0 2024-04-25T01:36:00.028 INFO:tasks.workunit.client.1.smithi138.stdout:0/233: symlink d6/df/d3a/l50 0 2024-04-25T01:36:00.030 INFO:tasks.workunit.client.1.smithi138.stdout:6/243: truncate dc/d2e/f3e 771538 0 2024-04-25T01:36:00.033 INFO:tasks.workunit.client.1.smithi138.stdout:3/291: creat db/df/d17/d23/d34/f52 x:0 0 0 2024-04-25T01:36:00.033 INFO:tasks.workunit.client.1.smithi138.stdout:0/234: write d6/d1d/f3c [786450,119261] 0 2024-04-25T01:36:00.035 INFO:tasks.workunit.client.1.smithi138.stdout:0/235: write d6/df/f4d [824023,3126] 0 2024-04-25T01:36:00.036 INFO:tasks.workunit.client.1.smithi138.stdout:3/292: mkdir db/df/d17/d23/d53 0 2024-04-25T01:36:00.038 INFO:tasks.workunit.client.1.smithi138.stdout:6/244: dread dc/d10/d28/f40 [0,4194304] 0 2024-04-25T01:36:00.040 INFO:tasks.workunit.client.1.smithi138.stdout:6/245: write dc/f18 [620758,25214] 0 2024-04-25T01:36:00.042 INFO:tasks.workunit.client.1.smithi138.stdout:5/204: dwrite d4/f18 [0,4194304] 0 2024-04-25T01:36:00.047 INFO:tasks.workunit.client.1.smithi138.stdout:6/246: mkdir dc/d2e/d55 0 2024-04-25T01:36:00.049 INFO:tasks.workunit.client.1.smithi138.stdout:6/247: write dc/d2e/f4c [60247,65865] 0 2024-04-25T01:36:00.050 INFO:tasks.workunit.client.1.smithi138.stdout:6/248: chown dc/d2e/d37/c3f 234635 1 2024-04-25T01:36:00.054 INFO:tasks.workunit.client.1.smithi138.stdout:5/205: write d4/da/d27/d1c/d22/f24 [5561437,20790] 0 2024-04-25T01:36:00.054 INFO:tasks.workunit.client.1.smithi138.stdout:6/249: chown dc/d2e/f2f 223204 1 2024-04-25T01:36:00.054 INFO:tasks.workunit.client.1.smithi138.stdout:5/206: stat d4/da/d27/d1c/d22/c2f 0 2024-04-25T01:36:00.056 INFO:tasks.workunit.client.1.smithi138.stdout:3/293: write db/df/f25 [572124,1163] 0 2024-04-25T01:36:00.057 INFO:tasks.workunit.client.1.smithi138.stdout:6/250: read fb [842179,75172] 0 2024-04-25T01:36:00.058 INFO:tasks.workunit.client.1.smithi138.stdout:5/207: creat d4/da/d27/d1c/f3d x:0 0 0 2024-04-25T01:36:00.058 INFO:tasks.workunit.client.1.smithi138.stdout:3/294: dread db/df/d19/d1a/f49 [0,4194304] 0 2024-04-25T01:36:00.059 INFO:tasks.workunit.client.1.smithi138.stdout:6/251: rmdir dc/d10 39 2024-04-25T01:36:00.062 INFO:tasks.workunit.client.1.smithi138.stdout:0/236: dread d6/d19/f33 [0,4194304] 0 2024-04-25T01:36:00.062 INFO:tasks.workunit.client.1.smithi138.stdout:3/295: write db/fe [3466832,34993] 0 2024-04-25T01:36:00.084 INFO:tasks.workunit.client.1.smithi138.stdout:2/313: sync 2024-04-25T01:36:00.088 INFO:tasks.workunit.client.1.smithi138.stdout:5/208: mknod d4/da/d27/c3e 0 2024-04-25T01:36:00.089 INFO:tasks.workunit.client.1.smithi138.stdout:3/296: readlink db/d14/l51 0 2024-04-25T01:36:00.090 INFO:tasks.workunit.client.1.smithi138.stdout:2/314: read - d0/d21/f5b zero size 2024-04-25T01:36:00.092 INFO:tasks.workunit.client.1.smithi138.stdout:6/252: getdents dc/d2e/d37 0 2024-04-25T01:36:00.095 INFO:tasks.workunit.client.1.smithi138.stdout:0/237: dwrite d6/f1e [0,4194304] 0 2024-04-25T01:36:00.095 INFO:tasks.workunit.client.1.smithi138.stdout:2/315: mknod d0/d5a/d15/d44/d4c/c5f 0 2024-04-25T01:36:00.096 INFO:tasks.workunit.client.1.smithi138.stdout:0/238: fsync d6/df/f15 0 2024-04-25T01:36:00.101 INFO:tasks.workunit.client.1.smithi138.stdout:6/253: mkdir dc/d2e/d37/d56 0 2024-04-25T01:36:00.101 INFO:tasks.workunit.client.1.smithi138.stdout:5/209: creat d4/da/d27/d1c/d22/f3f x:0 0 0 2024-04-25T01:36:00.104 INFO:tasks.workunit.client.1.smithi138.stdout:2/316: creat d0/d32/d35/f60 x:0 0 0 2024-04-25T01:36:00.105 INFO:tasks.workunit.client.1.smithi138.stdout:3/297: getdents db/df/d17 0 2024-04-25T01:36:00.105 INFO:tasks.workunit.client.1.smithi138.stdout:5/210: chown d4/da/d27/d1c/d22/l31 5645 1 2024-04-25T01:36:00.107 INFO:tasks.workunit.client.1.smithi138.stdout:6/254: symlink dc/d10/l57 0 2024-04-25T01:36:00.111 INFO:tasks.workunit.client.1.smithi138.stdout:6/255: dread - dc/d10/d28/f48 zero size 2024-04-25T01:36:00.114 INFO:tasks.workunit.client.1.smithi138.stdout:2/317: dwrite d0/d5a/d15/f5d [0,4194304] 0 2024-04-25T01:36:00.114 INFO:tasks.workunit.client.1.smithi138.stdout:6/256: write dc/d10/f26 [3076189,130203] 0 2024-04-25T01:36:00.114 INFO:tasks.workunit.client.1.smithi138.stdout:0/239: truncate d6/f28 1531402 0 2024-04-25T01:36:00.117 INFO:tasks.workunit.client.1.smithi138.stdout:5/211: dwrite d4/da/d27/d1c/d22/f24 [4194304,4194304] 0 2024-04-25T01:36:00.119 INFO:tasks.workunit.client.1.smithi138.stdout:6/257: symlink dc/d10/l58 0 2024-04-25T01:36:00.123 INFO:tasks.workunit.client.1.smithi138.stdout:3/298: dwrite db/df/d17/f1c [4194304,4194304] 0 2024-04-25T01:36:00.124 INFO:tasks.workunit.client.1.smithi138.stdout:0/240: link d6/df/d3a/f4a d6/df/d41/d49/f51 0 2024-04-25T01:36:00.125 INFO:tasks.workunit.client.1.smithi138.stdout:3/299: fdatasync db/df/d17/d23/d34/f52 0 2024-04-25T01:36:00.130 INFO:tasks.workunit.client.1.smithi138.stdout:0/241: mkdir d6/d39/d52 0 2024-04-25T01:36:00.131 INFO:tasks.workunit.client.1.smithi138.stdout:5/212: mkdir d4/da/d27/d1c/d3c/d40 0 2024-04-25T01:36:00.133 INFO:tasks.workunit.client.1.smithi138.stdout:3/300: creat db/df/d17/d23/f54 x:0 0 0 2024-04-25T01:36:00.134 INFO:tasks.workunit.client.1.smithi138.stdout:0/242: mknod d6/d2f/d31/c53 0 2024-04-25T01:36:00.137 INFO:tasks.workunit.client.1.smithi138.stdout:3/301: mknod db/d14/c55 0 2024-04-25T01:36:00.139 INFO:tasks.workunit.client.1.smithi138.stdout:6/258: link dc/d10/l45 dc/d2e/l59 0 2024-04-25T01:36:00.140 INFO:tasks.workunit.client.1.smithi138.stdout:2/318: dread d0/d5a/d15/d1f/f4a [0,4194304] 0 2024-04-25T01:36:00.142 INFO:tasks.workunit.client.1.smithi138.stdout:5/213: getdents d4/da/d27/d1c/d22 0 2024-04-25T01:36:00.143 INFO:tasks.workunit.client.1.smithi138.stdout:0/243: dwrite d6/fe [0,4194304] 0 2024-04-25T01:36:00.147 INFO:tasks.workunit.client.1.smithi138.stdout:0/244: chown d6/d2f/d31/c53 791 1 2024-04-25T01:36:00.148 INFO:tasks.workunit.client.1.smithi138.stdout:6/259: dread dc/d10/f25 [0,4194304] 0 2024-04-25T01:36:00.148 INFO:tasks.workunit.client.1.smithi138.stdout:2/319: rmdir d0/d21 39 2024-04-25T01:36:00.150 INFO:tasks.workunit.client.1.smithi138.stdout:3/302: dread db/f15 [4194304,4194304] 0 2024-04-25T01:36:00.154 INFO:tasks.workunit.client.1.smithi138.stdout:2/320: truncate d0/d21/f5b 7759 0 2024-04-25T01:36:00.157 INFO:tasks.workunit.client.1.smithi138.stdout:0/245: mkdir d6/d54 0 2024-04-25T01:36:00.157 INFO:tasks.workunit.client.1.smithi138.stdout:2/321: chown d0/d5a/d15/l2f 947 1 2024-04-25T01:36:00.157 INFO:tasks.workunit.client.1.smithi138.stdout:3/303: symlink db/df/d17/l56 0 2024-04-25T01:36:00.158 INFO:tasks.workunit.client.1.smithi138.stdout:6/260: creat dc/d2e/d55/f5a x:0 0 0 2024-04-25T01:36:00.161 INFO:tasks.workunit.client.0.smithi033.stdout:8/752: sync 2024-04-25T01:36:00.161 INFO:tasks.workunit.client.0.smithi033.stdout:4/844: sync 2024-04-25T01:36:00.162 INFO:tasks.workunit.client.0.smithi033.stdout:9/793: sync 2024-04-25T01:36:00.162 INFO:tasks.workunit.client.0.smithi033.stdout:6/914: sync 2024-04-25T01:36:00.162 INFO:tasks.workunit.client.0.smithi033.stdout:3/842: sync 2024-04-25T01:36:00.162 INFO:tasks.workunit.client.0.smithi033.stdout:7/792: sync 2024-04-25T01:36:00.162 INFO:tasks.workunit.client.0.smithi033.stdout:2/782: sync 2024-04-25T01:36:00.164 INFO:tasks.workunit.client.0.smithi033.stdout:2/783: chown da/d27/d31/d40/lfa 1 1 2024-04-25T01:36:00.164 INFO:tasks.workunit.client.0.smithi033.stdout:6/915: write d4/d24/d36/fb5 [417692,120857] 0 2024-04-25T01:36:00.165 INFO:tasks.workunit.client.1.smithi138.stdout:0/246: dwrite d6/f45 [0,4194304] 0 2024-04-25T01:36:00.165 INFO:tasks.workunit.client.1.smithi138.stdout:2/322: dwrite d0/d5a/f4e [0,4194304] 0 2024-04-25T01:36:00.165 INFO:tasks.workunit.client.1.smithi138.stdout:6/261: rename c9 to dc/d2e/d53/c5b 0 2024-04-25T01:36:00.168 INFO:tasks.workunit.client.0.smithi033.stdout:9/794: creat d0/d2/d43/d4b/dc5/de0/d7e/ddc/f117 x:0 0 0 2024-04-25T01:36:00.168 INFO:tasks.workunit.client.0.smithi033.stdout:2/784: chown da/d27/d31/d40/d77/cb7 2113008 1 2024-04-25T01:36:00.168 INFO:tasks.workunit.client.0.smithi033.stdout:9/795: fdatasync d0/d2/dd/fe 0 2024-04-25T01:36:00.168 INFO:tasks.workunit.client.1.smithi138.stdout:2/323: chown d0/d5a/ld 1 1 2024-04-25T01:36:00.171 INFO:tasks.workunit.client.0.smithi033.stdout:9/796: read - d0/d2/d43/d4b/dc5/de0/d42/dea/d105/f111 zero size 2024-04-25T01:36:00.172 INFO:tasks.workunit.client.1.smithi138.stdout:6/262: read f5 [2502814,129667] 0 2024-04-25T01:36:00.172 INFO:tasks.workunit.client.1.smithi138.stdout:0/247: creat d6/df/d41/f55 x:0 0 0 2024-04-25T01:36:00.174 INFO:tasks.workunit.client.0.smithi033.stdout:9/797: fsync d0/d2/d43/d4b/dc5/de0/d7e/f66 0 2024-04-25T01:36:00.175 INFO:tasks.workunit.client.1.smithi138.stdout:5/214: dread d4/da/d27/d1c/f26 [0,4194304] 0 2024-04-25T01:36:00.177 INFO:tasks.workunit.client.0.smithi033.stdout:2/785: creat da/d2c/ffe x:0 0 0 2024-04-25T01:36:00.181 INFO:tasks.workunit.client.1.smithi138.stdout:2/324: read d0/d18/f1b [717302,103843] 0 2024-04-25T01:36:00.181 INFO:tasks.workunit.client.1.smithi138.stdout:5/215: write d4/f1a [1698846,23129] 0 2024-04-25T01:36:00.181 INFO:tasks.workunit.client.0.smithi033.stdout:4/845: dread d6/d12/d84/fae [0,4194304] 0 2024-04-25T01:36:00.183 INFO:tasks.workunit.client.0.smithi033.stdout:2/786: creat da/d27/d31/d40/d8a/dbb/fff x:0 0 0 2024-04-25T01:36:00.187 INFO:tasks.workunit.client.1.smithi138.stdout:0/248: dwrite d6/d2f/f34 [4194304,4194304] 0 2024-04-25T01:36:00.187 INFO:tasks.workunit.client.0.smithi033.stdout:4/846: truncate d6/d14/d48/f99 3691256 0 2024-04-25T01:36:00.188 INFO:tasks.workunit.client.1.smithi138.stdout:5/216: symlink d4/da/d27/l41 0 2024-04-25T01:36:00.190 INFO:tasks.workunit.client.0.smithi033.stdout:4/847: unlink d6/d12/l42 0 2024-04-25T01:36:00.190 INFO:tasks.workunit.client.1.smithi138.stdout:6/263: dwrite dc/d2e/f49 [0,4194304] 0 2024-04-25T01:36:00.193 INFO:tasks.workunit.client.0.smithi033.stdout:4/848: symlink d6/d12/d38/l111 0 2024-04-25T01:36:00.195 INFO:tasks.workunit.client.1.smithi138.stdout:2/325: dread d0/d5a/f3c [0,4194304] 0 2024-04-25T01:36:00.197 INFO:tasks.workunit.client.1.smithi138.stdout:6/264: truncate dc/d2e/f4a 966676 0 2024-04-25T01:36:00.198 INFO:tasks.workunit.client.1.smithi138.stdout:3/304: dread db/df/d17/d23/d34/f50 [4194304,4194304] 0 2024-04-25T01:36:00.199 INFO:tasks.workunit.client.0.smithi033.stdout:4/849: mkdir d6/d14/d35/d44/d45/d46/d57/d70/d87/d112 0 2024-04-25T01:36:00.200 INFO:tasks.workunit.client.0.smithi033.stdout:7/793: write d4/d2e/d6b/d1f/fc7 [1009898,68262] 0 2024-04-25T01:36:00.201 INFO:tasks.workunit.client.1.smithi138.stdout:0/249: creat d6/f56 x:0 0 0 2024-04-25T01:36:00.202 INFO:tasks.workunit.client.0.smithi033.stdout:4/850: creat d6/d14/df6/d10e/f113 x:0 0 0 2024-04-25T01:36:00.203 INFO:tasks.workunit.client.0.smithi033.stdout:8/753: truncate d6/db/fc 1065409 0 2024-04-25T01:36:00.204 INFO:tasks.workunit.client.1.smithi138.stdout:3/305: mkdir db/df/d17/d20/d57 0 2024-04-25T01:36:00.206 INFO:tasks.workunit.client.1.smithi138.stdout:0/250: getdents d6/d39/d52 0 2024-04-25T01:36:00.207 INFO:tasks.workunit.client.1.smithi138.stdout:5/217: dwrite d4/da/d27/f1f [0,4194304] 0 2024-04-25T01:36:00.211 INFO:tasks.workunit.client.0.smithi033.stdout:3/843: truncate d8/dd/d3d/dce/fee 819290 0 2024-04-25T01:36:00.212 INFO:tasks.workunit.client.0.smithi033.stdout:9/798: write d0/d2/d43/d4b/f9d [1254956,52726] 0 2024-04-25T01:36:00.213 INFO:tasks.workunit.client.1.smithi138.stdout:0/251: mkdir d6/d19/d57 0 2024-04-25T01:36:00.214 INFO:tasks.workunit.client.1.smithi138.stdout:6/265: dwrite dc/f16 [0,4194304] 0 2024-04-25T01:36:00.214 INFO:tasks.workunit.client.0.smithi033.stdout:8/754: fdatasync d6/db/d80/fa3 0 2024-04-25T01:36:00.215 INFO:tasks.workunit.client.0.smithi033.stdout:4/851: mknod d6/d14/d35/d3a/de8/c114 0 2024-04-25T01:36:00.218 INFO:tasks.workunit.client.1.smithi138.stdout:0/252: mkdir d6/d39/d58 0 2024-04-25T01:36:00.218 INFO:tasks.workunit.client.1.smithi138.stdout:5/218: creat d4/da/d27/d1c/d3c/d40/f42 x:0 0 0 2024-04-25T01:36:00.218 INFO:tasks.workunit.client.1.smithi138.stdout:3/306: dwrite db/df/d17/d20/f26 [0,4194304] 0 2024-04-25T01:36:00.220 INFO:tasks.workunit.client.1.smithi138.stdout:6/266: write dc/d2e/d55/f5a [411595,91116] 0 2024-04-25T01:36:00.223 INFO:tasks.workunit.client.0.smithi033.stdout:6/916: dwrite d4/d24/d7c/fc3 [0,4194304] 0 2024-04-25T01:36:00.225 INFO:tasks.workunit.client.0.smithi033.stdout:8/755: rename d6/d45/d52/fd9 to d6/d45/d51/d5d/d9d/f118 0 2024-04-25T01:36:00.225 INFO:tasks.workunit.client.1.smithi138.stdout:6/267: mkdir dc/d2e/d53/d5c 0 2024-04-25T01:36:00.225 INFO:tasks.workunit.client.1.smithi138.stdout:5/219: mknod d4/da/d27/d1c/d3c/c43 0 2024-04-25T01:36:00.225 INFO:tasks.workunit.client.1.smithi138.stdout:6/268: chown dc/f35 86586 1 2024-04-25T01:36:00.226 INFO:tasks.workunit.client.0.smithi033.stdout:7/794: dwrite d4/d2e/d7f/d98/fd7 [0,4194304] 0 2024-04-25T01:36:00.226 INFO:tasks.workunit.client.0.smithi033.stdout:3/844: mkdir d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d12a 0 2024-04-25T01:36:00.231 INFO:tasks.workunit.client.1.smithi138.stdout:0/253: dwrite d6/d1d/f3c [0,4194304] 0 2024-04-25T01:36:00.231 INFO:tasks.workunit.client.1.smithi138.stdout:3/307: rename db/df/d17/d20/l28 to db/df/d17/d23/d53/l58 0 2024-04-25T01:36:00.233 INFO:tasks.workunit.client.1.smithi138.stdout:2/326: dread d0/d18/f1b [0,4194304] 0 2024-04-25T01:36:00.234 INFO:tasks.workunit.client.0.smithi033.stdout:9/799: creat d0/d2/d43/d4b/dc5/f118 x:0 0 0 2024-04-25T01:36:00.234 INFO:tasks.workunit.client.0.smithi033.stdout:8/756: write d6/db/f2d [401480,73469] 0 2024-04-25T01:36:00.234 INFO:tasks.workunit.client.1.smithi138.stdout:0/254: chown d6/d2f/d31/l4f 151787 1 2024-04-25T01:36:00.237 INFO:tasks.workunit.client.0.smithi033.stdout:6/917: truncate d4/d2a/f30 1319927 0 2024-04-25T01:36:00.240 INFO:tasks.workunit.client.1.smithi138.stdout:3/308: stat db/f3f 0 2024-04-25T01:36:00.240 INFO:tasks.workunit.client.1.smithi138.stdout:5/220: rename d4/da/d27/d1c/c29 to d4/da/c44 0 2024-04-25T01:36:00.241 INFO:tasks.workunit.client.0.smithi033.stdout:8/757: truncate d6/d45/d52/d2e/f30 1498574 0 2024-04-25T01:36:00.242 INFO:tasks.workunit.client.0.smithi033.stdout:8/758: readlink d6/d45/d52/l90 0 2024-04-25T01:36:00.242 INFO:tasks.workunit.client.1.smithi138.stdout:5/221: chown d4/da/d27/d1c/d22/l31 960 1 2024-04-25T01:36:00.244 INFO:tasks.workunit.client.0.smithi033.stdout:6/918: symlink d4/d1d/d3e/d41/d54/l135 0 2024-04-25T01:36:00.244 INFO:tasks.workunit.client.1.smithi138.stdout:0/255: mknod d6/d39/d52/c59 0 2024-04-25T01:36:00.244 INFO:tasks.workunit.client.1.smithi138.stdout:2/327: dread d0/d5a/d15/d1f/f2d [0,4194304] 0 2024-04-25T01:36:00.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/309: fdatasync db/d14/f32 0 2024-04-25T01:36:00.249 INFO:tasks.workunit.client.0.smithi033.stdout:6/919: unlink d4/d2a/l45 0 2024-04-25T01:36:00.249 INFO:tasks.workunit.client.1.smithi138.stdout:6/269: getdents dc/d2e/d37 0 2024-04-25T01:36:00.250 INFO:tasks.workunit.client.1.smithi138.stdout:0/256: dread d6/d19/f33 [4194304,4194304] 0 2024-04-25T01:36:00.251 INFO:tasks.workunit.client.0.smithi033.stdout:4/852: dread d6/d14/f33 [0,4194304] 0 2024-04-25T01:36:00.251 INFO:tasks.workunit.client.1.smithi138.stdout:5/222: mkdir d4/da/d27/d1c/d22/d45 0 2024-04-25T01:36:00.252 INFO:tasks.workunit.client.1.smithi138.stdout:5/223: dread - d4/da/f20 zero size 2024-04-25T01:36:00.252 INFO:tasks.workunit.client.1.smithi138.stdout:3/310: mknod db/d14/d40/c59 0 2024-04-25T01:36:00.254 INFO:tasks.workunit.client.0.smithi033.stdout:4/853: chown d6/d12/d38/dc0/dff 10084 1 2024-04-25T01:36:00.254 INFO:tasks.workunit.client.0.smithi033.stdout:8/759: dwrite d6/db/f10f [0,4194304] 0 2024-04-25T01:36:00.255 INFO:tasks.workunit.client.1.smithi138.stdout:6/270: dread dc/d10/d28/f3c [0,4194304] 0 2024-04-25T01:36:00.256 INFO:tasks.workunit.client.0.smithi033.stdout:4/854: readlink d6/d14/d35/d3a/l10a 0 2024-04-25T01:36:00.257 INFO:tasks.workunit.client.1.smithi138.stdout:3/311: write db/f15 [7930891,28506] 0 2024-04-25T01:36:00.259 INFO:tasks.workunit.client.1.smithi138.stdout:6/271: symlink dc/d2e/d37/l5d 0 2024-04-25T01:36:00.259 INFO:tasks.workunit.client.0.smithi033.stdout:5/836: sync 2024-04-25T01:36:00.260 INFO:tasks.workunit.client.0.smithi033.stdout:1/646: sync 2024-04-25T01:36:00.260 INFO:tasks.workunit.client.0.smithi033.stdout:0/735: sync 2024-04-25T01:36:00.260 INFO:tasks.workunit.client.0.smithi033.stdout:4/855: chown d6/l105 16749 1 2024-04-25T01:36:00.261 INFO:tasks.workunit.client.1.smithi138.stdout:6/272: write f8 [1360114,45945] 0 2024-04-25T01:36:00.262 INFO:tasks.workunit.client.0.smithi033.stdout:6/920: dread d4/d1d/d3e/d41/d5c/f8b [4194304,4194304] 0 2024-04-25T01:36:00.265 INFO:tasks.workunit.client.1.smithi138.stdout:5/224: dread d4/f23 [0,4194304] 0 2024-04-25T01:36:00.266 INFO:tasks.workunit.client.0.smithi033.stdout:4/856: symlink d6/d14/d6d/l115 0 2024-04-25T01:36:00.267 INFO:tasks.workunit.client.0.smithi033.stdout:6/921: fsync d4/d1d/d3e/d41/f10c 0 2024-04-25T01:36:00.268 INFO:tasks.workunit.client.0.smithi033.stdout:5/837: mkdir d1c/d2a/d50/d52/d10c 0 2024-04-25T01:36:00.269 INFO:tasks.workunit.client.1.smithi138.stdout:3/312: link db/df/d17/d23/d34/f50 db/df/d17/f5a 0 2024-04-25T01:36:00.269 INFO:tasks.workunit.client.1.smithi138.stdout:5/225: dread d4/da/d27/d1c/f28 [0,4194304] 0 2024-04-25T01:36:00.271 INFO:tasks.workunit.client.1.smithi138.stdout:6/273: link dc/d10/f33 dc/d54/f5e 0 2024-04-25T01:36:00.272 INFO:tasks.workunit.client.0.smithi033.stdout:4/857: write d6/d14/fd7 [475354,50027] 0 2024-04-25T01:36:00.274 INFO:tasks.workunit.client.0.smithi033.stdout:0/736: dwrite d7/da/d80/fe4 [0,4194304] 0 2024-04-25T01:36:00.274 INFO:tasks.workunit.client.0.smithi033.stdout:5/838: dread f1b [0,4194304] 0 2024-04-25T01:36:00.276 INFO:tasks.workunit.client.1.smithi138.stdout:8/444: sync 2024-04-25T01:36:00.278 INFO:tasks.workunit.client.0.smithi033.stdout:0/737: read d7/da/f48 [990093,118080] 0 2024-04-25T01:36:00.278 INFO:tasks.workunit.client.0.smithi033.stdout:1/647: creat d2/d5/d1b/fc7 x:0 0 0 2024-04-25T01:36:00.278 INFO:tasks.workunit.client.1.smithi138.stdout:8/445: stat d3 0 2024-04-25T01:36:00.279 INFO:tasks.workunit.client.0.smithi033.stdout:0/738: chown d7/cd 175407191 1 2024-04-25T01:36:00.279 INFO:tasks.workunit.client.1.smithi138.stdout:3/313: dwrite db/df/f4f [0,4194304] 0 2024-04-25T01:36:00.279 INFO:tasks.workunit.client.1.smithi138.stdout:5/226: dread d4/da/d27/d1c/f26 [0,4194304] 0 2024-04-25T01:36:00.282 INFO:tasks.workunit.client.0.smithi033.stdout:3/845: write d8/f37 [3749766,129348] 0 2024-04-25T01:36:00.284 INFO:tasks.workunit.client.0.smithi033.stdout:5/839: creat d1c/d2a/d50/d52/da6/d3e/d70/dae/df1/f10d x:0 0 0 2024-04-25T01:36:00.289 INFO:tasks.workunit.client.0.smithi033.stdout:1/648: creat d2/d5/d75/d59/d83/d98/fc8 x:0 0 0 2024-04-25T01:36:00.290 INFO:tasks.workunit.client.0.smithi033.stdout:7/795: truncate d4/d2e/d6b/d67/fd8 532843 0 2024-04-25T01:36:00.290 INFO:tasks.workunit.client.0.smithi033.stdout:1/649: chown d2/d5/d75/d59/d83/fb4 121319352 1 2024-04-25T01:36:00.292 INFO:tasks.workunit.client.0.smithi033.stdout:0/739: unlink d7/da/df/dc7/lda 0 2024-04-25T01:36:00.293 INFO:tasks.workunit.client.0.smithi033.stdout:9/800: write d0/d2/dd/d20/f38 [1162336,13558] 0 2024-04-25T01:36:00.295 INFO:tasks.workunit.client.1.smithi138.stdout:2/328: dread d0/f9 [0,4194304] 0 2024-04-25T01:36:00.295 INFO:tasks.workunit.client.1.smithi138.stdout:6/274: dwrite dc/d2e/d55/f5a [0,4194304] 0 2024-04-25T01:36:00.295 INFO:tasks.workunit.client.1.smithi138.stdout:5/227: symlink d4/da/d27/d1c/d22/d35/l46 0 2024-04-25T01:36:00.295 INFO:tasks.workunit.client.0.smithi033.stdout:4/858: symlink d6/d12/d38/l116 0 2024-04-25T01:36:00.298 INFO:tasks.workunit.client.1.smithi138.stdout:8/446: dwrite d3/f6d [0,4194304] 0 2024-04-25T01:36:00.298 INFO:tasks.workunit.client.1.smithi138.stdout:3/314: rename db/d14/f2a to db/f5b 0 2024-04-25T01:36:00.301 INFO:tasks.workunit.client.1.smithi138.stdout:5/228: chown d4/da/d27/d1c/d3c/d40/f42 104620 1 2024-04-25T01:36:00.301 INFO:tasks.workunit.client.1.smithi138.stdout:2/329: chown d0/d21/d45/c47 2 1 2024-04-25T01:36:00.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/315: read f1 [3099162,106476] 0 2024-04-25T01:36:00.301 INFO:tasks.workunit.client.1.smithi138.stdout:8/447: stat d3/d16/d88/c32 0 2024-04-25T01:36:00.302 INFO:tasks.workunit.client.0.smithi033.stdout:5/840: mkdir d1c/d2a/d50/d52/da6/d3e/d70/d9b/d10e 0 2024-04-25T01:36:00.305 INFO:tasks.workunit.client.1.smithi138.stdout:5/229: write d4/da/fe [4668089,66909] 0 2024-04-25T01:36:00.307 INFO:tasks.workunit.client.0.smithi033.stdout:9/801: dwrite d0/d2/d43/d4b/fd9 [0,4194304] 0 2024-04-25T01:36:00.307 INFO:tasks.workunit.client.0.smithi033.stdout:7/796: unlink d4/f39 0 2024-04-25T01:36:00.308 INFO:tasks.workunit.client.1.smithi138.stdout:0/257: dwrite d6/f28 [0,4194304] 0 2024-04-25T01:36:00.308 INFO:tasks.workunit.client.1.smithi138.stdout:8/448: chown d3/d16/d88/da/db/d15/f72 13653 1 2024-04-25T01:36:00.308 INFO:tasks.workunit.client.1.smithi138.stdout:6/275: link fb dc/d2e/d37/f5f 0 2024-04-25T01:36:00.309 INFO:tasks.workunit.client.0.smithi033.stdout:1/650: write d2/d5/d1b/f54 [3550207,71277] 0 2024-04-25T01:36:00.310 INFO:tasks.workunit.client.1.smithi138.stdout:6/276: chown dc/d10/l1b 2 1 2024-04-25T01:36:00.312 INFO:tasks.workunit.client.0.smithi033.stdout:4/859: symlink d6/d14/d35/d44/d45/d46/db0/dcc/l117 0 2024-04-25T01:36:00.315 INFO:tasks.workunit.client.1.smithi138.stdout:2/330: write d0/d21/f27 [321167,21109] 0 2024-04-25T01:36:00.315 INFO:tasks.workunit.client.1.smithi138.stdout:3/316: getdents db/d14/d40/d41 0 2024-04-25T01:36:00.316 INFO:tasks.workunit.client.0.smithi033.stdout:1/651: readlink d2/d5/d75/d59/l5d 0 2024-04-25T01:36:00.318 INFO:tasks.workunit.client.1.smithi138.stdout:6/277: unlink dc/d10/l1b 0 2024-04-25T01:36:00.319 INFO:tasks.workunit.client.0.smithi033.stdout:8/760: dwrite d6/db/f46 [0,4194304] 0 2024-04-25T01:36:00.319 INFO:tasks.workunit.client.0.smithi033.stdout:5/841: truncate f1b 134135 0 2024-04-25T01:36:00.321 INFO:tasks.workunit.client.0.smithi033.stdout:7/797: creat d4/d23/da7/dc2/ff8 x:0 0 0 2024-04-25T01:36:00.321 INFO:tasks.workunit.client.0.smithi033.stdout:0/740: symlink d7/da/df/d50/dfe/l102 0 2024-04-25T01:36:00.322 INFO:tasks.workunit.client.1.smithi138.stdout:2/331: dread - d0/d21/f41 zero size 2024-04-25T01:36:00.323 INFO:tasks.workunit.client.0.smithi033.stdout:6/922: write d4/d2a/f132 [892799,89414] 0 2024-04-25T01:36:00.323 INFO:tasks.workunit.client.1.smithi138.stdout:0/258: dwrite d6/df/f15 [4194304,4194304] 0 2024-04-25T01:36:00.324 INFO:tasks.workunit.client.1.smithi138.stdout:6/278: rmdir dc/d54 39 2024-04-25T01:36:00.326 INFO:tasks.workunit.client.0.smithi033.stdout:1/652: fsync d2/d5/d1b/f54 0 2024-04-25T01:36:00.327 INFO:tasks.workunit.client.0.smithi033.stdout:1/653: chown d2/d5/d75/d59/d83/fa9 102553019 1 2024-04-25T01:36:00.328 INFO:tasks.workunit.client.0.smithi033.stdout:6/923: write d4/d15/fd5 [269983,8340] 0 2024-04-25T01:36:00.329 INFO:tasks.workunit.client.1.smithi138.stdout:5/230: dwrite d4/da/fe [0,4194304] 0 2024-04-25T01:36:00.330 INFO:tasks.workunit.client.1.smithi138.stdout:8/449: getdents d3/d16/d88/da/db 0 2024-04-25T01:36:00.330 INFO:tasks.workunit.client.1.smithi138.stdout:6/279: fdatasync dc/d10/d28/f40 0 2024-04-25T01:36:00.333 INFO:tasks.workunit.client.0.smithi033.stdout:3/846: symlink d8/dd/d3d/d63/d7c/d4d/da5/l12b 0 2024-04-25T01:36:00.333 INFO:tasks.workunit.client.0.smithi033.stdout:2/787: sync 2024-04-25T01:36:00.334 INFO:tasks.workunit.client.1.smithi138.stdout:5/231: chown d4/da/d27/d1c/d22/l31 193 1 2024-04-25T01:36:00.334 INFO:tasks.workunit.client.1.smithi138.stdout:6/280: readlink dc/d10/l58 0 2024-04-25T01:36:00.335 INFO:tasks.workunit.client.1.smithi138.stdout:0/259: creat d6/d2f/f5a x:0 0 0 2024-04-25T01:36:00.336 INFO:tasks.workunit.client.0.smithi033.stdout:8/761: dwrite d6/db/f18 [0,4194304] 0 2024-04-25T01:36:00.336 INFO:tasks.workunit.client.0.smithi033.stdout:3/847: chown d8/dd/d11/ca6 15385 1 2024-04-25T01:36:00.336 INFO:tasks.workunit.client.1.smithi138.stdout:0/260: write f1 [1502713,29042] 0 2024-04-25T01:36:00.337 INFO:tasks.workunit.client.1.smithi138.stdout:5/232: chown d4/da/d27/d1c/d22/d35/l38 107651 1 2024-04-25T01:36:00.338 INFO:tasks.workunit.client.0.smithi033.stdout:9/802: unlink d0/d2/d43/f3c 0 2024-04-25T01:36:00.339 INFO:tasks.workunit.client.1.smithi138.stdout:8/450: creat d3/d16/d1f/d2a/f8b x:0 0 0 2024-04-25T01:36:00.340 INFO:tasks.workunit.client.1.smithi138.stdout:6/281: chown dc/d10/f15 11 1 2024-04-25T01:36:00.341 INFO:tasks.workunit.client.0.smithi033.stdout:4/860: creat d6/d14/d35/d44/d45/d46/d57/d70/d87/f118 x:0 0 0 2024-04-25T01:36:00.342 INFO:tasks.workunit.client.1.smithi138.stdout:2/332: link d0/d32/l42 d0/d5a/d15/d51/l61 0 2024-04-25T01:36:00.343 INFO:tasks.workunit.client.0.smithi033.stdout:9/803: fdatasync d0/d2/d43/d4b/dc5/de0/d6e/f95 0 2024-04-25T01:36:00.344 INFO:tasks.workunit.client.0.smithi033.stdout:0/741: creat d7/d33/dcd/f103 x:0 0 0 2024-04-25T01:36:00.344 INFO:tasks.workunit.client.1.smithi138.stdout:2/333: chown d0/d32/d35/c5e 3 1 2024-04-25T01:36:00.345 INFO:tasks.workunit.client.1.smithi138.stdout:2/334: stat d0/l1c 0 2024-04-25T01:36:00.346 INFO:tasks.workunit.client.1.smithi138.stdout:6/282: link dc/d10/f4b dc/f60 0 2024-04-25T01:36:00.348 INFO:tasks.workunit.client.0.smithi033.stdout:7/798: unlink d4/d2e/d7f/d98/dca/fe7 0 2024-04-25T01:36:00.349 INFO:tasks.workunit.client.1.smithi138.stdout:8/451: dwrite d3/d16/d1f/d2a/f8b [0,4194304] 0 2024-04-25T01:36:00.350 INFO:tasks.workunit.client.1.smithi138.stdout:2/335: creat d0/d5a/d15/d51/f62 x:0 0 0 2024-04-25T01:36:00.350 INFO:tasks.workunit.client.0.smithi033.stdout:7/799: chown d4/f7a 109328 1 2024-04-25T01:36:00.351 INFO:tasks.workunit.client.1.smithi138.stdout:6/283: write dc/d10/f25 [7341534,92437] 0 2024-04-25T01:36:00.352 INFO:tasks.workunit.client.1.smithi138.stdout:2/336: creat d0/d32/f63 x:0 0 0 2024-04-25T01:36:00.354 INFO:tasks.workunit.client.0.smithi033.stdout:6/924: rename d4/d24/d36/d42/d97/cc8 to d4/d24/d36/d42/d97/de0/c136 0 2024-04-25T01:36:00.355 INFO:tasks.workunit.client.0.smithi033.stdout:6/925: chown d4/dfc/c10f 621 1 2024-04-25T01:36:00.355 INFO:tasks.workunit.client.0.smithi033.stdout:2/788: fdatasync da/d5b/d5e/f9f 0 2024-04-25T01:36:00.356 INFO:tasks.workunit.client.1.smithi138.stdout:8/452: mknod d3/d16/d88/da/d70/c8c 0 2024-04-25T01:36:00.361 INFO:tasks.workunit.client.1.smithi138.stdout:8/453: rename d3/d16/d88/da/db/c2e to d3/d44/d48/c8d 0 2024-04-25T01:36:00.362 INFO:tasks.workunit.client.1.smithi138.stdout:2/337: dwrite d0/d5a/d15/d1f/f26 [0,4194304] 0 2024-04-25T01:36:00.363 INFO:tasks.workunit.client.0.smithi033.stdout:4/861: unlink d6/d12/c25 0 2024-04-25T01:36:00.364 INFO:tasks.workunit.client.1.smithi138.stdout:8/454: creat d3/d16/f8e x:0 0 0 2024-04-25T01:36:00.365 INFO:tasks.workunit.client.0.smithi033.stdout:4/862: chown d6/d14/d35/d44/d45/d46/d57/d70/d87/fd0 118 1 2024-04-25T01:36:00.365 INFO:tasks.workunit.client.0.smithi033.stdout:9/804: creat d0/d2/dd/da7/f119 x:0 0 0 2024-04-25T01:36:00.367 INFO:tasks.workunit.client.1.smithi138.stdout:8/455: write d3/d16/d88/f40 [376549,89165] 0 2024-04-25T01:36:00.368 INFO:tasks.workunit.client.1.smithi138.stdout:2/338: rename d0/d5a/d15/l2f to d0/d18/l64 0 2024-04-25T01:36:00.369 INFO:tasks.workunit.client.0.smithi033.stdout:1/654: mkdir d2/d11/dc9 0 2024-04-25T01:36:00.370 INFO:tasks.workunit.client.0.smithi033.stdout:7/800: mkdir d4/d23/da7/dc2/df9 0 2024-04-25T01:36:00.371 INFO:tasks.workunit.client.1.smithi138.stdout:8/456: fdatasync d3/d16/d1f/d2a/f8b 0 2024-04-25T01:36:00.372 INFO:tasks.workunit.client.0.smithi033.stdout:8/762: mkdir d6/d45/d52/d119 0 2024-04-25T01:36:00.373 INFO:tasks.workunit.client.1.smithi138.stdout:2/339: rename d0/d32/d35/c58 to d0/d32/c65 0 2024-04-25T01:36:00.374 INFO:tasks.workunit.client.1.smithi138.stdout:6/284: dread dc/d10/f22 [0,4194304] 0 2024-04-25T01:36:00.377 INFO:tasks.workunit.client.1.smithi138.stdout:3/317: dread db/df/d17/f2e [0,4194304] 0 2024-04-25T01:36:00.377 INFO:tasks.workunit.client.0.smithi033.stdout:1/655: fdatasync d2/d5/d1b/d46/d5f/d63/d8f/fbc 0 2024-04-25T01:36:00.378 INFO:tasks.workunit.client.0.smithi033.stdout:6/926: stat d4/d1d/d3e/d41/d54/da1/l127 0 2024-04-25T01:36:00.379 INFO:tasks.workunit.client.0.smithi033.stdout:4/863: dwrite d6/d14/d35/d3a/d4c/f78 [0,4194304] 0 2024-04-25T01:36:00.379 INFO:tasks.workunit.client.1.smithi138.stdout:2/340: mknod d0/d21/c66 0 2024-04-25T01:36:00.380 INFO:tasks.workunit.client.0.smithi033.stdout:3/848: dread d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/fa9 [0,4194304] 0 2024-04-25T01:36:00.382 INFO:tasks.workunit.client.0.smithi033.stdout:7/801: stat d4/d10/f26 0 2024-04-25T01:36:00.382 INFO:tasks.workunit.client.1.smithi138.stdout:2/341: fsync d0/d18/f2a 0 2024-04-25T01:36:00.382 INFO:tasks.workunit.client.0.smithi033.stdout:0/742: dwrite d7/d2d/d4d/d9c/d61/d70/fa9 [4194304,4194304] 0 2024-04-25T01:36:00.383 INFO:tasks.workunit.client.1.smithi138.stdout:3/318: symlink db/d14/d40/d41/l5c 0 2024-04-25T01:36:00.385 INFO:tasks.workunit.client.1.smithi138.stdout:6/285: dwrite dc/d2e/d37/f42 [0,4194304] 0 2024-04-25T01:36:00.385 INFO:tasks.workunit.client.0.smithi033.stdout:8/763: mknod d6/d45/d51/d9e/c11a 0 2024-04-25T01:36:00.392 INFO:tasks.workunit.client.0.smithi033.stdout:1/656: rename d2/d5/d1b/d46/l9c to d2/d7f/db0/lca 0 2024-04-25T01:36:00.392 INFO:tasks.workunit.client.0.smithi033.stdout:2/789: dread da/d27/d31/d40/d8a/dbb/f68 [0,4194304] 0 2024-04-25T01:36:00.393 INFO:tasks.workunit.client.1.smithi138.stdout:0/261: dwrite d6/d19/f33 [0,4194304] 0 2024-04-25T01:36:00.394 INFO:tasks.workunit.client.0.smithi033.stdout:9/805: creat d0/d2/d43/d4b/dc5/f11a x:0 0 0 2024-04-25T01:36:00.395 INFO:tasks.workunit.client.0.smithi033.stdout:1/657: write d2/d5/d75/d59/d83/fb4 [962480,108888] 0 2024-04-25T01:36:00.397 INFO:tasks.workunit.client.0.smithi033.stdout:5/842: truncate d1c/d53/f99 164046 0 2024-04-25T01:36:00.398 INFO:tasks.workunit.client.1.smithi138.stdout:0/262: readlink d6/d1d/l1b 0 2024-04-25T01:36:00.399 INFO:tasks.workunit.client.1.smithi138.stdout:0/263: write d6/f56 [1009581,112972] 0 2024-04-25T01:36:00.401 INFO:tasks.workunit.client.1.smithi138.stdout:0/264: readlink d6/la 0 2024-04-25T01:36:00.401 INFO:tasks.workunit.client.1.smithi138.stdout:0/265: fsync d6/d1d/f24 0 2024-04-25T01:36:00.403 INFO:tasks.workunit.client.0.smithi033.stdout:7/802: rename d4/d57/l5c to d4/d57/db7/lfa 0 2024-04-25T01:36:00.404 INFO:tasks.workunit.client.1.smithi138.stdout:3/319: getdents db/df/d17/d20 0 2024-04-25T01:36:00.408 INFO:tasks.workunit.client.1.smithi138.stdout:8/457: mkdir d3/d16/d88/da/db/d15/d56/d8f 0 2024-04-25T01:36:00.409 INFO:tasks.workunit.client.1.smithi138.stdout:8/458: readlink d3/d16/d88/lc 0 2024-04-25T01:36:00.411 INFO:tasks.workunit.client.0.smithi033.stdout:8/764: mkdir d6/d45/d52/dcc/d112/d11b 0 2024-04-25T01:36:00.411 INFO:tasks.workunit.client.1.smithi138.stdout:0/266: symlink d6/df/d41/d49/l5b 0 2024-04-25T01:36:00.412 INFO:tasks.workunit.client.0.smithi033.stdout:7/803: dwrite d4/d23/da7/dc2/fd6 [0,4194304] 0 2024-04-25T01:36:00.413 INFO:tasks.workunit.client.1.smithi138.stdout:8/459: unlink d3/d44/f7b 0 2024-04-25T01:36:00.416 INFO:tasks.workunit.client.1.smithi138.stdout:8/460: dread d3/d16/d1f/d2a/f6a [0,4194304] 0 2024-04-25T01:36:00.417 INFO:tasks.workunit.client.1.smithi138.stdout:8/461: readlink d3/l17 0 2024-04-25T01:36:00.418 INFO:tasks.workunit.client.0.smithi033.stdout:4/864: rename d6/d14/d35/d3a/d4d/d5e/d7c/d94/ff1 to d6/d14/d35/d44/f119 0 2024-04-25T01:36:00.418 INFO:tasks.workunit.client.0.smithi033.stdout:4/865: stat d6/d14 0 2024-04-25T01:36:00.423 INFO:tasks.workunit.client.0.smithi033.stdout:1/658: mknod d2/d5/ccb 0 2024-04-25T01:36:00.424 INFO:tasks.workunit.client.1.smithi138.stdout:6/286: dread dc/d10/f15 [0,4194304] 0 2024-04-25T01:36:00.425 INFO:tasks.workunit.client.0.smithi033.stdout:8/765: creat d6/d45/d52/db0/dbc/f11c x:0 0 0 2024-04-25T01:36:00.425 INFO:tasks.workunit.client.1.smithi138.stdout:6/287: rename dc/d2e to dc/d2e/d37/d56/d61 22 2024-04-25T01:36:00.426 INFO:tasks.workunit.client.1.smithi138.stdout:6/288: stat dc/d10/f33 0 2024-04-25T01:36:00.426 INFO:tasks.workunit.client.1.smithi138.stdout:6/289: readlink dc/d10/l58 0 2024-04-25T01:36:00.427 INFO:tasks.workunit.client.0.smithi033.stdout:6/927: getdents d4/d1d/d3e/d41/d54 0 2024-04-25T01:36:00.427 INFO:tasks.workunit.client.1.smithi138.stdout:6/290: fdatasync dc/d2e/d55/f5a 0 2024-04-25T01:36:00.429 INFO:tasks.workunit.client.0.smithi033.stdout:3/849: rename d8/dd/d3d/d4f/c97 to d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/c12c 0 2024-04-25T01:36:00.429 INFO:tasks.workunit.client.0.smithi033.stdout:7/804: dread d4/d2e/d6b/fbc [0,4194304] 0 2024-04-25T01:36:00.432 INFO:tasks.workunit.client.0.smithi033.stdout:0/743: link d7/da/df/lc9 d7/da/df/l104 0 2024-04-25T01:36:00.433 INFO:tasks.workunit.client.1.smithi138.stdout:3/320: write db/df/d17/f5a [5069993,35762] 0 2024-04-25T01:36:00.433 INFO:tasks.workunit.client.1.smithi138.stdout:6/291: dwrite dc/d10/f25 [4194304,4194304] 0 2024-04-25T01:36:00.434 INFO:tasks.workunit.client.0.smithi033.stdout:1/659: dwrite d2/d11/f5e [0,4194304] 0 2024-04-25T01:36:00.437 INFO:tasks.workunit.client.0.smithi033.stdout:5/843: creat d1c/d2a/f10f x:0 0 0 2024-04-25T01:36:00.439 INFO:tasks.workunit.client.0.smithi033.stdout:6/928: dread d4/d24/d63/fbb [0,4194304] 0 2024-04-25T01:36:00.441 INFO:tasks.workunit.client.1.smithi138.stdout:6/292: dread dc/d10/f15 [0,4194304] 0 2024-04-25T01:36:00.442 INFO:tasks.workunit.client.1.smithi138.stdout:3/321: rename db/df/d17/d23/d53 to db/df/d17/d5d 0 2024-04-25T01:36:00.444 INFO:tasks.workunit.client.0.smithi033.stdout:9/806: rename d0/d2/d43/d4b/dc5/de0/d8f to d0/d2/dd/d5f/d11b 0 2024-04-25T01:36:00.446 INFO:tasks.workunit.client.1.smithi138.stdout:6/293: creat dc/d2e/d53/f62 x:0 0 0 2024-04-25T01:36:00.446 INFO:tasks.workunit.client.0.smithi033.stdout:9/807: dread - d0/d2/dd/d5f/d11b/f102 zero size 2024-04-25T01:36:00.447 INFO:tasks.workunit.client.1.smithi138.stdout:6/294: mknod dc/d10/d28/c63 0 2024-04-25T01:36:00.448 INFO:tasks.workunit.client.0.smithi033.stdout:4/866: link d6/d14/f101 d6/d14/d35/d44/d45/d46/d57/d70/d87/d112/f11a 0 2024-04-25T01:36:00.450 INFO:tasks.workunit.client.0.smithi033.stdout:7/805: unlink d4/d10/f3b 0 2024-04-25T01:36:00.453 INFO:tasks.workunit.client.1.smithi138.stdout:8/462: dread d3/f1e [0,4194304] 0 2024-04-25T01:36:00.453 INFO:tasks.workunit.client.1.smithi138.stdout:3/322: dwrite db/df/d17/f2e [0,4194304] 0 2024-04-25T01:36:00.453 INFO:tasks.workunit.client.0.smithi033.stdout:1/660: unlink d2/d7f/lc2 0 2024-04-25T01:36:00.455 INFO:tasks.workunit.client.0.smithi033.stdout:8/766: mkdir d6/d45/d51/d5d/d11d 0 2024-04-25T01:36:00.456 INFO:tasks.workunit.client.1.smithi138.stdout:3/323: write db/df/d19/d1a/f44 [257615,124341] 0 2024-04-25T01:36:00.458 INFO:tasks.workunit.client.1.smithi138.stdout:6/295: dwrite dc/f1f [0,4194304] 0 2024-04-25T01:36:00.461 INFO:tasks.workunit.client.0.smithi033.stdout:5/844: rename d1c/d2a/d50/d52/da6/d4a/fbf to d1c/d2a/d50/d5b/d86/d10b/f110 0 2024-04-25T01:36:00.462 INFO:tasks.workunit.client.0.smithi033.stdout:9/808: mkdir d0/d2/dd/dba/dc1/d11c 0 2024-04-25T01:36:00.465 INFO:tasks.workunit.client.0.smithi033.stdout:4/867: read d6/d14/d35/d44/d45/d46/d57/d70/d87/dc7/fbf [19232,89971] 0 2024-04-25T01:36:00.468 INFO:tasks.workunit.client.0.smithi033.stdout:4/868: read d6/d14/d1d/f63 [1097078,86000] 0 2024-04-25T01:36:00.469 INFO:tasks.workunit.client.1.smithi138.stdout:3/324: dwrite db/d14/f32 [4194304,4194304] 0 2024-04-25T01:36:00.470 INFO:tasks.workunit.client.0.smithi033.stdout:1/661: mknod d2/d5/d1b/d46/d5f/d63/d64/ccc 0 2024-04-25T01:36:00.481 INFO:tasks.workunit.client.0.smithi033.stdout:0/744: write d7/d2d/d64/fad [5054062,117636] 0 2024-04-25T01:36:00.482 INFO:tasks.workunit.client.0.smithi033.stdout:0/745: stat d7/d2d/d4c/da0/da1/dd7 0 2024-04-25T01:36:00.483 INFO:tasks.workunit.client.0.smithi033.stdout:5/845: unlink d1c/d2a/d3d/fe7 0 2024-04-25T01:36:00.486 INFO:tasks.workunit.client.1.smithi138.stdout:6/296: dwrite dc/d2e/d37/f5f [0,4194304] 0 2024-04-25T01:36:00.486 INFO:tasks.workunit.client.0.smithi033.stdout:9/809: write d0/d2/dd/d20/d4e/f90 [2489259,10893] 0 2024-04-25T01:36:00.487 INFO:tasks.workunit.client.1.smithi138.stdout:3/325: dwrite f6 [0,4194304] 0 2024-04-25T01:36:00.487 INFO:tasks.workunit.client.0.smithi033.stdout:7/806: truncate d4/d23/d3f/d45/f5e 658271 0 2024-04-25T01:36:00.488 INFO:tasks.workunit.client.1.smithi138.stdout:8/463: dread d3/d16/d88/da/db/d15/f2f [0,4194304] 0 2024-04-25T01:36:00.488 INFO:tasks.workunit.client.1.smithi138.stdout:3/326: chown f1 2 1 2024-04-25T01:36:00.488 INFO:tasks.workunit.client.1.smithi138.stdout:8/464: readlink d3/l4 0 2024-04-25T01:36:00.493 INFO:tasks.workunit.client.0.smithi033.stdout:8/767: creat d6/db/d111/f11e x:0 0 0 2024-04-25T01:36:00.494 INFO:tasks.workunit.client.1.smithi138.stdout:8/465: unlink d3/d44/f46 0 2024-04-25T01:36:00.495 INFO:tasks.workunit.client.0.smithi033.stdout:0/746: dwrite d7/d2d/f67 [0,4194304] 0 2024-04-25T01:36:00.497 INFO:tasks.workunit.client.0.smithi033.stdout:5/846: dwrite d1c/d2a/d3d/f4d [0,4194304] 0 2024-04-25T01:36:00.500 INFO:tasks.workunit.client.1.smithi138.stdout:6/297: rename dc/d2e/f4a to dc/f64 0 2024-04-25T01:36:00.500 INFO:tasks.workunit.client.1.smithi138.stdout:3/327: write f1 [8201825,126767] 0 2024-04-25T01:36:00.500 INFO:tasks.workunit.client.1.smithi138.stdout:6/298: chown dc/d10/c44 167 1 2024-04-25T01:36:00.501 INFO:tasks.workunit.client.0.smithi033.stdout:0/747: chown d7/da/d80/lac 11486119 1 2024-04-25T01:36:00.503 INFO:tasks.workunit.client.1.smithi138.stdout:8/466: dwrite d3/d16/d1f/d2a/f8b [0,4194304] 0 2024-04-25T01:36:00.507 INFO:tasks.workunit.client.1.smithi138.stdout:6/299: getdents dc/d2e/d37 0 2024-04-25T01:36:00.508 INFO:tasks.workunit.client.0.smithi033.stdout:4/869: dread d6/d14/d1d/f63 [0,4194304] 0 2024-04-25T01:36:00.511 INFO:tasks.workunit.client.0.smithi033.stdout:1/662: rename d2/d5/d75/d59/d83/d98 to d2/d2e/dcd 0 2024-04-25T01:36:00.511 INFO:tasks.workunit.client.1.smithi138.stdout:3/328: rename db/df/d19/d1a/f47 to db/df/d17/f5e 0 2024-04-25T01:36:00.514 INFO:tasks.workunit.client.0.smithi033.stdout:6/929: dwrite d4/d1d/d3e/d41/d5c/d7f/f81 [0,4194304] 0 2024-04-25T01:36:00.514 INFO:tasks.workunit.client.0.smithi033.stdout:8/768: rmdir d6/d45/d51/d5d/d6e 39 2024-04-25T01:36:00.518 INFO:tasks.workunit.client.1.smithi138.stdout:3/329: creat db/df/d17/d23/d34/f5f x:0 0 0 2024-04-25T01:36:00.523 INFO:tasks.workunit.client.0.smithi033.stdout:9/810: rename d0/d2/l87 to d0/d2/d43/d4b/dc5/de0/d42/dea/l11d 0 2024-04-25T01:36:00.524 INFO:tasks.workunit.client.1.smithi138.stdout:3/330: dread db/df/f4f [0,4194304] 0 2024-04-25T01:36:00.526 INFO:tasks.workunit.client.0.smithi033.stdout:5/847: symlink d1c/d2a/d50/d52/d10c/l111 0 2024-04-25T01:36:00.528 INFO:tasks.workunit.client.1.smithi138.stdout:3/331: rename f1 to db/d14/d40/f60 0 2024-04-25T01:36:00.530 INFO:tasks.workunit.client.0.smithi033.stdout:1/663: mknod d2/d11/dc9/cce 0 2024-04-25T01:36:00.530 INFO:tasks.workunit.client.1.smithi138.stdout:3/332: dread - db/df/d17/d23/f54 zero size 2024-04-25T01:36:00.531 INFO:tasks.workunit.client.0.smithi033.stdout:8/769: symlink d6/d45/d51/d5d/l11f 0 2024-04-25T01:36:00.531 INFO:tasks.workunit.client.1.smithi138.stdout:3/333: dread - db/df/d17/d23/f54 zero size 2024-04-25T01:36:00.533 INFO:tasks.workunit.client.0.smithi033.stdout:5/848: creat d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e/f112 x:0 0 0 2024-04-25T01:36:00.535 INFO:tasks.workunit.client.0.smithi033.stdout:1/664: unlink d2/d7f/l8a 0 2024-04-25T01:36:00.536 INFO:tasks.workunit.client.1.smithi138.stdout:3/334: rename db/df/d17/d20/c24 to db/d14/d40/c61 0 2024-04-25T01:36:00.536 INFO:tasks.workunit.client.1.smithi138.stdout:6/300: dwrite dc/f19 [4194304,4194304] 0 2024-04-25T01:36:00.537 INFO:tasks.workunit.client.0.smithi033.stdout:8/770: read d6/d45/d51/d55/daa/fea [248573,14647] 0 2024-04-25T01:36:00.538 INFO:tasks.workunit.client.1.smithi138.stdout:7/222: sync 2024-04-25T01:36:00.538 INFO:tasks.workunit.client.0.smithi033.stdout:9/811: rmdir d0/d2/d43/d73/d10b 0 2024-04-25T01:36:00.538 INFO:tasks.workunit.client.1.smithi138.stdout:6/301: chown dc/d10/f4b 501848154 1 2024-04-25T01:36:00.540 INFO:tasks.workunit.client.0.smithi033.stdout:1/665: dread d2/d5/d1b/d46/d5f/d63/d64/fa5 [0,4194304] 0 2024-04-25T01:36:00.541 INFO:tasks.workunit.client.1.smithi138.stdout:6/302: write dc/d10/d28/f34 [2537437,18903] 0 2024-04-25T01:36:00.541 INFO:tasks.workunit.client.1.smithi138.stdout:7/223: truncate da/de/f2c 1108087 0 2024-04-25T01:36:00.542 INFO:tasks.workunit.client.0.smithi033.stdout:1/666: fdatasync d2/d11/f5e 0 2024-04-25T01:36:00.543 INFO:tasks.workunit.client.0.smithi033.stdout:8/771: rmdir d6/d45/d51 39 2024-04-25T01:36:00.543 INFO:tasks.workunit.client.1.smithi138.stdout:3/335: link db/df/d19/d1a/f49 db/f62 0 2024-04-25T01:36:00.544 INFO:tasks.workunit.client.1.smithi138.stdout:7/224: mknod da/c48 0 2024-04-25T01:36:00.545 INFO:tasks.workunit.client.0.smithi033.stdout:7/807: dread d4/d2e/d6b/d1f/f71 [0,4194304] 0 2024-04-25T01:36:00.545 INFO:tasks.workunit.client.1.smithi138.stdout:6/303: creat dc/d2e/d53/f65 x:0 0 0 2024-04-25T01:36:00.547 INFO:tasks.workunit.client.0.smithi033.stdout:5/849: creat d1c/d2a/d3d/f113 x:0 0 0 2024-04-25T01:36:00.547 INFO:tasks.workunit.client.0.smithi033.stdout:0/748: write d7/d33/f100 [1553694,66460] 0 2024-04-25T01:36:00.550 INFO:tasks.workunit.client.0.smithi033.stdout:1/667: creat d2/d5/d6c/fcf x:0 0 0 2024-04-25T01:36:00.550 INFO:tasks.workunit.client.0.smithi033.stdout:1/668: fsync d2/d5/fd 0 2024-04-25T01:36:00.551 INFO:tasks.workunit.client.0.smithi033.stdout:7/808: fsync f0 0 2024-04-25T01:36:00.552 INFO:tasks.workunit.client.0.smithi033.stdout:9/812: getdents d0/d2/dd/dba/dc1/d11c 0 2024-04-25T01:36:00.553 INFO:tasks.workunit.client.0.smithi033.stdout:9/813: fdatasync d0/fa2 0 2024-04-25T01:36:00.554 INFO:tasks.workunit.client.0.smithi033.stdout:0/749: mknod d7/d2d/d4c/da0/da1/c105 0 2024-04-25T01:36:00.557 INFO:tasks.workunit.client.0.smithi033.stdout:1/669: creat d2/d5/d75/d59/d83/fd0 x:0 0 0 2024-04-25T01:36:00.559 INFO:tasks.workunit.client.0.smithi033.stdout:7/809: unlink d4/d10/f89 0 2024-04-25T01:36:00.559 INFO:tasks.workunit.client.0.smithi033.stdout:5/850: symlink d1c/d2a/d50/d52/da6/d3e/d70/l114 0 2024-04-25T01:36:00.561 INFO:tasks.workunit.client.0.smithi033.stdout:7/810: write d4/f7a [3288717,19682] 0 2024-04-25T01:36:00.564 INFO:tasks.workunit.client.1.smithi138.stdout:6/304: dread dc/d10/d28/f34 [0,4194304] 0 2024-04-25T01:36:00.564 INFO:tasks.workunit.client.0.smithi033.stdout:0/750: dwrite d7/da/d80/fe4 [4194304,4194304] 0 2024-04-25T01:36:00.565 INFO:tasks.workunit.client.1.smithi138.stdout:6/305: write dc/d2e/d37/f5f [3238479,40161] 0 2024-04-25T01:36:00.565 INFO:tasks.workunit.client.0.smithi033.stdout:9/814: symlink d0/d2/dd/df/l11e 0 2024-04-25T01:36:00.566 INFO:tasks.workunit.client.0.smithi033.stdout:0/751: write d7/d2d/d4d/d9c/f83 [2427227,11218] 0 2024-04-25T01:36:00.568 INFO:tasks.workunit.client.0.smithi033.stdout:5/851: creat d1c/d2a/d50/d52/da6/f115 x:0 0 0 2024-04-25T01:36:00.570 INFO:tasks.workunit.client.0.smithi033.stdout:1/670: creat d2/d5/d1b/d46/d5f/d63/d9d/fd1 x:0 0 0 2024-04-25T01:36:00.572 INFO:tasks.workunit.client.0.smithi033.stdout:9/815: fdatasync d0/d2/d43/d4b/dc5/de0/d7e/fe4 0 2024-04-25T01:36:00.576 INFO:tasks.workunit.client.0.smithi033.stdout:0/752: dwrite d7/d2d/d43/dc0/fc3 [0,4194304] 0 2024-04-25T01:36:00.578 INFO:tasks.workunit.client.0.smithi033.stdout:0/753: chown d7/d2d/d4c/da0/da1/c105 3353 1 2024-04-25T01:36:00.578 INFO:tasks.workunit.client.0.smithi033.stdout:5/852: mknod d1c/d2a/d3d/d59/c116 0 2024-04-25T01:36:00.579 INFO:tasks.workunit.client.1.smithi138.stdout:3/336: getdents db/df/d17 0 2024-04-25T01:36:00.579 INFO:tasks.workunit.client.1.smithi138.stdout:3/337: chown db/df/d17/d23/d34 228509555 1 2024-04-25T01:36:00.581 INFO:tasks.workunit.client.1.smithi138.stdout:6/306: link dc/d2e/d53/f65 dc/f66 0 2024-04-25T01:36:00.581 INFO:tasks.workunit.client.1.smithi138.stdout:6/307: readlink dc/d10/l20 0 2024-04-25T01:36:00.582 INFO:tasks.workunit.client.1.smithi138.stdout:3/338: mknod db/df/d19/d1a/c63 0 2024-04-25T01:36:00.582 INFO:tasks.workunit.client.0.smithi033.stdout:5/853: chown d1c/d2a/d50/d52/d104 1624416 1 2024-04-25T01:36:00.582 INFO:tasks.workunit.client.1.smithi138.stdout:3/339: readlink db/d14/l51 0 2024-04-25T01:36:00.585 INFO:tasks.workunit.client.0.smithi033.stdout:9/816: rename d0/d2/d43/d4b/dc5/de0/fec to d0/d2/d43/d73/dab/f11f 0 2024-04-25T01:36:00.585 INFO:tasks.workunit.client.1.smithi138.stdout:6/308: truncate dc/d10/d28/f29 584704 0 2024-04-25T01:36:00.586 INFO:tasks.workunit.client.0.smithi033.stdout:6/930: write d4/d24/f5b [2883825,11943] 0 2024-04-25T01:36:00.587 INFO:tasks.workunit.client.1.smithi138.stdout:6/309: truncate dc/d2e/d53/f65 386548 0 2024-04-25T01:36:00.587 INFO:tasks.workunit.client.0.smithi033.stdout:1/671: mkdir d2/d5/d1b/dd2 0 2024-04-25T01:36:00.589 INFO:tasks.workunit.client.0.smithi033.stdout:4/870: dwrite d6/d14/d35/d44/fea [0,4194304] 0 2024-04-25T01:36:00.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/225: stat da/de/d13/f2d 0 2024-04-25T01:36:00.597 INFO:tasks.workunit.client.0.smithi033.stdout:5/854: creat d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/f117 x:0 0 0 2024-04-25T01:36:00.598 INFO:tasks.workunit.client.0.smithi033.stdout:1/672: readlink d2/d11/d28/l44 0 2024-04-25T01:36:00.600 INFO:tasks.workunit.client.0.smithi033.stdout:0/754: dwrite d7/d2d/d43/d44/fd5 [0,4194304] 0 2024-04-25T01:36:00.601 INFO:tasks.workunit.client.1.smithi138.stdout:6/310: rename dc/d2e/d53 to dc/d10/d67 0 2024-04-25T01:36:00.602 INFO:tasks.workunit.client.1.smithi138.stdout:6/311: fdatasync dc/d2e/f31 0 2024-04-25T01:36:00.603 INFO:tasks.workunit.client.1.smithi138.stdout:6/312: read fb [2464538,33668] 0 2024-04-25T01:36:00.605 INFO:tasks.workunit.client.1.smithi138.stdout:7/226: rename da/de/d13/d21/d2a to da/de/d1b/d49 0 2024-04-25T01:36:00.606 INFO:tasks.workunit.client.1.smithi138.stdout:7/227: write da/de/d13/f3c [495631,128585] 0 2024-04-25T01:36:00.608 INFO:tasks.workunit.client.1.smithi138.stdout:6/313: rename dc/d10/f25 to dc/d10/f68 0 2024-04-25T01:36:00.608 INFO:tasks.workunit.client.1.smithi138.stdout:7/228: chown da/de/d13/d21/l37 6640 1 2024-04-25T01:36:00.611 INFO:tasks.workunit.client.1.smithi138.stdout:3/340: dread db/df/d19/f36 [0,4194304] 0 2024-04-25T01:36:00.612 INFO:tasks.workunit.client.0.smithi033.stdout:5/855: dwrite d1c/d2a/f10f [0,4194304] 0 2024-04-25T01:36:00.612 INFO:tasks.workunit.client.1.smithi138.stdout:7/229: symlink da/de/d13/l4a 0 2024-04-25T01:36:00.613 INFO:tasks.workunit.client.1.smithi138.stdout:3/341: chown db/l3b 368 1 2024-04-25T01:36:00.615 INFO:tasks.workunit.client.0.smithi033.stdout:8/772: truncate d6/d45/d51/d5d/d6e/f85 976734 0 2024-04-25T01:36:00.615 INFO:tasks.workunit.client.0.smithi033.stdout:4/871: dwrite d6/d14/d35/d44/dfe/d103/fe0 [0,4194304] 0 2024-04-25T01:36:00.616 INFO:tasks.workunit.client.0.smithi033.stdout:7/811: dread d4/d10/fdb [0,4194304] 0 2024-04-25T01:36:00.619 INFO:tasks.workunit.client.1.smithi138.stdout:6/314: dwrite dc/d10/f15 [0,4194304] 0 2024-04-25T01:36:00.620 INFO:tasks.workunit.client.1.smithi138.stdout:3/342: write db/df/d19/d1a/f46 [632047,70996] 0 2024-04-25T01:36:00.622 INFO:tasks.workunit.client.1.smithi138.stdout:6/315: chown dc/c39 1035 1 2024-04-25T01:36:00.622 INFO:tasks.workunit.client.1.smithi138.stdout:3/343: stat db/f3f 0 2024-04-25T01:36:00.625 INFO:tasks.workunit.client.1.smithi138.stdout:6/316: truncate dc/f64 1188869 0 2024-04-25T01:36:00.626 INFO:tasks.workunit.client.1.smithi138.stdout:7/230: dwrite da/de/f2c [0,4194304] 0 2024-04-25T01:36:00.634 INFO:tasks.workunit.client.0.smithi033.stdout:0/755: creat d7/d2d/d4c/da0/da1/f106 x:0 0 0 2024-04-25T01:36:00.636 INFO:tasks.workunit.client.1.smithi138.stdout:6/317: dread fb [0,4194304] 0 2024-04-25T01:36:00.636 INFO:tasks.workunit.client.0.smithi033.stdout:6/931: rename d4/d1d/d3e/d41/d5c/fb7 to d4/d1d/d3e/d41/d93/f137 0 2024-04-25T01:36:00.637 INFO:tasks.workunit.client.1.smithi138.stdout:7/231: dwrite da/de/d13/f24 [0,4194304] 0 2024-04-25T01:36:00.640 INFO:tasks.workunit.client.1.smithi138.stdout:3/344: dread db/f3f [0,4194304] 0 2024-04-25T01:36:00.640 INFO:tasks.workunit.client.1.smithi138.stdout:7/232: mkdir da/de/d1b/d4b 0 2024-04-25T01:36:00.641 INFO:tasks.workunit.client.0.smithi033.stdout:7/812: creat d4/d2e/d7f/d98/dbf/ffb x:0 0 0 2024-04-25T01:36:00.641 INFO:tasks.workunit.client.1.smithi138.stdout:7/233: chown da/de/d13/f2d 319 1 2024-04-25T01:36:00.642 INFO:tasks.workunit.client.0.smithi033.stdout:9/817: mknod d0/d2/d43/d4b/dc5/c120 0 2024-04-25T01:36:00.643 INFO:tasks.workunit.client.1.smithi138.stdout:6/318: dread dc/d10/f68 [4194304,4194304] 0 2024-04-25T01:36:00.643 INFO:tasks.workunit.client.1.smithi138.stdout:3/345: dread - db/df/d17/d23/d34/f52 zero size 2024-04-25T01:36:00.643 INFO:tasks.workunit.client.1.smithi138.stdout:6/319: fdatasync dc/f18 0 2024-04-25T01:36:00.644 INFO:tasks.workunit.client.0.smithi033.stdout:0/756: creat d7/d2d/d4c/f107 x:0 0 0 2024-04-25T01:36:00.644 INFO:tasks.workunit.client.0.smithi033.stdout:9/818: truncate d0/d2/f32 353378 0 2024-04-25T01:36:00.645 INFO:tasks.workunit.client.1.smithi138.stdout:6/320: write dc/d2e/f3a [415558,86183] 0 2024-04-25T01:36:00.645 INFO:tasks.workunit.client.1.smithi138.stdout:7/234: creat da/de/d13/d3e/f4c x:0 0 0 2024-04-25T01:36:00.645 INFO:tasks.workunit.client.0.smithi033.stdout:0/757: stat d7/d2d/d4c/da0/da1/f106 0 2024-04-25T01:36:00.647 INFO:tasks.workunit.client.0.smithi033.stdout:8/773: symlink d6/db/d80/db4/l120 0 2024-04-25T01:36:00.648 INFO:tasks.workunit.client.0.smithi033.stdout:9/819: read d0/d2/d43/d4b/f9d [1040267,2377] 0 2024-04-25T01:36:00.649 INFO:tasks.workunit.client.0.smithi033.stdout:1/673: creat d2/d11/fd3 x:0 0 0 2024-04-25T01:36:00.649 INFO:tasks.workunit.client.1.smithi138.stdout:3/346: getdents db/d14/d40 0 2024-04-25T01:36:00.650 INFO:tasks.workunit.client.1.smithi138.stdout:7/235: symlink da/de/d1b/d4b/l4d 0 2024-04-25T01:36:00.651 INFO:tasks.workunit.client.0.smithi033.stdout:6/932: unlink d4/d24/d36/d42/d97/de0/c136 0 2024-04-25T01:36:00.654 INFO:tasks.workunit.client.0.smithi033.stdout:5/856: creat d1c/d2a/d50/d52/da6/d3e/d70/d7a/f118 x:0 0 0 2024-04-25T01:36:00.656 INFO:tasks.workunit.client.0.smithi033.stdout:7/813: mkdir d4/d2e/d6b/d1f/d78/d86/de4/dfc 0 2024-04-25T01:36:00.658 INFO:tasks.workunit.client.1.smithi138.stdout:3/347: dwrite db/d14/d40/f60 [0,4194304] 0 2024-04-25T01:36:00.658 INFO:tasks.workunit.client.1.smithi138.stdout:3/348: fsync db/df/d17/f2e 0 2024-04-25T01:36:00.658 INFO:tasks.workunit.client.0.smithi033.stdout:8/774: dwrite d6/db/d80/d89/f91 [0,4194304] 0 2024-04-25T01:36:00.658 INFO:tasks.workunit.client.0.smithi033.stdout:9/820: read - d0/d2/dd/f51 zero size 2024-04-25T01:36:00.664 INFO:tasks.workunit.client.1.smithi138.stdout:6/321: truncate f5 2710456 0 2024-04-25T01:36:00.665 INFO:tasks.workunit.client.1.smithi138.stdout:3/349: write db/df/d17/d23/f30 [2206199,129321] 0 2024-04-25T01:36:00.666 INFO:tasks.workunit.client.0.smithi033.stdout:6/933: creat d4/d1d/d3e/d41/d54/da1/f138 x:0 0 0 2024-04-25T01:36:00.666 INFO:tasks.workunit.client.0.smithi033.stdout:7/814: mkdir d4/d2e/d7f/dfd 0 2024-04-25T01:36:00.667 INFO:tasks.workunit.client.1.smithi138.stdout:6/322: mknod dc/d54/c69 0 2024-04-25T01:36:00.668 INFO:tasks.workunit.client.0.smithi033.stdout:6/934: dread - d4/df1/f134 zero size 2024-04-25T01:36:00.669 INFO:tasks.workunit.client.0.smithi033.stdout:7/815: stat d4/d2e/d65/df5 0 2024-04-25T01:36:00.670 INFO:tasks.workunit.client.1.smithi138.stdout:3/350: write db/df/d19/d1a/f21 [2808392,122698] 0 2024-04-25T01:36:00.670 INFO:tasks.workunit.client.0.smithi033.stdout:1/674: dwrite d2/d2e/d34/f6f [4194304,4194304] 0 2024-04-25T01:36:00.671 INFO:tasks.workunit.client.1.smithi138.stdout:6/323: symlink dc/d2e/d55/l6a 0 2024-04-25T01:36:00.673 INFO:tasks.workunit.client.1.smithi138.stdout:7/236: dwrite da/de/d1b/f31 [4194304,4194304] 0 2024-04-25T01:36:00.673 INFO:tasks.workunit.client.1.smithi138.stdout:3/351: symlink db/df/d19/d1a/l64 0 2024-04-25T01:36:00.676 INFO:tasks.workunit.client.1.smithi138.stdout:6/324: creat dc/d10/f6b x:0 0 0 2024-04-25T01:36:00.676 INFO:tasks.workunit.client.0.smithi033.stdout:8/775: truncate d6/d45/d51/d55/daa/fd7 1676426 0 2024-04-25T01:36:00.677 INFO:tasks.workunit.client.1.smithi138.stdout:6/325: chown dc/d10/l57 260 1 2024-04-25T01:36:00.677 INFO:tasks.workunit.client.1.smithi138.stdout:3/352: mkdir db/d14/d65 0 2024-04-25T01:36:00.678 INFO:tasks.workunit.client.0.smithi033.stdout:7/816: creat d4/d23/da7/ffe x:0 0 0 2024-04-25T01:36:00.678 INFO:tasks.workunit.client.1.smithi138.stdout:6/326: chown dc/d10/d67 84214009 1 2024-04-25T01:36:00.679 INFO:tasks.workunit.client.0.smithi033.stdout:7/817: chown d4/d2e/d6b/f62 10970110 1 2024-04-25T01:36:00.681 INFO:tasks.workunit.client.1.smithi138.stdout:6/327: chown f6 435054 1 2024-04-25T01:36:00.682 INFO:tasks.workunit.client.0.smithi033.stdout:8/776: dread - d6/d45/d51/d55/f5c zero size 2024-04-25T01:36:00.684 INFO:tasks.workunit.client.0.smithi033.stdout:6/935: link d4/d1d/d3e/d41/d93/fab d4/d24/d36/d42/d97/de0/f139 0 2024-04-25T01:36:00.684 INFO:tasks.workunit.client.1.smithi138.stdout:3/353: rename db/lc to db/df/l66 0 2024-04-25T01:36:00.686 INFO:tasks.workunit.client.0.smithi033.stdout:8/777: dread d6/d49/f5a [0,4194304] 0 2024-04-25T01:36:00.688 INFO:tasks.workunit.client.0.smithi033.stdout:7/818: mkdir d4/d2e/d65/dff 0 2024-04-25T01:36:00.688 INFO:tasks.workunit.client.1.smithi138.stdout:3/354: creat db/df/d17/d23/f67 x:0 0 0 2024-04-25T01:36:00.689 INFO:tasks.workunit.client.0.smithi033.stdout:7/819: stat d4/d23/d3f/f79 0 2024-04-25T01:36:00.691 INFO:tasks.workunit.client.1.smithi138.stdout:6/328: dread dc/f16 [0,4194304] 0 2024-04-25T01:36:00.691 INFO:tasks.workunit.client.0.smithi033.stdout:8/778: unlink d6/d45/d52/d2e/l71 0 2024-04-25T01:36:00.694 INFO:tasks.workunit.client.0.smithi033.stdout:7/820: creat d4/d57/db7/f100 x:0 0 0 2024-04-25T01:36:00.696 INFO:tasks.workunit.client.1.smithi138.stdout:6/329: creat dc/d2e/f6c x:0 0 0 2024-04-25T01:36:00.696 INFO:tasks.workunit.client.1.smithi138.stdout:3/355: dwrite db/df/d17/d23/d34/f52 [0,4194304] 0 2024-04-25T01:36:00.698 INFO:tasks.workunit.client.0.smithi033.stdout:1/675: getdents d2/d2e/dcd 0 2024-04-25T01:36:00.699 INFO:tasks.workunit.client.1.smithi138.stdout:7/237: dread f9 [0,4194304] 0 2024-04-25T01:36:00.702 INFO:tasks.workunit.client.1.smithi138.stdout:3/356: truncate db/df/d17/d20/f29 477783 0 2024-04-25T01:36:00.703 INFO:tasks.workunit.client.0.smithi033.stdout:1/676: read d2/d5/d1b/d46/f47 [3249985,41059] 0 2024-04-25T01:36:00.704 INFO:tasks.workunit.client.1.smithi138.stdout:3/357: write db/df/d17/d23/f30 [718284,3542] 0 2024-04-25T01:36:00.705 INFO:tasks.workunit.client.1.smithi138.stdout:4/159: sync 2024-04-25T01:36:00.705 INFO:tasks.workunit.client.1.smithi138.stdout:9/238: sync 2024-04-25T01:36:00.705 INFO:tasks.workunit.client.1.smithi138.stdout:1/228: sync 2024-04-25T01:36:00.706 INFO:tasks.workunit.client.0.smithi033.stdout:1/677: mkdir d2/d11/d28/dd4 0 2024-04-25T01:36:00.707 INFO:tasks.workunit.client.1.smithi138.stdout:7/238: getdents da/de/d13 0 2024-04-25T01:36:00.707 INFO:tasks.workunit.client.1.smithi138.stdout:1/229: stat c0 0 2024-04-25T01:36:00.708 INFO:tasks.workunit.client.1.smithi138.stdout:4/160: write d0/f13 [1294293,84593] 0 2024-04-25T01:36:00.708 INFO:tasks.workunit.client.0.smithi033.stdout:9/821: rmdir d0 39 2024-04-25T01:36:00.708 INFO:tasks.workunit.client.1.smithi138.stdout:3/358: creat db/df/d17/d23/d34/f68 x:0 0 0 2024-04-25T01:36:00.710 INFO:tasks.workunit.client.1.smithi138.stdout:4/161: stat d0/f12 0 2024-04-25T01:36:00.711 INFO:tasks.workunit.client.0.smithi033.stdout:7/821: dread d4/d2e/d6d/dbb/fc4 [0,4194304] 0 2024-04-25T01:36:00.712 INFO:tasks.workunit.client.1.smithi138.stdout:7/239: mknod da/de/d13/d30/c4e 0 2024-04-25T01:36:00.713 INFO:tasks.workunit.client.0.smithi033.stdout:7/822: chown d4/d2e/d6b/da9/dee 1886055 1 2024-04-25T01:36:00.713 INFO:tasks.workunit.client.1.smithi138.stdout:1/230: unlink d3/d27/d16/l21 0 2024-04-25T01:36:00.715 INFO:tasks.workunit.client.0.smithi033.stdout:2/790: sync 2024-04-25T01:36:00.715 INFO:tasks.workunit.client.0.smithi033.stdout:3/850: sync 2024-04-25T01:36:00.718 INFO:tasks.workunit.client.1.smithi138.stdout:7/240: rename da/de/d1b/f44 to da/de/d1b/f4f 0 2024-04-25T01:36:00.718 INFO:tasks.workunit.client.1.smithi138.stdout:7/241: chown da/de/d1b/d49 81225485 1 2024-04-25T01:36:00.718 INFO:tasks.workunit.client.1.smithi138.stdout:7/242: dread - da/de/d1b/d34/f3d zero size 2024-04-25T01:36:00.719 INFO:tasks.workunit.client.1.smithi138.stdout:7/243: write da/de/f29 [370342,43870] 0 2024-04-25T01:36:00.719 INFO:tasks.workunit.client.1.smithi138.stdout:3/359: creat db/f69 x:0 0 0 2024-04-25T01:36:00.719 INFO:tasks.workunit.client.0.smithi033.stdout:1/678: dread d2/f81 [0,4194304] 0 2024-04-25T01:36:00.719 INFO:tasks.workunit.client.0.smithi033.stdout:6/936: dread d4/d24/d36/d42/d9d/ff4 [0,4194304] 0 2024-04-25T01:36:00.719 INFO:tasks.workunit.client.0.smithi033.stdout:7/823: rmdir d4/d23/d3f/d45/d90 39 2024-04-25T01:36:00.721 INFO:tasks.workunit.client.1.smithi138.stdout:7/244: dread - da/de/d1b/f33 zero size 2024-04-25T01:36:00.721 INFO:tasks.workunit.client.0.smithi033.stdout:2/791: mkdir da/d8d/db0/d100 0 2024-04-25T01:36:00.721 INFO:tasks.workunit.client.0.smithi033.stdout:6/937: stat d4/d1d/d3e/d70/f86 0 2024-04-25T01:36:00.722 INFO:tasks.workunit.client.1.smithi138.stdout:7/245: chown da/de/d13/d30 541707 1 2024-04-25T01:36:00.723 INFO:tasks.workunit.client.1.smithi138.stdout:4/162: truncate d0/d16/f18 1316269 0 2024-04-25T01:36:00.726 INFO:tasks.workunit.client.1.smithi138.stdout:7/246: mkdir da/de/d1b/d34/d50 0 2024-04-25T01:36:00.731 INFO:tasks.workunit.client.1.smithi138.stdout:6/330: truncate f8 430006 0 2024-04-25T01:36:00.734 INFO:tasks.workunit.client.1.smithi138.stdout:6/331: stat dc/d10/d67/f65 0 2024-04-25T01:36:00.735 INFO:tasks.workunit.client.0.smithi033.stdout:9/822: dwrite d0/d2/d43/d73/dc4/fd5 [0,4194304] 0 2024-04-25T01:36:00.736 INFO:tasks.workunit.client.1.smithi138.stdout:7/247: dwrite da/de/d13/f24 [0,4194304] 0 2024-04-25T01:36:00.739 INFO:tasks.workunit.client.1.smithi138.stdout:7/248: mkdir da/de/d13/d3e/d51 0 2024-04-25T01:36:00.742 INFO:tasks.workunit.client.1.smithi138.stdout:1/231: write d3/d27/d16/f18 [670651,128375] 0 2024-04-25T01:36:00.743 INFO:tasks.workunit.client.1.smithi138.stdout:1/232: chown d3/db/fe 40742911 1 2024-04-25T01:36:00.744 INFO:tasks.workunit.client.1.smithi138.stdout:7/249: creat da/de/d13/f52 x:0 0 0 2024-04-25T01:36:00.744 INFO:tasks.workunit.client.1.smithi138.stdout:1/233: readlink d3/d27/d2d/l41 0 2024-04-25T01:36:00.746 INFO:tasks.workunit.client.1.smithi138.stdout:7/250: unlink da/de/d13/d30/c35 0 2024-04-25T01:36:00.749 INFO:tasks.workunit.client.1.smithi138.stdout:7/251: mknod da/de/d1b/d34/c53 0 2024-04-25T01:36:00.750 INFO:tasks.workunit.client.1.smithi138.stdout:7/252: chown da/de/d13/d3e/d51 16564409 1 2024-04-25T01:36:00.751 INFO:tasks.workunit.client.1.smithi138.stdout:7/253: symlink da/de/d1b/l54 0 2024-04-25T01:36:00.753 INFO:tasks.workunit.client.1.smithi138.stdout:7/254: creat da/de/d1b/d4b/f55 x:0 0 0 2024-04-25T01:36:00.753 INFO:tasks.workunit.client.1.smithi138.stdout:7/255: readlink l0 0 2024-04-25T01:36:00.781 INFO:tasks.workunit.client.0.smithi033.stdout:3/851: rmdir d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1 39 2024-04-25T01:36:00.783 INFO:tasks.workunit.client.1.smithi138.stdout:3/360: dread db/df/d17/f1c [0,4194304] 0 2024-04-25T01:36:00.790 INFO:tasks.workunit.client.1.smithi138.stdout:1/234: dread d3/db/fe [0,4194304] 0 2024-04-25T01:36:00.791 INFO:tasks.workunit.client.1.smithi138.stdout:3/361: mkdir db/df/d17/d20/d57/d6a 0 2024-04-25T01:36:00.791 INFO:tasks.workunit.client.1.smithi138.stdout:6/332: dwrite dc/d54/f5e [0,4194304] 0 2024-04-25T01:36:00.792 INFO:tasks.workunit.client.1.smithi138.stdout:1/235: symlink d3/d27/d24/l4d 0 2024-04-25T01:36:00.794 INFO:tasks.workunit.client.1.smithi138.stdout:6/333: rename dc/d54 to dc/d10/d6d 0 2024-04-25T01:36:00.795 INFO:tasks.workunit.client.1.smithi138.stdout:6/334: chown dc/d2e/f4d 0 1 2024-04-25T01:36:00.806 INFO:tasks.workunit.client.1.smithi138.stdout:3/362: read db/df/d17/d20/f29 [142772,117471] 0 2024-04-25T01:36:00.807 INFO:tasks.workunit.client.1.smithi138.stdout:6/335: fsync dc/d10/d67/f65 0 2024-04-25T01:36:00.808 INFO:tasks.workunit.client.1.smithi138.stdout:1/236: dread d3/f13 [0,4194304] 0 2024-04-25T01:36:00.809 INFO:tasks.workunit.client.1.smithi138.stdout:6/336: write dc/d10/d28/f29 [855314,88407] 0 2024-04-25T01:36:00.809 INFO:tasks.workunit.client.1.smithi138.stdout:3/363: creat db/df/d17/d5d/f6b x:0 0 0 2024-04-25T01:36:00.811 INFO:tasks.workunit.client.1.smithi138.stdout:1/237: symlink d3/d27/l4e 0 2024-04-25T01:36:00.813 INFO:tasks.workunit.client.1.smithi138.stdout:3/364: rename db/d14/d40/c61 to db/df/d17/d20/d57/c6c 0 2024-04-25T01:36:00.815 INFO:tasks.workunit.client.1.smithi138.stdout:3/365: rmdir db/df/d19/d1a 39 2024-04-25T01:36:00.815 INFO:tasks.workunit.client.1.smithi138.stdout:3/366: write db/f15 [4783279,50999] 0 2024-04-25T01:36:00.816 INFO:tasks.workunit.client.1.smithi138.stdout:3/367: chown db/f5b 185612 1 2024-04-25T01:36:00.816 INFO:tasks.workunit.client.1.smithi138.stdout:3/368: readlink db/df/d19/d1a/l22 0 2024-04-25T01:36:00.819 INFO:tasks.workunit.client.1.smithi138.stdout:3/369: creat db/df/d17/d23/d34/f6d x:0 0 0 2024-04-25T01:36:00.828 INFO:tasks.workunit.client.1.smithi138.stdout:6/337: dwrite dc/d10/d28/f34 [4194304,4194304] 0 2024-04-25T01:36:00.828 INFO:tasks.workunit.client.1.smithi138.stdout:3/370: dwrite db/d14/d40/f60 [8388608,4194304] 0 2024-04-25T01:36:00.834 INFO:tasks.workunit.client.1.smithi138.stdout:1/238: dread d3/d4/f9 [0,4194304] 0 2024-04-25T01:36:00.836 INFO:tasks.workunit.client.1.smithi138.stdout:6/338: dwrite dc/d10/f6b [0,4194304] 0 2024-04-25T01:36:00.839 INFO:tasks.workunit.client.1.smithi138.stdout:1/239: dread d3/d4/f9 [0,4194304] 0 2024-04-25T01:36:00.840 INFO:tasks.workunit.client.1.smithi138.stdout:3/371: read db/df/d17/f5e [1171536,91546] 0 2024-04-25T01:36:00.845 INFO:tasks.workunit.client.1.smithi138.stdout:1/240: rename d3/db/c3f to d3/d4/d47/c4f 0 2024-04-25T01:36:00.846 INFO:tasks.workunit.client.1.smithi138.stdout:6/339: dread dc/d10/f26 [0,4194304] 0 2024-04-25T01:36:00.847 INFO:tasks.workunit.client.1.smithi138.stdout:3/372: creat db/df/d17/d20/d57/d6a/f6e x:0 0 0 2024-04-25T01:36:00.848 INFO:tasks.workunit.client.1.smithi138.stdout:3/373: fdatasync db/df/f25 0 2024-04-25T01:36:00.850 INFO:tasks.workunit.client.1.smithi138.stdout:6/340: write dc/d2e/f49 [4169854,124194] 0 2024-04-25T01:36:00.851 INFO:tasks.workunit.client.1.smithi138.stdout:6/341: write dc/d2e/f4c [8885,54871] 0 2024-04-25T01:36:00.853 INFO:tasks.workunit.client.1.smithi138.stdout:3/374: dread - db/f35 zero size 2024-04-25T01:36:00.854 INFO:tasks.workunit.client.1.smithi138.stdout:3/375: dread - db/df/d17/f4a zero size 2024-04-25T01:36:00.854 INFO:tasks.workunit.client.1.smithi138.stdout:6/342: creat dc/d10/d67/f6e x:0 0 0 2024-04-25T01:36:00.857 INFO:tasks.workunit.client.1.smithi138.stdout:6/343: symlink dc/d2e/d55/l6f 0 2024-04-25T01:36:00.861 INFO:tasks.workunit.client.1.smithi138.stdout:3/376: dwrite db/df/d17/d23/d34/f6d [0,4194304] 0 2024-04-25T01:36:00.867 INFO:tasks.workunit.client.1.smithi138.stdout:3/377: read db/df/d19/d1a/f44 [256956,128043] 0 2024-04-25T01:36:00.871 INFO:tasks.workunit.client.1.smithi138.stdout:3/378: mknod db/d14/d40/d41/c6f 0 2024-04-25T01:36:00.871 INFO:tasks.workunit.client.1.smithi138.stdout:3/379: read f6 [2346340,10602] 0 2024-04-25T01:36:00.875 INFO:tasks.workunit.client.1.smithi138.stdout:1/241: dread d3/d4/fd [0,4194304] 0 2024-04-25T01:36:00.877 INFO:tasks.workunit.client.1.smithi138.stdout:6/344: write dc/d2e/f2f [927297,9199] 0 2024-04-25T01:36:00.878 INFO:tasks.workunit.client.1.smithi138.stdout:1/242: mknod d3/db/d2b/c50 0 2024-04-25T01:36:00.880 INFO:tasks.workunit.client.1.smithi138.stdout:3/380: creat db/df/d19/f70 x:0 0 0 2024-04-25T01:36:00.884 INFO:tasks.workunit.client.1.smithi138.stdout:1/243: unlink d3/d4/f26 0 2024-04-25T01:36:00.885 INFO:tasks.workunit.client.1.smithi138.stdout:3/381: rmdir db/df/d17/d20/d57 39 2024-04-25T01:36:00.887 INFO:tasks.workunit.client.1.smithi138.stdout:6/345: dwrite dc/d10/d28/f34 [4194304,4194304] 0 2024-04-25T01:36:00.887 INFO:tasks.workunit.client.1.smithi138.stdout:3/382: stat db/df/d17/d20 0 2024-04-25T01:36:00.888 INFO:tasks.workunit.client.1.smithi138.stdout:1/244: write d3/d4/fd [4796454,123606] 0 2024-04-25T01:36:00.890 INFO:tasks.workunit.client.1.smithi138.stdout:6/346: write dc/d10/d28/f48 [449553,128809] 0 2024-04-25T01:36:00.896 INFO:tasks.workunit.client.1.smithi138.stdout:6/347: rename dc/d10/d67/d5c to dc/d10/d28/d70 0 2024-04-25T01:36:00.897 INFO:tasks.workunit.client.1.smithi138.stdout:1/245: dwrite d3/d27/d24/f3b [0,4194304] 0 2024-04-25T01:36:00.898 INFO:tasks.workunit.client.1.smithi138.stdout:1/246: mkdir d3/d27/d3e/d51 0 2024-04-25T01:36:00.909 INFO:tasks.workunit.client.1.smithi138.stdout:1/247: write d3/d4/f9 [1251353,23433] 0 2024-04-25T01:36:00.909 INFO:tasks.workunit.client.1.smithi138.stdout:5/233: sync 2024-04-25T01:36:00.909 INFO:tasks.workunit.client.1.smithi138.stdout:3/383: dread db/df/d17/d23/f30 [0,4194304] 0 2024-04-25T01:36:00.909 INFO:tasks.workunit.client.1.smithi138.stdout:5/234: creat d4/da/f47 x:0 0 0 2024-04-25T01:36:00.911 INFO:tasks.workunit.client.1.smithi138.stdout:3/384: truncate db/f12 1557959 0 2024-04-25T01:36:00.912 INFO:tasks.workunit.client.1.smithi138.stdout:1/248: dwrite d3/d4/f37 [0,4194304] 0 2024-04-25T01:36:00.912 INFO:tasks.workunit.client.1.smithi138.stdout:5/235: dread d4/da/d27/d1c/f28 [0,4194304] 0 2024-04-25T01:36:00.912 INFO:tasks.workunit.client.1.smithi138.stdout:6/348: truncate dc/d10/d6d/f5e 3350896 0 2024-04-25T01:36:00.913 INFO:tasks.workunit.client.1.smithi138.stdout:1/249: write d3/d27/d2d/f30 [72713,14447] 0 2024-04-25T01:36:00.915 INFO:tasks.workunit.client.1.smithi138.stdout:3/385: chown db/df/d17/d23/c4b 974 1 2024-04-25T01:36:00.916 INFO:tasks.workunit.client.1.smithi138.stdout:5/236: mknod d4/da/d27/c48 0 2024-04-25T01:36:00.916 INFO:tasks.workunit.client.1.smithi138.stdout:6/349: fdatasync dc/d2e/d37/f42 0 2024-04-25T01:36:00.916 INFO:tasks.workunit.client.1.smithi138.stdout:3/386: truncate db/f5b 124551 0 2024-04-25T01:36:00.917 INFO:tasks.workunit.client.1.smithi138.stdout:1/250: unlink d3/f15 0 2024-04-25T01:36:00.922 INFO:tasks.workunit.client.1.smithi138.stdout:5/237: mkdir d4/da/d27/d1c/d3c/d40/d49 0 2024-04-25T01:36:00.923 INFO:tasks.workunit.client.1.smithi138.stdout:1/251: creat d3/db/d2b/f52 x:0 0 0 2024-04-25T01:36:00.924 INFO:tasks.workunit.client.1.smithi138.stdout:3/387: dread db/df/d19/f36 [0,4194304] 0 2024-04-25T01:36:00.926 INFO:tasks.workunit.client.1.smithi138.stdout:1/252: mkdir d3/d4/d53 0 2024-04-25T01:36:00.928 INFO:tasks.workunit.client.0.smithi033.stdout:2/792: rmdir da/d5b/d72/dd1/ded 39 2024-04-25T01:36:00.929 INFO:tasks.workunit.client.1.smithi138.stdout:3/388: dread db/f62 [0,4194304] 0 2024-04-25T01:36:00.930 INFO:tasks.workunit.client.1.smithi138.stdout:3/389: symlink db/df/d17/d5d/l71 0 2024-04-25T01:36:00.931 INFO:tasks.workunit.client.1.smithi138.stdout:1/253: link d3/d27/d24/f3b d3/d4/f54 0 2024-04-25T01:36:00.931 INFO:tasks.workunit.client.1.smithi138.stdout:6/350: write dc/d10/f4b [45878,42368] 0 2024-04-25T01:36:00.937 INFO:tasks.workunit.client.1.smithi138.stdout:3/390: symlink db/d14/d40/d41/l72 0 2024-04-25T01:36:00.937 INFO:tasks.workunit.client.0.smithi033.stdout:8/779: dwrite d6/f1d [0,4194304] 0 2024-04-25T01:36:00.939 INFO:tasks.workunit.client.1.smithi138.stdout:1/254: mknod d3/d27/d2d/c55 0 2024-04-25T01:36:00.939 INFO:tasks.workunit.client.1.smithi138.stdout:1/255: chown d3/d4/f54 82 1 2024-04-25T01:36:00.939 INFO:tasks.workunit.client.1.smithi138.stdout:5/238: dread d4/da/fd [0,4194304] 0 2024-04-25T01:36:00.940 INFO:tasks.workunit.client.1.smithi138.stdout:1/256: write d3/d4/fd [4035902,31242] 0 2024-04-25T01:36:00.940 INFO:tasks.workunit.client.1.smithi138.stdout:3/391: unlink db/d14/d40/c59 0 2024-04-25T01:36:00.940 INFO:tasks.workunit.client.1.smithi138.stdout:6/351: dread dc/d10/f6b [0,4194304] 0 2024-04-25T01:36:00.940 INFO:tasks.workunit.client.1.smithi138.stdout:1/257: read d3/db/f17 [2286782,73435] 0 2024-04-25T01:36:00.941 INFO:tasks.workunit.client.0.smithi033.stdout:6/938: mknod d4/d24/d36/dbd/d117/c13a 0 2024-04-25T01:36:00.946 INFO:tasks.workunit.client.0.smithi033.stdout:1/679: getdents d2/d5/d6c 0 2024-04-25T01:36:00.946 INFO:tasks.workunit.client.1.smithi138.stdout:3/392: unlink db/df/d19/d1a/f46 0 2024-04-25T01:36:00.947 INFO:tasks.workunit.client.1.smithi138.stdout:5/239: unlink d4/f3b 0 2024-04-25T01:36:00.948 INFO:tasks.workunit.client.0.smithi033.stdout:8/780: symlink d6/d45/d51/d5d/d9d/l121 0 2024-04-25T01:36:00.949 INFO:tasks.workunit.client.0.smithi033.stdout:6/939: dwrite d4/d24/d36/d42/d9d/fdd [0,4194304] 0 2024-04-25T01:36:00.950 INFO:tasks.workunit.client.0.smithi033.stdout:8/781: fsync d6/db/d111/f11e 0 2024-04-25T01:36:00.952 INFO:tasks.workunit.client.0.smithi033.stdout:6/940: write d4/d24/d36/d42/d97/db2/f10a [3149292,111260] 0 2024-04-25T01:36:00.953 INFO:tasks.workunit.client.1.smithi138.stdout:3/393: dwrite db/fe [4194304,4194304] 0 2024-04-25T01:36:00.953 INFO:tasks.workunit.client.1.smithi138.stdout:5/240: unlink d4/da/d27/d1c/d3c/d40/f42 0 2024-04-25T01:36:00.953 INFO:tasks.workunit.client.0.smithi033.stdout:1/680: write d2/f62 [1793114,57737] 0 2024-04-25T01:36:00.954 INFO:tasks.workunit.client.1.smithi138.stdout:1/258: getdents d3/d4/d47 0 2024-04-25T01:36:00.955 INFO:tasks.workunit.client.1.smithi138.stdout:3/394: mknod db/df/d17/c73 0 2024-04-25T01:36:00.956 INFO:tasks.workunit.client.0.smithi033.stdout:6/941: readlink d4/l56 0 2024-04-25T01:36:00.959 INFO:tasks.workunit.client.1.smithi138.stdout:1/259: unlink d3/f10 0 2024-04-25T01:36:00.960 INFO:tasks.workunit.client.0.smithi033.stdout:8/782: stat d6/d45/d51/d55/daa/cc0 0 2024-04-25T01:36:00.960 INFO:tasks.workunit.client.1.smithi138.stdout:1/260: readlink d3/d27/d2d/l43 0 2024-04-25T01:36:00.960 INFO:tasks.workunit.client.1.smithi138.stdout:1/261: chown d3/d4/f9 286230 1 2024-04-25T01:36:00.966 INFO:tasks.workunit.client.0.smithi033.stdout:6/942: mknod d4/df1/c13b 0 2024-04-25T01:36:00.967 INFO:tasks.workunit.client.1.smithi138.stdout:1/262: dwrite d3/d45/f48 [0,4194304] 0 2024-04-25T01:36:00.968 INFO:tasks.workunit.client.1.smithi138.stdout:1/263: chown d3/d27/d2d/l31 323 1 2024-04-25T01:36:00.968 INFO:tasks.workunit.client.1.smithi138.stdout:1/264: chown d3/d4/f9 0 1 2024-04-25T01:36:00.969 INFO:tasks.workunit.client.0.smithi033.stdout:2/793: dread da/d27/d31/f5a [0,4194304] 0 2024-04-25T01:36:00.972 INFO:tasks.workunit.client.0.smithi033.stdout:6/943: creat d4/d1d/d3e/d41/d5c/d7f/d116/f13c x:0 0 0 2024-04-25T01:36:00.973 INFO:tasks.workunit.client.0.smithi033.stdout:4/872: sync 2024-04-25T01:36:00.973 INFO:tasks.workunit.client.0.smithi033.stdout:8/783: creat d6/d45/d51/d5d/d11d/f122 x:0 0 0 2024-04-25T01:36:00.974 INFO:tasks.workunit.client.0.smithi033.stdout:8/784: stat d6/d45/d51/d5d/d9d 0 2024-04-25T01:36:00.974 INFO:tasks.workunit.client.1.smithi138.stdout:1/265: dwrite d3/f2c [0,4194304] 0 2024-04-25T01:36:00.975 INFO:tasks.workunit.client.1.smithi138.stdout:1/266: stat d3/d27/d2d 0 2024-04-25T01:36:00.975 INFO:tasks.workunit.client.0.smithi033.stdout:8/785: write d6/f1d [4060179,12307] 0 2024-04-25T01:36:00.977 INFO:tasks.workunit.client.1.smithi138.stdout:1/267: creat d3/d45/f56 x:0 0 0 2024-04-25T01:36:00.981 INFO:tasks.workunit.client.1.smithi138.stdout:1/268: mkdir d3/db/d2b/d57 0 2024-04-25T01:36:00.984 INFO:tasks.workunit.client.1.smithi138.stdout:6/352: truncate dc/f1f 2388931 0 2024-04-25T01:36:00.985 INFO:tasks.workunit.client.0.smithi033.stdout:8/786: creat d6/d45/d52/df2/dfa/f123 x:0 0 0 2024-04-25T01:36:00.986 INFO:tasks.workunit.client.0.smithi033.stdout:8/787: read - d6/d45/d52/db0/dbc/f11c zero size 2024-04-25T01:36:00.987 INFO:tasks.workunit.client.0.smithi033.stdout:8/788: chown d6/d49/fee 4203687 1 2024-04-25T01:36:00.989 INFO:tasks.workunit.client.1.smithi138.stdout:1/269: dwrite d3/d27/d2d/f30 [0,4194304] 0 2024-04-25T01:36:00.989 INFO:tasks.workunit.client.1.smithi138.stdout:5/241: truncate d4/da/d27/f1f 4860288 0 2024-04-25T01:36:00.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:00 smithi033 ceph-mon[26353]: pgmap v16: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 47 MiB/s rd, 156 MiB/s wr, 350 op/s 2024-04-25T01:36:00.995 INFO:tasks.workunit.client.1.smithi138.stdout:3/395: write db/df/d17/d20/d57/d6a/f6e [805212,89900] 0 2024-04-25T01:36:00.995 INFO:tasks.workunit.client.1.smithi138.stdout:5/242: symlink d4/da/d27/d1c/d22/l4a 0 2024-04-25T01:36:00.995 INFO:tasks.workunit.client.1.smithi138.stdout:6/353: getdents dc/d10/d67 0 2024-04-25T01:36:00.997 INFO:tasks.workunit.client.1.smithi138.stdout:1/270: mknod d3/d27/d3e/d51/c58 0 2024-04-25T01:36:00.997 INFO:tasks.workunit.client.1.smithi138.stdout:6/354: chown dc/d10/d28/f46 225731733 1 2024-04-25T01:36:00.997 INFO:tasks.workunit.client.0.smithi033.stdout:4/873: getdents d6 0 2024-04-25T01:36:00.998 INFO:tasks.workunit.client.0.smithi033.stdout:4/874: stat d6/d14/d35/d3a/d4d/d5e/d7c/ce4 0 2024-04-25T01:36:00.998 INFO:tasks.workunit.client.0.smithi033.stdout:8/789: mknod d6/db/c124 0 2024-04-25T01:36:00.999 INFO:tasks.workunit.client.0.smithi033.stdout:8/790: readlink d6/db/d80/la5 0 2024-04-25T01:36:00.999 INFO:tasks.workunit.client.1.smithi138.stdout:6/355: mknod dc/d2e/d55/c71 0 2024-04-25T01:36:01.001 INFO:tasks.workunit.client.1.smithi138.stdout:3/396: creat db/df/d17/d20/f74 x:0 0 0 2024-04-25T01:36:01.001 INFO:tasks.workunit.client.1.smithi138.stdout:5/243: getdents d4/da/d27/d1c 0 2024-04-25T01:36:01.004 INFO:tasks.workunit.client.0.smithi033.stdout:4/875: dwrite d6/d12/d38/f5d [4194304,4194304] 0 2024-04-25T01:36:01.004 INFO:tasks.workunit.client.1.smithi138.stdout:6/356: rename dc/d2e/f3a to dc/f72 0 2024-04-25T01:36:01.005 INFO:tasks.workunit.client.0.smithi033.stdout:9/823: symlink d0/d2/dd/dba/l121 0 2024-04-25T01:36:01.010 INFO:tasks.workunit.client.0.smithi033.stdout:9/824: mkdir d0/d2/dd/dba/dc1/d122 0 2024-04-25T01:36:01.017 INFO:tasks.workunit.client.0.smithi033.stdout:4/876: rmdir d6/d14/d1d/d3b/dd5 0 2024-04-25T01:36:01.017 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:00 smithi138 ceph-mon[32977]: pgmap v16: 65 pgs: 65 active+clean; 1.7 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 47 MiB/s rd, 156 MiB/s wr, 350 op/s 2024-04-25T01:36:01.017 INFO:tasks.workunit.client.0.smithi033.stdout:4/877: stat d6/d14/d35/d3a/d4c 0 2024-04-25T01:36:01.022 INFO:tasks.workunit.client.0.smithi033.stdout:4/878: mkdir d6/d14/d1d/d3b/d11b 0 2024-04-25T01:36:01.022 INFO:tasks.workunit.client.0.smithi033.stdout:4/879: readlink d6/d14/l37 0 2024-04-25T01:36:01.022 INFO:tasks.workunit.client.1.smithi138.stdout:5/244: getdents d4/da/d27/d1c/d22/d35 0 2024-04-25T01:36:01.023 INFO:tasks.workunit.client.1.smithi138.stdout:3/397: dwrite db/df/d17/f1c [4194304,4194304] 0 2024-04-25T01:36:01.025 INFO:tasks.workunit.client.0.smithi033.stdout:7/824: mkdir d4/d2e/d6b/d101 0 2024-04-25T01:36:01.025 INFO:tasks.workunit.client.1.smithi138.stdout:5/245: stat d4/da/d27/d1c/d22/c2f 0 2024-04-25T01:36:01.027 INFO:tasks.workunit.client.0.smithi033.stdout:7/825: truncate d4/d2e/d6b/da9/ff3 26816 0 2024-04-25T01:36:01.027 INFO:tasks.workunit.client.1.smithi138.stdout:6/357: link dc/d2e/f3e dc/d2e/d37/d56/f73 0 2024-04-25T01:36:01.028 INFO:tasks.workunit.client.1.smithi138.stdout:6/358: readlink la 0 2024-04-25T01:36:01.028 INFO:tasks.workunit.client.1.smithi138.stdout:5/246: mknod d4/da/d27/d1c/c4b 0 2024-04-25T01:36:01.029 INFO:tasks.workunit.client.1.smithi138.stdout:5/247: chown d4/f1a 2 1 2024-04-25T01:36:01.031 INFO:tasks.workunit.client.1.smithi138.stdout:3/398: getdents db/df/d19 0 2024-04-25T01:36:01.034 INFO:tasks.workunit.client.1.smithi138.stdout:6/359: dwrite dc/d2e/f2f [0,4194304] 0 2024-04-25T01:36:01.041 INFO:tasks.workunit.client.1.smithi138.stdout:3/399: link db/df/d17/f2e db/df/d19/f75 0 2024-04-25T01:36:01.046 INFO:tasks.workunit.client.1.smithi138.stdout:6/360: link dc/d10/f33 dc/d2e/d37/d56/f74 0 2024-04-25T01:36:01.048 INFO:tasks.workunit.client.1.smithi138.stdout:5/248: mkdir d4/da/d27/d1c/d3c/d4c 0 2024-04-25T01:36:01.048 INFO:tasks.workunit.client.0.smithi033.stdout:1/681: rename d2/d11/d28/c7b to d2/d5/d6c/d94/cd5 0 2024-04-25T01:36:01.050 INFO:tasks.workunit.client.1.smithi138.stdout:6/361: dread dc/d10/f6b [0,4194304] 0 2024-04-25T01:36:01.051 INFO:tasks.workunit.client.0.smithi033.stdout:1/682: truncate d2/d5/d75/d59/d83/fa9 4350915 0 2024-04-25T01:36:01.052 INFO:tasks.workunit.client.1.smithi138.stdout:6/362: write dc/d2e/f31 [2757220,82079] 0 2024-04-25T01:36:01.055 INFO:tasks.workunit.client.1.smithi138.stdout:5/249: dwrite d4/f1a [0,4194304] 0 2024-04-25T01:36:01.058 INFO:tasks.workunit.client.0.smithi033.stdout:1/683: dwrite d2/d2e/dcd/fc8 [0,4194304] 0 2024-04-25T01:36:01.065 INFO:tasks.workunit.client.0.smithi033.stdout:2/794: write da/d5b/d72/f95 [687243,82897] 0 2024-04-25T01:36:01.066 INFO:tasks.workunit.client.1.smithi138.stdout:6/363: dwrite dc/f60 [0,4194304] 0 2024-04-25T01:36:01.069 INFO:tasks.workunit.client.1.smithi138.stdout:3/400: dread db/df/d19/f75 [0,4194304] 0 2024-04-25T01:36:01.069 INFO:tasks.workunit.client.0.smithi033.stdout:2/795: creat da/d50/d6a/d43/f101 x:0 0 0 2024-04-25T01:36:01.071 INFO:tasks.workunit.client.1.smithi138.stdout:5/250: rename d4/da/d27/d1c/d3c/d40/d49 to d4/da/d27/d1c/d22/d35/d4d 0 2024-04-25T01:36:01.072 INFO:tasks.workunit.client.0.smithi033.stdout:6/944: dwrite d4/d24/d36/d42/d9d/fd1 [0,4194304] 0 2024-04-25T01:36:01.074 INFO:tasks.workunit.client.0.smithi033.stdout:2/796: mknod da/d27/d31/d40/db1/c102 0 2024-04-25T01:36:01.075 INFO:tasks.workunit.client.0.smithi033.stdout:2/797: fdatasync da/f42 0 2024-04-25T01:36:01.077 INFO:tasks.workunit.client.1.smithi138.stdout:6/364: dread dc/f16 [0,4194304] 0 2024-04-25T01:36:01.077 INFO:tasks.workunit.client.1.smithi138.stdout:3/401: fdatasync db/df/d19/f70 0 2024-04-25T01:36:01.077 INFO:tasks.workunit.client.0.smithi033.stdout:6/945: mkdir d4/d15/d13d 0 2024-04-25T01:36:01.081 INFO:tasks.workunit.client.0.smithi033.stdout:2/798: rmdir da/d8d/db0 39 2024-04-25T01:36:01.082 INFO:tasks.workunit.client.0.smithi033.stdout:2/799: chown da/d2c 445 1 2024-04-25T01:36:01.082 INFO:tasks.workunit.client.1.smithi138.stdout:6/365: getdents dc/d2e 0 2024-04-25T01:36:01.084 INFO:tasks.workunit.client.1.smithi138.stdout:3/402: rename db/df/d17/d23/d34/f68 to db/df/d17/d5d/f76 0 2024-04-25T01:36:01.084 INFO:tasks.workunit.client.1.smithi138.stdout:6/366: fsync dc/d10/f15 0 2024-04-25T01:36:01.087 INFO:tasks.workunit.client.0.smithi033.stdout:6/946: mkdir d4/dfc/d13e 0 2024-04-25T01:36:01.088 INFO:tasks.workunit.client.1.smithi138.stdout:3/403: stat db/df/d17/l56 0 2024-04-25T01:36:01.088 INFO:tasks.workunit.client.1.smithi138.stdout:6/367: dread - dc/d2e/f4d zero size 2024-04-25T01:36:01.088 INFO:tasks.workunit.client.1.smithi138.stdout:6/368: read dc/d10/d28/f3c [78210,116137] 0 2024-04-25T01:36:01.089 INFO:tasks.workunit.client.1.smithi138.stdout:3/404: mkdir db/d14/d77 0 2024-04-25T01:36:01.089 INFO:tasks.workunit.client.1.smithi138.stdout:6/369: dread - dc/d10/d67/f6e zero size 2024-04-25T01:36:01.090 INFO:tasks.workunit.client.1.smithi138.stdout:3/405: creat db/df/d17/d5d/f78 x:0 0 0 2024-04-25T01:36:01.090 INFO:tasks.workunit.client.0.smithi033.stdout:8/791: dwrite d6/d45/d51/d55/f5c [0,4194304] 0 2024-04-25T01:36:01.091 INFO:tasks.workunit.client.0.smithi033.stdout:3/852: rmdir d8/dd/d3d/d63/d7c/ded 39 2024-04-25T01:36:01.091 INFO:tasks.workunit.client.0.smithi033.stdout:8/792: write d6/d45/d52/fe2 [451374,122943] 0 2024-04-25T01:36:01.091 INFO:tasks.workunit.client.0.smithi033.stdout:3/853: readlink d8/dd/d40/l41 0 2024-04-25T01:36:01.091 INFO:tasks.workunit.client.0.smithi033.stdout:6/947: mkdir d4/d24/d36/d42/d97/de0/d13f 0 2024-04-25T01:36:01.092 INFO:tasks.workunit.client.1.smithi138.stdout:6/370: dread dc/d10/f15 [0,4194304] 0 2024-04-25T01:36:01.093 INFO:tasks.workunit.client.0.smithi033.stdout:2/800: creat da/d5b/d5e/f103 x:0 0 0 2024-04-25T01:36:01.093 INFO:tasks.workunit.client.0.smithi033.stdout:3/854: readlink d8/dd/d3d/d63/d7c/d9d/de8/df9/l106 0 2024-04-25T01:36:01.093 INFO:tasks.workunit.client.1.smithi138.stdout:3/406: rename db/df/d17/d23/f54 to db/df/d17/d23/f79 0 2024-04-25T01:36:01.098 INFO:tasks.workunit.client.1.smithi138.stdout:3/407: mkdir db/d14/d7a 0 2024-04-25T01:36:01.098 INFO:tasks.workunit.client.1.smithi138.stdout:6/371: truncate dc/d10/d67/f6e 221830 0 2024-04-25T01:36:01.099 INFO:tasks.workunit.client.0.smithi033.stdout:8/793: mknod d6/d45/d52/d2e/dd4/c125 0 2024-04-25T01:36:01.099 INFO:tasks.workunit.client.0.smithi033.stdout:6/948: read d4/d1d/d3e/d41/d5c/d7f/d116/f8c [434867,31271] 0 2024-04-25T01:36:01.099 INFO:tasks.workunit.client.1.smithi138.stdout:3/408: chown db/df/d17/d20 456408486 1 2024-04-25T01:36:01.100 INFO:tasks.workunit.client.1.smithi138.stdout:6/372: creat dc/d2e/d55/f75 x:0 0 0 2024-04-25T01:36:01.101 INFO:tasks.workunit.client.1.smithi138.stdout:3/409: mknod db/df/c7b 0 2024-04-25T01:36:01.101 INFO:tasks.workunit.client.1.smithi138.stdout:6/373: truncate dc/d2e/f4d 462687 0 2024-04-25T01:36:01.102 INFO:tasks.workunit.client.0.smithi033.stdout:9/825: dwrite d0/d2/dd/f41 [4194304,4194304] 0 2024-04-25T01:36:01.105 INFO:tasks.workunit.client.0.smithi033.stdout:2/801: creat da/d8d/db0/d100/f104 x:0 0 0 2024-04-25T01:36:01.105 INFO:tasks.workunit.client.1.smithi138.stdout:5/251: dread d4/f18 [0,4194304] 0 2024-04-25T01:36:01.105 INFO:tasks.workunit.client.1.smithi138.stdout:6/374: truncate dc/d2e/d37/f5f 1588481 0 2024-04-25T01:36:01.108 INFO:tasks.workunit.client.0.smithi033.stdout:2/802: fsync da/d27/d31/d40/d8a/dbb/f98 0 2024-04-25T01:36:01.109 INFO:tasks.workunit.client.1.smithi138.stdout:5/252: mknod d4/da/d27/d1c/d22/c4e 0 2024-04-25T01:36:01.110 INFO:tasks.workunit.client.0.smithi033.stdout:3/855: dwrite d8/dd/d3d/d63/f10a [0,4194304] 0 2024-04-25T01:36:01.115 INFO:tasks.workunit.client.0.smithi033.stdout:4/880: rename d6/d14/d1d/f2e to d6/d14/d35/d44/d45/d46/d57/d70/f11c 0 2024-04-25T01:36:01.116 INFO:tasks.workunit.client.0.smithi033.stdout:8/794: mknod d6/d45/c126 0 2024-04-25T01:36:01.118 INFO:tasks.workunit.client.1.smithi138.stdout:5/253: rename d4/da/d27/d1c/d22/d35/l37 to d4/da/d27/d1c/d3c/l4f 0 2024-04-25T01:36:01.119 INFO:tasks.workunit.client.0.smithi033.stdout:2/803: unlink da/d5b/la4 0 2024-04-25T01:36:01.123 INFO:tasks.workunit.client.0.smithi033.stdout:4/881: read - d6/d14/d35/d44/d45/d46/d57/d70/d87/f100 zero size 2024-04-25T01:36:01.123 INFO:tasks.workunit.client.0.smithi033.stdout:6/949: dwrite d4/d1d/d3e/d41/d93/f137 [0,4194304] 0 2024-04-25T01:36:01.126 INFO:tasks.workunit.client.1.smithi138.stdout:5/254: dwrite d4/da/d27/d1c/d22/f3f [0,4194304] 0 2024-04-25T01:36:01.130 INFO:tasks.workunit.client.0.smithi033.stdout:8/795: creat d6/db/f127 x:0 0 0 2024-04-25T01:36:01.132 INFO:tasks.workunit.client.0.smithi033.stdout:3/856: getdents d8/dd/d3d/d63 0 2024-04-25T01:36:01.132 INFO:tasks.workunit.client.1.smithi138.stdout:5/255: symlink d4/da/d27/d1c/d22/d45/l50 0 2024-04-25T01:36:01.133 INFO:tasks.workunit.client.0.smithi033.stdout:6/950: fdatasync d4/d1d/d3e/d41/d5c/d7f/f44 0 2024-04-25T01:36:01.135 INFO:tasks.workunit.client.0.smithi033.stdout:2/804: dwrite da/d27/fb2 [0,4194304] 0 2024-04-25T01:36:01.135 INFO:tasks.workunit.client.1.smithi138.stdout:6/375: dwrite dc/d10/f26 [0,4194304] 0 2024-04-25T01:36:01.137 INFO:tasks.workunit.client.1.smithi138.stdout:5/256: rmdir d4/da/d27 39 2024-04-25T01:36:01.138 INFO:tasks.workunit.client.0.smithi033.stdout:4/882: dwrite d6/d14/fd7 [0,4194304] 0 2024-04-25T01:36:01.139 INFO:tasks.workunit.client.0.smithi033.stdout:8/796: chown d6/db/d80/dd1/fe8 1 1 2024-04-25T01:36:01.142 INFO:tasks.workunit.client.1.smithi138.stdout:6/376: chown dc/d2e/f3e 1409 1 2024-04-25T01:36:01.143 INFO:tasks.workunit.client.1.smithi138.stdout:6/377: fsync dc/f1d 0 2024-04-25T01:36:01.144 INFO:tasks.workunit.client.0.smithi033.stdout:9/826: dread d0/d2/d43/d73/dab/dad/dee/ff7 [0,4194304] 0 2024-04-25T01:36:01.149 INFO:tasks.workunit.client.1.smithi138.stdout:3/410: dread db/df/d17/f5e [0,4194304] 0 2024-04-25T01:36:01.149 INFO:tasks.workunit.client.1.smithi138.stdout:3/411: dread - db/f69 zero size 2024-04-25T01:36:01.150 INFO:tasks.workunit.client.1.smithi138.stdout:6/378: unlink dc/d10/d28/f40 0 2024-04-25T01:36:01.152 INFO:tasks.workunit.client.0.smithi033.stdout:9/827: read d0/d2/d43/d73/dc4/dcb/ff1 [4187372,67099] 0 2024-04-25T01:36:01.152 INFO:tasks.workunit.client.0.smithi033.stdout:2/805: symlink da/d2c/l105 0 2024-04-25T01:36:01.153 INFO:tasks.workunit.client.1.smithi138.stdout:6/379: write dc/d2e/f4d [386107,91635] 0 2024-04-25T01:36:01.154 INFO:tasks.workunit.client.0.smithi033.stdout:4/883: dwrite d6/d14/d35/d44/d45/d46/d57/d70/d87/f100 [0,4194304] 0 2024-04-25T01:36:01.155 INFO:tasks.workunit.client.1.smithi138.stdout:3/412: creat db/d14/d40/d41/f7c x:0 0 0 2024-04-25T01:36:01.156 INFO:tasks.workunit.client.1.smithi138.stdout:3/413: stat db/df/d17/d23 0 2024-04-25T01:36:01.158 INFO:tasks.workunit.client.0.smithi033.stdout:8/797: mkdir d6/db/d80/d128 0 2024-04-25T01:36:01.159 INFO:tasks.workunit.client.1.smithi138.stdout:3/414: write db/df/f4f [4560557,23533] 0 2024-04-25T01:36:01.161 INFO:tasks.workunit.client.1.smithi138.stdout:6/380: dwrite dc/f3d [0,4194304] 0 2024-04-25T01:36:01.164 INFO:tasks.workunit.client.0.smithi033.stdout:3/857: mkdir d8/dd/d3d/d63/d7c/ded/dc7/ddf/d10b/d12d 0 2024-04-25T01:36:01.165 INFO:tasks.workunit.client.1.smithi138.stdout:3/415: creat db/df/d17/d20/f7d x:0 0 0 2024-04-25T01:36:01.166 INFO:tasks.workunit.client.1.smithi138.stdout:6/381: unlink dc/d10/c44 0 2024-04-25T01:36:01.167 INFO:tasks.workunit.client.1.smithi138.stdout:6/382: fsync dc/d2e/d37/f52 0 2024-04-25T01:36:01.170 INFO:tasks.workunit.client.1.smithi138.stdout:6/383: rename dc/d10/d28/f3c to dc/d10/f76 0 2024-04-25T01:36:01.171 INFO:tasks.workunit.client.1.smithi138.stdout:3/416: getdents db/df/d17/d23 0 2024-04-25T01:36:01.171 INFO:tasks.workunit.client.0.smithi033.stdout:8/798: creat d6/d45/d52/dae/dc3/f129 x:0 0 0 2024-04-25T01:36:01.172 INFO:tasks.workunit.client.1.smithi138.stdout:6/384: write dc/f64 [2015493,95486] 0 2024-04-25T01:36:01.173 INFO:tasks.workunit.client.0.smithi033.stdout:2/806: write da/d5b/d72/dd1/ded/ff8 [350141,25806] 0 2024-04-25T01:36:01.175 INFO:tasks.workunit.client.1.smithi138.stdout:6/385: creat dc/d10/d67/f77 x:0 0 0 2024-04-25T01:36:01.175 INFO:tasks.workunit.client.0.smithi033.stdout:3/858: mknod d8/dd/d3d/d63/d7c/d4d/da5/c12e 0 2024-04-25T01:36:01.176 INFO:tasks.workunit.client.1.smithi138.stdout:6/386: write dc/f4e [220267,63448] 0 2024-04-25T01:36:01.176 INFO:tasks.workunit.client.1.smithi138.stdout:6/387: stat dc/d10/d67/f6e 0 2024-04-25T01:36:01.178 INFO:tasks.workunit.client.1.smithi138.stdout:6/388: mknod dc/d10/d6d/c78 0 2024-04-25T01:36:01.179 INFO:tasks.workunit.client.1.smithi138.stdout:6/389: chown dc/f64 0 1 2024-04-25T01:36:01.179 INFO:tasks.workunit.client.0.smithi033.stdout:8/799: mkdir d6/d45/d52/d2e/d95/d12a 0 2024-04-25T01:36:01.180 INFO:tasks.workunit.client.1.smithi138.stdout:5/257: dread d4/f9 [0,4194304] 0 2024-04-25T01:36:01.181 INFO:tasks.workunit.client.1.smithi138.stdout:6/390: mknod dc/d10/d67/c79 0 2024-04-25T01:36:01.181 INFO:tasks.workunit.client.1.smithi138.stdout:5/258: write d4/da/f47 [963957,79933] 0 2024-04-25T01:36:01.182 INFO:tasks.workunit.client.0.smithi033.stdout:9/828: creat d0/d2/dd/d20/f123 x:0 0 0 2024-04-25T01:36:01.183 INFO:tasks.workunit.client.1.smithi138.stdout:6/391: unlink dc/d10/l20 0 2024-04-25T01:36:01.184 INFO:tasks.workunit.client.1.smithi138.stdout:5/259: unlink d4/f23 0 2024-04-25T01:36:01.184 INFO:tasks.workunit.client.0.smithi033.stdout:4/884: link d6/d14/d35/d3a/c8d d6/d14/d1d/c11d 0 2024-04-25T01:36:01.185 INFO:tasks.workunit.client.0.smithi033.stdout:9/829: truncate d0/d2/d43/d4b/dc5/de0/f70 4321273 0 2024-04-25T01:36:01.186 INFO:tasks.workunit.client.0.smithi033.stdout:8/800: mkdir d6/d45/d52/df2/d12b 0 2024-04-25T01:36:01.188 INFO:tasks.workunit.client.1.smithi138.stdout:5/260: mknod d4/da/d27/d1c/d22/d35/d4d/c51 0 2024-04-25T01:36:01.188 INFO:tasks.workunit.client.0.smithi033.stdout:8/801: write d6/db/fe0 [975522,89605] 0 2024-04-25T01:36:01.190 INFO:tasks.workunit.client.0.smithi033.stdout:9/830: dread - d0/d2/fc9 zero size 2024-04-25T01:36:01.190 INFO:tasks.workunit.client.1.smithi138.stdout:5/261: mkdir d4/da/d52 0 2024-04-25T01:36:01.191 INFO:tasks.workunit.client.1.smithi138.stdout:5/262: chown d4/da/d27/d1c/d3c/d40 1 1 2024-04-25T01:36:01.193 INFO:tasks.workunit.client.0.smithi033.stdout:4/885: symlink d6/d14/d1d/d3b/d11b/l11e 0 2024-04-25T01:36:01.194 INFO:tasks.workunit.client.0.smithi033.stdout:3/859: getdents d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86 0 2024-04-25T01:36:01.196 INFO:tasks.workunit.client.1.smithi138.stdout:5/263: rmdir d4/da/d27/d1c/d3c/d4c 0 2024-04-25T01:36:01.197 INFO:tasks.workunit.client.0.smithi033.stdout:6/951: dwrite d4/d24/d36/d42/d97/de0/f139 [0,4194304] 0 2024-04-25T01:36:01.197 INFO:tasks.workunit.client.0.smithi033.stdout:4/886: write d6/d14/d1d/f63 [686001,5669] 0 2024-04-25T01:36:01.198 INFO:tasks.workunit.client.0.smithi033.stdout:9/831: symlink d0/d2/dd/df/l124 0 2024-04-25T01:36:01.199 INFO:tasks.workunit.client.1.smithi138.stdout:5/264: mknod d4/da/d27/d1c/d22/d35/d4d/c53 0 2024-04-25T01:36:01.199 INFO:tasks.workunit.client.0.smithi033.stdout:5/857: sync 2024-04-25T01:36:01.199 INFO:tasks.workunit.client.0.smithi033.stdout:0/758: sync 2024-04-25T01:36:01.200 INFO:tasks.workunit.client.0.smithi033.stdout:2/807: dread da/d27/d31/f46 [0,4194304] 0 2024-04-25T01:36:01.204 INFO:tasks.workunit.client.0.smithi033.stdout:2/808: chown da/l1d 725018 1 2024-04-25T01:36:01.204 INFO:tasks.workunit.client.0.smithi033.stdout:9/832: mkdir d0/d2/dd/d5f/d11b/d125 0 2024-04-25T01:36:01.206 INFO:tasks.workunit.client.0.smithi033.stdout:4/887: rename d6/d14/ff to d6/d14/d35/d44/f11f 0 2024-04-25T01:36:01.207 INFO:tasks.workunit.client.0.smithi033.stdout:5/858: mknod d1c/d53/df6/c119 0 2024-04-25T01:36:01.207 INFO:tasks.workunit.client.0.smithi033.stdout:4/888: chown d6/d14/d35/d3a/d4d/fce 13 1 2024-04-25T01:36:01.208 INFO:tasks.workunit.client.1.smithi138.stdout:6/392: dwrite dc/d10/f33 [0,4194304] 0 2024-04-25T01:36:01.208 INFO:tasks.workunit.client.0.smithi033.stdout:2/809: rmdir da/d27/d31/dca 39 2024-04-25T01:36:01.209 INFO:tasks.workunit.client.1.smithi138.stdout:5/265: getdents d4/da/d27/d1c/d22/d35/d4d 0 2024-04-25T01:36:01.210 INFO:tasks.workunit.client.0.smithi033.stdout:2/810: dread - da/d27/d31/d40/d8a/dbb/fff zero size 2024-04-25T01:36:01.210 INFO:tasks.workunit.client.0.smithi033.stdout:0/759: truncate d7/da/df/d14/f69 1325975 0 2024-04-25T01:36:01.212 INFO:tasks.workunit.client.1.smithi138.stdout:6/393: unlink dc/f64 0 2024-04-25T01:36:01.217 INFO:tasks.workunit.client.0.smithi033.stdout:8/802: dread d6/d45/d52/f3b [0,4194304] 0 2024-04-25T01:36:01.217 INFO:tasks.workunit.client.1.smithi138.stdout:5/266: dwrite d4/da/f47 [0,4194304] 0 2024-04-25T01:36:01.219 INFO:tasks.workunit.client.1.smithi138.stdout:5/267: write d4/da/d27/d1c/d22/f24 [4839752,88728] 0 2024-04-25T01:36:01.223 INFO:tasks.workunit.client.0.smithi033.stdout:2/811: dwrite da/d27/d31/d40/d8a/dbb/f98 [0,4194304] 0 2024-04-25T01:36:01.224 INFO:tasks.workunit.client.1.smithi138.stdout:5/268: creat d4/da/d27/d1c/d22/d45/f54 x:0 0 0 2024-04-25T01:36:01.225 INFO:tasks.workunit.client.0.smithi033.stdout:0/760: getdents d7/d2d/d4d/d9c/d61/d70/dee 0 2024-04-25T01:36:01.227 INFO:tasks.workunit.client.1.smithi138.stdout:5/269: dread d4/f9 [0,4194304] 0 2024-04-25T01:36:01.227 INFO:tasks.workunit.client.1.smithi138.stdout:5/270: write d4/da/d27/d1c/d22/f24 [7125458,57248] 0 2024-04-25T01:36:01.228 INFO:tasks.workunit.client.0.smithi033.stdout:3/860: dread d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/f45 [0,4194304] 0 2024-04-25T01:36:01.229 INFO:tasks.workunit.client.1.smithi138.stdout:2/342: sync 2024-04-25T01:36:01.229 INFO:tasks.workunit.client.1.smithi138.stdout:8/467: sync 2024-04-25T01:36:01.229 INFO:tasks.workunit.client.1.smithi138.stdout:9/239: sync 2024-04-25T01:36:01.229 INFO:tasks.workunit.client.1.smithi138.stdout:4/163: sync 2024-04-25T01:36:01.229 INFO:tasks.workunit.client.1.smithi138.stdout:0/267: sync 2024-04-25T01:36:01.229 INFO:tasks.workunit.client.1.smithi138.stdout:7/256: sync 2024-04-25T01:36:01.231 INFO:tasks.workunit.client.1.smithi138.stdout:5/271: creat d4/da/d27/d1c/d3c/d40/f55 x:0 0 0 2024-04-25T01:36:01.235 INFO:tasks.workunit.client.0.smithi033.stdout:0/761: getdents d7/d2d/d43/d44/dd4 0 2024-04-25T01:36:01.285 INFO:tasks.workunit.client.0.smithi033.stdout:0/762: dread d7/d2d/d43/dc0/f60 [0,4194304] 0 2024-04-25T01:36:01.287 INFO:tasks.workunit.client.1.smithi138.stdout:9/240: creat d4/d1a/d3c/f51 x:0 0 0 2024-04-25T01:36:01.289 INFO:tasks.workunit.client.1.smithi138.stdout:8/468: creat d3/d16/d1f/d31/d82/f90 x:0 0 0 2024-04-25T01:36:01.290 INFO:tasks.workunit.client.0.smithi033.stdout:0/763: rename d7/d2d/d64/d51/f89 to d7/d2d/d4d/d9c/dcf/ddd/f108 0 2024-04-25T01:36:01.291 INFO:tasks.workunit.client.0.smithi033.stdout:0/764: mknod d7/d2d/d4c/da0/c109 0 2024-04-25T01:36:01.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/164: getdents d0/d16 0 2024-04-25T01:36:01.294 INFO:tasks.workunit.client.0.smithi033.stdout:0/765: unlink d7/d2d/d4d/d9c/lf4 0 2024-04-25T01:36:01.295 INFO:tasks.workunit.client.1.smithi138.stdout:8/469: write d3/d16/d88/f40 [1563892,81700] 0 2024-04-25T01:36:01.295 INFO:tasks.workunit.client.0.smithi033.stdout:0/766: read - d7/d2d/d43/d92/fea zero size 2024-04-25T01:36:01.296 INFO:tasks.workunit.client.1.smithi138.stdout:5/272: link d4/da/d27/d1c/d3c/d40/f55 d4/da/d27/d1c/d22/d35/f56 0 2024-04-25T01:36:01.296 INFO:tasks.workunit.client.1.smithi138.stdout:2/343: getdents d0/d32 0 2024-04-25T01:36:01.297 INFO:tasks.workunit.client.0.smithi033.stdout:0/767: write d7/d2d/d43/d44/f63 [3190405,122798] 0 2024-04-25T01:36:01.297 INFO:tasks.workunit.client.0.smithi033.stdout:5/859: dread fc [0,4194304] 0 2024-04-25T01:36:01.298 INFO:tasks.workunit.client.1.smithi138.stdout:4/165: unlink d0/f12 0 2024-04-25T01:36:01.300 INFO:tasks.workunit.client.1.smithi138.stdout:8/470: chown d3/d16/d1f/d2a/f8b 9 1 2024-04-25T01:36:01.301 INFO:tasks.workunit.client.1.smithi138.stdout:7/257: dwrite da/f15 [0,4194304] 0 2024-04-25T01:36:01.301 INFO:tasks.workunit.client.1.smithi138.stdout:9/241: dwrite d4/d1a/f20 [0,4194304] 0 2024-04-25T01:36:01.301 INFO:tasks.workunit.client.1.smithi138.stdout:2/344: unlink d0/d21/d45/c52 0 2024-04-25T01:36:01.302 INFO:tasks.workunit.client.0.smithi033.stdout:5/860: dread d1c/d2a/d50/d52/da6/d4a/f72 [0,4194304] 0 2024-04-25T01:36:01.303 INFO:tasks.workunit.client.1.smithi138.stdout:8/471: chown d3/d16/d1f/d31/d82 15541176 1 2024-04-25T01:36:01.303 INFO:tasks.workunit.client.1.smithi138.stdout:4/166: fsync d0/f13 0 2024-04-25T01:36:01.304 INFO:tasks.workunit.client.1.smithi138.stdout:9/242: dread - d4/d1a/d3c/f51 zero size 2024-04-25T01:36:01.304 INFO:tasks.workunit.client.1.smithi138.stdout:5/273: symlink d4/da/d27/d1c/d22/d45/l57 0 2024-04-25T01:36:01.305 INFO:tasks.workunit.client.1.smithi138.stdout:2/345: rmdir d0 39 2024-04-25T01:36:01.309 INFO:tasks.workunit.client.1.smithi138.stdout:7/258: symlink da/l56 0 2024-04-25T01:36:01.312 INFO:tasks.workunit.client.1.smithi138.stdout:2/346: write d0/d5a/d15/f28 [672017,73914] 0 2024-04-25T01:36:01.312 INFO:tasks.workunit.client.1.smithi138.stdout:8/472: creat d3/d16/d1f/d87/f91 x:0 0 0 2024-04-25T01:36:01.312 INFO:tasks.workunit.client.0.smithi033.stdout:8/803: write d6/d45/f57 [4556130,122287] 0 2024-04-25T01:36:01.313 INFO:tasks.workunit.client.0.smithi033.stdout:2/812: write da/d27/d31/f83 [1261230,83326] 0 2024-04-25T01:36:01.313 INFO:tasks.workunit.client.0.smithi033.stdout:5/861: dwrite d1c/d2a/d50/d52/da6/d3e/d70/d7a/fe5 [0,4194304] 0 2024-04-25T01:36:01.314 INFO:tasks.workunit.client.1.smithi138.stdout:5/274: write d4/da/d27/d1c/f3d [712031,70046] 0 2024-04-25T01:36:01.315 INFO:tasks.workunit.client.1.smithi138.stdout:8/473: write d3/d16/d88/da/db/d15/d20/f5a [1859834,71388] 0 2024-04-25T01:36:01.316 INFO:tasks.workunit.client.1.smithi138.stdout:5/275: dread - d4/da/d27/d1c/d22/d45/f54 zero size 2024-04-25T01:36:01.317 INFO:tasks.workunit.client.0.smithi033.stdout:5/862: chown d1c/d53/fd0 840 1 2024-04-25T01:36:01.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/243: mkdir d4/d1a/d33/d1d/d3f/d52 0 2024-04-25T01:36:01.321 INFO:tasks.workunit.client.1.smithi138.stdout:2/347: creat d0/d5a/d15/f67 x:0 0 0 2024-04-25T01:36:01.322 INFO:tasks.workunit.client.0.smithi033.stdout:3/861: dwrite d8/dd/fcf [0,4194304] 0 2024-04-25T01:36:01.322 INFO:tasks.workunit.client.1.smithi138.stdout:0/268: dread d6/df/d3a/f3d [0,4194304] 0 2024-04-25T01:36:01.323 INFO:tasks.workunit.client.0.smithi033.stdout:2/813: mknod da/d50/d6a/d43/c106 0 2024-04-25T01:36:01.325 INFO:tasks.workunit.client.0.smithi033.stdout:8/804: truncate d6/dd3/dfb/f105 3656627 0 2024-04-25T01:36:01.326 INFO:tasks.workunit.client.0.smithi033.stdout:2/814: truncate f8 1679382 0 2024-04-25T01:36:01.328 INFO:tasks.workunit.client.0.smithi033.stdout:2/815: dread - da/d27/d31/fe9 zero size 2024-04-25T01:36:01.331 INFO:tasks.workunit.client.0.smithi033.stdout:3/862: dwrite d8/dd/d3d/d63/d7c/f73 [4194304,4194304] 0 2024-04-25T01:36:01.333 INFO:tasks.workunit.client.1.smithi138.stdout:7/259: symlink da/de/d1b/d34/l57 0 2024-04-25T01:36:01.334 INFO:tasks.workunit.client.1.smithi138.stdout:2/348: truncate d0/d18/f1b 1707794 0 2024-04-25T01:36:01.334 INFO:tasks.workunit.client.0.smithi033.stdout:2/816: mkdir da/d2c/dcd/d107 0 2024-04-25T01:36:01.335 INFO:tasks.workunit.client.1.smithi138.stdout:0/269: rename d6/d2f/f34 to d6/df/f5c 0 2024-04-25T01:36:01.336 INFO:tasks.workunit.client.1.smithi138.stdout:7/260: write da/de/d1b/d34/f3d [605413,78891] 0 2024-04-25T01:36:01.337 INFO:tasks.workunit.client.1.smithi138.stdout:8/474: symlink d3/d16/d1f/d31/d82/d8a/l92 0 2024-04-25T01:36:01.337 INFO:tasks.workunit.client.1.smithi138.stdout:7/261: fsync da/de/f27 0 2024-04-25T01:36:01.338 INFO:tasks.workunit.client.0.smithi033.stdout:3/863: fsync d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/fbb 0 2024-04-25T01:36:01.338 INFO:tasks.workunit.client.1.smithi138.stdout:7/262: chown da/de/d13/d30 72125 1 2024-04-25T01:36:01.338 INFO:tasks.workunit.client.1.smithi138.stdout:2/349: rmdir d0/d5a/d15/d51 39 2024-04-25T01:36:01.340 INFO:tasks.workunit.client.1.smithi138.stdout:7/263: write da/de/d13/f3c [323592,111502] 0 2024-04-25T01:36:01.343 INFO:tasks.workunit.client.1.smithi138.stdout:5/276: link d4/da/d27/d1c/f26 d4/da/d27/d1c/d3c/d40/f58 0 2024-04-25T01:36:01.344 INFO:tasks.workunit.client.0.smithi033.stdout:3/864: mknod d8/dd/d11/d102/c12f 0 2024-04-25T01:36:01.344 INFO:tasks.workunit.client.1.smithi138.stdout:9/244: creat d4/d8/f53 x:0 0 0 2024-04-25T01:36:01.344 INFO:tasks.workunit.client.1.smithi138.stdout:9/245: stat d4/f3e 0 2024-04-25T01:36:01.345 INFO:tasks.workunit.client.0.smithi033.stdout:0/768: dwrite d7/d33/f36 [0,4194304] 0 2024-04-25T01:36:01.345 INFO:tasks.workunit.client.1.smithi138.stdout:4/167: dread d0/f1a [4194304,4194304] 0 2024-04-25T01:36:01.346 INFO:tasks.workunit.client.1.smithi138.stdout:2/350: symlink d0/d21/d45/l68 0 2024-04-25T01:36:01.346 INFO:tasks.workunit.client.0.smithi033.stdout:0/769: stat d7/d2d/d4d 0 2024-04-25T01:36:01.346 INFO:tasks.workunit.client.0.smithi033.stdout:3/865: dread - d8/dd/d3d/d63/d7c/ff6 zero size 2024-04-25T01:36:01.346 INFO:tasks.workunit.client.1.smithi138.stdout:5/277: truncate d4/da/d27/d1c/d3c/d40/f55 25715 0 2024-04-25T01:36:01.347 INFO:tasks.workunit.client.0.smithi033.stdout:2/817: read da/d8d/fda [65849,85443] 0 2024-04-25T01:36:01.348 INFO:tasks.workunit.client.1.smithi138.stdout:2/351: fdatasync d0/d32/f63 0 2024-04-25T01:36:01.349 INFO:tasks.workunit.client.1.smithi138.stdout:0/270: rmdir d6/d54 0 2024-04-25T01:36:01.349 INFO:tasks.workunit.client.0.smithi033.stdout:2/818: dread - da/d50/d6a/d43/f99 zero size 2024-04-25T01:36:01.350 INFO:tasks.workunit.client.1.smithi138.stdout:4/168: fdatasync d0/fa 0 2024-04-25T01:36:01.351 INFO:tasks.workunit.client.0.smithi033.stdout:2/819: write da/d27/f86 [3311819,8598] 0 2024-04-25T01:36:01.352 INFO:tasks.workunit.client.1.smithi138.stdout:7/264: getdents da/de/d1b/d34/d50 0 2024-04-25T01:36:01.353 INFO:tasks.workunit.client.0.smithi033.stdout:3/866: symlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/daa/l130 0 2024-04-25T01:36:01.353 INFO:tasks.workunit.client.0.smithi033.stdout:0/770: creat d7/d2d/d4d/d9c/d68/dd2/f10a x:0 0 0 2024-04-25T01:36:01.354 INFO:tasks.workunit.client.1.smithi138.stdout:9/246: dwrite d4/d8/f45 [0,4194304] 0 2024-04-25T01:36:01.355 INFO:tasks.workunit.client.1.smithi138.stdout:5/278: mkdir d4/da/d27/d1c/d3c/d40/d59 0 2024-04-25T01:36:01.357 INFO:tasks.workunit.client.0.smithi033.stdout:0/771: mknod d7/da/df/d14/c10b 0 2024-04-25T01:36:01.358 INFO:tasks.workunit.client.1.smithi138.stdout:0/271: mknod d6/d39/d58/c5d 0 2024-04-25T01:36:01.359 INFO:tasks.workunit.client.1.smithi138.stdout:9/247: chown d4/d8/l2e 6 1 2024-04-25T01:36:01.359 INFO:tasks.workunit.client.1.smithi138.stdout:7/265: mknod da/de/d2e/c58 0 2024-04-25T01:36:01.360 INFO:tasks.workunit.client.0.smithi033.stdout:0/772: write d7/d2d/d43/d44/f63 [3595500,29965] 0 2024-04-25T01:36:01.361 INFO:tasks.workunit.client.1.smithi138.stdout:5/279: read d4/da/d27/f2d [51465,99725] 0 2024-04-25T01:36:01.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/248: dread - d4/d1a/d33/f36 zero size 2024-04-25T01:36:01.361 INFO:tasks.workunit.client.1.smithi138.stdout:7/266: dread - da/de/d13/f52 zero size 2024-04-25T01:36:01.362 INFO:tasks.workunit.client.1.smithi138.stdout:2/352: dwrite d0/d32/d35/f60 [0,4194304] 0 2024-04-25T01:36:01.362 INFO:tasks.workunit.client.0.smithi033.stdout:0/773: dread - d7/d2d/d4d/d9c/d68/fb9 zero size 2024-04-25T01:36:01.364 INFO:tasks.workunit.client.0.smithi033.stdout:3/867: dwrite d8/dd/d3d/d63/d7c/f73 [0,4194304] 0 2024-04-25T01:36:01.365 INFO:tasks.workunit.client.1.smithi138.stdout:2/353: dread - d0/d32/f43 zero size 2024-04-25T01:36:01.373 INFO:tasks.workunit.client.1.smithi138.stdout:0/272: dwrite d6/d1d/f24 [0,4194304] 0 2024-04-25T01:36:01.376 INFO:tasks.workunit.client.1.smithi138.stdout:7/267: dwrite da/de/d1b/d34/f3d [0,4194304] 0 2024-04-25T01:36:01.376 INFO:tasks.workunit.client.1.smithi138.stdout:5/280: rename d4/da/d27/d1c/f3d to d4/da/d27/d1c/d3c/d40/f5a 0 2024-04-25T01:36:01.381 INFO:tasks.workunit.client.1.smithi138.stdout:9/249: rmdir d4/d1a/d33 39 2024-04-25T01:36:01.382 INFO:tasks.workunit.client.1.smithi138.stdout:0/273: rmdir d6/d1d 39 2024-04-25T01:36:01.385 INFO:tasks.workunit.client.1.smithi138.stdout:7/268: mkdir da/de/d13/d30/d59 0 2024-04-25T01:36:01.387 INFO:tasks.workunit.client.0.smithi033.stdout:7/826: sync 2024-04-25T01:36:01.389 INFO:tasks.workunit.client.0.smithi033.stdout:7/827: chown d4/d2e/d6b/f9a 11606 1 2024-04-25T01:36:01.390 INFO:tasks.workunit.client.1.smithi138.stdout:2/354: rename d0/d5a/f12 to d0/d32/d35/f69 0 2024-04-25T01:36:01.392 INFO:tasks.workunit.client.1.smithi138.stdout:7/269: dwrite da/de/d2e/f43 [0,4194304] 0 2024-04-25T01:36:01.395 INFO:tasks.workunit.client.1.smithi138.stdout:0/274: rmdir d6/d2f/d31 39 2024-04-25T01:36:01.399 INFO:tasks.workunit.client.1.smithi138.stdout:5/281: fsync d4/da/d27/f1f 0 2024-04-25T01:36:01.402 INFO:tasks.workunit.client.1.smithi138.stdout:7/270: symlink da/de/d1b/d34/l5a 0 2024-04-25T01:36:01.403 INFO:tasks.workunit.client.1.smithi138.stdout:8/475: truncate d3/d16/d88/da/db/d15/f30 1416440 0 2024-04-25T01:36:01.405 INFO:tasks.workunit.client.0.smithi033.stdout:5/863: write d1c/f51 [2342329,123596] 0 2024-04-25T01:36:01.405 INFO:tasks.workunit.client.1.smithi138.stdout:2/355: mknod d0/d5a/d15/d51/c6a 0 2024-04-25T01:36:01.406 INFO:tasks.workunit.client.0.smithi033.stdout:5/864: dread - d1c/d2a/d50/d52/da6/f39 zero size 2024-04-25T01:36:01.408 INFO:tasks.workunit.client.0.smithi033.stdout:7/828: fdatasync d4/d2e/d6b/f47 0 2024-04-25T01:36:01.408 INFO:tasks.workunit.client.1.smithi138.stdout:4/169: write d0/d16/f18 [2053655,539] 0 2024-04-25T01:36:01.409 INFO:tasks.workunit.client.1.smithi138.stdout:7/271: write da/de/d1b/d34/f47 [3884478,77566] 0 2024-04-25T01:36:01.410 INFO:tasks.workunit.client.0.smithi033.stdout:3/868: mknod d8/dd/d11/c131 0 2024-04-25T01:36:01.410 INFO:tasks.workunit.client.1.smithi138.stdout:7/272: write da/de/d13/d3e/f4c [272245,729] 0 2024-04-25T01:36:01.412 INFO:tasks.workunit.client.1.smithi138.stdout:1/271: sync 2024-04-25T01:36:01.416 INFO:tasks.workunit.client.0.smithi033.stdout:8/805: dwrite d6/d45/d52/f3b [0,4194304] 0 2024-04-25T01:36:01.417 INFO:tasks.workunit.client.1.smithi138.stdout:2/356: dwrite d0/d5a/f48 [0,4194304] 0 2024-04-25T01:36:01.419 INFO:tasks.workunit.client.1.smithi138.stdout:7/273: rename da/de/d2e to da/de/d1b/d4b/d5b 0 2024-04-25T01:36:01.419 INFO:tasks.workunit.client.0.smithi033.stdout:7/829: symlink d4/d2e/d65/dcb/l102 0 2024-04-25T01:36:01.420 INFO:tasks.workunit.client.0.smithi033.stdout:7/830: fdatasync d4/d2e/d6b/f56 0 2024-04-25T01:36:01.423 INFO:tasks.workunit.client.0.smithi033.stdout:8/806: symlink d6/d45/d51/d5d/d11d/l12c 0 2024-04-25T01:36:01.424 INFO:tasks.workunit.client.1.smithi138.stdout:7/274: dread f9 [0,4194304] 0 2024-04-25T01:36:01.429 INFO:tasks.workunit.client.0.smithi033.stdout:7/831: rename d4/d2e/d7f/c85 to d4/d23/d3f/d45/d6c/c103 0 2024-04-25T01:36:01.431 INFO:tasks.workunit.client.1.smithi138.stdout:9/250: write d4/d8/f1f [3724576,37505] 0 2024-04-25T01:36:01.434 INFO:tasks.workunit.client.0.smithi033.stdout:7/832: mknod d4/d2e/d65/df5/c104 0 2024-04-25T01:36:01.436 INFO:tasks.workunit.client.0.smithi033.stdout:2/820: dwrite da/d50/ff2 [0,4194304] 0 2024-04-25T01:36:01.436 INFO:tasks.workunit.client.1.smithi138.stdout:5/282: getdents d4/da/d27/d1c/d3c/d40 0 2024-04-25T01:36:01.437 INFO:tasks.workunit.client.0.smithi033.stdout:0/774: truncate d7/d2d/f67 765105 0 2024-04-25T01:36:01.438 INFO:tasks.workunit.client.0.smithi033.stdout:0/775: readlink d7/d2d/l74 0 2024-04-25T01:36:01.439 INFO:tasks.workunit.client.0.smithi033.stdout:2/821: write da/d27/f7a [3112585,25109] 0 2024-04-25T01:36:01.439 INFO:tasks.workunit.client.1.smithi138.stdout:5/283: creat d4/da/d27/d1c/f5b x:0 0 0 2024-04-25T01:36:01.439 INFO:tasks.workunit.client.0.smithi033.stdout:5/865: write d1c/d2a/d50/d52/da6/f6d [1695561,67930] 0 2024-04-25T01:36:01.441 INFO:tasks.workunit.client.0.smithi033.stdout:3/869: write d8/dd/d3d/d63/d7c/d9d/fb6 [1047314,126011] 0 2024-04-25T01:36:01.442 INFO:tasks.workunit.client.0.smithi033.stdout:0/776: dread - d7/d2d/d9b/fef zero size 2024-04-25T01:36:01.443 INFO:tasks.workunit.client.0.smithi033.stdout:8/807: dread d6/f41 [0,4194304] 0 2024-04-25T01:36:01.444 INFO:tasks.workunit.client.1.smithi138.stdout:8/476: truncate d3/d16/d88/da/db/d15/f30 1255527 0 2024-04-25T01:36:01.445 INFO:tasks.workunit.client.0.smithi033.stdout:8/808: fdatasync d6/db/f46 0 2024-04-25T01:36:01.446 INFO:tasks.workunit.client.0.smithi033.stdout:7/833: dread d4/d57/db7/fdc [0,4194304] 0 2024-04-25T01:36:01.446 INFO:tasks.workunit.client.0.smithi033.stdout:0/777: unlink d7/d33/f100 0 2024-04-25T01:36:01.446 INFO:tasks.workunit.client.0.smithi033.stdout:3/870: chown d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/daa/f100 409 1 2024-04-25T01:36:01.447 INFO:tasks.workunit.client.0.smithi033.stdout:5/866: creat d1c/d2a/d50/d52/da6/d3e/d70/dae/f11a x:0 0 0 2024-04-25T01:36:01.448 INFO:tasks.workunit.client.0.smithi033.stdout:0/778: read d7/d2d/d43/dc0/fc3 [3828025,1436] 0 2024-04-25T01:36:01.449 INFO:tasks.workunit.client.1.smithi138.stdout:1/272: dwrite d3/db/f14 [0,4194304] 0 2024-04-25T01:36:01.450 INFO:tasks.workunit.client.0.smithi033.stdout:7/834: truncate d4/d2e/d6d/fce 553635 0 2024-04-25T01:36:01.450 INFO:tasks.workunit.client.0.smithi033.stdout:3/871: creat d8/dff/f132 x:0 0 0 2024-04-25T01:36:01.451 INFO:tasks.workunit.client.0.smithi033.stdout:0/779: dread - d7/d2d/d4d/d9c/d61/fe1 zero size 2024-04-25T01:36:01.453 INFO:tasks.workunit.client.0.smithi033.stdout:5/867: fsync d1c/d53/f57 0 2024-04-25T01:36:01.455 INFO:tasks.workunit.client.1.smithi138.stdout:7/275: dread da/de/d13/f2d [0,4194304] 0 2024-04-25T01:36:01.455 INFO:tasks.workunit.client.1.smithi138.stdout:8/477: link d3/d16/d88/da/db/d15/l42 d3/d16/d1f/d87/l93 0 2024-04-25T01:36:01.457 INFO:tasks.workunit.client.1.smithi138.stdout:7/276: stat da/de/d1b/d34/d50 0 2024-04-25T01:36:01.457 INFO:tasks.workunit.client.1.smithi138.stdout:8/478: write d3/d16/d1f/d2a/f8b [3404451,63278] 0 2024-04-25T01:36:01.457 INFO:tasks.workunit.client.0.smithi033.stdout:8/809: dwrite d6/db/f18 [0,4194304] 0 2024-04-25T01:36:01.457 INFO:tasks.workunit.client.0.smithi033.stdout:5/868: dread - d1c/d2a/d50/d52/da6/f39 zero size 2024-04-25T01:36:01.458 INFO:tasks.workunit.client.1.smithi138.stdout:7/277: fsync da/de/d1b/f33 0 2024-04-25T01:36:01.458 INFO:tasks.workunit.client.0.smithi033.stdout:7/835: creat d4/d2e/d6b/d101/f105 x:0 0 0 2024-04-25T01:36:01.460 INFO:tasks.workunit.client.1.smithi138.stdout:8/479: mkdir d3/d16/d88/da/db/d15/d56/d94 0 2024-04-25T01:36:01.461 INFO:tasks.workunit.client.1.smithi138.stdout:7/278: creat da/de/d13/d3e/f5c x:0 0 0 2024-04-25T01:36:01.462 INFO:tasks.workunit.client.1.smithi138.stdout:8/480: mkdir d3/d95 0 2024-04-25T01:36:01.463 INFO:tasks.workunit.client.1.smithi138.stdout:7/279: symlink da/de/d13/d21/l5d 0 2024-04-25T01:36:01.463 INFO:tasks.workunit.client.0.smithi033.stdout:3/872: symlink d8/dd/d3d/d63/d7c/ded/dc7/ddf/d10b/l133 0 2024-04-25T01:36:01.464 INFO:tasks.workunit.client.0.smithi033.stdout:0/780: mknod d7/d2d/d4d/d9c/db2/c10c 0 2024-04-25T01:36:01.466 INFO:tasks.workunit.client.0.smithi033.stdout:2/822: link da/d27/d31/d40/db1/laf da/d27/d31/d40/d8a/dbb/l108 0 2024-04-25T01:36:01.467 INFO:tasks.workunit.client.0.smithi033.stdout:8/810: dwrite d6/d45/f57 [0,4194304] 0 2024-04-25T01:36:01.467 INFO:tasks.workunit.client.0.smithi033.stdout:2/823: stat da/f42 0 2024-04-25T01:36:01.467 INFO:tasks.workunit.client.1.smithi138.stdout:2/357: dread d0/d18/f1b [0,4194304] 0 2024-04-25T01:36:01.469 INFO:tasks.workunit.client.1.smithi138.stdout:8/481: creat d3/d16/d88/da/db/f96 x:0 0 0 2024-04-25T01:36:01.470 INFO:tasks.workunit.client.1.smithi138.stdout:7/280: dread da/de/d13/f3c [0,4194304] 0 2024-04-25T01:36:01.470 INFO:tasks.workunit.client.1.smithi138.stdout:8/482: write d3/d16/d1f/d50/f7c [120926,39865] 0 2024-04-25T01:36:01.470 INFO:tasks.workunit.client.0.smithi033.stdout:5/869: rmdir d1c/d2a/d50/d52 39 2024-04-25T01:36:01.472 INFO:tasks.workunit.client.1.smithi138.stdout:2/358: creat d0/d32/f6b x:0 0 0 2024-04-25T01:36:01.472 INFO:tasks.workunit.client.1.smithi138.stdout:2/359: chown d0/fe 463788 1 2024-04-25T01:36:01.473 INFO:tasks.workunit.client.0.smithi033.stdout:7/836: unlink d4/d2e/d6b/d67/f7d 0 2024-04-25T01:36:01.474 INFO:tasks.workunit.client.1.smithi138.stdout:7/281: symlink da/de/l5e 0 2024-04-25T01:36:01.474 INFO:tasks.workunit.client.0.smithi033.stdout:0/781: mkdir d7/de7/d10d 0 2024-04-25T01:36:01.475 INFO:tasks.workunit.client.1.smithi138.stdout:7/282: dread - da/de/d13/d3e/f5c zero size 2024-04-25T01:36:01.475 INFO:tasks.workunit.client.1.smithi138.stdout:1/273: fdatasync d3/db/f14 0 2024-04-25T01:36:01.476 INFO:tasks.workunit.client.0.smithi033.stdout:2/824: mkdir da/d27/d31/d40/d109 0 2024-04-25T01:36:01.479 INFO:tasks.workunit.client.0.smithi033.stdout:5/870: chown d1c/d2a/d50/d8d/de1/fe0 100379 1 2024-04-25T01:36:01.480 INFO:tasks.workunit.client.0.smithi033.stdout:8/811: symlink d6/d45/d51/d5d/db9/l12d 0 2024-04-25T01:36:01.480 INFO:tasks.workunit.client.0.smithi033.stdout:7/837: fdatasync d4/d2e/d6b/f47 0 2024-04-25T01:36:01.481 INFO:tasks.workunit.client.1.smithi138.stdout:8/483: creat d3/d16/d88/da/db/d15/d59/f97 x:0 0 0 2024-04-25T01:36:01.481 INFO:tasks.workunit.client.0.smithi033.stdout:7/838: stat d4/d23/da7/dc2/fd6 0 2024-04-25T01:36:01.482 INFO:tasks.workunit.client.0.smithi033.stdout:0/782: mknod d7/d2d/d4d/d9c/d68/dd2/c10e 0 2024-04-25T01:36:01.482 INFO:tasks.workunit.client.1.smithi138.stdout:2/360: link d0/d18/f2a d0/d18/f6c 0 2024-04-25T01:36:01.484 INFO:tasks.workunit.client.1.smithi138.stdout:7/283: dwrite da/de/d1b/f33 [0,4194304] 0 2024-04-25T01:36:01.486 INFO:tasks.workunit.client.0.smithi033.stdout:8/812: chown d6/d45/d52/d2e/f30 25545 1 2024-04-25T01:36:01.487 INFO:tasks.workunit.client.1.smithi138.stdout:1/274: getdents d3/db/d2b 0 2024-04-25T01:36:01.488 INFO:tasks.workunit.client.0.smithi033.stdout:0/783: creat d7/d2d/d64/f10f x:0 0 0 2024-04-25T01:36:01.489 INFO:tasks.workunit.client.1.smithi138.stdout:7/284: rmdir da/de/d1b/d4b/d5b 39 2024-04-25T01:36:01.489 INFO:tasks.workunit.client.1.smithi138.stdout:2/361: mkdir d0/d5a/d15/d44/d4c/d56/d6d 0 2024-04-25T01:36:01.491 INFO:tasks.workunit.client.1.smithi138.stdout:7/285: truncate da/de/f29 1320690 0 2024-04-25T01:36:01.491 INFO:tasks.workunit.client.1.smithi138.stdout:1/275: fsync d3/d27/d24/f3b 0 2024-04-25T01:36:01.492 INFO:tasks.workunit.client.0.smithi033.stdout:2/825: rename da/d5b/d5e/f6f to da/d5b/d5e/da8/f10a 0 2024-04-25T01:36:01.493 INFO:tasks.workunit.client.0.smithi033.stdout:2/826: chown da/d2c/dcd 3016 1 2024-04-25T01:36:01.494 INFO:tasks.workunit.client.1.smithi138.stdout:2/362: mkdir d0/d5a/d15/d44/d4c/d56/d6d/d6e 0 2024-04-25T01:36:01.496 INFO:tasks.workunit.client.1.smithi138.stdout:1/276: getdents d3/db/d2b/d3d 0 2024-04-25T01:36:01.496 INFO:tasks.workunit.client.1.smithi138.stdout:7/286: dread da/de/d13/d3e/f4c [0,4194304] 0 2024-04-25T01:36:01.498 INFO:tasks.workunit.client.0.smithi033.stdout:7/839: rename d4/d23/l27 to d4/d2e/d6b/d1f/d78/d86/de4/dfc/l106 0 2024-04-25T01:36:01.499 INFO:tasks.workunit.client.1.smithi138.stdout:2/363: write d0/f10 [2283692,5177] 0 2024-04-25T01:36:01.501 INFO:tasks.workunit.client.0.smithi033.stdout:2/827: mkdir da/d50/d6a/d43/d6c/d10b 0 2024-04-25T01:36:01.501 INFO:tasks.workunit.client.0.smithi033.stdout:7/840: dread d4/d2e/d6d/fce [0,4194304] 0 2024-04-25T01:36:01.502 INFO:tasks.workunit.client.0.smithi033.stdout:2/828: stat da/d5b/d72/cc0 0 2024-04-25T01:36:01.503 INFO:tasks.workunit.client.1.smithi138.stdout:2/364: read d0/d5a/d15/f28 [409356,46641] 0 2024-04-25T01:36:01.503 INFO:tasks.workunit.client.0.smithi033.stdout:3/873: dread d8/dd/d40/f43 [0,4194304] 0 2024-04-25T01:36:01.507 INFO:tasks.workunit.client.0.smithi033.stdout:7/841: mknod d4/d2e/d65/dcb/c107 0 2024-04-25T01:36:01.507 INFO:tasks.workunit.client.1.smithi138.stdout:8/484: dread d3/d16/d88/f60 [0,4194304] 0 2024-04-25T01:36:01.508 INFO:tasks.workunit.client.0.smithi033.stdout:5/871: link d1c/d2a/d50/d52/da6/lbc d1c/l11b 0 2024-04-25T01:36:01.509 INFO:tasks.workunit.client.0.smithi033.stdout:0/784: dread d7/da/df/d14/f69 [0,4194304] 0 2024-04-25T01:36:01.509 INFO:tasks.workunit.client.1.smithi138.stdout:7/287: dwrite da/de/d1b/d4b/f55 [0,4194304] 0 2024-04-25T01:36:01.510 INFO:tasks.workunit.client.1.smithi138.stdout:1/277: dwrite d3/d27/d16/f18 [4194304,4194304] 0 2024-04-25T01:36:01.510 INFO:tasks.workunit.client.1.smithi138.stdout:2/365: creat d0/d5a/d15/d51/f6f x:0 0 0 2024-04-25T01:36:01.510 INFO:tasks.workunit.client.0.smithi033.stdout:3/874: mkdir d8/dd/d3d/d63/d7c/d9d/de8/d134 0 2024-04-25T01:36:01.513 INFO:tasks.workunit.client.1.smithi138.stdout:8/485: chown d3/d16/d88/da/db/d15/d56/d8f 147 1 2024-04-25T01:36:01.515 INFO:tasks.workunit.client.1.smithi138.stdout:7/288: truncate da/de/d13/f52 543335 0 2024-04-25T01:36:01.519 INFO:tasks.workunit.client.0.smithi033.stdout:3/875: rmdir d8/dd/d3d/d4f/da4/dc4 39 2024-04-25T01:36:01.520 INFO:tasks.workunit.client.1.smithi138.stdout:1/278: link d3/d27/d16/c33 d3/db/d4a/c59 0 2024-04-25T01:36:01.520 INFO:tasks.workunit.client.1.smithi138.stdout:7/289: symlink da/de/d13/d30/l5f 0 2024-04-25T01:36:01.520 INFO:tasks.workunit.client.1.smithi138.stdout:8/486: mkdir d3/d16/d88/d98 0 2024-04-25T01:36:01.521 INFO:tasks.workunit.client.1.smithi138.stdout:8/487: chown d3/l73 4840 1 2024-04-25T01:36:01.521 INFO:tasks.workunit.client.1.smithi138.stdout:1/279: unlink c1 0 2024-04-25T01:36:01.523 INFO:tasks.workunit.client.0.smithi033.stdout:4/889: sync 2024-04-25T01:36:01.523 INFO:tasks.workunit.client.0.smithi033.stdout:6/952: sync 2024-04-25T01:36:01.523 INFO:tasks.workunit.client.0.smithi033.stdout:1/684: sync 2024-04-25T01:36:01.523 INFO:tasks.workunit.client.0.smithi033.stdout:9/833: sync 2024-04-25T01:36:01.523 INFO:tasks.workunit.client.0.smithi033.stdout:8/813: write d6/d45/d51/d5d/f6f [893315,34789] 0 2024-04-25T01:36:01.524 INFO:tasks.workunit.client.1.smithi138.stdout:7/290: rename da/de/d1b/l40 to da/de/d1b/d34/l60 0 2024-04-25T01:36:01.524 INFO:tasks.workunit.client.0.smithi033.stdout:3/876: chown d8/dd/d3d/d63/d7c/ded/dc7/ddf/cea 5 1 2024-04-25T01:36:01.525 INFO:tasks.workunit.client.1.smithi138.stdout:7/291: stat da/de/c32 0 2024-04-25T01:36:01.525 INFO:tasks.workunit.client.1.smithi138.stdout:1/280: chown d3/d4/c29 53 1 2024-04-25T01:36:01.526 INFO:tasks.workunit.client.0.smithi033.stdout:7/842: dread d4/f3d [0,4194304] 0 2024-04-25T01:36:01.526 INFO:tasks.workunit.client.1.smithi138.stdout:8/488: mknod d3/d16/d88/d98/c99 0 2024-04-25T01:36:01.527 INFO:tasks.workunit.client.0.smithi033.stdout:6/953: readlink d4/d24/l27 0 2024-04-25T01:36:01.527 INFO:tasks.workunit.client.1.smithi138.stdout:7/292: symlink da/de/d13/l61 0 2024-04-25T01:36:01.529 INFO:tasks.workunit.client.1.smithi138.stdout:1/281: rmdir d3/d27/d24 39 2024-04-25T01:36:01.531 INFO:tasks.workunit.client.0.smithi033.stdout:5/872: dread d1c/d2a/d50/d5b/d86/f7c [0,4194304] 0 2024-04-25T01:36:01.532 INFO:tasks.workunit.client.0.smithi033.stdout:1/685: mkdir d2/d7f/db0/dd6 0 2024-04-25T01:36:01.533 INFO:tasks.workunit.client.1.smithi138.stdout:1/282: dread d3/db/f14 [0,4194304] 0 2024-04-25T01:36:01.534 INFO:tasks.workunit.client.1.smithi138.stdout:7/293: link da/l56 da/de/d13/d30/l62 0 2024-04-25T01:36:01.535 INFO:tasks.workunit.client.0.smithi033.stdout:2/829: dread da/f13 [0,4194304] 0 2024-04-25T01:36:01.536 INFO:tasks.workunit.client.0.smithi033.stdout:9/834: fdatasync d0/d2/d43/f3f 0 2024-04-25T01:36:01.537 INFO:tasks.workunit.client.1.smithi138.stdout:7/294: creat da/f63 x:0 0 0 2024-04-25T01:36:01.538 INFO:tasks.workunit.client.1.smithi138.stdout:7/295: chown da/de/d13 6439667 1 2024-04-25T01:36:01.539 INFO:tasks.workunit.client.0.smithi033.stdout:8/814: read - d6/d45/d52/d2e/d87/fd5 zero size 2024-04-25T01:36:01.540 INFO:tasks.workunit.client.0.smithi033.stdout:6/954: rename d4/d24/d36/dbd/ce3 to d4/d24/d36/d42/d97/db2/c140 0 2024-04-25T01:36:01.543 INFO:tasks.workunit.client.0.smithi033.stdout:7/843: unlink d4/d2e/daa/cde 0 2024-04-25T01:36:01.547 INFO:tasks.workunit.client.1.smithi138.stdout:2/366: dwrite d0/d18/f1b [0,4194304] 0 2024-04-25T01:36:01.547 INFO:tasks.workunit.client.1.smithi138.stdout:8/489: truncate d3/d16/d88/da/f24 2772315 0 2024-04-25T01:36:01.548 INFO:tasks.workunit.client.0.smithi033.stdout:6/955: chown d4/d1d/d3e/d41/d54/da1/f138 100 1 2024-04-25T01:36:01.548 INFO:tasks.workunit.client.1.smithi138.stdout:7/296: dwrite da/de/d1b/d4b/d5b/f43 [0,4194304] 0 2024-04-25T01:36:01.549 INFO:tasks.workunit.client.0.smithi033.stdout:7/844: read - d4/d57/db7/f100 zero size 2024-04-25T01:36:01.550 INFO:tasks.workunit.client.0.smithi033.stdout:3/877: rename d8/dd/d11/ca6 to d8/dd/d3d/d63/d7c/d9d/de8/d108/c135 0 2024-04-25T01:36:01.552 INFO:tasks.workunit.client.0.smithi033.stdout:1/686: dwrite d2/d5/d1b/d46/d5f/d63/d8f/f9e [0,4194304] 0 2024-04-25T01:36:01.554 INFO:tasks.workunit.client.1.smithi138.stdout:1/283: dread d3/d27/f1e [0,4194304] 0 2024-04-25T01:36:01.560 INFO:tasks.workunit.client.1.smithi138.stdout:8/490: link d3/d16/d88/da/db/d15/f21 d3/d16/d88/da/db/d15/d5d/f9a 0 2024-04-25T01:36:01.560 INFO:tasks.workunit.client.1.smithi138.stdout:2/367: read d0/d5a/d15/d1f/f37 [199648,77632] 0 2024-04-25T01:36:01.561 INFO:tasks.workunit.client.1.smithi138.stdout:2/368: chown d0/d18/f1b 54 1 2024-04-25T01:36:01.562 INFO:tasks.workunit.client.1.smithi138.stdout:1/284: link d3/db/fe d3/db/f5a 0 2024-04-25T01:36:01.564 INFO:tasks.workunit.client.0.smithi033.stdout:5/873: dwrite d1c/d2a/d50/d52/da6/ddc/f10a [0,4194304] 0 2024-04-25T01:36:01.564 INFO:tasks.workunit.client.0.smithi033.stdout:8/815: symlink d6/dd3/dfb/l12e 0 2024-04-25T01:36:01.564 INFO:tasks.workunit.client.1.smithi138.stdout:8/491: creat d3/d16/f9b x:0 0 0 2024-04-25T01:36:01.564 INFO:tasks.workunit.client.1.smithi138.stdout:1/285: write d3/d4/fd [3052442,73763] 0 2024-04-25T01:36:01.565 INFO:tasks.workunit.client.0.smithi033.stdout:7/845: fsync f0 0 2024-04-25T01:36:01.566 INFO:tasks.workunit.client.0.smithi033.stdout:6/956: write d4/d1d/d3e/d41/d5c/d7f/da4/ffe [71164,122056] 0 2024-04-25T01:36:01.568 INFO:tasks.workunit.client.1.smithi138.stdout:7/297: write da/de/d13/d21/f41 [401199,105614] 0 2024-04-25T01:36:01.568 INFO:tasks.workunit.client.1.smithi138.stdout:2/369: rename d0/f14 to d0/d32/d35/f70 0 2024-04-25T01:36:01.571 INFO:tasks.workunit.client.0.smithi033.stdout:0/785: dwrite d7/da/f48 [4194304,4194304] 0 2024-04-25T01:36:01.571 INFO:tasks.workunit.client.0.smithi033.stdout:2/830: write da/d5b/d5e/da8/f10a [1447491,21725] 0 2024-04-25T01:36:01.571 INFO:tasks.workunit.client.1.smithi138.stdout:2/370: write d0/d5a/d15/f5d [1573078,12748] 0 2024-04-25T01:36:01.572 INFO:tasks.workunit.client.1.smithi138.stdout:1/286: rename d3/d45/c49 to d3/db/c5b 0 2024-04-25T01:36:01.573 INFO:tasks.workunit.client.0.smithi033.stdout:2/831: fdatasync da/fd 0 2024-04-25T01:36:01.573 INFO:tasks.workunit.client.0.smithi033.stdout:3/878: stat d8/dd/d3d/d63/d7c/la2 0 2024-04-25T01:36:01.574 INFO:tasks.workunit.client.0.smithi033.stdout:5/874: read - d1c/d2a/f73 zero size 2024-04-25T01:36:01.574 INFO:tasks.workunit.client.1.smithi138.stdout:1/287: chown d3/db/c11 7620 1 2024-04-25T01:36:01.575 INFO:tasks.workunit.client.0.smithi033.stdout:9/835: dwrite d0/d2/dd/d20/d4e/f90 [0,4194304] 0 2024-04-25T01:36:01.575 INFO:tasks.workunit.client.1.smithi138.stdout:7/298: dread da/de/d13/f2d [0,4194304] 0 2024-04-25T01:36:01.576 INFO:tasks.workunit.client.0.smithi033.stdout:2/832: write da/d27/d31/d40/db1/dbf/dd5/ff9 [687628,105242] 0 2024-04-25T01:36:01.577 INFO:tasks.workunit.client.1.smithi138.stdout:8/492: truncate f0 2251454 0 2024-04-25T01:36:01.578 INFO:tasks.workunit.client.0.smithi033.stdout:8/816: symlink d6/db/d111/l12f 0 2024-04-25T01:36:01.580 INFO:tasks.workunit.client.0.smithi033.stdout:0/786: fsync d7/d2d/d64/f6d 0 2024-04-25T01:36:01.581 INFO:tasks.workunit.client.0.smithi033.stdout:3/879: read d8/dd/d1d/f7e [527715,120930] 0 2024-04-25T01:36:01.583 INFO:tasks.workunit.client.1.smithi138.stdout:1/288: creat d3/d4/d47/f5c x:0 0 0 2024-04-25T01:36:01.584 INFO:tasks.workunit.client.0.smithi033.stdout:2/833: readlink da/d50/d6a/d43/d6c/lb9 0 2024-04-25T01:36:01.586 INFO:tasks.workunit.client.1.smithi138.stdout:8/493: dread d3/d16/d88/da/db/d15/d5d/f62 [0,4194304] 0 2024-04-25T01:36:01.586 INFO:tasks.workunit.client.1.smithi138.stdout:2/371: fsync d0/d5a/d15/f5d 0 2024-04-25T01:36:01.590 INFO:tasks.workunit.client.0.smithi033.stdout:1/687: dwrite d2/d5/d75/d59/d83/fd0 [0,4194304] 0 2024-04-25T01:36:01.591 INFO:tasks.workunit.client.1.smithi138.stdout:7/299: unlink da/de/d13/d3e/f5c 0 2024-04-25T01:36:01.591 INFO:tasks.workunit.client.0.smithi033.stdout:3/880: stat d8/dd/d3d/d63/d7c/ded/dc7/ddf/f112 0 2024-04-25T01:36:01.594 INFO:tasks.workunit.client.0.smithi033.stdout:6/957: truncate d4/d24/d36/d42/f83 4383216 0 2024-04-25T01:36:01.595 INFO:tasks.workunit.client.1.smithi138.stdout:7/300: write da/de/d1b/d4b/d5b/f46 [380893,121444] 0 2024-04-25T01:36:01.596 INFO:tasks.workunit.client.0.smithi033.stdout:0/787: creat d7/d2d/d43/d44/f110 x:0 0 0 2024-04-25T01:36:01.596 INFO:tasks.workunit.client.1.smithi138.stdout:1/289: creat d3/db/d2b/f5d x:0 0 0 2024-04-25T01:36:01.597 INFO:tasks.workunit.client.1.smithi138.stdout:2/372: rmdir d0/d32/d35/d4f 39 2024-04-25T01:36:01.599 INFO:tasks.workunit.client.1.smithi138.stdout:8/494: dwrite d3/d16/d88/da/db/f96 [0,4194304] 0 2024-04-25T01:36:01.602 INFO:tasks.workunit.client.1.smithi138.stdout:7/301: mknod da/de/d1b/d4b/d5b/c64 0 2024-04-25T01:36:01.602 INFO:tasks.workunit.client.1.smithi138.stdout:8/495: fsync d3/d16/d1f/d87/f91 0 2024-04-25T01:36:01.603 INFO:tasks.workunit.client.1.smithi138.stdout:2/373: symlink d0/d32/l71 0 2024-04-25T01:36:01.603 INFO:tasks.workunit.client.1.smithi138.stdout:2/374: readlink d0/l20 0 2024-04-25T01:36:01.603 INFO:tasks.workunit.client.1.smithi138.stdout:1/290: link d3/db/f14 d3/db/d2b/d3d/f5e 0 2024-04-25T01:36:01.605 INFO:tasks.workunit.client.1.smithi138.stdout:8/496: creat d3/d16/d88/da/db/d15/d56/f9c x:0 0 0 2024-04-25T01:36:01.606 INFO:tasks.workunit.client.1.smithi138.stdout:7/302: creat da/de/d13/d30/f65 x:0 0 0 2024-04-25T01:36:01.607 INFO:tasks.workunit.client.1.smithi138.stdout:8/497: chown d3/d16/d88/da/db/d15/d20/f5a 423256093 1 2024-04-25T01:36:01.607 INFO:tasks.workunit.client.0.smithi033.stdout:2/834: mkdir da/d8d/d10c 0 2024-04-25T01:36:01.607 INFO:tasks.workunit.client.0.smithi033.stdout:6/958: unlink d4/d24/d7c/f133 0 2024-04-25T01:36:01.609 INFO:tasks.workunit.client.1.smithi138.stdout:1/291: creat d3/db/f5f x:0 0 0 2024-04-25T01:36:01.611 INFO:tasks.workunit.client.1.smithi138.stdout:2/375: rename d0/d5a/d15/d1f/f2d to d0/d5a/d15/d51/f72 0 2024-04-25T01:36:01.613 INFO:tasks.workunit.client.0.smithi033.stdout:9/836: write d0/d2/dd/d5f/f9b [960772,90907] 0 2024-04-25T01:36:01.613 INFO:tasks.workunit.client.1.smithi138.stdout:7/303: mknod da/de/d13/d30/c66 0 2024-04-25T01:36:01.614 INFO:tasks.workunit.client.0.smithi033.stdout:7/846: write d4/d2e/d6b/f52 [936930,48497] 0 2024-04-25T01:36:01.617 INFO:tasks.workunit.client.0.smithi033.stdout:7/847: readlink d4/d2e/l99 0 2024-04-25T01:36:01.618 INFO:tasks.workunit.client.1.smithi138.stdout:2/376: mknod d0/d5a/d15/d44/d4c/c73 0 2024-04-25T01:36:01.619 INFO:tasks.workunit.client.0.smithi033.stdout:5/875: write d1c/d2a/d50/d52/fb3 [1117916,89878] 0 2024-04-25T01:36:01.620 INFO:tasks.workunit.client.0.smithi033.stdout:8/817: dwrite d6/f15 [0,4194304] 0 2024-04-25T01:36:01.620 INFO:tasks.workunit.client.1.smithi138.stdout:8/498: truncate d3/d16/d88/da/db/d15/f21 1925324 0 2024-04-25T01:36:01.622 INFO:tasks.workunit.client.1.smithi138.stdout:8/499: write d3/d16/d88/da/db/d15/d56/f9c [668302,67678] 0 2024-04-25T01:36:01.622 INFO:tasks.workunit.client.1.smithi138.stdout:8/500: readlink d3/d16/d88/l75 0 2024-04-25T01:36:01.624 INFO:tasks.workunit.client.1.smithi138.stdout:3/417: sync 2024-04-25T01:36:01.624 INFO:tasks.workunit.client.1.smithi138.stdout:6/394: sync 2024-04-25T01:36:01.625 INFO:tasks.workunit.client.1.smithi138.stdout:7/304: link da/de/d1b/d34/f47 da/de/d1b/d4b/d5b/f67 0 2024-04-25T01:36:01.626 INFO:tasks.workunit.client.1.smithi138.stdout:3/418: write db/df/d17/d5d/f78 [520305,129434] 0 2024-04-25T01:36:01.627 INFO:tasks.workunit.client.0.smithi033.stdout:7/848: dwrite d4/d23/d3f/f6f [4194304,4194304] 0 2024-04-25T01:36:01.628 INFO:tasks.workunit.client.1.smithi138.stdout:3/419: stat db/df/d19/f70 0 2024-04-25T01:36:01.629 INFO:tasks.workunit.client.1.smithi138.stdout:6/395: rename dc/d10/d28/c4f to dc/d2e/d55/c7a 0 2024-04-25T01:36:01.632 INFO:tasks.workunit.client.0.smithi033.stdout:5/876: truncate d1c/d2a/d50/d52/da6/d3e/d70/d7a/f118 566433 0 2024-04-25T01:36:01.632 INFO:tasks.workunit.client.1.smithi138.stdout:7/305: mknod da/c68 0 2024-04-25T01:36:01.633 INFO:tasks.workunit.client.0.smithi033.stdout:8/818: dwrite d6/d45/d52/f3f [0,4194304] 0 2024-04-25T01:36:01.634 INFO:tasks.workunit.client.1.smithi138.stdout:2/377: dwrite d0/d5a/d15/d1f/f4a [0,4194304] 0 2024-04-25T01:36:01.638 INFO:tasks.workunit.client.1.smithi138.stdout:6/396: mkdir dc/d7b 0 2024-04-25T01:36:01.641 INFO:tasks.workunit.client.1.smithi138.stdout:7/306: dread da/de/d13/f52 [0,4194304] 0 2024-04-25T01:36:01.644 INFO:tasks.workunit.client.1.smithi138.stdout:1/292: truncate d3/d27/d16/f28 134395 0 2024-04-25T01:36:01.644 INFO:tasks.workunit.client.1.smithi138.stdout:6/397: write dc/d10/d28/f48 [1500952,114114] 0 2024-04-25T01:36:01.645 INFO:tasks.workunit.client.1.smithi138.stdout:1/293: chown d3/d4/d47 59004500 1 2024-04-25T01:36:01.649 INFO:tasks.workunit.client.0.smithi033.stdout:1/688: mknod d2/d5/cd7 0 2024-04-25T01:36:01.649 INFO:tasks.workunit.client.1.smithi138.stdout:2/378: creat d0/d5a/d15/d44/d4c/d56/f74 x:0 0 0 2024-04-25T01:36:01.649 INFO:tasks.workunit.client.1.smithi138.stdout:2/379: readlink d0/l20 0 2024-04-25T01:36:01.650 INFO:tasks.workunit.client.0.smithi033.stdout:3/881: mknod d8/dd/d11/d102/c136 0 2024-04-25T01:36:01.652 INFO:tasks.workunit.client.0.smithi033.stdout:0/788: unlink d7/d2d/d43/f7b 0 2024-04-25T01:36:01.654 INFO:tasks.workunit.client.1.smithi138.stdout:6/398: unlink dc/d10/d67/f62 0 2024-04-25T01:36:01.656 INFO:tasks.workunit.client.1.smithi138.stdout:7/307: mknod da/de/d13/d30/d59/c69 0 2024-04-25T01:36:01.656 INFO:tasks.workunit.client.0.smithi033.stdout:2/835: mknod da/d8d/db0/d100/c10d 0 2024-04-25T01:36:01.656 INFO:tasks.workunit.client.0.smithi033.stdout:2/836: stat da/f45 0 2024-04-25T01:36:01.657 INFO:tasks.workunit.client.1.smithi138.stdout:3/420: getdents db/df/d17/d23/d34 0 2024-04-25T01:36:01.657 INFO:tasks.workunit.client.0.smithi033.stdout:2/837: write da/d27/d31/d40/fa1 [345468,70689] 0 2024-04-25T01:36:01.658 INFO:tasks.workunit.client.0.smithi033.stdout:9/837: rename d0/d2/dd/df/caf to d0/d11/da1/c126 0 2024-04-25T01:36:01.659 INFO:tasks.workunit.client.0.smithi033.stdout:9/838: readlink d0/d2/lb3 0 2024-04-25T01:36:01.659 INFO:tasks.workunit.client.1.smithi138.stdout:6/399: creat dc/d10/d6d/f7c x:0 0 0 2024-04-25T01:36:01.661 INFO:tasks.workunit.client.0.smithi033.stdout:7/849: symlink d4/d2e/d6b/d1f/d78/d86/l108 0 2024-04-25T01:36:01.663 INFO:tasks.workunit.client.1.smithi138.stdout:6/400: mknod dc/d2e/d55/c7d 0 2024-04-25T01:36:01.663 INFO:tasks.workunit.client.0.smithi033.stdout:5/877: fdatasync d1c/d2a/d50/d52/da6/d3e/d70/fb9 0 2024-04-25T01:36:01.664 INFO:tasks.workunit.client.1.smithi138.stdout:3/421: dread db/df/d17/d20/d57/d6a/f6e [0,4194304] 0 2024-04-25T01:36:01.664 INFO:tasks.workunit.client.1.smithi138.stdout:7/308: dwrite da/f63 [0,4194304] 0 2024-04-25T01:36:01.664 INFO:tasks.workunit.client.0.smithi033.stdout:1/689: symlink d2/d11/dc9/ld8 0 2024-04-25T01:36:01.665 INFO:tasks.workunit.client.1.smithi138.stdout:3/422: chown db/df/f4f 342297722 1 2024-04-25T01:36:01.665 INFO:tasks.workunit.client.0.smithi033.stdout:3/882: creat d8/dd/d3d/d63/d7c/d9d/de8/df9/f137 x:0 0 0 2024-04-25T01:36:01.666 INFO:tasks.workunit.client.0.smithi033.stdout:1/690: fsync d2/d5/d1b/d46/d5f/d63/d8f/fbc 0 2024-04-25T01:36:01.668 INFO:tasks.workunit.client.0.smithi033.stdout:6/959: creat d4/d1d/d3e/d41/d5c/df6/d12b/f141 x:0 0 0 2024-04-25T01:36:01.669 INFO:tasks.workunit.client.1.smithi138.stdout:3/423: symlink db/df/l7e 0 2024-04-25T01:36:01.669 INFO:tasks.workunit.client.0.smithi033.stdout:1/691: write d2/d5/d1b/f54 [3072669,35764] 0 2024-04-25T01:36:01.670 INFO:tasks.workunit.client.1.smithi138.stdout:6/401: dread dc/f35 [0,4194304] 0 2024-04-25T01:36:01.673 INFO:tasks.workunit.client.1.smithi138.stdout:7/309: dwrite da/de/d1b/d4b/f55 [0,4194304] 0 2024-04-25T01:36:01.674 INFO:tasks.workunit.client.0.smithi033.stdout:2/838: creat da/d50/d6a/d43/f10e x:0 0 0 2024-04-25T01:36:01.677 INFO:tasks.workunit.client.0.smithi033.stdout:8/819: rename d6/d45/d52/db0/dbc to d6/db/d80/d128/d130 0 2024-04-25T01:36:01.678 INFO:tasks.workunit.client.0.smithi033.stdout:3/883: rename d8/dd/d3d/d63 to d8/dd/d3d/d63/d7c/d9d/de8/df9/d138 22 2024-04-25T01:36:01.682 INFO:tasks.workunit.client.1.smithi138.stdout:3/424: dread db/df/d19/f75 [0,4194304] 0 2024-04-25T01:36:01.683 INFO:tasks.workunit.client.0.smithi033.stdout:4/890: sync 2024-04-25T01:36:01.684 INFO:tasks.workunit.client.1.smithi138.stdout:1/294: dwrite d3/db/fe [0,4194304] 0 2024-04-25T01:36:01.684 INFO:tasks.workunit.client.0.smithi033.stdout:7/850: write d4/d2e/d6d/f75 [4092105,100034] 0 2024-04-25T01:36:01.688 INFO:tasks.workunit.client.0.smithi033.stdout:5/878: dread d1c/d2a/d50/d52/da6/d4a/f64 [0,4194304] 0 2024-04-25T01:36:01.690 INFO:tasks.workunit.client.0.smithi033.stdout:5/879: dread - d1c/d2a/d50/d52/da6/d3e/d70/d7a/dab/ffe zero size 2024-04-25T01:36:01.690 INFO:tasks.workunit.client.0.smithi033.stdout:6/960: mknod d4/d1d/d3e/d70/c142 0 2024-04-25T01:36:01.691 INFO:tasks.workunit.client.1.smithi138.stdout:7/310: rmdir da/de/d1b/d34/d50 0 2024-04-25T01:36:01.691 INFO:tasks.workunit.client.1.smithi138.stdout:3/425: dread f6 [0,4194304] 0 2024-04-25T01:36:01.692 INFO:tasks.workunit.client.1.smithi138.stdout:7/311: stat da/de/d13/d30/l5f 0 2024-04-25T01:36:01.694 INFO:tasks.workunit.client.1.smithi138.stdout:6/402: dwrite dc/d10/f22 [0,4194304] 0 2024-04-25T01:36:01.694 INFO:tasks.workunit.client.1.smithi138.stdout:3/426: creat db/df/d17/d5d/f7f x:0 0 0 2024-04-25T01:36:01.699 INFO:tasks.workunit.client.0.smithi033.stdout:8/820: mkdir d6/d45/d51/d9e/d131 0 2024-04-25T01:36:01.699 INFO:tasks.workunit.client.1.smithi138.stdout:6/403: unlink dc/f3d 0 2024-04-25T01:36:01.701 INFO:tasks.workunit.client.0.smithi033.stdout:8/821: chown d6/d49/fee 1016736147 1 2024-04-25T01:36:01.701 INFO:tasks.workunit.client.0.smithi033.stdout:6/961: dwrite d4/d1d/d3e/d41/d93/f137 [0,4194304] 0 2024-04-25T01:36:01.703 INFO:tasks.workunit.client.1.smithi138.stdout:7/312: rename da/de/d1b/f4f to da/f6a 0 2024-04-25T01:36:01.703 INFO:tasks.workunit.client.0.smithi033.stdout:4/891: mkdir d6/d14/d35/d3a/d4d/d5e/d7c/d120 0 2024-04-25T01:36:01.704 INFO:tasks.workunit.client.0.smithi033.stdout:0/789: link d7/d2d/d4d/d9c/db2/ce0 d7/d2d/d4c/c111 0 2024-04-25T01:36:01.708 INFO:tasks.workunit.client.1.smithi138.stdout:6/404: rmdir dc/d10/d28 39 2024-04-25T01:36:01.709 INFO:tasks.workunit.client.1.smithi138.stdout:6/405: chown dc/c1a 185577661 1 2024-04-25T01:36:01.711 INFO:tasks.workunit.client.1.smithi138.stdout:6/406: read dc/d2e/f2f [3160799,60121] 0 2024-04-25T01:36:01.711 INFO:tasks.workunit.client.1.smithi138.stdout:3/427: rmdir db/d14/d65 0 2024-04-25T01:36:01.712 INFO:tasks.workunit.client.0.smithi033.stdout:2/839: creat da/d8d/d10c/f10f x:0 0 0 2024-04-25T01:36:01.713 INFO:tasks.workunit.client.1.smithi138.stdout:3/428: dread - db/df/d17/d20/f7d zero size 2024-04-25T01:36:01.714 INFO:tasks.workunit.client.1.smithi138.stdout:3/429: chown db/df/d17/d23/d34 0 1 2024-04-25T01:36:01.715 INFO:tasks.workunit.client.1.smithi138.stdout:3/430: write db/df/d17/d23/f67 [379726,57025] 0 2024-04-25T01:36:01.716 INFO:tasks.workunit.client.0.smithi033.stdout:7/851: creat d4/d57/da5/f109 x:0 0 0 2024-04-25T01:36:01.716 INFO:tasks.workunit.client.0.smithi033.stdout:4/892: dwrite d6/d14/d1d/f7a [0,4194304] 0 2024-04-25T01:36:01.717 INFO:tasks.workunit.client.0.smithi033.stdout:8/822: dwrite d6/f15 [0,4194304] 0 2024-04-25T01:36:01.719 INFO:tasks.workunit.client.0.smithi033.stdout:5/880: dread d1c/d2a/d50/d52/da6/d3e/d70/fc3 [0,4194304] 0 2024-04-25T01:36:01.723 INFO:tasks.workunit.client.1.smithi138.stdout:7/313: write da/f15 [4720374,5593] 0 2024-04-25T01:36:01.723 INFO:tasks.workunit.client.1.smithi138.stdout:3/431: dwrite db/df/d19/f70 [0,4194304] 0 2024-04-25T01:36:01.728 INFO:tasks.workunit.client.1.smithi138.stdout:3/432: creat db/df/d17/d23/d34/f80 x:0 0 0 2024-04-25T01:36:01.728 INFO:tasks.workunit.client.0.smithi033.stdout:4/893: dwrite d6/d12/f58 [12582912,4194304] 0 2024-04-25T01:36:01.729 INFO:tasks.workunit.client.0.smithi033.stdout:0/790: creat d7/d2d/d43/d44/f112 x:0 0 0 2024-04-25T01:36:01.730 INFO:tasks.workunit.client.0.smithi033.stdout:1/692: creat d2/d11/d28/fd9 x:0 0 0 2024-04-25T01:36:01.731 INFO:tasks.workunit.client.1.smithi138.stdout:7/314: rmdir da/de/d13/d21 39 2024-04-25T01:36:01.732 INFO:tasks.workunit.client.0.smithi033.stdout:3/884: creat d8/dd/d3d/d63/d7c/f139 x:0 0 0 2024-04-25T01:36:01.733 INFO:tasks.workunit.client.1.smithi138.stdout:3/433: write db/df/d17/d23/f30 [505847,6716] 0 2024-04-25T01:36:01.734 INFO:tasks.workunit.client.0.smithi033.stdout:2/840: mkdir da/d8d/d10c/d110 0 2024-04-25T01:36:01.734 INFO:tasks.workunit.client.0.smithi033.stdout:7/852: dwrite d4/d2e/d6d/f75 [0,4194304] 0 2024-04-25T01:36:01.734 INFO:tasks.workunit.client.0.smithi033.stdout:0/791: chown d7/da/df/dc7/cfd 72778 1 2024-04-25T01:36:01.734 INFO:tasks.workunit.client.0.smithi033.stdout:4/894: dread - d6/d14/df6/d10e/f113 zero size 2024-04-25T01:36:01.734 INFO:tasks.workunit.client.1.smithi138.stdout:7/315: write da/de/d13/f2d [4050241,22425] 0 2024-04-25T01:36:01.736 INFO:tasks.workunit.client.1.smithi138.stdout:3/434: mknod db/df/c81 0 2024-04-25T01:36:01.737 INFO:tasks.workunit.client.1.smithi138.stdout:7/316: dread da/de/d13/f3c [0,4194304] 0 2024-04-25T01:36:01.737 INFO:tasks.workunit.client.1.smithi138.stdout:3/435: chown db/df/c16 6550 1 2024-04-25T01:36:01.740 INFO:tasks.workunit.client.1.smithi138.stdout:3/436: mknod db/df/d17/d5d/c82 0 2024-04-25T01:36:01.741 INFO:tasks.workunit.client.0.smithi033.stdout:6/962: link d4/d24/d36/d42/d97/db2/f10a d4/d24/d36/dbd/dde/f143 0 2024-04-25T01:36:01.741 INFO:tasks.workunit.client.0.smithi033.stdout:0/792: chown d7/da/df/d14/lf2 15725 1 2024-04-25T01:36:01.742 INFO:tasks.workunit.client.1.smithi138.stdout:7/317: mknod da/de/d13/c6b 0 2024-04-25T01:36:01.742 INFO:tasks.workunit.client.1.smithi138.stdout:3/437: chown db/df/d17/d23/f67 379024 1 2024-04-25T01:36:01.742 INFO:tasks.workunit.client.0.smithi033.stdout:9/839: write d0/d2/dd/f51 [481348,33342] 0 2024-04-25T01:36:01.744 INFO:tasks.workunit.client.1.smithi138.stdout:7/318: creat da/de/f6c x:0 0 0 2024-04-25T01:36:01.745 INFO:tasks.workunit.client.0.smithi033.stdout:1/693: creat d2/d2e/d49/fda x:0 0 0 2024-04-25T01:36:01.745 INFO:tasks.workunit.client.1.smithi138.stdout:7/319: fdatasync da/de/f6c 0 2024-04-25T01:36:01.746 INFO:tasks.workunit.client.1.smithi138.stdout:7/320: chown da/de/d13/d30/d59 10643 1 2024-04-25T01:36:01.746 INFO:tasks.workunit.client.1.smithi138.stdout:7/321: chown da/f6a 7 1 2024-04-25T01:36:01.748 INFO:tasks.workunit.client.1.smithi138.stdout:7/322: unlink da/de/c32 0 2024-04-25T01:36:01.750 INFO:tasks.workunit.client.1.smithi138.stdout:7/323: mknod da/de/c6d 0 2024-04-25T01:36:01.750 INFO:tasks.workunit.client.0.smithi033.stdout:2/841: mkdir da/d27/d31/d40/d8a/d111 0 2024-04-25T01:36:01.751 INFO:tasks.workunit.client.0.smithi033.stdout:4/895: mkdir d6/d14/d35/d44/d45/d46/d57/d70/d87/db8/d121 0 2024-04-25T01:36:01.752 INFO:tasks.workunit.client.0.smithi033.stdout:6/963: dwrite d4/d1d/d3e/d41/d5c/dae/f106 [0,4194304] 0 2024-04-25T01:36:01.752 INFO:tasks.workunit.client.1.smithi138.stdout:7/324: mkdir da/de/d13/d3e/d51/d6e 0 2024-04-25T01:36:01.753 INFO:tasks.workunit.client.0.smithi033.stdout:7/853: mknod d4/d23/d3f/d45/d6c/c10a 0 2024-04-25T01:36:01.754 INFO:tasks.workunit.client.0.smithi033.stdout:7/854: readlink d4/d23/d3f/d45/d6c/l72 0 2024-04-25T01:36:01.754 INFO:tasks.workunit.client.0.smithi033.stdout:6/964: dread - d4/d1d/d3e/d41/d54/da1/f138 zero size 2024-04-25T01:36:01.758 INFO:tasks.workunit.client.0.smithi033.stdout:5/881: creat d1c/d2a/f11c x:0 0 0 2024-04-25T01:36:01.760 INFO:tasks.workunit.client.0.smithi033.stdout:4/896: dread - d6/d14/d1d/fb9 zero size 2024-04-25T01:36:01.762 INFO:tasks.workunit.client.0.smithi033.stdout:6/965: dwrite d4/df1/ff8 [0,4194304] 0 2024-04-25T01:36:01.764 INFO:tasks.workunit.client.1.smithi138.stdout:7/325: truncate da/de/d13/f2d 75445 0 2024-04-25T01:36:01.765 INFO:tasks.workunit.client.0.smithi033.stdout:4/897: fdatasync d6/d14/d35/d3a/d4c/f9f 0 2024-04-25T01:36:01.765 INFO:tasks.workunit.client.1.smithi138.stdout:7/326: dread - da/de/f27 zero size 2024-04-25T01:36:01.766 INFO:tasks.workunit.client.0.smithi033.stdout:4/898: rename d6 to d6/d14/d1d/d3b/d11b/d122 22 2024-04-25T01:36:01.767 INFO:tasks.workunit.client.0.smithi033.stdout:3/885: creat d8/dd/d3d/d63/d7c/d4d/f13a x:0 0 0 2024-04-25T01:36:01.773 INFO:tasks.workunit.client.0.smithi033.stdout:6/966: dread d4/d1d/d3e/d41/f10c [0,4194304] 0 2024-04-25T01:36:01.775 INFO:tasks.workunit.client.0.smithi033.stdout:6/967: chown d4/l110 69623 1 2024-04-25T01:36:01.797 INFO:tasks.workunit.client.0.smithi033.stdout:9/840: dread d0/d11/f18 [0,4194304] 0 2024-04-25T01:36:01.797 INFO:tasks.workunit.client.0.smithi033.stdout:1/694: link d2/d11/d28/c55 d2/d5/d6c/d94/cdb 0 2024-04-25T01:36:01.797 INFO:tasks.workunit.client.0.smithi033.stdout:6/968: unlink d4/d24/d36/dbd/dde/c10d 0 2024-04-25T01:36:01.798 INFO:tasks.workunit.client.0.smithi033.stdout:4/899: rename d6/d12/d38/la9 to d6/d14/d35/d44/d45/d46/d57/d70/d87/db8/d121/l123 0 2024-04-25T01:36:01.799 INFO:tasks.workunit.client.0.smithi033.stdout:2/842: getdents da/d27/d31/d40/d77 0 2024-04-25T01:36:01.913 INFO:tasks.workunit.client.0.smithi033.stdout:0/793: truncate d7/d33/fdf 3658265 0 2024-04-25T01:36:01.914 INFO:tasks.workunit.client.0.smithi033.stdout:8/823: dwrite d6/f34 [0,4194304] 0 2024-04-25T01:36:01.915 INFO:tasks.workunit.client.0.smithi033.stdout:7/855: getdents d4/d23/d3f/d45/d73/dbe 0 2024-04-25T01:36:01.917 INFO:tasks.workunit.client.0.smithi033.stdout:6/969: mkdir d4/d24/d7c/d144 0 2024-04-25T01:36:01.920 INFO:tasks.workunit.client.0.smithi033.stdout:1/695: write d2/d11/f30 [640043,66019] 0 2024-04-25T01:36:01.920 INFO:tasks.workunit.client.0.smithi033.stdout:7/856: chown d4/d2e/d6b/c59 39122 1 2024-04-25T01:36:01.923 INFO:tasks.workunit.client.0.smithi033.stdout:0/794: creat d7/d2d/d4d/d9c/d68/f113 x:0 0 0 2024-04-25T01:36:01.925 INFO:tasks.workunit.client.0.smithi033.stdout:2/843: write da/d8d/db0/fd0 [1333364,111679] 0 2024-04-25T01:36:01.926 INFO:tasks.workunit.client.0.smithi033.stdout:7/857: mknod d4/d57/c10b 0 2024-04-25T01:36:01.927 INFO:tasks.workunit.client.0.smithi033.stdout:0/795: stat d7/da/f48 0 2024-04-25T01:36:01.927 INFO:tasks.workunit.client.0.smithi033.stdout:4/900: creat d6/d12/f124 x:0 0 0 2024-04-25T01:36:01.928 INFO:tasks.workunit.client.0.smithi033.stdout:6/970: chown d4/d24/d36/d42/d97/db2/ld2 305 1 2024-04-25T01:36:01.928 INFO:tasks.workunit.client.0.smithi033.stdout:5/882: dwrite d1c/f37 [4194304,4194304] 0 2024-04-25T01:36:01.929 INFO:tasks.workunit.client.0.smithi033.stdout:1/696: mknod d2/d2e/dcd/cdc 0 2024-04-25T01:36:01.933 INFO:tasks.workunit.client.0.smithi033.stdout:6/971: readlink d4/d1d/d3e/d41/d5c/df6/lff 0 2024-04-25T01:36:01.934 INFO:tasks.workunit.client.0.smithi033.stdout:0/796: chown d7/d2d/d4d/fd6 15686 1 2024-04-25T01:36:01.935 INFO:tasks.workunit.client.0.smithi033.stdout:2/844: mknod da/d27/d31/d40/db1/dbf/dd5/c112 0 2024-04-25T01:36:01.938 INFO:tasks.workunit.client.0.smithi033.stdout:8/824: dwrite d6/d45/d52/f43 [0,4194304] 0 2024-04-25T01:36:01.939 INFO:tasks.workunit.client.0.smithi033.stdout:9/841: dread d0/d2/d43/d4b/fa8 [0,4194304] 0 2024-04-25T01:36:01.940 INFO:tasks.workunit.client.0.smithi033.stdout:7/858: symlink d4/d2e/d6b/d1f/d78/d86/de4/l10c 0 2024-04-25T01:36:01.941 INFO:tasks.workunit.client.0.smithi033.stdout:0/797: write d7/d2d/d4d/d9c/d61/fe1 [776093,9731] 0 2024-04-25T01:36:01.941 INFO:tasks.workunit.client.0.smithi033.stdout:6/972: mknod d4/d1d/dbf/c145 0 2024-04-25T01:36:01.944 INFO:tasks.workunit.client.0.smithi033.stdout:9/842: chown d0/d2/d43/d73/dab/lef 1707 1 2024-04-25T01:36:01.947 INFO:tasks.workunit.client.0.smithi033.stdout:6/973: creat d4/d1d/d3e/d70/f146 x:0 0 0 2024-04-25T01:36:01.947 INFO:tasks.workunit.client.0.smithi033.stdout:8/825: symlink d6/d45/d52/d2e/dd4/l132 0 2024-04-25T01:36:01.949 INFO:tasks.workunit.client.0.smithi033.stdout:0/798: creat d7/f114 x:0 0 0 2024-04-25T01:36:01.950 INFO:tasks.workunit.client.0.smithi033.stdout:9/843: mkdir d0/d11/d127 0 2024-04-25T01:36:01.950 INFO:tasks.workunit.client.0.smithi033.stdout:1/697: getdents d2/d2e/d49/dba 0 2024-04-25T01:36:01.952 INFO:tasks.workunit.client.0.smithi033.stdout:7/859: mknod d4/d23/d3f/d45/c10d 0 2024-04-25T01:36:01.953 INFO:tasks.workunit.client.0.smithi033.stdout:6/974: symlink d4/d1d/d3e/d41/d93/l147 0 2024-04-25T01:36:01.955 INFO:tasks.workunit.client.0.smithi033.stdout:4/901: write d6/d12/f23 [464611,22456] 0 2024-04-25T01:36:01.958 INFO:tasks.workunit.client.0.smithi033.stdout:8/826: dread d6/d45/f8b [0,4194304] 0 2024-04-25T01:36:01.960 INFO:tasks.workunit.client.0.smithi033.stdout:2/845: write da/d27/d31/d40/f64 [317260,63613] 0 2024-04-25T01:36:01.961 INFO:tasks.workunit.client.0.smithi033.stdout:6/975: creat d4/d1d/d3e/d41/d5c/d7f/d116/f148 x:0 0 0 2024-04-25T01:36:01.961 INFO:tasks.workunit.client.0.smithi033.stdout:5/883: link d1c/d2a/d50/d5b/fd7 d1c/d2a/f11d 0 2024-04-25T01:36:01.963 INFO:tasks.workunit.client.0.smithi033.stdout:9/844: mkdir d0/d2/d43/d4b/dc3/d128 0 2024-04-25T01:36:01.966 INFO:tasks.workunit.client.0.smithi033.stdout:4/902: read d6/d14/d1d/f6f [518054,23343] 0 2024-04-25T01:36:01.966 INFO:tasks.workunit.client.0.smithi033.stdout:6/976: mkdir d4/d24/d36/d42/d97/de0/d149 0 2024-04-25T01:36:01.967 INFO:tasks.workunit.client.0.smithi033.stdout:5/884: dread d1c/d2a/d50/d52/da6/d3e/d70/fc2 [0,4194304] 0 2024-04-25T01:36:01.971 INFO:tasks.workunit.client.0.smithi033.stdout:8/827: truncate d6/d45/d51/d5d/f62 246005 0 2024-04-25T01:36:01.971 INFO:tasks.workunit.client.0.smithi033.stdout:0/799: dread d7/da/df/d14/d45/f87 [0,4194304] 0 2024-04-25T01:36:01.972 INFO:tasks.workunit.client.0.smithi033.stdout:6/977: mkdir d4/d1d/d3e/d41/d54/da1/d14a 0 2024-04-25T01:36:01.973 INFO:tasks.workunit.client.0.smithi033.stdout:7/860: dwrite d4/d2e/d6b/d1f/f46 [4194304,4194304] 0 2024-04-25T01:36:01.974 INFO:tasks.workunit.client.0.smithi033.stdout:9/845: link d0/d2/ffa d0/d2/dd/dba/dc1/d11c/f129 0 2024-04-25T01:36:01.974 INFO:tasks.workunit.client.0.smithi033.stdout:6/978: read d4/d1d/f88 [798994,17263] 0 2024-04-25T01:36:01.984 INFO:tasks.workunit.client.0.smithi033.stdout:1/698: dread d2/d5/d1b/d46/f47 [0,4194304] 0 2024-04-25T01:36:01.985 INFO:tasks.workunit.client.0.smithi033.stdout:4/903: symlink d6/d14/d35/d44/d45/l125 0 2024-04-25T01:36:01.992 INFO:tasks.workunit.client.1.smithi138.stdout:0/275: sync 2024-04-25T01:36:02.095 INFO:tasks.workunit.client.0.smithi033.stdout:3/886: sync 2024-04-25T01:36:02.109 INFO:tasks.workunit.client.1.smithi138.stdout:5/284: sync 2024-04-25T01:36:02.109 INFO:tasks.workunit.client.1.smithi138.stdout:8/501: sync 2024-04-25T01:36:02.109 INFO:tasks.workunit.client.1.smithi138.stdout:9/251: sync 2024-04-25T01:36:02.109 INFO:tasks.workunit.client.1.smithi138.stdout:4/170: sync 2024-04-25T01:36:02.110 INFO:tasks.workunit.client.1.smithi138.stdout:1/295: sync 2024-04-25T01:36:02.110 INFO:tasks.workunit.client.1.smithi138.stdout:2/380: sync 2024-04-25T01:36:02.110 INFO:tasks.workunit.client.1.smithi138.stdout:7/327: sync 2024-04-25T01:36:02.110 INFO:tasks.workunit.client.1.smithi138.stdout:3/438: sync 2024-04-25T01:36:02.110 INFO:tasks.workunit.client.1.smithi138.stdout:6/407: sync 2024-04-25T01:36:02.112 INFO:tasks.workunit.client.1.smithi138.stdout:6/408: readlink dc/d10/l45 0 2024-04-25T01:36:02.113 INFO:tasks.workunit.client.1.smithi138.stdout:5/285: chown d4/da/d27/c48 31369 1 2024-04-25T01:36:02.115 INFO:tasks.workunit.client.1.smithi138.stdout:1/296: mknod d3/d4/d47/c60 0 2024-04-25T01:36:02.116 INFO:tasks.workunit.client.1.smithi138.stdout:1/297: chown d3/d27/d2d/l41 1 1 2024-04-25T01:36:02.121 INFO:tasks.workunit.client.1.smithi138.stdout:7/328: mkdir da/de/d13/d21/d6f 0 2024-04-25T01:36:02.123 INFO:tasks.workunit.client.1.smithi138.stdout:6/409: mkdir dc/d2e/d55/d7e 0 2024-04-25T01:36:02.124 INFO:tasks.workunit.client.1.smithi138.stdout:3/439: creat db/df/d17/f83 x:0 0 0 2024-04-25T01:36:02.125 INFO:tasks.workunit.client.1.smithi138.stdout:8/502: mknod d3/d16/d88/da/de/d57/c9d 0 2024-04-25T01:36:02.129 INFO:tasks.workunit.client.1.smithi138.stdout:7/329: mknod da/de/d1b/d34/c70 0 2024-04-25T01:36:02.134 INFO:tasks.workunit.client.1.smithi138.stdout:8/503: dwrite d3/d16/f3b [0,4194304] 0 2024-04-25T01:36:02.134 INFO:tasks.workunit.client.1.smithi138.stdout:3/440: read db/f3f [600738,81184] 0 2024-04-25T01:36:02.135 INFO:tasks.workunit.client.1.smithi138.stdout:2/381: link d0/d32/l46 d0/d18/l75 0 2024-04-25T01:36:02.136 INFO:tasks.workunit.client.1.smithi138.stdout:9/252: write d4/d1a/d33/f37 [3147611,16229] 0 2024-04-25T01:36:02.136 INFO:tasks.workunit.client.1.smithi138.stdout:4/171: getdents d0/d1c/d23 0 2024-04-25T01:36:02.138 INFO:tasks.workunit.client.1.smithi138.stdout:2/382: read d0/d32/f4b [1108705,49061] 0 2024-04-25T01:36:02.139 INFO:tasks.workunit.client.1.smithi138.stdout:7/330: mknod da/de/d13/d3e/d51/c71 0 2024-04-25T01:36:02.143 INFO:tasks.workunit.client.1.smithi138.stdout:9/253: symlink d4/d1a/d3c/l54 0 2024-04-25T01:36:02.146 INFO:tasks.workunit.client.1.smithi138.stdout:2/383: symlink d0/d5a/l76 0 2024-04-25T01:36:02.147 INFO:tasks.workunit.client.0.smithi033.stdout:0/800: dwrite d7/d2d/d43/dc0/f4b [0,4194304] 0 2024-04-25T01:36:02.147 INFO:tasks.workunit.client.0.smithi033.stdout:9/846: unlink d0/d2/dd/d5f/d11b/f102 0 2024-04-25T01:36:02.149 INFO:tasks.workunit.client.1.smithi138.stdout:5/286: dwrite d4/f9 [0,4194304] 0 2024-04-25T01:36:02.150 INFO:tasks.workunit.client.0.smithi033.stdout:2/846: write f8 [1501817,14320] 0 2024-04-25T01:36:02.151 INFO:tasks.workunit.client.1.smithi138.stdout:6/410: dread dc/d10/d28/f46 [0,4194304] 0 2024-04-25T01:36:02.151 INFO:tasks.workunit.client.0.smithi033.stdout:0/801: chown d7/d2d/d4d/d9c/db2 27110920 1 2024-04-25T01:36:02.152 INFO:tasks.workunit.client.0.smithi033.stdout:8/828: write d6/f3e [646248,2529] 0 2024-04-25T01:36:02.152 INFO:tasks.workunit.client.1.smithi138.stdout:1/298: truncate d3/d27/d2d/f32 4175 0 2024-04-25T01:36:02.153 INFO:tasks.workunit.client.0.smithi033.stdout:9/847: readlink d0/d2/d43/d4b/dc5/de0/d42/le5 0 2024-04-25T01:36:02.155 INFO:tasks.workunit.client.1.smithi138.stdout:2/384: symlink d0/d21/d45/l77 0 2024-04-25T01:36:02.156 INFO:tasks.workunit.client.0.smithi033.stdout:4/904: write d6/d14/f22 [229560,118193] 0 2024-04-25T01:36:02.156 INFO:tasks.workunit.client.0.smithi033.stdout:6/979: dwrite d4/d1d/d3e/d41/d5c/d7f/da4/dcc/f128 [0,4194304] 0 2024-04-25T01:36:02.157 INFO:tasks.workunit.client.1.smithi138.stdout:4/172: truncate d0/fa 985862 0 2024-04-25T01:36:02.159 INFO:tasks.workunit.client.1.smithi138.stdout:9/254: mkdir d4/d1a/d55 0 2024-04-25T01:36:02.160 INFO:tasks.workunit.client.1.smithi138.stdout:9/255: write d4/d8/f53 [714088,32009] 0 2024-04-25T01:36:02.161 INFO:tasks.workunit.client.1.smithi138.stdout:6/411: unlink dc/d10/f43 0 2024-04-25T01:36:02.162 INFO:tasks.workunit.client.0.smithi033.stdout:8/829: unlink d6/f3e 0 2024-04-25T01:36:02.162 INFO:tasks.workunit.client.1.smithi138.stdout:1/299: fdatasync d3/db/f14 0 2024-04-25T01:36:02.162 INFO:tasks.workunit.client.1.smithi138.stdout:6/412: write dc/d10/d67/f77 [1046650,125666] 0 2024-04-25T01:36:02.163 INFO:tasks.workunit.client.0.smithi033.stdout:3/887: symlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/l13b 0 2024-04-25T01:36:02.165 INFO:tasks.workunit.client.1.smithi138.stdout:5/287: mkdir d4/da/d27/d1c/d5c 0 2024-04-25T01:36:02.165 INFO:tasks.workunit.client.0.smithi033.stdout:9/848: chown d0/d2/dd/d20/d4e/f78 158648 1 2024-04-25T01:36:02.169 INFO:tasks.workunit.client.0.smithi033.stdout:0/802: dwrite d7/da/f48 [4194304,4194304] 0 2024-04-25T01:36:02.170 INFO:tasks.workunit.client.0.smithi033.stdout:9/849: dread - d0/d2/d43/fff zero size 2024-04-25T01:36:02.170 INFO:tasks.workunit.client.1.smithi138.stdout:7/331: getdents da/de/d13 0 2024-04-25T01:36:02.171 INFO:tasks.workunit.client.0.smithi033.stdout:2/847: creat da/d27/d31/d40/d8a/d111/f113 x:0 0 0 2024-04-25T01:36:02.171 INFO:tasks.workunit.client.1.smithi138.stdout:1/300: rmdir d3/db/d4a 39 2024-04-25T01:36:02.172 INFO:tasks.workunit.client.1.smithi138.stdout:7/332: stat da/de/l1d 0 2024-04-25T01:36:02.172 INFO:tasks.workunit.client.0.smithi033.stdout:0/803: dread - d7/d2d/d64/fd1 zero size 2024-04-25T01:36:02.173 INFO:tasks.workunit.client.0.smithi033.stdout:8/830: unlink d6/db/d80/d89/fca 0 2024-04-25T01:36:02.174 INFO:tasks.workunit.client.1.smithi138.stdout:6/413: creat dc/d10/d28/d70/f7f x:0 0 0 2024-04-25T01:36:02.174 INFO:tasks.workunit.client.0.smithi033.stdout:0/804: chown d7/d2d/d43/d92 55 1 2024-04-25T01:36:02.175 INFO:tasks.workunit.client.0.smithi033.stdout:8/831: chown d6/d45/d51/d5d/d6e/fbd 3866 1 2024-04-25T01:36:02.175 INFO:tasks.workunit.client.0.smithi033.stdout:6/980: unlink d4/f6 0 2024-04-25T01:36:02.176 INFO:tasks.workunit.client.0.smithi033.stdout:3/888: dread d8/dd/d3d/d63/d7c/f73 [4194304,4194304] 0 2024-04-25T01:36:02.177 INFO:tasks.workunit.client.1.smithi138.stdout:4/173: rename d0/l21 to d0/d16/l27 0 2024-04-25T01:36:02.178 INFO:tasks.workunit.client.0.smithi033.stdout:6/981: stat d4/d1d/d3e/d41/d9f/la8 0 2024-04-25T01:36:02.179 INFO:tasks.workunit.client.0.smithi033.stdout:1/699: getdents d2/d5/d75 0 2024-04-25T01:36:02.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/333: symlink da/de/d1b/d34/l72 0 2024-04-25T01:36:02.181 INFO:tasks.workunit.client.0.smithi033.stdout:7/861: dread d4/d2e/f1c [0,4194304] 0 2024-04-25T01:36:02.182 INFO:tasks.workunit.client.0.smithi033.stdout:4/905: creat d6/d14/d35/f126 x:0 0 0 2024-04-25T01:36:02.183 INFO:tasks.workunit.client.1.smithi138.stdout:1/301: creat d3/d27/d3e/f61 x:0 0 0 2024-04-25T01:36:02.184 INFO:tasks.workunit.client.1.smithi138.stdout:5/288: creat d4/da/d27/d1c/d5c/f5d x:0 0 0 2024-04-25T01:36:02.184 INFO:tasks.workunit.client.1.smithi138.stdout:1/302: chown d3/db/f17 16132564 1 2024-04-25T01:36:02.185 INFO:tasks.workunit.client.0.smithi033.stdout:9/850: creat d0/d2/dd/df/d1b/dc0/f12a x:0 0 0 2024-04-25T01:36:02.185 INFO:tasks.workunit.client.1.smithi138.stdout:6/414: mkdir dc/d10/d28/d80 0 2024-04-25T01:36:02.187 INFO:tasks.workunit.client.1.smithi138.stdout:4/174: creat d0/d1c/f28 x:0 0 0 2024-04-25T01:36:02.187 INFO:tasks.workunit.client.0.smithi033.stdout:6/982: rename d4/l13 to d4/d1d/d3e/d41/d93/l14b 0 2024-04-25T01:36:02.188 INFO:tasks.workunit.client.1.smithi138.stdout:9/256: creat d4/d1a/f56 x:0 0 0 2024-04-25T01:36:02.190 INFO:tasks.workunit.client.0.smithi033.stdout:7/862: mkdir d4/d2e/d6d/d10e 0 2024-04-25T01:36:02.190 INFO:tasks.workunit.client.0.smithi033.stdout:7/863: chown d4/d57/c91 4847732 1 2024-04-25T01:36:02.191 INFO:tasks.workunit.client.1.smithi138.stdout:7/334: mkdir da/de/d13/d3e/d73 0 2024-04-25T01:36:02.191 INFO:tasks.workunit.client.0.smithi033.stdout:1/700: creat d2/d5/d1b/d46/d5f/d63/d8f/fdd x:0 0 0 2024-04-25T01:36:02.192 INFO:tasks.workunit.client.0.smithi033.stdout:1/701: stat d2/d2e/d34/d60/fb8 0 2024-04-25T01:36:02.194 INFO:tasks.workunit.client.0.smithi033.stdout:1/702: dread - d2/d5/d6c/fcf zero size 2024-04-25T01:36:02.195 INFO:tasks.workunit.client.1.smithi138.stdout:6/415: unlink dc/d10/d67/f6e 0 2024-04-25T01:36:02.195 INFO:tasks.workunit.client.0.smithi033.stdout:2/848: creat da/d5b/d5e/f114 x:0 0 0 2024-04-25T01:36:02.196 INFO:tasks.workunit.client.1.smithi138.stdout:0/276: sync 2024-04-25T01:36:02.197 INFO:tasks.workunit.client.0.smithi033.stdout:8/832: fsync d6/d45/d52/f28 0 2024-04-25T01:36:02.197 INFO:tasks.workunit.client.1.smithi138.stdout:9/257: rmdir d4/d1a/d33 39 2024-04-25T01:36:02.199 INFO:tasks.workunit.client.0.smithi033.stdout:8/833: dread - d6/db/f127 zero size 2024-04-25T01:36:02.199 INFO:tasks.workunit.client.0.smithi033.stdout:9/851: write d0/d2/d43/d4b/dc5/de0/d7e/d58/f115 [2090131,113486] 0 2024-04-25T01:36:02.202 INFO:tasks.workunit.client.1.smithi138.stdout:6/416: dwrite dc/d2e/f6c [0,4194304] 0 2024-04-25T01:36:02.203 INFO:tasks.workunit.client.1.smithi138.stdout:6/417: stat dc/f66 0 2024-04-25T01:36:02.204 INFO:tasks.workunit.client.0.smithi033.stdout:7/864: dwrite d4/d2e/d7f/d98/fd9 [0,4194304] 0 2024-04-25T01:36:02.205 INFO:tasks.workunit.client.0.smithi033.stdout:1/703: mknod d2/d5/d1b/d46/cde 0 2024-04-25T01:36:02.206 INFO:tasks.workunit.client.0.smithi033.stdout:8/834: truncate d6/dd3/dfb/f107 891862 0 2024-04-25T01:36:02.211 INFO:tasks.workunit.client.0.smithi033.stdout:6/983: dwrite d4/d1d/d3e/d41/d93/f104 [0,4194304] 0 2024-04-25T01:36:02.211 INFO:tasks.workunit.client.0.smithi033.stdout:7/865: creat d4/d2e/d65/df5/f10f x:0 0 0 2024-04-25T01:36:02.212 INFO:tasks.workunit.client.1.smithi138.stdout:5/289: write d4/da/fd [1830916,3397] 0 2024-04-25T01:36:02.212 INFO:tasks.workunit.client.1.smithi138.stdout:7/335: link da/f63 da/de/f74 0 2024-04-25T01:36:02.213 INFO:tasks.workunit.client.0.smithi033.stdout:6/984: chown d4/d24/d36/d42/d97/de0/d13f 26575335 1 2024-04-25T01:36:02.214 INFO:tasks.workunit.client.0.smithi033.stdout:9/852: mkdir d0/d2/d43/d73/dab/dad/dee/d12b 0 2024-04-25T01:36:02.215 INFO:tasks.workunit.client.0.smithi033.stdout:1/704: dread - d2/d2e/d34/f82 zero size 2024-04-25T01:36:02.216 INFO:tasks.workunit.client.1.smithi138.stdout:0/277: dwrite d6/df/d3a/f3d [4194304,4194304] 0 2024-04-25T01:36:02.216 INFO:tasks.workunit.client.0.smithi033.stdout:1/705: chown d2/d5/d75/d59/d83/fa9 15248681 1 2024-04-25T01:36:02.219 INFO:tasks.workunit.client.0.smithi033.stdout:7/866: mknod d4/d2e/d65/c110 0 2024-04-25T01:36:02.223 INFO:tasks.workunit.client.0.smithi033.stdout:8/835: mknod d6/d45/d52/df2/d12b/c133 0 2024-04-25T01:36:02.224 INFO:tasks.workunit.client.1.smithi138.stdout:7/336: dwrite da/de/d13/f24 [4194304,4194304] 0 2024-04-25T01:36:02.224 INFO:tasks.workunit.client.1.smithi138.stdout:9/258: mkdir d4/d57 0 2024-04-25T01:36:02.224 INFO:tasks.workunit.client.1.smithi138.stdout:1/303: truncate d3/d45/f48 2767621 0 2024-04-25T01:36:02.224 INFO:tasks.workunit.client.1.smithi138.stdout:7/337: chown da/c48 3402 1 2024-04-25T01:36:02.226 INFO:tasks.workunit.client.1.smithi138.stdout:1/304: write d3/d4/f37 [3709698,89324] 0 2024-04-25T01:36:02.229 INFO:tasks.workunit.client.0.smithi033.stdout:7/867: mknod d4/d2e/d6b/d1f/d78/c111 0 2024-04-25T01:36:02.230 INFO:tasks.workunit.client.1.smithi138.stdout:4/175: getdents d0/d16 0 2024-04-25T01:36:02.230 INFO:tasks.workunit.client.0.smithi033.stdout:9/853: mkdir d0/d2/d43/d4b/dc5/d12c 0 2024-04-25T01:36:02.232 INFO:tasks.workunit.client.1.smithi138.stdout:9/259: read - d4/d1a/d33/f4d zero size 2024-04-25T01:36:02.234 INFO:tasks.workunit.client.0.smithi033.stdout:0/805: dwrite d7/da/fb7 [0,4194304] 0 2024-04-25T01:36:02.235 INFO:tasks.workunit.client.0.smithi033.stdout:8/836: mknod d6/d45/d51/d5d/d11d/c134 0 2024-04-25T01:36:02.240 INFO:tasks.workunit.client.0.smithi033.stdout:3/889: truncate d8/dd/d1d/f9a 4037139 0 2024-04-25T01:36:02.240 INFO:tasks.workunit.client.0.smithi033.stdout:2/849: dread da/d27/f5c [0,4194304] 0 2024-04-25T01:36:02.243 INFO:tasks.workunit.client.1.smithi138.stdout:4/176: chown d0/c19 7 1 2024-04-25T01:36:02.243 INFO:tasks.workunit.client.1.smithi138.stdout:9/260: dwrite d4/f22 [0,4194304] 0 2024-04-25T01:36:02.245 INFO:tasks.workunit.client.0.smithi033.stdout:4/906: dwrite d6/d12/fd1 [0,4194304] 0 2024-04-25T01:36:02.245 INFO:tasks.workunit.client.0.smithi033.stdout:9/854: truncate d0/d2/dd/f17 4095007 0 2024-04-25T01:36:02.248 INFO:tasks.workunit.client.0.smithi033.stdout:1/706: getdents d2/d5/d1b/d46/d5f/d63/d8f 0 2024-04-25T01:36:02.250 INFO:tasks.workunit.client.1.smithi138.stdout:0/278: creat d6/f5e x:0 0 0 2024-04-25T01:36:02.250 INFO:tasks.workunit.client.1.smithi138.stdout:6/418: getdents dc/d10/d6d 0 2024-04-25T01:36:02.251 INFO:tasks.workunit.client.1.smithi138.stdout:7/338: write da/de/d1b/d4b/d5b/f67 [4311312,95599] 0 2024-04-25T01:36:02.252 INFO:tasks.workunit.client.0.smithi033.stdout:2/850: getdents da/d2c 0 2024-04-25T01:36:02.253 INFO:tasks.workunit.client.0.smithi033.stdout:0/806: dwrite d7/d2d/d4d/d9c/d68/f113 [0,4194304] 0 2024-04-25T01:36:02.253 INFO:tasks.workunit.client.1.smithi138.stdout:5/290: getdents d4/da 0 2024-04-25T01:36:02.253 INFO:tasks.workunit.client.1.smithi138.stdout:4/177: creat d0/d16/f29 x:0 0 0 2024-04-25T01:36:02.255 INFO:tasks.workunit.client.0.smithi033.stdout:4/907: dread - d6/d14/d35/d44/d45/d46/db0/fbe zero size 2024-04-25T01:36:02.255 INFO:tasks.workunit.client.0.smithi033.stdout:9/855: truncate d0/d2/dd/d20/d4e/f104 34477 0 2024-04-25T01:36:02.256 INFO:tasks.workunit.client.0.smithi033.stdout:2/851: readlink da/d50/l60 0 2024-04-25T01:36:02.256 INFO:tasks.workunit.client.0.smithi033.stdout:3/890: mkdir d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/d13c 0 2024-04-25T01:36:02.257 INFO:tasks.workunit.client.1.smithi138.stdout:8/504: sync 2024-04-25T01:36:02.259 INFO:tasks.workunit.client.1.smithi138.stdout:6/419: write dc/d10/f76 [806324,47011] 0 2024-04-25T01:36:02.259 INFO:tasks.workunit.client.0.smithi033.stdout:3/891: write d8/fa [5215859,21654] 0 2024-04-25T01:36:02.260 INFO:tasks.workunit.client.1.smithi138.stdout:7/339: write da/f6a [605697,20661] 0 2024-04-25T01:36:02.261 INFO:tasks.workunit.client.1.smithi138.stdout:9/261: dwrite d4/f13 [0,4194304] 0 2024-04-25T01:36:02.263 INFO:tasks.workunit.client.0.smithi033.stdout:4/908: creat d6/d14/df6/d10e/f127 x:0 0 0 2024-04-25T01:36:02.264 INFO:tasks.workunit.client.1.smithi138.stdout:0/279: rename d6/df/d3a/f4a to d6/d1d/f5f 0 2024-04-25T01:36:02.265 INFO:tasks.workunit.client.1.smithi138.stdout:6/420: write dc/d10/f33 [4501534,65647] 0 2024-04-25T01:36:02.265 INFO:tasks.workunit.client.1.smithi138.stdout:9/262: chown d4/d8/d2c 856516192 1 2024-04-25T01:36:02.266 INFO:tasks.workunit.client.0.smithi033.stdout:2/852: creat da/d50/f115 x:0 0 0 2024-04-25T01:36:02.267 INFO:tasks.workunit.client.0.smithi033.stdout:0/807: creat d7/d2d/d4d/d9c/d68/dd2/f115 x:0 0 0 2024-04-25T01:36:02.267 INFO:tasks.workunit.client.1.smithi138.stdout:9/263: stat d4/d8/f23 0 2024-04-25T01:36:02.268 INFO:tasks.workunit.client.1.smithi138.stdout:6/421: stat dc/d2e/d37/d56 0 2024-04-25T01:36:02.270 INFO:tasks.workunit.client.1.smithi138.stdout:5/291: mkdir d4/da/d27/d1c/d22/d35/d4d/d5e 0 2024-04-25T01:36:02.270 INFO:tasks.workunit.client.0.smithi033.stdout:9/856: truncate d0/d2/dd/df/f2b 794127 0 2024-04-25T01:36:02.270 INFO:tasks.workunit.client.0.smithi033.stdout:5/885: sync 2024-04-25T01:36:02.272 INFO:tasks.workunit.client.1.smithi138.stdout:1/305: dwrite d3/db/f17 [0,4194304] 0 2024-04-25T01:36:02.275 INFO:tasks.workunit.client.0.smithi033.stdout:8/837: truncate d6/d45/f57 2827558 0 2024-04-25T01:36:02.275 INFO:tasks.workunit.client.1.smithi138.stdout:0/280: mknod d6/c60 0 2024-04-25T01:36:02.276 INFO:tasks.workunit.client.1.smithi138.stdout:5/292: write d4/da/d27/d1c/d5c/f5d [143566,65817] 0 2024-04-25T01:36:02.277 INFO:tasks.workunit.client.0.smithi033.stdout:3/892: fdatasync d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/fcd 0 2024-04-25T01:36:02.277 INFO:tasks.workunit.client.0.smithi033.stdout:4/909: mkdir d6/d14/d35/d44/d45/d46/d57/d70/d128 0 2024-04-25T01:36:02.281 INFO:tasks.workunit.client.0.smithi033.stdout:1/707: rename d2/d11/fd3 to d2/d2e/fdf 0 2024-04-25T01:36:02.282 INFO:tasks.workunit.client.0.smithi033.stdout:0/808: dwrite d7/d2d/d4d/f5c [0,4194304] 0 2024-04-25T01:36:02.282 INFO:tasks.workunit.client.0.smithi033.stdout:5/886: mknod d1c/d2a/d50/d52/da6/d3e/d70/dae/c11e 0 2024-04-25T01:36:02.283 INFO:tasks.workunit.client.0.smithi033.stdout:7/868: dwrite d4/d10/f29 [0,4194304] 0 2024-04-25T01:36:02.284 INFO:tasks.workunit.client.1.smithi138.stdout:1/306: write d3/db/f1d [2421136,54807] 0 2024-04-25T01:36:02.284 INFO:tasks.workunit.client.1.smithi138.stdout:6/422: dwrite dc/d10/f22 [0,4194304] 0 2024-04-25T01:36:02.285 INFO:tasks.workunit.client.1.smithi138.stdout:5/293: symlink d4/da/l5f 0 2024-04-25T01:36:02.287 INFO:tasks.workunit.client.0.smithi033.stdout:8/838: creat d6/db/d80/d89/dbf/f135 x:0 0 0 2024-04-25T01:36:02.288 INFO:tasks.workunit.client.1.smithi138.stdout:4/178: dread d0/fa [0,4194304] 0 2024-04-25T01:36:02.290 INFO:tasks.workunit.client.0.smithi033.stdout:9/857: mkdir d0/d2/d43/d4b/dc3/d128/d12d 0 2024-04-25T01:36:02.290 INFO:tasks.workunit.client.1.smithi138.stdout:6/423: read dc/d10/d28/f48 [572151,5186] 0 2024-04-25T01:36:02.291 INFO:tasks.workunit.client.0.smithi033.stdout:1/708: symlink d2/d7f/db0/le0 0 2024-04-25T01:36:02.291 INFO:tasks.workunit.client.0.smithi033.stdout:4/910: symlink d6/d14/d35/d44/d45/d46/db0/dcc/l129 0 2024-04-25T01:36:02.294 INFO:tasks.workunit.client.0.smithi033.stdout:0/809: truncate d7/d2d/d4d/d9c/d68/dd2/f115 305982 0 2024-04-25T01:36:02.295 INFO:tasks.workunit.client.1.smithi138.stdout:8/505: dread d3/d16/d88/f53 [0,4194304] 0 2024-04-25T01:36:02.296 INFO:tasks.workunit.client.1.smithi138.stdout:9/264: rmdir d4/d1a/d33/d1d/d50 0 2024-04-25T01:36:02.296 INFO:tasks.workunit.client.1.smithi138.stdout:1/307: dread d3/f2c [0,4194304] 0 2024-04-25T01:36:02.296 INFO:tasks.workunit.client.1.smithi138.stdout:8/506: fsync d3/f6d 0 2024-04-25T01:36:02.296 INFO:tasks.workunit.client.0.smithi033.stdout:2/853: creat da/d5b/d5e/f116 x:0 0 0 2024-04-25T01:36:02.297 INFO:tasks.workunit.client.1.smithi138.stdout:9/265: write d4/f22 [1728462,106814] 0 2024-04-25T01:36:02.298 INFO:tasks.workunit.client.1.smithi138.stdout:6/424: mkdir dc/d2e/d55/d81 0 2024-04-25T01:36:02.305 INFO:tasks.workunit.client.1.smithi138.stdout:8/507: creat d3/d16/d1f/d50/f9e x:0 0 0 2024-04-25T01:36:02.307 INFO:tasks.workunit.client.0.smithi033.stdout:7/869: creat d4/d2e/d6d/dbb/f112 x:0 0 0 2024-04-25T01:36:02.309 INFO:tasks.workunit.client.0.smithi033.stdout:0/810: mknod d7/d33/dcd/c116 0 2024-04-25T01:36:02.310 INFO:tasks.workunit.client.0.smithi033.stdout:1/709: dread d2/fb6 [0,4194304] 0 2024-04-25T01:36:02.311 INFO:tasks.workunit.client.0.smithi033.stdout:8/839: mkdir d6/d45/d51/d55/d136 0 2024-04-25T01:36:02.312 INFO:tasks.workunit.client.1.smithi138.stdout:7/340: dread da/de/d13/f23 [0,4194304] 0 2024-04-25T01:36:02.312 INFO:tasks.workunit.client.1.smithi138.stdout:8/508: dread d3/d16/d88/da/db/f96 [0,4194304] 0 2024-04-25T01:36:02.312 INFO:tasks.workunit.client.1.smithi138.stdout:4/179: creat d0/f2a x:0 0 0 2024-04-25T01:36:02.313 INFO:tasks.workunit.client.0.smithi033.stdout:2/854: dread da/d27/d31/d40/db1/fcc [0,4194304] 0 2024-04-25T01:36:02.313 INFO:tasks.workunit.client.1.smithi138.stdout:0/281: write d6/d1d/f3c [4366969,1900] 0 2024-04-25T01:36:02.314 INFO:tasks.workunit.client.0.smithi033.stdout:3/893: creat d8/dd/d3d/d4f/da4/f13d x:0 0 0 2024-04-25T01:36:02.314 INFO:tasks.workunit.client.0.smithi033.stdout:0/811: write d7/d2d/d4d/d9c/d61/fe1 [1769634,62331] 0 2024-04-25T01:36:02.314 INFO:tasks.workunit.client.1.smithi138.stdout:9/266: mknod d4/d8/c58 0 2024-04-25T01:36:02.317 INFO:tasks.workunit.client.1.smithi138.stdout:6/425: creat dc/d10/d28/d80/f82 x:0 0 0 2024-04-25T01:36:02.317 INFO:tasks.workunit.client.1.smithi138.stdout:0/282: write d6/f26 [5284039,37647] 0 2024-04-25T01:36:02.318 INFO:tasks.workunit.client.1.smithi138.stdout:9/267: truncate d4/d1a/f42 411090 0 2024-04-25T01:36:02.318 INFO:tasks.workunit.client.0.smithi033.stdout:2/855: fdatasync da/d27/d31/d40/db1/dbf/dd5/ff9 0 2024-04-25T01:36:02.319 INFO:tasks.workunit.client.0.smithi033.stdout:7/870: dwrite d4/d2e/d6d/fe5 [0,4194304] 0 2024-04-25T01:36:02.321 INFO:tasks.workunit.client.1.smithi138.stdout:4/180: creat d0/d1c/f2b x:0 0 0 2024-04-25T01:36:02.321 INFO:tasks.workunit.client.1.smithi138.stdout:0/283: write d6/f28 [1279136,21058] 0 2024-04-25T01:36:02.323 INFO:tasks.workunit.client.1.smithi138.stdout:4/181: stat d0/d16/f29 0 2024-04-25T01:36:02.324 INFO:tasks.workunit.client.1.smithi138.stdout:3/441: sync 2024-04-25T01:36:02.324 INFO:tasks.workunit.client.1.smithi138.stdout:2/385: sync 2024-04-25T01:36:02.328 INFO:tasks.workunit.client.0.smithi033.stdout:3/894: unlink d8/dd/f2c 0 2024-04-25T01:36:02.328 INFO:tasks.workunit.client.1.smithi138.stdout:8/509: creat d3/d16/d88/da/db/d15/d56/d94/f9f x:0 0 0 2024-04-25T01:36:02.329 INFO:tasks.workunit.client.1.smithi138.stdout:7/341: dwrite da/de/d13/d30/f65 [0,4194304] 0 2024-04-25T01:36:02.329 INFO:tasks.workunit.client.0.smithi033.stdout:5/887: link d1c/d2a/d50/d52/da6/d3e/l97 d1c/l11f 0 2024-04-25T01:36:02.331 INFO:tasks.workunit.client.0.smithi033.stdout:5/888: read - d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e/f112 zero size 2024-04-25T01:36:02.331 INFO:tasks.workunit.client.1.smithi138.stdout:4/182: write d0/d16/f18 [446700,80468] 0 2024-04-25T01:36:02.331 INFO:tasks.workunit.client.1.smithi138.stdout:6/426: mkdir dc/d10/d28/d70/d83 0 2024-04-25T01:36:02.332 INFO:tasks.workunit.client.0.smithi033.stdout:0/812: fsync d7/d2d/d4d/d9c/d61/f94 0 2024-04-25T01:36:02.333 INFO:tasks.workunit.client.1.smithi138.stdout:1/308: write d3/d27/d24/f3b [1104142,71739] 0 2024-04-25T01:36:02.335 INFO:tasks.workunit.client.0.smithi033.stdout:7/871: mkdir d4/d2e/d7f/d113 0 2024-04-25T01:36:02.337 INFO:tasks.workunit.client.1.smithi138.stdout:3/442: write db/df/d19/d1a/f49 [2418355,52137] 0 2024-04-25T01:36:02.338 INFO:tasks.workunit.client.1.smithi138.stdout:3/443: write db/d14/d40/d41/f7c [406531,64700] 0 2024-04-25T01:36:02.338 INFO:tasks.workunit.client.0.smithi033.stdout:4/911: write d6/d14/d48/f85 [771936,79796] 0 2024-04-25T01:36:02.341 INFO:tasks.workunit.client.1.smithi138.stdout:4/183: creat d0/d1c/f2c x:0 0 0 2024-04-25T01:36:02.345 INFO:tasks.workunit.client.0.smithi033.stdout:9/858: dwrite d0/d2/dd/df/f8c [0,4194304] 0 2024-04-25T01:36:02.345 INFO:tasks.workunit.client.1.smithi138.stdout:0/284: write d6/d1d/f5f [4647414,93568] 0 2024-04-25T01:36:02.346 INFO:tasks.workunit.client.1.smithi138.stdout:2/386: dread d0/d21/f3a [0,4194304] 0 2024-04-25T01:36:02.346 INFO:tasks.workunit.client.1.smithi138.stdout:6/427: symlink dc/d10/d6d/l84 0 2024-04-25T01:36:02.348 INFO:tasks.workunit.client.1.smithi138.stdout:3/444: dread db/df/d19/d1a/f44 [0,4194304] 0 2024-04-25T01:36:02.349 INFO:tasks.workunit.client.0.smithi033.stdout:0/813: dwrite d7/da/df/d50/f6b [0,4194304] 0 2024-04-25T01:36:02.350 INFO:tasks.workunit.client.0.smithi033.stdout:7/872: chown d4/fb 5373118 1 2024-04-25T01:36:02.352 INFO:tasks.workunit.client.1.smithi138.stdout:5/294: mkdir d4/da/d27/d1c/d22/d35/d4d/d5e/d60 0 2024-04-25T01:36:02.353 INFO:tasks.workunit.client.0.smithi033.stdout:1/710: creat d2/d11/d28/fe1 x:0 0 0 2024-04-25T01:36:02.354 INFO:tasks.workunit.client.1.smithi138.stdout:9/268: creat d4/d8/f59 x:0 0 0 2024-04-25T01:36:02.356 INFO:tasks.workunit.client.0.smithi033.stdout:5/889: unlink d1c/d2a/d50/db1/fce 0 2024-04-25T01:36:02.357 INFO:tasks.workunit.client.1.smithi138.stdout:7/342: creat da/de/d13/d3e/d51/d6e/f75 x:0 0 0 2024-04-25T01:36:02.357 INFO:tasks.workunit.client.0.smithi033.stdout:9/859: dread - d0/d2/dd/d5f/fd0 zero size 2024-04-25T01:36:02.357 INFO:tasks.workunit.client.1.smithi138.stdout:1/309: dwrite d3/f2c [4194304,4194304] 0 2024-04-25T01:36:02.358 INFO:tasks.workunit.client.1.smithi138.stdout:0/285: symlink d6/df/l61 0 2024-04-25T01:36:02.359 INFO:tasks.workunit.client.0.smithi033.stdout:0/814: mknod d7/d2d/d9b/c117 0 2024-04-25T01:36:02.359 INFO:tasks.workunit.client.1.smithi138.stdout:7/343: chown da/de/c25 0 1 2024-04-25T01:36:02.360 INFO:tasks.workunit.client.1.smithi138.stdout:2/387: symlink d0/d5a/d15/d44/d4c/l78 0 2024-04-25T01:36:02.361 INFO:tasks.workunit.client.0.smithi033.stdout:7/873: symlink d4/d2e/d7f/d98/l114 0 2024-04-25T01:36:02.362 INFO:tasks.workunit.client.1.smithi138.stdout:6/428: symlink dc/d10/d67/l85 0 2024-04-25T01:36:02.365 INFO:tasks.workunit.client.0.smithi033.stdout:5/890: fdatasync d1c/d2a/d50/d5b/fb7 0 2024-04-25T01:36:02.367 INFO:tasks.workunit.client.1.smithi138.stdout:3/445: chown db/df/d17/d23/c3a 1083 1 2024-04-25T01:36:02.370 INFO:tasks.workunit.client.1.smithi138.stdout:6/429: write dc/d10/d67/f77 [2073569,127209] 0 2024-04-25T01:36:02.371 INFO:tasks.workunit.client.1.smithi138.stdout:0/286: mknod d6/d39/c62 0 2024-04-25T01:36:02.372 INFO:tasks.workunit.client.1.smithi138.stdout:8/510: dwrite d3/d16/d1f/d2a/f6a [0,4194304] 0 2024-04-25T01:36:02.374 INFO:tasks.workunit.client.0.smithi033.stdout:1/711: mkdir d2/de2 0 2024-04-25T01:36:02.374 INFO:tasks.workunit.client.0.smithi033.stdout:2/856: write da/d5b/d5e/d76/ffd [925793,102891] 0 2024-04-25T01:36:02.374 INFO:tasks.workunit.client.0.smithi033.stdout:8/840: dwrite d6/d45/d52/d2e/f56 [0,4194304] 0 2024-04-25T01:36:02.375 INFO:tasks.workunit.client.0.smithi033.stdout:7/874: read d4/d23/d3f/f60 [2177295,51819] 0 2024-04-25T01:36:02.377 INFO:tasks.workunit.client.1.smithi138.stdout:2/388: dwrite d0/d5a/d15/d51/f62 [0,4194304] 0 2024-04-25T01:36:02.377 INFO:tasks.workunit.client.1.smithi138.stdout:7/344: creat da/de/d13/f76 x:0 0 0 2024-04-25T01:36:02.379 INFO:tasks.workunit.client.1.smithi138.stdout:2/389: fsync d0/d21/f41 0 2024-04-25T01:36:02.379 INFO:tasks.workunit.client.1.smithi138.stdout:1/310: mkdir d3/d27/d3e/d51/d62 0 2024-04-25T01:36:02.380 INFO:tasks.workunit.client.1.smithi138.stdout:3/446: chown db/df/d17/d20/d57 13 1 2024-04-25T01:36:02.381 INFO:tasks.workunit.client.0.smithi033.stdout:1/712: chown d2/d5/d75/d59/d83/cc3 115160459 1 2024-04-25T01:36:02.381 INFO:tasks.workunit.client.0.smithi033.stdout:8/841: creat d6/d45/d52/dcc/d112/f137 x:0 0 0 2024-04-25T01:36:02.381 INFO:tasks.workunit.client.0.smithi033.stdout:7/875: symlink d4/d2e/d7f/d98/dbf/l115 0 2024-04-25T01:36:02.382 INFO:tasks.workunit.client.0.smithi033.stdout:2/857: symlink da/d27/d31/l117 0 2024-04-25T01:36:02.383 INFO:tasks.workunit.client.0.smithi033.stdout:7/876: chown d4/d2e/d6b/d1f/f46 5001632 1 2024-04-25T01:36:02.383 INFO:tasks.workunit.client.0.smithi033.stdout:5/891: truncate d1c/d2a/d50/d5b/fd7 866327 0 2024-04-25T01:36:02.383 INFO:tasks.workunit.client.1.smithi138.stdout:6/430: symlink dc/d10/d28/d80/l86 0 2024-04-25T01:36:02.384 INFO:tasks.workunit.client.0.smithi033.stdout:1/713: creat d2/d5/d1b/d46/d5f/d63/fe3 x:0 0 0 2024-04-25T01:36:02.385 INFO:tasks.workunit.client.0.smithi033.stdout:7/877: chown d4/d2e/d50/f55 1068266351 1 2024-04-25T01:36:02.385 INFO:tasks.workunit.client.0.smithi033.stdout:2/858: truncate da/f22 5113722 0 2024-04-25T01:36:02.386 INFO:tasks.workunit.client.1.smithi138.stdout:4/184: getdents d0 0 2024-04-25T01:36:02.387 INFO:tasks.workunit.client.0.smithi033.stdout:1/714: fdatasync d2/d2e/dcd/fc8 0 2024-04-25T01:36:02.387 INFO:tasks.workunit.client.0.smithi033.stdout:2/859: dread - da/d27/fdf zero size 2024-04-25T01:36:02.388 INFO:tasks.workunit.client.0.smithi033.stdout:7/878: read - d4/d2e/d6b/d101/f105 zero size 2024-04-25T01:36:02.391 INFO:tasks.workunit.client.0.smithi033.stdout:1/715: creat d2/d5/d1b/d46/d5f/d63/d8f/fe4 x:0 0 0 2024-04-25T01:36:02.391 INFO:tasks.workunit.client.0.smithi033.stdout:2/860: dread da/f74 [0,4194304] 0 2024-04-25T01:36:02.392 INFO:tasks.workunit.client.0.smithi033.stdout:1/716: stat d2/d5/d1b/d46/d5f/d63/c70 0 2024-04-25T01:36:02.393 INFO:tasks.workunit.client.1.smithi138.stdout:4/185: truncate d0/d1c/f2c 817380 0 2024-04-25T01:36:02.394 INFO:tasks.workunit.client.0.smithi033.stdout:2/861: mknod da/d5b/d5e/da8/c118 0 2024-04-25T01:36:02.395 INFO:tasks.workunit.client.0.smithi033.stdout:1/717: read d2/d11/f91 [1942080,94476] 0 2024-04-25T01:36:02.395 INFO:tasks.workunit.client.0.smithi033.stdout:6/985: sync 2024-04-25T01:36:02.395 INFO:tasks.workunit.client.1.smithi138.stdout:1/311: rmdir d3/d27/d3e/d51 39 2024-04-25T01:36:02.396 INFO:tasks.workunit.client.1.smithi138.stdout:2/390: dwrite d0/d5a/d15/f3b [0,4194304] 0 2024-04-25T01:36:02.397 INFO:tasks.workunit.client.1.smithi138.stdout:1/312: fdatasync d3/db/d2b/f42 0 2024-04-25T01:36:02.397 INFO:tasks.workunit.client.1.smithi138.stdout:8/511: dread d3/d16/d88/da/db/d15/d56/f9c [0,4194304] 0 2024-04-25T01:36:02.398 INFO:tasks.workunit.client.0.smithi033.stdout:6/986: read - d4/df1/f134 zero size 2024-04-25T01:36:02.399 INFO:tasks.workunit.client.0.smithi033.stdout:2/862: fsync da/d5b/d5e/fbd 0 2024-04-25T01:36:02.403 INFO:tasks.workunit.client.1.smithi138.stdout:3/447: rename f6 to db/d14/d40/d41/f84 0 2024-04-25T01:36:02.404 INFO:tasks.workunit.client.1.smithi138.stdout:3/448: readlink db/d14/l51 0 2024-04-25T01:36:02.404 INFO:tasks.workunit.client.0.smithi033.stdout:8/842: dread d6/db/d80/fa3 [0,4194304] 0 2024-04-25T01:36:02.404 INFO:tasks.workunit.client.0.smithi033.stdout:0/815: write d7/d2d/d4d/f82 [676705,129056] 0 2024-04-25T01:36:02.405 INFO:tasks.workunit.client.1.smithi138.stdout:6/431: creat dc/d10/d6d/f87 x:0 0 0 2024-04-25T01:36:02.408 INFO:tasks.workunit.client.1.smithi138.stdout:7/345: mkdir da/de/d1b/d4b/d77 0 2024-04-25T01:36:02.410 INFO:tasks.workunit.client.0.smithi033.stdout:3/895: dread d8/dd/d11/fad [0,4194304] 0 2024-04-25T01:36:02.414 INFO:tasks.workunit.client.0.smithi033.stdout:4/912: dread d6/f18 [0,4194304] 0 2024-04-25T01:36:02.415 INFO:tasks.workunit.client.1.smithi138.stdout:4/186: rename d0/c19 to d0/d1c/d23/c2d 0 2024-04-25T01:36:02.416 INFO:tasks.workunit.client.0.smithi033.stdout:9/860: dread d0/d2/d43/d4b/dc5/de0/d7e/fe4 [4194304,4194304] 0 2024-04-25T01:36:02.416 INFO:tasks.workunit.client.1.smithi138.stdout:0/287: getdents d6/d39/d52 0 2024-04-25T01:36:02.417 INFO:tasks.workunit.client.0.smithi033.stdout:8/843: getdents d6/d45/d52/df2/d12b 0 2024-04-25T01:36:02.418 INFO:tasks.workunit.client.0.smithi033.stdout:1/718: dwrite d2/d5/d1b/d46/d5f/d63/d8f/fe4 [0,4194304] 0 2024-04-25T01:36:02.418 INFO:tasks.workunit.client.0.smithi033.stdout:0/816: creat d7/da/df/f118 x:0 0 0 2024-04-25T01:36:02.419 INFO:tasks.workunit.client.0.smithi033.stdout:5/892: dread d1c/d2a/d50/d5b/fa1 [0,4194304] 0 2024-04-25T01:36:02.420 INFO:tasks.workunit.client.1.smithi138.stdout:8/512: mknod d3/d95/ca0 0 2024-04-25T01:36:02.423 INFO:tasks.workunit.client.0.smithi033.stdout:8/844: creat d6/dd3/f138 x:0 0 0 2024-04-25T01:36:02.425 INFO:tasks.workunit.client.0.smithi033.stdout:1/719: rmdir d2/d5/d1b/d46/d5f/d63/d64 39 2024-04-25T01:36:02.426 INFO:tasks.workunit.client.1.smithi138.stdout:0/288: dwrite d6/df/f15 [8388608,4194304] 0 2024-04-25T01:36:02.426 INFO:tasks.workunit.client.1.smithi138.stdout:0/289: write d6/df/f15 [6728641,100608] 0 2024-04-25T01:36:02.426 INFO:tasks.workunit.client.1.smithi138.stdout:1/313: link d3/db/f5f d3/db/d2b/d57/f63 0 2024-04-25T01:36:02.427 INFO:tasks.workunit.client.0.smithi033.stdout:5/893: dread d1c/d2a/d50/d5b/d86/d10b/f110 [0,4194304] 0 2024-04-25T01:36:02.427 INFO:tasks.workunit.client.1.smithi138.stdout:5/295: fsync d4/da/d27/d1c/d3c/d40/f55 0 2024-04-25T01:36:02.427 INFO:tasks.workunit.client.0.smithi033.stdout:3/896: creat d8/dd/d3d/d63/d7c/ded/dc7/ddf/d10b/d12d/f13e x:0 0 0 2024-04-25T01:36:02.428 INFO:tasks.workunit.client.1.smithi138.stdout:1/314: chown d3/f44 87257 1 2024-04-25T01:36:02.428 INFO:tasks.workunit.client.1.smithi138.stdout:5/296: readlink d4/da/d27/d1c/d22/d45/l57 0 2024-04-25T01:36:02.429 INFO:tasks.workunit.client.0.smithi033.stdout:5/894: truncate d1c/d2a/d50/f90 859701 0 2024-04-25T01:36:02.429 INFO:tasks.workunit.client.0.smithi033.stdout:0/817: rmdir d7/d2d/d43/d44/dd4 39 2024-04-25T01:36:02.429 INFO:tasks.workunit.client.0.smithi033.stdout:3/897: dread - d8/dd/d3d/d63/f95 zero size 2024-04-25T01:36:02.429 INFO:tasks.workunit.client.0.smithi033.stdout:5/895: fsync f15 0 2024-04-25T01:36:02.432 INFO:tasks.workunit.client.0.smithi033.stdout:3/898: chown d8/dd/d3d/d63/d7c/ded/dbf/dc5 30 1 2024-04-25T01:36:02.432 INFO:tasks.workunit.client.1.smithi138.stdout:6/432: link dc/d10/f68 dc/d10/d28/d70/d83/f88 0 2024-04-25T01:36:02.435 INFO:tasks.workunit.client.0.smithi033.stdout:8/845: fsync d6/db/d80/d89/faf 0 2024-04-25T01:36:02.435 INFO:tasks.workunit.client.0.smithi033.stdout:8/846: readlink d6/d45/d51/d55/deb/lfc 0 2024-04-25T01:36:02.436 INFO:tasks.workunit.client.1.smithi138.stdout:8/513: unlink d3/d16/d1f/d50/f9e 0 2024-04-25T01:36:02.436 INFO:tasks.workunit.client.0.smithi033.stdout:3/899: dread - d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/fbb zero size 2024-04-25T01:36:02.438 INFO:tasks.workunit.client.0.smithi033.stdout:8/847: truncate d6/db/d80/d89/dbf/f135 438524 0 2024-04-25T01:36:02.438 INFO:tasks.workunit.client.0.smithi033.stdout:4/913: link d6/d14/d35/d3a/d4d/d5e/d7c/d94/fe7 d6/d14/d48/f12a 0 2024-04-25T01:36:02.440 INFO:tasks.workunit.client.0.smithi033.stdout:0/818: rename d7/d2d/d4d/d9c/f84 to d7/d2d/d4d/f119 0 2024-04-25T01:36:02.440 INFO:tasks.workunit.client.1.smithi138.stdout:4/187: write d0/d16/f18 [885838,55269] 0 2024-04-25T01:36:02.441 INFO:tasks.workunit.client.1.smithi138.stdout:2/391: dwrite d0/d5a/f17 [0,4194304] 0 2024-04-25T01:36:02.442 INFO:tasks.workunit.client.0.smithi033.stdout:0/819: fdatasync d7/da/d80/fe4 0 2024-04-25T01:36:02.443 INFO:tasks.workunit.client.1.smithi138.stdout:2/392: write d0/d21/f27 [822698,6003] 0 2024-04-25T01:36:02.446 INFO:tasks.workunit.client.1.smithi138.stdout:0/290: creat d6/df/d41/d49/f63 x:0 0 0 2024-04-25T01:36:02.446 INFO:tasks.workunit.client.1.smithi138.stdout:5/297: write d4/da/f20 [845206,108300] 0 2024-04-25T01:36:02.446 INFO:tasks.workunit.client.0.smithi033.stdout:1/720: mkdir d2/de2/de5 0 2024-04-25T01:36:02.448 INFO:tasks.workunit.client.0.smithi033.stdout:2/863: dwrite da/ff [0,4194304] 0 2024-04-25T01:36:02.448 INFO:tasks.workunit.client.1.smithi138.stdout:1/315: creat d3/d27/d24/f64 x:0 0 0 2024-04-25T01:36:02.450 INFO:tasks.workunit.client.0.smithi033.stdout:3/900: symlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/l13f 0 2024-04-25T01:36:02.452 INFO:tasks.workunit.client.1.smithi138.stdout:2/393: read d0/d5a/d15/d51/f72 [107138,111958] 0 2024-04-25T01:36:02.452 INFO:tasks.workunit.client.1.smithi138.stdout:7/346: creat da/de/d13/d30/d59/f78 x:0 0 0 2024-04-25T01:36:02.453 INFO:tasks.workunit.client.1.smithi138.stdout:7/347: dread - da/de/f6c zero size 2024-04-25T01:36:02.453 INFO:tasks.workunit.client.0.smithi033.stdout:0/820: dwrite d7/f114 [0,4194304] 0 2024-04-25T01:36:02.455 INFO:tasks.workunit.client.1.smithi138.stdout:4/188: dread - d0/d1c/f2b zero size 2024-04-25T01:36:02.456 INFO:tasks.workunit.client.1.smithi138.stdout:3/449: getdents db/d14/d40 0 2024-04-25T01:36:02.456 INFO:tasks.workunit.client.1.smithi138.stdout:4/189: write d0/d16/f18 [1465694,114761] 0 2024-04-25T01:36:02.458 INFO:tasks.workunit.client.1.smithi138.stdout:4/190: dread - d0/d1c/f28 zero size 2024-04-25T01:36:02.459 INFO:tasks.workunit.client.1.smithi138.stdout:6/433: write dc/d2e/d37/d56/f73 [597181,93606] 0 2024-04-25T01:36:02.460 INFO:tasks.workunit.client.0.smithi033.stdout:3/901: write d8/dff/f132 [300615,59983] 0 2024-04-25T01:36:02.460 INFO:tasks.workunit.client.0.smithi033.stdout:5/896: mknod d1c/d2a/d50/d52/da6/d3e/d70/dae/dfb/c120 0 2024-04-25T01:36:02.461 INFO:tasks.workunit.client.1.smithi138.stdout:3/450: read db/df/d19/d1a/f21 [1289890,29791] 0 2024-04-25T01:36:02.461 INFO:tasks.workunit.client.1.smithi138.stdout:0/291: creat d6/df/d41/f64 x:0 0 0 2024-04-25T01:36:02.462 INFO:tasks.workunit.client.1.smithi138.stdout:5/298: stat d4/da/fe 0 2024-04-25T01:36:02.462 INFO:tasks.workunit.client.0.smithi033.stdout:9/861: link d0/c1 d0/d2/d43/d73/dc4/dcb/c12e 0 2024-04-25T01:36:02.462 INFO:tasks.workunit.client.0.smithi033.stdout:1/721: write d2/d2e/fdf [924641,69929] 0 2024-04-25T01:36:02.462 INFO:tasks.workunit.client.0.smithi033.stdout:0/821: mknod d7/d2d/d64/d51/c11a 0 2024-04-25T01:36:02.464 INFO:tasks.workunit.client.0.smithi033.stdout:1/722: fsync d2/d11/d28/f79 0 2024-04-25T01:36:02.464 INFO:tasks.workunit.client.0.smithi033.stdout:2/864: rename da/d50/d6a/l2e to da/d27/d31/d40/db1/dbf/dd5/l119 0 2024-04-25T01:36:02.465 INFO:tasks.workunit.client.0.smithi033.stdout:9/862: chown d0/d2/d43/d4b/l5e 6 1 2024-04-25T01:36:02.465 INFO:tasks.workunit.client.1.smithi138.stdout:2/394: fdatasync d0/d32/d35/f40 0 2024-04-25T01:36:02.466 INFO:tasks.workunit.client.0.smithi033.stdout:3/902: rmdir d8/dd/d3d/d63/d7c/d9d 39 2024-04-25T01:36:02.467 INFO:tasks.workunit.client.0.smithi033.stdout:0/822: readlink d7/d2d/d4d/d9c/d68/lbf 0 2024-04-25T01:36:02.467 INFO:tasks.workunit.client.0.smithi033.stdout:9/863: fdatasync d0/d2/d43/d73/fd1 0 2024-04-25T01:36:02.468 INFO:tasks.workunit.client.1.smithi138.stdout:4/191: mknod d0/d1c/d1e/c2e 0 2024-04-25T01:36:02.469 INFO:tasks.workunit.client.1.smithi138.stdout:6/434: unlink dc/d10/d28/f46 0 2024-04-25T01:36:02.471 INFO:tasks.workunit.client.1.smithi138.stdout:2/395: symlink d0/d18/l79 0 2024-04-25T01:36:02.472 INFO:tasks.workunit.client.0.smithi033.stdout:9/864: chown d0/d2/d43/d4b/dc5/de0/faa 228 1 2024-04-25T01:36:02.472 INFO:tasks.workunit.client.1.smithi138.stdout:2/396: stat d0/d32 0 2024-04-25T01:36:02.473 INFO:tasks.workunit.client.1.smithi138.stdout:1/316: mkdir d3/d27/d3e/d51/d62/d65 0 2024-04-25T01:36:02.473 INFO:tasks.workunit.client.0.smithi033.stdout:2/865: dwrite da/f12 [0,4194304] 0 2024-04-25T01:36:02.475 INFO:tasks.workunit.client.0.smithi033.stdout:0/823: symlink d7/de7/l11b 0 2024-04-25T01:36:02.475 INFO:tasks.workunit.client.1.smithi138.stdout:7/348: creat da/de/d13/d21/d6f/f79 x:0 0 0 2024-04-25T01:36:02.475 INFO:tasks.workunit.client.1.smithi138.stdout:3/451: dwrite db/df/d17/d23/d34/f5f [0,4194304] 0 2024-04-25T01:36:02.477 INFO:tasks.workunit.client.1.smithi138.stdout:1/317: chown d3/db/d2b/d57/f63 490428 1 2024-04-25T01:36:02.478 INFO:tasks.workunit.client.0.smithi033.stdout:0/824: write d7/da/df/f118 [199689,109510] 0 2024-04-25T01:36:02.478 INFO:tasks.workunit.client.0.smithi033.stdout:1/723: mkdir d2/de6 0 2024-04-25T01:36:02.479 INFO:tasks.workunit.client.0.smithi033.stdout:5/897: creat d1c/d2a/f121 x:0 0 0 2024-04-25T01:36:02.481 INFO:tasks.workunit.client.1.smithi138.stdout:3/452: read db/f12 [631084,88718] 0 2024-04-25T01:36:02.482 INFO:tasks.workunit.client.0.smithi033.stdout:0/825: chown d7/d2d/fc4 0 1 2024-04-25T01:36:02.482 INFO:tasks.workunit.client.1.smithi138.stdout:7/349: truncate da/de/d13/f23 5248873 0 2024-04-25T01:36:02.483 INFO:tasks.workunit.client.0.smithi033.stdout:8/848: write d6/f40 [972838,73763] 0 2024-04-25T01:36:02.485 INFO:tasks.workunit.client.1.smithi138.stdout:5/299: dread d4/da/fe [0,4194304] 0 2024-04-25T01:36:02.487 INFO:tasks.workunit.client.0.smithi033.stdout:3/903: creat d8/dd/d3d/d4f/da4/dc4/d11f/f140 x:0 0 0 2024-04-25T01:36:02.488 INFO:tasks.workunit.client.1.smithi138.stdout:3/453: symlink db/df/l85 0 2024-04-25T01:36:02.489 INFO:tasks.workunit.client.1.smithi138.stdout:1/318: symlink d3/d27/d3e/d51/d62/d65/l66 0 2024-04-25T01:36:02.491 INFO:tasks.workunit.client.0.smithi033.stdout:2/866: symlink da/d27/d31/d40/db1/dbf/dd5/de0/l11a 0 2024-04-25T01:36:02.491 INFO:tasks.workunit.client.0.smithi033.stdout:8/849: symlink d6/db/d80/db4/l139 0 2024-04-25T01:36:02.492 INFO:tasks.workunit.client.0.smithi033.stdout:0/826: chown d7/d2d/d4c/c111 2868 1 2024-04-25T01:36:02.493 INFO:tasks.workunit.client.1.smithi138.stdout:3/454: readlink db/df/d17/d23/d34/l42 0 2024-04-25T01:36:02.493 INFO:tasks.workunit.client.1.smithi138.stdout:5/300: dwrite d4/f1a [0,4194304] 0 2024-04-25T01:36:02.494 INFO:tasks.workunit.client.0.smithi033.stdout:9/865: dwrite d0/d2/d43/d73/dab/dad/dee/ff7 [0,4194304] 0 2024-04-25T01:36:02.495 INFO:tasks.workunit.client.1.smithi138.stdout:7/350: creat da/f7a x:0 0 0 2024-04-25T01:36:02.496 INFO:tasks.workunit.client.0.smithi033.stdout:5/898: creat d1c/d2a/d50/d52/f122 x:0 0 0 2024-04-25T01:36:02.496 INFO:tasks.workunit.client.0.smithi033.stdout:4/914: truncate d6/d14/d35/d3a/d4c/f78 3178416 0 2024-04-25T01:36:02.501 INFO:tasks.workunit.client.1.smithi138.stdout:6/435: write fb [1341772,85334] 0 2024-04-25T01:36:02.502 INFO:tasks.workunit.client.0.smithi033.stdout:1/724: dwrite d2/d2e/d34/d60/f7a [8388608,4194304] 0 2024-04-25T01:36:02.503 INFO:tasks.workunit.client.1.smithi138.stdout:4/192: truncate d0/f1a 2954985 0 2024-04-25T01:36:02.503 INFO:tasks.workunit.client.1.smithi138.stdout:5/301: dwrite d4/f1a [0,4194304] 0 2024-04-25T01:36:02.506 INFO:tasks.workunit.client.0.smithi033.stdout:0/827: creat d7/da/df/d50/dbb/f11c x:0 0 0 2024-04-25T01:36:02.507 INFO:tasks.workunit.client.1.smithi138.stdout:7/351: chown da/de/d13/d3e/d51/c71 5 1 2024-04-25T01:36:02.508 INFO:tasks.workunit.client.1.smithi138.stdout:1/319: link d3/d27/d24/f3b d3/db/f67 0 2024-04-25T01:36:02.509 INFO:tasks.workunit.client.0.smithi033.stdout:8/850: rename d6/d45/d51/d55/daa/cc0 to d6/d45/d51/d55/deb/c13a 0 2024-04-25T01:36:02.510 INFO:tasks.workunit.client.0.smithi033.stdout:0/828: fsync d7/da/df/d50/dbb/ff7 0 2024-04-25T01:36:02.513 INFO:tasks.workunit.client.0.smithi033.stdout:0/829: fsync d7/d2d/d64/fd1 0 2024-04-25T01:36:02.513 INFO:tasks.workunit.client.0.smithi033.stdout:3/904: creat d8/dd/d3d/d63/d7c/ded/dc7/ddf/d10b/f141 x:0 0 0 2024-04-25T01:36:02.513 INFO:tasks.workunit.client.1.smithi138.stdout:6/436: mkdir dc/d10/d28/d80/d89 0 2024-04-25T01:36:02.514 INFO:tasks.workunit.client.1.smithi138.stdout:1/320: truncate d3/db/d2b/f5d 44951 0 2024-04-25T01:36:02.514 INFO:tasks.workunit.client.1.smithi138.stdout:3/455: write db/df/d19/d1a/f21 [4662900,22940] 0 2024-04-25T01:36:02.514 INFO:tasks.workunit.client.1.smithi138.stdout:5/302: readlink d4/da/l30 0 2024-04-25T01:36:02.515 INFO:tasks.workunit.client.1.smithi138.stdout:7/352: mkdir da/de/d13/d30/d59/d7b 0 2024-04-25T01:36:02.516 INFO:tasks.workunit.client.1.smithi138.stdout:1/321: chown d3/d45/f56 3590822 1 2024-04-25T01:36:02.516 INFO:tasks.workunit.client.0.smithi033.stdout:9/866: creat d0/d2/dd/df/d1b/dc0/f12f x:0 0 0 2024-04-25T01:36:02.517 INFO:tasks.workunit.client.0.smithi033.stdout:1/725: symlink d2/d5/d1b/d46/d5f/d63/d64/le7 0 2024-04-25T01:36:02.518 INFO:tasks.workunit.client.1.smithi138.stdout:4/193: dwrite d0/d16/f18 [0,4194304] 0 2024-04-25T01:36:02.521 INFO:tasks.workunit.client.1.smithi138.stdout:6/437: mknod dc/d2e/d55/d7e/c8a 0 2024-04-25T01:36:02.522 INFO:tasks.workunit.client.0.smithi033.stdout:5/899: rename d1c/d2a/d50/d52/da6/c93 to d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/c123 0 2024-04-25T01:36:02.522 INFO:tasks.workunit.client.1.smithi138.stdout:5/303: write d4/da/d27/d1c/d3c/d40/f55 [847037,35689] 0 2024-04-25T01:36:02.524 INFO:tasks.workunit.client.1.smithi138.stdout:3/456: rmdir db/d14/d77 0 2024-04-25T01:36:02.526 INFO:tasks.workunit.client.0.smithi033.stdout:4/915: dwrite d6/d14/d48/fdb [0,4194304] 0 2024-04-25T01:36:02.526 INFO:tasks.workunit.client.1.smithi138.stdout:1/322: creat d3/d27/d3e/d51/d62/f68 x:0 0 0 2024-04-25T01:36:02.526 INFO:tasks.workunit.client.1.smithi138.stdout:2/397: dread d0/f9 [0,4194304] 0 2024-04-25T01:36:02.527 INFO:tasks.workunit.client.1.smithi138.stdout:6/438: rmdir dc/d2e/d55/d7e 39 2024-04-25T01:36:02.527 INFO:tasks.workunit.client.1.smithi138.stdout:1/323: read - d3/f44 zero size 2024-04-25T01:36:02.527 INFO:tasks.workunit.client.0.smithi033.stdout:9/867: symlink d0/d2/dd/dba/l130 0 2024-04-25T01:36:02.529 INFO:tasks.workunit.client.1.smithi138.stdout:6/439: write dc/d10/f22 [1025302,3014] 0 2024-04-25T01:36:02.529 INFO:tasks.workunit.client.1.smithi138.stdout:2/398: truncate d0/d32/f63 582848 0 2024-04-25T01:36:02.532 INFO:tasks.workunit.client.1.smithi138.stdout:3/457: mknod db/d14/c86 0 2024-04-25T01:36:02.532 INFO:tasks.workunit.client.1.smithi138.stdout:5/304: write d4/da/d27/f33 [409220,18902] 0 2024-04-25T01:36:02.534 INFO:tasks.workunit.client.1.smithi138.stdout:3/458: chown db/f35 497614 1 2024-04-25T01:36:02.538 INFO:tasks.workunit.client.0.smithi033.stdout:5/900: chown d1c/d2a/d50/d52/c5a 58743 1 2024-04-25T01:36:02.541 INFO:tasks.workunit.client.0.smithi033.stdout:0/830: getdents d7/d2d/d4c/da0 0 2024-04-25T01:36:02.541 INFO:tasks.workunit.client.1.smithi138.stdout:3/459: creat db/d14/d40/f87 x:0 0 0 2024-04-25T01:36:02.542 INFO:tasks.workunit.client.0.smithi033.stdout:9/868: dread d0/d2/d43/d4b/dc5/de0/f70 [0,4194304] 0 2024-04-25T01:36:02.542 INFO:tasks.workunit.client.0.smithi033.stdout:4/916: symlink d6/d12/l12b 0 2024-04-25T01:36:02.542 INFO:tasks.workunit.client.1.smithi138.stdout:1/324: creat d3/db/d4a/f69 x:0 0 0 2024-04-25T01:36:02.542 INFO:tasks.workunit.client.1.smithi138.stdout:6/440: rmdir dc/d10/d28 39 2024-04-25T01:36:02.543 INFO:tasks.workunit.client.0.smithi033.stdout:3/905: dread d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/f8a [0,4194304] 0 2024-04-25T01:36:02.544 INFO:tasks.workunit.client.1.smithi138.stdout:3/460: write db/d14/d40/f60 [11821819,37948] 0 2024-04-25T01:36:02.545 INFO:tasks.workunit.client.1.smithi138.stdout:6/441: truncate dc/d10/d67/f65 682508 0 2024-04-25T01:36:02.546 INFO:tasks.workunit.client.1.smithi138.stdout:2/399: rename d0/d5a/d15/d44/l57 to d0/d5a/d15/d51/l7a 0 2024-04-25T01:36:02.546 INFO:tasks.workunit.client.0.smithi033.stdout:5/901: stat d1c/d2a/d50/d52/da6/d3e/d70/dae/dfb 0 2024-04-25T01:36:02.546 INFO:tasks.workunit.client.1.smithi138.stdout:2/400: fsync d0/d5a/d15/d51/f62 0 2024-04-25T01:36:02.547 INFO:tasks.workunit.client.1.smithi138.stdout:4/194: rename d0/f2a to d0/f2f 0 2024-04-25T01:36:02.550 INFO:tasks.workunit.client.0.smithi033.stdout:2/867: dwrite da/d27/d31/d40/f5f [0,4194304] 0 2024-04-25T01:36:02.550 INFO:tasks.workunit.client.1.smithi138.stdout:2/401: read d0/f10 [886298,39253] 0 2024-04-25T01:36:02.553 INFO:tasks.workunit.client.0.smithi033.stdout:1/726: dwrite d2/d5/d1b/d46/d5f/d63/d64/fa5 [4194304,4194304] 0 2024-04-25T01:36:02.553 INFO:tasks.workunit.client.0.smithi033.stdout:4/917: mkdir d6/d14/d35/d44/dfe/d12c 0 2024-04-25T01:36:02.555 INFO:tasks.workunit.client.1.smithi138.stdout:3/461: mkdir db/df/d17/d23/d88 0 2024-04-25T01:36:02.555 INFO:tasks.workunit.client.0.smithi033.stdout:9/869: creat d0/d2/d43/d4b/dc5/de0/d6e/f131 x:0 0 0 2024-04-25T01:36:02.557 INFO:tasks.workunit.client.0.smithi033.stdout:9/870: fsync d0/d2/dd/d20/f123 0 2024-04-25T01:36:02.560 INFO:tasks.workunit.client.1.smithi138.stdout:7/353: dwrite da/de/d13/f52 [0,4194304] 0 2024-04-25T01:36:02.561 INFO:tasks.workunit.client.1.smithi138.stdout:7/354: stat da/de/c25 0 2024-04-25T01:36:02.562 INFO:tasks.workunit.client.0.smithi033.stdout:3/906: creat d8/dd/d3d/d63/d7c/ded/dc7/ddf/f142 x:0 0 0 2024-04-25T01:36:02.562 INFO:tasks.workunit.client.0.smithi033.stdout:0/831: dwrite d7/d33/f36 [4194304,4194304] 0 2024-04-25T01:36:02.562 INFO:tasks.workunit.client.0.smithi033.stdout:5/902: fsync d1c/d2a/d50/d52/da6/d4a/f61 0 2024-04-25T01:36:02.564 INFO:tasks.workunit.client.0.smithi033.stdout:5/903: chown d1c/d2a/d3d/c41 12578464 1 2024-04-25T01:36:02.566 INFO:tasks.workunit.client.0.smithi033.stdout:2/868: creat da/d50/d6a/d43/d6c/df5/f11b x:0 0 0 2024-04-25T01:36:02.567 INFO:tasks.workunit.client.0.smithi033.stdout:1/727: write d2/d5/d1b/d46/d5f/d63/d9d/fd1 [200260,94741] 0 2024-04-25T01:36:02.570 INFO:tasks.workunit.client.0.smithi033.stdout:9/871: write d0/d2/d43/d4b/dc5/de0/d42/dea/d105/f111 [539771,10859] 0 2024-04-25T01:36:02.570 INFO:tasks.workunit.client.0.smithi033.stdout:1/728: write d2/d11/d28/f79 [7100600,42301] 0 2024-04-25T01:36:02.573 INFO:tasks.workunit.client.1.smithi138.stdout:4/195: chown d0/d1c 66261767 1 2024-04-25T01:36:02.574 INFO:tasks.workunit.client.1.smithi138.stdout:5/305: getdents d4/da/d27/d1c 0 2024-04-25T01:36:02.575 INFO:tasks.workunit.client.1.smithi138.stdout:2/402: symlink d0/d5a/d15/l7b 0 2024-04-25T01:36:02.577 INFO:tasks.workunit.client.1.smithi138.stdout:7/355: read da/de/d13/d30/f42 [2596500,127264] 0 2024-04-25T01:36:02.579 INFO:tasks.workunit.client.1.smithi138.stdout:4/196: fdatasync d0/fa 0 2024-04-25T01:36:02.580 INFO:tasks.workunit.client.1.smithi138.stdout:3/462: dwrite db/df/d17/d23/f79 [0,4194304] 0 2024-04-25T01:36:02.580 INFO:tasks.workunit.client.1.smithi138.stdout:7/356: chown da/de/d1b/d4b/f55 540619 1 2024-04-25T01:36:02.581 INFO:tasks.workunit.client.0.smithi033.stdout:9/872: dwrite d0/d2/dd/d5f/f9b [0,4194304] 0 2024-04-25T01:36:02.583 INFO:tasks.workunit.client.0.smithi033.stdout:1/729: rename d2/d5/d75/d59/d83/fa9 to d2/d5/d75/fe8 0 2024-04-25T01:36:02.583 INFO:tasks.workunit.client.1.smithi138.stdout:2/403: symlink d0/d5a/d15/d44/d4c/d56/l7c 0 2024-04-25T01:36:02.583 INFO:tasks.workunit.client.1.smithi138.stdout:4/197: dread - d0/d16/f29 zero size 2024-04-25T01:36:02.585 INFO:tasks.workunit.client.1.smithi138.stdout:4/198: chown d0/d16/f18 3644 1 2024-04-25T01:36:02.586 INFO:tasks.workunit.client.1.smithi138.stdout:5/306: rename d4/da/d27/d1c/d22/c2f to d4/da/d27/d1c/d3c/d40/c61 0 2024-04-25T01:36:02.587 INFO:tasks.workunit.client.1.smithi138.stdout:5/307: chown d4/da/d27/d1c/d3c/d40 92 1 2024-04-25T01:36:02.588 INFO:tasks.workunit.client.1.smithi138.stdout:7/357: symlink da/de/d13/d3e/d51/l7c 0 2024-04-25T01:36:02.593 INFO:tasks.workunit.client.0.smithi033.stdout:9/873: mknod d0/d2/d43/d73/dc4/c132 0 2024-04-25T01:36:02.593 INFO:tasks.workunit.client.0.smithi033.stdout:1/730: dread d2/d2e/d34/d60/f74 [0,4194304] 0 2024-04-25T01:36:02.594 INFO:tasks.workunit.client.1.smithi138.stdout:3/463: creat db/d14/d40/d41/f89 x:0 0 0 2024-04-25T01:36:02.595 INFO:tasks.workunit.client.1.smithi138.stdout:4/199: rmdir d0 39 2024-04-25T01:36:02.596 INFO:tasks.workunit.client.0.smithi033.stdout:3/907: mkdir d8/dd/d3d/d63/d7c/d9d/de8/d101/d143 0 2024-04-25T01:36:02.596 INFO:tasks.workunit.client.1.smithi138.stdout:3/464: chown db/f15 5819002 1 2024-04-25T01:36:02.597 INFO:tasks.workunit.client.1.smithi138.stdout:1/325: dread d3/d4/f54 [0,4194304] 0 2024-04-25T01:36:02.598 INFO:tasks.workunit.client.1.smithi138.stdout:5/308: dread d4/da/f20 [0,4194304] 0 2024-04-25T01:36:02.598 INFO:tasks.workunit.client.0.smithi033.stdout:3/908: chown d8/dd/d3d/d63/f71 1 1 2024-04-25T01:36:02.598 INFO:tasks.workunit.client.0.smithi033.stdout:2/869: link da/d5b/d5e/f114 da/d5b/d5e/f11c 0 2024-04-25T01:36:02.598 INFO:tasks.workunit.client.0.smithi033.stdout:4/918: link d6/d14/d35/d3a/l10a d6/d14/d35/l12d 0 2024-04-25T01:36:02.601 INFO:tasks.workunit.client.1.smithi138.stdout:7/358: rename da/de/l5e to da/de/d13/l7d 0 2024-04-25T01:36:02.602 INFO:tasks.workunit.client.0.smithi033.stdout:9/874: dwrite d0/d2/dd/df/d1b/f4d [4194304,4194304] 0 2024-04-25T01:36:02.603 INFO:tasks.workunit.client.1.smithi138.stdout:5/309: dread d4/da/d27/f33 [0,4194304] 0 2024-04-25T01:36:02.604 INFO:tasks.workunit.client.1.smithi138.stdout:3/465: symlink db/df/d17/d23/d88/l8a 0 2024-04-25T01:36:02.604 INFO:tasks.workunit.client.1.smithi138.stdout:7/359: stat da/de/d1b/d34/l72 0 2024-04-25T01:36:02.605 INFO:tasks.workunit.client.1.smithi138.stdout:4/200: creat d0/f30 x:0 0 0 2024-04-25T01:36:02.610 INFO:tasks.workunit.client.1.smithi138.stdout:2/404: dread d0/d5a/d15/f28 [0,4194304] 0 2024-04-25T01:36:02.612 INFO:tasks.workunit.client.1.smithi138.stdout:1/326: dwrite d3/db/d2b/f52 [0,4194304] 0 2024-04-25T01:36:02.612 INFO:tasks.workunit.client.1.smithi138.stdout:7/360: chown da/de/d13/d21/l5d 7929 1 2024-04-25T01:36:02.613 INFO:tasks.workunit.client.0.smithi033.stdout:0/832: getdents d7/d2d 0 2024-04-25T01:36:02.615 INFO:tasks.workunit.client.1.smithi138.stdout:5/310: dread d4/da/d27/d1c/d22/d35/f56 [0,4194304] 0 2024-04-25T01:36:02.617 INFO:tasks.workunit.client.1.smithi138.stdout:2/405: dread d0/d32/f63 [0,4194304] 0 2024-04-25T01:36:02.617 INFO:tasks.workunit.client.1.smithi138.stdout:3/466: dread db/d14/d40/d41/f7c [0,4194304] 0 2024-04-25T01:36:02.617 INFO:tasks.workunit.client.0.smithi033.stdout:2/870: dwrite da/d5b/d72/fbe [0,4194304] 0 2024-04-25T01:36:02.622 INFO:tasks.workunit.client.0.smithi033.stdout:5/904: write d1c/d2a/d50/d52/da6/d3e/d70/dae/fb5 [458328,66755] 0 2024-04-25T01:36:02.626 INFO:tasks.workunit.client.1.smithi138.stdout:1/327: dwrite d3/d4/f37 [0,4194304] 0 2024-04-25T01:36:02.629 INFO:tasks.workunit.client.1.smithi138.stdout:3/467: rename db/df/f25 to db/d14/d40/d41/f8b 0 2024-04-25T01:36:02.634 INFO:tasks.workunit.client.1.smithi138.stdout:2/406: dread d0/f9 [0,4194304] 0 2024-04-25T01:36:02.634 INFO:tasks.workunit.client.1.smithi138.stdout:2/407: fdatasync d0/d21/f3a 0 2024-04-25T01:36:02.634 INFO:tasks.workunit.client.1.smithi138.stdout:5/311: creat d4/f62 x:0 0 0 2024-04-25T01:36:02.634 INFO:tasks.workunit.client.1.smithi138.stdout:1/328: fsync d3/db/f67 0 2024-04-25T01:36:02.635 INFO:tasks.workunit.client.1.smithi138.stdout:5/312: stat d4/da/d27/c3e 0 2024-04-25T01:36:02.636 INFO:tasks.workunit.client.1.smithi138.stdout:4/201: dwrite d0/f13 [0,4194304] 0 2024-04-25T01:36:02.637 INFO:tasks.workunit.client.1.smithi138.stdout:5/313: stat d4/da/d27/d1c/d22/d45/l50 0 2024-04-25T01:36:02.637 INFO:tasks.workunit.client.1.smithi138.stdout:2/408: mknod d0/d5a/d15/d44/d4c/d56/c7d 0 2024-04-25T01:36:02.639 INFO:tasks.workunit.client.1.smithi138.stdout:1/329: symlink d3/d27/d3e/d51/d62/d65/l6a 0 2024-04-25T01:36:02.639 INFO:tasks.workunit.client.1.smithi138.stdout:4/202: creat d0/d16/f31 x:0 0 0 2024-04-25T01:36:02.644 INFO:tasks.workunit.client.0.smithi033.stdout:4/919: dwrite d6/d12/d84/f88 [0,4194304] 0 2024-04-25T01:36:02.653 INFO:tasks.workunit.client.0.smithi033.stdout:5/905: dread d1c/d2a/d3d/f46 [0,4194304] 0 2024-04-25T01:36:02.654 INFO:tasks.workunit.client.0.smithi033.stdout:5/906: chown d1c/d2a/d50/d52/da6/d3e/d70/d89/ff9 1 1 2024-04-25T01:36:02.662 INFO:tasks.workunit.client.0.smithi033.stdout:9/875: mkdir d0/d2/d43/d73/dab/dad/d133 0 2024-04-25T01:36:02.666 INFO:tasks.workunit.client.0.smithi033.stdout:4/920: fsync d6/d14/d1d/f20 0 2024-04-25T01:36:02.669 INFO:tasks.workunit.client.0.smithi033.stdout:3/909: rename d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/cb4 to d8/dff/c144 0 2024-04-25T01:36:02.670 INFO:tasks.workunit.client.0.smithi033.stdout:9/876: mknod d0/d2/dd/dba/c134 0 2024-04-25T01:36:02.671 INFO:tasks.workunit.client.0.smithi033.stdout:2/871: symlink da/d8d/d10c/d110/l11d 0 2024-04-25T01:36:02.673 INFO:tasks.workunit.client.0.smithi033.stdout:5/907: fsync d1c/d2a/d3d/f78 0 2024-04-25T01:36:02.675 INFO:tasks.workunit.client.0.smithi033.stdout:9/877: dread d0/d2/d43/d4b/dc5/de0/d42/dea/d105/f111 [0,4194304] 0 2024-04-25T01:36:02.676 INFO:tasks.workunit.client.0.smithi033.stdout:3/910: rmdir d8/dd/d3d/d4f/da4/dc4 39 2024-04-25T01:36:02.676 INFO:tasks.workunit.client.0.smithi033.stdout:9/878: chown d0/d2/dd/df/cb4 570 1 2024-04-25T01:36:02.683 INFO:tasks.workunit.client.0.smithi033.stdout:5/908: unlink d1c/d2a/d50/d52/da6/l33 0 2024-04-25T01:36:02.683 INFO:tasks.workunit.client.0.smithi033.stdout:5/909: stat d1c/d2a/l3b 0 2024-04-25T01:36:02.686 INFO:tasks.workunit.client.0.smithi033.stdout:9/879: unlink d0/d2/dd/d5f/fd0 0 2024-04-25T01:36:02.688 INFO:tasks.workunit.client.0.smithi033.stdout:1/731: link d2/d5/d75/c97 d2/d2e/ce9 0 2024-04-25T01:36:02.693 INFO:tasks.workunit.client.0.smithi033.stdout:5/910: dwrite d1c/d2a/d50/d52/da6/d3e/d70/d7a/f118 [0,4194304] 0 2024-04-25T01:36:02.695 INFO:tasks.workunit.client.0.smithi033.stdout:1/732: mknod d2/d5/d6c/cea 0 2024-04-25T01:36:02.696 INFO:tasks.workunit.client.0.smithi033.stdout:4/921: dread d6/d14/d1d/d3b/f93 [0,4194304] 0 2024-04-25T01:36:02.697 INFO:tasks.workunit.client.0.smithi033.stdout:1/733: write d2/d5/d1b/f5a [2313800,119984] 0 2024-04-25T01:36:02.699 INFO:tasks.workunit.client.0.smithi033.stdout:3/911: rename d8/dd/fcc to d8/dd/dfa/f145 0 2024-04-25T01:36:02.701 INFO:tasks.workunit.client.0.smithi033.stdout:5/911: mkdir d1c/d2a/d50/d5b/d86/d10b/d124 0 2024-04-25T01:36:02.706 INFO:tasks.workunit.client.0.smithi033.stdout:1/734: unlink d2/d5/d1b/d46/lc4 0 2024-04-25T01:36:02.708 INFO:tasks.workunit.client.0.smithi033.stdout:9/880: getdents d0/d2/d43/d73/dab/dad/dee 0 2024-04-25T01:36:02.711 INFO:tasks.workunit.client.0.smithi033.stdout:4/922: dwrite d6/d14/d35/d44/f119 [0,4194304] 0 2024-04-25T01:36:02.712 INFO:tasks.workunit.client.1.smithi138.stdout:1/330: unlink d3/d45/f56 0 2024-04-25T01:36:02.713 INFO:tasks.workunit.client.1.smithi138.stdout:5/314: dread d4/f1a [0,4194304] 0 2024-04-25T01:36:02.713 INFO:tasks.workunit.client.0.smithi033.stdout:5/912: unlink d1c/d2a/d50/d52/da6/d3e/d70/dae/dfb/c108 0 2024-04-25T01:36:02.715 INFO:tasks.workunit.client.0.smithi033.stdout:9/881: truncate d0/d2/d43/f8e 606946 0 2024-04-25T01:36:02.715 INFO:tasks.workunit.client.1.smithi138.stdout:5/315: read d4/da/d27/d1c/d3c/d40/f58 [1636169,43309] 0 2024-04-25T01:36:02.715 INFO:tasks.workunit.client.1.smithi138.stdout:7/361: truncate da/de/d13/f23 716761 0 2024-04-25T01:36:02.717 INFO:tasks.workunit.client.1.smithi138.stdout:7/362: chown da/de/f74 819 1 2024-04-25T01:36:02.717 INFO:tasks.workunit.client.0.smithi033.stdout:1/735: dwrite d2/d11/d28/fd9 [0,4194304] 0 2024-04-25T01:36:02.717 INFO:tasks.workunit.client.0.smithi033.stdout:4/923: symlink d6/d14/df6/d10e/l12e 0 2024-04-25T01:36:02.720 INFO:tasks.workunit.client.0.smithi033.stdout:4/924: dread d6/d14/d6d/f83 [0,4194304] 0 2024-04-25T01:36:02.723 INFO:tasks.workunit.client.0.smithi033.stdout:9/882: dread d0/d11/da1/fd3 [0,4194304] 0 2024-04-25T01:36:02.724 INFO:tasks.workunit.client.0.smithi033.stdout:4/925: chown d6/d14/d35/d44/fe1 8097655 1 2024-04-25T01:36:02.725 INFO:tasks.workunit.client.0.smithi033.stdout:7/879: sync 2024-04-25T01:36:02.730 INFO:tasks.workunit.client.0.smithi033.stdout:1/736: link d2/d5/d75/f8d d2/d5/d1b/dd2/feb 0 2024-04-25T01:36:02.735 INFO:tasks.workunit.client.0.smithi033.stdout:7/880: link d4/d2e/d50/f55 d4/d2e/f116 0 2024-04-25T01:36:02.738 INFO:tasks.workunit.client.0.smithi033.stdout:7/881: creat d4/d2e/d6b/da9/db3/f117 x:0 0 0 2024-04-25T01:36:02.741 INFO:tasks.workunit.client.0.smithi033.stdout:7/882: mkdir d4/d23/d3f/d45/d118 0 2024-04-25T01:36:02.742 INFO:tasks.workunit.client.1.smithi138.stdout:4/203: write d0/f2f [189998,21176] 0 2024-04-25T01:36:02.742 INFO:tasks.workunit.client.1.smithi138.stdout:5/316: write d4/da/f20 [844671,86627] 0 2024-04-25T01:36:02.744 INFO:tasks.workunit.client.0.smithi033.stdout:7/883: mkdir d4/d10/d119 0 2024-04-25T01:36:02.744 INFO:tasks.workunit.client.1.smithi138.stdout:1/331: link d3/d27/f1e d3/d27/d3e/f6b 0 2024-04-25T01:36:02.745 INFO:tasks.workunit.client.0.smithi033.stdout:7/884: stat d4/d2e/d6b/d1f/f46 0 2024-04-25T01:36:02.748 INFO:tasks.workunit.client.1.smithi138.stdout:1/332: read d3/db/f1d [1990998,93029] 0 2024-04-25T01:36:02.749 INFO:tasks.workunit.client.1.smithi138.stdout:5/317: symlink d4/da/d27/d1c/d22/d35/d4d/l63 0 2024-04-25T01:36:02.749 INFO:tasks.workunit.client.1.smithi138.stdout:2/409: dread d0/d32/d35/f60 [0,4194304] 0 2024-04-25T01:36:02.749 INFO:tasks.workunit.client.1.smithi138.stdout:1/333: fdatasync d3/f44 0 2024-04-25T01:36:02.753 INFO:tasks.workunit.client.1.smithi138.stdout:5/318: fdatasync d4/f1a 0 2024-04-25T01:36:02.754 INFO:tasks.workunit.client.1.smithi138.stdout:7/363: truncate da/de/d1b/d4b/f55 3136831 0 2024-04-25T01:36:02.757 INFO:tasks.workunit.client.1.smithi138.stdout:1/334: creat d3/d4/d53/f6c x:0 0 0 2024-04-25T01:36:02.757 INFO:tasks.workunit.client.1.smithi138.stdout:4/204: write d0/fa [808345,106005] 0 2024-04-25T01:36:02.758 INFO:tasks.workunit.client.1.smithi138.stdout:4/205: fsync d0/f2f 0 2024-04-25T01:36:02.758 INFO:tasks.workunit.client.1.smithi138.stdout:5/319: write d4/da/d27/d1c/d3c/d40/f5a [589812,45402] 0 2024-04-25T01:36:02.760 INFO:tasks.workunit.client.1.smithi138.stdout:4/206: dread - d0/d16/f29 zero size 2024-04-25T01:36:02.760 INFO:tasks.workunit.client.1.smithi138.stdout:5/320: write d4/da/f20 [887068,50048] 0 2024-04-25T01:36:02.764 INFO:tasks.workunit.client.1.smithi138.stdout:2/410: rename d0/d5a/d15/d1f to d0/d5a/d15/d7e 0 2024-04-25T01:36:02.765 INFO:tasks.workunit.client.1.smithi138.stdout:4/207: readlink d0/d1c/l1d 0 2024-04-25T01:36:02.767 INFO:tasks.workunit.client.1.smithi138.stdout:1/335: mkdir d3/d6d 0 2024-04-25T01:36:02.772 INFO:tasks.workunit.client.1.smithi138.stdout:7/364: write da/de/d13/f23 [1287488,47541] 0 2024-04-25T01:36:02.773 INFO:tasks.workunit.client.1.smithi138.stdout:4/208: dwrite d0/d16/f18 [0,4194304] 0 2024-04-25T01:36:02.774 INFO:tasks.workunit.client.1.smithi138.stdout:4/209: truncate d0/fa 1497095 0 2024-04-25T01:36:02.776 INFO:tasks.workunit.client.1.smithi138.stdout:2/411: symlink d0/d32/d35/l7f 0 2024-04-25T01:36:02.778 INFO:tasks.workunit.client.1.smithi138.stdout:4/210: creat d0/d1c/d23/f32 x:0 0 0 2024-04-25T01:36:02.779 INFO:tasks.workunit.client.1.smithi138.stdout:4/211: stat d0 0 2024-04-25T01:36:02.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/321: dread d4/da/d27/d1c/d22/f24 [4194304,4194304] 0 2024-04-25T01:36:02.785 INFO:tasks.workunit.client.1.smithi138.stdout:4/212: mkdir d0/d16/d33 0 2024-04-25T01:36:02.785 INFO:tasks.workunit.client.1.smithi138.stdout:4/213: stat d0/d1c/d23/f32 0 2024-04-25T01:36:02.787 INFO:tasks.workunit.client.1.smithi138.stdout:5/322: mknod d4/da/d27/d1c/d5c/c64 0 2024-04-25T01:36:02.792 INFO:tasks.workunit.client.1.smithi138.stdout:5/323: symlink d4/da/d27/d1c/d5c/l65 0 2024-04-25T01:36:02.793 INFO:tasks.workunit.client.1.smithi138.stdout:7/365: getdents da/de/d13/d3e/d51/d6e 0 2024-04-25T01:36:02.794 INFO:tasks.workunit.client.1.smithi138.stdout:1/336: dwrite d3/d27/f1e [0,4194304] 0 2024-04-25T01:36:02.794 INFO:tasks.workunit.client.1.smithi138.stdout:5/324: write d4/f9 [2137400,111566] 0 2024-04-25T01:36:02.797 INFO:tasks.workunit.client.1.smithi138.stdout:4/214: link d0/f13 d0/d16/f34 0 2024-04-25T01:36:02.798 INFO:tasks.workunit.client.1.smithi138.stdout:2/412: dread d0/d18/f6c [0,4194304] 0 2024-04-25T01:36:02.799 INFO:tasks.workunit.client.1.smithi138.stdout:1/337: write d3/d4/f37 [4083964,116319] 0 2024-04-25T01:36:02.801 INFO:tasks.workunit.client.1.smithi138.stdout:7/366: creat da/de/d13/d21/f7e x:0 0 0 2024-04-25T01:36:02.801 INFO:tasks.workunit.client.1.smithi138.stdout:5/325: stat d4/da/d27/d1c/d22/d35/l46 0 2024-04-25T01:36:02.802 INFO:tasks.workunit.client.1.smithi138.stdout:4/215: creat d0/d1c/d23/f35 x:0 0 0 2024-04-25T01:36:02.805 INFO:tasks.workunit.client.1.smithi138.stdout:2/413: creat d0/d5a/d15/d51/f80 x:0 0 0 2024-04-25T01:36:02.807 INFO:tasks.workunit.client.1.smithi138.stdout:7/367: write da/de/d13/d3e/f4c [315824,29160] 0 2024-04-25T01:36:02.808 INFO:tasks.workunit.client.1.smithi138.stdout:7/368: chown da/de/f6c 719 1 2024-04-25T01:36:02.808 INFO:tasks.workunit.client.1.smithi138.stdout:4/216: write d0/d16/f34 [1311709,59383] 0 2024-04-25T01:36:02.809 INFO:tasks.workunit.client.1.smithi138.stdout:5/326: unlink d4/c25 0 2024-04-25T01:36:02.812 INFO:tasks.workunit.client.1.smithi138.stdout:4/217: chown d0/fa 6115 1 2024-04-25T01:36:02.813 INFO:tasks.workunit.client.1.smithi138.stdout:7/369: symlink da/de/d13/d3e/l7f 0 2024-04-25T01:36:02.813 INFO:tasks.workunit.client.1.smithi138.stdout:4/218: write d0/d1c/f2b [496146,7255] 0 2024-04-25T01:36:02.814 INFO:tasks.workunit.client.1.smithi138.stdout:4/219: dread - d0/d16/f31 zero size 2024-04-25T01:36:02.814 INFO:tasks.workunit.client.1.smithi138.stdout:5/327: rmdir d4/da/d27/d1c 39 2024-04-25T01:36:02.818 INFO:tasks.workunit.client.1.smithi138.stdout:5/328: fdatasync d4/da/d27/d1c/d5c/f5d 0 2024-04-25T01:36:02.819 INFO:tasks.workunit.client.1.smithi138.stdout:1/338: write d3/db/f5f [89174,113515] 0 2024-04-25T01:36:02.822 INFO:tasks.workunit.client.1.smithi138.stdout:1/339: creat d3/db/d4a/f6e x:0 0 0 2024-04-25T01:36:02.824 INFO:tasks.workunit.client.1.smithi138.stdout:1/340: chown d3/d27/f1e 19 1 2024-04-25T01:36:02.825 INFO:tasks.workunit.client.1.smithi138.stdout:5/329: link d4/da/fe d4/da/d27/d1c/d22/d35/d4d/f66 0 2024-04-25T01:36:02.827 INFO:tasks.workunit.client.1.smithi138.stdout:4/220: dwrite d0/f13 [0,4194304] 0 2024-04-25T01:36:02.828 INFO:tasks.workunit.client.1.smithi138.stdout:1/341: symlink d3/db/d4a/l6f 0 2024-04-25T01:36:02.830 INFO:tasks.workunit.client.1.smithi138.stdout:5/330: creat d4/da/d27/d1c/d22/d45/f67 x:0 0 0 2024-04-25T01:36:02.830 INFO:tasks.workunit.client.1.smithi138.stdout:2/414: dwrite d0/d32/f4b [0,4194304] 0 2024-04-25T01:36:02.832 INFO:tasks.workunit.client.1.smithi138.stdout:7/370: dread da/f15 [0,4194304] 0 2024-04-25T01:36:02.832 INFO:tasks.workunit.client.1.smithi138.stdout:4/221: symlink d0/d1c/d23/l36 0 2024-04-25T01:36:02.833 INFO:tasks.workunit.client.1.smithi138.stdout:1/342: symlink d3/d6d/l70 0 2024-04-25T01:36:02.835 INFO:tasks.workunit.client.1.smithi138.stdout:2/415: creat d0/d32/f81 x:0 0 0 2024-04-25T01:36:02.836 INFO:tasks.workunit.client.1.smithi138.stdout:1/343: write d3/db/d2b/d57/f63 [375725,82066] 0 2024-04-25T01:36:02.839 INFO:tasks.workunit.client.1.smithi138.stdout:4/222: dread d0/fa [0,4194304] 0 2024-04-25T01:36:02.840 INFO:tasks.workunit.client.1.smithi138.stdout:7/371: dread da/de/f29 [0,4194304] 0 2024-04-25T01:36:02.842 INFO:tasks.workunit.client.1.smithi138.stdout:2/416: write d0/f9 [963921,37093] 0 2024-04-25T01:36:02.843 INFO:tasks.workunit.client.0.smithi033.stdout:7/885: mknod d4/d2e/d65/c11a 0 2024-04-25T01:36:02.843 INFO:tasks.workunit.client.0.smithi033.stdout:5/913: dread d1c/d2a/d3d/f92 [0,4194304] 0 2024-04-25T01:36:02.845 INFO:tasks.workunit.client.1.smithi138.stdout:5/331: creat d4/da/d27/d1c/d3c/d40/d59/f68 x:0 0 0 2024-04-25T01:36:02.846 INFO:tasks.workunit.client.1.smithi138.stdout:1/344: rmdir d3/d4 39 2024-04-25T01:36:02.847 INFO:tasks.workunit.client.1.smithi138.stdout:4/223: symlink d0/d16/l37 0 2024-04-25T01:36:02.850 INFO:tasks.workunit.client.1.smithi138.stdout:2/417: mknod d0/d21/d45/c82 0 2024-04-25T01:36:02.852 INFO:tasks.workunit.client.1.smithi138.stdout:2/418: dread - d0/d5a/d15/f67 zero size 2024-04-25T01:36:02.853 INFO:tasks.workunit.client.1.smithi138.stdout:5/332: rename d4/l2e to d4/da/d27/d1c/d3c/d40/d59/l69 0 2024-04-25T01:36:02.854 INFO:tasks.workunit.client.1.smithi138.stdout:2/419: chown d0/d21/c29 1017957754 1 2024-04-25T01:36:02.854 INFO:tasks.workunit.client.1.smithi138.stdout:1/345: dwrite d3/db/f67 [0,4194304] 0 2024-04-25T01:36:02.854 INFO:tasks.workunit.client.1.smithi138.stdout:4/224: mknod d0/d1c/c38 0 2024-04-25T01:36:02.856 INFO:tasks.workunit.client.1.smithi138.stdout:2/420: dread - d0/d5a/d15/d51/f80 zero size 2024-04-25T01:36:02.857 INFO:tasks.workunit.client.1.smithi138.stdout:2/421: read d0/d5a/d15/d51/f62 [128375,63704] 0 2024-04-25T01:36:02.860 INFO:tasks.workunit.client.1.smithi138.stdout:1/346: write d3/d4/f9 [4940002,123479] 0 2024-04-25T01:36:02.865 INFO:tasks.workunit.client.1.smithi138.stdout:5/333: mkdir d4/da/d52/d6a 0 2024-04-25T01:36:02.868 INFO:tasks.workunit.client.1.smithi138.stdout:1/347: rename d3/d4/f54 to d3/d27/d2d/f71 0 2024-04-25T01:36:02.869 INFO:tasks.workunit.client.1.smithi138.stdout:2/422: mkdir d0/d5a/d15/d44/d4c/d56/d6d/d6e/d83 0 2024-04-25T01:36:02.870 INFO:tasks.workunit.client.1.smithi138.stdout:7/372: write da/de/f74 [2221929,125559] 0 2024-04-25T01:36:02.872 INFO:tasks.workunit.client.1.smithi138.stdout:4/225: getdents d0/d1c/d23 0 2024-04-25T01:36:02.873 INFO:tasks.workunit.client.1.smithi138.stdout:4/226: fsync d0/d1c/d23/f32 0 2024-04-25T01:36:02.874 INFO:tasks.workunit.client.1.smithi138.stdout:5/334: creat d4/da/d27/d1c/d3c/d40/d59/f6b x:0 0 0 2024-04-25T01:36:02.877 INFO:tasks.workunit.client.1.smithi138.stdout:4/227: dread d0/d16/f18 [0,4194304] 0 2024-04-25T01:36:02.879 INFO:tasks.workunit.client.1.smithi138.stdout:5/335: unlink d4/da/d27/d1c/d5c/f5d 0 2024-04-25T01:36:02.879 INFO:tasks.workunit.client.1.smithi138.stdout:7/373: creat da/de/d13/d30/f80 x:0 0 0 2024-04-25T01:36:02.881 INFO:tasks.workunit.client.1.smithi138.stdout:7/374: truncate da/de/d1b/d4b/d5b/f46 1101822 0 2024-04-25T01:36:02.882 INFO:tasks.workunit.client.1.smithi138.stdout:4/228: rename d0/d1c/d23/l26 to d0/d16/d33/l39 0 2024-04-25T01:36:02.885 INFO:tasks.workunit.client.1.smithi138.stdout:4/229: mknod d0/d1c/c3a 0 2024-04-25T01:36:02.890 INFO:tasks.workunit.client.1.smithi138.stdout:7/375: dwrite da/de/d13/d3e/d51/d6e/f75 [0,4194304] 0 2024-04-25T01:36:02.892 INFO:tasks.workunit.client.1.smithi138.stdout:4/230: rename d0/d1c/d1e/c2e to d0/c3b 0 2024-04-25T01:36:02.893 INFO:tasks.workunit.client.1.smithi138.stdout:7/376: write da/fc [947636,50550] 0 2024-04-25T01:36:02.895 INFO:tasks.workunit.client.1.smithi138.stdout:7/377: dread - da/de/d13/f76 zero size 2024-04-25T01:36:02.900 INFO:tasks.workunit.client.1.smithi138.stdout:7/378: rename da/de/d13/d3e to da/de/d1b/d49/d81 0 2024-04-25T01:36:02.902 INFO:tasks.workunit.client.1.smithi138.stdout:4/231: creat d0/d1c/d23/f3c x:0 0 0 2024-04-25T01:36:02.904 INFO:tasks.workunit.client.1.smithi138.stdout:7/379: unlink da/de/f6c 0 2024-04-25T01:36:02.906 INFO:tasks.workunit.client.1.smithi138.stdout:4/232: mknod d0/c3d 0 2024-04-25T01:36:02.916 INFO:tasks.workunit.client.1.smithi138.stdout:5/336: dwrite d4/da/d27/d1c/f28 [0,4194304] 0 2024-04-25T01:36:02.917 INFO:tasks.workunit.client.1.smithi138.stdout:4/233: write d0/d1c/d23/f35 [1020768,49957] 0 2024-04-25T01:36:02.919 INFO:tasks.workunit.client.1.smithi138.stdout:4/234: chown d0/d1c/d23/f35 3 1 2024-04-25T01:36:02.923 INFO:tasks.workunit.client.1.smithi138.stdout:5/337: creat d4/f6c x:0 0 0 2024-04-25T01:36:02.925 INFO:tasks.workunit.client.1.smithi138.stdout:4/235: rename d0/d1c/d23/l36 to d0/d1c/l3e 0 2024-04-25T01:36:02.926 INFO:tasks.workunit.client.0.smithi033.stdout:6/987: sync 2024-04-25T01:36:02.926 INFO:tasks.workunit.client.0.smithi033.stdout:8/851: sync 2024-04-25T01:36:02.926 INFO:tasks.workunit.client.1.smithi138.stdout:7/380: dwrite da/de/d13/f2d [0,4194304] 0 2024-04-25T01:36:02.926 INFO:tasks.workunit.client.1.smithi138.stdout:5/338: fdatasync d4/da/d27/d1c/d22/f24 0 2024-04-25T01:36:02.930 INFO:tasks.workunit.client.1.smithi138.stdout:7/381: mknod da/de/c82 0 2024-04-25T01:36:02.932 INFO:tasks.workunit.client.1.smithi138.stdout:4/236: creat d0/f3f x:0 0 0 2024-04-25T01:36:02.934 INFO:tasks.workunit.client.1.smithi138.stdout:4/237: rmdir d0/d1c/d1e 39 2024-04-25T01:36:02.937 INFO:tasks.workunit.client.1.smithi138.stdout:5/339: rmdir d4/da/d52/d6a 0 2024-04-25T01:36:02.939 INFO:tasks.workunit.client.1.smithi138.stdout:4/238: dread d0/f13 [0,4194304] 0 2024-04-25T01:36:02.942 INFO:tasks.workunit.client.1.smithi138.stdout:4/239: unlink d0/d1c/c3a 0 2024-04-25T01:36:02.944 INFO:tasks.workunit.client.1.smithi138.stdout:4/240: truncate d0/d16/f29 625053 0 2024-04-25T01:36:02.947 INFO:tasks.workunit.client.1.smithi138.stdout:5/340: truncate d4/da/d27/f33 84920 0 2024-04-25T01:36:02.950 INFO:tasks.workunit.client.1.smithi138.stdout:4/241: rename d0/d1c/d23/f3c to d0/f40 0 2024-04-25T01:36:02.950 INFO:tasks.workunit.client.0.smithi033.stdout:0/833: dwrite d7/d2d/d4d/d9c/d61/f94 [0,4194304] 0 2024-04-25T01:36:02.951 INFO:tasks.workunit.client.0.smithi033.stdout:4/926: write d6/d14/f31 [2319149,121184] 0 2024-04-25T01:36:02.951 INFO:tasks.workunit.client.1.smithi138.stdout:5/341: symlink d4/da/d27/d1c/d22/l6d 0 2024-04-25T01:36:02.953 INFO:tasks.workunit.client.0.smithi033.stdout:7/886: creat d4/d2e/d7f/d98/dca/dd3/f11b x:0 0 0 2024-04-25T01:36:02.954 INFO:tasks.workunit.client.1.smithi138.stdout:7/382: dread da/de/d13/d30/f42 [0,4194304] 0 2024-04-25T01:36:02.956 INFO:tasks.workunit.client.1.smithi138.stdout:7/383: chown l5 34266682 1 2024-04-25T01:36:02.956 INFO:tasks.workunit.client.0.smithi033.stdout:6/988: chown d4/d1d/d3e/d41/d5c/d7f/lc5 0 1 2024-04-25T01:36:02.956 INFO:tasks.workunit.client.0.smithi033.stdout:2/872: dwrite da/f8e [0,4194304] 0 2024-04-25T01:36:02.959 INFO:tasks.workunit.client.1.smithi138.stdout:4/242: dwrite d0/f30 [0,4194304] 0 2024-04-25T01:36:02.960 INFO:tasks.workunit.client.0.smithi033.stdout:0/834: stat d7/d2d/f67 0 2024-04-25T01:36:02.961 INFO:tasks.workunit.client.0.smithi033.stdout:5/914: mkdir d1c/d2a/d50/d52/d104/d125 0 2024-04-25T01:36:02.965 INFO:tasks.workunit.client.0.smithi033.stdout:9/883: dwrite d0/d2/dd/d20/d4e/f76 [0,4194304] 0 2024-04-25T01:36:02.965 INFO:tasks.workunit.client.0.smithi033.stdout:0/835: read d7/da/df/f72 [2089249,66143] 0 2024-04-25T01:36:02.965 INFO:tasks.workunit.client.0.smithi033.stdout:7/887: rename d4/d2e/d6b to d4/d2e/d11c 0 2024-04-25T01:36:02.965 INFO:tasks.workunit.client.0.smithi033.stdout:3/912: read d8/dd/d3d/d63/d7c/f96 [5498065,125949] 0 2024-04-25T01:36:02.966 INFO:tasks.workunit.client.0.smithi033.stdout:8/852: symlink d6/d45/l13b 0 2024-04-25T01:36:02.968 INFO:tasks.workunit.client.0.smithi033.stdout:2/873: symlink da/d5b/d5e/d76/dfc/l11e 0 2024-04-25T01:36:02.969 INFO:tasks.workunit.client.1.smithi138.stdout:5/342: rename d4/da/d27/d1c to d4/da/d6e 0 2024-04-25T01:36:02.969 INFO:tasks.workunit.client.0.smithi033.stdout:9/884: chown d0/d2/d43/d4b/fe9 1489938 1 2024-04-25T01:36:02.970 INFO:tasks.workunit.client.0.smithi033.stdout:4/927: link d6/d12/d84/fbb d6/d14/d35/d44/d45/d46/d57/d70/d87/db8/f12f 0 2024-04-25T01:36:02.971 INFO:tasks.workunit.client.0.smithi033.stdout:6/989: creat d4/d24/f14c x:0 0 0 2024-04-25T01:36:02.974 INFO:tasks.workunit.client.0.smithi033.stdout:7/888: mknod d4/d2e/d65/df5/c11d 0 2024-04-25T01:36:02.974 INFO:tasks.workunit.client.1.smithi138.stdout:5/343: symlink d4/da/d6e/d22/d35/d4d/d5e/l6f 0 2024-04-25T01:36:02.974 INFO:tasks.workunit.client.1.smithi138.stdout:7/384: creat da/f83 x:0 0 0 2024-04-25T01:36:02.975 INFO:tasks.workunit.client.0.smithi033.stdout:6/990: chown d4/d1d/d3e/d4a/l82 1532 1 2024-04-25T01:36:02.975 INFO:tasks.workunit.client.1.smithi138.stdout:5/344: truncate d4/f6c 878841 0 2024-04-25T01:36:02.976 INFO:tasks.workunit.client.0.smithi033.stdout:5/915: rename d1c/d2a/d50/db1/lf2 to d1c/d2a/d50/d5b/d86/d10b/l126 0 2024-04-25T01:36:02.976 INFO:tasks.workunit.client.0.smithi033.stdout:0/836: dread d7/da/df/d50/f6b [0,4194304] 0 2024-04-25T01:36:02.978 INFO:tasks.workunit.client.0.smithi033.stdout:5/916: chown d1c/d2a/c43 13515 1 2024-04-25T01:36:02.979 INFO:tasks.workunit.client.1.smithi138.stdout:5/345: stat d4/da/d6e/d22/l31 0 2024-04-25T01:36:02.980 INFO:tasks.workunit.client.0.smithi033.stdout:8/853: mkdir d6/d45/d108/d13c 0 2024-04-25T01:36:02.981 INFO:tasks.workunit.client.1.smithi138.stdout:5/346: chown d4/da/d6e/d22/d35/d4d/l63 46829 1 2024-04-25T01:36:02.982 INFO:tasks.workunit.client.1.smithi138.stdout:5/347: write d4/f1a [1297901,19871] 0 2024-04-25T01:36:02.985 INFO:tasks.workunit.client.0.smithi033.stdout:7/889: rmdir d4/d2e/d7f 39 2024-04-25T01:36:02.986 INFO:tasks.workunit.client.0.smithi033.stdout:0/837: truncate d7/d2d/d43/d92/fea 561783 0 2024-04-25T01:36:02.986 INFO:tasks.workunit.client.0.smithi033.stdout:9/885: dwrite d0/d2/d43/d73/dc4/fd5 [0,4194304] 0 2024-04-25T01:36:02.986 INFO:tasks.workunit.client.1.smithi138.stdout:5/348: rename d4/da/d6e/d3c/d40/f55 to d4/da/d6e/d3c/d40/d59/f70 0 2024-04-25T01:36:02.988 INFO:tasks.workunit.client.0.smithi033.stdout:8/854: symlink d6/d45/d52/dae/l13d 0 2024-04-25T01:36:02.988 INFO:tasks.workunit.client.0.smithi033.stdout:5/917: truncate d1c/d2a/d3d/d83/fde 878219 0 2024-04-25T01:36:02.989 INFO:tasks.workunit.client.0.smithi033.stdout:4/928: creat d6/d12/f130 x:0 0 0 2024-04-25T01:36:02.989 INFO:tasks.workunit.client.0.smithi033.stdout:9/886: fsync d0/d11/f18 0 2024-04-25T01:36:02.991 INFO:tasks.workunit.client.0.smithi033.stdout:7/890: read d4/d23/d3f/d45/d73/dbe/fe0 [206128,38712] 0 2024-04-25T01:36:02.991 INFO:tasks.workunit.client.0.smithi033.stdout:0/838: mknod d7/d2d/d4d/d9c/d68/c11d 0 2024-04-25T01:36:02.992 INFO:tasks.workunit.client.1.smithi138.stdout:5/349: dread d4/da/d27/f1f [0,4194304] 0 2024-04-25T01:36:02.994 INFO:tasks.workunit.client.0.smithi033.stdout:7/891: read d4/f7a [891056,120097] 0 2024-04-25T01:36:02.994 INFO:tasks.workunit.client.0.smithi033.stdout:8/855: read d6/dd3/dfb/f105 [8764,38055] 0 2024-04-25T01:36:02.997 INFO:tasks.workunit.client.0.smithi033.stdout:4/929: truncate d6/d14/d48/f55 2017792 0 2024-04-25T01:36:02.997 INFO:tasks.workunit.client.1.smithi138.stdout:5/350: symlink d4/da/d6e/d22/d35/d4d/d5e/l71 0 2024-04-25T01:36:02.998 INFO:tasks.workunit.client.0.smithi033.stdout:5/918: truncate d1c/d2a/d50/d52/da6/d4a/f64 1058806 0 2024-04-25T01:36:03.004 INFO:tasks.workunit.client.0.smithi033.stdout:4/930: creat d6/d14/d35/d44/d45/f131 x:0 0 0 2024-04-25T01:36:03.005 INFO:tasks.workunit.client.0.smithi033.stdout:5/919: write d1c/d2a/d3d/d59/f84 [235259,11027] 0 2024-04-25T01:36:03.006 INFO:tasks.workunit.client.0.smithi033.stdout:0/839: dwrite d7/d2d/d4d/d9c/f59 [0,4194304] 0 2024-04-25T01:36:03.013 INFO:tasks.workunit.client.1.smithi138.stdout:4/243: truncate d0/d16/f34 3835604 0 2024-04-25T01:36:03.013 INFO:tasks.workunit.client.0.smithi033.stdout:0/840: creat d7/da/df/d14/f11e x:0 0 0 2024-04-25T01:36:03.013 INFO:tasks.workunit.client.0.smithi033.stdout:5/920: truncate d1c/d2a/d50/d52/da6/d3e/fc8 2933901 0 2024-04-25T01:36:03.016 INFO:tasks.workunit.client.1.smithi138.stdout:7/385: truncate da/de/f74 1083399 0 2024-04-25T01:36:03.016 INFO:tasks.workunit.client.0.smithi033.stdout:3/913: write d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/f8a [540066,29297] 0 2024-04-25T01:36:03.018 INFO:tasks.workunit.client.0.smithi033.stdout:5/921: read d1c/d2a/d50/d52/da6/d4a/fb4 [3106403,123100] 0 2024-04-25T01:36:03.019 INFO:tasks.workunit.client.0.smithi033.stdout:2/874: write da/d27/d31/d40/d8a/dbb/f68 [278510,76638] 0 2024-04-25T01:36:03.019 INFO:tasks.workunit.client.1.smithi138.stdout:7/386: symlink da/de/d13/d21/l84 0 2024-04-25T01:36:03.022 INFO:tasks.workunit.client.0.smithi033.stdout:0/841: creat d7/d2d/d4d/d9c/d61/f11f x:0 0 0 2024-04-25T01:36:03.022 INFO:tasks.workunit.client.1.smithi138.stdout:7/387: chown da/de/d13/f52 1606440 1 2024-04-25T01:36:03.023 INFO:tasks.workunit.client.1.smithi138.stdout:4/244: dwrite d0/f2f [0,4194304] 0 2024-04-25T01:36:03.024 INFO:tasks.workunit.client.0.smithi033.stdout:3/914: chown d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3 29 1 2024-04-25T01:36:03.027 INFO:tasks.workunit.client.0.smithi033.stdout:2/875: symlink da/d27/d31/d40/db1/dbf/dd5/de0/l11f 0 2024-04-25T01:36:03.027 INFO:tasks.workunit.client.0.smithi033.stdout:1/737: sync 2024-04-25T01:36:03.028 INFO:tasks.workunit.client.1.smithi138.stdout:4/245: mknod d0/d16/d33/c41 0 2024-04-25T01:36:03.029 INFO:tasks.workunit.client.0.smithi033.stdout:6/991: dwrite d4/f12 [8388608,4194304] 0 2024-04-25T01:36:03.031 INFO:tasks.workunit.client.0.smithi033.stdout:1/738: fdatasync d2/d5/d1b/d46/d5f/d63/d8f/fbc 0 2024-04-25T01:36:03.033 INFO:tasks.workunit.client.0.smithi033.stdout:6/992: chown d4/l84 154964 1 2024-04-25T01:36:03.036 INFO:tasks.workunit.client.1.smithi138.stdout:5/351: dread d4/da/d6e/f26 [0,4194304] 0 2024-04-25T01:36:03.036 INFO:tasks.workunit.client.0.smithi033.stdout:9/887: dread d0/d2/d43/d4b/f92 [0,4194304] 0 2024-04-25T01:36:03.037 INFO:tasks.workunit.client.1.smithi138.stdout:7/388: dwrite da/de/d1b/d34/f47 [0,4194304] 0 2024-04-25T01:36:03.039 INFO:tasks.workunit.client.1.smithi138.stdout:7/389: write da/f83 [4773,122399] 0 2024-04-25T01:36:03.041 INFO:tasks.workunit.client.1.smithi138.stdout:7/390: write da/f83 [325179,107143] 0 2024-04-25T01:36:03.042 INFO:tasks.workunit.client.0.smithi033.stdout:2/876: dread da/d27/d31/d40/d8a/dbb/f68 [0,4194304] 0 2024-04-25T01:36:03.044 INFO:tasks.workunit.client.0.smithi033.stdout:7/892: write d4/d23/d3f/d45/d73/dbe/fc1 [1818969,3270] 0 2024-04-25T01:36:03.047 INFO:tasks.workunit.client.1.smithi138.stdout:5/352: fdatasync d4/da/fe 0 2024-04-25T01:36:03.047 INFO:tasks.workunit.client.0.smithi033.stdout:6/993: creat d4/d24/d63/f14d x:0 0 0 2024-04-25T01:36:03.047 INFO:tasks.workunit.client.0.smithi033.stdout:1/739: fsync d2/d5/d1b/d46/d5f/d63/f65 0 2024-04-25T01:36:03.051 INFO:tasks.workunit.client.1.smithi138.stdout:7/391: mknod da/de/d1b/d49/d81/d73/c85 0 2024-04-25T01:36:03.053 INFO:tasks.workunit.client.1.smithi138.stdout:5/353: mknod d4/da/d6e/d3c/d40/c72 0 2024-04-25T01:36:03.053 INFO:tasks.workunit.client.0.smithi033.stdout:9/888: truncate d0/d2/dd/df/f5c 89568 0 2024-04-25T01:36:03.055 INFO:tasks.workunit.client.0.smithi033.stdout:5/922: creat d1c/d2a/d50/d52/da6/d3e/f127 x:0 0 0 2024-04-25T01:36:03.056 INFO:tasks.workunit.client.0.smithi033.stdout:1/740: fdatasync d2/d5/d6c/d94/fb5 0 2024-04-25T01:36:03.057 INFO:tasks.workunit.client.1.smithi138.stdout:7/392: rmdir da/de/d13/d30 39 2024-04-25T01:36:03.057 INFO:tasks.workunit.client.0.smithi033.stdout:8/856: dwrite d6/d45/d52/d2e/f98 [0,4194304] 0 2024-04-25T01:36:03.057 INFO:tasks.workunit.client.0.smithi033.stdout:3/915: rmdir d8/dd/d11 39 2024-04-25T01:36:03.058 INFO:tasks.workunit.client.1.smithi138.stdout:4/246: dread d0/d16/f34 [0,4194304] 0 2024-04-25T01:36:03.059 INFO:tasks.workunit.client.1.smithi138.stdout:5/354: creat d4/da/d6e/d3c/d40/f73 x:0 0 0 2024-04-25T01:36:03.060 INFO:tasks.workunit.client.1.smithi138.stdout:9/269: sync 2024-04-25T01:36:03.061 INFO:tasks.workunit.client.1.smithi138.stdout:5/355: write d4/da/d6e/d22/d45/f54 [348018,57857] 0 2024-04-25T01:36:03.061 INFO:tasks.workunit.client.0.smithi033.stdout:4/931: dwrite d6/d14/d1d/d5f/fed [0,4194304] 0 2024-04-25T01:36:03.062 INFO:tasks.workunit.client.1.smithi138.stdout:5/356: chown d4/da/d27/c3e 2029 1 2024-04-25T01:36:03.064 INFO:tasks.workunit.client.0.smithi033.stdout:1/741: chown d2/d5/d1b/d46/d5f/d63/d8f 1 1 2024-04-25T01:36:03.066 INFO:tasks.workunit.client.1.smithi138.stdout:9/270: creat d4/d57/f5a x:0 0 0 2024-04-25T01:36:03.066 INFO:tasks.workunit.client.0.smithi033.stdout:0/842: creat d7/d2d/d4d/d9c/d61/f120 x:0 0 0 2024-04-25T01:36:03.070 INFO:tasks.workunit.client.1.smithi138.stdout:5/357: dread d4/da/d27/f1f [0,4194304] 0 2024-04-25T01:36:03.072 INFO:tasks.workunit.client.1.smithi138.stdout:4/247: getdents d0/d1c/d1e 0 2024-04-25T01:36:03.075 INFO:tasks.workunit.client.1.smithi138.stdout:7/393: write da/de/f29 [614577,104652] 0 2024-04-25T01:36:03.078 INFO:tasks.workunit.client.0.smithi033.stdout:5/923: rename d1c/d2a/f11c to d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e/f128 0 2024-04-25T01:36:03.080 INFO:tasks.workunit.client.1.smithi138.stdout:9/271: dwrite d4/f26 [0,4194304] 0 2024-04-25T01:36:03.080 INFO:tasks.workunit.client.1.smithi138.stdout:7/394: read da/de/d13/d30/f42 [560341,68460] 0 2024-04-25T01:36:03.081 INFO:tasks.workunit.client.0.smithi033.stdout:1/742: symlink d2/d5/d1b/d46/db1/lec 0 2024-04-25T01:36:03.082 INFO:tasks.workunit.client.1.smithi138.stdout:4/248: rename d0/d1c/c38 to d0/d1c/c42 0 2024-04-25T01:36:03.083 INFO:tasks.workunit.client.0.smithi033.stdout:0/843: fsync d7/da/df/d14/f47 0 2024-04-25T01:36:03.083 INFO:tasks.workunit.client.1.smithi138.stdout:5/358: getdents d4/da/d6e/d5c 0 2024-04-25T01:36:03.085 INFO:tasks.workunit.client.0.smithi033.stdout:7/893: mknod d4/d23/d3f/d45/d118/c11e 0 2024-04-25T01:36:03.086 INFO:tasks.workunit.client.1.smithi138.stdout:4/249: creat d0/d16/f43 x:0 0 0 2024-04-25T01:36:03.087 INFO:tasks.workunit.client.1.smithi138.stdout:7/395: symlink da/de/d1b/d4b/d77/l86 0 2024-04-25T01:36:03.087 INFO:tasks.workunit.client.0.smithi033.stdout:0/844: write d7/da/df/d14/f11e [917973,26208] 0 2024-04-25T01:36:03.089 INFO:tasks.workunit.client.0.smithi033.stdout:9/889: mknod d0/d2/d43/d4b/c135 0 2024-04-25T01:36:03.090 INFO:tasks.workunit.client.0.smithi033.stdout:0/845: readlink d7/de7/l11b 0 2024-04-25T01:36:03.091 INFO:tasks.workunit.client.1.smithi138.stdout:5/359: rename d4/da/d6e/d3c/d40 to d4/da/d6e/d22/d74 0 2024-04-25T01:36:03.094 INFO:tasks.workunit.client.1.smithi138.stdout:9/272: dwrite d4/d1a/f2d [0,4194304] 0 2024-04-25T01:36:03.095 INFO:tasks.workunit.client.1.smithi138.stdout:5/360: chown d4/da/d6e/d22/d35 25001804 1 2024-04-25T01:36:03.095 INFO:tasks.workunit.client.1.smithi138.stdout:7/396: dread da/f83 [0,4194304] 0 2024-04-25T01:36:03.096 INFO:tasks.workunit.client.0.smithi033.stdout:4/932: dwrite d6/d14/d35/d3a/d4d/fd9 [0,4194304] 0 2024-04-25T01:36:03.096 INFO:tasks.workunit.client.0.smithi033.stdout:3/916: dread - d8/dd/d3d/d63/d7c/ded/dc7/ddf/d10b/f141 zero size 2024-04-25T01:36:03.099 INFO:tasks.workunit.client.0.smithi033.stdout:8/857: creat d6/d45/f13e x:0 0 0 2024-04-25T01:36:03.100 INFO:tasks.workunit.client.0.smithi033.stdout:8/858: dread - d6/db/d111/f11e zero size 2024-04-25T01:36:03.101 INFO:tasks.workunit.client.1.smithi138.stdout:4/250: dwrite d0/d1c/f2b [0,4194304] 0 2024-04-25T01:36:03.101 INFO:tasks.workunit.client.0.smithi033.stdout:7/894: rename d4/d2e/f63 to d4/d2e/d65/dcb/f11f 0 2024-04-25T01:36:03.104 INFO:tasks.workunit.client.1.smithi138.stdout:9/273: creat d4/d8/f5b x:0 0 0 2024-04-25T01:36:03.105 INFO:tasks.workunit.client.1.smithi138.stdout:4/251: creat d0/d16/f44 x:0 0 0 2024-04-25T01:36:03.107 INFO:tasks.workunit.client.1.smithi138.stdout:9/274: write d4/d1a/d33/f4d [739312,17086] 0 2024-04-25T01:36:03.108 INFO:tasks.workunit.client.1.smithi138.stdout:4/252: stat d0/c14 0 2024-04-25T01:36:03.109 INFO:tasks.workunit.client.0.smithi033.stdout:2/877: getdents da/d50/d6a/d43/d6c 0 2024-04-25T01:36:03.109 INFO:tasks.workunit.client.1.smithi138.stdout:9/275: read - d4/d8/f5b zero size 2024-04-25T01:36:03.111 INFO:tasks.workunit.client.0.smithi033.stdout:2/878: readlink da/d27/d31/d40/l7e 0 2024-04-25T01:36:03.113 INFO:tasks.workunit.client.1.smithi138.stdout:5/361: dread d4/da/d6e/d22/d74/f5a [0,4194304] 0 2024-04-25T01:36:03.115 INFO:tasks.workunit.client.0.smithi033.stdout:2/879: fsync da/f12 0 2024-04-25T01:36:03.115 INFO:tasks.workunit.client.0.smithi033.stdout:5/924: fsync f1b 0 2024-04-25T01:36:03.119 INFO:tasks.workunit.client.1.smithi138.stdout:4/253: rename d0/d1c/d23 to d0/d45 0 2024-04-25T01:36:03.120 INFO:tasks.workunit.client.1.smithi138.stdout:9/276: dwrite d4/f4a [0,4194304] 0 2024-04-25T01:36:03.121 INFO:tasks.workunit.client.1.smithi138.stdout:4/254: write d0/d16/f31 [688782,20689] 0 2024-04-25T01:36:03.121 INFO:tasks.workunit.client.0.smithi033.stdout:0/846: dwrite d7/da/df/d50/f6b [4194304,4194304] 0 2024-04-25T01:36:03.122 INFO:tasks.workunit.client.1.smithi138.stdout:7/397: write da/de/f74 [1252486,85084] 0 2024-04-25T01:36:03.122 INFO:tasks.workunit.client.0.smithi033.stdout:3/917: rename l1 to d8/dd/d3d/d63/d7c/d9d/de8/df9/l146 0 2024-04-25T01:36:03.124 INFO:tasks.workunit.client.0.smithi033.stdout:9/890: truncate d0/d2/d43/f8e 781652 0 2024-04-25T01:36:03.125 INFO:tasks.workunit.client.0.smithi033.stdout:1/743: link d2/d2e/l3f d2/d5/d6c/led 0 2024-04-25T01:36:03.125 INFO:tasks.workunit.client.0.smithi033.stdout:7/895: write d4/d2e/d7f/d98/fd7 [4371636,40472] 0 2024-04-25T01:36:03.126 INFO:tasks.workunit.client.0.smithi033.stdout:6/994: dwrite d4/d24/d36/d42/f83 [0,4194304] 0 2024-04-25T01:36:03.127 INFO:tasks.workunit.client.0.smithi033.stdout:8/859: symlink d6/l13f 0 2024-04-25T01:36:03.128 INFO:tasks.workunit.client.1.smithi138.stdout:8/514: sync 2024-04-25T01:36:03.130 INFO:tasks.workunit.client.0.smithi033.stdout:8/860: truncate d6/d45/d52/f43 5184326 0 2024-04-25T01:36:03.130 INFO:tasks.workunit.client.0.smithi033.stdout:9/891: chown d0/d2/dd/d20/c34 3 1 2024-04-25T01:36:03.133 INFO:tasks.workunit.client.0.smithi033.stdout:5/925: creat d1c/d2a/d50/d5b/d86/f129 x:0 0 0 2024-04-25T01:36:03.135 INFO:tasks.workunit.client.0.smithi033.stdout:0/847: symlink d7/d2d/d64/d51/l121 0 2024-04-25T01:36:03.140 INFO:tasks.workunit.client.0.smithi033.stdout:7/896: symlink d4/d2e/d11c/d101/l120 0 2024-04-25T01:36:03.140 INFO:tasks.workunit.client.0.smithi033.stdout:0/848: chown d7/d2d/d43/dc0/lfb 37816367 1 2024-04-25T01:36:03.142 INFO:tasks.workunit.client.0.smithi033.stdout:9/892: mkdir d0/d2/dd/dba/dc1/d136 0 2024-04-25T01:36:03.142 INFO:tasks.workunit.client.1.smithi138.stdout:9/277: dwrite d4/d1a/d33/f4d [0,4194304] 0 2024-04-25T01:36:03.144 INFO:tasks.workunit.client.1.smithi138.stdout:8/515: dwrite d3/d16/d1f/d50/f7c [0,4194304] 0 2024-04-25T01:36:03.145 INFO:tasks.workunit.client.0.smithi033.stdout:8/861: unlink d6/db/d80/dd1/fe3 0 2024-04-25T01:36:03.145 INFO:tasks.workunit.client.0.smithi033.stdout:3/918: dwrite d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/f8a [0,4194304] 0 2024-04-25T01:36:03.148 INFO:tasks.workunit.client.1.smithi138.stdout:9/278: stat d4/d8/d2c/l4c 0 2024-04-25T01:36:03.150 INFO:tasks.workunit.client.0.smithi033.stdout:2/880: rename da/d27/d31/c65 to da/d5b/d72/c120 0 2024-04-25T01:36:03.152 INFO:tasks.workunit.client.0.smithi033.stdout:6/995: dread d4/d1d/d3e/d41/d5c/f6d [0,4194304] 0 2024-04-25T01:36:03.152 INFO:tasks.workunit.client.1.smithi138.stdout:8/516: symlink d3/d16/d88/da/db/d15/d59/la1 0 2024-04-25T01:36:03.153 INFO:tasks.workunit.client.0.smithi033.stdout:9/893: fdatasync d0/d2/dd/dba/dc1/fcd 0 2024-04-25T01:36:03.155 INFO:tasks.workunit.client.0.smithi033.stdout:8/862: unlink d6/d45/d52/d2e/la8 0 2024-04-25T01:36:03.155 INFO:tasks.workunit.client.1.smithi138.stdout:8/517: rmdir d3/d16/d88/da/de 39 2024-04-25T01:36:03.157 INFO:tasks.workunit.client.0.smithi033.stdout:5/926: unlink d1c/d2a/d3d/d83/fde 0 2024-04-25T01:36:03.164 INFO:tasks.workunit.client.0.smithi033.stdout:0/849: rename d7/d2d/d4d/fd6 to d7/d2d/d43/f122 0 2024-04-25T01:36:03.165 INFO:tasks.workunit.client.1.smithi138.stdout:8/518: write d3/d16/d88/da/de/f36 [358539,55812] 0 2024-04-25T01:36:03.167 INFO:tasks.workunit.client.1.smithi138.stdout:7/398: dwrite da/de/d1b/d4b/f55 [0,4194304] 0 2024-04-25T01:36:03.168 INFO:tasks.workunit.client.0.smithi033.stdout:8/863: creat d6/d45/d52/df2/f140 x:0 0 0 2024-04-25T01:36:03.169 INFO:tasks.workunit.client.0.smithi033.stdout:5/927: mknod d1c/d2a/d50/d52/d104/c12a 0 2024-04-25T01:36:03.170 INFO:tasks.workunit.client.1.smithi138.stdout:7/399: write da/f83 [1330878,56948] 0 2024-04-25T01:36:03.170 INFO:tasks.workunit.client.1.smithi138.stdout:8/519: fdatasync d3/d16/d88/da/db/d15/d56/f9c 0 2024-04-25T01:36:03.171 INFO:tasks.workunit.client.0.smithi033.stdout:5/928: write d1c/f37 [9024037,67960] 0 2024-04-25T01:36:03.176 INFO:tasks.workunit.client.1.smithi138.stdout:7/400: getdents da/de/d1b/d49/d81/d73 0 2024-04-25T01:36:03.177 INFO:tasks.workunit.client.0.smithi033.stdout:7/897: getdents d4/d2e/d11c/d67 0 2024-04-25T01:36:03.178 INFO:tasks.workunit.client.1.smithi138.stdout:7/401: creat da/de/d1b/d49/d81/d51/d6e/f87 x:0 0 0 2024-04-25T01:36:03.178 INFO:tasks.workunit.client.0.smithi033.stdout:4/933: dwrite d6/d14/d35/d44/d45/d46/d57/d70/d87/dc7/fbf [0,4194304] 0 2024-04-25T01:36:03.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/402: write da/f18 [4548055,121074] 0 2024-04-25T01:36:03.183 INFO:tasks.workunit.client.0.smithi033.stdout:6/996: link d4/d1d/d3e/d41/d93/f104 d4/d1d/d3e/d41/d5c/dae/de1/f14e 0 2024-04-25T01:36:03.184 INFO:tasks.workunit.client.1.smithi138.stdout:4/255: dread d0/f1a [0,4194304] 0 2024-04-25T01:36:03.185 INFO:tasks.workunit.client.1.smithi138.stdout:7/403: getdents da/de/d1b 0 2024-04-25T01:36:03.185 INFO:tasks.workunit.client.1.smithi138.stdout:4/256: fsync d0/d1c/f2c 0 2024-04-25T01:36:03.187 INFO:tasks.workunit.client.0.smithi033.stdout:0/850: unlink d7/d2d/d43/d44/l78 0 2024-04-25T01:36:03.189 INFO:tasks.workunit.client.1.smithi138.stdout:7/404: unlink da/de/d1b/d34/l57 0 2024-04-25T01:36:03.189 INFO:tasks.workunit.client.0.smithi033.stdout:1/744: write d2/d2e/d34/d60/fa7 [131141,55813] 0 2024-04-25T01:36:03.190 INFO:tasks.workunit.client.1.smithi138.stdout:4/257: mknod d0/d16/d33/c46 0 2024-04-25T01:36:03.191 INFO:tasks.workunit.client.0.smithi033.stdout:3/919: getdents d8/dd/d3d 0 2024-04-25T01:36:03.191 INFO:tasks.workunit.client.1.smithi138.stdout:4/258: write d0/d1c/f2c [429192,107966] 0 2024-04-25T01:36:03.194 INFO:tasks.workunit.client.0.smithi033.stdout:6/997: unlink d4/d2a/f3f 0 2024-04-25T01:36:03.196 INFO:tasks.workunit.client.0.smithi033.stdout:5/929: mknod d1c/d2a/d50/d52/da6/d3e/c12b 0 2024-04-25T01:36:03.196 INFO:tasks.workunit.client.1.smithi138.stdout:4/259: rename d0/l5 to d0/l47 0 2024-04-25T01:36:03.197 INFO:tasks.workunit.client.1.smithi138.stdout:4/260: chown d0/d45 2106123193 1 2024-04-25T01:36:03.198 INFO:tasks.workunit.client.1.smithi138.stdout:9/279: dread d4/d1a/f20 [0,4194304] 0 2024-04-25T01:36:03.199 INFO:tasks.workunit.client.1.smithi138.stdout:7/405: truncate da/f18 928909 0 2024-04-25T01:36:03.199 INFO:tasks.workunit.client.1.smithi138.stdout:9/280: stat d4/l2b 0 2024-04-25T01:36:03.199 INFO:tasks.workunit.client.0.smithi033.stdout:4/934: mknod d6/d14/d35/c132 0 2024-04-25T01:36:03.203 INFO:tasks.workunit.client.1.smithi138.stdout:9/281: creat d4/d1a/d55/f5c x:0 0 0 2024-04-25T01:36:03.203 INFO:tasks.workunit.client.0.smithi033.stdout:0/851: link d7/d2d/d9b/fef d7/d2d/d4c/da0/da1/f123 0 2024-04-25T01:36:03.205 INFO:tasks.workunit.client.0.smithi033.stdout:6/998: write d4/d24/d36/d42/d97/db2/f11a [2644391,88217] 0 2024-04-25T01:36:03.205 INFO:tasks.workunit.client.1.smithi138.stdout:4/261: getdents d0/d16 0 2024-04-25T01:36:03.206 INFO:tasks.workunit.client.0.smithi033.stdout:5/930: mkdir d1c/d2a/d50/d52/d10c/d12c 0 2024-04-25T01:36:03.208 INFO:tasks.workunit.client.0.smithi033.stdout:5/931: chown d1c/d2a/d3d/d59/l74 483713301 1 2024-04-25T01:36:03.208 INFO:tasks.workunit.client.0.smithi033.stdout:7/898: link d4/f9 d4/d23/f121 0 2024-04-25T01:36:03.208 INFO:tasks.workunit.client.1.smithi138.stdout:9/282: creat d4/d1a/d33/d1d/d3f/f5d x:0 0 0 2024-04-25T01:36:03.211 INFO:tasks.workunit.client.1.smithi138.stdout:7/406: dwrite da/de/d13/d21/f41 [0,4194304] 0 2024-04-25T01:36:03.211 INFO:tasks.workunit.client.0.smithi033.stdout:4/935: fdatasync d6/d14/d35/d44/d45/f75 0 2024-04-25T01:36:03.213 INFO:tasks.workunit.client.0.smithi033.stdout:3/920: rename d8/dd/c11c to d8/dd/d3d/d63/d7c/ded/dbf/c147 0 2024-04-25T01:36:03.214 INFO:tasks.workunit.client.0.smithi033.stdout:0/852: mknod d7/da/df/d50/dfe/c124 0 2024-04-25T01:36:03.215 INFO:tasks.workunit.client.1.smithi138.stdout:7/407: fsync da/de/f27 0 2024-04-25T01:36:03.215 INFO:tasks.workunit.client.0.smithi033.stdout:9/894: write d0/d2/d43/d4b/f63 [2056304,121666] 0 2024-04-25T01:36:03.218 INFO:tasks.workunit.client.1.smithi138.stdout:4/262: dwrite d0/f2f [0,4194304] 0 2024-04-25T01:36:03.222 INFO:tasks.workunit.client.1.smithi138.stdout:9/283: dwrite d4/d1a/f2d [0,4194304] 0 2024-04-25T01:36:03.222 INFO:tasks.workunit.client.0.smithi033.stdout:3/921: write d8/dd/d11/f7f [290772,81831] 0 2024-04-25T01:36:03.223 INFO:tasks.workunit.client.1.smithi138.stdout:9/284: fsync d4/d8/f45 0 2024-04-25T01:36:03.224 INFO:tasks.workunit.client.1.smithi138.stdout:7/408: rename da/de/d1b/l54 to da/de/d13/d30/l88 0 2024-04-25T01:36:03.224 INFO:tasks.workunit.client.0.smithi033.stdout:0/853: unlink d7/d2d/d4d/d9c/d68/caf 0 2024-04-25T01:36:03.225 INFO:tasks.workunit.client.1.smithi138.stdout:4/263: mkdir d0/d1c/d48 0 2024-04-25T01:36:03.226 INFO:tasks.workunit.client.0.smithi033.stdout:8/864: dwrite d6/d45/d52/d2e/f3a [0,4194304] 0 2024-04-25T01:36:03.226 INFO:tasks.workunit.client.0.smithi033.stdout:0/854: chown d7/da/df/d14/f11e 12379 1 2024-04-25T01:36:03.227 INFO:tasks.workunit.client.1.smithi138.stdout:9/285: symlink d4/d1a/d33/l5e 0 2024-04-25T01:36:03.229 INFO:tasks.workunit.client.1.smithi138.stdout:9/286: read d4/d8/f53 [97564,67295] 0 2024-04-25T01:36:03.230 INFO:tasks.workunit.client.0.smithi033.stdout:7/899: link d4/d2e/d11c/f52 d4/d2e/d65/dff/f122 0 2024-04-25T01:36:03.232 INFO:tasks.workunit.client.1.smithi138.stdout:7/409: link da/de/d1b/d49/d81/f4c da/de/d1b/d49/f89 0 2024-04-25T01:36:03.234 INFO:tasks.workunit.client.1.smithi138.stdout:4/264: creat d0/d1c/f49 x:0 0 0 2024-04-25T01:36:03.238 INFO:tasks.workunit.client.0.smithi033.stdout:0/855: creat d7/da/df/d14/d9d/f125 x:0 0 0 2024-04-25T01:36:03.242 INFO:tasks.workunit.client.0.smithi033.stdout:7/900: creat d4/d2e/d65/f123 x:0 0 0 2024-04-25T01:36:03.243 INFO:tasks.workunit.client.1.smithi138.stdout:9/287: rmdir d4/d57 39 2024-04-25T01:36:03.243 INFO:tasks.workunit.client.1.smithi138.stdout:4/265: dwrite d0/d1c/f49 [0,4194304] 0 2024-04-25T01:36:03.246 INFO:tasks.workunit.client.1.smithi138.stdout:7/410: mkdir da/de/d1b/d49/d8a 0 2024-04-25T01:36:03.246 INFO:tasks.workunit.client.0.smithi033.stdout:9/895: creat d0/d2/dd/f137 x:0 0 0 2024-04-25T01:36:03.246 INFO:tasks.workunit.client.0.smithi033.stdout:1/745: dwrite d2/d2e/d34/d60/f74 [0,4194304] 0 2024-04-25T01:36:03.246 INFO:tasks.workunit.client.1.smithi138.stdout:9/288: creat d4/d1a/d55/f5f x:0 0 0 2024-04-25T01:36:03.247 INFO:tasks.workunit.client.1.smithi138.stdout:9/289: readlink d4/d1a/d3c/l54 0 2024-04-25T01:36:03.248 INFO:tasks.workunit.client.0.smithi033.stdout:8/865: link d6/d45/d52/d2e/f3a d6/d45/d108/d13c/f141 0 2024-04-25T01:36:03.250 INFO:tasks.workunit.client.1.smithi138.stdout:4/266: mkdir d0/d16/d4a 0 2024-04-25T01:36:03.253 INFO:tasks.workunit.client.0.smithi033.stdout:7/901: mkdir d4/d2e/d65/d124 0 2024-04-25T01:36:03.253 INFO:tasks.workunit.client.0.smithi033.stdout:6/999: write d4/d24/d36/d42/d9d/fb0 [90930,94940] 0 2024-04-25T01:36:03.256 INFO:tasks.workunit.client.0.smithi033.stdout:0/856: mknod d7/d2d/d43/d44/c126 0 2024-04-25T01:36:03.257 INFO:tasks.workunit.client.1.smithi138.stdout:9/290: rename d4/d8/c29 to d4/d1a/d33/d1d/c60 0 2024-04-25T01:36:03.258 INFO:tasks.workunit.client.1.smithi138.stdout:4/267: dread d0/f2f [0,4194304] 0 2024-04-25T01:36:03.260 INFO:tasks.workunit.client.1.smithi138.stdout:9/291: chown d4/d57 98 1 2024-04-25T01:36:03.260 INFO:tasks.workunit.client.1.smithi138.stdout:4/268: fsync d0/f2f 0 2024-04-25T01:36:03.260 INFO:tasks.workunit.client.1.smithi138.stdout:4/269: stat d0/d16/f44 0 2024-04-25T01:36:03.262 INFO:tasks.workunit.client.0.smithi033.stdout:7/902: creat d4/d2e/d65/d124/f125 x:0 0 0 2024-04-25T01:36:03.263 INFO:tasks.workunit.client.1.smithi138.stdout:9/292: creat d4/d8/f61 x:0 0 0 2024-04-25T01:36:03.265 INFO:tasks.workunit.client.0.smithi033.stdout:9/896: link d0/f9 d0/d2/dd/df/dd8/f138 0 2024-04-25T01:36:03.267 INFO:tasks.workunit.client.1.smithi138.stdout:4/270: dwrite d0/d16/f29 [0,4194304] 0 2024-04-25T01:36:03.267 INFO:tasks.workunit.client.1.smithi138.stdout:9/293: creat d4/d1a/d33/d1d/d3f/f62 x:0 0 0 2024-04-25T01:36:03.267 INFO:tasks.workunit.client.0.smithi033.stdout:7/903: creat d4/d2e/d65/d124/f126 x:0 0 0 2024-04-25T01:36:03.268 INFO:tasks.workunit.client.0.smithi033.stdout:8/866: getdents d6/d45/d51/d55/daa 0 2024-04-25T01:36:03.271 INFO:tasks.workunit.client.1.smithi138.stdout:4/271: mknod d0/d1c/d1e/c4b 0 2024-04-25T01:36:03.272 INFO:tasks.workunit.client.0.smithi033.stdout:0/857: getdents d7/d2d/d4d/d9c/d68 0 2024-04-25T01:36:03.272 INFO:tasks.workunit.client.1.smithi138.stdout:9/294: mkdir d4/d63 0 2024-04-25T01:36:03.273 INFO:tasks.workunit.client.0.smithi033.stdout:9/897: dwrite d0/d2/f48 [0,4194304] 0 2024-04-25T01:36:03.273 INFO:tasks.workunit.client.0.smithi033.stdout:8/867: fdatasync d6/d49/f5a 0 2024-04-25T01:36:03.277 INFO:tasks.workunit.client.0.smithi033.stdout:0/858: unlink d7/d2d/d4d/d9c/d61/fe1 0 2024-04-25T01:36:03.278 INFO:tasks.workunit.client.0.smithi033.stdout:0/859: write d7/f114 [4077757,116414] 0 2024-04-25T01:36:03.279 INFO:tasks.workunit.client.1.smithi138.stdout:4/272: dwrite d0/d1c/f49 [0,4194304] 0 2024-04-25T01:36:03.281 INFO:tasks.workunit.client.0.smithi033.stdout:0/860: truncate d7/d2d/d4d/d9c/dcf/fb1 4439928 0 2024-04-25T01:36:03.281 INFO:tasks.workunit.client.1.smithi138.stdout:4/273: dread - d0/d1c/f28 zero size 2024-04-25T01:36:03.282 INFO:tasks.workunit.client.0.smithi033.stdout:8/868: mknod d6/d45/d52/dcc/d112/d11b/c142 0 2024-04-25T01:36:03.284 INFO:tasks.workunit.client.1.smithi138.stdout:4/274: symlink d0/d16/d33/l4c 0 2024-04-25T01:36:03.284 INFO:tasks.workunit.client.0.smithi033.stdout:0/861: unlink l6 0 2024-04-25T01:36:03.285 INFO:tasks.workunit.client.1.smithi138.stdout:4/275: chown d0/c14 53 1 2024-04-25T01:36:03.286 INFO:tasks.workunit.client.0.smithi033.stdout:8/869: creat d6/d45/d52/df2/f143 x:0 0 0 2024-04-25T01:36:03.287 INFO:tasks.workunit.client.0.smithi033.stdout:9/898: getdents d0/d2/dd/df/d1b/dc0 0 2024-04-25T01:36:03.289 INFO:tasks.workunit.client.0.smithi033.stdout:0/862: mkdir d7/da/d80/d127 0 2024-04-25T01:36:03.291 INFO:tasks.workunit.client.1.smithi138.stdout:4/276: link d0/d1c/f2c d0/d16/d4a/f4d 0 2024-04-25T01:36:03.291 INFO:tasks.workunit.client.0.smithi033.stdout:9/899: creat d0/d11/f139 x:0 0 0 2024-04-25T01:36:03.292 INFO:tasks.workunit.client.0.smithi033.stdout:8/870: read d6/db/f2d [498338,126378] 0 2024-04-25T01:36:03.293 INFO:tasks.workunit.client.1.smithi138.stdout:4/277: symlink d0/d1c/d48/l4e 0 2024-04-25T01:36:03.293 INFO:tasks.workunit.client.0.smithi033.stdout:0/863: creat d7/d2d/f128 x:0 0 0 2024-04-25T01:36:03.296 INFO:tasks.workunit.client.0.smithi033.stdout:0/864: rename d7/d2d/d4c/da0/c8f to d7/de7/c129 0 2024-04-25T01:36:03.297 INFO:tasks.workunit.client.1.smithi138.stdout:4/278: dread d0/d16/d4a/f4d [0,4194304] 0 2024-04-25T01:36:03.297 INFO:tasks.workunit.client.0.smithi033.stdout:8/871: mkdir d6/d45/d51/d5d/d144 0 2024-04-25T01:36:03.298 INFO:tasks.workunit.client.1.smithi138.stdout:4/279: truncate d0/d1c/f2b 4353262 0 2024-04-25T01:36:03.298 INFO:tasks.workunit.client.0.smithi033.stdout:0/865: creat d7/de7/f12a x:0 0 0 2024-04-25T01:36:03.299 INFO:tasks.workunit.client.1.smithi138.stdout:4/280: mkdir d0/d16/d33/d4f 0 2024-04-25T01:36:03.301 INFO:tasks.workunit.client.0.smithi033.stdout:0/866: rename d7/d2d/d64/d51/c11a to d7/d8a/c12b 0 2024-04-25T01:36:03.301 INFO:tasks.workunit.client.0.smithi033.stdout:8/872: dread d6/d45/d52/f27 [0,4194304] 0 2024-04-25T01:36:03.302 INFO:tasks.workunit.client.0.smithi033.stdout:8/873: chown d6/d45/d52/d2e/dd4/c125 71785 1 2024-04-25T01:36:03.303 INFO:tasks.workunit.client.1.smithi138.stdout:4/281: link d0/c3d d0/d16/d33/c50 0 2024-04-25T01:36:03.304 INFO:tasks.workunit.client.0.smithi033.stdout:8/874: chown d6/db/d80/d117/ff9 56 1 2024-04-25T01:36:03.306 INFO:tasks.workunit.client.0.smithi033.stdout:8/875: fdatasync d6/db/d80/d89/fab 0 2024-04-25T01:36:03.306 INFO:tasks.workunit.client.0.smithi033.stdout:8/876: stat d6/d45/d52/d2e/f30 0 2024-04-25T01:36:03.308 INFO:tasks.workunit.client.0.smithi033.stdout:8/877: creat d6/db/d80/db4/f145 x:0 0 0 2024-04-25T01:36:03.310 INFO:tasks.workunit.client.0.smithi033.stdout:8/878: mknod d6/db/d80/d89/c146 0 2024-04-25T01:36:03.311 INFO:tasks.workunit.client.0.smithi033.stdout:8/879: mkdir d6/db/d111/d147 0 2024-04-25T01:36:03.312 INFO:tasks.workunit.client.0.smithi033.stdout:8/880: mknod d6/d45/d52/dae/c148 0 2024-04-25T01:36:03.315 INFO:tasks.workunit.client.0.smithi033.stdout:8/881: creat d6/db/d80/f149 x:0 0 0 2024-04-25T01:36:03.315 INFO:tasks.workunit.client.0.smithi033.stdout:1/746: write d2/d11/f6d [303299,10741] 0 2024-04-25T01:36:03.327 INFO:tasks.workunit.client.0.smithi033.stdout:1/747: dread d2/d11/f91 [0,4194304] 0 2024-04-25T01:36:03.327 INFO:tasks.workunit.client.0.smithi033.stdout:8/882: truncate d6/db/f1a 563241 0 2024-04-25T01:36:03.328 INFO:tasks.workunit.client.0.smithi033.stdout:8/883: chown d6/d45/d51/d5d/lf0 16 1 2024-04-25T01:36:03.329 INFO:tasks.workunit.client.0.smithi033.stdout:7/904: write d4/d23/d3f/f79 [4336394,124715] 0 2024-04-25T01:36:03.329 INFO:tasks.workunit.client.0.smithi033.stdout:1/748: read d2/d11/f1a [3751297,100404] 0 2024-04-25T01:36:03.332 INFO:tasks.workunit.client.0.smithi033.stdout:8/884: creat d6/db/d80/d89/f14a x:0 0 0 2024-04-25T01:36:03.335 INFO:tasks.workunit.client.1.smithi138.stdout:9/295: getdents d4 0 2024-04-25T01:36:03.335 INFO:tasks.workunit.client.1.smithi138.stdout:9/296: readlink d4/d1a/d33/d1d/d3f/l4f 0 2024-04-25T01:36:03.336 INFO:tasks.workunit.client.1.smithi138.stdout:4/282: creat d0/d16/d33/f51 x:0 0 0 2024-04-25T01:36:03.337 INFO:tasks.workunit.client.1.smithi138.stdout:4/283: write d0/d16/d33/f51 [1007376,99017] 0 2024-04-25T01:36:03.338 INFO:tasks.workunit.client.1.smithi138.stdout:9/297: mkdir d4/d1a/d33/d1d/d64 0 2024-04-25T01:36:03.339 INFO:tasks.workunit.client.0.smithi033.stdout:8/885: fdatasync d6/d45/d52/d2e/f30 0 2024-04-25T01:36:03.340 INFO:tasks.workunit.client.0.smithi033.stdout:1/749: dwrite d2/d5/d1b/d46/d5f/d63/d8f/fbc [0,4194304] 0 2024-04-25T01:36:03.340 INFO:tasks.workunit.client.0.smithi033.stdout:8/886: readlink d6/l13f 0 2024-04-25T01:36:03.341 INFO:tasks.workunit.client.1.smithi138.stdout:4/284: rmdir d0/d16/d33/d4f 0 2024-04-25T01:36:03.344 INFO:tasks.workunit.client.0.smithi033.stdout:8/887: dread d6/db/d80/d89/dbf/f135 [0,4194304] 0 2024-04-25T01:36:03.345 INFO:tasks.workunit.client.1.smithi138.stdout:0/292: sync 2024-04-25T01:36:03.346 INFO:tasks.workunit.client.0.smithi033.stdout:8/888: chown d6/d45/d51/d5d/d9d/fce 47548520 1 2024-04-25T01:36:03.347 INFO:tasks.workunit.client.0.smithi033.stdout:9/900: write d0/d2/d43/d73/f9f [2551051,12548] 0 2024-04-25T01:36:03.349 INFO:tasks.workunit.client.0.smithi033.stdout:7/905: dread d4/d23/fc5 [4194304,4194304] 0 2024-04-25T01:36:03.350 INFO:tasks.workunit.client.1.smithi138.stdout:4/285: dwrite d0/d16/d33/f51 [0,4194304] 0 2024-04-25T01:36:03.351 INFO:tasks.workunit.client.0.smithi033.stdout:8/889: fsync d6/d45/d52/d2e/f4a 0 2024-04-25T01:36:03.351 INFO:tasks.workunit.client.1.smithi138.stdout:4/286: fsync d0/d45/f35 0 2024-04-25T01:36:03.351 INFO:tasks.workunit.client.0.smithi033.stdout:1/750: mkdir d2/d5/dee 0 2024-04-25T01:36:03.353 INFO:tasks.workunit.client.1.smithi138.stdout:4/287: write d0/d1c/f2b [2292358,108922] 0 2024-04-25T01:36:03.356 INFO:tasks.workunit.client.0.smithi033.stdout:0/867: truncate d7/d2d/d4d/d9c/dcf/fb1 4051680 0 2024-04-25T01:36:03.357 INFO:tasks.workunit.client.1.smithi138.stdout:4/288: dread d0/d16/d4a/f4d [0,4194304] 0 2024-04-25T01:36:03.358 INFO:tasks.workunit.client.1.smithi138.stdout:0/293: dread d6/d1d/f5f [0,4194304] 0 2024-04-25T01:36:03.361 INFO:tasks.workunit.client.0.smithi033.stdout:8/890: rename d6/d45/d108/d13c to d6/dd3/dfb/d14b 0 2024-04-25T01:36:03.362 INFO:tasks.workunit.client.1.smithi138.stdout:4/289: mkdir d0/d1c/d52 0 2024-04-25T01:36:03.363 INFO:tasks.workunit.client.1.smithi138.stdout:0/294: dread d6/df/f4d [0,4194304] 0 2024-04-25T01:36:03.365 INFO:tasks.workunit.client.0.smithi033.stdout:7/906: truncate d4/d23/f121 4259633 0 2024-04-25T01:36:03.365 INFO:tasks.workunit.client.1.smithi138.stdout:9/298: truncate d4/d1a/f44 3495306 0 2024-04-25T01:36:03.366 INFO:tasks.workunit.client.1.smithi138.stdout:9/299: write d4/d8/f61 [179761,21045] 0 2024-04-25T01:36:03.366 INFO:tasks.workunit.client.1.smithi138.stdout:4/290: creat d0/d16/d4a/f53 x:0 0 0 2024-04-25T01:36:03.368 INFO:tasks.workunit.client.1.smithi138.stdout:0/295: symlink d6/d19/l65 0 2024-04-25T01:36:03.373 INFO:tasks.workunit.client.0.smithi033.stdout:7/907: dwrite d4/d23/da7/ff7 [0,4194304] 0 2024-04-25T01:36:03.380 INFO:tasks.workunit.client.0.smithi033.stdout:9/901: write d0/d11/f18 [1737236,61985] 0 2024-04-25T01:36:03.381 INFO:tasks.workunit.client.0.smithi033.stdout:9/902: stat d0/d2/dd/ca 0 2024-04-25T01:36:03.381 INFO:tasks.workunit.client.0.smithi033.stdout:0/868: chown d7/d2d/d64/d51/f52 10 1 2024-04-25T01:36:03.386 INFO:tasks.workunit.client.1.smithi138.stdout:4/291: symlink d0/d1c/d48/l54 0 2024-04-25T01:36:03.386 INFO:tasks.workunit.client.0.smithi033.stdout:8/891: link d6/db/d80/db4/l139 d6/d45/d52/d2e/d95/d12a/l14c 0 2024-04-25T01:36:03.387 INFO:tasks.workunit.client.0.smithi033.stdout:8/892: chown d6/db/d111/l12f 750 1 2024-04-25T01:36:03.388 INFO:tasks.workunit.client.0.smithi033.stdout:1/751: truncate d2/d2e/d34/d60/fa7 54935 0 2024-04-25T01:36:03.395 INFO:tasks.workunit.client.0.smithi033.stdout:9/903: write d0/d2/dd/d20/f86 [674807,130460] 0 2024-04-25T01:36:03.397 INFO:tasks.workunit.client.0.smithi033.stdout:0/869: dread d7/d2d/d4d/f119 [0,4194304] 0 2024-04-25T01:36:03.397 INFO:tasks.workunit.client.0.smithi033.stdout:7/908: truncate d4/f3d 1236151 0 2024-04-25T01:36:03.402 INFO:tasks.workunit.client.0.smithi033.stdout:8/893: link d6/l39 d6/d45/d51/d55/d84/l14d 0 2024-04-25T01:36:03.405 INFO:tasks.workunit.client.0.smithi033.stdout:0/870: unlink d7/da/df/c39 0 2024-04-25T01:36:03.406 INFO:tasks.workunit.client.0.smithi033.stdout:0/871: truncate d7/d2d/fc1 360330 0 2024-04-25T01:36:03.407 INFO:tasks.workunit.client.0.smithi033.stdout:7/909: symlink d4/d23/d3f/l127 0 2024-04-25T01:36:03.408 INFO:tasks.workunit.client.0.smithi033.stdout:7/910: dread - d4/d2e/d7f/d98/fe3 zero size 2024-04-25T01:36:03.410 INFO:tasks.workunit.client.0.smithi033.stdout:1/752: getdents d2/d5/d1b 0 2024-04-25T01:36:03.412 INFO:tasks.workunit.client.0.smithi033.stdout:8/894: creat d6/db/d80/dd1/f14e x:0 0 0 2024-04-25T01:36:03.414 INFO:tasks.workunit.client.0.smithi033.stdout:9/904: link d0/d2/dd/cdd d0/d2/d43/d4b/dc5/de0/d42/dea/d105/c13a 0 2024-04-25T01:36:03.415 INFO:tasks.workunit.client.0.smithi033.stdout:0/872: dwrite d7/d2d/d64/d51/f52 [4194304,4194304] 0 2024-04-25T01:36:03.415 INFO:tasks.workunit.client.0.smithi033.stdout:8/895: chown d6/d45/d51/d5d/d6e/fb1 9 1 2024-04-25T01:36:03.415 INFO:tasks.workunit.client.1.smithi138.stdout:9/300: creat d4/d1a/d33/d1d/d3f/d52/f65 x:0 0 0 2024-04-25T01:36:03.416 INFO:tasks.workunit.client.1.smithi138.stdout:9/301: write d4/d1a/d33/f4d [3879950,16888] 0 2024-04-25T01:36:03.417 INFO:tasks.workunit.client.0.smithi033.stdout:7/911: creat d4/d23/d3f/d45/d73/dbe/f128 x:0 0 0 2024-04-25T01:36:03.417 INFO:tasks.workunit.client.0.smithi033.stdout:1/753: dread d2/d11/f91 [0,4194304] 0 2024-04-25T01:36:03.424 INFO:tasks.workunit.client.1.smithi138.stdout:9/302: mknod d4/d63/c66 0 2024-04-25T01:36:03.425 INFO:tasks.workunit.client.1.smithi138.stdout:4/292: getdents d0/d1c/d1e 0 2024-04-25T01:36:03.427 INFO:tasks.workunit.client.0.smithi033.stdout:1/754: mkdir d2/d2e/d34/def 0 2024-04-25T01:36:03.429 INFO:tasks.workunit.client.1.smithi138.stdout:9/303: creat d4/d57/f67 x:0 0 0 2024-04-25T01:36:03.430 INFO:tasks.workunit.client.1.smithi138.stdout:9/304: dread - d4/d8/f23 zero size 2024-04-25T01:36:03.431 INFO:tasks.workunit.client.0.smithi033.stdout:8/896: rename d6/l19 to d6/d45/d52/d2e/l14f 0 2024-04-25T01:36:03.432 INFO:tasks.workunit.client.1.smithi138.stdout:9/305: write d4/f26 [4243616,112543] 0 2024-04-25T01:36:03.432 INFO:tasks.workunit.client.0.smithi033.stdout:1/755: creat d2/d5/d6c/d94/ff0 x:0 0 0 2024-04-25T01:36:03.433 INFO:tasks.workunit.client.1.smithi138.stdout:4/293: dwrite d0/f3f [0,4194304] 0 2024-04-25T01:36:03.435 INFO:tasks.workunit.client.1.smithi138.stdout:4/294: write d0/d45/f35 [814987,54421] 0 2024-04-25T01:36:03.442 INFO:tasks.workunit.client.0.smithi033.stdout:1/756: dwrite d2/d2e/d49/fbd [4194304,4194304] 0 2024-04-25T01:36:03.443 INFO:tasks.workunit.client.1.smithi138.stdout:4/295: rename d0/d45/f32 to d0/d1c/f55 0 2024-04-25T01:36:03.444 INFO:tasks.workunit.client.0.smithi033.stdout:0/873: dwrite d7/d2d/d9b/fef [0,4194304] 0 2024-04-25T01:36:03.444 INFO:tasks.workunit.client.0.smithi033.stdout:1/757: stat d2/d2e/f71 0 2024-04-25T01:36:03.447 INFO:tasks.workunit.client.1.smithi138.stdout:4/296: chown d0/fa 12 1 2024-04-25T01:36:03.449 INFO:tasks.workunit.client.0.smithi033.stdout:7/912: dwrite d4/d23/d3f/d45/f7c [0,4194304] 0 2024-04-25T01:36:03.450 INFO:tasks.workunit.client.0.smithi033.stdout:1/758: dread - d2/d2e/d34/d60/fb8 zero size 2024-04-25T01:36:03.450 INFO:tasks.workunit.client.1.smithi138.stdout:4/297: stat d0/d16/f34 0 2024-04-25T01:36:03.450 INFO:tasks.workunit.client.1.smithi138.stdout:4/298: chown d0/d1c/f2c 4439 1 2024-04-25T01:36:03.451 INFO:tasks.workunit.client.0.smithi033.stdout:0/874: mkdir d7/d2d/d4d/d9c/d68/dd2/d12c 0 2024-04-25T01:36:03.451 INFO:tasks.workunit.client.1.smithi138.stdout:4/299: unlink d0/d16/f18 0 2024-04-25T01:36:03.454 INFO:tasks.workunit.client.1.smithi138.stdout:4/300: rmdir d0/d16/d4a 39 2024-04-25T01:36:03.457 INFO:tasks.workunit.client.0.smithi033.stdout:7/913: write d4/d2e/ff [4365769,75677] 0 2024-04-25T01:36:03.460 INFO:tasks.workunit.client.1.smithi138.stdout:4/301: symlink d0/d1c/d52/l56 0 2024-04-25T01:36:03.460 INFO:tasks.workunit.client.0.smithi033.stdout:1/759: dwrite d2/d11/d28/fe1 [0,4194304] 0 2024-04-25T01:36:03.463 INFO:tasks.workunit.client.0.smithi033.stdout:0/875: symlink d7/da/df/d50/dfe/l12d 0 2024-04-25T01:36:03.464 INFO:tasks.workunit.client.1.smithi138.stdout:0/296: write d6/d39/d43/f21 [1034323,92245] 0 2024-04-25T01:36:03.466 INFO:tasks.workunit.client.0.smithi033.stdout:8/897: write d6/d45/d52/dcc/fdc [870127,84983] 0 2024-04-25T01:36:03.468 INFO:tasks.workunit.client.0.smithi033.stdout:0/876: symlink d7/da/df/dc7/l12e 0 2024-04-25T01:36:03.469 INFO:tasks.workunit.client.0.smithi033.stdout:8/898: write d6/d45/d51/d5d/d11d/f122 [853354,111160] 0 2024-04-25T01:36:03.469 INFO:tasks.workunit.client.1.smithi138.stdout:4/302: dread d0/d45/f35 [0,4194304] 0 2024-04-25T01:36:03.470 INFO:tasks.workunit.client.1.smithi138.stdout:0/297: mkdir d6/df/d66 0 2024-04-25T01:36:03.471 INFO:tasks.workunit.client.0.smithi033.stdout:1/760: dwrite d2/d11/d28/fd9 [0,4194304] 0 2024-04-25T01:36:03.473 INFO:tasks.workunit.client.1.smithi138.stdout:4/303: dread d0/d1c/f2c [0,4194304] 0 2024-04-25T01:36:03.473 INFO:tasks.workunit.client.1.smithi138.stdout:4/304: chown d0/f40 225638 1 2024-04-25T01:36:03.474 INFO:tasks.workunit.client.0.smithi033.stdout:1/761: chown d2/d5/d1b/d46/d5f/d63/d64/ccc 1055897322 1 2024-04-25T01:36:03.476 INFO:tasks.workunit.client.0.smithi033.stdout:8/899: creat d6/d45/d52/d2e/d95/d12a/f150 x:0 0 0 2024-04-25T01:36:03.476 INFO:tasks.workunit.client.1.smithi138.stdout:4/305: mknod d0/d1c/d1e/c57 0 2024-04-25T01:36:03.479 INFO:tasks.workunit.client.0.smithi033.stdout:1/762: unlink d2/fb6 0 2024-04-25T01:36:03.479 INFO:tasks.workunit.client.1.smithi138.stdout:4/306: mknod d0/d1c/d52/c58 0 2024-04-25T01:36:03.480 INFO:tasks.workunit.client.0.smithi033.stdout:8/900: fdatasync d6/db/d80/d89/dbf/f135 0 2024-04-25T01:36:03.482 INFO:tasks.workunit.client.0.smithi033.stdout:0/877: creat d7/d2d/d4d/d9c/d61/f12f x:0 0 0 2024-04-25T01:36:03.483 INFO:tasks.workunit.client.0.smithi033.stdout:1/763: chown d2/d5/d1b/c3a 6666103 1 2024-04-25T01:36:03.486 INFO:tasks.workunit.client.0.smithi033.stdout:0/878: symlink d7/da/df/d50/l130 0 2024-04-25T01:36:03.490 INFO:tasks.workunit.client.0.smithi033.stdout:7/914: truncate d4/d2e/d11c/d1f/f71 2251710 0 2024-04-25T01:36:03.490 INFO:tasks.workunit.client.0.smithi033.stdout:8/901: dwrite d6/d45/d52/dcc/d112/f137 [0,4194304] 0 2024-04-25T01:36:03.492 INFO:tasks.workunit.client.1.smithi138.stdout:4/307: mkdir d0/d59 0 2024-04-25T01:36:03.494 INFO:tasks.workunit.client.0.smithi033.stdout:0/879: rename d7/da/df/d14/f15 to d7/d2d/f131 0 2024-04-25T01:36:03.496 INFO:tasks.workunit.client.1.smithi138.stdout:9/306: mkdir d4/d1a/d68 0 2024-04-25T01:36:03.497 INFO:tasks.workunit.client.0.smithi033.stdout:7/915: mkdir d4/d23/d3f/d45/d73/dbe/d129 0 2024-04-25T01:36:03.499 INFO:tasks.workunit.client.1.smithi138.stdout:9/307: chown d4/d63/c66 7 1 2024-04-25T01:36:03.500 INFO:tasks.workunit.client.1.smithi138.stdout:4/308: dwrite d0/d1c/f2b [4194304,4194304] 0 2024-04-25T01:36:03.500 INFO:tasks.workunit.client.0.smithi033.stdout:0/880: rename d7/da/fb7 to d7/d2d/d4c/da0/da1/f132 0 2024-04-25T01:36:03.501 INFO:tasks.workunit.client.1.smithi138.stdout:4/309: fsync d0/d16/f44 0 2024-04-25T01:36:03.501 INFO:tasks.workunit.client.1.smithi138.stdout:9/308: symlink d4/d1a/d33/l69 0 2024-04-25T01:36:03.502 INFO:tasks.workunit.client.0.smithi033.stdout:0/881: read d7/d2d/d4d/d9c/f59 [2537576,112880] 0 2024-04-25T01:36:03.502 INFO:tasks.workunit.client.1.smithi138.stdout:9/309: stat d4/d1a/d33/d1d/d3f 0 2024-04-25T01:36:03.505 INFO:tasks.workunit.client.0.smithi033.stdout:8/902: getdents d6/d45/d51/d5d 0 2024-04-25T01:36:03.505 INFO:tasks.workunit.client.0.smithi033.stdout:8/903: fsync d6/db/fe0 0 2024-04-25T01:36:03.509 INFO:tasks.workunit.client.0.smithi033.stdout:7/916: rename d4/d23/d3f/d45/d6c/l72 to d4/d23/d3f/d45/d73/d8f/l12a 0 2024-04-25T01:36:03.512 INFO:tasks.workunit.client.0.smithi033.stdout:7/917: dread d4/d2e/d11c/da9/ff3 [0,4194304] 0 2024-04-25T01:36:03.514 INFO:tasks.workunit.client.0.smithi033.stdout:1/764: dread d2/d5/d75/fe8 [0,4194304] 0 2024-04-25T01:36:03.515 INFO:tasks.workunit.client.0.smithi033.stdout:7/918: creat d4/d2e/d65/dcb/f12b x:0 0 0 2024-04-25T01:36:03.517 INFO:tasks.workunit.client.0.smithi033.stdout:1/765: unlink d2/d5/d1b/d46/f69 0 2024-04-25T01:36:03.519 INFO:tasks.workunit.client.0.smithi033.stdout:7/919: creat d4/d57/da5/f12c x:0 0 0 2024-04-25T01:36:03.519 INFO:tasks.workunit.client.0.smithi033.stdout:1/766: creat d2/d5/d1b/d46/d5f/d63/d9d/ff1 x:0 0 0 2024-04-25T01:36:03.521 INFO:tasks.workunit.client.0.smithi033.stdout:7/920: write d4/d2e/d11c/d1f/f46 [1427070,63951] 0 2024-04-25T01:36:03.523 INFO:tasks.workunit.client.0.smithi033.stdout:1/767: unlink d2/d5/ca4 0 2024-04-25T01:36:03.524 INFO:tasks.workunit.client.0.smithi033.stdout:0/882: dread d7/d2d/d64/fad [0,4194304] 0 2024-04-25T01:36:03.530 INFO:tasks.workunit.client.0.smithi033.stdout:1/768: rename d2/d5/d75/d59/d83/fb4 to d2/de2/ff2 0 2024-04-25T01:36:03.530 INFO:tasks.workunit.client.0.smithi033.stdout:0/883: dwrite d7/d2d/d4d/d9c/dcf/ddd/ffa [0,4194304] 0 2024-04-25T01:36:03.536 INFO:tasks.workunit.client.0.smithi033.stdout:0/884: getdents d7/d2d/d4d/d9c/d61/d70/dee 0 2024-04-25T01:36:03.538 INFO:tasks.workunit.client.0.smithi033.stdout:0/885: write d7/da/df/d50/dbb/ff7 [851182,31457] 0 2024-04-25T01:36:03.538 INFO:tasks.workunit.client.0.smithi033.stdout:0/886: rmdir d7 39 2024-04-25T01:36:03.542 INFO:tasks.workunit.client.0.smithi033.stdout:0/887: getdents d7/d2d/d64/d51 0 2024-04-25T01:36:03.544 INFO:tasks.workunit.client.0.smithi033.stdout:0/888: symlink d7/d2d/d4d/d9c/d61/d70/l133 0 2024-04-25T01:36:03.545 INFO:tasks.workunit.client.0.smithi033.stdout:0/889: symlink d7/d2d/d4d/d9c/d61/d70/dee/l134 0 2024-04-25T01:36:03.546 INFO:tasks.workunit.client.0.smithi033.stdout:0/890: stat d7/da/df/d14/d45/c6a 0 2024-04-25T01:36:03.547 INFO:tasks.workunit.client.0.smithi033.stdout:0/891: creat d7/d2d/d4c/da0/da1/dd7/f135 x:0 0 0 2024-04-25T01:36:03.548 INFO:tasks.workunit.client.0.smithi033.stdout:0/892: chown d7/d2d/d4c/da0/da1/f106 220 1 2024-04-25T01:36:03.551 INFO:tasks.workunit.client.0.smithi033.stdout:0/893: rename d7/d2d/f128 to d7/d2d/d4d/d9c/d61/f136 0 2024-04-25T01:36:03.552 INFO:tasks.workunit.client.0.smithi033.stdout:0/894: chown d7/da/df/d50/c79 7 1 2024-04-25T01:36:03.554 INFO:tasks.workunit.client.1.smithi138.stdout:6/442: sync 2024-04-25T01:36:03.554 INFO:tasks.workunit.client.0.smithi033.stdout:0/895: mkdir d7/d2d/d43/d44/d137 0 2024-04-25T01:36:03.558 INFO:tasks.workunit.client.1.smithi138.stdout:9/310: unlink d4/d1a/d33/c1c 0 2024-04-25T01:36:03.558 INFO:tasks.workunit.client.1.smithi138.stdout:4/310: dread d0/d16/d4a/f4d [0,4194304] 0 2024-04-25T01:36:03.559 INFO:tasks.workunit.client.1.smithi138.stdout:9/311: read d4/d1a/f2d [935321,64579] 0 2024-04-25T01:36:03.559 INFO:tasks.workunit.client.1.smithi138.stdout:6/443: write dc/d10/f33 [154963,10942] 0 2024-04-25T01:36:03.561 INFO:tasks.workunit.client.1.smithi138.stdout:6/444: write dc/d10/d6d/f7c [149411,27961] 0 2024-04-25T01:36:03.563 INFO:tasks.workunit.client.0.smithi033.stdout:8/904: write d6/db/d80/d117/ff9 [710697,99300] 0 2024-04-25T01:36:03.564 INFO:tasks.workunit.client.1.smithi138.stdout:4/311: mknod d0/d59/c5a 0 2024-04-25T01:36:03.565 INFO:tasks.workunit.client.0.smithi033.stdout:8/905: dread - d6/d45/d51/d55/d84/fbe zero size 2024-04-25T01:36:03.566 INFO:tasks.workunit.client.0.smithi033.stdout:8/906: dread - d6/dd3/f138 zero size 2024-04-25T01:36:03.567 INFO:tasks.workunit.client.0.smithi033.stdout:7/921: write d4/d23/d3f/d45/d73/d8f/fba [470462,76327] 0 2024-04-25T01:36:03.571 INFO:tasks.workunit.client.0.smithi033.stdout:8/907: getdents d6/d45/d51/d55/deb 0 2024-04-25T01:36:03.572 INFO:tasks.workunit.client.1.smithi138.stdout:4/312: link d0/f2f d0/d1c/d48/f5b 0 2024-04-25T01:36:03.572 INFO:tasks.workunit.client.1.smithi138.stdout:6/445: dwrite dc/f4e [0,4194304] 0 2024-04-25T01:36:03.572 INFO:tasks.workunit.client.1.smithi138.stdout:9/312: dwrite d4/d1a/f20 [4194304,4194304] 0 2024-04-25T01:36:03.574 INFO:tasks.workunit.client.0.smithi033.stdout:0/896: getdents d7/d2d 0 2024-04-25T01:36:03.575 INFO:tasks.workunit.client.1.smithi138.stdout:4/313: truncate d0/d16/f44 606790 0 2024-04-25T01:36:03.575 INFO:tasks.workunit.client.1.smithi138.stdout:6/446: read dc/f1f [3976789,121659] 0 2024-04-25T01:36:03.580 INFO:tasks.workunit.client.1.smithi138.stdout:6/447: fsync dc/f1f 0 2024-04-25T01:36:03.582 INFO:tasks.workunit.client.1.smithi138.stdout:4/314: mkdir d0/d5c 0 2024-04-25T01:36:03.583 INFO:tasks.workunit.client.1.smithi138.stdout:9/313: dwrite d4/d1a/f42 [0,4194304] 0 2024-04-25T01:36:03.583 INFO:tasks.workunit.client.0.smithi033.stdout:8/908: truncate d6/d45/d52/d2e/f56 2880798 0 2024-04-25T01:36:03.585 INFO:tasks.workunit.client.1.smithi138.stdout:9/314: write d4/d8/f61 [396268,11430] 0 2024-04-25T01:36:03.587 INFO:tasks.workunit.client.1.smithi138.stdout:9/315: write d4/d8/f1f [1885717,90029] 0 2024-04-25T01:36:03.587 INFO:tasks.workunit.client.0.smithi033.stdout:0/897: dwrite d7/d2d/d4d/d9c/d61/f94 [0,4194304] 0 2024-04-25T01:36:03.590 INFO:tasks.workunit.client.0.smithi033.stdout:1/769: dread d2/d2e/f67 [0,4194304] 0 2024-04-25T01:36:03.590 INFO:tasks.workunit.client.0.smithi033.stdout:8/909: truncate d6/d45/d51/d55/fa9 1052264 0 2024-04-25T01:36:03.594 INFO:tasks.workunit.client.1.smithi138.stdout:4/315: mknod d0/d16/c5d 0 2024-04-25T01:36:03.595 INFO:tasks.workunit.client.1.smithi138.stdout:9/316: symlink d4/d57/l6a 0 2024-04-25T01:36:03.596 INFO:tasks.workunit.client.0.smithi033.stdout:8/910: mknod d6/db/d80/d89/c151 0 2024-04-25T01:36:03.597 INFO:tasks.workunit.client.1.smithi138.stdout:9/317: dread - d4/d1a/d3c/f51 zero size 2024-04-25T01:36:03.597 INFO:tasks.workunit.client.0.smithi033.stdout:1/770: rename d2/d5/d1b/d46/d5f/d63/c76 to d2/d7f/db0/cf3 0 2024-04-25T01:36:03.599 INFO:tasks.workunit.client.0.smithi033.stdout:8/911: getdents d6/d45/d51/d5d/d6e 0 2024-04-25T01:36:03.600 INFO:tasks.workunit.client.1.smithi138.stdout:4/316: truncate d0/f2f 2427294 0 2024-04-25T01:36:03.600 INFO:tasks.workunit.client.0.smithi033.stdout:1/771: mknod d2/d5/d6c/dc6/cf4 0 2024-04-25T01:36:03.601 INFO:tasks.workunit.client.0.smithi033.stdout:8/912: creat d6/d45/d51/d5d/d9d/f152 x:0 0 0 2024-04-25T01:36:03.603 INFO:tasks.workunit.client.1.smithi138.stdout:9/318: mkdir d4/d8/d6b 0 2024-04-25T01:36:03.604 INFO:tasks.workunit.client.0.smithi033.stdout:1/772: mkdir d2/d2e/df5 0 2024-04-25T01:36:03.607 INFO:tasks.workunit.client.1.smithi138.stdout:9/319: dread d4/d1a/d33/f16 [0,4194304] 0 2024-04-25T01:36:03.609 INFO:tasks.workunit.client.0.smithi033.stdout:1/773: rename d2/d11/d28/c55 to d2/d7f/cf6 0 2024-04-25T01:36:03.609 INFO:tasks.workunit.client.1.smithi138.stdout:9/320: write d4/d1a/d3c/f51 [227972,37601] 0 2024-04-25T01:36:03.609 INFO:tasks.workunit.client.0.smithi033.stdout:1/774: chown d2/f81 1951353550 1 2024-04-25T01:36:03.612 INFO:tasks.workunit.client.0.smithi033.stdout:1/775: link d2/d5/d75/f8d d2/d5/d1b/d46/d5f/d63/d64/ff7 0 2024-04-25T01:36:03.615 INFO:tasks.workunit.client.1.smithi138.stdout:9/321: dread d4/d1a/f21 [0,4194304] 0 2024-04-25T01:36:03.617 INFO:tasks.workunit.client.1.smithi138.stdout:9/322: creat d4/f6c x:0 0 0 2024-04-25T01:36:03.618 INFO:tasks.workunit.client.1.smithi138.stdout:9/323: write f1 [2655455,74283] 0 2024-04-25T01:36:03.627 INFO:tasks.workunit.client.1.smithi138.stdout:9/324: dread d4/d1a/f3b [0,4194304] 0 2024-04-25T01:36:03.629 INFO:tasks.workunit.client.1.smithi138.stdout:9/325: dread - d4/d57/f67 zero size 2024-04-25T01:36:03.630 INFO:tasks.workunit.client.1.smithi138.stdout:9/326: readlink d4/d1a/d33/d1d/d3f/l4f 0 2024-04-25T01:36:03.631 INFO:tasks.workunit.client.0.smithi033.stdout:0/898: write d7/d2d/fc4 [879273,10173] 0 2024-04-25T01:36:03.632 INFO:tasks.workunit.client.1.smithi138.stdout:9/327: rmdir d4/d8 39 2024-04-25T01:36:03.636 INFO:tasks.workunit.client.1.smithi138.stdout:9/328: getdents d4/d1a/d33 0 2024-04-25T01:36:03.637 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:03 smithi138 ceph-mon[32977]: pgmap v17: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 89 MiB/s rd, 297 MiB/s wr, 601 op/s 2024-04-25T01:36:03.639 INFO:tasks.workunit.client.1.smithi138.stdout:9/329: rmdir d4/d1a/d33/d1d/d3f/d52 39 2024-04-25T01:36:03.653 INFO:tasks.workunit.client.0.smithi033.stdout:1/776: chown d2/d5/d1b/l45 3679 1 2024-04-25T01:36:03.654 INFO:tasks.workunit.client.0.smithi033.stdout:1/777: read d2/f62 [1715999,24732] 0 2024-04-25T01:36:03.654 INFO:tasks.workunit.client.1.smithi138.stdout:9/330: dread d4/d8/f45 [0,4194304] 0 2024-04-25T01:36:03.656 INFO:tasks.workunit.client.1.smithi138.stdout:9/331: mknod d4/d57/c6d 0 2024-04-25T01:36:03.658 INFO:tasks.workunit.client.0.smithi033.stdout:1/778: dread d2/d5/d75/fe8 [0,4194304] 0 2024-04-25T01:36:03.658 INFO:tasks.workunit.client.1.smithi138.stdout:9/332: symlink d4/d1a/d33/d1d/d3f/l6e 0 2024-04-25T01:36:03.659 INFO:tasks.workunit.client.0.smithi033.stdout:1/779: write d2/f32 [3567153,102763] 0 2024-04-25T01:36:03.659 INFO:tasks.workunit.client.1.smithi138.stdout:9/333: getdents d4/d1a/d68 0 2024-04-25T01:36:03.660 INFO:tasks.workunit.client.1.smithi138.stdout:9/334: fdatasync d4/f4a 0 2024-04-25T01:36:03.664 INFO:tasks.workunit.client.1.smithi138.stdout:9/335: creat d4/d49/f6f x:0 0 0 2024-04-25T01:36:03.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/336: chown d4/d8/d2c/l41 2912 1 2024-04-25T01:36:03.665 INFO:tasks.workunit.client.0.smithi033.stdout:8/913: dwrite d6/d45/d51/d55/fa9 [0,4194304] 0 2024-04-25T01:36:03.667 INFO:tasks.workunit.client.0.smithi033.stdout:0/899: write d7/d2d/d4c/da0/fb0 [776008,70532] 0 2024-04-25T01:36:03.669 INFO:tasks.workunit.client.1.smithi138.stdout:9/337: rmdir d4/d1a/d33/d1d/d64 0 2024-04-25T01:36:03.670 INFO:tasks.workunit.client.1.smithi138.stdout:9/338: fsync d4/d1a/d33/d1d/d3f/f5d 0 2024-04-25T01:36:03.670 INFO:tasks.workunit.client.0.smithi033.stdout:8/914: truncate d6/d45/f8b 1002817 0 2024-04-25T01:36:03.671 INFO:tasks.workunit.client.1.smithi138.stdout:9/339: fdatasync d4/d57/f5a 0 2024-04-25T01:36:03.672 INFO:tasks.workunit.client.0.smithi033.stdout:8/915: dread - d6/d45/d51/d5d/db9/fc4 zero size 2024-04-25T01:36:03.674 INFO:tasks.workunit.client.0.smithi033.stdout:0/900: symlink d7/d2d/d43/dc0/l138 0 2024-04-25T01:36:03.676 INFO:tasks.workunit.client.1.smithi138.stdout:9/340: rename d4/f3e to d4/d1a/f70 0 2024-04-25T01:36:03.676 INFO:tasks.workunit.client.0.smithi033.stdout:0/901: mknod d7/d2d/d43/d44/dd4/c139 0 2024-04-25T01:36:03.677 INFO:tasks.workunit.client.0.smithi033.stdout:0/902: fdatasync d7/da/df/f9a 0 2024-04-25T01:36:03.680 INFO:tasks.workunit.client.0.smithi033.stdout:0/903: mkdir d7/d2d/d4d/d9c/d61/d70/d13a 0 2024-04-25T01:36:03.682 INFO:tasks.workunit.client.1.smithi138.stdout:9/341: dwrite d4/d1a/d55/f5c [0,4194304] 0 2024-04-25T01:36:03.683 INFO:tasks.workunit.client.0.smithi033.stdout:0/904: symlink d7/d33/l13b 0 2024-04-25T01:36:03.686 INFO:tasks.workunit.client.1.smithi138.stdout:9/342: mknod d4/d1a/d3c/c71 0 2024-04-25T01:36:03.687 INFO:tasks.workunit.client.0.smithi033.stdout:8/916: dread d6/d45/d52/f28 [4194304,4194304] 0 2024-04-25T01:36:03.689 INFO:tasks.workunit.client.1.smithi138.stdout:9/343: mkdir d4/d49/d72 0 2024-04-25T01:36:03.690 INFO:tasks.workunit.client.1.smithi138.stdout:9/344: chown d4/l2b 3259869 1 2024-04-25T01:36:03.693 INFO:tasks.workunit.client.1.smithi138.stdout:9/345: mkdir d4/d63/d73 0 2024-04-25T01:36:03.694 INFO:tasks.workunit.client.1.smithi138.stdout:9/346: chown d4/d1a/d55/f5f 4 1 2024-04-25T01:36:03.698 INFO:tasks.workunit.client.0.smithi033.stdout:0/905: dwrite d7/d2d/d4d/d9c/fb8 [0,4194304] 0 2024-04-25T01:36:03.698 INFO:tasks.workunit.client.0.smithi033.stdout:8/917: dwrite d6/f58 [0,4194304] 0 2024-04-25T01:36:03.699 INFO:tasks.workunit.client.1.smithi138.stdout:9/347: write d4/f17 [1930004,101715] 0 2024-04-25T01:36:03.702 INFO:tasks.workunit.client.0.smithi033.stdout:8/918: fdatasync d6/db/d80/d89/faf 0 2024-04-25T01:36:03.702 INFO:tasks.workunit.client.1.smithi138.stdout:9/348: creat d4/d1a/f74 x:0 0 0 2024-04-25T01:36:03.704 INFO:tasks.workunit.client.0.smithi033.stdout:8/919: readlink d6/d45/d51/d5d/d9d/l121 0 2024-04-25T01:36:03.707 INFO:tasks.workunit.client.0.smithi033.stdout:0/906: link d7/d2d/d4d/c5f d7/d2d/d4c/da0/da1/dd7/c13c 0 2024-04-25T01:36:03.708 INFO:tasks.workunit.client.0.smithi033.stdout:0/907: dread - d7/d2d/d4c/f107 zero size 2024-04-25T01:36:03.708 INFO:tasks.workunit.client.0.smithi033.stdout:8/920: creat d6/d45/d51/d5d/d9d/d10b/f153 x:0 0 0 2024-04-25T01:36:03.711 INFO:tasks.workunit.client.0.smithi033.stdout:8/921: mkdir d6/db/d154 0 2024-04-25T01:36:03.714 INFO:tasks.workunit.client.0.smithi033.stdout:8/922: symlink d6/l155 0 2024-04-25T01:36:03.715 INFO:tasks.workunit.client.0.smithi033.stdout:0/908: dwrite d7/d2d/d43/d92/fea [0,4194304] 0 2024-04-25T01:36:03.717 INFO:tasks.workunit.client.1.smithi138.stdout:3/468: sync 2024-04-25T01:36:03.718 INFO:tasks.workunit.client.0.smithi033.stdout:0/909: creat d7/d2d/f13d x:0 0 0 2024-04-25T01:36:03.720 INFO:tasks.workunit.client.1.smithi138.stdout:3/469: symlink db/df/d17/d23/d88/l8c 0 2024-04-25T01:36:03.721 INFO:tasks.workunit.client.1.smithi138.stdout:3/470: chown db/df/d17/d20/d57/d6a/f6e 14 1 2024-04-25T01:36:03.722 INFO:tasks.workunit.client.1.smithi138.stdout:3/471: chown db/df/d17/d23/d34/f50 74088057 1 2024-04-25T01:36:03.722 INFO:tasks.workunit.client.0.smithi033.stdout:0/910: rmdir d7/da/df/d50 39 2024-04-25T01:36:03.723 INFO:tasks.workunit.client.0.smithi033.stdout:8/923: dwrite d6/d45/d51/d55/f5c [0,4194304] 0 2024-04-25T01:36:03.724 INFO:tasks.workunit.client.1.smithi138.stdout:3/472: creat db/df/d17/d20/d57/d6a/f8d x:0 0 0 2024-04-25T01:36:03.725 INFO:tasks.workunit.client.1.smithi138.stdout:3/473: stat db/df/d17/d20/c45 0 2024-04-25T01:36:03.727 INFO:tasks.workunit.client.0.smithi033.stdout:8/924: creat d6/db/d80/db4/f156 x:0 0 0 2024-04-25T01:36:03.728 INFO:tasks.workunit.client.1.smithi138.stdout:3/474: mknod db/df/c8e 0 2024-04-25T01:36:03.729 INFO:tasks.workunit.client.0.smithi033.stdout:8/925: chown d6/d45/d51/d5d/d9d/cb6 59485894 1 2024-04-25T01:36:03.729 INFO:tasks.workunit.client.0.smithi033.stdout:0/911: link d7/d2d/d43/d44/c126 d7/d2d/d64/c13e 0 2024-04-25T01:36:03.732 INFO:tasks.workunit.client.1.smithi138.stdout:3/475: unlink db/l48 0 2024-04-25T01:36:03.732 INFO:tasks.workunit.client.0.smithi033.stdout:0/912: symlink d7/l13f 0 2024-04-25T01:36:03.737 INFO:tasks.workunit.client.0.smithi033.stdout:0/913: dread d7/d2d/d4c/da0/da1/f123 [0,4194304] 0 2024-04-25T01:36:03.738 INFO:tasks.workunit.client.0.smithi033.stdout:0/914: fsync d7/d2d/d64/fad 0 2024-04-25T01:36:03.740 INFO:tasks.workunit.client.0.smithi033.stdout:0/915: creat d7/d2d/d4c/da0/f140 x:0 0 0 2024-04-25T01:36:03.741 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:03 smithi033 ceph-mon[26353]: pgmap v17: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 89 MiB/s rd, 297 MiB/s wr, 601 op/s 2024-04-25T01:36:03.742 INFO:tasks.workunit.client.0.smithi033.stdout:0/916: fsync d7/d2d/d4d/d9c/d61/f12f 0 2024-04-25T01:36:03.743 INFO:tasks.workunit.client.0.smithi033.stdout:8/926: truncate d6/d45/d52/f3f 3003615 0 2024-04-25T01:36:03.744 INFO:tasks.workunit.client.0.smithi033.stdout:0/917: fsync d7/d2d/f131 0 2024-04-25T01:36:03.745 INFO:tasks.workunit.client.0.smithi033.stdout:8/927: mknod d6/d45/d52/d2e/c157 0 2024-04-25T01:36:03.752 INFO:tasks.workunit.client.0.smithi033.stdout:0/918: symlink d7/da/df/d50/dfe/l141 0 2024-04-25T01:36:03.755 INFO:tasks.workunit.client.0.smithi033.stdout:0/919: creat d7/da/df/d50/dfe/f142 x:0 0 0 2024-04-25T01:36:03.756 INFO:tasks.workunit.client.1.smithi138.stdout:3/476: dread db/df/d17/d23/d34/f6d [0,4194304] 0 2024-04-25T01:36:03.757 INFO:tasks.workunit.client.1.smithi138.stdout:3/477: chown db/df/l66 5085842 1 2024-04-25T01:36:03.760 INFO:tasks.workunit.client.1.smithi138.stdout:3/478: creat db/df/d17/d20/f8f x:0 0 0 2024-04-25T01:36:03.760 INFO:tasks.workunit.client.0.smithi033.stdout:8/928: dwrite d6/db/f61 [0,4194304] 0 2024-04-25T01:36:03.761 INFO:tasks.workunit.client.0.smithi033.stdout:0/920: rename d7/d2d/d4c/da0 to d7/d2d/d43/dc0/d143 0 2024-04-25T01:36:03.765 INFO:tasks.workunit.client.0.smithi033.stdout:8/929: unlink d6/db/d80/d89/dbf/f135 0 2024-04-25T01:36:03.768 INFO:tasks.workunit.client.0.smithi033.stdout:0/921: mknod d7/d2d/d43/d44/c144 0 2024-04-25T01:36:03.772 INFO:tasks.workunit.client.0.smithi033.stdout:0/922: link d7/da/df/d50/l130 d7/d2d/d4d/d9c/d61/d70/d13a/l145 0 2024-04-25T01:36:03.780 INFO:tasks.workunit.client.0.smithi033.stdout:0/923: dwrite d7/d2d/d4d/d9c/d61/f12f [0,4194304] 0 2024-04-25T01:36:03.782 INFO:tasks.workunit.client.0.smithi033.stdout:8/930: dwrite d6/db/d80/d89/fab [0,4194304] 0 2024-04-25T01:36:03.786 INFO:tasks.workunit.client.0.smithi033.stdout:8/931: unlink d6/d45/d51/d55/daa/fd7 0 2024-04-25T01:36:03.787 INFO:tasks.workunit.client.0.smithi033.stdout:8/932: chown d6/d45/d51/d5d/db9 53 1 2024-04-25T01:36:03.789 INFO:tasks.workunit.client.0.smithi033.stdout:8/933: chown d6/d45/d51/d55/c7a 7979497 1 2024-04-25T01:36:03.790 INFO:tasks.workunit.client.1.smithi138.stdout:3/479: dread db/d14/f32 [4194304,4194304] 0 2024-04-25T01:36:03.791 INFO:tasks.workunit.client.0.smithi033.stdout:0/924: dwrite d7/da/df/d14/fbe [0,4194304] 0 2024-04-25T01:36:03.791 INFO:tasks.workunit.client.0.smithi033.stdout:8/934: write d6/d45/d51/d55/f5c [1573840,31246] 0 2024-04-25T01:36:03.793 INFO:tasks.workunit.client.1.smithi138.stdout:3/480: dread db/f5b [0,4194304] 0 2024-04-25T01:36:03.794 INFO:tasks.workunit.client.0.smithi033.stdout:8/935: write d6/db/f1a [1566991,35440] 0 2024-04-25T01:36:03.795 INFO:tasks.workunit.client.0.smithi033.stdout:8/936: chown d6/d45/d51/f4f 485 1 2024-04-25T01:36:03.796 INFO:tasks.workunit.client.0.smithi033.stdout:8/937: read d6/db/f61 [3272389,79861] 0 2024-04-25T01:36:03.797 INFO:tasks.workunit.client.0.smithi033.stdout:0/925: creat d7/de7/d10d/f146 x:0 0 0 2024-04-25T01:36:03.797 INFO:tasks.workunit.client.1.smithi138.stdout:3/481: rename db/df/c7b to db/df/c90 0 2024-04-25T01:36:03.798 INFO:tasks.workunit.client.0.smithi033.stdout:8/938: chown d6/db/d80/d117/cd8 3 1 2024-04-25T01:36:03.799 INFO:tasks.workunit.client.0.smithi033.stdout:0/926: write d7/da/df/f9a [434339,100064] 0 2024-04-25T01:36:03.800 INFO:tasks.workunit.client.0.smithi033.stdout:0/927: truncate d7/d2d/d4d/f82 805781 0 2024-04-25T01:36:03.801 INFO:tasks.workunit.client.0.smithi033.stdout:0/928: chown d7/d2d/d64 2711 1 2024-04-25T01:36:03.802 INFO:tasks.workunit.client.1.smithi138.stdout:3/482: dread db/d14/f32 [4194304,4194304] 0 2024-04-25T01:36:03.807 INFO:tasks.workunit.client.1.smithi138.stdout:3/483: dread db/df/d17/d20/f29 [0,4194304] 0 2024-04-25T01:36:03.813 INFO:tasks.workunit.client.1.smithi138.stdout:3/484: dwrite db/df/d17/d23/d34/f6d [4194304,4194304] 0 2024-04-25T01:36:03.817 INFO:tasks.workunit.client.1.smithi138.stdout:3/485: creat db/d14/d7a/f91 x:0 0 0 2024-04-25T01:36:03.820 INFO:tasks.workunit.client.1.smithi138.stdout:3/486: creat db/d14/d7a/f92 x:0 0 0 2024-04-25T01:36:03.821 INFO:tasks.workunit.client.1.smithi138.stdout:3/487: write db/df/d17/d23/d34/f80 [813854,8084] 0 2024-04-25T01:36:03.830 INFO:tasks.workunit.client.1.smithi138.stdout:3/488: truncate db/df/d19/f36 319830 0 2024-04-25T01:36:03.832 INFO:tasks.workunit.client.0.smithi033.stdout:0/929: getdents d7/da/df/d50/dbb 0 2024-04-25T01:36:03.835 INFO:tasks.workunit.client.0.smithi033.stdout:0/930: rmdir d7/da/df/d50/dbb 39 2024-04-25T01:36:03.835 INFO:tasks.workunit.client.0.smithi033.stdout:0/931: dread - d7/d2d/d43/f122 zero size 2024-04-25T01:36:03.836 INFO:tasks.workunit.client.0.smithi033.stdout:8/939: dread d6/d45/d51/d55/ff6 [0,4194304] 0 2024-04-25T01:36:03.839 INFO:tasks.workunit.client.0.smithi033.stdout:8/940: symlink d6/d45/d52/d2e/dd4/l158 0 2024-04-25T01:36:03.840 INFO:tasks.workunit.client.1.smithi138.stdout:3/489: dread db/df/d17/d23/f30 [0,4194304] 0 2024-04-25T01:36:03.840 INFO:tasks.workunit.client.0.smithi033.stdout:0/932: write d7/da/df/d14/d45/da2/fc8 [701219,122346] 0 2024-04-25T01:36:03.841 INFO:tasks.workunit.client.0.smithi033.stdout:8/941: rmdir d6/d45/d52/d2e/d87 39 2024-04-25T01:36:03.841 INFO:tasks.workunit.client.0.smithi033.stdout:0/933: stat d7/d2d/d64/fd1 0 2024-04-25T01:36:03.842 INFO:tasks.workunit.client.1.smithi138.stdout:3/490: creat db/df/d19/f93 x:0 0 0 2024-04-25T01:36:03.845 INFO:tasks.workunit.client.1.smithi138.stdout:3/491: creat db/df/d19/d1a/f94 x:0 0 0 2024-04-25T01:36:03.847 INFO:tasks.workunit.client.0.smithi033.stdout:8/942: truncate d6/dd3/dfb/f105 473397 0 2024-04-25T01:36:03.848 INFO:tasks.workunit.client.0.smithi033.stdout:0/934: dwrite d7/da/f48 [0,4194304] 0 2024-04-25T01:36:03.849 INFO:tasks.workunit.client.0.smithi033.stdout:0/935: chown d7/de7/ce8 24646 1 2024-04-25T01:36:03.851 INFO:tasks.workunit.client.0.smithi033.stdout:8/943: creat d6/d45/d51/d9e/f159 x:0 0 0 2024-04-25T01:36:03.853 INFO:tasks.workunit.client.0.smithi033.stdout:0/936: rename d7/da/df/dc7/l12e to d7/d2d/d4c/l147 0 2024-04-25T01:36:03.854 INFO:tasks.workunit.client.0.smithi033.stdout:8/944: symlink d6/db/d80/db4/l15a 0 2024-04-25T01:36:03.855 INFO:tasks.workunit.client.0.smithi033.stdout:0/937: dread d7/d2d/d64/fad [4194304,4194304] 0 2024-04-25T01:36:03.857 INFO:tasks.workunit.client.0.smithi033.stdout:0/938: write d7/da/f5d [1732617,78868] 0 2024-04-25T01:36:03.858 INFO:tasks.workunit.client.0.smithi033.stdout:0/939: dread - d7/de7/f12a zero size 2024-04-25T01:36:03.859 INFO:tasks.workunit.client.0.smithi033.stdout:0/940: fsync d7/d2d/d4d/d9c/d61/d70/fa9 0 2024-04-25T01:36:03.860 INFO:tasks.workunit.client.0.smithi033.stdout:8/945: link d6/db/d111/l12f d6/d45/d52/d2e/l15b 0 2024-04-25T01:36:03.864 INFO:tasks.workunit.client.0.smithi033.stdout:0/941: creat d7/da/df/d50/dfe/f148 x:0 0 0 2024-04-25T01:36:03.866 INFO:tasks.workunit.client.0.smithi033.stdout:0/942: fdatasync d7/d2d/d4d/d9c/fae 0 2024-04-25T01:36:03.871 INFO:tasks.workunit.client.0.smithi033.stdout:0/943: creat d7/da/df/d50/dbb/f149 x:0 0 0 2024-04-25T01:36:03.878 INFO:tasks.workunit.client.0.smithi033.stdout:0/944: dwrite d7/da/df/d14/d9d/fe9 [0,4194304] 0 2024-04-25T01:36:03.880 INFO:tasks.workunit.client.0.smithi033.stdout:0/945: chown d7/da/df/d14/f69 8 1 2024-04-25T01:36:03.883 INFO:tasks.workunit.client.1.smithi138.stdout:1/348: sync 2024-04-25T01:36:03.884 INFO:tasks.workunit.client.1.smithi138.stdout:1/349: fdatasync d3/d27/d3e/f6b 0 2024-04-25T01:36:03.886 INFO:tasks.workunit.client.0.smithi033.stdout:0/946: dwrite d7/d2d/d43/d92/fea [0,4194304] 0 2024-04-25T01:36:03.890 INFO:tasks.workunit.client.1.smithi138.stdout:1/350: dwrite d3/f2c [0,4194304] 0 2024-04-25T01:36:03.895 INFO:tasks.workunit.client.0.smithi033.stdout:0/947: dwrite d7/d2d/d4d/d9c/d61/f120 [0,4194304] 0 2024-04-25T01:36:03.896 INFO:tasks.workunit.client.1.smithi138.stdout:1/351: dread d3/d27/d2d/f71 [0,4194304] 0 2024-04-25T01:36:03.934 INFO:tasks.workunit.client.1.smithi138.stdout:1/352: dwrite d3/d45/f48 [0,4194304] 0 2024-04-25T01:36:03.939 INFO:tasks.workunit.client.1.smithi138.stdout:1/353: getdents d3/d27/d2d 0 2024-04-25T01:36:03.945 INFO:tasks.workunit.client.1.smithi138.stdout:1/354: dwrite d3/d4/d53/f6c [0,4194304] 0 2024-04-25T01:36:03.950 INFO:tasks.workunit.client.1.smithi138.stdout:1/355: dread d3/d4/f37 [0,4194304] 0 2024-04-25T01:36:03.950 INFO:tasks.workunit.client.1.smithi138.stdout:1/356: chown d3/d27/d3e/d51/d62/d65 0 1 2024-04-25T01:36:03.953 INFO:tasks.workunit.client.1.smithi138.stdout:1/357: getdents d3/d6d 0 2024-04-25T01:36:03.957 INFO:tasks.workunit.client.1.smithi138.stdout:1/358: rename d3/db/d2b to d3/d45/d72 0 2024-04-25T01:36:03.961 INFO:tasks.workunit.client.1.smithi138.stdout:1/359: rename d3/d45/d72/c50 to d3/d27/d3e/d51/d62/c73 0 2024-04-25T01:36:03.962 INFO:tasks.workunit.client.1.smithi138.stdout:1/360: rename d3/d27/d16 to d3/d27/d16/d74 22 2024-04-25T01:36:03.975 INFO:tasks.workunit.client.1.smithi138.stdout:1/361: dread d3/db/f5a [0,4194304] 0 2024-04-25T01:36:03.976 INFO:tasks.workunit.client.1.smithi138.stdout:1/362: fdatasync d3/f13 0 2024-04-25T01:36:03.982 INFO:tasks.workunit.client.1.smithi138.stdout:1/363: dwrite d3/db/d4a/f6e [0,4194304] 0 2024-04-25T01:36:03.985 INFO:tasks.workunit.client.1.smithi138.stdout:1/364: creat d3/d27/d3e/d51/f75 x:0 0 0 2024-04-25T01:36:03.990 INFO:tasks.workunit.client.1.smithi138.stdout:1/365: link d3/d45/d72/d3d/f5e d3/d27/d3e/f76 0 2024-04-25T01:36:04.002 INFO:tasks.workunit.client.0.smithi033.stdout:2/881: sync 2024-04-25T01:36:04.003 INFO:tasks.workunit.client.0.smithi033.stdout:2/882: fsync da/d50/f115 0 2024-04-25T01:36:04.005 INFO:tasks.workunit.client.0.smithi033.stdout:2/883: fsync f7 0 2024-04-25T01:36:04.007 INFO:tasks.workunit.client.0.smithi033.stdout:2/884: symlink da/d27/d31/l121 0 2024-04-25T01:36:04.008 INFO:tasks.workunit.client.0.smithi033.stdout:2/885: fsync da/d2c/dcd/fc4 0 2024-04-25T01:36:04.009 INFO:tasks.workunit.client.0.smithi033.stdout:2/886: write da/d27/d31/d40/db1/dbf/dd5/ff9 [190586,56096] 0 2024-04-25T01:36:04.011 INFO:tasks.workunit.client.0.smithi033.stdout:2/887: write da/d27/d31/d40/db1/dbf/dd5/ff9 [1427718,80667] 0 2024-04-25T01:36:04.012 INFO:tasks.workunit.client.0.smithi033.stdout:2/888: chown da/d50/d6a/d43/f101 1352284 1 2024-04-25T01:36:04.016 INFO:tasks.workunit.client.0.smithi033.stdout:2/889: dread da/d5b/d5e/d76/ffd [0,4194304] 0 2024-04-25T01:36:04.017 INFO:tasks.workunit.client.0.smithi033.stdout:2/890: truncate da/d27/fdf 637749 0 2024-04-25T01:36:04.020 INFO:tasks.workunit.client.0.smithi033.stdout:2/891: rmdir da/d27/d31/d40/db1 39 2024-04-25T01:36:04.021 INFO:tasks.workunit.client.0.smithi033.stdout:2/892: stat da/d27/d31/d40/db1 0 2024-04-25T01:36:04.023 INFO:tasks.workunit.client.0.smithi033.stdout:2/893: fsync da/d5b/d5e/fbd 0 2024-04-25T01:36:04.025 INFO:tasks.workunit.client.0.smithi033.stdout:2/894: read da/d27/d31/f46 [1709237,23085] 0 2024-04-25T01:36:04.028 INFO:tasks.workunit.client.0.smithi033.stdout:2/895: link da/d27/f59 da/d50/d6a/d43/f122 0 2024-04-25T01:36:04.103 INFO:tasks.workunit.client.1.smithi138.stdout:2/423: sync 2024-04-25T01:36:04.103 INFO:tasks.workunit.client.1.smithi138.stdout:6/448: sync 2024-04-25T01:36:04.103 INFO:tasks.workunit.client.1.smithi138.stdout:9/349: sync 2024-04-25T01:36:04.103 INFO:tasks.workunit.client.1.smithi138.stdout:8/520: sync 2024-04-25T01:36:04.103 INFO:tasks.workunit.client.1.smithi138.stdout:7/411: sync 2024-04-25T01:36:04.103 INFO:tasks.workunit.client.1.smithi138.stdout:5/362: sync 2024-04-25T01:36:04.103 INFO:tasks.workunit.client.1.smithi138.stdout:4/317: sync 2024-04-25T01:36:04.103 INFO:tasks.workunit.client.1.smithi138.stdout:0/298: sync 2024-04-25T01:36:04.104 INFO:tasks.workunit.client.1.smithi138.stdout:6/449: readlink dc/d2e/d55/l6f 0 2024-04-25T01:36:04.106 INFO:tasks.workunit.client.1.smithi138.stdout:6/450: chown dc/d7b 929281 1 2024-04-25T01:36:04.106 INFO:tasks.workunit.client.1.smithi138.stdout:5/363: read d4/da/d6e/f26 [3008961,87420] 0 2024-04-25T01:36:04.107 INFO:tasks.workunit.client.1.smithi138.stdout:7/412: truncate da/de/d13/d21/f7e 664936 0 2024-04-25T01:36:04.107 INFO:tasks.workunit.client.1.smithi138.stdout:2/424: write d0/d32/d35/f69 [3716999,82286] 0 2024-04-25T01:36:04.107 INFO:tasks.workunit.client.1.smithi138.stdout:8/521: mknod d3/d16/d1f/d31/d82/ca2 0 2024-04-25T01:36:04.159 INFO:tasks.workunit.client.1.smithi138.stdout:7/413: dread - da/de/d13/d30/f80 zero size 2024-04-25T01:36:04.160 INFO:tasks.workunit.client.1.smithi138.stdout:2/425: mknod d0/d5a/d15/d51/c84 0 2024-04-25T01:36:04.160 INFO:tasks.workunit.client.1.smithi138.stdout:8/522: stat d3/d16/d88/da/de/f10 0 2024-04-25T01:36:04.161 INFO:tasks.workunit.client.1.smithi138.stdout:4/318: dread d0/d16/f29 [0,4194304] 0 2024-04-25T01:36:04.162 INFO:tasks.workunit.client.1.smithi138.stdout:7/414: fsync da/de/f1a 0 2024-04-25T01:36:04.163 INFO:tasks.workunit.client.1.smithi138.stdout:9/350: dwrite d4/d1a/f56 [0,4194304] 0 2024-04-25T01:36:04.164 INFO:tasks.workunit.client.0.smithi033.stdout:5/932: sync 2024-04-25T01:36:04.165 INFO:tasks.workunit.client.0.smithi033.stdout:5/933: stat d1c/d2a/d50/d52/da6/d3e/d70/d7a/dab/cec 0 2024-04-25T01:36:04.169 INFO:tasks.workunit.client.1.smithi138.stdout:2/426: mkdir d0/d85 0 2024-04-25T01:36:04.171 INFO:tasks.workunit.client.0.smithi033.stdout:5/934: dwrite d1c/f2e [0,4194304] 0 2024-04-25T01:36:04.171 INFO:tasks.workunit.client.1.smithi138.stdout:6/451: link dc/d2e/d55/f75 dc/d10/d28/d80/f8b 0 2024-04-25T01:36:04.172 INFO:tasks.workunit.client.1.smithi138.stdout:2/427: readlink d0/d5a/l76 0 2024-04-25T01:36:04.172 INFO:tasks.workunit.client.1.smithi138.stdout:5/364: dwrite d4/da/f47 [0,4194304] 0 2024-04-25T01:36:04.173 INFO:tasks.workunit.client.1.smithi138.stdout:8/523: creat d3/d44/d48/fa3 x:0 0 0 2024-04-25T01:36:04.173 INFO:tasks.workunit.client.1.smithi138.stdout:7/415: mknod da/de/d1b/d49/c8b 0 2024-04-25T01:36:04.175 INFO:tasks.workunit.client.0.smithi033.stdout:5/935: rmdir d1c/d2a/d50/d52/da6/d4a 39 2024-04-25T01:36:04.176 INFO:tasks.workunit.client.0.smithi033.stdout:5/936: readlink d1c/d2a/d50/d52/da6/d3e/d70/d9b/la4 0 2024-04-25T01:36:04.182 INFO:tasks.workunit.client.1.smithi138.stdout:6/452: mkdir dc/d2e/d37/d56/d8c 0 2024-04-25T01:36:04.182 INFO:tasks.workunit.client.1.smithi138.stdout:4/319: dwrite d0/d16/f44 [0,4194304] 0 2024-04-25T01:36:04.185 INFO:tasks.workunit.client.1.smithi138.stdout:9/351: dwrite d4/d1a/f74 [0,4194304] 0 2024-04-25T01:36:04.185 INFO:tasks.workunit.client.1.smithi138.stdout:7/416: chown da/de/d1b/d34/l60 1917 1 2024-04-25T01:36:04.185 INFO:tasks.workunit.client.1.smithi138.stdout:8/524: mkdir d3/d44/da4 0 2024-04-25T01:36:04.186 INFO:tasks.workunit.client.1.smithi138.stdout:6/453: chown dc/d10/c1e 1174 1 2024-04-25T01:36:04.187 INFO:tasks.workunit.client.1.smithi138.stdout:5/365: rmdir d4/da/d6e/d22/d35/d4d/d5e 39 2024-04-25T01:36:04.191 INFO:tasks.workunit.client.1.smithi138.stdout:4/320: dread - d0/d16/f43 zero size 2024-04-25T01:36:04.194 INFO:tasks.workunit.client.1.smithi138.stdout:6/454: mkdir dc/d8d 0 2024-04-25T01:36:04.196 INFO:tasks.workunit.client.1.smithi138.stdout:2/428: dwrite d0/d5a/d15/d7e/f4a [0,4194304] 0 2024-04-25T01:36:04.198 INFO:tasks.workunit.client.1.smithi138.stdout:2/429: chown d0/d32/f63 960 1 2024-04-25T01:36:04.199 INFO:tasks.workunit.client.1.smithi138.stdout:1/366: sync 2024-04-25T01:36:04.199 INFO:tasks.workunit.client.1.smithi138.stdout:3/492: sync 2024-04-25T01:36:04.200 INFO:tasks.workunit.client.1.smithi138.stdout:7/417: rename da/de/c82 to da/de/d1b/d49/d81/d51/c8c 0 2024-04-25T01:36:04.202 INFO:tasks.workunit.client.0.smithi033.stdout:5/937: dread d1c/d2a/d50/d52/da6/d4a/f8c [0,4194304] 0 2024-04-25T01:36:04.204 INFO:tasks.workunit.client.0.smithi033.stdout:5/938: creat d1c/d2a/d3d/f12d x:0 0 0 2024-04-25T01:36:04.207 INFO:tasks.workunit.client.1.smithi138.stdout:6/455: mkdir dc/d10/d67/d8e 0 2024-04-25T01:36:04.207 INFO:tasks.workunit.client.1.smithi138.stdout:8/525: getdents d3/d16/d88/da/db/d15/d56/d8f 0 2024-04-25T01:36:04.209 INFO:tasks.workunit.client.1.smithi138.stdout:2/430: chown d0/d5a/c19 2906 1 2024-04-25T01:36:04.210 INFO:tasks.workunit.client.0.smithi033.stdout:5/939: dread d1c/d2a/d50/f91 [0,4194304] 0 2024-04-25T01:36:04.211 INFO:tasks.workunit.client.1.smithi138.stdout:9/352: write d4/d1a/f2d [2291339,121533] 0 2024-04-25T01:36:04.211 INFO:tasks.workunit.client.1.smithi138.stdout:1/367: creat d3/d4/f77 x:0 0 0 2024-04-25T01:36:04.213 INFO:tasks.workunit.client.0.smithi033.stdout:5/940: rmdir d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0 39 2024-04-25T01:36:04.214 INFO:tasks.workunit.client.1.smithi138.stdout:5/366: rename d4/da/d6e/f5b to d4/da/d6e/d22/d35/d4d/d5e/f75 0 2024-04-25T01:36:04.215 INFO:tasks.workunit.client.1.smithi138.stdout:4/321: rmdir d0/d5c 0 2024-04-25T01:36:04.216 INFO:tasks.workunit.client.1.smithi138.stdout:4/322: chown d0/d1c/f55 42540394 1 2024-04-25T01:36:04.217 INFO:tasks.workunit.client.0.smithi033.stdout:5/941: rename d1c/f1f to d1c/d2a/d50/d52/d10c/d12c/f12e 0 2024-04-25T01:36:04.217 INFO:tasks.workunit.client.1.smithi138.stdout:6/456: unlink dc/f18 0 2024-04-25T01:36:04.218 INFO:tasks.workunit.client.1.smithi138.stdout:6/457: write dc/f66 [1230300,39185] 0 2024-04-25T01:36:04.219 INFO:tasks.workunit.client.0.smithi033.stdout:5/942: symlink d1c/d2a/d50/d5b/d86/d10b/d124/l12f 0 2024-04-25T01:36:04.220 INFO:tasks.workunit.client.1.smithi138.stdout:3/493: unlink db/l43 0 2024-04-25T01:36:04.221 INFO:tasks.workunit.client.0.smithi033.stdout:5/943: mkdir d1c/d2a/d50/db1/dd3/d130 0 2024-04-25T01:36:04.221 INFO:tasks.workunit.client.0.smithi033.stdout:4/936: sync 2024-04-25T01:36:04.221 INFO:tasks.workunit.client.0.smithi033.stdout:9/905: sync 2024-04-25T01:36:04.221 INFO:tasks.workunit.client.0.smithi033.stdout:7/922: sync 2024-04-25T01:36:04.221 INFO:tasks.workunit.client.0.smithi033.stdout:3/922: sync 2024-04-25T01:36:04.222 INFO:tasks.workunit.client.1.smithi138.stdout:1/368: unlink d3/d27/d2d/f71 0 2024-04-25T01:36:04.223 INFO:tasks.workunit.client.0.smithi033.stdout:9/906: readlink d0/d2/dd/d20/l81 0 2024-04-25T01:36:04.224 INFO:tasks.workunit.client.0.smithi033.stdout:5/944: symlink d1c/d2a/d3d/l131 0 2024-04-25T01:36:04.224 INFO:tasks.workunit.client.1.smithi138.stdout:8/526: symlink d3/d16/d88/da/db/d15/d6f/la5 0 2024-04-25T01:36:04.225 INFO:tasks.workunit.client.0.smithi033.stdout:9/907: stat d0/d2/d43/d4b/dc5/de0/d7e/d58/fa6 0 2024-04-25T01:36:04.225 INFO:tasks.workunit.client.0.smithi033.stdout:4/937: creat d6/d14/d35/d44/d45/f133 x:0 0 0 2024-04-25T01:36:04.226 INFO:tasks.workunit.client.0.smithi033.stdout:3/923: truncate d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/f3b 614077 0 2024-04-25T01:36:04.227 INFO:tasks.workunit.client.0.smithi033.stdout:5/945: symlink d1c/d2a/d50/d8d/de1/l132 0 2024-04-25T01:36:04.227 INFO:tasks.workunit.client.1.smithi138.stdout:7/418: dread da/de/d1b/f31 [0,4194304] 0 2024-04-25T01:36:04.228 INFO:tasks.workunit.client.1.smithi138.stdout:2/431: unlink d0/d5a/d15/d51/l61 0 2024-04-25T01:36:04.228 INFO:tasks.workunit.client.0.smithi033.stdout:5/946: write d1c/d2a/d50/d52/f122 [129782,29306] 0 2024-04-25T01:36:04.229 INFO:tasks.workunit.client.0.smithi033.stdout:5/947: chown d1c/d53/fd0 1 1 2024-04-25T01:36:04.230 INFO:tasks.workunit.client.0.smithi033.stdout:3/924: read d8/dd/d3d/d63/d7c/d9d/fb6 [621793,130906] 0 2024-04-25T01:36:04.230 INFO:tasks.workunit.client.1.smithi138.stdout:6/458: creat dc/d2e/d37/d56/f8f x:0 0 0 2024-04-25T01:36:04.230 INFO:tasks.workunit.client.0.smithi033.stdout:7/923: link d4/d2e/d7f/fab d4/d23/da7/dc2/f12d 0 2024-04-25T01:36:04.231 INFO:tasks.workunit.client.0.smithi033.stdout:4/938: fdatasync d6/d14/d35/d3a/f56 0 2024-04-25T01:36:04.232 INFO:tasks.workunit.client.1.smithi138.stdout:3/494: mknod db/d14/c95 0 2024-04-25T01:36:04.234 INFO:tasks.workunit.client.1.smithi138.stdout:1/369: creat d3/d27/d3e/d51/d62/d65/f78 x:0 0 0 2024-04-25T01:36:04.241 INFO:tasks.workunit.client.0.smithi033.stdout:7/924: unlink d4/d57/da5/f109 0 2024-04-25T01:36:04.242 INFO:tasks.workunit.client.0.smithi033.stdout:5/948: rmdir d1c/d2a/d3d 39 2024-04-25T01:36:04.243 INFO:tasks.workunit.client.1.smithi138.stdout:1/370: creat d3/d27/d2d/f79 x:0 0 0 2024-04-25T01:36:04.247 INFO:tasks.workunit.client.0.smithi033.stdout:7/925: write d4/d2e/d11c/f9a [1308438,130754] 0 2024-04-25T01:36:04.248 INFO:tasks.workunit.client.1.smithi138.stdout:3/495: dwrite db/df/d17/d5d/f7f [0,4194304] 0 2024-04-25T01:36:04.249 INFO:tasks.workunit.client.1.smithi138.stdout:3/496: stat db/df/d17/d23/d34 0 2024-04-25T01:36:04.249 INFO:tasks.workunit.client.0.smithi033.stdout:9/908: write d0/d2/dd/df/f3e [444222,57893] 0 2024-04-25T01:36:04.250 INFO:tasks.workunit.client.0.smithi033.stdout:3/925: write d8/dd/d3d/d4f/f69 [1797140,109962] 0 2024-04-25T01:36:04.251 INFO:tasks.workunit.client.1.smithi138.stdout:9/353: getdents d4/d57 0 2024-04-25T01:36:04.252 INFO:tasks.workunit.client.0.smithi033.stdout:7/926: symlink d4/d23/l12e 0 2024-04-25T01:36:04.252 INFO:tasks.workunit.client.1.smithi138.stdout:6/459: mknod dc/d2e/d55/c90 0 2024-04-25T01:36:04.253 INFO:tasks.workunit.client.1.smithi138.stdout:3/497: truncate db/df/d17/d20/d57/d6a/f8d 238483 0 2024-04-25T01:36:04.253 INFO:tasks.workunit.client.0.smithi033.stdout:3/926: stat d8/dd/d40/c7b 0 2024-04-25T01:36:04.253 INFO:tasks.workunit.client.1.smithi138.stdout:6/460: chown dc/d10/l45 2996861 1 2024-04-25T01:36:04.254 INFO:tasks.workunit.client.0.smithi033.stdout:3/927: chown l7 392151 1 2024-04-25T01:36:04.254 INFO:tasks.workunit.client.1.smithi138.stdout:6/461: stat dc/d2e/d37/l5d 0 2024-04-25T01:36:04.255 INFO:tasks.workunit.client.0.smithi033.stdout:7/927: chown d4/d23/f121 26689 1 2024-04-25T01:36:04.255 INFO:tasks.workunit.client.1.smithi138.stdout:5/367: creat d4/da/d6e/d22/d74/f76 x:0 0 0 2024-04-25T01:36:04.256 INFO:tasks.workunit.client.0.smithi033.stdout:9/909: dread d0/d2/dd/d20/d4e/f104 [0,4194304] 0 2024-04-25T01:36:04.256 INFO:tasks.workunit.client.1.smithi138.stdout:8/527: write d3/d16/d88/da/db/d15/d5d/f9a [1377865,108119] 0 2024-04-25T01:36:04.256 INFO:tasks.workunit.client.0.smithi033.stdout:4/939: dread d6/d14/d35/fb4 [0,4194304] 0 2024-04-25T01:36:04.257 INFO:tasks.workunit.client.0.smithi033.stdout:3/928: fsync d8/dd/d3d/d4f/fcb 0 2024-04-25T01:36:04.257 INFO:tasks.workunit.client.1.smithi138.stdout:4/323: getdents d0/d1c/d48 0 2024-04-25T01:36:04.257 INFO:tasks.workunit.client.1.smithi138.stdout:6/462: write dc/d10/f33 [299435,73863] 0 2024-04-25T01:36:04.257 INFO:tasks.workunit.client.1.smithi138.stdout:2/432: truncate d0/d32/f4b 1987029 0 2024-04-25T01:36:04.259 INFO:tasks.workunit.client.1.smithi138.stdout:1/371: chown d3/db/d4a/c59 234 1 2024-04-25T01:36:04.260 INFO:tasks.workunit.client.0.smithi033.stdout:3/929: write d8/dd/d3d/d63/d7c/ded/dc7/ddf/f112 [601884,44978] 0 2024-04-25T01:36:04.261 INFO:tasks.workunit.client.0.smithi033.stdout:4/940: chown d6/d14/d48/lba 587 1 2024-04-25T01:36:04.262 INFO:tasks.workunit.client.0.smithi033.stdout:7/928: creat d4/d2e/d11c/d1f/d78/f12f x:0 0 0 2024-04-25T01:36:04.263 INFO:tasks.workunit.client.0.smithi033.stdout:4/941: write d6/d12/f130 [335460,64252] 0 2024-04-25T01:36:04.264 INFO:tasks.workunit.client.1.smithi138.stdout:2/433: dread d0/d21/f3a [0,4194304] 0 2024-04-25T01:36:04.264 INFO:tasks.workunit.client.0.smithi033.stdout:3/930: dread - d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/daa/f100 zero size 2024-04-25T01:36:04.265 INFO:tasks.workunit.client.0.smithi033.stdout:7/929: creat d4/d23/d3f/d45/d73/dbe/f130 x:0 0 0 2024-04-25T01:36:04.266 INFO:tasks.workunit.client.1.smithi138.stdout:5/368: rename d4/da/d6e/d22/c32 to d4/da/d6e/d22/c77 0 2024-04-25T01:36:04.266 INFO:tasks.workunit.client.0.smithi033.stdout:7/930: stat d4/d2e/d7f/l8c 0 2024-04-25T01:36:04.267 INFO:tasks.workunit.client.1.smithi138.stdout:7/419: getdents da/de/d1b/d34 0 2024-04-25T01:36:04.269 INFO:tasks.workunit.client.0.smithi033.stdout:3/931: truncate d8/dd/d3d/d63/d7c/d9d/f127 604300 0 2024-04-25T01:36:04.269 INFO:tasks.workunit.client.0.smithi033.stdout:7/931: write d4/d23/da7/ff7 [2100878,52210] 0 2024-04-25T01:36:04.271 INFO:tasks.workunit.client.0.smithi033.stdout:5/949: dread d1c/d2a/d50/d52/d10c/d12c/f12e [0,4194304] 0 2024-04-25T01:36:04.272 INFO:tasks.workunit.client.0.smithi033.stdout:9/910: write d0/d2/dd/f56 [237023,63797] 0 2024-04-25T01:36:04.272 INFO:tasks.workunit.client.0.smithi033.stdout:4/942: dread d6/d12/f26 [0,4194304] 0 2024-04-25T01:36:04.273 INFO:tasks.workunit.client.1.smithi138.stdout:9/354: rename d4/d1a/d33/l1e to d4/d1a/d33/d1d/d3f/d52/l75 0 2024-04-25T01:36:04.274 INFO:tasks.workunit.client.1.smithi138.stdout:9/355: chown d4/d1a/d33/d1d 60076 1 2024-04-25T01:36:04.275 INFO:tasks.workunit.client.1.smithi138.stdout:4/324: symlink d0/l5e 0 2024-04-25T01:36:04.275 INFO:tasks.workunit.client.1.smithi138.stdout:8/528: dread d3/d16/d88/da/db/d15/d5d/f9a [0,4194304] 0 2024-04-25T01:36:04.276 INFO:tasks.workunit.client.1.smithi138.stdout:4/325: chown d0/d1c/f2c 62737 1 2024-04-25T01:36:04.277 INFO:tasks.workunit.client.1.smithi138.stdout:6/463: mkdir dc/d2e/d55/d81/d91 0 2024-04-25T01:36:04.278 INFO:tasks.workunit.client.0.smithi033.stdout:9/911: symlink d0/d2/d43/l13b 0 2024-04-25T01:36:04.278 INFO:tasks.workunit.client.1.smithi138.stdout:5/369: fdatasync d4/da/d6e/d22/d74/d59/f70 0 2024-04-25T01:36:04.279 INFO:tasks.workunit.client.1.smithi138.stdout:0/299: sync 2024-04-25T01:36:04.280 INFO:tasks.workunit.client.1.smithi138.stdout:1/372: symlink d3/l7a 0 2024-04-25T01:36:04.280 INFO:tasks.workunit.client.1.smithi138.stdout:3/498: link db/l3b db/d14/d7a/l96 0 2024-04-25T01:36:04.281 INFO:tasks.workunit.client.1.smithi138.stdout:4/326: dread d0/d1c/f2b [0,4194304] 0 2024-04-25T01:36:04.282 INFO:tasks.workunit.client.1.smithi138.stdout:9/356: write d4/d1a/f70 [266611,112884] 0 2024-04-25T01:36:04.282 INFO:tasks.workunit.client.1.smithi138.stdout:8/529: fsync d3/f3a 0 2024-04-25T01:36:04.283 INFO:tasks.workunit.client.0.smithi033.stdout:5/950: chown d1c/d2a/d3d/d59/lfc 3839 1 2024-04-25T01:36:04.284 INFO:tasks.workunit.client.0.smithi033.stdout:4/943: dwrite d6/d14/d35/d44/d45/d46/d57/d70/d87/dc7/fbf [4194304,4194304] 0 2024-04-25T01:36:04.286 INFO:tasks.workunit.client.0.smithi033.stdout:9/912: rename d0/d2/dd/df/d1b/dc0 to d0/d2/dd/d5f/d11b/d13c 0 2024-04-25T01:36:04.287 INFO:tasks.workunit.client.1.smithi138.stdout:6/464: mknod dc/d2e/d55/d81/c92 0 2024-04-25T01:36:04.288 INFO:tasks.workunit.client.0.smithi033.stdout:8/946: sync 2024-04-25T01:36:04.288 INFO:tasks.workunit.client.0.smithi033.stdout:1/780: sync 2024-04-25T01:36:04.288 INFO:tasks.workunit.client.0.smithi033.stdout:0/948: sync 2024-04-25T01:36:04.288 INFO:tasks.workunit.client.0.smithi033.stdout:2/896: sync 2024-04-25T01:36:04.288 INFO:tasks.workunit.client.1.smithi138.stdout:0/300: rmdir d6/d39/d52 39 2024-04-25T01:36:04.289 INFO:tasks.workunit.client.1.smithi138.stdout:1/373: mknod d3/d27/d2d/c7b 0 2024-04-25T01:36:04.289 INFO:tasks.workunit.client.0.smithi033.stdout:4/944: truncate d6/d14/d35/d3a/d4d/d5e/d7c/d94/fe7 671600 0 2024-04-25T01:36:04.292 INFO:tasks.workunit.client.0.smithi033.stdout:1/781: fdatasync d2/d11/d28/f79 0 2024-04-25T01:36:04.292 INFO:tasks.workunit.client.0.smithi033.stdout:2/897: fdatasync da/f45 0 2024-04-25T01:36:04.293 INFO:tasks.workunit.client.1.smithi138.stdout:3/499: creat db/df/d17/d5d/f97 x:0 0 0 2024-04-25T01:36:04.293 INFO:tasks.workunit.client.0.smithi033.stdout:9/913: mkdir d0/d2/d43/d4b/dc3/d13d 0 2024-04-25T01:36:04.294 INFO:tasks.workunit.client.1.smithi138.stdout:8/530: dwrite d3/d16/d88/da/db/d15/d56/f9c [0,4194304] 0 2024-04-25T01:36:04.296 INFO:tasks.workunit.client.0.smithi033.stdout:0/949: rmdir d7/da/df/dc7 39 2024-04-25T01:36:04.297 INFO:tasks.workunit.client.1.smithi138.stdout:9/357: creat d4/d49/f76 x:0 0 0 2024-04-25T01:36:04.297 INFO:tasks.workunit.client.0.smithi033.stdout:8/947: chown d6/d45/d51/d55/d84/l14d 13321130 1 2024-04-25T01:36:04.298 INFO:tasks.workunit.client.0.smithi033.stdout:5/951: dread d1c/d2a/d50/d52/da6/d3e/d70/dae/f103 [0,4194304] 0 2024-04-25T01:36:04.299 INFO:tasks.workunit.client.0.smithi033.stdout:9/914: fdatasync d0/d2/dd/d5f/d11b/d13c/f12f 0 2024-04-25T01:36:04.302 INFO:tasks.workunit.client.0.smithi033.stdout:3/932: dread d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/fc1 [0,4194304] 0 2024-04-25T01:36:04.304 INFO:tasks.workunit.client.0.smithi033.stdout:7/932: dwrite d4/d23/d3f/d45/f5e [0,4194304] 0 2024-04-25T01:36:04.306 INFO:tasks.workunit.client.1.smithi138.stdout:2/434: getdents d0/d21/d45 0 2024-04-25T01:36:04.306 INFO:tasks.workunit.client.0.smithi033.stdout:0/950: chown d7/da/df/f17 2914 1 2024-04-25T01:36:04.307 INFO:tasks.workunit.client.1.smithi138.stdout:9/358: symlink d4/d1a/d55/l77 0 2024-04-25T01:36:04.308 INFO:tasks.workunit.client.1.smithi138.stdout:7/420: dread da/de/d13/f24 [4194304,4194304] 0 2024-04-25T01:36:04.308 INFO:tasks.workunit.client.1.smithi138.stdout:6/465: mknod dc/c93 0 2024-04-25T01:36:04.309 INFO:tasks.workunit.client.0.smithi033.stdout:7/933: stat d4/d23/d3f/d45/f7c 0 2024-04-25T01:36:04.309 INFO:tasks.workunit.client.1.smithi138.stdout:9/359: dread - d4/d49/f76 zero size 2024-04-25T01:36:04.310 INFO:tasks.workunit.client.0.smithi033.stdout:8/948: mknod d6/dd3/dfb/d14b/c15c 0 2024-04-25T01:36:04.310 INFO:tasks.workunit.client.1.smithi138.stdout:6/466: fdatasync dc/d10/f76 0 2024-04-25T01:36:04.310 INFO:tasks.workunit.client.1.smithi138.stdout:1/374: truncate d3/db/f14 2818940 0 2024-04-25T01:36:04.311 INFO:tasks.workunit.client.0.smithi033.stdout:9/915: fsync d0/d2/d43/d4b/dc5/de0/d6e/f95 0 2024-04-25T01:36:04.311 INFO:tasks.workunit.client.0.smithi033.stdout:1/782: mknod d2/d5/d1b/dd2/cf8 0 2024-04-25T01:36:04.311 INFO:tasks.workunit.client.0.smithi033.stdout:4/945: dwrite d6/d14/d35/f126 [0,4194304] 0 2024-04-25T01:36:04.313 INFO:tasks.workunit.client.1.smithi138.stdout:2/435: fdatasync d0/d5a/f1a 0 2024-04-25T01:36:04.313 INFO:tasks.workunit.client.0.smithi033.stdout:0/951: mkdir d7/d2d/d64/d14a 0 2024-04-25T01:36:04.314 INFO:tasks.workunit.client.1.smithi138.stdout:9/360: write d4/d1a/d33/d1d/d3f/f5d [58898,91612] 0 2024-04-25T01:36:04.315 INFO:tasks.workunit.client.1.smithi138.stdout:2/436: truncate d0/d21/f41 757566 0 2024-04-25T01:36:04.315 INFO:tasks.workunit.client.1.smithi138.stdout:4/327: getdents d0/d59 0 2024-04-25T01:36:04.317 INFO:tasks.workunit.client.1.smithi138.stdout:2/437: readlink d0/d5a/lf 0 2024-04-25T01:36:04.317 INFO:tasks.workunit.client.0.smithi033.stdout:9/916: fsync d0/d2/d43/d4b/dc5/f118 0 2024-04-25T01:36:04.318 INFO:tasks.workunit.client.1.smithi138.stdout:7/421: mkdir da/de/d13/d21/d6f/d8d 0 2024-04-25T01:36:04.319 INFO:tasks.workunit.client.0.smithi033.stdout:5/952: write d1c/d2a/f34 [1412919,120319] 0 2024-04-25T01:36:04.320 INFO:tasks.workunit.client.1.smithi138.stdout:6/467: unlink dc/d10/d67/c79 0 2024-04-25T01:36:04.321 INFO:tasks.workunit.client.0.smithi033.stdout:0/952: symlink d7/d2d/d4d/d9c/db2/l14b 0 2024-04-25T01:36:04.322 INFO:tasks.workunit.client.1.smithi138.stdout:7/422: chown da/de/d13/d30/l5f 1 1 2024-04-25T01:36:04.322 INFO:tasks.workunit.client.1.smithi138.stdout:7/423: chown da/de 1386 1 2024-04-25T01:36:04.322 INFO:tasks.workunit.client.0.smithi033.stdout:0/953: chown d7/d2d/d4c/l76 738 1 2024-04-25T01:36:04.323 INFO:tasks.workunit.client.1.smithi138.stdout:3/500: rename db/l37 to db/df/d19/l98 0 2024-04-25T01:36:04.324 INFO:tasks.workunit.client.0.smithi033.stdout:4/946: dwrite d6/d14/d1d/f63 [0,4194304] 0 2024-04-25T01:36:04.324 INFO:tasks.workunit.client.1.smithi138.stdout:4/328: mknod d0/d45/c5f 0 2024-04-25T01:36:04.325 INFO:tasks.workunit.client.0.smithi033.stdout:9/917: fdatasync d0/d2/dd/d20/d4e/f94 0 2024-04-25T01:36:04.325 INFO:tasks.workunit.client.1.smithi138.stdout:2/438: fsync d0/d32/d35/f60 0 2024-04-25T01:36:04.325 INFO:tasks.workunit.client.1.smithi138.stdout:0/301: dread d6/df/f18 [0,4194304] 0 2024-04-25T01:36:04.327 INFO:tasks.workunit.client.1.smithi138.stdout:9/361: unlink d4/le 0 2024-04-25T01:36:04.329 INFO:tasks.workunit.client.1.smithi138.stdout:6/468: creat dc/d7b/f94 x:0 0 0 2024-04-25T01:36:04.329 INFO:tasks.workunit.client.0.smithi033.stdout:2/898: getdents da/d27/d31/d40/d8a/dbb 0 2024-04-25T01:36:04.329 INFO:tasks.workunit.client.1.smithi138.stdout:3/501: truncate db/df/d19/d1a/f49 2871555 0 2024-04-25T01:36:04.330 INFO:tasks.workunit.client.0.smithi033.stdout:1/783: creat d2/d5/d75/d9b/ff9 x:0 0 0 2024-04-25T01:36:04.331 INFO:tasks.workunit.client.1.smithi138.stdout:2/439: read d0/d32/d35/f60 [3728150,2975] 0 2024-04-25T01:36:04.331 INFO:tasks.workunit.client.0.smithi033.stdout:2/899: stat da/d50/d6a/d43/f10e 0 2024-04-25T01:36:04.332 INFO:tasks.workunit.client.1.smithi138.stdout:0/302: unlink d6/df/c4e 0 2024-04-25T01:36:04.333 INFO:tasks.workunit.client.0.smithi033.stdout:2/900: read - da/d5b/d5e/fbd zero size 2024-04-25T01:36:04.333 INFO:tasks.workunit.client.0.smithi033.stdout:4/947: rename d6/d14/df6/d10e/c10f to d6/d14/d35/d3a/d4d/d5e/dc5/c134 0 2024-04-25T01:36:04.333 INFO:tasks.workunit.client.1.smithi138.stdout:3/502: chown db/df/d19/f75 209447 1 2024-04-25T01:36:04.334 INFO:tasks.workunit.client.0.smithi033.stdout:1/784: write d2/d5/d1b/d46/d5f/d63/d8f/f96 [393237,12281] 0 2024-04-25T01:36:04.334 INFO:tasks.workunit.client.1.smithi138.stdout:5/370: dwrite d4/da/d6e/f26 [0,4194304] 0 2024-04-25T01:36:04.337 INFO:tasks.workunit.client.0.smithi033.stdout:0/954: unlink d7/d2d/d43/cd9 0 2024-04-25T01:36:04.338 INFO:tasks.workunit.client.1.smithi138.stdout:8/531: rename d3/d16/f1c to d3/d16/d88/da/db/fa6 0 2024-04-25T01:36:04.338 INFO:tasks.workunit.client.1.smithi138.stdout:2/440: chown d0/d5a/d15/d44/d4c/c5f 6321710 1 2024-04-25T01:36:04.338 INFO:tasks.workunit.client.1.smithi138.stdout:3/503: write db/df/d19/f93 [560926,129402] 0 2024-04-25T01:36:04.339 INFO:tasks.workunit.client.0.smithi033.stdout:5/953: write d1c/d2a/d50/d52/da6/f8f [658501,109520] 0 2024-04-25T01:36:04.344 INFO:tasks.workunit.client.1.smithi138.stdout:4/329: dread d0/d1c/d48/f5b [0,4194304] 0 2024-04-25T01:36:04.346 INFO:tasks.workunit.client.0.smithi033.stdout:4/948: dread d6/d14/d1d/f6f [0,4194304] 0 2024-04-25T01:36:04.347 INFO:tasks.workunit.client.1.smithi138.stdout:6/469: getdents dc/d10/d28/d70 0 2024-04-25T01:36:04.347 INFO:tasks.workunit.client.1.smithi138.stdout:3/504: creat db/df/d17/d23/d34/f99 x:0 0 0 2024-04-25T01:36:04.348 INFO:tasks.workunit.client.0.smithi033.stdout:7/934: dwrite d4/d2e/f116 [0,4194304] 0 2024-04-25T01:36:04.348 INFO:tasks.workunit.client.1.smithi138.stdout:9/362: dwrite d4/d49/f6f [0,4194304] 0 2024-04-25T01:36:04.350 INFO:tasks.workunit.client.0.smithi033.stdout:5/954: creat d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e/f133 x:0 0 0 2024-04-25T01:36:04.350 INFO:tasks.workunit.client.0.smithi033.stdout:1/785: dread d2/d5/d1b/d46/d5f/f61 [0,4194304] 0 2024-04-25T01:36:04.351 INFO:tasks.workunit.client.1.smithi138.stdout:6/470: write dc/d10/f51 [412786,81718] 0 2024-04-25T01:36:04.351 INFO:tasks.workunit.client.0.smithi033.stdout:8/949: dread d6/d45/d52/d2e/f5e [0,4194304] 0 2024-04-25T01:36:04.351 INFO:tasks.workunit.client.0.smithi033.stdout:2/901: dwrite da/d2c/dcd/fc4 [0,4194304] 0 2024-04-25T01:36:04.352 INFO:tasks.workunit.client.0.smithi033.stdout:4/949: chown d6/d14/d35/d44/fe1 1 1 2024-04-25T01:36:04.353 INFO:tasks.workunit.client.1.smithi138.stdout:4/330: symlink d0/l60 0 2024-04-25T01:36:04.354 INFO:tasks.workunit.client.0.smithi033.stdout:1/786: chown d2/d7f/db0/dd6 128 1 2024-04-25T01:36:04.356 INFO:tasks.workunit.client.1.smithi138.stdout:2/441: mknod d0/d32/d35/d4f/c86 0 2024-04-25T01:36:04.357 INFO:tasks.workunit.client.0.smithi033.stdout:7/935: unlink d4/d2e/d6d/fe5 0 2024-04-25T01:36:04.358 INFO:tasks.workunit.client.1.smithi138.stdout:6/471: mknod dc/d10/d28/c95 0 2024-04-25T01:36:04.359 INFO:tasks.workunit.client.0.smithi033.stdout:5/955: mkdir d1c/d2a/d134 0 2024-04-25T01:36:04.360 INFO:tasks.workunit.client.1.smithi138.stdout:4/331: truncate d0/d1c/f2c 314143 0 2024-04-25T01:36:04.361 INFO:tasks.workunit.client.1.smithi138.stdout:6/472: write dc/d2e/d37/d56/f73 [49074,86513] 0 2024-04-25T01:36:04.362 INFO:tasks.workunit.client.1.smithi138.stdout:6/473: chown dc/d2e/f3e 19 1 2024-04-25T01:36:04.362 INFO:tasks.workunit.client.1.smithi138.stdout:6/474: chown dc/d2e 30167909 1 2024-04-25T01:36:04.364 INFO:tasks.workunit.client.1.smithi138.stdout:6/475: chown dc/d7b/f94 51104 1 2024-04-25T01:36:04.364 INFO:tasks.workunit.client.1.smithi138.stdout:3/505: link db/df/d17/d23/f79 db/df/d17/d23/d88/f9a 0 2024-04-25T01:36:04.366 INFO:tasks.workunit.client.0.smithi033.stdout:0/955: dread d7/d2d/d43/d44/fd5 [0,4194304] 0 2024-04-25T01:36:04.367 INFO:tasks.workunit.client.0.smithi033.stdout:2/902: symlink da/d50/d6a/d43/d6c/df5/l123 0 2024-04-25T01:36:04.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/375: rename d3/d45/d72/d3d/f5e to d3/d6d/f7c 0 2024-04-25T01:36:04.370 INFO:tasks.workunit.client.1.smithi138.stdout:9/363: creat d4/d1a/d33/d1d/d3f/f78 x:0 0 0 2024-04-25T01:36:04.371 INFO:tasks.workunit.client.0.smithi033.stdout:7/936: dwrite d4/d2e/d11c/d1f/d78/f12f [0,4194304] 0 2024-04-25T01:36:04.371 INFO:tasks.workunit.client.0.smithi033.stdout:0/956: dread - d7/d2d/d4d/d9c/d68/fb9 zero size 2024-04-25T01:36:04.371 INFO:tasks.workunit.client.1.smithi138.stdout:0/303: dread d6/f1e [0,4194304] 0 2024-04-25T01:36:04.372 INFO:tasks.workunit.client.1.smithi138.stdout:9/364: stat d4/d1a 0 2024-04-25T01:36:04.372 INFO:tasks.workunit.client.1.smithi138.stdout:0/304: fdatasync d6/f3b 0 2024-04-25T01:36:04.372 INFO:tasks.workunit.client.0.smithi033.stdout:8/950: unlink d6/d45/d52/d2e/d87/c93 0 2024-04-25T01:36:04.373 INFO:tasks.workunit.client.1.smithi138.stdout:9/365: dread - d4/d49/f76 zero size 2024-04-25T01:36:04.373 INFO:tasks.workunit.client.1.smithi138.stdout:2/442: link d0/d5a/d15/f67 d0/d5a/f87 0 2024-04-25T01:36:04.375 INFO:tasks.workunit.client.0.smithi033.stdout:1/787: rename d2/d11/l2d to d2/d2e/d34/d60/lfa 0 2024-04-25T01:36:04.377 INFO:tasks.workunit.client.1.smithi138.stdout:7/424: rename da/c48 to da/de/d1b/d49/d81/d73/c8e 0 2024-04-25T01:36:04.378 INFO:tasks.workunit.client.0.smithi033.stdout:7/937: mkdir d4/d2e/d7f/d98/dca/dd3/d131 0 2024-04-25T01:36:04.380 INFO:tasks.workunit.client.0.smithi033.stdout:8/951: fsync d6/d45/d52/f36 0 2024-04-25T01:36:04.380 INFO:tasks.workunit.client.1.smithi138.stdout:7/425: chown da/de/d1b/d49/d81/f4c 55806234 1 2024-04-25T01:36:04.382 INFO:tasks.workunit.client.0.smithi033.stdout:5/956: dread d1c/d2a/d50/d52/fc9 [0,4194304] 0 2024-04-25T01:36:04.383 INFO:tasks.workunit.client.0.smithi033.stdout:4/950: rename d6/d14/d1d/d5f/fed to d6/d14/d35/d44/d45/d46/d57/d70/d128/f135 0 2024-04-25T01:36:04.383 INFO:tasks.workunit.client.1.smithi138.stdout:9/366: symlink d4/d1a/d55/l79 0 2024-04-25T01:36:04.384 INFO:tasks.workunit.client.1.smithi138.stdout:2/443: dwrite d0/d5a/f1a [4194304,4194304] 0 2024-04-25T01:36:04.385 INFO:tasks.workunit.client.0.smithi033.stdout:7/938: unlink d4/d2e/d6d/dbb/fc4 0 2024-04-25T01:36:04.386 INFO:tasks.workunit.client.1.smithi138.stdout:3/506: mkdir db/d9b 0 2024-04-25T01:36:04.388 INFO:tasks.workunit.client.1.smithi138.stdout:3/507: fsync db/df/d17/d5d/f97 0 2024-04-25T01:36:04.390 INFO:tasks.workunit.client.1.smithi138.stdout:3/508: fsync db/df/d19/f93 0 2024-04-25T01:36:04.391 INFO:tasks.workunit.client.0.smithi033.stdout:8/952: mknod d6/d45/d52/df2/c15d 0 2024-04-25T01:36:04.392 INFO:tasks.workunit.client.1.smithi138.stdout:3/509: read db/df/d17/d5d/f78 [256898,94957] 0 2024-04-25T01:36:04.392 INFO:tasks.workunit.client.1.smithi138.stdout:1/376: truncate d3/d27/d3e/f6b 13392 0 2024-04-25T01:36:04.397 INFO:tasks.workunit.client.1.smithi138.stdout:9/367: symlink d4/d1a/d33/l7a 0 2024-04-25T01:36:04.398 INFO:tasks.workunit.client.0.smithi033.stdout:5/957: rename d1c/d2a/d50/d8d/de1 to d1c/d2a/d50/d52/da6/d3e/d70/dae/d135 0 2024-04-25T01:36:04.399 INFO:tasks.workunit.client.1.smithi138.stdout:2/444: symlink d0/d5a/d15/d44/d4c/d56/d6d/d6e/l88 0 2024-04-25T01:36:04.400 INFO:tasks.workunit.client.0.smithi033.stdout:8/953: creat d6/d45/d52/dcc/d112/d11b/f15e x:0 0 0 2024-04-25T01:36:04.401 INFO:tasks.workunit.client.1.smithi138.stdout:7/426: dwrite da/f15 [0,4194304] 0 2024-04-25T01:36:04.402 INFO:tasks.workunit.client.0.smithi033.stdout:2/903: dwrite da/d5b/d5e/f9f [0,4194304] 0 2024-04-25T01:36:04.403 INFO:tasks.workunit.client.1.smithi138.stdout:8/532: dwrite d3/d16/d88/da/db/f14 [0,4194304] 0 2024-04-25T01:36:04.405 INFO:tasks.workunit.client.1.smithi138.stdout:8/533: fsync d3/f6d 0 2024-04-25T01:36:04.405 INFO:tasks.workunit.client.1.smithi138.stdout:7/427: fdatasync da/de/f27 0 2024-04-25T01:36:04.407 INFO:tasks.workunit.client.0.smithi033.stdout:7/939: mknod d4/d23/d3f/d45/d90/ded/c132 0 2024-04-25T01:36:04.407 INFO:tasks.workunit.client.1.smithi138.stdout:3/510: creat db/d14/d40/f9c x:0 0 0 2024-04-25T01:36:04.408 INFO:tasks.workunit.client.1.smithi138.stdout:3/511: chown db/df 4594877 1 2024-04-25T01:36:04.409 INFO:tasks.workunit.client.1.smithi138.stdout:1/377: symlink d3/d27/d3e/d51/d62/l7d 0 2024-04-25T01:36:04.409 INFO:tasks.workunit.client.0.smithi033.stdout:8/954: dwrite d6/f15 [4194304,4194304] 0 2024-04-25T01:36:04.410 INFO:tasks.workunit.client.1.smithi138.stdout:3/512: chown db/df/d17/l56 6046 1 2024-04-25T01:36:04.411 INFO:tasks.workunit.client.0.smithi033.stdout:2/904: mknod da/d5b/d5e/da8/c124 0 2024-04-25T01:36:04.412 INFO:tasks.workunit.client.1.smithi138.stdout:5/371: rename d4/da/d6e/f26 to d4/da/d27/f78 0 2024-04-25T01:36:04.413 INFO:tasks.workunit.client.0.smithi033.stdout:4/951: link d6/d14/lcb d6/d14/d35/d44/d45/d46/d57/l136 0 2024-04-25T01:36:04.414 INFO:tasks.workunit.client.0.smithi033.stdout:7/940: creat d4/d2e/d65/dcb/f133 x:0 0 0 2024-04-25T01:36:04.417 INFO:tasks.workunit.client.1.smithi138.stdout:2/445: creat d0/d5a/d15/d44/d4c/d56/d6d/f89 x:0 0 0 2024-04-25T01:36:04.418 INFO:tasks.workunit.client.0.smithi033.stdout:4/952: creat d6/f137 x:0 0 0 2024-04-25T01:36:04.419 INFO:tasks.workunit.client.1.smithi138.stdout:2/446: chown d0/d5a/ld 4083734 1 2024-04-25T01:36:04.420 INFO:tasks.workunit.client.1.smithi138.stdout:2/447: chown d0/d32/l71 0 1 2024-04-25T01:36:04.421 INFO:tasks.workunit.client.1.smithi138.stdout:9/368: mknod d4/d8/d6b/c7b 0 2024-04-25T01:36:04.422 INFO:tasks.workunit.client.1.smithi138.stdout:0/305: write d6/fc [3705432,69781] 0 2024-04-25T01:36:04.422 INFO:tasks.workunit.client.1.smithi138.stdout:8/534: dwrite d3/d16/d88/da/db/d15/d56/f86 [0,4194304] 0 2024-04-25T01:36:04.423 INFO:tasks.workunit.client.0.smithi033.stdout:8/955: rename d6/db/d80/db4/fd6 to d6/d45/d52/d2e/d95/d12a/f15f 0 2024-04-25T01:36:04.424 INFO:tasks.workunit.client.0.smithi033.stdout:1/788: dwrite d2/d11/d28/f85 [0,4194304] 0 2024-04-25T01:36:04.424 INFO:tasks.workunit.client.1.smithi138.stdout:2/448: chown d0/d21/l30 5491224 1 2024-04-25T01:36:04.425 INFO:tasks.workunit.client.0.smithi033.stdout:7/941: write d4/d23/d3f/d45/d73/d8f/fba [1064416,79046] 0 2024-04-25T01:36:04.427 INFO:tasks.workunit.client.1.smithi138.stdout:4/332: rename d0/d1c/d52/l56 to d0/d16/d4a/l61 0 2024-04-25T01:36:04.428 INFO:tasks.workunit.client.0.smithi033.stdout:5/958: link d1c/d2a/d50/d52/da6/d3e/d70/dae/fef d1c/d2a/f136 0 2024-04-25T01:36:04.430 INFO:tasks.workunit.client.0.smithi033.stdout:2/905: dwrite da/d5b/d72/fbe [0,4194304] 0 2024-04-25T01:36:04.434 INFO:tasks.workunit.client.0.smithi033.stdout:8/956: read d6/dd3/dfb/f107 [127359,27798] 0 2024-04-25T01:36:04.434 INFO:tasks.workunit.client.0.smithi033.stdout:1/789: mkdir d2/d5/dee/dfb 0 2024-04-25T01:36:04.435 INFO:tasks.workunit.client.0.smithi033.stdout:5/959: dread - d1c/d2a/f121 zero size 2024-04-25T01:36:04.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/369: dwrite f1 [0,4194304] 0 2024-04-25T01:36:04.441 INFO:tasks.workunit.client.0.smithi033.stdout:4/953: dwrite d6/d14/d35/d44/fe1 [0,4194304] 0 2024-04-25T01:36:04.443 INFO:tasks.workunit.client.0.smithi033.stdout:1/790: write d2/d5/d75/d9b/ff9 [763414,102308] 0 2024-04-25T01:36:04.443 INFO:tasks.workunit.client.0.smithi033.stdout:7/942: getdents d4/d2e/d6d 0 2024-04-25T01:36:04.443 INFO:tasks.workunit.client.0.smithi033.stdout:2/906: link da/d27/d31/d40/d8a/d111/f113 da/d5b/d72/f125 0 2024-04-25T01:36:04.445 INFO:tasks.workunit.client.0.smithi033.stdout:5/960: rename d1c/d53/c5c to d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e/c137 0 2024-04-25T01:36:04.447 INFO:tasks.workunit.client.1.smithi138.stdout:0/306: dwrite d6/d1d/f24 [0,4194304] 0 2024-04-25T01:36:04.447 INFO:tasks.workunit.client.1.smithi138.stdout:9/370: stat d4/d1a/d3c/c71 0 2024-04-25T01:36:04.449 INFO:tasks.workunit.client.0.smithi033.stdout:7/943: fsync d4/d23/f121 0 2024-04-25T01:36:04.450 INFO:tasks.workunit.client.0.smithi033.stdout:1/791: write d2/de2/ff2 [4500782,71734] 0 2024-04-25T01:36:04.450 INFO:tasks.workunit.client.0.smithi033.stdout:2/907: mknod da/d27/d31/d40/db1/dbf/dd5/de0/c126 0 2024-04-25T01:36:04.452 INFO:tasks.workunit.client.0.smithi033.stdout:2/908: readlink da/d27/d31/d40/d8a/dbb/l8c 0 2024-04-25T01:36:04.453 INFO:tasks.workunit.client.0.smithi033.stdout:8/957: link d6/d45/d52/d2e/fc9 d6/db/d111/f160 0 2024-04-25T01:36:04.453 INFO:tasks.workunit.client.0.smithi033.stdout:1/792: dread - d2/d5/d1b/d46/d5f/d63/fe3 zero size 2024-04-25T01:36:04.454 INFO:tasks.workunit.client.0.smithi033.stdout:5/961: chown cd 1054818291 1 2024-04-25T01:36:04.454 INFO:tasks.workunit.client.0.smithi033.stdout:8/958: stat d6/d49/c75 0 2024-04-25T01:36:04.454 INFO:tasks.workunit.client.0.smithi033.stdout:7/944: creat d4/d2e/d7f/d98/dbf/f134 x:0 0 0 2024-04-25T01:36:04.454 INFO:tasks.workunit.client.1.smithi138.stdout:9/371: truncate d4/d1a/d33/d1d/d3f/d52/f65 700819 0 2024-04-25T01:36:04.457 INFO:tasks.workunit.client.0.smithi033.stdout:2/909: mkdir da/d5b/d127 0 2024-04-25T01:36:04.457 INFO:tasks.workunit.client.0.smithi033.stdout:1/793: chown d2/d2e/d34/d60/c7e 2033 1 2024-04-25T01:36:04.459 INFO:tasks.workunit.client.0.smithi033.stdout:5/962: dread d1c/d2a/d50/d52/da6/d4a/f8c [0,4194304] 0 2024-04-25T01:36:04.460 INFO:tasks.workunit.client.1.smithi138.stdout:6/476: rename la to dc/d10/d28/d70/d83/l96 0 2024-04-25T01:36:04.464 INFO:tasks.workunit.client.1.smithi138.stdout:0/307: mkdir d6/df/d3a/d67 0 2024-04-25T01:36:04.464 INFO:tasks.workunit.client.0.smithi033.stdout:7/945: rename d4/d10/la4 to d4/d2e/d7f/dfd/l135 0 2024-04-25T01:36:04.465 INFO:tasks.workunit.client.0.smithi033.stdout:8/959: truncate d6/d45/d51/f8f 204591 0 2024-04-25T01:36:04.468 INFO:tasks.workunit.client.0.smithi033.stdout:8/960: dread - d6/db/d80/db4/f145 zero size 2024-04-25T01:36:04.468 INFO:tasks.workunit.client.1.smithi138.stdout:8/535: dwrite d3/d16/d88/da/db/d15/d5d/f9a [0,4194304] 0 2024-04-25T01:36:04.473 INFO:tasks.workunit.client.1.smithi138.stdout:7/428: rename da/de/c6d to da/de/d13/d30/c8f 0 2024-04-25T01:36:04.473 INFO:tasks.workunit.client.1.smithi138.stdout:0/308: dread d6/f1e [0,4194304] 0 2024-04-25T01:36:04.476 INFO:tasks.workunit.client.0.smithi033.stdout:8/961: mkdir d6/d45/d51/d55/d84/ddd/d161 0 2024-04-25T01:36:04.478 INFO:tasks.workunit.client.1.smithi138.stdout:6/477: truncate dc/d2e/d55/f5a 3751186 0 2024-04-25T01:36:04.479 INFO:tasks.workunit.client.1.smithi138.stdout:4/333: getdents d0/d16/d33 0 2024-04-25T01:36:04.480 INFO:tasks.workunit.client.1.smithi138.stdout:7/429: symlink da/de/d1b/d34/l90 0 2024-04-25T01:36:04.481 INFO:tasks.workunit.client.1.smithi138.stdout:7/430: write da/f15 [3685644,42642] 0 2024-04-25T01:36:04.482 INFO:tasks.workunit.client.0.smithi033.stdout:4/954: write d6/d14/f101 [566517,66699] 0 2024-04-25T01:36:04.483 INFO:tasks.workunit.client.1.smithi138.stdout:7/431: chown da/de/d13/d21/f41 3100 1 2024-04-25T01:36:04.484 INFO:tasks.workunit.client.1.smithi138.stdout:0/309: getdents d6/df/d66 0 2024-04-25T01:36:04.484 INFO:tasks.workunit.client.1.smithi138.stdout:0/310: read - d6/df/f14 zero size 2024-04-25T01:36:04.485 INFO:tasks.workunit.client.0.smithi033.stdout:4/955: creat d6/d14/d35/d44/d45/d46/d57/f138 x:0 0 0 2024-04-25T01:36:04.486 INFO:tasks.workunit.client.1.smithi138.stdout:0/311: write d6/f5e [625418,16370] 0 2024-04-25T01:36:04.486 INFO:tasks.workunit.client.0.smithi033.stdout:4/956: dread - d6/d14/df6/d10e/f127 zero size 2024-04-25T01:36:04.486 INFO:tasks.workunit.client.1.smithi138.stdout:6/478: fsync dc/d10/f6b 0 2024-04-25T01:36:04.487 INFO:tasks.workunit.client.0.smithi033.stdout:4/957: readlink d6/d14/d6d/le5 0 2024-04-25T01:36:04.489 INFO:tasks.workunit.client.0.smithi033.stdout:4/958: readlink d6/d14/d35/d44/d45/d46/d57/d70/d87/dc7/l8c 0 2024-04-25T01:36:04.493 INFO:tasks.workunit.client.1.smithi138.stdout:0/312: dwrite d6/f26 [4194304,4194304] 0 2024-04-25T01:36:04.495 INFO:tasks.workunit.client.1.smithi138.stdout:3/513: rename db/df/d17/f83 to db/f9d 0 2024-04-25T01:36:04.495 INFO:tasks.workunit.client.0.smithi033.stdout:4/959: dwrite d6/d14/df6/d10e/f113 [0,4194304] 0 2024-04-25T01:36:04.499 INFO:tasks.workunit.client.0.smithi033.stdout:8/962: dread d6/db/fc [0,4194304] 0 2024-04-25T01:36:04.499 INFO:tasks.workunit.client.1.smithi138.stdout:7/432: creat da/de/d1b/d4b/d77/f91 x:0 0 0 2024-04-25T01:36:04.500 INFO:tasks.workunit.client.1.smithi138.stdout:9/372: dread d4/d1a/d33/f37 [0,4194304] 0 2024-04-25T01:36:04.502 INFO:tasks.workunit.client.0.smithi033.stdout:8/963: rmdir d6/d45/d52/d2e/dd4 39 2024-04-25T01:36:04.504 INFO:tasks.workunit.client.1.smithi138.stdout:7/433: truncate da/f7a 730520 0 2024-04-25T01:36:04.505 INFO:tasks.workunit.client.1.smithi138.stdout:1/378: truncate d3/d27/d16/f18 3694043 0 2024-04-25T01:36:04.505 INFO:tasks.workunit.client.1.smithi138.stdout:2/449: dread d0/d5a/d15/d7e/f26 [4194304,4194304] 0 2024-04-25T01:36:04.506 INFO:tasks.workunit.client.0.smithi033.stdout:8/964: creat d6/d45/d51/d55/d84/f162 x:0 0 0 2024-04-25T01:36:04.506 INFO:tasks.workunit.client.1.smithi138.stdout:7/434: dread - da/de/d13/f76 zero size 2024-04-25T01:36:04.506 INFO:tasks.workunit.client.0.smithi033.stdout:5/963: write d1c/d2a/f73 [830144,9813] 0 2024-04-25T01:36:04.507 INFO:tasks.workunit.client.1.smithi138.stdout:7/435: chown da/de/d13/d21/l84 197292101 1 2024-04-25T01:36:04.508 INFO:tasks.workunit.client.1.smithi138.stdout:7/436: stat da/de/d13/d30/c4e 0 2024-04-25T01:36:04.511 INFO:tasks.workunit.client.1.smithi138.stdout:3/514: mkdir db/df/d19/d1a/d9e 0 2024-04-25T01:36:04.514 INFO:tasks.workunit.client.1.smithi138.stdout:6/479: dread f8 [0,4194304] 0 2024-04-25T01:36:04.514 INFO:tasks.workunit.client.1.smithi138.stdout:2/450: mkdir d0/d21/d45/d8a 0 2024-04-25T01:36:04.515 INFO:tasks.workunit.client.1.smithi138.stdout:6/480: dread - dc/d2e/d37/f52 zero size 2024-04-25T01:36:04.515 INFO:tasks.workunit.client.1.smithi138.stdout:1/379: creat d3/d45/f7e x:0 0 0 2024-04-25T01:36:04.516 INFO:tasks.workunit.client.0.smithi033.stdout:7/946: dread d4/d23/da7/dc2/fd6 [0,4194304] 0 2024-04-25T01:36:04.517 INFO:tasks.workunit.client.1.smithi138.stdout:0/313: mknod d6/d39/d52/c68 0 2024-04-25T01:36:04.519 INFO:tasks.workunit.client.1.smithi138.stdout:4/334: write d0/f1a [3553812,78969] 0 2024-04-25T01:36:04.523 INFO:tasks.workunit.client.1.smithi138.stdout:5/372: link d4/da/d6e/d22/f3f d4/da/d6e/d22/d74/f79 0 2024-04-25T01:36:04.523 INFO:tasks.workunit.client.1.smithi138.stdout:2/451: dwrite d0/d21/f41 [0,4194304] 0 2024-04-25T01:36:04.527 INFO:tasks.workunit.client.1.smithi138.stdout:0/314: mkdir d6/df/d69 0 2024-04-25T01:36:04.528 INFO:tasks.workunit.client.1.smithi138.stdout:8/536: rename d3/d16/d1f/l37 to d3/d16/d88/da/db/d15/la7 0 2024-04-25T01:36:04.529 INFO:tasks.workunit.client.1.smithi138.stdout:4/335: stat d0/d1c/c42 0 2024-04-25T01:36:04.530 INFO:tasks.workunit.client.1.smithi138.stdout:5/373: dread d4/da/d6e/d22/d45/f54 [0,4194304] 0 2024-04-25T01:36:04.531 INFO:tasks.workunit.client.0.smithi033.stdout:7/947: dread d4/f11 [4194304,4194304] 0 2024-04-25T01:36:04.532 INFO:tasks.workunit.client.0.smithi033.stdout:7/948: readlink d4/d2e/d11c/d101/l120 0 2024-04-25T01:36:04.535 INFO:tasks.workunit.client.1.smithi138.stdout:5/374: read d4/da/d6e/d22/d35/f56 [153515,83685] 0 2024-04-25T01:36:04.536 INFO:tasks.workunit.client.1.smithi138.stdout:6/481: creat dc/d8d/f97 x:0 0 0 2024-04-25T01:36:04.537 INFO:tasks.workunit.client.1.smithi138.stdout:6/482: fsync dc/d10/f4b 0 2024-04-25T01:36:04.537 INFO:tasks.workunit.client.1.smithi138.stdout:2/452: mkdir d0/d18/d8b 0 2024-04-25T01:36:04.538 INFO:tasks.workunit.client.1.smithi138.stdout:5/375: write d4/da/d6e/d22/d74/f73 [270204,78278] 0 2024-04-25T01:36:04.539 INFO:tasks.workunit.client.1.smithi138.stdout:2/453: write d0/d32/f6b [315761,25374] 0 2024-04-25T01:36:04.540 INFO:tasks.workunit.client.1.smithi138.stdout:8/537: creat d3/d16/d88/da/d70/fa8 x:0 0 0 2024-04-25T01:36:04.541 INFO:tasks.workunit.client.1.smithi138.stdout:7/437: link l0 da/de/d1b/d49/d81/l92 0 2024-04-25T01:36:04.543 INFO:tasks.workunit.client.1.smithi138.stdout:9/373: getdents d4/d1a/d33 0 2024-04-25T01:36:04.544 INFO:tasks.workunit.client.1.smithi138.stdout:9/374: fsync d4/d1a/d33/d1d/d3f/d52/f65 0 2024-04-25T01:36:04.545 INFO:tasks.workunit.client.1.smithi138.stdout:3/515: creat db/df/d17/f9f x:0 0 0 2024-04-25T01:36:04.550 INFO:tasks.workunit.client.1.smithi138.stdout:0/315: creat d6/df/d69/f6a x:0 0 0 2024-04-25T01:36:04.551 INFO:tasks.workunit.client.1.smithi138.stdout:0/316: write d6/d2f/f5a [968614,40146] 0 2024-04-25T01:36:04.556 INFO:tasks.workunit.client.1.smithi138.stdout:8/538: mkdir d3/d44/da9 0 2024-04-25T01:36:04.557 INFO:tasks.workunit.client.0.smithi033.stdout:7/949: dread d4/f17 [0,4194304] 0 2024-04-25T01:36:04.558 INFO:tasks.workunit.client.1.smithi138.stdout:7/438: creat da/de/d13/d30/f93 x:0 0 0 2024-04-25T01:36:04.561 INFO:tasks.workunit.client.1.smithi138.stdout:1/380: link d3/d27/d24/l4d d3/db/l7f 0 2024-04-25T01:36:04.562 INFO:tasks.workunit.client.1.smithi138.stdout:0/317: dwrite d6/f26 [0,4194304] 0 2024-04-25T01:36:04.564 INFO:tasks.workunit.client.1.smithi138.stdout:4/336: creat d0/d1c/f62 x:0 0 0 2024-04-25T01:36:04.565 INFO:tasks.workunit.client.1.smithi138.stdout:6/483: mknod dc/d2e/d37/d56/d8c/c98 0 2024-04-25T01:36:04.566 INFO:tasks.workunit.client.1.smithi138.stdout:5/376: dwrite d4/da/d6e/d22/d74/f58 [4194304,4194304] 0 2024-04-25T01:36:04.569 INFO:tasks.workunit.client.1.smithi138.stdout:9/375: dread d4/d8/f53 [0,4194304] 0 2024-04-25T01:36:04.569 INFO:tasks.workunit.client.1.smithi138.stdout:4/337: dread d0/d45/f35 [0,4194304] 0 2024-04-25T01:36:04.570 INFO:tasks.workunit.client.1.smithi138.stdout:7/439: rmdir da/de/d1b 39 2024-04-25T01:36:04.573 INFO:tasks.workunit.client.1.smithi138.stdout:1/381: chown d3/db/f14 249863 1 2024-04-25T01:36:04.575 INFO:tasks.workunit.client.1.smithi138.stdout:1/382: dread - d3/d27/d3e/d51/d62/d65/f78 zero size 2024-04-25T01:36:04.579 INFO:tasks.workunit.client.1.smithi138.stdout:3/516: creat db/df/d17/d20/fa0 x:0 0 0 2024-04-25T01:36:04.580 INFO:tasks.workunit.client.1.smithi138.stdout:4/338: symlink d0/d1c/d52/l63 0 2024-04-25T01:36:04.581 INFO:tasks.workunit.client.1.smithi138.stdout:2/454: dwrite d0/f10 [0,4194304] 0 2024-04-25T01:36:04.583 INFO:tasks.workunit.client.1.smithi138.stdout:3/517: write db/df/d17/d23/d34/f6d [1061911,121520] 0 2024-04-25T01:36:04.584 INFO:tasks.workunit.client.1.smithi138.stdout:1/383: creat d3/d27/d3e/d51/f80 x:0 0 0 2024-04-25T01:36:04.585 INFO:tasks.workunit.client.1.smithi138.stdout:9/376: truncate d4/f25 894081 0 2024-04-25T01:36:04.586 INFO:tasks.workunit.client.1.smithi138.stdout:7/440: write da/de/d1b/f31 [4524494,63437] 0 2024-04-25T01:36:04.588 INFO:tasks.workunit.client.1.smithi138.stdout:4/339: readlink d0/d1c/l3e 0 2024-04-25T01:36:04.589 INFO:tasks.workunit.client.1.smithi138.stdout:8/539: link d3/d16/d88/da/db/d15/d20/l6c d3/d16/d88/da/db/d15/d56/d8f/laa 0 2024-04-25T01:36:04.590 INFO:tasks.workunit.client.1.smithi138.stdout:9/377: write d4/d1a/d33/d1d/d3f/f5d [586087,29547] 0 2024-04-25T01:36:04.591 INFO:tasks.workunit.client.1.smithi138.stdout:2/455: write d0/f8 [1466240,41010] 0 2024-04-25T01:36:04.591 INFO:tasks.workunit.client.1.smithi138.stdout:0/318: getdents d6/df/d41/d49 0 2024-04-25T01:36:04.594 INFO:tasks.workunit.client.1.smithi138.stdout:8/540: chown d3/l4 1 1 2024-04-25T01:36:04.594 INFO:tasks.workunit.client.0.smithi033.stdout:8/965: unlink d6/d45/l13b 0 2024-04-25T01:36:04.596 INFO:tasks.workunit.client.1.smithi138.stdout:7/441: truncate da/de/d13/f76 949024 0 2024-04-25T01:36:04.596 INFO:tasks.workunit.client.0.smithi033.stdout:1/794: truncate d2/d11/d28/fe1 341949 0 2024-04-25T01:36:04.596 INFO:tasks.workunit.client.1.smithi138.stdout:3/518: fsync db/df/d17/d23/f79 0 2024-04-25T01:36:04.597 INFO:tasks.workunit.client.0.smithi033.stdout:5/964: rename d1c/d2a/d50/db1/lbd to d1c/d2a/d3d/d83/l138 0 2024-04-25T01:36:04.598 INFO:tasks.workunit.client.1.smithi138.stdout:6/484: getdents dc/d10/d28/d80 0 2024-04-25T01:36:04.601 INFO:tasks.workunit.client.1.smithi138.stdout:1/384: rename d3/d27/d2d/f30 to d3/d27/d24/f81 0 2024-04-25T01:36:04.602 INFO:tasks.workunit.client.1.smithi138.stdout:3/519: fsync db/df/d17/d5d/f97 0 2024-04-25T01:36:04.603 INFO:tasks.workunit.client.0.smithi033.stdout:8/966: unlink d6/f34 0 2024-04-25T01:36:04.605 INFO:tasks.workunit.client.0.smithi033.stdout:1/795: rename d2/d2e/d34/d60/f74 to d2/d5/d1b/ffc 0 2024-04-25T01:36:04.605 INFO:tasks.workunit.client.0.smithi033.stdout:4/960: write d6/d12/f1a [991381,119005] 0 2024-04-25T01:36:04.606 INFO:tasks.workunit.client.1.smithi138.stdout:5/377: dread d4/da/d6e/d22/f24 [0,4194304] 0 2024-04-25T01:36:04.607 INFO:tasks.workunit.client.1.smithi138.stdout:3/520: stat db/df/d17/d5d/f76 0 2024-04-25T01:36:04.607 INFO:tasks.workunit.client.1.smithi138.stdout:2/456: dwrite d0/d5a/f17 [0,4194304] 0 2024-04-25T01:36:04.611 INFO:tasks.workunit.client.1.smithi138.stdout:2/457: write d0/d5a/d15/d51/f62 [3741209,69262] 0 2024-04-25T01:36:04.612 INFO:tasks.workunit.client.1.smithi138.stdout:2/458: fdatasync d0/d21/f3a 0 2024-04-25T01:36:04.614 INFO:tasks.workunit.client.1.smithi138.stdout:2/459: chown d0/d5a/d15/d44/d4c/c5f 196 1 2024-04-25T01:36:04.618 INFO:tasks.workunit.client.1.smithi138.stdout:3/521: creat db/df/d17/d23/fa1 x:0 0 0 2024-04-25T01:36:04.619 INFO:tasks.workunit.client.0.smithi033.stdout:5/965: mknod d1c/d2a/d50/d52/da6/d3e/d70/c139 0 2024-04-25T01:36:04.621 INFO:tasks.workunit.client.1.smithi138.stdout:7/442: mkdir da/de/d1b/d49/d8a/d94 0 2024-04-25T01:36:04.621 INFO:tasks.workunit.client.1.smithi138.stdout:7/443: readlink l5 0 2024-04-25T01:36:04.621 INFO:tasks.workunit.client.1.smithi138.stdout:9/378: truncate d4/d1a/f56 4050308 0 2024-04-25T01:36:04.622 INFO:tasks.workunit.client.1.smithi138.stdout:9/379: write d4/d57/f5a [198001,32828] 0 2024-04-25T01:36:04.623 INFO:tasks.workunit.client.1.smithi138.stdout:8/541: creat d3/d16/d88/fab x:0 0 0 2024-04-25T01:36:04.624 INFO:tasks.workunit.client.1.smithi138.stdout:1/385: truncate d3/f2c 1943163 0 2024-04-25T01:36:04.625 INFO:tasks.workunit.client.1.smithi138.stdout:8/542: dread - d3/d16/d1f/d31/d82/f90 zero size 2024-04-25T01:36:04.626 INFO:tasks.workunit.client.1.smithi138.stdout:7/444: read da/de/d1b/d4b/d5b/f43 [856606,4980] 0 2024-04-25T01:36:04.626 INFO:tasks.workunit.client.1.smithi138.stdout:4/340: link d0/d16/f31 d0/f64 0 2024-04-25T01:36:04.628 INFO:tasks.workunit.client.1.smithi138.stdout:4/341: chown d0/d16/d33/f51 3666 1 2024-04-25T01:36:04.628 INFO:tasks.workunit.client.0.smithi033.stdout:4/961: unlink d6/d14/d35/d44/d45/f131 0 2024-04-25T01:36:04.629 INFO:tasks.workunit.client.1.smithi138.stdout:6/485: dread f6 [0,4194304] 0 2024-04-25T01:36:04.629 INFO:tasks.workunit.client.1.smithi138.stdout:5/378: link d4/da/d6e/d22/d74/f76 d4/da/d6e/d22/d35/d4d/d5e/f7a 0 2024-04-25T01:36:04.630 INFO:tasks.workunit.client.1.smithi138.stdout:0/319: getdents d6/d39/d43 0 2024-04-25T01:36:04.632 INFO:tasks.workunit.client.1.smithi138.stdout:5/379: write d4/da/f47 [4782291,83738] 0 2024-04-25T01:36:04.632 INFO:tasks.workunit.client.1.smithi138.stdout:9/380: creat d4/d63/f7c x:0 0 0 2024-04-25T01:36:04.632 INFO:tasks.workunit.client.0.smithi033.stdout:8/967: truncate d6/db/f18 953417 0 2024-04-25T01:36:04.634 INFO:tasks.workunit.client.0.smithi033.stdout:7/950: dwrite d4/d23/d3f/d45/d73/dbe/fe0 [0,4194304] 0 2024-04-25T01:36:04.634 INFO:tasks.workunit.client.1.smithi138.stdout:8/543: mkdir d3/d16/d1f/d2a/d4f/dac 0 2024-04-25T01:36:04.635 INFO:tasks.workunit.client.1.smithi138.stdout:9/381: dread d4/d1a/f3b [0,4194304] 0 2024-04-25T01:36:04.636 INFO:tasks.workunit.client.1.smithi138.stdout:7/445: dwrite da/de/d13/d21/f7e [0,4194304] 0 2024-04-25T01:36:04.639 INFO:tasks.workunit.client.1.smithi138.stdout:4/342: creat d0/d1c/d52/f65 x:0 0 0 2024-04-25T01:36:04.639 INFO:tasks.workunit.client.1.smithi138.stdout:7/446: write da/de/d1b/d49/d81/d51/d6e/f87 [1047021,108263] 0 2024-04-25T01:36:04.650 INFO:tasks.workunit.client.1.smithi138.stdout:0/320: creat d6/d39/d43/f6b x:0 0 0 2024-04-25T01:36:04.652 INFO:tasks.workunit.client.1.smithi138.stdout:1/386: fsync d3/d27/d3e/f6b 0 2024-04-25T01:36:04.654 INFO:tasks.workunit.client.1.smithi138.stdout:5/380: stat d4/da/c44 0 2024-04-25T01:36:04.654 INFO:tasks.workunit.client.1.smithi138.stdout:1/387: dread - d3/db/d4a/f69 zero size 2024-04-25T01:36:04.656 INFO:tasks.workunit.client.1.smithi138.stdout:5/381: chown d4/da/d6e/d22/d35/l46 29381590 1 2024-04-25T01:36:04.657 INFO:tasks.workunit.client.1.smithi138.stdout:5/382: chown d4/da/d6e/d22/d35/c36 738 1 2024-04-25T01:36:04.659 INFO:tasks.workunit.client.1.smithi138.stdout:0/321: dread d6/d1d/f5f [0,4194304] 0 2024-04-25T01:36:04.666 INFO:tasks.workunit.client.1.smithi138.stdout:1/388: write d3/d27/d24/f81 [1887022,74531] 0 2024-04-25T01:36:04.666 INFO:tasks.workunit.client.1.smithi138.stdout:6/486: write dc/d10/f68 [4577720,124379] 0 2024-04-25T01:36:04.668 INFO:tasks.workunit.client.1.smithi138.stdout:3/522: getdents db/d14/d7a 0 2024-04-25T01:36:04.670 INFO:tasks.workunit.client.1.smithi138.stdout:7/447: mkdir da/de/d13/d95 0 2024-04-25T01:36:04.674 INFO:tasks.workunit.client.1.smithi138.stdout:4/343: dwrite d0/d16/f29 [4194304,4194304] 0 2024-04-25T01:36:04.675 INFO:tasks.workunit.client.1.smithi138.stdout:0/322: symlink d6/d2f/d31/l6c 0 2024-04-25T01:36:04.677 INFO:tasks.workunit.client.1.smithi138.stdout:4/344: dread - d0/f40 zero size 2024-04-25T01:36:04.677 INFO:tasks.workunit.client.1.smithi138.stdout:2/460: dwrite d0/f31 [0,4194304] 0 2024-04-25T01:36:04.678 INFO:tasks.workunit.client.1.smithi138.stdout:4/345: write d0/f3f [4422429,10678] 0 2024-04-25T01:36:04.678 INFO:tasks.workunit.client.1.smithi138.stdout:1/389: creat d3/d45/d72/d3d/f82 x:0 0 0 2024-04-25T01:36:04.678 INFO:tasks.workunit.client.1.smithi138.stdout:0/323: truncate d6/df/d69/f6a 774938 0 2024-04-25T01:36:04.679 INFO:tasks.workunit.client.1.smithi138.stdout:0/324: chown d6/df/d3a/d67 14495 1 2024-04-25T01:36:04.679 INFO:tasks.workunit.client.1.smithi138.stdout:3/523: readlink db/df/d19/l4e 0 2024-04-25T01:36:04.680 INFO:tasks.workunit.client.1.smithi138.stdout:6/487: truncate dc/d10/d28/f48 603228 0 2024-04-25T01:36:04.683 INFO:tasks.workunit.client.1.smithi138.stdout:5/383: write d4/da/d27/f33 [176250,120923] 0 2024-04-25T01:36:04.685 INFO:tasks.workunit.client.0.smithi033.stdout:4/962: creat d6/d14/d35/d44/d45/d46/db0/f139 x:0 0 0 2024-04-25T01:36:04.687 INFO:tasks.workunit.client.0.smithi033.stdout:4/963: read d6/d12/d84/fbb [43402,63981] 0 2024-04-25T01:36:04.688 INFO:tasks.workunit.client.0.smithi033.stdout:8/968: mknod d6/dd3/c163 0 2024-04-25T01:36:04.689 INFO:tasks.workunit.client.0.smithi033.stdout:8/969: fdatasync d6/db/d80/d89/fab 0 2024-04-25T01:36:04.690 INFO:tasks.workunit.client.0.smithi033.stdout:8/970: dread - d6/d45/d51/d55/d84/f162 zero size 2024-04-25T01:36:04.691 INFO:tasks.workunit.client.0.smithi033.stdout:7/951: creat d4/d2e/d11c/d1f/d78/f136 x:0 0 0 2024-04-25T01:36:04.691 INFO:tasks.workunit.client.0.smithi033.stdout:8/971: chown d6/d45/c4c 0 1 2024-04-25T01:36:04.692 INFO:tasks.workunit.client.0.smithi033.stdout:7/952: read d4/d23/d3f/d45/f7c [246481,39235] 0 2024-04-25T01:36:04.694 INFO:tasks.workunit.client.0.smithi033.stdout:7/953: chown d4/d2e/d11c/da9/db3/fef 365 1 2024-04-25T01:36:04.697 INFO:tasks.workunit.client.0.smithi033.stdout:4/964: creat d6/d14/d35/d3a/d4d/d5e/ddc/f13a x:0 0 0 2024-04-25T01:36:04.700 INFO:tasks.workunit.client.0.smithi033.stdout:8/972: rmdir d6/d45/d52/df2/dfa 39 2024-04-25T01:36:04.702 INFO:tasks.workunit.client.0.smithi033.stdout:7/954: mknod d4/d23/da7/dc2/c137 0 2024-04-25T01:36:04.704 INFO:tasks.workunit.client.0.smithi033.stdout:4/965: rmdir d6/d14/d35/d44/d45/d46/d57/d70/d87/db8/d121 39 2024-04-25T01:36:04.708 INFO:tasks.workunit.client.0.smithi033.stdout:7/955: dwrite d4/d2e/d7f/fb2 [0,4194304] 0 2024-04-25T01:36:04.710 INFO:tasks.workunit.client.0.smithi033.stdout:4/966: mknod d6/d14/d35/d44/d45/d46/d57/d70/d87/db8/c13b 0 2024-04-25T01:36:04.710 INFO:tasks.workunit.client.0.smithi033.stdout:3/933: sync 2024-04-25T01:36:04.710 INFO:tasks.workunit.client.0.smithi033.stdout:9/918: sync 2024-04-25T01:36:04.711 INFO:tasks.workunit.client.0.smithi033.stdout:9/919: readlink d0/d2/dd/d20/l83 0 2024-04-25T01:36:04.718 INFO:tasks.workunit.client.0.smithi033.stdout:9/920: creat d0/d2/d43/d4b/dc5/de0/d7e/d58/f13e x:0 0 0 2024-04-25T01:36:04.720 INFO:tasks.workunit.client.0.smithi033.stdout:7/956: link d4/lc d4/d23/d3f/d45/d73/d8f/l138 0 2024-04-25T01:36:04.726 INFO:tasks.workunit.client.0.smithi033.stdout:9/921: rename d0/d2/dd/fde to d0/d2/d43/d73/dab/dad/d133/f13f 0 2024-04-25T01:36:04.728 INFO:tasks.workunit.client.0.smithi033.stdout:9/922: fdatasync d0/d2/d43/d4b/dc5/de0/d7e/d58/f115 0 2024-04-25T01:36:04.728 INFO:tasks.workunit.client.0.smithi033.stdout:9/923: dread - d0/d2/d43/d4b/dc5/f11a zero size 2024-04-25T01:36:04.739 INFO:tasks.workunit.client.0.smithi033.stdout:1/796: dread d2/d11/d28/fe1 [0,4194304] 0 2024-04-25T01:36:04.791 INFO:tasks.workunit.client.0.smithi033.stdout:2/910: sync 2024-04-25T01:36:04.791 INFO:tasks.workunit.client.0.smithi033.stdout:0/957: sync 2024-04-25T01:36:04.792 INFO:tasks.workunit.client.0.smithi033.stdout:0/958: rmdir d7/d2d/d43/d44/dd4 39 2024-04-25T01:36:04.793 INFO:tasks.workunit.client.0.smithi033.stdout:0/959: getdents d7/da/d80/d127 0 2024-04-25T01:36:04.797 INFO:tasks.workunit.client.0.smithi033.stdout:0/960: dwrite d7/da/d80/fe4 [0,4194304] 0 2024-04-25T01:36:04.800 INFO:tasks.workunit.client.0.smithi033.stdout:0/961: mknod d7/d2d/d4d/d9c/d61/d70/d13a/c14c 0 2024-04-25T01:36:04.802 INFO:tasks.workunit.client.0.smithi033.stdout:0/962: creat d7/da/f14d x:0 0 0 2024-04-25T01:36:04.862 INFO:tasks.workunit.client.1.smithi138.stdout:6/488: write dc/d8d/f97 [408359,26811] 0 2024-04-25T01:36:04.863 INFO:tasks.workunit.client.1.smithi138.stdout:2/461: fsync d0/d5a/d15/f67 0 2024-04-25T01:36:04.869 INFO:tasks.workunit.client.1.smithi138.stdout:9/382: dwrite d4/f38 [0,4194304] 0 2024-04-25T01:36:04.870 INFO:tasks.workunit.client.1.smithi138.stdout:1/390: dread d3/db/d4a/f6e [0,4194304] 0 2024-04-25T01:36:04.871 INFO:tasks.workunit.client.0.smithi033.stdout:5/966: write d1c/d2a/d50/d52/f9f [607599,29238] 0 2024-04-25T01:36:04.872 INFO:tasks.workunit.client.1.smithi138.stdout:7/448: mknod da/de/d13/d21/d6f/d8d/c96 0 2024-04-25T01:36:04.874 INFO:tasks.workunit.client.1.smithi138.stdout:9/383: truncate d4/f6c 404074 0 2024-04-25T01:36:04.874 INFO:tasks.workunit.client.1.smithi138.stdout:0/325: dwrite d6/fc [0,4194304] 0 2024-04-25T01:36:04.874 INFO:tasks.workunit.client.1.smithi138.stdout:1/391: chown d3/d4/d47/c60 157 1 2024-04-25T01:36:04.875 INFO:tasks.workunit.client.0.smithi033.stdout:5/967: creat d1c/d2a/d3d/d59/f13a x:0 0 0 2024-04-25T01:36:04.876 INFO:tasks.workunit.client.1.smithi138.stdout:7/449: readlink da/de/d1b/d49/d81/d51/l7c 0 2024-04-25T01:36:04.877 INFO:tasks.workunit.client.1.smithi138.stdout:0/326: chown d6/df/d69/f6a 6218794 1 2024-04-25T01:36:04.878 INFO:tasks.workunit.client.0.smithi033.stdout:8/973: write d6/d45/d51/f9a [66489,39225] 0 2024-04-25T01:36:04.879 INFO:tasks.workunit.client.1.smithi138.stdout:0/327: stat d6/df/d3a/d67 0 2024-04-25T01:36:04.881 INFO:tasks.workunit.client.0.smithi033.stdout:5/968: read d1c/d2a/d50/d52/da6/d3e/d70/fc2 [189168,94388] 0 2024-04-25T01:36:04.881 INFO:tasks.workunit.client.0.smithi033.stdout:8/974: write d6/db/d80/f149 [885211,48695] 0 2024-04-25T01:36:04.882 INFO:tasks.workunit.client.0.smithi033.stdout:3/934: write d8/dd/d3d/d4f/da4/dc4/ffb [1018202,77757] 0 2024-04-25T01:36:04.883 INFO:tasks.workunit.client.1.smithi138.stdout:0/328: dread d6/d19/f47 [4194304,4194304] 0 2024-04-25T01:36:04.883 INFO:tasks.workunit.client.0.smithi033.stdout:4/967: truncate d6/d14/d35/d44/f119 327751 0 2024-04-25T01:36:04.884 INFO:tasks.workunit.client.1.smithi138.stdout:5/384: creat d4/da/d6e/d22/f7b x:0 0 0 2024-04-25T01:36:04.885 INFO:tasks.workunit.client.0.smithi033.stdout:4/968: chown d6/d14/d1d/d3b/l3f 4 1 2024-04-25T01:36:04.888 INFO:tasks.workunit.client.0.smithi033.stdout:3/935: stat d8/dd/fcf 0 2024-04-25T01:36:04.888 INFO:tasks.workunit.client.1.smithi138.stdout:6/489: rmdir dc/d2e/d37/d56 39 2024-04-25T01:36:04.889 INFO:tasks.workunit.client.1.smithi138.stdout:5/385: read d4/da/fe [2852831,104855] 0 2024-04-25T01:36:04.890 INFO:tasks.workunit.client.1.smithi138.stdout:1/392: dwrite d3/d45/d72/d57/f63 [0,4194304] 0 2024-04-25T01:36:04.892 INFO:tasks.workunit.client.0.smithi033.stdout:0/963: write d7/d2d/d4d/d9c/d61/fdb [1727654,38244] 0 2024-04-25T01:36:04.892 INFO:tasks.workunit.client.0.smithi033.stdout:7/957: fdatasync d4/d2e/d7f/fb2 0 2024-04-25T01:36:04.893 INFO:tasks.workunit.client.1.smithi138.stdout:5/386: readlink d4/da/d6e/d22/l31 0 2024-04-25T01:36:04.894 INFO:tasks.workunit.client.1.smithi138.stdout:2/462: rename d0/d5a/f17 to d0/d5a/d15/d44/d4c/d56/d6d/f8c 0 2024-04-25T01:36:04.895 INFO:tasks.workunit.client.1.smithi138.stdout:6/490: write dc/d10/d67/f65 [287202,8955] 0 2024-04-25T01:36:04.896 INFO:tasks.workunit.client.0.smithi033.stdout:0/964: truncate d7/d2d/fc4 1580316 0 2024-04-25T01:36:04.896 INFO:tasks.workunit.client.1.smithi138.stdout:5/387: stat d4/da/d6e/d22/d35/d4d/l63 0 2024-04-25T01:36:04.896 INFO:tasks.workunit.client.0.smithi033.stdout:5/969: rename d1c/d2a/d50/d5b/d86 to d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/d13b 0 2024-04-25T01:36:04.898 INFO:tasks.workunit.client.0.smithi033.stdout:0/965: chown d7/d33/f36 13389610 1 2024-04-25T01:36:04.898 INFO:tasks.workunit.client.0.smithi033.stdout:4/969: dwrite d6/d12/d38/f5d [4194304,4194304] 0 2024-04-25T01:36:04.901 INFO:tasks.workunit.client.1.smithi138.stdout:7/450: stat da/de/d1b/d49/d81/l92 0 2024-04-25T01:36:04.901 INFO:tasks.workunit.client.0.smithi033.stdout:5/970: readlink d1c/l31 0 2024-04-25T01:36:04.902 INFO:tasks.workunit.client.1.smithi138.stdout:0/329: dwrite d6/f56 [0,4194304] 0 2024-04-25T01:36:04.903 INFO:tasks.workunit.client.0.smithi033.stdout:9/924: dread d0/d2/dd/df/d1b/f2e [0,4194304] 0 2024-04-25T01:36:04.904 INFO:tasks.workunit.client.0.smithi033.stdout:4/970: creat d6/d14/d35/d3a/d4d/d5e/dc5/f13c x:0 0 0 2024-04-25T01:36:04.905 INFO:tasks.workunit.client.1.smithi138.stdout:3/524: creat db/fa2 x:0 0 0 2024-04-25T01:36:04.906 INFO:tasks.workunit.client.0.smithi033.stdout:3/936: getdents d8/dd/d11/d102 0 2024-04-25T01:36:04.907 INFO:tasks.workunit.client.0.smithi033.stdout:5/971: write d1c/f32 [4485199,24533] 0 2024-04-25T01:36:04.908 INFO:tasks.workunit.client.1.smithi138.stdout:7/451: chown da/de/d13/d30/d59/f78 1002025 1 2024-04-25T01:36:04.909 INFO:tasks.workunit.client.0.smithi033.stdout:4/971: fsync d6/d12/d84/fae 0 2024-04-25T01:36:04.909 INFO:tasks.workunit.client.0.smithi033.stdout:9/925: read d0/d2/dd/d20/d4e/f90 [607995,59501] 0 2024-04-25T01:36:04.909 INFO:tasks.workunit.client.1.smithi138.stdout:4/346: dwrite d0/d1c/f2b [0,4194304] 0 2024-04-25T01:36:04.912 INFO:tasks.workunit.client.0.smithi033.stdout:4/972: truncate d6/d14/f22 620218 0 2024-04-25T01:36:04.912 INFO:tasks.workunit.client.0.smithi033.stdout:5/972: fdatasync d1c/d2a/d50/d52/da6/d3e/d70/fb9 0 2024-04-25T01:36:04.913 INFO:tasks.workunit.client.1.smithi138.stdout:1/393: mknod d3/d6d/c83 0 2024-04-25T01:36:04.913 INFO:tasks.workunit.client.1.smithi138.stdout:2/463: dwrite d0/d32/d35/f70 [0,4194304] 0 2024-04-25T01:36:04.914 INFO:tasks.workunit.client.1.smithi138.stdout:3/525: write db/d14/d40/f87 [459194,67487] 0 2024-04-25T01:36:04.914 INFO:tasks.workunit.client.0.smithi033.stdout:3/937: dread - d8/dd/d1d/dd1/f109 zero size 2024-04-25T01:36:04.915 INFO:tasks.workunit.client.1.smithi138.stdout:5/388: truncate d4/da/d6e/d22/d74/d59/f68 947121 0 2024-04-25T01:36:04.916 INFO:tasks.workunit.client.1.smithi138.stdout:2/464: chown d0/d5a/d15/d7e 3825130 1 2024-04-25T01:36:04.918 INFO:tasks.workunit.client.1.smithi138.stdout:9/384: link d4/d1a/d33/f36 d4/d1a/d33/d1d/d3f/d52/f7d 0 2024-04-25T01:36:04.918 INFO:tasks.workunit.client.0.smithi033.stdout:9/926: symlink d0/d2/dd/d5f/d11b/d13c/l140 0 2024-04-25T01:36:04.919 INFO:tasks.workunit.client.1.smithi138.stdout:1/394: chown d3/d4/f77 4373476 1 2024-04-25T01:36:04.920 INFO:tasks.workunit.client.0.smithi033.stdout:8/975: dread d6/d45/f57 [0,4194304] 0 2024-04-25T01:36:04.920 INFO:tasks.workunit.client.1.smithi138.stdout:9/385: chown d4/d1a/d33/c48 58 1 2024-04-25T01:36:04.920 INFO:tasks.workunit.client.1.smithi138.stdout:0/330: mknod d6/df/d41/c6d 0 2024-04-25T01:36:04.921 INFO:tasks.workunit.client.0.smithi033.stdout:9/927: stat d0/d2/dd/f137 0 2024-04-25T01:36:04.922 INFO:tasks.workunit.client.1.smithi138.stdout:2/465: write d0/d5a/d15/d44/d4c/d56/d6d/f89 [434894,103101] 0 2024-04-25T01:36:04.925 INFO:tasks.workunit.client.0.smithi033.stdout:3/938: stat d8/dd/d3d/d63/d7c/ded/ce2 0 2024-04-25T01:36:04.925 INFO:tasks.workunit.client.0.smithi033.stdout:5/973: dwrite d1c/d2a/d50/d52/f9f [0,4194304] 0 2024-04-25T01:36:04.926 INFO:tasks.workunit.client.1.smithi138.stdout:2/466: read d0/d32/f4b [3325377,73316] 0 2024-04-25T01:36:04.927 INFO:tasks.workunit.client.0.smithi033.stdout:4/973: dread d6/d12/feb [4194304,4194304] 0 2024-04-25T01:36:04.932 INFO:tasks.workunit.client.0.smithi033.stdout:5/974: mkdir d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/d13b/d13c 0 2024-04-25T01:36:04.933 INFO:tasks.workunit.client.0.smithi033.stdout:3/939: read d8/dd/d3d/d63/d7c/f96 [1107315,115318] 0 2024-04-25T01:36:04.933 INFO:tasks.workunit.client.1.smithi138.stdout:9/386: chown d4/d1a/d3c/l3d 907 1 2024-04-25T01:36:04.934 INFO:tasks.workunit.client.0.smithi033.stdout:4/974: rename d6/d14/d35/d3a/de8/ff3 to d6/d14/d35/d44/dfe/d103/f13d 0 2024-04-25T01:36:04.934 INFO:tasks.workunit.client.0.smithi033.stdout:5/975: dread - d1c/d2a/d50/d52/da6/d3e/f127 zero size 2024-04-25T01:36:04.935 INFO:tasks.workunit.client.0.smithi033.stdout:3/940: chown d8/c2a 47 1 2024-04-25T01:36:04.935 INFO:tasks.workunit.client.1.smithi138.stdout:3/526: dwrite db/df/d17/d5d/f7f [0,4194304] 0 2024-04-25T01:36:04.936 INFO:tasks.workunit.client.0.smithi033.stdout:4/975: dread - d6/ffa zero size 2024-04-25T01:36:04.936 INFO:tasks.workunit.client.1.smithi138.stdout:6/491: creat dc/d2e/d37/d56/d8c/f99 x:0 0 0 2024-04-25T01:36:04.938 INFO:tasks.workunit.client.0.smithi033.stdout:5/976: mkdir d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/d13d 0 2024-04-25T01:36:04.939 INFO:tasks.workunit.client.0.smithi033.stdout:4/976: symlink d6/d12/d38/l13e 0 2024-04-25T01:36:04.941 INFO:tasks.workunit.client.0.smithi033.stdout:5/977: truncate d1c/d53/fad 441532 0 2024-04-25T01:36:04.941 INFO:tasks.workunit.client.1.smithi138.stdout:0/331: dread d6/df/d41/d49/f51 [0,4194304] 0 2024-04-25T01:36:04.941 INFO:tasks.workunit.client.1.smithi138.stdout:7/452: mknod da/de/d1b/d49/d81/d73/c97 0 2024-04-25T01:36:04.943 INFO:tasks.workunit.client.0.smithi033.stdout:4/977: symlink d6/d14/d1d/l13f 0 2024-04-25T01:36:04.945 INFO:tasks.workunit.client.0.smithi033.stdout:5/978: rename d1c/d2a/fff to d1c/d2a/d50/d52/d104/f13e 0 2024-04-25T01:36:04.945 INFO:tasks.workunit.client.0.smithi033.stdout:3/941: link d8/f59 d8/f148 0 2024-04-25T01:36:04.947 INFO:tasks.workunit.client.0.smithi033.stdout:4/978: truncate d6/d14/d1d/d3b/f50 5387769 0 2024-04-25T01:36:04.947 INFO:tasks.workunit.client.1.smithi138.stdout:3/527: creat db/df/d17/d5d/fa3 x:0 0 0 2024-04-25T01:36:04.948 INFO:tasks.workunit.client.1.smithi138.stdout:9/387: mknod d4/d49/c7e 0 2024-04-25T01:36:04.951 INFO:tasks.workunit.client.1.smithi138.stdout:1/395: dwrite d3/d27/d3e/d51/d62/f68 [0,4194304] 0 2024-04-25T01:36:04.951 INFO:tasks.workunit.client.0.smithi033.stdout:0/966: write d7/da/df/f99 [3329712,55460] 0 2024-04-25T01:36:04.953 INFO:tasks.workunit.client.1.smithi138.stdout:6/492: dread - dc/d10/d6d/f87 zero size 2024-04-25T01:36:04.953 INFO:tasks.workunit.client.1.smithi138.stdout:7/453: write da/de/d13/d30/f80 [491007,21749] 0 2024-04-25T01:36:04.954 INFO:tasks.workunit.client.1.smithi138.stdout:3/528: dread - db/df/d17/d5d/f97 zero size 2024-04-25T01:36:04.956 INFO:tasks.workunit.client.0.smithi033.stdout:3/942: rename d8/dd/d11/c66 to d8/dd/d3d/dce/c149 0 2024-04-25T01:36:04.957 INFO:tasks.workunit.client.0.smithi033.stdout:7/958: dread d4/d57/fae [0,4194304] 0 2024-04-25T01:36:04.958 INFO:tasks.workunit.client.0.smithi033.stdout:3/943: read - d8/dd/d3d/d63/f95 zero size 2024-04-25T01:36:04.959 INFO:tasks.workunit.client.1.smithi138.stdout:0/332: read d6/df/f10 [3915854,123713] 0 2024-04-25T01:36:04.960 INFO:tasks.workunit.client.0.smithi033.stdout:5/979: dread d1c/d2a/f73 [0,4194304] 0 2024-04-25T01:36:04.961 INFO:tasks.workunit.client.1.smithi138.stdout:3/529: creat db/df/d17/d23/d88/fa4 x:0 0 0 2024-04-25T01:36:04.961 INFO:tasks.workunit.client.1.smithi138.stdout:2/467: dwrite d0/d5a/d15/f3b [0,4194304] 0 2024-04-25T01:36:04.961 INFO:tasks.workunit.client.1.smithi138.stdout:0/333: chown d6/d39/d58 221 1 2024-04-25T01:36:04.965 INFO:tasks.workunit.client.0.smithi033.stdout:4/979: dread d6/d14/d35/d3a/d4d/d5e/fdd [0,4194304] 0 2024-04-25T01:36:04.966 INFO:tasks.workunit.client.0.smithi033.stdout:0/967: creat d7/d2d/d43/dc0/d143/f14e x:0 0 0 2024-04-25T01:36:04.966 INFO:tasks.workunit.client.1.smithi138.stdout:2/468: fdatasync d0/d21/f59 0 2024-04-25T01:36:04.966 INFO:tasks.workunit.client.1.smithi138.stdout:7/454: rename da/de/d13/f52 to da/de/f98 0 2024-04-25T01:36:04.971 INFO:tasks.workunit.client.0.smithi033.stdout:5/980: unlink d1c/d2a/d50/d52/da6/d3e/d70/fb9 0 2024-04-25T01:36:04.974 INFO:tasks.workunit.client.0.smithi033.stdout:9/928: write d0/d2/ffa [749981,5806] 0 2024-04-25T01:36:04.975 INFO:tasks.workunit.client.0.smithi033.stdout:8/976: write d6/d45/d51/f4f [8475268,22255] 0 2024-04-25T01:36:04.976 INFO:tasks.workunit.client.0.smithi033.stdout:9/929: stat d0/d11/da1 0 2024-04-25T01:36:04.977 INFO:tasks.workunit.client.0.smithi033.stdout:0/968: rename d7/d2d/d64/f10f to d7/d2d/d43/dc0/d143/da1/f14f 0 2024-04-25T01:36:04.978 INFO:tasks.workunit.client.1.smithi138.stdout:2/469: write d0/f9 [3260105,59020] 0 2024-04-25T01:36:04.978 INFO:tasks.workunit.client.1.smithi138.stdout:9/388: dwrite d4/d1a/d33/d1d/d3f/d52/f65 [0,4194304] 0 2024-04-25T01:36:04.980 INFO:tasks.workunit.client.1.smithi138.stdout:1/396: dwrite d3/d45/d72/f52 [0,4194304] 0 2024-04-25T01:36:04.980 INFO:tasks.workunit.client.0.smithi033.stdout:9/930: dread - d0/d2/dd/d20/f106 zero size 2024-04-25T01:36:04.981 INFO:tasks.workunit.client.1.smithi138.stdout:0/334: creat d6/d19/d57/f6e x:0 0 0 2024-04-25T01:36:04.982 INFO:tasks.workunit.client.0.smithi033.stdout:2/911: sync 2024-04-25T01:36:04.982 INFO:tasks.workunit.client.0.smithi033.stdout:1/797: sync 2024-04-25T01:36:04.984 INFO:tasks.workunit.client.1.smithi138.stdout:6/493: dwrite dc/d10/d28/d80/f82 [0,4194304] 0 2024-04-25T01:36:04.986 INFO:tasks.workunit.client.0.smithi033.stdout:5/981: getdents d1c/d2a/d50/d5b 0 2024-04-25T01:36:04.986 INFO:tasks.workunit.client.1.smithi138.stdout:7/455: mkdir da/de/d13/d30/d59/d7b/d99 0 2024-04-25T01:36:04.988 INFO:tasks.workunit.client.0.smithi033.stdout:5/982: write d1c/d2a/f10f [2167136,21691] 0 2024-04-25T01:36:04.988 INFO:tasks.workunit.client.0.smithi033.stdout:9/931: mkdir d0/d2/d43/d4b/dc5/de0/d42/d141 0 2024-04-25T01:36:04.989 INFO:tasks.workunit.client.0.smithi033.stdout:5/983: stat d1c/d2a/d50/d52/da6/d3e/d70/d9b 0 2024-04-25T01:36:04.990 INFO:tasks.workunit.client.1.smithi138.stdout:0/335: truncate d6/df/f18 5765221 0 2024-04-25T01:36:04.990 INFO:tasks.workunit.client.1.smithi138.stdout:2/470: read d0/d21/f5b [5976,117688] 0 2024-04-25T01:36:04.991 INFO:tasks.workunit.client.1.smithi138.stdout:9/389: rmdir d4/d49 39 2024-04-25T01:36:04.991 INFO:tasks.workunit.client.1.smithi138.stdout:1/397: truncate d3/f13 1316307 0 2024-04-25T01:36:04.992 INFO:tasks.workunit.client.1.smithi138.stdout:7/456: fsync da/de/f27 0 2024-04-25T01:36:04.994 INFO:tasks.workunit.client.1.smithi138.stdout:4/347: dwrite d0/f64 [0,4194304] 0 2024-04-25T01:36:04.995 INFO:tasks.workunit.client.0.smithi033.stdout:2/912: rename da/d50/f115 to da/d27/d31/d40/d8a/dbb/f128 0 2024-04-25T01:36:04.997 INFO:tasks.workunit.client.0.smithi033.stdout:1/798: creat d2/de6/ffd x:0 0 0 2024-04-25T01:36:04.998 INFO:tasks.workunit.client.1.smithi138.stdout:2/471: dread - d0/d32/f81 zero size 2024-04-25T01:36:04.998 INFO:tasks.workunit.client.1.smithi138.stdout:0/336: write d6/f56 [1920431,39477] 0 2024-04-25T01:36:04.999 INFO:tasks.workunit.client.0.smithi033.stdout:5/984: symlink d1c/d2a/d50/d52/da6/d3e/d70/dae/d135/l13f 0 2024-04-25T01:36:05.000 INFO:tasks.workunit.client.0.smithi033.stdout:5/985: chown d1c/d2a/d50/d5b 57795510 1 2024-04-25T01:36:05.001 INFO:tasks.workunit.client.0.smithi033.stdout:9/932: symlink d0/d2/d43/d4b/dc5/de0/d7e/ddc/l142 0 2024-04-25T01:36:05.001 INFO:tasks.workunit.client.1.smithi138.stdout:1/398: fdatasync d3/d4/f77 0 2024-04-25T01:36:05.001 INFO:tasks.workunit.client.1.smithi138.stdout:9/390: mkdir d4/d1a/d55/d7f 0 2024-04-25T01:36:05.002 INFO:tasks.workunit.client.0.smithi033.stdout:9/933: chown d0/d2/d43/d4b/dc5/de0 0 1 2024-04-25T01:36:05.003 INFO:tasks.workunit.client.1.smithi138.stdout:3/530: dread db/df/d17/d20/f29 [0,4194304] 0 2024-04-25T01:36:05.003 INFO:tasks.workunit.client.0.smithi033.stdout:2/913: fsync da/d27/d31/d40/db1/fcc 0 2024-04-25T01:36:05.003 INFO:tasks.workunit.client.1.smithi138.stdout:2/472: chown d0/d21 0 1 2024-04-25T01:36:05.005 INFO:tasks.workunit.client.0.smithi033.stdout:9/934: write d0/d2/d43/d4b/dc5/de0/d7e/d58/f13e [1028733,75750] 0 2024-04-25T01:36:05.006 INFO:tasks.workunit.client.1.smithi138.stdout:4/348: creat d0/d1c/d52/f66 x:0 0 0 2024-04-25T01:36:05.007 INFO:tasks.workunit.client.0.smithi033.stdout:9/935: write d0/d2/d43/d73/fd1 [900932,76720] 0 2024-04-25T01:36:05.008 INFO:tasks.workunit.client.1.smithi138.stdout:6/494: dwrite dc/f72 [0,4194304] 0 2024-04-25T01:36:05.009 INFO:tasks.workunit.client.1.smithi138.stdout:0/337: fdatasync d6/df/f4d 0 2024-04-25T01:36:05.009 INFO:tasks.workunit.client.1.smithi138.stdout:9/391: chown d4/d1a/d33/d1d/d3f/f5d 0 1 2024-04-25T01:36:05.009 INFO:tasks.workunit.client.1.smithi138.stdout:1/399: write d3/d27/d2d/f79 [727776,77936] 0 2024-04-25T01:36:05.009 INFO:tasks.workunit.client.1.smithi138.stdout:3/531: stat db/df/l7e 0 2024-04-25T01:36:05.012 INFO:tasks.workunit.client.1.smithi138.stdout:0/338: write d6/df/f4d [1092346,3318] 0 2024-04-25T01:36:05.013 INFO:tasks.workunit.client.1.smithi138.stdout:4/349: mknod d0/d1c/c67 0 2024-04-25T01:36:05.013 INFO:tasks.workunit.client.1.smithi138.stdout:0/339: chown d6/d39/d43 0 1 2024-04-25T01:36:05.013 INFO:tasks.workunit.client.1.smithi138.stdout:3/532: chown db/df/d19/d1a/f94 7668 1 2024-04-25T01:36:05.013 INFO:tasks.workunit.client.0.smithi033.stdout:1/799: creat d2/d5/ffe x:0 0 0 2024-04-25T01:36:05.014 INFO:tasks.workunit.client.0.smithi033.stdout:4/980: dread d6/d14/d35/d44/d45/f75 [0,4194304] 0 2024-04-25T01:36:05.017 INFO:tasks.workunit.client.1.smithi138.stdout:4/350: creat d0/d16/d4a/f68 x:0 0 0 2024-04-25T01:36:05.019 INFO:tasks.workunit.client.1.smithi138.stdout:2/473: link d0/d32/d35/f40 d0/d18/d8b/f8d 0 2024-04-25T01:36:05.021 INFO:tasks.workunit.client.1.smithi138.stdout:0/340: symlink d6/df/d69/l6f 0 2024-04-25T01:36:05.021 INFO:tasks.workunit.client.1.smithi138.stdout:2/474: chown d0/d21/d45/l77 2276985 1 2024-04-25T01:36:05.022 INFO:tasks.workunit.client.0.smithi033.stdout:2/914: getdents da/d27/d31/d40/d109 0 2024-04-25T01:36:05.022 INFO:tasks.workunit.client.1.smithi138.stdout:2/475: chown d0/d18/d8b 737 1 2024-04-25T01:36:05.023 INFO:tasks.workunit.client.1.smithi138.stdout:0/341: chown f1 926 1 2024-04-25T01:36:05.024 INFO:tasks.workunit.client.0.smithi033.stdout:3/944: write d8/dd/d3d/d63/d7c/f96 [5831454,23760] 0 2024-04-25T01:36:05.024 INFO:tasks.workunit.client.0.smithi033.stdout:9/936: dread d0/d2/d43/f8e [0,4194304] 0 2024-04-25T01:36:05.026 INFO:tasks.workunit.client.1.smithi138.stdout:2/476: truncate d0/d5a/d15/d7e/f4a 4696424 0 2024-04-25T01:36:05.028 INFO:tasks.workunit.client.1.smithi138.stdout:3/533: rename db/d14/d40/d41/f84 to db/d14/d40/d41/fa5 0 2024-04-25T01:36:05.029 INFO:tasks.workunit.client.0.smithi033.stdout:1/800: rename d2/d5/d75/c39 to d2/d2e/d34/d60/cff 0 2024-04-25T01:36:05.030 INFO:tasks.workunit.client.1.smithi138.stdout:0/342: write d6/d1d/f5f [5202139,83366] 0 2024-04-25T01:36:05.030 INFO:tasks.workunit.client.1.smithi138.stdout:9/392: getdents d4/d1a/d55 0 2024-04-25T01:36:05.032 INFO:tasks.workunit.client.1.smithi138.stdout:3/534: chown db/df/d17/f2e 0 1 2024-04-25T01:36:05.034 INFO:tasks.workunit.client.1.smithi138.stdout:9/393: write d4/d1a/d55/f5c [370721,101662] 0 2024-04-25T01:36:05.035 INFO:tasks.workunit.client.0.smithi033.stdout:0/969: truncate d7/d2d/d4d/d9c/d61/fdb 23707 0 2024-04-25T01:36:05.036 INFO:tasks.workunit.client.0.smithi033.stdout:7/959: dwrite d4/d2e/d65/f68 [0,4194304] 0 2024-04-25T01:36:05.041 INFO:tasks.workunit.client.0.smithi033.stdout:9/937: rename d0/d2/d43/d4b/dc5/de0/c44 to d0/d2/d43/d4b/dc5/de0/d6e/c143 0 2024-04-25T01:36:05.043 INFO:tasks.workunit.client.1.smithi138.stdout:2/477: getdents d0/d32/d35/d4f 0 2024-04-25T01:36:05.046 INFO:tasks.workunit.client.0.smithi033.stdout:8/977: dwrite d6/db/d80/dd1/fe8 [0,4194304] 0 2024-04-25T01:36:05.047 INFO:tasks.workunit.client.1.smithi138.stdout:2/478: chown d0/d21/d45/l68 44758 1 2024-04-25T01:36:05.047 INFO:tasks.workunit.client.1.smithi138.stdout:4/351: dread d0/d1c/f49 [0,4194304] 0 2024-04-25T01:36:05.048 INFO:tasks.workunit.client.0.smithi033.stdout:7/960: symlink d4/d2e/d11c/d1f/d78/d86/de4/l139 0 2024-04-25T01:36:05.050 INFO:tasks.workunit.client.0.smithi033.stdout:9/938: creat d0/d2/d43/d4b/dc5/de0/d7e/ddc/f144 x:0 0 0 2024-04-25T01:36:05.050 INFO:tasks.workunit.client.1.smithi138.stdout:0/343: creat d6/d39/f70 x:0 0 0 2024-04-25T01:36:05.051 INFO:tasks.workunit.client.0.smithi033.stdout:8/978: chown d6/db/c4b 142569237 1 2024-04-25T01:36:05.053 INFO:tasks.workunit.client.1.smithi138.stdout:1/400: dread d3/db/f67 [0,4194304] 0 2024-04-25T01:36:05.054 INFO:tasks.workunit.client.1.smithi138.stdout:0/344: write d6/f26 [8174470,89231] 0 2024-04-25T01:36:05.054 INFO:tasks.workunit.client.0.smithi033.stdout:7/961: readlink d4/d2e/d11c/d1f/d78/l7b 0 2024-04-25T01:36:05.055 INFO:tasks.workunit.client.1.smithi138.stdout:3/535: creat db/d14/d40/d41/fa6 x:0 0 0 2024-04-25T01:36:05.055 INFO:tasks.workunit.client.0.smithi033.stdout:0/970: creat d7/d2d/d64/d14a/f150 x:0 0 0 2024-04-25T01:36:05.056 INFO:tasks.workunit.client.1.smithi138.stdout:9/394: rmdir d4/d63 39 2024-04-25T01:36:05.056 INFO:tasks.workunit.client.1.smithi138.stdout:1/401: write d3/d27/d2d/f79 [1800946,24348] 0 2024-04-25T01:36:05.056 INFO:tasks.workunit.client.1.smithi138.stdout:7/457: dread da/de/d1b/d34/f47 [0,4194304] 0 2024-04-25T01:36:05.057 INFO:tasks.workunit.client.0.smithi033.stdout:5/986: write d1c/d2a/d50/d52/f66 [2230213,61711] 0 2024-04-25T01:36:05.058 INFO:tasks.workunit.client.0.smithi033.stdout:2/915: dread da/d27/f86 [0,4194304] 0 2024-04-25T01:36:05.058 INFO:tasks.workunit.client.0.smithi033.stdout:3/945: dread d8/dd/d11/fa7 [0,4194304] 0 2024-04-25T01:36:05.059 INFO:tasks.workunit.client.1.smithi138.stdout:2/479: unlink d0/d32/d35/d4f/l53 0 2024-04-25T01:36:05.059 INFO:tasks.workunit.client.1.smithi138.stdout:0/345: write d6/df/f14 [766451,60484] 0 2024-04-25T01:36:05.059 INFO:tasks.workunit.client.0.smithi033.stdout:8/979: mkdir d6/d45/d51/d5d/d9d/d10b/d164 0 2024-04-25T01:36:05.060 INFO:tasks.workunit.client.1.smithi138.stdout:7/458: chown da/de/d13/d21 0 1 2024-04-25T01:36:05.060 INFO:tasks.workunit.client.1.smithi138.stdout:1/402: chown d3/d45/d72/f5d 252908 1 2024-04-25T01:36:05.060 INFO:tasks.workunit.client.1.smithi138.stdout:3/536: mknod db/df/d17/d20/d57/d6a/ca7 0 2024-04-25T01:36:05.061 INFO:tasks.workunit.client.0.smithi033.stdout:0/971: dread - d7/da/df/d50/dbb/f11c zero size 2024-04-25T01:36:05.063 INFO:tasks.workunit.client.1.smithi138.stdout:2/480: write d0/d5a/d15/d51/f72 [1229840,87738] 0 2024-04-25T01:36:05.063 INFO:tasks.workunit.client.1.smithi138.stdout:3/537: write db/df/d17/d20/f74 [795024,38582] 0 2024-04-25T01:36:05.063 INFO:tasks.workunit.client.1.smithi138.stdout:1/403: rmdir d3/d27 39 2024-04-25T01:36:05.065 INFO:tasks.workunit.client.1.smithi138.stdout:2/481: stat d0/d18/d8b 0 2024-04-25T01:36:05.066 INFO:tasks.workunit.client.1.smithi138.stdout:3/538: write db/df/d17/d20/d57/d6a/f8d [600817,1542] 0 2024-04-25T01:36:05.068 INFO:tasks.workunit.client.0.smithi033.stdout:4/981: dread d6/d12/f32 [0,4194304] 0 2024-04-25T01:36:05.069 INFO:tasks.workunit.client.1.smithi138.stdout:7/459: getdents da/de/d13/d30 0 2024-04-25T01:36:05.070 INFO:tasks.workunit.client.1.smithi138.stdout:1/404: truncate d3/f2c 1350515 0 2024-04-25T01:36:05.071 INFO:tasks.workunit.client.0.smithi033.stdout:8/980: creat d6/d45/d51/d5d/d9d/f165 x:0 0 0 2024-04-25T01:36:05.071 INFO:tasks.workunit.client.1.smithi138.stdout:2/482: mkdir d0/d21/d45/d8a/d8e 0 2024-04-25T01:36:05.072 INFO:tasks.workunit.client.1.smithi138.stdout:9/395: dread d4/d1a/f70 [0,4194304] 0 2024-04-25T01:36:05.073 INFO:tasks.workunit.client.1.smithi138.stdout:7/460: readlink da/de/d13/l19 0 2024-04-25T01:36:05.074 INFO:tasks.workunit.client.0.smithi033.stdout:3/946: creat d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/f14a x:0 0 0 2024-04-25T01:36:05.076 INFO:tasks.workunit.client.0.smithi033.stdout:5/987: dwrite d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/d13b/f129 [0,4194304] 0 2024-04-25T01:36:05.077 INFO:tasks.workunit.client.0.smithi033.stdout:1/801: chown d2/d5/d75/d59/d83/cc3 55308 1 2024-04-25T01:36:05.077 INFO:tasks.workunit.client.1.smithi138.stdout:1/405: unlink d3/db/f1d 0 2024-04-25T01:36:05.077 INFO:tasks.workunit.client.1.smithi138.stdout:9/396: write d4/d8/f59 [561570,37154] 0 2024-04-25T01:36:05.079 INFO:tasks.workunit.client.1.smithi138.stdout:3/539: dwrite db/f62 [0,4194304] 0 2024-04-25T01:36:05.079 INFO:tasks.workunit.client.0.smithi033.stdout:1/802: write d2/d5/d6c/f87 [207715,23845] 0 2024-04-25T01:36:05.080 INFO:tasks.workunit.client.0.smithi033.stdout:1/803: chown d2/d5/d6c/fcf 4401679 1 2024-04-25T01:36:05.082 INFO:tasks.workunit.client.1.smithi138.stdout:4/352: dread d0/f1a [0,4194304] 0 2024-04-25T01:36:05.082 INFO:tasks.workunit.client.1.smithi138.stdout:2/483: readlink d0/d5a/d15/d51/l7a 0 2024-04-25T01:36:05.082 INFO:tasks.workunit.client.0.smithi033.stdout:9/939: rename d0/d2/d43/d4b/dc5/de0/d42/dea/l11d to d0/d2/d43/d4b/dc3/l145 0 2024-04-25T01:36:05.083 INFO:tasks.workunit.client.1.smithi138.stdout:9/397: chown d4/d1a/c46 19767545 1 2024-04-25T01:36:05.085 INFO:tasks.workunit.client.1.smithi138.stdout:2/484: write d0/d5a/d15/d51/f72 [711573,20743] 0 2024-04-25T01:36:05.087 INFO:tasks.workunit.client.1.smithi138.stdout:3/540: fdatasync db/f12 0 2024-04-25T01:36:05.087 INFO:tasks.workunit.client.0.smithi033.stdout:4/982: creat d6/d12/d38/dc0/dff/f140 x:0 0 0 2024-04-25T01:36:05.089 INFO:tasks.workunit.client.1.smithi138.stdout:7/461: symlink da/de/d1b/d49/d81/d51/l9a 0 2024-04-25T01:36:05.091 INFO:tasks.workunit.client.0.smithi033.stdout:4/983: write d6/d14/d35/d44/d45/d46/d57/d70/d87/d112/f11a [1433913,46701] 0 2024-04-25T01:36:05.092 INFO:tasks.workunit.client.0.smithi033.stdout:2/916: mknod da/d50/d6a/d43/df1/c129 0 2024-04-25T01:36:05.094 INFO:tasks.workunit.client.0.smithi033.stdout:5/988: write d1c/d2a/d3d/f4d [710134,118456] 0 2024-04-25T01:36:05.096 INFO:tasks.workunit.client.0.smithi033.stdout:7/962: write d4/d2e/d6d/fdf [27805,28822] 0 2024-04-25T01:36:05.097 INFO:tasks.workunit.client.1.smithi138.stdout:2/485: rmdir d0/d32/d35/d4f 39 2024-04-25T01:36:05.097 INFO:tasks.workunit.client.1.smithi138.stdout:4/353: dwrite d0/d16/d4a/f53 [0,4194304] 0 2024-04-25T01:36:05.097 INFO:tasks.workunit.client.0.smithi033.stdout:8/981: mkdir d6/d45/d51/d5d/d9d/d10b/d164/d166 0 2024-04-25T01:36:05.098 INFO:tasks.workunit.client.0.smithi033.stdout:7/963: chown d4/d2e/d11c/d1f/d78/d86/de4 183 1 2024-04-25T01:36:05.104 INFO:tasks.workunit.client.1.smithi138.stdout:9/398: mkdir d4/d1a/d55/d7f/d80 0 2024-04-25T01:36:05.105 INFO:tasks.workunit.client.1.smithi138.stdout:9/399: read f1 [3452786,112162] 0 2024-04-25T01:36:05.106 INFO:tasks.workunit.client.1.smithi138.stdout:7/462: symlink da/de/d1b/l9b 0 2024-04-25T01:36:05.106 INFO:tasks.workunit.client.1.smithi138.stdout:1/406: dread d3/db/f5a [0,4194304] 0 2024-04-25T01:36:05.107 INFO:tasks.workunit.client.0.smithi033.stdout:4/984: mkdir d6/d14/d35/d44/dfe/d12c/d141 0 2024-04-25T01:36:05.107 INFO:tasks.workunit.client.1.smithi138.stdout:7/463: fsync da/de/f74 0 2024-04-25T01:36:05.108 INFO:tasks.workunit.client.0.smithi033.stdout:7/964: symlink d4/d23/d3f/d45/d73/dbe/l13a 0 2024-04-25T01:36:05.109 INFO:tasks.workunit.client.0.smithi033.stdout:0/972: dread d7/d2d/d43/dc0/fa3 [0,4194304] 0 2024-04-25T01:36:05.109 INFO:tasks.workunit.client.1.smithi138.stdout:2/486: unlink d0/c3 0 2024-04-25T01:36:05.112 INFO:tasks.workunit.client.0.smithi033.stdout:7/965: read - d4/d2e/d6d/dbb/f112 zero size 2024-04-25T01:36:05.112 INFO:tasks.workunit.client.0.smithi033.stdout:1/804: rename d2/d7f/db0/le0 to d2/d5/l100 0 2024-04-25T01:36:05.114 INFO:tasks.workunit.client.0.smithi033.stdout:0/973: readlink d7/d2d/d4d/d9c/l53 0 2024-04-25T01:36:05.114 INFO:tasks.workunit.client.0.smithi033.stdout:2/917: creat da/f12a x:0 0 0 2024-04-25T01:36:05.115 INFO:tasks.workunit.client.0.smithi033.stdout:8/982: dwrite d6/d45/d51/d55/d84/f162 [0,4194304] 0 2024-04-25T01:36:05.116 INFO:tasks.workunit.client.1.smithi138.stdout:7/464: readlink da/l1f 0 2024-04-25T01:36:05.117 INFO:tasks.workunit.client.1.smithi138.stdout:1/407: truncate d3/f34 1048167 0 2024-04-25T01:36:05.118 INFO:tasks.workunit.client.1.smithi138.stdout:7/465: write da/de/d1b/d4b/f55 [3840567,77998] 0 2024-04-25T01:36:05.118 INFO:tasks.workunit.client.0.smithi033.stdout:2/918: dread - da/d50/d6a/d43/f101 zero size 2024-04-25T01:36:05.119 INFO:tasks.workunit.client.0.smithi033.stdout:2/919: dread - da/d5b/d5e/f89 zero size 2024-04-25T01:36:05.120 INFO:tasks.workunit.client.1.smithi138.stdout:4/354: link d0/d16/f29 d0/d1c/d1e/f69 0 2024-04-25T01:36:05.120 INFO:tasks.workunit.client.0.smithi033.stdout:2/920: chown da/d5b/d5e/dd8 12191518 1 2024-04-25T01:36:05.121 INFO:tasks.workunit.client.0.smithi033.stdout:3/947: write d8/dd/d11/f122 [976789,55793] 0 2024-04-25T01:36:05.122 INFO:tasks.workunit.client.0.smithi033.stdout:1/805: mkdir d2/d5/d6c/d101 0 2024-04-25T01:36:05.123 INFO:tasks.workunit.client.1.smithi138.stdout:2/487: mknod d0/d5a/d15/d44/d4c/d56/d6d/c8f 0 2024-04-25T01:36:05.123 INFO:tasks.workunit.client.0.smithi033.stdout:1/806: chown d2/d2e/dcd/lb7 229136069 1 2024-04-25T01:36:05.125 INFO:tasks.workunit.client.0.smithi033.stdout:8/983: unlink d6/d45/d52/f2b 0 2024-04-25T01:36:05.128 INFO:tasks.workunit.client.0.smithi033.stdout:8/984: write d6/db/d80/d89/f91 [340732,119530] 0 2024-04-25T01:36:05.131 INFO:tasks.workunit.client.0.smithi033.stdout:7/966: fsync d4/d2e/d11c/d1f/f71 0 2024-04-25T01:36:05.131 INFO:tasks.workunit.client.0.smithi033.stdout:5/989: dread d1c/d2a/fb2 [0,4194304] 0 2024-04-25T01:36:05.131 INFO:tasks.workunit.client.0.smithi033.stdout:8/985: stat d6/db/d80/db4/f145 0 2024-04-25T01:36:05.132 INFO:tasks.workunit.client.1.smithi138.stdout:3/541: dread db/df/d17/d23/d34/f50 [4194304,4194304] 0 2024-04-25T01:36:05.132 INFO:tasks.workunit.client.0.smithi033.stdout:3/948: fdatasync d8/dd/d3d/d4f/fa0 0 2024-04-25T01:36:05.135 INFO:tasks.workunit.client.0.smithi033.stdout:1/807: mkdir d2/d5/d1b/dd2/d102 0 2024-04-25T01:36:05.135 INFO:tasks.workunit.client.1.smithi138.stdout:9/400: creat d4/d1a/f81 x:0 0 0 2024-04-25T01:36:05.137 INFO:tasks.workunit.client.1.smithi138.stdout:1/408: mkdir d3/d27/d3e/d51/d84 0 2024-04-25T01:36:05.138 INFO:tasks.workunit.client.1.smithi138.stdout:1/409: dread - d3/d27/d3e/f61 zero size 2024-04-25T01:36:05.139 INFO:tasks.workunit.client.1.smithi138.stdout:1/410: stat d3/d27/d16/l3a 0 2024-04-25T01:36:05.139 INFO:tasks.workunit.client.1.smithi138.stdout:1/411: chown d3/db 1606528 1 2024-04-25T01:36:05.140 INFO:tasks.workunit.client.0.smithi033.stdout:7/967: mkdir d4/d2e/d50/d13b 0 2024-04-25T01:36:05.140 INFO:tasks.workunit.client.1.smithi138.stdout:3/542: creat db/df/d17/d5d/fa8 x:0 0 0 2024-04-25T01:36:05.141 INFO:tasks.workunit.client.1.smithi138.stdout:9/401: rmdir d4/d1a/d33 39 2024-04-25T01:36:05.142 INFO:tasks.workunit.client.0.smithi033.stdout:3/949: creat d8/dd/d3d/dce/f14b x:0 0 0 2024-04-25T01:36:05.142 INFO:tasks.workunit.client.0.smithi033.stdout:7/968: stat d4/d23/d3f/d45/d73/dbe/l13a 0 2024-04-25T01:36:05.143 INFO:tasks.workunit.client.0.smithi033.stdout:5/990: truncate d1c/d2a/d50/d52/da6/d3e/d70/d89/dc0/fe6 934802 0 2024-04-25T01:36:05.143 INFO:tasks.workunit.client.1.smithi138.stdout:3/543: rmdir db/d14 39 2024-04-25T01:36:05.144 INFO:tasks.workunit.client.0.smithi033.stdout:2/921: rmdir da/d5b/d5e/dd8 0 2024-04-25T01:36:05.145 INFO:tasks.workunit.client.0.smithi033.stdout:8/986: mkdir d6/d45/d52/d119/d167 0 2024-04-25T01:36:05.146 INFO:tasks.workunit.client.1.smithi138.stdout:9/402: chown d4/d1a/d33/d1d/d3f/f78 357 1 2024-04-25T01:36:05.148 INFO:tasks.workunit.client.0.smithi033.stdout:7/969: dread - d4/d2e/d65/dcb/f11f zero size 2024-04-25T01:36:05.149 INFO:tasks.workunit.client.0.smithi033.stdout:4/985: dread d6/d14/d35/d3a/f49 [0,4194304] 0 2024-04-25T01:36:05.149 INFO:tasks.workunit.client.1.smithi138.stdout:9/403: fdatasync d4/d1a/d33/f36 0 2024-04-25T01:36:05.151 INFO:tasks.workunit.client.1.smithi138.stdout:3/544: symlink db/d14/d40/la9 0 2024-04-25T01:36:05.151 INFO:tasks.workunit.client.1.smithi138.stdout:3/545: chown db/df 520354 1 2024-04-25T01:36:05.152 INFO:tasks.workunit.client.1.smithi138.stdout:1/412: link d3/d27/d3e/d51/d62/d65/l66 d3/d27/d3e/d51/d62/d65/l85 0 2024-04-25T01:36:05.152 INFO:tasks.workunit.client.1.smithi138.stdout:9/404: read d4/d8/f1f [2580943,108025] 0 2024-04-25T01:36:05.152 INFO:tasks.workunit.client.0.smithi033.stdout:5/991: mknod d1c/d2a/d50/d52/da6/d3e/c140 0 2024-04-25T01:36:05.153 INFO:tasks.workunit.client.1.smithi138.stdout:3/546: write db/df/d17/d5d/f6b [953905,14502] 0 2024-04-25T01:36:05.153 INFO:tasks.workunit.client.0.smithi033.stdout:7/970: creat d4/d23/da7/dc2/f13c x:0 0 0 2024-04-25T01:36:05.156 INFO:tasks.workunit.client.1.smithi138.stdout:7/466: truncate da/de/d13/d30/f42 2587215 0 2024-04-25T01:36:05.156 INFO:tasks.workunit.client.1.smithi138.stdout:2/488: dwrite d0/d21/f22 [0,4194304] 0 2024-04-25T01:36:05.157 INFO:tasks.workunit.client.0.smithi033.stdout:2/922: truncate da/d5b/d5e/d76/f90 42553 0 2024-04-25T01:36:05.159 INFO:tasks.workunit.client.1.smithi138.stdout:9/405: mkdir d4/d8/d2c/d82 0 2024-04-25T01:36:05.159 INFO:tasks.workunit.client.1.smithi138.stdout:4/355: write d0/d16/d4a/f4d [9184,26802] 0 2024-04-25T01:36:05.161 INFO:tasks.workunit.client.1.smithi138.stdout:3/547: mkdir db/df/d17/d20/d57/daa 0 2024-04-25T01:36:05.162 INFO:tasks.workunit.client.0.smithi033.stdout:1/808: dwrite d2/f29 [4194304,4194304] 0 2024-04-25T01:36:05.163 INFO:tasks.workunit.client.1.smithi138.stdout:7/467: creat da/de/d1b/d49/d81/f9c x:0 0 0 2024-04-25T01:36:05.164 INFO:tasks.workunit.client.0.smithi033.stdout:8/987: dwrite d6/d45/d52/d2e/f3a [0,4194304] 0 2024-04-25T01:36:05.169 INFO:tasks.workunit.client.0.smithi033.stdout:1/809: symlink d2/d2e/d49/l103 0 2024-04-25T01:36:05.171 INFO:tasks.workunit.client.0.smithi033.stdout:3/950: dread d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/f7d [0,4194304] 0 2024-04-25T01:36:05.172 INFO:tasks.workunit.client.0.smithi033.stdout:1/810: dread - d2/d5/d6c/d94/ff0 zero size 2024-04-25T01:36:05.173 INFO:tasks.workunit.client.0.smithi033.stdout:4/986: dread d6/d12/fd1 [0,4194304] 0 2024-04-25T01:36:05.173 INFO:tasks.workunit.client.0.smithi033.stdout:8/988: creat d6/d45/d51/d5d/d144/f168 x:0 0 0 2024-04-25T01:36:05.176 INFO:tasks.workunit.client.0.smithi033.stdout:1/811: write d2/d11/f91 [2052509,106448] 0 2024-04-25T01:36:05.177 INFO:tasks.workunit.client.0.smithi033.stdout:4/987: unlink d6/d14/d35/d44/d45/d46/d57/d70/c89 0 2024-04-25T01:36:05.179 INFO:tasks.workunit.client.0.smithi033.stdout:1/812: symlink d2/d5/d6c/dc6/l104 0 2024-04-25T01:36:05.181 INFO:tasks.workunit.client.0.smithi033.stdout:3/951: truncate d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/f9f 11627273 0 2024-04-25T01:36:05.182 INFO:tasks.workunit.client.0.smithi033.stdout:4/988: creat d6/d14/d35/d44/d45/d46/db0/f142 x:0 0 0 2024-04-25T01:36:05.189 INFO:tasks.workunit.client.0.smithi033.stdout:1/813: read d2/d11/f6d [187344,8373] 0 2024-04-25T01:36:05.189 INFO:tasks.workunit.client.1.smithi138.stdout:2/489: mknod d0/d21/d45/c90 0 2024-04-25T01:36:05.189 INFO:tasks.workunit.client.1.smithi138.stdout:7/468: write da/de/f74 [2339946,35201] 0 2024-04-25T01:36:05.190 INFO:tasks.workunit.client.0.smithi033.stdout:7/971: write d4/d23/d3f/d45/d73/fea [40930,130685] 0 2024-04-25T01:36:05.191 INFO:tasks.workunit.client.1.smithi138.stdout:4/356: creat d0/d59/f6a x:0 0 0 2024-04-25T01:36:05.191 INFO:tasks.workunit.client.0.smithi033.stdout:1/814: mkdir d2/d5/d6c/d105 0 2024-04-25T01:36:05.192 INFO:tasks.workunit.client.1.smithi138.stdout:9/406: creat d4/d1a/d55/d7f/f83 x:0 0 0 2024-04-25T01:36:05.193 INFO:tasks.workunit.client.0.smithi033.stdout:7/972: mkdir d4/d2e/d6d/d13d 0 2024-04-25T01:36:05.193 INFO:tasks.workunit.client.0.smithi033.stdout:1/815: chown d2/d5/d75/d59/d83 459 1 2024-04-25T01:36:05.194 INFO:tasks.workunit.client.1.smithi138.stdout:4/357: write d0/d16/d33/f51 [748678,128335] 0 2024-04-25T01:36:05.194 INFO:tasks.workunit.client.1.smithi138.stdout:3/548: getdents db/df/d17/d20/d57/daa 0 2024-04-25T01:36:05.195 INFO:tasks.workunit.client.0.smithi033.stdout:1/816: mkdir d2/d2e/d34/d60/d106 0 2024-04-25T01:36:05.195 INFO:tasks.workunit.client.1.smithi138.stdout:4/358: fdatasync d0/f3f 0 2024-04-25T01:36:05.198 INFO:tasks.workunit.client.1.smithi138.stdout:2/490: dread d0/d5a/f48 [4194304,4194304] 0 2024-04-25T01:36:05.198 INFO:tasks.workunit.client.1.smithi138.stdout:7/469: mkdir da/de/d1b/d49/d8a/d94/d9d 0 2024-04-25T01:36:05.198 INFO:tasks.workunit.client.0.smithi033.stdout:2/923: write da/d5b/d5e/d76/fc9 [831160,76188] 0 2024-04-25T01:36:05.198 INFO:tasks.workunit.client.0.smithi033.stdout:7/973: link d4/d2e/d11c/c87 d4/d23/d3f/c13e 0 2024-04-25T01:36:05.199 INFO:tasks.workunit.client.1.smithi138.stdout:7/470: read - da/de/d1b/d4b/d77/f91 zero size 2024-04-25T01:36:05.201 INFO:tasks.workunit.client.1.smithi138.stdout:3/549: write db/df/d17/d20/d57/d6a/f6e [579518,70287] 0 2024-04-25T01:36:05.202 INFO:tasks.workunit.client.0.smithi033.stdout:7/974: symlink d4/d57/da5/l13f 0 2024-04-25T01:36:05.203 INFO:tasks.workunit.client.1.smithi138.stdout:9/407: creat d4/d1a/f84 x:0 0 0 2024-04-25T01:36:05.204 INFO:tasks.workunit.client.1.smithi138.stdout:4/359: unlink d0/d16/d4a/l61 0 2024-04-25T01:36:05.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/491: mknod d0/d21/c91 0 2024-04-25T01:36:05.207 INFO:tasks.workunit.client.1.smithi138.stdout:9/408: write d4/d1a/f21 [1641407,9306] 0 2024-04-25T01:36:05.207 INFO:tasks.workunit.client.0.smithi033.stdout:7/975: dwrite d4/d10/fdb [0,4194304] 0 2024-04-25T01:36:05.210 INFO:tasks.workunit.client.0.smithi033.stdout:7/976: write d4/d2e/d11c/fbc [316315,109326] 0 2024-04-25T01:36:05.210 INFO:tasks.workunit.client.1.smithi138.stdout:7/471: rename da/de/f98 to da/de/d1b/d4b/f9e 0 2024-04-25T01:36:05.212 INFO:tasks.workunit.client.1.smithi138.stdout:3/550: dwrite db/df/d17/d5d/fa8 [0,4194304] 0 2024-04-25T01:36:05.213 INFO:tasks.workunit.client.1.smithi138.stdout:4/360: creat d0/d1c/f6b x:0 0 0 2024-04-25T01:36:05.213 INFO:tasks.workunit.client.0.smithi033.stdout:7/977: creat d4/d2e/d6d/f140 x:0 0 0 2024-04-25T01:36:05.214 INFO:tasks.workunit.client.0.smithi033.stdout:7/978: write d4/d2e/d11c/fbc [1561104,87713] 0 2024-04-25T01:36:05.215 INFO:tasks.workunit.client.1.smithi138.stdout:4/361: chown d0/d1c/c67 1474 1 2024-04-25T01:36:05.216 INFO:tasks.workunit.client.0.smithi033.stdout:7/979: chown d4/d57/da5/l13f 7767050 1 2024-04-25T01:36:05.217 INFO:tasks.workunit.client.1.smithi138.stdout:3/551: symlink db/d14/lab 0 2024-04-25T01:36:05.217 INFO:tasks.workunit.client.0.smithi033.stdout:8/989: write d6/d45/d51/d55/daa/fea [2753527,64013] 0 2024-04-25T01:36:05.219 INFO:tasks.workunit.client.0.smithi033.stdout:7/980: creat d4/d2e/d11c/da9/db3/f141 x:0 0 0 2024-04-25T01:36:05.222 INFO:tasks.workunit.client.0.smithi033.stdout:7/981: creat d4/d2e/d11c/da9/db3/f142 x:0 0 0 2024-04-25T01:36:05.225 INFO:tasks.workunit.client.0.smithi033.stdout:4/989: write d6/d14/d35/d3a/d4d/d5e/d7c/d94/fd2 [913187,94907] 0 2024-04-25T01:36:05.226 INFO:tasks.workunit.client.1.smithi138.stdout:3/552: mkdir db/d9b/dac 0 2024-04-25T01:36:05.227 INFO:tasks.workunit.client.1.smithi138.stdout:7/472: link da/de/d13/l7d da/de/d1b/d4b/d5b/l9f 0 2024-04-25T01:36:05.228 INFO:tasks.workunit.client.1.smithi138.stdout:7/473: stat da/de/d1b/l45 0 2024-04-25T01:36:05.228 INFO:tasks.workunit.client.0.smithi033.stdout:7/982: creat d4/d23/d3f/d45/d90/ded/f143 x:0 0 0 2024-04-25T01:36:05.228 INFO:tasks.workunit.client.1.smithi138.stdout:7/474: fsync da/de/d13/f2d 0 2024-04-25T01:36:05.229 INFO:tasks.workunit.client.1.smithi138.stdout:4/362: getdents d0/d1c/d1e 0 2024-04-25T01:36:05.229 INFO:tasks.workunit.client.0.smithi033.stdout:8/990: getdents d6/d45/d51/d55/d84 0 2024-04-25T01:36:05.230 INFO:tasks.workunit.client.1.smithi138.stdout:3/553: mkdir db/d14/d40/d41/dad 0 2024-04-25T01:36:05.230 INFO:tasks.workunit.client.0.smithi033.stdout:7/983: chown d4/d2e/d11c/da9/ce2 218 1 2024-04-25T01:36:05.230 INFO:tasks.workunit.client.1.smithi138.stdout:7/475: dread - da/de/d1b/d4b/d77/f91 zero size 2024-04-25T01:36:05.231 INFO:tasks.workunit.client.1.smithi138.stdout:4/363: truncate d0/d1c/f2c 646021 0 2024-04-25T01:36:05.231 INFO:tasks.workunit.client.0.smithi033.stdout:3/952: dwrite d8/f59 [0,4194304] 0 2024-04-25T01:36:05.232 INFO:tasks.workunit.client.0.smithi033.stdout:4/990: creat d6/d14/d35/d44/dfe/d12c/f143 x:0 0 0 2024-04-25T01:36:05.234 INFO:tasks.workunit.client.0.smithi033.stdout:3/953: read d8/dd/d11/f122 [845193,54001] 0 2024-04-25T01:36:05.235 INFO:tasks.workunit.client.1.smithi138.stdout:3/554: mknod db/df/d17/d20/cae 0 2024-04-25T01:36:05.236 INFO:tasks.workunit.client.0.smithi033.stdout:2/924: write da/d50/d6a/f57 [401488,87996] 0 2024-04-25T01:36:05.238 INFO:tasks.workunit.client.0.smithi033.stdout:1/817: dread d2/d11/f1a [4194304,4194304] 0 2024-04-25T01:36:05.239 INFO:tasks.workunit.client.0.smithi033.stdout:1/818: stat d2/d11/d28/cac 0 2024-04-25T01:36:05.239 INFO:tasks.workunit.client.0.smithi033.stdout:4/991: rename d6/l105 to d6/d12/d84/l144 0 2024-04-25T01:36:05.240 INFO:tasks.workunit.client.1.smithi138.stdout:7/476: dwrite da/de/d1b/d4b/f9e [0,4194304] 0 2024-04-25T01:36:05.241 INFO:tasks.workunit.client.1.smithi138.stdout:7/477: chown da/de/d13/d30 1642 1 2024-04-25T01:36:05.242 INFO:tasks.workunit.client.0.smithi033.stdout:3/954: mkdir d8/dd/d3d/d4f/df0/d14c 0 2024-04-25T01:36:05.244 INFO:tasks.workunit.client.0.smithi033.stdout:7/984: rmdir d4/d2e/d6d 39 2024-04-25T01:36:05.246 INFO:tasks.workunit.client.0.smithi033.stdout:7/985: write d4/d2e/d11c/da9/db3/f117 [438971,9648] 0 2024-04-25T01:36:05.247 INFO:tasks.workunit.client.1.smithi138.stdout:7/478: creat da/de/d13/d30/d59/d7b/d99/fa0 x:0 0 0 2024-04-25T01:36:05.247 INFO:tasks.workunit.client.0.smithi033.stdout:7/986: chown d4/d23/d3f/d45 60 1 2024-04-25T01:36:05.248 INFO:tasks.workunit.client.0.smithi033.stdout:2/925: creat da/d5b/d72/dd1/f12b x:0 0 0 2024-04-25T01:36:05.249 INFO:tasks.workunit.client.1.smithi138.stdout:7/479: write da/de/d1b/d34/f3d [3536250,65188] 0 2024-04-25T01:36:05.252 INFO:tasks.workunit.client.1.smithi138.stdout:7/480: mknod da/de/d13/d21/d6f/d8d/ca1 0 2024-04-25T01:36:05.254 INFO:tasks.workunit.client.1.smithi138.stdout:7/481: mkdir da/de/d1b/d49/d81/d51/d6e/da2 0 2024-04-25T01:36:05.257 INFO:tasks.workunit.client.0.smithi033.stdout:7/987: dwrite d4/d2e/d65/dcb/f12b [0,4194304] 0 2024-04-25T01:36:05.259 INFO:tasks.workunit.client.0.smithi033.stdout:2/926: creat da/d27/d31/dca/f12c x:0 0 0 2024-04-25T01:36:05.262 INFO:tasks.workunit.client.0.smithi033.stdout:2/927: creat da/d2c/db8/f12d x:0 0 0 2024-04-25T01:36:05.262 INFO:tasks.workunit.client.0.smithi033.stdout:8/991: write d6/dd3/dfb/f102 [983283,95960] 0 2024-04-25T01:36:05.265 INFO:tasks.workunit.client.0.smithi033.stdout:8/992: read - d6/d45/f13e zero size 2024-04-25T01:36:05.265 INFO:tasks.workunit.client.0.smithi033.stdout:2/928: chown da/d27/f7a 254561900 1 2024-04-25T01:36:05.265 INFO:tasks.workunit.client.1.smithi138.stdout:3/555: write db/f12 [2518883,55390] 0 2024-04-25T01:36:05.266 INFO:tasks.workunit.client.0.smithi033.stdout:7/988: dwrite d4/d2e/d11c/fbc [0,4194304] 0 2024-04-25T01:36:05.269 INFO:tasks.workunit.client.1.smithi138.stdout:3/556: symlink db/df/d17/d20/d57/daa/laf 0 2024-04-25T01:36:05.270 INFO:tasks.workunit.client.1.smithi138.stdout:3/557: dread - db/df/d17/d5d/fa3 zero size 2024-04-25T01:36:05.271 INFO:tasks.workunit.client.1.smithi138.stdout:3/558: write db/df/d17/f9f [866340,65648] 0 2024-04-25T01:36:05.274 INFO:tasks.workunit.client.0.smithi033.stdout:2/929: mknod da/d27/d31/d40/c12e 0 2024-04-25T01:36:05.276 INFO:tasks.workunit.client.1.smithi138.stdout:3/559: link db/df/d19/d1a/f94 db/d14/d40/fb0 0 2024-04-25T01:36:05.279 INFO:tasks.workunit.client.0.smithi033.stdout:7/989: unlink d4/d2e/d11c/c25 0 2024-04-25T01:36:05.283 INFO:tasks.workunit.client.0.smithi033.stdout:1/819: dwrite d2/d2e/d34/d60/fa7 [0,4194304] 0 2024-04-25T01:36:05.284 INFO:tasks.workunit.client.0.smithi033.stdout:3/955: dwrite d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/f126 [0,4194304] 0 2024-04-25T01:36:05.285 INFO:tasks.workunit.client.1.smithi138.stdout:7/482: dread da/de/d13/f2d [0,4194304] 0 2024-04-25T01:36:05.286 INFO:tasks.workunit.client.0.smithi033.stdout:4/992: dwrite d6/d14/d6d/fc1 [0,4194304] 0 2024-04-25T01:36:05.286 INFO:tasks.workunit.client.0.smithi033.stdout:7/990: read d4/d10/fdb [2793795,54592] 0 2024-04-25T01:36:05.289 INFO:tasks.workunit.client.0.smithi033.stdout:1/820: chown d2/d11/lbe 36421793 1 2024-04-25T01:36:05.290 INFO:tasks.workunit.client.0.smithi033.stdout:8/993: dwrite d6/d45/d51/d9e/f159 [0,4194304] 0 2024-04-25T01:36:05.290 INFO:tasks.workunit.client.1.smithi138.stdout:7/483: creat da/de/d1b/fa3 x:0 0 0 2024-04-25T01:36:05.292 INFO:tasks.workunit.client.0.smithi033.stdout:3/956: symlink d8/dd/dfa/l14d 0 2024-04-25T01:36:05.299 INFO:tasks.workunit.client.1.smithi138.stdout:7/484: dwrite da/de/d13/d30/f65 [0,4194304] 0 2024-04-25T01:36:05.300 INFO:tasks.workunit.client.0.smithi033.stdout:1/821: mknod d2/d5/d1b/d46/d5f/d63/d8f/c107 0 2024-04-25T01:36:05.300 INFO:tasks.workunit.client.0.smithi033.stdout:3/957: truncate d8/dd/d3d/d4f/da4/dc4/d11f/f140 499200 0 2024-04-25T01:36:05.302 INFO:tasks.workunit.client.0.smithi033.stdout:7/991: dwrite d4/d57/db7/f100 [0,4194304] 0 2024-04-25T01:36:05.302 INFO:tasks.workunit.client.0.smithi033.stdout:2/930: link da/c37 da/d5b/d72/dd1/ded/c12f 0 2024-04-25T01:36:05.304 INFO:tasks.workunit.client.0.smithi033.stdout:3/958: read - d8/dd/d3d/d63/d7c/d4d/f13a zero size 2024-04-25T01:36:05.306 INFO:tasks.workunit.client.0.smithi033.stdout:8/994: mknod d6/d45/d51/d5d/d9d/d10b/d164/d166/c169 0 2024-04-25T01:36:05.308 INFO:tasks.workunit.client.0.smithi033.stdout:1/822: creat d2/d5/d1b/d46/d5f/d63/d64/f108 x:0 0 0 2024-04-25T01:36:05.310 INFO:tasks.workunit.client.0.smithi033.stdout:2/931: creat da/d50/d6a/d43/df1/f130 x:0 0 0 2024-04-25T01:36:05.310 INFO:tasks.workunit.client.0.smithi033.stdout:3/959: fdatasync d8/dd/d11/d102/f128 0 2024-04-25T01:36:05.313 INFO:tasks.workunit.client.0.smithi033.stdout:7/992: truncate d4/d2e/ff 493176 0 2024-04-25T01:36:05.314 INFO:tasks.workunit.client.0.smithi033.stdout:8/995: getdents d6/d45/d51/d5d/d6e 0 2024-04-25T01:36:05.315 INFO:tasks.workunit.client.0.smithi033.stdout:3/960: mknod d8/dd/d3d/d4f/df0/c14e 0 2024-04-25T01:36:05.316 INFO:tasks.workunit.client.0.smithi033.stdout:7/993: symlink d4/d23/l144 0 2024-04-25T01:36:05.318 INFO:tasks.workunit.client.0.smithi033.stdout:1/823: dwrite d2/d5/ffe [0,4194304] 0 2024-04-25T01:36:05.322 INFO:tasks.workunit.client.0.smithi033.stdout:8/996: truncate d6/d45/f57 681789 0 2024-04-25T01:36:05.323 INFO:tasks.workunit.client.0.smithi033.stdout:1/824: write d2/d5/d1b/d46/d5f/d63/d8f/fbc [2766804,40325] 0 2024-04-25T01:36:05.324 INFO:tasks.workunit.client.1.smithi138.stdout:3/560: dwrite db/df/d17/f2e [0,4194304] 0 2024-04-25T01:36:05.326 INFO:tasks.workunit.client.0.smithi033.stdout:7/994: rename d4/d2e/d11c/da9/db3 to d4/d2e/d50/d145 0 2024-04-25T01:36:05.330 INFO:tasks.workunit.client.0.smithi033.stdout:8/997: creat d6/d45/d52/dcc/f16a x:0 0 0 2024-04-25T01:36:05.330 INFO:tasks.workunit.client.0.smithi033.stdout:1/825: chown d2/d5/d75/l51 2932022 1 2024-04-25T01:36:05.331 INFO:tasks.workunit.client.0.smithi033.stdout:4/993: dread d6/d14/d35/d44/d45/d46/d57/d70/d128/f135 [0,4194304] 0 2024-04-25T01:36:05.334 INFO:tasks.workunit.client.1.smithi138.stdout:7/485: dread da/de/d13/f24 [0,4194304] 0 2024-04-25T01:36:05.335 INFO:tasks.workunit.client.1.smithi138.stdout:3/561: dwrite db/fe [8388608,4194304] 0 2024-04-25T01:36:05.338 INFO:tasks.workunit.client.0.smithi033.stdout:1/826: getdents d2/d7f/db0/dd6 0 2024-04-25T01:36:05.338 INFO:tasks.workunit.client.0.smithi033.stdout:8/998: getdents d6/db 0 2024-04-25T01:36:05.341 INFO:tasks.workunit.client.0.smithi033.stdout:1/827: mkdir d2/d5/d1b/d46/db1/d109 0 2024-04-25T01:36:05.343 INFO:tasks.workunit.client.0.smithi033.stdout:8/999: rename d6/d45/d51/d5d/f81 to d6/d45/d52/d2e/d95/f16b 0 2024-04-25T01:36:05.344 INFO:tasks.workunit.client.0.smithi033.stdout:1/828: write d2/d5/d1b/d46/d5f/d63/d9d/fd1 [1320732,13141] 0 2024-04-25T01:36:05.345 INFO:tasks.workunit.client.0.smithi033.stdout:4/994: dwrite d6/d14/d35/d44/d45/d46/d57/d70/d87/f100 [0,4194304] 0 2024-04-25T01:36:05.348 INFO:tasks.workunit.client.1.smithi138.stdout:7/486: write da/de/d1b/d49/d81/f4c [155707,103485] 0 2024-04-25T01:36:05.348 INFO:tasks.workunit.client.0.smithi033.stdout:1/829: stat d2/d2e/d49/dba 0 2024-04-25T01:36:05.349 INFO:tasks.workunit.client.0.smithi033.stdout:4/995: rmdir d6/d14/d35/d44/dfe 39 2024-04-25T01:36:05.350 INFO:tasks.workunit.client.1.smithi138.stdout:7/487: rmdir da/de/d13/d21/d6f 39 2024-04-25T01:36:05.352 INFO:tasks.workunit.client.0.smithi033.stdout:1/830: mkdir d2/d11/dc9/d10a 0 2024-04-25T01:36:05.352 INFO:tasks.workunit.client.1.smithi138.stdout:3/562: dwrite db/d14/d40/d41/fa5 [0,4194304] 0 2024-04-25T01:36:05.353 INFO:tasks.workunit.client.1.smithi138.stdout:7/488: truncate da/de/f27 83599 0 2024-04-25T01:36:05.354 INFO:tasks.workunit.client.1.smithi138.stdout:3/563: write db/df/d17/d5d/fa8 [4761890,60878] 0 2024-04-25T01:36:05.356 INFO:tasks.workunit.client.0.smithi033.stdout:1/831: read d2/d11/d28/f7d [304314,104167] 0 2024-04-25T01:36:05.357 INFO:tasks.workunit.client.1.smithi138.stdout:7/489: rename da/de/f1a to da/de/d1b/fa4 0 2024-04-25T01:36:05.366 INFO:tasks.workunit.client.0.smithi033.stdout:1/832: mknod d2/d2e/d49/dba/c10b 0 2024-04-25T01:36:05.369 INFO:tasks.workunit.client.0.smithi033.stdout:3/961: dwrite d8/dd/d3d/d63/d7c/d9d/fb6 [0,4194304] 0 2024-04-25T01:36:05.371 INFO:tasks.workunit.client.0.smithi033.stdout:2/932: truncate da/d27/d31/d40/db1/dbf/dd5/ff9 1152483 0 2024-04-25T01:36:05.372 INFO:tasks.workunit.client.0.smithi033.stdout:1/833: mknod d2/d5/d6c/d94/c10c 0 2024-04-25T01:36:05.373 INFO:tasks.workunit.client.0.smithi033.stdout:1/834: fsync d2/d11/f30 0 2024-04-25T01:36:05.374 INFO:tasks.workunit.client.0.smithi033.stdout:3/962: symlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/d13c/l14f 0 2024-04-25T01:36:05.376 INFO:tasks.workunit.client.0.smithi033.stdout:1/835: write d2/d5/d1b/d46/d5f/d63/d64/f108 [618238,116261] 0 2024-04-25T01:36:05.377 INFO:tasks.workunit.client.0.smithi033.stdout:3/963: chown d8/dff/c144 60898 1 2024-04-25T01:36:05.378 INFO:tasks.workunit.client.0.smithi033.stdout:2/933: dwrite da/d8d/db0/fd0 [0,4194304] 0 2024-04-25T01:36:05.380 INFO:tasks.workunit.client.1.smithi138.stdout:8/544: sync 2024-04-25T01:36:05.381 INFO:tasks.workunit.client.1.smithi138.stdout:7/490: write da/de/d1b/d4b/f55 [3905220,11586] 0 2024-04-25T01:36:05.382 INFO:tasks.workunit.client.1.smithi138.stdout:7/491: write da/de/d1b/f33 [420572,41982] 0 2024-04-25T01:36:05.385 INFO:tasks.workunit.client.0.smithi033.stdout:3/964: unlink d8/dd/d1d/dd1/c119 0 2024-04-25T01:36:05.386 INFO:tasks.workunit.client.0.smithi033.stdout:1/836: dwrite d2/d5/d1b/d46/d5f/d63/d64/fa5 [0,4194304] 0 2024-04-25T01:36:05.391 INFO:tasks.workunit.client.0.smithi033.stdout:7/995: write d4/d2e/d50/d145/fef [792886,36100] 0 2024-04-25T01:36:05.392 INFO:tasks.workunit.client.0.smithi033.stdout:3/965: fsync d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d86/d92/f75 0 2024-04-25T01:36:05.393 INFO:tasks.workunit.client.1.smithi138.stdout:7/492: mkdir da/de/d13/d21/da5 0 2024-04-25T01:36:05.394 INFO:tasks.workunit.client.1.smithi138.stdout:3/564: dread db/df/d19/f36 [0,4194304] 0 2024-04-25T01:36:05.396 INFO:tasks.workunit.client.1.smithi138.stdout:3/565: chown db/df/d17/d23/f67 8510711 1 2024-04-25T01:36:05.397 INFO:tasks.workunit.client.1.smithi138.stdout:7/493: creat da/de/d13/d30/d59/d7b/d99/fa6 x:0 0 0 2024-04-25T01:36:05.399 INFO:tasks.workunit.client.1.smithi138.stdout:3/566: symlink db/df/d17/d23/lb1 0 2024-04-25T01:36:05.400 INFO:tasks.workunit.client.1.smithi138.stdout:7/494: symlink da/de/d1b/d49/d8a/d94/d9d/la7 0 2024-04-25T01:36:05.401 INFO:tasks.workunit.client.0.smithi033.stdout:3/966: rename d8/dd/d3d/d4f/da4 to d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/d150 0 2024-04-25T01:36:05.401 INFO:tasks.workunit.client.1.smithi138.stdout:7/495: fsync da/de/d13/d21/f7e 0 2024-04-25T01:36:05.402 INFO:tasks.workunit.client.1.smithi138.stdout:3/567: creat db/d14/d40/d41/dad/fb2 x:0 0 0 2024-04-25T01:36:05.403 INFO:tasks.workunit.client.1.smithi138.stdout:7/496: rmdir da/de/d1b/d34 39 2024-04-25T01:36:05.403 INFO:tasks.workunit.client.0.smithi033.stdout:4/996: write d6/d14/d1d/d5f/fb3 [1713673,19050] 0 2024-04-25T01:36:05.405 INFO:tasks.workunit.client.0.smithi033.stdout:3/967: creat d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/d13c/f151 x:0 0 0 2024-04-25T01:36:05.406 INFO:tasks.workunit.client.0.smithi033.stdout:4/997: write d6/d14/d35/d44/d45/d46/d57/d70/d87/f100 [4925230,127034] 0 2024-04-25T01:36:05.407 INFO:tasks.workunit.client.1.smithi138.stdout:7/497: rename c2 to da/de/d13/d21/d6f/d8d/ca8 0 2024-04-25T01:36:05.409 INFO:tasks.workunit.client.0.smithi033.stdout:4/998: unlink d6/ffa 0 2024-04-25T01:36:05.410 INFO:tasks.workunit.client.0.smithi033.stdout:4/999: chown d6/d14/d6d/le5 48443 1 2024-04-25T01:36:05.411 INFO:tasks.workunit.client.1.smithi138.stdout:7/498: dread da/de/d13/d30/f65 [0,4194304] 0 2024-04-25T01:36:05.411 INFO:tasks.workunit.client.0.smithi033.stdout:3/968: rename d8/dd/d1d/f93 to d8/dd/d3d/d63/d7c/ded/dbf/dc5/f152 0 2024-04-25T01:36:05.413 INFO:tasks.workunit.client.0.smithi033.stdout:3/969: mknod d8/dd/d1d/c153 0 2024-04-25T01:36:05.414 INFO:tasks.workunit.client.0.smithi033.stdout:3/970: fdatasync d8/dd/d3d/d4f/f69 0 2024-04-25T01:36:05.414 INFO:tasks.workunit.client.0.smithi033.stdout:3/971: chown d8/dd/d3d 104258488 1 2024-04-25T01:36:05.419 INFO:tasks.workunit.client.0.smithi033.stdout:3/972: mknod d8/dd/d3d/d63/d7c/ded/dc7/ddf/d10b/d12d/c154 0 2024-04-25T01:36:05.420 INFO:tasks.workunit.client.1.smithi138.stdout:7/499: dwrite da/de/d13/d30/d59/d7b/d99/fa0 [0,4194304] 0 2024-04-25T01:36:05.421 INFO:tasks.workunit.client.0.smithi033.stdout:3/973: creat d8/dd/d3d/f155 x:0 0 0 2024-04-25T01:36:05.425 INFO:tasks.workunit.client.0.smithi033.stdout:1/837: write d2/d5/d1b/dd2/feb [607883,102652] 0 2024-04-25T01:36:05.430 INFO:tasks.workunit.client.1.smithi138.stdout:7/500: dwrite da/de/d1b/fa3 [0,4194304] 0 2024-04-25T01:36:05.431 INFO:tasks.workunit.client.1.smithi138.stdout:3/568: dread db/df/d19/d1a/f21 [0,4194304] 0 2024-04-25T01:36:05.432 INFO:tasks.workunit.client.1.smithi138.stdout:8/545: dwrite d3/d16/d88/da/de/f10 [4194304,4194304] 0 2024-04-25T01:36:05.433 INFO:tasks.workunit.client.0.smithi033.stdout:2/934: dwrite da/d2c/f33 [4194304,4194304] 0 2024-04-25T01:36:05.435 INFO:tasks.workunit.client.0.smithi033.stdout:7/996: dwrite d4/f7a [0,4194304] 0 2024-04-25T01:36:05.437 INFO:tasks.workunit.client.0.smithi033.stdout:1/838: dwrite d2/f29 [0,4194304] 0 2024-04-25T01:36:05.438 INFO:tasks.workunit.client.1.smithi138.stdout:8/546: read d3/f1e [2093136,110584] 0 2024-04-25T01:36:05.445 INFO:tasks.workunit.client.0.smithi033.stdout:1/839: rmdir d2/d5/d1b/d46/d5f 39 2024-04-25T01:36:05.446 INFO:tasks.workunit.client.0.smithi033.stdout:2/935: getdents da/d50/d6a/d43/d6c/d10b 0 2024-04-25T01:36:05.446 INFO:tasks.workunit.client.1.smithi138.stdout:7/501: dread da/de/d13/d30/f80 [0,4194304] 0 2024-04-25T01:36:05.448 INFO:tasks.workunit.client.0.smithi033.stdout:1/840: fdatasync d2/d5/d1b/ffc 0 2024-04-25T01:36:05.449 INFO:tasks.workunit.client.1.smithi138.stdout:7/502: truncate da/de/d1b/f33 4502626 0 2024-04-25T01:36:05.452 INFO:tasks.workunit.client.1.smithi138.stdout:3/569: dwrite db/df/d17/d5d/f7f [4194304,4194304] 0 2024-04-25T01:36:05.453 INFO:tasks.workunit.client.0.smithi033.stdout:3/974: mkdir d8/dd/d3d/d63/d7c/ded/dbf/dc5/d156 0 2024-04-25T01:36:05.454 INFO:tasks.workunit.client.0.smithi033.stdout:7/997: dwrite d4/d2e/d6d/f140 [0,4194304] 0 2024-04-25T01:36:05.454 INFO:tasks.workunit.client.0.smithi033.stdout:1/841: symlink d2/d5/d1b/d46/db1/d109/l10d 0 2024-04-25T01:36:05.460 INFO:tasks.workunit.client.0.smithi033.stdout:3/975: mknod d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/c157 0 2024-04-25T01:36:05.463 INFO:tasks.workunit.client.1.smithi138.stdout:7/503: dwrite da/de/f27 [0,4194304] 0 2024-04-25T01:36:05.463 INFO:tasks.workunit.client.0.smithi033.stdout:3/976: dread - d8/f118 zero size 2024-04-25T01:36:05.466 INFO:tasks.workunit.client.1.smithi138.stdout:3/570: dread db/df/d19/f93 [0,4194304] 0 2024-04-25T01:36:05.468 INFO:tasks.workunit.client.1.smithi138.stdout:7/504: creat da/de/fa9 x:0 0 0 2024-04-25T01:36:05.469 INFO:tasks.workunit.client.0.smithi033.stdout:7/998: dread d4/d2e/d11c/d1f/f71 [0,4194304] 0 2024-04-25T01:36:05.469 INFO:tasks.workunit.client.0.smithi033.stdout:3/977: mkdir d8/dd/d3d/d63/d7c/ded/dc7/ddf/d10b/d12d/d158 0 2024-04-25T01:36:05.470 INFO:tasks.workunit.client.1.smithi138.stdout:3/571: mkdir db/db3 0 2024-04-25T01:36:05.470 INFO:tasks.workunit.client.0.smithi033.stdout:1/842: dwrite d2/d5/d1b/d46/d5f/d63/d64/ff7 [0,4194304] 0 2024-04-25T01:36:05.471 INFO:tasks.workunit.client.1.smithi138.stdout:3/572: mkdir db/d9b/db4 0 2024-04-25T01:36:05.473 INFO:tasks.workunit.client.0.smithi033.stdout:1/843: truncate d2/d2e/d49/fda 919447 0 2024-04-25T01:36:05.474 INFO:tasks.workunit.client.0.smithi033.stdout:7/999: truncate d4/f41 2722500 0 2024-04-25T01:36:05.475 INFO:tasks.workunit.client.1.smithi138.stdout:7/505: dwrite da/de/d1b/d4b/f9e [0,4194304] 0 2024-04-25T01:36:05.475 INFO:tasks.workunit.client.0.smithi033.stdout:1/844: mkdir d2/d5/d75/d59/d10e 0 2024-04-25T01:36:05.477 INFO:tasks.workunit.client.1.smithi138.stdout:7/506: write da/f6a [1176028,107226] 0 2024-04-25T01:36:05.477 INFO:tasks.workunit.client.0.smithi033.stdout:1/845: truncate d2/d5/d6c/fcf 569180 0 2024-04-25T01:36:05.478 INFO:tasks.workunit.client.0.smithi033.stdout:1/846: dread - d2/d5/d75/d59/fae zero size 2024-04-25T01:36:05.479 INFO:tasks.workunit.client.1.smithi138.stdout:7/507: read da/f7a [663677,60202] 0 2024-04-25T01:36:05.483 INFO:tasks.workunit.client.1.smithi138.stdout:3/573: dread db/df/d17/d23/d34/f80 [0,4194304] 0 2024-04-25T01:36:05.484 INFO:tasks.workunit.client.1.smithi138.stdout:7/508: creat da/de/d1b/d49/d81/d51/faa x:0 0 0 2024-04-25T01:36:05.489 INFO:tasks.workunit.client.1.smithi138.stdout:7/509: link da/f7a da/de/d13/d30/d59/d7b/d99/fab 0 2024-04-25T01:36:05.494 INFO:tasks.workunit.client.0.smithi033.stdout:2/936: dwrite da/d50/d6a/d43/f62 [0,4194304] 0 2024-04-25T01:36:05.495 INFO:tasks.workunit.client.1.smithi138.stdout:7/510: dwrite da/de/d1b/d34/f47 [0,4194304] 0 2024-04-25T01:36:05.496 INFO:tasks.workunit.client.0.smithi033.stdout:2/937: mknod da/d27/d31/d40/d77/c131 0 2024-04-25T01:36:05.499 INFO:tasks.workunit.client.0.smithi033.stdout:2/938: mknod da/d5b/d72/c132 0 2024-04-25T01:36:05.502 INFO:tasks.workunit.client.1.smithi138.stdout:7/511: dwrite da/de/d1b/d4b/d77/f91 [0,4194304] 0 2024-04-25T01:36:05.508 INFO:tasks.workunit.client.0.smithi033.stdout:2/939: mknod da/d27/d31/d40/d8a/dbb/dd6/c133 0 2024-04-25T01:36:05.509 INFO:tasks.workunit.client.1.smithi138.stdout:7/512: unlink da/l11 0 2024-04-25T01:36:05.511 INFO:tasks.workunit.client.0.smithi033.stdout:2/940: unlink da/d50/d6a/d43/c106 0 2024-04-25T01:36:05.511 INFO:tasks.workunit.client.1.smithi138.stdout:8/547: write d3/d16/d88/da/f43 [5208327,67] 0 2024-04-25T01:36:05.514 INFO:tasks.workunit.client.0.smithi033.stdout:2/941: unlink da/d27/d31/c67 0 2024-04-25T01:36:05.515 INFO:tasks.workunit.client.1.smithi138.stdout:7/513: read da/f63 [4704,45171] 0 2024-04-25T01:36:05.516 INFO:tasks.workunit.client.0.smithi033.stdout:2/942: fsync da/f18 0 2024-04-25T01:36:05.517 INFO:tasks.workunit.client.1.smithi138.stdout:7/514: write da/de/d13/d21/f41 [1991860,92904] 0 2024-04-25T01:36:05.519 INFO:tasks.workunit.client.1.smithi138.stdout:8/548: dwrite d3/d16/f9b [0,4194304] 0 2024-04-25T01:36:05.521 INFO:tasks.workunit.client.1.smithi138.stdout:3/574: truncate db/df/d17/d20/f26 908819 0 2024-04-25T01:36:05.522 INFO:tasks.workunit.client.1.smithi138.stdout:8/549: read d3/d16/d88/da/db/d15/d20/f5a [358037,38947] 0 2024-04-25T01:36:05.524 INFO:tasks.workunit.client.1.smithi138.stdout:8/550: readlink d3/d16/d88/l75 0 2024-04-25T01:36:05.524 INFO:tasks.workunit.client.1.smithi138.stdout:7/515: chown da/de/d13/d30/f42 14064524 1 2024-04-25T01:36:05.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/575: write db/d14/d40/d41/f7c [104192,119092] 0 2024-04-25T01:36:05.526 INFO:tasks.workunit.client.1.smithi138.stdout:3/576: dread - db/d14/d7a/f92 zero size 2024-04-25T01:36:05.526 INFO:tasks.workunit.client.0.smithi033.stdout:2/943: dwrite da/d27/d31/d40/d8a/dbb/f128 [0,4194304] 0 2024-04-25T01:36:05.527 INFO:tasks.workunit.client.1.smithi138.stdout:3/577: fdatasync db/fa2 0 2024-04-25T01:36:05.529 INFO:tasks.workunit.client.1.smithi138.stdout:7/516: mknod da/de/d13/d30/d59/d7b/cac 0 2024-04-25T01:36:05.530 INFO:tasks.workunit.client.1.smithi138.stdout:8/551: creat d3/d16/d88/da/fad x:0 0 0 2024-04-25T01:36:05.531 INFO:tasks.workunit.client.1.smithi138.stdout:3/578: readlink db/l3b 0 2024-04-25T01:36:05.532 INFO:tasks.workunit.client.0.smithi033.stdout:2/944: mknod da/d27/d31/d40/c134 0 2024-04-25T01:36:05.532 INFO:tasks.workunit.client.1.smithi138.stdout:7/517: fsync da/de/d13/f24 0 2024-04-25T01:36:05.535 INFO:tasks.workunit.client.1.smithi138.stdout:3/579: symlink db/d14/lb5 0 2024-04-25T01:36:05.539 INFO:tasks.workunit.client.0.smithi033.stdout:2/945: dread da/f74 [0,4194304] 0 2024-04-25T01:36:05.540 INFO:tasks.workunit.client.1.smithi138.stdout:7/518: symlink da/de/d1b/d49/d8a/lad 0 2024-04-25T01:36:05.541 INFO:tasks.workunit.client.1.smithi138.stdout:3/580: symlink db/df/lb6 0 2024-04-25T01:36:05.543 INFO:tasks.workunit.client.0.smithi033.stdout:2/946: mknod da/c135 0 2024-04-25T01:36:05.543 INFO:tasks.workunit.client.1.smithi138.stdout:3/581: creat db/d9b/fb7 x:0 0 0 2024-04-25T01:36:05.544 INFO:tasks.workunit.client.0.smithi033.stdout:2/947: write f8 [759227,4383] 0 2024-04-25T01:36:05.544 INFO:tasks.workunit.client.1.smithi138.stdout:3/582: readlink db/df/d17/d23/lb1 0 2024-04-25T01:36:05.545 INFO:tasks.workunit.client.1.smithi138.stdout:3/583: dread - db/df/d17/f4a zero size 2024-04-25T01:36:05.546 INFO:tasks.workunit.client.0.smithi033.stdout:2/948: read da/d27/d31/f4f [24515,103817] 0 2024-04-25T01:36:05.547 INFO:tasks.workunit.client.1.smithi138.stdout:3/584: chown db/df/l7e 14 1 2024-04-25T01:36:05.548 INFO:tasks.workunit.client.0.smithi033.stdout:2/949: write da/d27/f7a [4016562,18999] 0 2024-04-25T01:36:05.550 INFO:tasks.workunit.client.1.smithi138.stdout:8/552: write d3/d16/d88/da/db/d15/f83 [33373,32786] 0 2024-04-25T01:36:05.551 INFO:tasks.workunit.client.0.smithi033.stdout:2/950: creat da/d50/d6a/df6/f136 x:0 0 0 2024-04-25T01:36:05.552 INFO:tasks.workunit.client.0.smithi033.stdout:2/951: truncate f7 1119931 0 2024-04-25T01:36:05.555 INFO:tasks.workunit.client.1.smithi138.stdout:3/585: dwrite db/d14/d40/d41/f89 [0,4194304] 0 2024-04-25T01:36:05.555 INFO:tasks.workunit.client.0.smithi033.stdout:2/952: rmdir da/d50/d6a/d43 39 2024-04-25T01:36:05.556 INFO:tasks.workunit.client.0.smithi033.stdout:2/953: chown da/d50/d6a/d43/d6c 36 1 2024-04-25T01:36:05.557 INFO:tasks.workunit.client.0.smithi033.stdout:2/954: write da/ff [1652319,102383] 0 2024-04-25T01:36:05.558 INFO:tasks.workunit.client.1.smithi138.stdout:7/519: chown da/de/d13/d30/d59/d7b/d99/fa6 4561970 1 2024-04-25T01:36:05.559 INFO:tasks.workunit.client.1.smithi138.stdout:7/520: chown da/de/d1b/d4b/d77/l86 2380 1 2024-04-25T01:36:05.560 INFO:tasks.workunit.client.1.smithi138.stdout:8/553: unlink d3/d16/d1f/d50/f81 0 2024-04-25T01:36:05.562 INFO:tasks.workunit.client.1.smithi138.stdout:3/586: rmdir db/d14/d40/d41 39 2024-04-25T01:36:05.562 INFO:tasks.workunit.client.0.smithi033.stdout:2/955: truncate da/d50/d6a/d43/f122 2063779 0 2024-04-25T01:36:05.564 INFO:tasks.workunit.client.1.smithi138.stdout:7/521: write da/de/d13/f2d [1704188,79480] 0 2024-04-25T01:36:05.564 INFO:tasks.workunit.client.1.smithi138.stdout:8/554: unlink d3/d16/d1f/d2a/l4a 0 2024-04-25T01:36:05.565 INFO:tasks.workunit.client.0.smithi033.stdout:2/956: truncate da/d27/d31/d40/d8a/f53 204762 0 2024-04-25T01:36:05.582 INFO:tasks.workunit.client.1.smithi138.stdout:3/587: dread db/df/d17/d23/d34/f6d [0,4194304] 0 2024-04-25T01:36:05.588 INFO:tasks.workunit.client.1.smithi138.stdout:8/555: dread d3/d16/d88/da/de/f10 [0,4194304] 0 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.628+0000 7ff9b1e52700 1 -- 172.21.15.33:0/3800822171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fe080 msgr2=0x7ff9ac0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.628+0000 7ff9b1e52700 1 --2- 172.21.15.33:0/3800822171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fe080 0x7ff9ac0fe4f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0004940 tx=0x7ff9a0009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.628+0000 7ff9b1e52700 1 -- 172.21.15.33:0/3800822171 shutdown_connections 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.628+0000 7ff9b1e52700 1 --2- 172.21.15.33:0/3800822171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fe080 0x7ff9ac0fe4f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.628+0000 7ff9b1e52700 1 --2- 172.21.15.33:0/3800822171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac0fce80 0x7ff9ac0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.628+0000 7ff9b1e52700 1 -- 172.21.15.33:0/3800822171 >> 172.21.15.33:0/3800822171 conn(0x7ff9ac0f8590 msgr2=0x7ff9ac0fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.628+0000 7ff9b1e52700 1 -- 172.21.15.33:0/3800822171 shutdown_connections 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.628+0000 7ff9b1e52700 1 -- 172.21.15.33:0/3800822171 wait complete. 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.629+0000 7ff9b1e52700 1 Processor -- start 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.629+0000 7ff9b1e52700 1 -- start start 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.629+0000 7ff9b1e52700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fce80 0x7ff9ac10d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.629+0000 7ff9b1e52700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac0fe080 0x7ff9ac10df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:05.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.629+0000 7ff9b1e52700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac10e500 con 0x7ff9ac0fce80 2024-04-25T01:36:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.629+0000 7ff9b1e52700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac10ed50 con 0x7ff9ac0fe080 2024-04-25T01:36:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.629+0000 7ff9ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fce80 0x7ff9ac10d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.630+0000 7ff9ab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fce80 0x7ff9ac10d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45058/0 (socket says 172.21.15.33:45058) 2024-04-25T01:36:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.630+0000 7ff9ab7fe700 1 -- 172.21.15.33:0/74722821 learned_addr learned my addr 172.21.15.33:0/74722821 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.630+0000 7ff9aaffd700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac0fe080 0x7ff9ac10df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.630+0000 7ff9ab7fe700 1 -- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac0fe080 msgr2=0x7ff9ac10df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:05.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.630+0000 7ff9ab7fe700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac0fe080 0x7ff9ac10df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:05.608 INFO:tasks.workunit.client.0.smithi033.stdout:9/940: sync 2024-04-25T01:36:05.608 INFO:tasks.workunit.client.0.smithi033.stdout:5/992: sync 2024-04-25T01:36:05.608 INFO:tasks.workunit.client.0.smithi033.stdout:0/974: sync 2024-04-25T01:36:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.630+0000 7ff9ab7fe700 1 -- 172.21.15.33:0/74722821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0005040 con 0x7ff9ac0fce80 2024-04-25T01:36:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.630+0000 7ff9ab7fe700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fce80 0x7ff9ac10d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00eb40 tx=0x7ff99c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.631+0000 7ff993fff700 1 -- 172.21.15.33:0/74722821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff99c007cc0 con 0x7ff9ac0fce80 2024-04-25T01:36:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.631+0000 7ff993fff700 1 -- 172.21.15.33:0/74722821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff99c00f040 con 0x7ff9ac0fce80 2024-04-25T01:36:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.631+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac10efd0 con 0x7ff9ac0fce80 2024-04-25T01:36:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.631+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac10f510 con 0x7ff9ac0fce80 2024-04-25T01:36:05.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.631+0000 7ff993fff700 1 -- 172.21.15.33:0/74722821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff99c00caa0 con 0x7ff9ac0fce80 2024-04-25T01:36:05.610 INFO:tasks.workunit.client.0.smithi033.stdout:0/975: chown d7/da/d80 56179674 1 2024-04-25T01:36:05.610 INFO:tasks.workunit.client.0.smithi033.stdout:9/941: readlink d0/d2/dd/d5f/d11b/lb8 0 2024-04-25T01:36:05.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.632+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac05f5c0 con 0x7ff9ac0fce80 2024-04-25T01:36:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.633+0000 7ff993fff700 1 -- 172.21.15.33:0/74722821 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7ff99c00cc00 con 0x7ff9ac0fce80 2024-04-25T01:36:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.633+0000 7ff993fff700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7ff99406b790 0x7ff99406dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:05.611 INFO:tasks.workunit.client.0.smithi033.stdout:0/976: write d7/d2d/d4d/d9c/d61/f120 [3066293,89745] 0 2024-04-25T01:36:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.633+0000 7ff993fff700 1 -- 172.21.15.33:0/74722821 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff99c089f50 con 0x7ff9ac0fce80 2024-04-25T01:36:05.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.634+0000 7ff9aaffd700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7ff99406b790 0x7ff99406dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:05.612 INFO:tasks.workunit.client.0.smithi033.stdout:5/993: creat d1c/d2a/d50/d52/da6/d3e/d70/dae/f141 x:0 0 0 2024-04-25T01:36:05.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.634+0000 7ff9aaffd700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7ff99406b790 0x7ff99406dc40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009b30 tx=0x7ff9a0009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:05.614 INFO:tasks.workunit.client.0.smithi033.stdout:9/942: rename d0/d2/d43/d73/dab/dad/d133 to d0/d11/da1/d146 0 2024-04-25T01:36:05.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:05.640+0000 7ff993fff700 1 -- 172.21.15.33:0/74722821 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff99c057590 con 0x7ff9ac0fce80 2024-04-25T01:36:05.621 INFO:tasks.workunit.client.0.smithi033.stdout:9/943: creat d0/d2/dd/d5f/d11b/d13c/df9/f147 x:0 0 0 2024-04-25T01:36:05.621 INFO:tasks.workunit.client.0.smithi033.stdout:5/994: write f17 [1411258,89489] 0 2024-04-25T01:36:05.623 INFO:tasks.workunit.client.0.smithi033.stdout:5/995: chown d1c/d2a/d50/d52/d10c 66801 1 2024-04-25T01:36:05.625 INFO:tasks.workunit.client.0.smithi033.stdout:0/977: dread d7/da/df/f99 [0,4194304] 0 2024-04-25T01:36:05.626 INFO:tasks.workunit.client.0.smithi033.stdout:5/996: dread - d1c/d2a/d50/d52/fd9 zero size 2024-04-25T01:36:05.636 INFO:tasks.workunit.client.0.smithi033.stdout:0/978: creat d7/da/df/d14/f151 x:0 0 0 2024-04-25T01:36:05.636 INFO:tasks.workunit.client.0.smithi033.stdout:5/997: fsync d1c/d2a/d50/d52/da6/d4a/f4c 0 2024-04-25T01:36:05.636 INFO:tasks.workunit.client.0.smithi033.stdout:9/944: write d0/d2/dd/d5f/d11b/f98 [381698,107708] 0 2024-04-25T01:36:05.657 INFO:tasks.workunit.client.0.smithi033.stdout:0/979: creat d7/d2d/d64/d51/f152 x:0 0 0 2024-04-25T01:36:05.658 INFO:tasks.workunit.client.0.smithi033.stdout:5/998: symlink d1c/d2a/d50/d52/d10c/d12c/l142 0 2024-04-25T01:36:05.659 INFO:tasks.workunit.client.0.smithi033.stdout:5/999: chown d1c/d2a/d50/d52/da6/d3e/d70/d7a/d7e/c9a 3 1 2024-04-25T01:36:05.660 INFO:tasks.workunit.client.0.smithi033.stdout:9/945: rename d0/d2/d43/d4b/dc5/de0/d7e/c114 to d0/d2/d43/d4b/dc3/d128/c148 0 2024-04-25T01:36:05.661 INFO:tasks.workunit.client.0.smithi033.stdout:0/980: symlink d7/d2d/d43/d44/l153 0 2024-04-25T01:36:05.663 INFO:tasks.workunit.client.0.smithi033.stdout:9/946: fsync d0/d2/d43/d4b/f5b 0 2024-04-25T01:36:05.663 INFO:tasks.workunit.client.0.smithi033.stdout:0/981: read d7/d2d/d4d/d9c/d61/f12f [751564,60157] 0 2024-04-25T01:36:05.668 INFO:tasks.workunit.client.0.smithi033.stdout:9/947: rename d0/d2/dd/d20/f86 to d0/d2/d43/d73/dab/f149 0 2024-04-25T01:36:05.669 INFO:tasks.workunit.client.0.smithi033.stdout:0/982: mknod d7/c154 0 2024-04-25T01:36:05.671 INFO:tasks.workunit.client.0.smithi033.stdout:9/948: rmdir d0/d2/dd/d20/d4e 39 2024-04-25T01:36:05.675 INFO:tasks.workunit.client.0.smithi033.stdout:0/983: creat d7/d2d/d43/d44/dd4/f155 x:0 0 0 2024-04-25T01:36:05.678 INFO:tasks.workunit.client.1.smithi138.stdout:3/588: symlink db/df/d17/d23/d88/lb8 0 2024-04-25T01:36:05.683 INFO:tasks.workunit.client.1.smithi138.stdout:7/522: dwrite da/de/d13/d30/f80 [0,4194304] 0 2024-04-25T01:36:05.684 INFO:tasks.workunit.client.1.smithi138.stdout:3/589: dread db/df/d19/d1a/f44 [0,4194304] 0 2024-04-25T01:36:05.687 INFO:tasks.workunit.client.1.smithi138.stdout:8/556: write d3/f3c [3282270,51576] 0 2024-04-25T01:36:05.688 INFO:tasks.workunit.client.1.smithi138.stdout:3/590: creat db/df/d19/d1a/d9e/fb9 x:0 0 0 2024-04-25T01:36:05.689 INFO:tasks.workunit.client.1.smithi138.stdout:7/523: dread da/de/d13/f24 [4194304,4194304] 0 2024-04-25T01:36:05.690 INFO:tasks.workunit.client.1.smithi138.stdout:8/557: mkdir d3/d44/dae 0 2024-04-25T01:36:05.691 INFO:tasks.workunit.client.1.smithi138.stdout:3/591: rmdir db/d14 39 2024-04-25T01:36:05.694 INFO:tasks.workunit.client.1.smithi138.stdout:3/592: getdents db/db3 0 2024-04-25T01:36:05.694 INFO:tasks.workunit.client.0.smithi033.stdout:0/984: truncate d7/d2d/d43/d44/f63 552096 0 2024-04-25T01:36:05.695 INFO:tasks.workunit.client.0.smithi033.stdout:9/949: dwrite d0/d2/dd/d20/f38 [4194304,4194304] 0 2024-04-25T01:36:05.697 INFO:tasks.workunit.client.0.smithi033.stdout:0/985: symlink d7/d2d/d4d/d9c/dcf/l156 0 2024-04-25T01:36:05.697 INFO:tasks.workunit.client.1.smithi138.stdout:3/593: write db/df/d17/d5d/f6b [1568938,14327] 0 2024-04-25T01:36:05.698 INFO:tasks.workunit.client.1.smithi138.stdout:7/524: dwrite da/f6a [0,4194304] 0 2024-04-25T01:36:05.701 INFO:tasks.workunit.client.1.smithi138.stdout:3/594: creat db/df/d17/d23/d34/fba x:0 0 0 2024-04-25T01:36:05.702 INFO:tasks.workunit.client.0.smithi033.stdout:9/950: mkdir d0/d2/dd/dba/dc1/d136/d14a 0 2024-04-25T01:36:05.703 INFO:tasks.workunit.client.1.smithi138.stdout:7/525: mkdir da/de/d1b/d49/d81/d51/d6e/da2/dae 0 2024-04-25T01:36:05.706 INFO:tasks.workunit.client.1.smithi138.stdout:8/558: dwrite d3/d16/d88/da/db/f11 [0,4194304] 0 2024-04-25T01:36:05.707 INFO:tasks.workunit.client.1.smithi138.stdout:7/526: write da/de/d13/f24 [7388143,43260] 0 2024-04-25T01:36:05.712 INFO:tasks.workunit.client.0.smithi033.stdout:9/951: dwrite d0/d2/dd/f51 [0,4194304] 0 2024-04-25T01:36:05.714 INFO:tasks.workunit.client.1.smithi138.stdout:3/595: dwrite db/f62 [0,4194304] 0 2024-04-25T01:36:05.714 INFO:tasks.workunit.client.0.smithi033.stdout:9/952: readlink d0/d2/dd/l4 0 2024-04-25T01:36:05.715 INFO:tasks.workunit.client.1.smithi138.stdout:8/559: creat d3/d16/d1f/d31/d82/d8a/faf x:0 0 0 2024-04-25T01:36:05.717 INFO:tasks.workunit.client.1.smithi138.stdout:7/527: mkdir da/de/d1b/d49/d81/d51/d6e/da2/dae/daf 0 2024-04-25T01:36:05.718 INFO:tasks.workunit.client.0.smithi033.stdout:9/953: creat d0/d2/dd/d5f/d11b/d125/f14b x:0 0 0 2024-04-25T01:36:05.720 INFO:tasks.workunit.client.1.smithi138.stdout:3/596: dread db/f62 [0,4194304] 0 2024-04-25T01:36:05.721 INFO:tasks.workunit.client.1.smithi138.stdout:3/597: stat la 0 2024-04-25T01:36:05.722 INFO:tasks.workunit.client.0.smithi033.stdout:9/954: mknod d0/d2/dd/dba/c14c 0 2024-04-25T01:36:05.722 INFO:tasks.workunit.client.1.smithi138.stdout:7/528: creat da/de/d13/d30/d59/d7b/d99/fb0 x:0 0 0 2024-04-25T01:36:05.724 INFO:tasks.workunit.client.1.smithi138.stdout:3/598: chown db/df/d17/d20 0 1 2024-04-25T01:36:05.724 INFO:tasks.workunit.client.1.smithi138.stdout:8/560: dwrite d3/d44/d48/fa3 [0,4194304] 0 2024-04-25T01:36:05.726 INFO:tasks.workunit.client.0.smithi033.stdout:9/955: mkdir d0/d2/dd/df/d14d 0 2024-04-25T01:36:05.728 INFO:tasks.workunit.client.1.smithi138.stdout:7/529: chown da/de/d13/d21 719 1 2024-04-25T01:36:05.728 INFO:tasks.workunit.client.1.smithi138.stdout:3/599: symlink db/d9b/lbb 0 2024-04-25T01:36:05.731 INFO:tasks.workunit.client.1.smithi138.stdout:3/600: mknod db/d9b/cbc 0 2024-04-25T01:36:05.733 INFO:tasks.workunit.client.1.smithi138.stdout:8/561: getdents d3/d95 0 2024-04-25T01:36:05.735 INFO:tasks.workunit.client.1.smithi138.stdout:7/530: dwrite da/de/d1b/d34/f47 [0,4194304] 0 2024-04-25T01:36:05.736 INFO:tasks.workunit.client.1.smithi138.stdout:8/562: truncate d3/d16/d88/da/db/d15/d59/f97 295902 0 2024-04-25T01:36:05.739 INFO:tasks.workunit.client.1.smithi138.stdout:8/563: write d3/d16/d1f/d31/d82/f90 [647979,91110] 0 2024-04-25T01:36:05.739 INFO:tasks.workunit.client.1.smithi138.stdout:7/531: creat da/de/d13/d30/d59/d7b/d99/fb1 x:0 0 0 2024-04-25T01:36:05.741 INFO:tasks.workunit.client.1.smithi138.stdout:7/532: write da/de/d1b/d49/d81/d51/faa [694558,17715] 0 2024-04-25T01:36:05.742 INFO:tasks.workunit.client.1.smithi138.stdout:8/564: fsync d3/d16/d88/da/db/d15/d5d/f6b 0 2024-04-25T01:36:05.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:05 smithi033 ceph-mon[26353]: pgmap v18: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 66 MiB/s rd, 238 MiB/s wr, 424 op/s 2024-04-25T01:36:05.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:05 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:36:05.744 INFO:tasks.workunit.client.1.smithi138.stdout:7/533: write da/de/d13/d30/d59/d7b/d99/fb0 [311780,76217] 0 2024-04-25T01:36:05.748 INFO:tasks.workunit.client.1.smithi138.stdout:7/534: symlink da/de/lb2 0 2024-04-25T01:36:05.751 INFO:tasks.workunit.client.1.smithi138.stdout:8/565: dwrite d3/d16/d88/fab [0,4194304] 0 2024-04-25T01:36:05.753 INFO:tasks.workunit.client.0.smithi033.stdout:9/956: dwrite d0/d2/dd/d20/f7d [0,4194304] 0 2024-04-25T01:36:05.753 INFO:tasks.workunit.client.1.smithi138.stdout:8/566: stat d3/d16/d1f/d2a/d4f 0 2024-04-25T01:36:05.755 INFO:tasks.workunit.client.1.smithi138.stdout:7/535: dread da/de/d13/f2d [0,4194304] 0 2024-04-25T01:36:05.758 INFO:tasks.workunit.client.1.smithi138.stdout:8/567: creat d3/d16/d88/da/db/d15/fb0 x:0 0 0 2024-04-25T01:36:05.761 INFO:tasks.workunit.client.1.smithi138.stdout:7/536: dwrite da/de/d1b/d4b/f55 [0,4194304] 0 2024-04-25T01:36:05.765 INFO:tasks.workunit.client.1.smithi138.stdout:7/537: creat da/de/d1b/d49/d8a/fb3 x:0 0 0 2024-04-25T01:36:05.767 INFO:tasks.workunit.client.0.smithi033.stdout:9/957: write d0/d2/dd/d20/f77 [4015628,118724] 0 2024-04-25T01:36:05.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:05 smithi138 ceph-mon[32977]: pgmap v18: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 66 MiB/s rd, 238 MiB/s wr, 424 op/s 2024-04-25T01:36:05.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:05 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:36:05.770 INFO:tasks.workunit.client.0.smithi033.stdout:9/958: mknod d0/d2/d43/d4b/dc5/de0/d6e/c14e 0 2024-04-25T01:36:05.771 INFO:tasks.workunit.client.1.smithi138.stdout:7/538: symlink da/de/d13/d30/d59/d7b/d99/lb4 0 2024-04-25T01:36:05.773 INFO:tasks.workunit.client.0.smithi033.stdout:9/959: rmdir d0/d2/d43/d4b/dc5/de0/d7e 39 2024-04-25T01:36:05.778 INFO:tasks.workunit.client.1.smithi138.stdout:7/539: dwrite da/de/d1b/d49/d81/f4c [0,4194304] 0 2024-04-25T01:36:05.779 INFO:tasks.workunit.client.1.smithi138.stdout:8/568: dread d3/d16/d88/da/db/f14 [0,4194304] 0 2024-04-25T01:36:05.781 INFO:tasks.workunit.client.1.smithi138.stdout:7/540: chown da/de/d1b/d4b/d5b/l9f 94 1 2024-04-25T01:36:05.782 INFO:tasks.workunit.client.1.smithi138.stdout:8/569: dread - d3/d16/d88/da/db/d15/f72 zero size 2024-04-25T01:36:05.793 INFO:tasks.workunit.client.1.smithi138.stdout:7/541: dwrite da/de/d13/d30/d59/f78 [0,4194304] 0 2024-04-25T01:36:05.794 INFO:tasks.workunit.client.1.smithi138.stdout:8/570: dwrite d3/d16/d88/f60 [4194304,4194304] 0 2024-04-25T01:36:05.801 INFO:tasks.workunit.client.1.smithi138.stdout:7/542: creat da/de/d13/fb5 x:0 0 0 2024-04-25T01:36:05.802 INFO:tasks.workunit.client.1.smithi138.stdout:8/571: mknod d3/d44/da4/cb1 0 2024-04-25T01:36:05.803 INFO:tasks.workunit.client.1.smithi138.stdout:8/572: chown d3/d44/da4/cb1 1 1 2024-04-25T01:36:05.810 INFO:tasks.workunit.client.1.smithi138.stdout:8/573: dread d3/d16/d88/da/db/d15/d5d/f6b [4194304,4194304] 0 2024-04-25T01:36:05.811 INFO:tasks.workunit.client.1.smithi138.stdout:7/543: dwrite da/f18 [0,4194304] 0 2024-04-25T01:36:05.814 INFO:tasks.workunit.client.1.smithi138.stdout:7/544: truncate da/de/d13/d21/d6f/f79 268493 0 2024-04-25T01:36:05.816 INFO:tasks.workunit.client.1.smithi138.stdout:8/574: symlink d3/d16/d88/da/db/d15/d56/d94/lb2 0 2024-04-25T01:36:05.820 INFO:tasks.workunit.client.1.smithi138.stdout:7/545: rename da/de/d1b/d49/d8a to da/de/d1b/d4b/d5b/db6 0 2024-04-25T01:36:05.821 INFO:tasks.workunit.client.1.smithi138.stdout:8/575: symlink d3/d16/d1f/d2a/lb3 0 2024-04-25T01:36:05.823 INFO:tasks.workunit.client.1.smithi138.stdout:7/546: fdatasync da/de/d1b/d49/d81/d51/d6e/f75 0 2024-04-25T01:36:05.823 INFO:tasks.workunit.client.1.smithi138.stdout:8/576: write d3/d44/d48/fa3 [2902903,87436] 0 2024-04-25T01:36:05.827 INFO:tasks.workunit.client.1.smithi138.stdout:7/547: mkdir da/de/d1b/d49/d81/d73/db7 0 2024-04-25T01:36:05.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/577: truncate d3/d16/d88/da/db/d15/f21 5109443 0 2024-04-25T01:36:05.831 INFO:tasks.workunit.client.1.smithi138.stdout:7/548: creat da/de/d1b/d49/d81/d51/d6e/da2/dae/fb8 x:0 0 0 2024-04-25T01:36:05.835 INFO:tasks.workunit.client.1.smithi138.stdout:8/578: link d3/d95/ca0 d3/d16/d88/da/cb4 0 2024-04-25T01:36:05.836 INFO:tasks.workunit.client.1.smithi138.stdout:8/579: dread - d3/d16/d1f/d31/d82/d8a/faf zero size 2024-04-25T01:36:05.837 INFO:tasks.workunit.client.1.smithi138.stdout:7/549: rename da/de/d13/d21/d6f/f79 to da/de/d1b/d49/d81/d51/d6e/da2/dae/daf/fb9 0 2024-04-25T01:36:05.837 INFO:tasks.workunit.client.0.smithi033.stdout:3/978: sync 2024-04-25T01:36:05.838 INFO:tasks.workunit.client.0.smithi033.stdout:2/957: sync 2024-04-25T01:36:05.838 INFO:tasks.workunit.client.0.smithi033.stdout:1/847: sync 2024-04-25T01:36:05.838 INFO:tasks.workunit.client.1.smithi138.stdout:7/550: chown da/de/d13/f76 29962 1 2024-04-25T01:36:05.840 INFO:tasks.workunit.client.1.smithi138.stdout:8/580: mknod d3/d16/d88/da/db/d15/d56/d94/cb5 0 2024-04-25T01:36:05.840 INFO:tasks.workunit.client.1.smithi138.stdout:5/389: sync 2024-04-25T01:36:05.840 INFO:tasks.workunit.client.1.smithi138.stdout:6/495: sync 2024-04-25T01:36:05.841 INFO:tasks.workunit.client.0.smithi033.stdout:2/958: write da/d5b/d5e/f9f [1863840,30625] 0 2024-04-25T01:36:05.841 INFO:tasks.workunit.client.1.smithi138.stdout:7/551: write da/de/d13/f2d [3393672,37058] 0 2024-04-25T01:36:05.843 INFO:tasks.workunit.client.1.smithi138.stdout:8/581: chown d3/d16/d88/da/db/d15/d20/c7f 88 1 2024-04-25T01:36:05.844 INFO:tasks.workunit.client.1.smithi138.stdout:5/390: rmdir d4/da/d6e/d22/d45 39 2024-04-25T01:36:05.846 INFO:tasks.workunit.client.1.smithi138.stdout:8/582: write d3/d16/d88/da/d70/fa8 [975288,70256] 0 2024-04-25T01:36:05.847 INFO:tasks.workunit.client.1.smithi138.stdout:7/552: fdatasync da/de/d13/f3c 0 2024-04-25T01:36:05.849 INFO:tasks.workunit.client.1.smithi138.stdout:6/496: dwrite f6 [4194304,4194304] 0 2024-04-25T01:36:05.851 INFO:tasks.workunit.client.1.smithi138.stdout:7/553: mknod da/de/d1b/d4b/d5b/db6/d94/cba 0 2024-04-25T01:36:05.853 INFO:tasks.workunit.client.1.smithi138.stdout:8/583: creat d3/d16/d88/da/db/d15/d59/fb6 x:0 0 0 2024-04-25T01:36:05.853 INFO:tasks.workunit.client.1.smithi138.stdout:7/554: chown da/de/d13/d21 4348 1 2024-04-25T01:36:05.856 INFO:tasks.workunit.client.1.smithi138.stdout:5/391: rename d4/da/d6e/d22/d35/f56 to d4/da/d52/f7c 0 2024-04-25T01:36:05.857 INFO:tasks.workunit.client.1.smithi138.stdout:6/497: rmdir dc/d10/d28 39 2024-04-25T01:36:05.861 INFO:tasks.workunit.client.1.smithi138.stdout:6/498: chown dc/d10/d28/d80/f8b 0 1 2024-04-25T01:36:05.861 INFO:tasks.workunit.client.1.smithi138.stdout:5/392: rmdir d4/da/d6e/d22/d35 39 2024-04-25T01:36:05.862 INFO:tasks.workunit.client.1.smithi138.stdout:7/555: dwrite da/de/d1b/d49/f89 [0,4194304] 0 2024-04-25T01:36:05.863 INFO:tasks.workunit.client.1.smithi138.stdout:8/584: mkdir d3/d16/d88/da/db7 0 2024-04-25T01:36:05.864 INFO:tasks.workunit.client.1.smithi138.stdout:5/393: dread - d4/da/d6e/d22/d74/f76 zero size 2024-04-25T01:36:05.871 INFO:tasks.workunit.client.1.smithi138.stdout:7/556: dwrite da/de/d1b/d49/d81/d51/faa [0,4194304] 0 2024-04-25T01:36:05.872 INFO:tasks.workunit.client.1.smithi138.stdout:8/585: mknod d3/d16/d1f/d2a/d4f/cb8 0 2024-04-25T01:36:05.874 INFO:tasks.workunit.client.1.smithi138.stdout:8/586: read d3/d16/d88/da/db/d15/f83 [4458231,25972] 0 2024-04-25T01:36:05.879 INFO:tasks.workunit.client.1.smithi138.stdout:7/557: unlink da/de/d1b/d4b/f9e 0 2024-04-25T01:36:05.881 INFO:tasks.workunit.client.1.smithi138.stdout:5/394: symlink d4/da/d6e/d22/d35/d4d/d5e/l7d 0 2024-04-25T01:36:05.883 INFO:tasks.workunit.client.1.smithi138.stdout:8/587: read d3/d16/d88/da/de/f10 [2188784,4176] 0 2024-04-25T01:36:05.884 INFO:tasks.workunit.client.0.smithi033.stdout:3/979: creat d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/daa/f159 x:0 0 0 2024-04-25T01:36:05.888 INFO:tasks.workunit.client.1.smithi138.stdout:8/588: fsync d3/d16/d88/da/db/d15/d5d/f6b 0 2024-04-25T01:36:05.889 INFO:tasks.workunit.client.1.smithi138.stdout:7/558: dwrite da/de/d1b/d4b/d5b/f46 [0,4194304] 0 2024-04-25T01:36:05.889 INFO:tasks.workunit.client.1.smithi138.stdout:5/395: symlink d4/l7e 0 2024-04-25T01:36:05.893 INFO:tasks.workunit.client.1.smithi138.stdout:8/589: fsync d3/d16/d88/da/db/d15/d20/f5a 0 2024-04-25T01:36:05.896 INFO:tasks.workunit.client.1.smithi138.stdout:5/396: dread d4/da/d27/f2d [0,4194304] 0 2024-04-25T01:36:05.901 INFO:tasks.workunit.client.1.smithi138.stdout:7/559: dread da/de/d1b/d34/f47 [0,4194304] 0 2024-04-25T01:36:05.902 INFO:tasks.workunit.client.1.smithi138.stdout:6/499: dread dc/d10/f15 [0,4194304] 0 2024-04-25T01:36:05.902 INFO:tasks.workunit.client.1.smithi138.stdout:0/346: sync 2024-04-25T01:36:05.902 INFO:tasks.workunit.client.1.smithi138.stdout:1/413: sync 2024-04-25T01:36:05.906 INFO:tasks.workunit.client.1.smithi138.stdout:0/347: fsync d6/d1d/f24 0 2024-04-25T01:36:05.911 INFO:tasks.workunit.client.1.smithi138.stdout:7/560: dwrite da/de/f27 [0,4194304] 0 2024-04-25T01:36:05.914 INFO:tasks.workunit.client.0.smithi033.stdout:3/980: mknod d8/dd/d11/c15a 0 2024-04-25T01:36:05.915 INFO:tasks.workunit.client.1.smithi138.stdout:0/348: rename d6/d19/d57/f6e to d6/df/d3a/f71 0 2024-04-25T01:36:05.919 INFO:tasks.workunit.client.0.smithi033.stdout:0/986: sync 2024-04-25T01:36:05.920 INFO:tasks.workunit.client.1.smithi138.stdout:0/349: creat d6/d39/d43/f72 x:0 0 0 2024-04-25T01:36:05.921 INFO:tasks.workunit.client.1.smithi138.stdout:6/500: dwrite dc/d10/d28/f34 [0,4194304] 0 2024-04-25T01:36:05.921 INFO:tasks.workunit.client.0.smithi033.stdout:0/987: readlink d7/d2d/d43/dc0/d143/da1/la7 0 2024-04-25T01:36:05.921 INFO:tasks.workunit.client.1.smithi138.stdout:1/414: dwrite d3/d45/d72/d57/f63 [0,4194304] 0 2024-04-25T01:36:05.922 INFO:tasks.workunit.client.1.smithi138.stdout:8/590: truncate d3/d16/d88/da/db/d15/d56/f9c 3938792 0 2024-04-25T01:36:05.923 INFO:tasks.workunit.client.1.smithi138.stdout:1/415: chown d3/d4/d47 926 1 2024-04-25T01:36:05.924 INFO:tasks.workunit.client.1.smithi138.stdout:5/397: truncate d4/da/fd 145918 0 2024-04-25T01:36:05.927 INFO:tasks.workunit.client.1.smithi138.stdout:8/591: stat d3/d16/d88/da/de 0 2024-04-25T01:36:05.927 INFO:tasks.workunit.client.1.smithi138.stdout:7/561: dwrite f9 [4194304,4194304] 0 2024-04-25T01:36:05.928 INFO:tasks.workunit.client.1.smithi138.stdout:0/350: symlink d6/d1d/l73 0 2024-04-25T01:36:05.928 INFO:tasks.workunit.client.1.smithi138.stdout:1/416: dread - d3/d27/d3e/d51/f80 zero size 2024-04-25T01:36:05.929 INFO:tasks.workunit.client.1.smithi138.stdout:1/417: readlink d3/d27/d2d/l31 0 2024-04-25T01:36:05.930 INFO:tasks.workunit.client.1.smithi138.stdout:5/398: write d4/da/d27/f2d [96195,75351] 0 2024-04-25T01:36:05.932 INFO:tasks.workunit.client.1.smithi138.stdout:6/501: creat dc/d2e/d37/d56/f9a x:0 0 0 2024-04-25T01:36:05.933 INFO:tasks.workunit.client.1.smithi138.stdout:1/418: stat d3/d27/d3e/d51/f75 0 2024-04-25T01:36:05.937 INFO:tasks.workunit.client.1.smithi138.stdout:8/592: mknod d3/d16/d88/da/db/cb9 0 2024-04-25T01:36:05.938 INFO:tasks.workunit.client.1.smithi138.stdout:6/502: chown dc/d10/f15 3428307 1 2024-04-25T01:36:05.939 INFO:tasks.workunit.client.1.smithi138.stdout:5/399: mkdir d4/da/d6e/d22/d74/d59/d7f 0 2024-04-25T01:36:05.943 INFO:tasks.workunit.client.1.smithi138.stdout:1/419: unlink d3/d27/c4c 0 2024-04-25T01:36:05.944 INFO:tasks.workunit.client.1.smithi138.stdout:0/351: dwrite d6/d19/f48 [0,4194304] 0 2024-04-25T01:36:05.945 INFO:tasks.workunit.client.0.smithi033.stdout:1/848: mknod d2/d2e/c10f 0 2024-04-25T01:36:05.945 INFO:tasks.workunit.client.1.smithi138.stdout:7/562: dwrite da/f18 [0,4194304] 0 2024-04-25T01:36:05.946 INFO:tasks.workunit.client.1.smithi138.stdout:6/503: dread - dc/d2e/d37/d56/f8f zero size 2024-04-25T01:36:05.946 INFO:tasks.workunit.client.0.smithi033.stdout:1/849: fsync d2/d11/d28/fd9 0 2024-04-25T01:36:05.947 INFO:tasks.workunit.client.0.smithi033.stdout:1/850: truncate d2/de6/ffd 504791 0 2024-04-25T01:36:05.948 INFO:tasks.workunit.client.0.smithi033.stdout:1/851: chown d2/d5/d6c/d94/l9f 5178141 1 2024-04-25T01:36:05.948 INFO:tasks.workunit.client.1.smithi138.stdout:7/563: chown da/de/d1b/c3b 1315 1 2024-04-25T01:36:05.950 INFO:tasks.workunit.client.1.smithi138.stdout:6/504: truncate dc/d10/d28/d70/f7f 817727 0 2024-04-25T01:36:05.951 INFO:tasks.workunit.client.1.smithi138.stdout:7/564: write da/de/f29 [396373,36520] 0 2024-04-25T01:36:05.951 INFO:tasks.workunit.client.0.smithi033.stdout:1/852: mkdir d2/d5/d75/d59/d10e/d110 0 2024-04-25T01:36:05.952 INFO:tasks.workunit.client.0.smithi033.stdout:1/853: write d2/f32 [97030,73339] 0 2024-04-25T01:36:05.959 INFO:tasks.workunit.client.0.smithi033.stdout:1/854: rmdir d2/d2e/d34/def 0 2024-04-25T01:36:05.959 INFO:tasks.workunit.client.0.smithi033.stdout:1/855: stat d2/d5/d6c 0 2024-04-25T01:36:05.960 INFO:tasks.workunit.client.1.smithi138.stdout:6/505: dwrite dc/d2e/d37/d56/f74 [0,4194304] 0 2024-04-25T01:36:05.960 INFO:tasks.workunit.client.1.smithi138.stdout:7/565: mkdir da/de/d1b/d49/dbb 0 2024-04-25T01:36:05.963 INFO:tasks.workunit.client.1.smithi138.stdout:6/506: stat f5 0 2024-04-25T01:36:05.964 INFO:tasks.workunit.client.0.smithi033.stdout:1/856: creat d2/d5/d1b/d46/d5f/d63/f111 x:0 0 0 2024-04-25T01:36:05.965 INFO:tasks.workunit.client.1.smithi138.stdout:6/507: readlink dc/d2e/d37/l5d 0 2024-04-25T01:36:05.965 INFO:tasks.workunit.client.1.smithi138.stdout:5/400: symlink d4/da/d6e/d22/d35/d4d/d5e/d60/l80 0 2024-04-25T01:36:05.969 INFO:tasks.workunit.client.1.smithi138.stdout:7/566: unlink da/de/d13/d30/d59/d7b/d99/fa0 0 2024-04-25T01:36:05.971 INFO:tasks.workunit.client.1.smithi138.stdout:6/508: creat dc/d8d/f9b x:0 0 0 2024-04-25T01:36:05.973 INFO:tasks.workunit.client.0.smithi033.stdout:1/857: truncate d2/d11/f5e 6953665 0 2024-04-25T01:36:05.974 INFO:tasks.workunit.client.1.smithi138.stdout:5/401: mknod d4/da/c81 0 2024-04-25T01:36:05.974 INFO:tasks.workunit.client.0.smithi033.stdout:2/959: dwrite da/d27/d31/f61 [0,4194304] 0 2024-04-25T01:36:05.976 INFO:tasks.workunit.client.1.smithi138.stdout:6/509: mknod dc/d10/d28/d70/d83/c9c 0 2024-04-25T01:36:05.977 INFO:tasks.workunit.client.1.smithi138.stdout:6/510: chown dc/d10/f26 221722707 1 2024-04-25T01:36:05.978 INFO:tasks.workunit.client.1.smithi138.stdout:7/567: dwrite da/fc [0,4194304] 0 2024-04-25T01:36:05.979 INFO:tasks.workunit.client.0.smithi033.stdout:9/960: sync 2024-04-25T01:36:05.981 INFO:tasks.workunit.client.1.smithi138.stdout:5/402: truncate d4/da/d6e/d22/d74/d59/f6b 400081 0 2024-04-25T01:36:05.983 INFO:tasks.workunit.client.0.smithi033.stdout:1/858: creat d2/d2e/d34/d60/d106/f112 x:0 0 0 2024-04-25T01:36:05.983 INFO:tasks.workunit.client.0.smithi033.stdout:2/960: link da/l21 da/d8d/db0/l137 0 2024-04-25T01:36:05.983 INFO:tasks.workunit.client.1.smithi138.stdout:7/568: chown da/de/d13/c6b 872494 1 2024-04-25T01:36:05.983 INFO:tasks.workunit.client.0.smithi033.stdout:1/859: chown d2/d5/dee/dfb 15 1 2024-04-25T01:36:05.986 INFO:tasks.workunit.client.0.smithi033.stdout:2/961: creat da/d5b/d5e/d76/f138 x:0 0 0 2024-04-25T01:36:05.992 INFO:tasks.workunit.client.0.smithi033.stdout:1/860: link d2/d11/d28/fbf d2/d11/d28/f113 0 2024-04-25T01:36:05.994 INFO:tasks.workunit.client.1.smithi138.stdout:7/569: dwrite da/de/d13/d30/d59/d7b/d99/fb0 [0,4194304] 0 2024-04-25T01:36:05.994 INFO:tasks.workunit.client.0.smithi033.stdout:3/981: write d8/dff/f11e [179349,8740] 0 2024-04-25T01:36:05.999 INFO:tasks.workunit.client.0.smithi033.stdout:1/861: fdatasync d2/d5/d75/d59/d83/fc1 0 2024-04-25T01:36:06.001 INFO:tasks.workunit.client.1.smithi138.stdout:1/420: truncate d3/d27/d24/f81 3041887 0 2024-04-25T01:36:06.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.026+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9ac10f840 con 0x7ff99406b790 2024-04-25T01:36:06.003 INFO:tasks.workunit.client.0.smithi033.stdout:3/982: creat d8/dd/d3d/d63/d7c/d4d/da5/f15b x:0 0 0 2024-04-25T01:36:06.003 INFO:tasks.workunit.client.1.smithi138.stdout:1/421: fdatasync d3/db/f67 0 2024-04-25T01:36:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.028+0000 7ff993fff700 1 -- 172.21.15.33:0/74722821 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7ff9ac10f840 con 0x7ff99406b790 2024-04-25T01:36:06.004 INFO:tasks.workunit.client.0.smithi033.stdout:0/988: dwrite d7/d2d/d4d/d9c/d61/fdb [0,4194304] 0 2024-04-25T01:36:06.005 INFO:tasks.workunit.client.1.smithi138.stdout:7/570: rename da/de/d1b/d4b/d5b/db6 to da/de/d13/d21/dbc 0 2024-04-25T01:36:06.005 INFO:tasks.workunit.client.0.smithi033.stdout:3/983: stat d8/dd/d3d/d63/d7c/ded/dc7/ddf/d10b 0 2024-04-25T01:36:06.007 INFO:tasks.workunit.client.1.smithi138.stdout:1/422: rmdir d3/db 39 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.031+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7ff99406b790 msgr2=0x7ff99406dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.031+0000 7ff9b1e52700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7ff99406b790 0x7ff99406dc40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0009b30 tx=0x7ff9a0009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.032+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fce80 msgr2=0x7ff9ac10d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.032+0000 7ff9b1e52700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fce80 0x7ff9ac10d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff99c00eb40 tx=0x7ff99c00b040 comp rx=0 tx=0).stop 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.032+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 shutdown_connections 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.032+0000 7ff9b1e52700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7ff99406b790 0x7ff99406dc40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.032+0000 7ff9b1e52700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff9ac0fce80 0x7ff9ac10d9c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.032+0000 7ff9b1e52700 1 --2- 172.21.15.33:0/74722821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9ac0fe080 0x7ff9ac10df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.032+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 >> 172.21.15.33:0/74722821 conn(0x7ff9ac0f8590 msgr2=0x7ff9ac0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.032+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 shutdown_connections 2024-04-25T01:36:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.032+0000 7ff9b1e52700 1 -- 172.21.15.33:0/74722821 wait complete. 2024-04-25T01:36:06.009 INFO:tasks.workunit.client.0.smithi033.stdout:9/961: dread d0/d2/dd/f15 [0,4194304] 0 2024-04-25T01:36:06.009 INFO:tasks.workunit.client.1.smithi138.stdout:7/571: rmdir da/de/d1b/d49/d81/d73 39 2024-04-25T01:36:06.010 INFO:tasks.workunit.client.0.smithi033.stdout:3/984: truncate d8/f5c 6302300 0 2024-04-25T01:36:06.011 INFO:tasks.workunit.client.0.smithi033.stdout:1/862: symlink d2/d11/d28/l114 0 2024-04-25T01:36:06.013 INFO:tasks.workunit.client.1.smithi138.stdout:1/423: truncate d3/f34 825856 0 2024-04-25T01:36:06.014 INFO:tasks.workunit.client.0.smithi033.stdout:9/962: rename d0/d2/dd/df/d14d to d0/d2/dd/d5f/d11b/d14f 0 2024-04-25T01:36:06.015 INFO:tasks.workunit.client.1.smithi138.stdout:5/403: write d4/da/d6e/d22/f3f [4089901,126429] 0 2024-04-25T01:36:06.017 INFO:tasks.workunit.client.0.smithi033.stdout:0/989: unlink d7/da/df/d50/f6b 0 2024-04-25T01:36:06.018 INFO:tasks.workunit.client.1.smithi138.stdout:1/424: rmdir d3/d4 39 2024-04-25T01:36:06.020 INFO:tasks.workunit.client.1.smithi138.stdout:7/572: link da/de/d1b/d4b/l4d da/de/d13/d30/d59/d7b/d99/lbd 0 2024-04-25T01:36:06.020 INFO:tasks.workunit.client.1.smithi138.stdout:7/573: fdatasync da/f18 0 2024-04-25T01:36:06.021 INFO:tasks.workunit.client.0.smithi033.stdout:0/990: write d7/da/f14d [224372,13522] 0 2024-04-25T01:36:06.022 INFO:tasks.workunit.client.0.smithi033.stdout:1/863: dwrite d2/d5/d6c/d94/fb5 [0,4194304] 0 2024-04-25T01:36:06.026 INFO:tasks.workunit.client.1.smithi138.stdout:6/511: dread dc/f16 [0,4194304] 0 2024-04-25T01:36:06.027 INFO:tasks.workunit.client.1.smithi138.stdout:6/512: chown dc/d10/c2d 7183 1 2024-04-25T01:36:06.028 INFO:tasks.workunit.client.0.smithi033.stdout:9/963: dread d0/d2/d43/d4b/dc5/de0/f70 [0,4194304] 0 2024-04-25T01:36:06.029 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:36:06.034 INFO:tasks.workunit.client.0.smithi033.stdout:9/964: creat d0/d2/d43/d4b/dc5/de0/d42/dea/f150 x:0 0 0 2024-04-25T01:36:06.035 INFO:tasks.workunit.client.1.smithi138.stdout:7/574: creat da/de/d13/d21/dbc/fbe x:0 0 0 2024-04-25T01:36:06.035 INFO:tasks.workunit.client.1.smithi138.stdout:1/425: unlink d3/db/d4a/f6e 0 2024-04-25T01:36:06.036 INFO:tasks.workunit.client.0.smithi033.stdout:1/864: getdents d2/d5/dee/dfb 0 2024-04-25T01:36:06.036 INFO:tasks.workunit.client.0.smithi033.stdout:9/965: stat d0/d2/d43/d4b/dc5/de0/f70 0 2024-04-25T01:36:06.037 INFO:tasks.workunit.client.1.smithi138.stdout:5/404: creat d4/f82 x:0 0 0 2024-04-25T01:36:06.039 INFO:tasks.workunit.client.0.smithi033.stdout:1/865: write d2/d5/d1b/d46/d5f/d63/d64/fa5 [2936161,25561] 0 2024-04-25T01:36:06.042 INFO:tasks.workunit.client.0.smithi033.stdout:0/991: link d7/d2d/d4c/lbd d7/d2d/d43/dc0/d143/da1/l157 0 2024-04-25T01:36:06.044 INFO:tasks.workunit.client.1.smithi138.stdout:7/575: unlink da/de/lb2 0 2024-04-25T01:36:06.044 INFO:tasks.workunit.client.0.smithi033.stdout:0/992: readlink d7/d2d/d4d/d9c/d61/d70/dee/l134 0 2024-04-25T01:36:06.044 INFO:tasks.workunit.client.1.smithi138.stdout:1/426: mknod d3/db/d4a/c86 0 2024-04-25T01:36:06.047 INFO:tasks.workunit.client.1.smithi138.stdout:1/427: fdatasync d3/d27/d3e/f61 0 2024-04-25T01:36:06.048 INFO:tasks.workunit.client.0.smithi033.stdout:9/966: rename d0/d2/d43/d73/f9f to d0/d2/dd/d5f/d11b/d14f/f151 0 2024-04-25T01:36:06.048 INFO:tasks.workunit.client.0.smithi033.stdout:2/962: dwrite da/d2c/db8/fc5 [0,4194304] 0 2024-04-25T01:36:06.050 INFO:tasks.workunit.client.1.smithi138.stdout:5/405: dwrite d4/da/d6e/d22/d74/d59/f6b [0,4194304] 0 2024-04-25T01:36:06.051 INFO:tasks.workunit.client.1.smithi138.stdout:5/406: read - d4/f62 zero size 2024-04-25T01:36:06.051 INFO:tasks.workunit.client.1.smithi138.stdout:7/576: dread da/f6a [0,4194304] 0 2024-04-25T01:36:06.052 INFO:tasks.workunit.client.1.smithi138.stdout:5/407: read - d4/da/d6e/d22/f7b zero size 2024-04-25T01:36:06.053 INFO:tasks.workunit.client.1.smithi138.stdout:7/577: chown da/de/d1b/d4b/d77 65022083 1 2024-04-25T01:36:06.055 INFO:tasks.workunit.client.1.smithi138.stdout:6/513: dread dc/f72 [0,4194304] 0 2024-04-25T01:36:06.058 INFO:tasks.workunit.client.0.smithi033.stdout:2/963: creat da/d5b/d5e/d76/dfc/f139 x:0 0 0 2024-04-25T01:36:06.063 INFO:tasks.workunit.client.0.smithi033.stdout:1/866: rmdir d2/d11/d28 39 2024-04-25T01:36:06.063 INFO:tasks.workunit.client.1.smithi138.stdout:1/428: mknod d3/d45/d72/d3d/c87 0 2024-04-25T01:36:06.065 INFO:tasks.workunit.client.1.smithi138.stdout:7/578: fdatasync da/de/d1b/d4b/d5b/f67 0 2024-04-25T01:36:06.065 INFO:tasks.workunit.client.1.smithi138.stdout:5/408: mkdir d4/da/d6e/d22/d74/d59/d83 0 2024-04-25T01:36:06.066 INFO:tasks.workunit.client.0.smithi033.stdout:1/867: mknod d2/d2e/d49/c115 0 2024-04-25T01:36:06.067 INFO:tasks.workunit.client.0.smithi033.stdout:2/964: truncate da/d50/d6a/d43/d6c/fc3 850106 0 2024-04-25T01:36:06.069 INFO:tasks.workunit.client.1.smithi138.stdout:6/514: dread dc/d10/d67/f65 [0,4194304] 0 2024-04-25T01:36:06.069 INFO:tasks.workunit.client.0.smithi033.stdout:3/985: write d8/dd/d3d/d63/d7c/ded/dbf/dc5/f152 [2396905,86478] 0 2024-04-25T01:36:06.071 INFO:tasks.workunit.client.1.smithi138.stdout:6/515: write dc/d2e/d37/d56/f9a [353253,128091] 0 2024-04-25T01:36:06.072 INFO:tasks.workunit.client.0.smithi033.stdout:1/868: creat d2/d5/d6c/d94/f116 x:0 0 0 2024-04-25T01:36:06.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/409: getdents d4/da/d52 0 2024-04-25T01:36:06.073 INFO:tasks.workunit.client.0.smithi033.stdout:2/965: mknod da/d2c/db8/c13a 0 2024-04-25T01:36:06.074 INFO:tasks.workunit.client.1.smithi138.stdout:7/579: unlink da/de/d1b/d49/d81/d73/c8e 0 2024-04-25T01:36:06.075 INFO:tasks.workunit.client.1.smithi138.stdout:7/580: read - da/de/d1b/d49/d81/f9c zero size 2024-04-25T01:36:06.078 INFO:tasks.workunit.client.1.smithi138.stdout:5/410: truncate d4/da/f20 1172416 0 2024-04-25T01:36:06.081 INFO:tasks.workunit.client.1.smithi138.stdout:5/411: chown d4/da/d6e/d22/d35/d4d/d5e/l7d 3 1 2024-04-25T01:36:06.082 INFO:tasks.workunit.client.1.smithi138.stdout:5/412: fsync d4/da/f20 0 2024-04-25T01:36:06.082 INFO:tasks.workunit.client.1.smithi138.stdout:7/581: link da/de/d13/d30/d59/f78 da/de/d1b/d49/dbb/fbf 0 2024-04-25T01:36:06.082 INFO:tasks.workunit.client.0.smithi033.stdout:1/869: write d2/d2e/f67 [4353890,41234] 0 2024-04-25T01:36:06.082 INFO:tasks.workunit.client.0.smithi033.stdout:2/966: fsync da/d50/d6a/d43/f66 0 2024-04-25T01:36:06.084 INFO:tasks.workunit.client.1.smithi138.stdout:7/582: symlink da/de/d1b/d49/d81/d73/lc0 0 2024-04-25T01:36:06.085 INFO:tasks.workunit.client.0.smithi033.stdout:1/870: symlink d2/d5/d1b/dd2/d102/l117 0 2024-04-25T01:36:06.085 INFO:tasks.workunit.client.1.smithi138.stdout:7/583: chown da/de/d1b/d34/l60 57412 1 2024-04-25T01:36:06.085 INFO:tasks.workunit.client.1.smithi138.stdout:9/409: sync 2024-04-25T01:36:06.085 INFO:tasks.workunit.client.1.smithi138.stdout:2/492: sync 2024-04-25T01:36:06.085 INFO:tasks.workunit.client.1.smithi138.stdout:4/364: sync 2024-04-25T01:36:06.085 INFO:tasks.workunit.client.1.smithi138.stdout:3/601: sync 2024-04-25T01:36:06.086 INFO:tasks.workunit.client.1.smithi138.stdout:5/413: symlink d4/da/l84 0 2024-04-25T01:36:06.086 INFO:tasks.workunit.client.1.smithi138.stdout:4/365: readlink d0/d16/l37 0 2024-04-25T01:36:06.087 INFO:tasks.workunit.client.1.smithi138.stdout:2/493: read - d0/d5a/d15/d51/f6f zero size 2024-04-25T01:36:06.087 INFO:tasks.workunit.client.1.smithi138.stdout:9/410: rmdir d4/d8 39 2024-04-25T01:36:06.087 INFO:tasks.workunit.client.1.smithi138.stdout:1/429: chown d3/d27/d3e/f76 113 1 2024-04-25T01:36:06.090 INFO:tasks.workunit.client.1.smithi138.stdout:1/430: dread - d3/d27/d3e/d51/f75 zero size 2024-04-25T01:36:06.092 INFO:tasks.workunit.client.1.smithi138.stdout:3/602: unlink db/df/d19/d1a/l22 0 2024-04-25T01:36:06.092 INFO:tasks.workunit.client.1.smithi138.stdout:2/494: read d0/d5a/d15/d7e/f37 [1555671,39318] 0 2024-04-25T01:36:06.093 INFO:tasks.workunit.client.0.smithi033.stdout:1/871: creat d2/d5/d1b/f118 x:0 0 0 2024-04-25T01:36:06.093 INFO:tasks.workunit.client.0.smithi033.stdout:0/993: dwrite d7/d33/dcd/f103 [0,4194304] 0 2024-04-25T01:36:06.095 INFO:tasks.workunit.client.0.smithi033.stdout:9/967: dwrite d0/d2/d43/d4b/dc5/de0/d7e/d58/fa6 [0,4194304] 0 2024-04-25T01:36:06.097 INFO:tasks.workunit.client.1.smithi138.stdout:7/584: mknod da/de/d1b/d49/dbb/cc1 0 2024-04-25T01:36:06.097 INFO:tasks.workunit.client.1.smithi138.stdout:1/431: readlink d3/d27/d24/l4d 0 2024-04-25T01:36:06.098 INFO:tasks.workunit.client.1.smithi138.stdout:6/516: dread dc/d10/f68 [0,4194304] 0 2024-04-25T01:36:06.099 INFO:tasks.workunit.client.0.smithi033.stdout:2/967: creat da/d27/d31/d40/d8a/dbb/dd6/f13b x:0 0 0 2024-04-25T01:36:06.100 INFO:tasks.workunit.client.1.smithi138.stdout:3/603: creat db/df/fbd x:0 0 0 2024-04-25T01:36:06.101 INFO:tasks.workunit.client.0.smithi033.stdout:1/872: creat d2/d5/d75/d59/d10e/f119 x:0 0 0 2024-04-25T01:36:06.105 INFO:tasks.workunit.client.0.smithi033.stdout:3/986: dwrite d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/fd0 [0,4194304] 0 2024-04-25T01:36:06.105 INFO:tasks.workunit.client.0.smithi033.stdout:1/873: dread - d2/d2e/d34/d60/d106/f112 zero size 2024-04-25T01:36:06.105 INFO:tasks.workunit.client.1.smithi138.stdout:2/495: mknod d0/d5a/d15/d44/c92 0 2024-04-25T01:36:06.105 INFO:tasks.workunit.client.1.smithi138.stdout:5/414: dread d4/da/f47 [0,4194304] 0 2024-04-25T01:36:06.108 INFO:tasks.workunit.client.0.smithi033.stdout:0/994: creat d7/da/df/d14/d45/da2/f158 x:0 0 0 2024-04-25T01:36:06.108 INFO:tasks.workunit.client.0.smithi033.stdout:2/968: mknod da/d5b/d5e/da8/c13c 0 2024-04-25T01:36:06.112 INFO:tasks.workunit.client.0.smithi033.stdout:1/874: dread - d2/d5/d75/d59/fae zero size 2024-04-25T01:36:06.114 INFO:tasks.workunit.client.1.smithi138.stdout:4/366: write d0/fa [1480840,7045] 0 2024-04-25T01:36:06.114 INFO:tasks.workunit.client.0.smithi033.stdout:3/987: truncate d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d6e/daa/f159 444570 0 2024-04-25T01:36:06.116 INFO:tasks.workunit.client.1.smithi138.stdout:7/585: rename da/de/d1b/d49/dbb/fbf to da/de/d1b/d34/fc2 0 2024-04-25T01:36:06.117 INFO:tasks.workunit.client.1.smithi138.stdout:3/604: symlink db/df/d17/lbe 0 2024-04-25T01:36:06.117 INFO:tasks.workunit.client.0.smithi033.stdout:3/988: fsync d8/dd/d3d/dce/f14b 0 2024-04-25T01:36:06.118 INFO:tasks.workunit.client.1.smithi138.stdout:1/432: creat d3/d27/d3e/f88 x:0 0 0 2024-04-25T01:36:06.118 INFO:tasks.workunit.client.1.smithi138.stdout:4/367: fsync d0/f40 0 2024-04-25T01:36:06.119 INFO:tasks.workunit.client.1.smithi138.stdout:1/433: readlink d3/d6d/l70 0 2024-04-25T01:36:06.120 INFO:tasks.workunit.client.0.smithi033.stdout:2/969: rename da/d27/d31/dca/cd9 to da/d50/d6a/c13d 0 2024-04-25T01:36:06.122 INFO:tasks.workunit.client.1.smithi138.stdout:4/368: rmdir d0/d1c 39 2024-04-25T01:36:06.123 INFO:tasks.workunit.client.0.smithi033.stdout:9/968: dread d0/d2/d43/d73/ffc [0,4194304] 0 2024-04-25T01:36:06.125 INFO:tasks.workunit.client.1.smithi138.stdout:2/496: rename d0/d32/d35/d4f to d0/d5a/d15/d44/d4c/d93 0 2024-04-25T01:36:06.125 INFO:tasks.workunit.client.0.smithi033.stdout:9/969: write d0/d2/d43/d4b/dc5/de0/d6e/f131 [540079,21845] 0 2024-04-25T01:36:06.125 INFO:tasks.workunit.client.0.smithi033.stdout:2/970: symlink da/d2c/dcd/l13e 0 2024-04-25T01:36:06.126 INFO:tasks.workunit.client.1.smithi138.stdout:1/434: creat d3/d6d/f89 x:0 0 0 2024-04-25T01:36:06.127 INFO:tasks.workunit.client.0.smithi033.stdout:3/989: symlink d8/dd/d3d/d63/d7c/ded/dbf/dc5/l15c 0 2024-04-25T01:36:06.127 INFO:tasks.workunit.client.0.smithi033.stdout:1/875: creat d2/d2e/f11a x:0 0 0 2024-04-25T01:36:06.127 INFO:tasks.workunit.client.0.smithi033.stdout:0/995: getdents d7/d2d/d43 0 2024-04-25T01:36:06.127 INFO:tasks.workunit.client.1.smithi138.stdout:1/435: dread - d3/d27/d3e/d51/f80 zero size 2024-04-25T01:36:06.128 INFO:tasks.workunit.client.1.smithi138.stdout:3/605: creat db/d9b/db4/fbf x:0 0 0 2024-04-25T01:36:06.128 INFO:tasks.workunit.client.0.smithi033.stdout:1/876: chown d2/d11/dc9/d10a 393994843 1 2024-04-25T01:36:06.129 INFO:tasks.workunit.client.1.smithi138.stdout:9/411: write d4/d8/f1f [5891023,4795] 0 2024-04-25T01:36:06.131 INFO:tasks.workunit.client.0.smithi033.stdout:3/990: stat d8/dd/c113 0 2024-04-25T01:36:06.132 INFO:tasks.workunit.client.1.smithi138.stdout:5/415: rename d4/da/d6e/d22/d74/d59/f6b to d4/da/d6e/d22/d74/f85 0 2024-04-25T01:36:06.132 INFO:tasks.workunit.client.0.smithi033.stdout:0/996: mkdir d7/d8a/d159 0 2024-04-25T01:36:06.134 INFO:tasks.workunit.client.0.smithi033.stdout:2/971: creat da/f13f x:0 0 0 2024-04-25T01:36:06.135 INFO:tasks.workunit.client.1.smithi138.stdout:6/517: dwrite dc/d2e/f2f [0,4194304] 0 2024-04-25T01:36:06.136 INFO:tasks.workunit.client.0.smithi033.stdout:1/877: mkdir d2/d2e/dcd/d11b 0 2024-04-25T01:36:06.136 INFO:tasks.workunit.client.1.smithi138.stdout:6/518: dread - dc/d10/d28/d80/f8b zero size 2024-04-25T01:36:06.138 INFO:tasks.workunit.client.0.smithi033.stdout:2/972: read da/d27/d31/d40/d77/f7b [216367,94452] 0 2024-04-25T01:36:06.140 INFO:tasks.workunit.client.0.smithi033.stdout:3/991: creat d8/dd/d11/d102/f15d x:0 0 0 2024-04-25T01:36:06.142 INFO:tasks.workunit.client.0.smithi033.stdout:1/878: rmdir d2/d5/d6c/d94 39 2024-04-25T01:36:06.143 INFO:tasks.workunit.client.1.smithi138.stdout:4/369: creat d0/d1c/d48/f6c x:0 0 0 2024-04-25T01:36:06.144 INFO:tasks.workunit.client.0.smithi033.stdout:1/879: chown d2/d2e/dcd/d11b 112 1 2024-04-25T01:36:06.144 INFO:tasks.workunit.client.1.smithi138.stdout:7/586: dwrite da/de/d1b/d4b/d5b/f43 [0,4194304] 0 2024-04-25T01:36:06.145 INFO:tasks.workunit.client.1.smithi138.stdout:4/370: stat d0/d1c/c67 0 2024-04-25T01:36:06.146 INFO:tasks.workunit.client.0.smithi033.stdout:0/997: creat d7/d2d/d4d/d9c/d68/dd2/d12c/f15a x:0 0 0 2024-04-25T01:36:06.148 INFO:tasks.workunit.client.1.smithi138.stdout:2/497: write d0/d5a/f3c [3595386,37069] 0 2024-04-25T01:36:06.149 INFO:tasks.workunit.client.0.smithi033.stdout:2/973: dwrite da/d27/d31/f83 [0,4194304] 0 2024-04-25T01:36:06.151 INFO:tasks.workunit.client.1.smithi138.stdout:9/412: write f1 [989622,65525] 0 2024-04-25T01:36:06.152 INFO:tasks.workunit.client.1.smithi138.stdout:1/436: link d3/db/fe d3/d45/d72/d57/f8a 0 2024-04-25T01:36:06.153 INFO:tasks.workunit.client.0.smithi033.stdout:0/998: write d7/d2d/d43/d44/fd5 [4320254,25594] 0 2024-04-25T01:36:06.153 INFO:tasks.workunit.client.1.smithi138.stdout:1/437: write d3/d45/f7e [215565,27109] 0 2024-04-25T01:36:06.154 INFO:tasks.workunit.client.1.smithi138.stdout:6/519: creat dc/d10/d28/d70/d83/f9d x:0 0 0 2024-04-25T01:36:06.156 INFO:tasks.workunit.client.1.smithi138.stdout:1/438: write d3/d27/d2d/f79 [787624,45947] 0 2024-04-25T01:36:06.157 INFO:tasks.workunit.client.0.smithi033.stdout:1/880: rename d2/d2e/d34/f50 to d2/d11/d28/dd4/f11c 0 2024-04-25T01:36:06.158 INFO:tasks.workunit.client.1.smithi138.stdout:3/606: creat db/fc0 x:0 0 0 2024-04-25T01:36:06.159 INFO:tasks.workunit.client.1.smithi138.stdout:2/498: unlink d0/d21/c29 0 2024-04-25T01:36:06.161 INFO:tasks.workunit.client.1.smithi138.stdout:2/499: write d0/d5a/d15/d51/f62 [2952211,113677] 0 2024-04-25T01:36:06.162 INFO:tasks.workunit.client.0.smithi033.stdout:2/974: creat da/d50/d6a/d43/d6c/df5/f140 x:0 0 0 2024-04-25T01:36:06.162 INFO:tasks.workunit.client.0.smithi033.stdout:1/881: write d2/d2e/d34/d60/fa7 [13524,85534] 0 2024-04-25T01:36:06.162 INFO:tasks.workunit.client.0.smithi033.stdout:2/975: fsync da/d5b/d5e/f103 0 2024-04-25T01:36:06.163 INFO:tasks.workunit.client.1.smithi138.stdout:5/416: rename d4/da/d6e/d22/c77 to d4/da/d6e/d3c/c86 0 2024-04-25T01:36:06.166 INFO:tasks.workunit.client.1.smithi138.stdout:6/520: mknod dc/d2e/d37/d56/d8c/c9e 0 2024-04-25T01:36:06.166 INFO:tasks.workunit.client.0.smithi033.stdout:1/882: fsync d2/d5/d75/d9b/ff9 0 2024-04-25T01:36:06.166 INFO:tasks.workunit.client.0.smithi033.stdout:2/976: fdatasync da/d27/d31/d40/d77/f9b 0 2024-04-25T01:36:06.169 INFO:tasks.workunit.client.0.smithi033.stdout:1/883: chown d2/d5/d6c/c92 507396 1 2024-04-25T01:36:06.171 INFO:tasks.workunit.client.1.smithi138.stdout:2/500: read d0/d5a/d15/d44/d4c/d56/d6d/f8c [3934807,39336] 0 2024-04-25T01:36:06.172 INFO:tasks.workunit.client.1.smithi138.stdout:9/413: rename d4/d8/c12 to d4/d1a/d33/c85 0 2024-04-25T01:36:06.172 INFO:tasks.workunit.client.0.smithi033.stdout:1/884: chown d2/d7f/db0/lca 140813148 1 2024-04-25T01:36:06.173 INFO:tasks.workunit.client.0.smithi033.stdout:1/885: readlink d2/d11/dc9/ld8 0 2024-04-25T01:36:06.173 INFO:tasks.workunit.client.0.smithi033.stdout:2/977: truncate da/d50/fad 167378 0 2024-04-25T01:36:06.173 INFO:tasks.workunit.client.0.smithi033.stdout:2/978: readlink da/l6b 0 2024-04-25T01:36:06.174 INFO:tasks.workunit.client.1.smithi138.stdout:2/501: fsync d0/d5a/d15/f5d 0 2024-04-25T01:36:06.174 INFO:tasks.workunit.client.0.smithi033.stdout:2/979: chown da/d27/d31/f5a 3121585 1 2024-04-25T01:36:06.174 INFO:tasks.workunit.client.0.smithi033.stdout:1/886: mkdir d2/d7f/d11d 0 2024-04-25T01:36:06.174 INFO:tasks.workunit.client.1.smithi138.stdout:5/417: mkdir d4/da/d52/d87 0 2024-04-25T01:36:06.175 INFO:tasks.workunit.client.0.smithi033.stdout:2/980: stat da/d5b/d5e/da8/c124 0 2024-04-25T01:36:06.176 INFO:tasks.workunit.client.1.smithi138.stdout:2/502: truncate d0/d5a/d15/d51/f80 372202 0 2024-04-25T01:36:06.181 INFO:tasks.workunit.client.0.smithi033.stdout:2/981: mknod da/d27/d31/c141 0 2024-04-25T01:36:06.183 INFO:tasks.workunit.client.1.smithi138.stdout:2/503: mknod d0/d85/c94 0 2024-04-25T01:36:06.184 INFO:tasks.workunit.client.1.smithi138.stdout:4/371: dread d0/d16/d4a/f53 [0,4194304] 0 2024-04-25T01:36:06.186 INFO:tasks.workunit.client.1.smithi138.stdout:2/504: write d0/f10 [662593,8042] 0 2024-04-25T01:36:06.186 INFO:tasks.workunit.client.0.smithi033.stdout:2/982: creat da/d27/d31/d40/d8a/dbb/dd6/f142 x:0 0 0 2024-04-25T01:36:06.187 INFO:tasks.workunit.client.1.smithi138.stdout:1/439: dwrite d3/d27/d3e/f76 [0,4194304] 0 2024-04-25T01:36:06.189 INFO:tasks.workunit.client.1.smithi138.stdout:6/521: truncate dc/d10/d28/f34 4335675 0 2024-04-25T01:36:06.189 INFO:tasks.workunit.client.1.smithi138.stdout:1/440: write d3/d45/d72/f42 [893385,48024] 0 2024-04-25T01:36:06.190 INFO:tasks.workunit.client.1.smithi138.stdout:4/372: creat d0/d16/d4a/f6d x:0 0 0 2024-04-25T01:36:06.190 INFO:tasks.workunit.client.0.smithi033.stdout:2/983: symlink da/d27/d31/dca/l143 0 2024-04-25T01:36:06.191 INFO:tasks.workunit.client.1.smithi138.stdout:6/522: dread - dc/d7b/f94 zero size 2024-04-25T01:36:06.193 INFO:tasks.workunit.client.1.smithi138.stdout:6/523: write dc/d2e/f4d [706095,7467] 0 2024-04-25T01:36:06.193 INFO:tasks.workunit.client.0.smithi033.stdout:2/984: symlink da/d50/d6a/d43/d6c/l144 0 2024-04-25T01:36:06.194 INFO:tasks.workunit.client.1.smithi138.stdout:2/505: fdatasync d0/d32/f63 0 2024-04-25T01:36:06.195 INFO:tasks.workunit.client.1.smithi138.stdout:4/373: mknod d0/d59/c6e 0 2024-04-25T01:36:06.195 INFO:tasks.workunit.client.0.smithi033.stdout:1/887: dread d2/d5/d75/d59/d83/fd0 [0,4194304] 0 2024-04-25T01:36:06.197 INFO:tasks.workunit.client.0.smithi033.stdout:2/985: mknod da/d8d/d10c/d110/c145 0 2024-04-25T01:36:06.199 INFO:tasks.workunit.client.1.smithi138.stdout:1/441: creat d3/d27/d3e/d51/f8b x:0 0 0 2024-04-25T01:36:06.199 INFO:tasks.workunit.client.1.smithi138.stdout:2/506: mknod d0/d5a/c95 0 2024-04-25T01:36:06.200 INFO:tasks.workunit.client.0.smithi033.stdout:2/986: creat da/d50/d6a/d43/df1/f146 x:0 0 0 2024-04-25T01:36:06.201 INFO:tasks.workunit.client.1.smithi138.stdout:5/418: dwrite d4/da/d6e/d22/d35/d4d/f66 [0,4194304] 0 2024-04-25T01:36:06.203 INFO:tasks.workunit.client.1.smithi138.stdout:5/419: dread - d4/f62 zero size 2024-04-25T01:36:06.203 INFO:tasks.workunit.client.0.smithi033.stdout:2/987: truncate da/d27/d31/d40/d8a/f94 1297451 0 2024-04-25T01:36:06.205 INFO:tasks.workunit.client.1.smithi138.stdout:4/374: write d0/f1a [1915696,38746] 0 2024-04-25T01:36:06.206 INFO:tasks.workunit.client.1.smithi138.stdout:6/524: creat dc/d2e/d55/d81/f9f x:0 0 0 2024-04-25T01:36:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.231+0000 7f673984b700 1 -- 172.21.15.33:0/1080702982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67340662a0 msgr2=0x7f67340ff9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.231+0000 7f673984b700 1 --2- 172.21.15.33:0/1080702982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67340662a0 0x7f67340ff9c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6724004940 tx=0x7f6724009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.231+0000 7f673984b700 1 -- 172.21.15.33:0/1080702982 shutdown_connections 2024-04-25T01:36:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.231+0000 7f673984b700 1 --2- 172.21.15.33:0/1080702982 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67340fff00 0x7f6734102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.231+0000 7f673984b700 1 --2- 172.21.15.33:0/1080702982 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67340662a0 0x7f67340ff9c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.231+0000 7f673984b700 1 -- 172.21.15.33:0/1080702982 >> 172.21.15.33:0/1080702982 conn(0x7f67340f7430 msgr2=0x7f67340f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:06.207 INFO:tasks.workunit.client.1.smithi138.stdout:9/414: dread d4/f17 [0,4194304] 0 2024-04-25T01:36:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.232+0000 7f673984b700 1 -- 172.21.15.33:0/1080702982 shutdown_connections 2024-04-25T01:36:06.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.232+0000 7f673984b700 1 -- 172.21.15.33:0/1080702982 wait complete. 2024-04-25T01:36:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.233+0000 7f673984b700 1 Processor -- start 2024-04-25T01:36:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.233+0000 7f673984b700 1 -- start start 2024-04-25T01:36:06.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.233+0000 7f673984b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67340662a0 0x7f673410dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.209 INFO:tasks.workunit.client.1.smithi138.stdout:5/420: chown d4/da/d6e/d22/d74/f79 7380984 1 2024-04-25T01:36:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.233+0000 7f673984b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67340fff00 0x7f673410e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.233+0000 7f673984b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673410e690 con 0x7f67340662a0 2024-04-25T01:36:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.233+0000 7f673984b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673410ee80 con 0x7f67340fff00 2024-04-25T01:36:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.233+0000 7f6732ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67340662a0 0x7f673410dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.233+0000 7f6732ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67340662a0 0x7f673410dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60206/0 (socket says 172.21.15.33:60206) 2024-04-25T01:36:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.233+0000 7f6732ffd700 1 -- 172.21.15.33:0/1555834556 learned_addr learned my addr 172.21.15.33:0/1555834556 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.234+0000 7f67327fc700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67340fff00 0x7f673410e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.234+0000 7f67327fc700 1 -- 172.21.15.33:0/1555834556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67340662a0 msgr2=0x7f673410dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.234+0000 7f67327fc700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67340662a0 0x7f673410dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.234+0000 7f67327fc700 1 -- 172.21.15.33:0/1555834556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6724005040 con 0x7f67340fff00 2024-04-25T01:36:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.234+0000 7f67327fc700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67340fff00 0x7f673410e000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f672800b530 tx=0x7f67280090f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.235+0000 7f671b7fe700 1 -- 172.21.15.33:0/1555834556 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f672800f6f0 con 0x7f67340fff00 2024-04-25T01:36:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.235+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673410f160 con 0x7f67340fff00 2024-04-25T01:36:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.235+0000 7f671b7fe700 1 -- 172.21.15.33:0/1555834556 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f672800fd20 con 0x7f67340fff00 2024-04-25T01:36:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.235+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673410f6c0 con 0x7f67340fff00 2024-04-25T01:36:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.235+0000 7f671b7fe700 1 -- 172.21.15.33:0/1555834556 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f672801ad00 con 0x7f67340fff00 2024-04-25T01:36:06.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.236+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673405f5c0 con 0x7f67340fff00 2024-04-25T01:36:06.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.237+0000 7f671b7fe700 1 -- 172.21.15.33:0/1555834556 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f672801a590 con 0x7f67340fff00 2024-04-25T01:36:06.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.238+0000 7f671b7fe700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f671c06b7e0 0x7f671c06dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.213 INFO:tasks.workunit.client.1.smithi138.stdout:9/415: read d4/f4a [435937,125796] 0 2024-04-25T01:36:06.213 INFO:tasks.workunit.client.0.smithi033.stdout:1/888: dread d2/d11/f30 [0,4194304] 0 2024-04-25T01:36:06.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.238+0000 7f6732ffd700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f671c06b7e0 0x7f671c06dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.214 INFO:tasks.workunit.client.1.smithi138.stdout:2/507: dwrite d0/d18/f2a [0,4194304] 0 2024-04-25T01:36:06.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.239+0000 7f6732ffd700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f671c06b7e0 0x7f671c06dc90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6724004f90 tx=0x7f672401c040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:06.215 INFO:tasks.workunit.client.1.smithi138.stdout:9/416: chown d4/d1a/d33/f37 8056 1 2024-04-25T01:36:06.215 INFO:tasks.workunit.client.1.smithi138.stdout:2/508: stat d0/d5a/d15/d51/f6f 0 2024-04-25T01:36:06.216 INFO:tasks.workunit.client.1.smithi138.stdout:6/525: mkdir dc/d10/d28/d80/d89/da0 0 2024-04-25T01:36:06.218 INFO:tasks.workunit.client.0.smithi033.stdout:1/889: dwrite d2/f32 [0,4194304] 0 2024-04-25T01:36:06.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.244+0000 7f671b7fe700 1 -- 172.21.15.33:0/1555834556 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f672800fe90 con 0x7f67340fff00 2024-04-25T01:36:06.221 INFO:tasks.workunit.client.0.smithi033.stdout:1/890: chown d2/d5/d75/c97 99528 1 2024-04-25T01:36:06.223 INFO:tasks.workunit.client.1.smithi138.stdout:5/421: getdents d4/da/d6e/d22/d74 0 2024-04-25T01:36:06.224 INFO:tasks.workunit.client.1.smithi138.stdout:9/417: getdents d4/d63/d73 0 2024-04-25T01:36:06.227 INFO:tasks.workunit.client.1.smithi138.stdout:6/526: creat dc/fa1 x:0 0 0 2024-04-25T01:36:06.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.254+0000 7f671b7fe700 1 -- 172.21.15.33:0/1555834556 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f6728057590 con 0x7f67340fff00 2024-04-25T01:36:06.235 INFO:tasks.workunit.client.1.smithi138.stdout:9/418: dwrite d4/d8/f45 [0,4194304] 0 2024-04-25T01:36:06.237 INFO:tasks.workunit.client.0.smithi033.stdout:1/891: dwrite d2/f62 [0,4194304] 0 2024-04-25T01:36:06.239 INFO:tasks.workunit.client.1.smithi138.stdout:5/422: getdents d4/da/d6e/d22/d45 0 2024-04-25T01:36:06.242 INFO:tasks.workunit.client.1.smithi138.stdout:4/375: dread d0/f2f [0,4194304] 0 2024-04-25T01:36:06.243 INFO:tasks.workunit.client.1.smithi138.stdout:2/509: dwrite d0/d5a/f87 [0,4194304] 0 2024-04-25T01:36:06.243 INFO:tasks.workunit.client.1.smithi138.stdout:4/376: chown d0/d1c/l3e 700 1 2024-04-25T01:36:06.243 INFO:tasks.workunit.client.1.smithi138.stdout:4/377: chown d0/d16/d4a 2526742 1 2024-04-25T01:36:06.244 INFO:tasks.workunit.client.0.smithi033.stdout:1/892: dread d2/d2e/d34/d60/fa7 [0,4194304] 0 2024-04-25T01:36:06.246 INFO:tasks.workunit.client.0.smithi033.stdout:2/988: dread da/d27/d31/d40/d8a/f75 [0,4194304] 0 2024-04-25T01:36:06.246 INFO:tasks.workunit.client.0.smithi033.stdout:1/893: chown d2/d2e/l80 510 1 2024-04-25T01:36:06.246 INFO:tasks.workunit.client.1.smithi138.stdout:9/419: rename d4/d8/d6b to d4/d63/d86 0 2024-04-25T01:36:06.246 INFO:tasks.workunit.client.1.smithi138.stdout:2/510: mknod d0/d5a/c96 0 2024-04-25T01:36:06.247 INFO:tasks.workunit.client.1.smithi138.stdout:9/420: stat d4/d1a/d3c/l54 0 2024-04-25T01:36:06.247 INFO:tasks.workunit.client.0.smithi033.stdout:2/989: fsync da/d5b/d5e/d76/fc9 0 2024-04-25T01:36:06.249 INFO:tasks.workunit.client.1.smithi138.stdout:4/378: creat d0/d1c/d48/f6f x:0 0 0 2024-04-25T01:36:06.254 INFO:tasks.workunit.client.1.smithi138.stdout:5/423: dwrite d4/da/d27/f78 [0,4194304] 0 2024-04-25T01:36:06.254 INFO:tasks.workunit.client.1.smithi138.stdout:2/511: write d0/d5a/f1a [7794450,107197] 0 2024-04-25T01:36:06.257 INFO:tasks.workunit.client.1.smithi138.stdout:9/421: creat d4/d1a/d33/d1d/d3f/f87 x:0 0 0 2024-04-25T01:36:06.259 INFO:tasks.workunit.client.1.smithi138.stdout:6/527: write dc/d10/d28/f2b [1679991,114019] 0 2024-04-25T01:36:06.261 INFO:tasks.workunit.client.0.smithi033.stdout:2/990: write da/d5b/d5e/ffb [381877,80901] 0 2024-04-25T01:36:06.263 INFO:tasks.workunit.client.1.smithi138.stdout:9/422: chown d4/f26 536 1 2024-04-25T01:36:06.264 INFO:tasks.workunit.client.0.smithi033.stdout:2/991: creat da/d5b/d127/f147 x:0 0 0 2024-04-25T01:36:06.265 INFO:tasks.workunit.client.1.smithi138.stdout:4/379: creat d0/f70 x:0 0 0 2024-04-25T01:36:06.265 INFO:tasks.workunit.client.1.smithi138.stdout:5/424: mkdir d4/da/d6e/d22/d88 0 2024-04-25T01:36:06.268 INFO:tasks.workunit.client.0.smithi033.stdout:2/992: rename da/d27/d31/d40/d8a/dbb/dd6/f142 to da/d27/d31/d40/db1/f148 0 2024-04-25T01:36:06.268 INFO:tasks.workunit.client.0.smithi033.stdout:2/993: dread - da/d5b/d5e/f114 zero size 2024-04-25T01:36:06.269 INFO:tasks.workunit.client.1.smithi138.stdout:4/380: mknod d0/d59/c71 0 2024-04-25T01:36:06.270 INFO:tasks.workunit.client.0.smithi033.stdout:2/994: fdatasync da/d5b/d72/dd1/ded/ff8 0 2024-04-25T01:36:06.272 INFO:tasks.workunit.client.1.smithi138.stdout:6/528: dwrite dc/d10/f76 [0,4194304] 0 2024-04-25T01:36:06.272 INFO:tasks.workunit.client.1.smithi138.stdout:4/381: readlink d0/d1c/d52/l63 0 2024-04-25T01:36:06.273 INFO:tasks.workunit.client.1.smithi138.stdout:9/423: rename d4/d8/f23 to d4/d49/d72/f88 0 2024-04-25T01:36:06.274 INFO:tasks.workunit.client.1.smithi138.stdout:9/424: fsync d4/d1a/d55/f5c 0 2024-04-25T01:36:06.275 INFO:tasks.workunit.client.0.smithi033.stdout:2/995: getdents da/d5b 0 2024-04-25T01:36:06.276 INFO:tasks.workunit.client.1.smithi138.stdout:5/425: mkdir d4/da/d6e/d22/d74/d59/d83/d89 0 2024-04-25T01:36:06.277 INFO:tasks.workunit.client.1.smithi138.stdout:6/529: dread dc/d10/d28/d70/f7f [0,4194304] 0 2024-04-25T01:36:06.280 INFO:tasks.workunit.client.1.smithi138.stdout:5/426: mkdir d4/da/d6e/d22/d74/d59/d83/d8a 0 2024-04-25T01:36:06.281 INFO:tasks.workunit.client.1.smithi138.stdout:5/427: truncate d4/f18 5131350 0 2024-04-25T01:36:06.282 INFO:tasks.workunit.client.1.smithi138.stdout:6/530: creat dc/d10/d28/d80/fa2 x:0 0 0 2024-04-25T01:36:06.284 INFO:tasks.workunit.client.1.smithi138.stdout:5/428: mknod d4/da/d6e/d3c/c8b 0 2024-04-25T01:36:06.286 INFO:tasks.workunit.client.1.smithi138.stdout:6/531: creat dc/d10/d28/d70/fa3 x:0 0 0 2024-04-25T01:36:06.287 INFO:tasks.workunit.client.1.smithi138.stdout:5/429: symlink d4/da/d6e/d22/d74/d59/l8c 0 2024-04-25T01:36:06.290 INFO:tasks.workunit.client.1.smithi138.stdout:5/430: symlink d4/da/d52/d87/l8d 0 2024-04-25T01:36:06.290 INFO:tasks.workunit.client.0.smithi033.stdout:1/894: dread d2/d5/fd [4194304,4194304] 0 2024-04-25T01:36:06.291 INFO:tasks.workunit.client.1.smithi138.stdout:6/532: creat dc/d2e/d55/fa4 x:0 0 0 2024-04-25T01:36:06.292 INFO:tasks.workunit.client.0.smithi033.stdout:1/895: creat d2/d2e/d49/dba/f11e x:0 0 0 2024-04-25T01:36:06.294 INFO:tasks.workunit.client.1.smithi138.stdout:2/512: dread d0/d5a/f3c [0,4194304] 0 2024-04-25T01:36:06.294 INFO:tasks.workunit.client.1.smithi138.stdout:5/431: mkdir d4/da/d6e/d22/d74/d8e 0 2024-04-25T01:36:06.297 INFO:tasks.workunit.client.1.smithi138.stdout:2/513: truncate d0/d32/f63 633439 0 2024-04-25T01:36:06.298 INFO:tasks.workunit.client.0.smithi033.stdout:1/896: creat d2/d5/d1b/d46/d5f/d63/f11f x:0 0 0 2024-04-25T01:36:06.298 INFO:tasks.workunit.client.0.smithi033.stdout:2/996: write da/d27/d31/f3d [1806725,84047] 0 2024-04-25T01:36:06.298 INFO:tasks.workunit.client.0.smithi033.stdout:2/997: chown da/l6b 201849 1 2024-04-25T01:36:06.299 INFO:tasks.workunit.client.1.smithi138.stdout:5/432: mkdir d4/da/d6e/d22/d35/d4d/d5e/d8f 0 2024-04-25T01:36:06.300 INFO:tasks.workunit.client.1.smithi138.stdout:6/533: dread dc/d2e/f3e [0,4194304] 0 2024-04-25T01:36:06.302 INFO:tasks.workunit.client.1.smithi138.stdout:9/425: dread d4/d1a/f2d [0,4194304] 0 2024-04-25T01:36:06.302 INFO:tasks.workunit.client.1.smithi138.stdout:4/382: rmdir d0/d59 39 2024-04-25T01:36:06.302 INFO:tasks.workunit.client.0.smithi033.stdout:1/897: getdents d2/d5/d1b/dd2/d102 0 2024-04-25T01:36:06.304 INFO:tasks.workunit.client.0.smithi033.stdout:2/998: write da/d8d/ff4 [108320,7111] 0 2024-04-25T01:36:06.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/514: dwrite d0/d5a/d15/f67 [0,4194304] 0 2024-04-25T01:36:06.310 INFO:tasks.workunit.client.0.smithi033.stdout:1/898: creat d2/d11/dc9/f120 x:0 0 0 2024-04-25T01:36:06.310 INFO:tasks.workunit.client.0.smithi033.stdout:2/999: truncate da/d5b/d5e/d76/f90 379363 0 2024-04-25T01:36:06.312 INFO:tasks.workunit.client.1.smithi138.stdout:9/426: dwrite d4/f38 [0,4194304] 0 2024-04-25T01:36:06.318 INFO:tasks.workunit.client.1.smithi138.stdout:2/515: symlink d0/d5a/d15/d44/l97 0 2024-04-25T01:36:06.320 INFO:tasks.workunit.client.1.smithi138.stdout:6/534: creat dc/d10/fa5 x:0 0 0 2024-04-25T01:36:06.322 INFO:tasks.workunit.client.1.smithi138.stdout:4/383: rename d0/d1c/d52/l63 to d0/l72 0 2024-04-25T01:36:06.323 INFO:tasks.workunit.client.1.smithi138.stdout:4/384: read - d0/d1c/d48/f6f zero size 2024-04-25T01:36:06.324 INFO:tasks.workunit.client.1.smithi138.stdout:9/427: symlink d4/l89 0 2024-04-25T01:36:06.327 INFO:tasks.workunit.client.1.smithi138.stdout:2/516: mkdir d0/d5a/d15/d44/d4c/d56/d6d/d6e/d83/d98 0 2024-04-25T01:36:06.328 INFO:tasks.workunit.client.1.smithi138.stdout:2/517: stat d0/d18/d8b/f8d 0 2024-04-25T01:36:06.328 INFO:tasks.workunit.client.1.smithi138.stdout:9/428: read d4/d1a/d33/f37 [2351528,20969] 0 2024-04-25T01:36:06.328 INFO:tasks.workunit.client.1.smithi138.stdout:5/433: truncate d4/da/fe 3391655 0 2024-04-25T01:36:06.330 INFO:tasks.workunit.client.1.smithi138.stdout:4/385: dread d0/d16/d4a/f4d [0,4194304] 0 2024-04-25T01:36:06.330 INFO:tasks.workunit.client.1.smithi138.stdout:6/535: symlink dc/d2e/d55/la6 0 2024-04-25T01:36:06.330 INFO:tasks.workunit.client.1.smithi138.stdout:5/434: chown d4/da/d6e/d22/d74/d59/f68 6491 1 2024-04-25T01:36:06.331 INFO:tasks.workunit.client.1.smithi138.stdout:2/518: symlink d0/d18/l99 0 2024-04-25T01:36:06.331 INFO:tasks.workunit.client.1.smithi138.stdout:9/429: rmdir d4/d8 39 2024-04-25T01:36:06.332 INFO:tasks.workunit.client.1.smithi138.stdout:2/519: chown d0/d32/d35/f69 518 1 2024-04-25T01:36:06.333 INFO:tasks.workunit.client.1.smithi138.stdout:4/386: unlink d0/d1c/d52/c58 0 2024-04-25T01:36:06.335 INFO:tasks.workunit.client.1.smithi138.stdout:9/430: symlink d4/d1a/d33/d1d/d3f/l8a 0 2024-04-25T01:36:06.338 INFO:tasks.workunit.client.1.smithi138.stdout:4/387: symlink d0/d1c/d1e/l73 0 2024-04-25T01:36:06.339 INFO:tasks.workunit.client.1.smithi138.stdout:5/435: dwrite d4/f82 [0,4194304] 0 2024-04-25T01:36:06.340 INFO:tasks.workunit.client.1.smithi138.stdout:6/536: rename f6 to dc/d2e/d55/fa7 0 2024-04-25T01:36:06.342 INFO:tasks.workunit.client.1.smithi138.stdout:9/431: creat d4/d63/f8b x:0 0 0 2024-04-25T01:36:06.343 INFO:tasks.workunit.client.1.smithi138.stdout:4/388: mkdir d0/d16/d4a/d74 0 2024-04-25T01:36:06.349 INFO:tasks.workunit.client.1.smithi138.stdout:4/389: fsync d0/d1c/d1e/f69 0 2024-04-25T01:36:06.350 INFO:tasks.workunit.client.1.smithi138.stdout:4/390: chown d0/d1c/d1e/c4b 0 1 2024-04-25T01:36:06.352 INFO:tasks.workunit.client.1.smithi138.stdout:4/391: read d0/f30 [2442803,99442] 0 2024-04-25T01:36:06.354 INFO:tasks.workunit.client.1.smithi138.stdout:9/432: symlink d4/d8/d2c/l8c 0 2024-04-25T01:36:06.355 INFO:tasks.workunit.client.1.smithi138.stdout:9/433: write d4/d8/f61 [993221,97412] 0 2024-04-25T01:36:06.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/392: dread d0/d45/f35 [0,4194304] 0 2024-04-25T01:36:06.359 INFO:tasks.workunit.client.1.smithi138.stdout:2/520: rename d0/d32/f6b to d0/f9a 0 2024-04-25T01:36:06.360 INFO:tasks.workunit.client.1.smithi138.stdout:4/393: chown d0/d1c/d52 6033461 1 2024-04-25T01:36:06.360 INFO:tasks.workunit.client.1.smithi138.stdout:4/394: stat d0/d45 0 2024-04-25T01:36:06.367 INFO:tasks.workunit.client.1.smithi138.stdout:9/434: mknod d4/d1a/d68/c8d 0 2024-04-25T01:36:06.369 INFO:tasks.workunit.client.1.smithi138.stdout:6/537: dwrite f5 [0,4194304] 0 2024-04-25T01:36:06.371 INFO:tasks.workunit.client.1.smithi138.stdout:5/436: rename d4/da/d52/d87 to d4/da/d6e/d22/d35/d4d/d90 0 2024-04-25T01:36:06.374 INFO:tasks.workunit.client.1.smithi138.stdout:4/395: dwrite d0/d1c/f55 [0,4194304] 0 2024-04-25T01:36:06.375 INFO:tasks.workunit.client.1.smithi138.stdout:6/538: creat dc/d2e/d37/d56/fa8 x:0 0 0 2024-04-25T01:36:06.377 INFO:tasks.workunit.client.1.smithi138.stdout:9/435: mkdir d4/d1a/d33/d1d/d3f/d8e 0 2024-04-25T01:36:06.378 INFO:tasks.workunit.client.1.smithi138.stdout:2/521: rename d0/d5a/d15/d44/d4c/d93 to d0/d5a/d15/d44/d4c/d56/d9b 0 2024-04-25T01:36:06.382 INFO:tasks.workunit.client.1.smithi138.stdout:9/436: write d4/d1a/d55/f5c [2407163,29590] 0 2024-04-25T01:36:06.382 INFO:tasks.workunit.client.1.smithi138.stdout:2/522: readlink d0/d18/l75 0 2024-04-25T01:36:06.385 INFO:tasks.workunit.client.1.smithi138.stdout:4/396: dwrite d0/d1c/d48/f6f [0,4194304] 0 2024-04-25T01:36:06.388 INFO:tasks.workunit.client.1.smithi138.stdout:2/523: write d0/d32/f43 [942149,39700] 0 2024-04-25T01:36:06.394 INFO:tasks.workunit.client.1.smithi138.stdout:5/437: write d4/da/d6e/d22/d45/f54 [116503,4666] 0 2024-04-25T01:36:06.396 INFO:tasks.workunit.client.1.smithi138.stdout:2/524: rename d0/d5a/d15/l25 to d0/d5a/d15/d51/l9c 0 2024-04-25T01:36:06.397 INFO:tasks.workunit.client.1.smithi138.stdout:5/438: symlink d4/da/d6e/d22/d35/l91 0 2024-04-25T01:36:06.398 INFO:tasks.workunit.client.1.smithi138.stdout:2/525: write d0/d5a/d15/d51/f6f [536261,95996] 0 2024-04-25T01:36:06.398 INFO:tasks.workunit.client.1.smithi138.stdout:4/397: write d0/f2f [252122,59999] 0 2024-04-25T01:36:06.399 INFO:tasks.workunit.client.1.smithi138.stdout:9/437: truncate d4/d1a/f20 391952 0 2024-04-25T01:36:06.401 INFO:tasks.workunit.client.1.smithi138.stdout:9/438: chown d4/d49/c7e 3427576 1 2024-04-25T01:36:06.402 INFO:tasks.workunit.client.1.smithi138.stdout:2/526: readlink d0/d5a/d15/d44/d4c/d56/l7c 0 2024-04-25T01:36:06.403 INFO:tasks.workunit.client.1.smithi138.stdout:4/398: write d0/d1c/f2b [9164394,14833] 0 2024-04-25T01:36:06.404 INFO:tasks.workunit.client.1.smithi138.stdout:6/539: dwrite dc/f19 [4194304,4194304] 0 2024-04-25T01:36:06.404 INFO:tasks.workunit.client.1.smithi138.stdout:0/352: sync 2024-04-25T01:36:06.405 INFO:tasks.workunit.client.1.smithi138.stdout:8/593: sync 2024-04-25T01:36:06.405 INFO:tasks.workunit.client.1.smithi138.stdout:7/587: sync 2024-04-25T01:36:06.405 INFO:tasks.workunit.client.1.smithi138.stdout:9/439: mkdir d4/d1a/d55/d7f/d8f 0 2024-04-25T01:36:06.407 INFO:tasks.workunit.client.1.smithi138.stdout:5/439: dread d4/da/f20 [0,4194304] 0 2024-04-25T01:36:06.408 INFO:tasks.workunit.client.1.smithi138.stdout:2/527: creat d0/d5a/f9d x:0 0 0 2024-04-25T01:36:06.409 INFO:tasks.workunit.client.1.smithi138.stdout:0/353: read d6/fe [3515968,31479] 0 2024-04-25T01:36:06.412 INFO:tasks.workunit.client.1.smithi138.stdout:2/528: write d0/d5a/d15/d7e/f26 [7258207,43331] 0 2024-04-25T01:36:06.412 INFO:tasks.workunit.client.1.smithi138.stdout:4/399: symlink d0/d16/l75 0 2024-04-25T01:36:06.414 INFO:tasks.workunit.client.1.smithi138.stdout:6/540: fsync dc/d10/d28/d70/f7f 0 2024-04-25T01:36:06.416 INFO:tasks.workunit.client.1.smithi138.stdout:7/588: unlink da/de/d13/d30/f80 0 2024-04-25T01:36:06.417 INFO:tasks.workunit.client.1.smithi138.stdout:5/440: creat d4/da/d52/f92 x:0 0 0 2024-04-25T01:36:06.423 INFO:tasks.workunit.client.1.smithi138.stdout:2/529: rename d0/l38 to d0/d5a/d15/d44/l9e 0 2024-04-25T01:36:06.425 INFO:tasks.workunit.client.1.smithi138.stdout:0/354: dwrite d6/fe [0,4194304] 0 2024-04-25T01:36:06.427 INFO:tasks.workunit.client.1.smithi138.stdout:9/440: dwrite d4/d63/f7c [0,4194304] 0 2024-04-25T01:36:06.436 INFO:tasks.workunit.client.1.smithi138.stdout:4/400: dwrite d0/d16/f31 [4194304,4194304] 0 2024-04-25T01:36:06.445 INFO:tasks.workunit.client.0.smithi033.stdout:0/999: sync 2024-04-25T01:36:06.445 INFO:tasks.workunit.client.0.smithi033.stdout:3/992: sync 2024-04-25T01:36:06.445 INFO:tasks.workunit.client.0.smithi033.stdout:1/899: sync 2024-04-25T01:36:06.445 INFO:tasks.workunit.client.0.smithi033.stdout:9/970: sync 2024-04-25T01:36:06.445 INFO:tasks.workunit.client.1.smithi138.stdout:7/589: mkdir da/de/d1b/d49/d81/dc3 0 2024-04-25T01:36:06.445 INFO:tasks.workunit.client.1.smithi138.stdout:0/355: dwrite d6/df/f4d [0,4194304] 0 2024-04-25T01:36:06.446 INFO:tasks.workunit.client.1.smithi138.stdout:5/441: creat d4/da/d6e/d22/f93 x:0 0 0 2024-04-25T01:36:06.446 INFO:tasks.workunit.client.1.smithi138.stdout:2/530: mknod d0/d18/d8b/c9f 0 2024-04-25T01:36:06.446 INFO:tasks.workunit.client.0.smithi033.stdout:9/971: chown d0/d2/dd/da7/f119 1 1 2024-04-25T01:36:06.447 INFO:tasks.workunit.client.0.smithi033.stdout:3/993: symlink d8/dff/l15e 0 2024-04-25T01:36:06.449 INFO:tasks.workunit.client.0.smithi033.stdout:9/972: dread - d0/d2/dd/d5f/d11b/d13c/df9/f147 zero size 2024-04-25T01:36:06.452 INFO:tasks.workunit.client.1.smithi138.stdout:8/594: dread d3/d16/d88/da/db/f11 [0,4194304] 0 2024-04-25T01:36:06.453 INFO:tasks.workunit.client.1.smithi138.stdout:2/531: chown d0/d21/d45/d8a/d8e 0 1 2024-04-25T01:36:06.453 INFO:tasks.workunit.client.1.smithi138.stdout:5/442: write d4/f62 [653706,75064] 0 2024-04-25T01:36:06.454 INFO:tasks.workunit.client.1.smithi138.stdout:7/590: symlink da/de/d1b/d4b/lc4 0 2024-04-25T01:36:06.454 INFO:tasks.workunit.client.0.smithi033.stdout:9/973: read - d0/d2/d43/d4b/dc5/de0/d7e/d58/fd7 zero size 2024-04-25T01:36:06.455 INFO:tasks.workunit.client.1.smithi138.stdout:9/441: dwrite d4/d57/f67 [0,4194304] 0 2024-04-25T01:36:06.455 INFO:tasks.workunit.client.0.smithi033.stdout:3/994: write d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d98/da1/dc3/f126 [3159977,93475] 0 2024-04-25T01:36:06.456 INFO:tasks.workunit.client.0.smithi033.stdout:1/900: dwrite d2/d2e/dcd/fc8 [4194304,4194304] 0 2024-04-25T01:36:06.457 INFO:tasks.workunit.client.1.smithi138.stdout:3/607: sync 2024-04-25T01:36:06.457 INFO:tasks.workunit.client.1.smithi138.stdout:1/442: sync 2024-04-25T01:36:06.458 INFO:tasks.workunit.client.1.smithi138.stdout:2/532: fdatasync d0/d5a/d15/d7e/f4a 0 2024-04-25T01:36:06.458 INFO:tasks.workunit.client.1.smithi138.stdout:8/595: chown d3/d16/d1f/d31/d82/d8a/faf 5056 1 2024-04-25T01:36:06.459 INFO:tasks.workunit.client.0.smithi033.stdout:1/901: fsync d2/d5/d75/d59/d10e/f119 0 2024-04-25T01:36:06.460 INFO:tasks.workunit.client.1.smithi138.stdout:5/443: stat d4/da/d6e/d5c 0 2024-04-25T01:36:06.464 INFO:tasks.workunit.client.0.smithi033.stdout:1/902: dread - d2/d2e/d34/d60/fb8 zero size 2024-04-25T01:36:06.467 INFO:tasks.workunit.client.1.smithi138.stdout:9/442: rename d4/d1a/d33/d1d/d3f to d4/d8/d90 0 2024-04-25T01:36:06.467 INFO:tasks.workunit.client.0.smithi033.stdout:1/903: creat d2/d5/dee/dfb/f121 x:0 0 0 2024-04-25T01:36:06.468 INFO:tasks.workunit.client.1.smithi138.stdout:3/608: chown db/df/f4f 89483 1 2024-04-25T01:36:06.470 INFO:tasks.workunit.client.1.smithi138.stdout:6/541: write dc/d10/d28/f34 [3166824,127706] 0 2024-04-25T01:36:06.472 INFO:tasks.workunit.client.0.smithi033.stdout:1/904: creat d2/d5/d75/d59/f122 x:0 0 0 2024-04-25T01:36:06.473 INFO:tasks.workunit.client.1.smithi138.stdout:8/596: creat d3/d16/d1f/fba x:0 0 0 2024-04-25T01:36:06.473 INFO:tasks.workunit.client.1.smithi138.stdout:9/443: fsync d4/d1a/d55/d7f/f83 0 2024-04-25T01:36:06.473 INFO:tasks.workunit.client.1.smithi138.stdout:2/533: symlink d0/d5a/d15/d44/d4c/d56/d6d/d6e/la0 0 2024-04-25T01:36:06.475 INFO:tasks.workunit.client.1.smithi138.stdout:4/401: write d0/d1c/d1e/f69 [8745974,102355] 0 2024-04-25T01:36:06.475 INFO:tasks.workunit.client.1.smithi138.stdout:0/356: dwrite d6/df/f5c [0,4194304] 0 2024-04-25T01:36:06.476 INFO:tasks.workunit.client.0.smithi033.stdout:1/905: creat d2/d5/d6c/f123 x:0 0 0 2024-04-25T01:36:06.476 INFO:tasks.workunit.client.0.smithi033.stdout:1/906: fdatasync d2/d11/f91 0 2024-04-25T01:36:06.479 INFO:tasks.workunit.client.0.smithi033.stdout:1/907: creat d2/d5/d6c/dc6/f124 x:0 0 0 2024-04-25T01:36:06.480 INFO:tasks.workunit.client.1.smithi138.stdout:3/609: creat db/df/d17/d20/d57/fc1 x:0 0 0 2024-04-25T01:36:06.482 INFO:tasks.workunit.client.1.smithi138.stdout:0/357: write d6/f26 [1190985,70250] 0 2024-04-25T01:36:06.482 INFO:tasks.workunit.client.1.smithi138.stdout:5/444: rename d4/da/d6e/d22/d35/d4d/l63 to d4/da/d6e/d22/d74/d8e/l94 0 2024-04-25T01:36:06.484 INFO:tasks.workunit.client.1.smithi138.stdout:1/443: dwrite d3/d27/d3e/f88 [0,4194304] 0 2024-04-25T01:36:06.486 INFO:tasks.workunit.client.1.smithi138.stdout:6/542: creat dc/d10/d28/d70/fa9 x:0 0 0 2024-04-25T01:36:06.488 INFO:tasks.workunit.client.1.smithi138.stdout:5/445: chown d4/da/d52/f7c 54060 1 2024-04-25T01:36:06.488 INFO:tasks.workunit.client.1.smithi138.stdout:7/591: dwrite da/de/d1b/d4b/d5b/f46 [0,4194304] 0 2024-04-25T01:36:06.490 INFO:tasks.workunit.client.1.smithi138.stdout:6/543: write dc/d10/f15 [3169841,23773] 0 2024-04-25T01:36:06.492 INFO:tasks.workunit.client.1.smithi138.stdout:3/610: chown db/d14/d40/fb0 9038 1 2024-04-25T01:36:06.493 INFO:tasks.workunit.client.0.smithi033.stdout:3/995: dwrite d8/dd/d11/f8e [0,4194304] 0 2024-04-25T01:36:06.497 INFO:tasks.workunit.client.0.smithi033.stdout:3/996: dread d8/dd/d11/fbc [0,4194304] 0 2024-04-25T01:36:06.497 INFO:tasks.workunit.client.1.smithi138.stdout:7/592: write da/de/d1b/d49/d81/d51/faa [2260293,130001] 0 2024-04-25T01:36:06.500 INFO:tasks.workunit.client.1.smithi138.stdout:5/446: write d4/da/d6e/d22/f7b [1032059,95253] 0 2024-04-25T01:36:06.500 INFO:tasks.workunit.client.1.smithi138.stdout:2/534: creat d0/fa1 x:0 0 0 2024-04-25T01:36:06.500 INFO:tasks.workunit.client.1.smithi138.stdout:0/358: link d6/df/f14 d6/d19/f74 0 2024-04-25T01:36:06.503 INFO:tasks.workunit.client.1.smithi138.stdout:9/444: dwrite d4/d1a/d55/f5f [0,4194304] 0 2024-04-25T01:36:06.508 INFO:tasks.workunit.client.0.smithi033.stdout:3/997: rename d8/dd/f83 to d8/dd/d3d/d63/f15f 0 2024-04-25T01:36:06.509 INFO:tasks.workunit.client.0.smithi033.stdout:9/974: dwrite d0/d2/dd/d20/d4e/f78 [0,4194304] 0 2024-04-25T01:36:06.514 INFO:tasks.workunit.client.1.smithi138.stdout:3/611: getdents db/df/d17/d20/d57/d6a 0 2024-04-25T01:36:06.514 INFO:tasks.workunit.client.1.smithi138.stdout:9/445: symlink d4/d8/d90/d52/l91 0 2024-04-25T01:36:06.514 INFO:tasks.workunit.client.1.smithi138.stdout:2/535: rename d0/d5a/d15/d7e to d0/d5a/d15/d44/d4c/d56/da2 0 2024-04-25T01:36:06.516 INFO:tasks.workunit.client.0.smithi033.stdout:9/975: creat d0/d2/d43/d73/dc4/dcb/df0/f152 x:0 0 0 2024-04-25T01:36:06.518 INFO:tasks.workunit.client.0.smithi033.stdout:1/908: write d2/d11/d28/dd4/f11c [1291541,127744] 0 2024-04-25T01:36:06.519 INFO:tasks.workunit.client.1.smithi138.stdout:7/593: dwrite da/de/d1b/d49/f89 [0,4194304] 0 2024-04-25T01:36:06.525 INFO:tasks.workunit.client.0.smithi033.stdout:1/909: creat d2/d5/d1b/d46/d5f/f125 x:0 0 0 2024-04-25T01:36:06.525 INFO:tasks.workunit.client.1.smithi138.stdout:2/536: unlink d0/d5a/f4e 0 2024-04-25T01:36:06.527 INFO:tasks.workunit.client.0.smithi033.stdout:1/910: truncate d2/d11/f6d 687332 0 2024-04-25T01:36:06.530 INFO:tasks.workunit.client.1.smithi138.stdout:0/359: write d6/df/d41/f55 [799869,95246] 0 2024-04-25T01:36:06.530 INFO:tasks.workunit.client.1.smithi138.stdout:1/444: dwrite d3/d27/d3e/f6b [0,4194304] 0 2024-04-25T01:36:06.531 INFO:tasks.workunit.client.0.smithi033.stdout:1/911: creat d2/d5/d1b/d46/d5f/d63/d64/f126 x:0 0 0 2024-04-25T01:36:06.535 INFO:tasks.workunit.client.0.smithi033.stdout:1/912: mkdir d2/d5/d75/d59/d127 0 2024-04-25T01:36:06.535 INFO:tasks.workunit.client.1.smithi138.stdout:3/612: dread db/df/d17/d5d/f6b [0,4194304] 0 2024-04-25T01:36:06.535 INFO:tasks.workunit.client.1.smithi138.stdout:2/537: read d0/d5a/d15/f3b [188430,81010] 0 2024-04-25T01:36:06.536 INFO:tasks.workunit.client.0.smithi033.stdout:1/913: chown d2/d5/d1b/f118 356 1 2024-04-25T01:36:06.537 INFO:tasks.workunit.client.0.smithi033.stdout:1/914: stat d2/d2e/d49/dba/f11e 0 2024-04-25T01:36:06.537 INFO:tasks.workunit.client.0.smithi033.stdout:3/998: dread d8/dd/d3d/d63/d7c/f70 [0,4194304] 0 2024-04-25T01:36:06.538 INFO:tasks.workunit.client.1.smithi138.stdout:7/594: dwrite da/de/f27 [4194304,4194304] 0 2024-04-25T01:36:06.538 INFO:tasks.workunit.client.1.smithi138.stdout:3/613: dread - db/df/d17/f4a zero size 2024-04-25T01:36:06.540 INFO:tasks.workunit.client.1.smithi138.stdout:9/446: link f1 d4/d1a/f92 0 2024-04-25T01:36:06.543 INFO:tasks.workunit.client.0.smithi033.stdout:3/999: mknod d8/dd/d3d/d63/d7c/ded/dbf/dc5/de5/d12a/c160 0 2024-04-25T01:36:06.544 INFO:tasks.workunit.client.1.smithi138.stdout:0/360: mkdir d6/d2f/d75 0 2024-04-25T01:36:06.544 INFO:tasks.workunit.client.0.smithi033.stdout:1/915: getdents d2/d11/d28/dd4 0 2024-04-25T01:36:06.547 INFO:tasks.workunit.client.0.smithi033.stdout:1/916: fsync d2/d11/d28/fe1 0 2024-04-25T01:36:06.547 INFO:tasks.workunit.client.1.smithi138.stdout:1/445: rename d3/d27/d24/f3b to d3/d4/d53/f8c 0 2024-04-25T01:36:06.549 INFO:tasks.workunit.client.1.smithi138.stdout:7/595: mknod da/de/d1b/d49/d81/d51/d6e/da2/dae/cc5 0 2024-04-25T01:36:06.549 INFO:tasks.workunit.client.1.smithi138.stdout:3/614: truncate db/f3f 427156 0 2024-04-25T01:36:06.552 INFO:tasks.workunit.client.1.smithi138.stdout:7/596: dread - da/de/d13/d30/f93 zero size 2024-04-25T01:36:06.552 INFO:tasks.workunit.client.1.smithi138.stdout:3/615: dread - db/fa2 zero size 2024-04-25T01:36:06.554 INFO:tasks.workunit.client.1.smithi138.stdout:0/361: link d6/d19/f4b d6/d2f/d75/f76 0 2024-04-25T01:36:06.555 INFO:tasks.workunit.client.0.smithi033.stdout:1/917: dwrite d2/d5/d1b/d46/d5f/d63/f111 [0,4194304] 0 2024-04-25T01:36:06.557 INFO:tasks.workunit.client.1.smithi138.stdout:2/538: dwrite d0/d32/d35/f60 [0,4194304] 0 2024-04-25T01:36:06.557 INFO:tasks.workunit.client.1.smithi138.stdout:3/616: fsync db/df/d17/d23/d34/f5f 0 2024-04-25T01:36:06.560 INFO:tasks.workunit.client.1.smithi138.stdout:5/447: dwrite d4/da/d27/f1f [0,4194304] 0 2024-04-25T01:36:06.561 INFO:tasks.workunit.client.0.smithi033.stdout:1/918: symlink d2/d5/d75/l128 0 2024-04-25T01:36:06.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.586+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f673410fa90 con 0x7f671c06b7e0 2024-04-25T01:36:06.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.588+0000 7f671b7fe700 1 -- 172.21.15.33:0/1555834556 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f673410fa90 con 0x7f671c06b7e0 2024-04-25T01:36:06.566 INFO:tasks.workunit.client.1.smithi138.stdout:6/544: dread dc/d2e/d55/f5a [0,4194304] 0 2024-04-25T01:36:06.568 INFO:tasks.workunit.client.1.smithi138.stdout:6/545: truncate fb 1827060 0 2024-04-25T01:36:06.568 INFO:tasks.workunit.client.1.smithi138.stdout:7/597: mknod da/de/d1b/d49/d81/dc3/cc6 0 2024-04-25T01:36:06.569 INFO:tasks.workunit.client.0.smithi033.stdout:1/919: rmdir d2/de2/de5 0 2024-04-25T01:36:06.569 INFO:tasks.workunit.client.0.smithi033.stdout:9/976: dread d0/d2/dd/d20/d4e/f76 [0,4194304] 0 2024-04-25T01:36:06.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.595+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f671c06b7e0 msgr2=0x7f671c06dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.595+0000 7f673984b700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f671c06b7e0 0x7f671c06dc90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6724004f90 tx=0x7f672401c040 comp rx=0 tx=0).stop 2024-04-25T01:36:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.595+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67340fff00 msgr2=0x7f673410e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.595+0000 7f673984b700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67340fff00 0x7f673410e000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f672800b530 tx=0x7f67280090f0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.595+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 shutdown_connections 2024-04-25T01:36:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.595+0000 7f673984b700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f671c06b7e0 0x7f671c06dc90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.595+0000 7f673984b700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67340662a0 0x7f673410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.595+0000 7f673984b700 1 --2- 172.21.15.33:0/1555834556 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67340fff00 0x7f673410e000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.595+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 >> 172.21.15.33:0/1555834556 conn(0x7f67340f7430 msgr2=0x7f67340f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:06.572 INFO:tasks.workunit.client.1.smithi138.stdout:2/539: mkdir d0/d5a/d15/d44/d4c/d56/d6d/da3 0 2024-04-25T01:36:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.596+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 shutdown_connections 2024-04-25T01:36:06.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.596+0000 7f673984b700 1 -- 172.21.15.33:0/1555834556 wait complete. 2024-04-25T01:36:06.577 INFO:tasks.workunit.client.0.smithi033.stdout:1/920: link d2/d5/l100 d2/d2e/d49/l129 0 2024-04-25T01:36:06.577 INFO:tasks.workunit.client.1.smithi138.stdout:9/447: write d4/d1a/f20 [1177269,54388] 0 2024-04-25T01:36:06.580 INFO:tasks.workunit.client.0.smithi033.stdout:1/921: chown d2/d5/d75/d59/l5d 66693 1 2024-04-25T01:36:06.580 INFO:tasks.workunit.client.1.smithi138.stdout:6/546: creat dc/d10/d67/d8e/faa x:0 0 0 2024-04-25T01:36:06.581 INFO:tasks.workunit.client.0.smithi033.stdout:1/922: readlink d2/d2e/d49/l103 0 2024-04-25T01:36:06.581 INFO:tasks.workunit.client.1.smithi138.stdout:7/598: mkdir da/de/d1b/d49/d81/dc7 0 2024-04-25T01:36:06.583 INFO:tasks.workunit.client.1.smithi138.stdout:6/547: creat dc/d10/d28/d70/fab x:0 0 0 2024-04-25T01:36:06.583 INFO:tasks.workunit.client.0.smithi033.stdout:1/923: write d2/d2e/d49/dba/f11e [907731,87966] 0 2024-04-25T01:36:06.583 INFO:tasks.workunit.client.1.smithi138.stdout:9/448: read d4/f25 [675273,47760] 0 2024-04-25T01:36:06.586 INFO:tasks.workunit.client.1.smithi138.stdout:9/449: fdatasync d4/d1a/d3c/f51 0 2024-04-25T01:36:06.587 INFO:tasks.workunit.client.0.smithi033.stdout:1/924: dread d2/d5/d75/d59/d83/fd0 [0,4194304] 0 2024-04-25T01:36:06.588 INFO:tasks.workunit.client.1.smithi138.stdout:2/540: getdents d0/d5a/d15/d44/d4c/d56 0 2024-04-25T01:36:06.592 INFO:tasks.workunit.client.1.smithi138.stdout:1/446: dread d3/d45/d72/d57/f8a [0,4194304] 0 2024-04-25T01:36:06.593 INFO:tasks.workunit.client.0.smithi033.stdout:9/977: truncate d0/d2/d43/d4b/dc5/de0/d7e/d58/fa6 1601745 0 2024-04-25T01:36:06.594 INFO:tasks.workunit.client.1.smithi138.stdout:3/617: truncate db/df/d17/d23/f67 240948 0 2024-04-25T01:36:06.595 INFO:tasks.workunit.client.1.smithi138.stdout:0/362: dread d6/df/d3a/f3d [0,4194304] 0 2024-04-25T01:36:06.595 INFO:tasks.workunit.client.1.smithi138.stdout:1/447: chown d3/db/f5a 104611 1 2024-04-25T01:36:06.596 INFO:tasks.workunit.client.1.smithi138.stdout:6/548: dwrite dc/d2e/d37/d56/fa8 [0,4194304] 0 2024-04-25T01:36:06.597 INFO:tasks.workunit.client.1.smithi138.stdout:0/363: readlink d6/df/d3a/l50 0 2024-04-25T01:36:06.599 INFO:tasks.workunit.client.0.smithi033.stdout:9/978: creat d0/d2/dd/d20/f153 x:0 0 0 2024-04-25T01:36:06.599 INFO:tasks.workunit.client.0.smithi033.stdout:1/925: dwrite d2/d5/d75/d59/d83/fc1 [0,4194304] 0 2024-04-25T01:36:06.599 INFO:tasks.workunit.client.1.smithi138.stdout:2/541: rename d0/d18/d8b/f8d to d0/d18/d8b/fa4 0 2024-04-25T01:36:06.601 INFO:tasks.workunit.client.1.smithi138.stdout:2/542: stat d0/d18/f6c 0 2024-04-25T01:36:06.603 INFO:tasks.workunit.client.1.smithi138.stdout:1/448: rename d3/d4/f9 to d3/d45/d72/d3d/f8d 0 2024-04-25T01:36:06.606 INFO:tasks.workunit.client.0.smithi033.stdout:9/979: symlink d0/d110/l154 0 2024-04-25T01:36:06.607 INFO:tasks.workunit.client.1.smithi138.stdout:9/450: getdents d4/d1a/d33/d1d 0 2024-04-25T01:36:06.608 INFO:tasks.workunit.client.1.smithi138.stdout:2/543: read d0/d5a/d15/d51/f6f [533778,45360] 0 2024-04-25T01:36:06.609 INFO:tasks.workunit.client.0.smithi033.stdout:9/980: mknod d0/d2/d43/d4b/dc5/de0/d6e/c155 0 2024-04-25T01:36:06.610 INFO:tasks.workunit.client.0.smithi033.stdout:9/981: chown d0/d2/d43/d4b/dc5/de0/f70 15889837 1 2024-04-25T01:36:06.611 INFO:tasks.workunit.client.1.smithi138.stdout:2/544: chown d0/d32/d35/f60 52686 1 2024-04-25T01:36:06.612 INFO:tasks.workunit.client.1.smithi138.stdout:0/364: link d6/f28 d6/d19/f77 0 2024-04-25T01:36:06.613 INFO:tasks.workunit.client.1.smithi138.stdout:0/365: readlink d6/d39/l40 0 2024-04-25T01:36:06.614 INFO:tasks.workunit.client.1.smithi138.stdout:1/449: rename d3/db/d4a to d3/d45/d72/d57/d8e 0 2024-04-25T01:36:06.617 INFO:tasks.workunit.client.1.smithi138.stdout:0/366: dread - d6/d39/f70 zero size 2024-04-25T01:36:06.618 INFO:tasks.workunit.client.1.smithi138.stdout:3/618: dwrite db/df/d19/d1a/f44 [0,4194304] 0 2024-04-25T01:36:06.618 INFO:tasks.workunit.client.1.smithi138.stdout:2/545: rmdir d0/d5a 39 2024-04-25T01:36:06.619 INFO:tasks.workunit.client.0.smithi033.stdout:1/926: dwrite d2/d2e/d34/d60/fb8 [0,4194304] 0 2024-04-25T01:36:06.619 INFO:tasks.workunit.client.0.smithi033.stdout:9/982: dwrite d0/d2/dd/d5f/d11b/f98 [0,4194304] 0 2024-04-25T01:36:06.622 INFO:tasks.workunit.client.1.smithi138.stdout:7/599: dread da/de/d1b/f33 [0,4194304] 0 2024-04-25T01:36:06.624 INFO:tasks.workunit.client.1.smithi138.stdout:1/450: truncate d3/d45/d72/d57/d8e/f69 786505 0 2024-04-25T01:36:06.624 INFO:tasks.workunit.client.1.smithi138.stdout:2/546: write d0/f9 [4899007,77890] 0 2024-04-25T01:36:06.625 INFO:tasks.workunit.client.0.smithi033.stdout:9/983: mkdir d0/d2/d43/d4b/dc5/de0/d6e/d156 0 2024-04-25T01:36:06.626 INFO:tasks.workunit.client.1.smithi138.stdout:0/367: symlink d6/d1d/l78 0 2024-04-25T01:36:06.628 INFO:tasks.workunit.client.0.smithi033.stdout:1/927: fdatasync d2/d5/d75/f53 0 2024-04-25T01:36:06.630 INFO:tasks.workunit.client.0.smithi033.stdout:9/984: fdatasync d0/f33 0 2024-04-25T01:36:06.631 INFO:tasks.workunit.client.1.smithi138.stdout:7/600: symlink da/de/d13/d21/d6f/lc8 0 2024-04-25T01:36:06.631 INFO:tasks.workunit.client.1.smithi138.stdout:6/549: truncate dc/d2e/d37/d56/fa8 201485 0 2024-04-25T01:36:06.631 INFO:tasks.workunit.client.1.smithi138.stdout:3/619: truncate db/df/d19/d1a/d9e/fb9 875184 0 2024-04-25T01:36:06.632 INFO:tasks.workunit.client.1.smithi138.stdout:3/620: readlink db/df/l13 0 2024-04-25T01:36:06.632 INFO:tasks.workunit.client.1.smithi138.stdout:1/451: rmdir d3/d27/d3e 39 2024-04-25T01:36:06.633 INFO:tasks.workunit.client.0.smithi033.stdout:1/928: write d2/d11/d28/dd4/f11c [1773785,38651] 0 2024-04-25T01:36:06.634 INFO:tasks.workunit.client.1.smithi138.stdout:3/621: chown db/df/lb6 0 1 2024-04-25T01:36:06.637 INFO:tasks.workunit.client.1.smithi138.stdout:0/368: dread d6/df/f4d [0,4194304] 0 2024-04-25T01:36:06.637 INFO:tasks.workunit.client.1.smithi138.stdout:2/547: read d0/d5a/f87 [2103598,39638] 0 2024-04-25T01:36:06.640 INFO:tasks.workunit.client.1.smithi138.stdout:6/550: write dc/f72 [20647,8309] 0 2024-04-25T01:36:06.641 INFO:tasks.workunit.client.1.smithi138.stdout:7/601: rmdir da/de/d13/d21/dbc/d94/d9d 39 2024-04-25T01:36:06.644 INFO:tasks.workunit.client.1.smithi138.stdout:1/452: mknod d3/d45/c8f 0 2024-04-25T01:36:06.647 INFO:tasks.workunit.client.1.smithi138.stdout:1/453: write d3/d6d/f89 [825651,34951] 0 2024-04-25T01:36:06.648 INFO:tasks.workunit.client.1.smithi138.stdout:2/548: dwrite d0/d32/d35/f70 [0,4194304] 0 2024-04-25T01:36:06.648 INFO:tasks.workunit.client.1.smithi138.stdout:0/369: dread d6/d39/d43/f21 [0,4194304] 0 2024-04-25T01:36:06.650 INFO:tasks.workunit.client.1.smithi138.stdout:9/451: dread d4/d1a/f44 [0,4194304] 0 2024-04-25T01:36:06.653 INFO:tasks.workunit.client.1.smithi138.stdout:0/370: read d6/d19/f33 [6690609,795] 0 2024-04-25T01:36:06.655 INFO:tasks.workunit.client.1.smithi138.stdout:9/452: dread d4/d1a/d33/f16 [0,4194304] 0 2024-04-25T01:36:06.663 INFO:tasks.workunit.client.1.smithi138.stdout:2/549: fsync d0/d32/f63 0 2024-04-25T01:36:06.664 INFO:tasks.workunit.client.1.smithi138.stdout:3/622: truncate db/f62 2972178 0 2024-04-25T01:36:06.668 INFO:tasks.workunit.client.1.smithi138.stdout:7/602: dwrite da/de/d13/d30/f65 [0,4194304] 0 2024-04-25T01:36:06.668 INFO:tasks.workunit.client.1.smithi138.stdout:3/623: fsync db/d14/d40/f87 0 2024-04-25T01:36:06.669 INFO:tasks.workunit.client.0.smithi033.stdout:9/985: dread d0/d2/dd/d20/d4e/f90 [0,4194304] 0 2024-04-25T01:36:06.669 INFO:tasks.workunit.client.1.smithi138.stdout:9/453: dwrite d4/d8/d90/f78 [0,4194304] 0 2024-04-25T01:36:06.670 INFO:tasks.workunit.client.1.smithi138.stdout:7/603: dread - da/de/d1b/d49/d81/d51/d6e/da2/dae/fb8 zero size 2024-04-25T01:36:06.671 INFO:tasks.workunit.client.1.smithi138.stdout:9/454: dread - d4/d1a/d55/d7f/f83 zero size 2024-04-25T01:36:06.671 INFO:tasks.workunit.client.0.smithi033.stdout:1/929: truncate d2/f81 2883640 0 2024-04-25T01:36:06.672 INFO:tasks.workunit.client.0.smithi033.stdout:1/930: chown d2/d7f/db0/dd6 1749626918 1 2024-04-25T01:36:06.674 INFO:tasks.workunit.client.0.smithi033.stdout:9/986: unlink d0/d2/dd/df/f2b 0 2024-04-25T01:36:06.676 INFO:tasks.workunit.client.0.smithi033.stdout:1/931: mknod d2/d5/d1b/c12a 0 2024-04-25T01:36:06.677 INFO:tasks.workunit.client.1.smithi138.stdout:6/551: dread dc/d10/d28/f2b [0,4194304] 0 2024-04-25T01:36:06.677 INFO:tasks.workunit.client.0.smithi033.stdout:1/932: chown d2/d5/d1b/dd2 4677068 1 2024-04-25T01:36:06.679 INFO:tasks.workunit.client.0.smithi033.stdout:1/933: fdatasync d2/d5/d1b/d46/d5f/d63/d64/f108 0 2024-04-25T01:36:06.686 INFO:tasks.workunit.client.0.smithi033.stdout:9/987: getdents d0/d2/dd/d20 0 2024-04-25T01:36:06.712 INFO:tasks.workunit.client.0.smithi033.stdout:1/934: dwrite d2/d5/d1b/d46/d5f/d63/f65 [0,4194304] 0 2024-04-25T01:36:06.713 INFO:tasks.workunit.client.0.smithi033.stdout:9/988: dwrite d0/d2/dd/dba/fdb [0,4194304] 0 2024-04-25T01:36:06.717 INFO:tasks.workunit.client.0.smithi033.stdout:9/989: mkdir d0/d2/d43/d4b/dc5/d157 0 2024-04-25T01:36:06.724 INFO:tasks.workunit.client.0.smithi033.stdout:9/990: truncate d0/d2/d43/d4b/d55/f8d 562816 0 2024-04-25T01:36:06.726 INFO:tasks.workunit.client.0.smithi033.stdout:9/991: mknod d0/d11/da1/c158 0 2024-04-25T01:36:06.728 INFO:tasks.workunit.client.0.smithi033.stdout:9/992: write d0/d2/d43/d4b/dc5/de0/d42/dea/f150 [669881,109050] 0 2024-04-25T01:36:06.731 INFO:tasks.workunit.client.0.smithi033.stdout:9/993: mkdir d0/d2/dd/df/d159 0 2024-04-25T01:36:06.733 INFO:tasks.workunit.client.0.smithi033.stdout:9/994: chown d0/d2/d43/d4b/dc5/de0/d7e/d58/c65 585660 1 2024-04-25T01:36:06.733 INFO:tasks.workunit.client.0.smithi033.stdout:1/935: dread d2/f81 [0,4194304] 0 2024-04-25T01:36:06.737 INFO:tasks.workunit.client.1.smithi138.stdout:0/371: link d6/d39/d43/c2a d6/d2f/d31/c79 0 2024-04-25T01:36:06.737 INFO:tasks.workunit.client.0.smithi033.stdout:9/995: rename d0/d2/dd/dba/dc1/fcd to d0/d2/d43/d4b/f15a 0 2024-04-25T01:36:06.737 INFO:tasks.workunit.client.1.smithi138.stdout:0/372: stat d6/fe 0 2024-04-25T01:36:06.739 INFO:tasks.workunit.client.1.smithi138.stdout:3/624: creat db/d9b/fc2 x:0 0 0 2024-04-25T01:36:06.739 INFO:tasks.workunit.client.1.smithi138.stdout:9/455: creat d4/d63/d86/f93 x:0 0 0 2024-04-25T01:36:06.740 INFO:tasks.workunit.client.0.smithi033.stdout:9/996: mknod d0/d2/d43/d73/c15b 0 2024-04-25T01:36:06.741 INFO:tasks.workunit.client.1.smithi138.stdout:6/552: rmdir dc/d2e/d55 39 2024-04-25T01:36:06.741 INFO:tasks.workunit.client.0.smithi033.stdout:1/936: rename d2/d2e/d49/c115 to d2/d2e/d34/c12b 0 2024-04-25T01:36:06.743 INFO:tasks.workunit.client.1.smithi138.stdout:0/373: unlink d6/c60 0 2024-04-25T01:36:06.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.769+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/1012929866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 msgr2=0x7fbd580fd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.769+0000 7fbd5e7f0700 1 --2- 172.21.15.33:0/1012929866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 0x7fbd580fd790 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbd48004880 tx=0x7fbd48009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:06.744 INFO:tasks.workunit.client.1.smithi138.stdout:3/625: creat db/df/d17/d23/d88/fc3 x:0 0 0 2024-04-25T01:36:06.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.770+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/1012929866 shutdown_connections 2024-04-25T01:36:06.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.770+0000 7fbd5e7f0700 1 --2- 172.21.15.33:0/1012929866 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd580fdcd0 0x7fbd58100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.770+0000 7fbd5e7f0700 1 --2- 172.21.15.33:0/1012929866 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 0x7fbd580fd790 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.770+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/1012929866 >> 172.21.15.33:0/1012929866 conn(0x7fbd580f7430 msgr2=0x7fbd580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:06.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.770+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/1012929866 shutdown_connections 2024-04-25T01:36:06.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.770+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/1012929866 wait complete. 2024-04-25T01:36:06.746 INFO:tasks.workunit.client.1.smithi138.stdout:9/456: dread d4/d8/d90/f78 [0,4194304] 0 2024-04-25T01:36:06.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.771+0000 7fbd5e7f0700 1 Processor -- start 2024-04-25T01:36:06.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.771+0000 7fbd5e7f0700 1 -- start start 2024-04-25T01:36:06.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.771+0000 7fbd5e7f0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 0x7fbd58114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.747 INFO:tasks.workunit.client.1.smithi138.stdout:0/374: truncate d6/df/d41/d49/f63 187399 0 2024-04-25T01:36:06.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.772+0000 7fbd5e7f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd580fdcd0 0x7fbd58114550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.772+0000 7fbd5e7f0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58114b50 con 0x7fbd580662c0 2024-04-25T01:36:06.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.772+0000 7fbd5e7f0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd581153a0 con 0x7fbd580fdcd0 2024-04-25T01:36:06.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.772+0000 7fbd57fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 0x7fbd58114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.772+0000 7fbd57fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 0x7fbd58114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60222/0 (socket says 172.21.15.33:60222) 2024-04-25T01:36:06.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.772+0000 7fbd57fff700 1 -- 172.21.15.33:0/2537095869 learned_addr learned my addr 172.21.15.33:0/2537095869 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:06.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.772+0000 7fbd57fff700 1 -- 172.21.15.33:0/2537095869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd580fdcd0 msgr2=0x7fbd58114550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:06.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.772+0000 7fbd57fff700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd580fdcd0 0x7fbd58114550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:06.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.772+0000 7fbd57fff700 1 -- 172.21.15.33:0/2537095869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd48005040 con 0x7fbd580662c0 2024-04-25T01:36:06.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.773+0000 7fbd57fff700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 0x7fbd58114010 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:06.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.773+0000 7fbd54ff9700 1 -- 172.21.15.33:0/2537095869 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd4800e030 con 0x7fbd580662c0 2024-04-25T01:36:06.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.773+0000 7fbd54ff9700 1 -- 172.21.15.33:0/2537095869 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd48013e60 con 0x7fbd580662c0 2024-04-25T01:36:06.750 INFO:tasks.workunit.client.1.smithi138.stdout:0/375: write d6/d1d/f5f [3248105,118632] 0 2024-04-25T01:36:06.750 INFO:tasks.workunit.client.1.smithi138.stdout:6/553: fsync dc/d2e/d55/d81/f9f 0 2024-04-25T01:36:06.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.774+0000 7fbd54ff9700 1 -- 172.21.15.33:0/2537095869 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd4801ce90 con 0x7fbd580662c0 2024-04-25T01:36:06.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.774+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58115620 con 0x7fbd580662c0 2024-04-25T01:36:06.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.774+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd58115aa0 con 0x7fbd580662c0 2024-04-25T01:36:06.750 INFO:tasks.workunit.client.0.smithi033.stdout:1/937: dwrite d2/d5/d75/d59/d83/fc1 [4194304,4194304] 0 2024-04-25T01:36:06.751 INFO:tasks.workunit.client.1.smithi138.stdout:0/376: truncate d6/f28 4578710 0 2024-04-25T01:36:06.751 INFO:tasks.workunit.client.1.smithi138.stdout:1/454: rename d3/d27/d3e/d51/d62/d65 to d3/d27/d3e/d90 0 2024-04-25T01:36:06.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.777+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5805f5c0 con 0x7fbd580662c0 2024-04-25T01:36:06.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.777+0000 7fbd54ff9700 1 -- 172.21.15.33:0/2537095869 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7fbd4801c400 con 0x7fbd580662c0 2024-04-25T01:36:06.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.777+0000 7fbd54ff9700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fbd4006b8b0 0x7fbd4006dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:06.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.778+0000 7fbd54ff9700 1 -- 172.21.15.33:0/2537095869 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fbd4808be20 con 0x7fbd580662c0 2024-04-25T01:36:06.753 INFO:tasks.workunit.client.1.smithi138.stdout:9/457: dread d4/d1a/d55/f5f [0,4194304] 0 2024-04-25T01:36:06.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.778+0000 7fbd577fe700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fbd4006b8b0 0x7fbd4006dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.779+0000 7fbd577fe700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fbd4006b8b0 0x7fbd4006dd60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009fd0 tx=0x7fbd4c00b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:06.755 INFO:tasks.workunit.client.1.smithi138.stdout:1/455: write d3/d27/d3e/d51/f80 [251928,79056] 0 2024-04-25T01:36:06.755 INFO:tasks.workunit.client.0.smithi033.stdout:1/938: mkdir d2/d5/d1b/d46/d5f/d63/d9d/d12c 0 2024-04-25T01:36:06.755 INFO:tasks.workunit.client.1.smithi138.stdout:1/456: stat d3/d4/d47/c4f 0 2024-04-25T01:36:06.756 INFO:tasks.workunit.client.0.smithi033.stdout:9/997: write d0/d2/dd/fe [307055,14857] 0 2024-04-25T01:36:06.756 INFO:tasks.workunit.client.1.smithi138.stdout:3/626: unlink db/d14/d40/la9 0 2024-04-25T01:36:06.759 INFO:tasks.workunit.client.0.smithi033.stdout:9/998: creat d0/d2/d43/d73/dab/dad/dee/f15c x:0 0 0 2024-04-25T01:36:06.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:06.784+0000 7fbd54ff9700 1 -- 172.21.15.33:0/2537095869 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fbd480594c0 con 0x7fbd580662c0 2024-04-25T01:36:06.760 INFO:tasks.workunit.client.0.smithi033.stdout:9/999: write d0/d11/f18 [1653938,75724] 0 2024-04-25T01:36:06.762 INFO:tasks.workunit.client.0.smithi033.stdout:1/939: getdents d2/d2e/dcd 0 2024-04-25T01:36:06.763 INFO:tasks.workunit.client.0.smithi033.stdout:1/940: readlink d2/d5/d1b/d46/d5f/lab 0 2024-04-25T01:36:06.763 INFO:tasks.workunit.client.1.smithi138.stdout:2/550: rename d0/d32 to d0/d85/da5 0 2024-04-25T01:36:06.764 INFO:tasks.workunit.client.1.smithi138.stdout:0/377: dread d6/f1f [0,4194304] 0 2024-04-25T01:36:06.765 INFO:tasks.workunit.client.1.smithi138.stdout:2/551: chown d0/d85/da5/d35/f69 3 1 2024-04-25T01:36:06.767 INFO:tasks.workunit.client.0.smithi033.stdout:1/941: mknod d2/d5/d1b/d46/d5f/d63/d9d/d12c/c12d 0 2024-04-25T01:36:06.768 INFO:tasks.workunit.client.1.smithi138.stdout:1/457: write d3/db/f67 [1420199,18493] 0 2024-04-25T01:36:06.770 INFO:tasks.workunit.client.1.smithi138.stdout:3/627: rmdir db/df/d17/d23/d88 39 2024-04-25T01:36:06.773 INFO:tasks.workunit.client.1.smithi138.stdout:9/458: dwrite d4/f17 [0,4194304] 0 2024-04-25T01:36:06.774 INFO:tasks.workunit.client.1.smithi138.stdout:6/554: rename dc/f4e to dc/d2e/d37/d56/fac 0 2024-04-25T01:36:06.775 INFO:tasks.workunit.client.1.smithi138.stdout:6/555: chown dc/c39 0 1 2024-04-25T01:36:06.775 INFO:tasks.workunit.client.1.smithi138.stdout:0/378: symlink d6/d39/d58/l7a 0 2024-04-25T01:36:06.777 INFO:tasks.workunit.client.1.smithi138.stdout:2/552: unlink d0/d5a/d15/d51/f6f 0 2024-04-25T01:36:06.779 INFO:tasks.workunit.client.1.smithi138.stdout:0/379: dread d6/d19/f47 [4194304,4194304] 0 2024-04-25T01:36:06.780 INFO:tasks.workunit.client.1.smithi138.stdout:2/553: chown d0/d5a/d15/d44/d4c/d56/c7d 527 1 2024-04-25T01:36:06.780 INFO:tasks.workunit.client.1.smithi138.stdout:1/458: creat d3/d45/d72/d57/f91 x:0 0 0 2024-04-25T01:36:06.786 INFO:tasks.workunit.client.1.smithi138.stdout:0/380: symlink d6/d1d/l7b 0 2024-04-25T01:36:06.787 INFO:tasks.workunit.client.1.smithi138.stdout:0/381: readlink d6/d1d/l36 0 2024-04-25T01:36:06.787 INFO:tasks.workunit.client.1.smithi138.stdout:2/554: symlink d0/d85/da5/la6 0 2024-04-25T01:36:06.788 INFO:tasks.workunit.client.1.smithi138.stdout:9/459: dread d4/d1a/d33/f37 [0,4194304] 0 2024-04-25T01:36:06.791 INFO:tasks.workunit.client.1.smithi138.stdout:0/382: chown d6/d2f/d31/c79 956919 1 2024-04-25T01:36:06.791 INFO:tasks.workunit.client.1.smithi138.stdout:6/556: dread dc/d2e/f49 [4194304,4194304] 0 2024-04-25T01:36:06.794 INFO:tasks.workunit.client.1.smithi138.stdout:3/628: write db/f15 [2979331,70129] 0 2024-04-25T01:36:06.796 INFO:tasks.workunit.client.1.smithi138.stdout:6/557: creat dc/d10/d67/d8e/fad x:0 0 0 2024-04-25T01:36:06.797 INFO:tasks.workunit.client.1.smithi138.stdout:9/460: mkdir d4/d1a/d55/d7f/d80/d94 0 2024-04-25T01:36:06.799 INFO:tasks.workunit.client.1.smithi138.stdout:3/629: mkdir db/df/d19/d1a/dc4 0 2024-04-25T01:36:06.801 INFO:tasks.workunit.client.1.smithi138.stdout:2/555: dwrite d0/d5a/d15/d44/d4c/d56/d6d/f8c [0,4194304] 0 2024-04-25T01:36:06.802 INFO:tasks.workunit.client.1.smithi138.stdout:6/558: unlink dc/d10/l47 0 2024-04-25T01:36:06.803 INFO:tasks.workunit.client.1.smithi138.stdout:9/461: chown d4/l4b 10 1 2024-04-25T01:36:06.804 INFO:tasks.workunit.client.1.smithi138.stdout:2/556: fdatasync d0/d85/da5/f81 0 2024-04-25T01:36:06.806 INFO:tasks.workunit.client.1.smithi138.stdout:6/559: fdatasync dc/d10/d28/f29 0 2024-04-25T01:36:06.808 INFO:tasks.workunit.client.1.smithi138.stdout:1/459: dwrite d3/d27/d16/f18 [0,4194304] 0 2024-04-25T01:36:06.808 INFO:tasks.workunit.client.1.smithi138.stdout:9/462: truncate d4/d49/d72/f88 975372 0 2024-04-25T01:36:06.810 INFO:tasks.workunit.client.1.smithi138.stdout:2/557: write d0/f9a [554615,23679] 0 2024-04-25T01:36:06.812 INFO:tasks.workunit.client.1.smithi138.stdout:6/560: read dc/d10/f33 [838524,90367] 0 2024-04-25T01:36:06.817 INFO:tasks.workunit.client.1.smithi138.stdout:0/383: dread d6/d19/f47 [0,4194304] 0 2024-04-25T01:36:06.817 INFO:tasks.workunit.client.1.smithi138.stdout:2/558: creat d0/d21/d45/d8a/fa7 x:0 0 0 2024-04-25T01:36:06.825 INFO:tasks.workunit.client.1.smithi138.stdout:3/630: dread db/df/d17/d23/d34/f6d [0,4194304] 0 2024-04-25T01:36:06.826 INFO:tasks.workunit.client.1.smithi138.stdout:6/561: creat dc/d2e/d55/d7e/fae x:0 0 0 2024-04-25T01:36:06.827 INFO:tasks.workunit.client.1.smithi138.stdout:3/631: dread - db/df/d17/d5d/f76 zero size 2024-04-25T01:36:06.828 INFO:tasks.workunit.client.1.smithi138.stdout:2/559: dwrite d0/f10 [0,4194304] 0 2024-04-25T01:36:06.829 INFO:tasks.workunit.client.1.smithi138.stdout:3/632: readlink db/d14/d7a/l96 0 2024-04-25T01:36:06.831 INFO:tasks.workunit.client.1.smithi138.stdout:2/560: unlink d0/d5a/c95 0 2024-04-25T01:36:06.833 INFO:tasks.workunit.client.1.smithi138.stdout:2/561: write d0/d5a/d15/d44/d4c/d56/da2/f26 [4120237,66700] 0 2024-04-25T01:36:06.834 INFO:tasks.workunit.client.1.smithi138.stdout:9/463: write d4/d8/d90/f78 [3849807,19160] 0 2024-04-25T01:36:06.835 INFO:tasks.workunit.client.1.smithi138.stdout:2/562: chown d0/d21/f22 739 1 2024-04-25T01:36:06.837 INFO:tasks.workunit.client.1.smithi138.stdout:0/384: write d6/d19/f77 [4602408,23101] 0 2024-04-25T01:36:06.840 INFO:tasks.workunit.client.1.smithi138.stdout:1/460: truncate d3/d4/d53/f8c 2888704 0 2024-04-25T01:36:06.840 INFO:tasks.workunit.client.1.smithi138.stdout:1/461: chown d3/db 3170503 1 2024-04-25T01:36:06.841 INFO:tasks.workunit.client.1.smithi138.stdout:1/462: write d3/f2c [1972999,56974] 0 2024-04-25T01:36:06.843 INFO:tasks.workunit.client.1.smithi138.stdout:2/563: rmdir d0/d18 39 2024-04-25T01:36:06.845 INFO:tasks.workunit.client.1.smithi138.stdout:0/385: creat d6/df/d41/d49/f7c x:0 0 0 2024-04-25T01:36:06.848 INFO:tasks.workunit.client.1.smithi138.stdout:6/562: truncate dc/d10/d28/f34 4004228 0 2024-04-25T01:36:06.849 INFO:tasks.workunit.client.1.smithi138.stdout:1/463: creat d3/d6d/f92 x:0 0 0 2024-04-25T01:36:06.849 INFO:tasks.workunit.client.1.smithi138.stdout:2/564: rmdir d0/d5a/d15/d44/d4c/d56/da2 39 2024-04-25T01:36:06.850 INFO:tasks.workunit.client.1.smithi138.stdout:2/565: dread - d0/d21/d45/d8a/fa7 zero size 2024-04-25T01:36:06.851 INFO:tasks.workunit.client.1.smithi138.stdout:3/633: rename db/d14/d40/d41/dad to db/dc5 0 2024-04-25T01:36:06.852 INFO:tasks.workunit.client.1.smithi138.stdout:6/563: mkdir dc/d10/d67/d8e/daf 0 2024-04-25T01:36:06.853 INFO:tasks.workunit.client.1.smithi138.stdout:6/564: stat dc/d10/d28/d70/fab 0 2024-04-25T01:36:06.853 INFO:tasks.workunit.client.1.smithi138.stdout:9/464: rename d4/d1a/c46 to d4/d57/c95 0 2024-04-25T01:36:06.855 INFO:tasks.workunit.client.1.smithi138.stdout:6/565: truncate dc/d2e/d55/d81/f9f 966836 0 2024-04-25T01:36:06.860 INFO:tasks.workunit.client.1.smithi138.stdout:9/465: mknod d4/d1a/d68/c96 0 2024-04-25T01:36:06.862 INFO:tasks.workunit.client.1.smithi138.stdout:3/634: symlink db/df/d17/d20/lc6 0 2024-04-25T01:36:06.864 INFO:tasks.workunit.client.1.smithi138.stdout:6/566: dwrite dc/d2e/d55/d7e/fae [0,4194304] 0 2024-04-25T01:36:06.866 INFO:tasks.workunit.client.1.smithi138.stdout:2/566: symlink d0/d5a/d15/d44/d4c/d56/da2/la8 0 2024-04-25T01:36:06.868 INFO:tasks.workunit.client.1.smithi138.stdout:2/567: dread d0/d21/f5b [0,4194304] 0 2024-04-25T01:36:06.869 INFO:tasks.workunit.client.1.smithi138.stdout:9/466: dwrite d4/d1a/d33/f4d [0,4194304] 0 2024-04-25T01:36:06.870 INFO:tasks.workunit.client.1.smithi138.stdout:0/386: rename d6/d19/f33 to d6/d2f/f7d 0 2024-04-25T01:36:06.871 INFO:tasks.workunit.client.1.smithi138.stdout:2/568: truncate d0/f9a 901775 0 2024-04-25T01:36:06.872 INFO:tasks.workunit.client.1.smithi138.stdout:2/569: stat d0/d85/da5/c3d 0 2024-04-25T01:36:06.874 INFO:tasks.workunit.client.1.smithi138.stdout:0/387: rmdir d6/df 39 2024-04-25T01:36:06.877 INFO:tasks.workunit.client.1.smithi138.stdout:1/464: rename d3/d27/d3e/d90/l6a to d3/d27/d3e/d51/l93 0 2024-04-25T01:36:06.879 INFO:tasks.workunit.client.1.smithi138.stdout:9/467: symlink d4/d8/d90/d52/l97 0 2024-04-25T01:36:06.880 INFO:tasks.workunit.client.1.smithi138.stdout:9/468: stat d4/d8/d90/d52/f7d 0 2024-04-25T01:36:06.880 INFO:tasks.workunit.client.1.smithi138.stdout:9/469: stat d4/d57/f67 0 2024-04-25T01:36:06.884 INFO:tasks.workunit.client.1.smithi138.stdout:2/570: creat d0/d21/d45/d8a/d8e/fa9 x:0 0 0 2024-04-25T01:36:06.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/470: dread d4/d8/f59 [0,4194304] 0 2024-04-25T01:36:06.886 INFO:tasks.workunit.client.1.smithi138.stdout:0/388: dread d6/d2f/d75/f76 [0,4194304] 0 2024-04-25T01:36:06.888 INFO:tasks.workunit.client.1.smithi138.stdout:1/465: link d3/db/l7f d3/d27/d2d/l94 0 2024-04-25T01:36:06.889 INFO:tasks.workunit.client.1.smithi138.stdout:1/466: write d3/d27/d3e/d51/f80 [491182,120678] 0 2024-04-25T01:36:06.891 INFO:tasks.workunit.client.1.smithi138.stdout:1/467: dread - d3/d45/d72/d57/f91 zero size 2024-04-25T01:36:06.891 INFO:tasks.workunit.client.1.smithi138.stdout:3/635: rename db/df/d19/d1a/f49 to db/fc7 0 2024-04-25T01:36:06.892 INFO:tasks.workunit.client.1.smithi138.stdout:0/389: creat d6/d2f/f7e x:0 0 0 2024-04-25T01:36:06.894 INFO:tasks.workunit.client.1.smithi138.stdout:6/567: rename dc/f72 to dc/d7b/fb0 0 2024-04-25T01:36:06.895 INFO:tasks.workunit.client.1.smithi138.stdout:1/468: chown d3/d4/d53/f8c 1162 1 2024-04-25T01:36:06.898 INFO:tasks.workunit.client.1.smithi138.stdout:2/571: rename d0/d85/da5/d35/l7f to d0/d5a/laa 0 2024-04-25T01:36:06.899 INFO:tasks.workunit.client.1.smithi138.stdout:6/568: dread dc/d2e/d55/d7e/fae [0,4194304] 0 2024-04-25T01:36:06.900 INFO:tasks.workunit.client.1.smithi138.stdout:3/636: link db/df/d17/d5d/fa8 db/d14/fc8 0 2024-04-25T01:36:06.903 INFO:tasks.workunit.client.1.smithi138.stdout:1/469: creat d3/d45/d72/f95 x:0 0 0 2024-04-25T01:36:06.904 INFO:tasks.workunit.client.1.smithi138.stdout:6/569: creat dc/d10/d28/fb1 x:0 0 0 2024-04-25T01:36:06.905 INFO:tasks.workunit.client.1.smithi138.stdout:2/572: dwrite d0/d21/f59 [0,4194304] 0 2024-04-25T01:36:06.906 INFO:tasks.workunit.client.1.smithi138.stdout:1/470: truncate d3/d27/d24/f64 161267 0 2024-04-25T01:36:06.910 INFO:tasks.workunit.client.1.smithi138.stdout:0/390: creat d6/df/d66/f7f x:0 0 0 2024-04-25T01:36:06.911 INFO:tasks.workunit.client.1.smithi138.stdout:6/570: mknod dc/d2e/d55/d7e/cb2 0 2024-04-25T01:36:06.913 INFO:tasks.workunit.client.1.smithi138.stdout:3/637: mknod db/d14/d40/cc9 0 2024-04-25T01:36:06.915 INFO:tasks.workunit.client.1.smithi138.stdout:1/471: dwrite d3/d45/d72/f42 [0,4194304] 0 2024-04-25T01:36:06.917 INFO:tasks.workunit.client.1.smithi138.stdout:0/391: rmdir d6/df/d41/d49 39 2024-04-25T01:36:06.919 INFO:tasks.workunit.client.1.smithi138.stdout:6/571: mknod dc/d10/d28/cb3 0 2024-04-25T01:36:06.922 INFO:tasks.workunit.client.1.smithi138.stdout:3/638: mknod db/d9b/db4/cca 0 2024-04-25T01:36:06.923 INFO:tasks.workunit.client.1.smithi138.stdout:1/472: rename d3/d27/l4e to d3/d45/d72/d3d/l96 0 2024-04-25T01:36:06.924 INFO:tasks.workunit.client.1.smithi138.stdout:0/392: mknod d6/d2f/d31/c80 0 2024-04-25T01:36:06.925 INFO:tasks.workunit.client.1.smithi138.stdout:1/473: chown d3/d6d/f92 22 1 2024-04-25T01:36:06.925 INFO:tasks.workunit.client.1.smithi138.stdout:1/474: chown d3/d45/d72/d57/d8e 7 1 2024-04-25T01:36:06.928 INFO:tasks.workunit.client.1.smithi138.stdout:6/572: dread dc/d2e/d37/d56/f9a [0,4194304] 0 2024-04-25T01:36:06.929 INFO:tasks.workunit.client.1.smithi138.stdout:3/639: rename db/df/d19/l4e to db/df/d19/d1a/d9e/lcb 0 2024-04-25T01:36:06.931 INFO:tasks.workunit.client.1.smithi138.stdout:2/573: link d0/d21/c91 d0/cab 0 2024-04-25T01:36:06.933 INFO:tasks.workunit.client.1.smithi138.stdout:0/393: truncate d6/df/f14 452804 0 2024-04-25T01:36:06.940 INFO:tasks.workunit.client.1.smithi138.stdout:1/475: rename d3/d4/fd to d3/db/f97 0 2024-04-25T01:36:06.940 INFO:tasks.workunit.client.0.smithi033.stdout:1/942: sync 2024-04-25T01:36:06.943 INFO:tasks.workunit.client.1.smithi138.stdout:1/476: write d3/d6d/f89 [1892932,44309] 0 2024-04-25T01:36:06.943 INFO:tasks.workunit.client.0.smithi033.stdout:1/943: mknod d2/d5/d1b/dd2/d102/c12e 0 2024-04-25T01:36:06.944 INFO:tasks.workunit.client.1.smithi138.stdout:6/573: dread dc/d2e/f6c [0,4194304] 0 2024-04-25T01:36:06.945 INFO:tasks.workunit.client.1.smithi138.stdout:3/640: dwrite db/df/d17/f1c [4194304,4194304] 0 2024-04-25T01:36:06.948 INFO:tasks.workunit.client.0.smithi033.stdout:1/944: rename d2/d5/d75/d59/f122 to d2/d7f/d11d/f12f 0 2024-04-25T01:36:06.948 INFO:tasks.workunit.client.1.smithi138.stdout:0/394: rename d6/d2f/d31/c79 to d6/c81 0 2024-04-25T01:36:06.950 INFO:tasks.workunit.client.1.smithi138.stdout:1/477: creat d3/d27/d3e/d51/d62/f98 x:0 0 0 2024-04-25T01:36:06.951 INFO:tasks.workunit.client.1.smithi138.stdout:1/478: write d3/f13 [828996,86808] 0 2024-04-25T01:36:06.951 INFO:tasks.workunit.client.0.smithi033.stdout:1/945: creat d2/d5/d1b/d46/d5f/d63/d9d/d12c/f130 x:0 0 0 2024-04-25T01:36:06.952 INFO:tasks.workunit.client.1.smithi138.stdout:6/574: creat dc/d2e/d55/d81/fb4 x:0 0 0 2024-04-25T01:36:06.952 INFO:tasks.workunit.client.1.smithi138.stdout:2/574: dread d0/fe [0,4194304] 0 2024-04-25T01:36:06.953 INFO:tasks.workunit.client.1.smithi138.stdout:0/395: rmdir d6/d39/d43 39 2024-04-25T01:36:06.954 INFO:tasks.workunit.client.1.smithi138.stdout:2/575: fsync d0/d85/da5/d35/f70 0 2024-04-25T01:36:06.956 INFO:tasks.workunit.client.1.smithi138.stdout:0/396: read d6/d1d/f3c [52627,58175] 0 2024-04-25T01:36:06.956 INFO:tasks.workunit.client.1.smithi138.stdout:1/479: rename d3/d27/d3e/d51/c58 to d3/d27/d2d/c99 0 2024-04-25T01:36:06.957 INFO:tasks.workunit.client.0.smithi033.stdout:1/946: symlink d2/d5/d75/d59/d10e/d110/l131 0 2024-04-25T01:36:06.963 INFO:tasks.workunit.client.1.smithi138.stdout:1/480: mknod d3/d4/c9a 0 2024-04-25T01:36:06.965 INFO:tasks.workunit.client.1.smithi138.stdout:2/576: symlink d0/d5a/d15/d44/d4c/d56/d6d/da3/lac 0 2024-04-25T01:36:06.967 INFO:tasks.workunit.client.1.smithi138.stdout:2/577: write d0/d85/da5/d35/f70 [1272161,49825] 0 2024-04-25T01:36:06.968 INFO:tasks.workunit.client.1.smithi138.stdout:1/481: dread d3/d45/d72/d57/f8a [0,4194304] 0 2024-04-25T01:36:06.969 INFO:tasks.workunit.client.1.smithi138.stdout:0/397: getdents d6/d1d 0 2024-04-25T01:36:06.970 INFO:tasks.workunit.client.1.smithi138.stdout:2/578: chown d0/d5a/d15/d44/d4c/d56/da2 2495593 1 2024-04-25T01:36:06.972 INFO:tasks.workunit.client.1.smithi138.stdout:2/579: write d0/f10 [1052465,23427] 0 2024-04-25T01:36:06.976 INFO:tasks.workunit.client.1.smithi138.stdout:2/580: symlink d0/d85/lad 0 2024-04-25T01:36:06.976 INFO:tasks.workunit.client.0.smithi033.stdout:1/947: dread d2/d5/d6c/f87 [0,4194304] 0 2024-04-25T01:36:06.977 INFO:tasks.workunit.client.1.smithi138.stdout:1/482: dwrite d3/db/f5f [0,4194304] 0 2024-04-25T01:36:06.979 INFO:tasks.workunit.client.1.smithi138.stdout:0/398: dwrite d6/df/d3a/f71 [0,4194304] 0 2024-04-25T01:36:06.980 INFO:tasks.workunit.client.1.smithi138.stdout:2/581: mknod d0/d21/d45/d8a/d8e/cae 0 2024-04-25T01:36:06.981 INFO:tasks.workunit.client.0.smithi033.stdout:1/948: dread d2/d5/d1b/d46/d5f/d63/d64/ff7 [0,4194304] 0 2024-04-25T01:36:06.981 INFO:tasks.workunit.client.1.smithi138.stdout:0/399: mkdir d6/d1d/d82 0 2024-04-25T01:36:06.984 INFO:tasks.workunit.client.1.smithi138.stdout:1/483: rename d3/d27/d3e/d51/d62 to d3/d4/d47/d9b 0 2024-04-25T01:36:06.986 INFO:tasks.workunit.client.0.smithi033.stdout:1/949: truncate d2/d5/d1b/d46/d5f/d63/d9d/ff1 987774 0 2024-04-25T01:36:06.993 INFO:tasks.workunit.client.0.smithi033.stdout:1/950: dwrite d2/d5/d75/d59/d10e/f119 [0,4194304] 0 2024-04-25T01:36:07.000 INFO:tasks.workunit.client.0.smithi033.stdout:1/951: dread d2/d2e/dcd/fc8 [4194304,4194304] 0 2024-04-25T01:36:07.003 INFO:tasks.workunit.client.0.smithi033.stdout:1/952: write d2/d2e/d34/d60/fa7 [3169714,84383] 0 2024-04-25T01:36:07.004 INFO:tasks.workunit.client.0.smithi033.stdout:1/953: truncate d2/d11/d28/f113 12988 0 2024-04-25T01:36:07.007 INFO:tasks.workunit.client.0.smithi033.stdout:1/954: creat d2/d11/d28/f132 x:0 0 0 2024-04-25T01:36:07.009 INFO:tasks.workunit.client.0.smithi033.stdout:1/955: mkdir d2/d5/d1b/d133 0 2024-04-25T01:36:07.011 INFO:tasks.workunit.client.0.smithi033.stdout:1/956: mknod d2/d2e/d34/d60/d106/c134 0 2024-04-25T01:36:07.017 INFO:tasks.workunit.client.0.smithi033.stdout:1/957: dwrite d2/f62 [0,4194304] 0 2024-04-25T01:36:07.020 INFO:tasks.workunit.client.0.smithi033.stdout:1/958: symlink d2/d5/d75/d59/d127/l135 0 2024-04-25T01:36:07.022 INFO:tasks.workunit.client.0.smithi033.stdout:1/959: creat d2/d5/d75/d59/d10e/d110/f136 x:0 0 0 2024-04-25T01:36:07.024 INFO:tasks.workunit.client.0.smithi033.stdout:1/960: creat d2/d5/d1b/d133/f137 x:0 0 0 2024-04-25T01:36:07.026 INFO:tasks.workunit.client.0.smithi033.stdout:1/961: mknod d2/d5/dee/dfb/c138 0 2024-04-25T01:36:07.039 INFO:tasks.workunit.client.0.smithi033.stdout:1/962: dread d2/d2e/f67 [0,4194304] 0 2024-04-25T01:36:07.041 INFO:tasks.workunit.client.0.smithi033.stdout:1/963: creat d2/d5/d1b/f139 x:0 0 0 2024-04-25T01:36:07.043 INFO:tasks.workunit.client.0.smithi033.stdout:1/964: mkdir d2/d5/d6c/d94/d13a 0 2024-04-25T01:36:07.048 INFO:tasks.workunit.client.1.smithi138.stdout:8/597: sync 2024-04-25T01:36:07.055 INFO:tasks.workunit.client.1.smithi138.stdout:0/400: creat d6/d19/d57/f83 x:0 0 0 2024-04-25T01:36:07.055 INFO:tasks.workunit.client.1.smithi138.stdout:1/484: dread d3/d45/d72/f42 [0,4194304] 0 2024-04-25T01:36:07.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.081+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd58115e60 con 0x7fbd4006b8b0 2024-04-25T01:36:07.064 INFO:tasks.workunit.client.1.smithi138.stdout:1/485: dwrite d3/d6d/f92 [0,4194304] 0 2024-04-25T01:36:07.070 INFO:tasks.workunit.client.1.smithi138.stdout:8/598: mkdir d3/d44/d48/dbb 0 2024-04-25T01:36:07.074 INFO:tasks.workunit.client.1.smithi138.stdout:2/582: getdents d0/d5a/d15 0 2024-04-25T01:36:07.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.099+0000 7fbd54ff9700 1 -- 172.21.15.33:0/2537095869 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7fbd58115e60 con 0x7fbd4006b8b0 2024-04-25T01:36:07.074 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:36:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (7m) 0s ago 8m 24.4M - 0.25.0 c8568f914cd2 50a86f263578 2024-04-25T01:36:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (8m) 0s ago 8m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:36:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (7m) 21s ago 7m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:36:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (7m) 0s ago 8m 87.2M - 9.4.7 954c08fa6188 4533d743079b 2024-04-25T01:36:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (3m) 0s ago 3m 61.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:36:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (3m) 0s ago 3m 99.5M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:36:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (3m) 21s ago 3m 110M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:36:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (2m) 21s ago 2m 67.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:36:07.075 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:9283 running (9m) 0s ago 9m 390M - 17.2.7-902-gc61472c8 0de407c1b3af 22f7bb21dc39 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283 running (64s) 21s ago 7m 488M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8ce48ef47e1e 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (9m) 0s ago 10m 48.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (7m) 21s ago 7m 41.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (8m) 0s ago 8m 16.6M - 1.5.0 0da6a335fe13 c2f75e5aaf0f 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 21s ago 7m 16.3M - 1.5.0 0da6a335fe13 ce8fbd90510d 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (6m) 0s ago 6m 126M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (6m) 0s ago 6m 146M 4289M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (5m) 0s ago 5m 124M 4289M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (5m) 21s ago 5m 156M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (5m) 21s ago 4m 135M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (4m) 21s ago 4m 128M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:36:07.076 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (6m) 0s ago 7m 58.7M - 2.43.0 a07b618ecd1d 9109b23faf81 2024-04-25T01:36:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.101+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fbd4006b8b0 msgr2=0x7fbd4006dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.101+0000 7fbd5e7f0700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fbd4006b8b0 0x7fbd4006dd60 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c009fd0 tx=0x7fbd4c00b040 comp rx=0 tx=0).stop 2024-04-25T01:36:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.101+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 msgr2=0x7fbd58114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.101+0000 7fbd5e7f0700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 0x7fbd58114010 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbd48000c00 tx=0x7fbd48009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.101+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 shutdown_connections 2024-04-25T01:36:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.101+0000 7fbd5e7f0700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fbd4006b8b0 0x7fbd4006dd60 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.101+0000 7fbd5e7f0700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd580662c0 0x7fbd58114010 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.101+0000 7fbd5e7f0700 1 --2- 172.21.15.33:0/2537095869 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd580fdcd0 0x7fbd58114550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.101+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 >> 172.21.15.33:0/2537095869 conn(0x7fbd580f7430 msgr2=0x7fbd580f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.102+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 shutdown_connections 2024-04-25T01:36:07.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.102+0000 7fbd5e7f0700 1 -- 172.21.15.33:0/2537095869 wait complete. 2024-04-25T01:36:07.078 INFO:tasks.workunit.client.1.smithi138.stdout:8/599: dwrite d3/d16/f9b [0,4194304] 0 2024-04-25T01:36:07.081 INFO:tasks.workunit.client.1.smithi138.stdout:8/600: symlink d3/d16/d1f/lbc 0 2024-04-25T01:36:07.083 INFO:tasks.workunit.client.1.smithi138.stdout:2/583: dwrite d0/d21/f59 [0,4194304] 0 2024-04-25T01:36:07.087 INFO:tasks.workunit.client.1.smithi138.stdout:8/601: unlink d3/d16/d1f/c23 0 2024-04-25T01:36:07.090 INFO:tasks.workunit.client.1.smithi138.stdout:0/401: dread d6/f1e [0,4194304] 0 2024-04-25T01:36:07.093 INFO:tasks.workunit.client.1.smithi138.stdout:2/584: dread d0/d5a/f87 [0,4194304] 0 2024-04-25T01:36:07.095 INFO:tasks.workunit.client.1.smithi138.stdout:2/585: creat d0/d21/d45/faf x:0 0 0 2024-04-25T01:36:07.101 INFO:tasks.workunit.client.1.smithi138.stdout:2/586: creat d0/d5a/d15/d44/d4c/d56/fb0 x:0 0 0 2024-04-25T01:36:07.102 INFO:tasks.workunit.client.1.smithi138.stdout:2/587: dread - d0/d5a/f9d zero size 2024-04-25T01:36:07.103 INFO:tasks.workunit.client.1.smithi138.stdout:2/588: fdatasync d0/d85/da5/d35/f70 0 2024-04-25T01:36:07.103 INFO:tasks.workunit.client.1.smithi138.stdout:8/602: dwrite d3/d16/d88/da/f24 [0,4194304] 0 2024-04-25T01:36:07.105 INFO:tasks.workunit.client.1.smithi138.stdout:2/589: write d0/f9a [1687063,67607] 0 2024-04-25T01:36:07.105 INFO:tasks.workunit.client.1.smithi138.stdout:2/590: write d0/fa1 [772525,25337] 0 2024-04-25T01:36:07.108 INFO:tasks.workunit.client.1.smithi138.stdout:8/603: truncate d3/d16/d88/da/db/d15/d59/fb6 369033 0 2024-04-25T01:36:07.108 INFO:tasks.workunit.client.1.smithi138.stdout:8/604: chown d3/d16 14 1 2024-04-25T01:36:07.112 INFO:tasks.workunit.client.1.smithi138.stdout:8/605: creat d3/d16/d1f/d31/d82/fbd x:0 0 0 2024-04-25T01:36:07.113 INFO:tasks.workunit.client.1.smithi138.stdout:2/591: write d0/d85/da5/f4b [1115957,75864] 0 2024-04-25T01:36:07.115 INFO:tasks.workunit.client.1.smithi138.stdout:8/606: unlink d3/d16/d1f/d2a/c64 0 2024-04-25T01:36:07.120 INFO:tasks.workunit.client.1.smithi138.stdout:8/607: truncate d3/d16/d1f/d50/f78 755109 0 2024-04-25T01:36:07.120 INFO:tasks.workunit.client.1.smithi138.stdout:2/592: getdents d0/d5a/d15/d44/d4c 0 2024-04-25T01:36:07.125 INFO:tasks.workunit.client.1.smithi138.stdout:8/608: rename d3/c6 to d3/d16/d1f/d2a/d4f/dac/cbe 0 2024-04-25T01:36:07.126 INFO:tasks.workunit.client.1.smithi138.stdout:8/609: dread - d3/d16/d88/da/db/d15/d56/d94/f9f zero size 2024-04-25T01:36:07.127 INFO:tasks.workunit.client.1.smithi138.stdout:8/610: write d3/d16/d1f/f61 [2967144,78577] 0 2024-04-25T01:36:07.128 INFO:tasks.workunit.client.1.smithi138.stdout:2/593: rename d0/d5a/d15/f67 to d0/d5a/d15/d44/d4c/d56/d6d/d6e/fb1 0 2024-04-25T01:36:07.130 INFO:tasks.workunit.client.0.smithi033.stdout:1/965: sync 2024-04-25T01:36:07.131 INFO:tasks.workunit.client.1.smithi138.stdout:8/611: chown d3/d16/d88/da/db/d15/c2c 117 1 2024-04-25T01:36:07.133 INFO:tasks.workunit.client.0.smithi033.stdout:1/966: creat d2/d11/d28/dd4/f13b x:0 0 0 2024-04-25T01:36:07.137 INFO:tasks.workunit.client.1.smithi138.stdout:8/612: dwrite d3/d16/d88/da/fad [0,4194304] 0 2024-04-25T01:36:07.138 INFO:tasks.workunit.client.0.smithi033.stdout:1/967: dwrite d2/d5/d1b/d46/d5f/d63/d9d/ff1 [0,4194304] 0 2024-04-25T01:36:07.141 INFO:tasks.workunit.client.0.smithi033.stdout:1/968: creat d2/d5/d75/d59/d10e/f13c x:0 0 0 2024-04-25T01:36:07.142 INFO:tasks.workunit.client.1.smithi138.stdout:8/613: symlink d3/d16/d1f/d87/lbf 0 2024-04-25T01:36:07.148 INFO:tasks.workunit.client.1.smithi138.stdout:2/594: link d0/d5a/d15/f3b d0/d85/da5/fb2 0 2024-04-25T01:36:07.149 INFO:tasks.workunit.client.0.smithi033.stdout:1/969: dread d2/de6/ffd [0,4194304] 0 2024-04-25T01:36:07.151 INFO:tasks.workunit.client.0.smithi033.stdout:1/970: chown d2/d11/d28/fd9 3 1 2024-04-25T01:36:07.152 INFO:tasks.workunit.client.0.smithi033.stdout:1/971: chown d2/d2e/d49/l103 854 1 2024-04-25T01:36:07.154 INFO:tasks.workunit.client.1.smithi138.stdout:8/614: dread d3/d16/d88/da/d70/fa8 [0,4194304] 0 2024-04-25T01:36:07.157 INFO:tasks.workunit.client.1.smithi138.stdout:8/615: symlink d3/d16/d88/da/d70/lc0 0 2024-04-25T01:36:07.157 INFO:tasks.workunit.client.1.smithi138.stdout:2/595: dwrite d0/d5a/d15/d51/f80 [0,4194304] 0 2024-04-25T01:36:07.161 INFO:tasks.workunit.client.1.smithi138.stdout:2/596: creat d0/d5a/fb3 x:0 0 0 2024-04-25T01:36:07.166 INFO:tasks.workunit.client.0.smithi033.stdout:1/972: dwrite d2/d5/d1b/d46/d5f/d63/d64/f6a [0,4194304] 0 2024-04-25T01:36:07.168 INFO:tasks.workunit.client.1.smithi138.stdout:8/616: dwrite d3/d16/d88/da/d70/fa8 [0,4194304] 0 2024-04-25T01:36:07.169 INFO:tasks.workunit.client.0.smithi033.stdout:1/973: mkdir d2/d11/d28/dd4/d13d 0 2024-04-25T01:36:07.170 INFO:tasks.workunit.client.1.smithi138.stdout:2/597: rename d0/d5a to d0/d85/da5/db4 0 2024-04-25T01:36:07.172 INFO:tasks.workunit.client.0.smithi033.stdout:1/974: write d2/d5/d1b/dd2/feb [309558,30437] 0 2024-04-25T01:36:07.173 INFO:tasks.workunit.client.1.smithi138.stdout:8/617: creat d3/d16/d88/fc1 x:0 0 0 2024-04-25T01:36:07.175 INFO:tasks.workunit.client.0.smithi033.stdout:1/975: creat d2/de6/f13e x:0 0 0 2024-04-25T01:36:07.175 INFO:tasks.workunit.client.1.smithi138.stdout:2/598: creat d0/d18/d8b/fb5 x:0 0 0 2024-04-25T01:36:07.176 INFO:tasks.workunit.client.0.smithi033.stdout:1/976: dread - d2/d5/d6c/dc6/f124 zero size 2024-04-25T01:36:07.177 INFO:tasks.workunit.client.1.smithi138.stdout:8/618: mkdir d3/dc2 0 2024-04-25T01:36:07.177 INFO:tasks.workunit.client.0.smithi033.stdout:1/977: readlink d2/d5/d75/d59/d10e/d110/l131 0 2024-04-25T01:36:07.178 INFO:tasks.workunit.client.1.smithi138.stdout:8/619: chown d3/d16/d88/da/db/d15/c4e 28250 1 2024-04-25T01:36:07.181 INFO:tasks.workunit.client.0.smithi033.stdout:1/978: symlink d2/d5/d1b/d46/d5f/d63/l13f 0 2024-04-25T01:36:07.183 INFO:tasks.workunit.client.0.smithi033.stdout:1/979: readlink d2/l15 0 2024-04-25T01:36:07.186 INFO:tasks.workunit.client.0.smithi033.stdout:1/980: mknod d2/c140 0 2024-04-25T01:36:07.186 INFO:tasks.workunit.client.1.smithi138.stdout:2/599: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/fb0 [0,4194304] 0 2024-04-25T01:36:07.188 INFO:tasks.workunit.client.1.smithi138.stdout:8/620: dwrite d3/d16/d1f/d31/d82/fbd [0,4194304] 0 2024-04-25T01:36:07.190 INFO:tasks.workunit.client.1.smithi138.stdout:2/600: rmdir d0/d21/d45/d8a 39 2024-04-25T01:36:07.191 INFO:tasks.workunit.client.1.smithi138.stdout:8/621: write d3/d16/d88/da/db/d15/d59/fb6 [743792,97930] 0 2024-04-25T01:36:07.191 INFO:tasks.workunit.client.0.smithi033.stdout:1/981: creat d2/f141 x:0 0 0 2024-04-25T01:36:07.193 INFO:tasks.workunit.client.1.smithi138.stdout:8/622: chown d3/d16/d88/da/db/d15/d5d/c79 272 1 2024-04-25T01:36:07.195 INFO:tasks.workunit.client.0.smithi033.stdout:1/982: creat d2/d2e/df5/f142 x:0 0 0 2024-04-25T01:36:07.195 INFO:tasks.workunit.client.1.smithi138.stdout:2/601: mknod d0/d85/da5/db4/d15/cb6 0 2024-04-25T01:36:07.196 INFO:tasks.workunit.client.1.smithi138.stdout:8/623: fsync d3/d16/d88/da/db/d15/f2f 0 2024-04-25T01:36:07.199 INFO:tasks.workunit.client.1.smithi138.stdout:2/602: rename d0/d21/d45/c82 to d0/d85/da5/db4/cb7 0 2024-04-25T01:36:07.203 INFO:tasks.workunit.client.1.smithi138.stdout:2/603: readlink d0/d85/da5/db4/d15/d44/d4c/d56/da2/la8 0 2024-04-25T01:36:07.205 INFO:tasks.workunit.client.1.smithi138.stdout:8/624: rename d3/f5 to d3/d16/d88/da/db/d15/d56/d8f/fc3 0 2024-04-25T01:36:07.208 INFO:tasks.workunit.client.1.smithi138.stdout:8/625: unlink d3/d16/d88/da/fad 0 2024-04-25T01:36:07.210 INFO:tasks.workunit.client.1.smithi138.stdout:2/604: creat d0/d85/da5/db4/d15/d44/d4c/d56/fb8 x:0 0 0 2024-04-25T01:36:07.210 INFO:tasks.workunit.client.1.smithi138.stdout:2/605: fsync d0/d21/d45/faf 0 2024-04-25T01:36:07.213 INFO:tasks.workunit.client.1.smithi138.stdout:8/626: getdents d3/d16/d1f 0 2024-04-25T01:36:07.214 INFO:tasks.workunit.client.1.smithi138.stdout:8/627: write d3/d16/d88/fc1 [723882,38538] 0 2024-04-25T01:36:07.216 INFO:tasks.workunit.client.1.smithi138.stdout:2/606: rename d0/d85/da5/db4/d15/d44/d4c/d56/d6d/f8c to d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fb9 0 2024-04-25T01:36:07.218 INFO:tasks.workunit.client.1.smithi138.stdout:4/402: sync 2024-04-25T01:36:07.218 INFO:tasks.workunit.client.1.smithi138.stdout:8/628: mknod d3/d16/d1f/d2a/d4f/dac/cc4 0 2024-04-25T01:36:07.220 INFO:tasks.workunit.client.1.smithi138.stdout:2/607: rename d0/d21/d45/c90 to d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/cba 0 2024-04-25T01:36:07.223 INFO:tasks.workunit.client.1.smithi138.stdout:4/403: rmdir d0 39 2024-04-25T01:36:07.224 INFO:tasks.workunit.client.0.smithi033.stdout:1/983: dread d2/d5/d1b/ffc [0,4194304] 0 2024-04-25T01:36:07.225 INFO:tasks.workunit.client.1.smithi138.stdout:8/629: read - d3/d16/f8e zero size 2024-04-25T01:36:07.225 INFO:tasks.workunit.client.1.smithi138.stdout:8/630: fdatasync d3/f6d 0 2024-04-25T01:36:07.227 INFO:tasks.workunit.client.0.smithi033.stdout:1/984: mkdir d2/d7f/db0/dd6/d143 0 2024-04-25T01:36:07.230 INFO:tasks.workunit.client.1.smithi138.stdout:8/631: stat d3/d16/d88/da/db/d15/f30 0 2024-04-25T01:36:07.232 INFO:tasks.workunit.client.0.smithi033.stdout:1/985: dread d2/d5/d1b/d46/d5f/d63/d9d/ff1 [0,4194304] 0 2024-04-25T01:36:07.233 INFO:tasks.workunit.client.1.smithi138.stdout:4/404: unlink d0/d1c/f55 0 2024-04-25T01:36:07.233 INFO:tasks.workunit.client.0.smithi033.stdout:1/986: chown d2/d11/d28/l90 87 1 2024-04-25T01:36:07.236 INFO:tasks.workunit.client.0.smithi033.stdout:1/987: mknod d2/d11/d28/dd4/d13d/c144 0 2024-04-25T01:36:07.236 INFO:tasks.workunit.client.1.smithi138.stdout:2/608: getdents d0/d85/da5/db4/d15/d44/d4c/d56/da2 0 2024-04-25T01:36:07.238 INFO:tasks.workunit.client.1.smithi138.stdout:4/405: creat d0/d16/d4a/f76 x:0 0 0 2024-04-25T01:36:07.238 INFO:tasks.workunit.client.1.smithi138.stdout:8/632: write d3/d16/d88/da/db/d15/f83 [2839645,76438] 0 2024-04-25T01:36:07.240 INFO:tasks.workunit.client.1.smithi138.stdout:2/609: creat d0/d85/da5/db4/d15/d44/d4c/fbb x:0 0 0 2024-04-25T01:36:07.240 INFO:tasks.workunit.client.0.smithi033.stdout:1/988: dread d2/d5/d75/d59/d83/fc1 [4194304,4194304] 0 2024-04-25T01:36:07.241 INFO:tasks.workunit.client.1.smithi138.stdout:4/406: creat d0/d16/d4a/f77 x:0 0 0 2024-04-25T01:36:07.242 INFO:tasks.workunit.client.0.smithi033.stdout:1/989: chown d2/d5/d1b/d46/d5f/d63/d9d/ff1 935105 1 2024-04-25T01:36:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.269+0000 7f2794440700 1 -- 172.21.15.33:0/1306205623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fcfa0 msgr2=0x7f278c0fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.269+0000 7f2794440700 1 --2- 172.21.15.33:0/1306205623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fcfa0 0x7f278c0fd410 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f277c004940 tx=0x7f277c009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.269+0000 7f2794440700 1 -- 172.21.15.33:0/1306205623 shutdown_connections 2024-04-25T01:36:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.269+0000 7f2794440700 1 --2- 172.21.15.33:0/1306205623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fcfa0 0x7f278c0fd410 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.269+0000 7f2794440700 1 --2- 172.21.15.33:0/1306205623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c0fbe30 0x7f278c0fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.269+0000 7f2794440700 1 -- 172.21.15.33:0/1306205623 >> 172.21.15.33:0/1306205623 conn(0x7f278c0f7440 msgr2=0x7f278c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.269+0000 7f2794440700 1 -- 172.21.15.33:0/1306205623 shutdown_connections 2024-04-25T01:36:07.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.269+0000 7f2794440700 1 -- 172.21.15.33:0/1306205623 wait complete. 2024-04-25T01:36:07.245 INFO:tasks.workunit.client.1.smithi138.stdout:4/407: mkdir d0/d78 0 2024-04-25T01:36:07.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f2794440700 1 Processor -- start 2024-04-25T01:36:07.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f2794440700 1 -- start start 2024-04-25T01:36:07.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f2794440700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fbe30 0x7f278c0ffe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:07.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f2794440700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c0fcfa0 0x7f278c0fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:07.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f2794440700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0fe890 con 0x7f278c0fbe30 2024-04-25T01:36:07.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f2794440700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c0fea00 con 0x7f278c0fcfa0 2024-04-25T01:36:07.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f27921dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fbe30 0x7f278c0ffe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:07.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f27921dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fbe30 0x7f278c0ffe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60246/0 (socket says 172.21.15.33:60246) 2024-04-25T01:36:07.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f27921dc700 1 -- 172.21.15.33:0/233363754 learned_addr learned my addr 172.21.15.33:0/233363754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:07.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f27919db700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c0fcfa0 0x7f278c0fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:07.247 INFO:tasks.workunit.client.0.smithi033.stdout:1/990: creat d2/d11/d28/dd4/d13d/f145 x:0 0 0 2024-04-25T01:36:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f27921dc700 1 -- 172.21.15.33:0/233363754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c0fcfa0 msgr2=0x7f278c0fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f27921dc700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c0fcfa0 0x7f278c0fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.270+0000 7f27921dc700 1 -- 172.21.15.33:0/233363754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f277c005040 con 0x7f278c0fbe30 2024-04-25T01:36:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.271+0000 7f27921dc700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fbe30 0x7f278c0ffe10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f278800eb40 tx=0x7f278800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:07.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.271+0000 7f2782ffd700 1 -- 172.21.15.33:0/233363754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2788007cc0 con 0x7f278c0fbe30 2024-04-25T01:36:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.271+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c0fec80 con 0x7f278c0fbe30 2024-04-25T01:36:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.271+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c115800 con 0x7f278c0fbe30 2024-04-25T01:36:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.271+0000 7f2782ffd700 1 -- 172.21.15.33:0/233363754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f278800f040 con 0x7f278c0fbe30 2024-04-25T01:36:07.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.271+0000 7f2782ffd700 1 -- 172.21.15.33:0/233363754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f278800ca10 con 0x7f278c0fbe30 2024-04-25T01:36:07.249 INFO:tasks.workunit.client.0.smithi033.stdout:1/991: dread - d2/d5/d6c/dc6/f124 zero size 2024-04-25T01:36:07.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.272+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27700052b0 con 0x7f278c0fbe30 2024-04-25T01:36:07.250 INFO:tasks.workunit.client.0.smithi033.stdout:1/992: read d2/d5/d75/d59/fae [1371286,127919] 0 2024-04-25T01:36:07.251 INFO:tasks.workunit.client.1.smithi138.stdout:4/408: chown d0/d1c/d48 4 1 2024-04-25T01:36:07.251 INFO:tasks.workunit.client.1.smithi138.stdout:8/633: write d3/d16/d88/da/db/d15/f30 [2042784,109005] 0 2024-04-25T01:36:07.251 INFO:tasks.workunit.client.1.smithi138.stdout:2/610: dwrite d0/d85/da5/db4/d15/d51/f72 [0,4194304] 0 2024-04-25T01:36:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.274+0000 7f2782ffd700 1 -- 172.21.15.33:0/233363754 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f278800cb70 con 0x7f278c0fbe30 2024-04-25T01:36:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.275+0000 7f2782ffd700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f277806b860 0x7f277806dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.275+0000 7f2782ffd700 1 -- 172.21.15.33:0/233363754 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f2788089f00 con 0x7f278c0fbe30 2024-04-25T01:36:07.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.275+0000 7f27919db700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f277806b860 0x7f277806dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:07.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.277+0000 7f27919db700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f277806b860 0x7f277806dd10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f277c009b30 tx=0x7f277c009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:07.252 INFO:tasks.workunit.client.1.smithi138.stdout:2/611: fsync d0/d85/da5/db4/f48 0 2024-04-25T01:36:07.252 INFO:tasks.workunit.client.0.smithi033.stdout:1/993: readlink d2/d2e/d49/l129 0 2024-04-25T01:36:07.253 INFO:tasks.workunit.client.1.smithi138.stdout:2/612: chown d0/d21/f59 424 1 2024-04-25T01:36:07.254 INFO:tasks.workunit.client.1.smithi138.stdout:4/409: truncate d0/d45/f35 385525 0 2024-04-25T01:36:07.255 INFO:tasks.workunit.client.0.smithi033.stdout:1/994: read d2/f26 [1585346,16134] 0 2024-04-25T01:36:07.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.283+0000 7f2782ffd700 1 -- 172.21.15.33:0/233363754 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f2788050100 con 0x7f278c0fbe30 2024-04-25T01:36:07.258 INFO:tasks.workunit.client.1.smithi138.stdout:4/410: creat d0/d16/f79 x:0 0 0 2024-04-25T01:36:07.259 INFO:tasks.workunit.client.1.smithi138.stdout:2/613: getdents d0/d85/da5/db4/d15/d44/d4c/d56/d6d 0 2024-04-25T01:36:07.259 INFO:tasks.workunit.client.1.smithi138.stdout:4/411: chown d0/d45/c2d 28550704 1 2024-04-25T01:36:07.260 INFO:tasks.workunit.client.1.smithi138.stdout:4/412: chown d0/d1c/c42 5558 1 2024-04-25T01:36:07.261 INFO:tasks.workunit.client.1.smithi138.stdout:2/614: write d0/d85/da5/db4/d15/d51/f72 [3520117,101848] 0 2024-04-25T01:36:07.267 INFO:tasks.workunit.client.1.smithi138.stdout:2/615: mknod d0/d85/da5/db4/d15/d44/d4c/d56/da2/cbc 0 2024-04-25T01:36:07.269 INFO:tasks.workunit.client.1.smithi138.stdout:2/616: truncate d0/d21/f5b 178416 0 2024-04-25T01:36:07.269 INFO:tasks.workunit.client.1.smithi138.stdout:4/413: dwrite d0/d16/f44 [4194304,4194304] 0 2024-04-25T01:36:07.272 INFO:tasks.workunit.client.1.smithi138.stdout:2/617: mknod d0/d85/da5/cbd 0 2024-04-25T01:36:07.276 INFO:tasks.workunit.client.1.smithi138.stdout:2/618: fsync d0/d85/da5/db4/f87 0 2024-04-25T01:36:07.282 INFO:tasks.workunit.client.1.smithi138.stdout:2/619: link d0/d85/da5/c65 d0/d85/da5/db4/d15/d44/d4c/d56/d6d/cbe 0 2024-04-25T01:36:07.285 INFO:tasks.workunit.client.1.smithi138.stdout:2/620: symlink d0/d21/d45/d8a/lbf 0 2024-04-25T01:36:07.287 INFO:tasks.workunit.client.1.smithi138.stdout:4/414: dread d0/d1c/d1e/f69 [0,4194304] 0 2024-04-25T01:36:07.287 INFO:tasks.workunit.client.1.smithi138.stdout:2/621: write d0/d85/da5/db4/d15/d44/d4c/d56/d6d/f89 [1369093,107763] 0 2024-04-25T01:36:07.288 INFO:tasks.workunit.client.1.smithi138.stdout:4/415: rmdir d0/d45 39 2024-04-25T01:36:07.289 INFO:tasks.workunit.client.1.smithi138.stdout:2/622: mknod d0/d18/cc0 0 2024-04-25T01:36:07.291 INFO:tasks.workunit.client.1.smithi138.stdout:4/416: rmdir d0/d78 0 2024-04-25T01:36:07.293 INFO:tasks.workunit.client.1.smithi138.stdout:2/623: rename d0/d85/da5/db4/d15/d51/c54 to d0/d21/d45/d8a/d8e/cc1 0 2024-04-25T01:36:07.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/417: creat d0/d16/d4a/d74/f7a x:0 0 0 2024-04-25T01:36:07.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/418: fdatasync d0/f3f 0 2024-04-25T01:36:07.295 INFO:tasks.workunit.client.1.smithi138.stdout:2/624: mknod d0/d85/da5/db4/d15/d44/cc2 0 2024-04-25T01:36:07.295 INFO:tasks.workunit.client.1.smithi138.stdout:4/419: write d0/d1c/d52/f65 [553486,91834] 0 2024-04-25T01:36:07.308 INFO:tasks.workunit.client.1.smithi138.stdout:2/625: dread d0/f9 [0,4194304] 0 2024-04-25T01:36:07.309 INFO:tasks.workunit.client.1.smithi138.stdout:4/420: dread d0/f30 [0,4194304] 0 2024-04-25T01:36:07.310 INFO:tasks.workunit.client.1.smithi138.stdout:2/626: write d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fb9 [3206693,122591] 0 2024-04-25T01:36:07.312 INFO:tasks.workunit.client.1.smithi138.stdout:2/627: write d0/d21/f59 [2823673,129489] 0 2024-04-25T01:36:07.312 INFO:tasks.workunit.client.1.smithi138.stdout:4/421: mknod d0/d1c/c7b 0 2024-04-25T01:36:07.314 INFO:tasks.workunit.client.1.smithi138.stdout:4/422: mknod d0/d16/d4a/d74/c7c 0 2024-04-25T01:36:07.318 INFO:tasks.workunit.client.1.smithi138.stdout:4/423: truncate d0/d16/f34 4154257 0 2024-04-25T01:36:07.320 INFO:tasks.workunit.client.1.smithi138.stdout:2/628: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fb9 [0,4194304] 0 2024-04-25T01:36:07.325 INFO:tasks.workunit.client.1.smithi138.stdout:2/629: creat d0/d21/d45/fc3 x:0 0 0 2024-04-25T01:36:07.327 INFO:tasks.workunit.client.1.smithi138.stdout:2/630: symlink d0/d18/lc4 0 2024-04-25T01:36:07.327 INFO:tasks.workunit.client.1.smithi138.stdout:4/424: write d0/d16/d4a/f4d [1545729,94247] 0 2024-04-25T01:36:07.331 INFO:tasks.workunit.client.1.smithi138.stdout:2/631: getdents d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/d98 0 2024-04-25T01:36:07.334 INFO:tasks.workunit.client.1.smithi138.stdout:4/425: link d0/d16/d33/c50 d0/d59/c7d 0 2024-04-25T01:36:07.334 INFO:tasks.workunit.client.1.smithi138.stdout:4/426: readlink d0/d1c/d48/l54 0 2024-04-25T01:36:07.337 INFO:tasks.workunit.client.1.smithi138.stdout:4/427: read - d0/d16/d4a/f77 zero size 2024-04-25T01:36:07.337 INFO:tasks.workunit.client.1.smithi138.stdout:2/632: symlink d0/d85/da5/db4/d15/d44/d4c/d56/da2/lc5 0 2024-04-25T01:36:07.338 INFO:tasks.workunit.client.1.smithi138.stdout:2/633: creat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fc6 x:0 0 0 2024-04-25T01:36:07.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/634: dread d0/d21/f5b [0,4194304] 0 2024-04-25T01:36:07.341 INFO:tasks.workunit.client.1.smithi138.stdout:4/428: dwrite d0/d1c/f2c [0,4194304] 0 2024-04-25T01:36:07.343 INFO:tasks.workunit.client.1.smithi138.stdout:2/635: write d0/d21/f5b [746125,63185] 0 2024-04-25T01:36:07.344 INFO:tasks.workunit.client.1.smithi138.stdout:2/636: fsync d0/d85/da5/db4/d15/d51/f80 0 2024-04-25T01:36:07.345 INFO:tasks.workunit.client.1.smithi138.stdout:4/429: mkdir d0/d16/d7e 0 2024-04-25T01:36:07.346 INFO:tasks.workunit.client.1.smithi138.stdout:4/430: dread - d0/d16/d4a/f68 zero size 2024-04-25T01:36:07.347 INFO:tasks.workunit.client.1.smithi138.stdout:2/637: creat d0/d18/d8b/fc7 x:0 0 0 2024-04-25T01:36:07.350 INFO:tasks.workunit.client.1.smithi138.stdout:4/431: mkdir d0/d45/d7f 0 2024-04-25T01:36:07.351 INFO:tasks.workunit.client.1.smithi138.stdout:4/432: truncate d0/d1c/f62 338485 0 2024-04-25T01:36:07.352 INFO:tasks.workunit.client.1.smithi138.stdout:2/638: mkdir d0/d85/da5/db4/d15/d44/d4c/d56/dc8 0 2024-04-25T01:36:07.353 INFO:tasks.workunit.client.1.smithi138.stdout:2/639: write d0/d18/f6c [1199700,15970] 0 2024-04-25T01:36:07.355 INFO:tasks.workunit.client.1.smithi138.stdout:2/640: fdatasync d0/d18/f1b 0 2024-04-25T01:36:07.358 INFO:tasks.workunit.client.1.smithi138.stdout:2/641: creat d0/d21/d45/d8a/fc9 x:0 0 0 2024-04-25T01:36:07.361 INFO:tasks.workunit.client.1.smithi138.stdout:2/642: getdents d0/d85/da5/db4/d15 0 2024-04-25T01:36:07.362 INFO:tasks.workunit.client.1.smithi138.stdout:2/643: truncate d0/d85/da5/db4/f9d 985018 0 2024-04-25T01:36:07.363 INFO:tasks.workunit.client.1.smithi138.stdout:2/644: stat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/c8f 0 2024-04-25T01:36:07.364 INFO:tasks.workunit.client.1.smithi138.stdout:2/645: readlink d0/d85/da5/db4/d15/l7b 0 2024-04-25T01:36:07.366 INFO:tasks.workunit.client.1.smithi138.stdout:2/646: getdents d0/d21/d45 0 2024-04-25T01:36:07.370 INFO:tasks.workunit.client.1.smithi138.stdout:2/647: dread d0/d85/da5/db4/d15/d51/f80 [0,4194304] 0 2024-04-25T01:36:07.371 INFO:tasks.workunit.client.1.smithi138.stdout:2/648: write d0/fa1 [549111,17541] 0 2024-04-25T01:36:07.375 INFO:tasks.workunit.client.1.smithi138.stdout:2/649: creat d0/d85/da5/db4/d15/d44/d4c/d56/da2/fca x:0 0 0 2024-04-25T01:36:07.376 INFO:tasks.workunit.client.1.smithi138.stdout:2/650: readlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/l88 0 2024-04-25T01:36:07.377 INFO:tasks.workunit.client.1.smithi138.stdout:2/651: chown d0/d85/da5/db4/d15/d44/d4c/d56/fb0 363 1 2024-04-25T01:36:07.378 INFO:tasks.workunit.client.1.smithi138.stdout:2/652: write d0/d85/da5/db4/d15/d44/d4c/d56/fb8 [826377,55098] 0 2024-04-25T01:36:07.381 INFO:tasks.workunit.client.0.smithi033.stdout:1/995: sync 2024-04-25T01:36:07.382 INFO:tasks.workunit.client.0.smithi033.stdout:1/996: write d2/de2/ff2 [2003819,55690] 0 2024-04-25T01:36:07.383 INFO:tasks.workunit.client.0.smithi033.stdout:1/997: dread - d2/d5/d1b/d46/d5f/f125 zero size 2024-04-25T01:36:07.386 INFO:tasks.workunit.client.0.smithi033.stdout:1/998: creat d2/d5/d1b/d46/d5f/d63/d64/f146 x:0 0 0 2024-04-25T01:36:07.388 INFO:tasks.workunit.client.0.smithi033.stdout:1/999: creat d2/d5/d1b/d46/d5f/d63/d64/f147 x:0 0 0 2024-04-25T01:36:07.390 INFO:tasks.workunit.client.1.smithi138.stdout:2/653: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/da2/f37 [0,4194304] 0 2024-04-25T01:36:07.392 INFO:tasks.workunit.client.0.smithi033.stderr:+ rm -rf -- ./tmp.IsQvjW8rMQ 2024-04-25T01:36:07.393 INFO:tasks.workunit.client.1.smithi138.stdout:2/654: rename d0/d85/da5/c33 to d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/ccb 0 2024-04-25T01:36:07.401 INFO:tasks.workunit.client.1.smithi138.stdout:2/655: write d0/d85/da5/f63 [489660,32660] 0 2024-04-25T01:36:07.408 INFO:tasks.workunit.client.1.smithi138.stdout:2/656: dwrite d0/d85/da5/f63 [0,4194304] 0 2024-04-25T01:36:07.414 INFO:tasks.workunit.client.1.smithi138.stdout:2/657: write d0/fe [2157545,21175] 0 2024-04-25T01:36:07.415 INFO:tasks.workunit.client.1.smithi138.stdout:2/658: dread - d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fc6 zero size 2024-04-25T01:36:07.425 INFO:tasks.workunit.client.1.smithi138.stdout:2/659: dwrite d0/d85/da5/f63 [4194304,4194304] 0 2024-04-25T01:36:07.426 INFO:tasks.workunit.client.1.smithi138.stdout:2/660: chown d0/d21/d45/d8a 2 1 2024-04-25T01:36:07.431 INFO:tasks.workunit.client.1.smithi138.stdout:2/661: mknod d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/d98/ccc 0 2024-04-25T01:36:07.434 INFO:tasks.workunit.client.1.smithi138.stdout:2/662: creat d0/d85/da5/db4/d15/d44/d4c/d56/da2/fcd x:0 0 0 2024-04-25T01:36:07.436 INFO:tasks.workunit.client.1.smithi138.stdout:2/663: symlink d0/d85/da5/lce 0 2024-04-25T01:36:07.438 INFO:tasks.workunit.client.1.smithi138.stdout:2/664: fsync d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fb1 0 2024-04-25T01:36:07.444 INFO:tasks.workunit.client.1.smithi138.stdout:2/665: truncate d0/d85/da5/db4/d15/d44/d4c/d56/da2/f4a 1278821 0 2024-04-25T01:36:07.447 INFO:tasks.workunit.client.1.smithi138.stdout:2/666: creat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/fcf x:0 0 0 2024-04-25T01:36:07.450 INFO:tasks.workunit.client.1.smithi138.stdout:2/667: rename d0/d85/da5/db4/d15/cb6 to d0/d85/da5/db4/d15/d44/cd0 0 2024-04-25T01:36:07.461 INFO:tasks.workunit.client.1.smithi138.stdout:2/668: dread d0/d18/f1b [0,4194304] 0 2024-04-25T01:36:07.463 INFO:tasks.workunit.client.1.smithi138.stdout:2/669: symlink d0/ld1 0 2024-04-25T01:36:07.473 INFO:tasks.workunit.client.1.smithi138.stdout:2/670: dread d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fb9 [0,4194304] 0 2024-04-25T01:36:07.476 INFO:tasks.workunit.client.1.smithi138.stdout:2/671: link d0/d85/da5/d35/f70 d0/d85/da5/db4/d15/d44/d4c/fd2 0 2024-04-25T01:36:07.478 INFO:tasks.workunit.client.1.smithi138.stdout:2/672: symlink d0/d85/da5/db4/d15/d44/d4c/d56/ld3 0 2024-04-25T01:36:07.481 INFO:tasks.workunit.client.1.smithi138.stdout:2/673: rename d0/d85/da5/db4/cb7 to d0/d21/d45/cd4 0 2024-04-25T01:36:07.483 INFO:tasks.workunit.client.1.smithi138.stdout:2/674: mkdir d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5 0 2024-04-25T01:36:07.488 INFO:tasks.workunit.client.1.smithi138.stdout:2/675: write d0/d18/f1b [3439796,19193] 0 2024-04-25T01:36:07.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:07 smithi033 ceph-mon[26353]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:07.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:07 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:07.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:07 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:07.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:07 smithi033 ceph-mon[26353]: pgmap v19: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 110 MiB/s rd, 352 MiB/s wr, 669 op/s 2024-04-25T01:36:07.493 INFO:tasks.workunit.client.1.smithi138.stdout:2/676: creat d0/d21/d45/d8a/d8e/fd6 x:0 0 0 2024-04-25T01:36:07.494 INFO:tasks.workunit.client.1.smithi138.stdout:2/677: symlink d0/d21/ld7 0 2024-04-25T01:36:07.495 INFO:tasks.workunit.client.1.smithi138.stdout:2/678: creat d0/d18/fd8 x:0 0 0 2024-04-25T01:36:07.497 INFO:tasks.workunit.client.1.smithi138.stdout:2/679: write d0/f9 [3321658,113292] 0 2024-04-25T01:36:07.499 INFO:tasks.workunit.client.1.smithi138.stdout:2/680: symlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/ld9 0 2024-04-25T01:36:07.504 INFO:tasks.workunit.client.1.smithi138.stdout:2/681: write d0/d85/da5/d35/f40 [1638344,128330] 0 2024-04-25T01:36:07.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:07 smithi138 ceph-mon[32977]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:07.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:07 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:07.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:07 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:07.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:07 smithi138 ceph-mon[32977]: pgmap v19: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 110 MiB/s rd, 352 MiB/s wr, 669 op/s 2024-04-25T01:36:07.594 INFO:tasks.workunit.client.1.smithi138.stdout:2/682: dwrite d0/d85/da5/d35/f69 [0,4194304] 0 2024-04-25T01:36:07.599 INFO:tasks.workunit.client.1.smithi138.stdout:2/683: creat d0/d85/da5/db4/d15/d44/d4c/d56/fda x:0 0 0 2024-04-25T01:36:07.602 INFO:tasks.workunit.client.1.smithi138.stdout:2/684: creat d0/d21/fdb x:0 0 0 2024-04-25T01:36:07.604 INFO:tasks.workunit.client.1.smithi138.stdout:2/685: mknod d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/cdc 0 2024-04-25T01:36:07.607 INFO:tasks.workunit.client.1.smithi138.stdout:2/686: link d0/d85/da5/l42 d0/d18/d8b/ldd 0 2024-04-25T01:36:07.651 INFO:tasks.workunit.client.1.smithi138.stdout:5/448: sync 2024-04-25T01:36:07.652 INFO:tasks.workunit.client.1.smithi138.stdout:5/449: fdatasync d4/da/f47 0 2024-04-25T01:36:07.654 INFO:tasks.workunit.client.1.smithi138.stdout:5/450: creat d4/da/d6e/d5c/f95 x:0 0 0 2024-04-25T01:36:07.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.681+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2770005830 con 0x7f278c0fbe30 2024-04-25T01:36:07.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.681+0000 7f2782ffd700 1 -- 172.21.15.33:0/233363754 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f2788053aa0 con 0x7f278c0fbe30 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:36:07.657 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:36:07.658 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:36:07.658 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:36:07.658 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:36:07.658 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-25T01:36:07.658 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-25T01:36:07.658 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:36:07.658 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:36:07.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.684+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f277806b860 msgr2=0x7f277806dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.684+0000 7f2794440700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f277806b860 0x7f277806dd10 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f277c009b30 tx=0x7f277c009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:07.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.684+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fbe30 msgr2=0x7f278c0ffe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.684+0000 7f2794440700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fbe30 0x7f278c0ffe10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f278800eb40 tx=0x7f278800b040 comp rx=0 tx=0).stop 2024-04-25T01:36:07.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.684+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 shutdown_connections 2024-04-25T01:36:07.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.684+0000 7f2794440700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f277806b860 0x7f277806dd10 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.684+0000 7f2794440700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f278c0fbe30 0x7f278c0ffe10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.684+0000 7f2794440700 1 --2- 172.21.15.33:0/233363754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f278c0fcfa0 0x7f278c0fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.684+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 >> 172.21.15.33:0/233363754 conn(0x7f278c0f7440 msgr2=0x7f278c0f86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:07.660 INFO:tasks.workunit.client.1.smithi138.stdout:5/451: dwrite d4/da/d52/f7c [0,4194304] 0 2024-04-25T01:36:07.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.685+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 shutdown_connections 2024-04-25T01:36:07.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.685+0000 7f2794440700 1 -- 172.21.15.33:0/233363754 wait complete. 2024-04-25T01:36:07.661 INFO:tasks.workunit.client.1.smithi138.stdout:5/452: write d4/da/d6e/d22/f24 [7408194,44013] 0 2024-04-25T01:36:07.708 INFO:tasks.workunit.client.1.smithi138.stdout:5/453: creat d4/da/d6e/d22/d74/d59/d83/d8a/f96 x:0 0 0 2024-04-25T01:36:07.708 INFO:tasks.workunit.client.1.smithi138.stdout:5/454: chown d4/da/d27/c3e 0 1 2024-04-25T01:36:07.709 INFO:tasks.workunit.client.1.smithi138.stdout:5/455: chown d4/da/d6e/d22/d35/d4d/c53 362349757 1 2024-04-25T01:36:07.709 INFO:tasks.workunit.client.1.smithi138.stdout:5/456: stat d4/da/d6e/d22/d45/f54 0 2024-04-25T01:36:07.712 INFO:tasks.workunit.client.1.smithi138.stdout:5/457: creat d4/da/d6e/d22/d74/d59/d7f/f97 x:0 0 0 2024-04-25T01:36:07.714 INFO:tasks.workunit.client.1.smithi138.stdout:5/458: creat d4/da/d6e/d22/f98 x:0 0 0 2024-04-25T01:36:07.717 INFO:tasks.workunit.client.1.smithi138.stdout:5/459: creat d4/da/d6e/d22/d74/d59/f99 x:0 0 0 2024-04-25T01:36:07.722 INFO:tasks.workunit.client.1.smithi138.stdout:5/460: dwrite d4/da/d27/f1f [0,4194304] 0 2024-04-25T01:36:07.724 INFO:tasks.workunit.client.1.smithi138.stdout:5/461: write d4/da/d6e/d22/d35/d4d/f66 [903122,126109] 0 2024-04-25T01:36:07.725 INFO:tasks.workunit.client.1.smithi138.stdout:5/462: chown d4/da/d6e/d22/d35/d4d/d90/l8d 1480840 1 2024-04-25T01:36:07.726 INFO:tasks.workunit.client.1.smithi138.stdout:5/463: read - d4/da/d6e/d22/d35/d4d/d5e/f75 zero size 2024-04-25T01:36:07.727 INFO:tasks.workunit.client.1.smithi138.stdout:5/464: write d4/da/d6e/d22/d74/d59/f68 [1936448,51031] 0 2024-04-25T01:36:07.730 INFO:tasks.workunit.client.1.smithi138.stdout:5/465: symlink d4/da/l9a 0 2024-04-25T01:36:07.731 INFO:tasks.workunit.client.1.smithi138.stdout:5/466: stat d4/da/d6e/d22/d74/d59/d83 0 2024-04-25T01:36:07.731 INFO:tasks.workunit.client.1.smithi138.stdout:5/467: stat d4/da/d6e/d22/d35 0 2024-04-25T01:36:07.734 INFO:tasks.workunit.client.1.smithi138.stdout:5/468: symlink d4/da/d27/l9b 0 2024-04-25T01:36:07.737 INFO:tasks.workunit.client.1.smithi138.stdout:5/469: creat d4/da/d6e/d22/d74/f9c x:0 0 0 2024-04-25T01:36:07.738 INFO:tasks.workunit.client.1.smithi138.stdout:5/470: mkdir d4/da/d6e/d22/d74/d59/d83/d89/d9d 0 2024-04-25T01:36:07.740 INFO:tasks.workunit.client.1.smithi138.stdout:5/471: mkdir d4/da/d6e/d22/d74/d59/d83/d89/d9d/d9e 0 2024-04-25T01:36:07.740 INFO:tasks.workunit.client.1.smithi138.stdout:5/472: chown d4/da 257591681 1 2024-04-25T01:36:07.746 INFO:tasks.workunit.client.1.smithi138.stdout:5/473: write d4/da/d6e/d22/d35/d4d/d5e/f7a [185933,55838] 0 2024-04-25T01:36:07.747 INFO:tasks.workunit.client.1.smithi138.stdout:5/474: chown d4/l8 5769 1 2024-04-25T01:36:07.752 INFO:tasks.workunit.client.1.smithi138.stdout:7/604: sync 2024-04-25T01:36:07.752 INFO:tasks.workunit.client.1.smithi138.stdout:9/471: sync 2024-04-25T01:36:07.752 INFO:tasks.workunit.client.1.smithi138.stdout:3/641: sync 2024-04-25T01:36:07.752 INFO:tasks.workunit.client.1.smithi138.stdout:1/486: sync 2024-04-25T01:36:07.752 INFO:tasks.workunit.client.1.smithi138.stdout:6/575: sync 2024-04-25T01:36:07.753 INFO:tasks.workunit.client.1.smithi138.stdout:9/472: fdatasync d4/d1a/d33/f4d 0 2024-04-25T01:36:07.754 INFO:tasks.workunit.client.1.smithi138.stdout:7/605: stat da/de/d1b/d49/d81/d51/d6e/da2 0 2024-04-25T01:36:07.755 INFO:tasks.workunit.client.1.smithi138.stdout:9/473: chown d4/d8/d90/d52/f7d 1 1 2024-04-25T01:36:07.755 INFO:tasks.workunit.client.1.smithi138.stdout:1/487: rmdir d3/d45/d72/d3d 39 2024-04-25T01:36:07.756 INFO:tasks.workunit.client.1.smithi138.stdout:9/474: truncate d4/d1a/f20 1984831 0 2024-04-25T01:36:07.757 INFO:tasks.workunit.client.1.smithi138.stdout:6/576: symlink dc/d2e/d55/d81/lb5 0 2024-04-25T01:36:07.758 INFO:tasks.workunit.client.1.smithi138.stdout:3/642: chown db/df/d17/d23/d88/l8a 426742 1 2024-04-25T01:36:07.758 INFO:tasks.workunit.client.1.smithi138.stdout:9/475: read d4/d1a/f92 [2645051,17059] 0 2024-04-25T01:36:07.759 INFO:tasks.workunit.client.1.smithi138.stdout:1/488: rmdir d3/db 39 2024-04-25T01:36:07.760 INFO:tasks.workunit.client.1.smithi138.stdout:3/643: fdatasync db/df/d17/f1c 0 2024-04-25T01:36:07.760 INFO:tasks.workunit.client.1.smithi138.stdout:9/476: chown d4/d1a/d55/d7f 17 1 2024-04-25T01:36:07.763 INFO:tasks.workunit.client.1.smithi138.stdout:7/606: mkdir da/de/d13/d21/dbc/d94/d9d/dc9 0 2024-04-25T01:36:07.764 INFO:tasks.workunit.client.1.smithi138.stdout:6/577: creat dc/d2e/d55/d81/fb6 x:0 0 0 2024-04-25T01:36:07.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/475: dread d4/da/d6e/d22/d74/f85 [0,4194304] 0 2024-04-25T01:36:07.765 INFO:tasks.workunit.client.1.smithi138.stdout:3/644: mknod db/df/ccc 0 2024-04-25T01:36:07.766 INFO:tasks.workunit.client.1.smithi138.stdout:9/477: creat d4/d1a/d68/f98 x:0 0 0 2024-04-25T01:36:07.768 INFO:tasks.workunit.client.1.smithi138.stdout:3/645: truncate db/df/d19/d1a/f94 542484 0 2024-04-25T01:36:07.768 INFO:tasks.workunit.client.1.smithi138.stdout:6/578: write dc/d2e/f6c [1095686,55004] 0 2024-04-25T01:36:07.768 INFO:tasks.workunit.client.1.smithi138.stdout:9/478: read d4/f26 [1423468,6667] 0 2024-04-25T01:36:07.770 INFO:tasks.workunit.client.1.smithi138.stdout:1/489: link d3/d45/d72/f5d d3/d4/d47/d9b/f9c 0 2024-04-25T01:36:07.771 INFO:tasks.workunit.client.1.smithi138.stdout:9/479: chown d4/d1a/f84 2 1 2024-04-25T01:36:07.772 INFO:tasks.workunit.client.1.smithi138.stdout:5/476: mkdir d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f 0 2024-04-25T01:36:07.775 INFO:tasks.workunit.client.1.smithi138.stdout:1/490: mkdir d3/d4/d53/d9d 0 2024-04-25T01:36:07.779 INFO:tasks.workunit.client.1.smithi138.stdout:7/607: dwrite da/de/d1b/d49/d81/d51/d6e/da2/dae/daf/fb9 [0,4194304] 0 2024-04-25T01:36:07.781 INFO:tasks.workunit.client.1.smithi138.stdout:6/579: dwrite dc/fa1 [0,4194304] 0 2024-04-25T01:36:07.781 INFO:tasks.workunit.client.1.smithi138.stdout:5/477: rmdir d4/da/d6e/d22/d74 39 2024-04-25T01:36:07.784 INFO:tasks.workunit.client.1.smithi138.stdout:6/580: chown dc/d10/d28/d70/d83 41151568 1 2024-04-25T01:36:07.785 INFO:tasks.workunit.client.1.smithi138.stdout:5/478: dread - d4/da/d52/f92 zero size 2024-04-25T01:36:07.787 INFO:tasks.workunit.client.1.smithi138.stdout:1/491: creat d3/d27/d3e/d51/f9e x:0 0 0 2024-04-25T01:36:07.789 INFO:tasks.workunit.client.1.smithi138.stdout:3/646: rename db/df/d19/d1a/d9e/lcb to db/df/d17/d20/lcd 0 2024-04-25T01:36:07.791 INFO:tasks.workunit.client.1.smithi138.stdout:6/581: mkdir dc/d2e/d37/d56/d8c/db7 0 2024-04-25T01:36:07.791 INFO:tasks.workunit.client.1.smithi138.stdout:9/480: dwrite d4/d49/d72/f88 [0,4194304] 0 2024-04-25T01:36:07.792 INFO:tasks.workunit.client.1.smithi138.stdout:7/608: dwrite da/de/d1b/d34/f3d [0,4194304] 0 2024-04-25T01:36:07.794 INFO:tasks.workunit.client.1.smithi138.stdout:9/481: fsync d4/d1a/f42 0 2024-04-25T01:36:07.796 INFO:tasks.workunit.client.1.smithi138.stdout:9/482: chown d4/d8/d90/f78 36 1 2024-04-25T01:36:07.802 INFO:tasks.workunit.client.1.smithi138.stdout:6/582: creat dc/d2e/d55/d81/fb8 x:0 0 0 2024-04-25T01:36:07.804 INFO:tasks.workunit.client.1.smithi138.stdout:6/583: chown dc/d10/d67/d8e/faa 808459333 1 2024-04-25T01:36:07.804 INFO:tasks.workunit.client.1.smithi138.stdout:7/609: unlink da/de/d1b/d34/c70 0 2024-04-25T01:36:07.805 INFO:tasks.workunit.client.1.smithi138.stdout:6/584: dread - dc/d7b/f94 zero size 2024-04-25T01:36:07.807 INFO:tasks.workunit.client.1.smithi138.stdout:5/479: creat d4/da/fa0 x:0 0 0 2024-04-25T01:36:07.808 INFO:tasks.workunit.client.1.smithi138.stdout:5/480: chown d4/da/d6e/d22/d35/l46 262092 1 2024-04-25T01:36:07.808 INFO:tasks.workunit.client.1.smithi138.stdout:1/492: rmdir d3/d27/d3e/d51/d84 0 2024-04-25T01:36:07.809 INFO:tasks.workunit.client.1.smithi138.stdout:1/493: stat d3/d45/d72/d57/f63 0 2024-04-25T01:36:07.810 INFO:tasks.workunit.client.1.smithi138.stdout:9/483: creat d4/d1a/d55/d7f/d8f/f99 x:0 0 0 2024-04-25T01:36:07.810 INFO:tasks.workunit.client.1.smithi138.stdout:7/610: read da/de/d1b/d49/f89 [738713,76058] 0 2024-04-25T01:36:07.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.836+0000 7f97bd123700 1 -- 172.21.15.33:0/697765670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b80fc290 msgr2=0x7f97b80fc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.836+0000 7f97bd123700 1 --2- 172.21.15.33:0/697765670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b80fc290 0x7f97b80fc6a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f97a8004940 tx=0x7f97a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:07.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.836+0000 7f97bd123700 1 -- 172.21.15.33:0/697765670 shutdown_connections 2024-04-25T01:36:07.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.836+0000 7f97bd123700 1 --2- 172.21.15.33:0/697765670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b80fcbe0 0x7f97b80fd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.836+0000 7f97bd123700 1 --2- 172.21.15.33:0/697765670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b80fc290 0x7f97b80fc6a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.836+0000 7f97bd123700 1 -- 172.21.15.33:0/697765670 >> 172.21.15.33:0/697765670 conn(0x7f97b80f6b80 msgr2=0x7f97b80f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:07.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.837+0000 7f97bd123700 1 -- 172.21.15.33:0/697765670 shutdown_connections 2024-04-25T01:36:07.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.837+0000 7f97bd123700 1 -- 172.21.15.33:0/697765670 wait complete. 2024-04-25T01:36:07.813 INFO:tasks.workunit.client.1.smithi138.stdout:1/494: chown d3/d4/d47/d9b/f68 271010 1 2024-04-25T01:36:07.813 INFO:tasks.workunit.client.1.smithi138.stdout:7/611: readlink da/de/d1b/d34/l60 0 2024-04-25T01:36:07.813 INFO:tasks.workunit.client.1.smithi138.stdout:5/481: mkdir d4/da/d6e/d22/d45/da1 0 2024-04-25T01:36:07.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.837+0000 7f97bd123700 1 Processor -- start 2024-04-25T01:36:07.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.837+0000 7f97bd123700 1 -- start start 2024-04-25T01:36:07.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97bd123700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b80fc290 0x7f97b810fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:07.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97bd123700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b80fcbe0 0x7f97b8110150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:07.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97bd123700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8110750 con 0x7f97b80fcbe0 2024-04-25T01:36:07.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97bd123700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8110f50 con 0x7f97b80fc290 2024-04-25T01:36:07.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97b6d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b80fc290 0x7f97b810fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97b6d9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b80fc290 0x7f97b810fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:36356/0 (socket says 172.21.15.33:36356) 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97b6d9d700 1 -- 172.21.15.33:0/159158699 learned_addr learned my addr 172.21.15.33:0/159158699 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97b659c700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b80fcbe0 0x7f97b8110150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97b6d9d700 1 -- 172.21.15.33:0/159158699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b80fcbe0 msgr2=0x7f97b8110150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97b6d9d700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b80fcbe0 0x7f97b8110150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.838+0000 7f97b6d9d700 1 -- 172.21.15.33:0/159158699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a8005040 con 0x7f97b80fc290 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.839+0000 7f97b6d9d700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b80fc290 0x7f97b810fc10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009040 tx=0x7f97a8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.839+0000 7f979f7fe700 1 -- 172.21.15.33:0/159158699 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a800e070 con 0x7f97b80fc290 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.839+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b81111d0 con 0x7f97b80fc290 2024-04-25T01:36:07.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.839+0000 7f979f7fe700 1 -- 172.21.15.33:0/159158699 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97a8013b50 con 0x7f97b80fc290 2024-04-25T01:36:07.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.839+0000 7f979f7fe700 1 -- 172.21.15.33:0/159158699 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97a80255e0 con 0x7f97b80fc290 2024-04-25T01:36:07.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.839+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b8111710 con 0x7f97b80fc290 2024-04-25T01:36:07.818 INFO:tasks.workunit.client.1.smithi138.stdout:9/484: mkdir d4/d1a/d33/d9a 0 2024-04-25T01:36:07.818 INFO:tasks.workunit.client.1.smithi138.stdout:9/485: chown d4/d8 7510284 1 2024-04-25T01:36:07.818 INFO:tasks.workunit.client.1.smithi138.stdout:7/612: mkdir da/de/d13/d21/dbc/dca 0 2024-04-25T01:36:07.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.841+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b81019e0 con 0x7f97b80fc290 2024-04-25T01:36:07.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.841+0000 7f979f7fe700 1 -- 172.21.15.33:0/159158699 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f97a8013cc0 con 0x7f97b80fc290 2024-04-25T01:36:07.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.842+0000 7f979f7fe700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f97a006b790 0x7f97a006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:07.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.842+0000 7f979f7fe700 1 -- 172.21.15.33:0/159158699 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f97a808bf10 con 0x7f97b80fc290 2024-04-25T01:36:07.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.842+0000 7f97b659c700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f97a006b790 0x7f97a006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:07.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.843+0000 7f97b659c700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f97a006b790 0x7f97a006dc40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009b80 tx=0x7f97ac00b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:07.819 INFO:tasks.workunit.client.1.smithi138.stdout:4/433: sync 2024-04-25T01:36:07.819 INFO:tasks.workunit.client.1.smithi138.stdout:2/687: sync 2024-04-25T01:36:07.819 INFO:tasks.workunit.client.1.smithi138.stdout:8/634: sync 2024-04-25T01:36:07.820 INFO:tasks.workunit.client.1.smithi138.stdout:9/486: chown d4/d8/d2c/l41 1922208 1 2024-04-25T01:36:07.821 INFO:tasks.workunit.client.1.smithi138.stdout:0/402: sync 2024-04-25T01:36:07.822 INFO:tasks.workunit.client.1.smithi138.stdout:6/585: dwrite dc/d2e/f3e [0,4194304] 0 2024-04-25T01:36:07.822 INFO:tasks.workunit.client.1.smithi138.stdout:3/647: readlink db/df/d17/d20/lcd 0 2024-04-25T01:36:07.823 INFO:tasks.workunit.client.1.smithi138.stdout:4/434: write d0/d1c/f2b [9632567,51789] 0 2024-04-25T01:36:07.823 INFO:tasks.workunit.client.1.smithi138.stdout:5/482: creat d4/da/d6e/d22/d35/d4d/d5e/d8f/fa2 x:0 0 0 2024-04-25T01:36:07.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:07.851+0000 7f979f7fe700 1 -- 172.21.15.33:0/159158699 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f97a80594f0 con 0x7f97b80fc290 2024-04-25T01:36:07.827 INFO:tasks.workunit.client.1.smithi138.stdout:7/613: unlink da/de/d13/c6b 0 2024-04-25T01:36:07.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/495: dwrite d3/d6d/f7c [0,4194304] 0 2024-04-25T01:36:07.906 INFO:tasks.workunit.client.1.smithi138.stdout:9/487: unlink d4/d8/d2c/l4c 0 2024-04-25T01:36:07.912 INFO:tasks.workunit.client.1.smithi138.stdout:5/483: dread d4/da/d52/f7c [0,4194304] 0 2024-04-25T01:36:07.913 INFO:tasks.workunit.client.1.smithi138.stdout:9/488: creat d4/d49/d72/f9b x:0 0 0 2024-04-25T01:36:07.914 INFO:tasks.workunit.client.1.smithi138.stdout:1/496: dread d3/d45/d72/d57/f8a [0,4194304] 0 2024-04-25T01:36:07.914 INFO:tasks.workunit.client.1.smithi138.stdout:2/688: mknod d0/d85/da5/db4/d15/d44/d4c/d56/da2/cde 0 2024-04-25T01:36:07.916 INFO:tasks.workunit.client.1.smithi138.stdout:7/614: write da/de/d1b/d49/f89 [2194542,117238] 0 2024-04-25T01:36:07.919 INFO:tasks.workunit.client.1.smithi138.stdout:4/435: dread d0/d1c/f49 [0,4194304] 0 2024-04-25T01:36:07.919 INFO:tasks.workunit.client.1.smithi138.stdout:0/403: getdents d6/d2f/d75 0 2024-04-25T01:36:07.922 INFO:tasks.workunit.client.1.smithi138.stdout:3/648: dwrite db/f3f [0,4194304] 0 2024-04-25T01:36:07.922 INFO:tasks.workunit.client.1.smithi138.stdout:0/404: chown d6/d19/d57 27973093 1 2024-04-25T01:36:07.923 INFO:tasks.workunit.client.1.smithi138.stdout:4/436: write d0/d16/f31 [5358607,4488] 0 2024-04-25T01:36:07.925 INFO:tasks.workunit.client.1.smithi138.stdout:1/497: creat d3/d27/d3e/d90/f9f x:0 0 0 2024-04-25T01:36:07.927 INFO:tasks.workunit.client.1.smithi138.stdout:6/586: dread dc/d10/d28/d80/f82 [0,4194304] 0 2024-04-25T01:36:07.929 INFO:tasks.workunit.client.1.smithi138.stdout:5/484: mknod d4/da/d6e/d22/d35/d4d/ca3 0 2024-04-25T01:36:07.933 INFO:tasks.workunit.client.1.smithi138.stdout:4/437: readlink d0/l47 0 2024-04-25T01:36:07.935 INFO:tasks.workunit.client.1.smithi138.stdout:1/498: creat d3/d45/fa0 x:0 0 0 2024-04-25T01:36:07.936 INFO:tasks.workunit.client.1.smithi138.stdout:9/489: creat d4/d1a/d55/d7f/d80/d94/f9c x:0 0 0 2024-04-25T01:36:07.936 INFO:tasks.workunit.client.1.smithi138.stdout:6/587: fdatasync dc/d10/f33 0 2024-04-25T01:36:07.938 INFO:tasks.workunit.client.1.smithi138.stdout:9/490: write d4/d1a/f3b [278330,55434] 0 2024-04-25T01:36:07.938 INFO:tasks.workunit.client.1.smithi138.stdout:7/615: write da/de/d13/d30/d59/f78 [4936105,94345] 0 2024-04-25T01:36:07.939 INFO:tasks.workunit.client.1.smithi138.stdout:2/689: write d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fb1 [3164830,41786] 0 2024-04-25T01:36:07.941 INFO:tasks.workunit.client.1.smithi138.stdout:0/405: mknod d6/d39/d43/c84 0 2024-04-25T01:36:07.945 INFO:tasks.workunit.client.1.smithi138.stdout:4/438: dwrite d0/d1c/d52/f65 [0,4194304] 0 2024-04-25T01:36:07.946 INFO:tasks.workunit.client.1.smithi138.stdout:1/499: truncate d3/d45/d72/d3d/f8d 5352169 0 2024-04-25T01:36:07.947 INFO:tasks.workunit.client.1.smithi138.stdout:4/439: write d0/f40 [938702,72231] 0 2024-04-25T01:36:07.948 INFO:tasks.workunit.client.1.smithi138.stdout:7/616: rmdir da/de/d13 39 2024-04-25T01:36:07.948 INFO:tasks.workunit.client.1.smithi138.stdout:3/649: link db/f35 db/df/d17/d20/d57/daa/fce 0 2024-04-25T01:36:07.949 INFO:tasks.workunit.client.1.smithi138.stdout:4/440: write d0/d16/f43 [550928,69473] 0 2024-04-25T01:36:07.950 INFO:tasks.workunit.client.1.smithi138.stdout:7/617: chown da/de/d1b/l9b 44220061 1 2024-04-25T01:36:07.950 INFO:tasks.workunit.client.1.smithi138.stdout:0/406: stat d6/df/d41/d49/f51 0 2024-04-25T01:36:07.950 INFO:tasks.workunit.client.1.smithi138.stdout:0/407: stat d6/f26 0 2024-04-25T01:36:07.952 INFO:tasks.workunit.client.1.smithi138.stdout:4/441: read d0/d1c/f2b [7758051,12944] 0 2024-04-25T01:36:07.954 INFO:tasks.workunit.client.1.smithi138.stdout:5/485: rmdir d4/da/d6e/d22/d45/da1 0 2024-04-25T01:36:07.956 INFO:tasks.workunit.client.1.smithi138.stdout:4/442: chown d0/d16/d4a/d74/c7c 108737 1 2024-04-25T01:36:07.956 INFO:tasks.workunit.client.1.smithi138.stdout:6/588: mkdir dc/d2e/db9 0 2024-04-25T01:36:07.957 INFO:tasks.workunit.client.1.smithi138.stdout:5/486: write d4/da/d6e/d5c/f95 [24570,91284] 0 2024-04-25T01:36:07.961 INFO:tasks.workunit.client.1.smithi138.stdout:3/650: fdatasync db/d14/d40/fb0 0 2024-04-25T01:36:07.961 INFO:tasks.workunit.client.1.smithi138.stdout:2/690: symlink d0/d85/da5/db4/d15/d44/d4c/d56/ldf 0 2024-04-25T01:36:07.962 INFO:tasks.workunit.client.1.smithi138.stdout:9/491: dread d4/f4a [0,4194304] 0 2024-04-25T01:36:07.963 INFO:tasks.workunit.client.1.smithi138.stdout:0/408: creat d6/d19/f85 x:0 0 0 2024-04-25T01:36:07.966 INFO:tasks.workunit.client.1.smithi138.stdout:0/409: stat d6/d1d/l22 0 2024-04-25T01:36:07.968 INFO:tasks.workunit.client.1.smithi138.stdout:6/589: write dc/d2e/d37/d56/fac [2081956,28176] 0 2024-04-25T01:36:07.968 INFO:tasks.workunit.client.1.smithi138.stdout:1/500: dwrite d3/db/f5f [4194304,4194304] 0 2024-04-25T01:36:07.970 INFO:tasks.workunit.client.1.smithi138.stdout:5/487: chown d4/da/c39 399 1 2024-04-25T01:36:07.972 INFO:tasks.workunit.client.1.smithi138.stdout:7/618: creat da/de/d1b/d49/d81/dc7/fcb x:0 0 0 2024-04-25T01:36:07.972 INFO:tasks.workunit.client.1.smithi138.stdout:6/590: truncate dc/d10/d28/d70/d83/f9d 403700 0 2024-04-25T01:36:07.973 INFO:tasks.workunit.client.1.smithi138.stdout:3/651: mknod db/df/d17/d20/d57/ccf 0 2024-04-25T01:36:07.978 INFO:tasks.workunit.client.1.smithi138.stdout:0/410: rename d6/d19/f4b to d6/d2f/d31/f86 0 2024-04-25T01:36:07.979 INFO:tasks.workunit.client.1.smithi138.stdout:5/488: readlink d4/da/l2c 0 2024-04-25T01:36:07.980 INFO:tasks.workunit.client.1.smithi138.stdout:4/443: dread d0/d16/d33/f51 [0,4194304] 0 2024-04-25T01:36:07.982 INFO:tasks.workunit.client.1.smithi138.stdout:6/591: mknod dc/d10/d28/d80/d89/cba 0 2024-04-25T01:36:07.983 INFO:tasks.workunit.client.1.smithi138.stdout:3/652: symlink db/df/d17/d20/d57/ld0 0 2024-04-25T01:36:07.983 INFO:tasks.workunit.client.1.smithi138.stdout:9/492: dwrite d4/d1a/d33/f37 [0,4194304] 0 2024-04-25T01:36:07.985 INFO:tasks.workunit.client.1.smithi138.stdout:4/444: write d0/d16/d4a/f76 [955856,113667] 0 2024-04-25T01:36:07.987 INFO:tasks.workunit.client.1.smithi138.stdout:2/691: getdents d0/d85/da5/db4 0 2024-04-25T01:36:07.989 INFO:tasks.workunit.client.1.smithi138.stdout:9/493: dread - d4/d8/d90/d52/f7d zero size 2024-04-25T01:36:07.990 INFO:tasks.workunit.client.1.smithi138.stdout:5/489: chown d4/da/d6e/d22/d74/d59/d83/d89/d9d 0 1 2024-04-25T01:36:07.991 INFO:tasks.workunit.client.1.smithi138.stdout:4/445: fdatasync d0/f3f 0 2024-04-25T01:36:07.992 INFO:tasks.workunit.client.1.smithi138.stdout:3/653: mkdir db/df/dd1 0 2024-04-25T01:36:07.993 INFO:tasks.workunit.client.1.smithi138.stdout:0/411: dwrite d6/fe [0,4194304] 0 2024-04-25T01:36:07.994 INFO:tasks.workunit.client.1.smithi138.stdout:6/592: mknod dc/d10/d67/d8e/cbb 0 2024-04-25T01:36:07.994 INFO:tasks.workunit.client.1.smithi138.stdout:2/692: read d0/fa1 [787786,67418] 0 2024-04-25T01:36:07.995 INFO:tasks.workunit.client.1.smithi138.stdout:3/654: readlink db/df/d17/d23/l39 0 2024-04-25T01:36:07.997 INFO:tasks.workunit.client.1.smithi138.stdout:6/593: dread - dc/d2e/d37/d56/f8f zero size 2024-04-25T01:36:07.997 INFO:tasks.workunit.client.1.smithi138.stdout:9/494: creat d4/d1a/d3c/f9d x:0 0 0 2024-04-25T01:36:07.999 INFO:tasks.workunit.client.1.smithi138.stdout:5/490: truncate d4/da/d52/f92 482340 0 2024-04-25T01:36:08.000 INFO:tasks.workunit.client.1.smithi138.stdout:5/491: fdatasync d4/da/d6e/d22/f93 0 2024-04-25T01:36:08.001 INFO:tasks.workunit.client.1.smithi138.stdout:0/412: fsync d6/df/f14 0 2024-04-25T01:36:08.002 INFO:tasks.workunit.client.1.smithi138.stdout:1/501: write d3/d4/d47/d9b/f9c [797755,8793] 0 2024-04-25T01:36:08.004 INFO:tasks.workunit.client.1.smithi138.stdout:2/693: symlink d0/le0 0 2024-04-25T01:36:08.006 INFO:tasks.workunit.client.1.smithi138.stdout:3/655: getdents db/df/d19/d1a/dc4 0 2024-04-25T01:36:08.007 INFO:tasks.workunit.client.1.smithi138.stdout:5/492: dread d4/da/f20 [0,4194304] 0 2024-04-25T01:36:08.007 INFO:tasks.workunit.client.1.smithi138.stdout:9/495: creat d4/d1a/d33/d9a/f9e x:0 0 0 2024-04-25T01:36:08.008 INFO:tasks.workunit.client.1.smithi138.stdout:3/656: write db/df/d17/d20/d57/fc1 [1039504,80884] 0 2024-04-25T01:36:08.009 INFO:tasks.workunit.client.1.smithi138.stdout:0/413: read d6/f26 [2187955,129718] 0 2024-04-25T01:36:08.009 INFO:tasks.workunit.client.1.smithi138.stdout:6/594: creat dc/d2e/db9/fbc x:0 0 0 2024-04-25T01:36:08.010 INFO:tasks.workunit.client.1.smithi138.stdout:2/694: readlink d0/l1c 0 2024-04-25T01:36:08.011 INFO:tasks.workunit.client.1.smithi138.stdout:0/414: stat d6/df/d41/d49 0 2024-04-25T01:36:08.014 INFO:tasks.workunit.client.1.smithi138.stdout:3/657: readlink db/df/d17/d23/d88/l8c 0 2024-04-25T01:36:08.021 INFO:tasks.workunit.client.1.smithi138.stdout:7/619: dread da/de/d13/d30/f65 [0,4194304] 0 2024-04-25T01:36:08.024 INFO:tasks.workunit.client.1.smithi138.stdout:6/595: link dc/d10/d28/c95 dc/d2e/db9/cbd 0 2024-04-25T01:36:08.027 INFO:tasks.workunit.client.1.smithi138.stdout:3/658: mknod db/df/d17/d20/cd2 0 2024-04-25T01:36:08.028 INFO:tasks.workunit.client.1.smithi138.stdout:9/496: dwrite d4/d63/f7c [0,4194304] 0 2024-04-25T01:36:08.029 INFO:tasks.workunit.client.1.smithi138.stdout:5/493: dwrite d4/da/d6e/d22/d35/d4d/d5e/f7a [0,4194304] 0 2024-04-25T01:36:08.031 INFO:tasks.workunit.client.1.smithi138.stdout:7/620: chown da/de/d1b/f33 330 1 2024-04-25T01:36:08.032 INFO:tasks.workunit.client.1.smithi138.stdout:2/695: truncate d0/d85/da5/db4/d15/d44/d4c/fd2 9189855 0 2024-04-25T01:36:08.032 INFO:tasks.workunit.client.1.smithi138.stdout:0/415: dwrite d6/f56 [0,4194304] 0 2024-04-25T01:36:08.032 INFO:tasks.workunit.client.1.smithi138.stdout:1/502: dread d3/d27/d16/f18 [0,4194304] 0 2024-04-25T01:36:08.034 INFO:tasks.workunit.client.1.smithi138.stdout:6/596: mknod dc/d10/d28/d80/d89/cbe 0 2024-04-25T01:36:08.038 INFO:tasks.workunit.client.1.smithi138.stdout:9/497: dread d4/f6c [0,4194304] 0 2024-04-25T01:36:08.039 INFO:tasks.workunit.client.1.smithi138.stdout:5/494: rmdir d4/da/d6e/d22/d74 39 2024-04-25T01:36:08.039 INFO:tasks.workunit.client.1.smithi138.stdout:3/659: creat db/df/d19/fd3 x:0 0 0 2024-04-25T01:36:08.039 INFO:tasks.workunit.client.1.smithi138.stdout:1/503: mkdir d3/d27/d2d/da1 0 2024-04-25T01:36:08.042 INFO:tasks.workunit.client.1.smithi138.stdout:0/416: unlink d6/l2b 0 2024-04-25T01:36:08.045 INFO:tasks.workunit.client.1.smithi138.stdout:7/621: dread da/de/d13/d30/d59/d7b/d99/fab [0,4194304] 0 2024-04-25T01:36:08.045 INFO:tasks.workunit.client.1.smithi138.stdout:2/696: getdents d0/d85/da5/db4/d15 0 2024-04-25T01:36:08.046 INFO:tasks.workunit.client.1.smithi138.stdout:7/622: chown da/de/d13/d30/c4e 2707258 1 2024-04-25T01:36:08.047 INFO:tasks.workunit.client.1.smithi138.stdout:9/498: mknod d4/d63/d86/c9f 0 2024-04-25T01:36:08.049 INFO:tasks.workunit.client.1.smithi138.stdout:0/417: read d6/d2f/f5a [203971,85304] 0 2024-04-25T01:36:08.049 INFO:tasks.workunit.client.1.smithi138.stdout:1/504: mknod d3/d6d/ca2 0 2024-04-25T01:36:08.051 INFO:tasks.workunit.client.1.smithi138.stdout:9/499: mkdir d4/d1a/d68/da0 0 2024-04-25T01:36:08.052 INFO:tasks.workunit.client.1.smithi138.stdout:0/418: chown d6/d39/d43/c84 0 1 2024-04-25T01:36:08.053 INFO:tasks.workunit.client.1.smithi138.stdout:3/660: mkdir db/df/d19/dd4 0 2024-04-25T01:36:08.059 INFO:tasks.workunit.client.1.smithi138.stdout:1/505: mkdir d3/d27/d16/da3 0 2024-04-25T01:36:08.059 INFO:tasks.workunit.client.1.smithi138.stdout:5/495: dwrite d4/da/d6e/d22/f98 [0,4194304] 0 2024-04-25T01:36:08.060 INFO:tasks.workunit.client.1.smithi138.stdout:0/419: chown d6/d39/d58/c5d 608716294 1 2024-04-25T01:36:08.061 INFO:tasks.workunit.client.1.smithi138.stdout:3/661: stat db/df/d17/d20/f74 0 2024-04-25T01:36:08.065 INFO:tasks.workunit.client.1.smithi138.stdout:7/623: write da/de/d13/d30/d59/d7b/d99/fab [864051,118659] 0 2024-04-25T01:36:08.072 INFO:tasks.workunit.client.1.smithi138.stdout:7/624: write da/f18 [936708,103258] 0 2024-04-25T01:36:08.075 INFO:tasks.workunit.client.1.smithi138.stdout:3/662: dwrite db/df/d17/f9f [0,4194304] 0 2024-04-25T01:36:08.076 INFO:tasks.workunit.client.1.smithi138.stdout:1/506: mknod d3/d27/d3e/d90/ca4 0 2024-04-25T01:36:08.076 INFO:tasks.workunit.client.1.smithi138.stdout:2/697: dread d0/d85/da5/d35/f60 [0,4194304] 0 2024-04-25T01:36:08.079 INFO:tasks.workunit.client.1.smithi138.stdout:2/698: fdatasync d0/d18/d8b/fb5 0 2024-04-25T01:36:08.081 INFO:tasks.workunit.client.1.smithi138.stdout:0/420: dread d6/d19/f74 [0,4194304] 0 2024-04-25T01:36:08.082 INFO:tasks.workunit.client.1.smithi138.stdout:7/625: mknod da/de/d13/d21/d6f/d8d/ccc 0 2024-04-25T01:36:08.083 INFO:tasks.workunit.client.1.smithi138.stdout:5/496: write d4/da/d6e/d22/d74/d59/f99 [627842,6286] 0 2024-04-25T01:36:08.084 INFO:tasks.workunit.client.1.smithi138.stdout:2/699: chown d0/d85/da5/l71 171 1 2024-04-25T01:36:08.086 INFO:tasks.workunit.client.1.smithi138.stdout:0/421: dread - d6/f3b zero size 2024-04-25T01:36:08.088 INFO:tasks.workunit.client.1.smithi138.stdout:6/597: truncate fb 1443266 0 2024-04-25T01:36:08.090 INFO:tasks.workunit.client.1.smithi138.stdout:9/500: getdents d4/d49 0 2024-04-25T01:36:08.092 INFO:tasks.workunit.client.1.smithi138.stdout:1/507: dwrite d3/d6d/f7c [4194304,4194304] 0 2024-04-25T01:36:08.094 INFO:tasks.workunit.client.1.smithi138.stdout:6/598: truncate dc/d10/f51 1462514 0 2024-04-25T01:36:08.097 INFO:tasks.workunit.client.1.smithi138.stdout:0/422: fsync d6/d39/d43/f21 0 2024-04-25T01:36:08.100 INFO:tasks.workunit.client.1.smithi138.stdout:7/626: rename da/de/d1b/d49/dbb to da/de/d1b/d49/d81/d51/d6e/da2/dae/dcd 0 2024-04-25T01:36:08.102 INFO:tasks.workunit.client.1.smithi138.stdout:1/508: mknod d3/d45/d72/d57/ca5 0 2024-04-25T01:36:08.102 INFO:tasks.workunit.client.1.smithi138.stdout:2/700: dwrite d0/d85/da5/db4/f87 [0,4194304] 0 2024-04-25T01:36:08.104 INFO:tasks.workunit.client.1.smithi138.stdout:3/663: write db/df/d17/d23/f79 [3543985,8553] 0 2024-04-25T01:36:08.110 INFO:tasks.workunit.client.1.smithi138.stdout:0/423: fdatasync d6/d19/f74 0 2024-04-25T01:36:08.111 INFO:tasks.workunit.client.1.smithi138.stdout:7/627: creat da/de/d1b/d49/d81/dc7/fce x:0 0 0 2024-04-25T01:36:08.113 INFO:tasks.workunit.client.1.smithi138.stdout:3/664: rename db/d14/d7a/l96 to db/df/d17/d23/d34/ld5 0 2024-04-25T01:36:08.114 INFO:tasks.workunit.client.1.smithi138.stdout:1/509: mknod d3/d4/d53/d9d/ca6 0 2024-04-25T01:36:08.114 INFO:tasks.workunit.client.1.smithi138.stdout:9/501: symlink d4/d8/d2c/d82/la1 0 2024-04-25T01:36:08.117 INFO:tasks.workunit.client.1.smithi138.stdout:2/701: mkdir d0/d21/d45/de1 0 2024-04-25T01:36:08.118 INFO:tasks.workunit.client.1.smithi138.stdout:9/502: dread d4/f4a [0,4194304] 0 2024-04-25T01:36:08.119 INFO:tasks.workunit.client.1.smithi138.stdout:7/628: rename da/de/d1b/d49/f89 to da/de/d13/d21/d6f/fcf 0 2024-04-25T01:36:08.123 INFO:tasks.workunit.client.1.smithi138.stdout:0/424: link d6/fc d6/d39/d58/f87 0 2024-04-25T01:36:08.123 INFO:tasks.workunit.client.1.smithi138.stdout:0/425: readlink d6/d39/l40 0 2024-04-25T01:36:08.124 INFO:tasks.workunit.client.1.smithi138.stdout:0/426: chown d6/df/d69/l6f 265 1 2024-04-25T01:36:08.127 INFO:tasks.workunit.client.1.smithi138.stdout:3/665: mkdir db/df/d19/dd6 0 2024-04-25T01:36:08.128 INFO:tasks.workunit.client.1.smithi138.stdout:7/629: unlink da/de/d1b/d49/d81/d73/c85 0 2024-04-25T01:36:08.129 INFO:tasks.workunit.client.1.smithi138.stdout:6/599: write dc/d10/d28/d80/f8b [87686,2828] 0 2024-04-25T01:36:08.130 INFO:tasks.workunit.client.1.smithi138.stdout:8/635: sync 2024-04-25T01:36:08.133 INFO:tasks.workunit.client.1.smithi138.stdout:5/497: truncate d4/da/d6e/d22/d74/f5a 304120 0 2024-04-25T01:36:08.133 INFO:tasks.workunit.client.1.smithi138.stdout:5/498: fdatasync d4/f82 0 2024-04-25T01:36:08.134 INFO:tasks.workunit.client.1.smithi138.stdout:3/666: creat db/df/d17/d20/d57/daa/fd7 x:0 0 0 2024-04-25T01:36:08.136 INFO:tasks.workunit.client.1.smithi138.stdout:5/499: stat d4/da/d6e/d22/d74/d59/d83/d89 0 2024-04-25T01:36:08.137 INFO:tasks.workunit.client.1.smithi138.stdout:7/630: symlink da/de/d1b/d4b/d77/ld0 0 2024-04-25T01:36:08.145 INFO:tasks.workunit.client.1.smithi138.stdout:9/503: creat d4/d1a/fa2 x:0 0 0 2024-04-25T01:36:08.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.171+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97b8104950 con 0x7f97a006b790 2024-04-25T01:36:08.147 INFO:tasks.workunit.client.1.smithi138.stdout:3/667: creat db/df/d17/d23/d34/fd8 x:0 0 0 2024-04-25T01:36:08.147 INFO:tasks.workunit.client.1.smithi138.stdout:1/510: write d3/db/f67 [525738,108128] 0 2024-04-25T01:36:08.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.172+0000 7f979f7fe700 1 -- 172.21.15.33:0/159158699 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f97ac00f040 con 0x7f97a006b790 2024-04-25T01:36:08.148 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:36:08.148 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:36:08.148 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:36:08.148 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-25T01:36:08.148 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [], 2024-04-25T01:36:08.148 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "1/2 daemons upgraded", 2024-04-25T01:36:08.148 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "", 2024-04-25T01:36:08.148 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:36:08.148 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:36:08.149 INFO:tasks.workunit.client.1.smithi138.stdout:6/600: creat dc/d10/d28/d80/d89/da0/fbf x:0 0 0 2024-04-25T01:36:08.149 INFO:tasks.workunit.client.1.smithi138.stdout:1/511: truncate d3/d4/d47/d9b/f9c 1799435 0 2024-04-25T01:36:08.150 INFO:tasks.workunit.client.1.smithi138.stdout:6/601: chown dc/d2e/d55/l6a 2424 1 2024-04-25T01:36:08.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.175+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f97a006b790 msgr2=0x7f97a006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.175+0000 7f97bd123700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f97a006b790 0x7f97a006dc40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009b80 tx=0x7f97ac00b040 comp rx=0 tx=0).stop 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.175+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b80fc290 msgr2=0x7f97b810fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.175+0000 7f97bd123700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b80fc290 0x7f97b810fc10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f97a8009040 tx=0x7f97a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.175+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 shutdown_connections 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.175+0000 7f97bd123700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f97a006b790 0x7f97a006dc40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.175+0000 7f97bd123700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97b80fc290 0x7f97b810fc10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.175+0000 7f97bd123700 1 --2- 172.21.15.33:0/159158699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97b80fcbe0 0x7f97b8110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.175+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 >> 172.21.15.33:0/159158699 conn(0x7f97b80f6b80 msgr2=0x7f97b80fb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.176+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 shutdown_connections 2024-04-25T01:36:08.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:08.176+0000 7f97bd123700 1 -- 172.21.15.33:0/159158699 wait complete. 2024-04-25T01:36:08.152 INFO:tasks.workunit.client.1.smithi138.stdout:5/500: rename d4/da/d6e/d22/d74/d59/d7f/f97 to d4/da/d6e/d22/d74/d59/d7f/fa4 0 2024-04-25T01:36:08.153 INFO:tasks.workunit.client.1.smithi138.stdout:8/636: rmdir d3/d44/da9 0 2024-04-25T01:36:08.154 INFO:tasks.workunit.client.1.smithi138.stdout:7/631: creat da/de/d13/d21/dbc/d94/d9d/dc9/fd1 x:0 0 0 2024-04-25T01:36:08.154 INFO:tasks.workunit.client.1.smithi138.stdout:7/632: stat da/de/d13/d30/d59/c69 0 2024-04-25T01:36:08.155 INFO:tasks.workunit.client.1.smithi138.stdout:7/633: chown da/de/d13/d30/d59/c69 132430 1 2024-04-25T01:36:08.155 INFO:tasks.workunit.client.1.smithi138.stdout:1/512: mkdir d3/d4/da7 0 2024-04-25T01:36:08.157 INFO:tasks.workunit.client.1.smithi138.stdout:6/602: creat dc/d10/d67/d8e/fc0 x:0 0 0 2024-04-25T01:36:08.158 INFO:tasks.workunit.client.1.smithi138.stdout:9/504: mkdir d4/d8/d2c/da3 0 2024-04-25T01:36:08.160 INFO:tasks.workunit.client.1.smithi138.stdout:9/505: dread - d4/d1a/d55/d7f/d80/d94/f9c zero size 2024-04-25T01:36:08.161 INFO:tasks.workunit.client.1.smithi138.stdout:8/637: mkdir d3/d16/d1f/dc5 0 2024-04-25T01:36:08.161 INFO:tasks.workunit.client.1.smithi138.stdout:9/506: fdatasync d4/d8/d90/f62 0 2024-04-25T01:36:08.162 INFO:tasks.workunit.client.1.smithi138.stdout:3/668: mknod db/cd9 0 2024-04-25T01:36:08.163 INFO:tasks.workunit.client.1.smithi138.stdout:1/513: dwrite d3/d27/d3e/d51/f80 [0,4194304] 0 2024-04-25T01:36:08.163 INFO:tasks.workunit.client.1.smithi138.stdout:2/702: truncate d0/fe 1102363 0 2024-04-25T01:36:08.163 INFO:tasks.workunit.client.1.smithi138.stdout:9/507: fsync d4/d1a/d33/f37 0 2024-04-25T01:36:08.167 INFO:tasks.workunit.client.1.smithi138.stdout:9/508: truncate d4/d8/d90/f5d 1526912 0 2024-04-25T01:36:08.170 INFO:tasks.workunit.client.1.smithi138.stdout:8/638: mknod d3/d16/d1f/d2a/cc6 0 2024-04-25T01:36:08.173 INFO:tasks.workunit.client.1.smithi138.stdout:2/703: dwrite d0/d85/da5/f4b [0,4194304] 0 2024-04-25T01:36:08.176 INFO:tasks.workunit.client.1.smithi138.stdout:1/514: unlink d3/d4/d53/f6c 0 2024-04-25T01:36:08.178 INFO:tasks.workunit.client.1.smithi138.stdout:6/603: mkdir dc/d2e/dc1 0 2024-04-25T01:36:08.178 INFO:tasks.workunit.client.1.smithi138.stdout:5/501: link d4/da/d6e/d22/d35/d4d/c51 d4/da/d6e/d22/d74/d59/d83/d89/d9d/d9e/ca5 0 2024-04-25T01:36:08.180 INFO:tasks.workunit.client.1.smithi138.stdout:2/704: dread d0/d85/da5/f63 [0,4194304] 0 2024-04-25T01:36:08.180 INFO:tasks.workunit.client.1.smithi138.stdout:9/509: symlink d4/d8/d2c/d82/la4 0 2024-04-25T01:36:08.181 INFO:tasks.workunit.client.1.smithi138.stdout:3/669: fsync db/f35 0 2024-04-25T01:36:08.181 INFO:tasks.workunit.client.1.smithi138.stdout:9/510: chown d4/d8/d90/d52/l91 10 1 2024-04-25T01:36:08.183 INFO:tasks.workunit.client.1.smithi138.stdout:3/670: write db/f15 [968603,75827] 0 2024-04-25T01:36:08.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/634: rename da/de/d1b/d49/d81/d51/d6e/da2/dae to da/dd2 0 2024-04-25T01:36:08.191 INFO:tasks.workunit.client.1.smithi138.stdout:5/502: mkdir d4/da/d52/da6 0 2024-04-25T01:36:08.192 INFO:tasks.workunit.client.1.smithi138.stdout:5/503: chown d4/da/d6e/d5c/c64 660503720 1 2024-04-25T01:36:08.193 INFO:tasks.workunit.client.1.smithi138.stdout:5/504: read d4/da/d27/f1f [169879,95556] 0 2024-04-25T01:36:08.195 INFO:tasks.workunit.client.1.smithi138.stdout:5/505: dread - d4/da/d6e/d22/d74/d59/d7f/fa4 zero size 2024-04-25T01:36:08.196 INFO:tasks.workunit.client.1.smithi138.stdout:7/635: dwrite da/de/d1b/d4b/d5b/f67 [0,4194304] 0 2024-04-25T01:36:08.198 INFO:tasks.workunit.client.1.smithi138.stdout:9/511: mkdir d4/d1a/d68/da5 0 2024-04-25T01:36:08.200 INFO:tasks.workunit.client.1.smithi138.stdout:3/671: rmdir db/df/d17/d20/d57 39 2024-04-25T01:36:08.201 INFO:tasks.workunit.client.1.smithi138.stdout:3/672: write db/df/d19/d1a/d9e/fb9 [628295,44980] 0 2024-04-25T01:36:08.202 INFO:tasks.workunit.client.1.smithi138.stdout:3/673: dread - db/d14/d40/d41/fa6 zero size 2024-04-25T01:36:08.203 INFO:tasks.workunit.client.1.smithi138.stdout:8/639: rename d3/f6d to d3/d44/d48/fc7 0 2024-04-25T01:36:08.205 INFO:tasks.workunit.client.1.smithi138.stdout:6/604: creat dc/d2e/d55/d81/d91/fc2 x:0 0 0 2024-04-25T01:36:08.210 INFO:tasks.workunit.client.1.smithi138.stdout:7/636: write da/f63 [1292973,117209] 0 2024-04-25T01:36:08.210 INFO:tasks.workunit.client.1.smithi138.stdout:3/674: dwrite db/df/d17/d5d/f97 [0,4194304] 0 2024-04-25T01:36:08.212 INFO:tasks.workunit.client.1.smithi138.stdout:5/506: creat d4/da/d6e/d22/d74/d59/d83/d8a/fa7 x:0 0 0 2024-04-25T01:36:08.213 INFO:tasks.workunit.client.1.smithi138.stdout:9/512: mknod d4/d1a/d55/ca6 0 2024-04-25T01:36:08.215 INFO:tasks.workunit.client.1.smithi138.stdout:9/513: write d4/f38 [3702384,52888] 0 2024-04-25T01:36:08.215 INFO:tasks.workunit.client.1.smithi138.stdout:9/514: readlink d4/d8/l2e 0 2024-04-25T01:36:08.216 INFO:tasks.workunit.client.1.smithi138.stdout:7/637: read da/f18 [177685,37484] 0 2024-04-25T01:36:08.220 INFO:tasks.workunit.client.1.smithi138.stdout:8/640: dread d3/d16/d1f/d50/f78 [0,4194304] 0 2024-04-25T01:36:08.222 INFO:tasks.workunit.client.1.smithi138.stdout:8/641: stat d3/d16/d88/da/db/d15/d20/l68 0 2024-04-25T01:36:08.223 INFO:tasks.workunit.client.1.smithi138.stdout:8/642: fsync d3/d16/d88/da/de/f36 0 2024-04-25T01:36:08.225 INFO:tasks.workunit.client.1.smithi138.stdout:3/675: mknod db/dc5/cda 0 2024-04-25T01:36:08.226 INFO:tasks.workunit.client.1.smithi138.stdout:9/515: fsync d4/d1a/f2d 0 2024-04-25T01:36:08.230 INFO:tasks.workunit.client.1.smithi138.stdout:2/705: truncate d0/d85/da5/f43 585592 0 2024-04-25T01:36:08.235 INFO:tasks.workunit.client.1.smithi138.stdout:1/515: rename d3/d27/d2d/l41 to d3/db/la8 0 2024-04-25T01:36:08.235 INFO:tasks.workunit.client.1.smithi138.stdout:9/516: rmdir d4/d1a/d33 39 2024-04-25T01:36:08.237 INFO:tasks.workunit.client.1.smithi138.stdout:8/643: mknod d3/d44/d48/dbb/cc8 0 2024-04-25T01:36:08.237 INFO:tasks.workunit.client.1.smithi138.stdout:6/605: write dc/d2e/d55/f5a [2821773,25053] 0 2024-04-25T01:36:08.238 INFO:tasks.workunit.client.1.smithi138.stdout:2/706: rmdir d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/d98 39 2024-04-25T01:36:08.240 INFO:tasks.workunit.client.1.smithi138.stdout:1/516: write d3/d45/d72/d57/f8a [2336296,96294] 0 2024-04-25T01:36:08.242 INFO:tasks.workunit.client.1.smithi138.stdout:9/517: creat d4/d1a/d68/fa7 x:0 0 0 2024-04-25T01:36:08.242 INFO:tasks.workunit.client.1.smithi138.stdout:6/606: write dc/d10/d67/d8e/fad [212763,11445] 0 2024-04-25T01:36:08.243 INFO:tasks.workunit.client.1.smithi138.stdout:7/638: link da/f83 da/fd3 0 2024-04-25T01:36:08.244 INFO:tasks.workunit.client.1.smithi138.stdout:3/676: dread db/d14/fc8 [0,4194304] 0 2024-04-25T01:36:08.245 INFO:tasks.workunit.client.1.smithi138.stdout:8/644: unlink d3/d16/d1f/d2a/c76 0 2024-04-25T01:36:08.250 INFO:tasks.workunit.client.1.smithi138.stdout:8/645: read d3/d44/d48/fa3 [1838889,422] 0 2024-04-25T01:36:08.251 INFO:tasks.workunit.client.1.smithi138.stdout:1/517: dwrite d3/d6d/f7c [4194304,4194304] 0 2024-04-25T01:36:08.252 INFO:tasks.workunit.client.1.smithi138.stdout:8/646: chown d3/d44/d48/fa3 2 1 2024-04-25T01:36:08.253 INFO:tasks.workunit.client.1.smithi138.stdout:7/639: mknod da/de/d1b/d49/d81/dc3/cd4 0 2024-04-25T01:36:08.256 INFO:tasks.workunit.client.1.smithi138.stdout:3/677: symlink db/df/d17/d23/d88/ldb 0 2024-04-25T01:36:08.257 INFO:tasks.workunit.client.1.smithi138.stdout:6/607: mknod dc/d2e/dc1/cc3 0 2024-04-25T01:36:08.259 INFO:tasks.workunit.client.1.smithi138.stdout:1/518: mkdir d3/d27/d24/da9 0 2024-04-25T01:36:08.261 INFO:tasks.workunit.client.1.smithi138.stdout:7/640: creat da/de/d13/d30/d59/d7b/d99/fd5 x:0 0 0 2024-04-25T01:36:08.263 INFO:tasks.workunit.client.1.smithi138.stdout:7/641: read da/f83 [737727,52315] 0 2024-04-25T01:36:08.263 INFO:tasks.workunit.client.1.smithi138.stdout:2/707: write d0/d85/da5/db4/d15/f28 [321740,21894] 0 2024-04-25T01:36:08.265 INFO:tasks.workunit.client.1.smithi138.stdout:8/647: dread d3/d16/d88/da/db/f11 [4194304,4194304] 0 2024-04-25T01:36:08.265 INFO:tasks.workunit.client.1.smithi138.stdout:6/608: symlink dc/d10/d28/d70/d83/lc4 0 2024-04-25T01:36:08.268 INFO:tasks.workunit.client.1.smithi138.stdout:1/519: mkdir d3/d4/d53/d9d/daa 0 2024-04-25T01:36:08.269 INFO:tasks.workunit.client.1.smithi138.stdout:3/678: creat db/df/d19/dd4/fdc x:0 0 0 2024-04-25T01:36:08.272 INFO:tasks.workunit.client.1.smithi138.stdout:8/648: truncate d3/d16/f51 1505903 0 2024-04-25T01:36:08.274 INFO:tasks.workunit.client.1.smithi138.stdout:7/642: link da/de/d1b/d34/f47 da/de/d13/d30/d59/fd6 0 2024-04-25T01:36:08.275 INFO:tasks.workunit.client.1.smithi138.stdout:1/520: creat d3/d45/d72/d57/d8e/fab x:0 0 0 2024-04-25T01:36:08.276 INFO:tasks.workunit.client.1.smithi138.stdout:2/708: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/da2/fcd [0,4194304] 0 2024-04-25T01:36:08.279 INFO:tasks.workunit.client.1.smithi138.stdout:3/679: creat db/d14/d40/d41/fdd x:0 0 0 2024-04-25T01:36:08.280 INFO:tasks.workunit.client.1.smithi138.stdout:7/643: write da/de/d13/d30/d59/d7b/d99/fab [1611009,20109] 0 2024-04-25T01:36:08.280 INFO:tasks.workunit.client.1.smithi138.stdout:6/609: dwrite dc/d10/d28/d70/fa9 [0,4194304] 0 2024-04-25T01:36:08.281 INFO:tasks.workunit.client.1.smithi138.stdout:1/521: truncate d3/d45/fa0 422987 0 2024-04-25T01:36:08.292 INFO:tasks.workunit.client.1.smithi138.stdout:1/522: mkdir d3/d27/d16/dac 0 2024-04-25T01:36:08.292 INFO:tasks.workunit.client.1.smithi138.stdout:6/610: dwrite dc/d2e/d37/d56/f8f [0,4194304] 0 2024-04-25T01:36:08.294 INFO:tasks.workunit.client.1.smithi138.stdout:6/611: write dc/f1f [427438,85281] 0 2024-04-25T01:36:08.295 INFO:tasks.workunit.client.1.smithi138.stdout:2/709: symlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/le2 0 2024-04-25T01:36:08.295 INFO:tasks.workunit.client.1.smithi138.stdout:2/710: read - d0/d18/fd8 zero size 2024-04-25T01:36:08.297 INFO:tasks.workunit.client.1.smithi138.stdout:2/711: dread - d0/d85/da5/db4/d15/d44/d4c/d56/da2/fca zero size 2024-04-25T01:36:08.304 INFO:tasks.workunit.client.1.smithi138.stdout:3/680: getdents db 0 2024-04-25T01:36:08.305 INFO:tasks.workunit.client.1.smithi138.stdout:6/612: symlink dc/d2e/d37/lc5 0 2024-04-25T01:36:08.305 INFO:tasks.workunit.client.1.smithi138.stdout:8/649: write d3/d16/d1f/d2a/f8b [5003381,18000] 0 2024-04-25T01:36:08.307 INFO:tasks.workunit.client.1.smithi138.stdout:3/681: truncate db/d14/d40/f60 2522972 0 2024-04-25T01:36:08.309 INFO:tasks.workunit.client.1.smithi138.stdout:6/613: fdatasync dc/d2e/d37/f42 0 2024-04-25T01:36:08.309 INFO:tasks.workunit.client.1.smithi138.stdout:8/650: dread d3/d16/d88/da/db/f11 [4194304,4194304] 0 2024-04-25T01:36:08.311 INFO:tasks.workunit.client.1.smithi138.stdout:3/682: write db/d14/d40/f9c [352649,13934] 0 2024-04-25T01:36:08.312 INFO:tasks.workunit.client.1.smithi138.stdout:3/683: readlink db/df/d17/d23/d88/l8a 0 2024-04-25T01:36:08.314 INFO:tasks.workunit.client.1.smithi138.stdout:6/614: truncate dc/d2e/d37/f5f 1220603 0 2024-04-25T01:36:08.314 INFO:tasks.workunit.client.1.smithi138.stdout:7/644: write da/f18 [1009833,111409] 0 2024-04-25T01:36:08.316 INFO:tasks.workunit.client.1.smithi138.stdout:6/615: write dc/d2e/d37/d56/d8c/f99 [763466,115236] 0 2024-04-25T01:36:08.318 INFO:tasks.workunit.client.1.smithi138.stdout:6/616: dread - dc/d10/d28/d80/fa2 zero size 2024-04-25T01:36:08.324 INFO:tasks.workunit.client.1.smithi138.stdout:2/712: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/da2/f4a [0,4194304] 0 2024-04-25T01:36:08.327 INFO:tasks.workunit.client.1.smithi138.stdout:8/651: write d3/d16/f3b [2227708,114599] 0 2024-04-25T01:36:08.330 INFO:tasks.workunit.client.1.smithi138.stdout:8/652: truncate d3/d16/d1f/fba 440523 0 2024-04-25T01:36:08.332 INFO:tasks.workunit.client.1.smithi138.stdout:2/713: symlink d0/d18/d8b/le3 0 2024-04-25T01:36:08.332 INFO:tasks.workunit.client.1.smithi138.stdout:7/645: truncate da/de/d13/d30/f65 2929248 0 2024-04-25T01:36:08.336 INFO:tasks.workunit.client.1.smithi138.stdout:8/653: mkdir d3/d16/d88/da/db/dc9 0 2024-04-25T01:36:08.337 INFO:tasks.workunit.client.1.smithi138.stdout:6/617: truncate dc/d2e/d37/d56/fac 4087940 0 2024-04-25T01:36:08.341 INFO:tasks.workunit.client.1.smithi138.stdout:7/646: getdents da/de/d13/d21/dbc/d94/d9d 0 2024-04-25T01:36:08.341 INFO:tasks.workunit.client.1.smithi138.stdout:2/714: dwrite d0/d21/d45/d8a/d8e/fd6 [0,4194304] 0 2024-04-25T01:36:08.343 INFO:tasks.workunit.client.1.smithi138.stdout:7/647: write da/dd2/fb8 [204209,81475] 0 2024-04-25T01:36:08.345 INFO:tasks.workunit.client.1.smithi138.stdout:2/715: write d0/fa1 [1488755,44681] 0 2024-04-25T01:36:08.345 INFO:tasks.workunit.client.1.smithi138.stdout:7/648: read - da/de/d13/d30/d59/d7b/d99/fb1 zero size 2024-04-25T01:36:08.347 INFO:tasks.workunit.client.1.smithi138.stdout:2/716: chown d0/d85/da5/c3d 187447361 1 2024-04-25T01:36:08.349 INFO:tasks.workunit.client.1.smithi138.stdout:8/654: creat d3/d16/d88/da/db/fca x:0 0 0 2024-04-25T01:36:08.358 INFO:tasks.workunit.client.1.smithi138.stdout:2/717: rename d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fb9 to d0/d21/d45/de1/fe4 0 2024-04-25T01:36:08.361 INFO:tasks.workunit.client.1.smithi138.stdout:6/618: dwrite dc/d2e/d55/d7e/fae [4194304,4194304] 0 2024-04-25T01:36:08.364 INFO:tasks.workunit.client.1.smithi138.stdout:7/649: truncate da/de/d13/d30/d59/d7b/d99/fb0 3946599 0 2024-04-25T01:36:08.366 INFO:tasks.workunit.client.1.smithi138.stdout:2/718: symlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5/le5 0 2024-04-25T01:36:08.375 INFO:tasks.workunit.client.1.smithi138.stdout:4/446: sync 2024-04-25T01:36:08.384 INFO:tasks.workunit.client.1.smithi138.stdout:2/719: rmdir d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5 39 2024-04-25T01:36:08.407 INFO:tasks.workunit.client.1.smithi138.stdout:7/650: mkdir da/de/d13/d95/dd7 0 2024-04-25T01:36:08.410 INFO:tasks.workunit.client.1.smithi138.stdout:4/447: chown d0/l1f 60 1 2024-04-25T01:36:08.411 INFO:tasks.workunit.client.1.smithi138.stdout:7/651: dread da/de/d13/f76 [0,4194304] 0 2024-04-25T01:36:08.414 INFO:tasks.workunit.client.1.smithi138.stdout:7/652: chown da/de/d1b/d49 9004 1 2024-04-25T01:36:08.414 INFO:tasks.workunit.client.1.smithi138.stdout:8/655: dwrite d3/d16/d88/da/db/fa6 [0,4194304] 0 2024-04-25T01:36:08.416 INFO:tasks.workunit.client.1.smithi138.stdout:7/653: truncate da/de/d13/d21/f7e 4793687 0 2024-04-25T01:36:08.417 INFO:tasks.workunit.client.1.smithi138.stdout:4/448: write d0/d1c/d1e/f69 [7438433,17336] 0 2024-04-25T01:36:08.419 INFO:tasks.workunit.client.1.smithi138.stdout:4/449: chown d0/d1c/c42 12304186 1 2024-04-25T01:36:08.420 INFO:tasks.workunit.client.1.smithi138.stdout:8/656: truncate d3/d16/d1f/d87/f91 262583 0 2024-04-25T01:36:08.424 INFO:tasks.workunit.client.1.smithi138.stdout:4/450: symlink d0/d16/d4a/d74/l80 0 2024-04-25T01:36:08.428 INFO:tasks.workunit.client.1.smithi138.stdout:6/619: dwrite fb [0,4194304] 0 2024-04-25T01:36:08.435 INFO:tasks.workunit.client.1.smithi138.stdout:7/654: creat da/de/d13/d21/dbc/d94/fd8 x:0 0 0 2024-04-25T01:36:08.440 INFO:tasks.workunit.client.1.smithi138.stdout:4/451: dread d0/d1c/d1e/f69 [0,4194304] 0 2024-04-25T01:36:08.440 INFO:tasks.workunit.client.1.smithi138.stdout:7/655: creat da/de/d1b/d49/d81/d51/d6e/da2/fd9 x:0 0 0 2024-04-25T01:36:08.443 INFO:tasks.workunit.client.1.smithi138.stdout:4/452: mknod d0/d16/d33/c81 0 2024-04-25T01:36:08.444 INFO:tasks.workunit.client.1.smithi138.stdout:4/453: readlink d0/d16/l37 0 2024-04-25T01:36:08.446 INFO:tasks.workunit.client.1.smithi138.stdout:4/454: chown d0/d45/c5f 1333900065 1 2024-04-25T01:36:08.447 INFO:tasks.workunit.client.1.smithi138.stdout:4/455: chown d0/f64 44394756 1 2024-04-25T01:36:08.448 INFO:tasks.workunit.client.1.smithi138.stdout:7/656: dwrite da/de/d1b/d49/d81/d51/faa [0,4194304] 0 2024-04-25T01:36:08.449 INFO:tasks.workunit.client.1.smithi138.stdout:4/456: fdatasync d0/d16/d4a/f77 0 2024-04-25T01:36:08.450 INFO:tasks.workunit.client.1.smithi138.stdout:4/457: write d0/d1c/d52/f66 [163658,10667] 0 2024-04-25T01:36:08.453 INFO:tasks.workunit.client.1.smithi138.stdout:7/657: write da/de/d1b/d34/fc2 [3963811,93314] 0 2024-04-25T01:36:08.474 INFO:tasks.workunit.client.1.smithi138.stdout:2/720: dwrite d0/d21/f27 [0,4194304] 0 2024-04-25T01:36:08.479 INFO:tasks.workunit.client.1.smithi138.stdout:8/657: rename d3/d95 to d3/d16/d1f/d2a/d4f/dcb 0 2024-04-25T01:36:08.479 INFO:tasks.workunit.client.1.smithi138.stdout:4/458: dread d0/d1c/d48/f6f [0,4194304] 0 2024-04-25T01:36:08.479 INFO:tasks.workunit.client.1.smithi138.stdout:2/721: write d0/d21/f22 [2739934,96673] 0 2024-04-25T01:36:08.479 INFO:tasks.workunit.client.1.smithi138.stdout:6/620: rename dc/d2e/d37/d56/d8c to dc/d2e/d37/d56/d8c/dc6 22 2024-04-25T01:36:08.481 INFO:tasks.workunit.client.1.smithi138.stdout:4/459: chown d0/l5e 2046 1 2024-04-25T01:36:08.482 INFO:tasks.workunit.client.1.smithi138.stdout:2/722: write d0/d85/da5/db4/d15/d44/d4c/fbb [372515,14314] 0 2024-04-25T01:36:08.483 INFO:tasks.workunit.client.1.smithi138.stdout:8/658: symlink d3/d16/d88/da/d70/lcc 0 2024-04-25T01:36:08.485 INFO:tasks.workunit.client.1.smithi138.stdout:8/659: chown d3/d16/d88/da/f43 1 1 2024-04-25T01:36:08.488 INFO:tasks.workunit.client.1.smithi138.stdout:8/660: write d3/d16/d88/da/db/d15/d20/f26 [3788105,5690] 0 2024-04-25T01:36:08.489 INFO:tasks.workunit.client.1.smithi138.stdout:2/723: mkdir d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/de6 0 2024-04-25T01:36:08.491 INFO:tasks.workunit.client.1.smithi138.stdout:2/724: chown d0/l1c 0 1 2024-04-25T01:36:08.491 INFO:tasks.workunit.client.1.smithi138.stdout:4/460: dwrite d0/d1c/f62 [0,4194304] 0 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/233363754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:08 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:08.493 INFO:tasks.workunit.client.1.smithi138.stdout:8/661: dread - d3/d16/d88/da/db/d15/f72 zero size 2024-04-25T01:36:08.493 INFO:tasks.workunit.client.1.smithi138.stdout:4/461: chown d0/l60 5 1 2024-04-25T01:36:08.494 INFO:tasks.workunit.client.1.smithi138.stdout:4/462: stat d0/d16/d7e 0 2024-04-25T01:36:08.494 INFO:tasks.workunit.client.1.smithi138.stdout:4/463: dread - d0/f70 zero size 2024-04-25T01:36:08.496 INFO:tasks.workunit.client.1.smithi138.stdout:8/662: fsync d3/d16/d88/da/db/f11 0 2024-04-25T01:36:08.498 INFO:tasks.workunit.client.1.smithi138.stdout:8/663: chown d3/d16/d1f/d2a/d4f/dac 1 1 2024-04-25T01:36:08.498 INFO:tasks.workunit.client.1.smithi138.stdout:2/725: unlink d0/d18/d8b/ldd 0 2024-04-25T01:36:08.499 INFO:tasks.workunit.client.1.smithi138.stdout:4/464: stat d0/d45/f35 0 2024-04-25T01:36:08.500 INFO:tasks.workunit.client.1.smithi138.stdout:2/726: readlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/l88 0 2024-04-25T01:36:08.502 INFO:tasks.workunit.client.1.smithi138.stdout:2/727: rmdir d0 39 2024-04-25T01:36:08.504 INFO:tasks.workunit.client.1.smithi138.stdout:4/465: dread d0/d1c/d1e/f69 [4194304,4194304] 0 2024-04-25T01:36:08.506 INFO:tasks.workunit.client.1.smithi138.stdout:2/728: unlink d0/d85/da5/db4/d15/d51/f80 0 2024-04-25T01:36:08.507 INFO:tasks.workunit.client.1.smithi138.stdout:8/664: link d3/d16/d88/da/db/d15/d20/f6e d3/d16/d88/fcd 0 2024-04-25T01:36:08.509 INFO:tasks.workunit.client.1.smithi138.stdout:4/466: unlink d0/c14 0 2024-04-25T01:36:08.511 INFO:tasks.workunit.client.1.smithi138.stdout:4/467: symlink d0/d45/l82 0 2024-04-25T01:36:08.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:08.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='client.14628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:08.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/233363754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:36:08.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:08.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:08.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:08.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:08 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:08.519 INFO:tasks.workunit.client.1.smithi138.stdout:4/468: dwrite d0/d16/d4a/f77 [0,4194304] 0 2024-04-25T01:36:08.521 INFO:tasks.workunit.client.1.smithi138.stdout:6/621: dwrite dc/d10/d67/f77 [0,4194304] 0 2024-04-25T01:36:08.524 INFO:tasks.workunit.client.1.smithi138.stdout:8/665: dread d3/d44/d48/fc7 [0,4194304] 0 2024-04-25T01:36:08.529 INFO:tasks.workunit.client.1.smithi138.stdout:6/622: rename dc/d2e/d37/f5f to dc/d10/d28/d80/d89/fc7 0 2024-04-25T01:36:08.536 INFO:tasks.workunit.client.1.smithi138.stdout:8/666: dwrite d3/d44/d48/fc7 [0,4194304] 0 2024-04-25T01:36:08.537 INFO:tasks.workunit.client.1.smithi138.stdout:8/667: fsync d3/d16/d88/da/db/d15/fb0 0 2024-04-25T01:36:08.539 INFO:tasks.workunit.client.1.smithi138.stdout:8/668: mkdir d3/d16/d1f/d2a/d4f/dcb/dce 0 2024-04-25T01:36:08.550 INFO:tasks.workunit.client.1.smithi138.stdout:4/469: dread d0/f30 [0,4194304] 0 2024-04-25T01:36:08.551 INFO:tasks.workunit.client.1.smithi138.stdout:4/470: dread - d0/d1c/f28 zero size 2024-04-25T01:36:08.554 INFO:tasks.workunit.client.1.smithi138.stdout:6/623: rmdir dc/d10/d28/d80 39 2024-04-25T01:36:08.554 INFO:tasks.workunit.client.1.smithi138.stdout:6/624: chown dc/d10/d67/d8e/cbb 9348 1 2024-04-25T01:36:08.554 INFO:tasks.workunit.client.1.smithi138.stdout:4/471: mknod d0/d1c/d52/c83 0 2024-04-25T01:36:08.557 INFO:tasks.workunit.client.1.smithi138.stdout:4/472: mknod d0/d45/d7f/c84 0 2024-04-25T01:36:08.560 INFO:tasks.workunit.client.1.smithi138.stdout:4/473: rmdir d0/d16/d4a 39 2024-04-25T01:36:08.562 INFO:tasks.workunit.client.1.smithi138.stdout:4/474: creat d0/d1c/d1e/f85 x:0 0 0 2024-04-25T01:36:08.562 INFO:tasks.workunit.client.1.smithi138.stdout:8/669: dread d3/d16/d88/f4d [0,4194304] 0 2024-04-25T01:36:08.565 INFO:tasks.workunit.client.1.smithi138.stdout:4/475: mkdir d0/d1c/d86 0 2024-04-25T01:36:08.565 INFO:tasks.workunit.client.1.smithi138.stdout:8/670: creat d3/d44/d48/fcf x:0 0 0 2024-04-25T01:36:08.566 INFO:tasks.workunit.client.1.smithi138.stdout:6/625: dwrite dc/d2e/d37/d56/fac [0,4194304] 0 2024-04-25T01:36:08.566 INFO:tasks.workunit.client.1.smithi138.stdout:4/476: getdents d0/d1c/d86 0 2024-04-25T01:36:08.568 INFO:tasks.workunit.client.1.smithi138.stdout:4/477: write d0/f1a [1825604,57295] 0 2024-04-25T01:36:08.569 INFO:tasks.workunit.client.1.smithi138.stdout:4/478: write d0/d1c/f62 [1679726,56070] 0 2024-04-25T01:36:08.570 INFO:tasks.workunit.client.1.smithi138.stdout:8/671: fdatasync d3/d16/d1f/d2a/f6a 0 2024-04-25T01:36:08.573 INFO:tasks.workunit.client.1.smithi138.stdout:6/626: mkdir dc/d10/dc8 0 2024-04-25T01:36:08.574 INFO:tasks.workunit.client.1.smithi138.stdout:8/672: mknod d3/cd0 0 2024-04-25T01:36:08.577 INFO:tasks.workunit.client.1.smithi138.stdout:8/673: symlink d3/d16/d1f/d50/ld1 0 2024-04-25T01:36:08.581 INFO:tasks.workunit.client.1.smithi138.stdout:6/627: dwrite dc/d7b/fb0 [0,4194304] 0 2024-04-25T01:36:08.585 INFO:tasks.workunit.client.1.smithi138.stdout:6/628: mkdir dc/dc9 0 2024-04-25T01:36:08.586 INFO:tasks.workunit.client.1.smithi138.stdout:6/629: mkdir dc/d8d/dca 0 2024-04-25T01:36:08.586 INFO:tasks.workunit.client.1.smithi138.stdout:6/630: readlink dc/d2e/l59 0 2024-04-25T01:36:08.587 INFO:tasks.workunit.client.1.smithi138.stdout:6/631: write dc/d7b/fb0 [2065200,128900] 0 2024-04-25T01:36:08.607 INFO:tasks.workunit.client.1.smithi138.stdout:8/674: dread d3/d16/d88/da/db/d15/d56/f86 [0,4194304] 0 2024-04-25T01:36:08.608 INFO:tasks.workunit.client.1.smithi138.stdout:8/675: mkdir d3/d16/d1f/dd2 0 2024-04-25T01:36:08.609 INFO:tasks.workunit.client.1.smithi138.stdout:8/676: read d3/d16/d88/da/db/fa6 [2753308,35192] 0 2024-04-25T01:36:08.612 INFO:tasks.workunit.client.1.smithi138.stdout:8/677: creat d3/d16/d88/da/db/dc9/fd3 x:0 0 0 2024-04-25T01:36:08.612 INFO:tasks.workunit.client.1.smithi138.stdout:6/632: dread dc/d10/d6d/f5e [0,4194304] 0 2024-04-25T01:36:08.619 INFO:tasks.workunit.client.1.smithi138.stdout:8/678: dwrite d3/d16/d88/da/db/fca [0,4194304] 0 2024-04-25T01:36:08.620 INFO:tasks.workunit.client.1.smithi138.stdout:6/633: write dc/d2e/f4c [1087384,87288] 0 2024-04-25T01:36:08.622 INFO:tasks.workunit.client.1.smithi138.stdout:8/679: rmdir d3/d16/d1f/d2a/d4f/dac 39 2024-04-25T01:36:08.625 INFO:tasks.workunit.client.1.smithi138.stdout:6/634: creat dc/d2e/d37/d56/d8c/db7/fcb x:0 0 0 2024-04-25T01:36:08.625 INFO:tasks.workunit.client.1.smithi138.stdout:8/680: truncate d3/d16/f8e 71262 0 2024-04-25T01:36:08.626 INFO:tasks.workunit.client.1.smithi138.stdout:8/681: write d3/f3a [355372,62699] 0 2024-04-25T01:36:08.631 INFO:tasks.workunit.client.1.smithi138.stdout:6/635: dwrite dc/d2e/d55/d81/fb4 [0,4194304] 0 2024-04-25T01:36:08.632 INFO:tasks.workunit.client.1.smithi138.stdout:6/636: chown dc/fa1 224 1 2024-04-25T01:36:08.633 INFO:tasks.workunit.client.1.smithi138.stdout:6/637: stat dc/c30 0 2024-04-25T01:36:08.659 INFO:tasks.workunit.client.1.smithi138.stdout:8/682: dwrite d3/d16/d88/da/db/d15/d20/f5a [0,4194304] 0 2024-04-25T01:36:08.662 INFO:tasks.workunit.client.1.smithi138.stdout:8/683: creat d3/d44/dae/fd4 x:0 0 0 2024-04-25T01:36:08.668 INFO:tasks.workunit.client.1.smithi138.stdout:8/684: dwrite d3/d16/d88/da/db/d15/f21 [0,4194304] 0 2024-04-25T01:36:08.729 INFO:tasks.workunit.client.1.smithi138.stdout:8/685: stat d3/d16/d88/da/db/d15/d5d/f6b 0 2024-04-25T01:36:08.738 INFO:tasks.workunit.client.1.smithi138.stdout:8/686: dread d3/d16/d88/da/db/d15/d56/d8f/fc3 [0,4194304] 0 2024-04-25T01:36:08.741 INFO:tasks.workunit.client.1.smithi138.stdout:8/687: rename d3/d16/d88/d98 to d3/d16/d1f/d31/d82/dd5 0 2024-04-25T01:36:08.743 INFO:tasks.workunit.client.1.smithi138.stdout:8/688: mkdir d3/d16/d1f/dd2/dd6 0 2024-04-25T01:36:08.747 INFO:tasks.workunit.client.1.smithi138.stdout:8/689: dread d3/d16/d88/da/db/d15/d59/fb6 [0,4194304] 0 2024-04-25T01:36:08.749 INFO:tasks.workunit.client.1.smithi138.stdout:8/690: write d3/d16/d88/da/db/d15/d59/f97 [555733,51932] 0 2024-04-25T01:36:08.751 INFO:tasks.workunit.client.1.smithi138.stdout:8/691: mkdir d3/d16/d1f/dd2/dd6/dd7 0 2024-04-25T01:36:08.753 INFO:tasks.workunit.client.1.smithi138.stdout:8/692: mknod d3/d16/d1f/d31/d82/dd5/cd8 0 2024-04-25T01:36:08.754 INFO:tasks.workunit.client.1.smithi138.stdout:8/693: write d3/d16/d1f/d31/d82/f90 [677927,53066] 0 2024-04-25T01:36:08.754 INFO:tasks.workunit.client.1.smithi138.stdout:8/694: chown d3/d44/d48/dbb 9250 1 2024-04-25T01:36:08.755 INFO:tasks.workunit.client.1.smithi138.stdout:8/695: write d3/d16/d88/f40 [5739389,130030] 0 2024-04-25T01:36:08.761 INFO:tasks.workunit.client.1.smithi138.stdout:8/696: dwrite d3/d16/d1f/d31/d82/fbd [0,4194304] 0 2024-04-25T01:36:08.764 INFO:tasks.workunit.client.1.smithi138.stdout:8/697: creat d3/d16/d1f/d2a/d4f/dac/fd9 x:0 0 0 2024-04-25T01:36:08.769 INFO:tasks.workunit.client.1.smithi138.stdout:8/698: getdents d3/d16/d88/da/d70 0 2024-04-25T01:36:08.771 INFO:tasks.workunit.client.1.smithi138.stdout:8/699: symlink d3/d16/d88/da/de/lda 0 2024-04-25T01:36:08.773 INFO:tasks.workunit.client.1.smithi138.stdout:8/700: unlink d3/d16/d88/da/de/ff 0 2024-04-25T01:36:08.981 INFO:tasks.workunit.client.1.smithi138.stdout:0/427: sync 2024-04-25T01:36:08.982 INFO:tasks.workunit.client.1.smithi138.stdout:5/507: sync 2024-04-25T01:36:08.983 INFO:tasks.workunit.client.1.smithi138.stdout:0/428: mknod d6/d2f/c88 0 2024-04-25T01:36:08.983 INFO:tasks.workunit.client.1.smithi138.stdout:0/429: dread - d6/f3b zero size 2024-04-25T01:36:08.984 INFO:tasks.workunit.client.1.smithi138.stdout:5/508: truncate d4/da/d6e/d22/d74/f9c 189190 0 2024-04-25T01:36:08.986 INFO:tasks.workunit.client.1.smithi138.stdout:5/509: creat d4/da/d52/fa8 x:0 0 0 2024-04-25T01:36:08.987 INFO:tasks.workunit.client.1.smithi138.stdout:5/510: write d4/da/d6e/d22/d35/d4d/d5e/d8f/fa2 [16368,101717] 0 2024-04-25T01:36:08.988 INFO:tasks.workunit.client.1.smithi138.stdout:5/511: chown d4/da/d6e/d22/d45/f67 4 1 2024-04-25T01:36:08.995 INFO:tasks.workunit.client.1.smithi138.stdout:5/512: truncate d4/da/d6e/d22/d74/f79 4111160 0 2024-04-25T01:36:08.996 INFO:tasks.workunit.client.1.smithi138.stdout:5/513: write d4/da/d52/fa8 [768587,79995] 0 2024-04-25T01:36:08.998 INFO:tasks.workunit.client.1.smithi138.stdout:5/514: symlink d4/da/d6e/d22/d74/la9 0 2024-04-25T01:36:08.999 INFO:tasks.workunit.client.1.smithi138.stdout:0/430: dread d6/f28 [0,4194304] 0 2024-04-25T01:36:08.999 INFO:tasks.workunit.client.1.smithi138.stdout:0/431: readlink d6/df/l38 0 2024-04-25T01:36:09.001 INFO:tasks.workunit.client.1.smithi138.stdout:0/432: mkdir d6/d2f/d75/d89 0 2024-04-25T01:36:09.002 INFO:tasks.workunit.client.1.smithi138.stdout:0/433: chown d6/df/d41/d49/f63 3228886 1 2024-04-25T01:36:09.003 INFO:tasks.workunit.client.1.smithi138.stdout:0/434: read d6/df/d69/f6a [524521,130272] 0 2024-04-25T01:36:09.003 INFO:tasks.workunit.client.1.smithi138.stdout:5/515: mknod d4/da/d6e/d22/d74/d59/d83/d89/d9d/caa 0 2024-04-25T01:36:09.011 INFO:tasks.workunit.client.1.smithi138.stdout:0/435: write d6/d19/f74 [48549,67942] 0 2024-04-25T01:36:09.022 INFO:tasks.workunit.client.1.smithi138.stdout:5/516: dread d4/da/d6e/d22/f3f [0,4194304] 0 2024-04-25T01:36:09.025 INFO:tasks.workunit.client.1.smithi138.stdout:5/517: creat d4/da/d6e/d22/d35/d4d/fab x:0 0 0 2024-04-25T01:36:09.027 INFO:tasks.workunit.client.1.smithi138.stdout:5/518: dread d4/da/d6e/d22/d35/d4d/d5e/d8f/fa2 [0,4194304] 0 2024-04-25T01:36:09.028 INFO:tasks.workunit.client.1.smithi138.stdout:5/519: dread d4/da/d6e/d22/d74/f9c [0,4194304] 0 2024-04-25T01:36:09.030 INFO:tasks.workunit.client.1.smithi138.stdout:5/520: mkdir d4/da/d6e/d5c/dac 0 2024-04-25T01:36:09.031 INFO:tasks.workunit.client.1.smithi138.stdout:5/521: mkdir d4/da/d6e/d3c/dad 0 2024-04-25T01:36:09.032 INFO:tasks.workunit.client.1.smithi138.stdout:5/522: chown d4/da/d27/f1f 57 1 2024-04-25T01:36:09.041 INFO:tasks.workunit.client.1.smithi138.stdout:5/523: dwrite d4/da/d6e/d22/d74/f79 [0,4194304] 0 2024-04-25T01:36:09.043 INFO:tasks.workunit.client.1.smithi138.stdout:5/524: rmdir d4/da/d6e/d22/d35 39 2024-04-25T01:36:09.045 INFO:tasks.workunit.client.1.smithi138.stdout:5/525: write d4/da/d6e/d22/d74/f9c [162468,61460] 0 2024-04-25T01:36:09.047 INFO:tasks.workunit.client.1.smithi138.stdout:5/526: write d4/da/d6e/d22/f24 [2338151,106470] 0 2024-04-25T01:36:09.052 INFO:tasks.workunit.client.1.smithi138.stdout:5/527: rename d4/da/d52/f7c to d4/da/d6e/d22/d74/d8e/fae 0 2024-04-25T01:36:09.057 INFO:tasks.workunit.client.1.smithi138.stdout:5/528: dwrite d4/da/d6e/d22/d74/d59/f68 [0,4194304] 0 2024-04-25T01:36:09.066 INFO:tasks.workunit.client.1.smithi138.stdout:5/529: read - d4/da/d6e/d22/d74/d59/d7f/fa4 zero size 2024-04-25T01:36:09.071 INFO:tasks.workunit.client.1.smithi138.stdout:5/530: dwrite d4/da/d6e/d22/f24 [0,4194304] 0 2024-04-25T01:36:09.077 INFO:tasks.workunit.client.1.smithi138.stdout:5/531: link d4/da/d52/fa8 d4/da/d6e/d22/d35/d4d/d5e/faf 0 2024-04-25T01:36:09.078 INFO:tasks.workunit.client.1.smithi138.stdout:5/532: readlink d4/da/d6e/d22/d35/d4d/d90/l8d 0 2024-04-25T01:36:09.082 INFO:tasks.workunit.client.1.smithi138.stdout:5/533: dread d4/da/d6e/d22/d74/d59/f68 [0,4194304] 0 2024-04-25T01:36:09.083 INFO:tasks.workunit.client.1.smithi138.stdout:5/534: stat d4/da/d6e/d22/d74/d59/f68 0 2024-04-25T01:36:09.101 INFO:tasks.workunit.client.1.smithi138.stdout:9/518: sync 2024-04-25T01:36:09.218 INFO:tasks.workunit.client.1.smithi138.stdout:2/729: sync 2024-04-25T01:36:09.219 INFO:tasks.workunit.client.1.smithi138.stdout:1/523: sync 2024-04-25T01:36:09.219 INFO:tasks.workunit.client.1.smithi138.stdout:7/658: sync 2024-04-25T01:36:09.219 INFO:tasks.workunit.client.1.smithi138.stdout:3/684: sync 2024-04-25T01:36:09.219 INFO:tasks.workunit.client.1.smithi138.stdout:6/638: sync 2024-04-25T01:36:09.219 INFO:tasks.workunit.client.1.smithi138.stdout:4/479: sync 2024-04-25T01:36:09.219 INFO:tasks.workunit.client.1.smithi138.stdout:8/701: sync 2024-04-25T01:36:09.220 INFO:tasks.workunit.client.1.smithi138.stdout:8/702: readlink d3/l4 0 2024-04-25T01:36:09.221 INFO:tasks.workunit.client.1.smithi138.stdout:3/685: unlink db/df/lb6 0 2024-04-25T01:36:09.223 INFO:tasks.workunit.client.1.smithi138.stdout:4/480: rmdir d0/d1c/d1e 39 2024-04-25T01:36:09.225 INFO:tasks.workunit.client.1.smithi138.stdout:8/703: creat d3/d16/d1f/dd2/dd6/fdb x:0 0 0 2024-04-25T01:36:09.227 INFO:tasks.workunit.client.1.smithi138.stdout:6/639: rename dc/d2e/d37/d56/f9a to dc/d10/d67/d8e/daf/fcc 0 2024-04-25T01:36:09.228 INFO:tasks.workunit.client.1.smithi138.stdout:1/524: dwrite d3/d45/d72/d57/d8e/fab [0,4194304] 0 2024-04-25T01:36:09.229 INFO:tasks.workunit.client.1.smithi138.stdout:1/525: write d3/d45/d72/f95 [60732,11083] 0 2024-04-25T01:36:09.232 INFO:tasks.workunit.client.1.smithi138.stdout:3/686: unlink db/df/d17/d5d/l71 0 2024-04-25T01:36:09.236 INFO:tasks.workunit.client.1.smithi138.stdout:8/704: rmdir d3/d16/d88/da/de/d57 39 2024-04-25T01:36:09.239 INFO:tasks.workunit.client.1.smithi138.stdout:6/640: fdatasync dc/f35 0 2024-04-25T01:36:09.240 INFO:tasks.workunit.client.1.smithi138.stdout:8/705: dread d3/d16/d88/da/db/d15/f30 [0,4194304] 0 2024-04-25T01:36:09.240 INFO:tasks.workunit.client.1.smithi138.stdout:1/526: chown d3/db/la8 672167530 1 2024-04-25T01:36:09.243 INFO:tasks.workunit.client.1.smithi138.stdout:0/436: sync 2024-04-25T01:36:09.243 INFO:tasks.workunit.client.1.smithi138.stdout:5/535: sync 2024-04-25T01:36:09.244 INFO:tasks.workunit.client.1.smithi138.stdout:9/519: sync 2024-04-25T01:36:09.246 INFO:tasks.workunit.client.1.smithi138.stdout:6/641: dread dc/d10/d6d/f7c [0,4194304] 0 2024-04-25T01:36:09.248 INFO:tasks.workunit.client.1.smithi138.stdout:0/437: fsync d6/d2f/f7d 0 2024-04-25T01:36:09.249 INFO:tasks.workunit.client.1.smithi138.stdout:8/706: link d3/d16/d1f/d50/ld1 d3/d16/d88/da/db/dc9/ldc 0 2024-04-25T01:36:09.250 INFO:tasks.workunit.client.1.smithi138.stdout:1/527: dwrite d3/d45/fa0 [0,4194304] 0 2024-04-25T01:36:09.251 INFO:tasks.workunit.client.1.smithi138.stdout:6/642: write dc/d2e/d55/d81/fb4 [4547425,94769] 0 2024-04-25T01:36:09.251 INFO:tasks.workunit.client.1.smithi138.stdout:0/438: chown d6/d39/d43 1 1 2024-04-25T01:36:09.252 INFO:tasks.workunit.client.1.smithi138.stdout:5/536: symlink d4/da/d6e/d22/d35/d4d/d5e/d60/lb0 0 2024-04-25T01:36:09.255 INFO:tasks.workunit.client.1.smithi138.stdout:9/520: creat d4/d8/d2c/fa8 x:0 0 0 2024-04-25T01:36:09.256 INFO:tasks.workunit.client.1.smithi138.stdout:8/707: mknod d3/d16/d88/da/db/d15/d56/d94/cdd 0 2024-04-25T01:36:09.258 INFO:tasks.workunit.client.1.smithi138.stdout:8/708: write d3/d16/d1f/d31/d82/f90 [658080,91389] 0 2024-04-25T01:36:09.259 INFO:tasks.workunit.client.1.smithi138.stdout:6/643: mknod dc/ccd 0 2024-04-25T01:36:09.263 INFO:tasks.workunit.client.1.smithi138.stdout:6/644: chown dc/d2e/d55/d7e/fae 34 1 2024-04-25T01:36:09.264 INFO:tasks.workunit.client.1.smithi138.stdout:5/537: creat d4/da/d6e/d22/d74/d59/d83/d89/d9d/d9e/fb1 x:0 0 0 2024-04-25T01:36:09.265 INFO:tasks.workunit.client.1.smithi138.stdout:9/521: rmdir d4/d1a/d33/d1d 39 2024-04-25T01:36:09.268 INFO:tasks.workunit.client.1.smithi138.stdout:6/645: write dc/d10/d28/d80/d89/da0/fbf [671146,21604] 0 2024-04-25T01:36:09.273 INFO:tasks.workunit.client.1.smithi138.stdout:9/522: creat d4/d1a/d55/fa9 x:0 0 0 2024-04-25T01:36:09.274 INFO:tasks.workunit.client.1.smithi138.stdout:5/538: fdatasync d4/da/d6e/d22/d74/d8e/fae 0 2024-04-25T01:36:09.276 INFO:tasks.workunit.client.1.smithi138.stdout:8/709: dwrite d3/d16/d88/fc1 [0,4194304] 0 2024-04-25T01:36:09.277 INFO:tasks.workunit.client.1.smithi138.stdout:1/528: dread d3/db/fe [0,4194304] 0 2024-04-25T01:36:09.278 INFO:tasks.workunit.client.1.smithi138.stdout:7/659: truncate da/de/d1b/d4b/f55 3900474 0 2024-04-25T01:36:09.279 INFO:tasks.workunit.client.1.smithi138.stdout:9/523: write d4/d57/f5a [616324,42682] 0 2024-04-25T01:36:09.282 INFO:tasks.workunit.client.1.smithi138.stdout:6/646: getdents dc/d10/dc8 0 2024-04-25T01:36:09.288 INFO:tasks.workunit.client.1.smithi138.stdout:6/647: truncate fb 5034447 0 2024-04-25T01:36:09.288 INFO:tasks.workunit.client.1.smithi138.stdout:1/529: write d3/d4/d53/f8c [1591567,28436] 0 2024-04-25T01:36:09.294 INFO:tasks.workunit.client.1.smithi138.stdout:2/730: dwrite d0/d85/da5/f43 [0,4194304] 0 2024-04-25T01:36:09.294 INFO:tasks.workunit.client.1.smithi138.stdout:6/648: write dc/d10/d28/d80/f82 [3946847,106271] 0 2024-04-25T01:36:09.296 INFO:tasks.workunit.client.1.smithi138.stdout:8/710: dread d3/d16/d88/f40 [0,4194304] 0 2024-04-25T01:36:09.296 INFO:tasks.workunit.client.1.smithi138.stdout:4/481: dwrite d0/f13 [0,4194304] 0 2024-04-25T01:36:09.296 INFO:tasks.workunit.client.1.smithi138.stdout:1/530: symlink d3/db/lad 0 2024-04-25T01:36:09.299 INFO:tasks.workunit.client.1.smithi138.stdout:8/711: fdatasync d3/d16/d1f/f61 0 2024-04-25T01:36:09.301 INFO:tasks.workunit.client.1.smithi138.stdout:2/731: read d0/f9a [709260,95582] 0 2024-04-25T01:36:09.303 INFO:tasks.workunit.client.1.smithi138.stdout:4/482: write d0/d16/d4a/d74/f7a [132892,113777] 0 2024-04-25T01:36:09.306 INFO:tasks.workunit.client.1.smithi138.stdout:2/732: truncate d0/d21/d45/d8a/fa7 502383 0 2024-04-25T01:36:09.307 INFO:tasks.workunit.client.1.smithi138.stdout:6/649: truncate dc/d10/f22 2062493 0 2024-04-25T01:36:09.308 INFO:tasks.workunit.client.1.smithi138.stdout:0/439: dread d6/f26 [0,4194304] 0 2024-04-25T01:36:09.308 INFO:tasks.workunit.client.1.smithi138.stdout:8/712: write f0 [1001868,90125] 0 2024-04-25T01:36:09.309 INFO:tasks.workunit.client.1.smithi138.stdout:4/483: unlink d0/d59/c6e 0 2024-04-25T01:36:09.310 INFO:tasks.workunit.client.1.smithi138.stdout:6/650: write dc/f1f [1982150,46551] 0 2024-04-25T01:36:09.311 INFO:tasks.workunit.client.1.smithi138.stdout:0/440: dread - d6/df/d66/f7f zero size 2024-04-25T01:36:09.312 INFO:tasks.workunit.client.1.smithi138.stdout:4/484: write d0/f40 [1228123,5951] 0 2024-04-25T01:36:09.312 INFO:tasks.workunit.client.1.smithi138.stdout:1/531: link d3/d27/d3e/f88 d3/d27/d24/fae 0 2024-04-25T01:36:09.313 INFO:tasks.workunit.client.1.smithi138.stdout:0/441: chown d6/fc 7 1 2024-04-25T01:36:09.313 INFO:tasks.workunit.client.1.smithi138.stdout:6/651: chown dc/d10/d67/d8e/fad 0 1 2024-04-25T01:36:09.318 INFO:tasks.workunit.client.1.smithi138.stdout:2/733: write d0/d85/da5/db4/f3c [3836744,656] 0 2024-04-25T01:36:09.322 INFO:tasks.workunit.client.1.smithi138.stdout:0/442: read d6/df/d41/d49/f51 [3665242,65494] 0 2024-04-25T01:36:09.322 INFO:tasks.workunit.client.1.smithi138.stdout:1/532: truncate d3/d27/d3e/f61 615498 0 2024-04-25T01:36:09.326 INFO:tasks.workunit.client.1.smithi138.stdout:0/443: read - d6/df/d41/d49/f7c zero size 2024-04-25T01:36:09.330 INFO:tasks.workunit.client.1.smithi138.stdout:1/533: write d3/d6d/f7c [8611590,94113] 0 2024-04-25T01:36:09.332 INFO:tasks.workunit.client.1.smithi138.stdout:4/485: dread d0/d16/f43 [0,4194304] 0 2024-04-25T01:36:09.340 INFO:tasks.workunit.client.1.smithi138.stdout:2/734: dwrite d0/d85/da5/f4b [0,4194304] 0 2024-04-25T01:36:09.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/534: write d3/d27/d2d/f79 [801108,129167] 0 2024-04-25T01:36:09.341 INFO:tasks.workunit.client.1.smithi138.stdout:5/539: dwrite d4/f6c [0,4194304] 0 2024-04-25T01:36:09.348 INFO:tasks.workunit.client.1.smithi138.stdout:7/660: dwrite da/f83 [0,4194304] 0 2024-04-25T01:36:09.353 INFO:tasks.workunit.client.1.smithi138.stdout:6/652: dread dc/d10/d28/d70/d83/f88 [0,4194304] 0 2024-04-25T01:36:09.353 INFO:tasks.workunit.client.1.smithi138.stdout:6/653: stat c4 0 2024-04-25T01:36:09.355 INFO:tasks.workunit.client.1.smithi138.stdout:1/535: chown d3/d4/d47/d9b/f68 25300 1 2024-04-25T01:36:09.355 INFO:tasks.workunit.client.1.smithi138.stdout:0/444: dwrite d6/d19/f77 [0,4194304] 0 2024-04-25T01:36:09.356 INFO:tasks.workunit.client.1.smithi138.stdout:2/735: write d0/d18/d8b/fb5 [807936,58025] 0 2024-04-25T01:36:09.357 INFO:tasks.workunit.client.1.smithi138.stdout:5/540: creat d4/da/d6e/d22/d35/d4d/d5e/d8f/fb2 x:0 0 0 2024-04-25T01:36:09.360 INFO:tasks.workunit.client.1.smithi138.stdout:6/654: write dc/d2e/f6c [2692650,128724] 0 2024-04-25T01:36:09.362 INFO:tasks.workunit.client.1.smithi138.stdout:1/536: mkdir d3/d27/d16/da3/daf 0 2024-04-25T01:36:09.365 INFO:tasks.workunit.client.1.smithi138.stdout:8/713: dread d3/d16/d88/da/db/d15/d20/f26 [0,4194304] 0 2024-04-25T01:36:09.366 INFO:tasks.workunit.client.1.smithi138.stdout:7/661: creat da/de/d1b/d49/fda x:0 0 0 2024-04-25T01:36:09.367 INFO:tasks.workunit.client.1.smithi138.stdout:0/445: rename d6/d2f/d75/d89 to d6/df/d3a/d67/d8a 0 2024-04-25T01:36:09.368 INFO:tasks.workunit.client.1.smithi138.stdout:4/486: dread d0/d1c/d1e/f69 [4194304,4194304] 0 2024-04-25T01:36:09.368 INFO:tasks.workunit.client.1.smithi138.stdout:5/541: rmdir d4/da/d6e/d22/d35/d4d/d90 39 2024-04-25T01:36:09.369 INFO:tasks.workunit.client.1.smithi138.stdout:1/537: mkdir d3/d45/d72/d57/db0 0 2024-04-25T01:36:09.369 INFO:tasks.workunit.client.1.smithi138.stdout:2/736: dread - d0/d85/da5/db4/d15/d44/d4c/d56/fda zero size 2024-04-25T01:36:09.370 INFO:tasks.workunit.client.1.smithi138.stdout:6/655: truncate dc/d10/d28/d80/fa2 1037102 0 2024-04-25T01:36:09.373 INFO:tasks.workunit.client.1.smithi138.stdout:0/446: stat d6/d39/d43/f6b 0 2024-04-25T01:36:09.373 INFO:tasks.workunit.client.1.smithi138.stdout:8/714: mkdir d3/dde 0 2024-04-25T01:36:09.373 INFO:tasks.workunit.client.1.smithi138.stdout:0/447: chown d6/d1d/d82 0 1 2024-04-25T01:36:09.374 INFO:tasks.workunit.client.1.smithi138.stdout:2/737: mknod d0/d85/da5/db4/ce7 0 2024-04-25T01:36:09.374 INFO:tasks.workunit.client.1.smithi138.stdout:0/448: stat d6/d39/f70 0 2024-04-25T01:36:09.375 INFO:tasks.workunit.client.1.smithi138.stdout:1/538: mkdir d3/d4/da7/db1 0 2024-04-25T01:36:09.375 INFO:tasks.workunit.client.1.smithi138.stdout:2/738: stat d0/d21/d45/d8a/lbf 0 2024-04-25T01:36:09.376 INFO:tasks.workunit.client.1.smithi138.stdout:5/542: mkdir d4/da/d6e/d22/d74/d59/d7f/db3 0 2024-04-25T01:36:09.376 INFO:tasks.workunit.client.1.smithi138.stdout:1/539: chown d3/d4/c9a 34682313 1 2024-04-25T01:36:09.377 INFO:tasks.workunit.client.1.smithi138.stdout:1/540: stat d3/d45/f48 0 2024-04-25T01:36:09.378 INFO:tasks.workunit.client.1.smithi138.stdout:7/662: rename da/de/d1b/d4b/d5b/c58 to da/de/d1b/d49/d81/dc7/cdb 0 2024-04-25T01:36:09.381 INFO:tasks.workunit.client.1.smithi138.stdout:2/739: creat d0/d85/da5/db4/d15/d44/d4c/fe8 x:0 0 0 2024-04-25T01:36:09.383 INFO:tasks.workunit.client.1.smithi138.stdout:8/715: dwrite d3/d16/d88/da/db/d15/f21 [0,4194304] 0 2024-04-25T01:36:09.383 INFO:tasks.workunit.client.1.smithi138.stdout:1/541: rmdir d3/d4/d53 39 2024-04-25T01:36:09.385 INFO:tasks.workunit.client.1.smithi138.stdout:1/542: write d3/d27/d3e/d51/f75 [111105,84986] 0 2024-04-25T01:36:09.386 INFO:tasks.workunit.client.1.smithi138.stdout:5/543: mknod d4/da/d6e/d22/d35/d4d/cb4 0 2024-04-25T01:36:09.389 INFO:tasks.workunit.client.1.smithi138.stdout:2/740: rmdir d0/d18 39 2024-04-25T01:36:09.390 INFO:tasks.workunit.client.1.smithi138.stdout:1/543: chown d3/d27/d3e/d51/f9e 965529 1 2024-04-25T01:36:09.393 INFO:tasks.workunit.client.1.smithi138.stdout:2/741: truncate d0/d18/fd8 885617 0 2024-04-25T01:36:09.397 INFO:tasks.workunit.client.1.smithi138.stdout:2/742: read - d0/d85/da5/db4/d15/d44/d4c/d56/da2/fca zero size 2024-04-25T01:36:09.398 INFO:tasks.workunit.client.1.smithi138.stdout:5/544: mkdir d4/da/d6e/d22/d35/d4d/d5e/d60/db5 0 2024-04-25T01:36:09.399 INFO:tasks.workunit.client.1.smithi138.stdout:1/544: mkdir d3/d45/d72/db2 0 2024-04-25T01:36:09.404 INFO:tasks.workunit.client.1.smithi138.stdout:0/449: write d6/df/f4d [3898071,123918] 0 2024-04-25T01:36:09.406 INFO:tasks.workunit.client.1.smithi138.stdout:6/656: dread dc/f19 [0,4194304] 0 2024-04-25T01:36:09.406 INFO:tasks.workunit.client.1.smithi138.stdout:5/545: dwrite d4/da/d6e/d22/d74/f73 [0,4194304] 0 2024-04-25T01:36:09.406 INFO:tasks.workunit.client.1.smithi138.stdout:2/743: mkdir d0/de9 0 2024-04-25T01:36:09.409 INFO:tasks.workunit.client.1.smithi138.stdout:1/545: creat d3/d27/d16/da3/daf/fb3 x:0 0 0 2024-04-25T01:36:09.409 INFO:tasks.workunit.client.1.smithi138.stdout:8/716: write d3/d16/d88/da/db/d15/d56/f9c [3333716,24085] 0 2024-04-25T01:36:09.411 INFO:tasks.workunit.client.1.smithi138.stdout:2/744: chown d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/l88 5 1 2024-04-25T01:36:09.411 INFO:tasks.workunit.client.1.smithi138.stdout:0/450: creat d6/df/d41/d49/f8b x:0 0 0 2024-04-25T01:36:09.415 INFO:tasks.workunit.client.1.smithi138.stdout:6/657: creat dc/d7b/fce x:0 0 0 2024-04-25T01:36:09.417 INFO:tasks.workunit.client.1.smithi138.stdout:8/717: chown d3/d16/d1f/d2a/d4f/dcb/ca0 357777150 1 2024-04-25T01:36:09.417 INFO:tasks.workunit.client.1.smithi138.stdout:1/546: write d3/d4/d47/d9b/f68 [653053,120148] 0 2024-04-25T01:36:09.422 INFO:tasks.workunit.client.1.smithi138.stdout:2/745: creat d0/d85/fea x:0 0 0 2024-04-25T01:36:09.423 INFO:tasks.workunit.client.1.smithi138.stdout:6/658: symlink dc/dc9/lcf 0 2024-04-25T01:36:09.423 INFO:tasks.workunit.client.1.smithi138.stdout:7/663: dwrite da/de/d13/d30/f65 [0,4194304] 0 2024-04-25T01:36:09.424 INFO:tasks.workunit.client.1.smithi138.stdout:8/718: unlink d3/d16/d88/da/d70/fa8 0 2024-04-25T01:36:09.428 INFO:tasks.workunit.client.1.smithi138.stdout:2/746: write d0/d21/d45/d8a/fa7 [47186,43841] 0 2024-04-25T01:36:09.430 INFO:tasks.workunit.client.1.smithi138.stdout:1/547: rename d3/db/la8 to d3/d4/da7/lb4 0 2024-04-25T01:36:09.438 INFO:tasks.workunit.client.1.smithi138.stdout:5/546: dwrite d4/da/d6e/d22/d35/d4d/f66 [0,4194304] 0 2024-04-25T01:36:09.439 INFO:tasks.workunit.client.1.smithi138.stdout:1/548: unlink d3/d6d/ca2 0 2024-04-25T01:36:09.440 INFO:tasks.workunit.client.1.smithi138.stdout:8/719: mkdir d3/d16/d88/da/db/ddf 0 2024-04-25T01:36:09.453 INFO:tasks.workunit.client.1.smithi138.stdout:5/547: dwrite d4/da/d6e/d22/d74/f58 [0,4194304] 0 2024-04-25T01:36:09.454 INFO:tasks.workunit.client.1.smithi138.stdout:0/451: dread d6/df/f15 [8388608,4194304] 0 2024-04-25T01:36:09.455 INFO:tasks.workunit.client.1.smithi138.stdout:8/720: dread d3/d16/d88/da/db/d15/d56/f86 [0,4194304] 0 2024-04-25T01:36:09.457 INFO:tasks.workunit.client.1.smithi138.stdout:5/548: read - d4/da/d6e/d22/d35/d4d/fab zero size 2024-04-25T01:36:09.459 INFO:tasks.workunit.client.1.smithi138.stdout:5/549: chown d4/da/d6e/d22/d35/d4d/d5e/f7a 1 1 2024-04-25T01:36:09.461 INFO:tasks.workunit.client.1.smithi138.stdout:6/659: dread dc/f1f [0,4194304] 0 2024-04-25T01:36:09.463 INFO:tasks.workunit.client.1.smithi138.stdout:8/721: unlink d3/d16/f9b 0 2024-04-25T01:36:09.466 INFO:tasks.workunit.client.1.smithi138.stdout:2/747: truncate d0/d21/f22 679400 0 2024-04-25T01:36:09.467 INFO:tasks.workunit.client.1.smithi138.stdout:1/549: write d3/d45/f48 [4300543,34108] 0 2024-04-25T01:36:09.470 INFO:tasks.workunit.client.1.smithi138.stdout:6/660: mknod dc/d2e/db9/cd0 0 2024-04-25T01:36:09.472 INFO:tasks.workunit.client.1.smithi138.stdout:8/722: unlink d3/d16/d1f/d87/l74 0 2024-04-25T01:36:09.473 INFO:tasks.workunit.client.1.smithi138.stdout:7/664: dread f9 [4194304,4194304] 0 2024-04-25T01:36:09.475 INFO:tasks.workunit.client.1.smithi138.stdout:2/748: creat d0/d21/d45/d8a/feb x:0 0 0 2024-04-25T01:36:09.475 INFO:tasks.workunit.client.1.smithi138.stdout:5/550: dread d4/da/d6e/d22/d74/d59/f68 [0,4194304] 0 2024-04-25T01:36:09.476 INFO:tasks.workunit.client.1.smithi138.stdout:0/452: dread d6/d2f/f5a [0,4194304] 0 2024-04-25T01:36:09.480 INFO:tasks.workunit.client.1.smithi138.stdout:1/550: creat d3/d27/d3e/fb5 x:0 0 0 2024-04-25T01:36:09.481 INFO:tasks.workunit.client.1.smithi138.stdout:0/453: mknod d6/df/d3a/d67/c8c 0 2024-04-25T01:36:09.485 INFO:tasks.workunit.client.1.smithi138.stdout:8/723: getdents d3/d16/d1f/dc5 0 2024-04-25T01:36:09.487 INFO:tasks.workunit.client.1.smithi138.stdout:1/551: write d3/d27/d3e/d90/f78 [342255,62929] 0 2024-04-25T01:36:09.487 INFO:tasks.workunit.client.1.smithi138.stdout:5/551: mkdir d4/da/d6e/d5c/dac/db6 0 2024-04-25T01:36:09.488 INFO:tasks.workunit.client.1.smithi138.stdout:7/665: dwrite da/f7a [0,4194304] 0 2024-04-25T01:36:09.489 INFO:tasks.workunit.client.1.smithi138.stdout:6/661: dwrite dc/d10/d67/d8e/daf/fcc [0,4194304] 0 2024-04-25T01:36:09.494 INFO:tasks.workunit.client.1.smithi138.stdout:6/662: read - dc/d7b/fce zero size 2024-04-25T01:36:09.498 INFO:tasks.workunit.client.1.smithi138.stdout:2/749: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/da2/f37 [0,4194304] 0 2024-04-25T01:36:09.501 INFO:tasks.workunit.client.1.smithi138.stdout:5/552: dread d4/da/d6e/d22/d35/d4d/d5e/d8f/fa2 [0,4194304] 0 2024-04-25T01:36:09.505 INFO:tasks.workunit.client.1.smithi138.stdout:1/552: mkdir d3/d45/d72/d57/d8e/db6 0 2024-04-25T01:36:09.505 INFO:tasks.workunit.client.1.smithi138.stdout:2/750: write d0/d85/da5/db4/d15/f28 [1524938,81262] 0 2024-04-25T01:36:09.505 INFO:tasks.workunit.client.1.smithi138.stdout:7/666: symlink da/de/d1b/d49/d81/d51/d6e/ldc 0 2024-04-25T01:36:09.506 INFO:tasks.workunit.client.1.smithi138.stdout:0/454: dread d6/d39/d43/f21 [0,4194304] 0 2024-04-25T01:36:09.506 INFO:tasks.workunit.client.1.smithi138.stdout:5/553: read d4/da/d6e/d22/d45/f54 [129732,48950] 0 2024-04-25T01:36:09.508 INFO:tasks.workunit.client.1.smithi138.stdout:7/667: chown da/de/d13/d21/d6f 285 1 2024-04-25T01:36:09.509 INFO:tasks.workunit.client.1.smithi138.stdout:6/663: symlink dc/d8d/dca/ld1 0 2024-04-25T01:36:09.512 INFO:tasks.workunit.client.1.smithi138.stdout:1/553: mknod d3/d45/cb7 0 2024-04-25T01:36:09.513 INFO:tasks.workunit.client.1.smithi138.stdout:0/455: symlink d6/d19/l8d 0 2024-04-25T01:36:09.515 INFO:tasks.workunit.client.1.smithi138.stdout:5/554: rename d4/l8 to d4/da/d52/lb7 0 2024-04-25T01:36:09.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:09 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi033 to 1559M 2024-04-25T01:36:09.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:09 smithi138 ceph-mon[32977]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:09.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:09 smithi138 ceph-mon[32977]: pgmap v20: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 86 MiB/s rd, 255 MiB/s wr, 496 op/s 2024-04-25T01:36:09.517 INFO:tasks.workunit.client.1.smithi138.stdout:7/668: creat da/de/d13/d30/d59/d7b/fdd x:0 0 0 2024-04-25T01:36:09.518 INFO:tasks.workunit.client.1.smithi138.stdout:1/554: mknod d3/d4/d47/cb8 0 2024-04-25T01:36:09.520 INFO:tasks.workunit.client.1.smithi138.stdout:6/664: dwrite dc/d2e/db9/fbc [0,4194304] 0 2024-04-25T01:36:09.521 INFO:tasks.workunit.client.1.smithi138.stdout:7/669: chown da/de/d13/d30/f42 22 1 2024-04-25T01:36:09.522 INFO:tasks.workunit.client.1.smithi138.stdout:5/555: symlink d4/da/d6e/d22/d35/d4d/d5e/d8f/lb8 0 2024-04-25T01:36:09.522 INFO:tasks.workunit.client.1.smithi138.stdout:8/724: link d3/c2d d3/d16/d88/da/db/d15/ce0 0 2024-04-25T01:36:09.531 INFO:tasks.workunit.client.1.smithi138.stdout:6/665: unlink dc/d2e/d37/d56/f8f 0 2024-04-25T01:36:09.533 INFO:tasks.workunit.client.1.smithi138.stdout:7/670: dwrite da/de/d13/f2d [0,4194304] 0 2024-04-25T01:36:09.533 INFO:tasks.workunit.client.1.smithi138.stdout:5/556: chown d4/da/d6e/d3c/l4f 1941 1 2024-04-25T01:36:09.535 INFO:tasks.workunit.client.1.smithi138.stdout:2/751: chown d0/d21/f22 0 1 2024-04-25T01:36:09.535 INFO:tasks.workunit.client.1.smithi138.stdout:5/557: rename d4/da to d4/da/d6e/d5c/db9 22 2024-04-25T01:36:09.538 INFO:tasks.workunit.client.1.smithi138.stdout:7/671: write da/de/d13/d21/dbc/fb3 [268049,59090] 0 2024-04-25T01:36:09.542 INFO:tasks.workunit.client.1.smithi138.stdout:5/558: read d4/da/d6e/d22/d35/d4d/d5e/d8f/fa2 [90132,2148] 0 2024-04-25T01:36:09.543 INFO:tasks.workunit.client.1.smithi138.stdout:6/666: creat dc/d10/d28/d80/d89/fd2 x:0 0 0 2024-04-25T01:36:09.543 INFO:tasks.workunit.client.1.smithi138.stdout:8/725: link d3/cd0 d3/dde/ce1 0 2024-04-25T01:36:09.544 INFO:tasks.workunit.client.1.smithi138.stdout:0/456: dread d6/df/d3a/f71 [0,4194304] 0 2024-04-25T01:36:09.545 INFO:tasks.workunit.client.1.smithi138.stdout:1/555: creat d3/db/fb9 x:0 0 0 2024-04-25T01:36:09.546 INFO:tasks.workunit.client.1.smithi138.stdout:8/726: fdatasync d3/d16/d1f/dd2/dd6/fdb 0 2024-04-25T01:36:09.547 INFO:tasks.workunit.client.1.smithi138.stdout:2/752: symlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/lec 0 2024-04-25T01:36:09.552 INFO:tasks.workunit.client.1.smithi138.stdout:8/727: chown d3/d16/d1f/d31/d82/dd5/cd8 593263979 1 2024-04-25T01:36:09.552 INFO:tasks.workunit.client.1.smithi138.stdout:0/457: symlink d6/d1d/l8e 0 2024-04-25T01:36:09.553 INFO:tasks.workunit.client.1.smithi138.stdout:1/556: mknod d3/d27/d16/cba 0 2024-04-25T01:36:09.555 INFO:tasks.workunit.client.1.smithi138.stdout:8/728: readlink d3/d44/d48/l58 0 2024-04-25T01:36:09.555 INFO:tasks.workunit.client.1.smithi138.stdout:2/753: write d0/f10 [1610053,101494] 0 2024-04-25T01:36:09.557 INFO:tasks.workunit.client.1.smithi138.stdout:5/559: dwrite d4/da/d6e/d22/f98 [4194304,4194304] 0 2024-04-25T01:36:09.558 INFO:tasks.workunit.client.1.smithi138.stdout:7/672: getdents da/de/d1b/d49/d81/dc3 0 2024-04-25T01:36:09.560 INFO:tasks.workunit.client.1.smithi138.stdout:0/458: mknod d6/d2f/c8f 0 2024-04-25T01:36:09.563 INFO:tasks.workunit.client.1.smithi138.stdout:6/667: dwrite dc/d10/d67/f77 [0,4194304] 0 2024-04-25T01:36:09.563 INFO:tasks.workunit.client.1.smithi138.stdout:2/754: fdatasync d0/d85/da5/db4/f9d 0 2024-04-25T01:36:09.564 INFO:tasks.workunit.client.1.smithi138.stdout:0/459: dread - d6/d19/f85 zero size 2024-04-25T01:36:09.567 INFO:tasks.workunit.client.1.smithi138.stdout:5/560: chown d4/da/d6e/d5c/f95 1 1 2024-04-25T01:36:09.573 INFO:tasks.workunit.client.1.smithi138.stdout:0/460: creat d6/d39/d43/f90 x:0 0 0 2024-04-25T01:36:09.576 INFO:tasks.workunit.client.1.smithi138.stdout:1/557: rename d3/d6d/f7c to d3/d45/d72/db2/fbb 0 2024-04-25T01:36:09.576 INFO:tasks.workunit.client.1.smithi138.stdout:8/729: read d3/d16/d88/da/db/d15/d20/f6e [2569938,98466] 0 2024-04-25T01:36:09.577 INFO:tasks.workunit.client.1.smithi138.stdout:1/558: dread - d3/d4/d47/d9b/f98 zero size 2024-04-25T01:36:09.578 INFO:tasks.workunit.client.1.smithi138.stdout:5/561: truncate d4/da/d6e/d22/d74/d59/d83/d89/d9d/d9e/fb1 676588 0 2024-04-25T01:36:09.579 INFO:tasks.workunit.client.1.smithi138.stdout:7/673: dwrite da/f83 [0,4194304] 0 2024-04-25T01:36:09.579 INFO:tasks.workunit.client.1.smithi138.stdout:5/562: stat d4/da/fe 0 2024-04-25T01:36:09.585 INFO:tasks.workunit.client.1.smithi138.stdout:8/730: mkdir d3/d16/d1f/de2 0 2024-04-25T01:36:09.586 INFO:tasks.workunit.client.1.smithi138.stdout:1/559: mknod d3/d45/d72/db2/cbc 0 2024-04-25T01:36:09.588 INFO:tasks.workunit.client.1.smithi138.stdout:7/674: mknod da/de/d1b/d49/cde 0 2024-04-25T01:36:09.588 INFO:tasks.workunit.client.1.smithi138.stdout:1/560: write d3/d4/d47/d9b/f9c [798033,18550] 0 2024-04-25T01:36:09.590 INFO:tasks.workunit.client.1.smithi138.stdout:6/668: link dc/d10/d67/d8e/cbb dc/d10/d67/d8e/daf/cd3 0 2024-04-25T01:36:09.591 INFO:tasks.workunit.client.1.smithi138.stdout:2/755: getdents d0/d85/da5 0 2024-04-25T01:36:09.592 INFO:tasks.workunit.client.1.smithi138.stdout:8/731: fdatasync d3/d16/d88/da/db/f96 0 2024-04-25T01:36:09.595 INFO:tasks.workunit.client.1.smithi138.stdout:8/732: dread - d3/d16/d1f/d2a/d4f/dac/fd9 zero size 2024-04-25T01:36:09.595 INFO:tasks.workunit.client.1.smithi138.stdout:5/563: mkdir d4/da/d6e/d22/d35/d4d/d90/dba 0 2024-04-25T01:36:09.599 INFO:tasks.workunit.client.1.smithi138.stdout:6/669: dread dc/d8d/f97 [0,4194304] 0 2024-04-25T01:36:09.599 INFO:tasks.workunit.client.1.smithi138.stdout:5/564: chown d4/da/d6e/d22/d35/d4d/d90/dba 1537059 1 2024-04-25T01:36:09.600 INFO:tasks.workunit.client.1.smithi138.stdout:8/733: fdatasync d3/d16/d88/fcd 0 2024-04-25T01:36:09.601 INFO:tasks.workunit.client.1.smithi138.stdout:6/670: read - dc/d10/fa5 zero size 2024-04-25T01:36:09.604 INFO:tasks.workunit.client.1.smithi138.stdout:7/675: link da/de/d13/d21/dbc/fb3 da/de/d13/d21/fdf 0 2024-04-25T01:36:09.605 INFO:tasks.workunit.client.1.smithi138.stdout:2/756: dwrite d0/d85/da5/d35/f69 [4194304,4194304] 0 2024-04-25T01:36:09.607 INFO:tasks.workunit.client.1.smithi138.stdout:5/565: write d4/da/d6e/d22/d74/d59/d83/d89/d9d/d9e/fb1 [241978,4411] 0 2024-04-25T01:36:09.611 INFO:tasks.workunit.client.1.smithi138.stdout:3/687: sync 2024-04-25T01:36:09.613 INFO:tasks.workunit.client.1.smithi138.stdout:5/566: read d4/da/d6e/d5c/f95 [114234,1766] 0 2024-04-25T01:36:09.615 INFO:tasks.workunit.client.1.smithi138.stdout:1/561: link d3/d4/c3c d3/d4/da7/db1/cbd 0 2024-04-25T01:36:09.617 INFO:tasks.workunit.client.1.smithi138.stdout:2/757: creat d0/d18/d8b/fed x:0 0 0 2024-04-25T01:36:09.618 INFO:tasks.workunit.client.1.smithi138.stdout:1/562: fdatasync d3/d45/d72/f52 0 2024-04-25T01:36:09.620 INFO:tasks.workunit.client.1.smithi138.stdout:6/671: dwrite dc/d8d/f9b [0,4194304] 0 2024-04-25T01:36:09.621 INFO:tasks.workunit.client.1.smithi138.stdout:8/734: dwrite d3/d16/d88/fab [0,4194304] 0 2024-04-25T01:36:09.621 INFO:tasks.workunit.client.1.smithi138.stdout:7/676: rename da/de/d13/d21/fdf to da/dd2/fe0 0 2024-04-25T01:36:09.628 INFO:tasks.workunit.client.1.smithi138.stdout:5/567: dread d4/da/d6e/d22/d74/f79 [0,4194304] 0 2024-04-25T01:36:09.630 INFO:tasks.workunit.client.1.smithi138.stdout:1/563: write d3/d45/d72/d57/f8a [3872136,7951] 0 2024-04-25T01:36:09.630 INFO:tasks.workunit.client.1.smithi138.stdout:3/688: getdents db/df/dd1 0 2024-04-25T01:36:09.631 INFO:tasks.workunit.client.1.smithi138.stdout:8/735: creat d3/d16/d1f/d31/d82/dd5/fe3 x:0 0 0 2024-04-25T01:36:09.632 INFO:tasks.workunit.client.1.smithi138.stdout:5/568: write d4/da/d6e/d22/d74/f85 [1966553,30407] 0 2024-04-25T01:36:09.634 INFO:tasks.workunit.client.1.smithi138.stdout:8/736: chown d3/d16/d88/da/db/d15/d56/f69 31 1 2024-04-25T01:36:09.636 INFO:tasks.workunit.client.1.smithi138.stdout:6/672: rename dc/d10/f51 to dc/d2e/d55/fd4 0 2024-04-25T01:36:09.640 INFO:tasks.workunit.client.1.smithi138.stdout:1/564: read d3/d4/d47/d9b/f68 [3708225,28012] 0 2024-04-25T01:36:09.642 INFO:tasks.workunit.client.1.smithi138.stdout:8/737: mknod d3/d16/d1f/dd2/dd6/ce4 0 2024-04-25T01:36:09.645 INFO:tasks.workunit.client.1.smithi138.stdout:8/738: chown d3/d16/d1f/d2a/d4f/dac/fd9 102 1 2024-04-25T01:36:09.646 INFO:tasks.workunit.client.1.smithi138.stdout:1/565: creat d3/d45/d72/d57/d8e/fbe x:0 0 0 2024-04-25T01:36:09.647 INFO:tasks.workunit.client.1.smithi138.stdout:5/569: dwrite d4/da/d6e/d22/f24 [0,4194304] 0 2024-04-25T01:36:09.648 INFO:tasks.workunit.client.1.smithi138.stdout:5/570: chown d4/da/f47 417 1 2024-04-25T01:36:09.653 INFO:tasks.workunit.client.1.smithi138.stdout:1/566: rmdir d3/d6d 39 2024-04-25T01:36:09.654 INFO:tasks.workunit.client.1.smithi138.stdout:1/567: read d3/d45/fa0 [3053190,78638] 0 2024-04-25T01:36:09.656 INFO:tasks.workunit.client.1.smithi138.stdout:2/758: rename d0/d18 to d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee 0 2024-04-25T01:36:09.661 INFO:tasks.workunit.client.1.smithi138.stdout:1/568: chown d3/d4/d53/d9d/daa 51077 1 2024-04-25T01:36:09.664 INFO:tasks.workunit.client.1.smithi138.stdout:5/571: mkdir d4/da/d6e/d22/d88/dbb 0 2024-04-25T01:36:09.665 INFO:tasks.workunit.client.1.smithi138.stdout:2/759: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/da2/fcd [0,4194304] 0 2024-04-25T01:36:09.666 INFO:tasks.workunit.client.1.smithi138.stdout:5/572: write d4/da/fa0 [482114,56979] 0 2024-04-25T01:36:09.674 INFO:tasks.workunit.client.1.smithi138.stdout:1/569: unlink d3/d45/d72/d3d/f82 0 2024-04-25T01:36:09.675 INFO:tasks.workunit.client.1.smithi138.stdout:1/570: chown d3/f44 2115 1 2024-04-25T01:36:09.676 INFO:tasks.workunit.client.1.smithi138.stdout:6/673: dread dc/d2e/d55/f5a [0,4194304] 0 2024-04-25T01:36:09.678 INFO:tasks.workunit.client.1.smithi138.stdout:8/739: link d3/d16/d88/da/db/d15/d5d/c79 d3/d16/d88/da/db/d15/d20/ce5 0 2024-04-25T01:36:09.681 INFO:tasks.workunit.client.1.smithi138.stdout:5/573: dwrite d4/f62 [0,4194304] 0 2024-04-25T01:36:09.682 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:09 smithi033 ceph-mon[26353]: Adjusting osd_memory_target on smithi033 to 1559M 2024-04-25T01:36:09.682 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:09 smithi033 ceph-mon[26353]: from='client.24445 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:09.682 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:09 smithi033 ceph-mon[26353]: pgmap v20: 65 pgs: 65 active+clean; 3.0 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 86 MiB/s rd, 255 MiB/s wr, 496 op/s 2024-04-25T01:36:09.682 INFO:tasks.workunit.client.1.smithi138.stdout:2/760: dwrite d0/f31 [0,4194304] 0 2024-04-25T01:36:09.682 INFO:tasks.workunit.client.1.smithi138.stdout:8/740: rmdir d3/d44/da4 39 2024-04-25T01:36:09.689 INFO:tasks.workunit.client.1.smithi138.stdout:1/571: link d3/f2c d3/d27/d3e/fbf 0 2024-04-25T01:36:09.690 INFO:tasks.workunit.client.1.smithi138.stdout:6/674: truncate dc/d10/f6b 3539311 0 2024-04-25T01:36:09.695 INFO:tasks.workunit.client.1.smithi138.stdout:1/572: dread d3/d27/d16/f28 [0,4194304] 0 2024-04-25T01:36:09.700 INFO:tasks.workunit.client.1.smithi138.stdout:2/761: dread d0/f9 [0,4194304] 0 2024-04-25T01:36:09.702 INFO:tasks.workunit.client.1.smithi138.stdout:5/574: rmdir d4/da/d6e/d5c/dac/db6 0 2024-04-25T01:36:09.703 INFO:tasks.workunit.client.1.smithi138.stdout:5/575: read d4/da/f20 [988752,116038] 0 2024-04-25T01:36:09.704 INFO:tasks.workunit.client.1.smithi138.stdout:1/573: creat d3/d45/d72/d57/fc0 x:0 0 0 2024-04-25T01:36:09.710 INFO:tasks.workunit.client.1.smithi138.stdout:2/762: creat d0/d85/da5/db4/d15/d44/d4c/d56/da2/fef x:0 0 0 2024-04-25T01:36:09.712 INFO:tasks.workunit.client.1.smithi138.stdout:5/576: dwrite d4/da/d6e/d22/d74/f76 [0,4194304] 0 2024-04-25T01:36:09.713 INFO:tasks.workunit.client.1.smithi138.stdout:2/763: mkdir d0/de9/df0 0 2024-04-25T01:36:09.714 INFO:tasks.workunit.client.1.smithi138.stdout:6/675: dread dc/d2e/d55/d7e/fae [0,4194304] 0 2024-04-25T01:36:09.716 INFO:tasks.workunit.client.1.smithi138.stdout:5/577: symlink d4/da/lbc 0 2024-04-25T01:36:09.718 INFO:tasks.workunit.client.1.smithi138.stdout:2/764: creat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/ff1 x:0 0 0 2024-04-25T01:36:09.719 INFO:tasks.workunit.client.1.smithi138.stdout:5/578: mkdir d4/da/d6e/d22/d35/d4d/d5e/d60/dbd 0 2024-04-25T01:36:09.720 INFO:tasks.workunit.client.1.smithi138.stdout:2/765: mkdir d0/d85/da5/db4/d15/d51/df2 0 2024-04-25T01:36:09.721 INFO:tasks.workunit.client.1.smithi138.stdout:2/766: read - d0/d85/da5/f81 zero size 2024-04-25T01:36:09.728 INFO:tasks.workunit.client.1.smithi138.stdout:5/579: dwrite d4/f6c [0,4194304] 0 2024-04-25T01:36:09.731 INFO:tasks.workunit.client.1.smithi138.stdout:8/741: truncate d3/d16/d88/da/f43 4247614 0 2024-04-25T01:36:09.733 INFO:tasks.workunit.client.1.smithi138.stdout:8/742: stat d3/d16/d88/da/d70/l7d 0 2024-04-25T01:36:09.735 INFO:tasks.workunit.client.1.smithi138.stdout:5/580: mknod d4/da/d6e/d3c/dad/cbe 0 2024-04-25T01:36:09.736 INFO:tasks.workunit.client.1.smithi138.stdout:2/767: dwrite d0/f9a [0,4194304] 0 2024-04-25T01:36:09.741 INFO:tasks.workunit.client.1.smithi138.stdout:8/743: rmdir d3/d16/d1f/d2a 39 2024-04-25T01:36:09.742 INFO:tasks.workunit.client.1.smithi138.stdout:8/744: write f0 [1527544,128540] 0 2024-04-25T01:36:09.743 INFO:tasks.workunit.client.1.smithi138.stdout:5/581: symlink d4/da/d52/lbf 0 2024-04-25T01:36:09.746 INFO:tasks.workunit.client.1.smithi138.stdout:8/745: chown d3/d16/d88/da/db/d15/d5d/f6b 5036 1 2024-04-25T01:36:09.749 INFO:tasks.workunit.client.1.smithi138.stdout:2/768: getdents d0/d21/d45 0 2024-04-25T01:36:09.753 INFO:tasks.workunit.client.1.smithi138.stdout:2/769: mknod d0/de9/cf3 0 2024-04-25T01:36:09.761 INFO:tasks.workunit.client.1.smithi138.stdout:2/770: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b/fa4 [0,4194304] 0 2024-04-25T01:36:09.764 INFO:tasks.workunit.client.1.smithi138.stdout:5/582: truncate d4/da/d6e/d22/f98 1155174 0 2024-04-25T01:36:09.765 INFO:tasks.workunit.client.1.smithi138.stdout:2/771: creat d0/d21/d45/ff4 x:0 0 0 2024-04-25T01:36:09.765 INFO:tasks.workunit.client.1.smithi138.stdout:8/746: write d3/d16/d88/da/db/d15/d56/f69 [1387726,90388] 0 2024-04-25T01:36:09.767 INFO:tasks.workunit.client.1.smithi138.stdout:5/583: creat d4/da/d6e/d22/d74/d59/d83/d8a/fc0 x:0 0 0 2024-04-25T01:36:09.771 INFO:tasks.workunit.client.1.smithi138.stdout:8/747: creat d3/d16/d88/da/db/d15/d56/d8f/fe6 x:0 0 0 2024-04-25T01:36:09.772 INFO:tasks.workunit.client.1.smithi138.stdout:8/748: chown d3/d16/d88/da/d70/l7d 432674879 1 2024-04-25T01:36:09.772 INFO:tasks.workunit.client.1.smithi138.stdout:5/584: creat d4/fc1 x:0 0 0 2024-04-25T01:36:09.774 INFO:tasks.workunit.client.1.smithi138.stdout:8/749: truncate d3/d16/d1f/d31/d82/f90 1231452 0 2024-04-25T01:36:09.776 INFO:tasks.workunit.client.1.smithi138.stdout:5/585: unlink d4/da/d6e/d22/f93 0 2024-04-25T01:36:09.779 INFO:tasks.workunit.client.1.smithi138.stdout:5/586: mkdir d4/da/d6e/d22/d74/d59/d83/d8a/dc2 0 2024-04-25T01:36:09.780 INFO:tasks.workunit.client.1.smithi138.stdout:8/750: creat d3/d16/d1f/d2a/d4f/dcb/dce/fe7 x:0 0 0 2024-04-25T01:36:09.781 INFO:tasks.workunit.client.1.smithi138.stdout:8/751: readlink d3/l17 0 2024-04-25T01:36:09.783 INFO:tasks.workunit.client.1.smithi138.stdout:5/587: rename d4/da/d6e/d22/d74/d8e/l94 to d4/da/d6e/d22/d45/lc3 0 2024-04-25T01:36:09.786 INFO:tasks.workunit.client.1.smithi138.stdout:5/588: mkdir d4/da/d6e/d22/d35/d4d/d5e/d60/dc4 0 2024-04-25T01:36:09.794 INFO:tasks.workunit.client.1.smithi138.stdout:5/589: rename d4/da/d6e/d22/d74/c72 to d4/da/d6e/d22/d35/d4d/d90/dba/cc5 0 2024-04-25T01:36:09.797 INFO:tasks.workunit.client.1.smithi138.stdout:2/772: dread - d0/d21/d45/d8a/fc9 zero size 2024-04-25T01:36:09.800 INFO:tasks.workunit.client.1.smithi138.stdout:5/590: creat d4/da/d6e/d22/d35/d4d/d90/fc6 x:0 0 0 2024-04-25T01:36:09.800 INFO:tasks.workunit.client.1.smithi138.stdout:5/591: chown d4/da 49859 1 2024-04-25T01:36:09.801 INFO:tasks.workunit.client.1.smithi138.stdout:5/592: chown d4/da/d6e/d22/d45/l57 39 1 2024-04-25T01:36:09.803 INFO:tasks.workunit.client.1.smithi138.stdout:2/773: unlink d0/d85/da5/db4/d15/d51/l7a 0 2024-04-25T01:36:09.804 INFO:tasks.workunit.client.1.smithi138.stdout:8/752: link d3/d16/d88/da/f43 d3/d16/d88/da/de/d57/fe8 0 2024-04-25T01:36:09.806 INFO:tasks.workunit.client.1.smithi138.stdout:8/753: fsync d3/d16/d1f/d31/d82/f90 0 2024-04-25T01:36:09.809 INFO:tasks.workunit.client.1.smithi138.stdout:8/754: readlink d3/d16/d88/da/d70/lcc 0 2024-04-25T01:36:09.809 INFO:tasks.workunit.client.1.smithi138.stdout:5/593: dwrite d4/da/d6e/d22/d74/f85 [0,4194304] 0 2024-04-25T01:36:09.811 INFO:tasks.workunit.client.1.smithi138.stdout:2/774: dread d0/d85/da5/db4/d15/d44/d4c/d56/da2/fcd [0,4194304] 0 2024-04-25T01:36:09.812 INFO:tasks.workunit.client.1.smithi138.stdout:8/755: fsync d3/f1e 0 2024-04-25T01:36:09.813 INFO:tasks.workunit.client.1.smithi138.stdout:2/775: write d0/d85/da5/db4/d15/d51/f72 [2412614,58586] 0 2024-04-25T01:36:09.814 INFO:tasks.workunit.client.1.smithi138.stdout:8/756: read d3/f1e [336203,64499] 0 2024-04-25T01:36:09.818 INFO:tasks.workunit.client.1.smithi138.stdout:2/776: mkdir d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5/df5 0 2024-04-25T01:36:09.820 INFO:tasks.workunit.client.1.smithi138.stdout:2/777: chown d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/d98 12254 1 2024-04-25T01:36:09.824 INFO:tasks.workunit.client.1.smithi138.stdout:2/778: dread d0/d85/da5/d35/f70 [8388608,4194304] 0 2024-04-25T01:36:09.825 INFO:tasks.workunit.client.1.smithi138.stdout:2/779: write d0/f9a [4917796,15585] 0 2024-04-25T01:36:09.828 INFO:tasks.workunit.client.1.smithi138.stdout:2/780: mkdir d0/d85/da5/db4/d15/d51/df2/df6 0 2024-04-25T01:36:09.830 INFO:tasks.workunit.client.1.smithi138.stdout:2/781: creat d0/d21/d45/d8a/d8e/ff7 x:0 0 0 2024-04-25T01:36:09.834 INFO:tasks.workunit.client.1.smithi138.stdout:2/782: creat d0/d85/da5/db4/d15/d44/d4c/d56/ff8 x:0 0 0 2024-04-25T01:36:09.835 INFO:tasks.workunit.client.1.smithi138.stdout:2/783: creat d0/d85/ff9 x:0 0 0 2024-04-25T01:36:09.836 INFO:tasks.workunit.client.1.smithi138.stdout:2/784: read - d0/d85/ff9 zero size 2024-04-25T01:36:09.844 INFO:tasks.workunit.client.1.smithi138.stdout:2/785: truncate d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b/fb5 125567 0 2024-04-25T01:36:09.845 INFO:tasks.workunit.client.1.smithi138.stdout:2/786: fsync d0/d85/ff9 0 2024-04-25T01:36:09.848 INFO:tasks.workunit.client.1.smithi138.stdout:2/787: getdents d0/d21/d45/d8a/d8e 0 2024-04-25T01:36:09.849 INFO:tasks.workunit.client.1.smithi138.stdout:2/788: write d0/d21/d45/ff4 [24546,51385] 0 2024-04-25T01:36:09.851 INFO:tasks.workunit.client.1.smithi138.stdout:2/789: read d0/d85/da5/fb2 [3430896,6375] 0 2024-04-25T01:36:09.859 INFO:tasks.workunit.client.1.smithi138.stdout:2/790: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fc6 [0,4194304] 0 2024-04-25T01:36:09.863 INFO:tasks.workunit.client.1.smithi138.stdout:2/791: symlink d0/d85/da5/db4/d15/d51/lfa 0 2024-04-25T01:36:09.865 INFO:tasks.workunit.client.1.smithi138.stdout:2/792: unlink d0/d21/d45/fc3 0 2024-04-25T01:36:09.867 INFO:tasks.workunit.client.1.smithi138.stdout:2/793: dread - d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b/fed zero size 2024-04-25T01:36:09.868 INFO:tasks.workunit.client.1.smithi138.stdout:2/794: write d0/d85/da5/db4/f3c [4431364,128247] 0 2024-04-25T01:36:09.879 INFO:tasks.workunit.client.1.smithi138.stdout:2/795: getdents d0/d21/d45 0 2024-04-25T01:36:09.881 INFO:tasks.workunit.client.1.smithi138.stdout:2/796: mknod d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/de6/cfb 0 2024-04-25T01:36:09.885 INFO:tasks.workunit.client.1.smithi138.stdout:2/797: rename d0/d85/da5/f43 to d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b/ffc 0 2024-04-25T01:36:09.892 INFO:tasks.workunit.client.1.smithi138.stdout:2/798: link d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b/ffc d0/de9/ffd 0 2024-04-25T01:36:09.892 INFO:tasks.workunit.client.1.smithi138.stdout:2/799: chown d0/l20 147 1 2024-04-25T01:36:09.898 INFO:tasks.workunit.client.1.smithi138.stdout:2/800: dwrite d0/d85/da5/d35/f60 [0,4194304] 0 2024-04-25T01:36:09.901 INFO:tasks.workunit.client.1.smithi138.stdout:2/801: creat d0/d85/ffe x:0 0 0 2024-04-25T01:36:09.903 INFO:tasks.workunit.client.1.smithi138.stdout:2/802: getdents d0/de9/df0 0 2024-04-25T01:36:09.903 INFO:tasks.workunit.client.1.smithi138.stdout:2/803: write d0/d85/da5/db4/f9d [1742487,26619] 0 2024-04-25T01:36:09.904 INFO:tasks.workunit.client.1.smithi138.stdout:2/804: write d0/d85/ffe [590217,58671] 0 2024-04-25T01:36:09.923 INFO:tasks.workunit.client.1.smithi138.stdout:2/805: truncate d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/f6c 2968738 0 2024-04-25T01:36:09.990 INFO:tasks.workunit.client.1.smithi138.stdout:9/524: sync 2024-04-25T01:36:09.991 INFO:tasks.workunit.client.1.smithi138.stdout:4/487: sync 2024-04-25T01:36:09.999 INFO:tasks.workunit.client.1.smithi138.stdout:2/806: mknod d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/cff 0 2024-04-25T01:36:09.999 INFO:tasks.workunit.client.1.smithi138.stdout:9/525: readlink d4/d8/d2c/l8c 0 2024-04-25T01:36:10.000 INFO:tasks.workunit.client.1.smithi138.stdout:2/807: read d0/d85/da5/db4/f1a [8843996,29837] 0 2024-04-25T01:36:10.005 INFO:tasks.workunit.client.1.smithi138.stdout:2/808: dread d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fc6 [0,4194304] 0 2024-04-25T01:36:10.012 INFO:tasks.workunit.client.1.smithi138.stdout:9/526: getdents d4/d63/d86 0 2024-04-25T01:36:10.013 INFO:tasks.workunit.client.1.smithi138.stdout:4/488: truncate d0/d16/d4a/f4d 635444 0 2024-04-25T01:36:10.016 INFO:tasks.workunit.client.1.smithi138.stdout:2/809: symlink d0/d85/da5/db4/d15/d44/d4c/d56/dc8/l100 0 2024-04-25T01:36:10.017 INFO:tasks.workunit.client.1.smithi138.stdout:2/810: write d0/d21/f59 [2062333,35890] 0 2024-04-25T01:36:10.145 INFO:tasks.workunit.client.1.smithi138.stdout:2/811: fdatasync d0/d21/f22 0 2024-04-25T01:36:10.146 INFO:tasks.workunit.client.1.smithi138.stdout:9/527: write d4/d8/d90/d52/f7d [557843,87071] 0 2024-04-25T01:36:10.162 INFO:tasks.workunit.client.1.smithi138.stdout:2/812: readlink d0/d85/da5/db4/d15/d44/l9e 0 2024-04-25T01:36:10.163 INFO:tasks.workunit.client.1.smithi138.stdout:9/528: stat d4/f13 0 2024-04-25T01:36:10.163 INFO:tasks.workunit.client.1.smithi138.stdout:2/813: dread - d0/d21/fdb zero size 2024-04-25T01:36:10.181 INFO:tasks.workunit.client.1.smithi138.stdout:2/814: write d0/d85/da5/fb2 [1621359,107199] 0 2024-04-25T01:36:10.187 INFO:tasks.workunit.client.1.smithi138.stdout:9/529: dwrite d4/d1a/d33/f16 [0,4194304] 0 2024-04-25T01:36:10.193 INFO:tasks.workunit.client.1.smithi138.stdout:2/815: rename d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b/fc7 to d0/d85/da5/db4/d15/d44/d4c/d56/f101 0 2024-04-25T01:36:10.195 INFO:tasks.workunit.client.1.smithi138.stdout:2/816: rmdir d0/d85/da5/db4/d15/d44 39 2024-04-25T01:36:10.195 INFO:tasks.workunit.client.1.smithi138.stdout:1/574: sync 2024-04-25T01:36:10.195 INFO:tasks.workunit.client.1.smithi138.stdout:7/677: sync 2024-04-25T01:36:10.195 INFO:tasks.workunit.client.1.smithi138.stdout:0/461: sync 2024-04-25T01:36:10.195 INFO:tasks.workunit.client.1.smithi138.stdout:3/689: sync 2024-04-25T01:36:10.195 INFO:tasks.workunit.client.1.smithi138.stdout:3/690: readlink db/d9b/lbb 0 2024-04-25T01:36:10.196 INFO:tasks.workunit.client.1.smithi138.stdout:1/575: truncate d3/d27/d3e/d51/f75 859702 0 2024-04-25T01:36:10.197 INFO:tasks.workunit.client.1.smithi138.stdout:9/530: link d4/d8/d2c/d82/la1 d4/d1a/d33/d9a/laa 0 2024-04-25T01:36:10.198 INFO:tasks.workunit.client.1.smithi138.stdout:7/678: readlink da/de/d13/l26 0 2024-04-25T01:36:10.199 INFO:tasks.workunit.client.1.smithi138.stdout:0/462: rmdir d6/d2f/d75 39 2024-04-25T01:36:10.200 INFO:tasks.workunit.client.1.smithi138.stdout:7/679: chown da/de/d1b/d34/l90 1877173314 1 2024-04-25T01:36:10.202 INFO:tasks.workunit.client.1.smithi138.stdout:9/531: dread d4/d1a/d33/f36 [0,4194304] 0 2024-04-25T01:36:10.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/680: chown da/de/d1b/d49/d81/d51/l9a 106450 1 2024-04-25T01:36:10.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/681: readlink da/de/l1d 0 2024-04-25T01:36:10.203 INFO:tasks.workunit.client.1.smithi138.stdout:3/691: creat db/d14/d40/d41/fde x:0 0 0 2024-04-25T01:36:10.205 INFO:tasks.workunit.client.1.smithi138.stdout:1/576: mknod d3/d45/cc1 0 2024-04-25T01:36:10.209 INFO:tasks.workunit.client.1.smithi138.stdout:2/817: write d0/d85/da5/db4/d15/d44/d4c/d56/d6d/ff1 [124215,6855] 0 2024-04-25T01:36:10.209 INFO:tasks.workunit.client.1.smithi138.stdout:1/577: dread - d3/d4/d47/f5c zero size 2024-04-25T01:36:10.212 INFO:tasks.workunit.client.1.smithi138.stdout:9/532: creat d4/d1a/d68/da0/fab x:0 0 0 2024-04-25T01:36:10.213 INFO:tasks.workunit.client.1.smithi138.stdout:3/692: truncate db/df/d17/d20/d57/d6a/f6e 1676773 0 2024-04-25T01:36:10.214 INFO:tasks.workunit.client.1.smithi138.stdout:0/463: rename d6/d1d/l1b to d6/d39/l91 0 2024-04-25T01:36:10.218 INFO:tasks.workunit.client.1.smithi138.stdout:9/533: creat d4/d1a/d3c/fac x:0 0 0 2024-04-25T01:36:10.222 INFO:tasks.workunit.client.1.smithi138.stdout:2/818: mknod d0/d85/da5/db4/d15/d44/d4c/d56/da2/c102 0 2024-04-25T01:36:10.224 INFO:tasks.workunit.client.1.smithi138.stdout:3/693: write db/df/d17/d5d/f6b [1487988,48145] 0 2024-04-25T01:36:10.226 INFO:tasks.workunit.client.1.smithi138.stdout:2/819: fdatasync d0/d21/d45/d8a/fa7 0 2024-04-25T01:36:10.226 INFO:tasks.workunit.client.1.smithi138.stdout:3/694: chown db/df/d17/d23/d34/f80 0 1 2024-04-25T01:36:10.227 INFO:tasks.workunit.client.1.smithi138.stdout:0/464: dread d6/f1e [0,4194304] 0 2024-04-25T01:36:10.228 INFO:tasks.workunit.client.1.smithi138.stdout:3/695: fdatasync db/df/d17/d5d/fa3 0 2024-04-25T01:36:10.230 INFO:tasks.workunit.client.1.smithi138.stdout:7/682: rename da/de/d1b/c3b to da/de/ce1 0 2024-04-25T01:36:10.231 INFO:tasks.workunit.client.1.smithi138.stdout:1/578: dread d3/d6d/f92 [0,4194304] 0 2024-04-25T01:36:10.232 INFO:tasks.workunit.client.1.smithi138.stdout:3/696: dread - db/df/d17/d20/d57/daa/fd7 zero size 2024-04-25T01:36:10.238 INFO:tasks.workunit.client.1.smithi138.stdout:7/683: creat da/dd2/daf/fe2 x:0 0 0 2024-04-25T01:36:10.239 INFO:tasks.workunit.client.1.smithi138.stdout:2/820: dwrite d0/d21/d45/d8a/feb [0,4194304] 0 2024-04-25T01:36:10.242 INFO:tasks.workunit.client.1.smithi138.stdout:7/684: fsync da/de/d13/f76 0 2024-04-25T01:36:10.243 INFO:tasks.workunit.client.1.smithi138.stdout:3/697: getdents db/df/d19/d1a/dc4 0 2024-04-25T01:36:10.246 INFO:tasks.workunit.client.1.smithi138.stdout:3/698: read - db/df/d17/d23/d34/fba zero size 2024-04-25T01:36:10.246 INFO:tasks.workunit.client.1.smithi138.stdout:0/465: write d6/df/f10 [4616092,129905] 0 2024-04-25T01:36:10.248 INFO:tasks.workunit.client.1.smithi138.stdout:1/579: write d3/d27/d2d/f32 [509686,96908] 0 2024-04-25T01:36:10.251 INFO:tasks.workunit.client.1.smithi138.stdout:9/534: dwrite d4/f25 [0,4194304] 0 2024-04-25T01:36:10.253 INFO:tasks.workunit.client.1.smithi138.stdout:0/466: write d6/d19/f77 [4563062,97749] 0 2024-04-25T01:36:10.253 INFO:tasks.workunit.client.1.smithi138.stdout:1/580: stat d3/d27/d16/f28 0 2024-04-25T01:36:10.256 INFO:tasks.workunit.client.1.smithi138.stdout:0/467: write d6/f28 [2403534,5128] 0 2024-04-25T01:36:10.257 INFO:tasks.workunit.client.1.smithi138.stdout:7/685: dwrite da/f18 [0,4194304] 0 2024-04-25T01:36:10.259 INFO:tasks.workunit.client.1.smithi138.stdout:9/535: rmdir d4/d1a/d55 39 2024-04-25T01:36:10.263 INFO:tasks.workunit.client.1.smithi138.stdout:9/536: write d4/d1a/f21 [2557707,52039] 0 2024-04-25T01:36:10.264 INFO:tasks.workunit.client.1.smithi138.stdout:7/686: write da/de/d1b/d34/f3d [3159107,15684] 0 2024-04-25T01:36:10.268 INFO:tasks.workunit.client.1.smithi138.stdout:0/468: dwrite d6/df/d41/d49/f63 [0,4194304] 0 2024-04-25T01:36:10.272 INFO:tasks.workunit.client.1.smithi138.stdout:9/537: truncate d4/d63/f8b 236456 0 2024-04-25T01:36:10.276 INFO:tasks.workunit.client.1.smithi138.stdout:2/821: getdents d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b 0 2024-04-25T01:36:10.278 INFO:tasks.workunit.client.1.smithi138.stdout:9/538: creat d4/d63/d86/fad x:0 0 0 2024-04-25T01:36:10.281 INFO:tasks.workunit.client.1.smithi138.stdout:1/581: write d3/f44 [209861,65279] 0 2024-04-25T01:36:10.281 INFO:tasks.workunit.client.1.smithi138.stdout:7/687: dwrite da/de/d1b/d34/fc2 [0,4194304] 0 2024-04-25T01:36:10.283 INFO:tasks.workunit.client.1.smithi138.stdout:9/539: truncate d4/f6c 1095471 0 2024-04-25T01:36:10.286 INFO:tasks.workunit.client.1.smithi138.stdout:1/582: creat d3/d4/d47/fc2 x:0 0 0 2024-04-25T01:36:10.338 INFO:tasks.workunit.client.1.smithi138.stdout:9/540: mknod d4/cae 0 2024-04-25T01:36:10.339 INFO:tasks.workunit.client.1.smithi138.stdout:3/699: dread db/df/d17/d5d/f97 [0,4194304] 0 2024-04-25T01:36:10.341 INFO:tasks.workunit.client.1.smithi138.stdout:7/688: dwrite da/de/d13/d30/f93 [0,4194304] 0 2024-04-25T01:36:10.342 INFO:tasks.workunit.client.1.smithi138.stdout:7/689: fdatasync da/de/d13/fb5 0 2024-04-25T01:36:10.343 INFO:tasks.workunit.client.1.smithi138.stdout:3/700: dread - db/df/fbd zero size 2024-04-25T01:36:10.344 INFO:tasks.workunit.client.1.smithi138.stdout:9/541: rmdir d4/d1a/d68/da0 39 2024-04-25T01:36:10.348 INFO:tasks.workunit.client.1.smithi138.stdout:0/469: dread d6/df/f10 [0,4194304] 0 2024-04-25T01:36:10.349 INFO:tasks.workunit.client.1.smithi138.stdout:7/690: stat da/l3a 0 2024-04-25T01:36:10.351 INFO:tasks.workunit.client.1.smithi138.stdout:1/583: link d3/d4/d47/c4f d3/d4/da7/db1/cc3 0 2024-04-25T01:36:10.352 INFO:tasks.workunit.client.1.smithi138.stdout:0/470: write d6/df/d3a/f3d [5583823,16293] 0 2024-04-25T01:36:10.353 INFO:tasks.workunit.client.1.smithi138.stdout:0/471: chown d6/df/d41/d49 235 1 2024-04-25T01:36:10.353 INFO:tasks.workunit.client.1.smithi138.stdout:3/701: mkdir db/df/dd1/ddf 0 2024-04-25T01:36:10.355 INFO:tasks.workunit.client.1.smithi138.stdout:7/691: creat da/de/d13/d21/da5/fe3 x:0 0 0 2024-04-25T01:36:10.356 INFO:tasks.workunit.client.1.smithi138.stdout:0/472: fdatasync d6/d39/d43/f6b 0 2024-04-25T01:36:10.357 INFO:tasks.workunit.client.1.smithi138.stdout:3/702: truncate db/df/d17/d23/d88/fc3 517505 0 2024-04-25T01:36:10.359 INFO:tasks.workunit.client.1.smithi138.stdout:0/473: dread - d6/df/d41/d49/f7c zero size 2024-04-25T01:36:10.360 INFO:tasks.workunit.client.1.smithi138.stdout:9/542: dread d4/d1a/f20 [0,4194304] 0 2024-04-25T01:36:10.361 INFO:tasks.workunit.client.1.smithi138.stdout:1/584: rename d3/d45/l40 to d3/lc4 0 2024-04-25T01:36:10.361 INFO:tasks.workunit.client.1.smithi138.stdout:9/543: stat d4/d8/d90/l8a 0 2024-04-25T01:36:10.362 INFO:tasks.workunit.client.1.smithi138.stdout:1/585: chown d3/d6d/c83 0 1 2024-04-25T01:36:10.365 INFO:tasks.workunit.client.1.smithi138.stdout:0/474: dread d6/df/f18 [4194304,4194304] 0 2024-04-25T01:36:10.365 INFO:tasks.workunit.client.1.smithi138.stdout:9/544: readlink d4/d1a/d33/d1d/l39 0 2024-04-25T01:36:10.366 INFO:tasks.workunit.client.1.smithi138.stdout:0/475: chown d6/d2f/d31/l6c 2 1 2024-04-25T01:36:10.366 INFO:tasks.workunit.client.1.smithi138.stdout:0/476: read - d6/df/d66/f7f zero size 2024-04-25T01:36:10.367 INFO:tasks.workunit.client.1.smithi138.stdout:1/586: rmdir d3/d4/d47 39 2024-04-25T01:36:10.368 INFO:tasks.workunit.client.1.smithi138.stdout:1/587: fsync d3/d27/d3e/d51/f80 0 2024-04-25T01:36:10.370 INFO:tasks.workunit.client.1.smithi138.stdout:3/703: getdents db/df/d17/d20/d57/d6a 0 2024-04-25T01:36:10.371 INFO:tasks.workunit.client.1.smithi138.stdout:7/692: truncate da/de/d13/d30/d59/f78 2025925 0 2024-04-25T01:36:10.373 INFO:tasks.workunit.client.1.smithi138.stdout:9/545: write d4/d1a/d55/d7f/d8f/f99 [113403,23563] 0 2024-04-25T01:36:10.374 INFO:tasks.workunit.client.1.smithi138.stdout:9/546: fdatasync d4/f38 0 2024-04-25T01:36:10.375 INFO:tasks.workunit.client.1.smithi138.stdout:7/693: symlink da/de/d13/d21/dbc/d94/d9d/le4 0 2024-04-25T01:36:10.381 INFO:tasks.workunit.client.1.smithi138.stdout:9/547: fdatasync d4/d49/f76 0 2024-04-25T01:36:10.382 INFO:tasks.workunit.client.1.smithi138.stdout:7/694: symlink da/de/d1b/d49/d81/d51/d6e/le5 0 2024-04-25T01:36:10.383 INFO:tasks.workunit.client.1.smithi138.stdout:3/704: truncate db/df/d17/d5d/f6b 188455 0 2024-04-25T01:36:10.383 INFO:tasks.workunit.client.1.smithi138.stdout:1/588: getdents d3/d45/d72/db2 0 2024-04-25T01:36:10.385 INFO:tasks.workunit.client.1.smithi138.stdout:1/589: write d3/d27/d16/da3/daf/fb3 [855220,76215] 0 2024-04-25T01:36:10.386 INFO:tasks.workunit.client.1.smithi138.stdout:7/695: symlink da/de/d13/d30/le6 0 2024-04-25T01:36:10.389 INFO:tasks.workunit.client.1.smithi138.stdout:1/590: creat d3/d4/d53/d9d/fc5 x:0 0 0 2024-04-25T01:36:10.392 INFO:tasks.workunit.client.1.smithi138.stdout:3/705: getdents db/df/d19/d1a/d9e 0 2024-04-25T01:36:10.395 INFO:tasks.workunit.client.1.smithi138.stdout:3/706: fdatasync db/d14/d40/d41/f7c 0 2024-04-25T01:36:10.397 INFO:tasks.workunit.client.1.smithi138.stdout:1/591: dwrite d3/d4/d53/f8c [0,4194304] 0 2024-04-25T01:36:10.398 INFO:tasks.workunit.client.1.smithi138.stdout:3/707: readlink db/df/d17/d23/l39 0 2024-04-25T01:36:10.400 INFO:tasks.workunit.client.1.smithi138.stdout:9/548: dread d4/f38 [0,4194304] 0 2024-04-25T01:36:10.401 INFO:tasks.workunit.client.1.smithi138.stdout:7/696: write da/f6a [4230307,117036] 0 2024-04-25T01:36:10.402 INFO:tasks.workunit.client.1.smithi138.stdout:6/676: sync 2024-04-25T01:36:10.404 INFO:tasks.workunit.client.1.smithi138.stdout:9/549: read d4/d1a/f2d [1795764,44128] 0 2024-04-25T01:36:10.405 INFO:tasks.workunit.client.1.smithi138.stdout:7/697: rename da/de/d1b/d49/d81/d51/d6e/ldc to da/de/d1b/le7 0 2024-04-25T01:36:10.408 INFO:tasks.workunit.client.1.smithi138.stdout:6/677: rename dc/d10/l3b to dc/d2e/d37/d56/ld5 0 2024-04-25T01:36:10.412 INFO:tasks.workunit.client.1.smithi138.stdout:7/698: dwrite da/de/d1b/d4b/d77/f91 [0,4194304] 0 2024-04-25T01:36:10.423 INFO:tasks.workunit.client.1.smithi138.stdout:1/592: dread d3/d45/f48 [0,4194304] 0 2024-04-25T01:36:10.424 INFO:tasks.workunit.client.1.smithi138.stdout:1/593: truncate d3/d27/d3e/d51/f8b 536926 0 2024-04-25T01:36:10.436 INFO:tasks.workunit.client.1.smithi138.stdout:7/699: dread da/de/d13/d30/f65 [0,4194304] 0 2024-04-25T01:36:10.459 INFO:tasks.workunit.client.1.smithi138.stdout:9/550: getdents d4/d1a/d55 0 2024-04-25T01:36:10.460 INFO:tasks.workunit.client.1.smithi138.stdout:9/551: truncate d4/d1a/f84 310065 0 2024-04-25T01:36:10.461 INFO:tasks.workunit.client.1.smithi138.stdout:1/594: creat d3/d27/d16/da3/daf/fc6 x:0 0 0 2024-04-25T01:36:10.463 INFO:tasks.workunit.client.1.smithi138.stdout:6/678: symlink dc/d10/ld6 0 2024-04-25T01:36:10.543 INFO:tasks.workunit.client.1.smithi138.stdout:9/552: symlink d4/d49/d72/laf 0 2024-04-25T01:36:10.544 INFO:tasks.workunit.client.1.smithi138.stdout:1/595: creat d3/d45/d72/db2/fc7 x:0 0 0 2024-04-25T01:36:10.545 INFO:tasks.workunit.client.1.smithi138.stdout:7/700: dread da/de/d13/f3c [0,4194304] 0 2024-04-25T01:36:10.547 INFO:tasks.workunit.client.1.smithi138.stdout:9/553: rename d4/d8/f53 to d4/d8/d2c/da3/fb0 0 2024-04-25T01:36:10.550 INFO:tasks.workunit.client.1.smithi138.stdout:6/679: truncate dc/d10/f33 391444 0 2024-04-25T01:36:10.552 INFO:tasks.workunit.client.1.smithi138.stdout:6/680: read - dc/d10/fa5 zero size 2024-04-25T01:36:10.555 INFO:tasks.workunit.client.1.smithi138.stdout:7/701: dwrite da/de/d1b/d49/d81/d51/d6e/da2/fd9 [0,4194304] 0 2024-04-25T01:36:10.556 INFO:tasks.workunit.client.1.smithi138.stdout:9/554: rename d4/d1a/d33/d1d/l32 to d4/d63/d86/lb1 0 2024-04-25T01:36:10.562 INFO:tasks.workunit.client.1.smithi138.stdout:7/702: chown da/de/d1b/d49/d81/d73/lc0 6940 1 2024-04-25T01:36:10.562 INFO:tasks.workunit.client.1.smithi138.stdout:1/596: dread d3/d45/fa0 [0,4194304] 0 2024-04-25T01:36:10.565 INFO:tasks.workunit.client.1.smithi138.stdout:7/703: rename da/de/d1b/d49/d81/dc3 to da/de/d13/d21/da5/de8 0 2024-04-25T01:36:10.566 INFO:tasks.workunit.client.1.smithi138.stdout:1/597: mknod d3/db/cc8 0 2024-04-25T01:36:10.570 INFO:tasks.workunit.client.1.smithi138.stdout:6/681: dwrite dc/d2e/d55/d7e/fae [4194304,4194304] 0 2024-04-25T01:36:10.577 INFO:tasks.workunit.client.1.smithi138.stdout:1/598: dwrite d3/db/f5a [0,4194304] 0 2024-04-25T01:36:10.579 INFO:tasks.workunit.client.1.smithi138.stdout:9/555: dwrite d4/d1a/d55/f5f [0,4194304] 0 2024-04-25T01:36:10.610 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:10.610 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:10.610 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.610 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.610 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.611 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.611 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.611 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.611 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: Adjusting osd_memory_target on smithi138 to 2581M 2024-04-25T01:36:10.611 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:10.611 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:36:10.611 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:36:10.611 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-25T01:36:10.611 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:10 smithi033 ceph-mon[26353]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T01:36:10.622 INFO:tasks.workunit.client.1.smithi138.stdout:5/594: sync 2024-04-25T01:36:10.622 INFO:tasks.workunit.client.1.smithi138.stdout:8/757: sync 2024-04-25T01:36:10.625 INFO:tasks.workunit.client.1.smithi138.stdout:9/556: getdents d4/d1a/d55 0 2024-04-25T01:36:10.628 INFO:tasks.workunit.client.1.smithi138.stdout:1/599: rename d3/d27/d24/f81 to d3/fc9 0 2024-04-25T01:36:10.630 INFO:tasks.workunit.client.1.smithi138.stdout:5/595: creat d4/da/d6e/d22/d74/d59/fc7 x:0 0 0 2024-04-25T01:36:10.631 INFO:tasks.workunit.client.1.smithi138.stdout:8/758: symlink d3/d16/d88/da/db/d15/le9 0 2024-04-25T01:36:10.631 INFO:tasks.workunit.client.1.smithi138.stdout:6/682: dwrite dc/d2e/d55/fa7 [4194304,4194304] 0 2024-04-25T01:36:10.633 INFO:tasks.workunit.client.1.smithi138.stdout:6/683: fsync dc/d10/fa5 0 2024-04-25T01:36:10.634 INFO:tasks.workunit.client.1.smithi138.stdout:9/557: mkdir d4/d1a/d55/d7f/d80/d94/db2 0 2024-04-25T01:36:10.636 INFO:tasks.workunit.client.1.smithi138.stdout:5/596: dread d4/da/d27/f2d [0,4194304] 0 2024-04-25T01:36:10.638 INFO:tasks.workunit.client.1.smithi138.stdout:8/759: symlink d3/d16/d88/da/d70/lea 0 2024-04-25T01:36:10.640 INFO:tasks.workunit.client.1.smithi138.stdout:1/600: mkdir d3/d27/d2d/da1/dca 0 2024-04-25T01:36:10.644 INFO:tasks.workunit.client.1.smithi138.stdout:8/760: mknod d3/d16/d1f/d2a/d4f/dcb/dce/ceb 0 2024-04-25T01:36:10.644 INFO:tasks.workunit.client.1.smithi138.stdout:6/684: dwrite dc/d10/d28/d70/fa9 [0,4194304] 0 2024-04-25T01:36:10.653 INFO:tasks.workunit.client.1.smithi138.stdout:6/685: symlink dc/d10/d67/ld7 0 2024-04-25T01:36:10.654 INFO:tasks.workunit.client.1.smithi138.stdout:1/601: dwrite d3/d27/d3e/d90/f78 [0,4194304] 0 2024-04-25T01:36:10.654 INFO:tasks.workunit.client.1.smithi138.stdout:8/761: creat d3/d44/da4/fec x:0 0 0 2024-04-25T01:36:10.659 INFO:tasks.workunit.client.1.smithi138.stdout:9/558: rename d4/d1a/d33/c85 to d4/d1a/cb3 0 2024-04-25T01:36:10.665 INFO:tasks.workunit.client.1.smithi138.stdout:5/597: rename d4/da/d6e/d22/d88 to d4/da/d6e/d22/d35/d4d/d5e/d60/dbd/dc8 0 2024-04-25T01:36:10.665 INFO:tasks.workunit.client.1.smithi138.stdout:8/762: dwrite d3/d16/d1f/d31/d82/fbd [0,4194304] 0 2024-04-25T01:36:10.667 INFO:tasks.workunit.client.1.smithi138.stdout:8/763: stat d3/d44/d48/dbb 0 2024-04-25T01:36:10.668 INFO:tasks.workunit.client.1.smithi138.stdout:9/559: creat d4/d1a/d68/da5/fb4 x:0 0 0 2024-04-25T01:36:10.670 INFO:tasks.workunit.client.1.smithi138.stdout:8/764: write d3/d16/d1f/d2a/d4f/dcb/dce/fe7 [683851,103906] 0 2024-04-25T01:36:10.671 INFO:tasks.workunit.client.1.smithi138.stdout:9/560: mknod d4/d1a/d55/d7f/cb5 0 2024-04-25T01:36:10.677 INFO:tasks.workunit.client.1.smithi138.stdout:5/598: dwrite d4/da/d6e/d22/d35/d4d/d5e/d8f/fa2 [0,4194304] 0 2024-04-25T01:36:10.679 INFO:tasks.workunit.client.1.smithi138.stdout:6/686: write dc/f60 [666211,41134] 0 2024-04-25T01:36:10.688 INFO:tasks.workunit.client.1.smithi138.stdout:6/687: write dc/d2e/d37/d56/fa8 [32030,55368] 0 2024-04-25T01:36:10.691 INFO:tasks.workunit.client.1.smithi138.stdout:5/599: dwrite d4/da/d6e/d22/d74/d8e/fae [0,4194304] 0 2024-04-25T01:36:10.695 INFO:tasks.workunit.client.1.smithi138.stdout:6/688: read dc/d2e/f31 [1375232,116197] 0 2024-04-25T01:36:10.701 INFO:tasks.workunit.client.1.smithi138.stdout:5/600: dwrite d4/da/d6e/d22/d74/d59/f68 [0,4194304] 0 2024-04-25T01:36:10.703 INFO:tasks.workunit.client.1.smithi138.stdout:9/561: dread d4/f13 [4194304,4194304] 0 2024-04-25T01:36:10.705 INFO:tasks.workunit.client.1.smithi138.stdout:9/562: write d4/f25 [2715462,115416] 0 2024-04-25T01:36:10.705 INFO:tasks.workunit.client.1.smithi138.stdout:8/765: dread d3/d16/d88/fcd [0,4194304] 0 2024-04-25T01:36:10.707 INFO:tasks.workunit.client.1.smithi138.stdout:9/563: rmdir d4/d8/d2c/da3 39 2024-04-25T01:36:10.707 INFO:tasks.workunit.client.1.smithi138.stdout:8/766: write d3/d16/d1f/d2a/f8b [6051656,41568] 0 2024-04-25T01:36:10.709 INFO:tasks.workunit.client.1.smithi138.stdout:8/767: dread - d3/d16/d1f/d2a/d4f/dac/fd9 zero size 2024-04-25T01:36:10.711 INFO:tasks.workunit.client.1.smithi138.stdout:5/601: link d4/da/fe d4/da/d6e/d22/d35/d4d/d90/dba/fc9 0 2024-04-25T01:36:10.713 INFO:tasks.workunit.client.1.smithi138.stdout:9/564: rename d4/d1a/d3c/l54 to d4/d1a/d55/lb6 0 2024-04-25T01:36:10.714 INFO:tasks.workunit.client.1.smithi138.stdout:6/689: write dc/d2e/d37/d56/f74 [147988,86745] 0 2024-04-25T01:36:10.714 INFO:tasks.workunit.client.1.smithi138.stdout:8/768: mknod d3/d16/d1f/dd2/dd6/ced 0 2024-04-25T01:36:10.716 INFO:tasks.workunit.client.1.smithi138.stdout:9/565: write d4/d1a/d55/fa9 [643317,19444] 0 2024-04-25T01:36:10.717 INFO:tasks.workunit.client.1.smithi138.stdout:6/690: chown dc/d2e/d55/d81/lb5 23423 1 2024-04-25T01:36:10.717 INFO:tasks.workunit.client.1.smithi138.stdout:5/602: mkdir d4/da/d6e/d22/d35/d4d/dca 0 2024-04-25T01:36:10.718 INFO:tasks.workunit.client.1.smithi138.stdout:8/769: symlink d3/d44/d48/lee 0 2024-04-25T01:36:10.721 INFO:tasks.workunit.client.1.smithi138.stdout:9/566: chown d4/d8/d90/d52/l97 382 1 2024-04-25T01:36:10.722 INFO:tasks.workunit.client.1.smithi138.stdout:5/603: write d4/da/d6e/d22/d74/d59/f99 [932914,114021] 0 2024-04-25T01:36:10.722 INFO:tasks.workunit.client.1.smithi138.stdout:6/691: creat dc/d2e/db9/fd8 x:0 0 0 2024-04-25T01:36:10.722 INFO:tasks.workunit.client.1.smithi138.stdout:8/770: write d3/d16/d1f/d31/d82/f90 [1874736,126986] 0 2024-04-25T01:36:10.723 INFO:tasks.workunit.client.1.smithi138.stdout:5/604: fdatasync d4/da/d6e/d22/d74/f9c 0 2024-04-25T01:36:10.724 INFO:tasks.workunit.client.1.smithi138.stdout:6/692: read dc/d2e/d55/d81/f9f [44455,96870] 0 2024-04-25T01:36:10.725 INFO:tasks.workunit.client.1.smithi138.stdout:8/771: stat d3/d16/d88/da/db/d15/d20/f6e 0 2024-04-25T01:36:10.726 INFO:tasks.workunit.client.1.smithi138.stdout:9/567: truncate d4/d8/d90/d52/f7d 1356404 0 2024-04-25T01:36:10.726 INFO:tasks.workunit.client.1.smithi138.stdout:5/605: dread - d4/da/d6e/d22/d74/d59/d7f/fa4 zero size 2024-04-25T01:36:10.727 INFO:tasks.workunit.client.1.smithi138.stdout:9/568: fsync d4/d1a/f81 0 2024-04-25T01:36:10.729 INFO:tasks.workunit.client.1.smithi138.stdout:8/772: mkdir d3/d16/d88/da/db/d15/d56/d8f/def 0 2024-04-25T01:36:10.732 INFO:tasks.workunit.client.1.smithi138.stdout:5/606: fdatasync d4/da/d6e/d22/d35/d4d/d5e/f75 0 2024-04-25T01:36:10.733 INFO:tasks.workunit.client.1.smithi138.stdout:6/693: dwrite dc/d2e/d55/fa7 [0,4194304] 0 2024-04-25T01:36:10.734 INFO:tasks.workunit.client.1.smithi138.stdout:9/569: creat d4/d1a/d68/da0/fb7 x:0 0 0 2024-04-25T01:36:10.739 INFO:tasks.workunit.client.1.smithi138.stdout:8/773: unlink d3/d16/d88/da/db/d15/d20/ce5 0 2024-04-25T01:36:10.743 INFO:tasks.workunit.client.1.smithi138.stdout:6/694: mknod dc/d10/d28/cd9 0 2024-04-25T01:36:10.746 INFO:tasks.workunit.client.1.smithi138.stdout:8/774: mkdir d3/d16/d1f/d2a/d4f/dcb/dce/df0 0 2024-04-25T01:36:10.747 INFO:tasks.workunit.client.1.smithi138.stdout:5/607: creat d4/da/d6e/d22/fcb x:0 0 0 2024-04-25T01:36:10.749 INFO:tasks.workunit.client.1.smithi138.stdout:6/695: mknod dc/d10/d28/d80/d89/cda 0 2024-04-25T01:36:10.751 INFO:tasks.workunit.client.1.smithi138.stdout:8/775: mknod d3/d16/d1f/dc5/cf1 0 2024-04-25T01:36:10.754 INFO:tasks.workunit.client.1.smithi138.stdout:8/776: truncate d3/d44/d48/fa3 4810049 0 2024-04-25T01:36:10.755 INFO:tasks.workunit.client.1.smithi138.stdout:8/777: chown d3/d16/d88/da/db/d15/d20/f26 320784455 1 2024-04-25T01:36:10.762 INFO:tasks.workunit.client.1.smithi138.stdout:6/696: truncate dc/d10/f33 249204 0 2024-04-25T01:36:10.763 INFO:tasks.workunit.client.1.smithi138.stdout:9/570: dread d4/d8/d90/f78 [0,4194304] 0 2024-04-25T01:36:10.764 INFO:tasks.workunit.client.1.smithi138.stdout:6/697: creat dc/d2e/dc1/fdb x:0 0 0 2024-04-25T01:36:10.765 INFO:tasks.workunit.client.1.smithi138.stdout:5/608: dwrite d4/da/fe [0,4194304] 0 2024-04-25T01:36:10.765 INFO:tasks.workunit.client.1.smithi138.stdout:9/571: rename d4/d1a to d4/d1a/d33/d1d/db8 22 2024-04-25T01:36:10.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:10.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:10.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T01:36:10.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: Adjusting osd_memory_target on smithi138 to 2581M 2024-04-25T01:36:10.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:10.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:36:10.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:36:10.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-25T01:36:10.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:10 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T01:36:10.770 INFO:tasks.workunit.client.1.smithi138.stdout:6/698: mknod dc/d2e/d55/d81/cdc 0 2024-04-25T01:36:10.774 INFO:tasks.workunit.client.1.smithi138.stdout:9/572: rename d4/d1a/d55/d7f/d80 to d4/d8/d2c/d82/db9 0 2024-04-25T01:36:10.776 INFO:tasks.workunit.client.1.smithi138.stdout:5/609: dwrite d4/da/d6e/d22/d74/d59/f70 [4194304,4194304] 0 2024-04-25T01:36:10.780 INFO:tasks.workunit.client.1.smithi138.stdout:9/573: creat d4/d8/d2c/da3/fba x:0 0 0 2024-04-25T01:36:10.786 INFO:tasks.workunit.client.1.smithi138.stdout:6/699: dwrite dc/d8d/f97 [0,4194304] 0 2024-04-25T01:36:10.788 INFO:tasks.workunit.client.1.smithi138.stdout:9/574: rmdir d4/d1a/d55/d7f/d8f 39 2024-04-25T01:36:10.793 INFO:tasks.workunit.client.1.smithi138.stdout:5/610: dwrite d4/da/d6e/d22/d74/d59/fc7 [0,4194304] 0 2024-04-25T01:36:10.793 INFO:tasks.workunit.client.1.smithi138.stdout:6/700: write dc/d10/f26 [1628872,96166] 0 2024-04-25T01:36:10.802 INFO:tasks.workunit.client.1.smithi138.stdout:5/611: mknod d4/da/d6e/d22/d35/d4d/d5e/d8f/ccc 0 2024-04-25T01:36:10.810 INFO:tasks.workunit.client.1.smithi138.stdout:6/701: dwrite dc/d10/d28/d70/d83/f9d [0,4194304] 0 2024-04-25T01:36:10.814 INFO:tasks.workunit.client.1.smithi138.stdout:5/612: rename d4/da/d6e/d5c/dac to d4/da/d6e/d22/d74/d59/d83/dcd 0 2024-04-25T01:36:10.823 INFO:tasks.workunit.client.1.smithi138.stdout:6/702: rename dc/d10/d28/d80/fa2 to dc/d10/d67/d8e/daf/fdd 0 2024-04-25T01:36:10.823 INFO:tasks.workunit.client.1.smithi138.stdout:9/575: dread d4/d57/f67 [0,4194304] 0 2024-04-25T01:36:10.826 INFO:tasks.workunit.client.1.smithi138.stdout:9/576: dread - d4/d1a/fa2 zero size 2024-04-25T01:36:10.826 INFO:tasks.workunit.client.1.smithi138.stdout:6/703: fdatasync dc/d10/d67/f77 0 2024-04-25T01:36:10.826 INFO:tasks.workunit.client.1.smithi138.stdout:9/577: dread - d4/d1a/f81 zero size 2024-04-25T01:36:10.827 INFO:tasks.workunit.client.1.smithi138.stdout:5/613: dwrite d4/fc1 [0,4194304] 0 2024-04-25T01:36:10.834 INFO:tasks.workunit.client.1.smithi138.stdout:9/578: creat d4/d63/d86/fbb x:0 0 0 2024-04-25T01:36:10.837 INFO:tasks.workunit.client.1.smithi138.stdout:5/614: mkdir d4/da/d6e/d22/d35/dce 0 2024-04-25T01:36:10.839 INFO:tasks.workunit.client.1.smithi138.stdout:9/579: symlink d4/d8/lbc 0 2024-04-25T01:36:10.842 INFO:tasks.workunit.client.1.smithi138.stdout:9/580: truncate d4/d1a/d68/da5/fb4 255865 0 2024-04-25T01:36:10.848 INFO:tasks.workunit.client.1.smithi138.stdout:9/581: mknod d4/d63/d86/cbd 0 2024-04-25T01:36:10.848 INFO:tasks.workunit.client.1.smithi138.stdout:5/615: getdents d4/da/d6e/d22/d35/d4d/d5e/d60 0 2024-04-25T01:36:10.849 INFO:tasks.workunit.client.1.smithi138.stdout:9/582: chown d4/d63/d86/f93 374691 1 2024-04-25T01:36:10.854 INFO:tasks.workunit.client.1.smithi138.stdout:9/583: readlink d4/d8/d2c/l34 0 2024-04-25T01:36:10.860 INFO:tasks.workunit.client.1.smithi138.stdout:9/584: mkdir d4/d1a/d68/dbe 0 2024-04-25T01:36:10.864 INFO:tasks.workunit.client.1.smithi138.stdout:6/704: dwrite f8 [0,4194304] 0 2024-04-25T01:36:10.869 INFO:tasks.workunit.client.1.smithi138.stdout:6/705: getdents dc/d10/d67/d8e/daf 0 2024-04-25T01:36:10.874 INFO:tasks.workunit.client.1.smithi138.stdout:6/706: rename dc/d2e/d55/c90 to dc/d2e/d37/d56/cde 0 2024-04-25T01:36:10.881 INFO:tasks.workunit.client.1.smithi138.stdout:6/707: creat dc/d10/d28/d70/fdf x:0 0 0 2024-04-25T01:36:10.885 INFO:tasks.workunit.client.1.smithi138.stdout:5/616: dwrite d4/da/fd [0,4194304] 0 2024-04-25T01:36:10.887 INFO:tasks.workunit.client.1.smithi138.stdout:6/708: symlink dc/d2e/d37/d56/d8c/le0 0 2024-04-25T01:36:10.888 INFO:tasks.workunit.client.1.smithi138.stdout:6/709: dread - dc/d10/d28/d70/fdf zero size 2024-04-25T01:36:10.893 INFO:tasks.workunit.client.1.smithi138.stdout:6/710: read dc/d10/d6d/f5e [79003,52309] 0 2024-04-25T01:36:10.894 INFO:tasks.workunit.client.1.smithi138.stdout:6/711: chown dc/d2e/d37/d56 5295 1 2024-04-25T01:36:10.898 INFO:tasks.workunit.client.1.smithi138.stdout:6/712: getdents dc/d10/dc8 0 2024-04-25T01:36:10.903 INFO:tasks.workunit.client.1.smithi138.stdout:6/713: getdents dc/d7b 0 2024-04-25T01:36:10.908 INFO:tasks.workunit.client.1.smithi138.stdout:6/714: rename dc/d2e/d55/c7a to dc/d8d/dca/ce1 0 2024-04-25T01:36:10.911 INFO:tasks.workunit.client.1.smithi138.stdout:6/715: truncate dc/d2e/d37/d56/fa8 639407 0 2024-04-25T01:36:10.912 INFO:tasks.workunit.client.1.smithi138.stdout:6/716: chown dc/d8d/f9b 0 1 2024-04-25T01:36:10.924 INFO:tasks.workunit.client.1.smithi138.stdout:6/717: truncate dc/d10/d28/d80/f82 638866 0 2024-04-25T01:36:10.943 INFO:tasks.workunit.client.1.smithi138.stdout:6/718: dread dc/f60 [0,4194304] 0 2024-04-25T01:36:10.950 INFO:tasks.workunit.client.1.smithi138.stdout:6/719: dwrite dc/d10/d28/d80/d89/da0/fbf [0,4194304] 0 2024-04-25T01:36:10.955 INFO:tasks.workunit.client.1.smithi138.stdout:6/720: link dc/f66 dc/d8d/fe2 0 2024-04-25T01:36:10.956 INFO:tasks.workunit.client.1.smithi138.stdout:6/721: dread - dc/d2e/db9/fd8 zero size 2024-04-25T01:36:10.958 INFO:tasks.workunit.client.1.smithi138.stdout:6/722: mkdir dc/d2e/d55/d81/d91/de3 0 2024-04-25T01:36:10.960 INFO:tasks.workunit.client.1.smithi138.stdout:6/723: mkdir dc/d7b/de4 0 2024-04-25T01:36:10.961 INFO:tasks.workunit.client.1.smithi138.stdout:6/724: fdatasync dc/d2e/d37/d56/f73 0 2024-04-25T01:36:10.963 INFO:tasks.workunit.client.1.smithi138.stdout:6/725: creat dc/d10/d67/fe5 x:0 0 0 2024-04-25T01:36:10.969 INFO:tasks.workunit.client.1.smithi138.stdout:6/726: rename dc/d10/d28/f34 to dc/d10/d28/fe6 0 2024-04-25T01:36:10.973 INFO:tasks.workunit.client.1.smithi138.stdout:6/727: mkdir dc/d2e/d55/de7 0 2024-04-25T01:36:10.973 INFO:tasks.workunit.client.1.smithi138.stdout:6/728: stat dc/d10/d28/d70/fa3 0 2024-04-25T01:36:10.978 INFO:tasks.workunit.client.1.smithi138.stdout:6/729: rmdir dc/d2e/d55/de7 0 2024-04-25T01:36:10.988 INFO:tasks.workunit.client.1.smithi138.stdout:6/730: dwrite dc/f35 [0,4194304] 0 2024-04-25T01:36:10.991 INFO:tasks.workunit.client.1.smithi138.stdout:6/731: read dc/d2e/d37/d56/f73 [2318438,69156] 0 2024-04-25T01:36:10.994 INFO:tasks.workunit.client.1.smithi138.stdout:6/732: mknod dc/d8d/ce8 0 2024-04-25T01:36:10.998 INFO:tasks.workunit.client.1.smithi138.stdout:6/733: link dc/d10/l57 dc/d10/d28/d80/d89/le9 0 2024-04-25T01:36:10.998 INFO:tasks.workunit.client.1.smithi138.stdout:6/734: chown dc/d2e/db9/fd8 1 1 2024-04-25T01:36:10.999 INFO:tasks.workunit.client.1.smithi138.stdout:4/489: sync 2024-04-25T01:36:11.000 INFO:tasks.workunit.client.1.smithi138.stdout:6/735: rmdir dc/d10/d28/d80/d89/da0 39 2024-04-25T01:36:11.003 INFO:tasks.workunit.client.1.smithi138.stdout:4/490: rename d0/d1c/d52/c83 to d0/d59/c87 0 2024-04-25T01:36:11.003 INFO:tasks.workunit.client.1.smithi138.stdout:4/491: chown d0/d45/f35 33063461 1 2024-04-25T01:36:11.004 INFO:tasks.workunit.client.1.smithi138.stdout:6/736: mknod dc/d2e/d55/d7e/cea 0 2024-04-25T01:36:11.004 INFO:tasks.workunit.client.1.smithi138.stdout:4/492: stat d0/d1c/d1e/c57 0 2024-04-25T01:36:11.005 INFO:tasks.workunit.client.1.smithi138.stdout:4/493: write d0/d16/f79 [861746,39904] 0 2024-04-25T01:36:11.005 INFO:tasks.workunit.client.1.smithi138.stdout:6/737: creat dc/d10/d67/d8e/feb x:0 0 0 2024-04-25T01:36:11.006 INFO:tasks.workunit.client.1.smithi138.stdout:4/494: rename d0/d45 to d0/d45/d7f/d88 22 2024-04-25T01:36:11.007 INFO:tasks.workunit.client.1.smithi138.stdout:6/738: read - dc/d2e/d37/d56/d8c/db7/fcb zero size 2024-04-25T01:36:11.008 INFO:tasks.workunit.client.1.smithi138.stdout:6/739: fdatasync dc/d10/d67/d8e/feb 0 2024-04-25T01:36:11.009 INFO:tasks.workunit.client.1.smithi138.stdout:4/495: creat d0/d16/d33/f89 x:0 0 0 2024-04-25T01:36:11.012 INFO:tasks.workunit.client.1.smithi138.stdout:4/496: dread d0/d16/f43 [0,4194304] 0 2024-04-25T01:36:11.017 INFO:tasks.workunit.client.1.smithi138.stdout:6/740: truncate dc/d10/d67/d8e/daf/fdd 892732 0 2024-04-25T01:36:11.018 INFO:tasks.workunit.client.1.smithi138.stdout:4/497: dwrite d0/d16/d4a/d74/f7a [0,4194304] 0 2024-04-25T01:36:11.020 INFO:tasks.workunit.client.1.smithi138.stdout:6/741: mkdir dc/d10/d28/dec 0 2024-04-25T01:36:11.022 INFO:tasks.workunit.client.1.smithi138.stdout:4/498: creat d0/d45/d7f/f8a x:0 0 0 2024-04-25T01:36:11.023 INFO:tasks.workunit.client.1.smithi138.stdout:6/742: creat dc/d8d/fed x:0 0 0 2024-04-25T01:36:11.024 INFO:tasks.workunit.client.1.smithi138.stdout:6/743: stat dc/d7b/de4 0 2024-04-25T01:36:11.024 INFO:tasks.workunit.client.1.smithi138.stdout:6/744: readlink dc/d2e/d55/d81/lb5 0 2024-04-25T01:36:11.025 INFO:tasks.workunit.client.1.smithi138.stdout:4/499: creat d0/d1c/d52/f8b x:0 0 0 2024-04-25T01:36:11.026 INFO:tasks.workunit.client.1.smithi138.stdout:6/745: fdatasync dc/d10/d6d/f7c 0 2024-04-25T01:36:11.044 INFO:tasks.workunit.client.1.smithi138.stdout:6/746: dread dc/d2e/f31 [0,4194304] 0 2024-04-25T01:36:11.104 INFO:tasks.workunit.client.1.smithi138.stdout:2/822: sync 2024-04-25T01:36:11.104 INFO:tasks.workunit.client.1.smithi138.stdout:7/704: sync 2024-04-25T01:36:11.104 INFO:tasks.workunit.client.1.smithi138.stdout:0/477: sync 2024-04-25T01:36:11.104 INFO:tasks.workunit.client.1.smithi138.stdout:3/708: sync 2024-04-25T01:36:11.104 INFO:tasks.workunit.client.1.smithi138.stdout:3/709: fdatasync db/df/d19/d1a/f44 0 2024-04-25T01:36:11.105 INFO:tasks.workunit.client.1.smithi138.stdout:4/500: write d0/d16/d4a/f4d [636478,70730] 0 2024-04-25T01:36:11.124 INFO:tasks.workunit.client.1.smithi138.stdout:6/747: mkdir dc/d2e/d37/dee 0 2024-04-25T01:36:11.124 INFO:tasks.workunit.client.1.smithi138.stdout:0/478: dread - d6/d39/d43/f72 zero size 2024-04-25T01:36:11.126 INFO:tasks.workunit.client.1.smithi138.stdout:6/748: truncate dc/d2e/d55/d7e/fae 9182782 0 2024-04-25T01:36:11.126 INFO:tasks.workunit.client.1.smithi138.stdout:2/823: mknod d0/d85/da5/db4/c103 0 2024-04-25T01:36:11.134 INFO:tasks.workunit.client.1.smithi138.stdout:0/479: mknod d6/d2f/c92 0 2024-04-25T01:36:11.137 INFO:tasks.workunit.client.1.smithi138.stdout:0/480: read d6/df/d3a/f3d [7298750,85602] 0 2024-04-25T01:36:11.148 INFO:tasks.workunit.client.1.smithi138.stdout:0/481: dwrite d6/f1e [0,4194304] 0 2024-04-25T01:36:11.149 INFO:tasks.workunit.client.1.smithi138.stdout:2/824: dread d0/d21/d45/d8a/d8e/fd6 [0,4194304] 0 2024-04-25T01:36:11.156 INFO:tasks.workunit.client.1.smithi138.stdout:4/501: write d0/d45/f35 [1309689,82905] 0 2024-04-25T01:36:11.160 INFO:tasks.workunit.client.1.smithi138.stdout:6/749: read dc/d10/d67/d8e/daf/fdd [180959,9401] 0 2024-04-25T01:36:11.165 INFO:tasks.workunit.client.1.smithi138.stdout:0/482: rename d6/d19/l30 to d6/d2f/l93 0 2024-04-25T01:36:11.165 INFO:tasks.workunit.client.1.smithi138.stdout:6/750: write dc/d7b/fb0 [2927294,75281] 0 2024-04-25T01:36:11.165 INFO:tasks.workunit.client.1.smithi138.stdout:3/710: dwrite db/df/d17/f9f [4194304,4194304] 0 2024-04-25T01:36:11.167 INFO:tasks.workunit.client.1.smithi138.stdout:2/825: dwrite d0/d21/d45/faf [0,4194304] 0 2024-04-25T01:36:11.169 INFO:tasks.workunit.client.1.smithi138.stdout:4/502: fsync d0/d1c/f6b 0 2024-04-25T01:36:11.189 INFO:tasks.workunit.client.1.smithi138.stdout:0/483: unlink d6/df/f14 0 2024-04-25T01:36:11.189 INFO:tasks.workunit.client.1.smithi138.stdout:6/751: creat dc/d10/d67/d8e/daf/fef x:0 0 0 2024-04-25T01:36:11.189 INFO:tasks.workunit.client.1.smithi138.stdout:0/484: chown d6/d2f 18274318 1 2024-04-25T01:36:11.203 INFO:tasks.workunit.client.1.smithi138.stdout:2/826: mknod d0/d85/da5/db4/d15/d44/d4c/d56/da2/c104 0 2024-04-25T01:36:11.210 INFO:tasks.workunit.client.1.smithi138.stdout:0/485: creat d6/d1d/d82/f94 x:0 0 0 2024-04-25T01:36:11.211 INFO:tasks.workunit.client.1.smithi138.stdout:3/711: link db/df/d17/d5d/f7f db/df/d19/dd4/fe0 0 2024-04-25T01:36:11.214 INFO:tasks.workunit.client.1.smithi138.stdout:6/752: creat dc/d2e/ff0 x:0 0 0 2024-04-25T01:36:11.217 INFO:tasks.workunit.client.1.smithi138.stdout:2/827: dwrite d0/d21/d45/d8a/fc9 [0,4194304] 0 2024-04-25T01:36:11.217 INFO:tasks.workunit.client.1.smithi138.stdout:6/753: write dc/fa1 [3934292,82314] 0 2024-04-25T01:36:11.218 INFO:tasks.workunit.client.1.smithi138.stdout:3/712: chown db/df/d17/d23/d34/f80 29207 1 2024-04-25T01:36:11.219 INFO:tasks.workunit.client.1.smithi138.stdout:4/503: dwrite d0/d1c/f2b [8388608,4194304] 0 2024-04-25T01:36:11.224 INFO:tasks.workunit.client.1.smithi138.stdout:6/754: fdatasync dc/d10/d28/d70/fa9 0 2024-04-25T01:36:11.230 INFO:tasks.workunit.client.1.smithi138.stdout:2/828: creat d0/d85/f105 x:0 0 0 2024-04-25T01:36:11.232 INFO:tasks.workunit.client.1.smithi138.stdout:4/504: fdatasync d0/d16/d33/f51 0 2024-04-25T01:36:11.235 INFO:tasks.workunit.client.1.smithi138.stdout:3/713: rename db/df/d17/d20/d57/daa/laf to db/le1 0 2024-04-25T01:36:11.236 INFO:tasks.workunit.client.1.smithi138.stdout:6/755: symlink dc/d10/lf1 0 2024-04-25T01:36:11.239 INFO:tasks.workunit.client.1.smithi138.stdout:4/505: creat d0/d1c/d48/f8c x:0 0 0 2024-04-25T01:36:11.245 INFO:tasks.workunit.client.1.smithi138.stdout:6/756: unlink dc/d10/d28/c63 0 2024-04-25T01:36:11.247 INFO:tasks.workunit.client.1.smithi138.stdout:0/486: truncate d6/f45 2967131 0 2024-04-25T01:36:11.248 INFO:tasks.workunit.client.1.smithi138.stdout:4/506: dwrite d0/f3f [0,4194304] 0 2024-04-25T01:36:11.255 INFO:tasks.workunit.client.1.smithi138.stdout:0/487: rename d6/d39/d43/c84 to d6/d2f/d31/c95 0 2024-04-25T01:36:11.257 INFO:tasks.workunit.client.1.smithi138.stdout:2/829: truncate d0/f10 3234300 0 2024-04-25T01:36:11.258 INFO:tasks.workunit.client.1.smithi138.stdout:4/507: unlink d0/f70 0 2024-04-25T01:36:11.259 INFO:tasks.workunit.client.1.smithi138.stdout:3/714: truncate db/f35 818454 0 2024-04-25T01:36:11.266 INFO:tasks.workunit.client.1.smithi138.stdout:4/508: unlink d0/d45/d7f/c84 0 2024-04-25T01:36:11.266 INFO:tasks.workunit.client.1.smithi138.stdout:2/830: dwrite d0/d21/f3a [0,4194304] 0 2024-04-25T01:36:11.266 INFO:tasks.workunit.client.1.smithi138.stdout:3/715: rmdir db/d14 39 2024-04-25T01:36:11.276 INFO:tasks.workunit.client.1.smithi138.stdout:4/509: dwrite d0/d16/f31 [4194304,4194304] 0 2024-04-25T01:36:11.277 INFO:tasks.workunit.client.1.smithi138.stdout:3/716: chown db/df/d17/d20/l3e 60 1 2024-04-25T01:36:11.280 INFO:tasks.workunit.client.1.smithi138.stdout:4/510: chown d0/d16/f29 3928764 1 2024-04-25T01:36:11.280 INFO:tasks.workunit.client.1.smithi138.stdout:2/831: mkdir d0/d85/da5/db4/d15/d44/d4c/d56/d106 0 2024-04-25T01:36:11.283 INFO:tasks.workunit.client.1.smithi138.stdout:3/717: mkdir db/df/de2 0 2024-04-25T01:36:11.287 INFO:tasks.workunit.client.1.smithi138.stdout:4/511: link d0/d1c/f28 d0/d16/d4a/d74/f8d 0 2024-04-25T01:36:11.287 INFO:tasks.workunit.client.1.smithi138.stdout:4/512: stat d0/d16 0 2024-04-25T01:36:11.289 INFO:tasks.workunit.client.1.smithi138.stdout:0/488: dwrite d6/df/f5c [0,4194304] 0 2024-04-25T01:36:11.296 INFO:tasks.workunit.client.1.smithi138.stdout:2/832: link d0/d85/da5/db4/d15/d44/d4c/d56/fb0 d0/d85/da5/db4/d15/d44/d4c/d56/da2/f107 0 2024-04-25T01:36:11.297 INFO:tasks.workunit.client.1.smithi138.stdout:3/718: creat db/df/d17/d23/fe3 x:0 0 0 2024-04-25T01:36:11.298 INFO:tasks.workunit.client.1.smithi138.stdout:0/489: symlink d6/d2f/d75/l96 0 2024-04-25T01:36:11.298 INFO:tasks.workunit.client.1.smithi138.stdout:4/513: dwrite d0/d1c/d48/f5b [0,4194304] 0 2024-04-25T01:36:11.303 INFO:tasks.workunit.client.1.smithi138.stdout:0/490: symlink d6/df/d69/l97 0 2024-04-25T01:36:11.308 INFO:tasks.workunit.client.1.smithi138.stdout:2/833: creat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/f108 x:0 0 0 2024-04-25T01:36:11.308 INFO:tasks.workunit.client.1.smithi138.stdout:3/719: dwrite db/df/d17/d20/d57/fc1 [0,4194304] 0 2024-04-25T01:36:11.310 INFO:tasks.workunit.client.1.smithi138.stdout:4/514: dwrite d0/d16/f29 [4194304,4194304] 0 2024-04-25T01:36:11.313 INFO:tasks.workunit.client.1.smithi138.stdout:2/834: mkdir d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d109 0 2024-04-25T01:36:11.315 INFO:tasks.workunit.client.1.smithi138.stdout:4/515: creat d0/d45/f8e x:0 0 0 2024-04-25T01:36:11.318 INFO:tasks.workunit.client.1.smithi138.stdout:3/720: dread db/df/d17/d23/d34/f80 [0,4194304] 0 2024-04-25T01:36:11.320 INFO:tasks.workunit.client.1.smithi138.stdout:3/721: readlink db/df/l85 0 2024-04-25T01:36:11.322 INFO:tasks.workunit.client.1.smithi138.stdout:2/835: getdents d0/d85/da5/db4/d15/d44/d4c/d56 0 2024-04-25T01:36:11.323 INFO:tasks.workunit.client.1.smithi138.stdout:4/516: dwrite d0/d16/f79 [0,4194304] 0 2024-04-25T01:36:11.324 INFO:tasks.workunit.client.1.smithi138.stdout:3/722: chown db/d9b/fb7 54637 1 2024-04-25T01:36:11.325 INFO:tasks.workunit.client.1.smithi138.stdout:0/491: read d6/fe [1801589,38887] 0 2024-04-25T01:36:11.327 INFO:tasks.workunit.client.1.smithi138.stdout:4/517: write d0/d1c/d1e/f69 [1535579,56969] 0 2024-04-25T01:36:11.329 INFO:tasks.workunit.client.1.smithi138.stdout:1/602: sync 2024-04-25T01:36:11.329 INFO:tasks.workunit.client.1.smithi138.stdout:9/585: sync 2024-04-25T01:36:11.329 INFO:tasks.workunit.client.1.smithi138.stdout:8/778: sync 2024-04-25T01:36:11.334 INFO:tasks.workunit.client.1.smithi138.stdout:3/723: getdents db/df/d19/d1a/dc4 0 2024-04-25T01:36:11.334 INFO:tasks.workunit.client.1.smithi138.stdout:2/836: read d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/f6c [2914154,82738] 0 2024-04-25T01:36:11.342 INFO:tasks.workunit.client.1.smithi138.stdout:0/492: rename d6/d19/d57 to d6/d1d/d98 0 2024-04-25T01:36:11.344 INFO:tasks.workunit.client.1.smithi138.stdout:9/586: getdents d4/d1a/d3c 0 2024-04-25T01:36:11.345 INFO:tasks.workunit.client.1.smithi138.stdout:3/724: dwrite db/df/d17/d23/d88/fc3 [0,4194304] 0 2024-04-25T01:36:11.345 INFO:tasks.workunit.client.1.smithi138.stdout:2/837: mknod d0/d85/da5/db4/d15/d51/c10a 0 2024-04-25T01:36:11.347 INFO:tasks.workunit.client.1.smithi138.stdout:9/587: dread - d4/d8/d90/f87 zero size 2024-04-25T01:36:11.351 INFO:tasks.workunit.client.1.smithi138.stdout:3/725: dread - db/df/d17/d23/d34/f99 zero size 2024-04-25T01:36:11.353 INFO:tasks.workunit.client.1.smithi138.stdout:9/588: creat d4/d8/d2c/da3/fbf x:0 0 0 2024-04-25T01:36:11.354 INFO:tasks.workunit.client.1.smithi138.stdout:0/493: symlink d6/l99 0 2024-04-25T01:36:11.355 INFO:tasks.workunit.client.1.smithi138.stdout:8/779: dread d3/d16/d88/da/db/d15/f21 [0,4194304] 0 2024-04-25T01:36:11.356 INFO:tasks.workunit.client.1.smithi138.stdout:1/603: dread d3/d45/d72/f52 [0,4194304] 0 2024-04-25T01:36:11.356 INFO:tasks.workunit.client.1.smithi138.stdout:2/838: mkdir d0/d21/d10b 0 2024-04-25T01:36:11.358 INFO:tasks.workunit.client.1.smithi138.stdout:3/726: symlink db/df/d17/d5d/le4 0 2024-04-25T01:36:11.359 INFO:tasks.workunit.client.1.smithi138.stdout:1/604: write d3/d45/d72/d57/d8e/fbe [520411,43506] 0 2024-04-25T01:36:11.359 INFO:tasks.workunit.client.1.smithi138.stdout:9/589: creat d4/d8/d90/d52/fc0 x:0 0 0 2024-04-25T01:36:11.360 INFO:tasks.workunit.client.1.smithi138.stdout:3/727: stat db/df/d17/d23/d88/ldb 0 2024-04-25T01:36:11.362 INFO:tasks.workunit.client.1.smithi138.stdout:8/780: rmdir d3/d16/d88/da/de 39 2024-04-25T01:36:11.362 INFO:tasks.workunit.client.1.smithi138.stdout:0/494: read d6/d2f/d75/f76 [204310,47924] 0 2024-04-25T01:36:11.362 INFO:tasks.workunit.client.1.smithi138.stdout:1/605: write d3/d45/d72/d57/d8e/f69 [1113080,38472] 0 2024-04-25T01:36:11.364 INFO:tasks.workunit.client.1.smithi138.stdout:1/606: dread - d3/db/fb9 zero size 2024-04-25T01:36:11.368 INFO:tasks.workunit.client.1.smithi138.stdout:9/590: mkdir d4/d1a/dc1 0 2024-04-25T01:36:11.370 INFO:tasks.workunit.client.1.smithi138.stdout:3/728: rmdir db/d14/d40 39 2024-04-25T01:36:11.372 INFO:tasks.workunit.client.1.smithi138.stdout:2/839: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b/fa4 [4194304,4194304] 0 2024-04-25T01:36:11.372 INFO:tasks.workunit.client.1.smithi138.stdout:8/781: creat d3/d16/d1f/d2a/d4f/dac/ff2 x:0 0 0 2024-04-25T01:36:11.375 INFO:tasks.workunit.client.1.smithi138.stdout:3/729: mkdir db/df/d19/d1a/de5 0 2024-04-25T01:36:11.376 INFO:tasks.workunit.client.1.smithi138.stdout:9/591: rename d4/d8/d2c/d82/db9/d94 to d4/d1a/d3c/dc2 0 2024-04-25T01:36:11.378 INFO:tasks.workunit.client.1.smithi138.stdout:9/592: readlink d4/l2b 0 2024-04-25T01:36:11.378 INFO:tasks.workunit.client.1.smithi138.stdout:2/840: mknod d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5/c10c 0 2024-04-25T01:36:11.378 INFO:tasks.workunit.client.1.smithi138.stdout:8/782: write d3/d16/d88/da/db/d15/d56/f86 [2707497,46393] 0 2024-04-25T01:36:11.379 INFO:tasks.workunit.client.1.smithi138.stdout:3/730: unlink db/d14/d40/cc9 0 2024-04-25T01:36:11.382 INFO:tasks.workunit.client.1.smithi138.stdout:1/607: dwrite d3/f2c [4194304,4194304] 0 2024-04-25T01:36:11.382 INFO:tasks.workunit.client.1.smithi138.stdout:2/841: chown d0/d85/da5/db4/f3c 6 1 2024-04-25T01:36:11.385 INFO:tasks.workunit.client.1.smithi138.stdout:9/593: dread d4/d1a/f20 [0,4194304] 0 2024-04-25T01:36:11.385 INFO:tasks.workunit.client.1.smithi138.stdout:8/783: write d3/d16/d88/da/de/f36 [815862,108160] 0 2024-04-25T01:36:11.386 INFO:tasks.workunit.client.1.smithi138.stdout:3/731: mknod db/df/dd1/ce6 0 2024-04-25T01:36:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:36:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:36:11.388 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: pgmap v21: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 93 MiB/s rd, 273 MiB/s wr, 558 op/s 2024-04-25T01:36:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.389 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:11 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.389 INFO:tasks.workunit.client.1.smithi138.stdout:2/842: mkdir d0/d85/da5/db4/d15/d44/d4c/d10d 0 2024-04-25T01:36:11.391 INFO:tasks.workunit.client.1.smithi138.stdout:2/843: write d0/d85/da5/db4/d15/d44/d4c/fbb [124518,101884] 0 2024-04-25T01:36:11.394 INFO:tasks.workunit.client.1.smithi138.stdout:1/608: mknod d3/d4/d47/d9b/ccb 0 2024-04-25T01:36:11.397 INFO:tasks.workunit.client.1.smithi138.stdout:9/594: symlink d4/d8/d90/lc3 0 2024-04-25T01:36:11.400 INFO:tasks.workunit.client.1.smithi138.stdout:3/732: mkdir db/df/d19/d1a/de5/de7 0 2024-04-25T01:36:11.401 INFO:tasks.workunit.client.1.smithi138.stdout:1/609: mkdir d3/dcc 0 2024-04-25T01:36:11.404 INFO:tasks.workunit.client.1.smithi138.stdout:8/784: truncate f0 1258557 0 2024-04-25T01:36:11.404 INFO:tasks.workunit.client.1.smithi138.stdout:9/595: rmdir d4/d1a/d55/d7f/d8f 39 2024-04-25T01:36:11.406 INFO:tasks.workunit.client.1.smithi138.stdout:1/610: rmdir d3/d4/da7/db1 39 2024-04-25T01:36:11.409 INFO:tasks.workunit.client.1.smithi138.stdout:1/611: write d3/d27/d3e/fbf [3388313,77949] 0 2024-04-25T01:36:11.411 INFO:tasks.workunit.client.1.smithi138.stdout:3/733: getdents db 0 2024-04-25T01:36:11.412 INFO:tasks.workunit.client.1.smithi138.stdout:1/612: stat d3/d27/d16/cba 0 2024-04-25T01:36:11.414 INFO:tasks.workunit.client.1.smithi138.stdout:8/785: dread d3/d16/d88/da/db/d15/f21 [0,4194304] 0 2024-04-25T01:36:11.418 INFO:tasks.workunit.client.1.smithi138.stdout:9/596: dwrite d4/d1a/f84 [0,4194304] 0 2024-04-25T01:36:11.418 INFO:tasks.workunit.client.1.smithi138.stdout:2/844: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/f6c [0,4194304] 0 2024-04-25T01:36:11.423 INFO:tasks.workunit.client.1.smithi138.stdout:1/613: mkdir d3/d45/d72/d57/db0/dcd 0 2024-04-25T01:36:11.423 INFO:tasks.workunit.client.1.smithi138.stdout:9/597: rmdir d4/d1a/d33 39 2024-04-25T01:36:11.425 INFO:tasks.workunit.client.1.smithi138.stdout:3/734: write db/df/d17/d23/d88/fa4 [669315,41031] 0 2024-04-25T01:36:11.428 INFO:tasks.workunit.client.1.smithi138.stdout:2/845: dread d0/d85/da5/db4/d15/d44/d4c/d56/fb8 [0,4194304] 0 2024-04-25T01:36:11.430 INFO:tasks.workunit.client.1.smithi138.stdout:8/786: rename d3/d16/d88/da/de/f36 to d3/d16/d1f/dd2/dd6/dd7/ff3 0 2024-04-25T01:36:11.432 INFO:tasks.workunit.client.1.smithi138.stdout:9/598: unlink d4/d1a/d55/l77 0 2024-04-25T01:36:11.433 INFO:tasks.workunit.client.1.smithi138.stdout:8/787: read - d3/d44/dae/fd4 zero size 2024-04-25T01:36:11.434 INFO:tasks.workunit.client.1.smithi138.stdout:1/614: dwrite d3/d27/d3e/fbf [8388608,4194304] 0 2024-04-25T01:36:11.434 INFO:tasks.workunit.client.1.smithi138.stdout:3/735: unlink db/df/d17/d20/f8f 0 2024-04-25T01:36:11.436 INFO:tasks.workunit.client.1.smithi138.stdout:1/615: chown d3/d6d/l70 4070641 1 2024-04-25T01:36:11.437 INFO:tasks.workunit.client.1.smithi138.stdout:1/616: chown d3/d27/d16/c33 7009 1 2024-04-25T01:36:11.438 INFO:tasks.workunit.client.1.smithi138.stdout:1/617: readlink d3/d27/d2d/l31 0 2024-04-25T01:36:11.439 INFO:tasks.workunit.client.1.smithi138.stdout:3/736: dread db/df/d17/d23/d34/f80 [0,4194304] 0 2024-04-25T01:36:11.441 INFO:tasks.workunit.client.1.smithi138.stdout:8/788: mknod d3/d16/d88/da/db/dc9/cf4 0 2024-04-25T01:36:11.441 INFO:tasks.workunit.client.1.smithi138.stdout:1/618: write d3/d27/d3e/fbf [5811455,75375] 0 2024-04-25T01:36:11.445 INFO:tasks.workunit.client.1.smithi138.stdout:9/599: creat d4/d63/d73/fc4 x:0 0 0 2024-04-25T01:36:11.449 INFO:tasks.workunit.client.1.smithi138.stdout:0/495: read d6/df/d69/f6a [635777,110409] 0 2024-04-25T01:36:11.451 INFO:tasks.workunit.client.1.smithi138.stdout:2/846: getdents d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3 0 2024-04-25T01:36:11.451 INFO:tasks.workunit.client.1.smithi138.stdout:3/737: creat db/df/d19/dd6/fe8 x:0 0 0 2024-04-25T01:36:11.455 INFO:tasks.workunit.client.1.smithi138.stdout:8/789: link d3/d16/d88/da/db/d15/d59/fb6 d3/d16/d1f/d31/d82/dd5/ff5 0 2024-04-25T01:36:11.463 INFO:tasks.workunit.client.1.smithi138.stdout:2/847: fsync d0/d85/da5/db4/d15/d44/d4c/d56/da2/f26 0 2024-04-25T01:36:11.465 INFO:tasks.workunit.client.1.smithi138.stdout:9/600: dread d4/d1a/d33/d1d/f31 [0,4194304] 0 2024-04-25T01:36:11.465 INFO:tasks.workunit.client.1.smithi138.stdout:8/790: readlink d3/l17 0 2024-04-25T01:36:11.466 INFO:tasks.workunit.client.1.smithi138.stdout:1/619: dwrite d3/d27/d3e/d90/f78 [0,4194304] 0 2024-04-25T01:36:11.467 INFO:tasks.workunit.client.1.smithi138.stdout:2/848: chown d0/d85/da5/db4/d15/d51/c6a 309867 1 2024-04-25T01:36:11.468 INFO:tasks.workunit.client.1.smithi138.stdout:1/620: stat d3/d27/d16/f18 0 2024-04-25T01:36:11.468 INFO:tasks.workunit.client.1.smithi138.stdout:9/601: chown d4/d8/d2c/da3/fba 6985 1 2024-04-25T01:36:11.469 INFO:tasks.workunit.client.1.smithi138.stdout:1/621: chown d3/f34 21 1 2024-04-25T01:36:11.470 INFO:tasks.workunit.client.1.smithi138.stdout:9/602: dread - d4/d63/d73/fc4 zero size 2024-04-25T01:36:11.472 INFO:tasks.workunit.client.1.smithi138.stdout:0/496: dread d6/d19/f77 [4194304,4194304] 0 2024-04-25T01:36:11.473 INFO:tasks.workunit.client.1.smithi138.stdout:2/849: dread d0/d85/da5/db4/f1a [8388608,4194304] 0 2024-04-25T01:36:11.474 INFO:tasks.workunit.client.1.smithi138.stdout:1/622: symlink d3/d45/d72/d57/db0/lce 0 2024-04-25T01:36:11.476 INFO:tasks.workunit.client.1.smithi138.stdout:1/623: write d3/d4/d53/f8c [3552485,9402] 0 2024-04-25T01:36:11.478 INFO:tasks.workunit.client.1.smithi138.stdout:9/603: mknod d4/d1a/d33/d1d/cc5 0 2024-04-25T01:36:11.483 INFO:tasks.workunit.client.1.smithi138.stdout:0/497: truncate d6/f5e 1000792 0 2024-04-25T01:36:11.485 INFO:tasks.workunit.client.1.smithi138.stdout:2/850: dwrite d0/d85/da5/db4/d15/d44/d4c/fe8 [0,4194304] 0 2024-04-25T01:36:11.486 INFO:tasks.workunit.client.1.smithi138.stdout:1/624: symlink d3/d27/d3e/d51/lcf 0 2024-04-25T01:36:11.488 INFO:tasks.workunit.client.1.smithi138.stdout:8/791: link d3/d16/d88/da/db/d15/d56/d94/lb2 d3/d16/d88/da/db/d15/d59/lf6 0 2024-04-25T01:36:11.490 INFO:tasks.workunit.client.1.smithi138.stdout:1/625: write d3/d4/d47/fc2 [831939,34953] 0 2024-04-25T01:36:11.492 INFO:tasks.workunit.client.1.smithi138.stdout:1/626: chown d3/d27/d2d/c2e 8 1 2024-04-25T01:36:11.493 INFO:tasks.workunit.client.1.smithi138.stdout:0/498: truncate d6/d2f/d31/f86 1213695 0 2024-04-25T01:36:11.493 INFO:tasks.workunit.client.1.smithi138.stdout:1/627: fdatasync d3/d6d/f89 0 2024-04-25T01:36:11.496 INFO:tasks.workunit.client.1.smithi138.stdout:1/628: truncate d3/d45/d72/d57/f91 238711 0 2024-04-25T01:36:11.496 INFO:tasks.workunit.client.1.smithi138.stdout:9/604: creat d4/d1a/d33/fc6 x:0 0 0 2024-04-25T01:36:11.498 INFO:tasks.workunit.client.1.smithi138.stdout:1/629: fsync d3/d27/d2d/f32 0 2024-04-25T01:36:11.502 INFO:tasks.workunit.client.1.smithi138.stdout:9/605: creat d4/d1a/d33/d9a/fc7 x:0 0 0 2024-04-25T01:36:11.505 INFO:tasks.workunit.client.1.smithi138.stdout:2/851: dread d0/de9/ffd [0,4194304] 0 2024-04-25T01:36:11.505 INFO:tasks.workunit.client.1.smithi138.stdout:3/738: dwrite db/df/d17/d20/f7d [0,4194304] 0 2024-04-25T01:36:11.507 INFO:tasks.workunit.client.1.smithi138.stdout:1/630: rename d3/d45/d72/d57/f8a to d3/d45/d72/d57/db0/dcd/fd0 0 2024-04-25T01:36:11.507 INFO:tasks.workunit.client.1.smithi138.stdout:3/739: chown db/df/d19/f93 15 1 2024-04-25T01:36:11.517 INFO:tasks.workunit.client.1.smithi138.stdout:1/631: creat d3/d45/fd1 x:0 0 0 2024-04-25T01:36:11.517 INFO:tasks.workunit.client.1.smithi138.stdout:8/792: write f0 [1775796,57366] 0 2024-04-25T01:36:11.518 INFO:tasks.workunit.client.1.smithi138.stdout:1/632: stat d3/d45/d72/d57/f91 0 2024-04-25T01:36:11.520 INFO:tasks.workunit.client.1.smithi138.stdout:1/633: chown d3/db/fb9 327 1 2024-04-25T01:36:11.524 INFO:tasks.workunit.client.1.smithi138.stdout:0/499: dread d6/d19/f77 [0,4194304] 0 2024-04-25T01:36:11.527 INFO:tasks.workunit.client.1.smithi138.stdout:0/500: chown d6/d39/d43/c3e 3119026 1 2024-04-25T01:36:11.530 INFO:tasks.workunit.client.1.smithi138.stdout:2/852: dread d0/d21/f27 [0,4194304] 0 2024-04-25T01:36:11.531 INFO:tasks.workunit.client.1.smithi138.stdout:9/606: truncate d4/d1a/f84 1407835 0 2024-04-25T01:36:11.531 INFO:tasks.workunit.client.1.smithi138.stdout:1/634: dwrite d3/d27/d16/da3/daf/fc6 [0,4194304] 0 2024-04-25T01:36:11.534 INFO:tasks.workunit.client.1.smithi138.stdout:7/705: sync 2024-04-25T01:36:11.534 INFO:tasks.workunit.client.1.smithi138.stdout:5/617: sync 2024-04-25T01:36:11.536 INFO:tasks.workunit.client.1.smithi138.stdout:2/853: truncate d0/f10 145010 0 2024-04-25T01:36:11.538 INFO:tasks.workunit.client.1.smithi138.stdout:1/635: rename d3/db/f97 to d3/d27/d16/da3/fd2 0 2024-04-25T01:36:11.538 INFO:tasks.workunit.client.1.smithi138.stdout:2/854: read d0/f8 [4432412,18713] 0 2024-04-25T01:36:11.540 INFO:tasks.workunit.client.1.smithi138.stdout:1/636: stat d3/d27/d16/da3/fd2 0 2024-04-25T01:36:11.541 INFO:tasks.workunit.client.1.smithi138.stdout:7/706: creat da/de/d13/d21/dbc/dca/fe9 x:0 0 0 2024-04-25T01:36:11.542 INFO:tasks.workunit.client.1.smithi138.stdout:1/637: fsync d3/d45/d72/d57/f63 0 2024-04-25T01:36:11.542 INFO:tasks.workunit.client.1.smithi138.stdout:5/618: symlink d4/da/d27/lcf 0 2024-04-25T01:36:11.543 INFO:tasks.workunit.client.1.smithi138.stdout:3/740: dread db/df/d17/d23/d34/f52 [0,4194304] 0 2024-04-25T01:36:11.545 INFO:tasks.workunit.client.1.smithi138.stdout:8/793: dread d3/d16/d88/fab [0,4194304] 0 2024-04-25T01:36:11.545 INFO:tasks.workunit.client.1.smithi138.stdout:1/638: chown d3/d45/d72/d57/ca5 2741250 1 2024-04-25T01:36:11.545 INFO:tasks.workunit.client.1.smithi138.stdout:7/707: chown da/de/d13/d21/dbc/fbe 23 1 2024-04-25T01:36:11.546 INFO:tasks.workunit.client.1.smithi138.stdout:2/855: symlink d0/d85/da5/db4/d15/d44/l10e 0 2024-04-25T01:36:11.546 INFO:tasks.workunit.client.1.smithi138.stdout:1/639: stat d3/d45/d72/d3d 0 2024-04-25T01:36:11.548 INFO:tasks.workunit.client.1.smithi138.stdout:0/501: dread d6/d1d/f24 [0,4194304] 0 2024-04-25T01:36:11.548 INFO:tasks.workunit.client.1.smithi138.stdout:3/741: mkdir db/df/d17/d23/de9 0 2024-04-25T01:36:11.550 INFO:tasks.workunit.client.1.smithi138.stdout:5/619: creat d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/fd0 x:0 0 0 2024-04-25T01:36:11.550 INFO:tasks.workunit.client.1.smithi138.stdout:7/708: creat da/fea x:0 0 0 2024-04-25T01:36:11.576 INFO:tasks.workunit.client.1.smithi138.stdout:0/502: creat d6/d39/d43/f9a x:0 0 0 2024-04-25T01:36:11.577 INFO:tasks.workunit.client.1.smithi138.stdout:5/620: truncate d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/fd0 685562 0 2024-04-25T01:36:11.577 INFO:tasks.workunit.client.1.smithi138.stdout:7/709: read da/de/d1b/d49/d81/f4c [249604,55598] 0 2024-04-25T01:36:11.579 INFO:tasks.workunit.client.1.smithi138.stdout:8/794: unlink d3/d16/d88/da/db/d15/d5d/c79 0 2024-04-25T01:36:11.581 INFO:tasks.workunit.client.1.smithi138.stdout:3/742: fdatasync db/df/d19/f70 0 2024-04-25T01:36:11.603 INFO:tasks.workunit.client.1.smithi138.stdout:2/856: dwrite d0/d21/d45/d8a/d8e/fd6 [0,4194304] 0 2024-04-25T01:36:11.605 INFO:tasks.workunit.client.1.smithi138.stdout:5/621: dread d4/da/d6e/d22/d35/d4d/f66 [0,4194304] 0 2024-04-25T01:36:11.605 INFO:tasks.workunit.client.1.smithi138.stdout:7/710: mknod da/de/d13/d30/d59/ceb 0 2024-04-25T01:36:11.609 INFO:tasks.workunit.client.1.smithi138.stdout:0/503: mknod d6/d19/c9b 0 2024-04-25T01:36:11.614 INFO:tasks.workunit.client.1.smithi138.stdout:7/711: mknod da/de/d1b/d49/d81/d51/d6e/da2/cec 0 2024-04-25T01:36:11.615 INFO:tasks.workunit.client.1.smithi138.stdout:7/712: write da/de/d1b/d49/d81/f9c [549120,54238] 0 2024-04-25T01:36:11.616 INFO:tasks.workunit.client.1.smithi138.stdout:2/857: mkdir d0/de9/df0/d10f 0 2024-04-25T01:36:11.619 INFO:tasks.workunit.client.1.smithi138.stdout:0/504: mkdir d6/d39/d52/d9c 0 2024-04-25T01:36:11.619 INFO:tasks.workunit.client.1.smithi138.stdout:7/713: write da/de/d13/d30/d59/d7b/d99/fb1 [1046547,5572] 0 2024-04-25T01:36:11.621 INFO:tasks.workunit.client.1.smithi138.stdout:2/858: stat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/cc0 0 2024-04-25T01:36:11.621 INFO:tasks.workunit.client.1.smithi138.stdout:3/743: truncate db/df/d17/d23/d34/f50 2660666 0 2024-04-25T01:36:11.622 INFO:tasks.workunit.client.1.smithi138.stdout:7/714: stat da/de/d13/d21/dbc/d94/d9d 0 2024-04-25T01:36:11.622 INFO:tasks.workunit.client.1.smithi138.stdout:1/640: truncate d3/d45/d72/d57/db0/dcd/fd0 386179 0 2024-04-25T01:36:11.625 INFO:tasks.workunit.client.1.smithi138.stdout:5/622: rename d4/l11 to d4/da/d6e/d22/d35/d4d/d5e/d8f/ld1 0 2024-04-25T01:36:11.627 INFO:tasks.workunit.client.1.smithi138.stdout:5/623: write d4/f6c [4067145,88711] 0 2024-04-25T01:36:11.627 INFO:tasks.workunit.client.1.smithi138.stdout:8/795: write d3/d16/d1f/d50/f7c [1288061,120862] 0 2024-04-25T01:36:11.629 INFO:tasks.workunit.client.1.smithi138.stdout:3/744: rmdir db/dc5 39 2024-04-25T01:36:11.630 INFO:tasks.workunit.client.1.smithi138.stdout:5/624: write d4/da/d6e/d22/d35/d4d/d5e/f75 [615424,129609] 0 2024-04-25T01:36:11.631 INFO:tasks.workunit.client.1.smithi138.stdout:2/859: symlink d0/d85/da5/db4/l110 0 2024-04-25T01:36:11.636 INFO:tasks.workunit.client.1.smithi138.stdout:0/505: dread d6/d2f/d31/f86 [0,4194304] 0 2024-04-25T01:36:11.639 INFO:tasks.workunit.client.1.smithi138.stdout:8/796: dwrite d3/d16/d1f/d31/d82/dd5/fe3 [0,4194304] 0 2024-04-25T01:36:11.639 INFO:tasks.workunit.client.1.smithi138.stdout:2/860: chown d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b/le3 28 1 2024-04-25T01:36:11.645 INFO:tasks.workunit.client.1.smithi138.stdout:0/506: readlink d6/d19/l25 0 2024-04-25T01:36:11.645 INFO:tasks.workunit.client.1.smithi138.stdout:7/715: write da/de/d13/d30/d59/fd6 [3211193,8892] 0 2024-04-25T01:36:11.647 INFO:tasks.workunit.client.1.smithi138.stdout:5/625: rename d4/da/d27/l9b to d4/da/d6e/d22/d74/d59/d7f/db3/ld2 0 2024-04-25T01:36:11.648 INFO:tasks.workunit.client.1.smithi138.stdout:7/716: chown da/de/d1b/d49/cde 832 1 2024-04-25T01:36:11.650 INFO:tasks.workunit.client.1.smithi138.stdout:7/717: chown da/de/d13/d30/l5f 0 1 2024-04-25T01:36:11.653 INFO:tasks.workunit.client.1.smithi138.stdout:3/745: write db/d14/d40/d41/fdd [810477,24594] 0 2024-04-25T01:36:11.653 INFO:tasks.workunit.client.1.smithi138.stdout:1/641: truncate d3/d27/d16/f18 347845 0 2024-04-25T01:36:11.654 INFO:tasks.workunit.client.1.smithi138.stdout:0/507: dwrite d6/d1d/d82/f94 [0,4194304] 0 2024-04-25T01:36:11.657 INFO:tasks.workunit.client.1.smithi138.stdout:2/861: mkdir d0/d85/da5/db4/d15/d44/d4c/d10d/d111 0 2024-04-25T01:36:11.658 INFO:tasks.workunit.client.1.smithi138.stdout:7/718: read da/de/d1b/d49/d81/f4c [2571443,68818] 0 2024-04-25T01:36:11.663 INFO:tasks.workunit.client.1.smithi138.stdout:1/642: mkdir d3/d4/d47/dd3 0 2024-04-25T01:36:11.665 INFO:tasks.workunit.client.1.smithi138.stdout:8/797: link d3/d16/d88/f4d d3/d16/d88/da/db/d15/d5d/ff7 0 2024-04-25T01:36:11.667 INFO:tasks.workunit.client.1.smithi138.stdout:5/626: dwrite d4/f18 [0,4194304] 0 2024-04-25T01:36:11.670 INFO:tasks.workunit.client.1.smithi138.stdout:0/508: dwrite d6/df/f4d [0,4194304] 0 2024-04-25T01:36:11.674 INFO:tasks.workunit.client.1.smithi138.stdout:0/509: stat d6/fe 0 2024-04-25T01:36:11.674 INFO:tasks.workunit.client.1.smithi138.stdout:1/643: truncate d3/d4/d47/d9b/f98 216046 0 2024-04-25T01:36:11.676 INFO:tasks.workunit.client.1.smithi138.stdout:8/798: symlink d3/d16/d1f/d2a/d4f/dcb/dce/lf8 0 2024-04-25T01:36:11.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/644: chown d3/d45/fa0 55043267 1 2024-04-25T01:36:11.679 INFO:tasks.workunit.client.1.smithi138.stdout:1/645: stat d3/d4/d47/d9b/f68 0 2024-04-25T01:36:11.679 INFO:tasks.workunit.client.1.smithi138.stdout:2/862: dwrite d0/d85/da5/db4/f87 [0,4194304] 0 2024-04-25T01:36:11.680 INFO:tasks.workunit.client.1.smithi138.stdout:0/510: creat d6/df/d69/f9d x:0 0 0 2024-04-25T01:36:11.681 INFO:tasks.workunit.client.1.smithi138.stdout:1/646: readlink d3/d27/d16/l3a 0 2024-04-25T01:36:11.694 INFO:tasks.workunit.client.1.smithi138.stdout:1/647: creat d3/d27/d3e/fd4 x:0 0 0 2024-04-25T01:36:11.695 INFO:tasks.workunit.client.1.smithi138.stdout:3/746: write db/d9b/fb7 [141606,82429] 0 2024-04-25T01:36:11.695 INFO:tasks.workunit.client.1.smithi138.stdout:7/719: truncate da/f6a 57317 0 2024-04-25T01:36:11.697 INFO:tasks.workunit.client.1.smithi138.stdout:7/720: stat da/de/d13/d30/d59/d7b/d99/lb4 0 2024-04-25T01:36:11.698 INFO:tasks.workunit.client.1.smithi138.stdout:1/648: unlink d3/d4/d47/d9b/ccb 0 2024-04-25T01:36:11.699 INFO:tasks.workunit.client.1.smithi138.stdout:3/747: dread - db/df/d17/d23/d34/f99 zero size 2024-04-25T01:36:11.700 INFO:tasks.workunit.client.1.smithi138.stdout:8/799: write d3/d16/d88/da/de/f10 [5858197,47929] 0 2024-04-25T01:36:11.703 INFO:tasks.workunit.client.1.smithi138.stdout:7/721: creat da/de/d13/fed x:0 0 0 2024-04-25T01:36:11.703 INFO:tasks.workunit.client.1.smithi138.stdout:8/800: readlink d3/d16/d1f/d2a/d4f/l5c 0 2024-04-25T01:36:11.706 INFO:tasks.workunit.client.1.smithi138.stdout:2/863: truncate d0/d85/da5/db4/d15/d44/d4c/d56/fb8 121645 0 2024-04-25T01:36:11.706 INFO:tasks.workunit.client.1.smithi138.stdout:0/511: write d6/d2f/d75/f76 [178332,80840] 0 2024-04-25T01:36:11.707 INFO:tasks.workunit.client.1.smithi138.stdout:8/801: write d3/d16/d1f/d31/d82/dd5/fe3 [5180598,37543] 0 2024-04-25T01:36:11.707 INFO:tasks.workunit.client.1.smithi138.stdout:7/722: rmdir da/de/d1b 39 2024-04-25T01:36:11.709 INFO:tasks.workunit.client.1.smithi138.stdout:2/864: write d0/d85/da5/db4/d15/d44/d4c/fbb [1352226,117802] 0 2024-04-25T01:36:11.712 INFO:tasks.workunit.client.1.smithi138.stdout:0/512: mknod d6/df/d41/d49/c9e 0 2024-04-25T01:36:11.712 INFO:tasks.workunit.client.1.smithi138.stdout:5/627: dread d4/f62 [0,4194304] 0 2024-04-25T01:36:11.717 INFO:tasks.workunit.client.1.smithi138.stdout:3/748: truncate db/df/d17/d23/d88/fc3 2061039 0 2024-04-25T01:36:11.717 INFO:tasks.workunit.client.1.smithi138.stdout:7/723: read da/de/d13/d21/f41 [2543543,520] 0 2024-04-25T01:36:11.719 INFO:tasks.workunit.client.1.smithi138.stdout:0/513: creat d6/d2f/d75/f9f x:0 0 0 2024-04-25T01:36:11.722 INFO:tasks.workunit.client.1.smithi138.stdout:5/628: dwrite d4/da/d6e/d22/d35/d4d/d5e/f7a [0,4194304] 0 2024-04-25T01:36:11.725 INFO:tasks.workunit.client.1.smithi138.stdout:3/749: symlink db/df/d17/d23/d34/lea 0 2024-04-25T01:36:11.726 INFO:tasks.workunit.client.1.smithi138.stdout:8/802: creat d3/d16/d88/da/ff9 x:0 0 0 2024-04-25T01:36:11.729 INFO:tasks.workunit.client.1.smithi138.stdout:5/629: rmdir d4/da/d6e/d22/d74/d59 39 2024-04-25T01:36:11.731 INFO:tasks.workunit.client.1.smithi138.stdout:0/514: symlink d6/la0 0 2024-04-25T01:36:11.738 INFO:tasks.workunit.client.1.smithi138.stdout:3/750: rename db/d14/d40/d41/f89 to db/df/d19/d1a/d9e/feb 0 2024-04-25T01:36:11.738 INFO:tasks.workunit.client.1.smithi138.stdout:2/865: dread d0/d85/da5/db4/d15/d44/d4c/d56/da2/f26 [4194304,4194304] 0 2024-04-25T01:36:11.740 INFO:tasks.workunit.client.1.smithi138.stdout:2/866: chown d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e 7360 1 2024-04-25T01:36:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:36:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:36:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: pgmap v21: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 93 MiB/s rd, 273 MiB/s wr, 558 op/s 2024-04-25T01:36:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:36:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:11 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:11.744 INFO:tasks.workunit.client.1.smithi138.stdout:8/803: rmdir d3/d16/d88/da 39 2024-04-25T01:36:11.748 INFO:tasks.workunit.client.1.smithi138.stdout:5/630: symlink d4/da/d6e/ld3 0 2024-04-25T01:36:11.750 INFO:tasks.workunit.client.1.smithi138.stdout:3/751: chown db/df/d19/d1a/d9e/feb 3958037 1 2024-04-25T01:36:11.752 INFO:tasks.workunit.client.1.smithi138.stdout:0/515: write d6/d1d/f3c [3977506,81508] 0 2024-04-25T01:36:11.753 INFO:tasks.workunit.client.1.smithi138.stdout:2/867: symlink d0/d85/da5/db4/d15/d44/d4c/d56/d106/l112 0 2024-04-25T01:36:11.755 INFO:tasks.workunit.client.1.smithi138.stdout:7/724: getdents da/de/d1b/d49/d81 0 2024-04-25T01:36:11.757 INFO:tasks.workunit.client.1.smithi138.stdout:8/804: symlink d3/d16/d88/da/db/d15/d56/d94/lfa 0 2024-04-25T01:36:11.761 INFO:tasks.workunit.client.1.smithi138.stdout:3/752: creat db/df/de2/fec x:0 0 0 2024-04-25T01:36:11.761 INFO:tasks.workunit.client.1.smithi138.stdout:7/725: mknod da/dd2/cee 0 2024-04-25T01:36:11.765 INFO:tasks.workunit.client.1.smithi138.stdout:5/631: dread d4/da/d6e/d22/d74/d59/f99 [0,4194304] 0 2024-04-25T01:36:11.766 INFO:tasks.workunit.client.1.smithi138.stdout:8/805: dwrite d3/d16/d88/da/db/d15/d56/d94/f9f [0,4194304] 0 2024-04-25T01:36:11.767 INFO:tasks.workunit.client.1.smithi138.stdout:0/516: rename d6/fc to d6/d2f/fa1 0 2024-04-25T01:36:11.774 INFO:tasks.workunit.client.1.smithi138.stdout:5/632: mkdir d4/da/d6e/d22/d74/d59/dd4 0 2024-04-25T01:36:11.774 INFO:tasks.workunit.client.1.smithi138.stdout:8/806: stat d3/d16/d1f/d2a/d4f/dcb/dce/lf8 0 2024-04-25T01:36:11.776 INFO:tasks.workunit.client.1.smithi138.stdout:2/868: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/fb0 [0,4194304] 0 2024-04-25T01:36:11.779 INFO:tasks.workunit.client.1.smithi138.stdout:3/753: dwrite db/d14/fc8 [4194304,4194304] 0 2024-04-25T01:36:11.782 INFO:tasks.workunit.client.1.smithi138.stdout:0/517: rename d6/d39/d43/f90 to d6/d39/fa2 0 2024-04-25T01:36:11.782 INFO:tasks.workunit.client.1.smithi138.stdout:5/633: mkdir d4/da/d6e/d3c/dd5 0 2024-04-25T01:36:11.783 INFO:tasks.workunit.client.1.smithi138.stdout:8/807: stat d3/d16/d88/da/db/d15/d20/l6c 0 2024-04-25T01:36:11.784 INFO:tasks.workunit.client.1.smithi138.stdout:3/754: creat db/df/de2/fed x:0 0 0 2024-04-25T01:36:11.785 INFO:tasks.workunit.client.1.smithi138.stdout:0/518: read - d6/d1d/d98/f83 zero size 2024-04-25T01:36:11.787 INFO:tasks.workunit.client.1.smithi138.stdout:5/634: unlink d4/da/fa0 0 2024-04-25T01:36:11.788 INFO:tasks.workunit.client.1.smithi138.stdout:0/519: chown d6/d39/d52/c68 30301523 1 2024-04-25T01:36:11.789 INFO:tasks.workunit.client.1.smithi138.stdout:2/869: rename d0/d85/da5/db4/d15/d51/f72 to d0/d21/d45/de1/f113 0 2024-04-25T01:36:11.792 INFO:tasks.workunit.client.1.smithi138.stdout:3/755: truncate db/df/d17/d20/f74 651058 0 2024-04-25T01:36:11.793 INFO:tasks.workunit.client.1.smithi138.stdout:0/520: rename d6/d2f/d31/c53 to d6/d39/d52/ca3 0 2024-04-25T01:36:11.794 INFO:tasks.workunit.client.1.smithi138.stdout:0/521: chown d6/d39/f70 209987 1 2024-04-25T01:36:11.797 INFO:tasks.workunit.client.1.smithi138.stdout:8/808: link f0 d3/d16/d88/da/db/d15/ffb 0 2024-04-25T01:36:11.799 INFO:tasks.workunit.client.1.smithi138.stdout:5/635: dread d4/da/d6e/d22/f98 [0,4194304] 0 2024-04-25T01:36:11.799 INFO:tasks.workunit.client.1.smithi138.stdout:5/636: readlink d4/da/l2c 0 2024-04-25T01:36:11.800 INFO:tasks.workunit.client.1.smithi138.stdout:3/756: rename db/f69 to db/d14/d40/d41/fee 0 2024-04-25T01:36:11.801 INFO:tasks.workunit.client.1.smithi138.stdout:8/809: creat d3/d16/d1f/d31/ffc x:0 0 0 2024-04-25T01:36:11.803 INFO:tasks.workunit.client.1.smithi138.stdout:8/810: write d3/d16/d1f/d2a/d4f/dac/ff2 [943859,88511] 0 2024-04-25T01:36:11.805 INFO:tasks.workunit.client.1.smithi138.stdout:5/637: truncate d4/da/f20 433166 0 2024-04-25T01:36:11.805 INFO:tasks.workunit.client.1.smithi138.stdout:8/811: write d3/d16/d1f/f61 [2381911,103013] 0 2024-04-25T01:36:11.807 INFO:tasks.workunit.client.1.smithi138.stdout:0/522: getdents d6/df/d69 0 2024-04-25T01:36:11.808 INFO:tasks.workunit.client.1.smithi138.stdout:8/812: chown d3/d16/d88/da/db/fa6 22012115 1 2024-04-25T01:36:11.809 INFO:tasks.workunit.client.1.smithi138.stdout:8/813: write d3/d16/f3b [1218690,1887] 0 2024-04-25T01:36:11.810 INFO:tasks.workunit.client.1.smithi138.stdout:5/638: creat d4/da/d6e/d22/d35/d4d/dca/fd6 x:0 0 0 2024-04-25T01:36:11.814 INFO:tasks.workunit.client.1.smithi138.stdout:5/639: write d4/da/d6e/d22/d74/d59/f68 [984153,3073] 0 2024-04-25T01:36:11.817 INFO:tasks.workunit.client.1.smithi138.stdout:0/523: dwrite d6/d39/d43/f9a [0,4194304] 0 2024-04-25T01:36:11.819 INFO:tasks.workunit.client.1.smithi138.stdout:5/640: rmdir d4/da/d6e/d22/d74/d59/d83 39 2024-04-25T01:36:11.823 INFO:tasks.workunit.client.1.smithi138.stdout:5/641: write d4/da/d27/f2d [786790,59659] 0 2024-04-25T01:36:11.825 INFO:tasks.workunit.client.1.smithi138.stdout:8/814: link d3/d16/d88/da/db/d15/f2f d3/d16/d88/da/db/d15/d6f/ffd 0 2024-04-25T01:36:11.828 INFO:tasks.workunit.client.1.smithi138.stdout:2/870: truncate d0/d85/da5/db4/d15/d44/d4c/d56/fb0 2389799 0 2024-04-25T01:36:11.829 INFO:tasks.workunit.client.1.smithi138.stdout:2/871: stat d0/d21/d45/de1/f113 0 2024-04-25T01:36:11.831 INFO:tasks.workunit.client.1.smithi138.stdout:5/642: rename d4/da/d52 to d4/da/d6e/d22/d74/d8e/dd7 0 2024-04-25T01:36:11.834 INFO:tasks.workunit.client.1.smithi138.stdout:2/872: mkdir d0/d85/da5/db4/d15/d44/d114 0 2024-04-25T01:36:11.835 INFO:tasks.workunit.client.1.smithi138.stdout:5/643: mkdir d4/da/d6e/d22/d74/d8e/dd8 0 2024-04-25T01:36:11.887 INFO:tasks.workunit.client.1.smithi138.stdout:6/757: sync 2024-04-25T01:36:11.888 INFO:tasks.workunit.client.1.smithi138.stdout:3/757: dwrite db/df/d17/d23/d34/f80 [0,4194304] 0 2024-04-25T01:36:11.888 INFO:tasks.workunit.client.1.smithi138.stdout:2/873: rename d0/d21/d45/faf to d0/d21/d45/f115 0 2024-04-25T01:36:11.894 INFO:tasks.workunit.client.1.smithi138.stdout:5/644: write d4/da/d6e/d22/d74/d59/d83/d89/d9d/d9e/fb1 [175218,9971] 0 2024-04-25T01:36:11.897 INFO:tasks.workunit.client.1.smithi138.stdout:5/645: chown d4/da/d6e/d22/d74/d8e/dd7/f92 5999108 1 2024-04-25T01:36:11.898 INFO:tasks.workunit.client.1.smithi138.stdout:2/874: mknod d0/d85/da5/db4/d15/d44/c116 0 2024-04-25T01:36:11.900 INFO:tasks.workunit.client.1.smithi138.stdout:3/758: dread db/df/d17/d23/d34/f6d [4194304,4194304] 0 2024-04-25T01:36:11.900 INFO:tasks.workunit.client.1.smithi138.stdout:0/524: truncate d6/d1d/f3c 3022169 0 2024-04-25T01:36:11.904 INFO:tasks.workunit.client.1.smithi138.stdout:8/815: write d3/d16/d88/da/db/d15/d59/fb6 [222358,91916] 0 2024-04-25T01:36:11.908 INFO:tasks.workunit.client.1.smithi138.stdout:5/646: unlink d4/da/d6e/d22/d74/d8e/dd7/lbf 0 2024-04-25T01:36:11.909 INFO:tasks.workunit.client.1.smithi138.stdout:8/816: dread d3/d16/d88/da/db/d15/d5d/f9a [0,4194304] 0 2024-04-25T01:36:11.912 INFO:tasks.workunit.client.1.smithi138.stdout:5/647: truncate d4/da/d6e/d22/d74/d59/d83/d8a/f96 360201 0 2024-04-25T01:36:11.914 INFO:tasks.workunit.client.1.smithi138.stdout:0/525: symlink d6/la4 0 2024-04-25T01:36:11.919 INFO:tasks.workunit.client.1.smithi138.stdout:6/758: getdents dc/d10/d67 0 2024-04-25T01:36:11.920 INFO:tasks.workunit.client.1.smithi138.stdout:5/648: creat d4/da/d6e/d3c/dad/fd9 x:0 0 0 2024-04-25T01:36:11.921 INFO:tasks.workunit.client.1.smithi138.stdout:0/526: mknod d6/df/d66/ca5 0 2024-04-25T01:36:11.924 INFO:tasks.workunit.client.1.smithi138.stdout:5/649: write d4/da/d6e/d22/d74/f58 [4207699,17359] 0 2024-04-25T01:36:11.925 INFO:tasks.workunit.client.1.smithi138.stdout:5/650: chown d4/f6c 177031572 1 2024-04-25T01:36:11.926 INFO:tasks.workunit.client.1.smithi138.stdout:2/875: rename d0/f31 to d0/d21/d45/d8a/d8e/f117 0 2024-04-25T01:36:11.927 INFO:tasks.workunit.client.1.smithi138.stdout:6/759: dwrite dc/d10/d28/d70/fab [0,4194304] 0 2024-04-25T01:36:11.929 INFO:tasks.workunit.client.1.smithi138.stdout:0/527: creat d6/d1d/fa6 x:0 0 0 2024-04-25T01:36:11.932 INFO:tasks.workunit.client.1.smithi138.stdout:2/876: fdatasync d0/d85/da5/db4/d15/d44/d4c/fbb 0 2024-04-25T01:36:11.936 INFO:tasks.workunit.client.1.smithi138.stdout:2/877: write d0/f8 [3937358,11132] 0 2024-04-25T01:36:11.937 INFO:tasks.workunit.client.1.smithi138.stdout:3/759: dwrite db/df/d17/d23/fa1 [0,4194304] 0 2024-04-25T01:36:11.937 INFO:tasks.workunit.client.1.smithi138.stdout:6/760: dread - dc/d2e/d37/f52 zero size 2024-04-25T01:36:11.946 INFO:tasks.workunit.client.1.smithi138.stdout:8/817: dwrite d3/d16/d88/da/db/d15/d6f/ffd [0,4194304] 0 2024-04-25T01:36:11.946 INFO:tasks.workunit.client.1.smithi138.stdout:2/878: link d0/d21/f27 d0/d21/d10b/f118 0 2024-04-25T01:36:11.949 INFO:tasks.workunit.client.1.smithi138.stdout:0/528: dwrite d6/d1d/d98/f83 [0,4194304] 0 2024-04-25T01:36:11.952 INFO:tasks.workunit.client.1.smithi138.stdout:6/761: rename dc/d2e/l59 to dc/d2e/d55/d81/lf2 0 2024-04-25T01:36:11.952 INFO:tasks.workunit.client.1.smithi138.stdout:2/879: write d0/d85/da5/db4/d15/d44/d4c/d56/d6d/ff1 [904788,44456] 0 2024-04-25T01:36:11.954 INFO:tasks.workunit.client.1.smithi138.stdout:3/760: dread db/df/d17/d23/fa1 [0,4194304] 0 2024-04-25T01:36:11.956 INFO:tasks.workunit.client.1.smithi138.stdout:6/762: stat dc/d7b/fb0 0 2024-04-25T01:36:11.956 INFO:tasks.workunit.client.1.smithi138.stdout:6/763: read - dc/d2e/ff0 zero size 2024-04-25T01:36:11.957 INFO:tasks.workunit.client.1.smithi138.stdout:0/529: symlink d6/d19/la7 0 2024-04-25T01:36:11.958 INFO:tasks.workunit.client.1.smithi138.stdout:6/764: write dc/d2e/f4c [622247,92682] 0 2024-04-25T01:36:11.958 INFO:tasks.workunit.client.1.smithi138.stdout:3/761: creat db/df/d17/d20/d57/daa/fef x:0 0 0 2024-04-25T01:36:11.961 INFO:tasks.workunit.client.1.smithi138.stdout:8/818: creat d3/ffe x:0 0 0 2024-04-25T01:36:11.965 INFO:tasks.workunit.client.1.smithi138.stdout:2/880: creat d0/d21/f119 x:0 0 0 2024-04-25T01:36:11.967 INFO:tasks.workunit.client.1.smithi138.stdout:6/765: creat dc/d2e/d37/d56/d8c/db7/ff3 x:0 0 0 2024-04-25T01:36:11.968 INFO:tasks.workunit.client.1.smithi138.stdout:5/651: dwrite d4/da/f20 [0,4194304] 0 2024-04-25T01:36:11.972 INFO:tasks.workunit.client.1.smithi138.stdout:0/530: dwrite d6/df/d41/f55 [0,4194304] 0 2024-04-25T01:36:11.972 INFO:tasks.workunit.client.1.smithi138.stdout:8/819: creat d3/d16/d88/da/d70/fff x:0 0 0 2024-04-25T01:36:11.972 INFO:tasks.workunit.client.1.smithi138.stdout:3/762: symlink db/d14/d40/lf0 0 2024-04-25T01:36:11.972 INFO:tasks.workunit.client.1.smithi138.stdout:2/881: stat d0/d85/da5/db4/d15/d44/d4c/d10d/d111 0 2024-04-25T01:36:11.972 INFO:tasks.workunit.client.1.smithi138.stdout:6/766: chown dc/d10/d28/d70/fdf 8 1 2024-04-25T01:36:11.977 INFO:tasks.workunit.client.1.smithi138.stdout:3/763: mkdir db/d9b/db4/df1 0 2024-04-25T01:36:11.977 INFO:tasks.workunit.client.1.smithi138.stdout:6/767: fsync dc/d10/d28/d80/f8b 0 2024-04-25T01:36:11.980 INFO:tasks.workunit.client.1.smithi138.stdout:2/882: truncate d0/d21/d45/d8a/d8e/fa9 34765 0 2024-04-25T01:36:11.981 INFO:tasks.workunit.client.1.smithi138.stdout:0/531: creat d6/df/d41/fa8 x:0 0 0 2024-04-25T01:36:11.984 INFO:tasks.workunit.client.1.smithi138.stdout:3/764: mkdir db/df/d19/dd4/df2 0 2024-04-25T01:36:11.989 INFO:tasks.workunit.client.1.smithi138.stdout:5/652: dwrite d4/da/d6e/d22/d74/f73 [0,4194304] 0 2024-04-25T01:36:11.990 INFO:tasks.workunit.client.1.smithi138.stdout:8/820: dwrite d3/d16/f3b [0,4194304] 0 2024-04-25T01:36:11.990 INFO:tasks.workunit.client.1.smithi138.stdout:0/532: mknod d6/df/d3a/d67/d8a/ca9 0 2024-04-25T01:36:11.991 INFO:tasks.workunit.client.1.smithi138.stdout:2/883: rmdir d0/d85/da5/db4/d15/d44/d4c/d56 39 2024-04-25T01:36:11.995 INFO:tasks.workunit.client.1.smithi138.stdout:2/884: fsync d0/d21/f59 0 2024-04-25T01:36:12.001 INFO:tasks.workunit.client.1.smithi138.stdout:6/768: dwrite dc/d10/d28/f2b [0,4194304] 0 2024-04-25T01:36:12.004 INFO:tasks.workunit.client.1.smithi138.stdout:8/821: getdents d3/d16/d1f/d87 0 2024-04-25T01:36:12.006 INFO:tasks.workunit.client.1.smithi138.stdout:6/769: fdatasync dc/d10/d67/f65 0 2024-04-25T01:36:12.006 INFO:tasks.workunit.client.1.smithi138.stdout:0/533: dread d6/f9 [0,4194304] 0 2024-04-25T01:36:12.007 INFO:tasks.workunit.client.1.smithi138.stdout:2/885: creat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5/df5/f11a x:0 0 0 2024-04-25T01:36:12.009 INFO:tasks.workunit.client.1.smithi138.stdout:8/822: chown d3/d16/d88/da/db/d15/f83 78 1 2024-04-25T01:36:12.012 INFO:tasks.workunit.client.1.smithi138.stdout:5/653: dwrite d4/da/d6e/d22/d74/d59/d83/d8a/fa7 [0,4194304] 0 2024-04-25T01:36:12.014 INFO:tasks.workunit.client.1.smithi138.stdout:8/823: write d3/d16/d88/da/f24 [1487088,94226] 0 2024-04-25T01:36:12.014 INFO:tasks.workunit.client.1.smithi138.stdout:6/770: symlink dc/d2e/d37/lf4 0 2024-04-25T01:36:12.021 INFO:tasks.workunit.client.1.smithi138.stdout:2/886: mknod d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/c11b 0 2024-04-25T01:36:12.024 INFO:tasks.workunit.client.1.smithi138.stdout:8/824: fdatasync d3/d16/d88/da/f43 0 2024-04-25T01:36:12.026 INFO:tasks.workunit.client.1.smithi138.stdout:8/825: write d3/d16/d88/da/d70/fff [275400,36204] 0 2024-04-25T01:36:12.026 INFO:tasks.workunit.client.1.smithi138.stdout:3/765: write db/df/d17/d23/d88/fc3 [2484727,39349] 0 2024-04-25T01:36:12.031 INFO:tasks.workunit.client.1.smithi138.stdout:8/826: mkdir d3/d16/d88/da/de/d57/d100 0 2024-04-25T01:36:12.034 INFO:tasks.workunit.client.1.smithi138.stdout:2/887: unlink d0/d85/da5/db4/c19 0 2024-04-25T01:36:12.037 INFO:tasks.workunit.client.1.smithi138.stdout:8/827: fdatasync d3/d16/d88/da/db/d15/f30 0 2024-04-25T01:36:12.038 INFO:tasks.workunit.client.1.smithi138.stdout:0/534: dread d6/f1e [0,4194304] 0 2024-04-25T01:36:12.039 INFO:tasks.workunit.client.1.smithi138.stdout:8/828: write d3/d16/d1f/f61 [324828,61280] 0 2024-04-25T01:36:12.040 INFO:tasks.workunit.client.1.smithi138.stdout:3/766: creat db/df/d17/d23/ff3 x:0 0 0 2024-04-25T01:36:12.041 INFO:tasks.workunit.client.1.smithi138.stdout:5/654: truncate d4/da/d6e/d22/d74/d59/d83/d8a/fa7 1753435 0 2024-04-25T01:36:12.042 INFO:tasks.workunit.client.1.smithi138.stdout:6/771: dread dc/d2e/d55/d81/fb4 [0,4194304] 0 2024-04-25T01:36:12.043 INFO:tasks.workunit.client.1.smithi138.stdout:6/772: chown dc/d8d 425 1 2024-04-25T01:36:12.043 INFO:tasks.workunit.client.1.smithi138.stdout:2/888: creat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5/df5/f11c x:0 0 0 2024-04-25T01:36:12.045 INFO:tasks.workunit.client.1.smithi138.stdout:0/535: creat d6/df/d69/faa x:0 0 0 2024-04-25T01:36:12.046 INFO:tasks.workunit.client.1.smithi138.stdout:5/655: dread d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/fd0 [0,4194304] 0 2024-04-25T01:36:12.048 INFO:tasks.workunit.client.1.smithi138.stdout:5/656: fdatasync d4/da/d6e/d22/d74/d59/f70 0 2024-04-25T01:36:12.049 INFO:tasks.workunit.client.1.smithi138.stdout:6/773: mkdir dc/d8d/df5 0 2024-04-25T01:36:12.050 INFO:tasks.workunit.client.1.smithi138.stdout:3/767: mknod db/d14/cf4 0 2024-04-25T01:36:12.054 INFO:tasks.workunit.client.1.smithi138.stdout:8/829: truncate d3/d16/d88/da/db/d15/d5d/ff7 4342969 0 2024-04-25T01:36:12.056 INFO:tasks.workunit.client.1.smithi138.stdout:5/657: write d4/da/d6e/d22/d74/f79 [2885353,90635] 0 2024-04-25T01:36:12.057 INFO:tasks.workunit.client.1.smithi138.stdout:8/830: write d3/d16/d88/da/db/d15/d56/f9c [2136830,91894] 0 2024-04-25T01:36:12.061 INFO:tasks.workunit.client.1.smithi138.stdout:6/774: creat dc/d10/dc8/ff6 x:0 0 0 2024-04-25T01:36:12.063 INFO:tasks.workunit.client.1.smithi138.stdout:6/775: write dc/d10/d67/d8e/daf/fcc [4350755,3360] 0 2024-04-25T01:36:12.065 INFO:tasks.workunit.client.1.smithi138.stdout:5/658: unlink d4/da/d6e/d22/f7b 0 2024-04-25T01:36:12.065 INFO:tasks.workunit.client.1.smithi138.stdout:6/776: read dc/f60 [3305058,62952] 0 2024-04-25T01:36:12.065 INFO:tasks.workunit.client.1.smithi138.stdout:3/768: mknod db/d9b/db4/df1/cf5 0 2024-04-25T01:36:12.066 INFO:tasks.workunit.client.1.smithi138.stdout:8/831: symlink d3/d16/d1f/d2a/d4f/dac/l101 0 2024-04-25T01:36:12.068 INFO:tasks.workunit.client.1.smithi138.stdout:2/889: getdents d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83 0 2024-04-25T01:36:12.068 INFO:tasks.workunit.client.1.smithi138.stdout:8/832: chown d3/d16/d88/da/db/d15/fb0 0 1 2024-04-25T01:36:12.078 INFO:tasks.workunit.client.1.smithi138.stdout:6/777: symlink dc/d10/d67/d8e/lf7 0 2024-04-25T01:36:12.079 INFO:tasks.workunit.client.1.smithi138.stdout:0/536: truncate d6/df/f10 2769050 0 2024-04-25T01:36:12.080 INFO:tasks.workunit.client.1.smithi138.stdout:8/833: dwrite d3/f1e [0,4194304] 0 2024-04-25T01:36:12.103 INFO:tasks.workunit.client.1.smithi138.stdout:5/659: mkdir d4/da/d6e/d22/d35/dce/dda 0 2024-04-25T01:36:12.105 INFO:tasks.workunit.client.1.smithi138.stdout:6/778: truncate dc/d10/d28/f29 1436773 0 2024-04-25T01:36:12.108 INFO:tasks.workunit.client.1.smithi138.stdout:0/537: rename d6/d39/d43/c3f to d6/d2f/d31/cab 0 2024-04-25T01:36:12.108 INFO:tasks.workunit.client.1.smithi138.stdout:6/779: dread - dc/d10/fa5 zero size 2024-04-25T01:36:12.109 INFO:tasks.workunit.client.1.smithi138.stdout:8/834: truncate d3/f38 2155524 0 2024-04-25T01:36:12.112 INFO:tasks.workunit.client.1.smithi138.stdout:5/660: dwrite d4/f6c [0,4194304] 0 2024-04-25T01:36:12.116 INFO:tasks.workunit.client.1.smithi138.stdout:6/780: unlink dc/d7b/fce 0 2024-04-25T01:36:12.117 INFO:tasks.workunit.client.1.smithi138.stdout:0/538: creat d6/df/d69/fac x:0 0 0 2024-04-25T01:36:12.121 INFO:tasks.workunit.client.1.smithi138.stdout:2/890: dread d0/d21/d45/de1/fe4 [0,4194304] 0 2024-04-25T01:36:12.122 INFO:tasks.workunit.client.1.smithi138.stdout:8/835: unlink d3/d16/d88/da/db/d15/d56/l84 0 2024-04-25T01:36:12.126 INFO:tasks.workunit.client.1.smithi138.stdout:0/539: write d6/f28 [5579509,85052] 0 2024-04-25T01:36:12.131 INFO:tasks.workunit.client.1.smithi138.stdout:8/836: write d3/d16/d88/f40 [2724259,69621] 0 2024-04-25T01:36:12.132 INFO:tasks.workunit.client.1.smithi138.stdout:6/781: mkdir dc/d2e/d55/df8 0 2024-04-25T01:36:12.132 INFO:tasks.workunit.client.1.smithi138.stdout:9/607: sync 2024-04-25T01:36:12.133 INFO:tasks.workunit.client.1.smithi138.stdout:4/518: sync 2024-04-25T01:36:12.133 INFO:tasks.workunit.client.1.smithi138.stdout:0/540: creat d6/d1d/d82/fad x:0 0 0 2024-04-25T01:36:12.133 INFO:tasks.workunit.client.1.smithi138.stdout:4/519: stat d0/d16/d4a/f76 0 2024-04-25T01:36:12.134 INFO:tasks.workunit.client.1.smithi138.stdout:0/541: chown d6/d1d 87418 1 2024-04-25T01:36:12.137 INFO:tasks.workunit.client.1.smithi138.stdout:8/837: fdatasync f0 0 2024-04-25T01:36:12.143 INFO:tasks.workunit.client.1.smithi138.stdout:9/608: rename d4/d8/d90/d52/l91 to d4/d63/lc8 0 2024-04-25T01:36:12.145 INFO:tasks.workunit.client.1.smithi138.stdout:2/891: read d0/d85/da5/db4/d15/d44/d4c/fbb [310196,98743] 0 2024-04-25T01:36:12.146 INFO:tasks.workunit.client.1.smithi138.stdout:8/838: chown d3/dde/ce1 14703587 1 2024-04-25T01:36:12.151 INFO:tasks.workunit.client.1.smithi138.stdout:5/661: dread d4/da/d6e/d22/d74/f85 [0,4194304] 0 2024-04-25T01:36:12.151 INFO:tasks.workunit.client.1.smithi138.stdout:0/542: rename d6/f1e to d6/df/d3a/fae 0 2024-04-25T01:36:12.153 INFO:tasks.workunit.client.1.smithi138.stdout:9/609: mknod d4/d1a/d68/cc9 0 2024-04-25T01:36:12.155 INFO:tasks.workunit.client.1.smithi138.stdout:2/892: dread d0/d85/da5/db4/d15/d44/d4c/d56/fb0 [0,4194304] 0 2024-04-25T01:36:12.156 INFO:tasks.workunit.client.1.smithi138.stdout:8/839: symlink d3/d16/d88/da/db/d15/d6f/l102 0 2024-04-25T01:36:12.159 INFO:tasks.workunit.client.1.smithi138.stdout:9/610: fsync d4/d1a/f20 0 2024-04-25T01:36:12.159 INFO:tasks.workunit.client.1.smithi138.stdout:2/893: chown d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fc6 515536 1 2024-04-25T01:36:12.161 INFO:tasks.workunit.client.1.smithi138.stdout:5/662: creat d4/da/d6e/d22/d35/d4d/d5e/d60/dbd/fdb x:0 0 0 2024-04-25T01:36:12.166 INFO:tasks.workunit.client.1.smithi138.stdout:0/543: link d6/d39/fa2 d6/df/faf 0 2024-04-25T01:36:12.167 INFO:tasks.workunit.client.1.smithi138.stdout:6/782: dread dc/d10/d67/d8e/daf/fcc [0,4194304] 0 2024-04-25T01:36:12.169 INFO:tasks.workunit.client.1.smithi138.stdout:0/544: chown d6/d2f/d31/f86 31506 1 2024-04-25T01:36:12.169 INFO:tasks.workunit.client.1.smithi138.stdout:9/611: mknod d4/d49/d72/cca 0 2024-04-25T01:36:12.170 INFO:tasks.workunit.client.1.smithi138.stdout:4/520: dread d0/f3f [0,4194304] 0 2024-04-25T01:36:12.170 INFO:tasks.workunit.client.1.smithi138.stdout:9/612: readlink d4/d8/d90/lc3 0 2024-04-25T01:36:12.171 INFO:tasks.workunit.client.1.smithi138.stdout:4/521: fdatasync d0/d16/d4a/f4d 0 2024-04-25T01:36:12.172 INFO:tasks.workunit.client.1.smithi138.stdout:9/613: chown d4/d1a/d33/d9a 3058 1 2024-04-25T01:36:12.172 INFO:tasks.workunit.client.1.smithi138.stdout:5/663: creat d4/da/d6e/d3c/fdc x:0 0 0 2024-04-25T01:36:12.176 INFO:tasks.workunit.client.1.smithi138.stdout:0/545: chown d6/d2f/d31/c95 3 1 2024-04-25T01:36:12.180 INFO:tasks.workunit.client.1.smithi138.stdout:3/769: write db/df/d17/d5d/f6b [1058043,96124] 0 2024-04-25T01:36:12.181 INFO:tasks.workunit.client.1.smithi138.stdout:4/522: chown d0/d59/c87 470 1 2024-04-25T01:36:12.185 INFO:tasks.workunit.client.1.smithi138.stdout:3/770: read db/df/d17/d20/f29 [308759,129511] 0 2024-04-25T01:36:12.190 INFO:tasks.workunit.client.1.smithi138.stdout:9/614: creat d4/d63/d86/fcb x:0 0 0 2024-04-25T01:36:12.191 INFO:tasks.workunit.client.1.smithi138.stdout:9/615: dread - d4/d63/d86/fad zero size 2024-04-25T01:36:12.194 INFO:tasks.workunit.client.1.smithi138.stdout:0/546: symlink d6/d1d/lb0 0 2024-04-25T01:36:12.195 INFO:tasks.workunit.client.1.smithi138.stdout:2/894: creat d0/d85/da5/d35/f11d x:0 0 0 2024-04-25T01:36:12.197 INFO:tasks.workunit.client.1.smithi138.stdout:4/523: creat d0/d16/d4a/f8f x:0 0 0 2024-04-25T01:36:12.197 INFO:tasks.workunit.client.1.smithi138.stdout:8/840: link d3/d44/d48/l58 d3/d16/d1f/d2a/d4f/l103 0 2024-04-25T01:36:12.200 INFO:tasks.workunit.client.1.smithi138.stdout:5/664: creat d4/da/d6e/d22/d35/d4d/d5e/d60/db5/fdd x:0 0 0 2024-04-25T01:36:12.201 INFO:tasks.workunit.client.1.smithi138.stdout:8/841: write d3/d16/d88/da/db/f11 [1979117,8563] 0 2024-04-25T01:36:12.202 INFO:tasks.workunit.client.1.smithi138.stdout:3/771: creat db/df/d17/d23/d34/ff6 x:0 0 0 2024-04-25T01:36:12.204 INFO:tasks.workunit.client.1.smithi138.stdout:2/895: dwrite d0/fa1 [0,4194304] 0 2024-04-25T01:36:12.204 INFO:tasks.workunit.client.1.smithi138.stdout:6/783: rmdir dc/d2e/d55/df8 0 2024-04-25T01:36:12.206 INFO:tasks.workunit.client.1.smithi138.stdout:1/649: sync 2024-04-25T01:36:12.206 INFO:tasks.workunit.client.1.smithi138.stdout:9/616: truncate d4/d49/f76 119875 0 2024-04-25T01:36:12.208 INFO:tasks.workunit.client.1.smithi138.stdout:0/547: stat d6/d2f/l93 0 2024-04-25T01:36:12.210 INFO:tasks.workunit.client.1.smithi138.stdout:0/548: chown d6/d39 37375 1 2024-04-25T01:36:12.210 INFO:tasks.workunit.client.1.smithi138.stdout:4/524: rmdir d0/d45/d7f 39 2024-04-25T01:36:12.211 INFO:tasks.workunit.client.1.smithi138.stdout:0/549: readlink d6/d2f/l93 0 2024-04-25T01:36:12.214 INFO:tasks.workunit.client.1.smithi138.stdout:3/772: rmdir db/df/d19 39 2024-04-25T01:36:12.214 INFO:tasks.workunit.client.1.smithi138.stdout:4/525: write d0/d16/d4a/f77 [1041590,41156] 0 2024-04-25T01:36:12.214 INFO:tasks.workunit.client.1.smithi138.stdout:2/896: dwrite d0/d21/fdb [0,4194304] 0 2024-04-25T01:36:12.216 INFO:tasks.workunit.client.1.smithi138.stdout:0/550: truncate d6/df/d69/f9d 909671 0 2024-04-25T01:36:12.217 INFO:tasks.workunit.client.1.smithi138.stdout:5/665: rename d4/da/d6e/d3c/c8b to d4/da/d6e/d3c/dad/cde 0 2024-04-25T01:36:12.218 INFO:tasks.workunit.client.1.smithi138.stdout:6/784: rmdir dc/d10/d28/d80 39 2024-04-25T01:36:12.222 INFO:tasks.workunit.client.1.smithi138.stdout:9/617: unlink d4/d8/d90/f5d 0 2024-04-25T01:36:12.227 INFO:tasks.workunit.client.1.smithi138.stdout:3/773: write db/df/d19/f70 [4230206,9538] 0 2024-04-25T01:36:12.230 INFO:tasks.workunit.client.1.smithi138.stdout:2/897: mknod d0/d85/da5/c11e 0 2024-04-25T01:36:12.231 INFO:tasks.workunit.client.1.smithi138.stdout:5/666: unlink d4/da/lbc 0 2024-04-25T01:36:12.232 INFO:tasks.workunit.client.1.smithi138.stdout:4/526: dwrite d0/f3f [0,4194304] 0 2024-04-25T01:36:12.233 INFO:tasks.workunit.client.1.smithi138.stdout:6/785: mkdir dc/d10/d67/d8e/df9 0 2024-04-25T01:36:12.234 INFO:tasks.workunit.client.1.smithi138.stdout:2/898: write d0/d85/da5/db4/d15/d44/d4c/fe8 [2521134,72517] 0 2024-04-25T01:36:12.239 INFO:tasks.workunit.client.1.smithi138.stdout:9/618: symlink d4/d1a/d3c/dc2/lcc 0 2024-04-25T01:36:12.239 INFO:tasks.workunit.client.1.smithi138.stdout:9/619: readlink d4/d8/l2e 0 2024-04-25T01:36:12.242 INFO:tasks.workunit.client.1.smithi138.stdout:4/527: mknod d0/d1c/d48/c90 0 2024-04-25T01:36:12.246 INFO:tasks.workunit.client.1.smithi138.stdout:5/667: mknod d4/da/d6e/d22/d35/cdf 0 2024-04-25T01:36:12.250 INFO:tasks.workunit.client.1.smithi138.stdout:9/620: unlink d4/d1a/d55/ca6 0 2024-04-25T01:36:12.253 INFO:tasks.workunit.client.1.smithi138.stdout:8/842: dwrite d3/d16/d88/da/db/d15/f72 [0,4194304] 0 2024-04-25T01:36:12.254 INFO:tasks.workunit.client.1.smithi138.stdout:9/621: truncate d4/d1a/d33/f37 4530014 0 2024-04-25T01:36:12.254 INFO:tasks.workunit.client.1.smithi138.stdout:4/528: rename d0/d1c/d48/f8c to d0/d16/d7e/f91 0 2024-04-25T01:36:12.255 INFO:tasks.workunit.client.1.smithi138.stdout:8/843: write d3/d16/d1f/d31/d82/dd5/ff5 [475191,2307] 0 2024-04-25T01:36:12.263 INFO:tasks.workunit.client.1.smithi138.stdout:6/786: write dc/f16 [2028568,98503] 0 2024-04-25T01:36:12.264 INFO:tasks.workunit.client.1.smithi138.stdout:4/529: fsync d0/d45/d7f/f8a 0 2024-04-25T01:36:12.265 INFO:tasks.workunit.client.1.smithi138.stdout:1/650: dwrite d3/d27/d3e/fbf [0,4194304] 0 2024-04-25T01:36:12.266 INFO:tasks.workunit.client.1.smithi138.stdout:8/844: creat d3/d16/d1f/dd2/dd6/f104 x:0 0 0 2024-04-25T01:36:12.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/622: symlink d4/lcd 0 2024-04-25T01:36:12.276 INFO:tasks.workunit.client.1.smithi138.stdout:5/668: dread d4/da/d6e/d22/d74/d59/fc7 [0,4194304] 0 2024-04-25T01:36:12.276 INFO:tasks.workunit.client.1.smithi138.stdout:3/774: dwrite db/f35 [0,4194304] 0 2024-04-25T01:36:12.276 INFO:tasks.workunit.client.1.smithi138.stdout:1/651: fsync d3/d45/f48 0 2024-04-25T01:36:12.277 INFO:tasks.workunit.client.1.smithi138.stdout:4/530: creat d0/d1c/d48/f92 x:0 0 0 2024-04-25T01:36:12.278 INFO:tasks.workunit.client.1.smithi138.stdout:8/845: rename d3/d16/d88/da/d70/fff to d3/d44/dae/f105 0 2024-04-25T01:36:12.279 INFO:tasks.workunit.client.1.smithi138.stdout:9/623: creat d4/d57/fce x:0 0 0 2024-04-25T01:36:12.280 INFO:tasks.workunit.client.1.smithi138.stdout:4/531: dread - d0/d16/d7e/f91 zero size 2024-04-25T01:36:12.281 INFO:tasks.workunit.client.1.smithi138.stdout:1/652: write d3/d27/d3e/d90/f78 [3449763,72603] 0 2024-04-25T01:36:12.284 INFO:tasks.workunit.client.1.smithi138.stdout:5/669: mknod d4/da/d6e/d22/d35/d4d/dca/ce0 0 2024-04-25T01:36:12.284 INFO:tasks.workunit.client.1.smithi138.stdout:9/624: mkdir d4/d63/dcf 0 2024-04-25T01:36:12.284 INFO:tasks.workunit.client.1.smithi138.stdout:1/653: write d3/d45/d72/d57/f63 [9122725,113995] 0 2024-04-25T01:36:12.287 INFO:tasks.workunit.client.1.smithi138.stdout:4/532: read - d0/d16/d33/f89 zero size 2024-04-25T01:36:12.287 INFO:tasks.workunit.client.1.smithi138.stdout:6/787: getdents dc/d10/d28 0 2024-04-25T01:36:12.288 INFO:tasks.workunit.client.1.smithi138.stdout:9/625: chown d4/d49/f6f 4562 1 2024-04-25T01:36:12.289 INFO:tasks.workunit.client.1.smithi138.stdout:5/670: fsync d4/da/d6e/d22/d74/f76 0 2024-04-25T01:36:12.289 INFO:tasks.workunit.client.1.smithi138.stdout:8/846: rename d3/d16/d88/da/d70/lcc to d3/d16/d1f/d50/l106 0 2024-04-25T01:36:12.291 INFO:tasks.workunit.client.1.smithi138.stdout:6/788: creat dc/d10/dc8/ffa x:0 0 0 2024-04-25T01:36:12.293 INFO:tasks.workunit.client.1.smithi138.stdout:1/654: mknod d3/d45/d72/d3d/cd5 0 2024-04-25T01:36:12.299 INFO:tasks.workunit.client.1.smithi138.stdout:6/789: chown dc/d10/d28/d70/f7f 73434 1 2024-04-25T01:36:12.300 INFO:tasks.workunit.client.1.smithi138.stdout:4/533: dwrite d0/d16/d4a/f68 [0,4194304] 0 2024-04-25T01:36:12.304 INFO:tasks.workunit.client.1.smithi138.stdout:1/655: mkdir d3/d4/d47/dd6 0 2024-04-25T01:36:12.304 INFO:tasks.workunit.client.1.smithi138.stdout:5/671: creat d4/da/d6e/d22/d74/d59/d7f/db3/fe1 x:0 0 0 2024-04-25T01:36:12.306 INFO:tasks.workunit.client.1.smithi138.stdout:4/534: fdatasync d0/d16/d33/f89 0 2024-04-25T01:36:12.312 INFO:tasks.workunit.client.1.smithi138.stdout:6/790: chown dc/d10/d28/d80/l86 55 1 2024-04-25T01:36:12.313 INFO:tasks.workunit.client.1.smithi138.stdout:3/775: dread db/df/d19/d1a/f21 [0,4194304] 0 2024-04-25T01:36:12.313 INFO:tasks.workunit.client.1.smithi138.stdout:6/791: write dc/d2e/db9/fbc [1340810,31252] 0 2024-04-25T01:36:12.314 INFO:tasks.workunit.client.1.smithi138.stdout:9/626: dwrite d4/d49/d72/f9b [0,4194304] 0 2024-04-25T01:36:12.318 INFO:tasks.workunit.client.1.smithi138.stdout:8/847: dread d3/d16/d88/da/db/d15/d56/d94/f9f [0,4194304] 0 2024-04-25T01:36:12.321 INFO:tasks.workunit.client.1.smithi138.stdout:4/535: mknod d0/d16/d7e/c93 0 2024-04-25T01:36:12.323 INFO:tasks.workunit.client.1.smithi138.stdout:3/776: rename db/df/d17/f4a to db/df/d19/d1a/d9e/ff7 0 2024-04-25T01:36:12.325 INFO:tasks.workunit.client.1.smithi138.stdout:1/656: rmdir d3/d4/d47/dd3 0 2024-04-25T01:36:12.327 INFO:tasks.workunit.client.1.smithi138.stdout:8/848: chown d3/d16/d88/da/db/d15/d56/d94/cdd 52 1 2024-04-25T01:36:12.328 INFO:tasks.workunit.client.1.smithi138.stdout:6/792: dwrite dc/d2e/d55/f75 [0,4194304] 0 2024-04-25T01:36:12.331 INFO:tasks.workunit.client.1.smithi138.stdout:4/536: symlink d0/d1c/d86/l94 0 2024-04-25T01:36:12.331 INFO:tasks.workunit.client.1.smithi138.stdout:6/793: write dc/d2e/d55/d81/d91/fc2 [106094,75614] 0 2024-04-25T01:36:12.333 INFO:tasks.workunit.client.1.smithi138.stdout:1/657: symlink d3/d27/d2d/ld7 0 2024-04-25T01:36:12.335 INFO:tasks.workunit.client.1.smithi138.stdout:4/537: fdatasync d0/d16/d33/f89 0 2024-04-25T01:36:12.335 INFO:tasks.workunit.client.1.smithi138.stdout:8/849: mknod d3/d16/d88/da/db/d15/d59/c107 0 2024-04-25T01:36:12.344 INFO:tasks.workunit.client.1.smithi138.stdout:3/777: write db/df/d17/d5d/f76 [793834,75749] 0 2024-04-25T01:36:12.346 INFO:tasks.workunit.client.1.smithi138.stdout:6/794: dwrite dc/d10/d28/fe6 [8388608,4194304] 0 2024-04-25T01:36:12.349 INFO:tasks.workunit.client.1.smithi138.stdout:8/850: dwrite d3/d16/d88/da/ff9 [0,4194304] 0 2024-04-25T01:36:12.351 INFO:tasks.workunit.client.1.smithi138.stdout:3/778: symlink db/df/dd1/lf8 0 2024-04-25T01:36:12.351 INFO:tasks.workunit.client.1.smithi138.stdout:5/672: dread d4/da/d6e/d22/d74/f79 [0,4194304] 0 2024-04-25T01:36:12.353 INFO:tasks.workunit.client.1.smithi138.stdout:8/851: rmdir d3/d16/d1f/d31/d82 39 2024-04-25T01:36:12.355 INFO:tasks.workunit.client.1.smithi138.stdout:5/673: readlink d4/da/d6e/d22/d35/d4d/d5e/l71 0 2024-04-25T01:36:12.356 INFO:tasks.workunit.client.1.smithi138.stdout:3/779: mkdir db/d9b/dac/df9 0 2024-04-25T01:36:12.357 INFO:tasks.workunit.client.1.smithi138.stdout:1/658: truncate d3/d4/d47/d9b/f68 3960089 0 2024-04-25T01:36:12.360 INFO:tasks.workunit.client.1.smithi138.stdout:6/795: dwrite dc/d10/d67/d8e/fc0 [0,4194304] 0 2024-04-25T01:36:12.365 INFO:tasks.workunit.client.1.smithi138.stdout:1/659: chown d3/d27/d3e/d51/f75 216978 1 2024-04-25T01:36:12.365 INFO:tasks.workunit.client.1.smithi138.stdout:8/852: rename d3/d16/d88/da/de/d57/fe8 to d3/d16/d88/da/d70/f108 0 2024-04-25T01:36:12.367 INFO:tasks.workunit.client.1.smithi138.stdout:6/796: creat dc/d8d/ffb x:0 0 0 2024-04-25T01:36:12.368 INFO:tasks.workunit.client.1.smithi138.stdout:8/853: stat d3/d16/d88/da/db/d15/d20/c7f 0 2024-04-25T01:36:12.370 INFO:tasks.workunit.client.1.smithi138.stdout:5/674: dwrite d4/f18 [4194304,4194304] 0 2024-04-25T01:36:12.372 INFO:tasks.workunit.client.1.smithi138.stdout:6/797: unlink dc/d2e/db9/cd0 0 2024-04-25T01:36:12.374 INFO:tasks.workunit.client.1.smithi138.stdout:5/675: read d4/da/d6e/d22/d74/d59/f68 [772784,17191] 0 2024-04-25T01:36:12.381 INFO:tasks.workunit.client.1.smithi138.stdout:5/676: fdatasync d4/da/d6e/d22/d45/f54 0 2024-04-25T01:36:12.381 INFO:tasks.workunit.client.1.smithi138.stdout:8/854: dwrite d3/d16/d88/da/db/d15/d59/f97 [0,4194304] 0 2024-04-25T01:36:12.387 INFO:tasks.workunit.client.1.smithi138.stdout:6/798: dwrite dc/d2e/db9/fd8 [0,4194304] 0 2024-04-25T01:36:12.389 INFO:tasks.workunit.client.1.smithi138.stdout:8/855: symlink d3/d16/d88/da/db/d15/d5d/l109 0 2024-04-25T01:36:12.390 INFO:tasks.workunit.client.1.smithi138.stdout:6/799: write fb [2758110,25477] 0 2024-04-25T01:36:12.392 INFO:tasks.workunit.client.1.smithi138.stdout:6/800: read dc/d2e/d55/d81/fb4 [2573232,71049] 0 2024-04-25T01:36:12.395 INFO:tasks.workunit.client.1.smithi138.stdout:8/856: chown d3/d44/d48/c8d 5113638 1 2024-04-25T01:36:12.397 INFO:tasks.workunit.client.1.smithi138.stdout:5/677: link d4/da/d6e/d3c/c43 d4/da/d6e/d22/d74/d59/d83/d89/d9d/ce2 0 2024-04-25T01:36:12.398 INFO:tasks.workunit.client.1.smithi138.stdout:3/780: dread db/df/d19/d1a/f44 [0,4194304] 0 2024-04-25T01:36:12.401 INFO:tasks.workunit.client.1.smithi138.stdout:5/678: write d4/da/d6e/d22/d74/d59/f99 [1245720,44459] 0 2024-04-25T01:36:12.401 INFO:tasks.workunit.client.1.smithi138.stdout:7/726: sync 2024-04-25T01:36:12.405 INFO:tasks.workunit.client.1.smithi138.stdout:3/781: dread db/df/d17/d23/d88/fa4 [0,4194304] 0 2024-04-25T01:36:12.410 INFO:tasks.workunit.client.1.smithi138.stdout:7/727: dwrite da/de/d1b/d49/d81/f9c [0,4194304] 0 2024-04-25T01:36:12.413 INFO:tasks.workunit.client.1.smithi138.stdout:7/728: truncate da/de/d13/d21/dbc/d94/fd8 963748 0 2024-04-25T01:36:12.417 INFO:tasks.workunit.client.1.smithi138.stdout:7/729: creat da/de/d1b/fef x:0 0 0 2024-04-25T01:36:12.417 INFO:tasks.workunit.client.1.smithi138.stdout:8/857: write d3/d16/d88/f60 [6701397,124257] 0 2024-04-25T01:36:12.421 INFO:tasks.workunit.client.1.smithi138.stdout:7/730: mknod da/de/d1b/d4b/d77/cf0 0 2024-04-25T01:36:12.425 INFO:tasks.workunit.client.1.smithi138.stdout:7/731: symlink da/de/d13/d30/d59/d7b/lf1 0 2024-04-25T01:36:12.425 INFO:tasks.workunit.client.1.smithi138.stdout:3/782: dwrite db/d14/d40/d41/fa5 [4194304,4194304] 0 2024-04-25T01:36:12.428 INFO:tasks.workunit.client.1.smithi138.stdout:6/801: dread dc/d10/f76 [0,4194304] 0 2024-04-25T01:36:12.428 INFO:tasks.workunit.client.1.smithi138.stdout:5/679: dread d4/da/d6e/d22/d74/f58 [0,4194304] 0 2024-04-25T01:36:12.428 INFO:tasks.workunit.client.1.smithi138.stdout:8/858: read d3/d16/d1f/d31/d82/f90 [637050,121387] 0 2024-04-25T01:36:12.431 INFO:tasks.workunit.client.1.smithi138.stdout:7/732: mknod da/de/d13/d21/d6f/d8d/cf2 0 2024-04-25T01:36:12.431 INFO:tasks.workunit.client.1.smithi138.stdout:5/680: readlink d4/da/d6e/d22/l4a 0 2024-04-25T01:36:12.432 INFO:tasks.workunit.client.1.smithi138.stdout:6/802: creat dc/d10/dc8/ffc x:0 0 0 2024-04-25T01:36:12.434 INFO:tasks.workunit.client.1.smithi138.stdout:6/803: stat dc/d10/d67/d8e/faa 0 2024-04-25T01:36:12.434 INFO:tasks.workunit.client.1.smithi138.stdout:5/681: write d4/da/d6e/d22/d35/d4d/dca/fd6 [825258,97629] 0 2024-04-25T01:36:12.436 INFO:tasks.workunit.client.1.smithi138.stdout:7/733: creat da/de/d13/d30/d59/ff3 x:0 0 0 2024-04-25T01:36:12.437 INFO:tasks.workunit.client.1.smithi138.stdout:3/783: fsync db/f62 0 2024-04-25T01:36:12.439 INFO:tasks.workunit.client.1.smithi138.stdout:8/859: creat d3/d16/d1f/de2/f10a x:0 0 0 2024-04-25T01:36:12.440 INFO:tasks.workunit.client.1.smithi138.stdout:5/682: mknod d4/da/d6e/d22/d45/ce3 0 2024-04-25T01:36:12.442 INFO:tasks.workunit.client.1.smithi138.stdout:7/734: rmdir da/de/d1b/d4b 39 2024-04-25T01:36:12.442 INFO:tasks.workunit.client.1.smithi138.stdout:3/784: symlink db/df/de2/lfa 0 2024-04-25T01:36:12.445 INFO:tasks.workunit.client.1.smithi138.stdout:5/683: unlink d4/f1a 0 2024-04-25T01:36:12.446 INFO:tasks.workunit.client.1.smithi138.stdout:6/804: dwrite dc/d10/f4b [0,4194304] 0 2024-04-25T01:36:12.449 INFO:tasks.workunit.client.1.smithi138.stdout:8/860: mknod d3/d16/d88/c10b 0 2024-04-25T01:36:12.452 INFO:tasks.workunit.client.1.smithi138.stdout:6/805: fsync dc/d2e/d55/fd4 0 2024-04-25T01:36:12.458 INFO:tasks.workunit.client.1.smithi138.stdout:5/684: rename d4/da/d6e/d22/d74/f79 to d4/da/d6e/d22/d35/d4d/fe4 0 2024-04-25T01:36:12.460 INFO:tasks.workunit.client.1.smithi138.stdout:8/861: dwrite d3/d16/d88/da/db/d15/d20/f26 [0,4194304] 0 2024-04-25T01:36:12.461 INFO:tasks.workunit.client.1.smithi138.stdout:7/735: write da/de/d13/d21/dbc/fb3 [1297030,1555] 0 2024-04-25T01:36:12.462 INFO:tasks.workunit.client.1.smithi138.stdout:5/685: symlink d4/da/d6e/le5 0 2024-04-25T01:36:12.463 INFO:tasks.workunit.client.1.smithi138.stdout:8/862: fsync d3/d16/d88/da/db/fca 0 2024-04-25T01:36:12.467 INFO:tasks.workunit.client.1.smithi138.stdout:7/736: fdatasync da/de/d13/d30/d59/d7b/d99/fd5 0 2024-04-25T01:36:12.469 INFO:tasks.workunit.client.1.smithi138.stdout:5/686: creat d4/da/d6e/d22/d74/d8e/dd7/fe6 x:0 0 0 2024-04-25T01:36:12.475 INFO:tasks.workunit.client.1.smithi138.stdout:6/806: dwrite dc/d10/d67/d8e/faa [0,4194304] 0 2024-04-25T01:36:12.476 INFO:tasks.workunit.client.1.smithi138.stdout:5/687: mknod d4/da/d6e/ce7 0 2024-04-25T01:36:12.478 INFO:tasks.workunit.client.1.smithi138.stdout:8/863: truncate d3/d16/d88/da/db/f96 372242 0 2024-04-25T01:36:12.478 INFO:tasks.workunit.client.1.smithi138.stdout:3/785: dwrite db/d9b/db4/fbf [0,4194304] 0 2024-04-25T01:36:12.481 INFO:tasks.workunit.client.1.smithi138.stdout:3/786: fdatasync db/df/d17/d5d/fa3 0 2024-04-25T01:36:12.482 INFO:tasks.workunit.client.1.smithi138.stdout:7/737: dread da/de/d1b/d49/d81/f9c [0,4194304] 0 2024-04-25T01:36:12.483 INFO:tasks.workunit.client.1.smithi138.stdout:8/864: creat d3/dde/f10c x:0 0 0 2024-04-25T01:36:12.490 INFO:tasks.workunit.client.1.smithi138.stdout:7/738: symlink da/de/d13/d21/d6f/d8d/lf4 0 2024-04-25T01:36:12.496 INFO:tasks.workunit.client.1.smithi138.stdout:5/688: dwrite d4/da/d6e/d22/d35/d4d/f66 [0,4194304] 0 2024-04-25T01:36:12.501 INFO:tasks.workunit.client.1.smithi138.stdout:3/787: dwrite db/df/d17/d23/d34/fd8 [0,4194304] 0 2024-04-25T01:36:12.501 INFO:tasks.workunit.client.1.smithi138.stdout:6/807: dwrite dc/d10/f15 [4194304,4194304] 0 2024-04-25T01:36:12.509 INFO:tasks.workunit.client.1.smithi138.stdout:8/865: dwrite d3/d16/d1f/f61 [0,4194304] 0 2024-04-25T01:36:12.514 INFO:tasks.workunit.client.1.smithi138.stdout:7/739: dwrite da/de/d13/d30/d59/ff3 [0,4194304] 0 2024-04-25T01:36:12.515 INFO:tasks.workunit.client.1.smithi138.stdout:3/788: creat db/df/d19/d1a/ffb x:0 0 0 2024-04-25T01:36:12.517 INFO:tasks.workunit.client.1.smithi138.stdout:6/808: mknod dc/d2e/d37/dee/cfd 0 2024-04-25T01:36:12.520 INFO:tasks.workunit.client.1.smithi138.stdout:8/866: creat d3/dc2/f10d x:0 0 0 2024-04-25T01:36:12.523 INFO:tasks.workunit.client.1.smithi138.stdout:8/867: symlink d3/d44/d48/dbb/l10e 0 2024-04-25T01:36:12.525 INFO:tasks.workunit.client.1.smithi138.stdout:3/789: rename la to db/lfc 0 2024-04-25T01:36:12.528 INFO:tasks.workunit.client.1.smithi138.stdout:6/809: dwrite dc/d2e/d55/d81/d91/fc2 [0,4194304] 0 2024-04-25T01:36:12.530 INFO:tasks.workunit.client.1.smithi138.stdout:3/790: write db/df/d17/d23/d34/ff6 [323489,20306] 0 2024-04-25T01:36:12.530 INFO:tasks.workunit.client.1.smithi138.stdout:8/868: creat d3/d16/d88/da/db/d15/d6f/f10f x:0 0 0 2024-04-25T01:36:12.535 INFO:tasks.workunit.client.1.smithi138.stdout:8/869: dread d3/d16/d88/f4d [4194304,4194304] 0 2024-04-25T01:36:12.536 INFO:tasks.workunit.client.1.smithi138.stdout:3/791: truncate db/df/d17/d20/f29 1282903 0 2024-04-25T01:36:12.537 INFO:tasks.workunit.client.1.smithi138.stdout:8/870: stat d3/d16/d1f/dd2/dd6/dd7/ff3 0 2024-04-25T01:36:12.540 INFO:tasks.workunit.client.1.smithi138.stdout:8/871: mknod d3/d16/d1f/d2a/d4f/dac/c110 0 2024-04-25T01:36:12.541 INFO:tasks.workunit.client.1.smithi138.stdout:3/792: dread db/df/d17/d23/f67 [0,4194304] 0 2024-04-25T01:36:12.542 INFO:tasks.workunit.client.1.smithi138.stdout:8/872: creat d3/d16/d1f/d2a/d4f/dcb/dce/f111 x:0 0 0 2024-04-25T01:36:12.544 INFO:tasks.workunit.client.1.smithi138.stdout:8/873: write d3/ffe [1043018,87397] 0 2024-04-25T01:36:12.546 INFO:tasks.workunit.client.1.smithi138.stdout:3/793: creat db/df/d17/d20/ffd x:0 0 0 2024-04-25T01:36:12.548 INFO:tasks.workunit.client.1.smithi138.stdout:3/794: write db/df/d19/d1a/d9e/fb9 [1807410,54809] 0 2024-04-25T01:36:12.549 INFO:tasks.workunit.client.1.smithi138.stdout:3/795: chown db/df/d17/d23/de9 8927208 1 2024-04-25T01:36:12.550 INFO:tasks.workunit.client.1.smithi138.stdout:5/689: dread d4/da/d6e/d22/d35/d4d/d5e/d8f/fa2 [0,4194304] 0 2024-04-25T01:36:12.550 INFO:tasks.workunit.client.1.smithi138.stdout:3/796: chown db/df/dd1/ddf 2893751 1 2024-04-25T01:36:12.551 INFO:tasks.workunit.client.1.smithi138.stdout:7/740: write da/de/d13/d30/f65 [2684363,702] 0 2024-04-25T01:36:12.551 INFO:tasks.workunit.client.1.smithi138.stdout:3/797: fsync db/df/de2/fed 0 2024-04-25T01:36:12.554 INFO:tasks.workunit.client.1.smithi138.stdout:5/690: unlink d4/da/d6e/d3c/fdc 0 2024-04-25T01:36:12.554 INFO:tasks.workunit.client.1.smithi138.stdout:6/810: write dc/d2e/f31 [1399202,16167] 0 2024-04-25T01:36:12.560 INFO:tasks.workunit.client.1.smithi138.stdout:5/691: mkdir d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/de8 0 2024-04-25T01:36:12.564 INFO:tasks.workunit.client.1.smithi138.stdout:8/874: dread d3/d16/d88/f4d [0,4194304] 0 2024-04-25T01:36:12.567 INFO:tasks.workunit.client.1.smithi138.stdout:7/741: truncate da/de/d1b/f33 464968 0 2024-04-25T01:36:12.568 INFO:tasks.workunit.client.1.smithi138.stdout:5/692: write d4/da/d6e/d22/d45/f54 [430403,68205] 0 2024-04-25T01:36:12.568 INFO:tasks.workunit.client.1.smithi138.stdout:8/875: truncate d3/d16/d1f/de2/f10a 1026875 0 2024-04-25T01:36:12.573 INFO:tasks.workunit.client.1.smithi138.stdout:7/742: fdatasync da/de/d1b/d49/d81/f9c 0 2024-04-25T01:36:12.575 INFO:tasks.workunit.client.1.smithi138.stdout:6/811: dwrite dc/d10/d67/f65 [0,4194304] 0 2024-04-25T01:36:12.575 INFO:tasks.workunit.client.1.smithi138.stdout:3/798: dwrite db/df/d17/d20/f26 [0,4194304] 0 2024-04-25T01:36:12.582 INFO:tasks.workunit.client.1.smithi138.stdout:7/743: read da/de/d13/d21/dbc/fb3 [607625,45226] 0 2024-04-25T01:36:12.583 INFO:tasks.workunit.client.1.smithi138.stdout:6/812: creat dc/d2e/d55/d7e/ffe x:0 0 0 2024-04-25T01:36:12.585 INFO:tasks.workunit.client.1.smithi138.stdout:7/744: stat f9 0 2024-04-25T01:36:12.587 INFO:tasks.workunit.client.1.smithi138.stdout:8/876: dwrite d3/d16/d88/da/db/d15/f21 [0,4194304] 0 2024-04-25T01:36:12.587 INFO:tasks.workunit.client.1.smithi138.stdout:5/693: truncate d4/da/d6e/d22/d74/f9c 16042 0 2024-04-25T01:36:12.590 INFO:tasks.workunit.client.1.smithi138.stdout:6/813: fdatasync dc/d10/d28/d80/d89/da0/fbf 0 2024-04-25T01:36:12.592 INFO:tasks.workunit.client.1.smithi138.stdout:3/799: creat db/db3/ffe x:0 0 0 2024-04-25T01:36:12.593 INFO:tasks.workunit.client.1.smithi138.stdout:7/745: symlink da/de/d13/d30/d59/d7b/d99/lf5 0 2024-04-25T01:36:12.596 INFO:tasks.workunit.client.1.smithi138.stdout:8/877: rename d3/d16/d88/da/db/d15/c4e to d3/d16/d88/da/db/d15/d59/c112 0 2024-04-25T01:36:12.597 INFO:tasks.workunit.client.1.smithi138.stdout:6/814: dread dc/d10/d67/f65 [0,4194304] 0 2024-04-25T01:36:12.598 INFO:tasks.workunit.client.1.smithi138.stdout:6/815: chown dc/d8d 0 1 2024-04-25T01:36:12.599 INFO:tasks.workunit.client.1.smithi138.stdout:7/746: read da/de/d13/d30/f93 [1503655,110821] 0 2024-04-25T01:36:12.599 INFO:tasks.workunit.client.1.smithi138.stdout:6/816: stat dc/d10/d67/d8e/fc0 0 2024-04-25T01:36:12.600 INFO:tasks.workunit.client.1.smithi138.stdout:3/800: mknod db/df/d19/d1a/cff 0 2024-04-25T01:36:12.602 INFO:tasks.workunit.client.1.smithi138.stdout:5/694: dwrite d4/f62 [0,4194304] 0 2024-04-25T01:36:12.604 INFO:tasks.workunit.client.1.smithi138.stdout:7/747: dread - da/de/fa9 zero size 2024-04-25T01:36:12.605 INFO:tasks.workunit.client.1.smithi138.stdout:6/817: mkdir dc/d2e/d37/d56/d8c/dff 0 2024-04-25T01:36:12.607 INFO:tasks.workunit.client.1.smithi138.stdout:6/818: dread - dc/d10/d28/fb1 zero size 2024-04-25T01:36:12.608 INFO:tasks.workunit.client.1.smithi138.stdout:6/819: dread - dc/d10/d28/d70/fdf zero size 2024-04-25T01:36:12.610 INFO:tasks.workunit.client.1.smithi138.stdout:7/748: creat da/de/d1b/d49/d81/ff6 x:0 0 0 2024-04-25T01:36:12.610 INFO:tasks.workunit.client.1.smithi138.stdout:3/801: dread db/d14/d40/d41/f7c [0,4194304] 0 2024-04-25T01:36:12.611 INFO:tasks.workunit.client.1.smithi138.stdout:7/749: dread - da/dd2/daf/fe2 zero size 2024-04-25T01:36:12.611 INFO:tasks.workunit.client.1.smithi138.stdout:5/695: creat d4/da/d6e/d22/d35/d4d/d5e/d60/dbd/dc8/dbb/fe9 x:0 0 0 2024-04-25T01:36:12.615 INFO:tasks.workunit.client.1.smithi138.stdout:3/802: creat db/df/d17/d20/d57/d6a/f100 x:0 0 0 2024-04-25T01:36:12.616 INFO:tasks.workunit.client.1.smithi138.stdout:3/803: chown db/df/d17/d23/fe3 53 1 2024-04-25T01:36:12.621 INFO:tasks.workunit.client.1.smithi138.stdout:5/696: rename d4/da/d6e/d22/d45/lc3 to d4/lea 0 2024-04-25T01:36:12.623 INFO:tasks.workunit.client.1.smithi138.stdout:5/697: rmdir d4/da/d6e/d22/d74/d59/d7f 39 2024-04-25T01:36:12.625 INFO:tasks.workunit.client.1.smithi138.stdout:5/698: symlink d4/da/d6e/d3c/dad/leb 0 2024-04-25T01:36:12.626 INFO:tasks.workunit.client.1.smithi138.stdout:5/699: chown d4/da/d6e/d22/d74/d59/d83/d8a/fc0 37 1 2024-04-25T01:36:12.626 INFO:tasks.workunit.client.1.smithi138.stdout:5/700: fdatasync d4/f62 0 2024-04-25T01:36:12.627 INFO:tasks.workunit.client.1.smithi138.stdout:5/701: readlink d4/da/d6e/d22/d74/d59/l69 0 2024-04-25T01:36:12.629 INFO:tasks.workunit.client.1.smithi138.stdout:5/702: mknod d4/da/d6e/d22/d74/d59/d83/d89/d9d/d9e/cec 0 2024-04-25T01:36:12.631 INFO:tasks.workunit.client.1.smithi138.stdout:5/703: write d4/da/d6e/d22/d35/d4d/d5e/d60/dbd/dc8/dbb/fe9 [597261,102003] 0 2024-04-25T01:36:12.633 INFO:tasks.workunit.client.1.smithi138.stdout:5/704: unlink d4/da/d27/l41 0 2024-04-25T01:36:12.636 INFO:tasks.workunit.client.1.smithi138.stdout:3/804: dread db/df/d17/d23/d34/f80 [0,4194304] 0 2024-04-25T01:36:12.637 INFO:tasks.workunit.client.1.smithi138.stdout:5/705: unlink d4/da/d6e/d22/d74/f5a 0 2024-04-25T01:36:12.640 INFO:tasks.workunit.client.1.smithi138.stdout:5/706: mkdir d4/da/d6e/d22/d74/d59/d83/dcd/ded 0 2024-04-25T01:36:12.645 INFO:tasks.workunit.client.1.smithi138.stdout:5/707: rename d4/da/d6e/d22/d74/d59/d83/d89/d9d/d9e/ca5 to d4/da/d6e/d22/d35/dce/dda/cee 0 2024-04-25T01:36:12.646 INFO:tasks.workunit.client.1.smithi138.stdout:3/805: write db/d14/d40/d41/fee [762837,7788] 0 2024-04-25T01:36:12.647 INFO:tasks.workunit.client.1.smithi138.stdout:5/708: symlink d4/da/lef 0 2024-04-25T01:36:12.648 INFO:tasks.workunit.client.1.smithi138.stdout:3/806: mkdir db/db3/d101 0 2024-04-25T01:36:12.651 INFO:tasks.workunit.client.1.smithi138.stdout:3/807: truncate db/df/d19/f93 925295 0 2024-04-25T01:36:12.653 INFO:tasks.workunit.client.1.smithi138.stdout:3/808: write db/df/d17/d23/d88/fa4 [114154,70884] 0 2024-04-25T01:36:12.655 INFO:tasks.workunit.client.1.smithi138.stdout:3/809: mkdir db/df/d102 0 2024-04-25T01:36:12.660 INFO:tasks.workunit.client.1.smithi138.stdout:3/810: rename db/df/d17/d5d/f76 to db/df/d19/d1a/dc4/f103 0 2024-04-25T01:36:12.660 INFO:tasks.workunit.client.1.smithi138.stdout:5/709: dwrite d4/da/d6e/d5c/f95 [0,4194304] 0 2024-04-25T01:36:12.661 INFO:tasks.workunit.client.1.smithi138.stdout:5/710: write d4/da/d6e/d22/d74/f76 [1426362,67989] 0 2024-04-25T01:36:12.662 INFO:tasks.workunit.client.1.smithi138.stdout:5/711: chown d4/da/c44 24176807 1 2024-04-25T01:36:12.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:12 smithi033 ceph-mon[26353]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:36:12.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:12 smithi033 ceph-mon[26353]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:36:12.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:12 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:12.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:12 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:12.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:12 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:12.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:12 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:12.666 INFO:tasks.workunit.client.1.smithi138.stdout:5/712: symlink d4/da/d6e/d22/d35/d4d/d90/lf0 0 2024-04-25T01:36:12.667 INFO:tasks.workunit.client.1.smithi138.stdout:3/811: creat db/dc5/f104 x:0 0 0 2024-04-25T01:36:12.670 INFO:tasks.workunit.client.1.smithi138.stdout:5/713: unlink d4/f62 0 2024-04-25T01:36:12.673 INFO:tasks.workunit.client.1.smithi138.stdout:3/812: link db/dc5/fb2 db/df/d17/d23/d88/f105 0 2024-04-25T01:36:12.674 INFO:tasks.workunit.client.1.smithi138.stdout:3/813: stat db/df/d17/d23/fe3 0 2024-04-25T01:36:12.678 INFO:tasks.workunit.client.1.smithi138.stdout:5/714: dwrite d4/da/d6e/d22/f3f [0,4194304] 0 2024-04-25T01:36:12.681 INFO:tasks.workunit.client.1.smithi138.stdout:5/715: rename d4/da/d6e/d22/d35/d4d/d5e to d4/da/d6e/d22/d35/d4d/d5e/d8f/df1 22 2024-04-25T01:36:12.682 INFO:tasks.workunit.client.1.smithi138.stdout:5/716: readlink d4/da/d6e/d22/l4a 0 2024-04-25T01:36:12.682 INFO:tasks.workunit.client.1.smithi138.stdout:3/814: write db/d14/d40/fb0 [1446542,6254] 0 2024-04-25T01:36:12.684 INFO:tasks.workunit.client.1.smithi138.stdout:3/815: write db/d14/d40/d41/f7c [898081,104613] 0 2024-04-25T01:36:12.685 INFO:tasks.workunit.client.1.smithi138.stdout:5/717: readlink d4/da/d6e/d22/d74/d59/d7f/db3/ld2 0 2024-04-25T01:36:12.687 INFO:tasks.workunit.client.1.smithi138.stdout:5/718: mknod d4/da/cf2 0 2024-04-25T01:36:12.688 INFO:tasks.workunit.client.1.smithi138.stdout:5/719: write d4/da/fd [4904181,51055] 0 2024-04-25T01:36:12.690 INFO:tasks.workunit.client.1.smithi138.stdout:5/720: mknod d4/da/d6e/d22/d74/d59/d83/dcd/ded/cf3 0 2024-04-25T01:36:12.693 INFO:tasks.workunit.client.1.smithi138.stdout:5/721: mkdir d4/da/d6e/d22/d45/df4 0 2024-04-25T01:36:12.694 INFO:tasks.workunit.client.1.smithi138.stdout:5/722: mkdir d4/da/d6e/d22/d45/df5 0 2024-04-25T01:36:12.696 INFO:tasks.workunit.client.1.smithi138.stdout:5/723: mknod d4/da/d27/cf6 0 2024-04-25T01:36:12.697 INFO:tasks.workunit.client.1.smithi138.stdout:5/724: fsync d4/da/d6e/d22/d74/f76 0 2024-04-25T01:36:12.699 INFO:tasks.workunit.client.1.smithi138.stdout:5/725: creat d4/da/d6e/d22/d74/d8e/dd8/ff7 x:0 0 0 2024-04-25T01:36:12.714 INFO:tasks.workunit.client.1.smithi138.stdout:5/726: dread d4/da/d6e/d22/d74/d59/f68 [0,4194304] 0 2024-04-25T01:36:12.717 INFO:tasks.workunit.client.1.smithi138.stdout:5/727: rename d4/lea to d4/da/d6e/d22/d45/lf8 0 2024-04-25T01:36:12.718 INFO:tasks.workunit.client.1.smithi138.stdout:5/728: chown d4/da/d6e/d22/d74/d8e/dd7/fa8 4680 1 2024-04-25T01:36:12.720 INFO:tasks.workunit.client.1.smithi138.stdout:5/729: read d4/da/d6e/d22/d74/d8e/fae [5302230,98847] 0 2024-04-25T01:36:12.724 INFO:tasks.workunit.client.1.smithi138.stdout:5/730: rename d4/da/d6e/d22/d74/d59 to d4/da/d6e/d22/d35/dce/df9 0 2024-04-25T01:36:12.730 INFO:tasks.workunit.client.1.smithi138.stdout:5/731: dwrite d4/da/fd [0,4194304] 0 2024-04-25T01:36:12.730 INFO:tasks.workunit.client.1.smithi138.stdout:5/732: chown d4/f6c 3 1 2024-04-25T01:36:12.735 INFO:tasks.workunit.client.1.smithi138.stdout:5/733: read d4/da/d6e/d22/d35/d4d/d5e/d60/dbd/dc8/dbb/fe9 [394018,104390] 0 2024-04-25T01:36:12.740 INFO:tasks.workunit.client.1.smithi138.stdout:5/734: link d4/da/d6e/d22/d35/dce/df9/d83/dcd/ded/cf3 d4/da/d6e/d22/d35/dce/cfa 0 2024-04-25T01:36:12.743 INFO:tasks.workunit.client.1.smithi138.stdout:5/735: mkdir d4/da/d6e/d22/d35/d4d/d5e/dfb 0 2024-04-25T01:36:12.744 INFO:tasks.workunit.client.1.smithi138.stdout:5/736: stat d4/da/d6e/d22/f3f 0 2024-04-25T01:36:12.744 INFO:tasks.workunit.client.1.smithi138.stdout:5/737: write d4/da/f47 [293668,84630] 0 2024-04-25T01:36:12.746 INFO:tasks.workunit.client.1.smithi138.stdout:5/738: dread - d4/da/d6e/d22/d35/d4d/d5e/d8f/fb2 zero size 2024-04-25T01:36:12.753 INFO:tasks.workunit.client.1.smithi138.stdout:5/739: dwrite d4/da/d6e/d22/d35/dce/df9/d83/d8a/fa7 [0,4194304] 0 2024-04-25T01:36:12.756 INFO:tasks.workunit.client.1.smithi138.stdout:5/740: rmdir d4/da/d6e/d22/d35/d4d/d90/dba 39 2024-04-25T01:36:12.758 INFO:tasks.workunit.client.1.smithi138.stdout:5/741: unlink d4/da/d6e/d3c/dad/leb 0 2024-04-25T01:36:12.761 INFO:tasks.workunit.client.1.smithi138.stdout:5/742: getdents d4/da/d6e/d22/d35/d4d 0 2024-04-25T01:36:12.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:12 smithi138 ceph-mon[32977]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:36:12.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:12 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:36:12.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:12 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:12.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:12 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:12.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:12 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:12.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:12 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:12.768 INFO:tasks.workunit.client.1.smithi138.stdout:5/743: dwrite d4/da/d6e/d22/d74/f73 [0,4194304] 0 2024-04-25T01:36:12.825 INFO:tasks.workunit.client.1.smithi138.stdout:5/744: write d4/da/d6e/d22/f98 [687526,27965] 0 2024-04-25T01:36:12.828 INFO:tasks.workunit.client.1.smithi138.stdout:5/745: rmdir d4/da/d6e/d22/d35/dce/df9/d83/d8a/dc2 0 2024-04-25T01:36:12.828 INFO:tasks.workunit.client.1.smithi138.stdout:0/551: sync 2024-04-25T01:36:12.830 INFO:tasks.workunit.client.1.smithi138.stdout:0/552: write d6/df/d3a/fae [5034609,26324] 0 2024-04-25T01:36:12.830 INFO:tasks.workunit.client.1.smithi138.stdout:5/746: dread d4/da/d6e/d22/d74/f9c [0,4194304] 0 2024-04-25T01:36:12.837 INFO:tasks.workunit.client.1.smithi138.stdout:0/553: write d6/d39/fa2 [885986,92709] 0 2024-04-25T01:36:12.838 INFO:tasks.workunit.client.1.smithi138.stdout:5/747: dread d4/da/d6e/d22/d35/dce/df9/fc7 [0,4194304] 0 2024-04-25T01:36:12.838 INFO:tasks.workunit.client.1.smithi138.stdout:0/554: rmdir d6/d1d 39 2024-04-25T01:36:12.838 INFO:tasks.workunit.client.1.smithi138.stdout:5/748: symlink d4/da/d6e/d22/d74/d8e/lfc 0 2024-04-25T01:36:12.838 INFO:tasks.workunit.client.1.smithi138.stdout:0/555: read d6/df/d41/f55 [1461943,73654] 0 2024-04-25T01:36:12.843 INFO:tasks.workunit.client.1.smithi138.stdout:5/749: link d4/da/f20 d4/da/d6e/d22/d35/dce/df9/dd4/ffd 0 2024-04-25T01:36:12.844 INFO:tasks.workunit.client.1.smithi138.stdout:0/556: dwrite d6/d19/f77 [4194304,4194304] 0 2024-04-25T01:36:12.847 INFO:tasks.workunit.client.1.smithi138.stdout:5/750: unlink d4/da/d6e/d22/d74/d8e/dd8/ff7 0 2024-04-25T01:36:12.848 INFO:tasks.workunit.client.1.smithi138.stdout:0/557: write d6/d39/d58/f87 [5507551,84167] 0 2024-04-25T01:36:12.849 INFO:tasks.workunit.client.1.smithi138.stdout:5/751: creat d4/da/d6e/d5c/ffe x:0 0 0 2024-04-25T01:36:12.852 INFO:tasks.workunit.client.1.smithi138.stdout:0/558: creat d6/d2f/fb1 x:0 0 0 2024-04-25T01:36:12.863 INFO:tasks.workunit.client.1.smithi138.stdout:5/752: write d4/da/d6e/d22/d35/dce/df9/f68 [858070,7246] 0 2024-04-25T01:36:12.865 INFO:tasks.workunit.client.1.smithi138.stdout:5/753: write d4/da/d6e/d22/d35/d4d/f66 [2148644,71193] 0 2024-04-25T01:36:12.866 INFO:tasks.workunit.client.1.smithi138.stdout:5/754: read d4/da/d6e/d22/f3f [1855116,22952] 0 2024-04-25T01:36:12.869 INFO:tasks.workunit.client.1.smithi138.stdout:0/559: dread d6/d2f/fa1 [0,4194304] 0 2024-04-25T01:36:12.873 INFO:tasks.workunit.client.1.smithi138.stdout:0/560: link d6/df/d69/f6a d6/d2f/d31/fb2 0 2024-04-25T01:36:12.875 INFO:tasks.workunit.client.1.smithi138.stdout:5/755: write d4/da/d6e/d22/d35/dce/df9/dd4/ffd [130896,74762] 0 2024-04-25T01:36:12.879 INFO:tasks.workunit.client.1.smithi138.stdout:0/561: dwrite d6/d2f/fa1 [0,4194304] 0 2024-04-25T01:36:12.880 INFO:tasks.workunit.client.1.smithi138.stdout:0/562: chown d6/d2f/f7d 262122461 1 2024-04-25T01:36:12.883 INFO:tasks.workunit.client.1.smithi138.stdout:0/563: unlink d6/d19/l8d 0 2024-04-25T01:36:12.890 INFO:tasks.workunit.client.1.smithi138.stdout:0/564: dwrite d6/d1d/d98/f83 [4194304,4194304] 0 2024-04-25T01:36:12.893 INFO:tasks.workunit.client.1.smithi138.stdout:0/565: fsync d6/df/f2c 0 2024-04-25T01:36:12.899 INFO:tasks.workunit.client.1.smithi138.stdout:0/566: dwrite d6/df/d41/d49/f51 [4194304,4194304] 0 2024-04-25T01:36:12.901 INFO:tasks.workunit.client.1.smithi138.stdout:0/567: chown d6/f1f 46 1 2024-04-25T01:36:12.901 INFO:tasks.workunit.client.1.smithi138.stdout:0/568: chown d6/d2f 1557 1 2024-04-25T01:36:12.915 INFO:tasks.workunit.client.1.smithi138.stdout:0/569: dread d6/df/f15 [0,4194304] 0 2024-04-25T01:36:12.917 INFO:tasks.workunit.client.1.smithi138.stdout:0/570: creat d6/d39/d43/fb3 x:0 0 0 2024-04-25T01:36:12.921 INFO:tasks.workunit.client.1.smithi138.stdout:0/571: write d6/f5e [1482880,56908] 0 2024-04-25T01:36:12.925 INFO:tasks.workunit.client.1.smithi138.stdout:0/572: rename d6/df/d3a/l50 to d6/d1d/d98/lb4 0 2024-04-25T01:36:12.927 INFO:tasks.workunit.client.1.smithi138.stdout:0/573: rename d6/la0 to d6/d2f/d75/lb5 0 2024-04-25T01:36:12.928 INFO:tasks.workunit.client.1.smithi138.stdout:0/574: write d6/d39/fa2 [967583,14732] 0 2024-04-25T01:36:12.930 INFO:tasks.workunit.client.1.smithi138.stdout:0/575: dread d6/df/f2c [0,4194304] 0 2024-04-25T01:36:12.935 INFO:tasks.workunit.client.1.smithi138.stdout:0/576: dwrite d6/df/d69/f9d [0,4194304] 0 2024-04-25T01:36:12.938 INFO:tasks.workunit.client.1.smithi138.stdout:0/577: mkdir d6/d2f/d75/db6 0 2024-04-25T01:36:13.011 INFO:tasks.workunit.client.1.smithi138.stdout:2/899: sync 2024-04-25T01:36:13.016 INFO:tasks.workunit.client.1.smithi138.stdout:2/900: dwrite d0/d21/f119 [0,4194304] 0 2024-04-25T01:36:13.018 INFO:tasks.workunit.client.1.smithi138.stdout:2/901: readlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/l34 0 2024-04-25T01:36:13.026 INFO:tasks.workunit.client.1.smithi138.stdout:2/902: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fc6 [4194304,4194304] 0 2024-04-25T01:36:13.034 INFO:tasks.workunit.client.1.smithi138.stdout:2/903: link d0/d85/da5/db4/d15/d44/d4c/d56/d6d/ld9 d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/l11f 0 2024-04-25T01:36:13.037 INFO:tasks.workunit.client.1.smithi138.stdout:2/904: rename d0/d85/da5/db4/l110 to d0/d85/l120 0 2024-04-25T01:36:13.040 INFO:tasks.workunit.client.1.smithi138.stdout:2/905: rename d0/d85/da5/db4/d15/d44/d4c/fbb to d0/d21/d10b/f121 0 2024-04-25T01:36:13.159 INFO:tasks.workunit.client.1.smithi138.stdout:1/660: sync 2024-04-25T01:36:13.159 INFO:tasks.workunit.client.1.smithi138.stdout:4/538: sync 2024-04-25T01:36:13.159 INFO:tasks.workunit.client.1.smithi138.stdout:9/627: sync 2024-04-25T01:36:13.165 INFO:tasks.workunit.client.1.smithi138.stdout:1/661: dwrite d3/f44 [0,4194304] 0 2024-04-25T01:36:13.166 INFO:tasks.workunit.client.1.smithi138.stdout:9/628: link d4/l2b d4/d8/d90/ld0 0 2024-04-25T01:36:13.168 INFO:tasks.workunit.client.1.smithi138.stdout:9/629: mknod d4/d49/cd1 0 2024-04-25T01:36:13.171 INFO:tasks.workunit.client.1.smithi138.stdout:9/630: write d4/d8/d2c/da3/fba [93603,64470] 0 2024-04-25T01:36:13.171 INFO:tasks.workunit.client.1.smithi138.stdout:9/631: chown d4/d8/d90/d52 201202484 1 2024-04-25T01:36:13.172 INFO:tasks.workunit.client.1.smithi138.stdout:1/662: rename d3/d27/d24/f64 to d3/d4/d53/d9d/daa/fd8 0 2024-04-25T01:36:13.174 INFO:tasks.workunit.client.1.smithi138.stdout:9/632: creat d4/d49/fd2 x:0 0 0 2024-04-25T01:36:13.175 INFO:tasks.workunit.client.1.smithi138.stdout:1/663: mkdir d3/d45/d72/db2/dd9 0 2024-04-25T01:36:13.177 INFO:tasks.workunit.client.1.smithi138.stdout:9/633: rename d4/d8/d90/d52/f65 to d4/d1a/d3c/fd3 0 2024-04-25T01:36:13.179 INFO:tasks.workunit.client.1.smithi138.stdout:9/634: dread - d4/d1a/d3c/dc2/f9c zero size 2024-04-25T01:36:13.181 INFO:tasks.workunit.client.1.smithi138.stdout:1/664: rename d3/f44 to d3/d27/d2d/da1/dca/fda 0 2024-04-25T01:36:13.183 INFO:tasks.workunit.client.1.smithi138.stdout:9/635: stat d4/f6c 0 2024-04-25T01:36:13.183 INFO:tasks.workunit.client.1.smithi138.stdout:3/816: sync 2024-04-25T01:36:13.183 INFO:tasks.workunit.client.1.smithi138.stdout:5/756: sync 2024-04-25T01:36:13.183 INFO:tasks.workunit.client.1.smithi138.stdout:2/906: sync 2024-04-25T01:36:13.184 INFO:tasks.workunit.client.1.smithi138.stdout:8/878: sync 2024-04-25T01:36:13.184 INFO:tasks.workunit.client.1.smithi138.stdout:6/820: sync 2024-04-25T01:36:13.184 INFO:tasks.workunit.client.1.smithi138.stdout:7/750: sync 2024-04-25T01:36:13.184 INFO:tasks.workunit.client.1.smithi138.stdout:0/578: sync 2024-04-25T01:36:13.186 INFO:tasks.workunit.client.1.smithi138.stdout:8/879: readlink d3/d16/d1f/d87/lbf 0 2024-04-25T01:36:13.186 INFO:tasks.workunit.client.1.smithi138.stdout:0/579: fsync d6/d39/fa2 0 2024-04-25T01:36:13.187 INFO:tasks.workunit.client.1.smithi138.stdout:5/757: write d4/da/fd [3283956,28990] 0 2024-04-25T01:36:13.187 INFO:tasks.workunit.client.1.smithi138.stdout:9/636: read d4/d49/f76 [51244,8563] 0 2024-04-25T01:36:13.190 INFO:tasks.workunit.client.1.smithi138.stdout:5/758: chown d4/da/d27/c48 13746 1 2024-04-25T01:36:13.191 INFO:tasks.workunit.client.1.smithi138.stdout:1/665: rename d3/d4/d47 to d3/ddb 0 2024-04-25T01:36:13.193 INFO:tasks.workunit.client.1.smithi138.stdout:5/759: write d4/da/d6e/d5c/ffe [214564,63103] 0 2024-04-25T01:36:13.194 INFO:tasks.workunit.client.1.smithi138.stdout:7/751: dwrite da/de/d13/fb5 [0,4194304] 0 2024-04-25T01:36:13.197 INFO:tasks.workunit.client.1.smithi138.stdout:0/580: mkdir d6/d39/d52/db7 0 2024-04-25T01:36:13.198 INFO:tasks.workunit.client.1.smithi138.stdout:7/752: write da/de/d13/d21/da5/fe3 [525853,92854] 0 2024-04-25T01:36:13.203 INFO:tasks.workunit.client.1.smithi138.stdout:9/637: rename d4/d49/fd2 to d4/d8/d2c/da3/fd4 0 2024-04-25T01:36:13.204 INFO:tasks.workunit.client.1.smithi138.stdout:1/666: creat d3/d45/d72/db2/fdc x:0 0 0 2024-04-25T01:36:13.204 INFO:tasks.workunit.client.1.smithi138.stdout:1/667: chown d3/d27/d24 4 1 2024-04-25T01:36:13.205 INFO:tasks.workunit.client.1.smithi138.stdout:2/907: write d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/f108 [910958,77325] 0 2024-04-25T01:36:13.208 INFO:tasks.workunit.client.1.smithi138.stdout:5/760: readlink d4/da/d6e/d22/d74/d8e/dd7/lb7 0 2024-04-25T01:36:13.209 INFO:tasks.workunit.client.1.smithi138.stdout:5/761: stat d4/da/d6e/d22/d35/dce/df9/d83/dcd 0 2024-04-25T01:36:13.209 INFO:tasks.workunit.client.1.smithi138.stdout:9/638: truncate d4/d1a/d55/f5c 1957604 0 2024-04-25T01:36:13.211 INFO:tasks.workunit.client.1.smithi138.stdout:0/581: symlink d6/d2f/lb8 0 2024-04-25T01:36:13.212 INFO:tasks.workunit.client.1.smithi138.stdout:7/753: mkdir da/de/d13/df7 0 2024-04-25T01:36:13.212 INFO:tasks.workunit.client.1.smithi138.stdout:2/908: symlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/l122 0 2024-04-25T01:36:13.213 INFO:tasks.workunit.client.1.smithi138.stdout:1/668: creat d3/dcc/fdd x:0 0 0 2024-04-25T01:36:13.214 INFO:tasks.workunit.client.1.smithi138.stdout:4/539: truncate d0/d1c/d1e/f69 3982550 0 2024-04-25T01:36:13.215 INFO:tasks.workunit.client.1.smithi138.stdout:2/909: chown d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/d8b/fb5 722 1 2024-04-25T01:36:13.220 INFO:tasks.workunit.client.1.smithi138.stdout:4/540: dwrite d0/f2f [4194304,4194304] 0 2024-04-25T01:36:13.234 INFO:tasks.workunit.client.1.smithi138.stdout:9/639: symlink d4/d1a/d3c/dc2/ld5 0 2024-04-25T01:36:13.238 INFO:tasks.workunit.client.1.smithi138.stdout:5/762: mknod d4/da/d6e/d3c/dd5/cff 0 2024-04-25T01:36:13.240 INFO:tasks.workunit.client.1.smithi138.stdout:1/669: write d3/db/f14 [551452,127309] 0 2024-04-25T01:36:13.241 INFO:tasks.workunit.client.1.smithi138.stdout:1/670: write d3/d45/d72/f5d [536944,41145] 0 2024-04-25T01:36:13.243 INFO:tasks.workunit.client.1.smithi138.stdout:1/671: write d3/d45/d72/f5d [2098990,10527] 0 2024-04-25T01:36:13.245 INFO:tasks.workunit.client.1.smithi138.stdout:7/754: readlink da/de/d1b/d49/d81/l92 0 2024-04-25T01:36:13.245 INFO:tasks.workunit.client.1.smithi138.stdout:5/763: dread d4/da/d6e/d22/d35/d4d/dca/fd6 [0,4194304] 0 2024-04-25T01:36:13.246 INFO:tasks.workunit.client.1.smithi138.stdout:5/764: write d4/fc1 [2345245,38491] 0 2024-04-25T01:36:13.250 INFO:tasks.workunit.client.1.smithi138.stdout:2/910: symlink d0/d85/da5/db4/d15/d44/d4c/d56/da2/l123 0 2024-04-25T01:36:13.251 INFO:tasks.workunit.client.1.smithi138.stdout:9/640: getdents d4/d1a/d3c/dc2/db2 0 2024-04-25T01:36:13.251 INFO:tasks.workunit.client.1.smithi138.stdout:8/880: write d3/d16/d1f/d31/d82/d8a/faf [792009,46805] 0 2024-04-25T01:36:13.254 INFO:tasks.workunit.client.1.smithi138.stdout:3/817: dwrite db/df/d17/d20/f29 [0,4194304] 0 2024-04-25T01:36:13.258 INFO:tasks.workunit.client.1.smithi138.stdout:7/755: rename da/dd2/daf to da/de/d13/d30/df8 0 2024-04-25T01:36:13.261 INFO:tasks.workunit.client.1.smithi138.stdout:5/765: mkdir d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/d100 0 2024-04-25T01:36:13.261 INFO:tasks.workunit.client.1.smithi138.stdout:1/672: mkdir d3/ddb/dd6/dde 0 2024-04-25T01:36:13.262 INFO:tasks.workunit.client.1.smithi138.stdout:7/756: truncate da/de/d13/d30/d59/d7b/d99/fd5 53291 0 2024-04-25T01:36:13.263 INFO:tasks.workunit.client.1.smithi138.stdout:1/673: chown d3/d6d/l70 41 1 2024-04-25T01:36:13.263 INFO:tasks.workunit.client.1.smithi138.stdout:1/674: chown d3/d6d/c83 0 1 2024-04-25T01:36:13.264 INFO:tasks.workunit.client.1.smithi138.stdout:9/641: mknod d4/d1a/d33/d1d/cd6 0 2024-04-25T01:36:13.274 INFO:tasks.workunit.client.1.smithi138.stdout:3/818: fsync db/df/d19/dd4/fe0 0 2024-04-25T01:36:13.274 INFO:tasks.workunit.client.1.smithi138.stdout:1/675: chown d3/d45/d72/d57/db0/dcd/fd0 0 1 2024-04-25T01:36:13.276 INFO:tasks.workunit.client.1.smithi138.stdout:9/642: dwrite d4/d1a/d3c/dc2/f9c [0,4194304] 0 2024-04-25T01:36:13.278 INFO:tasks.workunit.client.1.smithi138.stdout:1/676: chown d3/d27/d16/l3a 129439 1 2024-04-25T01:36:13.279 INFO:tasks.workunit.client.1.smithi138.stdout:7/757: truncate da/de/d13/d30/d59/f78 2657064 0 2024-04-25T01:36:13.282 INFO:tasks.workunit.client.1.smithi138.stdout:3/819: creat db/df/d19/d1a/de5/f106 x:0 0 0 2024-04-25T01:36:13.283 INFO:tasks.workunit.client.1.smithi138.stdout:5/766: dwrite d4/da/d6e/d22/d74/f58 [0,4194304] 0 2024-04-25T01:36:13.290 INFO:tasks.workunit.client.1.smithi138.stdout:9/643: creat d4/d63/dcf/fd7 x:0 0 0 2024-04-25T01:36:13.292 INFO:tasks.workunit.client.1.smithi138.stdout:1/677: rename d3/db/f5a to d3/d45/d72/d57/d8e/db6/fdf 0 2024-04-25T01:36:13.294 INFO:tasks.workunit.client.1.smithi138.stdout:9/644: read d4/d1a/d55/fa9 [545721,124328] 0 2024-04-25T01:36:13.298 INFO:tasks.workunit.client.1.smithi138.stdout:7/758: rename da/de/d1b/d4b/d77/l86 to da/dd2/lf9 0 2024-04-25T01:36:13.299 INFO:tasks.workunit.client.1.smithi138.stdout:5/767: truncate d4/da/d6e/d22/d35/d4d/d5e/f7a 3437911 0 2024-04-25T01:36:13.301 INFO:tasks.workunit.client.1.smithi138.stdout:1/678: mknod d3/db/ce0 0 2024-04-25T01:36:13.303 INFO:tasks.workunit.client.1.smithi138.stdout:9/645: rename d4/d8/d2c/d82/db9 to d4/d1a/d3c/dc2/db2/dd8 0 2024-04-25T01:36:13.304 INFO:tasks.workunit.client.1.smithi138.stdout:7/759: fsync da/de/d13/d21/dbc/fb3 0 2024-04-25T01:36:13.305 INFO:tasks.workunit.client.1.smithi138.stdout:5/768: mkdir d4/da/d6e/d22/d35/dce/df9/d83/d89/d9d/d9e/d101 0 2024-04-25T01:36:13.306 INFO:tasks.workunit.client.1.smithi138.stdout:3/820: getdents db/df/d17/d23/d88 0 2024-04-25T01:36:13.307 INFO:tasks.workunit.client.1.smithi138.stdout:5/769: write d4/da/d6e/d22/d35/dce/df9/d83/d8a/fc0 [379677,22396] 0 2024-04-25T01:36:13.308 INFO:tasks.workunit.client.1.smithi138.stdout:1/679: mkdir d3/d4/da7/db1/de1 0 2024-04-25T01:36:13.310 INFO:tasks.workunit.client.1.smithi138.stdout:3/821: truncate db/df/d17/d5d/f6b 1742674 0 2024-04-25T01:36:13.312 INFO:tasks.workunit.client.1.smithi138.stdout:5/770: dread d4/da/d6e/d22/d35/dce/df9/f68 [0,4194304] 0 2024-04-25T01:36:13.313 INFO:tasks.workunit.client.1.smithi138.stdout:5/771: chown d4/da/d6e/d22/d74/d8e/dd7/da6 3692 1 2024-04-25T01:36:13.315 INFO:tasks.workunit.client.1.smithi138.stdout:9/646: mkdir d4/d8/d90/d8e/dd9 0 2024-04-25T01:36:13.315 INFO:tasks.workunit.client.1.smithi138.stdout:7/760: mknod da/de/d1b/d49/d81/d73/cfa 0 2024-04-25T01:36:13.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/647: rename d4/d1a/d3c/dc2/ld5 to d4/d1a/d3c/dc2/db2/lda 0 2024-04-25T01:36:13.318 INFO:tasks.workunit.client.1.smithi138.stdout:1/680: creat d3/d27/d3e/fe2 x:0 0 0 2024-04-25T01:36:13.319 INFO:tasks.workunit.client.1.smithi138.stdout:9/648: mkdir d4/d1a/d33/d9a/ddb 0 2024-04-25T01:36:13.320 INFO:tasks.workunit.client.1.smithi138.stdout:3/822: getdents db/d9b 0 2024-04-25T01:36:13.322 INFO:tasks.workunit.client.1.smithi138.stdout:1/681: mkdir d3/d45/d72/d57/db0/dcd/de3 0 2024-04-25T01:36:13.323 INFO:tasks.workunit.client.1.smithi138.stdout:1/682: write d3/db/f14 [7056155,127711] 0 2024-04-25T01:36:13.325 INFO:tasks.workunit.client.1.smithi138.stdout:1/683: truncate d3/d45/d72/d57/f91 801832 0 2024-04-25T01:36:13.325 INFO:tasks.workunit.client.1.smithi138.stdout:9/649: symlink d4/d1a/d3c/dc2/db2/dd8/ldc 0 2024-04-25T01:36:13.327 INFO:tasks.workunit.client.1.smithi138.stdout:1/684: write d3/f34 [8690237,76836] 0 2024-04-25T01:36:13.327 INFO:tasks.workunit.client.1.smithi138.stdout:9/650: chown d4/d1a/d33/l47 26093 1 2024-04-25T01:36:13.329 INFO:tasks.workunit.client.1.smithi138.stdout:3/823: mkdir db/df/d17/d107 0 2024-04-25T01:36:13.331 INFO:tasks.workunit.client.1.smithi138.stdout:9/651: mkdir d4/d8/d90/d52/ddd 0 2024-04-25T01:36:13.332 INFO:tasks.workunit.client.1.smithi138.stdout:9/652: chown d4/d1a/d33/d9a/f9e 12105948 1 2024-04-25T01:36:13.333 INFO:tasks.workunit.client.1.smithi138.stdout:1/685: rename d3/d4/d53/d9d to d3/d4/da7/db1/de4 0 2024-04-25T01:36:13.334 INFO:tasks.workunit.client.1.smithi138.stdout:3/824: dread - db/df/d19/fd3 zero size 2024-04-25T01:36:13.340 INFO:tasks.workunit.client.1.smithi138.stdout:3/825: symlink db/d9b/l108 0 2024-04-25T01:36:13.340 INFO:tasks.workunit.client.1.smithi138.stdout:1/686: dread d3/ddb/d9b/f9c [0,4194304] 0 2024-04-25T01:36:13.342 INFO:tasks.workunit.client.1.smithi138.stdout:1/687: creat d3/d27/d2d/fe5 x:0 0 0 2024-04-25T01:36:13.345 INFO:tasks.workunit.client.1.smithi138.stdout:3/826: rename db/df/d17/d23/ff3 to db/df/d19/d1a/de5/f109 0 2024-04-25T01:36:13.347 INFO:tasks.workunit.client.1.smithi138.stdout:1/688: rename d3/db/cc to d3/d27/d16/da3/ce6 0 2024-04-25T01:36:13.347 INFO:tasks.workunit.client.1.smithi138.stdout:9/653: truncate d4/f25 1966276 0 2024-04-25T01:36:13.348 INFO:tasks.workunit.client.1.smithi138.stdout:3/827: mkdir db/df/de2/d10a 0 2024-04-25T01:36:13.349 INFO:tasks.workunit.client.1.smithi138.stdout:9/654: truncate d4/d63/dcf/fd7 484296 0 2024-04-25T01:36:13.351 INFO:tasks.workunit.client.1.smithi138.stdout:3/828: mknod db/df/d17/d20/d57/daa/c10b 0 2024-04-25T01:36:13.352 INFO:tasks.workunit.client.1.smithi138.stdout:9/655: mknod d4/d8/d90/d52/ddd/cde 0 2024-04-25T01:36:13.358 INFO:tasks.workunit.client.1.smithi138.stdout:1/689: truncate d3/ddb/d9b/f9c 1894077 0 2024-04-25T01:36:13.361 INFO:tasks.workunit.client.1.smithi138.stdout:1/690: creat d3/d27/d24/da9/fe7 x:0 0 0 2024-04-25T01:36:13.361 INFO:tasks.workunit.client.1.smithi138.stdout:3/829: rmdir db/df/d17/d20 39 2024-04-25T01:36:13.364 INFO:tasks.workunit.client.1.smithi138.stdout:9/656: dread d4/d8/f61 [0,4194304] 0 2024-04-25T01:36:13.366 INFO:tasks.workunit.client.1.smithi138.stdout:3/830: symlink db/df/d17/d23/de9/l10c 0 2024-04-25T01:36:13.367 INFO:tasks.workunit.client.1.smithi138.stdout:1/691: creat d3/d4/fe8 x:0 0 0 2024-04-25T01:36:13.381 INFO:tasks.workunit.client.1.smithi138.stdout:3/831: dread db/df/d17/d23/f30 [0,4194304] 0 2024-04-25T01:36:13.414 INFO:tasks.workunit.client.1.smithi138.stdout:6/821: sync 2024-04-25T01:36:13.423 INFO:tasks.workunit.client.1.smithi138.stdout:1/692: creat d3/d45/d72/d57/db0/fe9 x:0 0 0 2024-04-25T01:36:13.428 INFO:tasks.workunit.client.1.smithi138.stdout:6/822: chown dc/d2e/d37/d56/cde 322 1 2024-04-25T01:36:13.432 INFO:tasks.workunit.client.1.smithi138.stdout:6/823: mkdir dc/d8d/d100 0 2024-04-25T01:36:13.435 INFO:tasks.workunit.client.1.smithi138.stdout:3/832: getdents db/df/de2 0 2024-04-25T01:36:13.435 INFO:tasks.workunit.client.1.smithi138.stdout:9/657: dwrite d4/d8/d90/d52/f7d [0,4194304] 0 2024-04-25T01:36:13.437 INFO:tasks.workunit.client.1.smithi138.stdout:3/833: chown db/df/d17/d107 5 1 2024-04-25T01:36:13.444 INFO:tasks.workunit.client.1.smithi138.stdout:3/834: unlink db/df/d17/d23/c3a 0 2024-04-25T01:36:13.451 INFO:tasks.workunit.client.1.smithi138.stdout:9/658: getdents d4 0 2024-04-25T01:36:13.452 INFO:tasks.workunit.client.1.smithi138.stdout:3/835: creat db/d14/f10d x:0 0 0 2024-04-25T01:36:13.457 INFO:tasks.workunit.client.1.smithi138.stdout:3/836: symlink db/db3/d101/l10e 0 2024-04-25T01:36:13.458 INFO:tasks.workunit.client.1.smithi138.stdout:9/659: dwrite d4/d8/d2c/fa8 [0,4194304] 0 2024-04-25T01:36:13.460 INFO:tasks.workunit.client.1.smithi138.stdout:3/837: truncate db/df/f4f 3547385 0 2024-04-25T01:36:13.461 INFO:tasks.workunit.client.1.smithi138.stdout:9/660: creat d4/d57/fdf x:0 0 0 2024-04-25T01:36:13.462 INFO:tasks.workunit.client.1.smithi138.stdout:9/661: stat d4/f6c 0 2024-04-25T01:36:13.462 INFO:tasks.workunit.client.1.smithi138.stdout:3/838: truncate db/d14/f10d 210191 0 2024-04-25T01:36:13.467 INFO:tasks.workunit.client.1.smithi138.stdout:3/839: symlink db/d14/l10f 0 2024-04-25T01:36:13.473 INFO:tasks.workunit.client.1.smithi138.stdout:1/693: rmdir d3/d27/d24 39 2024-04-25T01:36:13.474 INFO:tasks.workunit.client.1.smithi138.stdout:9/662: dread d4/d1a/f70 [0,4194304] 0 2024-04-25T01:36:13.478 INFO:tasks.workunit.client.1.smithi138.stdout:9/663: truncate d4/d1a/f74 4097743 0 2024-04-25T01:36:13.485 INFO:tasks.workunit.client.1.smithi138.stdout:3/840: dread db/df/d17/d23/d88/f9a [0,4194304] 0 2024-04-25T01:36:13.487 INFO:tasks.workunit.client.1.smithi138.stdout:3/841: truncate db/df/d19/d1a/d9e/fb9 2820410 0 2024-04-25T01:36:13.489 INFO:tasks.workunit.client.1.smithi138.stdout:3/842: fsync db/df/d17/d20/f26 0 2024-04-25T01:36:13.490 INFO:tasks.workunit.client.1.smithi138.stdout:3/843: chown db/d14/d40/f9c 0 1 2024-04-25T01:36:13.491 INFO:tasks.workunit.client.1.smithi138.stdout:3/844: stat db/f5b 0 2024-04-25T01:36:13.493 INFO:tasks.workunit.client.1.smithi138.stdout:3/845: mkdir db/df/d17/d20/d57/d6a/d110 0 2024-04-25T01:36:13.495 INFO:tasks.workunit.client.1.smithi138.stdout:3/846: rmdir db/d9b/db4 39 2024-04-25T01:36:13.503 INFO:tasks.workunit.client.1.smithi138.stdout:6/824: write dc/d10/d28/d70/f7f [689458,40873] 0 2024-04-25T01:36:13.551 INFO:tasks.workunit.client.1.smithi138.stdout:0/582: sync 2024-04-25T01:36:13.551 INFO:tasks.workunit.client.1.smithi138.stdout:4/541: sync 2024-04-25T01:36:13.557 INFO:tasks.workunit.client.1.smithi138.stdout:1/694: write d3/d27/f1e [729498,54767] 0 2024-04-25T01:36:13.557 INFO:tasks.workunit.client.1.smithi138.stdout:9/664: rename d4/d1a/d33/f36 to d4/d1a/d3c/dc2/fe0 0 2024-04-25T01:36:13.559 INFO:tasks.workunit.client.1.smithi138.stdout:0/583: dread d6/d39/d58/f87 [4194304,4194304] 0 2024-04-25T01:36:13.560 INFO:tasks.workunit.client.1.smithi138.stdout:4/542: symlink d0/d45/d7f/l95 0 2024-04-25T01:36:13.560 INFO:tasks.workunit.client.1.smithi138.stdout:9/665: truncate d4/d1a/d68/da5/fb4 352334 0 2024-04-25T01:36:13.660 INFO:tasks.workunit.client.1.smithi138.stdout:3/847: rename db/df/d17/d23/d34/f5f to db/df/d17/d20/d57/d6a/d110/f111 0 2024-04-25T01:36:13.660 INFO:tasks.workunit.client.1.smithi138.stdout:6/825: write dc/d10/d67/f65 [1128810,94665] 0 2024-04-25T01:36:13.660 INFO:tasks.workunit.client.1.smithi138.stdout:4/543: chown d0/d16/l75 276484258 1 2024-04-25T01:36:13.661 INFO:tasks.workunit.client.1.smithi138.stdout:0/584: write d6/df/d66/f7f [549638,110045] 0 2024-04-25T01:36:13.661 INFO:tasks.workunit.client.1.smithi138.stdout:1/695: creat d3/d4/da7/db1/de1/fea x:0 0 0 2024-04-25T01:36:13.661 INFO:tasks.workunit.client.1.smithi138.stdout:0/585: stat d6/d39/fa2 0 2024-04-25T01:36:13.662 INFO:tasks.workunit.client.1.smithi138.stdout:6/826: chown dc/d2e/d37/f52 788633642 1 2024-04-25T01:36:13.664 INFO:tasks.workunit.client.1.smithi138.stdout:9/666: creat d4/d8/d90/d8e/fe1 x:0 0 0 2024-04-25T01:36:13.665 INFO:tasks.workunit.client.1.smithi138.stdout:4/544: mknod d0/d1c/d52/c96 0 2024-04-25T01:36:13.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/667: fdatasync d4/f4a 0 2024-04-25T01:36:13.665 INFO:tasks.workunit.client.1.smithi138.stdout:4/545: write d0/f64 [2703351,76759] 0 2024-04-25T01:36:13.666 INFO:tasks.workunit.client.1.smithi138.stdout:1/696: mkdir d3/ddb/dd6/deb 0 2024-04-25T01:36:13.667 INFO:tasks.workunit.client.1.smithi138.stdout:0/586: rename d6/f28 to d6/d2f/d75/fb9 0 2024-04-25T01:36:13.673 INFO:tasks.workunit.client.1.smithi138.stdout:6/827: rename dc/d10/d6d/l84 to dc/d2e/d37/dee/l101 0 2024-04-25T01:36:13.674 INFO:tasks.workunit.client.1.smithi138.stdout:6/828: mknod dc/d10/d28/d80/c102 0 2024-04-25T01:36:13.678 INFO:tasks.workunit.client.1.smithi138.stdout:6/829: link dc/d10/f15 dc/d2e/db9/f103 0 2024-04-25T01:36:13.732 INFO:tasks.workunit.client.1.smithi138.stdout:2/911: sync 2024-04-25T01:36:13.736 INFO:tasks.workunit.client.1.smithi138.stdout:4/546: write d0/f30 [374993,15594] 0 2024-04-25T01:36:13.737 INFO:tasks.workunit.client.1.smithi138.stdout:1/697: write d3/f13 [120545,6050] 0 2024-04-25T01:36:13.738 INFO:tasks.workunit.client.1.smithi138.stdout:3/848: dread db/d14/f32 [4194304,4194304] 0 2024-04-25T01:36:13.739 INFO:tasks.workunit.client.1.smithi138.stdout:6/830: symlink dc/d2e/d55/l104 0 2024-04-25T01:36:13.740 INFO:tasks.workunit.client.1.smithi138.stdout:4/547: rmdir d0/d1c/d48 39 2024-04-25T01:36:13.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:13 smithi033 ceph-mon[26353]: Reconfiguring node-exporter.smithi033 (dependencies changed)... 2024-04-25T01:36:13.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:13 smithi033 ceph-mon[26353]: Reconfiguring daemon node-exporter.smithi033 on smithi033 2024-04-25T01:36:13.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:13 smithi033 ceph-mon[26353]: pgmap v22: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 119 MiB/s rd, 335 MiB/s wr, 687 op/s 2024-04-25T01:36:13.742 INFO:tasks.workunit.client.1.smithi138.stdout:1/698: creat d3/d45/d72/d57/fec x:0 0 0 2024-04-25T01:36:13.744 INFO:tasks.workunit.client.1.smithi138.stdout:3/849: rename db/d14 to db/df/de2/d10a/d112 0 2024-04-25T01:36:13.744 INFO:tasks.workunit.client.1.smithi138.stdout:3/850: dread - db/d9b/fc2 zero size 2024-04-25T01:36:13.745 INFO:tasks.workunit.client.1.smithi138.stdout:3/851: stat db/df/de2/d10a/d112/d7a 0 2024-04-25T01:36:13.746 INFO:tasks.workunit.client.1.smithi138.stdout:4/548: creat d0/d16/d4a/d74/f97 x:0 0 0 2024-04-25T01:36:13.746 INFO:tasks.workunit.client.1.smithi138.stdout:3/852: dread - db/dc5/f104 zero size 2024-04-25T01:36:13.748 INFO:tasks.workunit.client.1.smithi138.stdout:1/699: mkdir d3/d27/d16/da3/daf/ded 0 2024-04-25T01:36:13.750 INFO:tasks.workunit.client.1.smithi138.stdout:4/549: stat d0/d1c/d48/f92 0 2024-04-25T01:36:13.751 INFO:tasks.workunit.client.1.smithi138.stdout:1/700: dread d3/d45/d72/d57/f91 [0,4194304] 0 2024-04-25T01:36:13.751 INFO:tasks.workunit.client.1.smithi138.stdout:4/550: write d0/d16/d4a/f77 [2614516,91156] 0 2024-04-25T01:36:13.753 INFO:tasks.workunit.client.1.smithi138.stdout:3/853: mknod db/df/d19/c113 0 2024-04-25T01:36:13.753 INFO:tasks.workunit.client.1.smithi138.stdout:2/912: dread d0/d21/f3a [4194304,4194304] 0 2024-04-25T01:36:13.756 INFO:tasks.workunit.client.1.smithi138.stdout:3/854: write db/df/d17/d5d/fa8 [6105581,85817] 0 2024-04-25T01:36:13.757 INFO:tasks.workunit.client.1.smithi138.stdout:4/551: unlink d0/d16/d7e/f91 0 2024-04-25T01:36:13.760 INFO:tasks.workunit.client.1.smithi138.stdout:1/701: dwrite d3/d27/d3e/f61 [0,4194304] 0 2024-04-25T01:36:13.763 INFO:tasks.workunit.client.1.smithi138.stdout:3/855: unlink db/d9b/db4/df1/cf5 0 2024-04-25T01:36:13.764 INFO:tasks.workunit.client.1.smithi138.stdout:2/913: rename d0/d21/c66 to d0/d85/da5/db4/d15/d51/df2/c124 0 2024-04-25T01:36:13.766 INFO:tasks.workunit.client.1.smithi138.stdout:3/856: symlink db/df/dd1/l114 0 2024-04-25T01:36:13.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:13 smithi138 ceph-mon[32977]: Reconfiguring node-exporter.smithi033 (dependencies changed)... 2024-04-25T01:36:13.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:13 smithi138 ceph-mon[32977]: Reconfiguring daemon node-exporter.smithi033 on smithi033 2024-04-25T01:36:13.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:13 smithi138 ceph-mon[32977]: pgmap v22: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 119 MiB/s rd, 335 MiB/s wr, 687 op/s 2024-04-25T01:36:13.767 INFO:tasks.workunit.client.1.smithi138.stdout:3/857: dread - db/df/d19/d1a/de5/f106 zero size 2024-04-25T01:36:13.769 INFO:tasks.workunit.client.1.smithi138.stdout:1/702: rename d3/d27/d2d/c2e to d3/d4/da7/db1/de4/daa/cee 0 2024-04-25T01:36:13.776 INFO:tasks.workunit.client.1.smithi138.stdout:3/858: link db/db3/d101/l10e db/df/d19/d1a/l115 0 2024-04-25T01:36:13.776 INFO:tasks.workunit.client.1.smithi138.stdout:1/703: dwrite d3/d45/d72/d57/d8e/fbe [0,4194304] 0 2024-04-25T01:36:13.778 INFO:tasks.workunit.client.1.smithi138.stdout:3/859: rmdir db/df/d17/d23/d88 39 2024-04-25T01:36:13.779 INFO:tasks.workunit.client.1.smithi138.stdout:4/552: write d0/d1c/d52/f65 [2880839,29547] 0 2024-04-25T01:36:13.780 INFO:tasks.workunit.client.1.smithi138.stdout:1/704: mknod d3/d45/d72/cef 0 2024-04-25T01:36:13.781 INFO:tasks.workunit.client.1.smithi138.stdout:1/705: write d3/d27/d3e/fbf [2621399,67121] 0 2024-04-25T01:36:13.783 INFO:tasks.workunit.client.1.smithi138.stdout:1/706: write d3/d27/d3e/d51/f80 [3373284,59602] 0 2024-04-25T01:36:13.783 INFO:tasks.workunit.client.1.smithi138.stdout:3/860: mknod db/df/d19/d1a/dc4/c116 0 2024-04-25T01:36:13.785 INFO:tasks.workunit.client.1.smithi138.stdout:1/707: dread - d3/d45/d72/d57/db0/fe9 zero size 2024-04-25T01:36:13.789 INFO:tasks.workunit.client.1.smithi138.stdout:2/914: truncate d0/f8 1906399 0 2024-04-25T01:36:13.791 INFO:tasks.workunit.client.1.smithi138.stdout:4/553: rename d0/d1c/d1e/c4b to d0/c98 0 2024-04-25T01:36:13.793 INFO:tasks.workunit.client.1.smithi138.stdout:1/708: dwrite d3/db/f5f [4194304,4194304] 0 2024-04-25T01:36:13.794 INFO:tasks.workunit.client.1.smithi138.stdout:1/709: stat d3/d27/d16/l4b 0 2024-04-25T01:36:13.795 INFO:tasks.workunit.client.1.smithi138.stdout:1/710: read - d3/d4/fe8 zero size 2024-04-25T01:36:13.798 INFO:tasks.workunit.client.1.smithi138.stdout:4/554: rename d0/f2f to d0/d16/d33/f99 0 2024-04-25T01:36:13.799 INFO:tasks.workunit.client.1.smithi138.stdout:1/711: rmdir d3/d27/d16/da3 39 2024-04-25T01:36:13.803 INFO:tasks.workunit.client.1.smithi138.stdout:4/555: rename d0/d16/l27 to d0/d1c/d48/l9a 0 2024-04-25T01:36:13.803 INFO:tasks.workunit.client.1.smithi138.stdout:1/712: fsync d3/d27/d2d/da1/dca/fda 0 2024-04-25T01:36:13.804 INFO:tasks.workunit.client.1.smithi138.stdout:4/556: fdatasync d0/f64 0 2024-04-25T01:36:13.804 INFO:tasks.workunit.client.1.smithi138.stdout:3/861: dread db/df/de2/d10a/d112/d40/d41/f8b [0,4194304] 0 2024-04-25T01:36:13.811 INFO:tasks.workunit.client.1.smithi138.stdout:3/862: mkdir db/d9b/db4/d117 0 2024-04-25T01:36:13.815 INFO:tasks.workunit.client.1.smithi138.stdout:4/557: dwrite d0/d16/d4a/d74/f7a [0,4194304] 0 2024-04-25T01:36:13.816 INFO:tasks.workunit.client.1.smithi138.stdout:1/713: dwrite d3/d45/d72/d57/fc0 [0,4194304] 0 2024-04-25T01:36:13.818 INFO:tasks.workunit.client.1.smithi138.stdout:1/714: write d3/d27/d3e/fb5 [80758,76757] 0 2024-04-25T01:36:13.823 INFO:tasks.workunit.client.1.smithi138.stdout:4/558: read d0/d1c/d1e/f69 [3145424,117614] 0 2024-04-25T01:36:13.826 INFO:tasks.workunit.client.1.smithi138.stdout:3/863: dwrite db/df/d17/d23/d88/fc3 [0,4194304] 0 2024-04-25T01:36:13.827 INFO:tasks.workunit.client.1.smithi138.stdout:1/715: rename d3/ddb/c60 to d3/d4/da7/cf0 0 2024-04-25T01:36:13.828 INFO:tasks.workunit.client.1.smithi138.stdout:1/716: write d3/d4/fe8 [925256,32357] 0 2024-04-25T01:36:13.833 INFO:tasks.workunit.client.1.smithi138.stdout:8/881: sync 2024-04-25T01:36:13.837 INFO:tasks.workunit.client.1.smithi138.stdout:2/915: dread d0/d85/da5/db4/d15/d44/d4c/d56/da2/f26 [0,4194304] 0 2024-04-25T01:36:13.841 INFO:tasks.workunit.client.1.smithi138.stdout:8/882: unlink d3/d44/d48/fa3 0 2024-04-25T01:36:13.841 INFO:tasks.workunit.client.1.smithi138.stdout:2/916: chown d0/d85/da5/db4/f3c 1309192418 1 2024-04-25T01:36:13.841 INFO:tasks.workunit.client.1.smithi138.stdout:3/864: dwrite db/df/de2/d10a/d112/d40/d41/f8b [4194304,4194304] 0 2024-04-25T01:36:13.844 INFO:tasks.workunit.client.1.smithi138.stdout:3/865: stat db/d9b/fb7 0 2024-04-25T01:36:13.844 INFO:tasks.workunit.client.1.smithi138.stdout:1/717: dwrite d3/ddb/fc2 [0,4194304] 0 2024-04-25T01:36:13.848 INFO:tasks.workunit.client.1.smithi138.stdout:1/718: truncate d3/d27/d2d/f79 2268858 0 2024-04-25T01:36:13.852 INFO:tasks.workunit.client.1.smithi138.stdout:2/917: symlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/l125 0 2024-04-25T01:36:13.853 INFO:tasks.workunit.client.1.smithi138.stdout:3/866: dread db/fc7 [0,4194304] 0 2024-04-25T01:36:13.857 INFO:tasks.workunit.client.1.smithi138.stdout:3/867: readlink l4 0 2024-04-25T01:36:13.857 INFO:tasks.workunit.client.1.smithi138.stdout:3/868: chown db/df/d17/d23/f79 27 1 2024-04-25T01:36:13.857 INFO:tasks.workunit.client.1.smithi138.stdout:2/918: rmdir d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e 39 2024-04-25T01:36:13.861 INFO:tasks.workunit.client.1.smithi138.stdout:2/919: creat d0/d21/d45/f126 x:0 0 0 2024-04-25T01:36:13.864 INFO:tasks.workunit.client.1.smithi138.stdout:3/869: rename db/c4c to db/df/d17/d23/c118 0 2024-04-25T01:36:13.865 INFO:tasks.workunit.client.1.smithi138.stdout:8/883: dwrite d3/d16/d88/da/de/f13 [0,4194304] 0 2024-04-25T01:36:13.868 INFO:tasks.workunit.client.1.smithi138.stdout:8/884: dread - d3/d44/dae/fd4 zero size 2024-04-25T01:36:13.868 INFO:tasks.workunit.client.1.smithi138.stdout:4/559: dread d0/d16/f31 [0,4194304] 0 2024-04-25T01:36:13.870 INFO:tasks.workunit.client.1.smithi138.stdout:3/870: symlink db/df/de2/d10a/d112/d7a/l119 0 2024-04-25T01:36:13.870 INFO:tasks.workunit.client.1.smithi138.stdout:1/719: truncate d3/d6d/f92 3663083 0 2024-04-25T01:36:13.872 INFO:tasks.workunit.client.1.smithi138.stdout:4/560: creat d0/d16/d33/f9b x:0 0 0 2024-04-25T01:36:13.873 INFO:tasks.workunit.client.1.smithi138.stdout:4/561: read d0/d16/d4a/d74/f7a [2262991,61667] 0 2024-04-25T01:36:13.875 INFO:tasks.workunit.client.1.smithi138.stdout:8/885: mknod d3/d16/d1f/d31/d82/d8a/c113 0 2024-04-25T01:36:13.877 INFO:tasks.workunit.client.1.smithi138.stdout:3/871: chown db/df/d17/f5a 1021 1 2024-04-25T01:36:13.877 INFO:tasks.workunit.client.1.smithi138.stdout:2/920: write d0/fe [1313108,130748] 0 2024-04-25T01:36:13.878 INFO:tasks.workunit.client.1.smithi138.stdout:1/720: mknod d3/d45/d72/d57/db0/cf1 0 2024-04-25T01:36:13.880 INFO:tasks.workunit.client.1.smithi138.stdout:3/872: rmdir db/db3/d101 39 2024-04-25T01:36:13.881 INFO:tasks.workunit.client.1.smithi138.stdout:1/721: write d3/d27/d3e/fb5 [183419,48666] 0 2024-04-25T01:36:13.882 INFO:tasks.workunit.client.1.smithi138.stdout:8/886: symlink d3/d16/d88/da/db/d15/l114 0 2024-04-25T01:36:13.882 INFO:tasks.workunit.client.1.smithi138.stdout:4/562: dwrite d0/f3f [0,4194304] 0 2024-04-25T01:36:13.884 INFO:tasks.workunit.client.1.smithi138.stdout:8/887: fsync d3/d16/f3b 0 2024-04-25T01:36:13.885 INFO:tasks.workunit.client.1.smithi138.stdout:2/921: fsync d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fb1 0 2024-04-25T01:36:13.889 INFO:tasks.workunit.client.1.smithi138.stdout:1/722: write d3/d27/d24/da9/fe7 [911944,105604] 0 2024-04-25T01:36:13.896 INFO:tasks.workunit.client.1.smithi138.stdout:3/873: dwrite db/df/fbd [0,4194304] 0 2024-04-25T01:36:13.897 INFO:tasks.workunit.client.1.smithi138.stdout:4/563: dwrite d0/d1c/d52/f66 [0,4194304] 0 2024-04-25T01:36:13.900 INFO:tasks.workunit.client.1.smithi138.stdout:3/874: write db/f35 [1883749,88420] 0 2024-04-25T01:36:13.905 INFO:tasks.workunit.client.1.smithi138.stdout:4/564: rmdir d0/d16/d7e 39 2024-04-25T01:36:13.905 INFO:tasks.workunit.client.1.smithi138.stdout:1/723: dwrite d3/d45/d72/d57/f63 [0,4194304] 0 2024-04-25T01:36:13.908 INFO:tasks.workunit.client.1.smithi138.stdout:2/922: dwrite d0/d21/d45/d8a/d8e/fd6 [0,4194304] 0 2024-04-25T01:36:13.913 INFO:tasks.workunit.client.1.smithi138.stdout:1/724: symlink d3/d4/d53/lf2 0 2024-04-25T01:36:13.914 INFO:tasks.workunit.client.1.smithi138.stdout:1/725: dread - d3/d45/d72/db2/fdc zero size 2024-04-25T01:36:13.916 INFO:tasks.workunit.client.1.smithi138.stdout:1/726: write d3/f2c [2932980,119437] 0 2024-04-25T01:36:13.918 INFO:tasks.workunit.client.1.smithi138.stdout:2/923: rename d0/d85/da5/db4/d15/d44/d4c/d56/d6d/lec to d0/d85/da5/db4/d15/d44/d4c/d56/d106/l127 0 2024-04-25T01:36:13.923 INFO:tasks.workunit.client.1.smithi138.stdout:3/875: dwrite db/f9d [0,4194304] 0 2024-04-25T01:36:13.923 INFO:tasks.workunit.client.1.smithi138.stdout:4/565: link d0/d1c/f62 d0/d16/d33/f9c 0 2024-04-25T01:36:13.933 INFO:tasks.workunit.client.1.smithi138.stdout:2/924: rename d0/d85/da5/db4/d15/d44/d4c/d56/c7d to d0/de9/df0/d10f/c128 0 2024-04-25T01:36:13.933 INFO:tasks.workunit.client.1.smithi138.stdout:4/566: dwrite d0/f30 [0,4194304] 0 2024-04-25T01:36:13.935 INFO:tasks.workunit.client.1.smithi138.stdout:2/925: fdatasync d0/d85/da5/d35/f11d 0 2024-04-25T01:36:13.937 INFO:tasks.workunit.client.1.smithi138.stdout:2/926: write d0/d85/da5/f81 [450329,13803] 0 2024-04-25T01:36:13.940 INFO:tasks.workunit.client.1.smithi138.stdout:1/727: creat d3/ddb/dd6/dde/ff3 x:0 0 0 2024-04-25T01:36:13.942 INFO:tasks.workunit.client.1.smithi138.stdout:3/876: mknod db/df/d17/c11a 0 2024-04-25T01:36:13.942 INFO:tasks.workunit.client.1.smithi138.stdout:1/728: stat d3/d45/d72/f95 0 2024-04-25T01:36:13.942 INFO:tasks.workunit.client.1.smithi138.stdout:4/567: mknod d0/d1c/d86/c9d 0 2024-04-25T01:36:13.942 INFO:tasks.workunit.client.1.smithi138.stdout:1/729: stat d3/d45/d72/f42 0 2024-04-25T01:36:13.944 INFO:tasks.workunit.client.1.smithi138.stdout:1/730: fsync d3/d45/d72/d57/db0/fe9 0 2024-04-25T01:36:13.945 INFO:tasks.workunit.client.1.smithi138.stdout:4/568: symlink d0/d16/d33/l9e 0 2024-04-25T01:36:13.947 INFO:tasks.workunit.client.1.smithi138.stdout:1/731: truncate d3/d45/d72/d57/db0/fe9 93126 0 2024-04-25T01:36:13.947 INFO:tasks.workunit.client.1.smithi138.stdout:2/927: rmdir d0/d85/da5/db4/d15/d44/d114 0 2024-04-25T01:36:13.951 INFO:tasks.workunit.client.1.smithi138.stdout:1/732: mkdir d3/d27/d3e/df4 0 2024-04-25T01:36:13.953 INFO:tasks.workunit.client.1.smithi138.stdout:2/928: dwrite d0/d85/da5/db4/f9d [0,4194304] 0 2024-04-25T01:36:13.954 INFO:tasks.workunit.client.1.smithi138.stdout:1/733: symlink d3/ddb/dd6/dde/lf5 0 2024-04-25T01:36:13.958 INFO:tasks.workunit.client.1.smithi138.stdout:1/734: symlink d3/d4/da7/db1/de1/lf6 0 2024-04-25T01:36:13.965 INFO:tasks.workunit.client.1.smithi138.stdout:1/735: dwrite d3/d4/da7/db1/de4/fc5 [0,4194304] 0 2024-04-25T01:36:13.970 INFO:tasks.workunit.client.1.smithi138.stdout:4/569: getdents d0/d16/d33 0 2024-04-25T01:36:13.970 INFO:tasks.workunit.client.1.smithi138.stdout:1/736: dread d3/d45/f48 [4194304,4194304] 0 2024-04-25T01:36:13.971 INFO:tasks.workunit.client.1.smithi138.stdout:4/570: fsync d0/f1a 0 2024-04-25T01:36:13.972 INFO:tasks.workunit.client.1.smithi138.stdout:4/571: readlink d0/d16/d4a/d74/l80 0 2024-04-25T01:36:13.975 INFO:tasks.workunit.client.1.smithi138.stdout:2/929: rename d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/f6c to d0/d85/da5/db4/d15/d44/d4c/d56/d106/f129 0 2024-04-25T01:36:13.977 INFO:tasks.workunit.client.1.smithi138.stdout:4/572: unlink d0/d16/f43 0 2024-04-25T01:36:13.978 INFO:tasks.workunit.client.1.smithi138.stdout:4/573: chown d0/d1c/l1d 80 1 2024-04-25T01:36:13.978 INFO:tasks.workunit.client.1.smithi138.stdout:1/737: mknod d3/d27/d3e/df4/cf7 0 2024-04-25T01:36:13.980 INFO:tasks.workunit.client.1.smithi138.stdout:4/574: truncate d0/d16/f31 8770392 0 2024-04-25T01:36:13.986 INFO:tasks.workunit.client.1.smithi138.stdout:1/738: dwrite d3/ddb/fc2 [0,4194304] 0 2024-04-25T01:36:13.986 INFO:tasks.workunit.client.1.smithi138.stdout:4/575: getdents d0/d16 0 2024-04-25T01:36:13.991 INFO:tasks.workunit.client.1.smithi138.stdout:4/576: rename d0/d16 to d0/d1c/d1e/d9f 0 2024-04-25T01:36:13.991 INFO:tasks.workunit.client.1.smithi138.stdout:1/739: creat d3/d27/d16/da3/daf/ff8 x:0 0 0 2024-04-25T01:36:13.992 INFO:tasks.workunit.client.1.smithi138.stdout:1/740: readlink d3/db/lad 0 2024-04-25T01:36:13.995 INFO:tasks.workunit.client.1.smithi138.stdout:4/577: rename d0/d1c/d48/c90 to d0/ca0 0 2024-04-25T01:36:13.996 INFO:tasks.workunit.client.1.smithi138.stdout:7/761: sync 2024-04-25T01:36:13.996 INFO:tasks.workunit.client.1.smithi138.stdout:5/772: sync 2024-04-25T01:36:13.996 INFO:tasks.workunit.client.1.smithi138.stdout:1/741: dread d3/d27/d3e/fb5 [0,4194304] 0 2024-04-25T01:36:13.997 INFO:tasks.workunit.client.1.smithi138.stdout:4/578: write d0/d45/d7f/f8a [610745,121780] 0 2024-04-25T01:36:13.999 INFO:tasks.workunit.client.1.smithi138.stdout:2/930: dread d0/d85/da5/d35/f60 [0,4194304] 0 2024-04-25T01:36:14.000 INFO:tasks.workunit.client.1.smithi138.stdout:7/762: mknod da/de/d1b/d4b/cfb 0 2024-04-25T01:36:14.002 INFO:tasks.workunit.client.1.smithi138.stdout:4/579: creat d0/d1c/d52/fa1 x:0 0 0 2024-04-25T01:36:14.002 INFO:tasks.workunit.client.1.smithi138.stdout:1/742: mkdir d3/d45/d72/db2/dd9/df9 0 2024-04-25T01:36:14.004 INFO:tasks.workunit.client.1.smithi138.stdout:7/763: creat da/de/d1b/d34/ffc x:0 0 0 2024-04-25T01:36:14.005 INFO:tasks.workunit.client.1.smithi138.stdout:4/580: symlink d0/d1c/d1e/d9f/d33/la2 0 2024-04-25T01:36:14.008 INFO:tasks.workunit.client.1.smithi138.stdout:2/931: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/d6d/ff1 [0,4194304] 0 2024-04-25T01:36:14.008 INFO:tasks.workunit.client.1.smithi138.stdout:7/764: mknod da/de/d13/d21/dbc/cfd 0 2024-04-25T01:36:14.010 INFO:tasks.workunit.client.1.smithi138.stdout:2/932: chown d0/d21/l30 1015 1 2024-04-25T01:36:14.014 INFO:tasks.workunit.client.1.smithi138.stdout:1/743: dwrite d3/d27/d3e/fb5 [0,4194304] 0 2024-04-25T01:36:14.018 INFO:tasks.workunit.client.1.smithi138.stdout:2/933: stat d0/d85/da5/db4/d15/d44/d4c/d56/da2/cbc 0 2024-04-25T01:36:14.019 INFO:tasks.workunit.client.1.smithi138.stdout:4/581: write d0/d1c/d48/f6c [909183,104163] 0 2024-04-25T01:36:14.020 INFO:tasks.workunit.client.1.smithi138.stdout:4/582: write d0/d1c/d1e/d9f/d4a/f68 [2803707,35029] 0 2024-04-25T01:36:14.024 INFO:tasks.workunit.client.1.smithi138.stdout:4/583: mkdir d0/d1c/d52/da3 0 2024-04-25T01:36:14.025 INFO:tasks.workunit.client.1.smithi138.stdout:4/584: unlink d0/d59/c5a 0 2024-04-25T01:36:14.026 INFO:tasks.workunit.client.1.smithi138.stdout:2/934: write d0/d85/da5/db4/fb3 [907075,579] 0 2024-04-25T01:36:14.027 INFO:tasks.workunit.client.1.smithi138.stdout:2/935: readlink d0/d85/da5/db4/ld 0 2024-04-25T01:36:14.028 INFO:tasks.workunit.client.1.smithi138.stdout:4/585: creat d0/d1c/d52/da3/fa4 x:0 0 0 2024-04-25T01:36:14.029 INFO:tasks.workunit.client.1.smithi138.stdout:4/586: chown d0/f13 1763 1 2024-04-25T01:36:14.040 INFO:tasks.workunit.client.1.smithi138.stdout:1/744: dread d3/ddb/d9b/f68 [0,4194304] 0 2024-04-25T01:36:14.041 INFO:tasks.workunit.client.1.smithi138.stdout:1/745: write d3/d4/da7/db1/de1/fea [884933,130458] 0 2024-04-25T01:36:14.045 INFO:tasks.workunit.client.1.smithi138.stdout:1/746: dread d3/d27/d24/da9/fe7 [0,4194304] 0 2024-04-25T01:36:14.135 INFO:tasks.workunit.client.1.smithi138.stdout:2/936: readlink d0/d21/d45/l77 0 2024-04-25T01:36:14.137 INFO:tasks.workunit.client.1.smithi138.stdout:4/587: creat d0/d1c/d1e/d9f/fa5 x:0 0 0 2024-04-25T01:36:14.137 INFO:tasks.workunit.client.1.smithi138.stdout:4/588: stat d0/fa 0 2024-04-25T01:36:14.164 INFO:tasks.workunit.client.1.smithi138.stdout:4/589: rename d0/d1c/d1e/d9f/d33/l4c to d0/d1c/d1e/d9f/d4a/la6 0 2024-04-25T01:36:14.167 INFO:tasks.workunit.client.1.smithi138.stdout:1/747: unlink d3/d27/d2d/l94 0 2024-04-25T01:36:14.168 INFO:tasks.workunit.client.1.smithi138.stdout:2/937: symlink d0/d85/da5/db4/d15/d51/df2/df6/l12a 0 2024-04-25T01:36:14.172 INFO:tasks.workunit.client.1.smithi138.stdout:2/938: creat d0/de9/df0/f12b x:0 0 0 2024-04-25T01:36:14.174 INFO:tasks.workunit.client.1.smithi138.stdout:2/939: chown d0/d85/da5/d35/f40 3 1 2024-04-25T01:36:14.175 INFO:tasks.workunit.client.1.smithi138.stdout:1/748: dwrite d3/d27/d2d/f32 [0,4194304] 0 2024-04-25T01:36:14.175 INFO:tasks.workunit.client.1.smithi138.stdout:9/668: sync 2024-04-25T01:36:14.175 INFO:tasks.workunit.client.1.smithi138.stdout:8/888: sync 2024-04-25T01:36:14.175 INFO:tasks.workunit.client.1.smithi138.stdout:6/831: sync 2024-04-25T01:36:14.175 INFO:tasks.workunit.client.1.smithi138.stdout:0/587: sync 2024-04-25T01:36:14.178 INFO:tasks.workunit.client.1.smithi138.stdout:9/669: read - d4/d1a/d68/da0/fab zero size 2024-04-25T01:36:14.180 INFO:tasks.workunit.client.1.smithi138.stdout:0/588: truncate d6/df/d3a/f71 4425081 0 2024-04-25T01:36:14.184 INFO:tasks.workunit.client.1.smithi138.stdout:2/940: unlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/cff 0 2024-04-25T01:36:14.187 INFO:tasks.workunit.client.1.smithi138.stdout:6/832: rename dc/ccd to dc/d10/d67/d8e/c105 0 2024-04-25T01:36:14.190 INFO:tasks.workunit.client.1.smithi138.stdout:1/749: mkdir d3/d45/d72/db2/dfa 0 2024-04-25T01:36:14.193 INFO:tasks.workunit.client.1.smithi138.stdout:8/889: symlink d3/d16/d1f/dc5/l115 0 2024-04-25T01:36:14.195 INFO:tasks.workunit.client.1.smithi138.stdout:8/890: write d3/d44/dae/fd4 [99199,117857] 0 2024-04-25T01:36:14.196 INFO:tasks.workunit.client.1.smithi138.stdout:0/589: fsync d6/d39/d43/f6b 0 2024-04-25T01:36:14.198 INFO:tasks.workunit.client.1.smithi138.stdout:1/750: rmdir d3/d27/d3e/d51 39 2024-04-25T01:36:14.203 INFO:tasks.workunit.client.1.smithi138.stdout:6/833: getdents dc/d10/d67/d8e/df9 0 2024-04-25T01:36:14.203 INFO:tasks.workunit.client.1.smithi138.stdout:8/891: dwrite d3/ffe [0,4194304] 0 2024-04-25T01:36:14.206 INFO:tasks.workunit.client.1.smithi138.stdout:0/590: truncate d6/df/f2c 707644 0 2024-04-25T01:36:14.207 INFO:tasks.workunit.client.1.smithi138.stdout:0/591: read d6/d19/f77 [3827287,105699] 0 2024-04-25T01:36:14.211 INFO:tasks.workunit.client.1.smithi138.stdout:6/834: truncate dc/d2e/d37/d56/fa8 1190235 0 2024-04-25T01:36:14.212 INFO:tasks.workunit.client.1.smithi138.stdout:2/941: dread d0/d85/da5/db4/d15/d44/d4c/d56/da2/f107 [0,4194304] 0 2024-04-25T01:36:14.214 INFO:tasks.workunit.client.1.smithi138.stdout:0/592: truncate f1 5431574 0 2024-04-25T01:36:14.216 INFO:tasks.workunit.client.1.smithi138.stdout:6/835: mkdir dc/d10/d28/d70/d83/d106 0 2024-04-25T01:36:14.216 INFO:tasks.workunit.client.1.smithi138.stdout:8/892: dwrite d3/d16/d88/da/db/d15/f72 [0,4194304] 0 2024-04-25T01:36:14.216 INFO:tasks.workunit.client.1.smithi138.stdout:0/593: read d6/df/faf [503044,124088] 0 2024-04-25T01:36:14.218 INFO:tasks.workunit.client.1.smithi138.stdout:2/942: readlink d0/d21/d45/l68 0 2024-04-25T01:36:14.220 INFO:tasks.workunit.client.1.smithi138.stdout:4/590: write d0/d1c/d1e/d9f/d4a/f53 [318835,110274] 0 2024-04-25T01:36:14.220 INFO:tasks.workunit.client.1.smithi138.stdout:8/893: dread - d3/d16/d1f/dd2/dd6/f104 zero size 2024-04-25T01:36:14.222 INFO:tasks.workunit.client.1.smithi138.stdout:6/836: mknod dc/d10/d28/d70/d83/c107 0 2024-04-25T01:36:14.222 INFO:tasks.workunit.client.1.smithi138.stdout:0/594: write d6/d2f/fa1 [1633273,103658] 0 2024-04-25T01:36:14.226 INFO:tasks.workunit.client.1.smithi138.stdout:6/837: truncate dc/d8d/ffb 1001213 0 2024-04-25T01:36:14.226 INFO:tasks.workunit.client.1.smithi138.stdout:4/591: unlink d0/d1c/d1e/d9f/fa5 0 2024-04-25T01:36:14.227 INFO:tasks.workunit.client.1.smithi138.stdout:4/592: stat d0/d1c/d1e/d9f/f34 0 2024-04-25T01:36:14.228 INFO:tasks.workunit.client.1.smithi138.stdout:0/595: creat d6/d39/d52/fba x:0 0 0 2024-04-25T01:36:14.235 INFO:tasks.workunit.client.1.smithi138.stdout:8/894: rename d3/d16/d88/da/db/d15/c7e to d3/d16/d88/da/db/d15/d56/d8f/def/c116 0 2024-04-25T01:36:14.236 INFO:tasks.workunit.client.1.smithi138.stdout:8/895: fsync d3/d16/d88/da/f24 0 2024-04-25T01:36:14.237 INFO:tasks.workunit.client.1.smithi138.stdout:0/596: mkdir d6/d19/dbb 0 2024-04-25T01:36:14.238 INFO:tasks.workunit.client.1.smithi138.stdout:4/593: truncate d0/d1c/d1e/d9f/d33/f9c 1934435 0 2024-04-25T01:36:14.240 INFO:tasks.workunit.client.1.smithi138.stdout:2/943: write d0/d85/da5/db4/d15/d44/d4c/d56/d6d/f89 [1838972,110911] 0 2024-04-25T01:36:14.242 INFO:tasks.workunit.client.1.smithi138.stdout:6/838: creat dc/d2e/d55/f108 x:0 0 0 2024-04-25T01:36:14.244 INFO:tasks.workunit.client.1.smithi138.stdout:6/839: write dc/d7b/fb0 [3987602,104721] 0 2024-04-25T01:36:14.244 INFO:tasks.workunit.client.1.smithi138.stdout:8/896: creat d3/d16/d88/da/db/d15/d59/f117 x:0 0 0 2024-04-25T01:36:14.247 INFO:tasks.workunit.client.1.smithi138.stdout:4/594: dwrite d0/d1c/d1e/d9f/f31 [8388608,4194304] 0 2024-04-25T01:36:14.249 INFO:tasks.workunit.client.1.smithi138.stdout:0/597: mkdir d6/d39/d52/d9c/dbc 0 2024-04-25T01:36:14.250 INFO:tasks.workunit.client.1.smithi138.stdout:4/595: write d0/d1c/d1e/d9f/d4a/f8f [493902,19894] 0 2024-04-25T01:36:14.252 INFO:tasks.workunit.client.1.smithi138.stdout:2/944: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fb1 [0,4194304] 0 2024-04-25T01:36:14.253 INFO:tasks.workunit.client.1.smithi138.stdout:6/840: rename dc/d10/d6d/f87 to dc/dc9/f109 0 2024-04-25T01:36:14.255 INFO:tasks.workunit.client.1.smithi138.stdout:8/897: mkdir d3/d16/d1f/d2a/d4f/dac/d118 0 2024-04-25T01:36:14.258 INFO:tasks.workunit.client.1.smithi138.stdout:4/596: fsync d0/d59/f6a 0 2024-04-25T01:36:14.260 INFO:tasks.workunit.client.1.smithi138.stdout:0/598: mknod d6/cbd 0 2024-04-25T01:36:14.261 INFO:tasks.workunit.client.1.smithi138.stdout:8/898: read - d3/d16/d88/da/db/d15/d6f/f10f zero size 2024-04-25T01:36:14.261 INFO:tasks.workunit.client.1.smithi138.stdout:6/841: write dc/f19 [7051701,69215] 0 2024-04-25T01:36:14.265 INFO:tasks.workunit.client.1.smithi138.stdout:4/597: write d0/d1c/d48/f92 [1001776,112477] 0 2024-04-25T01:36:14.265 INFO:tasks.workunit.client.1.smithi138.stdout:2/945: dwrite d0/d85/ffe [0,4194304] 0 2024-04-25T01:36:14.267 INFO:tasks.workunit.client.1.smithi138.stdout:8/899: write d3/d16/d88/da/db/d15/d59/f117 [582795,84014] 0 2024-04-25T01:36:14.269 INFO:tasks.workunit.client.1.smithi138.stdout:0/599: rename d6/d39/d43/c4c to d6/df/d41/d49/cbe 0 2024-04-25T01:36:14.272 INFO:tasks.workunit.client.1.smithi138.stdout:4/598: unlink d0/d1c/d1e/c57 0 2024-04-25T01:36:14.273 INFO:tasks.workunit.client.1.smithi138.stdout:4/599: chown d0/d1c/d52/f8b 3950 1 2024-04-25T01:36:14.275 INFO:tasks.workunit.client.1.smithi138.stdout:6/842: rename dc/d2e/d37/d56 to dc/d10/d28/d70/d83/d10a 0 2024-04-25T01:36:14.278 INFO:tasks.workunit.client.1.smithi138.stdout:6/843: write dc/d10/f26 [4348584,5565] 0 2024-04-25T01:36:14.278 INFO:tasks.workunit.client.1.smithi138.stdout:0/600: chown d6/d39/d43/c2a 3057 1 2024-04-25T01:36:14.279 INFO:tasks.workunit.client.1.smithi138.stdout:0/601: chown d6/la4 0 1 2024-04-25T01:36:14.279 INFO:tasks.workunit.client.1.smithi138.stdout:6/844: write dc/d10/d28/d70/d83/d10a/f73 [4450814,18054] 0 2024-04-25T01:36:14.287 INFO:tasks.workunit.client.1.smithi138.stdout:6/845: dwrite dc/f19 [4194304,4194304] 0 2024-04-25T01:36:14.288 INFO:tasks.workunit.client.1.smithi138.stdout:6/846: chown dc/d2e/d55/d7e 3 1 2024-04-25T01:36:14.289 INFO:tasks.workunit.client.1.smithi138.stdout:6/847: dread - dc/d10/d67/fe5 zero size 2024-04-25T01:36:14.290 INFO:tasks.workunit.client.1.smithi138.stdout:8/900: dread d3/d16/d88/da/db/d15/d56/d8f/fc3 [0,4194304] 0 2024-04-25T01:36:14.293 INFO:tasks.workunit.client.1.smithi138.stdout:2/946: link d0/d85/da5/db4/d15/d44/d4c/d56/da2/cbc d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5/c12c 0 2024-04-25T01:36:14.295 INFO:tasks.workunit.client.1.smithi138.stdout:3/877: sync 2024-04-25T01:36:14.295 INFO:tasks.workunit.client.1.smithi138.stdout:4/600: getdents d0/d1c/d1e/d9f/d7e 0 2024-04-25T01:36:14.297 INFO:tasks.workunit.client.1.smithi138.stdout:6/848: mknod dc/d10/d28/d70/d83/d10a/d8c/db7/c10b 0 2024-04-25T01:36:14.299 INFO:tasks.workunit.client.1.smithi138.stdout:0/602: truncate d6/df/d3a/fae 1800497 0 2024-04-25T01:36:14.300 INFO:tasks.workunit.client.1.smithi138.stdout:0/603: truncate d6/d39/d43/fb3 584511 0 2024-04-25T01:36:14.302 INFO:tasks.workunit.client.1.smithi138.stdout:8/901: dread d3/d16/d88/da/db/d15/d5d/f62 [0,4194304] 0 2024-04-25T01:36:14.306 INFO:tasks.workunit.client.1.smithi138.stdout:6/849: rename dc/f60 to dc/d7b/f10c 0 2024-04-25T01:36:14.307 INFO:tasks.workunit.client.1.smithi138.stdout:2/947: mkdir d0/d85/da5/db4/d15/d44/d4c/d56/d9b/d12d 0 2024-04-25T01:36:14.309 INFO:tasks.workunit.client.1.smithi138.stdout:4/601: write d0/d1c/d1e/f69 [3602355,16384] 0 2024-04-25T01:36:14.311 INFO:tasks.workunit.client.1.smithi138.stdout:4/602: write d0/d1c/d1e/d9f/d4a/f8f [565736,47029] 0 2024-04-25T01:36:14.313 INFO:tasks.workunit.client.1.smithi138.stdout:8/902: rename d3/d16/d88/da/db/d15/d56/f9c to d3/d16/d88/da/de/d57/d100/f119 0 2024-04-25T01:36:14.316 INFO:tasks.workunit.client.1.smithi138.stdout:6/850: dread - dc/d2e/d55/d81/fb6 zero size 2024-04-25T01:36:14.318 INFO:tasks.workunit.client.1.smithi138.stdout:0/604: write d6/d2f/f7e [905632,51922] 0 2024-04-25T01:36:14.319 INFO:tasks.workunit.client.1.smithi138.stdout:4/603: creat d0/d1c/d1e/d9f/fa7 x:0 0 0 2024-04-25T01:36:14.320 INFO:tasks.workunit.client.1.smithi138.stdout:3/878: dwrite db/df/d17/d20/d57/d6a/f8d [0,4194304] 0 2024-04-25T01:36:14.322 INFO:tasks.workunit.client.1.smithi138.stdout:8/903: mknod d3/dde/c11a 0 2024-04-25T01:36:14.324 INFO:tasks.workunit.client.1.smithi138.stdout:0/605: stat d6/f26 0 2024-04-25T01:36:14.325 INFO:tasks.workunit.client.1.smithi138.stdout:0/606: chown d6/df/d3a/d67 0 1 2024-04-25T01:36:14.326 INFO:tasks.workunit.client.1.smithi138.stdout:6/851: truncate dc/d10/d28/d70/d83/d10a/f74 232449 0 2024-04-25T01:36:14.326 INFO:tasks.workunit.client.1.smithi138.stdout:4/604: mknod d0/d45/d7f/ca8 0 2024-04-25T01:36:14.327 INFO:tasks.workunit.client.1.smithi138.stdout:3/879: creat db/dc5/f11b x:0 0 0 2024-04-25T01:36:14.329 INFO:tasks.workunit.client.1.smithi138.stdout:2/948: rename d0/d21/d45/f115 to d0/d85/da5/db4/d15/d44/f12e 0 2024-04-25T01:36:14.331 INFO:tasks.workunit.client.1.smithi138.stdout:6/852: unlink dc/d10/d28/d70/d83/d10a/d8c/f99 0 2024-04-25T01:36:14.332 INFO:tasks.workunit.client.1.smithi138.stdout:2/949: stat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/l99 0 2024-04-25T01:36:14.333 INFO:tasks.workunit.client.1.smithi138.stdout:6/853: chown dc/d2e/ff0 62568 1 2024-04-25T01:36:14.336 INFO:tasks.workunit.client.1.smithi138.stdout:8/904: rename d3/d16/d88/lc to d3/d16/l11b 0 2024-04-25T01:36:14.337 INFO:tasks.workunit.client.1.smithi138.stdout:0/607: dread d6/d39/fa2 [0,4194304] 0 2024-04-25T01:36:14.337 INFO:tasks.workunit.client.1.smithi138.stdout:0/608: chown d6/d2f/c88 23 1 2024-04-25T01:36:14.338 INFO:tasks.workunit.client.1.smithi138.stdout:4/605: unlink d0/d1c/d1e/d9f/d33/f9c 0 2024-04-25T01:36:14.339 INFO:tasks.workunit.client.1.smithi138.stdout:4/606: chown d0/f40 84692689 1 2024-04-25T01:36:14.339 INFO:tasks.workunit.client.1.smithi138.stdout:2/950: symlink d0/d21/d45/d8a/d8e/l12f 0 2024-04-25T01:36:14.345 INFO:tasks.workunit.client.1.smithi138.stdout:4/607: truncate d0/fa 1560027 0 2024-04-25T01:36:14.346 INFO:tasks.workunit.client.1.smithi138.stdout:3/880: write db/df/de2/d10a/d112/f32 [6552188,68578] 0 2024-04-25T01:36:14.347 INFO:tasks.workunit.client.1.smithi138.stdout:2/951: creat d0/d85/da5/db4/d15/d44/d4c/d10d/f130 x:0 0 0 2024-04-25T01:36:14.348 INFO:tasks.workunit.client.1.smithi138.stdout:3/881: fsync db/df/d17/d20/d57/daa/fef 0 2024-04-25T01:36:14.350 INFO:tasks.workunit.client.1.smithi138.stdout:0/609: mkdir d6/d39/d52/d9c/dbc/dbf 0 2024-04-25T01:36:14.352 INFO:tasks.workunit.client.1.smithi138.stdout:2/952: dread d0/d85/da5/db4/d15/d44/d4c/d56/da2/f107 [0,4194304] 0 2024-04-25T01:36:14.352 INFO:tasks.workunit.client.1.smithi138.stdout:6/854: creat dc/d2e/d37/f10d x:0 0 0 2024-04-25T01:36:14.355 INFO:tasks.workunit.client.1.smithi138.stdout:8/905: creat d3/d16/d88/f11c x:0 0 0 2024-04-25T01:36:14.355 INFO:tasks.workunit.client.1.smithi138.stdout:3/882: mknod db/d9b/db4/df1/c11c 0 2024-04-25T01:36:14.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/608: symlink d0/d45/la9 0 2024-04-25T01:36:14.359 INFO:tasks.workunit.client.1.smithi138.stdout:3/883: write db/df/de2/d10a/d112/d40/d41/f8b [2300639,95548] 0 2024-04-25T01:36:14.361 INFO:tasks.workunit.client.1.smithi138.stdout:3/884: chown db/df/dd1/ce6 0 1 2024-04-25T01:36:14.361 INFO:tasks.workunit.client.1.smithi138.stdout:0/610: link d6/d39/d43/f21 d6/df/d66/fc0 0 2024-04-25T01:36:14.362 INFO:tasks.workunit.client.1.smithi138.stdout:3/885: chown db/df/d17/d20 22602010 1 2024-04-25T01:36:14.363 INFO:tasks.workunit.client.1.smithi138.stdout:4/609: chown d0/d1c/d1e/d9f/d33/f89 12273 1 2024-04-25T01:36:14.363 INFO:tasks.workunit.client.1.smithi138.stdout:3/886: dread - db/df/d17/d23/fe3 zero size 2024-04-25T01:36:14.364 INFO:tasks.workunit.client.1.smithi138.stdout:2/953: mkdir d0/d85/da5/db4/d15/d131 0 2024-04-25T01:36:14.366 INFO:tasks.workunit.client.1.smithi138.stdout:2/954: fsync d0/d85/da5/db4/d15/d44/d4c/d56/da2/fca 0 2024-04-25T01:36:14.368 INFO:tasks.workunit.client.1.smithi138.stdout:8/906: getdents d3/d16/d1f/dd2/dd6/dd7 0 2024-04-25T01:36:14.369 INFO:tasks.workunit.client.1.smithi138.stdout:6/855: dread dc/d2e/f31 [0,4194304] 0 2024-04-25T01:36:14.371 INFO:tasks.workunit.client.1.smithi138.stdout:2/955: creat d0/d21/d10b/f132 x:0 0 0 2024-04-25T01:36:14.372 INFO:tasks.workunit.client.1.smithi138.stdout:8/907: mknod d3/dde/c11d 0 2024-04-25T01:36:14.372 INFO:tasks.workunit.client.1.smithi138.stdout:6/856: chown dc/d10/f6b 1662966 1 2024-04-25T01:36:14.374 INFO:tasks.workunit.client.1.smithi138.stdout:4/610: rename d0/fa to d0/d1c/d1e/d9f/d4a/faa 0 2024-04-25T01:36:14.374 INFO:tasks.workunit.client.1.smithi138.stdout:8/908: write d3/d16/d88/da/db/fca [4023528,100793] 0 2024-04-25T01:36:14.377 INFO:tasks.workunit.client.1.smithi138.stdout:2/956: chown d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/l11f 74668855 1 2024-04-25T01:36:14.379 INFO:tasks.workunit.client.1.smithi138.stdout:8/909: symlink d3/d44/d48/l11e 0 2024-04-25T01:36:14.379 INFO:tasks.workunit.client.1.smithi138.stdout:0/611: dread d6/d1d/f3c [0,4194304] 0 2024-04-25T01:36:14.379 INFO:tasks.workunit.client.1.smithi138.stdout:4/611: write d0/d1c/d1e/d9f/d4a/f53 [5125052,6117] 0 2024-04-25T01:36:14.381 INFO:tasks.workunit.client.1.smithi138.stdout:2/957: creat d0/d85/f133 x:0 0 0 2024-04-25T01:36:14.383 INFO:tasks.workunit.client.1.smithi138.stdout:6/857: dwrite dc/d10/d28/d70/fdf [0,4194304] 0 2024-04-25T01:36:14.384 INFO:tasks.workunit.client.1.smithi138.stdout:2/958: chown d0/d21/l30 3632114 1 2024-04-25T01:36:14.386 INFO:tasks.workunit.client.1.smithi138.stdout:8/910: symlink d3/d16/d1f/d2a/d4f/dac/l11f 0 2024-04-25T01:36:14.390 INFO:tasks.workunit.client.1.smithi138.stdout:4/612: rename d0/d1c/f2b to d0/d1c/d1e/d9f/d7e/fab 0 2024-04-25T01:36:14.391 INFO:tasks.workunit.client.1.smithi138.stdout:6/858: symlink dc/d8d/l10e 0 2024-04-25T01:36:14.393 INFO:tasks.workunit.client.1.smithi138.stdout:6/859: write dc/d2e/dc1/fdb [930903,50961] 0 2024-04-25T01:36:14.394 INFO:tasks.workunit.client.1.smithi138.stdout:4/613: write d0/d1c/d1e/d9f/d33/f89 [926483,63026] 0 2024-04-25T01:36:14.395 INFO:tasks.workunit.client.1.smithi138.stdout:0/612: dwrite d6/d19/f85 [0,4194304] 0 2024-04-25T01:36:14.396 INFO:tasks.workunit.client.1.smithi138.stdout:2/959: symlink d0/d21/l134 0 2024-04-25T01:36:14.397 INFO:tasks.workunit.client.1.smithi138.stdout:4/614: dread - d0/d1c/d52/fa1 zero size 2024-04-25T01:36:14.397 INFO:tasks.workunit.client.1.smithi138.stdout:2/960: dread - d0/d85/ff9 zero size 2024-04-25T01:36:14.398 INFO:tasks.workunit.client.1.smithi138.stdout:0/613: chown d6/d2f/f7e 0 1 2024-04-25T01:36:14.400 INFO:tasks.workunit.client.1.smithi138.stdout:8/911: getdents d3/d44 0 2024-04-25T01:36:14.405 INFO:tasks.workunit.client.1.smithi138.stdout:4/615: creat d0/d1c/d52/da3/fac x:0 0 0 2024-04-25T01:36:14.406 INFO:tasks.workunit.client.1.smithi138.stdout:0/614: read d6/d39/d43/f9a [1285976,8100] 0 2024-04-25T01:36:14.407 INFO:tasks.workunit.client.1.smithi138.stdout:6/860: dwrite dc/d10/d28/f2b [4194304,4194304] 0 2024-04-25T01:36:14.414 INFO:tasks.workunit.client.1.smithi138.stdout:0/615: unlink d6/d39/c62 0 2024-04-25T01:36:14.416 INFO:tasks.workunit.client.1.smithi138.stdout:0/616: readlink d6/d2f/lb8 0 2024-04-25T01:36:14.416 INFO:tasks.workunit.client.1.smithi138.stdout:8/912: dread d3/d16/d1f/d2a/d4f/dac/ff2 [0,4194304] 0 2024-04-25T01:36:14.416 INFO:tasks.workunit.client.1.smithi138.stdout:6/861: truncate dc/d10/d28/f48 1488363 0 2024-04-25T01:36:14.417 INFO:tasks.workunit.client.1.smithi138.stdout:2/961: truncate d0/d85/da5/db4/d15/d44/d4c/d56/f74 248839 0 2024-04-25T01:36:14.418 INFO:tasks.workunit.client.1.smithi138.stdout:6/862: chown dc/d10/d28/d70/d83/f88 1702923 1 2024-04-25T01:36:14.420 INFO:tasks.workunit.client.1.smithi138.stdout:0/617: unlink d6/df/d69/l97 0 2024-04-25T01:36:14.423 INFO:tasks.workunit.client.1.smithi138.stdout:8/913: truncate d3/d16/d88/da/db/d15/d5d/f6b 4143481 0 2024-04-25T01:36:14.425 INFO:tasks.workunit.client.1.smithi138.stdout:4/616: link d0/d1c/d48/l9a d0/d1c/d1e/d9f/d4a/d74/lad 0 2024-04-25T01:36:14.425 INFO:tasks.workunit.client.1.smithi138.stdout:8/914: stat d3/d16/d1f/d2a/d4f/dcb/dce/f111 0 2024-04-25T01:36:14.426 INFO:tasks.workunit.client.1.smithi138.stdout:4/617: write d0/f3f [2618904,32876] 0 2024-04-25T01:36:14.426 INFO:tasks.workunit.client.1.smithi138.stdout:6/863: mknod dc/c10f 0 2024-04-25T01:36:14.428 INFO:tasks.workunit.client.1.smithi138.stdout:2/962: creat d0/d85/da5/d35/f135 x:0 0 0 2024-04-25T01:36:14.430 INFO:tasks.workunit.client.1.smithi138.stdout:8/915: fsync d3/d16/d88/da/db/d15/d5d/f62 0 2024-04-25T01:36:14.433 INFO:tasks.workunit.client.1.smithi138.stdout:0/618: rename d6/df/l27 to d6/lc1 0 2024-04-25T01:36:14.433 INFO:tasks.workunit.client.1.smithi138.stdout:4/618: mkdir d0/d1c/d52/dae 0 2024-04-25T01:36:14.434 INFO:tasks.workunit.client.1.smithi138.stdout:8/916: read d3/d16/d88/da/db/d15/f30 [1236698,99265] 0 2024-04-25T01:36:14.437 INFO:tasks.workunit.client.1.smithi138.stdout:0/619: unlink d6/df/f4d 0 2024-04-25T01:36:14.439 INFO:tasks.workunit.client.1.smithi138.stdout:6/864: dwrite dc/d10/dc8/ffc [0,4194304] 0 2024-04-25T01:36:14.440 INFO:tasks.workunit.client.1.smithi138.stdout:6/865: truncate dc/d2e/ff0 826288 0 2024-04-25T01:36:14.443 INFO:tasks.workunit.client.1.smithi138.stdout:8/917: symlink d3/dde/l120 0 2024-04-25T01:36:14.446 INFO:tasks.workunit.client.1.smithi138.stdout:6/866: fsync dc/d10/d28/f29 0 2024-04-25T01:36:14.446 INFO:tasks.workunit.client.1.smithi138.stdout:8/918: dread d3/d16/d1f/d2a/d4f/dac/ff2 [0,4194304] 0 2024-04-25T01:36:14.448 INFO:tasks.workunit.client.1.smithi138.stdout:8/919: chown d3/d16/d88/da/db/d15 0 1 2024-04-25T01:36:14.449 INFO:tasks.workunit.client.1.smithi138.stdout:6/867: write dc/d2e/d55/d81/d91/fc2 [2774458,37928] 0 2024-04-25T01:36:14.449 INFO:tasks.workunit.client.1.smithi138.stdout:8/920: chown d3/dde/ce1 0 1 2024-04-25T01:36:14.455 INFO:tasks.workunit.client.1.smithi138.stdout:0/620: symlink d6/d2f/lc2 0 2024-04-25T01:36:14.455 INFO:tasks.workunit.client.1.smithi138.stdout:2/963: rmdir d0/d85/da5/d35 39 2024-04-25T01:36:14.460 INFO:tasks.workunit.client.1.smithi138.stdout:6/868: mknod dc/d2e/d55/d81/c110 0 2024-04-25T01:36:14.461 INFO:tasks.workunit.client.1.smithi138.stdout:6/869: chown dc/d7b/f10c 10419856 1 2024-04-25T01:36:14.464 INFO:tasks.workunit.client.1.smithi138.stdout:0/621: rename d6/d1d/d98/f83 to d6/d39/d52/d9c/dbc/dbf/fc3 0 2024-04-25T01:36:14.465 INFO:tasks.workunit.client.1.smithi138.stdout:4/619: link d0/ca0 d0/d1c/d1e/d9f/caf 0 2024-04-25T01:36:14.473 INFO:tasks.workunit.client.1.smithi138.stdout:2/964: rename d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/dee/l34 to d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/l136 0 2024-04-25T01:36:14.474 INFO:tasks.workunit.client.1.smithi138.stdout:8/921: creat d3/d16/d88/da/de/f121 x:0 0 0 2024-04-25T01:36:14.476 INFO:tasks.workunit.client.1.smithi138.stdout:4/620: unlink d0/d1c/d1e/d9f/l37 0 2024-04-25T01:36:14.478 INFO:tasks.workunit.client.1.smithi138.stdout:6/870: creat dc/d10/d28/d70/d83/d106/f111 x:0 0 0 2024-04-25T01:36:14.482 INFO:tasks.workunit.client.1.smithi138.stdout:8/922: rename d3/d16/d1f/dd2/dd6/f104 to d3/d16/d1f/dd2/dd6/f122 0 2024-04-25T01:36:14.482 INFO:tasks.workunit.client.1.smithi138.stdout:0/622: write d6/f26 [5879505,61087] 0 2024-04-25T01:36:14.483 INFO:tasks.workunit.client.1.smithi138.stdout:6/871: rmdir dc/d10/d67 39 2024-04-25T01:36:14.487 INFO:tasks.workunit.client.1.smithi138.stdout:0/623: mknod d6/d1d/d98/cc4 0 2024-04-25T01:36:14.488 INFO:tasks.workunit.client.1.smithi138.stdout:4/621: link d0/c25 d0/d1c/d52/da3/cb0 0 2024-04-25T01:36:14.489 INFO:tasks.workunit.client.1.smithi138.stdout:2/965: getdents d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5 0 2024-04-25T01:36:14.491 INFO:tasks.workunit.client.1.smithi138.stdout:6/872: dwrite dc/d10/d28/f2b [8388608,4194304] 0 2024-04-25T01:36:14.491 INFO:tasks.workunit.client.1.smithi138.stdout:0/624: creat d6/d19/dbb/fc5 x:0 0 0 2024-04-25T01:36:14.495 INFO:tasks.workunit.client.1.smithi138.stdout:4/622: creat d0/d1c/d1e/d9f/d4a/fb1 x:0 0 0 2024-04-25T01:36:14.497 INFO:tasks.workunit.client.1.smithi138.stdout:6/873: chown dc/d10/d28/c32 25976 1 2024-04-25T01:36:14.498 INFO:tasks.workunit.client.1.smithi138.stdout:6/874: dread - dc/d2e/d55/f108 zero size 2024-04-25T01:36:14.498 INFO:tasks.workunit.client.1.smithi138.stdout:0/625: mkdir d6/d1d/d82/dc6 0 2024-04-25T01:36:14.499 INFO:tasks.workunit.client.1.smithi138.stdout:0/626: rename d6/d1d to d6/d1d/d98/dc7 22 2024-04-25T01:36:14.500 INFO:tasks.workunit.client.1.smithi138.stdout:0/627: dread - d6/d1d/d82/fad zero size 2024-04-25T01:36:14.504 INFO:tasks.workunit.client.1.smithi138.stdout:0/628: mknod d6/df/d69/cc8 0 2024-04-25T01:36:14.504 INFO:tasks.workunit.client.1.smithi138.stdout:8/923: dread d3/d44/d48/fc7 [0,4194304] 0 2024-04-25T01:36:14.505 INFO:tasks.workunit.client.1.smithi138.stdout:6/875: getdents dc/d8d/df5 0 2024-04-25T01:36:14.507 INFO:tasks.workunit.client.1.smithi138.stdout:2/966: dwrite d0/d85/da5/d35/f60 [0,4194304] 0 2024-04-25T01:36:14.507 INFO:tasks.workunit.client.1.smithi138.stdout:6/876: fdatasync dc/d10/d28/d70/f7f 0 2024-04-25T01:36:14.508 INFO:tasks.workunit.client.1.smithi138.stdout:5/773: sync 2024-04-25T01:36:14.508 INFO:tasks.workunit.client.1.smithi138.stdout:7/765: sync 2024-04-25T01:36:14.509 INFO:tasks.workunit.client.1.smithi138.stdout:8/924: fsync d3/d16/d88/da/db/d15/f21 0 2024-04-25T01:36:14.512 INFO:tasks.workunit.client.1.smithi138.stdout:8/925: chown d3 105 1 2024-04-25T01:36:14.513 INFO:tasks.workunit.client.1.smithi138.stdout:0/629: rename d6/lc1 to d6/d1d/d98/lc9 0 2024-04-25T01:36:14.513 INFO:tasks.workunit.client.1.smithi138.stdout:5/774: read - d4/da/d6e/d22/d35/dce/df9/d7f/fa4 zero size 2024-04-25T01:36:14.514 INFO:tasks.workunit.client.1.smithi138.stdout:4/623: dwrite d0/d1c/d48/f5b [4194304,4194304] 0 2024-04-25T01:36:14.514 INFO:tasks.workunit.client.1.smithi138.stdout:7/766: rmdir da/de/d13/d21/dbc/d94 39 2024-04-25T01:36:14.517 INFO:tasks.workunit.client.1.smithi138.stdout:0/630: symlink d6/df/d66/lca 0 2024-04-25T01:36:14.520 INFO:tasks.workunit.client.1.smithi138.stdout:7/767: stat da/de/d1b/d4b 0 2024-04-25T01:36:14.524 INFO:tasks.workunit.client.1.smithi138.stdout:5/775: symlink d4/da/d6e/d22/d35/d4d/d5e/l102 0 2024-04-25T01:36:14.526 INFO:tasks.workunit.client.1.smithi138.stdout:4/624: mknod d0/d1c/d1e/d9f/d4a/cb2 0 2024-04-25T01:36:14.526 INFO:tasks.workunit.client.1.smithi138.stdout:6/877: dwrite dc/d2e/d55/d7e/ffe [0,4194304] 0 2024-04-25T01:36:14.528 INFO:tasks.workunit.client.1.smithi138.stdout:8/926: dread d3/d16/d1f/d2a/d4f/dcb/dce/fe7 [0,4194304] 0 2024-04-25T01:36:14.528 INFO:tasks.workunit.client.1.smithi138.stdout:7/768: mknod da/de/d13/d30/df8/cfe 0 2024-04-25T01:36:14.530 INFO:tasks.workunit.client.1.smithi138.stdout:4/625: truncate d0/d1c/d1e/d9f/d4a/f77 4678683 0 2024-04-25T01:36:14.533 INFO:tasks.workunit.client.1.smithi138.stdout:6/878: write dc/d2e/d37/f10d [367497,66427] 0 2024-04-25T01:36:14.535 INFO:tasks.workunit.client.1.smithi138.stdout:0/631: creat d6/d2f/fcb x:0 0 0 2024-04-25T01:36:14.536 INFO:tasks.workunit.client.1.smithi138.stdout:2/967: link d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/l11f d0/d85/da5/db4/d15/d51/l137 0 2024-04-25T01:36:14.538 INFO:tasks.workunit.client.1.smithi138.stdout:1/751: sync 2024-04-25T01:36:14.539 INFO:tasks.workunit.client.1.smithi138.stdout:4/626: creat d0/d1c/d1e/d9f/fb3 x:0 0 0 2024-04-25T01:36:14.539 INFO:tasks.workunit.client.1.smithi138.stdout:3/887: sync 2024-04-25T01:36:14.539 INFO:tasks.workunit.client.1.smithi138.stdout:9/670: sync 2024-04-25T01:36:14.541 INFO:tasks.workunit.client.1.smithi138.stdout:4/627: write d0/d1c/d1e/d9f/d4a/f8f [448490,53399] 0 2024-04-25T01:36:14.544 INFO:tasks.workunit.client.1.smithi138.stdout:6/879: creat dc/d10/d28/d80/d89/da0/f112 x:0 0 0 2024-04-25T01:36:14.553 INFO:tasks.workunit.client.1.smithi138.stdout:6/880: dwrite dc/d8d/f9b [0,4194304] 0 2024-04-25T01:36:14.553 INFO:tasks.workunit.client.1.smithi138.stdout:9/671: stat d4/d8/d90/d52/l75 0 2024-04-25T01:36:14.555 INFO:tasks.workunit.client.1.smithi138.stdout:6/881: stat dc/d2e/d37/f10d 0 2024-04-25T01:36:14.557 INFO:tasks.workunit.client.1.smithi138.stdout:5/776: write d4/da/d27/f33 [942225,59061] 0 2024-04-25T01:36:14.558 INFO:tasks.workunit.client.1.smithi138.stdout:9/672: dread - d4/d8/d90/d52/fc0 zero size 2024-04-25T01:36:14.561 INFO:tasks.workunit.client.1.smithi138.stdout:8/927: dread d3/d16/d1f/d2a/f6a [0,4194304] 0 2024-04-25T01:36:14.562 INFO:tasks.workunit.client.1.smithi138.stdout:7/769: dread da/de/d13/fb5 [0,4194304] 0 2024-04-25T01:36:14.564 INFO:tasks.workunit.client.1.smithi138.stdout:6/882: mkdir dc/d10/d6d/d113 0 2024-04-25T01:36:14.565 INFO:tasks.workunit.client.1.smithi138.stdout:1/752: write d3/d27/d3e/d90/f9f [416458,120315] 0 2024-04-25T01:36:14.565 INFO:tasks.workunit.client.1.smithi138.stdout:0/632: dread d6/df/f10 [0,4194304] 0 2024-04-25T01:36:14.570 INFO:tasks.workunit.client.1.smithi138.stdout:5/777: creat d4/da/d6e/d3c/dd5/f103 x:0 0 0 2024-04-25T01:36:14.570 INFO:tasks.workunit.client.1.smithi138.stdout:3/888: dread db/df/d17/d20/f7d [0,4194304] 0 2024-04-25T01:36:14.571 INFO:tasks.workunit.client.1.smithi138.stdout:8/928: stat d3/d16/d88/da/de/l49 0 2024-04-25T01:36:14.571 INFO:tasks.workunit.client.1.smithi138.stdout:4/628: write d0/d1c/f28 [503923,21125] 0 2024-04-25T01:36:14.571 INFO:tasks.workunit.client.1.smithi138.stdout:5/778: stat d4/f9 0 2024-04-25T01:36:14.572 INFO:tasks.workunit.client.1.smithi138.stdout:2/968: creat d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/f138 x:0 0 0 2024-04-25T01:36:14.574 INFO:tasks.workunit.client.1.smithi138.stdout:7/770: rmdir da/de/d13/d95 39 2024-04-25T01:36:14.574 INFO:tasks.workunit.client.1.smithi138.stdout:2/969: chown d0/d21/d10b/f118 192 1 2024-04-25T01:36:14.577 INFO:tasks.workunit.client.1.smithi138.stdout:0/633: rmdir d6/d2f 39 2024-04-25T01:36:14.582 INFO:tasks.workunit.client.1.smithi138.stdout:7/771: chown da/de/d1b/d49/d81/l92 276 1 2024-04-25T01:36:14.584 INFO:tasks.workunit.client.1.smithi138.stdout:6/883: getdents dc/d2e/d55/d81/d91/de3 0 2024-04-25T01:36:14.587 INFO:tasks.workunit.client.1.smithi138.stdout:3/889: rmdir db/d9b/dac/df9 0 2024-04-25T01:36:14.587 INFO:tasks.workunit.client.1.smithi138.stdout:0/634: read d6/d39/d52/d9c/dbc/dbf/fc3 [313058,118924] 0 2024-04-25T01:36:14.590 INFO:tasks.workunit.client.1.smithi138.stdout:9/673: dwrite d4/d49/f76 [0,4194304] 0 2024-04-25T01:36:14.592 INFO:tasks.workunit.client.1.smithi138.stdout:8/929: truncate d3/d16/d88/da/db/d15/d56/f69 1067721 0 2024-04-25T01:36:14.594 INFO:tasks.workunit.client.1.smithi138.stdout:5/779: dwrite d4/da/d6e/d5c/ffe [0,4194304] 0 2024-04-25T01:36:14.596 INFO:tasks.workunit.client.1.smithi138.stdout:6/884: chown dc/d10/d28/d70/d83/d10a/fa8 364 1 2024-04-25T01:36:14.598 INFO:tasks.workunit.client.1.smithi138.stdout:0/635: symlink d6/df/lcc 0 2024-04-25T01:36:14.599 INFO:tasks.workunit.client.1.smithi138.stdout:4/629: dwrite d0/d1c/d1e/d9f/d4a/f53 [0,4194304] 0 2024-04-25T01:36:14.604 INFO:tasks.workunit.client.1.smithi138.stdout:7/772: mkdir da/de/d13/d21/dbc/dff 0 2024-04-25T01:36:14.604 INFO:tasks.workunit.client.1.smithi138.stdout:9/674: truncate d4/d8/f1f 1246955 0 2024-04-25T01:36:14.607 INFO:tasks.workunit.client.1.smithi138.stdout:6/885: rmdir dc/d2e/d55 39 2024-04-25T01:36:14.608 INFO:tasks.workunit.client.1.smithi138.stdout:3/890: dwrite db/df/de2/d10a/d112/d40/d41/fde [0,4194304] 0 2024-04-25T01:36:14.608 INFO:tasks.workunit.client.1.smithi138.stdout:1/753: dread d3/d45/d72/d57/d8e/fab [0,4194304] 0 2024-04-25T01:36:14.610 INFO:tasks.workunit.client.1.smithi138.stdout:0/636: mknod d6/d1d/d98/ccd 0 2024-04-25T01:36:14.610 INFO:tasks.workunit.client.1.smithi138.stdout:5/780: fsync d4/da/d6e/d22/d35/d4d/d5e/d8f/fb2 0 2024-04-25T01:36:14.613 INFO:tasks.workunit.client.1.smithi138.stdout:2/970: truncate d0/d85/da5/db4/d15/d44/d4c/fe8 2084420 0 2024-04-25T01:36:14.614 INFO:tasks.workunit.client.1.smithi138.stdout:6/886: dread - dc/d10/dc8/ff6 zero size 2024-04-25T01:36:14.614 INFO:tasks.workunit.client.1.smithi138.stdout:3/891: readlink db/df/d17/d20/l3e 0 2024-04-25T01:36:14.617 INFO:tasks.workunit.client.1.smithi138.stdout:2/971: chown d0/d85/da5/d35/f11d 249 1 2024-04-25T01:36:14.619 INFO:tasks.workunit.client.1.smithi138.stdout:9/675: read d4/d8/d90/d52/f7d [2912441,119196] 0 2024-04-25T01:36:14.621 INFO:tasks.workunit.client.1.smithi138.stdout:7/773: rename da/de/d1b/d4b/cfb to da/de/d1b/d49/c100 0 2024-04-25T01:36:14.624 INFO:tasks.workunit.client.1.smithi138.stdout:5/781: dwrite d4/da/d6e/d22/f24 [0,4194304] 0 2024-04-25T01:36:14.626 INFO:tasks.workunit.client.1.smithi138.stdout:2/972: symlink d0/d21/d10b/l139 0 2024-04-25T01:36:14.630 INFO:tasks.workunit.client.1.smithi138.stdout:0/637: dwrite d6/d2f/f7e [0,4194304] 0 2024-04-25T01:36:14.632 INFO:tasks.workunit.client.1.smithi138.stdout:9/676: rmdir d4/d8/d90/d52 39 2024-04-25T01:36:14.632 INFO:tasks.workunit.client.1.smithi138.stdout:4/630: link d0/d45/c5f d0/d1c/d1e/cb4 0 2024-04-25T01:36:14.634 INFO:tasks.workunit.client.1.smithi138.stdout:9/677: dread - d4/d1a/d68/da0/fab zero size 2024-04-25T01:36:14.634 INFO:tasks.workunit.client.1.smithi138.stdout:3/892: getdents db/df/dd1/ddf 0 2024-04-25T01:36:14.636 INFO:tasks.workunit.client.1.smithi138.stdout:3/893: read db/df/de2/d10a/d112/d40/fb0 [1412668,10410] 0 2024-04-25T01:36:14.636 INFO:tasks.workunit.client.1.smithi138.stdout:6/887: mkdir dc/d10/d28/d70/d83/d10a/d8c/dff/d114 0 2024-04-25T01:36:14.640 INFO:tasks.workunit.client.1.smithi138.stdout:1/754: write d3/d45/f7e [257059,81014] 0 2024-04-25T01:36:14.642 INFO:tasks.workunit.client.1.smithi138.stdout:5/782: unlink d4/da/d6e/d22/d74/d8e/fae 0 2024-04-25T01:36:14.644 INFO:tasks.workunit.client.1.smithi138.stdout:0/638: chown d6/d19/l65 19 1 2024-04-25T01:36:14.645 INFO:tasks.workunit.client.1.smithi138.stdout:0/639: dread - d6/d1d/d82/fad zero size 2024-04-25T01:36:14.646 INFO:tasks.workunit.client.1.smithi138.stdout:4/631: mkdir d0/d1c/d52/da3/db5 0 2024-04-25T01:36:14.652 INFO:tasks.workunit.client.1.smithi138.stdout:9/678: rename d4/d1a/d68/da5/fb4 to d4/d8/d2c/d82/fe2 0 2024-04-25T01:36:14.652 INFO:tasks.workunit.client.1.smithi138.stdout:3/894: dread - db/df/de2/d10a/d112/d7a/f92 zero size 2024-04-25T01:36:14.654 INFO:tasks.workunit.client.1.smithi138.stdout:8/930: truncate d3/d16/d88/da/db/d15/d56/f86 338380 0 2024-04-25T01:36:14.658 INFO:tasks.workunit.client.1.smithi138.stdout:1/755: read d3/d4/da7/db1/de4/daa/fd8 [132696,57520] 0 2024-04-25T01:36:14.660 INFO:tasks.workunit.client.1.smithi138.stdout:3/895: dwrite db/df/d17/d23/d34/fd8 [0,4194304] 0 2024-04-25T01:36:14.660 INFO:tasks.workunit.client.1.smithi138.stdout:0/640: symlink d6/d39/d43/lce 0 2024-04-25T01:36:14.662 INFO:tasks.workunit.client.1.smithi138.stdout:0/641: chown c2 24 1 2024-04-25T01:36:14.662 INFO:tasks.workunit.client.1.smithi138.stdout:4/632: symlink d0/d1c/d52/da3/lb6 0 2024-04-25T01:36:14.662 INFO:tasks.workunit.client.1.smithi138.stdout:3/896: chown db/df/d19/fd3 984 1 2024-04-25T01:36:14.664 INFO:tasks.workunit.client.1.smithi138.stdout:6/888: write dc/d2e/db9/f103 [2619429,8646] 0 2024-04-25T01:36:14.668 INFO:tasks.workunit.client.1.smithi138.stdout:7/774: write da/de/d1b/d49/d81/d51/faa [2938883,84586] 0 2024-04-25T01:36:14.668 INFO:tasks.workunit.client.1.smithi138.stdout:5/783: dread d4/da/d6e/d22/d74/f85 [0,4194304] 0 2024-04-25T01:36:14.673 INFO:tasks.workunit.client.1.smithi138.stdout:3/897: dread db/df/d17/d23/d34/fd8 [0,4194304] 0 2024-04-25T01:36:14.675 INFO:tasks.workunit.client.1.smithi138.stdout:7/775: rmdir da/de/d1b 39 2024-04-25T01:36:14.675 INFO:tasks.workunit.client.1.smithi138.stdout:9/679: dread d4/f17 [0,4194304] 0 2024-04-25T01:36:14.680 INFO:tasks.workunit.client.1.smithi138.stdout:2/973: truncate d0/d85/da5/d35/f60 279312 0 2024-04-25T01:36:14.686 INFO:tasks.workunit.client.1.smithi138.stdout:7/776: chown da/ld 49733025 1 2024-04-25T01:36:14.686 INFO:tasks.workunit.client.1.smithi138.stdout:0/642: dwrite d6/d1d/d82/fad [0,4194304] 0 2024-04-25T01:36:14.686 INFO:tasks.workunit.client.1.smithi138.stdout:4/633: dwrite d0/d1c/d52/da3/fa4 [0,4194304] 0 2024-04-25T01:36:14.698 INFO:tasks.workunit.client.1.smithi138.stdout:9/680: mkdir d4/d8/de3 0 2024-04-25T01:36:14.701 INFO:tasks.workunit.client.1.smithi138.stdout:1/756: rename d3/db/l7f to d3/d4/da7/lfb 0 2024-04-25T01:36:14.701 INFO:tasks.workunit.client.1.smithi138.stdout:9/681: fsync d4/d57/fce 0 2024-04-25T01:36:14.704 INFO:tasks.workunit.client.1.smithi138.stdout:2/974: write d0/d21/f27 [2437376,12452] 0 2024-04-25T01:36:14.705 INFO:tasks.workunit.client.1.smithi138.stdout:8/931: dread d3/d16/d88/da/db/d15/d20/f5a [0,4194304] 0 2024-04-25T01:36:14.705 INFO:tasks.workunit.client.1.smithi138.stdout:6/889: dwrite dc/d10/f22 [0,4194304] 0 2024-04-25T01:36:14.708 INFO:tasks.workunit.client.1.smithi138.stdout:3/898: dwrite db/f62 [0,4194304] 0 2024-04-25T01:36:14.713 INFO:tasks.workunit.client.1.smithi138.stdout:5/784: getdents d4/da/d6e/d22/d74/d8e/dd8 0 2024-04-25T01:36:14.716 INFO:tasks.workunit.client.1.smithi138.stdout:8/932: read - d3/d16/d1f/dd2/dd6/f122 zero size 2024-04-25T01:36:14.716 INFO:tasks.workunit.client.1.smithi138.stdout:6/890: creat dc/d10/d6d/f115 x:0 0 0 2024-04-25T01:36:14.718 INFO:tasks.workunit.client.1.smithi138.stdout:9/682: creat d4/d1a/fe4 x:0 0 0 2024-04-25T01:36:14.719 INFO:tasks.workunit.client.1.smithi138.stdout:2/975: truncate d0/d21/d45/d8a/d8e/ff7 212054 0 2024-04-25T01:36:14.720 INFO:tasks.workunit.client.1.smithi138.stdout:0/643: dread d6/d39/d58/f87 [0,4194304] 0 2024-04-25T01:36:14.720 INFO:tasks.workunit.client.1.smithi138.stdout:3/899: rename db/df/d17/d20/lc6 to db/d9b/db4/df1/l11d 0 2024-04-25T01:36:14.722 INFO:tasks.workunit.client.1.smithi138.stdout:2/976: chown d0/d21/d10b/f118 877086 1 2024-04-25T01:36:14.722 INFO:tasks.workunit.client.1.smithi138.stdout:6/891: rmdir dc 39 2024-04-25T01:36:14.725 INFO:tasks.workunit.client.1.smithi138.stdout:2/977: write d0/d85/da5/db4/d15/d44/d4c/d56/d6d/da3/dd5/df5/f11a [235303,61918] 0 2024-04-25T01:36:14.728 INFO:tasks.workunit.client.1.smithi138.stdout:5/785: symlink d4/da/d6e/d22/d35/dce/df9/l104 0 2024-04-25T01:36:14.728 INFO:tasks.workunit.client.1.smithi138.stdout:2/978: chown d0/d85/da5/db4/d15/d44/d4c/d56/da2/c104 963190430 1 2024-04-25T01:36:14.730 INFO:tasks.workunit.client.1.smithi138.stdout:3/900: creat db/df/d19/d1a/d9e/f11e x:0 0 0 2024-04-25T01:36:14.733 INFO:tasks.workunit.client.1.smithi138.stdout:7/777: dread da/de/d13/d30/f42 [0,4194304] 0 2024-04-25T01:36:14.736 INFO:tasks.workunit.client.1.smithi138.stdout:3/901: symlink db/df/d17/d20/d57/d6a/l11f 0 2024-04-25T01:36:14.737 INFO:tasks.workunit.client.1.smithi138.stdout:3/902: fdatasync db/df/d17/f9f 0 2024-04-25T01:36:14.739 INFO:tasks.workunit.client.1.smithi138.stdout:9/683: write d4/d49/d72/f88 [1447492,64853] 0 2024-04-25T01:36:14.740 INFO:tasks.workunit.client.1.smithi138.stdout:0/644: write d6/f3b [272854,12992] 0 2024-04-25T01:36:14.740 INFO:tasks.workunit.client.1.smithi138.stdout:3/903: read db/df/d17/d23/d88/f9a [3933791,40799] 0 2024-04-25T01:36:14.741 INFO:tasks.workunit.client.1.smithi138.stdout:4/634: dread d0/d1c/d1e/d9f/d4a/d74/f7a [0,4194304] 0 2024-04-25T01:36:14.742 INFO:tasks.workunit.client.1.smithi138.stdout:6/892: creat dc/d10/d67/f116 x:0 0 0 2024-04-25T01:36:14.742 INFO:tasks.workunit.client.1.smithi138.stdout:2/979: dread d0/d85/da5/db4/d15/d44/d4c/d56/da2/f26 [0,4194304] 0 2024-04-25T01:36:14.744 INFO:tasks.workunit.client.1.smithi138.stdout:0/645: fsync d6/f3b 0 2024-04-25T01:36:14.744 INFO:tasks.workunit.client.1.smithi138.stdout:2/980: chown d0/d21/ld7 196 1 2024-04-25T01:36:14.744 INFO:tasks.workunit.client.1.smithi138.stdout:6/893: rename dc/dc9 to dc/dc9/d117 22 2024-04-25T01:36:14.745 INFO:tasks.workunit.client.1.smithi138.stdout:5/786: rmdir d4/da/d6e/d22/d35/d4d/d5e/dfb 0 2024-04-25T01:36:14.746 INFO:tasks.workunit.client.1.smithi138.stdout:7/778: chown da/de/d1b/d49/d81/d51/d6e/f75 18143898 1 2024-04-25T01:36:14.749 INFO:tasks.workunit.client.1.smithi138.stdout:0/646: mknod d6/d1d/d82/ccf 0 2024-04-25T01:36:14.750 INFO:tasks.workunit.client.1.smithi138.stdout:2/981: truncate d0/de9/df0/f12b 21684 0 2024-04-25T01:36:14.752 INFO:tasks.workunit.client.1.smithi138.stdout:4/635: dread d0/d45/f35 [0,4194304] 0 2024-04-25T01:36:14.753 INFO:tasks.workunit.client.1.smithi138.stdout:5/787: stat d4/da/d6e/d22/d74/f76 0 2024-04-25T01:36:14.753 INFO:tasks.workunit.client.1.smithi138.stdout:8/933: dwrite d3/d16/d88/da/db/d15/d5d/f6b [0,4194304] 0 2024-04-25T01:36:14.755 INFO:tasks.workunit.client.1.smithi138.stdout:5/788: stat d4/da/d6e/d22/d35/dce/df9/d83/d89/d9d/d9e/d101 0 2024-04-25T01:36:14.757 INFO:tasks.workunit.client.1.smithi138.stdout:9/684: write d4/d8/d90/f62 [112423,102177] 0 2024-04-25T01:36:14.757 INFO:tasks.workunit.client.1.smithi138.stdout:5/789: read d4/f6c [2005638,30925] 0 2024-04-25T01:36:14.759 INFO:tasks.workunit.client.1.smithi138.stdout:2/982: creat d0/d21/d45/d8a/f13a x:0 0 0 2024-04-25T01:36:14.759 INFO:tasks.workunit.client.1.smithi138.stdout:5/790: chown d4/da/d6e/d22/d35 102 1 2024-04-25T01:36:14.761 INFO:tasks.workunit.client.1.smithi138.stdout:3/904: write db/df/d19/f75 [1492519,56942] 0 2024-04-25T01:36:14.761 INFO:tasks.workunit.client.1.smithi138.stdout:4/636: creat d0/d1c/d1e/fb7 x:0 0 0 2024-04-25T01:36:14.764 INFO:tasks.workunit.client.1.smithi138.stdout:3/905: write db/df/d19/d1a/d9e/fb9 [1418636,90403] 0 2024-04-25T01:36:14.765 INFO:tasks.workunit.client.1.smithi138.stdout:7/779: write da/de/d1b/f31 [8073933,91777] 0 2024-04-25T01:36:14.767 INFO:tasks.workunit.client.1.smithi138.stdout:7/780: write da/de/d13/d21/dbc/fb3 [2312331,88298] 0 2024-04-25T01:36:14.770 INFO:tasks.workunit.client.1.smithi138.stdout:9/685: creat d4/d63/d86/fe5 x:0 0 0 2024-04-25T01:36:14.771 INFO:tasks.workunit.client.1.smithi138.stdout:0/647: dread d6/f1f [0,4194304] 0 2024-04-25T01:36:14.775 INFO:tasks.workunit.client.1.smithi138.stdout:8/934: creat d3/d16/d88/da/de/f123 x:0 0 0 2024-04-25T01:36:14.777 INFO:tasks.workunit.client.1.smithi138.stdout:7/781: dwrite da/de/d13/d30/d59/fd6 [0,4194304] 0 2024-04-25T01:36:14.777 INFO:tasks.workunit.client.1.smithi138.stdout:0/648: truncate d6/d39/d43/f72 370184 0 2024-04-25T01:36:14.779 INFO:tasks.workunit.client.1.smithi138.stdout:0/649: chown d6/df/d41/f55 3731761 1 2024-04-25T01:36:14.780 INFO:tasks.workunit.client.1.smithi138.stdout:0/650: chown d6/df/f15 71 1 2024-04-25T01:36:14.780 INFO:tasks.workunit.client.1.smithi138.stdout:6/894: truncate dc/d10/f15 314678 0 2024-04-25T01:36:14.781 INFO:tasks.workunit.client.1.smithi138.stdout:9/686: dread d4/d1a/d55/d7f/d8f/f99 [0,4194304] 0 2024-04-25T01:36:14.782 INFO:tasks.workunit.client.1.smithi138.stdout:8/935: symlink d3/dde/l124 0 2024-04-25T01:36:14.782 INFO:tasks.workunit.client.1.smithi138.stdout:6/895: chown dc/d10/d67/d8e/faa 33816 1 2024-04-25T01:36:14.784 INFO:tasks.workunit.client.1.smithi138.stdout:7/782: mknod da/de/d13/d30/c101 0 2024-04-25T01:36:14.784 INFO:tasks.workunit.client.1.smithi138.stdout:8/936: truncate d3/d16/d1f/dd2/dd6/fdb 7091 0 2024-04-25T01:36:14.784 INFO:tasks.workunit.client.1.smithi138.stdout:5/791: write d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/fd0 [707790,116572] 0 2024-04-25T01:36:14.785 INFO:tasks.workunit.client.1.smithi138.stdout:0/651: rmdir d6/d39/d52 39 2024-04-25T01:36:14.786 INFO:tasks.workunit.client.1.smithi138.stdout:6/896: truncate dc/d10/d28/d70/f7f 1659976 0 2024-04-25T01:36:14.789 INFO:tasks.workunit.client.1.smithi138.stdout:6/897: stat dc/d10/d6d/c78 0 2024-04-25T01:36:14.789 INFO:tasks.workunit.client.1.smithi138.stdout:5/792: fsync d4/da/d6e/d22/d35/d4d/f66 0 2024-04-25T01:36:14.789 INFO:tasks.workunit.client.1.smithi138.stdout:3/906: write db/df/de2/d10a/d112/d40/d41/fa6 [945812,88242] 0 2024-04-25T01:36:14.791 INFO:tasks.workunit.client.1.smithi138.stdout:3/907: chown db/df/d19/dd6 0 1 2024-04-25T01:36:14.791 INFO:tasks.workunit.client.1.smithi138.stdout:6/898: write dc/d10/d67/f77 [435697,89526] 0 2024-04-25T01:36:14.793 INFO:tasks.workunit.client.1.smithi138.stdout:8/937: mknod d3/d16/d88/da/de/d57/d100/c125 0 2024-04-25T01:36:14.793 INFO:tasks.workunit.client.1.smithi138.stdout:2/983: truncate d0/f9a 3533434 0 2024-04-25T01:36:14.794 INFO:tasks.workunit.client.1.smithi138.stdout:8/938: fdatasync d3/d16/d88/da/db/d15/d6f/f10f 0 2024-04-25T01:36:14.798 INFO:tasks.workunit.client.1.smithi138.stdout:0/652: mknod d6/d39/d52/cd0 0 2024-04-25T01:36:14.801 INFO:tasks.workunit.client.1.smithi138.stdout:5/793: unlink d4/da/d6e/d22/d35/cdf 0 2024-04-25T01:36:14.803 INFO:tasks.workunit.client.1.smithi138.stdout:3/908: mkdir db/df/d19/d1a/d9e/d120 0 2024-04-25T01:36:14.805 INFO:tasks.workunit.client.1.smithi138.stdout:3/909: write db/df/d19/d1a/ffb [670249,32041] 0 2024-04-25T01:36:14.805 INFO:tasks.workunit.client.1.smithi138.stdout:6/899: symlink dc/d8d/l118 0 2024-04-25T01:36:14.806 INFO:tasks.workunit.client.1.smithi138.stdout:3/910: fdatasync db/d9b/db4/fbf 0 2024-04-25T01:36:14.809 INFO:tasks.workunit.client.1.smithi138.stdout:8/939: creat d3/d16/d1f/d87/f126 x:0 0 0 2024-04-25T01:36:14.809 INFO:tasks.workunit.client.1.smithi138.stdout:7/783: write da/de/d1b/f33 [616951,3924] 0 2024-04-25T01:36:14.812 INFO:tasks.workunit.client.1.smithi138.stdout:0/653: write d6/f1f [7966211,76324] 0 2024-04-25T01:36:14.812 INFO:tasks.workunit.client.1.smithi138.stdout:5/794: mknod d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/c105 0 2024-04-25T01:36:14.813 INFO:tasks.workunit.client.1.smithi138.stdout:7/784: chown da/de/d13/d30/l88 162610304 1 2024-04-25T01:36:14.813 INFO:tasks.workunit.client.1.smithi138.stdout:9/687: dwrite d4/d1a/d55/f5c [0,4194304] 0 2024-04-25T01:36:14.816 INFO:tasks.workunit.client.1.smithi138.stdout:7/785: chown da/de/d1b/d49/d81/d73/cfa 1951 1 2024-04-25T01:36:14.817 INFO:tasks.workunit.client.1.smithi138.stdout:9/688: write d4/d1a/d3c/dc2/f9c [1668577,84543] 0 2024-04-25T01:36:14.818 INFO:tasks.workunit.client.1.smithi138.stdout:6/900: mknod dc/d10/d28/d70/d83/d10a/d8c/db7/c119 0 2024-04-25T01:36:14.820 INFO:tasks.workunit.client.1.smithi138.stdout:2/984: mknod d0/d85/da5/db4/d15/d44/d4c/d56/d6d/c13b 0 2024-04-25T01:36:14.821 INFO:tasks.workunit.client.1.smithi138.stdout:9/689: write d4/d1a/d3c/fac [179053,100518] 0 2024-04-25T01:36:14.822 INFO:tasks.workunit.client.1.smithi138.stdout:6/901: truncate dc/d10/dc8/ffa 743102 0 2024-04-25T01:36:14.823 INFO:tasks.workunit.client.1.smithi138.stdout:8/940: creat d3/d16/d1f/d31/d82/dd5/f127 x:0 0 0 2024-04-25T01:36:14.824 INFO:tasks.workunit.client.1.smithi138.stdout:6/902: write dc/d2e/db9/fd8 [4628070,35245] 0 2024-04-25T01:36:14.827 INFO:tasks.workunit.client.1.smithi138.stdout:8/941: truncate d3/d16/d1f/d31/d82/dd5/ff5 1518073 0 2024-04-25T01:36:14.828 INFO:tasks.workunit.client.1.smithi138.stdout:5/795: mknod d4/da/d6e/d22/d35/dce/df9/d83/c106 0 2024-04-25T01:36:14.831 INFO:tasks.workunit.client.1.smithi138.stdout:7/786: mkdir da/de/d1b/d49/d81/d51/d6e/da2/d102 0 2024-04-25T01:36:14.834 INFO:tasks.workunit.client.1.smithi138.stdout:9/690: truncate d4/d1a/d3c/f9d 996120 0 2024-04-25T01:36:14.837 INFO:tasks.workunit.client.1.smithi138.stdout:5/796: dwrite d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/fd0 [0,4194304] 0 2024-04-25T01:36:14.845 INFO:tasks.workunit.client.1.smithi138.stdout:9/691: creat d4/d1a/d33/d9a/fe6 x:0 0 0 2024-04-25T01:36:14.847 INFO:tasks.workunit.client.1.smithi138.stdout:2/985: readlink d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/d83/le2 0 2024-04-25T01:36:14.847 INFO:tasks.workunit.client.1.smithi138.stdout:9/692: stat d4/d8/d2c/da3 0 2024-04-25T01:36:14.849 INFO:tasks.workunit.client.1.smithi138.stdout:0/654: dwrite d6/fe [0,4194304] 0 2024-04-25T01:36:14.852 INFO:tasks.workunit.client.1.smithi138.stdout:3/911: dwrite db/fc0 [0,4194304] 0 2024-04-25T01:36:14.855 INFO:tasks.workunit.client.1.smithi138.stdout:3/912: truncate db/df/d19/dd6/fe8 133044 0 2024-04-25T01:36:14.857 INFO:tasks.workunit.client.1.smithi138.stdout:3/913: chown db/df/c16 45886256 1 2024-04-25T01:36:14.858 INFO:tasks.workunit.client.1.smithi138.stdout:2/986: mknod d0/d21/c13c 0 2024-04-25T01:36:14.861 INFO:tasks.workunit.client.1.smithi138.stdout:9/693: rename d4/d8/f45 to d4/d63/dcf/fe7 0 2024-04-25T01:36:14.864 INFO:tasks.workunit.client.1.smithi138.stdout:5/797: rename d4/da/d6e/d22/d74/f73 to d4/da/d6e/d22/d74/f107 0 2024-04-25T01:36:14.866 INFO:tasks.workunit.client.1.smithi138.stdout:7/787: dwrite da/de/d1b/d4b/d5b/f43 [0,4194304] 0 2024-04-25T01:36:14.866 INFO:tasks.workunit.client.1.smithi138.stdout:3/914: unlink db/df/d19/f70 0 2024-04-25T01:36:14.870 INFO:tasks.workunit.client.1.smithi138.stdout:9/694: truncate d4/d1a/d68/fa7 373419 0 2024-04-25T01:36:14.871 INFO:tasks.workunit.client.1.smithi138.stdout:2/987: symlink d0/d85/da5/d35/l13d 0 2024-04-25T01:36:14.872 INFO:tasks.workunit.client.1.smithi138.stdout:8/942: dwrite d3/d16/f8e [0,4194304] 0 2024-04-25T01:36:14.875 INFO:tasks.workunit.client.1.smithi138.stdout:5/798: truncate d4/da/d6e/d22/d35/d4d/d5e/faf 1332346 0 2024-04-25T01:36:14.875 INFO:tasks.workunit.client.1.smithi138.stdout:9/695: chown d4/d57/fdf 88 1 2024-04-25T01:36:14.880 INFO:tasks.workunit.client.1.smithi138.stdout:9/696: chown d4/d1a/d3c/dc2/lcc 10 1 2024-04-25T01:36:14.880 INFO:tasks.workunit.client.1.smithi138.stdout:3/915: rmdir db/df/d19/d1a/d9e/d120 0 2024-04-25T01:36:14.882 INFO:tasks.workunit.client.1.smithi138.stdout:0/655: dread f1 [0,4194304] 0 2024-04-25T01:36:14.883 INFO:tasks.workunit.client.1.smithi138.stdout:3/916: write db/df/d17/d20/ffd [569601,64481] 0 2024-04-25T01:36:14.884 INFO:tasks.workunit.client.1.smithi138.stdout:7/788: mkdir da/de/d13/d21/dbc/d94/d103 0 2024-04-25T01:36:14.888 INFO:tasks.workunit.client.1.smithi138.stdout:5/799: creat d4/da/d6e/d22/d35/dce/df9/d83/f108 x:0 0 0 2024-04-25T01:36:14.888 INFO:tasks.workunit.client.1.smithi138.stdout:9/697: rmdir d4/d1a/d3c 39 2024-04-25T01:36:14.892 INFO:tasks.workunit.client.1.smithi138.stdout:2/988: mknod d0/d85/da5/db4/d15/d44/d4c/d56/dc8/c13e 0 2024-04-25T01:36:14.893 INFO:tasks.workunit.client.1.smithi138.stdout:5/800: stat d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/d100 0 2024-04-25T01:36:14.893 INFO:tasks.workunit.client.1.smithi138.stdout:3/917: dwrite db/fc7 [0,4194304] 0 2024-04-25T01:36:14.893 INFO:tasks.workunit.client.1.smithi138.stdout:8/943: mkdir d3/d16/d1f/d2a/d4f/dcb/dce/df0/d128 0 2024-04-25T01:36:14.894 INFO:tasks.workunit.client.1.smithi138.stdout:2/989: dread - d0/d85/da5/d35/f11d zero size 2024-04-25T01:36:14.894 INFO:tasks.workunit.client.1.smithi138.stdout:7/789: creat da/de/d13/d30/d59/d7b/d99/f104 x:0 0 0 2024-04-25T01:36:14.897 INFO:tasks.workunit.client.1.smithi138.stdout:9/698: fdatasync d4/f22 0 2024-04-25T01:36:14.897 INFO:tasks.workunit.client.1.smithi138.stdout:7/790: read da/de/d1b/d4b/d77/f91 [2554689,77725] 0 2024-04-25T01:36:14.900 INFO:tasks.workunit.client.1.smithi138.stdout:2/990: mkdir d0/de9/d13f 0 2024-04-25T01:36:14.903 INFO:tasks.workunit.client.1.smithi138.stdout:8/944: rename d3/dde/f10c to d3/d16/d1f/d2a/d4f/dcb/dce/df0/f129 0 2024-04-25T01:36:14.906 INFO:tasks.workunit.client.1.smithi138.stdout:9/699: unlink d4/d8/d2c/da3/fbf 0 2024-04-25T01:36:14.909 INFO:tasks.workunit.client.1.smithi138.stdout:5/801: mkdir d4/da/d6e/d22/d109 0 2024-04-25T01:36:14.911 INFO:tasks.workunit.client.1.smithi138.stdout:0/656: getdents d6/d39/d43 0 2024-04-25T01:36:14.912 INFO:tasks.workunit.client.1.smithi138.stdout:7/791: dread da/de/d13/f23 [0,4194304] 0 2024-04-25T01:36:14.913 INFO:tasks.workunit.client.1.smithi138.stdout:8/945: mkdir d3/d44/da4/d12a 0 2024-04-25T01:36:14.914 INFO:tasks.workunit.client.1.smithi138.stdout:9/700: mkdir d4/d8/d2c/d82/de8 0 2024-04-25T01:36:14.915 INFO:tasks.workunit.client.1.smithi138.stdout:9/701: read d4/d1a/d55/f5c [2863619,162] 0 2024-04-25T01:36:14.919 INFO:tasks.workunit.client.1.smithi138.stdout:8/946: creat d3/d44/f12b x:0 0 0 2024-04-25T01:36:14.919 INFO:tasks.workunit.client.1.smithi138.stdout:7/792: dwrite da/de/d13/d30/f65 [0,4194304] 0 2024-04-25T01:36:14.919 INFO:tasks.workunit.client.1.smithi138.stdout:9/702: truncate d4/d8/d90/d8e/fe1 149097 0 2024-04-25T01:36:14.920 INFO:tasks.workunit.client.1.smithi138.stdout:5/802: symlink d4/l10a 0 2024-04-25T01:36:14.922 INFO:tasks.workunit.client.1.smithi138.stdout:0/657: symlink d6/d1d/d82/dc6/ld1 0 2024-04-25T01:36:14.923 INFO:tasks.workunit.client.1.smithi138.stdout:2/991: creat d0/d85/da5/db4/d15/d44/d4c/f140 x:0 0 0 2024-04-25T01:36:14.925 INFO:tasks.workunit.client.1.smithi138.stdout:9/703: write d4/d1a/d55/f5c [148419,110294] 0 2024-04-25T01:36:14.928 INFO:tasks.workunit.client.1.smithi138.stdout:0/658: mknod d6/d2f/d31/cd2 0 2024-04-25T01:36:14.933 INFO:tasks.workunit.client.1.smithi138.stdout:9/704: symlink d4/d8/d90/d8e/le9 0 2024-04-25T01:36:14.934 INFO:tasks.workunit.client.1.smithi138.stdout:5/803: mknod d4/da/d6e/d22/d45/df5/c10b 0 2024-04-25T01:36:14.934 INFO:tasks.workunit.client.1.smithi138.stdout:9/705: truncate d4/d57/fce 646570 0 2024-04-25T01:36:14.936 INFO:tasks.workunit.client.1.smithi138.stdout:0/659: unlink d6/d39/d43/f72 0 2024-04-25T01:36:14.937 INFO:tasks.workunit.client.1.smithi138.stdout:2/992: creat d0/d21/d45/de1/f141 x:0 0 0 2024-04-25T01:36:14.938 INFO:tasks.workunit.client.1.smithi138.stdout:2/993: readlink d0/d85/da5/db4/laa 0 2024-04-25T01:36:14.940 INFO:tasks.workunit.client.1.smithi138.stdout:7/793: creat da/de/d1b/d4b/f105 x:0 0 0 2024-04-25T01:36:14.941 INFO:tasks.workunit.client.1.smithi138.stdout:9/706: creat d4/d1a/d33/d9a/fea x:0 0 0 2024-04-25T01:36:14.942 INFO:tasks.workunit.client.1.smithi138.stdout:5/804: symlink d4/da/d6e/d22/d45/df5/l10c 0 2024-04-25T01:36:14.944 INFO:tasks.workunit.client.1.smithi138.stdout:2/994: mkdir d0/d85/da5/db4/d15/d44/d4c/d56/d106/d142 0 2024-04-25T01:36:14.944 INFO:tasks.workunit.client.1.smithi138.stdout:5/805: truncate d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/fd0 4486329 0 2024-04-25T01:36:14.946 INFO:tasks.workunit.client.1.smithi138.stdout:5/806: readlink d4/da/d6e/d22/d35/l38 0 2024-04-25T01:36:14.951 INFO:tasks.workunit.client.1.smithi138.stdout:2/995: dwrite d0/d85/da5/db4/d15/d44/d4c/d56/d6d/d6e/fc6 [0,4194304] 0 2024-04-25T01:36:14.951 INFO:tasks.workunit.client.1.smithi138.stdout:0/660: creat d6/d2f/fd3 x:0 0 0 2024-04-25T01:36:14.952 INFO:tasks.workunit.client.1.smithi138.stdout:8/947: truncate d3/d16/d88/da/db/d15/d20/f26 96195 0 2024-04-25T01:36:14.953 INFO:tasks.workunit.client.1.smithi138.stdout:0/661: dread - d6/d2f/d75/f9f zero size 2024-04-25T01:36:14.953 INFO:tasks.workunit.client.1.smithi138.stdout:7/794: creat da/de/d13/d95/dd7/f106 x:0 0 0 2024-04-25T01:36:14.954 INFO:tasks.workunit.client.1.smithi138.stdout:5/807: creat d4/da/d6e/d22/d35/d4d/d5e/d60/dbd/dc8/dbb/f10d x:0 0 0 2024-04-25T01:36:14.956 INFO:tasks.workunit.client.1.smithi138.stdout:9/707: getdents d4/d1a/d68/da5 0 2024-04-25T01:36:14.960 INFO:tasks.workunit.client.1.smithi138.stdout:7/795: rename da/de/d13/l7d to da/de/d13/d30/l107 0 2024-04-25T01:36:14.962 INFO:tasks.workunit.client.1.smithi138.stdout:7/796: write da/de/d13/d30/f65 [4781719,38509] 0 2024-04-25T01:36:14.962 INFO:tasks.workunit.client.1.smithi138.stdout:9/708: mknod d4/d1a/d33/ceb 0 2024-04-25T01:36:14.965 INFO:tasks.workunit.client.1.smithi138.stdout:8/948: getdents d3/d16/d1f/d2a 0 2024-04-25T01:36:14.968 INFO:tasks.workunit.client.1.smithi138.stdout:2/996: read d0/d21/d10b/f118 [609322,6524] 0 2024-04-25T01:36:14.968 INFO:tasks.workunit.client.1.smithi138.stdout:7/797: creat da/de/d13/d30/f108 x:0 0 0 2024-04-25T01:36:14.971 INFO:tasks.workunit.client.1.smithi138.stdout:9/709: dwrite d4/d1a/d55/f5c [0,4194304] 0 2024-04-25T01:36:14.979 INFO:tasks.workunit.client.1.smithi138.stdout:2/997: dwrite d0/d21/f59 [0,4194304] 0 2024-04-25T01:36:14.980 INFO:tasks.workunit.client.1.smithi138.stdout:9/710: symlink d4/d8/de3/lec 0 2024-04-25T01:36:14.985 INFO:tasks.workunit.client.1.smithi138.stdout:5/808: dread d4/da/d6e/d22/d35/d4d/d5e/f7a [0,4194304] 0 2024-04-25T01:36:14.988 INFO:tasks.workunit.client.1.smithi138.stdout:7/798: write da/de/d13/d30/d59/d7b/d99/fb0 [2009070,5048] 0 2024-04-25T01:36:14.991 INFO:tasks.workunit.client.1.smithi138.stdout:5/809: mkdir d4/da/d6e/d3c/d10e 0 2024-04-25T01:36:14.993 INFO:tasks.workunit.client.1.smithi138.stdout:9/711: dwrite d4/d1a/d33/f37 [0,4194304] 0 2024-04-25T01:36:14.994 INFO:tasks.workunit.client.1.smithi138.stdout:8/949: dwrite d3/d16/d88/da/db/d15/d56/f86 [0,4194304] 0 2024-04-25T01:36:14.995 INFO:tasks.workunit.client.1.smithi138.stdout:7/799: write da/de/d13/d30/d59/d7b/d99/fd5 [612804,78549] 0 2024-04-25T01:36:15.001 INFO:tasks.workunit.client.1.smithi138.stdout:5/810: mknod d4/da/d6e/d22/d74/d8e/c10f 0 2024-04-25T01:36:15.001 INFO:tasks.workunit.client.1.smithi138.stdout:8/950: rmdir d3/d16/d1f/d2a 39 2024-04-25T01:36:15.002 INFO:tasks.workunit.client.1.smithi138.stdout:2/998: dwrite d0/d85/da5/db4/d15/d44/d4c/fd2 [0,4194304] 0 2024-04-25T01:36:15.004 INFO:tasks.workunit.client.1.smithi138.stdout:5/811: read - d4/da/d6e/d22/d35/d4d/d5e/d60/db5/fdd zero size 2024-04-25T01:36:15.006 INFO:tasks.workunit.client.1.smithi138.stdout:9/712: unlink d4/d1a/d3c/dc2/fe0 0 2024-04-25T01:36:15.006 INFO:tasks.workunit.client.1.smithi138.stdout:8/951: write d3/d16/d1f/d31/d82/dd5/fe3 [2294125,112043] 0 2024-04-25T01:36:15.006 INFO:tasks.workunit.client.1.smithi138.stdout:7/800: mkdir da/dd2/dcd/d109 0 2024-04-25T01:36:15.010 INFO:tasks.workunit.client.1.smithi138.stdout:2/999: unlink d0/f9 0 2024-04-25T01:36:15.010 INFO:tasks.workunit.client.1.smithi138.stdout:7/801: read da/de/d1b/d4b/d5b/f43 [3900982,77630] 0 2024-04-25T01:36:15.012 INFO:tasks.workunit.client.1.smithi138.stdout:5/812: rename d4/da/d6e/d22/c4e to d4/da/d6e/d22/d35/d4d/d5e/d60/dc4/c110 0 2024-04-25T01:36:15.014 INFO:tasks.workunit.client.1.smithi138.stdout:7/802: creat da/dd2/f10a x:0 0 0 2024-04-25T01:36:15.016 INFO:tasks.workunit.client.1.smithi138.stdout:9/713: unlink d4/d1a/d55/d7f/f83 0 2024-04-25T01:36:15.016 INFO:tasks.workunit.client.1.smithi138.stdout:7/803: write da/de/d13/d30/d59/d7b/d99/fb0 [2828986,77819] 0 2024-04-25T01:36:15.018 INFO:tasks.workunit.client.1.smithi138.stdout:8/952: rename d3/d16/d88/da/db/d15/d20/c7f to d3/d16/d88/da/db/dc9/c12c 0 2024-04-25T01:36:15.020 INFO:tasks.workunit.client.1.smithi138.stdout:5/813: creat d4/da/d6e/d22/d35/d4d/d5e/d60/dc4/f111 x:0 0 0 2024-04-25T01:36:15.021 INFO:tasks.workunit.client.1.smithi138.stdout:5/814: fsync d4/da/d6e/d22/f24 0 2024-04-25T01:36:15.024 INFO:tasks.workunit.client.1.smithi138.stdout:9/714: dwrite d4/d1a/d3c/dc2/f9c [0,4194304] 0 2024-04-25T01:36:15.027 INFO:tasks.workunit.client.1.smithi138.stdout:5/815: creat d4/da/d6e/d22/d74/d8e/f112 x:0 0 0 2024-04-25T01:36:15.031 INFO:tasks.workunit.client.1.smithi138.stdout:9/715: symlink d4/d8/d90/d52/led 0 2024-04-25T01:36:15.038 INFO:tasks.workunit.client.1.smithi138.stdout:5/816: dread d4/da/d6e/d22/f98 [0,4194304] 0 2024-04-25T01:36:15.039 INFO:tasks.workunit.client.1.smithi138.stdout:9/716: rename d4/d8/l2e to d4/d8/d2c/lee 0 2024-04-25T01:36:15.040 INFO:tasks.workunit.client.1.smithi138.stdout:5/817: chown d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/d100 17015648 1 2024-04-25T01:36:15.041 INFO:tasks.workunit.client.1.smithi138.stdout:9/717: creat d4/d1a/d68/da5/fef x:0 0 0 2024-04-25T01:36:15.042 INFO:tasks.workunit.client.1.smithi138.stdout:5/818: chown d4/da/d6e/d22/d35/d4d/d5e/d8f/fb2 0 1 2024-04-25T01:36:15.045 INFO:tasks.workunit.client.1.smithi138.stdout:9/718: rename d4/d1a/f92 to d4/d8/d2c/d82/de8/ff0 0 2024-04-25T01:36:15.048 INFO:tasks.workunit.client.1.smithi138.stdout:5/819: mknod d4/da/d6e/d22/d35/dce/df9/dd4/c113 0 2024-04-25T01:36:15.049 INFO:tasks.workunit.client.1.smithi138.stdout:7/804: dwrite da/de/d13/d30/d59/d7b/d99/fa6 [0,4194304] 0 2024-04-25T01:36:15.052 INFO:tasks.workunit.client.1.smithi138.stdout:9/719: rmdir d4/d8 39 2024-04-25T01:36:15.053 INFO:tasks.workunit.client.1.smithi138.stdout:8/953: dwrite d3/f38 [0,4194304] 0 2024-04-25T01:36:15.058 INFO:tasks.workunit.client.1.smithi138.stdout:5/820: mknod d4/da/d6e/d22/d35/d4d/d90/dba/c114 0 2024-04-25T01:36:15.063 INFO:tasks.workunit.client.1.smithi138.stdout:5/821: mkdir d4/da/d6e/d22/d35/dce/d115 0 2024-04-25T01:36:15.067 INFO:tasks.workunit.client.1.smithi138.stdout:5/822: rmdir d4/da/d6e/d22/d35/dce/df9/dd4 39 2024-04-25T01:36:15.069 INFO:tasks.workunit.client.1.smithi138.stdout:8/954: write d3/d16/d88/da/db/d15/f30 [1406330,6881] 0 2024-04-25T01:36:15.073 INFO:tasks.workunit.client.1.smithi138.stdout:7/805: dwrite da/de/d13/d30/f42 [0,4194304] 0 2024-04-25T01:36:15.074 INFO:tasks.workunit.client.1.smithi138.stdout:9/720: write d4/d1a/d3c/f9d [1812894,108567] 0 2024-04-25T01:36:15.075 INFO:tasks.workunit.client.1.smithi138.stdout:8/955: write d3/d44/f12b [413517,125159] 0 2024-04-25T01:36:15.076 INFO:tasks.workunit.client.1.smithi138.stdout:5/823: creat d4/da/d6e/d22/d35/dce/df9/dd4/f116 x:0 0 0 2024-04-25T01:36:15.079 INFO:tasks.workunit.client.1.smithi138.stdout:7/806: creat da/de/d1b/d49/d81/d51/d6e/da2/f10b x:0 0 0 2024-04-25T01:36:15.080 INFO:tasks.workunit.client.1.smithi138.stdout:5/824: dread d4/da/d6e/d22/f98 [0,4194304] 0 2024-04-25T01:36:15.080 INFO:tasks.workunit.client.1.smithi138.stdout:8/956: mkdir d3/d16/d88/da/db/d15/d56/d8f/d12d 0 2024-04-25T01:36:15.081 INFO:tasks.workunit.client.1.smithi138.stdout:9/721: rmdir d4/d8/de3 39 2024-04-25T01:36:15.083 INFO:tasks.workunit.client.1.smithi138.stdout:8/957: fsync d3/d16/d88/da/db/d15/f30 0 2024-04-25T01:36:15.086 INFO:tasks.workunit.client.1.smithi138.stdout:9/722: creat d4/d1a/d68/dbe/ff1 x:0 0 0 2024-04-25T01:36:15.101 INFO:tasks.workunit.client.1.smithi138.stdout:7/807: dwrite da/de/d13/d30/d59/d7b/d99/fb1 [0,4194304] 0 2024-04-25T01:36:15.102 INFO:tasks.workunit.client.1.smithi138.stdout:8/958: dwrite d3/d16/d88/da/de/f121 [0,4194304] 0 2024-04-25T01:36:15.104 INFO:tasks.workunit.client.1.smithi138.stdout:5/825: dwrite d4/f6c [4194304,4194304] 0 2024-04-25T01:36:15.108 INFO:tasks.workunit.client.1.smithi138.stdout:5/826: read d4/da/f20 [88426,44683] 0 2024-04-25T01:36:15.110 INFO:tasks.workunit.client.1.smithi138.stdout:5/827: write d4/da/d6e/d22/d35/dce/df9/f99 [917791,33271] 0 2024-04-25T01:36:15.112 INFO:tasks.workunit.client.1.smithi138.stdout:5/828: chown d4/da/d6e/d22/d35/c36 37353563 1 2024-04-25T01:36:15.214 INFO:tasks.workunit.client.1.smithi138.stdout:7/808: rmdir da/de/d13/d21 39 2024-04-25T01:36:15.220 INFO:tasks.workunit.client.1.smithi138.stdout:9/723: symlink d4/d8/d90/d8e/dd9/lf2 0 2024-04-25T01:36:15.222 INFO:tasks.workunit.client.1.smithi138.stdout:5/829: read d4/f9 [3911924,110417] 0 2024-04-25T01:36:15.226 INFO:tasks.workunit.client.1.smithi138.stdout:8/959: rename d3/f3a to d3/d44/da4/d12a/f12e 0 2024-04-25T01:36:15.230 INFO:tasks.workunit.client.1.smithi138.stdout:5/830: dwrite d4/f6c [4194304,4194304] 0 2024-04-25T01:36:15.232 INFO:tasks.workunit.client.1.smithi138.stdout:5/831: chown d4/da/d6e/d22/d74/d8e/dd7/lb7 5 1 2024-04-25T01:36:15.234 INFO:tasks.workunit.client.1.smithi138.stdout:8/960: readlink d3/d44/d48/dbb/l10e 0 2024-04-25T01:36:15.235 INFO:tasks.workunit.client.1.smithi138.stdout:9/724: rename d4/d1a/d68/da0/fb7 to d4/d1a/d3c/dc2/db2/ff3 0 2024-04-25T01:36:15.236 INFO:tasks.workunit.client.1.smithi138.stdout:7/809: creat da/de/d13/f10c x:0 0 0 2024-04-25T01:36:15.241 INFO:tasks.workunit.client.1.smithi138.stdout:8/961: write d3/d16/d1f/d2a/d4f/dac/fd9 [1010846,957] 0 2024-04-25T01:36:15.241 INFO:tasks.workunit.client.1.smithi138.stdout:7/810: truncate da/de/d1b/d49/d81/d51/d6e/f87 1320007 0 2024-04-25T01:36:15.245 INFO:tasks.workunit.client.1.smithi138.stdout:5/832: dwrite d4/da/d6e/d22/d74/f85 [4194304,4194304] 0 2024-04-25T01:36:15.247 INFO:tasks.workunit.client.1.smithi138.stdout:9/725: rename d4/d63/d86/fad to d4/d8/d2c/d82/ff4 0 2024-04-25T01:36:15.250 INFO:tasks.workunit.client.1.smithi138.stdout:5/833: dread d4/da/d6e/d22/d35/dce/df9/d83/d8a/f96 [0,4194304] 0 2024-04-25T01:36:15.251 INFO:tasks.workunit.client.1.smithi138.stdout:5/834: chown d4/da 0 1 2024-04-25T01:36:15.251 INFO:tasks.workunit.client.1.smithi138.stdout:8/962: symlink d3/d16/d88/da/db/d15/d56/d8f/def/l12f 0 2024-04-25T01:36:15.252 INFO:tasks.workunit.client.1.smithi138.stdout:8/963: stat d3/d16/d1f/dc5/cf1 0 2024-04-25T01:36:15.254 INFO:tasks.workunit.client.1.smithi138.stdout:9/726: dread d4/d1a/d3c/dc2/f9c [0,4194304] 0 2024-04-25T01:36:15.255 INFO:tasks.workunit.client.1.smithi138.stdout:7/811: rmdir da/de/d13/d21 39 2024-04-25T01:36:15.259 INFO:tasks.workunit.client.1.smithi138.stdout:5/835: dread d4/da/d6e/d22/d35/d4d/d5e/faf [0,4194304] 0 2024-04-25T01:36:15.263 INFO:tasks.workunit.client.1.smithi138.stdout:1/757: sync 2024-04-25T01:36:15.263 INFO:tasks.workunit.client.1.smithi138.stdout:5/836: fdatasync d4/da/d6e/d22/d35/dce/df9/d83/d8a/f96 0 2024-04-25T01:36:15.265 INFO:tasks.workunit.client.1.smithi138.stdout:7/812: mknod da/de/d1b/d49/d81/dc7/c10d 0 2024-04-25T01:36:15.267 INFO:tasks.workunit.client.1.smithi138.stdout:5/837: unlink d4/da/d6e/d22/d74/f107 0 2024-04-25T01:36:15.268 INFO:tasks.workunit.client.1.smithi138.stdout:1/758: rename d3/d45/d72/f95 to d3/d45/d72/d57/db0/dcd/de3/ffc 0 2024-04-25T01:36:15.271 INFO:tasks.workunit.client.1.smithi138.stdout:5/838: creat d4/da/d6e/d22/d35/d4d/d5e/d8f/f117 x:0 0 0 2024-04-25T01:36:15.273 INFO:tasks.workunit.client.1.smithi138.stdout:7/813: rename da/de/d1b/d49/d81/d51/d6e/da2/f10b to da/de/d13/d21/dbc/d94/d103/f10e 0 2024-04-25T01:36:15.275 INFO:tasks.workunit.client.1.smithi138.stdout:1/759: mknod d3/d27/d16/dac/cfd 0 2024-04-25T01:36:15.276 INFO:tasks.workunit.client.1.smithi138.stdout:5/839: creat d4/da/d6e/d22/d35/d4d/dca/f118 x:0 0 0 2024-04-25T01:36:15.279 INFO:tasks.workunit.client.1.smithi138.stdout:7/814: rename f9 to da/de/d13/d30/d59/d7b/f10f 0 2024-04-25T01:36:15.280 INFO:tasks.workunit.client.1.smithi138.stdout:1/760: creat d3/ddb/ffe x:0 0 0 2024-04-25T01:36:15.282 INFO:tasks.workunit.client.1.smithi138.stdout:5/840: mknod d4/da/d6e/d22/d35/dce/df9/d83/d89/d9d/d9e/c119 0 2024-04-25T01:36:15.282 INFO:tasks.workunit.client.1.smithi138.stdout:1/761: read d3/db/f5f [7304395,55085] 0 2024-04-25T01:36:15.290 INFO:tasks.workunit.client.1.smithi138.stdout:9/727: write d4/d8/f1f [2158188,86796] 0 2024-04-25T01:36:15.290 INFO:tasks.workunit.client.1.smithi138.stdout:5/841: mknod d4/da/d6e/d22/d74/d8e/dd7/da6/c11a 0 2024-04-25T01:36:15.290 INFO:tasks.workunit.client.1.smithi138.stdout:8/964: dwrite d3/d16/d88/f53 [0,4194304] 0 2024-04-25T01:36:15.294 INFO:tasks.workunit.client.1.smithi138.stdout:5/842: symlink d4/da/d6e/d22/d45/df4/l11b 0 2024-04-25T01:36:15.295 INFO:tasks.workunit.client.1.smithi138.stdout:8/965: mknod d3/d16/d1f/d31/c130 0 2024-04-25T01:36:15.297 INFO:tasks.workunit.client.1.smithi138.stdout:8/966: dread - d3/d16/d1f/d2a/d4f/dcb/dce/f111 zero size 2024-04-25T01:36:15.299 INFO:tasks.workunit.client.1.smithi138.stdout:7/815: dread da/de/d1b/d4b/f55 [0,4194304] 0 2024-04-25T01:36:15.303 INFO:tasks.workunit.client.1.smithi138.stdout:5/843: getdents d4/da/d6e/d22/d35/dce/df9/dd4 0 2024-04-25T01:36:15.303 INFO:tasks.workunit.client.1.smithi138.stdout:1/762: dread d3/fc9 [0,4194304] 0 2024-04-25T01:36:15.304 INFO:tasks.workunit.client.1.smithi138.stdout:8/967: dread d3/d16/d88/f53 [0,4194304] 0 2024-04-25T01:36:15.305 INFO:tasks.workunit.client.1.smithi138.stdout:7/816: unlink da/de/d1b/d49/d81/d51/d6e/da2/cec 0 2024-04-25T01:36:15.306 INFO:tasks.workunit.client.1.smithi138.stdout:8/968: chown d3/d16/d1f/d2a/cc6 841 1 2024-04-25T01:36:15.306 INFO:tasks.workunit.client.1.smithi138.stdout:5/844: fsync d4/da/d6e/d22/d35/d4d/d5e/d60/dbd/dc8/dbb/f10d 0 2024-04-25T01:36:15.306 INFO:tasks.workunit.client.1.smithi138.stdout:1/763: symlink d3/d4/d53/lff 0 2024-04-25T01:36:15.307 INFO:tasks.workunit.client.1.smithi138.stdout:8/969: write d3/f3c [4300170,42084] 0 2024-04-25T01:36:15.311 INFO:tasks.workunit.client.1.smithi138.stdout:5/845: read d4/da/d27/f2d [838885,115266] 0 2024-04-25T01:36:15.313 INFO:tasks.workunit.client.1.smithi138.stdout:9/728: dread d4/d1a/d3c/fd3 [0,4194304] 0 2024-04-25T01:36:15.314 INFO:tasks.workunit.client.1.smithi138.stdout:7/817: rename da/de/d1b/d49/d81/dc7/fce to da/de/d13/d95/dd7/f110 0 2024-04-25T01:36:15.315 INFO:tasks.workunit.client.1.smithi138.stdout:8/970: creat d3/d16/d88/da/db/d15/d5d/f131 x:0 0 0 2024-04-25T01:36:15.316 INFO:tasks.workunit.client.1.smithi138.stdout:1/764: dwrite d3/d45/d72/d57/d8e/f69 [0,4194304] 0 2024-04-25T01:36:15.322 INFO:tasks.workunit.client.1.smithi138.stdout:7/818: creat da/de/d13/d21/dbc/dca/f111 x:0 0 0 2024-04-25T01:36:15.324 INFO:tasks.workunit.client.1.smithi138.stdout:7/819: chown da/de/d1b/d34/f47 58384 1 2024-04-25T01:36:15.326 INFO:tasks.workunit.client.1.smithi138.stdout:1/765: dwrite d3/d27/d16/f18 [0,4194304] 0 2024-04-25T01:36:15.327 INFO:tasks.workunit.client.1.smithi138.stdout:1/766: stat d3/d27/d3e/d90/ca4 0 2024-04-25T01:36:15.331 INFO:tasks.workunit.client.1.smithi138.stdout:1/767: rmdir d3/d4/da7 39 2024-04-25T01:36:15.333 INFO:tasks.workunit.client.1.smithi138.stdout:7/820: unlink da/c16 0 2024-04-25T01:36:15.334 INFO:tasks.workunit.client.1.smithi138.stdout:9/729: getdents d4/d1a/d3c/dc2/db2 0 2024-04-25T01:36:15.336 INFO:tasks.workunit.client.1.smithi138.stdout:7/821: mknod da/de/d1b/d49/c112 0 2024-04-25T01:36:15.340 INFO:tasks.workunit.client.1.smithi138.stdout:8/971: dread d3/d16/d88/da/de/f13 [0,4194304] 0 2024-04-25T01:36:15.342 INFO:tasks.workunit.client.1.smithi138.stdout:7/822: fdatasync da/de/d13/d30/d59/fd6 0 2024-04-25T01:36:15.344 INFO:tasks.workunit.client.1.smithi138.stdout:5/846: write d4/da/d6e/d22/d35/dce/df9/f70 [8825995,1143] 0 2024-04-25T01:36:15.347 INFO:tasks.workunit.client.1.smithi138.stdout:3/918: sync 2024-04-25T01:36:15.347 INFO:tasks.workunit.client.1.smithi138.stdout:4/637: sync 2024-04-25T01:36:15.347 INFO:tasks.workunit.client.1.smithi138.stdout:6/903: sync 2024-04-25T01:36:15.348 INFO:tasks.workunit.client.1.smithi138.stdout:4/638: readlink d0/d1c/d1e/l24 0 2024-04-25T01:36:15.348 INFO:tasks.workunit.client.1.smithi138.stdout:1/768: link d3/d27/d3e/d90/l66 d3/d27/d16/dac/l100 0 2024-04-25T01:36:15.350 INFO:tasks.workunit.client.1.smithi138.stdout:7/823: fdatasync da/f6a 0 2024-04-25T01:36:15.354 INFO:tasks.workunit.client.1.smithi138.stdout:6/904: rmdir dc/d2e/d37 39 2024-04-25T01:36:15.356 INFO:tasks.workunit.client.1.smithi138.stdout:5/847: dwrite d4/da/d6e/d22/d35/dce/df9/d83/d8a/fc0 [0,4194304] 0 2024-04-25T01:36:15.357 INFO:tasks.workunit.client.1.smithi138.stdout:9/730: dwrite d4/d8/d2c/da3/fb0 [0,4194304] 0 2024-04-25T01:36:15.366 INFO:tasks.workunit.client.1.smithi138.stdout:8/972: truncate d3/d16/d88/da/db/d15/f2f 1314632 0 2024-04-25T01:36:15.367 INFO:tasks.workunit.client.1.smithi138.stdout:7/824: creat da/de/d13/d95/dd7/f113 x:0 0 0 2024-04-25T01:36:15.367 INFO:tasks.workunit.client.1.smithi138.stdout:4/639: symlink d0/d1c/d52/da3/db5/lb8 0 2024-04-25T01:36:15.369 INFO:tasks.workunit.client.1.smithi138.stdout:3/919: write db/df/f4f [3015847,16938] 0 2024-04-25T01:36:15.369 INFO:tasks.workunit.client.1.smithi138.stdout:5/848: truncate d4/da/d6e/d22/d74/f9c 720830 0 2024-04-25T01:36:15.370 INFO:tasks.workunit.client.1.smithi138.stdout:7/825: chown da/dd2/dcd/d109 1 1 2024-04-25T01:36:15.371 INFO:tasks.workunit.client.1.smithi138.stdout:6/905: dwrite fb [4194304,4194304] 0 2024-04-25T01:36:15.373 INFO:tasks.workunit.client.1.smithi138.stdout:1/769: symlink d3/db/l101 0 2024-04-25T01:36:15.374 INFO:tasks.workunit.client.1.smithi138.stdout:7/826: dread - da/de/d1b/fef zero size 2024-04-25T01:36:15.376 INFO:tasks.workunit.client.1.smithi138.stdout:9/731: unlink d4/d57/c95 0 2024-04-25T01:36:15.376 INFO:tasks.workunit.client.1.smithi138.stdout:1/770: stat d3/d4/da7 0 2024-04-25T01:36:15.377 INFO:tasks.workunit.client.1.smithi138.stdout:5/849: write d4/da/d6e/d22/d35/dce/df9/d83/f108 [376170,77826] 0 2024-04-25T01:36:15.378 INFO:tasks.workunit.client.1.smithi138.stdout:8/973: creat d3/d16/d1f/d2a/d4f/dcb/dce/df0/d128/f132 x:0 0 0 2024-04-25T01:36:15.384 INFO:tasks.workunit.client.1.smithi138.stdout:8/974: creat d3/d16/d88/da/db/d15/d56/d94/f133 x:0 0 0 2024-04-25T01:36:15.390 INFO:tasks.workunit.client.1.smithi138.stdout:8/975: write d3/d16/d1f/d31/d82/f90 [1849773,61425] 0 2024-04-25T01:36:15.390 INFO:tasks.workunit.client.1.smithi138.stdout:6/906: link dc/d10/d28/f29 dc/d10/d28/f11a 0 2024-04-25T01:36:15.394 INFO:tasks.workunit.client.1.smithi138.stdout:3/920: write db/df/de2/d10a/d112/d7a/f92 [537058,78958] 0 2024-04-25T01:36:15.395 INFO:tasks.workunit.client.1.smithi138.stdout:1/771: getdents d3/dcc 0 2024-04-25T01:36:15.395 INFO:tasks.workunit.client.1.smithi138.stdout:0/662: sync 2024-04-25T01:36:15.396 INFO:tasks.workunit.client.1.smithi138.stdout:5/850: write d4/da/d6e/d22/d45/f67 [848938,50806] 0 2024-04-25T01:36:15.396 INFO:tasks.workunit.client.1.smithi138.stdout:3/921: truncate db/df/d17/d23/f30 4719821 0 2024-04-25T01:36:15.399 INFO:tasks.workunit.client.1.smithi138.stdout:6/907: creat dc/d10/d28/d70/d83/d106/f11b x:0 0 0 2024-04-25T01:36:15.401 INFO:tasks.workunit.client.1.smithi138.stdout:9/732: link d4/d8/d90/l4f d4/d49/d72/lf5 0 2024-04-25T01:36:15.401 INFO:tasks.workunit.client.1.smithi138.stdout:5/851: dread d4/da/d6e/d22/f98 [0,4194304] 0 2024-04-25T01:36:15.402 INFO:tasks.workunit.client.1.smithi138.stdout:0/663: unlink d6/d19/l25 0 2024-04-25T01:36:15.403 INFO:tasks.workunit.client.1.smithi138.stdout:5/852: chown d4/da/d6e/d5c 1406373594 1 2024-04-25T01:36:15.405 INFO:tasks.workunit.client.1.smithi138.stdout:3/922: write db/df/d17/d23/d88/f9a [4404634,23898] 0 2024-04-25T01:36:15.405 INFO:tasks.workunit.client.1.smithi138.stdout:6/908: mkdir dc/d10/d28/d11c 0 2024-04-25T01:36:15.406 INFO:tasks.workunit.client.1.smithi138.stdout:7/827: dread da/dd2/fe0 [0,4194304] 0 2024-04-25T01:36:15.406 INFO:tasks.workunit.client.1.smithi138.stdout:8/976: getdents d3/d16/d1f/d2a 0 2024-04-25T01:36:15.407 INFO:tasks.workunit.client.1.smithi138.stdout:5/853: truncate d4/da/d6e/d22/f98 1994254 0 2024-04-25T01:36:15.407 INFO:tasks.workunit.client.1.smithi138.stdout:6/909: write dc/d2e/f3e [372672,27128] 0 2024-04-25T01:36:15.408 INFO:tasks.workunit.client.1.smithi138.stdout:1/772: rename d3/d45/d72/d3d/l96 to d3/d45/d72/d57/db0/dcd/l102 0 2024-04-25T01:36:15.409 INFO:tasks.workunit.client.1.smithi138.stdout:0/664: rmdir d6/df/d41/d49 39 2024-04-25T01:36:15.410 INFO:tasks.workunit.client.1.smithi138.stdout:4/640: dread d0/d1c/d1e/d9f/f34 [0,4194304] 0 2024-04-25T01:36:15.410 INFO:tasks.workunit.client.1.smithi138.stdout:1/773: chown d3/d27/f1e 299727072 1 2024-04-25T01:36:15.411 INFO:tasks.workunit.client.1.smithi138.stdout:0/665: write d6/f3b [1028080,104333] 0 2024-04-25T01:36:15.412 INFO:tasks.workunit.client.1.smithi138.stdout:1/774: readlink d3/d27/d16/l3a 0 2024-04-25T01:36:15.413 INFO:tasks.workunit.client.1.smithi138.stdout:8/977: chown d3/c2d 8802 1 2024-04-25T01:36:15.413 INFO:tasks.workunit.client.1.smithi138.stdout:6/910: mkdir dc/d10/dc8/d11d 0 2024-04-25T01:36:15.415 INFO:tasks.workunit.client.1.smithi138.stdout:9/733: rename d4/d1a/d68/c96 to d4/d1a/d3c/dc2/db2/dd8/cf6 0 2024-04-25T01:36:15.418 INFO:tasks.workunit.client.1.smithi138.stdout:7/828: unlink da/de/d13/d21/dbc/dca/f111 0 2024-04-25T01:36:15.419 INFO:tasks.workunit.client.1.smithi138.stdout:4/641: symlink d0/d45/d7f/lb9 0 2024-04-25T01:36:15.423 INFO:tasks.workunit.client.1.smithi138.stdout:1/775: mkdir d3/d27/d2d/d103 0 2024-04-25T01:36:15.424 INFO:tasks.workunit.client.1.smithi138.stdout:4/642: dread d0/d1c/f28 [0,4194304] 0 2024-04-25T01:36:15.426 INFO:tasks.workunit.client.1.smithi138.stdout:3/923: dread db/df/d17/f5e [0,4194304] 0 2024-04-25T01:36:15.426 INFO:tasks.workunit.client.1.smithi138.stdout:7/829: symlink da/de/d13/d21/da5/l114 0 2024-04-25T01:36:15.428 INFO:tasks.workunit.client.1.smithi138.stdout:6/911: creat dc/d10/d28/dec/f11e x:0 0 0 2024-04-25T01:36:15.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/643: unlink d0/f30 0 2024-04-25T01:36:15.430 INFO:tasks.workunit.client.1.smithi138.stdout:4/644: readlink d0/d1c/l1d 0 2024-04-25T01:36:15.432 INFO:tasks.workunit.client.1.smithi138.stdout:1/776: chown d3/d45/d72/d57/db0/dcd/de3/ffc 390083 1 2024-04-25T01:36:15.435 INFO:tasks.workunit.client.1.smithi138.stdout:3/924: write db/df/d17/d23/fa1 [1377228,65000] 0 2024-04-25T01:36:15.436 INFO:tasks.workunit.client.1.smithi138.stdout:3/925: readlink db/d9b/lbb 0 2024-04-25T01:36:15.436 INFO:tasks.workunit.client.1.smithi138.stdout:7/830: mknod da/de/d13/d30/df8/c115 0 2024-04-25T01:36:15.438 INFO:tasks.workunit.client.1.smithi138.stdout:7/831: readlink l5 0 2024-04-25T01:36:15.438 INFO:tasks.workunit.client.1.smithi138.stdout:4/645: dwrite d0/d1c/d1e/d9f/d33/f99 [0,4194304] 0 2024-04-25T01:36:15.438 INFO:tasks.workunit.client.1.smithi138.stdout:3/926: dread - db/d9b/fc2 zero size 2024-04-25T01:36:15.438 INFO:tasks.workunit.client.1.smithi138.stdout:7/832: chown da/de/d1b/f33 214 1 2024-04-25T01:36:15.442 INFO:tasks.workunit.client.1.smithi138.stdout:5/854: rename d4/da/d6e/d22/d35/dce/df9/d83/d89/d9d/d9e/fb1 to d4/da/d6e/f11c 0 2024-04-25T01:36:15.443 INFO:tasks.workunit.client.1.smithi138.stdout:0/666: write d6/df/d3a/fae [840844,92343] 0 2024-04-25T01:36:15.446 INFO:tasks.workunit.client.1.smithi138.stdout:8/978: getdents d3/d16/d1f/dd2/dd6 0 2024-04-25T01:36:15.451 INFO:tasks.workunit.client.1.smithi138.stdout:6/912: rename dc/d10/d28/d70/d83/d10a/fa8 to dc/d10/d28/d70/d83/d10a/d8c/dff/d114/f11f 0 2024-04-25T01:36:15.452 INFO:tasks.workunit.client.1.smithi138.stdout:0/667: creat d6/df/d69/fd4 x:0 0 0 2024-04-25T01:36:15.454 INFO:tasks.workunit.client.1.smithi138.stdout:5/855: creat d4/da/d6e/d22/d35/d4d/d5e/d60/dbd/dc8/f11d x:0 0 0 2024-04-25T01:36:15.455 INFO:tasks.workunit.client.1.smithi138.stdout:7/833: write da/de/d13/d21/f41 [2812916,45069] 0 2024-04-25T01:36:15.456 INFO:tasks.workunit.client.1.smithi138.stdout:8/979: mkdir d3/d44/da4/d12a/d134 0 2024-04-25T01:36:15.458 INFO:tasks.workunit.client.1.smithi138.stdout:4/646: link d0/d1c/d1e/d9f/d33/f51 d0/d1c/d48/fba 0 2024-04-25T01:36:15.461 INFO:tasks.workunit.client.1.smithi138.stdout:0/668: rename d6/la4 to d6/d39/d52/db7/ld5 0 2024-04-25T01:36:15.465 INFO:tasks.workunit.client.1.smithi138.stdout:8/980: stat d3/d44/l7a 0 2024-04-25T01:36:15.466 INFO:tasks.workunit.client.1.smithi138.stdout:1/777: truncate d3/d45/d72/db2/fbb 4856089 0 2024-04-25T01:36:15.468 INFO:tasks.workunit.client.1.smithi138.stdout:1/778: write d3/d27/d16/da3/daf/ff8 [738505,72267] 0 2024-04-25T01:36:15.468 INFO:tasks.workunit.client.1.smithi138.stdout:1/779: fdatasync d3/d27/d3e/f6b 0 2024-04-25T01:36:15.470 INFO:tasks.workunit.client.1.smithi138.stdout:4/647: rename d0/d1c/d52 to d0/d1c/d1e/dbb 0 2024-04-25T01:36:15.471 INFO:tasks.workunit.client.1.smithi138.stdout:5/856: mknod d4/da/d6e/c11e 0 2024-04-25T01:36:15.472 INFO:tasks.workunit.client.1.smithi138.stdout:0/669: fsync d6/d39/d52/d9c/dbc/dbf/fc3 0 2024-04-25T01:36:15.474 INFO:tasks.workunit.client.1.smithi138.stdout:6/913: symlink dc/d2e/d55/l120 0 2024-04-25T01:36:15.475 INFO:tasks.workunit.client.1.smithi138.stdout:8/981: mkdir d3/d16/d88/da/db/d15/d56/d8f/def/d135 0 2024-04-25T01:36:15.479 INFO:tasks.workunit.client.1.smithi138.stdout:6/914: dread dc/d10/d67/d8e/daf/fdd [0,4194304] 0 2024-04-25T01:36:15.481 INFO:tasks.workunit.client.1.smithi138.stdout:7/834: dwrite da/de/d13/d21/dbc/d94/d9d/dc9/fd1 [0,4194304] 0 2024-04-25T01:36:15.482 INFO:tasks.workunit.client.1.smithi138.stdout:6/915: dread - dc/d10/d28/d70/d83/d10a/d8c/db7/fcb zero size 2024-04-25T01:36:15.483 INFO:tasks.workunit.client.1.smithi138.stdout:6/916: chown dc/d10/d67/d8e/daf/fef 44098 1 2024-04-25T01:36:15.484 INFO:tasks.workunit.client.1.smithi138.stdout:6/917: fsync dc/d10/d28/d80/d89/da0/f112 0 2024-04-25T01:36:15.485 INFO:tasks.workunit.client.1.smithi138.stdout:1/780: mkdir d3/d4/da7/d104 0 2024-04-25T01:36:15.487 INFO:tasks.workunit.client.1.smithi138.stdout:1/781: write d3/d27/d3e/fd4 [513304,3415] 0 2024-04-25T01:36:15.489 INFO:tasks.workunit.client.1.smithi138.stdout:5/857: creat d4/da/d6e/d3c/d10e/f11f x:0 0 0 2024-04-25T01:36:15.490 INFO:tasks.workunit.client.1.smithi138.stdout:6/918: symlink dc/d8d/dca/l121 0 2024-04-25T01:36:15.490 INFO:tasks.workunit.client.1.smithi138.stdout:6/919: stat dc/d10/d67/fe5 0 2024-04-25T01:36:15.491 INFO:tasks.workunit.client.1.smithi138.stdout:1/782: rmdir d3/ddb/dd6 39 2024-04-25T01:36:15.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:15 smithi033 ceph-mon[26353]: pgmap v23: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 78 MiB/s rd, 194 MiB/s wr, 436 op/s 2024-04-25T01:36:15.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:15 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:15.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:15 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:15.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:15 smithi033 ceph-mon[26353]: Reconfiguring alertmanager.smithi033 (dependencies changed)... 2024-04-25T01:36:15.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:15 smithi033 ceph-mon[26353]: Reconfiguring daemon alertmanager.smithi033 on smithi033 2024-04-25T01:36:15.493 INFO:tasks.workunit.client.1.smithi138.stdout:7/835: symlink da/de/d13/d21/dbc/dff/l116 0 2024-04-25T01:36:15.493 INFO:tasks.workunit.client.1.smithi138.stdout:0/670: write d6/f56 [2732454,88040] 0 2024-04-25T01:36:15.497 INFO:tasks.workunit.client.1.smithi138.stdout:4/648: getdents d0/d1c/d1e/dbb/da3 0 2024-04-25T01:36:15.499 INFO:tasks.workunit.client.1.smithi138.stdout:4/649: mkdir d0/d59/dbc 0 2024-04-25T01:36:15.501 INFO:tasks.workunit.client.1.smithi138.stdout:1/783: symlink d3/d4/da7/l105 0 2024-04-25T01:36:15.503 INFO:tasks.workunit.client.1.smithi138.stdout:0/671: link d6/d2f/f7e d6/d19/dbb/fd6 0 2024-04-25T01:36:15.504 INFO:tasks.workunit.client.1.smithi138.stdout:1/784: fdatasync d3/d27/f1e 0 2024-04-25T01:36:15.507 INFO:tasks.workunit.client.1.smithi138.stdout:8/982: dwrite d3/d16/d1f/d87/f91 [0,4194304] 0 2024-04-25T01:36:15.507 INFO:tasks.workunit.client.1.smithi138.stdout:5/858: getdents d4/da/d6e/d5c 0 2024-04-25T01:36:15.508 INFO:tasks.workunit.client.1.smithi138.stdout:6/920: dwrite dc/d2e/f31 [0,4194304] 0 2024-04-25T01:36:15.509 INFO:tasks.workunit.client.1.smithi138.stdout:5/859: stat d4/da/fd 0 2024-04-25T01:36:15.511 INFO:tasks.workunit.client.1.smithi138.stdout:4/650: symlink d0/d1c/d1e/d9f/d4a/lbd 0 2024-04-25T01:36:15.515 INFO:tasks.workunit.client.1.smithi138.stdout:7/836: rename da/de/d1b/fa3 to da/de/d13/f117 0 2024-04-25T01:36:15.518 INFO:tasks.workunit.client.1.smithi138.stdout:0/672: chown d6/d19/c37 36589459 1 2024-04-25T01:36:15.520 INFO:tasks.workunit.client.1.smithi138.stdout:1/785: creat d3/d45/f106 x:0 0 0 2024-04-25T01:36:15.521 INFO:tasks.workunit.client.1.smithi138.stdout:0/673: write d6/df/d41/fa8 [742456,130965] 0 2024-04-25T01:36:15.521 INFO:tasks.workunit.client.1.smithi138.stdout:6/921: creat dc/d2e/d55/d7e/f122 x:0 0 0 2024-04-25T01:36:15.523 INFO:tasks.workunit.client.1.smithi138.stdout:8/983: truncate d3/d16/d88/fc1 3764633 0 2024-04-25T01:36:15.524 INFO:tasks.workunit.client.1.smithi138.stdout:4/651: mknod d0/d1c/d1e/d9f/d4a/d74/cbe 0 2024-04-25T01:36:15.527 INFO:tasks.workunit.client.1.smithi138.stdout:1/786: creat d3/d27/d2d/da1/dca/f107 x:0 0 0 2024-04-25T01:36:15.528 INFO:tasks.workunit.client.1.smithi138.stdout:6/922: unlink dc/d10/d28/cd9 0 2024-04-25T01:36:15.529 INFO:tasks.workunit.client.1.smithi138.stdout:6/923: chown dc/d10/d67/d8e/lf7 30 1 2024-04-25T01:36:15.530 INFO:tasks.workunit.client.1.smithi138.stdout:6/924: readlink dc/d10/lf1 0 2024-04-25T01:36:15.531 INFO:tasks.workunit.client.1.smithi138.stdout:8/984: creat d3/d16/d88/da/de/d57/f136 x:0 0 0 2024-04-25T01:36:15.532 INFO:tasks.workunit.client.1.smithi138.stdout:8/985: chown d3/d16/d1f/dc5 6978122 1 2024-04-25T01:36:15.533 INFO:tasks.workunit.client.1.smithi138.stdout:1/787: mknod d3/d27/d2d/c108 0 2024-04-25T01:36:15.536 INFO:tasks.workunit.client.1.smithi138.stdout:6/925: unlink dc/d7b/f94 0 2024-04-25T01:36:15.537 INFO:tasks.workunit.client.1.smithi138.stdout:4/652: mknod d0/d1c/d1e/dbb/dae/cbf 0 2024-04-25T01:36:15.538 INFO:tasks.workunit.client.1.smithi138.stdout:6/926: write dc/d10/d28/d80/d89/da0/f112 [850533,30238] 0 2024-04-25T01:36:15.539 INFO:tasks.workunit.client.1.smithi138.stdout:1/788: dread d3/d4/da7/db1/de1/fea [0,4194304] 0 2024-04-25T01:36:15.541 INFO:tasks.workunit.client.1.smithi138.stdout:4/653: write d0/d1c/d1e/d9f/fa7 [17350,14330] 0 2024-04-25T01:36:15.542 INFO:tasks.workunit.client.1.smithi138.stdout:7/837: write da/de/d13/d30/d59/d7b/d99/fab [1295103,31081] 0 2024-04-25T01:36:15.552 INFO:tasks.workunit.client.1.smithi138.stdout:4/654: creat d0/d1c/d1e/d9f/d4a/fc0 x:0 0 0 2024-04-25T01:36:15.552 INFO:tasks.workunit.client.1.smithi138.stdout:6/927: dwrite dc/d10/f22 [0,4194304] 0 2024-04-25T01:36:15.552 INFO:tasks.workunit.client.1.smithi138.stdout:0/674: dwrite d6/d39/d43/f6b [0,4194304] 0 2024-04-25T01:36:15.552 INFO:tasks.workunit.client.1.smithi138.stdout:7/838: mknod da/dd2/dcd/c118 0 2024-04-25T01:36:15.557 INFO:tasks.workunit.client.1.smithi138.stdout:1/789: link d3/ddb/d9b/f98 d3/d27/d16/da3/daf/ded/f109 0 2024-04-25T01:36:15.558 INFO:tasks.workunit.client.1.smithi138.stdout:8/986: truncate d3/d16/d88/da/de/f121 3716751 0 2024-04-25T01:36:15.563 INFO:tasks.workunit.client.1.smithi138.stdout:4/655: mkdir d0/d1c/d1e/dbb/dc1 0 2024-04-25T01:36:15.565 INFO:tasks.workunit.client.1.smithi138.stdout:7/839: creat da/de/d1b/d49/f119 x:0 0 0 2024-04-25T01:36:15.566 INFO:tasks.workunit.client.1.smithi138.stdout:7/840: stat da/fea 0 2024-04-25T01:36:15.567 INFO:tasks.workunit.client.1.smithi138.stdout:7/841: stat da/de/d13/d30/f93 0 2024-04-25T01:36:15.570 INFO:tasks.workunit.client.1.smithi138.stdout:8/987: rename d3/d16/d88/da/f43 to d3/d16/d1f/d50/f137 0 2024-04-25T01:36:15.571 INFO:tasks.workunit.client.1.smithi138.stdout:7/842: mknod da/de/d1b/d49/d81/d73/c11a 0 2024-04-25T01:36:15.571 INFO:tasks.workunit.client.1.smithi138.stdout:0/675: getdents d6/d1d/d82 0 2024-04-25T01:36:15.573 INFO:tasks.workunit.client.1.smithi138.stdout:8/988: creat d3/d16/d88/da/db/d15/d6f/f138 x:0 0 0 2024-04-25T01:36:15.574 INFO:tasks.workunit.client.1.smithi138.stdout:0/676: rename d6/d2f/fb1 to d6/df/d3a/fd7 0 2024-04-25T01:36:15.576 INFO:tasks.workunit.client.1.smithi138.stdout:7/843: truncate da/f15 6346755 0 2024-04-25T01:36:15.576 INFO:tasks.workunit.client.1.smithi138.stdout:7/844: readlink da/de/d13/d21/l37 0 2024-04-25T01:36:15.576 INFO:tasks.workunit.client.1.smithi138.stdout:8/989: symlink d3/d44/l139 0 2024-04-25T01:36:15.578 INFO:tasks.workunit.client.1.smithi138.stdout:7/845: read - da/de/d13/d21/dbc/dca/fe9 zero size 2024-04-25T01:36:15.579 INFO:tasks.workunit.client.1.smithi138.stdout:8/990: mkdir d3/d44/d13a 0 2024-04-25T01:36:15.580 INFO:tasks.workunit.client.1.smithi138.stdout:0/677: getdents d6/d39/d52/d9c/dbc 0 2024-04-25T01:36:15.582 INFO:tasks.workunit.client.1.smithi138.stdout:8/991: symlink d3/d16/d1f/d50/l13b 0 2024-04-25T01:36:15.583 INFO:tasks.workunit.client.1.smithi138.stdout:8/992: chown d3/d16/d1f/de2/f10a 2 1 2024-04-25T01:36:15.587 INFO:tasks.workunit.client.1.smithi138.stdout:6/928: dread dc/d2e/db9/fbc [0,4194304] 0 2024-04-25T01:36:15.589 INFO:tasks.workunit.client.1.smithi138.stdout:6/929: mknod dc/d10/d28/c123 0 2024-04-25T01:36:15.590 INFO:tasks.workunit.client.1.smithi138.stdout:4/656: write d0/d1c/d48/f6f [1353518,36893] 0 2024-04-25T01:36:15.595 INFO:tasks.workunit.client.1.smithi138.stdout:1/790: dread d3/d27/f1e [0,4194304] 0 2024-04-25T01:36:15.600 INFO:tasks.workunit.client.1.smithi138.stdout:4/657: dwrite d0/d1c/d1e/d9f/f31 [0,4194304] 0 2024-04-25T01:36:15.601 INFO:tasks.workunit.client.1.smithi138.stdout:1/791: mkdir d3/d27/d16/d10a 0 2024-04-25T01:36:15.606 INFO:tasks.workunit.client.1.smithi138.stdout:4/658: mkdir d0/d1c/d1e/d9f/d4a/dc2 0 2024-04-25T01:36:15.612 INFO:tasks.workunit.client.1.smithi138.stdout:7/846: dwrite da/de/d13/d30/df8/fb9 [4194304,4194304] 0 2024-04-25T01:36:15.612 INFO:tasks.workunit.client.1.smithi138.stdout:1/792: mknod d3/d45/d72/c10b 0 2024-04-25T01:36:15.613 INFO:tasks.workunit.client.1.smithi138.stdout:8/993: dread d3/d16/d88/da/de/d57/d100/f119 [0,4194304] 0 2024-04-25T01:36:15.615 INFO:tasks.workunit.client.1.smithi138.stdout:0/678: dwrite d6/d2f/d31/fb2 [0,4194304] 0 2024-04-25T01:36:15.616 INFO:tasks.workunit.client.1.smithi138.stdout:6/930: dwrite dc/d10/d28/d70/d83/d10a/fac [0,4194304] 0 2024-04-25T01:36:15.623 INFO:tasks.workunit.client.1.smithi138.stdout:1/793: rename d3/d27/d16/da3/daf/fb3 to d3/dcc/f10c 0 2024-04-25T01:36:15.624 INFO:tasks.workunit.client.1.smithi138.stdout:7/847: dread da/de/d13/f23 [0,4194304] 0 2024-04-25T01:36:15.624 INFO:tasks.workunit.client.1.smithi138.stdout:1/794: fsync d3/d45/d72/f52 0 2024-04-25T01:36:15.628 INFO:tasks.workunit.client.1.smithi138.stdout:6/931: unlink dc/d10/d28/d70/d83/d10a/d8c/c9e 0 2024-04-25T01:36:15.632 INFO:tasks.workunit.client.1.smithi138.stdout:0/679: creat d6/d2f/d75/db6/fd8 x:0 0 0 2024-04-25T01:36:15.632 INFO:tasks.workunit.client.1.smithi138.stdout:4/659: write d0/d1c/f49 [4916646,72794] 0 2024-04-25T01:36:15.633 INFO:tasks.workunit.client.1.smithi138.stdout:1/795: mknod d3/d45/c10d 0 2024-04-25T01:36:15.634 INFO:tasks.workunit.client.1.smithi138.stdout:7/848: rmdir da/de/d1b/d49/d81/d51 39 2024-04-25T01:36:15.634 INFO:tasks.workunit.client.1.smithi138.stdout:0/680: fdatasync d6/df/d66/f7f 0 2024-04-25T01:36:15.636 INFO:tasks.workunit.client.1.smithi138.stdout:1/796: dread - d3/ddb/ffe zero size 2024-04-25T01:36:15.637 INFO:tasks.workunit.client.1.smithi138.stdout:4/660: dread - d0/d1c/d1e/d9f/d33/f9b zero size 2024-04-25T01:36:15.637 INFO:tasks.workunit.client.1.smithi138.stdout:6/932: dwrite dc/d10/d28/d70/d83/f9d [0,4194304] 0 2024-04-25T01:36:15.638 INFO:tasks.workunit.client.1.smithi138.stdout:7/849: symlink da/dd2/dcd/l11b 0 2024-04-25T01:36:15.639 INFO:tasks.workunit.client.1.smithi138.stdout:4/661: readlink d0/d45/d7f/l95 0 2024-04-25T01:36:15.640 INFO:tasks.workunit.client.1.smithi138.stdout:8/994: link d3/d16/d1f/d31/c130 d3/d16/d1f/d87/c13c 0 2024-04-25T01:36:15.640 INFO:tasks.workunit.client.1.smithi138.stdout:0/681: creat d6/d1d/d98/fd9 x:0 0 0 2024-04-25T01:36:15.644 INFO:tasks.workunit.client.1.smithi138.stdout:1/797: rename d3/d27/d16/c33 to d3/d4/da7/db1/c10e 0 2024-04-25T01:36:15.647 INFO:tasks.workunit.client.1.smithi138.stdout:7/850: mknod da/de/d13/d30/df8/c11c 0 2024-04-25T01:36:15.648 INFO:tasks.workunit.client.1.smithi138.stdout:4/662: rmdir d0/d1c/d1e/dbb/dae 39 2024-04-25T01:36:15.650 INFO:tasks.workunit.client.1.smithi138.stdout:6/933: dwrite dc/d10/d67/f77 [0,4194304] 0 2024-04-25T01:36:15.652 INFO:tasks.workunit.client.1.smithi138.stdout:8/995: creat d3/d16/d1f/d2a/d4f/dcb/f13d x:0 0 0 2024-04-25T01:36:15.658 INFO:tasks.workunit.client.1.smithi138.stdout:6/934: mkdir dc/d10/d6d/d124 0 2024-04-25T01:36:15.658 INFO:tasks.workunit.client.1.smithi138.stdout:0/682: dread d6/d19/f74 [0,4194304] 0 2024-04-25T01:36:15.659 INFO:tasks.workunit.client.1.smithi138.stdout:7/851: dwrite da/de/d13/d30/d59/d7b/d99/fa6 [0,4194304] 0 2024-04-25T01:36:15.664 INFO:tasks.workunit.client.1.smithi138.stdout:6/935: mknod dc/d10/c125 0 2024-04-25T01:36:15.665 INFO:tasks.workunit.client.1.smithi138.stdout:4/663: dread d0/d1c/d1e/d9f/d4a/f4d [0,4194304] 0 2024-04-25T01:36:15.665 INFO:tasks.workunit.client.1.smithi138.stdout:0/683: read d6/df/d41/f55 [666324,41684] 0 2024-04-25T01:36:15.666 INFO:tasks.workunit.client.1.smithi138.stdout:8/996: mkdir d3/d44/da4/d12a/d13e 0 2024-04-25T01:36:15.669 INFO:tasks.workunit.client.1.smithi138.stdout:0/684: creat d6/d39/d52/fda x:0 0 0 2024-04-25T01:36:15.670 INFO:tasks.workunit.client.1.smithi138.stdout:6/936: read dc/d10/d28/f29 [1100666,41462] 0 2024-04-25T01:36:15.670 INFO:tasks.workunit.client.1.smithi138.stdout:4/664: creat d0/d1c/d1e/fc3 x:0 0 0 2024-04-25T01:36:15.672 INFO:tasks.workunit.client.1.smithi138.stdout:0/685: rmdir d6/d39/d58 39 2024-04-25T01:36:15.672 INFO:tasks.workunit.client.1.smithi138.stdout:6/937: stat dc/d10/d28/d80/d89/fd2 0 2024-04-25T01:36:15.673 INFO:tasks.workunit.client.1.smithi138.stdout:7/852: getdents da/de/d13 0 2024-04-25T01:36:15.676 INFO:tasks.workunit.client.1.smithi138.stdout:0/686: mkdir d6/df/d41/d49/ddb 0 2024-04-25T01:36:15.677 INFO:tasks.workunit.client.1.smithi138.stdout:1/798: dread d3/d45/d72/d57/d8e/fbe [0,4194304] 0 2024-04-25T01:36:15.678 INFO:tasks.workunit.client.1.smithi138.stdout:6/938: truncate dc/d10/d28/d70/d83/d10a/d8c/dff/d114/f11f 1371967 0 2024-04-25T01:36:15.680 INFO:tasks.workunit.client.1.smithi138.stdout:4/665: link d0/d1c/d1e/d9f/d7e/fab d0/d1c/d1e/d9f/fc4 0 2024-04-25T01:36:15.680 INFO:tasks.workunit.client.1.smithi138.stdout:8/997: dread d3/d16/d88/da/db/f96 [0,4194304] 0 2024-04-25T01:36:15.682 INFO:tasks.workunit.client.1.smithi138.stdout:7/853: chown da/de/d1b/d49/d81/l7f 45 1 2024-04-25T01:36:15.684 INFO:tasks.workunit.client.1.smithi138.stdout:6/939: symlink dc/d2e/d55/d7e/l126 0 2024-04-25T01:36:15.685 INFO:tasks.workunit.client.1.smithi138.stdout:0/687: unlink d6/d1d/d98/lc9 0 2024-04-25T01:36:15.687 INFO:tasks.workunit.client.1.smithi138.stdout:8/998: mkdir d3/d16/d1f/d2a/d4f/dac/d13f 0 2024-04-25T01:36:15.687 INFO:tasks.workunit.client.1.smithi138.stdout:4/666: dwrite d0/d1c/d1e/d9f/d4a/f8f [0,4194304] 0 2024-04-25T01:36:15.688 INFO:tasks.workunit.client.1.smithi138.stdout:8/999: write d3/f3c [2233426,47478] 0 2024-04-25T01:36:15.689 INFO:tasks.workunit.client.1.smithi138.stdout:1/799: mkdir d3/ddb/d10f 0 2024-04-25T01:36:15.691 INFO:tasks.workunit.client.1.smithi138.stdout:0/688: mkdir d6/d2f/d31/ddc 0 2024-04-25T01:36:15.693 INFO:tasks.workunit.client.1.smithi138.stdout:4/667: creat d0/d1c/d1e/d9f/d4a/d74/fc5 x:0 0 0 2024-04-25T01:36:15.694 INFO:tasks.workunit.client.1.smithi138.stdout:6/940: getdents dc/d10/dc8/d11d 0 2024-04-25T01:36:15.697 INFO:tasks.workunit.client.1.smithi138.stdout:0/689: creat d6/d39/d52/d9c/dbc/dbf/fdd x:0 0 0 2024-04-25T01:36:15.701 INFO:tasks.workunit.client.1.smithi138.stdout:1/800: rename d3/d27/d2d/da1/dca/f107 to d3/ddb/dd6/f110 0 2024-04-25T01:36:15.702 INFO:tasks.workunit.client.1.smithi138.stdout:7/854: getdents da/dd2/dcd 0 2024-04-25T01:36:15.703 INFO:tasks.workunit.client.1.smithi138.stdout:4/668: rename d0/d45 to d0/d1c/d1e/dbb/dae/dc6 0 2024-04-25T01:36:15.709 INFO:tasks.workunit.client.1.smithi138.stdout:7/855: fsync da/de/d13/d30/d59/f78 0 2024-04-25T01:36:15.711 INFO:tasks.workunit.client.1.smithi138.stdout:1/801: rename d3/d4/da7/db1/cbd to d3/d45/d72/db2/dd9/c111 0 2024-04-25T01:36:15.714 INFO:tasks.workunit.client.1.smithi138.stdout:7/856: symlink da/de/d13/d21/d6f/d8d/l11d 0 2024-04-25T01:36:15.714 INFO:tasks.workunit.client.1.smithi138.stdout:1/802: creat d3/d45/d72/d57/db0/f112 x:0 0 0 2024-04-25T01:36:15.717 INFO:tasks.workunit.client.1.smithi138.stdout:7/857: stat da/de/d1b/d49/d81/dc7/cdb 0 2024-04-25T01:36:15.718 INFO:tasks.workunit.client.1.smithi138.stdout:7/858: dread - da/de/d1b/d4b/f105 zero size 2024-04-25T01:36:15.719 INFO:tasks.workunit.client.1.smithi138.stdout:1/803: dwrite d3/d27/d2d/f32 [0,4194304] 0 2024-04-25T01:36:15.746 INFO:tasks.workunit.client.1.smithi138.stdout:7/859: mknod da/de/d13/d21/da5/c11e 0 2024-04-25T01:36:15.749 INFO:tasks.workunit.client.1.smithi138.stdout:1/804: rename d3/d4/da7/db1/de4 to d3/d45/d72/d57/d8e/d113 0 2024-04-25T01:36:15.749 INFO:tasks.workunit.client.1.smithi138.stdout:6/941: fsync dc/d10/d28/d70/d83/d10a/d8c/dff/d114/f11f 0 2024-04-25T01:36:15.756 INFO:tasks.workunit.client.1.smithi138.stdout:1/805: rmdir d3/ddb/dd6/deb 0 2024-04-25T01:36:15.758 INFO:tasks.workunit.client.1.smithi138.stdout:4/669: dwrite d0/d1c/d1e/f85 [0,4194304] 0 2024-04-25T01:36:15.758 INFO:tasks.workunit.client.1.smithi138.stdout:6/942: dwrite dc/d10/d28/dec/f11e [0,4194304] 0 2024-04-25T01:36:15.759 INFO:tasks.workunit.client.1.smithi138.stdout:1/806: fdatasync d3/d45/f48 0 2024-04-25T01:36:15.759 INFO:tasks.workunit.client.1.smithi138.stdout:6/943: fsync dc/d10/f22 0 2024-04-25T01:36:15.762 INFO:tasks.workunit.client.1.smithi138.stdout:6/944: stat dc/d2e/d55/fa4 0 2024-04-25T01:36:15.763 INFO:tasks.workunit.client.1.smithi138.stdout:1/807: chown d3/d45/d72/d57/d8e/d113/daa/fd8 456 1 2024-04-25T01:36:15.764 INFO:tasks.workunit.client.1.smithi138.stdout:1/808: dread - d3/d45/d72/db2/fc7 zero size 2024-04-25T01:36:15.764 INFO:tasks.workunit.client.1.smithi138.stdout:7/860: write da/de/d13/d21/f7e [4666209,47449] 0 2024-04-25T01:36:15.765 INFO:tasks.workunit.client.1.smithi138.stdout:4/670: truncate d0/d1c/d1e/d9f/d4a/d74/fc5 965924 0 2024-04-25T01:36:15.765 INFO:tasks.workunit.client.1.smithi138.stdout:6/945: stat dc/d2e/d55/d81/lb5 0 2024-04-25T01:36:15.766 INFO:tasks.workunit.client.1.smithi138.stdout:1/809: chown d3/d4/da7/d104 5346750 1 2024-04-25T01:36:15.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:15 smithi138 ceph-mon[32977]: pgmap v23: 65 pgs: 65 active+clean; 3.2 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 78 MiB/s rd, 194 MiB/s wr, 436 op/s 2024-04-25T01:36:15.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:15 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:15.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:15 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:15.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:15 smithi138 ceph-mon[32977]: Reconfiguring alertmanager.smithi033 (dependencies changed)... 2024-04-25T01:36:15.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:15 smithi138 ceph-mon[32977]: Reconfiguring daemon alertmanager.smithi033 on smithi033 2024-04-25T01:36:15.774 INFO:tasks.workunit.client.1.smithi138.stdout:7/861: link da/de/d13/d30/d59/ff3 da/de/d13/d21/da5/de8/f11f 0 2024-04-25T01:36:15.776 INFO:tasks.workunit.client.1.smithi138.stdout:4/671: link d0/d59/c7d d0/d1c/d1e/d9f/d4a/dc2/cc7 0 2024-04-25T01:36:15.776 INFO:tasks.workunit.client.1.smithi138.stdout:7/862: chown da/de/d13/d30/f65 6558 1 2024-04-25T01:36:15.777 INFO:tasks.workunit.client.1.smithi138.stdout:7/863: stat da/de/d1b/d4b/d5b/l36 0 2024-04-25T01:36:15.778 INFO:tasks.workunit.client.1.smithi138.stdout:4/672: creat d0/d1c/d1e/dbb/da3/db5/fc8 x:0 0 0 2024-04-25T01:36:15.782 INFO:tasks.workunit.client.1.smithi138.stdout:4/673: dread d0/d1c/d1e/f85 [0,4194304] 0 2024-04-25T01:36:15.784 INFO:tasks.workunit.client.1.smithi138.stdout:6/946: dread dc/f16 [0,4194304] 0 2024-04-25T01:36:15.785 INFO:tasks.workunit.client.1.smithi138.stdout:4/674: symlink d0/d1c/d1e/d9f/lc9 0 2024-04-25T01:36:15.786 INFO:tasks.workunit.client.1.smithi138.stdout:6/947: mknod dc/d2e/dc1/c127 0 2024-04-25T01:36:15.787 INFO:tasks.workunit.client.1.smithi138.stdout:6/948: write dc/d10/dc8/ffc [1417695,39311] 0 2024-04-25T01:36:15.792 INFO:tasks.workunit.client.1.smithi138.stdout:6/949: fdatasync dc/d10/d28/d80/d89/fd2 0 2024-04-25T01:36:15.793 INFO:tasks.workunit.client.1.smithi138.stdout:4/675: link d0/d1c/d1e/dbb/dae/dc6/d7f/l95 d0/d59/dbc/lca 0 2024-04-25T01:36:15.796 INFO:tasks.workunit.client.1.smithi138.stdout:4/676: rename d0/d1c/d1e/dbb/dae/dc6 to d0/d1c/d1e/dbb/dae/dc6/dcb 22 2024-04-25T01:36:15.796 INFO:tasks.workunit.client.1.smithi138.stdout:6/950: creat dc/d10/d28/d70/d83/d10a/d8c/db7/f128 x:0 0 0 2024-04-25T01:36:15.797 INFO:tasks.workunit.client.1.smithi138.stdout:4/677: chown d0/d1c/d1e/d9f/lc9 0 1 2024-04-25T01:36:15.798 INFO:tasks.workunit.client.1.smithi138.stdout:1/810: dread d3/d45/d72/d57/d8e/d113/fc5 [0,4194304] 0 2024-04-25T01:36:15.799 INFO:tasks.workunit.client.1.smithi138.stdout:6/951: mknod dc/d2e/d55/d81/c129 0 2024-04-25T01:36:15.804 INFO:tasks.workunit.client.1.smithi138.stdout:6/952: getdents dc/d10/d67 0 2024-04-25T01:36:15.806 INFO:tasks.workunit.client.1.smithi138.stdout:1/811: rename d3/d45/d72/d57/db0/dcd/fd0 to d3/d4/da7/f114 0 2024-04-25T01:36:15.807 INFO:tasks.workunit.client.1.smithi138.stdout:4/678: truncate d0/d1c/d1e/d9f/f44 8099975 0 2024-04-25T01:36:15.810 INFO:tasks.workunit.client.1.smithi138.stdout:1/812: link d3/dcc/fdd d3/d27/d16/da3/daf/ded/f115 0 2024-04-25T01:36:15.813 INFO:tasks.workunit.client.1.smithi138.stdout:1/813: truncate d3/db/f14 3079958 0 2024-04-25T01:36:15.814 INFO:tasks.workunit.client.1.smithi138.stdout:1/814: fdatasync d3/d27/d2d/f32 0 2024-04-25T01:36:15.815 INFO:tasks.workunit.client.1.smithi138.stdout:1/815: write d3/d45/d72/d57/d8e/fbe [4909601,31872] 0 2024-04-25T01:36:15.816 INFO:tasks.workunit.client.1.smithi138.stdout:1/816: chown d3/d27/d16/cba 8 1 2024-04-25T01:36:15.820 INFO:tasks.workunit.client.1.smithi138.stdout:4/679: truncate d0/d1c/d48/f5b 323089 0 2024-04-25T01:36:15.823 INFO:tasks.workunit.client.1.smithi138.stdout:1/817: rename d3/d27/d3e/d51/f80 to d3/d45/d72/d57/d8e/db6/f116 0 2024-04-25T01:36:15.828 INFO:tasks.workunit.client.1.smithi138.stdout:4/680: creat d0/d1c/d1e/d9f/d4a/dc2/fcc x:0 0 0 2024-04-25T01:36:15.834 INFO:tasks.workunit.client.1.smithi138.stdout:4/681: rename d0/d59/c71 to d0/d1c/d1e/d9f/ccd 0 2024-04-25T01:36:15.836 INFO:tasks.workunit.client.1.smithi138.stdout:1/818: rmdir d3/d27/d2d/d103 0 2024-04-25T01:36:15.838 INFO:tasks.workunit.client.1.smithi138.stdout:4/682: write d0/d1c/d1e/dbb/dae/dc6/f35 [587701,110441] 0 2024-04-25T01:36:15.840 INFO:tasks.workunit.client.1.smithi138.stdout:4/683: chown d0/d1c/d1e/dbb/da3/db5/fc8 18474601 1 2024-04-25T01:36:15.843 INFO:tasks.workunit.client.1.smithi138.stdout:4/684: rmdir d0/d1c/d1e/dbb/dc1 0 2024-04-25T01:36:15.844 INFO:tasks.workunit.client.1.smithi138.stdout:4/685: write d0/d1c/f49 [722098,43680] 0 2024-04-25T01:36:15.847 INFO:tasks.workunit.client.1.smithi138.stdout:4/686: rmdir d0/d1c/d1e/d9f/d4a/d74 39 2024-04-25T01:36:15.853 INFO:tasks.workunit.client.1.smithi138.stdout:1/819: dread d3/d27/d16/da3/fd2 [0,4194304] 0 2024-04-25T01:36:15.855 INFO:tasks.workunit.client.1.smithi138.stdout:4/687: truncate d0/d1c/d1e/d9f/d4a/f8f 2110287 0 2024-04-25T01:36:15.855 INFO:tasks.workunit.client.1.smithi138.stdout:4/688: dread - d0/d1c/f6b zero size 2024-04-25T01:36:15.856 INFO:tasks.workunit.client.1.smithi138.stdout:1/820: mkdir d3/ddb/dd6/dde/d117 0 2024-04-25T01:36:15.857 INFO:tasks.workunit.client.1.smithi138.stdout:4/689: mknod d0/d1c/cce 0 2024-04-25T01:36:15.859 INFO:tasks.workunit.client.1.smithi138.stdout:4/690: read d0/f3f [4103988,118396] 0 2024-04-25T01:36:15.859 INFO:tasks.workunit.client.1.smithi138.stdout:1/821: dread d3/d27/d16/da3/fd2 [0,4194304] 0 2024-04-25T01:36:15.861 INFO:tasks.workunit.client.1.smithi138.stdout:4/691: creat d0/d1c/d1e/dbb/dae/fcf x:0 0 0 2024-04-25T01:36:15.862 INFO:tasks.workunit.client.1.smithi138.stdout:4/692: chown d0/f13 1770838561 1 2024-04-25T01:36:15.862 INFO:tasks.workunit.client.1.smithi138.stdout:1/822: symlink d3/d27/d16/l118 0 2024-04-25T01:36:15.863 INFO:tasks.workunit.client.1.smithi138.stdout:4/693: rmdir d0/d1c/d1e/dbb/dae/dc6 39 2024-04-25T01:36:15.864 INFO:tasks.workunit.client.1.smithi138.stdout:1/823: dread - d3/d45/d72/db2/fdc zero size 2024-04-25T01:36:15.866 INFO:tasks.workunit.client.1.smithi138.stdout:1/824: write d3/d45/d72/d57/d8e/fab [3689342,105192] 0 2024-04-25T01:36:15.869 INFO:tasks.workunit.client.1.smithi138.stdout:1/825: symlink d3/d27/d16/da3/daf/ded/l119 0 2024-04-25T01:36:15.870 INFO:tasks.workunit.client.1.smithi138.stdout:1/826: stat d3/d45/cc1 0 2024-04-25T01:36:15.871 INFO:tasks.workunit.client.1.smithi138.stdout:4/694: dwrite d0/d1c/d1e/d9f/d4a/dc2/fcc [0,4194304] 0 2024-04-25T01:36:15.874 INFO:tasks.workunit.client.1.smithi138.stdout:1/827: creat d3/d45/d72/d57/d8e/db6/f11a x:0 0 0 2024-04-25T01:36:15.876 INFO:tasks.workunit.client.1.smithi138.stdout:4/695: symlink d0/d1c/d1e/dbb/ld0 0 2024-04-25T01:36:15.878 INFO:tasks.workunit.client.1.smithi138.stdout:4/696: dread d0/d1c/d1e/d9f/d4a/f4d [0,4194304] 0 2024-04-25T01:36:15.879 INFO:tasks.workunit.client.1.smithi138.stdout:1/828: link d3/d45/d72/f42 d3/ddb/dd6/f11b 0 2024-04-25T01:36:15.880 INFO:tasks.workunit.client.1.smithi138.stdout:1/829: chown d3/dcc/f10c 111685922 1 2024-04-25T01:36:15.880 INFO:tasks.workunit.client.1.smithi138.stdout:4/697: read d0/d1c/d48/f6c [69206,50184] 0 2024-04-25T01:36:15.882 INFO:tasks.workunit.client.1.smithi138.stdout:4/698: dread - d0/d1c/d1e/d9f/d4a/fc0 zero size 2024-04-25T01:36:15.883 INFO:tasks.workunit.client.1.smithi138.stdout:1/830: mkdir d3/db/d11c 0 2024-04-25T01:36:15.884 INFO:tasks.workunit.client.1.smithi138.stdout:1/831: write d3/d45/d72/d57/db0/f112 [616480,90867] 0 2024-04-25T01:36:15.885 INFO:tasks.workunit.client.1.smithi138.stdout:1/832: fdatasync d3/d27/f1e 0 2024-04-25T01:36:15.893 INFO:tasks.workunit.client.1.smithi138.stdout:4/699: dwrite d0/d1c/d48/fba [0,4194304] 0 2024-04-25T01:36:15.894 INFO:tasks.workunit.client.1.smithi138.stdout:4/700: dread - d0/d1c/d1e/d9f/d4a/fb1 zero size 2024-04-25T01:36:15.901 INFO:tasks.workunit.client.1.smithi138.stdout:4/701: dwrite d0/d1c/d1e/d9f/f79 [0,4194304] 0 2024-04-25T01:36:15.906 INFO:tasks.workunit.client.1.smithi138.stdout:4/702: creat d0/d1c/d1e/dbb/dae/dc6/fd1 x:0 0 0 2024-04-25T01:36:15.908 INFO:tasks.workunit.client.1.smithi138.stdout:4/703: write d0/d1c/d48/f6c [560835,12069] 0 2024-04-25T01:36:15.910 INFO:tasks.workunit.client.1.smithi138.stdout:4/704: mknod d0/d1c/d1e/dbb/da3/cd2 0 2024-04-25T01:36:15.911 INFO:tasks.workunit.client.1.smithi138.stdout:4/705: write d0/d1c/d1e/d9f/d4a/f77 [639809,21618] 0 2024-04-25T01:36:15.914 INFO:tasks.workunit.client.1.smithi138.stdout:9/734: sync 2024-04-25T01:36:15.914 INFO:tasks.workunit.client.1.smithi138.stdout:9/735: truncate d4/d63/d86/fbb 303897 0 2024-04-25T01:36:15.915 INFO:tasks.workunit.client.1.smithi138.stdout:9/736: write d4/d57/fce [263323,9925] 0 2024-04-25T01:36:15.947 INFO:tasks.workunit.client.1.smithi138.stdout:4/706: dwrite d0/d1c/f28 [0,4194304] 0 2024-04-25T01:36:15.947 INFO:tasks.workunit.client.1.smithi138.stdout:9/737: dwrite d4/d63/d86/f93 [0,4194304] 0 2024-04-25T01:36:15.967 INFO:tasks.workunit.client.1.smithi138.stdout:9/738: rmdir d4/d1a/d33/d9a 39 2024-04-25T01:36:15.969 INFO:tasks.workunit.client.1.smithi138.stdout:9/739: mkdir d4/d1a/d55/d7f/d8f/df7 0 2024-04-25T01:36:15.970 INFO:tasks.workunit.client.1.smithi138.stdout:4/707: read d0/d1c/d1e/d9f/d4a/faa [842425,93410] 0 2024-04-25T01:36:15.972 INFO:tasks.workunit.client.1.smithi138.stdout:4/708: rename d0/d1c/d86/c9d to d0/d1c/d1e/d9f/d33/cd3 0 2024-04-25T01:36:15.973 INFO:tasks.workunit.client.1.smithi138.stdout:4/709: rename d0/d1c/d1e/dbb/dae/dc6/d7f to d0/d1c/d1e/dbb/dae/dc6/d7f/dd4 22 2024-04-25T01:36:15.975 INFO:tasks.workunit.client.1.smithi138.stdout:9/740: write d4/f26 [2619561,54939] 0 2024-04-25T01:36:15.977 INFO:tasks.workunit.client.1.smithi138.stdout:4/710: rename d0/d1c/d1e/d9f/d4a to d0/d1c/d1e/d9f/d7e/dd5 0 2024-04-25T01:36:15.978 INFO:tasks.workunit.client.1.smithi138.stdout:4/711: readlink d0/d1c/d48/l4e 0 2024-04-25T01:36:15.979 INFO:tasks.workunit.client.1.smithi138.stdout:9/741: creat d4/d1a/d3c/dc2/ff8 x:0 0 0 2024-04-25T01:36:15.982 INFO:tasks.workunit.client.1.smithi138.stdout:9/742: unlink d4/d1a/d3c/l3d 0 2024-04-25T01:36:15.984 INFO:tasks.workunit.client.1.smithi138.stdout:4/712: dwrite d0/f3f [0,4194304] 0 2024-04-25T01:36:15.985 INFO:tasks.workunit.client.1.smithi138.stdout:9/743: creat d4/d8/d2c/ff9 x:0 0 0 2024-04-25T01:36:15.986 INFO:tasks.workunit.client.1.smithi138.stdout:4/713: write d0/d1c/f28 [2506945,128506] 0 2024-04-25T01:36:15.994 INFO:tasks.workunit.client.1.smithi138.stdout:4/714: rename d0/d1c/d1e/d9f/d7e/c93 to d0/d1c/d1e/dbb/dae/dc6/d7f/cd6 0 2024-04-25T01:36:15.994 INFO:tasks.workunit.client.1.smithi138.stdout:9/744: dwrite d4/d8/d90/f87 [0,4194304] 0 2024-04-25T01:36:15.998 INFO:tasks.workunit.client.1.smithi138.stdout:9/745: unlink d4/d8/d2c/l8c 0 2024-04-25T01:36:16.000 INFO:tasks.workunit.client.1.smithi138.stdout:4/715: rename d0/d1c/f49 to d0/d1c/d1e/fd7 0 2024-04-25T01:36:16.001 INFO:tasks.workunit.client.1.smithi138.stdout:4/716: fdatasync d0/d1c/d48/f6c 0 2024-04-25T01:36:16.006 INFO:tasks.workunit.client.1.smithi138.stdout:9/746: unlink d4/d8/d2c/d82/la1 0 2024-04-25T01:36:16.009 INFO:tasks.workunit.client.1.smithi138.stdout:9/747: truncate d4/d1a/f84 1849327 0 2024-04-25T01:36:16.017 INFO:tasks.workunit.client.1.smithi138.stdout:4/717: dwrite d0/d1c/f62 [0,4194304] 0 2024-04-25T01:36:16.031 INFO:tasks.workunit.client.1.smithi138.stdout:4/718: truncate d0/d1c/d1e/d9f/d33/f89 680253 0 2024-04-25T01:36:16.034 INFO:tasks.workunit.client.1.smithi138.stdout:4/719: rename d0/d1c/d1e/d9f/d7e/dd5/d74/f97 to d0/d1c/d1e/fd8 0 2024-04-25T01:36:16.040 INFO:tasks.workunit.client.1.smithi138.stdout:4/720: dwrite d0/d1c/d1e/d9f/f79 [4194304,4194304] 0 2024-04-25T01:36:16.042 INFO:tasks.workunit.client.1.smithi138.stdout:4/721: mkdir d0/d1c/d86/dd9 0 2024-04-25T01:36:16.049 INFO:tasks.workunit.client.1.smithi138.stdout:4/722: dwrite d0/d1c/f2c [0,4194304] 0 2024-04-25T01:36:16.052 INFO:tasks.workunit.client.1.smithi138.stdout:4/723: truncate d0/f1a 917195 0 2024-04-25T01:36:16.062 INFO:tasks.workunit.client.1.smithi138.stdout:4/724: write d0/d1c/d1e/d9f/d7e/fab [6707773,54754] 0 2024-04-25T01:36:16.064 INFO:tasks.workunit.client.1.smithi138.stdout:4/725: symlink d0/d1c/d1e/lda 0 2024-04-25T01:36:16.065 INFO:tasks.workunit.client.1.smithi138.stdout:4/726: readlink d0/l5e 0 2024-04-25T01:36:16.067 INFO:tasks.workunit.client.1.smithi138.stdout:4/727: symlink d0/d1c/d1e/d9f/d7e/dd5/ldb 0 2024-04-25T01:36:16.070 INFO:tasks.workunit.client.1.smithi138.stdout:4/728: rename d0/d1c/d1e/dbb/dae/dc6/d7f/lb9 to d0/d1c/d86/ldc 0 2024-04-25T01:36:16.071 INFO:tasks.workunit.client.1.smithi138.stdout:4/729: mknod d0/d59/dbc/cdd 0 2024-04-25T01:36:16.074 INFO:tasks.workunit.client.1.smithi138.stdout:4/730: rename d0/d1c/d48/f5b to d0/d1c/d1e/d9f/fde 0 2024-04-25T01:36:16.079 INFO:tasks.workunit.client.1.smithi138.stdout:4/731: dwrite d0/d1c/d1e/d9f/f31 [8388608,4194304] 0 2024-04-25T01:36:16.087 INFO:tasks.workunit.client.1.smithi138.stdout:4/732: dread d0/f40 [0,4194304] 0 2024-04-25T01:36:16.088 INFO:tasks.workunit.client.1.smithi138.stdout:4/733: write d0/d1c/d1e/dbb/da3/fac [544027,116402] 0 2024-04-25T01:36:16.090 INFO:tasks.workunit.client.1.smithi138.stdout:4/734: mkdir d0/d1c/d48/ddf 0 2024-04-25T01:36:16.092 INFO:tasks.workunit.client.1.smithi138.stdout:4/735: mkdir d0/d1c/d1e/d9f/d7e/de0 0 2024-04-25T01:36:16.093 INFO:tasks.workunit.client.1.smithi138.stdout:4/736: chown d0/f3f 622210046 1 2024-04-25T01:36:16.094 INFO:tasks.workunit.client.1.smithi138.stdout:4/737: read - d0/d1c/d1e/d9f/d7e/dd5/f6d zero size 2024-04-25T01:36:16.166 INFO:tasks.workunit.client.1.smithi138.stdout:3/927: sync 2024-04-25T01:36:16.171 INFO:tasks.workunit.client.1.smithi138.stdout:4/738: unlink d0/d1c/d1e/d9f/caf 0 2024-04-25T01:36:16.174 INFO:tasks.workunit.client.1.smithi138.stdout:3/928: symlink db/db3/d101/l121 0 2024-04-25T01:36:16.175 INFO:tasks.workunit.client.1.smithi138.stdout:4/739: readlink d0/d1c/d1e/dbb/dae/dc6/d7f/l95 0 2024-04-25T01:36:16.176 INFO:tasks.workunit.client.1.smithi138.stdout:4/740: chown d0/d1c/d1e/d9f/d33/c46 56395467 1 2024-04-25T01:36:16.178 INFO:tasks.workunit.client.1.smithi138.stdout:3/929: creat db/df/d19/d1a/de5/de7/f122 x:0 0 0 2024-04-25T01:36:16.179 INFO:tasks.workunit.client.1.smithi138.stdout:4/741: creat d0/d59/fe1 x:0 0 0 2024-04-25T01:36:16.181 INFO:tasks.workunit.client.1.smithi138.stdout:4/742: chown d0/d1c/d1e/f85 124 1 2024-04-25T01:36:16.182 INFO:tasks.workunit.client.1.smithi138.stdout:4/743: dread - d0/d59/f6a zero size 2024-04-25T01:36:16.184 INFO:tasks.workunit.client.1.smithi138.stdout:4/744: fsync d0/d1c/d1e/dbb/fa1 0 2024-04-25T01:36:16.186 INFO:tasks.workunit.client.1.smithi138.stdout:3/930: dwrite db/df/d17/d5d/fa3 [0,4194304] 0 2024-04-25T01:36:16.188 INFO:tasks.workunit.client.1.smithi138.stdout:4/745: rmdir d0/d1c/d1e/dbb/dae/dc6/d7f 39 2024-04-25T01:36:16.206 INFO:tasks.workunit.client.1.smithi138.stdout:4/746: truncate d0/d1c/d1e/dbb/f8b 219603 0 2024-04-25T01:36:16.212 INFO:tasks.workunit.client.1.smithi138.stdout:4/747: symlink d0/d1c/d1e/d9f/d7e/dd5/le2 0 2024-04-25T01:36:16.215 INFO:tasks.workunit.client.1.smithi138.stdout:4/748: unlink d0/d1c/c7b 0 2024-04-25T01:36:16.216 INFO:tasks.workunit.client.1.smithi138.stdout:4/749: readlink d0/d1c/d1e/dbb/da3/db5/lb8 0 2024-04-25T01:36:16.268 INFO:tasks.workunit.client.1.smithi138.stdout:4/750: dread d0/d1c/d48/f6f [0,4194304] 0 2024-04-25T01:36:16.269 INFO:tasks.workunit.client.1.smithi138.stdout:4/751: fdatasync d0/d1c/d1e/d9f/d7e/dd5/d74/f8d 0 2024-04-25T01:36:16.332 INFO:tasks.workunit.client.1.smithi138.stdout:5/860: sync 2024-04-25T01:36:16.333 INFO:tasks.workunit.client.1.smithi138.stdout:5/861: rmdir d4/da/d27 39 2024-04-25T01:36:16.336 INFO:tasks.workunit.client.1.smithi138.stdout:5/862: write d4/da/d6e/d22/d35/dce/df9/f68 [5179057,104858] 0 2024-04-25T01:36:16.337 INFO:tasks.workunit.client.1.smithi138.stdout:5/863: chown d4/da/d6e/d22/d35/d4d/d5e/d60/db5 1555781 1 2024-04-25T01:36:16.339 INFO:tasks.workunit.client.1.smithi138.stdout:5/864: mknod d4/da/d6e/d3c/d10e/c120 0 2024-04-25T01:36:16.342 INFO:tasks.workunit.client.1.smithi138.stdout:5/865: dread d4/da/d6e/d22/d45/f67 [0,4194304] 0 2024-04-25T01:36:16.343 INFO:tasks.workunit.client.1.smithi138.stdout:5/866: write d4/f6c [3557666,15104] 0 2024-04-25T01:36:16.344 INFO:tasks.workunit.client.1.smithi138.stdout:5/867: chown d4/da/d6e/d22/d35/d4d/d5e/d8f/d9f/c105 237 1 2024-04-25T01:36:16.347 INFO:tasks.workunit.client.1.smithi138.stdout:5/868: truncate d4/da/d6e/d22/f24 5223059 0 2024-04-25T01:36:16.351 INFO:tasks.workunit.client.1.smithi138.stdout:5/869: creat d4/da/d6e/d22/d35/dce/df9/d83/d89/d9d/d9e/f121 x:0 0 0 2024-04-25T01:36:16.352 INFO:tasks.workunit.client.1.smithi138.stdout:5/870: fdatasync d4/da/d6e/d22/d45/f67 0 2024-04-25T01:36:16.359 INFO:tasks.workunit.client.1.smithi138.stdout:5/871: rename d4/da/d6e/d22/d35/d4d to d4/d122 0 2024-04-25T01:36:16.360 INFO:tasks.workunit.client.1.smithi138.stdout:5/872: fdatasync d4/da/d6e/d22/d35/dce/df9/d7f/db3/fe1 0 2024-04-25T01:36:16.362 INFO:tasks.workunit.client.1.smithi138.stdout:5/873: mknod d4/d122/d5e/d60/dbd/c123 0 2024-04-25T01:36:16.365 INFO:tasks.workunit.client.1.smithi138.stdout:5/874: creat d4/da/d6e/d22/d35/dce/df9/d7f/f124 x:0 0 0 2024-04-25T01:36:16.368 INFO:tasks.workunit.client.1.smithi138.stdout:5/875: link d4/da/cf2 d4/da/d6e/d22/d74/d8e/dd7/c125 0 2024-04-25T01:36:16.370 INFO:tasks.workunit.client.1.smithi138.stdout:5/876: creat d4/da/d6e/d22/d45/f126 x:0 0 0 2024-04-25T01:36:16.371 INFO:tasks.workunit.client.1.smithi138.stdout:5/877: mknod d4/da/d6e/d22/d35/dce/df9/d83/c127 0 2024-04-25T01:36:16.376 INFO:tasks.workunit.client.1.smithi138.stdout:5/878: dwrite d4/d122/d5e/f75 [0,4194304] 0 2024-04-25T01:36:16.382 INFO:tasks.workunit.client.1.smithi138.stdout:5/879: link d4/d122/dca/ce0 d4/da/d6e/d22/d74/c128 0 2024-04-25T01:36:16.395 INFO:tasks.workunit.client.1.smithi138.stdout:5/880: truncate d4/da/d6e/d22/d74/f85 5750310 0 2024-04-25T01:36:16.396 INFO:tasks.workunit.client.1.smithi138.stdout:5/881: getdents d4/da/d6e/d22/d35/dce/df9/dd4 0 2024-04-25T01:36:16.398 INFO:tasks.workunit.client.1.smithi138.stdout:5/882: fdatasync d4/da/d27/f33 0 2024-04-25T01:36:16.398 INFO:tasks.workunit.client.1.smithi138.stdout:5/883: fdatasync d4/da/d6e/d5c/f95 0 2024-04-25T01:36:16.400 INFO:tasks.workunit.client.1.smithi138.stdout:5/884: mkdir d4/da/d6e/d3c/d129 0 2024-04-25T01:36:16.405 INFO:tasks.workunit.client.1.smithi138.stdout:5/885: getdents d4/da/d6e 0 2024-04-25T01:36:16.406 INFO:tasks.workunit.client.1.smithi138.stdout:5/886: mknod d4/da/d6e/d5c/c12a 0 2024-04-25T01:36:16.421 INFO:tasks.workunit.client.1.smithi138.stdout:5/887: dread d4/da/d6e/d22/d35/dce/df9/d83/d8a/fa7 [0,4194304] 0 2024-04-25T01:36:16.422 INFO:tasks.workunit.client.1.smithi138.stdout:5/888: truncate d4/d122/d5e/d60/dbd/dc8/dbb/f10d 13609 0 2024-04-25T01:36:16.427 INFO:tasks.workunit.client.1.smithi138.stdout:5/889: dwrite d4/da/d6e/d22/d74/f58 [0,4194304] 0 2024-04-25T01:36:16.433 INFO:tasks.workunit.client.1.smithi138.stdout:5/890: rename d4/da/d6e/d22/d74/d8e/dd7/da6 to d4/da/d6e/d22/d35/dce/df9/d12b 0 2024-04-25T01:36:16.434 INFO:tasks.workunit.client.1.smithi138.stdout:7/864: sync 2024-04-25T01:36:16.434 INFO:tasks.workunit.client.1.smithi138.stdout:0/690: sync 2024-04-25T01:36:16.437 INFO:tasks.workunit.client.1.smithi138.stdout:7/865: chown da/de/d13/d30/d59/d7b/f10f 53925 1 2024-04-25T01:36:16.437 INFO:tasks.workunit.client.1.smithi138.stdout:5/891: mknod d4/da/d6e/d22/d35/dce/df9/d83/d89/d9d/d9e/c12c 0 2024-04-25T01:36:16.438 INFO:tasks.workunit.client.1.smithi138.stdout:5/892: chown d4/d122/d5e/d8f/fa2 1826619537 1 2024-04-25T01:36:16.439 INFO:tasks.workunit.client.1.smithi138.stdout:0/691: creat d6/d39/fde x:0 0 0 2024-04-25T01:36:16.448 INFO:tasks.workunit.client.1.smithi138.stdout:0/692: write d6/df/f2c [698648,27491] 0 2024-04-25T01:36:16.449 INFO:tasks.workunit.client.1.smithi138.stdout:0/693: fsync d6/d2f/fd3 0 2024-04-25T01:36:16.450 INFO:tasks.workunit.client.1.smithi138.stdout:0/694: write d6/df/d3a/fd7 [991298,100715] 0 2024-04-25T01:36:16.452 INFO:tasks.workunit.client.1.smithi138.stdout:7/866: dread da/de/f2c [0,4194304] 0 2024-04-25T01:36:16.453 INFO:tasks.workunit.client.1.smithi138.stdout:0/695: creat d6/d39/fdf x:0 0 0 2024-04-25T01:36:16.454 INFO:tasks.workunit.client.1.smithi138.stdout:5/893: dread d4/f9 [0,4194304] 0 2024-04-25T01:36:16.455 INFO:tasks.workunit.client.1.smithi138.stdout:0/696: mknod d6/d19/dbb/ce0 0 2024-04-25T01:36:16.457 INFO:tasks.workunit.client.1.smithi138.stdout:5/894: mkdir d4/da/d6e/d22/d35/dce/df9/d7f/db3/d12d 0 2024-04-25T01:36:16.458 INFO:tasks.workunit.client.1.smithi138.stdout:0/697: mknod d6/df/d66/ce1 0 2024-04-25T01:36:16.459 INFO:tasks.workunit.client.1.smithi138.stdout:5/895: creat d4/d122/d5e/d60/db5/f12e x:0 0 0 2024-04-25T01:36:16.461 INFO:tasks.workunit.client.1.smithi138.stdout:0/698: mkdir d6/d2f/de2 0 2024-04-25T01:36:16.467 INFO:tasks.workunit.client.1.smithi138.stdout:5/896: dwrite d4/da/d27/f78 [0,4194304] 0 2024-04-25T01:36:16.469 INFO:tasks.workunit.client.1.smithi138.stdout:7/867: dread da/de/d13/f24 [4194304,4194304] 0 2024-04-25T01:36:16.473 INFO:tasks.workunit.client.1.smithi138.stdout:9/748: sync 2024-04-25T01:36:16.473 INFO:tasks.workunit.client.1.smithi138.stdout:7/868: stat da/de/d13/d21/d6f/d8d/ccc 0 2024-04-25T01:36:16.473 INFO:tasks.workunit.client.1.smithi138.stdout:1/833: sync 2024-04-25T01:36:16.473 INFO:tasks.workunit.client.1.smithi138.stdout:3/931: sync 2024-04-25T01:36:16.473 INFO:tasks.workunit.client.1.smithi138.stdout:6/953: sync 2024-04-25T01:36:16.473 INFO:tasks.workunit.client.1.smithi138.stdout:4/752: sync 2024-04-25T01:36:16.476 INFO:tasks.workunit.client.1.smithi138.stdout:5/897: creat d4/d122/d90/f12f x:0 0 0 2024-04-25T01:36:16.477 INFO:tasks.workunit.client.1.smithi138.stdout:4/753: chown d0/c98 9109213 1 2024-04-25T01:36:16.478 INFO:tasks.workunit.client.1.smithi138.stdout:7/869: unlink da/de/d1b/f33 0 2024-04-25T01:36:16.480 INFO:tasks.workunit.client.1.smithi138.stdout:5/898: dread - d4/d122/d5e/d60/dbd/dc8/f11d zero size 2024-04-25T01:36:16.480 INFO:tasks.workunit.client.1.smithi138.stdout:6/954: creat dc/d10/d28/d70/d83/d10a/d8c/db7/f12a x:0 0 0 2024-04-25T01:36:16.480 INFO:tasks.workunit.client.1.smithi138.stdout:0/699: dwrite d6/df/d66/fc0 [0,4194304] 0 2024-04-25T01:36:16.485 INFO:tasks.workunit.client.1.smithi138.stdout:1/834: rename d3/ddb/d9b/f98 to d3/d27/d24/da9/f11d 0 2024-04-25T01:36:16.485 INFO:tasks.workunit.client.1.smithi138.stdout:3/932: mknod db/dc5/c123 0 2024-04-25T01:36:16.488 INFO:tasks.workunit.client.1.smithi138.stdout:6/955: read dc/d2e/db9/fd8 [1577228,30403] 0 2024-04-25T01:36:16.488 INFO:tasks.workunit.client.1.smithi138.stdout:7/870: creat da/de/d1b/d49/d81/dc7/f120 x:0 0 0 2024-04-25T01:36:16.489 INFO:tasks.workunit.client.1.smithi138.stdout:0/700: creat d6/df/d3a/d67/fe3 x:0 0 0 2024-04-25T01:36:16.492 INFO:tasks.workunit.client.1.smithi138.stdout:4/754: dwrite d0/d1c/d1e/d9f/d7e/dd5/d74/fc5 [0,4194304] 0 2024-04-25T01:36:16.495 INFO:tasks.workunit.client.1.smithi138.stdout:1/835: fdatasync d3/ddb/d9b/f68 0 2024-04-25T01:36:16.498 INFO:tasks.workunit.client.1.smithi138.stdout:0/701: mkdir d6/df/d41/d49/de4 0 2024-04-25T01:36:16.500 INFO:tasks.workunit.client.1.smithi138.stdout:0/702: write d6/df/d69/faa [827230,76601] 0 2024-04-25T01:36:16.500 INFO:tasks.workunit.client.1.smithi138.stdout:7/871: truncate da/de/d13/f3c 260399 0 2024-04-25T01:36:16.501 INFO:tasks.workunit.client.1.smithi138.stdout:7/872: chown da/de/d13/d21/dbc/d94/d103 13 1 2024-04-25T01:36:16.503 INFO:tasks.workunit.client.1.smithi138.stdout:5/899: creat d4/d122/d5e/d8f/d9f/de8/f130 x:0 0 0 2024-04-25T01:36:16.504 INFO:tasks.workunit.client.1.smithi138.stdout:5/900: chown d4/da/lef 14963 1 2024-04-25T01:36:16.505 INFO:tasks.workunit.client.1.smithi138.stdout:0/703: dread d6/d39/d43/f21 [0,4194304] 0 2024-04-25T01:36:16.509 INFO:tasks.workunit.client.1.smithi138.stdout:3/933: creat db/df/d19/f124 x:0 0 0 2024-04-25T01:36:16.509 INFO:tasks.workunit.client.1.smithi138.stdout:5/901: dread d4/da/d6e/d22/d35/dce/df9/d83/f108 [0,4194304] 0 2024-04-25T01:36:16.513 INFO:tasks.workunit.client.1.smithi138.stdout:7/873: dread da/f6a [0,4194304] 0 2024-04-25T01:36:16.513 INFO:tasks.workunit.client.1.smithi138.stdout:0/704: mkdir d6/d2f/d75/db6/de5 0 2024-04-25T01:36:16.514 INFO:tasks.workunit.client.1.smithi138.stdout:4/755: write d0/d1c/d1e/d9f/d7e/dd5/f76 [276712,70354] 0 2024-04-25T01:36:16.517 INFO:tasks.workunit.client.1.smithi138.stdout:4/756: read - d0/d1c/d1e/fd8 zero size 2024-04-25T01:36:16.517 INFO:tasks.workunit.client.1.smithi138.stdout:5/902: dread d4/d122/d5e/faf [0,4194304] 0 2024-04-25T01:36:16.517 INFO:tasks.workunit.client.1.smithi138.stdout:9/749: rename d4/d1a/d33/d9a/ddb to d4/d8/d90/d52/dfa 0 2024-04-25T01:36:16.518 INFO:tasks.workunit.client.1.smithi138.stdout:1/836: write d3/d27/d3e/d51/f8b [1280520,2765] 0 2024-04-25T01:36:16.519 INFO:tasks.workunit.client.1.smithi138.stdout:3/934: mknod db/df/d17/d5d/c125 0 2024-04-25T01:36:16.523 INFO:tasks.workunit.client.1.smithi138.stdout:5/903: dread - d4/da/d6e/d22/d35/dce/df9/d7f/fa4 zero size 2024-04-25T01:36:16.524 INFO:tasks.workunit.client.1.smithi138.stdout:7/874: dwrite da/de/d13/fed [0,4194304] 0 2024-04-25T01:36:16.526 INFO:tasks.workunit.client.1.smithi138.stdout:1/837: truncate d3/d45/d72/d57/d8e/d113/fc5 4779365 0 2024-04-25T01:36:16.527 INFO:tasks.workunit.client.1.smithi138.stdout:6/956: rename dc/d7b/f10c to dc/d10/dc8/d11d/f12b 0 2024-04-25T01:36:16.527 INFO:tasks.workunit.client.1.smithi138.stdout:5/904: write d4/da/d6e/d22/d35/dce/df9/f68 [1127077,57126] 0 2024-04-25T01:36:16.531 INFO:tasks.workunit.client.1.smithi138.stdout:3/935: fsync db/df/de2/d10a/d112/d40/f87 0 2024-04-25T01:36:16.534 INFO:tasks.workunit.client.1.smithi138.stdout:0/705: write d6/df/d41/f64 [927187,8896] 0 2024-04-25T01:36:16.535 INFO:tasks.workunit.client.1.smithi138.stdout:9/750: write d4/d1a/f44 [93431,53674] 0 2024-04-25T01:36:16.536 INFO:tasks.workunit.client.1.smithi138.stdout:4/757: creat d0/d1c/d86/dd9/fe3 x:0 0 0 2024-04-25T01:36:16.537 INFO:tasks.workunit.client.1.smithi138.stdout:6/957: unlink dc/d10/d28/fb1 0 2024-04-25T01:36:16.540 INFO:tasks.workunit.client.1.smithi138.stdout:1/838: rename d3/d27/d3e/d90/f78 to d3/d45/f11e 0 2024-04-25T01:36:16.541 INFO:tasks.workunit.client.1.smithi138.stdout:5/905: mknod d4/da/d6e/d22/d35/dce/df9/d83/d8a/c131 0 2024-04-25T01:36:16.544 INFO:tasks.workunit.client.1.smithi138.stdout:4/758: mkdir d0/d1c/d1e/de4 0 2024-04-25T01:36:16.545 INFO:tasks.workunit.client.1.smithi138.stdout:0/706: dwrite d6/d39/fdf [0,4194304] 0 2024-04-25T01:36:16.545 INFO:tasks.workunit.client.1.smithi138.stdout:6/958: mknod dc/d10/d28/d70/d83/d10a/c12c 0 2024-04-25T01:36:16.551 INFO:tasks.workunit.client.1.smithi138.stdout:4/759: mkdir d0/d1c/d1e/d9f/de5 0 2024-04-25T01:36:16.551 INFO:tasks.workunit.client.1.smithi138.stdout:5/906: dwrite d4/d122/d5e/d60/db5/fdd [0,4194304] 0 2024-04-25T01:36:16.553 INFO:tasks.workunit.client.1.smithi138.stdout:5/907: truncate d4/d122/f66 5134323 0 2024-04-25T01:36:16.555 INFO:tasks.workunit.client.1.smithi138.stdout:7/875: write da/de/d1b/d34/fc2 [605347,127445] 0 2024-04-25T01:36:16.556 INFO:tasks.workunit.client.1.smithi138.stdout:0/707: mknod d6/df/d3a/d67/d8a/ce6 0 2024-04-25T01:36:16.563 INFO:tasks.workunit.client.1.smithi138.stdout:5/908: dwrite d4/d122/d90/f12f [0,4194304] 0 2024-04-25T01:36:16.566 INFO:tasks.workunit.client.1.smithi138.stdout:1/839: rename d3/db/ce0 to d3/d27/d3e/c11f 0 2024-04-25T01:36:16.569 INFO:tasks.workunit.client.1.smithi138.stdout:1/840: truncate d3/d27/d3e/fe2 301835 0 2024-04-25T01:36:16.571 INFO:tasks.workunit.client.1.smithi138.stdout:3/936: write db/f15 [5710558,20895] 0 2024-04-25T01:36:16.573 INFO:tasks.workunit.client.1.smithi138.stdout:6/959: write dc/d10/d28/f11a [1646113,113900] 0 2024-04-25T01:36:16.574 INFO:tasks.workunit.client.1.smithi138.stdout:6/960: fdatasync dc/d2e/d55/f108 0 2024-04-25T01:36:16.576 INFO:tasks.workunit.client.1.smithi138.stdout:1/841: mkdir d3/d45/d72/d57/db0/dcd/d120 0 2024-04-25T01:36:16.579 INFO:tasks.workunit.client.1.smithi138.stdout:7/876: write da/f15 [1581065,8897] 0 2024-04-25T01:36:16.580 INFO:tasks.workunit.client.1.smithi138.stdout:0/708: getdents d6/d1d/d82 0 2024-04-25T01:36:16.584 INFO:tasks.workunit.client.1.smithi138.stdout:4/760: getdents d0/d1c/d1e/d9f 0 2024-04-25T01:36:16.586 INFO:tasks.workunit.client.1.smithi138.stdout:7/877: creat da/de/d13/f121 x:0 0 0 2024-04-25T01:36:16.587 INFO:tasks.workunit.client.1.smithi138.stdout:5/909: creat d4/da/f132 x:0 0 0 2024-04-25T01:36:16.589 INFO:tasks.workunit.client.1.smithi138.stdout:0/709: fsync d6/d1d/f3c 0 2024-04-25T01:36:16.589 INFO:tasks.workunit.client.1.smithi138.stdout:6/961: truncate dc/d2e/db9/f103 741383 0 2024-04-25T01:36:16.589 INFO:tasks.workunit.client.1.smithi138.stdout:7/878: write da/f15 [5162173,70641] 0 2024-04-25T01:36:16.590 INFO:tasks.workunit.client.1.smithi138.stdout:1/842: symlink d3/d45/d72/d57/db0/dcd/d120/l121 0 2024-04-25T01:36:16.592 INFO:tasks.workunit.client.1.smithi138.stdout:0/710: dread - d6/d39/fde zero size 2024-04-25T01:36:16.593 INFO:tasks.workunit.client.1.smithi138.stdout:3/937: dread db/df/d19/f75 [0,4194304] 0 2024-04-25T01:36:16.598 INFO:tasks.workunit.client.1.smithi138.stdout:4/761: mkdir d0/d1c/d1e/dbb/dae/de6 0 2024-04-25T01:36:16.598 INFO:tasks.workunit.client.1.smithi138.stdout:0/711: write d6/d39/d52/d9c/dbc/dbf/fdd [613112,54383] 0 2024-04-25T01:36:16.599 INFO:tasks.workunit.client.1.smithi138.stdout:1/843: mknod d3/d6d/c122 0 2024-04-25T01:36:16.600 INFO:tasks.workunit.client.1.smithi138.stdout:4/762: chown d0/d59/f6a 4069 1 2024-04-25T01:36:16.602 INFO:tasks.workunit.client.1.smithi138.stdout:1/844: readlink d3/d4/d53/lff 0 2024-04-25T01:36:16.603 INFO:tasks.workunit.client.1.smithi138.stdout:6/962: creat dc/d10/d28/d70/d83/d106/f12d x:0 0 0 2024-04-25T01:36:16.604 INFO:tasks.workunit.client.1.smithi138.stdout:0/712: mknod d6/d2f/d31/ce7 0 2024-04-25T01:36:16.604 INFO:tasks.workunit.client.1.smithi138.stdout:3/938: dwrite db/f62 [0,4194304] 0 2024-04-25T01:36:16.608 INFO:tasks.workunit.client.1.smithi138.stdout:4/763: truncate d0/d1c/d1e/d9f/fa7 290713 0 2024-04-25T01:36:16.609 INFO:tasks.workunit.client.1.smithi138.stdout:6/963: chown dc/d10/d28/d80/d89/da0/fbf 41372 1 2024-04-25T01:36:16.609 INFO:tasks.workunit.client.1.smithi138.stdout:1/845: unlink d3/d45/d72/d57/db0/dcd/d120/l121 0 2024-04-25T01:36:16.611 INFO:tasks.workunit.client.1.smithi138.stdout:5/910: dread d4/f6c [0,4194304] 0 2024-04-25T01:36:16.613 INFO:tasks.workunit.client.1.smithi138.stdout:1/846: chown d3/d27/d24/da9 376867 1 2024-04-25T01:36:16.613 INFO:tasks.workunit.client.1.smithi138.stdout:0/713: creat d6/d2f/fe8 x:0 0 0 2024-04-25T01:36:16.616 INFO:tasks.workunit.client.1.smithi138.stdout:7/879: dwrite da/fc [0,4194304] 0 2024-04-25T01:36:16.617 INFO:tasks.workunit.client.1.smithi138.stdout:6/964: dread - dc/d2e/d55/d81/fb8 zero size 2024-04-25T01:36:16.617 INFO:tasks.workunit.client.1.smithi138.stdout:5/911: chown d4/d122/c53 0 1 2024-04-25T01:36:16.617 INFO:tasks.workunit.client.1.smithi138.stdout:0/714: fdatasync d6/d39/d43/fb3 0 2024-04-25T01:36:16.621 INFO:tasks.workunit.client.1.smithi138.stdout:1/847: unlink d3/d45/d72/cef 0 2024-04-25T01:36:16.626 INFO:tasks.workunit.client.1.smithi138.stdout:1/848: fsync d3/d45/d72/d57/d8e/db6/f11a 0 2024-04-25T01:36:16.626 INFO:tasks.workunit.client.1.smithi138.stdout:0/715: chown d6/df/d3a/d67/fe3 851997798 1 2024-04-25T01:36:16.626 INFO:tasks.workunit.client.1.smithi138.stdout:4/764: dwrite d0/d1c/f28 [0,4194304] 0 2024-04-25T01:36:16.631 INFO:tasks.workunit.client.1.smithi138.stdout:6/965: fdatasync dc/d10/d28/d70/d83/d10a/f73 0 2024-04-25T01:36:16.635 INFO:tasks.workunit.client.1.smithi138.stdout:7/880: mknod da/de/d13/d95/dd7/c122 0 2024-04-25T01:36:16.635 INFO:tasks.workunit.client.1.smithi138.stdout:3/939: dread db/df/d19/d1a/d9e/fb9 [0,4194304] 0 2024-04-25T01:36:16.636 INFO:tasks.workunit.client.1.smithi138.stdout:5/912: dwrite d4/d122/d5e/d60/db5/f12e [0,4194304] 0 2024-04-25T01:36:16.641 INFO:tasks.workunit.client.1.smithi138.stdout:4/765: write d0/d1c/d1e/dbb/dae/dc6/d7f/f8a [373204,55364] 0 2024-04-25T01:36:16.644 INFO:tasks.workunit.client.1.smithi138.stdout:0/716: rename d6/d39/l40 to d6/d2f/de2/le9 0 2024-04-25T01:36:16.647 INFO:tasks.workunit.client.1.smithi138.stdout:5/913: read d4/da/d6e/f28 [3917832,18824] 0 2024-04-25T01:36:16.649 INFO:tasks.workunit.client.1.smithi138.stdout:0/717: symlink d6/d2f/d31/ddc/lea 0 2024-04-25T01:36:16.652 INFO:tasks.workunit.client.1.smithi138.stdout:5/914: unlink d4/da/d6e/d3c/dad/fd9 0 2024-04-25T01:36:16.652 INFO:tasks.workunit.client.1.smithi138.stdout:4/766: dwrite d0/d1c/d1e/dbb/dae/dc6/f8e [0,4194304] 0 2024-04-25T01:36:16.660 INFO:tasks.workunit.client.1.smithi138.stdout:5/915: getdents d4/da/d6e/d22/d45/df5 0 2024-04-25T01:36:16.660 INFO:tasks.workunit.client.1.smithi138.stdout:4/767: dread d0/d1c/d1e/d9f/d7e/dd5/f8f [0,4194304] 0 2024-04-25T01:36:16.661 INFO:tasks.workunit.client.1.smithi138.stdout:1/849: write d3/ddb/dd6/f11b [1895596,45082] 0 2024-04-25T01:36:16.662 INFO:tasks.workunit.client.1.smithi138.stdout:7/881: truncate da/f7a 3656295 0 2024-04-25T01:36:16.664 INFO:tasks.workunit.client.1.smithi138.stdout:7/882: dread - da/de/d13/f121 zero size 2024-04-25T01:36:16.664 INFO:tasks.workunit.client.1.smithi138.stdout:1/850: unlink d3/d6d/c122 0 2024-04-25T01:36:16.669 INFO:tasks.workunit.client.1.smithi138.stdout:6/966: dwrite dc/d10/d28/d70/d83/d10a/f74 [0,4194304] 0 2024-04-25T01:36:16.670 INFO:tasks.workunit.client.1.smithi138.stdout:5/916: rename d4/da/d6e/d3c/d129 to d4/da/d6e/d22/d74/d8e/d133 0 2024-04-25T01:36:16.670 INFO:tasks.workunit.client.1.smithi138.stdout:3/940: dread db/df/d19/d1a/f44 [0,4194304] 0 2024-04-25T01:36:16.672 INFO:tasks.workunit.client.1.smithi138.stdout:3/941: chown db/df/de2/d10a/d112/f10d 15 1 2024-04-25T01:36:16.676 INFO:tasks.workunit.client.1.smithi138.stdout:0/718: dwrite d6/d19/dbb/fd6 [0,4194304] 0 2024-04-25T01:36:16.678 INFO:tasks.workunit.client.1.smithi138.stdout:6/967: rename dc/d10/dc8/d11d/f12b to dc/d10/d28/f12e 0 2024-04-25T01:36:16.683 INFO:tasks.workunit.client.1.smithi138.stdout:7/883: creat da/de/d13/d21/da5/de8/f123 x:0 0 0 2024-04-25T01:36:16.683 INFO:tasks.workunit.client.1.smithi138.stdout:6/968: write dc/d10/d28/d70/d83/d106/f111 [2361,1167] 0 2024-04-25T01:36:16.686 INFO:tasks.workunit.client.1.smithi138.stdout:6/969: dread - dc/d2e/d55/d81/fb6 zero size 2024-04-25T01:36:16.686 INFO:tasks.workunit.client.1.smithi138.stdout:4/768: dwrite d0/d1c/d1e/d9f/f34 [0,4194304] 0 2024-04-25T01:36:16.687 INFO:tasks.workunit.client.1.smithi138.stdout:0/719: mkdir d6/d39/d52/db7/deb 0 2024-04-25T01:36:16.691 INFO:tasks.workunit.client.1.smithi138.stdout:3/942: link db/db3/ffe db/df/d17/d107/f126 0 2024-04-25T01:36:16.692 INFO:tasks.workunit.client.1.smithi138.stdout:3/943: write db/df/d19/d1a/ffb [956141,11574] 0 2024-04-25T01:36:16.695 INFO:tasks.workunit.client.1.smithi138.stdout:4/769: mknod d0/d1c/d1e/dbb/da3/db5/ce7 0 2024-04-25T01:36:16.696 INFO:tasks.workunit.client.1.smithi138.stdout:4/770: chown d0/d1c/d1e/l73 2 1 2024-04-25T01:36:16.696 INFO:tasks.workunit.client.1.smithi138.stdout:6/970: mknod dc/d2e/c12f 0 2024-04-25T01:36:16.697 INFO:tasks.workunit.client.1.smithi138.stdout:4/771: fdatasync d0/d1c/d48/f6c 0 2024-04-25T01:36:16.698 INFO:tasks.workunit.client.1.smithi138.stdout:0/720: mkdir d6/df/d41/d49/ddb/dec 0 2024-04-25T01:36:16.701 INFO:tasks.workunit.client.1.smithi138.stdout:3/944: mkdir db/df/dd1/d127 0 2024-04-25T01:36:16.703 INFO:tasks.workunit.client.1.smithi138.stdout:5/917: rename d4/da/d6e/d3c/c86 to d4/d122/d90/c134 0 2024-04-25T01:36:16.704 INFO:tasks.workunit.client.1.smithi138.stdout:0/721: symlink d6/d39/d52/db7/led 0 2024-04-25T01:36:16.705 INFO:tasks.workunit.client.1.smithi138.stdout:3/945: rmdir db/df/d17/d20 39 2024-04-25T01:36:16.706 INFO:tasks.workunit.client.1.smithi138.stdout:6/971: link dc/d10/d67/fe5 dc/d10/dc8/d11d/f130 0 2024-04-25T01:36:16.710 INFO:tasks.workunit.client.1.smithi138.stdout:0/722: dread d6/df/f2c [0,4194304] 0 2024-04-25T01:36:16.711 INFO:tasks.workunit.client.1.smithi138.stdout:3/946: rmdir db 39 2024-04-25T01:36:16.712 INFO:tasks.workunit.client.1.smithi138.stdout:9/751: sync 2024-04-25T01:36:16.713 INFO:tasks.workunit.client.1.smithi138.stdout:1/851: dread d3/db/f67 [0,4194304] 0 2024-04-25T01:36:16.714 INFO:tasks.workunit.client.1.smithi138.stdout:5/918: mkdir d4/da/d6e/d22/d109/d135 0 2024-04-25T01:36:16.715 INFO:tasks.workunit.client.1.smithi138.stdout:4/772: getdents d0/d1c/d86/dd9 0 2024-04-25T01:36:16.717 INFO:tasks.workunit.client.1.smithi138.stdout:1/852: fdatasync d3/d45/f106 0 2024-04-25T01:36:16.722 INFO:tasks.workunit.client.1.smithi138.stdout:6/972: dwrite dc/d10/d67/d8e/daf/fef [0,4194304] 0 2024-04-25T01:36:16.722 INFO:tasks.workunit.client.1.smithi138.stdout:6/973: readlink dc/d2e/d55/l120 0 2024-04-25T01:36:16.722 INFO:tasks.workunit.client.1.smithi138.stdout:7/884: dread da/de/d1b/d49/d81/d51/d6e/f75 [0,4194304] 0 2024-04-25T01:36:16.724 INFO:tasks.workunit.client.1.smithi138.stdout:4/773: unlink d0/d1c/d48/fba 0 2024-04-25T01:36:16.726 INFO:tasks.workunit.client.1.smithi138.stdout:9/752: read d4/d1a/f56 [3620018,40872] 0 2024-04-25T01:36:16.727 INFO:tasks.workunit.client.1.smithi138.stdout:3/947: symlink db/df/d17/d23/d34/l128 0 2024-04-25T01:36:16.728 INFO:tasks.workunit.client.1.smithi138.stdout:0/723: dwrite d6/d39/d43/f6b [4194304,4194304] 0 2024-04-25T01:36:16.730 INFO:tasks.workunit.client.1.smithi138.stdout:3/948: stat db/d9b/fc2 0 2024-04-25T01:36:16.735 INFO:tasks.workunit.client.1.smithi138.stdout:7/885: creat da/de/d1b/d49/d81/dc7/f124 x:0 0 0 2024-04-25T01:36:16.735 INFO:tasks.workunit.client.1.smithi138.stdout:9/753: chown d4/d1a/d68/fa7 56394451 1 2024-04-25T01:36:16.735 INFO:tasks.workunit.client.1.smithi138.stdout:1/853: dread d3/d4/d53/f8c [0,4194304] 0 2024-04-25T01:36:16.736 INFO:tasks.workunit.client.1.smithi138.stdout:4/774: symlink d0/d1c/d1e/d9f/d7e/dd5/d74/le8 0 2024-04-25T01:36:16.737 INFO:tasks.workunit.client.1.smithi138.stdout:0/724: write d6/d39/d43/f6b [6375016,110824] 0 2024-04-25T01:36:16.740 INFO:tasks.workunit.client.1.smithi138.stdout:1/854: truncate d3/d27/d16/f18 4589246 0 2024-04-25T01:36:16.741 INFO:tasks.workunit.client.1.smithi138.stdout:6/974: truncate dc/d10/f22 844907 0 2024-04-25T01:36:16.744 INFO:tasks.workunit.client.1.smithi138.stdout:9/754: creat d4/d63/d86/ffb x:0 0 0 2024-04-25T01:36:16.745 INFO:tasks.workunit.client.1.smithi138.stdout:9/755: dread - d4/d8/d2c/ff9 zero size 2024-04-25T01:36:16.746 INFO:tasks.workunit.client.1.smithi138.stdout:0/725: dread d6/df/d41/fa8 [0,4194304] 0 2024-04-25T01:36:16.747 INFO:tasks.workunit.client.1.smithi138.stdout:1/855: fdatasync d3/d45/d72/d57/f63 0 2024-04-25T01:36:16.748 INFO:tasks.workunit.client.1.smithi138.stdout:0/726: write d6/df/d3a/d67/fe3 [670286,105641] 0 2024-04-25T01:36:16.749 INFO:tasks.workunit.client.1.smithi138.stdout:5/919: dread d4/da/d6e/d22/d35/dce/df9/d83/d8a/fc0 [0,4194304] 0 2024-04-25T01:36:16.750 INFO:tasks.workunit.client.1.smithi138.stdout:0/727: write d6/d1d/f24 [1830308,4588] 0 2024-04-25T01:36:16.751 INFO:tasks.workunit.client.1.smithi138.stdout:6/975: creat dc/d10/d28/d70/d83/d10a/f131 x:0 0 0 2024-04-25T01:36:16.752 INFO:tasks.workunit.client.1.smithi138.stdout:6/976: readlink dc/d8d/dca/ld1 0 2024-04-25T01:36:16.752 INFO:tasks.workunit.client.1.smithi138.stdout:3/949: mknod db/df/d17/d20/d57/d6a/c129 0 2024-04-25T01:36:16.752 INFO:tasks.workunit.client.1.smithi138.stdout:3/950: stat db/fc7 0 2024-04-25T01:36:16.754 INFO:tasks.workunit.client.1.smithi138.stdout:7/886: link da/de/d1b/fef da/de/d13/d95/f125 0 2024-04-25T01:36:16.756 INFO:tasks.workunit.client.1.smithi138.stdout:9/756: creat d4/d49/d72/ffc x:0 0 0 2024-04-25T01:36:16.759 INFO:tasks.workunit.client.1.smithi138.stdout:3/951: dwrite db/df/d17/d23/f30 [0,4194304] 0 2024-04-25T01:36:16.765 INFO:tasks.workunit.client.1.smithi138.stdout:4/775: creat d0/d1c/d1e/dbb/dae/dc6/fe9 x:0 0 0 2024-04-25T01:36:16.766 INFO:tasks.workunit.client.1.smithi138.stdout:1/856: dread d3/d27/d3e/d51/f75 [0,4194304] 0 2024-04-25T01:36:16.771 INFO:tasks.workunit.client.1.smithi138.stdout:0/728: mknod d6/d39/cee 0 2024-04-25T01:36:16.771 INFO:tasks.workunit.client.1.smithi138.stdout:5/920: write d4/d122/d5e/f7a [2309497,27646] 0 2024-04-25T01:36:16.773 INFO:tasks.workunit.client.1.smithi138.stdout:6/977: link dc/d2e/f31 dc/d2e/dc1/f132 0 2024-04-25T01:36:16.774 INFO:tasks.workunit.client.1.smithi138.stdout:3/952: read db/df/d17/d20/d57/d6a/f8d [1003038,31168] 0 2024-04-25T01:36:16.775 INFO:tasks.workunit.client.1.smithi138.stdout:4/776: symlink d0/d1c/d86/dd9/lea 0 2024-04-25T01:36:16.776 INFO:tasks.workunit.client.1.smithi138.stdout:1/857: mkdir d3/d27/d3e/df4/d123 0 2024-04-25T01:36:16.778 INFO:tasks.workunit.client.1.smithi138.stdout:0/729: truncate d6/d1d/f3c 678451 0 2024-04-25T01:36:16.781 INFO:tasks.workunit.client.1.smithi138.stdout:6/978: dwrite dc/d10/d6d/f5e [0,4194304] 0 2024-04-25T01:36:16.782 INFO:tasks.workunit.client.1.smithi138.stdout:1/858: mkdir d3/d4/da7/db1/d124 0 2024-04-25T01:36:16.785 INFO:tasks.workunit.client.1.smithi138.stdout:4/777: creat d0/d1c/d48/ddf/feb x:0 0 0 2024-04-25T01:36:16.788 INFO:tasks.workunit.client.1.smithi138.stdout:1/859: mknod d3/d45/d72/d57/d8e/d113/c125 0 2024-04-25T01:36:16.790 INFO:tasks.workunit.client.1.smithi138.stdout:9/757: dwrite d4/d8/f59 [0,4194304] 0 2024-04-25T01:36:16.795 INFO:tasks.workunit.client.1.smithi138.stdout:7/887: dread da/de/d1b/d34/f3d [0,4194304] 0 2024-04-25T01:36:16.797 INFO:tasks.workunit.client.1.smithi138.stdout:1/860: readlink d3/d27/d24/l4d 0 2024-04-25T01:36:16.802 INFO:tasks.workunit.client.1.smithi138.stdout:7/888: creat da/de/d1b/d49/d81/d51/d6e/f126 x:0 0 0 2024-04-25T01:36:16.803 INFO:tasks.workunit.client.1.smithi138.stdout:0/730: write d6/df/d41/f55 [4777450,17063] 0 2024-04-25T01:36:16.805 INFO:tasks.workunit.client.1.smithi138.stdout:1/861: rename d3/d27/d3e/d51 to d3/d45/d72/db2/dd9/df9/d126 0 2024-04-25T01:36:16.805 INFO:tasks.workunit.client.1.smithi138.stdout:0/731: chown d6/d1d/d82/f94 159 1 2024-04-25T01:36:16.806 INFO:tasks.workunit.client.1.smithi138.stdout:3/953: dwrite db/df/d17/d20/d57/daa/fd7 [0,4194304] 0 2024-04-25T01:36:16.807 INFO:tasks.workunit.client.1.smithi138.stdout:0/732: fsync d6/df/d69/f6a 0 2024-04-25T01:36:16.809 INFO:tasks.workunit.client.1.smithi138.stdout:9/758: link d4/d8/d90/f87 d4/d8/d90/d8e/dd9/ffd 0 2024-04-25T01:36:16.813 INFO:tasks.workunit.client.1.smithi138.stdout:6/979: getdents dc/d2e/d37 0 2024-04-25T01:36:16.814 INFO:tasks.workunit.client.1.smithi138.stdout:1/862: readlink d3/d4/da7/lb4 0 2024-04-25T01:36:16.816 INFO:tasks.workunit.client.1.smithi138.stdout:6/980: chown dc/d10/d28/d70/d83/f9d 587 1 2024-04-25T01:36:16.817 INFO:tasks.workunit.client.1.smithi138.stdout:3/954: creat db/df/d17/d20/d57/f12a x:0 0 0 2024-04-25T01:36:16.818 INFO:tasks.workunit.client.1.smithi138.stdout:4/778: dwrite d0/d1c/d1e/d9f/d7e/dd5/d74/f7a [0,4194304] 0 2024-04-25T01:36:16.819 INFO:tasks.workunit.client.1.smithi138.stdout:0/733: creat d6/d39/d52/d9c/dbc/dbf/fef x:0 0 0 2024-04-25T01:36:16.825 INFO:tasks.workunit.client.1.smithi138.stdout:0/734: dread d6/d2f/f7e [0,4194304] 0 2024-04-25T01:36:16.828 INFO:tasks.workunit.client.1.smithi138.stdout:1/863: rename d3/d45/fa0 to d3/d45/d72/d57/d8e/d113/daa/f127 0 2024-04-25T01:36:16.830 INFO:tasks.workunit.client.1.smithi138.stdout:3/955: creat db/df/d19/d1a/f12b x:0 0 0 2024-04-25T01:36:16.833 INFO:tasks.workunit.client.1.smithi138.stdout:6/981: symlink dc/d2e/l133 0 2024-04-25T01:36:16.835 INFO:tasks.workunit.client.1.smithi138.stdout:0/735: symlink d6/d39/d52/d9c/dbc/dbf/lf0 0 2024-04-25T01:36:16.837 INFO:tasks.workunit.client.1.smithi138.stdout:6/982: read dc/f19 [3433462,38217] 0 2024-04-25T01:36:16.838 INFO:tasks.workunit.client.1.smithi138.stdout:7/889: dwrite da/de/d13/f76 [0,4194304] 0 2024-04-25T01:36:16.839 INFO:tasks.workunit.client.1.smithi138.stdout:3/956: fdatasync db/df/d17/d23/d34/f6d 0 2024-04-25T01:36:16.839 INFO:tasks.workunit.client.1.smithi138.stdout:9/759: creat d4/d1a/d55/d7f/ffe x:0 0 0 2024-04-25T01:36:16.842 INFO:tasks.workunit.client.1.smithi138.stdout:4/779: dwrite d0/d1c/d1e/f85 [0,4194304] 0 2024-04-25T01:36:16.842 INFO:tasks.workunit.client.1.smithi138.stdout:7/890: write da/fc [5364308,69803] 0 2024-04-25T01:36:16.847 INFO:tasks.workunit.client.1.smithi138.stdout:1/864: fdatasync d3/f34 0 2024-04-25T01:36:16.849 INFO:tasks.workunit.client.1.smithi138.stdout:1/865: readlink d3/d45/d72/d57/db0/lce 0 2024-04-25T01:36:16.855 INFO:tasks.workunit.client.1.smithi138.stdout:6/983: dwrite dc/d2e/d55/d7e/f122 [0,4194304] 0 2024-04-25T01:36:16.855 INFO:tasks.workunit.client.1.smithi138.stdout:3/957: dwrite db/df/fbd [0,4194304] 0 2024-04-25T01:36:16.857 INFO:tasks.workunit.client.1.smithi138.stdout:0/736: mkdir d6/df/d41/df1 0 2024-04-25T01:36:16.858 INFO:tasks.workunit.client.1.smithi138.stdout:6/984: read dc/d10/f4b [2778353,23618] 0 2024-04-25T01:36:16.866 INFO:tasks.workunit.client.1.smithi138.stdout:7/891: mkdir da/de/d13/d30/d59/d7b/d99/d127 0 2024-04-25T01:36:16.867 INFO:tasks.workunit.client.1.smithi138.stdout:6/985: dwrite dc/d10/d28/d70/f7f [0,4194304] 0 2024-04-25T01:36:16.869 INFO:tasks.workunit.client.1.smithi138.stdout:6/986: chown dc/d10/d28/d70/d83/d106/f111 28365 1 2024-04-25T01:36:16.869 INFO:tasks.workunit.client.1.smithi138.stdout:0/737: rmdir d6/d2f/de2 39 2024-04-25T01:36:16.871 INFO:tasks.workunit.client.1.smithi138.stdout:1/866: mkdir d3/d27/d3e/df4/d123/d128 0 2024-04-25T01:36:16.874 INFO:tasks.workunit.client.1.smithi138.stdout:3/958: creat db/df/d102/f12c x:0 0 0 2024-04-25T01:36:16.875 INFO:tasks.workunit.client.1.smithi138.stdout:7/892: mkdir da/de/d1b/d4b/d5b/d128 0 2024-04-25T01:36:16.876 INFO:tasks.workunit.client.1.smithi138.stdout:0/738: mkdir d6/d1d/d82/df2 0 2024-04-25T01:36:16.879 INFO:tasks.workunit.client.1.smithi138.stdout:0/739: chown d6/d1d/d98/fd9 16 1 2024-04-25T01:36:16.881 INFO:tasks.workunit.client.1.smithi138.stdout:7/893: unlink da/de/d13/d30/d59/d7b/d99/lf5 0 2024-04-25T01:36:16.883 INFO:tasks.workunit.client.1.smithi138.stdout:6/987: dwrite dc/d10/d28/d70/d83/d10a/d8c/dff/d114/f11f [0,4194304] 0 2024-04-25T01:36:16.884 INFO:tasks.workunit.client.1.smithi138.stdout:0/740: creat d6/d1d/d98/ff3 x:0 0 0 2024-04-25T01:36:16.885 INFO:tasks.workunit.client.1.smithi138.stdout:3/959: dread db/df/d19/d1a/ffb [0,4194304] 0 2024-04-25T01:36:16.887 INFO:tasks.workunit.client.1.smithi138.stdout:7/894: truncate da/de/d13/d21/dbc/d94/d103/f10e 357361 0 2024-04-25T01:36:16.887 INFO:tasks.workunit.client.1.smithi138.stdout:9/760: write d4/f13 [392291,22982] 0 2024-04-25T01:36:16.888 INFO:tasks.workunit.client.1.smithi138.stdout:0/741: creat d6/d2f/d75/db6/ff4 x:0 0 0 2024-04-25T01:36:16.890 INFO:tasks.workunit.client.1.smithi138.stdout:0/742: readlink d6/df/l61 0 2024-04-25T01:36:16.891 INFO:tasks.workunit.client.1.smithi138.stdout:3/960: rmdir db/df/d19/d1a/de5/de7 39 2024-04-25T01:36:16.896 INFO:tasks.workunit.client.1.smithi138.stdout:9/761: unlink d4/d63/d86/fbb 0 2024-04-25T01:36:16.896 INFO:tasks.workunit.client.1.smithi138.stdout:4/780: dwrite d0/d1c/d1e/d9f/d33/f89 [0,4194304] 0 2024-04-25T01:36:16.897 INFO:tasks.workunit.client.1.smithi138.stdout:0/743: mkdir d6/d1d/df5 0 2024-04-25T01:36:16.906 INFO:tasks.workunit.client.1.smithi138.stdout:9/762: mkdir d4/d1a/d68/dbe/dff 0 2024-04-25T01:36:16.907 INFO:tasks.workunit.client.1.smithi138.stdout:1/867: dwrite d3/d45/d72/d57/d8e/d113/daa/fd8 [0,4194304] 0 2024-04-25T01:36:16.907 INFO:tasks.workunit.client.1.smithi138.stdout:0/744: creat d6/d2f/d31/ff6 x:0 0 0 2024-04-25T01:36:16.909 INFO:tasks.workunit.client.1.smithi138.stdout:7/895: creat da/de/d1b/d4b/f129 x:0 0 0 2024-04-25T01:36:16.912 INFO:tasks.workunit.client.1.smithi138.stdout:9/763: mknod d4/d1a/d3c/dc2/c100 0 2024-04-25T01:36:16.916 INFO:tasks.workunit.client.1.smithi138.stdout:9/764: dread - d4/d63/d86/fe5 zero size 2024-04-25T01:36:16.919 INFO:tasks.workunit.client.1.smithi138.stdout:6/988: dwrite dc/d10/f76 [4194304,4194304] 0 2024-04-25T01:36:16.919 INFO:tasks.workunit.client.1.smithi138.stdout:0/745: getdents d6/d39/d52/d9c/dbc 0 2024-04-25T01:36:16.925 INFO:tasks.workunit.client.1.smithi138.stdout:3/961: dwrite db/df/de2/d10a/d112/d40/fb0 [0,4194304] 0 2024-04-25T01:36:16.925 INFO:tasks.workunit.client.1.smithi138.stdout:9/765: creat d4/d1a/d55/d7f/d8f/f101 x:0 0 0 2024-04-25T01:36:16.926 INFO:tasks.workunit.client.1.smithi138.stdout:6/989: stat dc/d10/d28/d70/d83/d10a/d8c/dff/d114/f11f 0 2024-04-25T01:36:16.929 INFO:tasks.workunit.client.1.smithi138.stdout:3/962: dread - db/df/d19/d1a/de5/f106 zero size 2024-04-25T01:36:16.929 INFO:tasks.workunit.client.1.smithi138.stdout:9/766: read d4/d8/f1f [1802665,115178] 0 2024-04-25T01:36:16.932 INFO:tasks.workunit.client.1.smithi138.stdout:9/767: stat d4/d49/c7e 0 2024-04-25T01:36:16.934 INFO:tasks.workunit.client.1.smithi138.stdout:1/868: dwrite d3/db/f5f [8388608,4194304] 0 2024-04-25T01:36:16.937 INFO:tasks.workunit.client.1.smithi138.stdout:6/990: rmdir dc/d10/d6d/d124 0 2024-04-25T01:36:16.940 INFO:tasks.workunit.client.1.smithi138.stdout:7/896: dwrite da/de/f2c [0,4194304] 0 2024-04-25T01:36:16.943 INFO:tasks.workunit.client.1.smithi138.stdout:9/768: read - d4/d8/d90/d52/fc0 zero size 2024-04-25T01:36:16.945 INFO:tasks.workunit.client.1.smithi138.stdout:1/869: truncate d3/d6d/f92 3088445 0 2024-04-25T01:36:16.945 INFO:tasks.workunit.client.1.smithi138.stdout:0/746: dread d6/d19/f85 [0,4194304] 0 2024-04-25T01:36:16.947 INFO:tasks.workunit.client.1.smithi138.stdout:7/897: mknod da/de/d13/d21/da5/de8/c12a 0 2024-04-25T01:36:16.948 INFO:tasks.workunit.client.1.smithi138.stdout:6/991: dread dc/d10/d67/d8e/fad [0,4194304] 0 2024-04-25T01:36:16.949 INFO:tasks.workunit.client.1.smithi138.stdout:6/992: chown dc/d7b 15910 1 2024-04-25T01:36:16.951 INFO:tasks.workunit.client.1.smithi138.stdout:7/898: mkdir da/de/d13/d21/dbc/dca/d12b 0 2024-04-25T01:36:16.951 INFO:tasks.workunit.client.1.smithi138.stdout:0/747: creat d6/d2f/d75/db6/de5/ff7 x:0 0 0 2024-04-25T01:36:16.953 INFO:tasks.workunit.client.1.smithi138.stdout:6/993: rmdir dc/d2e/d37 39 2024-04-25T01:36:16.955 INFO:tasks.workunit.client.1.smithi138.stdout:6/994: mkdir dc/d2e/d134 0 2024-04-25T01:36:16.962 INFO:tasks.workunit.client.1.smithi138.stdout:6/995: rename dc/d10/dc8/ffa to dc/d10/d28/d70/d83/d10a/d8c/dff/d114/f135 0 2024-04-25T01:36:16.962 INFO:tasks.workunit.client.1.smithi138.stdout:9/769: write d4/f17 [2295113,36040] 0 2024-04-25T01:36:16.966 INFO:tasks.workunit.client.1.smithi138.stdout:6/996: creat dc/d10/d28/dec/f136 x:0 0 0 2024-04-25T01:36:16.971 INFO:tasks.workunit.client.1.smithi138.stdout:0/748: truncate d6/f9 2859826 0 2024-04-25T01:36:16.971 INFO:tasks.workunit.client.1.smithi138.stdout:1/870: dwrite d3/d4/f77 [0,4194304] 0 2024-04-25T01:36:16.974 INFO:tasks.workunit.client.1.smithi138.stdout:1/871: symlink d3/d4/da7/db1/de1/l129 0 2024-04-25T01:36:16.978 INFO:tasks.workunit.client.1.smithi138.stdout:0/749: link d6/f3b d6/d1d/d82/df2/ff8 0 2024-04-25T01:36:16.979 INFO:tasks.workunit.client.1.smithi138.stdout:7/899: dread da/de/d1b/d34/fc2 [0,4194304] 0 2024-04-25T01:36:16.979 INFO:tasks.workunit.client.1.smithi138.stdout:1/872: creat d3/d45/d72/d57/d8e/f12a x:0 0 0 2024-04-25T01:36:16.983 INFO:tasks.workunit.client.1.smithi138.stdout:0/750: write d6/d2f/d31/ff6 [93080,41956] 0 2024-04-25T01:36:16.983 INFO:tasks.workunit.client.1.smithi138.stdout:9/770: dwrite d4/d1a/f21 [0,4194304] 0 2024-04-25T01:36:16.983 INFO:tasks.workunit.client.1.smithi138.stdout:1/873: write d3/d27/d3e/fbf [5481986,35816] 0 2024-04-25T01:36:16.987 INFO:tasks.workunit.client.1.smithi138.stdout:6/997: dread dc/d2e/d55/d81/fb4 [0,4194304] 0 2024-04-25T01:36:16.989 INFO:tasks.workunit.client.1.smithi138.stdout:7/900: creat da/de/d13/d21/dbc/d94/d9d/f12c x:0 0 0 2024-04-25T01:36:16.991 INFO:tasks.workunit.client.1.smithi138.stdout:0/751: rename d6/f45 to d6/d39/d52/ff9 0 2024-04-25T01:36:16.992 INFO:tasks.workunit.client.1.smithi138.stdout:0/752: read d6/f3b [646325,51445] 0 2024-04-25T01:36:16.995 INFO:tasks.workunit.client.1.smithi138.stdout:7/901: rmdir da/dd2 39 2024-04-25T01:36:16.996 INFO:tasks.workunit.client.1.smithi138.stdout:7/902: stat da/de/d1b/d49/f119 0 2024-04-25T01:36:16.997 INFO:tasks.workunit.client.1.smithi138.stdout:7/903: fdatasync da/de/d13/d21/dbc/dca/fe9 0 2024-04-25T01:36:16.997 INFO:tasks.workunit.client.1.smithi138.stdout:6/998: fsync dc/d2e/db9/f103 0 2024-04-25T01:36:16.998 INFO:tasks.workunit.client.1.smithi138.stdout:0/753: dread d6/d39/fa2 [0,4194304] 0 2024-04-25T01:36:17.000 INFO:tasks.workunit.client.1.smithi138.stdout:7/904: chown da/de/d13/d21/d6f/d8d/cf2 78871883 1 2024-04-25T01:36:17.001 INFO:tasks.workunit.client.1.smithi138.stdout:6/999: creat dc/d10/d28/d70/d83/f137 x:0 0 0 2024-04-25T01:36:17.002 INFO:tasks.workunit.client.1.smithi138.stdout:0/754: write d6/df/d3a/f3d [8239592,106031] 0 2024-04-25T01:36:17.003 INFO:tasks.workunit.client.1.smithi138.stdout:7/905: mknod da/de/d1b/d49/d81/dc7/c12d 0 2024-04-25T01:36:17.003 INFO:tasks.workunit.client.1.smithi138.stdout:0/755: chown d6/df/d3a/d67/c8c 2755 1 2024-04-25T01:36:17.010 INFO:tasks.workunit.client.1.smithi138.stdout:0/756: getdents d6/d1d/d82/df2 0 2024-04-25T01:36:17.011 INFO:tasks.workunit.client.1.smithi138.stdout:7/906: link da/de/d13/d21/d6f/d8d/cf2 da/de/d13/d21/da5/de8/c12e 0 2024-04-25T01:36:17.015 INFO:tasks.workunit.client.1.smithi138.stdout:1/874: dwrite d3/d4/da7/f114 [0,4194304] 0 2024-04-25T01:36:17.015 INFO:tasks.workunit.client.1.smithi138.stdout:7/907: mknod da/de/d13/d21/d6f/c12f 0 2024-04-25T01:36:17.018 INFO:tasks.workunit.client.1.smithi138.stdout:0/757: rename d6/df/faf to d6/d39/d58/ffa 0 2024-04-25T01:36:17.018 INFO:tasks.workunit.client.1.smithi138.stdout:9/771: dread d4/d1a/f74 [0,4194304] 0 2024-04-25T01:36:17.021 INFO:tasks.workunit.client.1.smithi138.stdout:1/875: read d3/d45/d72/d57/d8e/fbe [4688294,58291] 0 2024-04-25T01:36:17.024 INFO:tasks.workunit.client.1.smithi138.stdout:7/908: rename da/de/d13/d21/dbc/fb3 to da/de/d1b/d34/f130 0 2024-04-25T01:36:17.024 INFO:tasks.workunit.client.1.smithi138.stdout:9/772: truncate d4/d1a/f56 33624 0 2024-04-25T01:36:17.026 INFO:tasks.workunit.client.1.smithi138.stdout:0/758: mknod d6/df/d41/cfb 0 2024-04-25T01:36:17.026 INFO:tasks.workunit.client.1.smithi138.stdout:0/759: fdatasync d6/d19/dbb/fc5 0 2024-04-25T01:36:17.028 INFO:tasks.workunit.client.1.smithi138.stdout:1/876: rename d3/d27/d16/f18 to d3/d27/d16/da3/daf/ded/f12b 0 2024-04-25T01:36:17.029 INFO:tasks.workunit.client.1.smithi138.stdout:9/773: mknod d4/d1a/d55/d7f/d8f/c102 0 2024-04-25T01:36:17.032 INFO:tasks.workunit.client.1.smithi138.stdout:1/877: mknod d3/d45/d72/d57/db0/c12c 0 2024-04-25T01:36:17.034 INFO:tasks.workunit.client.1.smithi138.stdout:1/878: mkdir d3/ddb/dd6/d12d 0 2024-04-25T01:36:17.036 INFO:tasks.workunit.client.1.smithi138.stdout:1/879: dread - d3/d45/d72/d57/d8e/db6/f11a zero size 2024-04-25T01:36:17.038 INFO:tasks.workunit.client.1.smithi138.stdout:1/880: mkdir d3/d45/d72/d57/d12e 0 2024-04-25T01:36:17.041 INFO:tasks.workunit.client.1.smithi138.stdout:1/881: symlink d3/d27/d3e/df4/d123/l12f 0 2024-04-25T01:36:17.043 INFO:tasks.workunit.client.1.smithi138.stdout:7/909: dread da/de/d1b/d4b/d77/f91 [0,4194304] 0 2024-04-25T01:36:17.044 INFO:tasks.workunit.client.1.smithi138.stdout:1/882: unlink d3/d45/d72/d57/db0/lce 0 2024-04-25T01:36:17.045 INFO:tasks.workunit.client.1.smithi138.stdout:0/760: write d6/d19/dbb/fd6 [3075493,121231] 0 2024-04-25T01:36:17.046 INFO:tasks.workunit.client.1.smithi138.stdout:1/883: write d3/d45/d72/d57/db0/f112 [1018431,85598] 0 2024-04-25T01:36:17.048 INFO:tasks.workunit.client.1.smithi138.stdout:7/910: unlink da/de/d13/d21/dbc/fbe 0 2024-04-25T01:36:17.050 INFO:tasks.workunit.client.1.smithi138.stdout:9/774: dwrite d4/d49/f6f [0,4194304] 0 2024-04-25T01:36:17.051 INFO:tasks.workunit.client.1.smithi138.stdout:0/761: mknod d6/df/d41/d49/ddb/cfc 0 2024-04-25T01:36:17.053 INFO:tasks.workunit.client.1.smithi138.stdout:7/911: creat da/de/d13/d21/d6f/d8d/f131 x:0 0 0 2024-04-25T01:36:17.055 INFO:tasks.workunit.client.1.smithi138.stdout:0/762: dread d6/d2f/f7e [0,4194304] 0 2024-04-25T01:36:17.057 INFO:tasks.workunit.client.1.smithi138.stdout:9/775: symlink d4/d8/d90/d52/l103 0 2024-04-25T01:36:17.057 INFO:tasks.workunit.client.1.smithi138.stdout:7/912: truncate da/dd2/f10a 898037 0 2024-04-25T01:36:17.058 INFO:tasks.workunit.client.1.smithi138.stdout:7/913: dread - da/de/d13/d95/f125 zero size 2024-04-25T01:36:17.063 INFO:tasks.workunit.client.1.smithi138.stdout:0/763: dwrite d6/d1d/d82/fad [0,4194304] 0 2024-04-25T01:36:17.066 INFO:tasks.workunit.client.1.smithi138.stdout:7/914: creat da/de/d13/d21/dbc/d94/f132 x:0 0 0 2024-04-25T01:36:17.071 INFO:tasks.workunit.client.1.smithi138.stdout:7/915: read da/de/d1b/d4b/d5b/f43 [1550848,67276] 0 2024-04-25T01:36:17.072 INFO:tasks.workunit.client.1.smithi138.stdout:7/916: chown da/de/d1b/d49/d81/d51/d6e/da2 3622310 1 2024-04-25T01:36:17.074 INFO:tasks.workunit.client.1.smithi138.stdout:0/764: link d6/l2e d6/df/d3a/lfd 0 2024-04-25T01:36:17.078 INFO:tasks.workunit.client.1.smithi138.stdout:0/765: getdents d6/d39/d58 0 2024-04-25T01:36:17.078 INFO:tasks.workunit.client.1.smithi138.stdout:0/766: fdatasync d6/d2f/d31/f86 0 2024-04-25T01:36:17.079 INFO:tasks.workunit.client.1.smithi138.stdout:0/767: write d6/d39/d43/fb3 [1163433,83439] 0 2024-04-25T01:36:17.085 INFO:tasks.workunit.client.1.smithi138.stdout:7/917: write da/de/d1b/d4b/d77/f91 [1990669,38501] 0 2024-04-25T01:36:17.088 INFO:tasks.workunit.client.1.smithi138.stdout:9/776: dwrite d4/d8/d2c/d82/de8/ff0 [0,4194304] 0 2024-04-25T01:36:17.089 INFO:tasks.workunit.client.1.smithi138.stdout:9/777: fsync d4/d1a/d55/d7f/d8f/f101 0 2024-04-25T01:36:17.095 INFO:tasks.workunit.client.1.smithi138.stdout:9/778: rename d4/d8/d2c/d82/ff4 to d4/d1a/f104 0 2024-04-25T01:36:17.102 INFO:tasks.workunit.client.1.smithi138.stdout:9/779: dwrite d4/d1a/f104 [0,4194304] 0 2024-04-25T01:36:17.102 INFO:tasks.workunit.client.1.smithi138.stdout:9/780: chown d4/d8 12608449 1 2024-04-25T01:36:17.106 INFO:tasks.workunit.client.1.smithi138.stdout:7/918: dread da/de/d1b/f31 [4194304,4194304] 0 2024-04-25T01:36:17.107 INFO:tasks.workunit.client.1.smithi138.stdout:9/781: rmdir d4/d1a/d68/dbe 39 2024-04-25T01:36:17.111 INFO:tasks.workunit.client.1.smithi138.stdout:7/919: creat da/f133 x:0 0 0 2024-04-25T01:36:17.111 INFO:tasks.workunit.client.1.smithi138.stdout:9/782: link d4/d1a/d33/c48 d4/d8/d90/d8e/c105 0 2024-04-25T01:36:17.112 INFO:tasks.workunit.client.1.smithi138.stdout:9/783: chown d4/d49/d72/cca 61563149 1 2024-04-25T01:36:17.114 INFO:tasks.workunit.client.1.smithi138.stdout:9/784: creat d4/d49/f106 x:0 0 0 2024-04-25T01:36:17.115 INFO:tasks.workunit.client.1.smithi138.stdout:7/920: creat da/de/d1b/d49/d81/d73/db7/f134 x:0 0 0 2024-04-25T01:36:17.116 INFO:tasks.workunit.client.1.smithi138.stdout:9/785: creat d4/d8/d2c/d82/f107 x:0 0 0 2024-04-25T01:36:17.118 INFO:tasks.workunit.client.1.smithi138.stdout:7/921: rename da/de/d13/d30/d59/d7b/d99/f104 to da/de/d1b/d34/f135 0 2024-04-25T01:36:17.120 INFO:tasks.workunit.client.1.smithi138.stdout:9/786: getdents d4/d8/d90 0 2024-04-25T01:36:17.125 INFO:tasks.workunit.client.1.smithi138.stdout:7/922: rename da/de/d13/d21/da5/de8 to da/de/d13/d21/dbc/d94/d136 0 2024-04-25T01:36:17.126 INFO:tasks.workunit.client.1.smithi138.stdout:7/923: stat da/de/d13/d21/d6f/d8d/ccc 0 2024-04-25T01:36:17.126 INFO:tasks.workunit.client.1.smithi138.stdout:7/924: fdatasync da/de/d1b/d4b/f105 0 2024-04-25T01:36:17.127 INFO:tasks.workunit.client.1.smithi138.stdout:9/787: creat d4/d1a/d55/d7f/f108 x:0 0 0 2024-04-25T01:36:17.128 INFO:tasks.workunit.client.1.smithi138.stdout:7/925: write da/de/d13/d30/d59/d7b/f10f [5258767,107377] 0 2024-04-25T01:36:17.128 INFO:tasks.workunit.client.1.smithi138.stdout:9/788: write d4/d8/d2c/d82/f107 [615274,100477] 0 2024-04-25T01:36:17.130 INFO:tasks.workunit.client.1.smithi138.stdout:9/789: dread - d4/d1a/fa2 zero size 2024-04-25T01:36:17.138 INFO:tasks.workunit.client.1.smithi138.stdout:9/790: dwrite d4/d63/d86/ffb [0,4194304] 0 2024-04-25T01:36:17.142 INFO:tasks.workunit.client.1.smithi138.stdout:7/926: dread da/de/d1b/d49/d81/d51/d6e/f87 [0,4194304] 0 2024-04-25T01:36:17.148 INFO:tasks.workunit.client.1.smithi138.stdout:9/791: dwrite d4/d8/d2c/fa8 [0,4194304] 0 2024-04-25T01:36:17.154 INFO:tasks.workunit.client.1.smithi138.stdout:9/792: symlink d4/d8/d90/d8e/l109 0 2024-04-25T01:36:17.154 INFO:tasks.workunit.client.1.smithi138.stdout:9/793: dread - d4/d1a/d55/d7f/d8f/f101 zero size 2024-04-25T01:36:17.163 INFO:tasks.workunit.client.1.smithi138.stdout:7/927: truncate da/de/d13/d21/d6f/fcf 713531 0 2024-04-25T01:36:17.164 INFO:tasks.workunit.client.1.smithi138.stdout:9/794: write d4/d8/d90/d8e/dd9/ffd [1108450,52320] 0 2024-04-25T01:36:17.166 INFO:tasks.workunit.client.1.smithi138.stdout:9/795: readlink d4/d8/d2c/d82/la4 0 2024-04-25T01:36:17.180 INFO:tasks.workunit.client.1.smithi138.stdout:7/928: dwrite da/de/d13/d21/dbc/d94/f132 [0,4194304] 0 2024-04-25T01:36:17.184 INFO:tasks.workunit.client.1.smithi138.stdout:9/796: dwrite d4/d1a/d68/dbe/ff1 [0,4194304] 0 2024-04-25T01:36:17.199 INFO:tasks.workunit.client.1.smithi138.stdout:5/921: sync 2024-04-25T01:36:17.200 INFO:tasks.workunit.client.1.smithi138.stdout:5/922: fsync d4/da/d6e/d22/d45/f67 0 2024-04-25T01:36:17.202 INFO:tasks.workunit.client.1.smithi138.stdout:7/929: symlink da/de/d1b/d4b/l137 0 2024-04-25T01:36:17.203 INFO:tasks.workunit.client.1.smithi138.stdout:4/781: sync 2024-04-25T01:36:17.203 INFO:tasks.workunit.client.1.smithi138.stdout:3/963: sync 2024-04-25T01:36:17.205 INFO:tasks.workunit.client.1.smithi138.stdout:5/923: fdatasync d4/da/d6e/f28 0 2024-04-25T01:36:17.207 INFO:tasks.workunit.client.1.smithi138.stdout:3/964: mkdir db/df/d19/d1a/dc4/d12d 0 2024-04-25T01:36:17.211 INFO:tasks.workunit.client.1.smithi138.stdout:5/924: symlink d4/da/d6e/d22/d35/dce/l136 0 2024-04-25T01:36:17.215 INFO:tasks.workunit.client.1.smithi138.stdout:3/965: rmdir db 39 2024-04-25T01:36:17.219 INFO:tasks.workunit.client.1.smithi138.stdout:4/782: creat d0/d1c/d1e/d9f/d7e/de0/fec x:0 0 0 2024-04-25T01:36:17.221 INFO:tasks.workunit.client.1.smithi138.stdout:7/930: creat da/de/d13/df7/f138 x:0 0 0 2024-04-25T01:36:17.222 INFO:tasks.workunit.client.1.smithi138.stdout:7/931: fdatasync da/de/d13/d21/dbc/dca/fe9 0 2024-04-25T01:36:17.224 INFO:tasks.workunit.client.1.smithi138.stdout:5/925: rename d4/da/d6e/d22/d35/dce/dda to d4/da/d6e/d22/d35/dce/df9/d83/d89/d9d/d9e/d101/d137 0 2024-04-25T01:36:17.228 INFO:tasks.workunit.client.1.smithi138.stdout:4/783: mkdir d0/d59/dbc/ded 0 2024-04-25T01:36:17.229 INFO:tasks.workunit.client.1.smithi138.stdout:4/784: dread - d0/d1c/d1e/fb7 zero size 2024-04-25T01:36:17.232 INFO:tasks.workunit.client.1.smithi138.stdout:5/926: rename d4/f82 to d4/d122/d5e/d8f/f138 0 2024-04-25T01:36:17.234 INFO:tasks.workunit.client.1.smithi138.stdout:3/966: stat db/d9b/db4/df1/l11d 0 2024-04-25T01:36:17.238 INFO:tasks.workunit.client.1.smithi138.stdout:4/785: write d0/d1c/d1e/fd7 [1725376,82530] 0 2024-04-25T01:36:17.238 INFO:tasks.workunit.client.1.smithi138.stdout:7/932: dwrite da/dd2/fe0 [0,4194304] 0 2024-04-25T01:36:17.243 INFO:tasks.workunit.client.1.smithi138.stdout:3/967: rmdir db/d9b/db4 39 2024-04-25T01:36:17.248 INFO:tasks.workunit.client.1.smithi138.stdout:4/786: rename d0/d1c/d1e/d9f/d7e/dd5/f4d to d0/d1c/d48/ddf/fee 0 2024-04-25T01:36:17.250 INFO:tasks.workunit.client.1.smithi138.stdout:7/933: dwrite da/de/d1b/d49/d81/d51/d6e/f126 [0,4194304] 0 2024-04-25T01:36:17.253 INFO:tasks.workunit.client.1.smithi138.stdout:4/787: write d0/d1c/d1e/dbb/dae/fcf [916089,28790] 0 2024-04-25T01:36:17.258 INFO:tasks.workunit.client.1.smithi138.stdout:3/968: mkdir db/df/de2/d10a/d112/d12e 0 2024-04-25T01:36:17.262 INFO:tasks.workunit.client.1.smithi138.stdout:3/969: readlink db/df/l13 0 2024-04-25T01:36:17.262 INFO:tasks.workunit.client.1.smithi138.stdout:7/934: dread - da/de/d1b/d49/d81/dc7/fcb zero size 2024-04-25T01:36:17.267 INFO:tasks.workunit.client.1.smithi138.stdout:3/970: creat db/df/de2/d10a/f12f x:0 0 0 2024-04-25T01:36:17.270 INFO:tasks.workunit.client.1.smithi138.stdout:5/927: dread d4/f18 [0,4194304] 0 2024-04-25T01:36:17.272 INFO:tasks.workunit.client.1.smithi138.stdout:3/971: creat db/df/d17/f130 x:0 0 0 2024-04-25T01:36:17.274 INFO:tasks.workunit.client.1.smithi138.stdout:3/972: fsync db/df/de2/d10a/d112/f10d 0 2024-04-25T01:36:17.280 INFO:tasks.workunit.client.1.smithi138.stdout:3/973: dread - db/df/de2/fec zero size 2024-04-25T01:36:17.285 INFO:tasks.workunit.client.1.smithi138.stdout:4/788: dread d0/d1c/d1e/dbb/dae/fcf [0,4194304] 0 2024-04-25T01:36:17.285 INFO:tasks.workunit.client.1.smithi138.stdout:9/797: dwrite d4/d8/d90/d52/f7d [0,4194304] 0 2024-04-25T01:36:17.288 INFO:tasks.workunit.client.1.smithi138.stdout:3/974: rmdir db/df/de2/d10a/d112/d40 39 2024-04-25T01:36:17.290 INFO:tasks.workunit.client.1.smithi138.stdout:9/798: dread - d4/d1a/d68/f98 zero size 2024-04-25T01:36:17.292 INFO:tasks.workunit.client.1.smithi138.stdout:4/789: mkdir d0/d1c/d1e/d9f/de5/def 0 2024-04-25T01:36:17.294 INFO:tasks.workunit.client.1.smithi138.stdout:3/975: fsync db/df/de2/d10a/d112/d40/f9c 0 2024-04-25T01:36:17.296 INFO:tasks.workunit.client.1.smithi138.stdout:4/790: mkdir d0/d1c/d1e/d9f/de5/df0 0 2024-04-25T01:36:17.299 INFO:tasks.workunit.client.1.smithi138.stdout:3/976: rmdir db/df/d19/dd4 39 2024-04-25T01:36:17.299 INFO:tasks.workunit.client.1.smithi138.stdout:3/977: chown db/d9b/fc2 10961186 1 2024-04-25T01:36:17.301 INFO:tasks.workunit.client.1.smithi138.stdout:9/799: getdents d4/d63/d86 0 2024-04-25T01:36:17.304 INFO:tasks.workunit.client.1.smithi138.stdout:3/978: creat db/df/dd1/ddf/f131 x:0 0 0 2024-04-25T01:36:17.305 INFO:tasks.workunit.client.1.smithi138.stdout:9/800: symlink d4/d1a/d33/l10a 0 2024-04-25T01:36:17.307 INFO:tasks.workunit.client.1.smithi138.stdout:9/801: read d4/d49/f76 [3478860,89371] 0 2024-04-25T01:36:17.312 INFO:tasks.workunit.client.1.smithi138.stdout:3/979: getdents db/df/d17/d23/d34 0 2024-04-25T01:36:17.313 INFO:tasks.workunit.client.1.smithi138.stdout:3/980: stat db/df/dd1/d127 0 2024-04-25T01:36:17.315 INFO:tasks.workunit.client.1.smithi138.stdout:5/928: dwrite d4/da/d6e/d22/d74/f85 [0,4194304] 0 2024-04-25T01:36:17.319 INFO:tasks.workunit.client.1.smithi138.stdout:4/791: dwrite d0/d1c/f6b [0,4194304] 0 2024-04-25T01:36:17.320 INFO:tasks.workunit.client.1.smithi138.stdout:5/929: mkdir d4/d122/d5e/d60/dbd/dc8/dbb/d139 0 2024-04-25T01:36:17.324 INFO:tasks.workunit.client.1.smithi138.stdout:7/935: dwrite da/de/d13/d21/d6f/fcf [0,4194304] 0 2024-04-25T01:36:17.327 INFO:tasks.workunit.client.1.smithi138.stdout:4/792: rename d0/f1a to d0/ff1 0 2024-04-25T01:36:17.333 INFO:tasks.workunit.client.1.smithi138.stdout:4/793: symlink d0/d1c/lf2 0 2024-04-25T01:36:17.336 INFO:tasks.workunit.client.1.smithi138.stdout:4/794: mknod d0/d1c/d1e/cf3 0 2024-04-25T01:36:17.342 INFO:tasks.workunit.client.1.smithi138.stdout:7/936: dwrite da/de/d1b/d4b/d5b/f46 [0,4194304] 0 2024-04-25T01:36:17.343 INFO:tasks.workunit.client.1.smithi138.stdout:4/795: dread d0/f40 [0,4194304] 0 2024-04-25T01:36:17.348 INFO:tasks.workunit.client.1.smithi138.stdout:4/796: creat d0/d1c/d48/ddf/ff4 x:0 0 0 2024-04-25T01:36:17.349 INFO:tasks.workunit.client.1.smithi138.stdout:4/797: write d0/d1c/d1e/dbb/dae/dc6/fe9 [644658,56498] 0 2024-04-25T01:36:17.350 INFO:tasks.workunit.client.1.smithi138.stdout:4/798: fdatasync d0/d1c/f62 0 2024-04-25T01:36:17.351 INFO:tasks.workunit.client.1.smithi138.stdout:7/937: link da/de/d13/d30/df8/fb9 da/de/d13/d30/df8/f139 0 2024-04-25T01:36:17.352 INFO:tasks.workunit.client.1.smithi138.stdout:4/799: chown d0/d1c/d1e/d9f/d33/c46 38 1 2024-04-25T01:36:17.355 INFO:tasks.workunit.client.1.smithi138.stdout:7/938: link da/de/fa9 da/de/d13/d21/d6f/d8d/f13a 0 2024-04-25T01:36:17.356 INFO:tasks.workunit.client.1.smithi138.stdout:7/939: write da/de/d13/d21/f41 [1863934,11196] 0 2024-04-25T01:36:17.360 INFO:tasks.workunit.client.1.smithi138.stdout:4/800: fsync d0/d1c/f6b 0 2024-04-25T01:36:17.364 INFO:tasks.workunit.client.1.smithi138.stdout:4/801: rename d0/f13 to d0/d1c/d1e/dbb/dae/dc6/d7f/ff5 0 2024-04-25T01:36:17.370 INFO:tasks.workunit.client.1.smithi138.stdout:7/940: write da/de/d13/d95/dd7/f110 [238304,63352] 0 2024-04-25T01:36:17.375 INFO:tasks.workunit.client.1.smithi138.stdout:7/941: write da/de/d13/d30/df8/f139 [3732623,55620] 0 2024-04-25T01:36:17.376 INFO:tasks.workunit.client.1.smithi138.stdout:4/802: dwrite d0/d1c/d1e/d9f/d33/f99 [0,4194304] 0 2024-04-25T01:36:17.378 INFO:tasks.workunit.client.1.smithi138.stdout:4/803: chown d0/d1c/d1e/dbb/dae/dc6/d7f/l95 2239236 1 2024-04-25T01:36:17.379 INFO:tasks.workunit.client.1.smithi138.stdout:7/942: mkdir da/de/d1b/d49/d81/d51/d13b 0 2024-04-25T01:36:17.381 INFO:tasks.workunit.client.1.smithi138.stdout:4/804: mknod d0/cf6 0 2024-04-25T01:36:17.383 INFO:tasks.workunit.client.1.smithi138.stdout:4/805: chown d0/d1c/d1e/dbb/da3/db5/fc8 6728286 1 2024-04-25T01:36:17.383 INFO:tasks.workunit.client.1.smithi138.stdout:4/806: chown d0/d1c/d1e/dbb/da3/fa4 52854 1 2024-04-25T01:36:17.383 INFO:tasks.workunit.client.1.smithi138.stdout:7/943: mknod da/de/d1b/d49/d81/d51/d6e/da2/d102/c13c 0 2024-04-25T01:36:17.386 INFO:tasks.workunit.client.1.smithi138.stdout:7/944: creat da/de/d1b/d34/f13d x:0 0 0 2024-04-25T01:36:17.388 INFO:tasks.workunit.client.1.smithi138.stdout:7/945: mkdir da/de/d1b/d49/d81/d13e 0 2024-04-25T01:36:17.389 INFO:tasks.workunit.client.1.smithi138.stdout:7/946: dread - da/de/d1b/d34/ffc zero size 2024-04-25T01:36:17.393 INFO:tasks.workunit.client.1.smithi138.stdout:7/947: unlink da/de/d13/d21/d6f/d8d/cf2 0 2024-04-25T01:36:17.394 INFO:tasks.workunit.client.1.smithi138.stdout:7/948: chown da/de/d13/d30/d59/ff3 259232236 1 2024-04-25T01:36:17.398 INFO:tasks.workunit.client.1.smithi138.stdout:4/807: dread d0/f64 [4194304,4194304] 0 2024-04-25T01:36:17.400 INFO:tasks.workunit.client.1.smithi138.stdout:4/808: write d0/d1c/f2c [3314411,37378] 0 2024-04-25T01:36:17.401 INFO:tasks.workunit.client.1.smithi138.stdout:7/949: dwrite da/de/d1b/d49/d81/d51/faa [0,4194304] 0 2024-04-25T01:36:17.401 INFO:tasks.workunit.client.1.smithi138.stdout:4/809: fsync d0/d1c/d1e/d9f/fde 0 2024-04-25T01:36:17.405 INFO:tasks.workunit.client.1.smithi138.stdout:7/950: truncate da/de/d1b/d4b/d5b/f67 4846913 0 2024-04-25T01:36:17.407 INFO:tasks.workunit.client.1.smithi138.stdout:4/810: creat d0/d1c/d1e/d9f/de5/def/ff7 x:0 0 0 2024-04-25T01:36:17.413 INFO:tasks.workunit.client.1.smithi138.stdout:4/811: getdents d0/d1c/d1e/dbb/dae/dc6 0 2024-04-25T01:36:17.414 INFO:tasks.workunit.client.1.smithi138.stdout:4/812: mknod d0/d59/cf8 0 2024-04-25T01:36:17.421 INFO:tasks.workunit.client.1.smithi138.stdout:7/951: write da/de/d1b/d34/f47 [713426,78012] 0 2024-04-25T01:36:17.421 INFO:tasks.workunit.client.1.smithi138.stdout:7/952: dread - da/de/d13/df7/f138 zero size 2024-04-25T01:36:17.422 INFO:tasks.workunit.client.1.smithi138.stdout:7/953: stat da/de/d1b/d49/f119 0 2024-04-25T01:36:17.426 INFO:tasks.workunit.client.1.smithi138.stdout:7/954: getdents da/de/d13/d95/dd7 0 2024-04-25T01:36:17.428 INFO:tasks.workunit.client.1.smithi138.stdout:7/955: fsync da/de/fa9 0 2024-04-25T01:36:17.433 INFO:tasks.workunit.client.1.smithi138.stdout:4/813: dread d0/d1c/f28 [0,4194304] 0 2024-04-25T01:36:17.434 INFO:tasks.workunit.client.1.smithi138.stdout:7/956: dwrite da/de/d1b/d34/f47 [4194304,4194304] 0 2024-04-25T01:36:17.435 INFO:tasks.workunit.client.1.smithi138.stdout:4/814: write d0/d1c/d1e/d9f/d33/f89 [3800368,102253] 0 2024-04-25T01:36:17.438 INFO:tasks.workunit.client.1.smithi138.stdout:7/957: creat da/de/d1b/d4b/d5b/f13f x:0 0 0 2024-04-25T01:36:17.438 INFO:tasks.workunit.client.1.smithi138.stdout:4/815: rmdir d0/d1c/d1e/d9f/d7e/dd5/d74 39 2024-04-25T01:36:17.439 INFO:tasks.workunit.client.1.smithi138.stdout:7/958: chown da/de/d1b/d49 409227 1 2024-04-25T01:36:17.440 INFO:tasks.workunit.client.1.smithi138.stdout:4/816: mknod d0/d1c/d1e/dbb/dae/dc6/cf9 0 2024-04-25T01:36:17.443 INFO:tasks.workunit.client.1.smithi138.stdout:7/959: rename da/de/d1b/d49/d81/d51/l7c to da/de/d13/d30/d59/d7b/l140 0 2024-04-25T01:36:17.443 INFO:tasks.workunit.client.1.smithi138.stdout:1/884: sync 2024-04-25T01:36:17.445 INFO:tasks.workunit.client.1.smithi138.stdout:4/817: truncate d0/d1c/f62 3788261 0 2024-04-25T01:36:17.446 INFO:tasks.workunit.client.1.smithi138.stdout:1/885: creat d3/d45/f130 x:0 0 0 2024-04-25T01:36:17.448 INFO:tasks.workunit.client.1.smithi138.stdout:1/886: chown d3/db/c5b 10552 1 2024-04-25T01:36:17.449 INFO:tasks.workunit.client.1.smithi138.stdout:1/887: readlink d3/d27/d3e/d90/l66 0 2024-04-25T01:36:17.449 INFO:tasks.workunit.client.1.smithi138.stdout:1/888: readlink d3/d45/d72/db2/dd9/df9/d126/lcf 0 2024-04-25T01:36:17.467 INFO:tasks.workunit.client.1.smithi138.stdout:1/889: dwrite d3/d45/d72/db2/dd9/df9/d126/f75 [0,4194304] 0 2024-04-25T01:36:17.474 INFO:tasks.workunit.client.1.smithi138.stdout:7/960: dread da/de/d13/d30/f42 [0,4194304] 0 2024-04-25T01:36:17.477 INFO:tasks.workunit.client.1.smithi138.stdout:4/818: dread d0/d1c/d1e/dbb/da3/fa4 [0,4194304] 0 2024-04-25T01:36:17.477 INFO:tasks.workunit.client.1.smithi138.stdout:7/961: chown da/de/d13/d21/dbc/d94/d9d/le4 3946656 1 2024-04-25T01:36:17.478 INFO:tasks.workunit.client.1.smithi138.stdout:1/890: dwrite d3/d27/d16/da3/fd2 [0,4194304] 0 2024-04-25T01:36:17.479 INFO:tasks.workunit.client.1.smithi138.stdout:4/819: truncate d0/d1c/d48/f6f 4853871 0 2024-04-25T01:36:17.483 INFO:tasks.workunit.client.1.smithi138.stdout:4/820: write d0/d1c/d1e/fb7 [889191,93230] 0 2024-04-25T01:36:17.484 INFO:tasks.workunit.client.1.smithi138.stdout:1/891: unlink d3/d27/d3e/df4/d123/l12f 0 2024-04-25T01:36:17.485 INFO:tasks.workunit.client.1.smithi138.stdout:7/962: dread da/de/f2c [0,4194304] 0 2024-04-25T01:36:17.487 INFO:tasks.workunit.client.1.smithi138.stdout:4/821: chown d0/d1c/d1e/dbb/dae/dc6/la9 1 1 2024-04-25T01:36:17.489 INFO:tasks.workunit.client.1.smithi138.stdout:7/963: mknod da/de/d1b/d4b/c141 0 2024-04-25T01:36:17.490 INFO:tasks.workunit.client.1.smithi138.stdout:4/822: creat d0/d1c/d1e/d9f/d7e/dd5/ffa x:0 0 0 2024-04-25T01:36:17.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:17 smithi033 ceph-mon[26353]: pgmap v24: 65 pgs: 65 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 105 MiB/s rd, 262 MiB/s wr, 631 op/s 2024-04-25T01:36:17.492 INFO:tasks.workunit.client.1.smithi138.stdout:4/823: chown d0/c25 615250654 1 2024-04-25T01:36:17.493 INFO:tasks.workunit.client.1.smithi138.stdout:7/964: dread da/de/d1b/d4b/d77/f91 [0,4194304] 0 2024-04-25T01:36:17.494 INFO:tasks.workunit.client.1.smithi138.stdout:7/965: stat da/de/d13/f121 0 2024-04-25T01:36:17.494 INFO:tasks.workunit.client.1.smithi138.stdout:1/892: write d3/ddb/d9b/f68 [2491419,4742] 0 2024-04-25T01:36:17.496 INFO:tasks.workunit.client.1.smithi138.stdout:1/893: mknod d3/d45/d72/db2/c131 0 2024-04-25T01:36:17.501 INFO:tasks.workunit.client.1.smithi138.stdout:1/894: dread d3/d45/d72/d57/f63 [8388608,4194304] 0 2024-04-25T01:36:17.506 INFO:tasks.workunit.client.1.smithi138.stdout:1/895: dwrite d3/d27/d2d/f32 [0,4194304] 0 2024-04-25T01:36:17.512 INFO:tasks.workunit.client.1.smithi138.stdout:7/966: dread da/de/d13/f2d [0,4194304] 0 2024-04-25T01:36:17.514 INFO:tasks.workunit.client.1.smithi138.stdout:1/896: rename d3/d4/da7/db1/c10e to d3/d27/d3e/c132 0 2024-04-25T01:36:17.515 INFO:tasks.workunit.client.1.smithi138.stdout:1/897: dread - d3/d45/d72/d57/d8e/db6/f11a zero size 2024-04-25T01:36:17.518 INFO:tasks.workunit.client.1.smithi138.stdout:1/898: dread d3/d45/d72/d57/f91 [0,4194304] 0 2024-04-25T01:36:17.519 INFO:tasks.workunit.client.1.smithi138.stdout:1/899: fdatasync d3/f2c 0 2024-04-25T01:36:17.526 INFO:tasks.workunit.client.1.smithi138.stdout:7/967: dwrite da/f18 [4194304,4194304] 0 2024-04-25T01:36:17.527 INFO:tasks.workunit.client.1.smithi138.stdout:1/900: dwrite d3/d27/d16/f28 [0,4194304] 0 2024-04-25T01:36:17.530 INFO:tasks.workunit.client.1.smithi138.stdout:1/901: chown d3/d4/da7/lb4 128 1 2024-04-25T01:36:17.532 INFO:tasks.workunit.client.1.smithi138.stdout:7/968: creat da/de/d13/d30/d59/d7b/f142 x:0 0 0 2024-04-25T01:36:17.535 INFO:tasks.workunit.client.1.smithi138.stdout:7/969: mkdir da/de/d1b/d49/d143 0 2024-04-25T01:36:17.541 INFO:tasks.workunit.client.1.smithi138.stdout:7/970: dwrite da/de/d13/d30/f108 [0,4194304] 0 2024-04-25T01:36:17.545 INFO:tasks.workunit.client.1.smithi138.stdout:7/971: mkdir da/de/d1b/d49/d81/d51/d6e/da2/d102/d144 0 2024-04-25T01:36:17.553 INFO:tasks.workunit.client.1.smithi138.stdout:7/972: truncate da/de/d13/d30/df8/fb9 7471165 0 2024-04-25T01:36:17.553 INFO:tasks.workunit.client.1.smithi138.stdout:1/902: dwrite d3/d27/d3e/f88 [0,4194304] 0 2024-04-25T01:36:17.555 INFO:tasks.workunit.client.1.smithi138.stdout:1/903: write d3/d45/d72/d57/fec [77352,1704] 0 2024-04-25T01:36:17.563 INFO:tasks.workunit.client.1.smithi138.stdout:7/973: dwrite da/de/d1b/d4b/f105 [0,4194304] 0 2024-04-25T01:36:17.572 INFO:tasks.workunit.client.1.smithi138.stdout:1/904: dwrite d3/d45/d72/d57/db0/dcd/de3/ffc [0,4194304] 0 2024-04-25T01:36:17.573 INFO:tasks.workunit.client.1.smithi138.stdout:7/974: dwrite da/de/d13/d95/dd7/f110 [0,4194304] 0 2024-04-25T01:36:17.575 INFO:tasks.workunit.client.1.smithi138.stdout:1/905: write d3/d27/d16/da3/fd2 [5653903,94225] 0 2024-04-25T01:36:17.579 INFO:tasks.workunit.client.1.smithi138.stdout:7/975: mknod da/de/d1b/d34/c145 0 2024-04-25T01:36:17.581 INFO:tasks.workunit.client.1.smithi138.stdout:1/906: unlink d3/db/f5f 0 2024-04-25T01:36:17.585 INFO:tasks.workunit.client.1.smithi138.stdout:1/907: mknod d3/d45/d72/d3d/c133 0 2024-04-25T01:36:17.588 INFO:tasks.workunit.client.1.smithi138.stdout:7/976: dwrite da/de/d13/d30/f65 [0,4194304] 0 2024-04-25T01:36:17.589 INFO:tasks.workunit.client.1.smithi138.stdout:1/908: rename d3/ddb/d9b/f68 to d3/d27/d16/d10a/f134 0 2024-04-25T01:36:17.592 INFO:tasks.workunit.client.1.smithi138.stdout:7/977: mkdir da/de/d1b/d49/d146 0 2024-04-25T01:36:17.594 INFO:tasks.workunit.client.1.smithi138.stdout:7/978: unlink da/de/d1b/d49/d81/d51/c8c 0 2024-04-25T01:36:17.595 INFO:tasks.workunit.client.1.smithi138.stdout:7/979: fdatasync da/dd2/f10a 0 2024-04-25T01:36:17.609 INFO:tasks.workunit.client.1.smithi138.stdout:1/909: dwrite d3/d27/d16/da3/daf/fc6 [0,4194304] 0 2024-04-25T01:36:17.614 INFO:tasks.workunit.client.1.smithi138.stdout:1/910: truncate d3/ddb/fc2 1186645 0 2024-04-25T01:36:17.615 INFO:tasks.workunit.client.1.smithi138.stdout:1/911: write d3/ddb/dd6/f110 [740885,27450] 0 2024-04-25T01:36:17.620 INFO:tasks.workunit.client.1.smithi138.stdout:1/912: rename d3/d27/d16/da3/fd2 to d3/d45/d72/d57/db0/f135 0 2024-04-25T01:36:17.632 INFO:tasks.workunit.client.1.smithi138.stdout:1/913: write d3/d45/d72/f5d [2764776,85776] 0 2024-04-25T01:36:17.635 INFO:tasks.workunit.client.1.smithi138.stdout:1/914: unlink d3/d27/d3e/d90/l66 0 2024-04-25T01:36:17.635 INFO:tasks.workunit.client.1.smithi138.stdout:1/915: write d3/d45/f48 [1798956,17422] 0 2024-04-25T01:36:17.639 INFO:tasks.workunit.client.1.smithi138.stdout:1/916: unlink d3/d45/d72/d57/d8e/fbe 0 2024-04-25T01:36:17.641 INFO:tasks.workunit.client.1.smithi138.stdout:1/917: creat d3/d45/d72/db2/dfa/f136 x:0 0 0 2024-04-25T01:36:17.643 INFO:tasks.workunit.client.1.smithi138.stdout:1/918: unlink d3/d45/d72/d57/db0/dcd/de3/ffc 0 2024-04-25T01:36:17.643 INFO:tasks.workunit.client.1.smithi138.stdout:1/919: stat d3/d45/d72/d57/d8e/db6/fdf 0 2024-04-25T01:36:17.651 INFO:tasks.workunit.client.1.smithi138.stdout:0/768: sync 2024-04-25T01:36:17.652 INFO:tasks.workunit.client.1.smithi138.stdout:0/769: chown d6/df/d41/cfb 211 1 2024-04-25T01:36:17.655 INFO:tasks.workunit.client.1.smithi138.stdout:0/770: fdatasync d6/d39/f70 0 2024-04-25T01:36:17.658 INFO:tasks.workunit.client.1.smithi138.stdout:1/920: dwrite d3/d45/d72/d57/db0/fe9 [0,4194304] 0 2024-04-25T01:36:17.669 INFO:tasks.workunit.client.1.smithi138.stdout:1/921: mknod d3/db/c137 0 2024-04-25T01:36:17.670 INFO:tasks.workunit.client.1.smithi138.stdout:1/922: chown d3/d45/f130 8322063 1 2024-04-25T01:36:17.673 INFO:tasks.workunit.client.1.smithi138.stdout:1/923: creat d3/d27/d3e/df4/d123/d128/f138 x:0 0 0 2024-04-25T01:36:17.675 INFO:tasks.workunit.client.1.smithi138.stdout:0/771: dwrite d6/df/d41/d49/f63 [0,4194304] 0 2024-04-25T01:36:17.678 INFO:tasks.workunit.client.1.smithi138.stdout:1/924: mkdir d3/d27/d2d/da1/d139 0 2024-04-25T01:36:17.682 INFO:tasks.workunit.client.1.smithi138.stdout:1/925: mkdir d3/d27/d2d/da1/d139/d13a 0 2024-04-25T01:36:17.684 INFO:tasks.workunit.client.1.smithi138.stdout:1/926: write d3/d45/d72/d57/fec [551895,101260] 0 2024-04-25T01:36:17.688 INFO:tasks.workunit.client.1.smithi138.stdout:1/927: dread d3/d27/d24/fae [0,4194304] 0 2024-04-25T01:36:17.689 INFO:tasks.workunit.client.1.smithi138.stdout:1/928: chown d3/d27/d3e/fd4 2610 1 2024-04-25T01:36:17.691 INFO:tasks.workunit.client.1.smithi138.stdout:1/929: symlink d3/ddb/dd6/l13b 0 2024-04-25T01:36:17.692 INFO:tasks.workunit.client.1.smithi138.stdout:0/772: truncate d6/f56 3534063 0 2024-04-25T01:36:17.693 INFO:tasks.workunit.client.1.smithi138.stdout:0/773: chown d6/df/d69/faa 1522 1 2024-04-25T01:36:17.693 INFO:tasks.workunit.client.1.smithi138.stdout:1/930: mkdir d3/ddb/dd6/d12d/d13c 0 2024-04-25T01:36:17.693 INFO:tasks.workunit.client.1.smithi138.stdout:0/774: stat d6/d39/d52/cd0 0 2024-04-25T01:36:17.694 INFO:tasks.workunit.client.1.smithi138.stdout:1/931: write d3/d27/d3e/f61 [4837756,103503] 0 2024-04-25T01:36:17.697 INFO:tasks.workunit.client.1.smithi138.stdout:0/775: unlink d6/d2f/d75/fb9 0 2024-04-25T01:36:17.698 INFO:tasks.workunit.client.1.smithi138.stdout:1/932: creat d3/d27/d3e/df4/d123/d128/f13d x:0 0 0 2024-04-25T01:36:17.700 INFO:tasks.workunit.client.1.smithi138.stdout:1/933: fsync d3/d27/d16/da3/daf/ff8 0 2024-04-25T01:36:17.700 INFO:tasks.workunit.client.1.smithi138.stdout:0/776: creat d6/df/d3a/d67/d8a/ffe x:0 0 0 2024-04-25T01:36:17.702 INFO:tasks.workunit.client.1.smithi138.stdout:0/777: write d6/df/d3a/fd7 [1778998,2406] 0 2024-04-25T01:36:17.702 INFO:tasks.workunit.client.1.smithi138.stdout:1/934: mkdir d3/d4/da7/db1/de1/d13e 0 2024-04-25T01:36:17.705 INFO:tasks.workunit.client.1.smithi138.stdout:0/778: mknod d6/df/d3a/cff 0 2024-04-25T01:36:17.711 INFO:tasks.workunit.client.1.smithi138.stdout:0/779: creat d6/d2f/de2/f100 x:0 0 0 2024-04-25T01:36:17.713 INFO:tasks.workunit.client.1.smithi138.stdout:0/780: creat d6/d39/d52/db7/f101 x:0 0 0 2024-04-25T01:36:17.717 INFO:tasks.workunit.client.1.smithi138.stdout:0/781: creat d6/df/d41/f102 x:0 0 0 2024-04-25T01:36:17.719 INFO:tasks.workunit.client.1.smithi138.stdout:0/782: creat d6/d1d/d82/f103 x:0 0 0 2024-04-25T01:36:17.725 INFO:tasks.workunit.client.1.smithi138.stdout:0/783: dwrite d6/df/d66/f7f [0,4194304] 0 2024-04-25T01:36:17.725 INFO:tasks.workunit.client.1.smithi138.stdout:1/935: dread d3/d45/d72/d57/f63 [0,4194304] 0 2024-04-25T01:36:17.725 INFO:tasks.workunit.client.1.smithi138.stdout:0/784: readlink d6/l99 0 2024-04-25T01:36:17.726 INFO:tasks.workunit.client.1.smithi138.stdout:1/936: chown d3/d27/d3e/f88 1151674 1 2024-04-25T01:36:17.726 INFO:tasks.workunit.client.1.smithi138.stdout:0/785: fsync d6/d39/d52/d9c/dbc/dbf/fdd 0 2024-04-25T01:36:17.729 INFO:tasks.workunit.client.1.smithi138.stdout:1/937: creat d3/d45/d72/db2/f13f x:0 0 0 2024-04-25T01:36:17.732 INFO:tasks.workunit.client.1.smithi138.stdout:0/786: creat d6/d1d/d82/dc6/f104 x:0 0 0 2024-04-25T01:36:17.735 INFO:tasks.workunit.client.1.smithi138.stdout:0/787: symlink d6/d2f/d31/l105 0 2024-04-25T01:36:17.736 INFO:tasks.workunit.client.1.smithi138.stdout:0/788: truncate d6/d39/fa2 1937956 0 2024-04-25T01:36:17.737 INFO:tasks.workunit.client.1.smithi138.stdout:1/938: write d3/d45/fd1 [208966,77290] 0 2024-04-25T01:36:17.737 INFO:tasks.workunit.client.1.smithi138.stdout:0/789: write d6/df/d66/f7f [1029650,61489] 0 2024-04-25T01:36:17.741 INFO:tasks.workunit.client.1.smithi138.stdout:1/939: mknod d3/d27/d3e/df4/d123/d128/c140 0 2024-04-25T01:36:17.742 INFO:tasks.workunit.client.1.smithi138.stdout:1/940: mknod d3/ddb/d10f/c141 0 2024-04-25T01:36:17.744 INFO:tasks.workunit.client.1.smithi138.stdout:1/941: mknod d3/ddb/dd6/d12d/d13c/c142 0 2024-04-25T01:36:17.752 INFO:tasks.workunit.client.1.smithi138.stdout:1/942: dread d3/f13 [0,4194304] 0 2024-04-25T01:36:17.753 INFO:tasks.workunit.client.1.smithi138.stdout:1/943: write d3/d45/f130 [609734,41199] 0 2024-04-25T01:36:17.754 INFO:tasks.workunit.client.1.smithi138.stdout:1/944: write d3/d27/f1e [3315723,60544] 0 2024-04-25T01:36:17.756 INFO:tasks.workunit.client.1.smithi138.stdout:0/790: dread d6/d19/f47 [0,4194304] 0 2024-04-25T01:36:17.757 INFO:tasks.workunit.client.1.smithi138.stdout:1/945: rmdir d3/d45/d72/d57/d8e/d113 39 2024-04-25T01:36:17.758 INFO:tasks.workunit.client.1.smithi138.stdout:0/791: read d6/fe [2050575,106144] 0 2024-04-25T01:36:17.759 INFO:tasks.workunit.client.1.smithi138.stdout:0/792: write d6/d39/fdf [178220,116454] 0 2024-04-25T01:36:17.760 INFO:tasks.workunit.client.1.smithi138.stdout:1/946: creat d3/d45/d72/d57/f143 x:0 0 0 2024-04-25T01:36:17.764 INFO:tasks.workunit.client.1.smithi138.stdout:1/947: creat d3/d4/da7/db1/de1/d13e/f144 x:0 0 0 2024-04-25T01:36:17.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:17 smithi138 ceph-mon[32977]: pgmap v24: 65 pgs: 65 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 105 MiB/s rd, 262 MiB/s wr, 631 op/s 2024-04-25T01:36:17.767 INFO:tasks.workunit.client.1.smithi138.stdout:0/793: write d6/f9 [3127286,110272] 0 2024-04-25T01:36:17.768 INFO:tasks.workunit.client.1.smithi138.stdout:0/794: creat d6/d2f/d75/db6/de5/f106 x:0 0 0 2024-04-25T01:36:17.770 INFO:tasks.workunit.client.1.smithi138.stdout:0/795: creat d6/d2f/d75/f107 x:0 0 0 2024-04-25T01:36:17.771 INFO:tasks.workunit.client.1.smithi138.stdout:0/796: readlink d6/d2f/l93 0 2024-04-25T01:36:17.776 INFO:tasks.workunit.client.1.smithi138.stdout:0/797: dwrite d6/d39/d52/db7/f101 [0,4194304] 0 2024-04-25T01:36:17.779 INFO:tasks.workunit.client.1.smithi138.stdout:0/798: mknod d6/df/d41/d49/de4/c108 0 2024-04-25T01:36:17.783 INFO:tasks.workunit.client.1.smithi138.stdout:0/799: creat d6/d19/f109 x:0 0 0 2024-04-25T01:36:17.784 INFO:tasks.workunit.client.1.smithi138.stdout:0/800: mknod d6/d2f/d75/db6/c10a 0 2024-04-25T01:36:17.786 INFO:tasks.workunit.client.1.smithi138.stdout:0/801: unlink d6/d39/d58/c5d 0 2024-04-25T01:36:17.787 INFO:tasks.workunit.client.1.smithi138.stdout:0/802: creat d6/d1d/d82/f10b x:0 0 0 2024-04-25T01:36:17.788 INFO:tasks.workunit.client.1.smithi138.stdout:0/803: chown d6/d2f/l93 22410205 1 2024-04-25T01:36:17.789 INFO:tasks.workunit.client.1.smithi138.stdout:0/804: write d6/df/d41/f102 [662347,10679] 0 2024-04-25T01:36:17.789 INFO:tasks.workunit.client.1.smithi138.stdout:0/805: dread - d6/d2f/d75/db6/fd8 zero size 2024-04-25T01:36:17.790 INFO:tasks.workunit.client.1.smithi138.stdout:0/806: chown d6/d39/d52/db7/led 4 1 2024-04-25T01:36:17.796 INFO:tasks.workunit.client.1.smithi138.stdout:0/807: dwrite d6/d2f/d75/db6/de5/ff7 [0,4194304] 0 2024-04-25T01:36:17.797 INFO:tasks.workunit.client.1.smithi138.stdout:0/808: write d6/df/d69/f9d [4100662,1537] 0 2024-04-25T01:36:17.802 INFO:tasks.workunit.client.1.smithi138.stdout:0/809: mkdir d6/d19/dbb/d10c 0 2024-04-25T01:36:17.803 INFO:tasks.workunit.client.1.smithi138.stdout:0/810: read d6/df/d41/d49/f63 [3840856,99581] 0 2024-04-25T01:36:17.805 INFO:tasks.workunit.client.1.smithi138.stdout:9/802: sync 2024-04-25T01:36:17.806 INFO:tasks.workunit.client.1.smithi138.stdout:0/811: rename d6/d39/d52/c59 to d6/d39/d52/d9c/dbc/dbf/c10d 0 2024-04-25T01:36:17.807 INFO:tasks.workunit.client.1.smithi138.stdout:9/803: write d4/d8/d2c/d82/fe2 [1253616,123478] 0 2024-04-25T01:36:17.813 INFO:tasks.workunit.client.1.smithi138.stdout:0/812: dwrite d6/d39/d43/fb3 [0,4194304] 0 2024-04-25T01:36:17.814 INFO:tasks.workunit.client.1.smithi138.stdout:9/804: dwrite d4/f4a [0,4194304] 0 2024-04-25T01:36:17.817 INFO:tasks.workunit.client.1.smithi138.stdout:0/813: mkdir d6/d39/d52/db7/d10e 0 2024-04-25T01:36:17.819 INFO:tasks.workunit.client.1.smithi138.stdout:0/814: chown d6/df/d3a/d67/d8a/ce6 104 1 2024-04-25T01:36:17.821 INFO:tasks.workunit.client.1.smithi138.stdout:9/805: getdents d4 0 2024-04-25T01:36:17.823 INFO:tasks.workunit.client.1.smithi138.stdout:0/815: rename d6/d2f/d75/db6/de5/f106 to d6/d19/f10f 0 2024-04-25T01:36:17.823 INFO:tasks.workunit.client.1.smithi138.stdout:0/816: fdatasync d6/f9 0 2024-04-25T01:36:17.828 INFO:tasks.workunit.client.1.smithi138.stdout:0/817: rename d6/d2f/d31/l4f to d6/d39/d52/l110 0 2024-04-25T01:36:17.828 INFO:tasks.workunit.client.1.smithi138.stdout:9/806: link d4/d8/f61 d4/d8/f10b 0 2024-04-25T01:36:17.830 INFO:tasks.workunit.client.1.smithi138.stdout:0/818: creat d6/df/d41/d49/ddb/dec/f111 x:0 0 0 2024-04-25T01:36:17.832 INFO:tasks.workunit.client.1.smithi138.stdout:0/819: symlink d6/df/d41/l112 0 2024-04-25T01:36:17.833 INFO:tasks.workunit.client.1.smithi138.stdout:0/820: chown d6/d19/c9b 1272503 1 2024-04-25T01:36:17.834 INFO:tasks.workunit.client.1.smithi138.stdout:0/821: unlink d6/d2f/c8f 0 2024-04-25T01:36:17.836 INFO:tasks.workunit.client.1.smithi138.stdout:0/822: rmdir d6/d39/d43 39 2024-04-25T01:36:17.839 INFO:tasks.workunit.client.1.smithi138.stdout:0/823: rename d6/d39/d52/cd0 to d6/d1d/d82/c113 0 2024-04-25T01:36:17.844 INFO:tasks.workunit.client.1.smithi138.stdout:0/824: rename d6/d39/d52/ca3 to d6/c114 0 2024-04-25T01:36:17.844 INFO:tasks.workunit.client.1.smithi138.stdout:0/825: fdatasync d6/d1d/d82/f10b 0 2024-04-25T01:36:17.847 INFO:tasks.workunit.client.1.smithi138.stdout:0/826: rmdir d6/d1d/d98 39 2024-04-25T01:36:17.849 INFO:tasks.workunit.client.1.smithi138.stdout:0/827: creat d6/d2f/d75/db6/de5/f115 x:0 0 0 2024-04-25T01:36:17.849 INFO:tasks.workunit.client.1.smithi138.stdout:0/828: dread - d6/d19/f10f zero size 2024-04-25T01:36:17.852 INFO:tasks.workunit.client.1.smithi138.stdout:0/829: creat d6/d1d/df5/f116 x:0 0 0 2024-04-25T01:36:17.852 INFO:tasks.workunit.client.1.smithi138.stdout:0/830: fsync d6/d1d/d82/f10b 0 2024-04-25T01:36:17.854 INFO:tasks.workunit.client.1.smithi138.stdout:0/831: symlink d6/d39/d52/db7/d10e/l117 0 2024-04-25T01:36:17.855 INFO:tasks.workunit.client.1.smithi138.stdout:0/832: mknod d6/d2f/d31/c118 0 2024-04-25T01:36:17.856 INFO:tasks.workunit.client.1.smithi138.stdout:0/833: write d6/d1d/d98/ff3 [430347,84750] 0 2024-04-25T01:36:17.857 INFO:tasks.workunit.client.1.smithi138.stdout:0/834: dread - d6/d2f/d75/db6/de5/f115 zero size 2024-04-25T01:36:17.859 INFO:tasks.workunit.client.1.smithi138.stdout:0/835: write d6/d2f/d75/db6/de5/f115 [377780,28460] 0 2024-04-25T01:36:17.859 INFO:tasks.workunit.client.1.smithi138.stdout:0/836: readlink d6/d39/d52/db7/led 0 2024-04-25T01:36:17.861 INFO:tasks.workunit.client.1.smithi138.stdout:0/837: read d6/f56 [1716688,33124] 0 2024-04-25T01:36:17.867 INFO:tasks.workunit.client.1.smithi138.stdout:0/838: dwrite d6/f26 [0,4194304] 0 2024-04-25T01:36:17.868 INFO:tasks.workunit.client.1.smithi138.stdout:0/839: fdatasync d6/d39/d58/ffa 0 2024-04-25T01:36:17.871 INFO:tasks.workunit.client.1.smithi138.stdout:0/840: mkdir d6/df/d66/d119 0 2024-04-25T01:36:17.872 INFO:tasks.workunit.client.1.smithi138.stdout:0/841: chown d6/df/d3a/f71 43321 1 2024-04-25T01:36:17.875 INFO:tasks.workunit.client.1.smithi138.stdout:0/842: unlink d6/d2f/d31/cd2 0 2024-04-25T01:36:17.877 INFO:tasks.workunit.client.1.smithi138.stdout:0/843: symlink d6/d2f/de2/l11a 0 2024-04-25T01:36:17.877 INFO:tasks.workunit.client.1.smithi138.stdout:0/844: dread - d6/d2f/d75/db6/ff4 zero size 2024-04-25T01:36:17.878 INFO:tasks.workunit.client.1.smithi138.stdout:0/845: fsync d6/d2f/de2/f100 0 2024-04-25T01:36:17.880 INFO:tasks.workunit.client.1.smithi138.stdout:0/846: rename d6/d19/f74 to d6/d39/d58/f11b 0 2024-04-25T01:36:17.881 INFO:tasks.workunit.client.1.smithi138.stdout:0/847: mkdir d6/d39/d58/d11c 0 2024-04-25T01:36:17.883 INFO:tasks.workunit.client.1.smithi138.stdout:0/848: creat d6/df/d41/d49/f11d x:0 0 0 2024-04-25T01:36:17.885 INFO:tasks.workunit.client.1.smithi138.stdout:0/849: mkdir d6/df/d3a/d11e 0 2024-04-25T01:36:17.890 INFO:tasks.workunit.client.1.smithi138.stdout:0/850: dwrite d6/df/d3a/d67/d8a/ffe [0,4194304] 0 2024-04-25T01:36:17.894 INFO:tasks.workunit.client.1.smithi138.stdout:0/851: dread d6/df/d41/fa8 [0,4194304] 0 2024-04-25T01:36:17.897 INFO:tasks.workunit.client.1.smithi138.stdout:0/852: write d6/d1d/d98/fd9 [11447,107936] 0 2024-04-25T01:36:17.901 INFO:tasks.workunit.client.1.smithi138.stdout:0/853: rename d6/d39/d52/d9c/dbc/dbf/fdd to d6/df/d3a/d67/d8a/f11f 0 2024-04-25T01:36:17.902 INFO:tasks.workunit.client.1.smithi138.stdout:0/854: write d6/d2f/fe8 [826930,34090] 0 2024-04-25T01:36:17.903 INFO:tasks.workunit.client.1.smithi138.stdout:0/855: readlink d6/d2f/lc2 0 2024-04-25T01:36:17.904 INFO:tasks.workunit.client.1.smithi138.stdout:0/856: write d6/df/f18 [3141364,86675] 0 2024-04-25T01:36:17.905 INFO:tasks.workunit.client.1.smithi138.stdout:0/857: fsync d6/d39/d58/ffa 0 2024-04-25T01:36:17.907 INFO:tasks.workunit.client.1.smithi138.stdout:0/858: symlink d6/df/d41/d49/l120 0 2024-04-25T01:36:17.909 INFO:tasks.workunit.client.1.smithi138.stdout:0/859: symlink d6/d39/l121 0 2024-04-25T01:36:17.911 INFO:tasks.workunit.client.1.smithi138.stdout:0/860: rename d6/df/f5c to d6/d1d/d82/dc6/f122 0 2024-04-25T01:36:17.912 INFO:tasks.workunit.client.1.smithi138.stdout:0/861: read d6/d2f/d31/f86 [360279,92633] 0 2024-04-25T01:36:17.915 INFO:tasks.workunit.client.1.smithi138.stdout:0/862: getdents d6/d1d 0 2024-04-25T01:36:17.916 INFO:tasks.workunit.client.1.smithi138.stdout:0/863: symlink d6/df/d41/d49/de4/l123 0 2024-04-25T01:36:17.918 INFO:tasks.workunit.client.1.smithi138.stdout:0/864: symlink d6/d39/d58/d11c/l124 0 2024-04-25T01:36:17.928 INFO:tasks.workunit.client.1.smithi138.stdout:0/865: dwrite d6/d1d/f3c [0,4194304] 0 2024-04-25T01:36:17.935 INFO:tasks.workunit.client.1.smithi138.stdout:0/866: dwrite d6/df/d41/f102 [0,4194304] 0 2024-04-25T01:36:17.936 INFO:tasks.workunit.client.1.smithi138.stdout:0/867: chown d6/d2f/fd3 94495 1 2024-04-25T01:36:17.937 INFO:tasks.workunit.client.1.smithi138.stdout:0/868: dread - d6/d1d/d82/f103 zero size 2024-04-25T01:36:17.946 INFO:tasks.workunit.client.1.smithi138.stdout:0/869: dwrite d6/d19/f10f [0,4194304] 0 2024-04-25T01:36:17.946 INFO:tasks.workunit.client.1.smithi138.stdout:0/870: write d6/d2f/de2/f100 [970287,61877] 0 2024-04-25T01:36:17.947 INFO:tasks.workunit.client.1.smithi138.stdout:0/871: mkdir d6/d2f/d31/d125 0 2024-04-25T01:36:17.954 INFO:tasks.workunit.client.1.smithi138.stdout:0/872: dwrite d6/d19/dbb/fc5 [0,4194304] 0 2024-04-25T01:36:17.958 INFO:tasks.workunit.client.1.smithi138.stdout:0/873: mkdir d6/d39/d52/d9c/d126 0 2024-04-25T01:36:17.959 INFO:tasks.workunit.client.1.smithi138.stdout:0/874: chown d6/d1d/fa6 7575 1 2024-04-25T01:36:17.961 INFO:tasks.workunit.client.1.smithi138.stdout:0/875: write d6/d1d/d82/fad [2762321,63791] 0 2024-04-25T01:36:17.962 INFO:tasks.workunit.client.1.smithi138.stdout:0/876: fdatasync d6/d2f/d75/f76 0 2024-04-25T01:36:17.962 INFO:tasks.workunit.client.1.smithi138.stdout:0/877: read - d6/d2f/d75/f9f zero size 2024-04-25T01:36:18.004 INFO:tasks.workunit.client.1.smithi138.stdout:3/981: sync 2024-04-25T01:36:18.006 INFO:tasks.workunit.client.1.smithi138.stdout:3/982: creat db/f132 x:0 0 0 2024-04-25T01:36:18.007 INFO:tasks.workunit.client.1.smithi138.stdout:3/983: rmdir db/df/de2 39 2024-04-25T01:36:18.009 INFO:tasks.workunit.client.1.smithi138.stdout:3/984: unlink db/df/de2/d10a/d112/c95 0 2024-04-25T01:36:18.019 INFO:tasks.workunit.client.1.smithi138.stdout:3/985: dwrite db/dc5/fb2 [0,4194304] 0 2024-04-25T01:36:18.025 INFO:tasks.workunit.client.1.smithi138.stdout:3/986: truncate db/df/de2/d10a/d112/d40/d41/fa6 436068 0 2024-04-25T01:36:18.026 INFO:tasks.workunit.client.1.smithi138.stdout:3/987: dread - db/df/de2/fed zero size 2024-04-25T01:36:18.046 INFO:tasks.workunit.client.1.smithi138.stdout:3/988: dread db/df/d17/f5e [0,4194304] 0 2024-04-25T01:36:18.131 INFO:tasks.workunit.client.1.smithi138.stdout:1/948: sync 2024-04-25T01:36:18.131 INFO:tasks.workunit.client.1.smithi138.stdout:9/807: sync 2024-04-25T01:36:18.131 INFO:tasks.workunit.client.1.smithi138.stdout:4/824: sync 2024-04-25T01:36:18.131 INFO:tasks.workunit.client.1.smithi138.stdout:7/980: sync 2024-04-25T01:36:18.131 INFO:tasks.workunit.client.1.smithi138.stdout:5/930: sync 2024-04-25T01:36:18.132 INFO:tasks.workunit.client.1.smithi138.stdout:9/808: fdatasync d4/d1a/d68/da5/fef 0 2024-04-25T01:36:18.132 INFO:tasks.workunit.client.1.smithi138.stdout:1/949: write d3/ddb/dd6/f110 [290907,7580] 0 2024-04-25T01:36:18.135 INFO:tasks.workunit.client.1.smithi138.stdout:4/825: creat d0/d1c/d1e/d9f/ffb x:0 0 0 2024-04-25T01:36:18.136 INFO:tasks.workunit.client.1.smithi138.stdout:1/950: write d3/d45/d72/f5d [2713322,11029] 0 2024-04-25T01:36:18.137 INFO:tasks.workunit.client.1.smithi138.stdout:5/931: rename d4/da/d6e/f28 to d4/da/d6e/d22/d74/d8e/d133/f13a 0 2024-04-25T01:36:18.138 INFO:tasks.workunit.client.1.smithi138.stdout:7/981: dread da/de/d13/d30/f108 [0,4194304] 0 2024-04-25T01:36:18.141 INFO:tasks.workunit.client.1.smithi138.stdout:9/809: symlink d4/d1a/d3c/dc2/db2/l10c 0 2024-04-25T01:36:18.145 INFO:tasks.workunit.client.1.smithi138.stdout:7/982: symlink da/de/d1b/d49/d81/d73/db7/l147 0 2024-04-25T01:36:18.147 INFO:tasks.workunit.client.1.smithi138.stdout:1/951: mknod d3/c145 0 2024-04-25T01:36:18.147 INFO:tasks.workunit.client.1.smithi138.stdout:9/810: dread d4/d63/f8b [0,4194304] 0 2024-04-25T01:36:18.147 INFO:tasks.workunit.client.1.smithi138.stdout:4/826: fsync d0/d1c/d1e/d9f/d7e/dd5/d74/f8d 0 2024-04-25T01:36:18.148 INFO:tasks.workunit.client.1.smithi138.stdout:7/983: truncate da/de/d13/df7/f138 201560 0 2024-04-25T01:36:18.153 INFO:tasks.workunit.client.1.smithi138.stdout:7/984: write da/de/d13/d30/d59/d7b/d99/fb0 [3100334,25979] 0 2024-04-25T01:36:18.156 INFO:tasks.workunit.client.1.smithi138.stdout:5/932: getdents d4/d122/d90 0 2024-04-25T01:36:18.157 INFO:tasks.workunit.client.1.smithi138.stdout:7/985: fdatasync da/de/d1b/d49/f119 0 2024-04-25T01:36:18.158 INFO:tasks.workunit.client.1.smithi138.stdout:1/952: truncate d3/d45/d72/d57/f91 413186 0 2024-04-25T01:36:18.159 INFO:tasks.workunit.client.1.smithi138.stdout:4/827: dwrite d0/d1c/d1e/dbb/da3/db5/fc8 [0,4194304] 0 2024-04-25T01:36:18.161 INFO:tasks.workunit.client.1.smithi138.stdout:4/828: chown d0/d1c/d1e/d9f/d7e/de0/fec 1 1 2024-04-25T01:36:18.165 INFO:tasks.workunit.client.1.smithi138.stdout:7/986: write da/de/d13/f23 [6398023,124466] 0 2024-04-25T01:36:18.167 INFO:tasks.workunit.client.1.smithi138.stdout:1/953: creat d3/d6d/f146 x:0 0 0 2024-04-25T01:36:18.173 INFO:tasks.workunit.client.1.smithi138.stdout:5/933: dread d4/d122/dca/fd6 [0,4194304] 0 2024-04-25T01:36:18.174 INFO:tasks.workunit.client.1.smithi138.stdout:4/829: dwrite d0/d1c/d1e/d9f/d7e/de0/fec [0,4194304] 0 2024-04-25T01:36:18.185 INFO:tasks.workunit.client.1.smithi138.stdout:5/934: mkdir d4/d122/dca/d13b 0 2024-04-25T01:36:18.186 INFO:tasks.workunit.client.1.smithi138.stdout:5/935: readlink d4/d122/d5e/l7d 0 2024-04-25T01:36:18.186 INFO:tasks.workunit.client.1.smithi138.stdout:7/987: write da/de/d1b/d49/d81/dc7/fcb [488252,72286] 0 2024-04-25T01:36:18.188 INFO:tasks.workunit.client.1.smithi138.stdout:7/988: stat da/de/d1b/d49/d81/d51 0 2024-04-25T01:36:18.197 INFO:tasks.workunit.client.1.smithi138.stdout:3/989: sync 2024-04-25T01:36:18.197 INFO:tasks.workunit.client.1.smithi138.stdout:0/878: sync 2024-04-25T01:36:18.199 INFO:tasks.workunit.client.1.smithi138.stdout:0/879: write d6/d2f/fcb [853337,105379] 0 2024-04-25T01:36:18.208 INFO:tasks.workunit.client.1.smithi138.stdout:1/954: dread d3/d45/d72/f42 [0,4194304] 0 2024-04-25T01:36:18.215 INFO:tasks.workunit.client.1.smithi138.stdout:9/811: sync 2024-04-25T01:36:18.218 INFO:tasks.workunit.client.1.smithi138.stdout:4/830: unlink d0/d1c/d1e/d9f/d7e/fab 0 2024-04-25T01:36:18.223 INFO:tasks.workunit.client.1.smithi138.stdout:7/989: rename da/de/d13/d95/dd7/c122 to da/de/d1b/d49/d81/d51/d6e/da2/d102/d144/c148 0 2024-04-25T01:36:18.229 INFO:tasks.workunit.client.1.smithi138.stdout:4/831: mknod d0/d1c/d1e/d9f/de5/cfc 0 2024-04-25T01:36:18.231 INFO:tasks.workunit.client.1.smithi138.stdout:3/990: dread db/fc0 [0,4194304] 0 2024-04-25T01:36:18.236 INFO:tasks.workunit.client.1.smithi138.stdout:9/812: rename d4/d49/c7e to d4/d1a/d68/da5/c10d 0 2024-04-25T01:36:18.241 INFO:tasks.workunit.client.1.smithi138.stdout:1/955: creat d3/d27/d2d/da1/d139/d13a/f147 x:0 0 0 2024-04-25T01:36:18.244 INFO:tasks.workunit.client.1.smithi138.stdout:1/956: chown d3/d45/d72/d3d/c87 0 1 2024-04-25T01:36:18.245 INFO:tasks.workunit.client.1.smithi138.stdout:7/990: dwrite da/de/d13/d30/d59/f78 [0,4194304] 0 2024-04-25T01:36:18.247 INFO:tasks.workunit.client.1.smithi138.stdout:3/991: chown db/d9b/db4/d117 16758 1 2024-04-25T01:36:18.251 INFO:tasks.workunit.client.1.smithi138.stdout:4/832: mkdir d0/dfd 0 2024-04-25T01:36:18.255 INFO:tasks.workunit.client.1.smithi138.stdout:1/957: dread d3/ddb/dd6/f110 [0,4194304] 0 2024-04-25T01:36:18.257 INFO:tasks.workunit.client.1.smithi138.stdout:4/833: mkdir d0/d1c/d86/dfe 0 2024-04-25T01:36:18.260 INFO:tasks.workunit.client.1.smithi138.stdout:7/991: mknod da/de/d13/d21/dbc/dff/c149 0 2024-04-25T01:36:18.261 INFO:tasks.workunit.client.1.smithi138.stdout:7/992: fdatasync da/f18 0 2024-04-25T01:36:18.262 INFO:tasks.workunit.client.1.smithi138.stdout:1/958: rmdir d3/d4/da7 39 2024-04-25T01:36:18.263 INFO:tasks.workunit.client.1.smithi138.stdout:4/834: dread d0/d1c/d1e/dbb/da3/fac [0,4194304] 0 2024-04-25T01:36:18.265 INFO:tasks.workunit.client.1.smithi138.stdout:3/992: mkdir db/df/d19/d1a/de5/de7/d133 0 2024-04-25T01:36:18.265 INFO:tasks.workunit.client.1.smithi138.stdout:4/835: truncate d0/d1c/d1e/d9f/de5/def/ff7 624336 0 2024-04-25T01:36:18.278 INFO:tasks.workunit.client.1.smithi138.stdout:9/813: dread d4/d1a/f2d [0,4194304] 0 2024-04-25T01:36:18.279 INFO:tasks.workunit.client.1.smithi138.stdout:5/936: rename d4/da/d27/c2b to d4/da/d6e/d22/d74/c13c 0 2024-04-25T01:36:18.284 INFO:tasks.workunit.client.1.smithi138.stdout:7/993: truncate da/de/d13/f3c 930422 0 2024-04-25T01:36:18.286 INFO:tasks.workunit.client.1.smithi138.stdout:9/814: chown d4/d1a/d33/d9a/fe6 1 1 2024-04-25T01:36:18.287 INFO:tasks.workunit.client.1.smithi138.stdout:9/815: stat d4/d49/d72/laf 0 2024-04-25T01:36:18.287 INFO:tasks.workunit.client.1.smithi138.stdout:0/880: dwrite d6/df/d66/fc0 [0,4194304] 0 2024-04-25T01:36:18.294 INFO:tasks.workunit.client.1.smithi138.stdout:4/836: creat d0/dfd/fff x:0 0 0 2024-04-25T01:36:18.295 INFO:tasks.workunit.client.1.smithi138.stdout:7/994: mkdir da/de/d1b/d49/d81/d51/d6e/da2/d14a 0 2024-04-25T01:36:18.296 INFO:tasks.workunit.client.1.smithi138.stdout:9/816: readlink d4/d1a/d55/lb6 0 2024-04-25T01:36:18.297 INFO:tasks.workunit.client.1.smithi138.stdout:0/881: symlink d6/d2f/d31/ddc/l127 0 2024-04-25T01:36:18.298 INFO:tasks.workunit.client.1.smithi138.stdout:1/959: link d3/d27/d3e/fbf d3/d4/da7/db1/d124/f148 0 2024-04-25T01:36:18.299 INFO:tasks.workunit.client.1.smithi138.stdout:1/960: chown d3/ddb/dd6 15 1 2024-04-25T01:36:18.301 INFO:tasks.workunit.client.1.smithi138.stdout:3/993: rename db/d9b/lbb to db/df/d17/d23/l134 0 2024-04-25T01:36:18.302 INFO:tasks.workunit.client.1.smithi138.stdout:7/995: dwrite da/de/d1b/d34/fc2 [0,4194304] 0 2024-04-25T01:36:18.303 INFO:tasks.workunit.client.1.smithi138.stdout:4/837: rmdir d0/d1c/d1e/dbb/da3/db5 39 2024-04-25T01:36:18.306 INFO:tasks.workunit.client.1.smithi138.stdout:9/817: fsync d4/d1a/d55/f5f 0 2024-04-25T01:36:18.309 INFO:tasks.workunit.client.1.smithi138.stdout:1/961: mknod d3/d27/d16/da3/c149 0 2024-04-25T01:36:18.310 INFO:tasks.workunit.client.1.smithi138.stdout:3/994: fsync db/df/d17/d23/d34/f52 0 2024-04-25T01:36:18.317 INFO:tasks.workunit.client.1.smithi138.stdout:0/882: link d6/d1d/f5f d6/df/d3a/d11e/f128 0 2024-04-25T01:36:18.320 INFO:tasks.workunit.client.1.smithi138.stdout:1/962: write d3/ddb/dd6/f11b [2833763,54894] 0 2024-04-25T01:36:18.321 INFO:tasks.workunit.client.1.smithi138.stdout:5/937: write d4/da/d6e/d22/d35/dce/df9/dd4/ffd [3496798,23777] 0 2024-04-25T01:36:18.322 INFO:tasks.workunit.client.1.smithi138.stdout:1/963: readlink d3/l7a 0 2024-04-25T01:36:18.323 INFO:tasks.workunit.client.1.smithi138.stdout:3/995: write db/df/d19/f75 [12853104,16044] 0 2024-04-25T01:36:18.329 INFO:tasks.workunit.client.1.smithi138.stdout:1/964: dread d3/d27/d16/da3/daf/fc6 [0,4194304] 0 2024-04-25T01:36:18.330 INFO:tasks.workunit.client.1.smithi138.stdout:1/965: stat d3/d45/d72/d3d/f8d 0 2024-04-25T01:36:18.331 INFO:tasks.workunit.client.1.smithi138.stdout:1/966: truncate d3/ddb/ffe 267890 0 2024-04-25T01:36:18.333 INFO:tasks.workunit.client.1.smithi138.stdout:9/818: write d4/f22 [932143,46349] 0 2024-04-25T01:36:18.334 INFO:tasks.workunit.client.1.smithi138.stdout:0/883: dread d6/d19/dbb/fd6 [0,4194304] 0 2024-04-25T01:36:18.335 INFO:tasks.workunit.client.1.smithi138.stdout:0/884: stat d6/d2f/d31/f86 0 2024-04-25T01:36:18.335 INFO:tasks.workunit.client.1.smithi138.stdout:3/996: rename db/df/d19/d1a/de5/f109 to db/df/d19/d1a/de5/de7/d133/f135 0 2024-04-25T01:36:18.336 INFO:tasks.workunit.client.1.smithi138.stdout:0/885: fsync d6/d2f/fd3 0 2024-04-25T01:36:18.338 INFO:tasks.workunit.client.1.smithi138.stdout:1/967: rmdir d3/ddb/d9b 39 2024-04-25T01:36:18.339 INFO:tasks.workunit.client.1.smithi138.stdout:7/996: write da/de/d1b/d49/d81/f9c [4333269,71408] 0 2024-04-25T01:36:18.340 INFO:tasks.workunit.client.1.smithi138.stdout:5/938: write d4/d122/fab [916399,101925] 0 2024-04-25T01:36:18.341 INFO:tasks.workunit.client.1.smithi138.stdout:9/819: truncate d4/d1a/d3c/dc2/f9c 492784 0 2024-04-25T01:36:18.341 INFO:tasks.workunit.client.1.smithi138.stdout:1/968: stat d3/d45/d72/d57/db0/dcd 0 2024-04-25T01:36:18.342 INFO:tasks.workunit.client.1.smithi138.stdout:3/997: mknod db/df/d17/d20/d57/d6a/c136 0 2024-04-25T01:36:18.344 INFO:tasks.workunit.client.1.smithi138.stdout:1/969: chown d3/d27/d24 126 1 2024-04-25T01:36:18.346 INFO:tasks.workunit.client.1.smithi138.stdout:9/820: write d4/d57/fce [616603,46842] 0 2024-04-25T01:36:18.346 INFO:tasks.workunit.client.1.smithi138.stdout:0/886: dwrite d6/d39/d52/fba [0,4194304] 0 2024-04-25T01:36:18.347 INFO:tasks.workunit.client.1.smithi138.stdout:5/939: mknod d4/d122/d5e/c13d 0 2024-04-25T01:36:18.347 INFO:tasks.workunit.client.1.smithi138.stdout:1/970: readlink d3/d27/d16/l3a 0 2024-04-25T01:36:18.352 INFO:tasks.workunit.client.1.smithi138.stdout:4/838: dread d0/d1c/d1e/d9f/f44 [4194304,4194304] 0 2024-04-25T01:36:18.358 INFO:tasks.workunit.client.1.smithi138.stdout:5/940: dread d4/da/d6e/f11c [0,4194304] 0 2024-04-25T01:36:18.363 INFO:tasks.workunit.client.1.smithi138.stdout:9/821: link d4/d1a/d55/d7f/d8f/f101 d4/d1a/d33/d1d/f10e 0 2024-04-25T01:36:18.366 INFO:tasks.workunit.client.1.smithi138.stdout:7/997: dread da/de/d1b/d4b/d5b/f46 [0,4194304] 0 2024-04-25T01:36:18.366 INFO:tasks.workunit.client.1.smithi138.stdout:3/998: rename db/df/d17/d23 to db/df/d17/d137 0 2024-04-25T01:36:18.367 INFO:tasks.workunit.client.1.smithi138.stdout:0/887: truncate d6/df/d41/d49/f51 3953989 0 2024-04-25T01:36:18.368 INFO:tasks.workunit.client.1.smithi138.stdout:0/888: dread - d6/d2f/d75/f107 zero size 2024-04-25T01:36:18.370 INFO:tasks.workunit.client.1.smithi138.stdout:9/822: dread d4/d1a/f70 [0,4194304] 0 2024-04-25T01:36:18.371 INFO:tasks.workunit.client.1.smithi138.stdout:4/839: creat d0/f100 x:0 0 0 2024-04-25T01:36:18.374 INFO:tasks.workunit.client.1.smithi138.stdout:0/889: dread d6/df/d69/faa [0,4194304] 0 2024-04-25T01:36:18.375 INFO:tasks.workunit.client.1.smithi138.stdout:0/890: chown d6/df/lcc 0 1 2024-04-25T01:36:18.377 INFO:tasks.workunit.client.1.smithi138.stdout:0/891: chown d6/df/d41/d49/ddb/dec/f111 74044 1 2024-04-25T01:36:18.382 INFO:tasks.workunit.client.1.smithi138.stdout:0/892: dread d6/df/d3a/d67/d8a/ffe [0,4194304] 0 2024-04-25T01:36:18.385 INFO:tasks.workunit.client.1.smithi138.stdout:5/941: dread d4/d122/d5e/f75 [0,4194304] 0 2024-04-25T01:36:18.386 INFO:tasks.workunit.client.1.smithi138.stdout:0/893: chown d6/df/d3a/d67/c8c 4304751 1 2024-04-25T01:36:18.389 INFO:tasks.workunit.client.1.smithi138.stdout:9/823: dread d4/d1a/d68/dbe/ff1 [0,4194304] 0 2024-04-25T01:36:18.432 INFO:tasks.workunit.client.1.smithi138.stdout:1/971: truncate d3/d45/d72/d57/d8e/f69 3177643 0 2024-04-25T01:36:18.435 INFO:tasks.workunit.client.1.smithi138.stdout:5/942: mkdir d4/d122/dca/d13e 0 2024-04-25T01:36:18.436 INFO:tasks.workunit.client.1.smithi138.stdout:5/943: chown d4/da/d6e/d22/d35/dce/df9/d83/d89/d9d/d9e/d101 692007 1 2024-04-25T01:36:18.438 INFO:tasks.workunit.client.1.smithi138.stdout:9/824: creat d4/d8/d90/d8e/dd9/f10f x:0 0 0 2024-04-25T01:36:18.440 INFO:tasks.workunit.client.1.smithi138.stdout:7/998: symlink da/dd2/dcd/d109/l14b 0 2024-04-25T01:36:18.442 INFO:tasks.workunit.client.1.smithi138.stdout:7/999: stat da/de/d1b/d4b/d5b/c38 0 2024-04-25T01:36:18.447 INFO:tasks.workunit.client.1.smithi138.stdout:1/972: symlink d3/d27/d24/da9/l14a 0 2024-04-25T01:36:18.450 INFO:tasks.workunit.client.1.smithi138.stdout:0/894: getdents d6/df/d41/df1 0 2024-04-25T01:36:18.451 INFO:tasks.workunit.client.1.smithi138.stdout:0/895: chown d6/d39/d52/d9c/dbc 698 1 2024-04-25T01:36:18.451 INFO:tasks.workunit.client.1.smithi138.stdout:5/944: dread d4/f18 [0,4194304] 0 2024-04-25T01:36:18.453 INFO:tasks.workunit.client.1.smithi138.stdout:3/999: link db/df/de2/fec db/df/de2/d10a/d112/d12e/f138 0 2024-04-25T01:36:18.453 INFO:tasks.workunit.client.1.smithi138.stdout:5/945: read d4/da/d6e/d22/f98 [1251878,40492] 0 2024-04-25T01:36:18.457 INFO:tasks.workunit.client.1.smithi138.stdout:4/840: rename d0/d1c/d1e/f69 to d0/d1c/d1e/d9f/d7e/f101 0 2024-04-25T01:36:18.461 INFO:tasks.workunit.client.1.smithi138.stdout:5/946: dwrite d4/da/d6e/d22/d74/f76 [0,4194304] 0 2024-04-25T01:36:18.461 INFO:tasks.workunit.client.1.smithi138.stdout:0/896: write d6/d39/d52/ff9 [3735388,127090] 0 2024-04-25T01:36:18.462 INFO:tasks.workunit.client.1.smithi138.stdout:0/897: dread - d6/d39/f70 zero size 2024-04-25T01:36:18.472 INFO:tasks.workunit.client.1.smithi138.stdout:4/841: mknod d0/d1c/d1e/d9f/de5/df0/c102 0 2024-04-25T01:36:18.473 INFO:tasks.workunit.client.1.smithi138.stdout:1/973: dwrite d3/d45/d72/d57/d8e/d113/fc5 [0,4194304] 0 2024-04-25T01:36:18.476 INFO:tasks.workunit.client.1.smithi138.stdout:5/947: rename d4/da/d6e/d3c/d10e/c120 to d4/d122/d5e/d60/db5/c13f 0 2024-04-25T01:36:18.478 INFO:tasks.workunit.client.1.smithi138.stdout:0/898: getdents d6/df/d66 0 2024-04-25T01:36:18.479 INFO:tasks.workunit.client.1.smithi138.stdout:1/974: mkdir d3/dcc/d14b 0 2024-04-25T01:36:18.483 INFO:tasks.workunit.client.1.smithi138.stdout:0/899: creat d6/df/d41/d49/de4/f129 x:0 0 0 2024-04-25T01:36:18.485 INFO:tasks.workunit.client.1.smithi138.stdout:5/948: symlink d4/da/d6e/d22/d35/dce/df9/d83/d8a/l140 0 2024-04-25T01:36:18.487 INFO:tasks.workunit.client.1.smithi138.stdout:4/842: symlink d0/d1c/d1e/dbb/dae/de6/l103 0 2024-04-25T01:36:18.489 INFO:tasks.workunit.client.1.smithi138.stdout:1/975: symlink d3/d45/d72/d57/d8e/db6/l14c 0 2024-04-25T01:36:18.490 INFO:tasks.workunit.client.1.smithi138.stdout:1/976: readlink d3/d4/da7/db1/de1/lf6 0 2024-04-25T01:36:18.491 INFO:tasks.workunit.client.1.smithi138.stdout:0/900: dwrite d6/d19/f77 [0,4194304] 0 2024-04-25T01:36:18.494 INFO:tasks.workunit.client.1.smithi138.stdout:5/949: link d4/d122/f66 d4/d122/d5e/d60/dbd/dc8/dbb/f141 0 2024-04-25T01:36:18.497 INFO:tasks.workunit.client.1.smithi138.stdout:1/977: unlink d3/ddb/dd6/dde/ff3 0 2024-04-25T01:36:18.497 INFO:tasks.workunit.client.1.smithi138.stdout:1/978: read - d3/d6d/f146 zero size 2024-04-25T01:36:18.500 INFO:tasks.workunit.client.1.smithi138.stdout:4/843: creat d0/d1c/d1e/d9f/d7e/dd5/f104 x:0 0 0 2024-04-25T01:36:18.502 INFO:tasks.workunit.client.1.smithi138.stdout:5/950: write d4/da/d6e/d22/f98 [434511,17942] 0 2024-04-25T01:36:18.507 INFO:tasks.workunit.client.1.smithi138.stdout:1/979: creat d3/ddb/dd6/d12d/f14d x:0 0 0 2024-04-25T01:36:18.507 INFO:tasks.workunit.client.1.smithi138.stdout:1/980: dread - d3/d27/d2d/fe5 zero size 2024-04-25T01:36:18.510 INFO:tasks.workunit.client.1.smithi138.stdout:0/901: dread d6/df/d3a/fae [0,4194304] 0 2024-04-25T01:36:18.512 INFO:tasks.workunit.client.1.smithi138.stdout:1/981: symlink d3/d27/d16/da3/daf/ded/l14e 0 2024-04-25T01:36:18.516 INFO:tasks.workunit.client.1.smithi138.stdout:0/902: readlink d6/la 0 2024-04-25T01:36:18.520 INFO:tasks.workunit.client.1.smithi138.stdout:0/903: dread d6/f26 [0,4194304] 0 2024-04-25T01:36:18.521 INFO:tasks.workunit.client.1.smithi138.stdout:1/982: truncate d3/d27/d24/da9/f11d 944836 0 2024-04-25T01:36:18.525 INFO:tasks.workunit.client.1.smithi138.stdout:1/983: creat d3/ddb/dd6/dde/d117/f14f x:0 0 0 2024-04-25T01:36:18.526 INFO:tasks.workunit.client.1.smithi138.stdout:1/984: readlink d3/d4/da7/db1/de1/lf6 0 2024-04-25T01:36:18.527 INFO:tasks.workunit.client.1.smithi138.stdout:1/985: chown d3/d4/da7/db1/cc3 162629 1 2024-04-25T01:36:18.529 INFO:tasks.workunit.client.1.smithi138.stdout:1/986: write d3/d45/d72/d57/d8e/d113/daa/fd8 [3788794,92451] 0 2024-04-25T01:36:18.531 INFO:tasks.workunit.client.1.smithi138.stdout:1/987: truncate d3/d45/d72/d57/d8e/db6/fdf 4922857 0 2024-04-25T01:36:18.535 INFO:tasks.workunit.client.1.smithi138.stdout:1/988: link d3/d45/d72/d57/d8e/d113/daa/cee d3/d27/d2d/da1/c150 0 2024-04-25T01:36:18.543 INFO:tasks.workunit.client.1.smithi138.stdout:9/825: rmdir d4/d8/d90/d8e 39 2024-04-25T01:36:18.543 INFO:tasks.workunit.client.1.smithi138.stdout:1/989: dwrite d3/d45/d72/d57/f63 [12582912,4194304] 0 2024-04-25T01:36:18.548 INFO:tasks.workunit.client.1.smithi138.stdout:9/826: chown d4/d8/d90/d52/l97 29812 1 2024-04-25T01:36:18.553 INFO:tasks.workunit.client.1.smithi138.stdout:1/990: dwrite d3/d27/d16/d10a/f134 [4194304,4194304] 0 2024-04-25T01:36:18.556 INFO:tasks.workunit.client.1.smithi138.stdout:9/827: unlink d4/d1a/d33/c48 0 2024-04-25T01:36:18.557 INFO:tasks.workunit.client.1.smithi138.stdout:9/828: fdatasync d4/d1a/d68/da0/fab 0 2024-04-25T01:36:18.573 INFO:tasks.workunit.client.1.smithi138.stdout:4/844: write d0/d1c/d1e/d9f/d7e/dd5/faa [2451847,128919] 0 2024-04-25T01:36:18.574 INFO:tasks.workunit.client.1.smithi138.stdout:4/845: write d0/d1c/d1e/fd7 [5075089,28147] 0 2024-04-25T01:36:18.576 INFO:tasks.workunit.client.1.smithi138.stdout:5/951: write d4/d122/d5e/d8f/f138 [5097454,106937] 0 2024-04-25T01:36:18.577 INFO:tasks.workunit.client.1.smithi138.stdout:5/952: read - d4/d122/d5e/d8f/d9f/de8/f130 zero size 2024-04-25T01:36:18.579 INFO:tasks.workunit.client.1.smithi138.stdout:4/846: rename d0/d1c/d1e/dbb/dae to d0/d1c/d86/dd9/d105 0 2024-04-25T01:36:18.589 INFO:tasks.workunit.client.1.smithi138.stdout:9/829: truncate d4/d1a/d55/f5c 301200 0 2024-04-25T01:36:18.592 INFO:tasks.workunit.client.1.smithi138.stdout:5/953: dread d4/d122/d5e/d60/db5/fdd [0,4194304] 0 2024-04-25T01:36:18.592 INFO:tasks.workunit.client.1.smithi138.stdout:5/954: readlink d4/da/l5f 0 2024-04-25T01:36:18.595 INFO:tasks.workunit.client.1.smithi138.stdout:5/955: creat d4/d122/d5e/d60/db5/f142 x:0 0 0 2024-04-25T01:36:18.600 INFO:tasks.workunit.client.1.smithi138.stdout:9/830: truncate d4/f22 2507971 0 2024-04-25T01:36:18.602 INFO:tasks.workunit.client.1.smithi138.stdout:5/956: truncate d4/da/d27/f2d 166075 0 2024-04-25T01:36:18.602 INFO:tasks.workunit.client.1.smithi138.stdout:9/831: creat d4/d49/d72/f110 x:0 0 0 2024-04-25T01:36:18.609 INFO:tasks.workunit.client.1.smithi138.stdout:5/957: dwrite d4/da/d6e/d22/d35/dce/df9/dd4/f116 [0,4194304] 0 2024-04-25T01:36:18.618 INFO:tasks.workunit.client.1.smithi138.stdout:5/958: dwrite d4/da/d6e/d22/d45/f126 [0,4194304] 0 2024-04-25T01:36:18.639 INFO:tasks.workunit.client.1.smithi138.stdout:9/832: dread d4/d63/dcf/fe7 [0,4194304] 0 2024-04-25T01:36:18.640 INFO:tasks.workunit.client.1.smithi138.stdout:9/833: write d4/d1a/d3c/dc2/db2/ff3 [933292,64209] 0 2024-04-25T01:36:18.642 INFO:tasks.workunit.client.1.smithi138.stdout:9/834: fdatasync d4/d57/f67 0 2024-04-25T01:36:18.646 INFO:tasks.workunit.client.1.smithi138.stdout:9/835: rename d4/d1a/d33/f37 to d4/d8/d90/d52/dfa/f111 0 2024-04-25T01:36:18.647 INFO:tasks.workunit.client.1.smithi138.stdout:9/836: chown d4/d1a/d68/dbe 28993 1 2024-04-25T01:36:18.657 INFO:tasks.workunit.client.1.smithi138.stdout:9/837: dread f1 [0,4194304] 0 2024-04-25T01:36:18.658 INFO:tasks.workunit.client.1.smithi138.stdout:9/838: chown d4/d1a/d3c/dc2/db2/dd8 11317872 1 2024-04-25T01:36:18.659 INFO:tasks.workunit.client.1.smithi138.stdout:9/839: mkdir d4/d8/d112 0 2024-04-25T01:36:18.661 INFO:tasks.workunit.client.1.smithi138.stdout:9/840: fdatasync d4/d63/dcf/fe7 0 2024-04-25T01:36:18.663 INFO:tasks.workunit.client.1.smithi138.stdout:9/841: creat d4/d1a/f113 x:0 0 0 2024-04-25T01:36:18.665 INFO:tasks.workunit.client.1.smithi138.stdout:9/842: truncate d4/d1a/d33/d1d/f10e 367060 0 2024-04-25T01:36:18.666 INFO:tasks.workunit.client.1.smithi138.stdout:9/843: creat d4/d1a/d3c/dc2/db2/f114 x:0 0 0 2024-04-25T01:36:18.667 INFO:tasks.workunit.client.1.smithi138.stdout:9/844: creat d4/d57/f115 x:0 0 0 2024-04-25T01:36:18.668 INFO:tasks.workunit.client.1.smithi138.stdout:9/845: truncate d4/d49/d72/ffc 311014 0 2024-04-25T01:36:18.671 INFO:tasks.workunit.client.1.smithi138.stdout:9/846: link d4/d1a/d3c/dc2/ff8 d4/d8/d2c/d82/f116 0 2024-04-25T01:36:18.672 INFO:tasks.workunit.client.1.smithi138.stdout:9/847: fsync d4/d1a/d68/f98 0 2024-04-25T01:36:18.675 INFO:tasks.workunit.client.1.smithi138.stdout:9/848: mkdir d4/d8/de3/d117 0 2024-04-25T01:36:18.677 INFO:tasks.workunit.client.1.smithi138.stdout:9/849: mknod d4/d8/d90/d52/c118 0 2024-04-25T01:36:18.678 INFO:tasks.workunit.client.1.smithi138.stdout:9/850: read - d4/d1a/d55/d7f/ffe zero size 2024-04-25T01:36:18.679 INFO:tasks.workunit.client.1.smithi138.stdout:9/851: truncate d4/d63/d86/fcb 640140 0 2024-04-25T01:36:18.681 INFO:tasks.workunit.client.1.smithi138.stdout:9/852: creat d4/d1a/f119 x:0 0 0 2024-04-25T01:36:18.682 INFO:tasks.workunit.client.1.smithi138.stdout:9/853: fdatasync d4/d57/fdf 0 2024-04-25T01:36:18.684 INFO:tasks.workunit.client.1.smithi138.stdout:9/854: rmdir d4/d1a/d55/d7f 39 2024-04-25T01:36:18.684 INFO:tasks.workunit.client.1.smithi138.stdout:9/855: stat d4/d8/d90/d52/fc0 0 2024-04-25T01:36:18.686 INFO:tasks.workunit.client.1.smithi138.stdout:9/856: chown d4/d63/d86/fcb 8134 1 2024-04-25T01:36:18.689 INFO:tasks.workunit.client.1.smithi138.stdout:9/857: symlink d4/d1a/d55/d7f/l11a 0 2024-04-25T01:36:18.691 INFO:tasks.workunit.client.1.smithi138.stdout:9/858: unlink d4/d1a/d3c/dc2/db2/f114 0 2024-04-25T01:36:18.692 INFO:tasks.workunit.client.1.smithi138.stdout:9/859: mknod d4/d8/d2c/d82/c11b 0 2024-04-25T01:36:18.693 INFO:tasks.workunit.client.1.smithi138.stdout:9/860: dread - d4/d1a/d68/da0/fab zero size 2024-04-25T01:36:18.694 INFO:tasks.workunit.client.1.smithi138.stdout:9/861: rmdir d4/d1a/d55/d7f 39 2024-04-25T01:36:18.701 INFO:tasks.workunit.client.1.smithi138.stdout:9/862: dwrite d4/d1a/d55/f5f [0,4194304] 0 2024-04-25T01:36:18.999 INFO:tasks.workunit.client.1.smithi138.stdout:0/904: sync 2024-04-25T01:36:19.001 INFO:tasks.workunit.client.1.smithi138.stdout:0/905: symlink d6/d19/dbb/l12a 0 2024-04-25T01:36:19.001 INFO:tasks.workunit.client.1.smithi138.stdout:0/906: stat d6/d1d 0 2024-04-25T01:36:19.006 INFO:tasks.workunit.client.1.smithi138.stdout:0/907: dwrite d6/df/d69/f6a [0,4194304] 0 2024-04-25T01:36:19.008 INFO:tasks.workunit.client.1.smithi138.stdout:0/908: write d6/d19/dbb/fd6 [3287497,80886] 0 2024-04-25T01:36:19.014 INFO:tasks.workunit.client.1.smithi138.stdout:0/909: dread d6/d39/d58/f87 [0,4194304] 0 2024-04-25T01:36:19.019 INFO:tasks.workunit.client.1.smithi138.stdout:0/910: dwrite d6/d2f/fe8 [0,4194304] 0 2024-04-25T01:36:19.022 INFO:tasks.workunit.client.1.smithi138.stdout:0/911: symlink d6/df/d41/l12b 0 2024-04-25T01:36:19.023 INFO:tasks.workunit.client.1.smithi138.stdout:0/912: write d6/df/d69/fd4 [30190,35179] 0 2024-04-25T01:36:19.029 INFO:tasks.workunit.client.1.smithi138.stdout:0/913: dwrite d6/d1d/d82/fad [0,4194304] 0 2024-04-25T01:36:19.032 INFO:tasks.workunit.client.1.smithi138.stdout:0/914: creat d6/d39/d52/d9c/f12c x:0 0 0 2024-04-25T01:36:19.042 INFO:tasks.workunit.client.1.smithi138.stdout:0/915: dwrite d6/d2f/d75/f107 [0,4194304] 0 2024-04-25T01:36:19.053 INFO:tasks.workunit.client.1.smithi138.stdout:9/863: sync 2024-04-25T01:36:19.054 INFO:tasks.workunit.client.1.smithi138.stdout:1/991: sync 2024-04-25T01:36:19.054 INFO:tasks.workunit.client.1.smithi138.stdout:4/847: sync 2024-04-25T01:36:19.054 INFO:tasks.workunit.client.1.smithi138.stdout:5/959: sync 2024-04-25T01:36:19.054 INFO:tasks.workunit.client.1.smithi138.stdout:1/992: chown d3/d45/f106 635 1 2024-04-25T01:36:19.056 INFO:tasks.workunit.client.1.smithi138.stdout:1/993: write d3/d27/d2d/f32 [2238831,31507] 0 2024-04-25T01:36:19.056 INFO:tasks.workunit.client.1.smithi138.stdout:5/960: symlink d4/d122/d5e/d60/dbd/dc8/l143 0 2024-04-25T01:36:19.059 INFO:tasks.workunit.client.1.smithi138.stdout:9/864: creat d4/d8/d90/d52/ddd/f11c x:0 0 0 2024-04-25T01:36:19.064 INFO:tasks.workunit.client.1.smithi138.stdout:0/916: write d6/d2f/f5a [1591595,72529] 0 2024-04-25T01:36:19.065 INFO:tasks.workunit.client.1.smithi138.stdout:5/961: fdatasync d4/da/d27/f2d 0 2024-04-25T01:36:19.067 INFO:tasks.workunit.client.1.smithi138.stdout:5/962: truncate d4/da/d27/f33 1751249 0 2024-04-25T01:36:19.067 INFO:tasks.workunit.client.1.smithi138.stdout:1/994: creat d3/f151 x:0 0 0 2024-04-25T01:36:19.068 INFO:tasks.workunit.client.1.smithi138.stdout:4/848: getdents d0/d1c/d1e/d9f/de5/df0 0 2024-04-25T01:36:19.069 INFO:tasks.workunit.client.1.smithi138.stdout:9/865: write d4/d1a/d33/f16 [622408,21913] 0 2024-04-25T01:36:19.070 INFO:tasks.workunit.client.1.smithi138.stdout:0/917: dread - d6/df/d41/d49/f8b zero size 2024-04-25T01:36:19.073 INFO:tasks.workunit.client.1.smithi138.stdout:5/963: mkdir d4/d122/d5e/d60/dbd/dc8/dbb/d139/d144 0 2024-04-25T01:36:19.073 INFO:tasks.workunit.client.1.smithi138.stdout:0/918: write d6/d39/d52/fda [808940,23925] 0 2024-04-25T01:36:19.074 INFO:tasks.workunit.client.1.smithi138.stdout:1/995: mkdir d3/d4/da7/db1/d124/d152 0 2024-04-25T01:36:19.077 INFO:tasks.workunit.client.1.smithi138.stdout:9/866: dwrite d4/d1a/fe4 [0,4194304] 0 2024-04-25T01:36:19.077 INFO:tasks.workunit.client.1.smithi138.stdout:0/919: symlink d6/d19/dbb/l12d 0 2024-04-25T01:36:19.079 INFO:tasks.workunit.client.1.smithi138.stdout:1/996: mkdir d3/d45/d72/d57/d8e/d113/d153 0 2024-04-25T01:36:19.080 INFO:tasks.workunit.client.1.smithi138.stdout:1/997: chown d3/d6d/c83 0 1 2024-04-25T01:36:19.080 INFO:tasks.workunit.client.1.smithi138.stdout:5/964: symlink d4/d122/d5e/d60/dbd/dc8/dbb/d139/l145 0 2024-04-25T01:36:19.082 INFO:tasks.workunit.client.1.smithi138.stdout:5/965: write d4/da/fd [2904662,99111] 0 2024-04-25T01:36:19.083 INFO:tasks.workunit.client.1.smithi138.stdout:9/867: creat d4/d1a/d3c/dc2/db2/f11d x:0 0 0 2024-04-25T01:36:19.085 INFO:tasks.workunit.client.1.smithi138.stdout:0/920: dwrite d6/fe [4194304,4194304] 0 2024-04-25T01:36:19.087 INFO:tasks.workunit.client.1.smithi138.stdout:4/849: truncate d0/d1c/d1e/dbb/f66 3015561 0 2024-04-25T01:36:19.088 INFO:tasks.workunit.client.1.smithi138.stdout:0/921: truncate d6/d39/fde 782948 0 2024-04-25T01:36:19.091 INFO:tasks.workunit.client.1.smithi138.stdout:5/966: read d4/da/d6e/d22/f24 [279647,104680] 0 2024-04-25T01:36:19.092 INFO:tasks.workunit.client.1.smithi138.stdout:9/868: truncate d4/d1a/f81 977111 0 2024-04-25T01:36:19.093 INFO:tasks.workunit.client.1.smithi138.stdout:0/922: creat d6/d1d/d82/dc6/f12e x:0 0 0 2024-04-25T01:36:19.095 INFO:tasks.workunit.client.1.smithi138.stdout:1/998: write d3/d45/d72/d3d/f8d [2189544,85338] 0 2024-04-25T01:36:19.096 INFO:tasks.workunit.client.1.smithi138.stdout:9/869: chown d4/d1a/d33/d9a/fea 5578411 1 2024-04-25T01:36:19.101 INFO:tasks.workunit.client.1.smithi138.stdout:9/870: mkdir d4/d1a/d33/d1d/d11e 0 2024-04-25T01:36:19.101 INFO:tasks.workunit.client.1.smithi138.stdout:5/967: link d4/da/d6e/ld3 d4/da/d6e/d22/d109/l146 0 2024-04-25T01:36:19.102 INFO:tasks.workunit.client.1.smithi138.stdout:0/923: getdents d6/df/d41/d49/ddb/dec 0 2024-04-25T01:36:19.103 INFO:tasks.workunit.client.1.smithi138.stdout:0/924: write d6/d2f/d75/db6/de5/ff7 [1716404,75749] 0 2024-04-25T01:36:19.104 INFO:tasks.workunit.client.1.smithi138.stdout:9/871: creat d4/d63/d86/f11f x:0 0 0 2024-04-25T01:36:19.107 INFO:tasks.workunit.client.1.smithi138.stdout:4/850: dread d0/f64 [0,4194304] 0 2024-04-25T01:36:19.109 INFO:tasks.workunit.client.1.smithi138.stdout:4/851: dread - d0/d1c/d1e/d9f/d7e/dd5/f6d zero size 2024-04-25T01:36:19.110 INFO:tasks.workunit.client.1.smithi138.stdout:0/925: rename d6/d2f/f5a to d6/d39/d58/f12f 0 2024-04-25T01:36:19.112 INFO:tasks.workunit.client.1.smithi138.stdout:9/872: unlink d4/d1a/d55/l79 0 2024-04-25T01:36:19.113 INFO:tasks.workunit.client.1.smithi138.stdout:5/968: creat d4/da/f147 x:0 0 0 2024-04-25T01:36:19.118 INFO:tasks.workunit.client.1.smithi138.stdout:0/926: write d6/d39/d43/f21 [2545009,7914] 0 2024-04-25T01:36:19.120 INFO:tasks.workunit.client.1.smithi138.stdout:9/873: chown d4/d1a/f81 4 1 2024-04-25T01:36:19.121 INFO:tasks.workunit.client.1.smithi138.stdout:1/999: dwrite d3/d6d/f89 [0,4194304] 0 2024-04-25T01:36:19.121 INFO:tasks.workunit.client.1.smithi138.stdout:4/852: mkdir d0/d1c/d1e/d9f/d106 0 2024-04-25T01:36:19.121 INFO:tasks.workunit.client.1.smithi138.stdout:9/874: chown d4/d1a/d33/f4d 227 1 2024-04-25T01:36:19.122 INFO:tasks.workunit.client.1.smithi138.stdout:4/853: readlink d0/d1c/d1e/lda 0 2024-04-25T01:36:19.125 INFO:tasks.workunit.client.1.smithi138.stdout:5/969: mknod d4/da/d6e/d22/d35/dce/df9/d7f/db3/d12d/c148 0 2024-04-25T01:36:19.127 INFO:tasks.workunit.client.1.smithi138.stdout:9/875: dread - d4/d1a/d55/d7f/f108 zero size 2024-04-25T01:36:19.129 INFO:tasks.workunit.client.1.smithi138.stdout:0/927: creat d6/d19/dbb/d10c/f130 x:0 0 0 2024-04-25T01:36:19.131 INFO:tasks.workunit.client.1.smithi138.stdout:4/854: creat d0/d1c/d1e/d9f/d33/f107 x:0 0 0 2024-04-25T01:36:19.132 INFO:tasks.workunit.client.1.smithi138.stdout:9/876: rmdir d4/d1a/d3c/dc2/db2/dd8 39 2024-04-25T01:36:19.134 INFO:tasks.workunit.client.1.smithi138.stdout:4/855: readlink d0/d1c/d86/ldc 0 2024-04-25T01:36:19.138 INFO:tasks.workunit.client.1.smithi138.stdout:9/877: mkdir d4/d8/d90/d8e/d120 0 2024-04-25T01:36:19.140 INFO:tasks.workunit.client.1.smithi138.stdout:4/856: link d0/d1c/d1e/d9f/f29 d0/d1c/d1e/d9f/d106/f108 0 2024-04-25T01:36:19.144 INFO:tasks.workunit.client.1.smithi138.stdout:0/928: rmdir d6/d19 39 2024-04-25T01:36:19.145 INFO:tasks.workunit.client.1.smithi138.stdout:5/970: write d4/da/d27/f1f [4107552,122133] 0 2024-04-25T01:36:19.148 INFO:tasks.workunit.client.1.smithi138.stdout:5/971: creat d4/da/d6e/d22/d35/f149 x:0 0 0 2024-04-25T01:36:19.151 INFO:tasks.workunit.client.1.smithi138.stdout:5/972: symlink d4/da/d6e/d22/d35/l14a 0 2024-04-25T01:36:19.153 INFO:tasks.workunit.client.1.smithi138.stdout:4/857: write d0/d1c/d1e/d9f/f44 [2769517,24213] 0 2024-04-25T01:36:19.154 INFO:tasks.workunit.client.1.smithi138.stdout:9/878: unlink d4/d8/d90/d8e/c105 0 2024-04-25T01:36:19.154 INFO:tasks.workunit.client.1.smithi138.stdout:5/973: rename d4/d122/dca to d4/d122/dca/d13e/d14b 22 2024-04-25T01:36:19.157 INFO:tasks.workunit.client.1.smithi138.stdout:9/879: write d4/d1a/d68/da0/fab [692847,18256] 0 2024-04-25T01:36:19.157 INFO:tasks.workunit.client.1.smithi138.stdout:4/858: mknod d0/d1c/d1e/d9f/d7e/de0/c109 0 2024-04-25T01:36:19.157 INFO:tasks.workunit.client.1.smithi138.stdout:0/929: dread d6/d1d/f24 [0,4194304] 0 2024-04-25T01:36:19.159 INFO:tasks.workunit.client.1.smithi138.stdout:9/880: chown d4/d8/d90/ld0 7431675 1 2024-04-25T01:36:19.161 INFO:tasks.workunit.client.1.smithi138.stdout:0/930: mknod d6/c131 0 2024-04-25T01:36:19.162 INFO:tasks.workunit.client.1.smithi138.stdout:9/881: creat d4/d49/d72/f121 x:0 0 0 2024-04-25T01:36:19.163 INFO:tasks.workunit.client.1.smithi138.stdout:0/931: write d6/d39/fdf [5227918,13935] 0 2024-04-25T01:36:19.165 INFO:tasks.workunit.client.1.smithi138.stdout:9/882: chown d4/d8/d2c/l34 1056 1 2024-04-25T01:36:19.166 INFO:tasks.workunit.client.1.smithi138.stdout:4/859: unlink d0/d1c/d86/dd9/d105/dc6/c5f 0 2024-04-25T01:36:19.168 INFO:tasks.workunit.client.1.smithi138.stdout:5/974: link d4/da/d6e/d5c/f95 d4/da/d6e/d22/d35/dce/df9/d83/f14c 0 2024-04-25T01:36:19.174 INFO:tasks.workunit.client.1.smithi138.stdout:4/860: dread d0/d1c/d1e/dbb/f8b [0,4194304] 0 2024-04-25T01:36:19.174 INFO:tasks.workunit.client.1.smithi138.stdout:5/975: symlink d4/da/d6e/d22/d74/d8e/dd7/l14d 0 2024-04-25T01:36:19.179 INFO:tasks.workunit.client.1.smithi138.stdout:5/976: chown d4/da/d6e/d22/d35/dce/df9/d83/d89/d9d/d9e/d101/d137/cee 0 1 2024-04-25T01:36:19.189 INFO:tasks.workunit.client.1.smithi138.stdout:4/861: dwrite d0/d1c/d1e/d9f/d7e/dd5/f53 [4194304,4194304] 0 2024-04-25T01:36:19.192 INFO:tasks.workunit.client.1.smithi138.stdout:0/932: truncate d6/df/d69/f6a 3200905 0 2024-04-25T01:36:19.199 INFO:tasks.workunit.client.1.smithi138.stdout:9/883: dwrite d4/d8/d2c/d82/f116 [0,4194304] 0 2024-04-25T01:36:19.219 INFO:tasks.workunit.client.1.smithi138.stdout:0/933: creat d6/df/d41/d49/de4/f132 x:0 0 0 2024-04-25T01:36:19.222 INFO:tasks.workunit.client.1.smithi138.stdout:9/884: rename d4/d63/d86 to d4/d63/d122 0 2024-04-25T01:36:19.226 INFO:tasks.workunit.client.1.smithi138.stdout:0/934: rename f1 to d6/d39/d52/d9c/dbc/dbf/f133 0 2024-04-25T01:36:19.232 INFO:tasks.workunit.client.1.smithi138.stdout:9/885: mknod d4/d8/c123 0 2024-04-25T01:36:19.233 INFO:tasks.workunit.client.1.smithi138.stdout:9/886: dread - d4/d1a/d33/d9a/fe6 zero size 2024-04-25T01:36:19.235 INFO:tasks.workunit.client.1.smithi138.stdout:0/935: dread d6/d19/f47 [0,4194304] 0 2024-04-25T01:36:19.236 INFO:tasks.workunit.client.1.smithi138.stdout:9/887: dread d4/d1a/d68/fa7 [0,4194304] 0 2024-04-25T01:36:19.237 INFO:tasks.workunit.client.1.smithi138.stdout:0/936: readlink d6/l2e 0 2024-04-25T01:36:19.238 INFO:tasks.workunit.client.1.smithi138.stdout:0/937: read d6/d2f/d75/f107 [859964,80419] 0 2024-04-25T01:36:19.243 INFO:tasks.workunit.client.1.smithi138.stdout:9/888: mknod d4/d8/d90/d8e/dd9/c124 0 2024-04-25T01:36:19.247 INFO:tasks.workunit.client.1.smithi138.stdout:9/889: readlink d4/d8/d90/lc3 0 2024-04-25T01:36:19.248 INFO:tasks.workunit.client.1.smithi138.stdout:9/890: chown d4/d8/d2c/d82/f116 58498490 1 2024-04-25T01:36:19.250 INFO:tasks.workunit.client.1.smithi138.stdout:0/938: creat d6/df/d66/d119/f134 x:0 0 0 2024-04-25T01:36:19.256 INFO:tasks.workunit.client.1.smithi138.stdout:5/977: truncate d4/da/d27/f33 728725 0 2024-04-25T01:36:19.257 INFO:tasks.workunit.client.1.smithi138.stdout:5/978: dread - d4/d122/d5e/d8f/fb2 zero size 2024-04-25T01:36:19.263 INFO:tasks.workunit.client.1.smithi138.stdout:4/862: write d0/d1c/d1e/d9f/d7e/dd5/f53 [5052254,40223] 0 2024-04-25T01:36:19.271 INFO:tasks.workunit.client.1.smithi138.stdout:4/863: dread - d0/d1c/d1e/d9f/d33/f9b zero size 2024-04-25T01:36:19.272 INFO:tasks.workunit.client.1.smithi138.stdout:5/979: creat d4/da/d27/f14e x:0 0 0 2024-04-25T01:36:19.273 INFO:tasks.workunit.client.1.smithi138.stdout:5/980: dread - d4/d122/d90/fc6 zero size 2024-04-25T01:36:19.274 INFO:tasks.workunit.client.1.smithi138.stdout:5/981: write d4/d122/d90/dba/fc9 [4410424,34328] 0 2024-04-25T01:36:19.275 INFO:tasks.workunit.client.1.smithi138.stdout:5/982: chown d4/d122/d5e/d60/dbd 14 1 2024-04-25T01:36:19.277 INFO:tasks.workunit.client.1.smithi138.stdout:4/864: mkdir d0/d1c/d1e/d9f/d7e/dd5/d10a 0 2024-04-25T01:36:19.281 INFO:tasks.workunit.client.1.smithi138.stdout:5/983: fdatasync d4/da/d6e/d22/d74/d8e/d133/f13a 0 2024-04-25T01:36:19.288 INFO:tasks.workunit.client.1.smithi138.stdout:5/984: dread d4/d122/d5e/f7a [0,4194304] 0 2024-04-25T01:36:19.289 INFO:tasks.workunit.client.1.smithi138.stdout:9/891: dwrite d4/d1a/f20 [0,4194304] 0 2024-04-25T01:36:19.291 INFO:tasks.workunit.client.1.smithi138.stdout:9/892: rmdir d4/d1a/d68 39 2024-04-25T01:36:19.292 INFO:tasks.workunit.client.1.smithi138.stdout:9/893: chown d4/d49/d72/ffc 31809220 1 2024-04-25T01:36:19.299 INFO:tasks.workunit.client.1.smithi138.stdout:5/985: dwrite d4/d122/d5e/d60/db5/fdd [0,4194304] 0 2024-04-25T01:36:19.301 INFO:tasks.workunit.client.1.smithi138.stdout:4/865: dread d0/d1c/d1e/d9f/d7e/f101 [0,4194304] 0 2024-04-25T01:36:19.302 INFO:tasks.workunit.client.1.smithi138.stdout:9/894: write d4/d8/f10b [1132248,116808] 0 2024-04-25T01:36:19.303 INFO:tasks.workunit.client.1.smithi138.stdout:9/895: chown d4/d49/d72/f121 65444937 1 2024-04-25T01:36:19.307 INFO:tasks.workunit.client.1.smithi138.stdout:5/986: creat d4/da/d6e/d22/d35/dce/d115/f14f x:0 0 0 2024-04-25T01:36:19.312 INFO:tasks.workunit.client.1.smithi138.stdout:4/866: rename d0/d1c/d1e/d9f/fc4 to d0/d1c/f10b 0 2024-04-25T01:36:19.314 INFO:tasks.workunit.client.1.smithi138.stdout:9/896: dwrite d4/d8/d2c/da3/fd4 [0,4194304] 0 2024-04-25T01:36:19.315 INFO:tasks.workunit.client.1.smithi138.stdout:5/987: getdents d4/da/d6e/d22/d45 0 2024-04-25T01:36:19.317 INFO:tasks.workunit.client.1.smithi138.stdout:9/897: rmdir d4/d57 39 2024-04-25T01:36:19.319 INFO:tasks.workunit.client.1.smithi138.stdout:4/867: dread d0/d1c/d1e/d9f/d106/f108 [0,4194304] 0 2024-04-25T01:36:19.319 INFO:tasks.workunit.client.1.smithi138.stdout:5/988: creat d4/da/d6e/d3c/d10e/f150 x:0 0 0 2024-04-25T01:36:19.320 INFO:tasks.workunit.client.1.smithi138.stdout:5/989: read - d4/d122/dca/f118 zero size 2024-04-25T01:36:19.323 INFO:tasks.workunit.client.1.smithi138.stdout:4/868: creat d0/d1c/d86/dfe/f10c x:0 0 0 2024-04-25T01:36:19.329 INFO:tasks.workunit.client.1.smithi138.stdout:5/990: link d4/l7e d4/da/d6e/d22/d109/d135/l151 0 2024-04-25T01:36:19.330 INFO:tasks.workunit.client.1.smithi138.stdout:4/869: truncate d0/d1c/d1e/dbb/da3/fac 327314 0 2024-04-25T01:36:19.331 INFO:tasks.workunit.client.1.smithi138.stdout:5/991: symlink d4/da/d6e/d22/d35/dce/d115/l152 0 2024-04-25T01:36:19.333 INFO:tasks.workunit.client.1.smithi138.stdout:5/992: mkdir d4/da/d6e/d3c/d153 0 2024-04-25T01:36:19.335 INFO:tasks.workunit.client.1.smithi138.stdout:9/898: write d4/d1a/f84 [2772442,26933] 0 2024-04-25T01:36:19.336 INFO:tasks.workunit.client.1.smithi138.stdout:5/993: rmdir d4/da/d6e/d22/d35/dce/df9/d83/dcd 39 2024-04-25T01:36:19.338 INFO:tasks.workunit.client.1.smithi138.stdout:4/870: dwrite d0/d1c/d1e/d9f/d33/f9b [0,4194304] 0 2024-04-25T01:36:19.345 INFO:tasks.workunit.client.1.smithi138.stdout:5/994: rename d4/da/cf2 to d4/d122/d5e/c154 0 2024-04-25T01:36:19.345 INFO:tasks.workunit.client.1.smithi138.stdout:9/899: dwrite d4/d49/f6f [0,4194304] 0 2024-04-25T01:36:19.348 INFO:tasks.workunit.client.1.smithi138.stdout:4/871: link d0/d1c/d86/dd9/d105/dc6/fd1 d0/d1c/d86/f10d 0 2024-04-25T01:36:19.349 INFO:tasks.workunit.client.1.smithi138.stdout:5/995: mknod d4/da/d6e/d22/d74/d8e/dd8/c155 0 2024-04-25T01:36:19.350 INFO:tasks.workunit.client.1.smithi138.stdout:9/900: creat d4/d63/dcf/f125 x:0 0 0 2024-04-25T01:36:19.351 INFO:tasks.workunit.client.1.smithi138.stdout:5/996: chown d4/da/d6e/d22/d35/dce/d115/f14f 141 1 2024-04-25T01:36:19.353 INFO:tasks.workunit.client.1.smithi138.stdout:5/997: rmdir d4/d122/d5e/d60/dbd/dc8 39 2024-04-25T01:36:19.354 INFO:tasks.workunit.client.1.smithi138.stdout:4/872: getdents d0/d1c/d1e/d9f/d106 0 2024-04-25T01:36:19.355 INFO:tasks.workunit.client.1.smithi138.stdout:5/998: truncate d4/d122/dca/f118 273375 0 2024-04-25T01:36:19.356 INFO:tasks.workunit.client.1.smithi138.stdout:4/873: rmdir d0/d1c/d86/dd9 39 2024-04-25T01:36:19.358 INFO:tasks.workunit.client.1.smithi138.stdout:5/999: creat d4/da/d6e/d22/d45/df4/f156 x:0 0 0 2024-04-25T01:36:19.362 INFO:tasks.workunit.client.1.smithi138.stdout:9/901: write d4/d1a/f74 [1208139,86061] 0 2024-04-25T01:36:19.366 INFO:tasks.workunit.client.1.smithi138.stdout:9/902: rename d4/d63/d122 to d4/d1a/d33/d126 0 2024-04-25T01:36:19.368 INFO:tasks.workunit.client.1.smithi138.stdout:9/903: rmdir d4/d8/d90/d8e/dd9 39 2024-04-25T01:36:19.369 INFO:tasks.workunit.client.1.smithi138.stdout:9/904: mknod d4/d8/d2c/d82/c127 0 2024-04-25T01:36:19.370 INFO:tasks.workunit.client.1.smithi138.stdout:9/905: write d4/d1a/d33/f16 [243544,85798] 0 2024-04-25T01:36:19.372 INFO:tasks.workunit.client.1.smithi138.stdout:9/906: fdatasync d4/d8/f5b 0 2024-04-25T01:36:19.374 INFO:tasks.workunit.client.1.smithi138.stdout:9/907: mkdir d4/d1a/d33/d1d/d128 0 2024-04-25T01:36:19.381 INFO:tasks.workunit.client.1.smithi138.stdout:9/908: dwrite d4/d1a/d33/d9a/fe6 [0,4194304] 0 2024-04-25T01:36:19.381 INFO:tasks.workunit.client.1.smithi138.stdout:4/874: dread d0/d1c/d48/ddf/fee [0,4194304] 0 2024-04-25T01:36:19.382 INFO:tasks.workunit.client.1.smithi138.stdout:9/909: fdatasync d4/d8/d2c/da3/fba 0 2024-04-25T01:36:19.384 INFO:tasks.workunit.client.1.smithi138.stdout:9/910: chown d4/d1a/d33/d126/ffb 503019 1 2024-04-25T01:36:19.386 INFO:tasks.workunit.client.1.smithi138.stdout:4/875: mkdir d0/d1c/d86/dd9/d105/de6/d10e 0 2024-04-25T01:36:19.387 INFO:tasks.workunit.client.1.smithi138.stdout:4/876: dread - d0/d1c/d1e/dbb/fa1 zero size 2024-04-25T01:36:19.388 INFO:tasks.workunit.client.1.smithi138.stdout:9/911: mknod d4/d8/d112/c129 0 2024-04-25T01:36:19.388 INFO:tasks.workunit.client.1.smithi138.stdout:4/877: chown d0/d1c/d1e/d9f/f29 0 1 2024-04-25T01:36:19.389 INFO:tasks.workunit.client.1.smithi138.stdout:4/878: fsync d0/d1c/d1e/dbb/fa1 0 2024-04-25T01:36:19.390 INFO:tasks.workunit.client.1.smithi138.stdout:9/912: symlink d4/d1a/d33/d1d/d128/l12a 0 2024-04-25T01:36:19.393 INFO:tasks.workunit.client.1.smithi138.stdout:4/879: link d0/d1c/d1e/d9f/d33/la2 d0/d59/l10f 0 2024-04-25T01:36:19.394 INFO:tasks.workunit.client.1.smithi138.stdout:9/913: creat d4/d1a/d3c/dc2/db2/f12b x:0 0 0 2024-04-25T01:36:19.398 INFO:tasks.workunit.client.1.smithi138.stdout:9/914: dread d4/d1a/fe4 [0,4194304] 0 2024-04-25T01:36:19.399 INFO:tasks.workunit.client.1.smithi138.stdout:9/915: dread - d4/d1a/d3c/dc2/db2/f12b zero size 2024-04-25T01:36:19.403 INFO:tasks.workunit.client.1.smithi138.stdout:9/916: rename d4/d1a/d55/d7f/d8f/f99 to d4/d49/d72/f12c 0 2024-04-25T01:36:19.405 INFO:tasks.workunit.client.1.smithi138.stdout:9/917: creat d4/d1a/d33/d1d/d128/f12d x:0 0 0 2024-04-25T01:36:19.406 INFO:tasks.workunit.client.1.smithi138.stdout:4/880: write d0/d1c/d1e/d9f/d33/f51 [4505876,96546] 0 2024-04-25T01:36:19.408 INFO:tasks.workunit.client.1.smithi138.stdout:9/918: symlink d4/d8/d90/d52/dfa/l12e 0 2024-04-25T01:36:19.409 INFO:tasks.workunit.client.1.smithi138.stdout:4/881: creat d0/d59/f110 x:0 0 0 2024-04-25T01:36:19.411 INFO:tasks.workunit.client.1.smithi138.stdout:9/919: symlink d4/d8/d90/d8e/l12f 0 2024-04-25T01:36:19.412 INFO:tasks.workunit.client.1.smithi138.stdout:4/882: fsync d0/d1c/d1e/fd7 0 2024-04-25T01:36:19.414 INFO:tasks.workunit.client.1.smithi138.stdout:9/920: creat d4/d1a/d55/d7f/d8f/f130 x:0 0 0 2024-04-25T01:36:19.417 INFO:tasks.workunit.client.1.smithi138.stdout:4/883: creat d0/f111 x:0 0 0 2024-04-25T01:36:19.421 INFO:tasks.workunit.client.1.smithi138.stdout:9/921: dwrite d4/d1a/d33/d9a/fea [0,4194304] 0 2024-04-25T01:36:19.422 INFO:tasks.workunit.client.1.smithi138.stdout:4/884: rename d0/d1c/d1e/fd7 to d0/d1c/d1e/d9f/d7e/dd5/d10a/f112 0 2024-04-25T01:36:19.423 INFO:tasks.workunit.client.1.smithi138.stdout:9/922: write d4/d1a/d3c/dc2/db2/ff3 [1006542,81932] 0 2024-04-25T01:36:19.424 INFO:tasks.workunit.client.1.smithi138.stdout:9/923: readlink d4/d1a/d33/l69 0 2024-04-25T01:36:19.431 INFO:tasks.workunit.client.1.smithi138.stdout:9/924: fsync d4/d1a/f56 0 2024-04-25T01:36:19.434 INFO:tasks.workunit.client.1.smithi138.stdout:9/925: rename d4/d1a/f104 to d4/d49/f131 0 2024-04-25T01:36:19.435 INFO:tasks.workunit.client.1.smithi138.stdout:9/926: dread - d4/d63/d73/fc4 zero size 2024-04-25T01:36:19.436 INFO:tasks.workunit.client.1.smithi138.stdout:9/927: read d4/d1a/f20 [1401915,10871] 0 2024-04-25T01:36:19.438 INFO:tasks.workunit.client.1.smithi138.stdout:4/885: write d0/d1c/f10b [11618996,77935] 0 2024-04-25T01:36:19.442 INFO:tasks.workunit.client.1.smithi138.stdout:4/886: mkdir d0/d1c/d1e/d9f/d7e/dd5/d10a/d113 0 2024-04-25T01:36:19.443 INFO:tasks.workunit.client.1.smithi138.stdout:9/928: dwrite d4/f26 [0,4194304] 0 2024-04-25T01:36:19.450 INFO:tasks.workunit.client.1.smithi138.stdout:4/887: truncate d0/d1c/d1e/d9f/d33/f9b 2678733 0 2024-04-25T01:36:19.452 INFO:tasks.workunit.client.1.smithi138.stdout:4/888: unlink d0/f100 0 2024-04-25T01:36:19.453 INFO:tasks.workunit.client.1.smithi138.stdout:4/889: symlink d0/d1c/d1e/d9f/d7e/dd5/d10a/l114 0 2024-04-25T01:36:19.455 INFO:tasks.workunit.client.1.smithi138.stdout:4/890: dread - d0/d1c/d1e/d9f/ffb zero size 2024-04-25T01:36:19.455 INFO:tasks.workunit.client.1.smithi138.stdout:4/891: fsync d0/d1c/d1e/d9f/d33/f99 0 2024-04-25T01:36:19.456 INFO:tasks.workunit.client.1.smithi138.stdout:4/892: dread - d0/d59/fe1 zero size 2024-04-25T01:36:19.461 INFO:tasks.workunit.client.1.smithi138.stdout:4/893: dwrite d0/d1c/f6b [0,4194304] 0 2024-04-25T01:36:19.462 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:19 smithi033 ceph-mon[26353]: pgmap v25: 65 pgs: 65 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 61 MiB/s rd, 148 MiB/s wr, 386 op/s 2024-04-25T01:36:19.464 INFO:tasks.workunit.client.1.smithi138.stdout:4/894: link d0/f40 d0/d1c/d1e/d9f/de5/def/f115 0 2024-04-25T01:36:19.466 INFO:tasks.workunit.client.1.smithi138.stdout:4/895: creat d0/d1c/d1e/d9f/d7e/dd5/d74/f116 x:0 0 0 2024-04-25T01:36:19.611 INFO:tasks.workunit.client.1.smithi138.stdout:0/939: sync 2024-04-25T01:36:19.612 INFO:tasks.workunit.client.1.smithi138.stdout:0/940: mkdir d6/d1d/d82/d135 0 2024-04-25T01:36:19.614 INFO:tasks.workunit.client.1.smithi138.stdout:0/941: truncate d6/d2f/d31/fb2 3559893 0 2024-04-25T01:36:19.617 INFO:tasks.workunit.client.1.smithi138.stdout:0/942: getdents d6/df/d66/d119 0 2024-04-25T01:36:19.624 INFO:tasks.workunit.client.1.smithi138.stdout:0/943: dwrite d6/d1d/f24 [0,4194304] 0 2024-04-25T01:36:19.725 INFO:tasks.workunit.client.1.smithi138.stdout:4/896: sync 2024-04-25T01:36:19.725 INFO:tasks.workunit.client.1.smithi138.stdout:9/929: sync 2024-04-25T01:36:19.729 INFO:tasks.workunit.client.1.smithi138.stdout:9/930: dwrite d4/d1a/d55/f5f [0,4194304] 0 2024-04-25T01:36:19.734 INFO:tasks.workunit.client.1.smithi138.stdout:9/931: dread d4/f6c [0,4194304] 0 2024-04-25T01:36:19.736 INFO:tasks.workunit.client.1.smithi138.stdout:9/932: symlink d4/d8/d90/d8e/l132 0 2024-04-25T01:36:19.737 INFO:tasks.workunit.client.1.smithi138.stdout:9/933: chown d4/d1a/d33/d1d/cc5 1121 1 2024-04-25T01:36:19.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:19 smithi138 ceph-mon[32977]: pgmap v25: 65 pgs: 65 active+clean; 3.0 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 61 MiB/s rd, 148 MiB/s wr, 386 op/s 2024-04-25T01:36:19.798 INFO:tasks.workunit.client.1.smithi138.stdout:9/934: dread d4/f38 [0,4194304] 0 2024-04-25T01:36:19.799 INFO:tasks.workunit.client.1.smithi138.stdout:9/935: write d4/d1a/d33/d126/fe5 [700847,60272] 0 2024-04-25T01:36:19.810 INFO:tasks.workunit.client.1.smithi138.stdout:9/936: dwrite d4/d1a/d3c/f51 [0,4194304] 0 2024-04-25T01:36:19.814 INFO:tasks.workunit.client.1.smithi138.stdout:9/937: rename d4/d8/d2c/da3/fba to d4/d63/d73/f133 0 2024-04-25T01:36:19.817 INFO:tasks.workunit.client.1.smithi138.stdout:9/938: mknod d4/d1a/d55/d7f/c134 0 2024-04-25T01:36:19.819 INFO:tasks.workunit.client.1.smithi138.stdout:9/939: mknod d4/d8/d90/d52/ddd/c135 0 2024-04-25T01:36:19.821 INFO:tasks.workunit.client.1.smithi138.stdout:9/940: creat d4/d8/de3/f136 x:0 0 0 2024-04-25T01:36:19.823 INFO:tasks.workunit.client.1.smithi138.stdout:9/941: fsync d4/d49/f76 0 2024-04-25T01:36:19.824 INFO:tasks.workunit.client.1.smithi138.stdout:0/944: sync 2024-04-25T01:36:19.826 INFO:tasks.workunit.client.1.smithi138.stdout:9/942: creat d4/d8/d2c/da3/f137 x:0 0 0 2024-04-25T01:36:19.829 INFO:tasks.workunit.client.1.smithi138.stdout:0/945: fsync d6/d39/d58/f11b 0 2024-04-25T01:36:19.832 INFO:tasks.workunit.client.1.smithi138.stdout:0/946: mkdir d6/d19/dbb/d10c/d136 0 2024-04-25T01:36:19.833 INFO:tasks.workunit.client.1.smithi138.stdout:9/943: rename d4/d8/d2c/l41 to d4/d8/d90/d52/l138 0 2024-04-25T01:36:19.834 INFO:tasks.workunit.client.1.smithi138.stdout:0/947: mknod d6/df/d3a/d67/c137 0 2024-04-25T01:36:19.834 INFO:tasks.workunit.client.1.smithi138.stdout:9/944: write d4/d1a/d3c/dc2/db2/f12b [407585,61511] 0 2024-04-25T01:36:19.838 INFO:tasks.workunit.client.1.smithi138.stdout:9/945: mknod d4/d1a/d55/d7f/d8f/df7/c139 0 2024-04-25T01:36:19.840 INFO:tasks.workunit.client.1.smithi138.stdout:9/946: write d4/d1a/d33/d9a/fe6 [4409422,111307] 0 2024-04-25T01:36:19.841 INFO:tasks.workunit.client.1.smithi138.stdout:0/948: dwrite d6/d2f/fd3 [0,4194304] 0 2024-04-25T01:36:19.842 INFO:tasks.workunit.client.1.smithi138.stdout:0/949: fsync d6/d19/dbb/fd6 0 2024-04-25T01:36:19.843 INFO:tasks.workunit.client.1.smithi138.stdout:0/950: fdatasync d6/d39/fdf 0 2024-04-25T01:36:19.845 INFO:tasks.workunit.client.1.smithi138.stdout:9/947: symlink d4/d1a/d68/dbe/l13a 0 2024-04-25T01:36:19.845 INFO:tasks.workunit.client.1.smithi138.stdout:0/951: dread - d6/df/d41/d49/de4/f132 zero size 2024-04-25T01:36:19.846 INFO:tasks.workunit.client.1.smithi138.stdout:9/948: fsync d4/d8/d2c/da3/f137 0 2024-04-25T01:36:19.848 INFO:tasks.workunit.client.1.smithi138.stdout:0/952: rmdir d6/df/d69 39 2024-04-25T01:36:19.854 INFO:tasks.workunit.client.1.smithi138.stdout:9/949: dwrite d4/d8/d2c/d82/de8/ff0 [0,4194304] 0 2024-04-25T01:36:19.856 INFO:tasks.workunit.client.1.smithi138.stdout:0/953: dread d6/df/d69/f6a [0,4194304] 0 2024-04-25T01:36:19.857 INFO:tasks.workunit.client.1.smithi138.stdout:0/954: chown d6/df/d66/ca5 796628 1 2024-04-25T01:36:19.859 INFO:tasks.workunit.client.1.smithi138.stdout:4/897: sync 2024-04-25T01:36:19.860 INFO:tasks.workunit.client.1.smithi138.stdout:0/955: creat d6/d39/d52/d9c/dbc/dbf/f138 x:0 0 0 2024-04-25T01:36:19.862 INFO:tasks.workunit.client.1.smithi138.stdout:0/956: read d6/d19/dbb/fd6 [3036310,13879] 0 2024-04-25T01:36:19.863 INFO:tasks.workunit.client.1.smithi138.stdout:0/957: write d6/df/d41/d49/de4/f132 [457118,16454] 0 2024-04-25T01:36:19.865 INFO:tasks.workunit.client.1.smithi138.stdout:0/958: write d6/df/f2c [1395036,108417] 0 2024-04-25T01:36:19.866 INFO:tasks.workunit.client.1.smithi138.stdout:9/950: write d4/d1a/d68/fa7 [1404111,95586] 0 2024-04-25T01:36:19.867 INFO:tasks.workunit.client.1.smithi138.stdout:0/959: write d6/d39/d52/fba [208117,32562] 0 2024-04-25T01:36:19.871 INFO:tasks.workunit.client.1.smithi138.stdout:0/960: rename d6/l99 to d6/df/d41/d49/ddb/dec/l139 0 2024-04-25T01:36:19.872 INFO:tasks.workunit.client.1.smithi138.stdout:9/951: write d4/d1a/f56 [795673,108602] 0 2024-04-25T01:36:19.872 INFO:tasks.workunit.client.1.smithi138.stdout:0/961: chown d6/d1d/d82/d135 1552 1 2024-04-25T01:36:19.874 INFO:tasks.workunit.client.1.smithi138.stdout:9/952: write d4/d1a/d3c/f9d [1357035,58700] 0 2024-04-25T01:36:19.881 INFO:tasks.workunit.client.1.smithi138.stdout:9/953: link d4/d8/d90/d8e/le9 d4/d1a/dc1/l13b 0 2024-04-25T01:36:19.884 INFO:tasks.workunit.client.1.smithi138.stdout:9/954: symlink d4/d8/d112/l13c 0 2024-04-25T01:36:19.885 INFO:tasks.workunit.client.1.smithi138.stdout:0/962: dwrite d6/d1d/f5f [4194304,4194304] 0 2024-04-25T01:36:19.886 INFO:tasks.workunit.client.1.smithi138.stdout:9/955: mkdir d4/d49/d72/d13d 0 2024-04-25T01:36:19.893 INFO:tasks.workunit.client.1.smithi138.stdout:4/898: dread d0/d1c/d86/dd9/d105/dc6/f8e [0,4194304] 0 2024-04-25T01:36:19.895 INFO:tasks.workunit.client.1.smithi138.stdout:0/963: dwrite d6/d2f/fa1 [0,4194304] 0 2024-04-25T01:36:19.898 INFO:tasks.workunit.client.1.smithi138.stdout:9/956: dwrite d4/d8/d90/d52/ddd/f11c [0,4194304] 0 2024-04-25T01:36:19.903 INFO:tasks.workunit.client.1.smithi138.stdout:9/957: dread f1 [0,4194304] 0 2024-04-25T01:36:19.906 INFO:tasks.workunit.client.1.smithi138.stdout:0/964: truncate d6/d39/d43/f6b 6393159 0 2024-04-25T01:36:19.909 INFO:tasks.workunit.client.1.smithi138.stdout:9/958: getdents d4/d1a/d33/d1d/d128 0 2024-04-25T01:36:19.909 INFO:tasks.workunit.client.1.smithi138.stdout:9/959: dread - d4/d8/d2c/ff9 zero size 2024-04-25T01:36:19.910 INFO:tasks.workunit.client.1.smithi138.stdout:9/960: readlink d4/d1a/d33/l69 0 2024-04-25T01:36:19.913 INFO:tasks.workunit.client.1.smithi138.stdout:9/961: creat d4/d1a/d3c/dc2/db2/dd8/f13e x:0 0 0 2024-04-25T01:36:19.919 INFO:tasks.workunit.client.1.smithi138.stdout:9/962: dwrite d4/d8/d2c/d82/f116 [0,4194304] 0 2024-04-25T01:36:19.919 INFO:tasks.workunit.client.1.smithi138.stdout:9/963: chown d4/d8/d90/d52/l103 2 1 2024-04-25T01:36:19.923 INFO:tasks.workunit.client.1.smithi138.stdout:9/964: creat d4/d1a/d3c/f13f x:0 0 0 2024-04-25T01:36:19.926 INFO:tasks.workunit.client.1.smithi138.stdout:9/965: rmdir d4/d57 39 2024-04-25T01:36:19.928 INFO:tasks.workunit.client.1.smithi138.stdout:9/966: write d4/d57/fdf [595,1654] 0 2024-04-25T01:36:19.937 INFO:tasks.workunit.client.1.smithi138.stdout:0/965: dread d6/d39/d43/f9a [0,4194304] 0 2024-04-25T01:36:19.939 INFO:tasks.workunit.client.1.smithi138.stdout:0/966: rename d6/d2f/lc2 to d6/d2f/de2/l13a 0 2024-04-25T01:36:19.940 INFO:tasks.workunit.client.1.smithi138.stdout:0/967: symlink d6/d1d/d82/dc6/l13b 0 2024-04-25T01:36:19.942 INFO:tasks.workunit.client.1.smithi138.stdout:0/968: creat d6/df/d3a/d67/d8a/f13c x:0 0 0 2024-04-25T01:36:19.949 INFO:tasks.workunit.client.1.smithi138.stdout:9/967: dread d4/d1a/f21 [0,4194304] 0 2024-04-25T01:36:19.951 INFO:tasks.workunit.client.1.smithi138.stdout:9/968: rmdir d4/d8/d2c 39 2024-04-25T01:36:19.955 INFO:tasks.workunit.client.1.smithi138.stdout:0/969: dread d6/d39/d43/f21 [0,4194304] 0 2024-04-25T01:36:19.957 INFO:tasks.workunit.client.1.smithi138.stdout:9/969: write d4/d1a/d33/f4d [2941777,64668] 0 2024-04-25T01:36:19.958 INFO:tasks.workunit.client.1.smithi138.stdout:9/970: chown d4/d1a/d33/l7a 20 1 2024-04-25T01:36:19.958 INFO:tasks.workunit.client.1.smithi138.stdout:0/970: dread d6/d2f/f7e [0,4194304] 0 2024-04-25T01:36:19.960 INFO:tasks.workunit.client.1.smithi138.stdout:9/971: symlink d4/d1a/d33/d1d/l140 0 2024-04-25T01:36:19.962 INFO:tasks.workunit.client.1.smithi138.stdout:9/972: write d4/d1a/d3c/dc2/db2/ff3 [494360,87685] 0 2024-04-25T01:36:19.962 INFO:tasks.workunit.client.1.smithi138.stdout:0/971: rename d6/d2f/d31/c95 to d6/c13d 0 2024-04-25T01:36:19.971 INFO:tasks.workunit.client.1.smithi138.stdout:9/973: dwrite d4/d1a/d33/d9a/fc7 [0,4194304] 0 2024-04-25T01:36:19.978 INFO:tasks.workunit.client.1.smithi138.stdout:9/974: dwrite d4/d1a/d33/f16 [0,4194304] 0 2024-04-25T01:36:19.980 INFO:tasks.workunit.client.1.smithi138.stdout:9/975: symlink d4/d1a/d33/l141 0 2024-04-25T01:36:19.983 INFO:tasks.workunit.client.1.smithi138.stdout:9/976: mkdir d4/d8/d142 0 2024-04-25T01:36:19.983 INFO:tasks.workunit.client.1.smithi138.stdout:9/977: rmdir d4/d8/d90/d52 39 2024-04-25T01:36:19.985 INFO:tasks.workunit.client.1.smithi138.stdout:9/978: write d4/d8/d2c/d82/f107 [1524456,26631] 0 2024-04-25T01:36:19.991 INFO:tasks.workunit.client.1.smithi138.stdout:9/979: rename d4/d8/d2c/l34 to d4/d8/d2c/d82/l143 0 2024-04-25T01:36:19.991 INFO:tasks.workunit.client.1.smithi138.stdout:9/980: chown d4/d8/lbc 2884184 1 2024-04-25T01:36:19.991 INFO:tasks.workunit.client.1.smithi138.stdout:9/981: fsync d4/d63/f8b 0 2024-04-25T01:36:19.992 INFO:tasks.workunit.client.1.smithi138.stdout:9/982: unlink d4/d1a/d55/d7f/d8f/c102 0 2024-04-25T01:36:19.993 INFO:tasks.workunit.client.1.smithi138.stdout:9/983: rmdir d4/d1a/dc1 39 2024-04-25T01:36:19.995 INFO:tasks.workunit.client.1.smithi138.stdout:9/984: write d4/d1a/f21 [5242607,72941] 0 2024-04-25T01:36:19.996 INFO:tasks.workunit.client.1.smithi138.stdout:9/985: readlink d4/d8/l40 0 2024-04-25T01:36:19.997 INFO:tasks.workunit.client.1.smithi138.stdout:9/986: chown d4/d8/d90/d52/dfa/f111 0 1 2024-04-25T01:36:19.998 INFO:tasks.workunit.client.1.smithi138.stdout:9/987: write d4/d8/f10b [237652,30471] 0 2024-04-25T01:36:20.001 INFO:tasks.workunit.client.1.smithi138.stdout:9/988: read d4/d8/f61 [763864,11417] 0 2024-04-25T01:36:20.003 INFO:tasks.workunit.client.1.smithi138.stdout:9/989: creat d4/d8/d90/d8e/d120/f144 x:0 0 0 2024-04-25T01:36:20.008 INFO:tasks.workunit.client.1.smithi138.stdout:9/990: write d4/d57/f5a [832090,55498] 0 2024-04-25T01:36:20.011 INFO:tasks.workunit.client.1.smithi138.stdout:9/991: read d4/d1a/d3c/dc2/f9c [408122,54199] 0 2024-04-25T01:36:20.013 INFO:tasks.workunit.client.1.smithi138.stdout:9/992: rename d4/d1a/d55/d7f/d8f/df7 to d4/d1a/d55/d7f/d8f/d145 0 2024-04-25T01:36:20.014 INFO:tasks.workunit.client.1.smithi138.stdout:9/993: write d4/d1a/d33/d1d/d128/f12d [940972,78158] 0 2024-04-25T01:36:20.016 INFO:tasks.workunit.client.1.smithi138.stdout:9/994: creat d4/d8/d2c/d82/de8/f146 x:0 0 0 2024-04-25T01:36:20.018 INFO:tasks.workunit.client.1.smithi138.stdout:9/995: symlink d4/d1a/d3c/dc2/db2/l147 0 2024-04-25T01:36:20.019 INFO:tasks.workunit.client.1.smithi138.stdout:9/996: dread d4/f6c [0,4194304] 0 2024-04-25T01:36:20.022 INFO:tasks.workunit.client.1.smithi138.stdout:9/997: rename d4/d49/f76 to d4/d8/d112/f148 0 2024-04-25T01:36:20.024 INFO:tasks.workunit.client.1.smithi138.stdout:9/998: dread d4/d1a/fe4 [0,4194304] 0 2024-04-25T01:36:20.026 INFO:tasks.workunit.client.1.smithi138.stdout:9/999: creat d4/d1a/dc1/f149 x:0 0 0 2024-04-25T01:36:20.151 INFO:tasks.workunit.client.1.smithi138.stdout:4/899: sync 2024-04-25T01:36:20.154 INFO:tasks.workunit.client.1.smithi138.stdout:4/900: getdents d0/d1c/d48 0 2024-04-25T01:36:20.157 INFO:tasks.workunit.client.1.smithi138.stdout:4/901: rename d0/d1c/d1e/d9f/fa7 to d0/d1c/d1e/d9f/d7e/dd5/d10a/f117 0 2024-04-25T01:36:20.163 INFO:tasks.workunit.client.1.smithi138.stdout:4/902: dwrite d0/d1c/d48/f92 [0,4194304] 0 2024-04-25T01:36:20.164 INFO:tasks.workunit.client.1.smithi138.stdout:4/903: read d0/f64 [340702,80300] 0 2024-04-25T01:36:20.167 INFO:tasks.workunit.client.1.smithi138.stdout:4/904: unlink d0/d1c/lf2 0 2024-04-25T01:36:20.170 INFO:tasks.workunit.client.1.smithi138.stdout:4/905: mkdir d0/d1c/d1e/d9f/de5/def/d118 0 2024-04-25T01:36:20.174 INFO:tasks.workunit.client.1.smithi138.stdout:4/906: rmdir d0/d1c/d1e/dbb/da3/db5 39 2024-04-25T01:36:20.177 INFO:tasks.workunit.client.1.smithi138.stdout:4/907: symlink d0/l119 0 2024-04-25T01:36:20.177 INFO:tasks.workunit.client.1.smithi138.stdout:4/908: stat d0/d1c/d86/dd9/d105/dc6/f8e 0 2024-04-25T01:36:20.179 INFO:tasks.workunit.client.1.smithi138.stdout:4/909: truncate d0/d1c/d1e/d9f/d7e/dd5/d74/f116 795950 0 2024-04-25T01:36:20.181 INFO:tasks.workunit.client.1.smithi138.stdout:4/910: symlink d0/d1c/d1e/l11a 0 2024-04-25T01:36:20.185 INFO:tasks.workunit.client.1.smithi138.stdout:4/911: creat d0/d1c/d1e/d9f/d7e/dd5/f11b x:0 0 0 2024-04-25T01:36:20.186 INFO:tasks.workunit.client.1.smithi138.stdout:4/912: read - d0/d59/fe1 zero size 2024-04-25T01:36:20.195 INFO:tasks.workunit.client.1.smithi138.stdout:0/972: sync 2024-04-25T01:36:20.195 INFO:tasks.workunit.client.1.smithi138.stdout:4/913: symlink d0/d1c/d86/dd9/d105/de6/l11c 0 2024-04-25T01:36:20.203 INFO:tasks.workunit.client.1.smithi138.stdout:0/973: dwrite d6/d19/f109 [0,4194304] 0 2024-04-25T01:36:20.269 INFO:tasks.workunit.client.1.smithi138.stdout:0/974: creat d6/d39/d52/db7/d10e/f13e x:0 0 0 2024-04-25T01:36:20.269 INFO:tasks.workunit.client.1.smithi138.stdout:4/914: creat d0/d59/dbc/ded/f11d x:0 0 0 2024-04-25T01:36:20.275 INFO:tasks.workunit.client.1.smithi138.stdout:0/975: creat d6/d39/d58/d11c/f13f x:0 0 0 2024-04-25T01:36:20.276 INFO:tasks.workunit.client.1.smithi138.stdout:4/915: symlink d0/d1c/d86/dd9/d105/dc6/l11e 0 2024-04-25T01:36:20.277 INFO:tasks.workunit.client.1.smithi138.stdout:0/976: creat d6/d2f/de2/f140 x:0 0 0 2024-04-25T01:36:20.277 INFO:tasks.workunit.client.1.smithi138.stdout:0/977: chown d6/df/d3a/d11e/f128 1391 1 2024-04-25T01:36:20.283 INFO:tasks.workunit.client.1.smithi138.stdout:0/978: creat d6/f141 x:0 0 0 2024-04-25T01:36:20.287 INFO:tasks.workunit.client.1.smithi138.stdout:0/979: rename d6/df/d41/d49/de4/l123 to d6/d2f/l142 0 2024-04-25T01:36:20.288 INFO:tasks.workunit.client.1.smithi138.stdout:0/980: creat d6/df/d3a/d67/f143 x:0 0 0 2024-04-25T01:36:20.306 INFO:tasks.workunit.client.1.smithi138.stdout:4/916: rmdir d0 39 2024-04-25T01:36:20.309 INFO:tasks.workunit.client.1.smithi138.stdout:4/917: getdents d0/d1c/d1e/d9f/d7e 0 2024-04-25T01:36:20.312 INFO:tasks.workunit.client.1.smithi138.stdout:4/918: unlink d0/d1c/d86/dd9/d105/dc6/cf9 0 2024-04-25T01:36:20.313 INFO:tasks.workunit.client.1.smithi138.stdout:4/919: write d0/d1c/d1e/d9f/d7e/dd5/d74/fc5 [4811463,514] 0 2024-04-25T01:36:20.316 INFO:tasks.workunit.client.1.smithi138.stdout:4/920: symlink d0/d1c/d1e/de4/l11f 0 2024-04-25T01:36:20.319 INFO:tasks.workunit.client.1.smithi138.stdout:4/921: mkdir d0/d1c/d1e/d9f/de5/def/d118/d120 0 2024-04-25T01:36:20.321 INFO:tasks.workunit.client.1.smithi138.stdout:4/922: creat d0/d1c/d86/f121 x:0 0 0 2024-04-25T01:36:20.322 INFO:tasks.workunit.client.1.smithi138.stdout:4/923: write d0/d1c/d48/f92 [1339184,57831] 0 2024-04-25T01:36:20.324 INFO:tasks.workunit.client.1.smithi138.stdout:4/924: unlink d0/d1c/d86/l94 0 2024-04-25T01:36:20.327 INFO:tasks.workunit.client.1.smithi138.stdout:4/925: symlink d0/d1c/d1e/d9f/de5/def/l122 0 2024-04-25T01:36:20.328 INFO:tasks.workunit.client.1.smithi138.stdout:4/926: mkdir d0/d1c/d1e/d9f/d106/d123 0 2024-04-25T01:36:20.330 INFO:tasks.workunit.client.1.smithi138.stdout:4/927: read d0/d1c/d86/dd9/d105/dc6/d7f/ff5 [3388283,17706] 0 2024-04-25T01:36:20.341 INFO:tasks.workunit.client.1.smithi138.stdout:4/928: dread d0/d1c/d1e/dbb/f65 [0,4194304] 0 2024-04-25T01:36:20.349 INFO:tasks.workunit.client.1.smithi138.stdout:4/929: dwrite d0/d1c/d1e/dbb/f65 [4194304,4194304] 0 2024-04-25T01:36:20.355 INFO:tasks.workunit.client.1.smithi138.stdout:4/930: rename d0/d1c/c42 to d0/d1c/d1e/d9f/c124 0 2024-04-25T01:36:20.356 INFO:tasks.workunit.client.1.smithi138.stdout:4/931: chown d0/d1c/d1e/d9f/d7e/dd5/d10a 30816002 1 2024-04-25T01:36:20.357 INFO:tasks.workunit.client.1.smithi138.stdout:4/932: fdatasync d0/d1c/d1e/dbb/f8b 0 2024-04-25T01:36:20.359 INFO:tasks.workunit.client.1.smithi138.stdout:4/933: chown d0/d1c/d1e/fd8 115 1 2024-04-25T01:36:20.360 INFO:tasks.workunit.client.1.smithi138.stdout:4/934: symlink d0/d1c/d48/ddf/l125 0 2024-04-25T01:36:20.366 INFO:tasks.workunit.client.1.smithi138.stdout:4/935: dwrite d0/d1c/d1e/d9f/d7e/dd5/f53 [0,4194304] 0 2024-04-25T01:36:20.367 INFO:tasks.workunit.client.1.smithi138.stdout:4/936: write d0/d59/f110 [194992,14735] 0 2024-04-25T01:36:20.372 INFO:tasks.workunit.client.1.smithi138.stdout:4/937: dwrite d0/d1c/d1e/dbb/f65 [0,4194304] 0 2024-04-25T01:36:20.380 INFO:tasks.workunit.client.1.smithi138.stdout:4/938: dwrite d0/f111 [0,4194304] 0 2024-04-25T01:36:20.388 INFO:tasks.workunit.client.1.smithi138.stdout:4/939: link d0/d1c/d1e/d9f/d7e/dd5/dc2/cc7 d0/d1c/d1e/c126 0 2024-04-25T01:36:20.389 INFO:tasks.workunit.client.1.smithi138.stdout:4/940: dread - d0/d1c/d1e/d9f/d33/f107 zero size 2024-04-25T01:36:20.413 INFO:tasks.workunit.client.1.smithi138.stdout:4/941: dread d0/d1c/d1e/d9f/d33/f99 [0,4194304] 0 2024-04-25T01:36:20.418 INFO:tasks.workunit.client.1.smithi138.stdout:4/942: dwrite d0/d1c/d1e/d9f/f44 [4194304,4194304] 0 2024-04-25T01:36:20.423 INFO:tasks.workunit.client.1.smithi138.stdout:4/943: read d0/d1c/d86/dd9/d105/dc6/fe9 [256987,30430] 0 2024-04-25T01:36:20.424 INFO:tasks.workunit.client.1.smithi138.stdout:4/944: readlink d0/d1c/d48/ddf/l125 0 2024-04-25T01:36:20.426 INFO:tasks.workunit.client.1.smithi138.stdout:4/945: dread d0/d1c/d48/ddf/fee [0,4194304] 0 2024-04-25T01:36:20.429 INFO:tasks.workunit.client.1.smithi138.stdout:4/946: creat d0/d1c/d1e/d9f/d7e/dd5/d74/f127 x:0 0 0 2024-04-25T01:36:20.442 INFO:tasks.workunit.client.1.smithi138.stdout:4/947: write d0/f40 [961092,2521] 0 2024-04-25T01:36:20.448 INFO:tasks.workunit.client.1.smithi138.stdout:4/948: dwrite d0/d1c/d1e/d9f/d7e/dd5/ffa [0,4194304] 0 2024-04-25T01:36:20.454 INFO:tasks.workunit.client.1.smithi138.stdout:4/949: dwrite d0/d1c/d1e/dbb/f65 [4194304,4194304] 0 2024-04-25T01:36:20.465 INFO:tasks.workunit.client.1.smithi138.stdout:4/950: truncate d0/d1c/d1e/d9f/d7e/dd5/d74/fc5 1718493 0 2024-04-25T01:36:20.467 INFO:tasks.workunit.client.1.smithi138.stdout:4/951: write d0/d1c/d1e/dbb/da3/fa4 [2704375,103741] 0 2024-04-25T01:36:20.468 INFO:tasks.workunit.client.1.smithi138.stdout:4/952: write d0/d1c/d1e/d9f/d33/f107 [210484,58359] 0 2024-04-25T01:36:20.470 INFO:tasks.workunit.client.1.smithi138.stdout:4/953: creat d0/d1c/d1e/d9f/d7e/de0/f128 x:0 0 0 2024-04-25T01:36:20.476 INFO:tasks.workunit.client.1.smithi138.stdout:4/954: truncate d0/d1c/d1e/d9f/fde 2725200 0 2024-04-25T01:36:20.478 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:20 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:36:20.482 INFO:tasks.workunit.client.1.smithi138.stdout:4/955: dread d0/d1c/d86/dd9/d105/dc6/d7f/f8a [0,4194304] 0 2024-04-25T01:36:20.485 INFO:tasks.workunit.client.1.smithi138.stdout:4/956: dread d0/d1c/d1e/dbb/f65 [4194304,4194304] 0 2024-04-25T01:36:20.486 INFO:tasks.workunit.client.1.smithi138.stdout:4/957: write d0/d1c/d86/f121 [676520,113629] 0 2024-04-25T01:36:20.487 INFO:tasks.workunit.client.1.smithi138.stdout:4/958: write d0/d1c/d1e/d9f/f44 [8861545,1247] 0 2024-04-25T01:36:20.489 INFO:tasks.workunit.client.1.smithi138.stdout:4/959: truncate d0/d59/f110 227839 0 2024-04-25T01:36:20.490 INFO:tasks.workunit.client.1.smithi138.stdout:4/960: fsync d0/d1c/d86/f10d 0 2024-04-25T01:36:20.492 INFO:tasks.workunit.client.1.smithi138.stdout:4/961: readlink d0/d59/l10f 0 2024-04-25T01:36:20.492 INFO:tasks.workunit.client.1.smithi138.stdout:4/962: stat d0/d1c/d1e/d9f/de5/def/l122 0 2024-04-25T01:36:20.495 INFO:tasks.workunit.client.1.smithi138.stdout:4/963: rename d0/d1c/d1e/d9f/c124 to d0/d1c/d1e/d9f/de5/df0/c129 0 2024-04-25T01:36:20.500 INFO:tasks.workunit.client.1.smithi138.stdout:4/964: link d0/d1c/d1e/dbb/da3/db5/ce7 d0/d1c/d1e/d9f/d106/d123/c12a 0 2024-04-25T01:36:20.500 INFO:tasks.workunit.client.1.smithi138.stdout:4/965: stat d0/d1c/d1e/dbb 0 2024-04-25T01:36:20.504 INFO:tasks.workunit.client.1.smithi138.stdout:0/981: sync 2024-04-25T01:36:20.506 INFO:tasks.workunit.client.1.smithi138.stdout:0/982: mknod d6/df/d66/d119/c144 0 2024-04-25T01:36:20.507 INFO:tasks.workunit.client.1.smithi138.stdout:0/983: write d6/df/f2c [2518851,14904] 0 2024-04-25T01:36:20.509 INFO:tasks.workunit.client.1.smithi138.stdout:4/966: write d0/ff1 [666914,79801] 0 2024-04-25T01:36:20.511 INFO:tasks.workunit.client.1.smithi138.stdout:0/984: rmdir d6/d39/d52/d9c/d126 0 2024-04-25T01:36:20.513 INFO:tasks.workunit.client.1.smithi138.stdout:4/967: rename d0/d1c/d1e/d9f/d7e/dd5/f53 to d0/d1c/d1e/d9f/d106/d123/f12b 0 2024-04-25T01:36:20.515 INFO:tasks.workunit.client.1.smithi138.stdout:0/985: rename d6/df/d41/d49/ddb/dec to d6/df/d3a/d67/d145 0 2024-04-25T01:36:20.516 INFO:tasks.workunit.client.1.smithi138.stdout:0/986: mkdir d6/d1d/d82/df2/d146 0 2024-04-25T01:36:20.518 INFO:tasks.workunit.client.1.smithi138.stdout:4/968: rename d0/d1c/l1d to d0/d1c/d1e/d9f/d7e/dd5/l12c 0 2024-04-25T01:36:20.519 INFO:tasks.workunit.client.1.smithi138.stdout:4/969: readlink d0/l5e 0 2024-04-25T01:36:20.521 INFO:tasks.workunit.client.1.smithi138.stdout:4/970: mkdir d0/d1c/d48/d12d 0 2024-04-25T01:36:20.531 INFO:tasks.workunit.client.1.smithi138.stdout:0/987: dwrite d6/d39/d43/f6b [0,4194304] 0 2024-04-25T01:36:20.531 INFO:tasks.workunit.client.1.smithi138.stdout:4/971: dread d0/d1c/d1e/d9f/d7e/dd5/d74/fc5 [0,4194304] 0 2024-04-25T01:36:20.532 INFO:tasks.workunit.client.1.smithi138.stdout:4/972: chown d0/d1c/d1e/d9f/de5/def 230886 1 2024-04-25T01:36:20.534 INFO:tasks.workunit.client.1.smithi138.stdout:0/988: symlink d6/d39/d58/d11c/l147 0 2024-04-25T01:36:20.536 INFO:tasks.workunit.client.1.smithi138.stdout:4/973: mknod d0/d1c/d48/d12d/c12e 0 2024-04-25T01:36:20.536 INFO:tasks.workunit.client.1.smithi138.stdout:0/989: read d6/d2f/d31/ff6 [107999,29218] 0 2024-04-25T01:36:20.539 INFO:tasks.workunit.client.1.smithi138.stdout:0/990: mkdir d6/d2f/d31/ddc/d148 0 2024-04-25T01:36:20.540 INFO:tasks.workunit.client.1.smithi138.stdout:4/974: unlink d0/d1c/d1e/d9f/d7e/dd5/d74/lad 0 2024-04-25T01:36:20.542 INFO:tasks.workunit.client.1.smithi138.stdout:4/975: chown d0/d1c/d1e/d9f/d7e/dd5/d10a/f117 5640417 1 2024-04-25T01:36:20.545 INFO:tasks.workunit.client.1.smithi138.stdout:0/991: dwrite d6/d2f/fe8 [0,4194304] 0 2024-04-25T01:36:20.551 INFO:tasks.workunit.client.1.smithi138.stdout:0/992: write d6/df/d69/f6a [3411240,116706] 0 2024-04-25T01:36:20.555 INFO:tasks.workunit.client.1.smithi138.stdout:0/993: rename d6/df/d3a/cff to d6/d39/d52/c149 0 2024-04-25T01:36:20.556 INFO:tasks.workunit.client.1.smithi138.stdout:0/994: truncate d6/d2f/de2/f140 676221 0 2024-04-25T01:36:20.561 INFO:tasks.workunit.client.1.smithi138.stdout:0/995: dwrite d6/d2f/fe8 [0,4194304] 0 2024-04-25T01:36:20.565 INFO:tasks.workunit.client.1.smithi138.stdout:0/996: creat d6/d39/d52/f14a x:0 0 0 2024-04-25T01:36:20.580 INFO:tasks.workunit.client.1.smithi138.stdout:0/997: dread d6/f26 [4194304,4194304] 0 2024-04-25T01:36:20.583 INFO:tasks.workunit.client.1.smithi138.stdout:0/998: rename d6/d2f/d31/l6c to d6/d1d/df5/l14b 0 2024-04-25T01:36:20.584 INFO:tasks.workunit.client.1.smithi138.stdout:0/999: dread - d6/d1d/d82/f10b zero size 2024-04-25T01:36:20.662 INFO:tasks.workunit.client.1.smithi138.stdout:4/976: sync 2024-04-25T01:36:20.663 INFO:tasks.workunit.client.1.smithi138.stdout:4/977: dread - d0/d1c/d86/dfe/f10c zero size 2024-04-25T01:36:20.664 INFO:tasks.workunit.client.1.smithi138.stdout:4/978: creat d0/d59/f12f x:0 0 0 2024-04-25T01:36:20.668 INFO:tasks.workunit.client.1.smithi138.stdout:4/979: rename d0/d1c/d86/ldc to d0/d1c/d1e/dbb/l130 0 2024-04-25T01:36:20.671 INFO:tasks.workunit.client.1.smithi138.stdout:4/980: read d0/d1c/d1e/d9f/f34 [3296877,111764] 0 2024-04-25T01:36:20.673 INFO:tasks.workunit.client.1.smithi138.stdout:4/981: mkdir d0/d1c/d86/dd9/d105/de6/d10e/d131 0 2024-04-25T01:36:20.674 INFO:tasks.workunit.client.1.smithi138.stdout:4/982: chown d0/d1c/d1e/d9f/de5/cfc 7065 1 2024-04-25T01:36:20.678 INFO:tasks.workunit.client.1.smithi138.stdout:4/983: link d0/d1c/d1e/d9f/c5d d0/d1c/d86/dfe/c132 0 2024-04-25T01:36:20.679 INFO:tasks.workunit.client.1.smithi138.stdout:4/984: stat d0/d1c/d1e/d9f/d7e/dd5/d74/le8 0 2024-04-25T01:36:20.690 INFO:tasks.workunit.client.1.smithi138.stdout:4/985: dread d0/d1c/f28 [0,4194304] 0 2024-04-25T01:36:20.692 INFO:tasks.workunit.client.1.smithi138.stdout:4/986: dread d0/d59/f110 [0,4194304] 0 2024-04-25T01:36:20.694 INFO:tasks.workunit.client.1.smithi138.stdout:4/987: creat d0/d1c/d1e/d9f/f133 x:0 0 0 2024-04-25T01:36:20.696 INFO:tasks.workunit.client.1.smithi138.stdout:4/988: creat d0/d1c/d1e/de4/f134 x:0 0 0 2024-04-25T01:36:20.705 INFO:tasks.workunit.client.1.smithi138.stdout:4/989: write d0/d1c/d1e/d9f/d7e/dd5/f8f [2313539,70425] 0 2024-04-25T01:36:20.708 INFO:tasks.workunit.client.1.smithi138.stdout:4/990: rename d0/d1c/cce to d0/d1c/c135 0 2024-04-25T01:36:20.713 INFO:tasks.workunit.client.1.smithi138.stdout:4/991: dwrite d0/d1c/d1e/dbb/da3/db5/fc8 [0,4194304] 0 2024-04-25T01:36:20.717 INFO:tasks.workunit.client.1.smithi138.stdout:4/992: dread d0/f111 [0,4194304] 0 2024-04-25T01:36:20.719 INFO:tasks.workunit.client.1.smithi138.stdout:4/993: truncate d0/d1c/d1e/d9f/d7e/de0/fec 4198600 0 2024-04-25T01:36:20.721 INFO:tasks.workunit.client.1.smithi138.stdout:4/994: mkdir d0/d1c/d1e/d9f/de5/def/d118/d136 0 2024-04-25T01:36:20.726 INFO:tasks.workunit.client.1.smithi138.stdout:4/995: link d0/l72 d0/d1c/d86/dd9/d105/de6/d10e/l137 0 2024-04-25T01:36:20.729 INFO:tasks.workunit.client.1.smithi138.stdout:4/996: read d0/d1c/d1e/d9f/d7e/dd5/d10a/f112 [478497,102419] 0 2024-04-25T01:36:20.731 INFO:tasks.workunit.client.1.smithi138.stdout:4/997: write d0/d1c/d1e/d9f/d7e/dd5/f8f [3070186,30799] 0 2024-04-25T01:36:20.733 INFO:tasks.workunit.client.1.smithi138.stdout:4/998: fdatasync d0/d1c/d1e/d9f/d106/f108 0 2024-04-25T01:36:20.734 INFO:tasks.workunit.client.1.smithi138.stdout:4/999: write d0/d1c/d1e/d9f/d7e/de0/fec [2974040,115491] 0 2024-04-25T01:36:20.737 INFO:tasks.workunit.client.1.smithi138.stderr:+ rm -rf -- ./tmp.ebWhjI7yKi 2024-04-25T01:36:20.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:20 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:36:21.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:21 smithi033 ceph-mon[26353]: pgmap v26: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 67 MiB/s rd, 162 MiB/s wr, 416 op/s 2024-04-25T01:36:21.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:21 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:21.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:21 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:21.743 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:36:21.743 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T01:36:21.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:21 smithi138 ceph-mon[32977]: pgmap v26: 65 pgs: 65 active+clean; 2.9 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 67 MiB/s rd, 162 MiB/s wr, 416 op/s 2024-04-25T01:36:21.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:21 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:21.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:21 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:22.616 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:22 smithi033 ceph-mon[26353]: Reconfiguring grafana.smithi033 (dependencies changed)... 2024-04-25T01:36:22.616 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:22 smithi033 ceph-mon[26353]: Reconfiguring daemon grafana.smithi033 on smithi033 2024-04-25T01:36:22.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:22 smithi138 ceph-mon[32977]: Reconfiguring grafana.smithi033 (dependencies changed)... 2024-04-25T01:36:22.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:22 smithi138 ceph-mon[32977]: Reconfiguring daemon grafana.smithi033 on smithi033 2024-04-25T01:36:23.667 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:23 smithi138 ceph-mon[32977]: pgmap v27: 65 pgs: 65 active+clean; 2.6 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 78 MiB/s rd, 186 MiB/s wr, 498 op/s 2024-04-25T01:36:23.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:23 smithi033 ceph-mon[26353]: pgmap v27: 65 pgs: 65 active+clean; 2.6 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 78 MiB/s rd, 186 MiB/s wr, 498 op/s 2024-04-25T01:36:25.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:25 smithi033 ceph-mon[26353]: pgmap v28: 65 pgs: 65 active+clean; 2.6 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 52 MiB/s rd, 124 MiB/s wr, 369 op/s 2024-04-25T01:36:25.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:25 smithi138 ceph-mon[32977]: pgmap v28: 65 pgs: 65 active+clean; 2.6 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 52 MiB/s rd, 124 MiB/s wr, 369 op/s 2024-04-25T01:36:27.242 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:26 smithi033 ceph-mon[26353]: pgmap v29: 65 pgs: 65 active+clean; 1.7 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 54 MiB/s rd, 128 MiB/s wr, 499 op/s 2024-04-25T01:36:27.267 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:26 smithi138 ceph-mon[32977]: pgmap v29: 65 pgs: 65 active+clean; 1.7 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 54 MiB/s rd, 128 MiB/s wr, 499 op/s 2024-04-25T01:36:29.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:29 smithi033 ceph-mon[26353]: pgmap v30: 65 pgs: 65 active+clean; 1.7 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 60 MiB/s wr, 304 op/s 2024-04-25T01:36:29.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:29 smithi138 ceph-mon[32977]: pgmap v30: 65 pgs: 65 active+clean; 1.7 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 60 MiB/s wr, 304 op/s 2024-04-25T01:36:31.677 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T01:36:31.677 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-25T01:36:31.739 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:31 smithi138 ceph-mon[32977]: pgmap v31: 65 pgs: 65 active+clean; 1.4 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 27 MiB/s rd, 60 MiB/s wr, 355 op/s 2024-04-25T01:36:31.741 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:31 smithi033 ceph-mon[26353]: pgmap v31: 65 pgs: 65 active+clean; 1.4 GiB data, 5.8 GiB used, 531 GiB / 536 GiB avail; 27 MiB/s rd, 60 MiB/s wr, 355 op/s 2024-04-25T01:36:33.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:33 smithi033 ceph-mon[26353]: pgmap v32: 65 pgs: 65 active+clean; 831 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 21 MiB/s rd, 47 MiB/s wr, 436 op/s 2024-04-25T01:36:33.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:33 smithi138 ceph-mon[32977]: pgmap v32: 65 pgs: 65 active+clean; 831 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 21 MiB/s rd, 47 MiB/s wr, 436 op/s 2024-04-25T01:36:35.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:35 smithi033 ceph-mon[26353]: pgmap v33: 65 pgs: 65 active+clean; 831 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.4 MiB/s wr, 291 op/s 2024-04-25T01:36:35.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:36:35.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:35 smithi138 ceph-mon[32977]: pgmap v33: 65 pgs: 65 active+clean; 831 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 3.0 MiB/s rd, 5.4 MiB/s wr, 291 op/s 2024-04-25T01:36:35.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:36:37.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:37 smithi033 ceph-mon[26353]: pgmap v34: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 463 op/s 2024-04-25T01:36:37.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:37 smithi138 ceph-mon[32977]: pgmap v34: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 3.8 MiB/s rd, 6.2 MiB/s wr, 463 op/s 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.323+0000 7f732ebd5700 1 -- 172.21.15.33:0/2908267185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 msgr2=0x7f73280fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.323+0000 7f732ebd5700 1 --2- 172.21.15.33:0/2908267185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 0x7f73280fae70 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7314004880 tx=0x7f7314009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.323+0000 7f732ebd5700 1 -- 172.21.15.33:0/2908267185 shutdown_connections 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.323+0000 7f732ebd5700 1 --2- 172.21.15.33:0/2908267185 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 0x7f73280fae70 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.323+0000 7f732ebd5700 1 --2- 172.21.15.33:0/2908267185 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73280fe3e0 0x7f73280fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.323+0000 7f732ebd5700 1 -- 172.21.15.33:0/2908267185 >> 172.21.15.33:0/2908267185 conn(0x7f73280f6b80 msgr2=0x7f73280f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.323+0000 7f732ebd5700 1 -- 172.21.15.33:0/2908267185 shutdown_connections 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.323+0000 7f732ebd5700 1 -- 172.21.15.33:0/2908267185 wait complete. 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f732ebd5700 1 Processor -- start 2024-04-25T01:36:38.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f732ebd5700 1 -- start start 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f732ebd5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73280fe3e0 0x7f732810d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f732ebd5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 0x7f732810df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f732ebd5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732810e500 con 0x7f73280fed30 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f732ebd5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732810ed50 con 0x7f73280fe3e0 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f7327fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 0x7f732810df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f732c971700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73280fe3e0 0x7f732810d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f732c971700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73280fe3e0 0x7f732810d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:55774/0 (socket says 172.21.15.33:55774) 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f7327fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 0x7f732810df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54286/0 (socket says 172.21.15.33:54286) 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f732c971700 1 -- 172.21.15.33:0/84844418 learned_addr learned my addr 172.21.15.33:0/84844418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f7327fff700 1 -- 172.21.15.33:0/84844418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73280fe3e0 msgr2=0x7f732810d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.324+0000 7f7327fff700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73280fe3e0 0x7f732810d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.325+0000 7f7327fff700 1 -- 172.21.15.33:0/84844418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f731c005040 con 0x7f73280fed30 2024-04-25T01:36:38.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.325+0000 7f7327fff700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 0x7f732810df00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f73140137c0 tx=0x7f7314009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:38.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.325+0000 7f73257fa700 1 -- 172.21.15.33:0/84844418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f731400e030 con 0x7f73280fed30 2024-04-25T01:36:38.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.325+0000 7f73257fa700 1 -- 172.21.15.33:0/84844418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7314013e60 con 0x7f73280fed30 2024-04-25T01:36:38.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.325+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7314005040 con 0x7f73280fed30 2024-04-25T01:36:38.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.325+0000 7f73257fa700 1 -- 172.21.15.33:0/84844418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f731401ce20 con 0x7f73280fed30 2024-04-25T01:36:38.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.325+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f732810f380 con 0x7f73280fed30 2024-04-25T01:36:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.326+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f732805f5c0 con 0x7f73280fed30 2024-04-25T01:36:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.326+0000 7f73257fa700 1 -- 172.21.15.33:0/84844418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f731401c690 con 0x7f73280fed30 2024-04-25T01:36:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.327+0000 7f73257fa700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f731806b790 0x7f731806dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.327+0000 7f73257fa700 1 -- 172.21.15.33:0/84844418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f731408b8b0 con 0x7f73280fed30 2024-04-25T01:36:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.327+0000 7f732c971700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f731806b790 0x7f731806dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:38.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.328+0000 7f732c971700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f731806b790 0x7f731806dc40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f731c0045a0 tx=0x7f731c00b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:38.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.332+0000 7f73257fa700 1 -- 172.21.15.33:0/84844418 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f7314059130 con 0x7f73280fed30 2024-04-25T01:36:38.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.654+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f732810f650 con 0x7f731806b790 2024-04-25T01:36:38.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.655+0000 7f73257fa700 1 -- 172.21.15.33:0/84844418 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f732810f650 con 0x7f731806b790 2024-04-25T01:36:38.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f731806b790 msgr2=0x7f731806dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:38.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f731806b790 0x7f731806dc40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f731c0045a0 tx=0x7f731c00b040 comp rx=0 tx=0).stop 2024-04-25T01:36:38.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 msgr2=0x7f732810df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:38.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 0x7f732810df00 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f73140137c0 tx=0x7f7314009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:38.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 shutdown_connections 2024-04-25T01:36:38.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f731806b790 0x7f731806dc40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73280fe3e0 0x7f732810d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 --2- 172.21.15.33:0/84844418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73280fed30 0x7f732810df00 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 >> 172.21.15.33:0/84844418 conn(0x7f73280f6b80 msgr2=0x7f73280f8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:38.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 shutdown_connections 2024-04-25T01:36:38.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.658+0000 7f732ebd5700 1 -- 172.21.15.33:0/84844418 wait complete. 2024-04-25T01:36:38.644 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:36:38.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.768+0000 7f530d87d700 1 -- 172.21.15.33:0/3722861749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 msgr2=0x7f5308102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:38.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.768+0000 7f530d87d700 1 --2- 172.21.15.33:0/3722861749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 0x7f5308102300 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52f0004940 tx=0x7f52f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:38.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.768+0000 7f530d87d700 1 -- 172.21.15.33:0/3722861749 shutdown_connections 2024-04-25T01:36:38.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.768+0000 7f530d87d700 1 --2- 172.21.15.33:0/3722861749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 0x7f5308102300 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.768+0000 7f530d87d700 1 --2- 172.21.15.33:0/3722861749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53080662c0 0x7f53080ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.768+0000 7f530d87d700 1 -- 172.21.15.33:0/3722861749 >> 172.21.15.33:0/3722861749 conn(0x7f53080f7430 msgr2=0x7f53080f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.768+0000 7f530d87d700 1 -- 172.21.15.33:0/3722861749 shutdown_connections 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.768+0000 7f530d87d700 1 -- 172.21.15.33:0/3722861749 wait complete. 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f530d87d700 1 Processor -- start 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f530d87d700 1 -- start start 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f530d87d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53080662c0 0x7f53080fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f530d87d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 0x7f53080fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f530d87d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080fbe50 con 0x7f53080ffee0 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f530d87d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080fbfc0 con 0x7f53080662c0 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f53067fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 0x7f53080fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f53067fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 0x7f53080fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54308/0 (socket says 172.21.15.33:54308) 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f53067fc700 1 -- 172.21.15.33:0/62387358 learned_addr learned my addr 172.21.15.33:0/62387358 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:38.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.769+0000 7f5306ffd700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53080662c0 0x7f53080fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.770+0000 7f53067fc700 1 -- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53080662c0 msgr2=0x7f53080fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.770+0000 7f53067fc700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53080662c0 0x7f53080fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.770+0000 7f53067fc700 1 -- 172.21.15.33:0/62387358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0005040 con 0x7f53080ffee0 2024-04-25T01:36:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.770+0000 7f53067fc700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 0x7f53080fb8a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52f0013870 tx=0x7f52f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.770+0000 7f52ff7fe700 1 -- 172.21.15.33:0/62387358 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52f0021070 con 0x7f53080ffee0 2024-04-25T01:36:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.770+0000 7f52ff7fe700 1 -- 172.21.15.33:0/62387358 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52f0009430 con 0x7f53080ffee0 2024-04-25T01:36:38.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.770+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53080fc240 con 0x7f53080ffee0 2024-04-25T01:36:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.770+0000 7f52ff7fe700 1 -- 172.21.15.33:0/62387358 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52f000e070 con 0x7f53080ffee0 2024-04-25T01:36:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.770+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53081159e0 con 0x7f53080ffee0 2024-04-25T01:36:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.771+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530805f5c0 con 0x7f53080ffee0 2024-04-25T01:36:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.771+0000 7f52ff7fe700 1 -- 172.21.15.33:0/62387358 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f52f001c750 con 0x7f53080ffee0 2024-04-25T01:36:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.772+0000 7f52ff7fe700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f52f406b730 0x7f52f406dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.772+0000 7f52ff7fe700 1 -- 172.21.15.33:0/62387358 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f52f008b5a0 con 0x7f53080ffee0 2024-04-25T01:36:38.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.772+0000 7f5306ffd700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f52f406b730 0x7f52f406dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:38.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.773+0000 7f5306ffd700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f52f406b730 0x7f52f406dbe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53080fd160 tx=0x7f52f800b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:38.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:38.777+0000 7f52ff7fe700 1 -- 172.21.15.33:0/62387358 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f52f0055250 con 0x7f53080ffee0 2024-04-25T01:36:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.022+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53080f8f20 con 0x7f52f406b730 2024-04-25T01:36:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.023+0000 7f52ff7fe700 1 -- 172.21.15.33:0/62387358 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f53080f8f20 con 0x7f52f406b730 2024-04-25T01:36:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f52f406b730 msgr2=0x7f52f406dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f52f406b730 0x7f52f406dbe0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f53080fd160 tx=0x7f52f800b040 comp rx=0 tx=0).stop 2024-04-25T01:36:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 msgr2=0x7f53080fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 0x7f53080fb8a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f52f0013870 tx=0x7f52f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 shutdown_connections 2024-04-25T01:36:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f52f406b730 0x7f52f406dbe0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53080662c0 0x7f53080fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 --2- 172.21.15.33:0/62387358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53080ffee0 0x7f53080fb8a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 >> 172.21.15.33:0/62387358 conn(0x7f53080f7430 msgr2=0x7f53080f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:39.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 shutdown_connections 2024-04-25T01:36:39.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.025+0000 7f530d87d700 1 -- 172.21.15.33:0/62387358 wait complete. 2024-04-25T01:36:39.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.135+0000 7f384da21700 1 -- 172.21.15.33:0/722694958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 msgr2=0x7f3848102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.135+0000 7f384da21700 1 --2- 172.21.15.33:0/722694958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 0x7f3848102300 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f383c004940 tx=0x7f383c009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:39.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.136+0000 7f384da21700 1 -- 172.21.15.33:0/722694958 shutdown_connections 2024-04-25T01:36:39.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.136+0000 7f384da21700 1 --2- 172.21.15.33:0/722694958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 0x7f3848102300 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.136+0000 7f384da21700 1 --2- 172.21.15.33:0/722694958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38480662c0 0x7f38480ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.136+0000 7f384da21700 1 -- 172.21.15.33:0/722694958 >> 172.21.15.33:0/722694958 conn(0x7f38480f7430 msgr2=0x7f38480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:39.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.136+0000 7f384da21700 1 -- 172.21.15.33:0/722694958 shutdown_connections 2024-04-25T01:36:39.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.136+0000 7f384da21700 1 -- 172.21.15.33:0/722694958 wait complete. 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.136+0000 7f384da21700 1 Processor -- start 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.136+0000 7f384da21700 1 -- start start 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f384da21700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38480662c0 0x7f38480fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f384da21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 0x7f38480fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f384da21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38480fbe50 con 0x7f38480ffee0 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f384da21700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38480fbfc0 con 0x7f38480662c0 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f38467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 0x7f38480fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f38467fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 0x7f38480fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54328/0 (socket says 172.21.15.33:54328) 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f38467fc700 1 -- 172.21.15.33:0/2837899731 learned_addr learned my addr 172.21.15.33:0/2837899731 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f3846ffd700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38480662c0 0x7f38480fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:39.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f38467fc700 1 -- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38480662c0 msgr2=0x7f38480fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f38467fc700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38480662c0 0x7f38480fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f38467fc700 1 -- 172.21.15.33:0/2837899731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c005040 con 0x7f38480ffee0 2024-04-25T01:36:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f38467fc700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 0x7f38480fb8a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f383c00b3b0 tx=0x7f383c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f382f7fe700 1 -- 172.21.15.33:0/2837899731 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f383c00e040 con 0x7f38480ffee0 2024-04-25T01:36:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f382f7fe700 1 -- 172.21.15.33:0/2837899731 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f383c004290 con 0x7f38480ffee0 2024-04-25T01:36:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.137+0000 7f382f7fe700 1 -- 172.21.15.33:0/2837899731 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f383c01c3f0 con 0x7f38480ffee0 2024-04-25T01:36:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.138+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38480fc240 con 0x7f38480ffee0 2024-04-25T01:36:39.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.138+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3848115a30 con 0x7f38480ffee0 2024-04-25T01:36:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.139+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f384805f5c0 con 0x7f38480ffee0 2024-04-25T01:36:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.139+0000 7f382f7fe700 1 -- 172.21.15.33:0/2837899731 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f383c00bbf0 con 0x7f38480ffee0 2024-04-25T01:36:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.140+0000 7f382f7fe700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f383006b790 0x7f383006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.140+0000 7f382f7fe700 1 -- 172.21.15.33:0/2837899731 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f383c08b660 con 0x7f38480ffee0 2024-04-25T01:36:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.140+0000 7f3846ffd700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f383006b790 0x7f383006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:39.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.140+0000 7f3846ffd700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f383006b790 0x7f383006dc40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38480fd160 tx=0x7f383800b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:39.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.145+0000 7f382f7fe700 1 -- 172.21.15.33:0/2837899731 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f383c055290 con 0x7f38480ffee0 2024-04-25T01:36:39.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.377+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f38480f8f20 con 0x7f383006b790 2024-04-25T01:36:39.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.390+0000 7f382f7fe700 1 -- 172.21.15.33:0/2837899731 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f38480f8f20 con 0x7f383006b790 2024-04-25T01:36:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:36:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 starting - - - - 2024-04-25T01:36:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (9m) 33s ago 9m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:36:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (8m) 54s ago 8m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:36:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (7m) 33s ago 8m 87.2M - 9.4.7 954c08fa6188 4533d743079b 2024-04-25T01:36:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (3m) 33s ago 3m 61.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:36:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (3m) 33s ago 3m 99.5M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:36:39.366 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (3m) 54s ago 3m 110M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (3m) 54s ago 3m 67.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:9283 running (10m) 33s ago 10m 390M - 17.2.7-902-gc61472c8 0de407c1b3af 22f7bb21dc39 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283 running (96s) 54s ago 8m 488M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8ce48ef47e1e 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (10m) 33s ago 10m 48.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (8m) 54s ago 8m 41.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 starting - - - - 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (8m) 54s ago 8m 16.3M - 1.5.0 0da6a335fe13 ce8fbd90510d 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (7m) 33s ago 7m 126M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (6m) 33s ago 6m 146M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (6m) 33s ago 6m 124M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (5m) 54s ago 5m 156M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (5m) 54s ago 5m 135M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (5m) 54s ago 5m 128M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:36:39.367 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (7m) 33s ago 8m 58.7M - 2.43.0 a07b618ecd1d 9109b23faf81 2024-04-25T01:36:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f383006b790 msgr2=0x7f383006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f383006b790 0x7f383006dc40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38480fd160 tx=0x7f383800b040 comp rx=0 tx=0).stop 2024-04-25T01:36:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 msgr2=0x7f38480fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 0x7f38480fb8a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f383c00b3b0 tx=0x7f383c009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:39.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 shutdown_connections 2024-04-25T01:36:39.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f383006b790 0x7f383006dc40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38480662c0 0x7f38480fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 --2- 172.21.15.33:0/2837899731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f38480ffee0 0x7f38480fb8a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 >> 172.21.15.33:0/2837899731 conn(0x7f38480f7430 msgr2=0x7f38480f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:39.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 shutdown_connections 2024-04-25T01:36:39.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.392+0000 7f384da21700 1 -- 172.21.15.33:0/2837899731 wait complete. 2024-04-25T01:36:39.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.507+0000 7f366e24c700 1 -- 172.21.15.33:0/1679038873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 msgr2=0x7f36680fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.507+0000 7f366e24c700 1 --2- 172.21.15.33:0/1679038873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 0x7f36680fe4f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f365c004940 tx=0x7f365c009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.507+0000 7f366e24c700 1 -- 172.21.15.33:0/1679038873 shutdown_connections 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.507+0000 7f366e24c700 1 --2- 172.21.15.33:0/1679038873 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 0x7f36680fe4f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.507+0000 7f366e24c700 1 --2- 172.21.15.33:0/1679038873 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36680fcf10 0x7f36680fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.507+0000 7f366e24c700 1 -- 172.21.15.33:0/1679038873 >> 172.21.15.33:0/1679038873 conn(0x7f36680f8560 msgr2=0x7f36680fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.507+0000 7f366e24c700 1 -- 172.21.15.33:0/1679038873 shutdown_connections 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.507+0000 7f366e24c700 1 -- 172.21.15.33:0/1679038873 wait complete. 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.508+0000 7f366e24c700 1 Processor -- start 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.508+0000 7f366e24c700 1 -- start start 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.508+0000 7f366e24c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36680fcf10 0x7f366810d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.508+0000 7f366e24c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 0x7f366810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.508+0000 7f366e24c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366810e4c0 con 0x7f36680fe080 2024-04-25T01:36:39.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.508+0000 7f366e24c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366810ed10 con 0x7f36680fcf10 2024-04-25T01:36:39.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.508+0000 7f3666ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 0x7f366810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:39.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f3666ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 0x7f366810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54336/0 (socket says 172.21.15.33:54336) 2024-04-25T01:36:39.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f36677fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36680fcf10 0x7f366810d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:39.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f3666ffd700 1 -- 172.21.15.33:0/70171106 learned_addr learned my addr 172.21.15.33:0/70171106 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:39.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f3666ffd700 1 -- 172.21.15.33:0/70171106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36680fcf10 msgr2=0x7f366810d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f3666ffd700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36680fcf10 0x7f366810d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f3666ffd700 1 -- 172.21.15.33:0/70171106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658005040 con 0x7f36680fe080 2024-04-25T01:36:39.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f3666ffd700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 0x7f366810dec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f365c013870 tx=0x7f365c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:39.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f364ffff700 1 -- 172.21.15.33:0/70171106 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365c021070 con 0x7f36680fe080 2024-04-25T01:36:39.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f364ffff700 1 -- 172.21.15.33:0/70171106 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f365c009430 con 0x7f36680fe080 2024-04-25T01:36:39.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f365c005040 con 0x7f36680fe080 2024-04-25T01:36:39.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f364ffff700 1 -- 172.21.15.33:0/70171106 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365c00e070 con 0x7f36680fe080 2024-04-25T01:36:39.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.509+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366810f340 con 0x7f36680fe080 2024-04-25T01:36:39.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.510+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366805f5c0 con 0x7f36680fe080 2024-04-25T01:36:39.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.511+0000 7f364ffff700 1 -- 172.21.15.33:0/70171106 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f365c01c750 con 0x7f36680fe080 2024-04-25T01:36:39.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.511+0000 7f364ffff700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f365006b790 0x7f365006dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:39.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.511+0000 7f364ffff700 1 -- 172.21.15.33:0/70171106 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f365c08b4e0 con 0x7f36680fe080 2024-04-25T01:36:39.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.511+0000 7f36677fe700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f365006b790 0x7f365006dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:39.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.512+0000 7f36677fe700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f365006b790 0x7f365006dc40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3658009fd0 tx=0x7f365800b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:39.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.516+0000 7f364ffff700 1 -- 172.21.15.33:0/70171106 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f365c058d60 con 0x7f36680fe080 2024-04-25T01:36:39.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:39 smithi033 ceph-mon[26353]: pgmap v35: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 333 op/s 2024-04-25T01:36:39.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:39 smithi138 ceph-mon[32977]: pgmap v35: 65 pgs: 65 active+clean; 282 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 333 op/s 2024-04-25T01:36:39.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.921+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f366804bed0 con 0x7f36680fe080 2024-04-25T01:36:39.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.922+0000 7f364ffff700 1 -- 172.21.15.33:0/70171106 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f365c004490 con 0x7f36680fe080 2024-04-25T01:36:39.898 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:36:39.898 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:36:39.898 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:36:39.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:36:39.898 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:36:39.898 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:36:39.898 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-25T01:36:39.898 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-25T01:36:39.899 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:36:39.900 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:36:39.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.924+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f365006b790 msgr2=0x7f365006dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.924+0000 7f366e24c700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f365006b790 0x7f365006dc40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3658009fd0 tx=0x7f365800b040 comp rx=0 tx=0).stop 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.924+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 msgr2=0x7f366810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.924+0000 7f366e24c700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 0x7f366810dec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f365c013870 tx=0x7f365c009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.925+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 shutdown_connections 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.925+0000 7f366e24c700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f365006b790 0x7f365006dc40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.925+0000 7f366e24c700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36680fcf10 0x7f366810d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.925+0000 7f366e24c700 1 --2- 172.21.15.33:0/70171106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36680fe080 0x7f366810dec0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.925+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 >> 172.21.15.33:0/70171106 conn(0x7f36680f8560 msgr2=0x7f36680f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.925+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 shutdown_connections 2024-04-25T01:36:39.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:39.925+0000 7f366e24c700 1 -- 172.21.15.33:0/70171106 wait complete. 2024-04-25T01:36:40.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.064+0000 7f0f35296700 1 -- 172.21.15.33:0/122868025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 msgr2=0x7f0f300fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:40.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.064+0000 7f0f35296700 1 --2- 172.21.15.33:0/122868025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 0x7f0f300fd2a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0f20004940 tx=0x7f0f20009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:40.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.064+0000 7f0f35296700 1 -- 172.21.15.33:0/122868025 shutdown_connections 2024-04-25T01:36:40.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.064+0000 7f0f35296700 1 --2- 172.21.15.33:0/122868025 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f300fe090 0x7f0f300fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:40.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.064+0000 7f0f35296700 1 --2- 172.21.15.33:0/122868025 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 0x7f0f300fd2a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:40.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.064+0000 7f0f35296700 1 -- 172.21.15.33:0/122868025 >> 172.21.15.33:0/122868025 conn(0x7f0f300f85c0 msgr2=0x7f0f300fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:40.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.064+0000 7f0f35296700 1 -- 172.21.15.33:0/122868025 shutdown_connections 2024-04-25T01:36:40.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.064+0000 7f0f35296700 1 -- 172.21.15.33:0/122868025 wait complete. 2024-04-25T01:36:40.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f35296700 1 Processor -- start 2024-04-25T01:36:40.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f35296700 1 -- start start 2024-04-25T01:36:40.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f35296700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 0x7f0f3010d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:40.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f35296700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f300fe090 0x7f0f3010df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:40.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f35296700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3010e510 con 0x7f0f300fce90 2024-04-25T01:36:40.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f35296700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3010ed60 con 0x7f0f300fe090 2024-04-25T01:36:40.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f2effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 0x7f0f3010d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:40.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f2effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 0x7f0f3010d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54352/0 (socket says 172.21.15.33:54352) 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f2effd700 1 -- 172.21.15.33:0/1689635883 learned_addr learned my addr 172.21.15.33:0/1689635883 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.065+0000 7f0f2e7fc700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f300fe090 0x7f0f3010df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.066+0000 7f0f2effd700 1 -- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f300fe090 msgr2=0x7f0f3010df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.066+0000 7f0f2effd700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f300fe090 0x7f0f3010df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.066+0000 7f0f2effd700 1 -- 172.21.15.33:0/1689635883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f20005040 con 0x7f0f300fce90 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.066+0000 7f0f2effd700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 0x7f0f3010d9d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000c00 tx=0x7f0f20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.066+0000 7f0f177fe700 1 -- 172.21.15.33:0/1689635883 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f2000e030 con 0x7f0f300fce90 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.066+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3010efe0 con 0x7f0f300fce90 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.066+0000 7f0f177fe700 1 -- 172.21.15.33:0/1689635883 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f20013de0 con 0x7f0f300fce90 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.066+0000 7f0f177fe700 1 -- 172.21.15.33:0/1689635883 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f2001ce50 con 0x7f0f300fce90 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.066+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3010f520 con 0x7f0f300fce90 2024-04-25T01:36:40.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.067+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3005f5c0 con 0x7f0f300fce90 2024-04-25T01:36:40.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.068+0000 7f0f177fe700 1 -- 172.21.15.33:0/1689635883 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 25) v1 ==== 89381+0+0 (secure 0 0 0) 0x7f0f2001c400 con 0x7f0f300fce90 2024-04-25T01:36:40.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.068+0000 7f0f177fe700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f0f1806b730 0x7f0f1806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:36:40.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.068+0000 7f0f177fe700 1 -- 172.21.15.33:0/1689635883 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f0f2008bd70 con 0x7f0f300fce90 2024-04-25T01:36:40.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.069+0000 7f0f2e7fc700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f0f1806b730 0x7f0f1806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:36:40.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.069+0000 7f0f2e7fc700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f0f1806b730 0x7f0f1806dbe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009b80 tx=0x7f0f2400b040 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:36:40.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.074+0000 7f0f177fe700 1 -- 172.21.15.33:0/1689635883 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0f200594d0 con 0x7f0f300fce90 2024-04-25T01:36:40.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.349+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f3010f800 con 0x7f0f1806b730 2024-04-25T01:36:40.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.350+0000 7f0f177fe700 1 -- 172.21.15.33:0/1689635883 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f0f3010f800 con 0x7f0f1806b730 2024-04-25T01:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:36:40.326 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-25T01:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [], 2024-04-25T01:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "1/2 daemons upgraded", 2024-04-25T01:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "", 2024-04-25T01:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:36:40.327 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.352+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f0f1806b730 msgr2=0x7f0f1806dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.352+0000 7f0f35296700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f0f1806b730 0x7f0f1806dbe0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0f24009b80 tx=0x7f0f2400b040 comp rx=0 tx=0).stop 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.352+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 msgr2=0x7f0f3010d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.352+0000 7f0f35296700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 0x7f0f3010d9d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000c00 tx=0x7f0f20009e30 comp rx=0 tx=0).stop 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.352+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 shutdown_connections 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.352+0000 7f0f35296700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f0f1806b730 0x7f0f1806dbe0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.352+0000 7f0f35296700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f300fce90 0x7f0f3010d9d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.352+0000 7f0f35296700 1 --2- 172.21.15.33:0/1689635883 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f300fe090 0x7f0f3010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.352+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 >> 172.21.15.33:0/1689635883 conn(0x7f0f300f85c0 msgr2=0x7f0f300f9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.353+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 shutdown_connections 2024-04-25T01:36:40.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:36:40.353+0000 7f0f35296700 1 -- 172.21.15.33:0/1689635883 wait complete. 2024-04-25T01:36:40.679 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:40 smithi033 ceph-mon[26353]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:40.680 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:40 smithi033 ceph-mon[26353]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:40.680 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:40 smithi033 ceph-mon[26353]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:40.680 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:40 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/70171106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:36:40.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:40 smithi138 ceph-mon[32977]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:40.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:40 smithi138 ceph-mon[32977]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:40.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:40 smithi138 ceph-mon[32977]: from='client.14648 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:40.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:40 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/70171106' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:36:41.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:41 smithi033 ceph-mon[26353]: pgmap v36: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 414 op/s 2024-04-25T01:36:41.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:41 smithi033 ceph-mon[26353]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:41.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:41 smithi138 ceph-mon[32977]: pgmap v36: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 414 op/s 2024-04-25T01:36:41.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:41 smithi138 ceph-mon[32977]: from='client.14656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:36:43.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:43 smithi033 ceph-mon[26353]: pgmap v37: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 516 op/s 2024-04-25T01:36:43.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:43 smithi138 ceph-mon[32977]: pgmap v37: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 516 op/s 2024-04-25T01:36:44.696 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:44 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:36:44.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:44 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:36:45.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:45 smithi033 ceph-mon[26353]: pgmap v38: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 405 op/s 2024-04-25T01:36:45.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:45 smithi138 ceph-mon[32977]: pgmap v38: 65 pgs: 65 active+clean; 286 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 405 op/s 2024-04-25T01:36:47.493 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:47 smithi033 ceph-mon[26353]: pgmap v39: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 610 op/s 2024-04-25T01:36:47.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:47 smithi138 ceph-mon[32977]: pgmap v39: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.3 MiB/s wr, 610 op/s 2024-04-25T01:36:49.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:49 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:49.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:49 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:49.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:49 smithi033 ceph-mon[26353]: Reconfiguring prometheus.smithi033 (dependencies changed)... 2024-04-25T01:36:49.397 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:49 smithi033 ceph-mon[26353]: Reconfiguring daemon prometheus.smithi033 on smithi033 2024-04-25T01:36:49.397 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:49 smithi033 ceph-mon[26353]: pgmap v40: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 438 op/s 2024-04-25T01:36:49.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:49 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:49.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:49 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:49.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:49 smithi138 ceph-mon[32977]: Reconfiguring prometheus.smithi033 (dependencies changed)... 2024-04-25T01:36:49.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:49 smithi138 ceph-mon[32977]: Reconfiguring daemon prometheus.smithi033 on smithi033 2024-04-25T01:36:49.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:49 smithi138 ceph-mon[32977]: pgmap v40: 65 pgs: 65 active+clean; 285 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 438 op/s 2024-04-25T01:36:50.242 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:50 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:36:50.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:50 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:36:51.332 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:51 smithi033 ceph-mon[26353]: pgmap v41: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 496 op/s 2024-04-25T01:36:51.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:51 smithi138 ceph-mon[32977]: pgmap v41: 65 pgs: 65 active+clean; 287 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 496 op/s 2024-04-25T01:36:53.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:53 smithi033 ceph-mon[26353]: pgmap v42: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 541 op/s 2024-04-25T01:36:53.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:53 smithi138 ceph-mon[32977]: pgmap v42: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 541 op/s 2024-04-25T01:36:55.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:55 smithi138 ceph-mon[32977]: pgmap v43: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 389 op/s 2024-04-25T01:36:55.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:55 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:55.517 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:55 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:55.741 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:55 smithi033 ceph-mon[26353]: pgmap v43: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 389 op/s 2024-04-25T01:36:55.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:55 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:55.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:55 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:36:56.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:56 smithi033 ceph-mon[26353]: Reconfiguring node-exporter.smithi138 (dependencies changed)... 2024-04-25T01:36:56.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:56 smithi033 ceph-mon[26353]: Reconfiguring daemon node-exporter.smithi138 on smithi138 2024-04-25T01:36:56.759 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:56 smithi138 ceph-mon[32977]: Reconfiguring node-exporter.smithi138 (dependencies changed)... 2024-04-25T01:36:56.759 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:56 smithi138 ceph-mon[32977]: Reconfiguring daemon node-exporter.smithi138 on smithi138 2024-04-25T01:36:57.643 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:57 smithi033 ceph-mon[26353]: pgmap v44: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 668 op/s 2024-04-25T01:36:57.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:57 smithi138 ceph-mon[32977]: pgmap v44: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 668 op/s 2024-04-25T01:36:59.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:36:59 smithi033 ceph-mon[26353]: pgmap v45: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 463 op/s 2024-04-25T01:36:59.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:36:59 smithi138 ceph-mon[32977]: pgmap v45: 65 pgs: 65 active+clean; 271 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 463 op/s 2024-04-25T01:37:01.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:01 smithi033 ceph-mon[26353]: pgmap v46: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 565 op/s 2024-04-25T01:37:01.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:01 smithi138 ceph-mon[32977]: pgmap v46: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 565 op/s 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: pgmap v47: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 608 op/s 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033.front.sepia.ceph.com:9093"}]: dispatch 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033.front.sepia.ceph.com:3000"}]: dispatch 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:37:03.695 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033.front.sepia.ceph.com:9095"}]: dispatch 2024-04-25T01:37:03.696 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.696 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:03 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:03.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: pgmap v47: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 608 op/s 2024-04-25T01:37:03.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033.front.sepia.ceph.com:9093"}]: dispatch 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033.front.sepia.ceph.com:3000"}]: dispatch 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033.front.sepia.ceph.com:9095"}]: dispatch 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:03.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:03 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:04.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:04 smithi033 ceph-mon[26353]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:37:04.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:04 smithi033 ceph-mon[26353]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033.front.sepia.ceph.com:9093"}]: dispatch 2024-04-25T01:37:04.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:04 smithi033 ceph-mon[26353]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:37:04.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:04 smithi033 ceph-mon[26353]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033.front.sepia.ceph.com:3000"}]: dispatch 2024-04-25T01:37:04.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:04 smithi033 ceph-mon[26353]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:37:04.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:04 smithi033 ceph-mon[26353]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033.front.sepia.ceph.com:9095"}]: dispatch 2024-04-25T01:37:04.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:04 smithi033 ceph-mon[26353]: Upgrade: Need to upgrade myself (mgr.smithi138.mmfhyi) 2024-04-25T01:37:04.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:04 smithi033 ceph-mon[26353]: Upgrade: Need to upgrade myself (mgr.smithi138.mmfhyi) 2024-04-25T01:37:04.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:04 smithi138 ceph-mon[32977]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:37:04.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:04 smithi138 ceph-mon[32977]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033.front.sepia.ceph.com:9093"}]: dispatch 2024-04-25T01:37:04.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:04 smithi138 ceph-mon[32977]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:37:04.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:04 smithi138 ceph-mon[32977]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033.front.sepia.ceph.com:3000"}]: dispatch 2024-04-25T01:37:04.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:04 smithi138 ceph-mon[32977]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:37:04.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:04 smithi138 ceph-mon[32977]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033.front.sepia.ceph.com:9095"}]: dispatch 2024-04-25T01:37:04.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:04 smithi138 ceph-mon[32977]: Upgrade: Need to upgrade myself (mgr.smithi138.mmfhyi) 2024-04-25T01:37:04.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:04 smithi138 ceph-mon[32977]: Upgrade: Need to upgrade myself (mgr.smithi138.mmfhyi) 2024-04-25T01:37:05.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:05 smithi033 ceph-mon[26353]: pgmap v48: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 481 op/s 2024-04-25T01:37:05.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:05 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:05.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:05 smithi138 ceph-mon[32977]: pgmap v48: 65 pgs: 65 active+clean; 273 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 481 op/s 2024-04-25T01:37:05.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:05 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:07.688 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:07 smithi033 ceph-mon[26353]: pgmap v49: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 700 op/s 2024-04-25T01:37:07.689 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:07 smithi033 ceph-mon[26353]: mgrmap e26: smithi138.mmfhyi(active, since 92s), standbys: smithi033.asemba 2024-04-25T01:37:07.689 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:07 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:07.689 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:07 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.asemba", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:07.689 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:07 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.asemba", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:07.689 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:07 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:07.689 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:07 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:07.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:07 smithi138 ceph-mon[32977]: pgmap v49: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 2.5 MiB/s rd, 2.4 MiB/s wr, 700 op/s 2024-04-25T01:37:07.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:07 smithi138 ceph-mon[32977]: mgrmap e26: smithi138.mmfhyi(active, since 92s), standbys: smithi033.asemba 2024-04-25T01:37:07.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:07 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:07.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:07 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.asemba", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:07.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:07 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.asemba", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:07.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:07 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:07.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:07 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:08.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:08 smithi033 ceph-mon[26353]: Upgrade: Updating mgr.smithi033.asemba 2024-04-25T01:37:08.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:08 smithi033 ceph-mon[26353]: Deploying daemon mgr.smithi033.asemba on smithi033 2024-04-25T01:37:08.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:08 smithi138 ceph-mon[32977]: Upgrade: Updating mgr.smithi033.asemba 2024-04-25T01:37:08.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:08 smithi138 ceph-mon[32977]: Deploying daemon mgr.smithi033.asemba on smithi033 2024-04-25T01:37:09.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:09 smithi033 ceph-mon[26353]: pgmap v50: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 421 op/s 2024-04-25T01:37:09.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:09 smithi138 ceph-mon[32977]: pgmap v50: 65 pgs: 65 active+clean; 274 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 421 op/s 2024-04-25T01:37:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.485+0000 7fcddee38700 1 -- 172.21.15.33:0/2911876875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd80fcf60 msgr2=0x7fcdd80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.485+0000 7fcddee38700 1 --2- 172.21.15.33:0/2911876875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd80fcf60 0x7fcdd80fd3d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4004940 tx=0x7fcdc4009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.485+0000 7fcddee38700 1 -- 172.21.15.33:0/2911876875 shutdown_connections 2024-04-25T01:37:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.485+0000 7fcddee38700 1 --2- 172.21.15.33:0/2911876875 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd80fcf60 0x7fcdd80fd3d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.485+0000 7fcddee38700 1 --2- 172.21.15.33:0/2911876875 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd80fbcb0 0x7fcdd80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.485+0000 7fcddee38700 1 -- 172.21.15.33:0/2911876875 >> 172.21.15.33:0/2911876875 conn(0x7fcdd80f7430 msgr2=0x7fcdd80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:10.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.485+0000 7fcddee38700 1 -- 172.21.15.33:0/2911876875 shutdown_connections 2024-04-25T01:37:10.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.485+0000 7fcddee38700 1 -- 172.21.15.33:0/2911876875 wait complete. 2024-04-25T01:37:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.486+0000 7fcddee38700 1 Processor -- start 2024-04-25T01:37:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.486+0000 7fcddee38700 1 -- start start 2024-04-25T01:37:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.486+0000 7fcddee38700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd80fbcb0 0x7fcdd80ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.486+0000 7fcddee38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd80fcf60 0x7fcdd80fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.486+0000 7fcddee38700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80fe830 con 0x7fcdd80fcf60 2024-04-25T01:37:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.486+0000 7fcddee38700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd80fe970 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.487+0000 7fcddcbd4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd80fbcb0 0x7fcdd80ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.487+0000 7fcddcbd4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd80fbcb0 0x7fcdd80ffd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:47962/0 (socket says 172.21.15.33:47962) 2024-04-25T01:37:10.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.487+0000 7fcddcbd4700 1 -- 172.21.15.33:0/2656343525 learned_addr learned my addr 172.21.15.33:0/2656343525 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.487+0000 7fcdd7fff700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd80fcf60 0x7fcdd80fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcddcbd4700 1 -- 172.21.15.33:0/2656343525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd80fcf60 msgr2=0x7fcdd80fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcddcbd4700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd80fcf60 0x7fcdd80fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcddcbd4700 1 -- 172.21.15.33:0/2656343525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc4005040 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcdd7fff700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd80fcf60 0x7fcdd80fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:37:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcddcbd4700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd80fbcb0 0x7fcdd80ffd80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00eb40 tx=0x7fcdcc00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcdd57fa700 1 -- 172.21.15.33:0/2656343525 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdcc007cc0 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcdd57fa700 1 -- 172.21.15.33:0/2656343525 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdcc00f040 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcdd57fa700 1 -- 172.21.15.33:0/2656343525 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdcc009620 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd80febf0 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.488+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd80ff0b0 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.489+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd8101af0 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.490+0000 7fcdd57fa700 1 -- 172.21.15.33:0/2656343525 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) v1 ==== 89279+0+0 (secure 0 0 0) 0x7fcdcc009780 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.491+0000 7fcdd57fa700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fcdc806b670 0x7fcdc806db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.491+0000 7fcdd57fa700 1 -- 172.21.15.33:0/2656343525 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fcdcc089e70 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.491+0000 7fcdd7fff700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fcdc806b670 0x7fcdc806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:10.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.492+0000 7fcdd7fff700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fcdc806b670 0x7fcdc806db20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009b30 tx=0x7fcdc4009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:10.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.496+0000 7fcdd57fa700 1 -- 172.21.15.33:0/2656343525 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fcdcc050100 con 0x7fcdd80fbcb0 2024-04-25T01:37:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.925+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdd8104910 con 0x7fcdc806b670 2024-04-25T01:37:10.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.927+0000 7fcdd57fa700 1 -- 172.21.15.33:0/2656343525 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fcdd8104910 con 0x7fcdc806b670 2024-04-25T01:37:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.929+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fcdc806b670 msgr2=0x7fcdc806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.929+0000 7fcddee38700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fcdc806b670 0x7fcdc806db20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4009b30 tx=0x7fcdc4009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.929+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd80fbcb0 msgr2=0x7fcdd80ffd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:10.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.929+0000 7fcddee38700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd80fbcb0 0x7fcdd80ffd80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc00eb40 tx=0x7fcdcc00b040 comp rx=0 tx=0).stop 2024-04-25T01:37:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.929+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 shutdown_connections 2024-04-25T01:37:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.929+0000 7fcddee38700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fcdc806b670 0x7fcdc806db20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.929+0000 7fcddee38700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdd80fbcb0 0x7fcdd80ffd80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.929+0000 7fcddee38700 1 --2- 172.21.15.33:0/2656343525 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdd80fcf60 0x7fcdd80fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.929+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 >> 172.21.15.33:0/2656343525 conn(0x7fcdd80f7430 msgr2=0x7fcdd8065b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.930+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 shutdown_connections 2024-04-25T01:37:10.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:10.930+0000 7fcddee38700 1 -- 172.21.15.33:0/2656343525 wait complete. 2024-04-25T01:37:10.918 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:37:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.103+0000 7fb82d039700 1 -- 172.21.15.33:0/4012887972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fb420 msgr2=0x7fb8280fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.103+0000 7fb82d039700 1 --2- 172.21.15.33:0/4012887972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fb420 0x7fb8280fb830 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb818004940 tx=0x7fb818009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.103+0000 7fb82d039700 1 -- 172.21.15.33:0/4012887972 shutdown_connections 2024-04-25T01:37:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.103+0000 7fb82d039700 1 --2- 172.21.15.33:0/4012887972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8280fc6d0 0x7fb8280fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.103+0000 7fb82d039700 1 --2- 172.21.15.33:0/4012887972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fb420 0x7fb8280fb830 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.103+0000 7fb82d039700 1 -- 172.21.15.33:0/4012887972 >> 172.21.15.33:0/4012887972 conn(0x7fb8280f6b80 msgr2=0x7fb8280f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:11.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.103+0000 7fb82d039700 1 -- 172.21.15.33:0/4012887972 shutdown_connections 2024-04-25T01:37:11.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.103+0000 7fb82d039700 1 -- 172.21.15.33:0/4012887972 wait complete. 2024-04-25T01:37:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.104+0000 7fb82d039700 1 Processor -- start 2024-04-25T01:37:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.104+0000 7fb82d039700 1 -- start start 2024-04-25T01:37:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.104+0000 7fb82d039700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8280fb420 0x7fb8280f8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.104+0000 7fb82d039700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fc6d0 0x7fb8280f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:11.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.104+0000 7fb82d039700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280f7a50 con 0x7fb8280fc6d0 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.104+0000 7fb82d039700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8280f7bc0 con 0x7fb8280fb420 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb82659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fc6d0 0x7fb8280f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb82659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fc6d0 0x7fb8280f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48770/0 (socket says 172.21.15.33:48770) 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb82659c700 1 -- 172.21.15.33:0/1511774663 learned_addr learned my addr 172.21.15.33:0/1511774663 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb82659c700 1 -- 172.21.15.33:0/1511774663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8280fb420 msgr2=0x7fb8280f8fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb82659c700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8280fb420 0x7fb8280f8fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb82659c700 1 -- 172.21.15.33:0/1511774663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c005040 con 0x7fb8280fc6d0 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb82659c700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fc6d0 0x7fb8280f74a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0096d0 tx=0x7fb81c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb80f7fe700 1 -- 172.21.15.33:0/1511774663 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb81c00e840 con 0x7fb8280fc6d0 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb80f7fe700 1 -- 172.21.15.33:0/1511774663 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb81c00f040 con 0x7fb8280fc6d0 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb80f7fe700 1 -- 172.21.15.33:0/1511774663 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb81c013e50 con 0x7fb8280fc6d0 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.105+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb818005040 con 0x7fb8280fc6d0 2024-04-25T01:37:11.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.106+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8280f81c0 con 0x7fb8280fc6d0 2024-04-25T01:37:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.107+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb828107ac0 con 0x7fb8280fc6d0 2024-04-25T01:37:11.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.108+0000 7fb80f7fe700 1 -- 172.21.15.33:0/1511774663 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 89279+0+0 (secure 0 0 0) 0x7fb81c00e9a0 con 0x7fb8280fc6d0 2024-04-25T01:37:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.108+0000 7fb80f7fe700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fb81006b810 0x7fb81006dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.108+0000 7fb80f7fe700 1 -- 172.21.15.33:0/1511774663 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fb81c089f30 con 0x7fb8280fc6d0 2024-04-25T01:37:11.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.108+0000 7fb826d9d700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fb81006b810 0x7fb81006dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:11.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.110+0000 7fb826d9d700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fb81006b810 0x7fb81006dcc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb818013870 tx=0x7fb818009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:11.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.114+0000 7fb80f7fe700 1 -- 172.21.15.33:0/1511774663 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fb81c053cc0 con 0x7fb8280fc6d0 2024-04-25T01:37:11.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.452+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb82805e3a0 con 0x7fb81006b810 2024-04-25T01:37:11.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.453+0000 7fb80f7fe700 1 -- 172.21.15.33:0/1511774663 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fb82805e3a0 con 0x7fb81006b810 2024-04-25T01:37:11.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.455+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fb81006b810 msgr2=0x7fb81006dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fb81006b810 0x7fb81006dcc0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb818013870 tx=0x7fb818009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fc6d0 msgr2=0x7fb8280f74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fc6d0 0x7fb8280f74a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0096d0 tx=0x7fb81c009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 shutdown_connections 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fb81006b810 0x7fb81006dcc0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb8280fb420 0x7fb8280f8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 --2- 172.21.15.33:0/1511774663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb8280fc6d0 0x7fb8280f74a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 >> 172.21.15.33:0/1511774663 conn(0x7fb8280f6b80 msgr2=0x7fb828100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 shutdown_connections 2024-04-25T01:37:11.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.456+0000 7fb82d039700 1 -- 172.21.15.33:0/1511774663 wait complete. 2024-04-25T01:37:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.586+0000 7f90ff387700 1 -- 172.21.15.33:0/2446142163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f8066220 msgr2=0x7f90f80ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.586+0000 7f90ff387700 1 --2- 172.21.15.33:0/2446142163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f8066220 0x7f90f80ffa40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f90f4004940 tx=0x7f90f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.586+0000 7f90ff387700 1 -- 172.21.15.33:0/2446142163 shutdown_connections 2024-04-25T01:37:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.586+0000 7f90ff387700 1 --2- 172.21.15.33:0/2446142163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f80fff80 0x7f90f8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.586+0000 7f90ff387700 1 --2- 172.21.15.33:0/2446142163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f8066220 0x7f90f80ffa40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.586+0000 7f90ff387700 1 -- 172.21.15.33:0/2446142163 >> 172.21.15.33:0/2446142163 conn(0x7f90f80f7430 msgr2=0x7f90f80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:11.564 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:11 smithi033 ceph-mon[26353]: pgmap v51: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 541 op/s 2024-04-25T01:37:11.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90ff387700 1 -- 172.21.15.33:0/2446142163 shutdown_connections 2024-04-25T01:37:11.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90ff387700 1 -- 172.21.15.33:0/2446142163 wait complete. 2024-04-25T01:37:11.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90ff387700 1 Processor -- start 2024-04-25T01:37:11.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90ff387700 1 -- start start 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90ff387700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f8066220 0x7f90f810da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90ff387700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fff80 0x7f90f810df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90ff387700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f810e570 con 0x7f90f80fff80 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90ff387700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f810edc0 con 0x7f90f8066220 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90fc922700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fff80 0x7f90f810df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90fc922700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fff80 0x7f90f810df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48784/0 (socket says 172.21.15.33:48784) 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90fc922700 1 -- 172.21.15.33:0/1974038346 learned_addr learned my addr 172.21.15.33:0/1974038346 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.587+0000 7f90fd123700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f8066220 0x7f90f810da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.588+0000 7f90fc922700 1 -- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f8066220 msgr2=0x7f90f810da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.588+0000 7f90fc922700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f8066220 0x7f90f810da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.588+0000 7f90fc922700 1 -- 172.21.15.33:0/1974038346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4005040 con 0x7f90f80fff80 2024-04-25T01:37:11.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.588+0000 7f90fc922700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fff80 0x7f90f810df70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f90ec002a50 tx=0x7f90ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:11.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.588+0000 7f90e9ffb700 1 -- 172.21.15.33:0/1974038346 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ec007cc0 con 0x7f90f80fff80 2024-04-25T01:37:11.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.588+0000 7f90e9ffb700 1 -- 172.21.15.33:0/1974038346 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90ec00f040 con 0x7f90f80fff80 2024-04-25T01:37:11.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.588+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f810f0a0 con 0x7f90f80fff80 2024-04-25T01:37:11.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.588+0000 7f90e9ffb700 1 -- 172.21.15.33:0/1974038346 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ec00da30 con 0x7f90f80fff80 2024-04-25T01:37:11.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.588+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f810f5e0 con 0x7f90f80fff80 2024-04-25T01:37:11.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.589+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f805f5c0 con 0x7f90f80fff80 2024-04-25T01:37:11.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.590+0000 7f90e9ffb700 1 -- 172.21.15.33:0/1974038346 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 89279+0+0 (secure 0 0 0) 0x7f90ec00db90 con 0x7f90f80fff80 2024-04-25T01:37:11.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.590+0000 7f90e9ffb700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f90e406b670 0x7f90e406db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:11.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.590+0000 7f90e9ffb700 1 -- 172.21.15.33:0/1974038346 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f90ec089e40 con 0x7f90f80fff80 2024-04-25T01:37:11.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.590+0000 7f90fd123700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f90e406b670 0x7f90e406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:11.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.591+0000 7f90fd123700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f90e406b670 0x7f90e406db20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f90f4004890 tx=0x7f90f4009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:11.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.595+0000 7f90e9ffb700 1 -- 172.21.15.33:0/1974038346 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f90ec057720 con 0x7f90f80fff80 2024-04-25T01:37:11.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:11 smithi138 ceph-mon[32977]: pgmap v51: 65 pgs: 65 active+clean; 270 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 541 op/s 2024-04-25T01:37:11.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.857+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f90f810f960 con 0x7f90e406b670 2024-04-25T01:37:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.867+0000 7f90e9ffb700 1 -- 172.21.15.33:0/1974038346 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f90f810f960 con 0x7f90e406b670 2024-04-25T01:37:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:37:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 starting - - - - 2024-04-25T01:37:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (9m) 65s ago 9m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:37:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (8m) 86s ago 8m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:37:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 starting - - - - 2024-04-25T01:37:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (4m) 65s ago 4m 61.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:37:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (4m) 65s ago 4m 99.5M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:37:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (4m) 86s ago 4m 110M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (4m) 86s ago 4m 67.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:9283 running (10m) 65s ago 10m 390M - 17.2.7-902-gc61472c8 0de407c1b3af 22f7bb21dc39 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283 running (2m) 86s ago 8m 488M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8ce48ef47e1e 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (10m) 65s ago 11m 48.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (8m) 86s ago 8m 41.3M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 starting - - - - 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 starting - - - - 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (7m) 65s ago 7m 126M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (7m) 65s ago 7m 146M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (6m) 65s ago 6m 124M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (6m) 86s ago 6m 156M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (6m) 86s ago 6m 135M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:37:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (5m) 86s ago 5m 128M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:37:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 starting - - - - 2024-04-25T01:37:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.869+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f90e406b670 msgr2=0x7f90e406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.869+0000 7f90ff387700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f90e406b670 0x7f90e406db20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f90f4004890 tx=0x7f90f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:11.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.869+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fff80 msgr2=0x7f90f810df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.869+0000 7f90ff387700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fff80 0x7f90f810df70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f90ec002a50 tx=0x7f90ec0046b0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.869+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 shutdown_connections 2024-04-25T01:37:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.869+0000 7f90ff387700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7f90e406b670 0x7f90e406db20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.869+0000 7f90ff387700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f90f8066220 0x7f90f810da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.869+0000 7f90ff387700 1 --2- 172.21.15.33:0/1974038346 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f90f80fff80 0x7f90f810df70 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.869+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 >> 172.21.15.33:0/1974038346 conn(0x7f90f80f7430 msgr2=0x7f90f80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.870+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 shutdown_connections 2024-04-25T01:37:11.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.870+0000 7f90ff387700 1 -- 172.21.15.33:0/1974038346 wait complete. 2024-04-25T01:37:11.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.982+0000 7fdcd2c44700 1 -- 172.21.15.33:0/2957902520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdccc066b50 msgr2=0x7fdccc0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.982+0000 7fdcd2c44700 1 --2- 172.21.15.33:0/2957902520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdccc066b50 0x7fdccc0ffa30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0004940 tx=0x7fdcc0009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.982+0000 7fdcd2c44700 1 -- 172.21.15.33:0/2957902520 shutdown_connections 2024-04-25T01:37:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.982+0000 7fdcd2c44700 1 --2- 172.21.15.33:0/2957902520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdccc0fff70 0x7fdccc102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.982+0000 7fdcd2c44700 1 --2- 172.21.15.33:0/2957902520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdccc066b50 0x7fdccc0ffa30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.982+0000 7fdcd2c44700 1 -- 172.21.15.33:0/2957902520 >> 172.21.15.33:0/2957902520 conn(0x7fdccc0f74c0 msgr2=0x7fdccc0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.982+0000 7fdcd2c44700 1 -- 172.21.15.33:0/2957902520 shutdown_connections 2024-04-25T01:37:11.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.982+0000 7fdcd2c44700 1 -- 172.21.15.33:0/2957902520 wait complete. 2024-04-25T01:37:11.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.982+0000 7fdcd2c44700 1 Processor -- start 2024-04-25T01:37:11.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdcd2c44700 1 -- start start 2024-04-25T01:37:11.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdcd2c44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdccc066b50 0x7fdccc10fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:11.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdcd2c44700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdccc0fff70 0x7fdccc110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:11.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdcd2c44700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc110760 con 0x7fdccc066b50 2024-04-25T01:37:11.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdcd2c44700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc110f60 con 0x7fdccc0fff70 2024-04-25T01:37:11.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdccbfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdccc0fff70 0x7fdccc110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdcd09e0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdccc066b50 0x7fdccc10fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdccbfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdccc0fff70 0x7fdccc110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:48006/0 (socket says 172.21.15.33:48006) 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdccbfff700 1 -- 172.21.15.33:0/193375530 learned_addr learned my addr 172.21.15.33:0/193375530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdccbfff700 1 -- 172.21.15.33:0/193375530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdccc066b50 msgr2=0x7fdccc10fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdccbfff700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdccc066b50 0x7fdccc10fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdccbfff700 1 -- 172.21.15.33:0/193375530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc0005040 con 0x7fdccc0fff70 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.983+0000 7fdcd09e0700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdccc066b50 0x7fdccc10fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.984+0000 7fdccbfff700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdccc0fff70 0x7fdccc110160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800eb40 tx=0x7fdcb800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.984+0000 7fdcc97fa700 1 -- 172.21.15.33:0/193375530 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb8007cc0 con 0x7fdccc0fff70 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.984+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc1111e0 con 0x7fdccc0fff70 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.984+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc111720 con 0x7fdccc0fff70 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.984+0000 7fdcc97fa700 1 -- 172.21.15.33:0/193375530 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcb800f040 con 0x7fdccc0fff70 2024-04-25T01:37:11.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.984+0000 7fdcc97fa700 1 -- 172.21.15.33:0/193375530 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb800ca10 con 0x7fdccc0fff70 2024-04-25T01:37:11.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.985+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb00052e0 con 0x7fdccc0fff70 2024-04-25T01:37:11.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.986+0000 7fdcc97fa700 1 -- 172.21.15.33:0/193375530 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 26) v1 ==== 89279+0+0 (secure 0 0 0) 0x7fdcb800cb70 con 0x7fdccc0fff70 2024-04-25T01:37:11.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.986+0000 7fdcc97fa700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fdcbc06b670 0x7fdcbc06db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:11.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.986+0000 7fdcc97fa700 1 -- 172.21.15.33:0/193375530 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fdcb8089d00 con 0x7fdccc0fff70 2024-04-25T01:37:11.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.987+0000 7fdcd09e0700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fdcbc06b670 0x7fdcbc06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:11.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.987+0000 7fdcd09e0700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fdcbc06b670 0x7fdcbc06db20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0004890 tx=0x7fdcc0009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:11.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:11.991+0000 7fdcc97fa700 1 -- 172.21.15.33:0/193375530 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fdcb8050100 con 0x7fdccc0fff70 2024-04-25T01:37:12.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.418+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdcb0005da0 con 0x7fdccc0fff70 2024-04-25T01:37:12.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.418+0000 7fdcc97fa700 1 -- 172.21.15.33:0/193375530 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7fdcb8057630 con 0x7fdccc0fff70 2024-04-25T01:37:12.395 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:37:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:37:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:37:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:37:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:37:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:37:12.395 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:37:12.396 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:37:12.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fdcbc06b670 msgr2=0x7fdcbc06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:12.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fdcbc06b670 0x7fdcbc06db20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0004890 tx=0x7fdcc0009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:12.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdccc0fff70 msgr2=0x7fdccc110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:12.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdccc0fff70 0x7fdccc110160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800eb40 tx=0x7fdcb800b040 comp rx=0 tx=0).stop 2024-04-25T01:37:12.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 shutdown_connections 2024-04-25T01:37:12.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fdcbc06b670 0x7fdcbc06db20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:12.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdccc066b50 0x7fdccc10fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:12.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 --2- 172.21.15.33:0/193375530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdccc0fff70 0x7fdccc110160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:12.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 >> 172.21.15.33:0/193375530 conn(0x7fdccc0f74c0 msgr2=0x7fdccc0fbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:12.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 shutdown_connections 2024-04-25T01:37:12.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.421+0000 7fdcd2c44700 1 -- 172.21.15.33:0/193375530 wait complete. 2024-04-25T01:37:12.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.543+0000 7fa216e1e700 1 -- 172.21.15.33:0/2170030388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100fbd40 msgr2=0x7fa2100fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:12.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.543+0000 7fa216e1e700 1 --2- 172.21.15.33:0/2170030388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100fbd40 0x7fa2100fc150 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa204004940 tx=0x7fa204009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.543+0000 7fa216e1e700 1 -- 172.21.15.33:0/2170030388 shutdown_connections 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.543+0000 7fa216e1e700 1 --2- 172.21.15.33:0/2170030388 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100fcff0 0x7fa2100fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.543+0000 7fa216e1e700 1 --2- 172.21.15.33:0/2170030388 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100fbd40 0x7fa2100fc150 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.543+0000 7fa216e1e700 1 -- 172.21.15.33:0/2170030388 >> 172.21.15.33:0/2170030388 conn(0x7fa2100f74c0 msgr2=0x7fa2100f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.543+0000 7fa216e1e700 1 -- 172.21.15.33:0/2170030388 shutdown_connections 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.543+0000 7fa216e1e700 1 -- 172.21.15.33:0/2170030388 wait complete. 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.543+0000 7fa216e1e700 1 Processor -- start 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa216e1e700 1 -- start start 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa216e1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100fbd40 0x7fa21010da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa216e1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100fcff0 0x7fa21010df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa216e1e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21010e580 con 0x7fa2100fcff0 2024-04-25T01:37:12.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa216e1e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21010ed30 con 0x7fa2100fbd40 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa214bba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100fbd40 0x7fa21010da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa20ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100fcff0 0x7fa21010df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa214bba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100fbd40 0x7fa21010da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:48024/0 (socket says 172.21.15.33:48024) 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa214bba700 1 -- 172.21.15.33:0/2498969037 learned_addr learned my addr 172.21.15.33:0/2498969037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa20ffff700 1 -- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100fbd40 msgr2=0x7fa21010da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa20ffff700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100fbd40 0x7fa21010da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa20ffff700 1 -- 172.21.15.33:0/2498969037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204005040 con 0x7fa2100fcff0 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.544+0000 7fa214bba700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100fbd40 0x7fa21010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.545+0000 7fa20ffff700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100fcff0 0x7fa21010df80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fd0 tx=0x7fa1fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.545+0000 7fa20d7fa700 1 -- 172.21.15.33:0/2498969037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1fc004e90 con 0x7fa2100fcff0 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.545+0000 7fa20d7fa700 1 -- 172.21.15.33:0/2498969037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1fc00f040 con 0x7fa2100fcff0 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.545+0000 7fa20d7fa700 1 -- 172.21.15.33:0/2498969037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1fc00ca50 con 0x7fa2100fcff0 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.545+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21010efb0 con 0x7fa2100fcff0 2024-04-25T01:37:12.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.545+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21010f4a0 con 0x7fa2100fcff0 2024-04-25T01:37:12.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.546+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa210101b80 con 0x7fa2100fcff0 2024-04-25T01:37:12.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.547+0000 7fa20d7fa700 1 -- 172.21.15.33:0/2498969037 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 26) v1 ==== 89279+0+0 (secure 0 0 0) 0x7fa1fc007cc0 con 0x7fa2100fcff0 2024-04-25T01:37:12.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.547+0000 7fa20d7fa700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fa20006b670 0x7fa20006db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:12.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.548+0000 7fa20d7fa700 1 -- 172.21.15.33:0/2498969037 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa1fc089b90 con 0x7fa2100fcff0 2024-04-25T01:37:12.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.548+0000 7fa214bba700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fa20006b670 0x7fa20006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:12.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.548+0000 7fa214bba700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fa20006b670 0x7fa20006db20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa20400b0d0 tx=0x7fa204009e30 comp rx=0 tx=0).ready entity=mgr.24383 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:12.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.556+0000 7fa20d7fa700 1 -- 172.21.15.33:0/2498969037 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa1fc050100 con 0x7fa2100fcff0 2024-04-25T01:37:12.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:12 smithi033 ceph-mon[26353]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:12.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:12 smithi033 ceph-mon[26353]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:12.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:12 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/193375530' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:37:12.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:12 smithi138 ceph-mon[32977]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:12.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:12 smithi138 ceph-mon[32977]: from='client.14664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:12.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:12 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/193375530' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:37:12.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.852+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 --> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa21010f710 con 0x7fa20006b670 2024-04-25T01:37:12.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.854+0000 7fa20d7fa700 1 -- 172.21.15.33:0/2498969037 <== mgr.24383 v2:172.21.15.138:6828/1792929713 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fa21010f710 con 0x7fa20006b670 2024-04-25T01:37:12.831 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:37:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:37:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:37:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-25T01:37:12.831 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [], 2024-04-25T01:37:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "1/2 daemons upgraded", 2024-04-25T01:37:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mgr daemons", 2024-04-25T01:37:12.832 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:37:12.832 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:37:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fa20006b670 msgr2=0x7fa20006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fa20006b670 0x7fa20006db20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa20400b0d0 tx=0x7fa204009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100fcff0 msgr2=0x7fa21010df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100fcff0 0x7fa21010df80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc009fd0 tx=0x7fa1fc00b040 comp rx=0 tx=0).stop 2024-04-25T01:37:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 shutdown_connections 2024-04-25T01:37:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:6828/1792929713,v1:172.21.15.138:6829/1792929713] conn(0x7fa20006b670 0x7fa20006db20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2100fbd40 0x7fa21010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 --2- 172.21.15.33:0/2498969037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2100fcff0 0x7fa21010df80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:12.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 >> 172.21.15.33:0/2498969037 conn(0x7fa2100f74c0 msgr2=0x7fa210066350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:12.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 shutdown_connections 2024-04-25T01:37:12.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:12.857+0000 7fa216e1e700 1 -- 172.21.15.33:0/2498969037 wait complete. 2024-04-25T01:37:13.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:13 smithi033 ceph-mon[26353]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:13.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:13 smithi033 ceph-mon[26353]: pgmap v52: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 482 op/s 2024-04-25T01:37:13.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:13 smithi138 ceph-mon[32977]: from='client.14668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:13.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:13 smithi138 ceph-mon[32977]: pgmap v52: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 482 op/s 2024-04-25T01:37:14.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:14 smithi033 ceph-mon[26353]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:14.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:14 smithi138 ceph-mon[32977]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:15.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:15 smithi033 ceph-mon[26353]: pgmap v53: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 381 op/s 2024-04-25T01:37:15.767 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:15 smithi138 ceph-mon[32977]: pgmap v53: 65 pgs: 65 active+clean; 268 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 381 op/s 2024-04-25T01:37:17.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:17 smithi033 ceph-mon[26353]: pgmap v54: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 493 op/s 2024-04-25T01:37:17.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:17 smithi138 ceph-mon[32977]: pgmap v54: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 493 op/s 2024-04-25T01:37:19.452 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-25T01:37:19.453 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T01:37:19.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:19 smithi033 ceph-mon[26353]: pgmap v55: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 903 KiB/s rd, 909 KiB/s wr, 274 op/s 2024-04-25T01:37:19.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:19 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:19.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:19 smithi138 ceph-mon[32977]: pgmap v55: 65 pgs: 65 active+clean; 269 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 903 KiB/s rd, 909 KiB/s wr, 274 op/s 2024-04-25T01:37:19.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:19 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:19.777 DEBUG:teuthology.parallel:result is None 2024-04-25T01:37:21.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:21 smithi033 ceph-mon[26353]: pgmap v56: 65 pgs: 65 active+clean; 264 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 372 op/s 2024-04-25T01:37:21.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:21 smithi138 ceph-mon[32977]: pgmap v56: 65 pgs: 65 active+clean; 264 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 372 op/s 2024-04-25T01:37:23.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:23 smithi033 ceph-mon[26353]: pgmap v57: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-04-25T01:37:23.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:23 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:23.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:23 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:23.665 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:23 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:23.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:23 smithi138 ceph-mon[32977]: pgmap v57: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 299 op/s 2024-04-25T01:37:23.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:23 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:23.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:23 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:23.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:23 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:24.878 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:24 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:24.878 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:24 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:24.878 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:24 smithi138 ceph-mon[32977]: pgmap v58: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-04-25T01:37:24.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:24 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:24.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:24 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:24.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:24 smithi033 ceph-mon[26353]: pgmap v58: 65 pgs: 65 active+clean; 265 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 257 op/s 2024-04-25T01:37:26.854 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-25T01:37:26.855 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-25T01:37:27.253 DEBUG:teuthology.parallel:result is None 2024-04-25T01:37:27.253 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:37:27.291 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T01:37:27.291 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T01:37:27.347 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T01:37:27.347 DEBUG:teuthology.parallel:result is None 2024-04-25T01:37:27.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:27 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:27.518 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:27 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:27.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:27 smithi138 ceph-mon[32977]: pgmap v59: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 400 op/s 2024-04-25T01:37:27.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:27 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:27.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:27 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:27.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:27 smithi033 ceph-mon[26353]: pgmap v59: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 400 op/s 2024-04-25T01:37:29.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:29 smithi033 ceph-mon[26353]: pgmap v60: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 288 op/s 2024-04-25T01:37:29.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:29 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:29.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:29 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:29.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:29 smithi138 ceph-mon[32977]: pgmap v60: 65 pgs: 65 active+clean; 262 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.0 MiB/s rd, 1.1 MiB/s wr, 288 op/s 2024-04-25T01:37:29.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:29 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:29.768 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:29 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:31.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:31 smithi033 ceph-mon[26353]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 343 op/s 2024-04-25T01:37:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:31 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:31 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:31 smithi033 ceph-mon[26353]: Standby manager daemon smithi033.asemba restarted 2024-04-25T01:37:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:31 smithi033 ceph-mon[26353]: Standby manager daemon smithi033.asemba started 2024-04-25T01:37:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:31 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.asemba/crt"}]: dispatch 2024-04-25T01:37:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:31 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:31 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.asemba/key"}]: dispatch 2024-04-25T01:37:31.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:31 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:31 smithi138 ceph-mon[32977]: pgmap v61: 65 pgs: 65 active+clean; 258 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 343 op/s 2024-04-25T01:37:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:31 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:31 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:31 smithi138 ceph-mon[32977]: Standby manager daemon smithi033.asemba restarted 2024-04-25T01:37:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:31 smithi138 ceph-mon[32977]: Standby manager daemon smithi033.asemba started 2024-04-25T01:37:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:31 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.asemba/crt"}]: dispatch 2024-04-25T01:37:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:31 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:31 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi033.asemba/key"}]: dispatch 2024-04-25T01:37:31.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:31 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:32.658 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:32 smithi033 ceph-mon[26353]: mgrmap e27: smithi138.mmfhyi(active, since 117s), standbys: smithi033.asemba 2024-04-25T01:37:32.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:32 smithi138 ceph-mon[32977]: mgrmap e27: smithi138.mmfhyi(active, since 117s), standbys: smithi033.asemba 2024-04-25T01:37:33.728 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:33 smithi033 ceph-mon[26353]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 967 KiB/s rd, 978 KiB/s wr, 264 op/s 2024-04-25T01:37:33.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:33 smithi138 ceph-mon[32977]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 967 KiB/s rd, 978 KiB/s wr, 264 op/s 2024-04-25T01:37:34.743 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:34 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:34.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:34 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:35.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 617 KiB/s rd, 620 KiB/s wr, 216 op/s 2024-04-25T01:37:35.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:35.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:35.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:35.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:35.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:35.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr fail", "who": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr fail", "who": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: Activating manager daemon smithi033.asemba 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: osdmap e47: 6 total, 6 up, 6 in 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd='[{"prefix": "mgr fail", "who": "smithi138.mmfhyi"}]': finished 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: mgrmap e28: smithi033.asemba(active, starting, since 0.0093562s) 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:37:35.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: Manager daemon smithi033.asemba is now available 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:37:35.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/trash_purge_schedule"}]: dispatch 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 617 KiB/s rd, 620 KiB/s wr, 216 op/s 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.24383 172.21.15.138:0/3968177202' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr fail", "who": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "mgr fail", "who": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:37:35.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: Activating manager daemon smithi033.asemba 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: osdmap e47: 6 total, 6 up, 6 in 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.24383 ' entity='mgr.smithi138.mmfhyi' cmd='[{"prefix": "mgr fail", "who": "smithi138.mmfhyi"}]': finished 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: mgrmap e28: smithi033.asemba(active, starting, since 0.0093562s) 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:37:35.770 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: Manager daemon smithi033.asemba is now available 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:37:35.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/trash_purge_schedule"}]: dispatch 2024-04-25T01:37:36.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:36 smithi138 ceph-mon[32977]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-25T01:37:36.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:36 smithi138 ceph-mon[32977]: mgrmap e29: smithi033.asemba(active, since 1.0115s) 2024-04-25T01:37:36.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:36 smithi033 ceph-mon[26353]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-25T01:37:36.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:36 smithi033 ceph-mon[26353]: mgrmap e29: smithi033.asemba(active, since 1.0115s) 2024-04-25T01:37:37.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:37 smithi033 ceph-mon[26353]: [25/Apr/2024:01:37:36] ENGINE Bus STARTING 2024-04-25T01:37:37.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:37 smithi033 ceph-mon[26353]: [25/Apr/2024:01:37:36] ENGINE Serving on http://172.21.15.33:8765 2024-04-25T01:37:37.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:37 smithi033 ceph-mon[26353]: [25/Apr/2024:01:37:36] ENGINE Serving on https://172.21.15.33:7150 2024-04-25T01:37:37.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:37 smithi033 ceph-mon[26353]: [25/Apr/2024:01:37:36] ENGINE Bus STARTED 2024-04-25T01:37:37.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:37 smithi033 ceph-mon[26353]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-25T01:37:37.742 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:37 smithi033 ceph-mon[26353]: Cluster is now healthy 2024-04-25T01:37:37.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:37 smithi138 ceph-mon[32977]: [25/Apr/2024:01:37:36] ENGINE Bus STARTING 2024-04-25T01:37:37.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:37 smithi138 ceph-mon[32977]: [25/Apr/2024:01:37:36] ENGINE Serving on http://172.21.15.33:8765 2024-04-25T01:37:37.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:37 smithi138 ceph-mon[32977]: [25/Apr/2024:01:37:36] ENGINE Serving on https://172.21.15.33:7150 2024-04-25T01:37:37.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:37 smithi138 ceph-mon[32977]: [25/Apr/2024:01:37:36] ENGINE Bus STARTED 2024-04-25T01:37:37.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:37 smithi138 ceph-mon[32977]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-25T01:37:37.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:37 smithi138 ceph-mon[32977]: Cluster is now healthy 2024-04-25T01:37:38.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:38 smithi033 ceph-mon[26353]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:37:38.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:38 smithi033 ceph-mon[26353]: mgrmap e30: smithi033.asemba(active, since 2s) 2024-04-25T01:37:38.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:38 smithi138 ceph-mon[32977]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:37:38.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:38 smithi138 ceph-mon[32977]: mgrmap e30: smithi033.asemba(active, since 2s) 2024-04-25T01:37:39.708 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:39 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:39.708 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:39 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:39.709 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:39 smithi033 ceph-mon[26353]: mgrmap e31: smithi033.asemba(active, since 4s) 2024-04-25T01:37:39.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:39 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:39.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:39 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:39.769 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:39 smithi138 ceph-mon[32977]: mgrmap e31: smithi033.asemba(active, since 4s) 2024-04-25T01:37:40.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:40 smithi033 ceph-mon[26353]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:37:40.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:40 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:40.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:40 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:40.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:40 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:40.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:40 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:41.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:40 smithi138 ceph-mon[32977]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:37:41.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:40 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:41.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:40 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:41.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:40 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:41.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:40 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:42.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:42 smithi138 ceph-mon[32977]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:37:42.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:42 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:42.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:42 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:42.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:42 smithi138 ceph-mon[32977]: Standby manager daemon smithi138.mmfhyi started 2024-04-25T01:37:42.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:42 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/1440103186' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/crt"}]: dispatch 2024-04-25T01:37:42.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:42 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/1440103186' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:42.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:42 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/1440103186' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/key"}]: dispatch 2024-04-25T01:37:42.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:42 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/1440103186' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:42.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:42 smithi033 ceph-mon[26353]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:37:42.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:42 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:42.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:42 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:42.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:42 smithi033 ceph-mon[26353]: Standby manager daemon smithi138.mmfhyi started 2024-04-25T01:37:42.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:42 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/1440103186' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/crt"}]: dispatch 2024-04-25T01:37:42.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:42 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/1440103186' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:37:42.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:42 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/1440103186' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/key"}]: dispatch 2024-04-25T01:37:42.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:42 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/1440103186' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:37:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.972+0000 7fc3981b9700 1 -- 172.21.15.33:0/626803953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 msgr2=0x7fc3900faf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.972+0000 7fc3981b9700 1 --2- 172.21.15.33:0/626803953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 0x7fc3900faf80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc380004940 tx=0x7fc380009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:42.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.972+0000 7fc3981b9700 1 -- 172.21.15.33:0/626803953 shutdown_connections 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.972+0000 7fc3981b9700 1 --2- 172.21.15.33:0/626803953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 0x7fc3900faf80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.972+0000 7fc3981b9700 1 --2- 172.21.15.33:0/626803953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900fe4f0 0x7fc3900fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.972+0000 7fc3981b9700 1 -- 172.21.15.33:0/626803953 >> 172.21.15.33:0/626803953 conn(0x7fc3900f6bd0 msgr2=0x7fc3900f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.972+0000 7fc3981b9700 1 -- 172.21.15.33:0/626803953 shutdown_connections 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.972+0000 7fc3981b9700 1 -- 172.21.15.33:0/626803953 wait complete. 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc3981b9700 1 Processor -- start 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc3981b9700 1 -- start start 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc3981b9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900fe4f0 0x7fc39010fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc3981b9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 0x7fc390110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc3981b9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc390110610 con 0x7fc3900fee40 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc3981b9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc390110780 con 0x7fc3900fe4f0 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc395754700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 0x7fc390110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc395754700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 0x7fc390110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53340/0 (socket says 172.21.15.33:53340) 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc395754700 1 -- 172.21.15.33:0/748343985 learned_addr learned my addr 172.21.15.33:0/748343985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc395f55700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900fe4f0 0x7fc39010fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc395754700 1 -- 172.21.15.33:0/748343985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900fe4f0 msgr2=0x7fc39010fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc395754700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900fe4f0 0x7fc39010fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc395754700 1 -- 172.21.15.33:0/748343985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc380005040 con 0x7fc3900fee40 2024-04-25T01:37:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.973+0000 7fc395754700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 0x7fc390110010 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc380009150 tx=0x7fc380009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:42.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.974+0000 7fc3867fc700 1 -- 172.21.15.33:0/748343985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc38000e040 con 0x7fc3900fee40 2024-04-25T01:37:42.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.974+0000 7fc3867fc700 1 -- 172.21.15.33:0/748343985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc380004290 con 0x7fc3900fee40 2024-04-25T01:37:42.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.974+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc390115180 con 0x7fc3900fee40 2024-04-25T01:37:42.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.974+0000 7fc3867fc700 1 -- 172.21.15.33:0/748343985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc38001c630 con 0x7fc3900fee40 2024-04-25T01:37:42.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.974+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc390115740 con 0x7fc3900fee40 2024-04-25T01:37:42.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.975+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc390101c30 con 0x7fc3900fee40 2024-04-25T01:37:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.975+0000 7fc3867fc700 1 -- 172.21.15.33:0/748343985 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fc38000bbf0 con 0x7fc3900fee40 2024-04-25T01:37:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.976+0000 7fc3867fc700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fc37c06db70 0x7fc37c070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.976+0000 7fc3867fc700 1 -- 172.21.15.33:0/748343985 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fc38008dec0 con 0x7fc3900fee40 2024-04-25T01:37:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.976+0000 7fc395f55700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fc37c06db70 0x7fc37c070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:42.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.976+0000 7fc395f55700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fc37c06db70 0x7fc37c070020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc38c007b20 tx=0x7fc38c009e20 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:42.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:42.981+0000 7fc3867fc700 1 -- 172.21.15.33:0/748343985 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fc38005b520 con 0x7fc3900fee40 2024-04-25T01:37:43.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.243+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc39002b390 con 0x7fc37c06db70 2024-04-25T01:37:43.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.244+0000 7fc3867fc700 1 -- 172.21.15.33:0/748343985 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fc39002b390 con 0x7fc37c06db70 2024-04-25T01:37:43.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fc37c06db70 msgr2=0x7fc37c070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fc37c06db70 0x7fc37c070020 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc38c007b20 tx=0x7fc38c009e20 comp rx=0 tx=0).stop 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 msgr2=0x7fc390110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 0x7fc390110010 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fc380009150 tx=0x7fc380009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 shutdown_connections 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fc37c06db70 0x7fc37c070020 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc3900fe4f0 0x7fc39010fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 --2- 172.21.15.33:0/748343985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc3900fee40 0x7fc390110010 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 >> 172.21.15.33:0/748343985 conn(0x7fc3900f6bd0 msgr2=0x7fc3900f8140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 shutdown_connections 2024-04-25T01:37:43.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.246+0000 7fc3981b9700 1 -- 172.21.15.33:0/748343985 wait complete. 2024-04-25T01:37:43.235 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:37:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.352+0000 7f470dca6700 1 -- 172.21.15.33:0/4156824500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fed60 msgr2=0x7f47080faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.352+0000 7f470dca6700 1 --2- 172.21.15.33:0/4156824500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fed60 0x7f47080faea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f46f8004940 tx=0x7f46f8009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.352+0000 7f470dca6700 1 -- 172.21.15.33:0/4156824500 shutdown_connections 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.352+0000 7f470dca6700 1 --2- 172.21.15.33:0/4156824500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fed60 0x7f47080faea0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.352+0000 7f470dca6700 1 --2- 172.21.15.33:0/4156824500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47080fe410 0x7f47080fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.352+0000 7f470dca6700 1 -- 172.21.15.33:0/4156824500 >> 172.21.15.33:0/4156824500 conn(0x7f47080f6bd0 msgr2=0x7f47080f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.353+0000 7f470dca6700 1 -- 172.21.15.33:0/4156824500 shutdown_connections 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.353+0000 7f470dca6700 1 -- 172.21.15.33:0/4156824500 wait complete. 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.353+0000 7f470dca6700 1 Processor -- start 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.353+0000 7f470dca6700 1 -- start start 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.353+0000 7f470dca6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fe410 0x7f47080fda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.353+0000 7f470dca6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47080fed60 0x7f47080fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.353+0000 7f470dca6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47080fe020 con 0x7f47080fe410 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.353+0000 7f470dca6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47080fc610 con 0x7f47080fed60 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f47077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fe410 0x7f47080fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f47077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fe410 0x7f47080fda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53356/0 (socket says 172.21.15.33:53356) 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f47077fe700 1 -- 172.21.15.33:0/3309321146 learned_addr learned my addr 172.21.15.33:0/3309321146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f4706ffd700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47080fed60 0x7f47080fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f47077fe700 1 -- 172.21.15.33:0/3309321146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47080fed60 msgr2=0x7f47080fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f47077fe700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47080fed60 0x7f47080fc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f47077fe700 1 -- 172.21.15.33:0/3309321146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f8005040 con 0x7f47080fe410 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f47077fe700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fe410 0x7f47080fda20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f46fc002a50 tx=0x7f46fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f46effff700 1 -- 172.21.15.33:0/3309321146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46fc007cc0 con 0x7f47080fe410 2024-04-25T01:37:43.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f46effff700 1 -- 172.21.15.33:0/3309321146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46fc00f040 con 0x7f47080fe410 2024-04-25T01:37:43.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f46effff700 1 -- 172.21.15.33:0/3309321146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46fc00da30 con 0x7f47080fe410 2024-04-25T01:37:43.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47080fc8c0 con 0x7f47080fe410 2024-04-25T01:37:43.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.354+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47080fce30 con 0x7f47080fe410 2024-04-25T01:37:43.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.355+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470804c710 con 0x7f47080fe410 2024-04-25T01:37:43.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.356+0000 7f46effff700 1 -- 172.21.15.33:0/3309321146 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f46fc00db90 con 0x7f47080fe410 2024-04-25T01:37:43.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.356+0000 7f46effff700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f46f006db70 0x7f46f0070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.356+0000 7f46effff700 1 -- 172.21.15.33:0/3309321146 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f46fc08d180 con 0x7f47080fe410 2024-04-25T01:37:43.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.356+0000 7f4706ffd700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f46f006db70 0x7f46f0070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:43.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.357+0000 7f4706ffd700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f46f006db70 0x7f46f0070020 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009fd0 tx=0x7f46f8009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:43.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.361+0000 7f46effff700 1 -- 172.21.15.33:0/3309321146 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f46fc05a6c0 con 0x7f47080fe410 2024-04-25T01:37:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.592+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47081024a0 con 0x7f46f006db70 2024-04-25T01:37:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.593+0000 7f46effff700 1 -- 172.21.15.33:0/3309321146 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f47081024a0 con 0x7f46f006db70 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.595+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f46f006db70 msgr2=0x7f46f0070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.595+0000 7f470dca6700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f46f006db70 0x7f46f0070020 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009fd0 tx=0x7f46f8009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.595+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fe410 msgr2=0x7f47080fda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.595+0000 7f470dca6700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fe410 0x7f47080fda20 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f46fc002a50 tx=0x7f46fc0046b0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.596+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 shutdown_connections 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.596+0000 7f470dca6700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f46f006db70 0x7f46f0070020 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.596+0000 7f470dca6700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47080fe410 0x7f47080fda20 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.596+0000 7f470dca6700 1 --2- 172.21.15.33:0/3309321146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47080fed60 0x7f47080fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.596+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 >> 172.21.15.33:0/3309321146 conn(0x7f47080f6bd0 msgr2=0x7f47080f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.596+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 shutdown_connections 2024-04-25T01:37:43.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.596+0000 7f470dca6700 1 -- 172.21.15.33:0/3309321146 wait complete. 2024-04-25T01:37:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.702+0000 7f26a0acf700 1 -- 172.21.15.33:0/261426851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 msgr2=0x7f269c102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.702+0000 7f26a0acf700 1 --2- 172.21.15.33:0/261426851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 0x7f269c102340 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2684004940 tx=0x7f2684009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.702+0000 7f26a0acf700 1 -- 172.21.15.33:0/261426851 shutdown_connections 2024-04-25T01:37:43.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.702+0000 7f26a0acf700 1 --2- 172.21.15.33:0/261426851 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 0x7f269c102340 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.702+0000 7f26a0acf700 1 --2- 172.21.15.33:0/261426851 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c066e50 0x7f269c0ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.702+0000 7f26a0acf700 1 -- 172.21.15.33:0/261426851 >> 172.21.15.33:0/261426851 conn(0x7f269c0f7430 msgr2=0x7f269c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.702+0000 7f26a0acf700 1 -- 172.21.15.33:0/261426851 shutdown_connections 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.702+0000 7f26a0acf700 1 -- 172.21.15.33:0/261426851 wait complete. 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f26a0acf700 1 Processor -- start 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f26a0acf700 1 -- start start 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f26a0acf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c066e50 0x7f269c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f26a0acf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 0x7f269c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f26a0acf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c10e440 con 0x7f269c0fff60 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f26a0acf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c10e5b0 con 0x7f269c066e50 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f2699d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 0x7f269c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f2699d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 0x7f269c10de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53386/0 (socket says 172.21.15.33:53386) 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f2699d9b700 1 -- 172.21.15.33:0/1007335846 learned_addr learned my addr 172.21.15.33:0/1007335846 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:43.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f269a59c700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c066e50 0x7f269c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f2699d9b700 1 -- 172.21.15.33:0/1007335846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c066e50 msgr2=0x7f269c10d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f2699d9b700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c066e50 0x7f269c10d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.703+0000 7f2699d9b700 1 -- 172.21.15.33:0/1007335846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2684005040 con 0x7f269c0fff60 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.704+0000 7f2699d9b700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 0x7f269c10de40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f268400b3b0 tx=0x7f2684009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.704+0000 7f2692ffd700 1 -- 172.21.15.33:0/1007335846 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f268400e040 con 0x7f269c0fff60 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.704+0000 7f2692ffd700 1 -- 172.21.15.33:0/1007335846 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f268400bbf0 con 0x7f269c0fff60 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.704+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f269c112fb0 con 0x7f269c0fff60 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.704+0000 7f2692ffd700 1 -- 172.21.15.33:0/1007335846 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f268401c630 con 0x7f269c0fff60 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.704+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f269c113540 con 0x7f269c0fff60 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.705+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f269c04c710 con 0x7f269c0fff60 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.705+0000 7f2692ffd700 1 -- 172.21.15.33:0/1007335846 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f268400bd60 con 0x7f269c0fff60 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.706+0000 7f2692ffd700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f268806db70 0x7f2688070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:43.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.706+0000 7f2692ffd700 1 -- 172.21.15.33:0/1007335846 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f268408dfc0 con 0x7f269c0fff60 2024-04-25T01:37:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.706+0000 7f269a59c700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f268806db70 0x7f2688070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:43.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.706+0000 7f269a59c700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f268806db70 0x7f2688070020 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f269c066b90 tx=0x7f268c009040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:43.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.711+0000 7f2692ffd700 1 -- 172.21.15.33:0/1007335846 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f268405b580 con 0x7f269c0fff60 2024-04-25T01:37:43.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.928+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f269c1137f0 con 0x7f268806db70 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.939+0000 7f2692ffd700 1 -- 172.21.15.33:0/1007335846 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f269c1137f0 con 0x7f268806db70 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (83s) 3s ago 10m 18.4M - 0.25.0 c8568f914cd2 c74b90b8b6ca 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (10m) 3s ago 10m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (9m) 5s ago 9m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 unknown 3s ago 9m - - 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (4m) 3s ago 4m 61.8M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (4m) 3s ago 4m 114M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (4m) 5s ago 4m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (4m) 5s ago 4m 60.9M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:37:43.917 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (21s) 3s ago 11m 472M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283 running (2m) 5s ago 9m 73.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 8ce48ef47e1e 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (11m) 3s ago 11m 54.9M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (9m) 5s ago 9m 43.0M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (90s) 3s ago 10m 8014k - 1.5.0 0da6a335fe13 fc5ceaf7d933 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (41s) 5s ago 9m 7860k - 1.5.0 0da6a335fe13 f072f46b495e 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (8m) 3s ago 8m 221M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (7m) 3s ago 7m 234M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (7m) 3s ago 7m 188M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (7m) 5s ago 7m 262M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (6m) 5s ago 6m 230M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (6m) 5s ago 6m 216M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:37:43.918 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (49s) 3s ago 9m 44.5M - 2.43.0 a07b618ecd1d 6a19fffbbab6 2024-04-25T01:37:43.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f268806db70 msgr2=0x7f2688070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f268806db70 0x7f2688070020 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f269c066b90 tx=0x7f268c009040 comp rx=0 tx=0).stop 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 msgr2=0x7f269c10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 0x7f269c10de40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f268400b3b0 tx=0x7f2684009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 shutdown_connections 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f268806db70 0x7f2688070020 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f269c066e50 0x7f269c10d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 --2- 172.21.15.33:0/1007335846 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f269c0fff60 0x7f269c10de40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 >> 172.21.15.33:0/1007335846 conn(0x7f269c0f7430 msgr2=0x7f269c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 shutdown_connections 2024-04-25T01:37:43.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:43.941+0000 7f26a0acf700 1 -- 172.21.15.33:0/1007335846 wait complete. 2024-04-25T01:37:43.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:43 smithi033 ceph-mon[26353]: mgrmap e32: smithi033.asemba(active, since 7s), standbys: smithi138.mmfhyi 2024-04-25T01:37:43.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:43 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi138.mmfhyi", "id": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:37:44.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:43 smithi138 ceph-mon[32977]: mgrmap e32: smithi033.asemba(active, since 7s), standbys: smithi138.mmfhyi 2024-04-25T01:37:44.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:43 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi138.mmfhyi", "id": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:37:44.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.056+0000 7f25752c7700 1 -- 172.21.15.33:0/2134376289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 msgr2=0x7f25700fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.056+0000 7f25752c7700 1 --2- 172.21.15.33:0/2134376289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 0x7f25700fe7f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2560004940 tx=0x7f2560009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.056+0000 7f25752c7700 1 -- 172.21.15.33:0/2134376289 shutdown_connections 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.056+0000 7f25752c7700 1 --2- 172.21.15.33:0/2134376289 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25700fed30 0x7f25700fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.056+0000 7f25752c7700 1 --2- 172.21.15.33:0/2134376289 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 0x7f25700fe7f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.056+0000 7f25752c7700 1 -- 172.21.15.33:0/2134376289 >> 172.21.15.33:0/2134376289 conn(0x7f25700f6b80 msgr2=0x7f25700f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.056+0000 7f25752c7700 1 -- 172.21.15.33:0/2134376289 shutdown_connections 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.056+0000 7f25752c7700 1 -- 172.21.15.33:0/2134376289 wait complete. 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.057+0000 7f25752c7700 1 Processor -- start 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.057+0000 7f25752c7700 1 -- start start 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.057+0000 7f25752c7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 0x7f257010da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:44.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.057+0000 7f25752c7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25700fed30 0x7f257010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.057+0000 7f25752c7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257010e560 con 0x7f25700fe3e0 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.057+0000 7f25752c7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257010edb0 con 0x7f25700fed30 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.057+0000 7f256effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 0x7f257010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.057+0000 7f256effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 0x7f257010da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53402/0 (socket says 172.21.15.33:53402) 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.057+0000 7f256effd700 1 -- 172.21.15.33:0/2054733494 learned_addr learned my addr 172.21.15.33:0/2054733494 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f256e7fc700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25700fed30 0x7f257010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f256effd700 1 -- 172.21.15.33:0/2054733494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25700fed30 msgr2=0x7f257010df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f256effd700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25700fed30 0x7f257010df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f256effd700 1 -- 172.21.15.33:0/2054733494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2560005040 con 0x7f25700fe3e0 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f256e7fc700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25700fed30 0x7f257010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:37:44.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f256effd700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 0x7f257010da20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2560009150 tx=0x7f2560009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:44.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f25577fe700 1 -- 172.21.15.33:0/2054733494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256000e040 con 0x7f25700fe3e0 2024-04-25T01:37:44.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257010f030 con 0x7f25700fe3e0 2024-04-25T01:37:44.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f25577fe700 1 -- 172.21.15.33:0/2054733494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f256000bcc0 con 0x7f25700fe3e0 2024-04-25T01:37:44.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257010f570 con 0x7f25700fe3e0 2024-04-25T01:37:44.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.058+0000 7f25577fe700 1 -- 172.21.15.33:0/2054733494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256001c630 con 0x7f25700fe3e0 2024-04-25T01:37:44.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.059+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257005f5c0 con 0x7f25700fe3e0 2024-04-25T01:37:44.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.060+0000 7f25577fe700 1 -- 172.21.15.33:0/2054733494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f2560026440 con 0x7f25700fe3e0 2024-04-25T01:37:44.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.061+0000 7f25577fe700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f255806db60 0x7f2558070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:44.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.061+0000 7f25577fe700 1 -- 172.21.15.33:0/2054733494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f256008e100 con 0x7f25700fe3e0 2024-04-25T01:37:44.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.061+0000 7f256e7fc700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f255806db60 0x7f2558070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:44.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.061+0000 7f256e7fc700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f255806db60 0x7f2558070010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2564009fd0 tx=0x7f256400b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:44.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.065+0000 7f25577fe700 1 -- 172.21.15.33:0/2054733494 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f256005b730 con 0x7f25700fe3e0 2024-04-25T01:37:44.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.405+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2570105890 con 0x7f25700fe3e0 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.405+0000 7f25577fe700 1 -- 172.21.15.33:0/2054733494 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f256005b2c0 con 0x7f25700fe3e0 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:37:44.384 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:37:44.385 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:37:44.385 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:37:44.385 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:37:44.385 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:37:44.385 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-25T01:37:44.385 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:37:44.385 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:37:44.385 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:37:44.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f255806db60 msgr2=0x7f2558070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:44.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f255806db60 0x7f2558070010 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2564009fd0 tx=0x7f256400b040 comp rx=0 tx=0).stop 2024-04-25T01:37:44.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 msgr2=0x7f257010da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:44.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 0x7f257010da20 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2560009150 tx=0x7f2560009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:44.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 shutdown_connections 2024-04-25T01:37:44.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f255806db60 0x7f2558070010 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25700fe3e0 0x7f257010da20 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 --2- 172.21.15.33:0/2054733494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25700fed30 0x7f257010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 >> 172.21.15.33:0/2054733494 conn(0x7f25700f6b80 msgr2=0x7f25700f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:44.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 shutdown_connections 2024-04-25T01:37:44.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.408+0000 7f25752c7700 1 -- 172.21.15.33:0/2054733494 wait complete. 2024-04-25T01:37:44.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.522+0000 7f4bf537f700 1 -- 172.21.15.33:0/1769913652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 msgr2=0x7f4bf00fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:44.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.522+0000 7f4bf537f700 1 --2- 172.21.15.33:0/1769913652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 0x7f4bf00fd320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f4be0004940 tx=0x7f4be0009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:44.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.522+0000 7f4bf537f700 1 -- 172.21.15.33:0/1769913652 shutdown_connections 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.522+0000 7f4bf537f700 1 --2- 172.21.15.33:0/1769913652 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf00fe080 0x7f4bf00fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.522+0000 7f4bf537f700 1 --2- 172.21.15.33:0/1769913652 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 0x7f4bf00fd320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.522+0000 7f4bf537f700 1 -- 172.21.15.33:0/1769913652 >> 172.21.15.33:0/1769913652 conn(0x7f4bf00f8560 msgr2=0x7f4bf00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.522+0000 7f4bf537f700 1 -- 172.21.15.33:0/1769913652 shutdown_connections 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.522+0000 7f4bf537f700 1 -- 172.21.15.33:0/1769913652 wait complete. 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4bf537f700 1 Processor -- start 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4bf537f700 1 -- start start 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4bf537f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 0x7f4bf010d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4bf537f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf00fe080 0x7f4bf010df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4bf537f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf010e520 con 0x7f4bf00fcf10 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4bf537f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf010ed70 con 0x7f4bf00fe080 2024-04-25T01:37:44.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4beeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 0x7f4bf010d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4beeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 0x7f4bf010d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53422/0 (socket says 172.21.15.33:53422) 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4beeffd700 1 -- 172.21.15.33:0/3743527823 learned_addr learned my addr 172.21.15.33:0/3743527823 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4be7fff700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf00fe080 0x7f4bf010df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4beeffd700 1 -- 172.21.15.33:0/3743527823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf00fe080 msgr2=0x7f4bf010df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4beeffd700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf00fe080 0x7f4bf010df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.523+0000 7f4beeffd700 1 -- 172.21.15.33:0/3743527823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be0005040 con 0x7f4bf00fcf10 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.524+0000 7f4beeffd700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 0x7f4bf010d9e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009150 tx=0x7f4be0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.524+0000 7f4be77fe700 1 -- 172.21.15.33:0/3743527823 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be000e040 con 0x7f4bf00fcf10 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.524+0000 7f4be77fe700 1 -- 172.21.15.33:0/3743527823 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4be0004290 con 0x7f4bf00fcf10 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.524+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf010eff0 con 0x7f4bf00fcf10 2024-04-25T01:37:44.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.524+0000 7f4be77fe700 1 -- 172.21.15.33:0/3743527823 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be001c630 con 0x7f4bf00fcf10 2024-04-25T01:37:44.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.524+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf010f530 con 0x7f4bf00fcf10 2024-04-25T01:37:44.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.525+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf005f5c0 con 0x7f4bf00fcf10 2024-04-25T01:37:44.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.525+0000 7f4be77fe700 1 -- 172.21.15.33:0/3743527823 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 32) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f4be000bbf0 con 0x7f4bf00fcf10 2024-04-25T01:37:44.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.526+0000 7f4be77fe700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f4bdc06db60 0x7f4bdc070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:37:44.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.526+0000 7f4be77fe700 1 -- 172.21.15.33:0/3743527823 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f4be008deb0 con 0x7f4bf00fcf10 2024-04-25T01:37:44.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.526+0000 7f4be7fff700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f4bdc06db60 0x7f4bdc070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:37:44.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.527+0000 7f4be7fff700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f4bdc06db60 0x7f4bdc070010 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8004f40 tx=0x7f4bd800b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:37:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.531+0000 7f4be77fe700 1 -- 172.21.15.33:0/3743527823 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f4be005b510 con 0x7f4bf00fcf10 2024-04-25T01:37:44.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.755+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bf010f860 con 0x7f4bdc06db60 2024-04-25T01:37:44.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.756+0000 7f4be77fe700 1 -- 172.21.15.33:0/3743527823 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f4bf010f860 con 0x7f4bdc06db60 2024-04-25T01:37:44.735 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr" 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "2/2 daemons upgraded", 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "", 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:37:44.736 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:37:44.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f4bdc06db60 msgr2=0x7f4bdc070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:44.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f4bdc06db60 0x7f4bdc070010 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8004f40 tx=0x7f4bd800b040 comp rx=0 tx=0).stop 2024-04-25T01:37:44.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 msgr2=0x7f4bf010d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:37:44.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 0x7f4bf010d9e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009150 tx=0x7f4be0009e30 comp rx=0 tx=0).stop 2024-04-25T01:37:44.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 shutdown_connections 2024-04-25T01:37:44.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f4bdc06db60 0x7f4bdc070010 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf00fcf10 0x7f4bf010d9e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 --2- 172.21.15.33:0/3743527823 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf00fe080 0x7f4bf010df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:37:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 >> 172.21.15.33:0/3743527823 conn(0x7f4bf00f8560 msgr2=0x7f4bf00f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:37:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 shutdown_connections 2024-04-25T01:37:44.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:37:44.759+0000 7f4bf537f700 1 -- 172.21.15.33:0/3743527823 wait complete. 2024-04-25T01:37:44.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:44 smithi033 ceph-mon[26353]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-25T01:37:44.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:44 smithi033 ceph-mon[26353]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:44.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:44 smithi033 ceph-mon[26353]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:44.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:44 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2054733494' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:37:45.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:44 smithi138 ceph-mon[32977]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-25T01:37:45.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:44 smithi138 ceph-mon[32977]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:45.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:44 smithi138 ceph-mon[32977]: from='client.14716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:45.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:44 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2054733494' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:37:46.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:45 smithi033 ceph-mon[26353]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:46.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:45 smithi033 ceph-mon[26353]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:46.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:45 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:46.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:45 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:46.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:45 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:46.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:45 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:46.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:45 smithi138 ceph-mon[32977]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:46.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:45 smithi138 ceph-mon[32977]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:37:46.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:45 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:46.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:45 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:46.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:45 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:46.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:45 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:37:46.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-04-25T01:37:46.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-25T01:37:46.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T01:37:46.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:46.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:46.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:37:46.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:37:46.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:46.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:46.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:46.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:46.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:46 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:47.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-04-25T01:37:47.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-25T01:37:47.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T01:37:47.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:47.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:37:47.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:37:47.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:37:47.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:47.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:47.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:47.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:47.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:46 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:47.863 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:47 smithi033 ceph-mon[26353]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:37:47.863 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:47 smithi033 ceph-mon[26353]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:37:47.863 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:47 smithi033 ceph-mon[26353]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 0 B/s wr, 15 op/s 2024-04-25T01:37:47.863 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:47 smithi033 ceph-mon[26353]: Reconfiguring prometheus.smithi033 (dependencies changed)... 2024-04-25T01:37:47.863 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:47 smithi033 ceph-mon[26353]: Reconfiguring daemon prometheus.smithi033 on smithi033 2024-04-25T01:37:47.863 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:47 smithi033 ceph-mon[26353]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-25T01:37:48.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:47 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:37:48.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:47 smithi138 ceph-mon[32977]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:37:48.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:47 smithi138 ceph-mon[32977]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 22 KiB/s rd, 0 B/s wr, 15 op/s 2024-04-25T01:37:48.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:47 smithi138 ceph-mon[32977]: Reconfiguring prometheus.smithi033 (dependencies changed)... 2024-04-25T01:37:48.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:47 smithi138 ceph-mon[32977]: Reconfiguring daemon prometheus.smithi033 on smithi033 2024-04-25T01:37:48.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:47 smithi138 ceph-mon[32977]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-25T01:37:50.019 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:49 smithi138 ceph-mon[32977]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-25T01:37:50.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:49 smithi033 ceph-mon[26353]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-25T01:37:51.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:51 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:51.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:51 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:51.244 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:51 smithi033 ceph-mon[26353]: pgmap v11: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-25T01:37:51.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:51 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:51.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:51 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:37:51.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:51 smithi138 ceph-mon[32977]: pgmap v11: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 13 op/s 2024-04-25T01:37:53.372 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:53 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:53.372 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:53 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:53.372 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:53 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:37:53.372 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:53 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:37:53.373 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:53 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033:9095"}]: dispatch 2024-04-25T01:37:53.373 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:53 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033:9095"}]: dispatch 2024-04-25T01:37:53.373 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:53 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:53.373 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:53 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:53.373 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:53 smithi138 ceph-mon[32977]: pgmap v12: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 272 B/s wr, 16 op/s 2024-04-25T01:37:53.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:53 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:53.494 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:53 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:53 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:37:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:53 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:37:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:53 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033:9095"}]: dispatch 2024-04-25T01:37:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:53 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi033:9095"}]: dispatch 2024-04-25T01:37:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:53 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:53 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:53.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:53 smithi033 ceph-mon[26353]: pgmap v12: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 272 B/s wr, 16 op/s 2024-04-25T01:37:54.519 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:54 smithi138 ceph-mon[32977]: Upgrade: Updating mgr.smithi138.mmfhyi 2024-04-25T01:37:54.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:54 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:54.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:54 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:54.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:54 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:54.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:54 smithi138 ceph-mon[32977]: Deploying daemon mgr.smithi138.mmfhyi on smithi138 2024-04-25T01:37:54.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:54 smithi033 ceph-mon[26353]: Upgrade: Updating mgr.smithi138.mmfhyi 2024-04-25T01:37:54.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:54 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:37:54.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:54 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:37:54.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:54 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:37:54.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:54 smithi033 ceph-mon[26353]: Deploying daemon mgr.smithi138.mmfhyi on smithi138 2024-04-25T01:37:55.520 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:55 smithi138 ceph-mon[32977]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 272 B/s wr, 3 op/s 2024-04-25T01:37:55.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:55 smithi033 ceph-mon[26353]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 272 B/s wr, 3 op/s 2024-04-25T01:37:57.416 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:57 smithi138 ceph-mon[32977]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 272 B/s wr, 5 op/s 2024-04-25T01:37:57.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:57 smithi033 ceph-mon[26353]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 272 B/s wr, 5 op/s 2024-04-25T01:37:58.907 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:58 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:58.908 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:58 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:58.908 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:58 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:59.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:58 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:59.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:58 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:37:59.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:58 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:37:59.994 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:37:59 smithi033 ceph-mon[26353]: pgmap v15: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T01:38:00.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:37:59 smithi138 ceph-mon[32977]: pgmap v15: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T01:38:01.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:01 smithi033 ceph-mon[26353]: pgmap v16: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T01:38:01.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:01 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:01.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:01 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:02.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:01 smithi138 ceph-mon[32977]: pgmap v16: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 255 B/s wr, 3 op/s 2024-04-25T01:38:02.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:01 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:02.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:01 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:03.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:03.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:03.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:03 smithi033 ceph-mon[26353]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:38:03.340 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:03.340 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:03.340 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:03 smithi138 ceph-mon[32977]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T01:38:04.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:04 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:04.986 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:04 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:04.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:04 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:04.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:04 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:05.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:05 smithi033 ceph-mon[26353]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:05.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:05 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/3501444848' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/crt"}]: dispatch 2024-04-25T01:38:05.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:05 smithi033 ceph-mon[26353]: Standby manager daemon smithi138.mmfhyi restarted 2024-04-25T01:38:05.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:05 smithi033 ceph-mon[26353]: Standby manager daemon smithi138.mmfhyi started 2024-04-25T01:38:05.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:05 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/3501444848' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:38:05.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:05 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/3501444848' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/key"}]: dispatch 2024-04-25T01:38:05.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:05 smithi033 ceph-mon[26353]: from='mgr.? 172.21.15.138:0/3501444848' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:38:05.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:05 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:38:06.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:05 smithi138 ceph-mon[32977]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:06.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:05 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3501444848' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/crt"}]: dispatch 2024-04-25T01:38:06.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:05 smithi138 ceph-mon[32977]: Standby manager daemon smithi138.mmfhyi restarted 2024-04-25T01:38:06.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:05 smithi138 ceph-mon[32977]: Standby manager daemon smithi138.mmfhyi started 2024-04-25T01:38:06.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:05 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3501444848' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:38:06.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:05 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3501444848' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/key"}]: dispatch 2024-04-25T01:38:06.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:05 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/3501444848' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:38:06.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:05 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:38:06.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: mgrmap e33: smithi033.asemba(active, since 30s), standbys: smithi138.mmfhyi 2024-04-25T01:38:06.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: Upgrade: Setting container_image for all mgr 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi033.asemba"}]: dispatch 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi033.asemba"}]': finished 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.mmfhyi"}]: dispatch 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.mmfhyi"}]': finished 2024-04-25T01:38:06.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:06.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:06.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:06.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:06.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:06.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:06.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:06.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:06.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:06 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: mgrmap e33: smithi033.asemba(active, since 30s), standbys: smithi138.mmfhyi 2024-04-25T01:38:07.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:07.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:07.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:38:07.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T01:38:07.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: Upgrade: Setting container_image for all mgr 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi033.asemba"}]: dispatch 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi033.asemba"}]': finished 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.mmfhyi"}]: dispatch 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi138.mmfhyi"}]': finished 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:07.022 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:06 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:07.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:07 smithi033 ceph-mon[26353]: Upgrade: Setting container_image for all rgw 2024-04-25T01:38:07.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:07 smithi033 ceph-mon[26353]: Upgrade: Setting container_image for all rbd-mirror 2024-04-25T01:38:07.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:07 smithi033 ceph-mon[26353]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-25T01:38:07.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:07 smithi033 ceph-mon[26353]: Upgrade: Setting container_image for all ceph-exporter 2024-04-25T01:38:07.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:07 smithi033 ceph-mon[26353]: Upgrade: Setting container_image for all iscsi 2024-04-25T01:38:07.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:07 smithi033 ceph-mon[26353]: Upgrade: Setting container_image for all nfs 2024-04-25T01:38:07.995 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:07 smithi033 ceph-mon[26353]: Upgrade: Setting container_image for all nvmeof 2024-04-25T01:38:08.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:07 smithi138 ceph-mon[32977]: Upgrade: Setting container_image for all rgw 2024-04-25T01:38:08.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:07 smithi138 ceph-mon[32977]: Upgrade: Setting container_image for all rbd-mirror 2024-04-25T01:38:08.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:07 smithi138 ceph-mon[32977]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-25T01:38:08.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:07 smithi138 ceph-mon[32977]: Upgrade: Setting container_image for all ceph-exporter 2024-04-25T01:38:08.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:07 smithi138 ceph-mon[32977]: Upgrade: Setting container_image for all iscsi 2024-04-25T01:38:08.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:07 smithi138 ceph-mon[32977]: Upgrade: Setting container_image for all nfs 2024-04-25T01:38:08.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:07 smithi138 ceph-mon[32977]: Upgrade: Setting container_image for all nvmeof 2024-04-25T01:38:08.970 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:08 smithi033 ceph-mon[26353]: Upgrade: Updating node-exporter.smithi033 (1/2) 2024-04-25T01:38:08.970 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:08 smithi033 ceph-mon[26353]: Deploying daemon node-exporter.smithi033 on smithi033 2024-04-25T01:38:08.970 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:08 smithi033 ceph-mon[26353]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:09.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:08 smithi138 ceph-mon[32977]: Upgrade: Updating node-exporter.smithi033 (1/2) 2024-04-25T01:38:09.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:08 smithi138 ceph-mon[32977]: Deploying daemon node-exporter.smithi033 on smithi033 2024-04-25T01:38:09.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:08 smithi138 ceph-mon[32977]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:10.745 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:10 smithi033 ceph-mon[26353]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:11.020 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:10 smithi138 ceph-mon[32977]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:12.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:12 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:12.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:12 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:12.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:12 smithi033 ceph-mon[26353]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:12.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:12 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:12.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:12 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:12.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:12 smithi138 ceph-mon[32977]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:13.270 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:13 smithi138 ceph-mon[32977]: Upgrade: Updating node-exporter.smithi138 (2/2) 2024-04-25T01:38:13.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:13 smithi138 ceph-mon[32977]: Deploying daemon node-exporter.smithi138 on smithi138 2024-04-25T01:38:13.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:13 smithi033 ceph-mon[26353]: Upgrade: Updating node-exporter.smithi138 (2/2) 2024-04-25T01:38:13.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:13 smithi033 ceph-mon[26353]: Deploying daemon node-exporter.smithi138 on smithi138 2024-04-25T01:38:14.210 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:14 smithi138 ceph-mon[32977]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:14.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:14 smithi033 ceph-mon[26353]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.881+0000 7f0074fa9700 1 -- 172.21.15.33:0/2096221811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070070810 msgr2=0x7f0070070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.881+0000 7f0074fa9700 1 --2- 172.21.15.33:0/2096221811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070070810 0x7f0070070c80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0064004940 tx=0x7f0064009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.881+0000 7f0074fa9700 1 -- 172.21.15.33:0/2096221811 shutdown_connections 2024-04-25T01:38:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.881+0000 7f0074fa9700 1 --2- 172.21.15.33:0/2096221811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070070810 0x7f0070070c80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.881+0000 7f0074fa9700 1 --2- 172.21.15.33:0/2096221811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0070071dc0 0x7f00700702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.881+0000 7f0074fa9700 1 -- 172.21.15.33:0/2096221811 >> 172.21.15.33:0/2096221811 conn(0x7f00700f8540 msgr2=0x7f00700fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:14.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.881+0000 7f0074fa9700 1 -- 172.21.15.33:0/2096221811 shutdown_connections 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.881+0000 7f0074fa9700 1 -- 172.21.15.33:0/2096221811 wait complete. 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f0074fa9700 1 Processor -- start 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f0074fa9700 1 -- start start 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f0074fa9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0070070810 0x7f007010fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f0074fa9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070071dc0 0x7f0070110060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f0074fa9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0070110660 con 0x7f0070071dc0 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f0074fa9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00701107d0 con 0x7f0070070810 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f006dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070071dc0 0x7f0070110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f006e59c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0070070810 0x7f007010fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f006dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070071dc0 0x7f0070110060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37050/0 (socket says 172.21.15.33:37050) 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f006dd9b700 1 -- 172.21.15.33:0/1398661707 learned_addr learned my addr 172.21.15.33:0/1398661707 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f006dd9b700 1 -- 172.21.15.33:0/1398661707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0070070810 msgr2=0x7f007010fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f006dd9b700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0070070810 0x7f007010fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.882+0000 7f006dd9b700 1 -- 172.21.15.33:0/1398661707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064005040 con 0x7f0070071dc0 2024-04-25T01:38:14.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.883+0000 7f006dd9b700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070071dc0 0x7f0070110060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f006400b3b0 tx=0x7f0064009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.883+0000 7f005effd700 1 -- 172.21.15.33:0/1398661707 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006400e040 con 0x7f0070071dc0 2024-04-25T01:38:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.883+0000 7f005effd700 1 -- 172.21.15.33:0/1398661707 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f006400bbf0 con 0x7f0070071dc0 2024-04-25T01:38:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.883+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00701151d0 con 0x7f0070071dc0 2024-04-25T01:38:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.883+0000 7f005effd700 1 -- 172.21.15.33:0/1398661707 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006401c630 con 0x7f0070071dc0 2024-04-25T01:38:14.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.883+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0070115790 con 0x7f0070071dc0 2024-04-25T01:38:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.884+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007004c710 con 0x7f0070071dc0 2024-04-25T01:38:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.884+0000 7f005effd700 1 -- 172.21.15.33:0/1398661707 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f006400bd60 con 0x7f0070071dc0 2024-04-25T01:38:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.885+0000 7f005effd700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f005806db70 0x7f0058070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.885+0000 7f005effd700 1 -- 172.21.15.33:0/1398661707 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f006408dfa0 con 0x7f0070071dc0 2024-04-25T01:38:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.885+0000 7f006e59c700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f005806db70 0x7f0058070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:14.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.885+0000 7f006e59c700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f005806db70 0x7f0058070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0070071b00 tx=0x7f006000b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:14.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:14.890+0000 7f005effd700 1 -- 172.21.15.33:0/1398661707 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f006405b560 con 0x7f0070071dc0 2024-04-25T01:38:15.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.113+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0070103e60 con 0x7f005806db70 2024-04-25T01:38:15.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.114+0000 7f005effd700 1 -- 172.21.15.33:0/1398661707 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f0070103e60 con 0x7f005806db70 2024-04-25T01:38:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.116+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f005806db70 msgr2=0x7f0058070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.116+0000 7f0074fa9700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f005806db70 0x7f0058070020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0070071b00 tx=0x7f006000b040 comp rx=0 tx=0).stop 2024-04-25T01:38:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.116+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070071dc0 msgr2=0x7f0070110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.116+0000 7f0074fa9700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070071dc0 0x7f0070110060 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f006400b3b0 tx=0x7f0064009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.116+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 shutdown_connections 2024-04-25T01:38:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.116+0000 7f0074fa9700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f005806db70 0x7f0058070020 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.116+0000 7f0074fa9700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0070070810 0x7f007010fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.116+0000 7f0074fa9700 1 --2- 172.21.15.33:0/1398661707 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0070071dc0 0x7f0070110060 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.116+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 >> 172.21.15.33:0/1398661707 conn(0x7f00700f8540 msgr2=0x7f00700f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.117+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 shutdown_connections 2024-04-25T01:38:15.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.117+0000 7f0074fa9700 1 -- 172.21.15.33:0/1398661707 wait complete. 2024-04-25T01:38:15.107 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.227+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/2242429793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fbcb0 msgr2=0x7fbe080fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.227+0000 7fbe0eb7c700 1 --2- 172.21.15.33:0/2242429793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fbcb0 0x7fbe080fc0c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc004940 tx=0x7fbdfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.227+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/2242429793 shutdown_connections 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.227+0000 7fbe0eb7c700 1 --2- 172.21.15.33:0/2242429793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe080fcf60 0x7fbe080fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.227+0000 7fbe0eb7c700 1 --2- 172.21.15.33:0/2242429793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fbcb0 0x7fbe080fc0c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.227+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/2242429793 >> 172.21.15.33:0/2242429793 conn(0x7fbe080f7430 msgr2=0x7fbe080f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.227+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/2242429793 shutdown_connections 2024-04-25T01:38:15.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.227+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/2242429793 wait complete. 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe0eb7c700 1 Processor -- start 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe0eb7c700 1 -- start start 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe0eb7c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe080fbcb0 0x7fbe0810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe0eb7c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fcf60 0x7fbe0810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe0eb7c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0810e4b0 con 0x7fbe080fcf60 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe0eb7c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0810ed00 con 0x7fbe080fbcb0 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fcf60 0x7fbe0810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fcf60 0x7fbe0810deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37066/0 (socket says 172.21.15.33:37066) 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe07fff700 1 -- 172.21.15.33:0/3615188754 learned_addr learned my addr 172.21.15.33:0/3615188754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:15.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.228+0000 7fbe0c918700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe080fbcb0 0x7fbe0810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe0c918700 1 -- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fcf60 msgr2=0x7fbe0810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe0c918700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fcf60 0x7fbe0810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe0c918700 1 -- 172.21.15.33:0/3615188754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc005040 con 0x7fbe080fbcb0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe07fff700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fcf60 0x7fbe0810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe0c918700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe080fbcb0 0x7fbe0810d970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc000c00 tx=0x7fbdfc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe057fa700 1 -- 172.21.15.33:0/3615188754 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdfc00e030 con 0x7fbe080fbcb0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe057fa700 1 -- 172.21.15.33:0/3615188754 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbdfc013de0 con 0x7fbe080fbcb0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0810ef80 con 0x7fbe080fbcb0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe057fa700 1 -- 172.21.15.33:0/3615188754 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbdfc01ce50 con 0x7fbe080fbcb0 2024-04-25T01:38:15.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.229+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe0810f4c0 con 0x7fbe080fbcb0 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.231+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe08101af0 con 0x7fbe080fbcb0 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.231+0000 7fbe057fa700 1 -- 172.21.15.33:0/3615188754 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fbdfc01c400 con 0x7fbe080fbcb0 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.232+0000 7fbe057fa700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbdf806db00 0x7fbdf806ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.232+0000 7fbe057fa700 1 -- 172.21.15.33:0/3615188754 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fbdfc08ed20 con 0x7fbe080fbcb0 2024-04-25T01:38:15.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.232+0000 7fbe07fff700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbdf806db00 0x7fbdf806ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.232+0000 7fbe07fff700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbdf806db00 0x7fbdf806ffb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40041d0 tx=0x7fbdf400b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.236+0000 7fbe057fa700 1 -- 172.21.15.33:0/3615188754 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fbdfc05c1c0 con 0x7fbe080fbcb0 2024-04-25T01:38:15.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.462+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe081048b0 con 0x7fbdf806db00 2024-04-25T01:38:15.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.463+0000 7fbe057fa700 1 -- 172.21.15.33:0/3615188754 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fbe081048b0 con 0x7fbdf806db00 2024-04-25T01:38:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.465+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbdf806db00 msgr2=0x7fbdf806ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.465+0000 7fbe0eb7c700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbdf806db00 0x7fbdf806ffb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40041d0 tx=0x7fbdf400b040 comp rx=0 tx=0).stop 2024-04-25T01:38:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.465+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe080fbcb0 msgr2=0x7fbe0810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.465+0000 7fbe0eb7c700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe080fbcb0 0x7fbe0810d970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc000c00 tx=0x7fbdfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.466+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 shutdown_connections 2024-04-25T01:38:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.466+0000 7fbe0eb7c700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbdf806db00 0x7fbdf806ffb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.466+0000 7fbe0eb7c700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe080fbcb0 0x7fbe0810d970 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.466+0000 7fbe0eb7c700 1 --2- 172.21.15.33:0/3615188754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe080fcf60 0x7fbe0810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.466+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 >> 172.21.15.33:0/3615188754 conn(0x7fbe080f7430 msgr2=0x7fbe080f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.466+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 shutdown_connections 2024-04-25T01:38:15.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.466+0000 7fbe0eb7c700 1 -- 172.21.15.33:0/3615188754 wait complete. 2024-04-25T01:38:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.573+0000 7efe2d5a8700 1 -- 172.21.15.33:0/1726203086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fed30 msgr2=0x7efe280fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.573+0000 7efe2d5a8700 1 --2- 172.21.15.33:0/1726203086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fed30 0x7efe280fae70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7efe10004940 tx=0x7efe10009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.573+0000 7efe2d5a8700 1 -- 172.21.15.33:0/1726203086 shutdown_connections 2024-04-25T01:38:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.573+0000 7efe2d5a8700 1 --2- 172.21.15.33:0/1726203086 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fed30 0x7efe280fae70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.573+0000 7efe2d5a8700 1 --2- 172.21.15.33:0/1726203086 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe280fe3e0 0x7efe280fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.573+0000 7efe2d5a8700 1 -- 172.21.15.33:0/1726203086 >> 172.21.15.33:0/1726203086 conn(0x7efe280f6b80 msgr2=0x7efe280f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.573+0000 7efe2d5a8700 1 -- 172.21.15.33:0/1726203086 shutdown_connections 2024-04-25T01:38:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.573+0000 7efe2d5a8700 1 -- 172.21.15.33:0/1726203086 wait complete. 2024-04-25T01:38:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe2d5a8700 1 Processor -- start 2024-04-25T01:38:15.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe2d5a8700 1 -- start start 2024-04-25T01:38:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe2d5a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fe3e0 0x7efe280fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe2d5a8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe280fed30 0x7efe280fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe2d5a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe280fc650 con 0x7efe280fe3e0 2024-04-25T01:38:15.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe2d5a8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe280fc7c0 con 0x7efe280fed30 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe26ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fe3e0 0x7efe280fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe26ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fe3e0 0x7efe280fdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37086/0 (socket says 172.21.15.33:37086) 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe26ffd700 1 -- 172.21.15.33:0/3563603318 learned_addr learned my addr 172.21.15.33:0/3563603318 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe267fc700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe280fed30 0x7efe280fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe267fc700 1 -- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fe3e0 msgr2=0x7efe280fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe267fc700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fe3e0 0x7efe280fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe267fc700 1 -- 172.21.15.33:0/3563603318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10005040 con 0x7efe280fed30 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.574+0000 7efe26ffd700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fe3e0 0x7efe280fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.575+0000 7efe267fc700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe280fed30 0x7efe280fc0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe10000c00 tx=0x7efe10009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.575+0000 7efe1f7fe700 1 -- 172.21.15.33:0/3563603318 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1000e030 con 0x7efe280fed30 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.575+0000 7efe1f7fe700 1 -- 172.21.15.33:0/3563603318 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe10013de0 con 0x7efe280fed30 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.575+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe280fca40 con 0x7efe280fed30 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.575+0000 7efe1f7fe700 1 -- 172.21.15.33:0/3563603318 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1001ce50 con 0x7efe280fed30 2024-04-25T01:38:15.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.575+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe28115990 con 0x7efe280fed30 2024-04-25T01:38:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.576+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2805f5c0 con 0x7efe280fed30 2024-04-25T01:38:15.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.576+0000 7efe1f7fe700 1 -- 172.21.15.33:0/3563603318 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7efe10013910 con 0x7efe280fed30 2024-04-25T01:38:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.577+0000 7efe1f7fe700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7efe1406db00 0x7efe1406ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.577+0000 7efe1f7fe700 1 -- 172.21.15.33:0/3563603318 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7efe1008ed10 con 0x7efe280fed30 2024-04-25T01:38:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.577+0000 7efe26ffd700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7efe1406db00 0x7efe1406ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.577+0000 7efe26ffd700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7efe1406db00 0x7efe1406ffb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efe18004fd0 tx=0x7efe1800b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:15.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.581+0000 7efe1f7fe700 1 -- 172.21.15.33:0/3563603318 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7efe100586c0 con 0x7efe280fed30 2024-04-25T01:38:15.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.811+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efe2805f5c0 con 0x7efe1406db00 2024-04-25T01:38:15.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.821+0000 7efe1f7fe700 1 -- 172.21.15.33:0/3563603318 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7efe2805f5c0 con 0x7efe1406db00 2024-04-25T01:38:15.801 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:38:15.801 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (115s) 12s ago 10m 18.4M - 0.25.0 c8568f914cd2 c74b90b8b6ca 2024-04-25T01:38:15.801 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (10m) 12s ago 10m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:38:15.801 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (9m) 14s ago 9m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 unknown 12s ago 10m - - 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (5m) 12s ago 5m 61.8M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (5m) 12s ago 5m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (5m) 14s ago 5m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (5m) 14s ago 5m 60.8M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (53s) 12s ago 11m 492M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (18s) 14s ago 9m 76.6M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (11m) 12s ago 12m 55.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (9m) 14s ago 9m 44.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 starting - - - - 2024-04-25T01:38:15.802 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 starting - - - - 2024-04-25T01:38:15.803 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (8m) 12s ago 8m 222M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:38:15.803 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (8m) 12s ago 8m 235M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:38:15.803 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (7m) 12s ago 7m 188M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:38:15.803 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (7m) 14s ago 7m 262M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:38:15.803 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (7m) 14s ago 7m 229M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:38:15.803 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (6m) 14s ago 6m 217M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:38:15.803 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (24s) 12s ago 10m 40.7M - 2.43.0 a07b618ecd1d 2e4c853b61b5 2024-04-25T01:38:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.823+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7efe1406db00 msgr2=0x7efe1406ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.823+0000 7efe2d5a8700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7efe1406db00 0x7efe1406ffb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efe18004fd0 tx=0x7efe1800b040 comp rx=0 tx=0).stop 2024-04-25T01:38:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.823+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe280fed30 msgr2=0x7efe280fc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.823+0000 7efe2d5a8700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe280fed30 0x7efe280fc0a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efe10000c00 tx=0x7efe10009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:15.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.823+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 shutdown_connections 2024-04-25T01:38:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.823+0000 7efe2d5a8700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7efe1406db00 0x7efe1406ffb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.824+0000 7efe2d5a8700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe280fe3e0 0x7efe280fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.824+0000 7efe2d5a8700 1 --2- 172.21.15.33:0/3563603318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe280fed30 0x7efe280fc0a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.824+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 >> 172.21.15.33:0/3563603318 conn(0x7efe280f6b80 msgr2=0x7efe280f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.824+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 shutdown_connections 2024-04-25T01:38:15.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.824+0000 7efe2d5a8700 1 -- 172.21.15.33:0/3563603318 wait complete. 2024-04-25T01:38:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.927+0000 7f771eff6700 1 -- 172.21.15.33:0/1893271372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fbd40 msgr2=0x7f77180fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.927+0000 7f771eff6700 1 --2- 172.21.15.33:0/1893271372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fbd40 0x7f77180fc150 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f770c004940 tx=0x7f770c009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.927+0000 7f771eff6700 1 -- 172.21.15.33:0/1893271372 shutdown_connections 2024-04-25T01:38:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.927+0000 7f771eff6700 1 --2- 172.21.15.33:0/1893271372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77180fcff0 0x7f77180fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.927+0000 7f771eff6700 1 --2- 172.21.15.33:0/1893271372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fbd40 0x7f77180fc150 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.927+0000 7f771eff6700 1 -- 172.21.15.33:0/1893271372 >> 172.21.15.33:0/1893271372 conn(0x7f77180f74c0 msgr2=0x7f77180f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.927+0000 7f771eff6700 1 -- 172.21.15.33:0/1893271372 shutdown_connections 2024-04-25T01:38:15.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f771eff6700 1 -- 172.21.15.33:0/1893271372 wait complete. 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f771eff6700 1 Processor -- start 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f771eff6700 1 -- start start 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f771eff6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77180fbd40 0x7f771810fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f771eff6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fcff0 0x7f7718110190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f771eff6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718110790 con 0x7f77180fcff0 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f771eff6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7718110f90 con 0x7f77180fbd40 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f7717fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fcff0 0x7f7718110190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f7717fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fcff0 0x7f7718110190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46750/0 (socket says 172.21.15.33:46750) 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.928+0000 7f7717fff700 1 -- 172.21.15.33:0/3813666053 learned_addr learned my addr 172.21.15.33:0/3813666053 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f771cd92700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77180fbd40 0x7f771810fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f7717fff700 1 -- 172.21.15.33:0/3813666053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77180fbd40 msgr2=0x7f771810fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f7717fff700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77180fbd40 0x7f771810fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f7717fff700 1 -- 172.21.15.33:0/3813666053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f770c005040 con 0x7f77180fcff0 2024-04-25T01:38:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f771cd92700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77180fbd40 0x7f771810fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:38:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f7717fff700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fcff0 0x7f7718110190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7704002a50 tx=0x7f77040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f77157fa700 1 -- 172.21.15.33:0/3813666053 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7704007cc0 con 0x7f77180fcff0 2024-04-25T01:38:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f77157fa700 1 -- 172.21.15.33:0/3813666053 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f770400f040 con 0x7f77180fcff0 2024-04-25T01:38:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7718111270 con 0x7f77180fcff0 2024-04-25T01:38:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f77157fa700 1 -- 172.21.15.33:0/3813666053 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7704009640 con 0x7f77180fcff0 2024-04-25T01:38:15.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.929+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77181117b0 con 0x7f77180fcff0 2024-04-25T01:38:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.930+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7718101980 con 0x7f77180fcff0 2024-04-25T01:38:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.931+0000 7f77157fa700 1 -- 172.21.15.33:0/3813666053 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f77040097a0 con 0x7f77180fcff0 2024-04-25T01:38:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.931+0000 7f77157fa700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f770806db60 0x7f7708070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.931+0000 7f77157fa700 1 -- 172.21.15.33:0/3813666053 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f770408d200 con 0x7f77180fcff0 2024-04-25T01:38:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.931+0000 7f771cd92700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f770806db60 0x7f7708070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.932+0000 7f771cd92700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f770806db60 0x7f7708070010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f770c00b970 tx=0x7f770c009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:15.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:15.936+0000 7f77157fa700 1 -- 172.21.15.33:0/3813666053 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f770405a740 con 0x7f77180fcff0 2024-04-25T01:38:16.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.237+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f771804bed0 con 0x7f77180fcff0 2024-04-25T01:38:16.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.238+0000 7f77157fa700 1 -- 172.21.15.33:0/3813666053 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f770405a2d0 con 0x7f77180fcff0 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:38:16.218 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:38:16.219 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:38:16.219 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:38:16.219 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:38:16.219 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-25T01:38:16.219 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:38:16.219 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:38:16.219 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f770806db60 msgr2=0x7f7708070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f770806db60 0x7f7708070010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f770c00b970 tx=0x7f770c009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fcff0 msgr2=0x7f7718110190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fcff0 0x7f7718110190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7704002a50 tx=0x7f77040046b0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 shutdown_connections 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f770806db60 0x7f7708070010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77180fbd40 0x7f771810fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 --2- 172.21.15.33:0/3813666053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77180fcff0 0x7f7718110190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 >> 172.21.15.33:0/3813666053 conn(0x7f77180f74c0 msgr2=0x7f7718066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:16.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 shutdown_connections 2024-04-25T01:38:16.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.240+0000 7f771eff6700 1 -- 172.21.15.33:0/3813666053 wait complete. 2024-04-25T01:38:16.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:16 smithi138 ceph-mon[32977]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:16.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:16 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:16.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:16 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:16.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:16 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:16.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:16 smithi138 ceph-mon[32977]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:16.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:16 smithi138 ceph-mon[32977]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T01:38:16.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:16 smithi138 ceph-mon[32977]: from='client.24545 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.347+0000 7f7e009d8700 1 -- 172.21.15.33:0/3962351717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 msgr2=0x7f7dfc070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.347+0000 7f7e009d8700 1 --2- 172.21.15.33:0/3962351717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 0x7f7dfc070ca0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7df0004940 tx=0x7f7df0009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 -- 172.21.15.33:0/3962351717 shutdown_connections 2024-04-25T01:38:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 --2- 172.21.15.33:0/3962351717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 0x7f7dfc070ca0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 --2- 172.21.15.33:0/3962351717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dfc071de0 0x7f7dfc0702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 -- 172.21.15.33:0/3962351717 >> 172.21.15.33:0/3962351717 conn(0x7f7dfc0f8540 msgr2=0x7f7dfc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 -- 172.21.15.33:0/3962351717 shutdown_connections 2024-04-25T01:38:16.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 -- 172.21.15.33:0/3962351717 wait complete. 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 Processor -- start 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 -- start start 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 0x7f7dfc10fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dfc071de0 0x7f7dfc110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc110650 con 0x7f7dfc070830 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7e009d8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dfc1107c0 con 0x7f7dfc071de0 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7dfa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 0x7f7dfc10fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7dfa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 0x7f7dfc10fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46770/0 (socket says 172.21.15.33:46770) 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.348+0000 7f7dfa59c700 1 -- 172.21.15.33:0/2567002333 learned_addr learned my addr 172.21.15.33:0/2567002333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7df9d9b700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dfc071de0 0x7f7dfc110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7dfa59c700 1 -- 172.21.15.33:0/2567002333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dfc071de0 msgr2=0x7f7dfc110050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7dfa59c700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dfc071de0 0x7f7dfc110050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7dfa59c700 1 -- 172.21.15.33:0/2567002333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df0005040 con 0x7f7dfc070830 2024-04-25T01:38:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7dfa59c700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 0x7f7dfc10fb10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7dec002a80 tx=0x7f7dec004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7deaffd700 1 -- 172.21.15.33:0/2567002333 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dec007cc0 con 0x7f7dfc070830 2024-04-25T01:38:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dfc115220 con 0x7f7dfc070830 2024-04-25T01:38:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7deaffd700 1 -- 172.21.15.33:0/2567002333 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dec00f040 con 0x7f7dfc070830 2024-04-25T01:38:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7deaffd700 1 -- 172.21.15.33:0/2567002333 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dec00da30 con 0x7f7dfc070830 2024-04-25T01:38:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.349+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dfc1157e0 con 0x7f7dfc070830 2024-04-25T01:38:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.350+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dfc04c710 con 0x7f7dfc070830 2024-04-25T01:38:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.351+0000 7f7deaffd700 1 -- 172.21.15.33:0/2567002333 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f7dec00db90 con 0x7f7dfc070830 2024-04-25T01:38:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.351+0000 7f7deaffd700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f7de406db70 0x7f7de4070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.351+0000 7f7deaffd700 1 -- 172.21.15.33:0/2567002333 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f7dec08d180 con 0x7f7dfc070830 2024-04-25T01:38:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.351+0000 7f7df9d9b700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f7de406db70 0x7f7de4070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.352+0000 7f7df9d9b700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f7de406db70 0x7f7de4070020 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7df000b970 tx=0x7f7df0009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:16.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.356+0000 7f7deaffd700 1 -- 172.21.15.33:0/2567002333 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f7dec05a6c0 con 0x7f7dfc070830 2024-04-25T01:38:16.495 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:16 smithi033 ceph-mon[26353]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:16 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:16 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:16 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:16 smithi033 ceph-mon[26353]: from='client.24541 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:16 smithi033 ceph-mon[26353]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T01:38:16.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:16 smithi033 ceph-mon[26353]: from='client.24545 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:16.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.593+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dfc103eb0 con 0x7f7de406db70 2024-04-25T01:38:16.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.594+0000 7f7deaffd700 1 -- 172.21.15.33:0/2567002333 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f7dfc103eb0 con 0x7f7de406db70 2024-04-25T01:38:16.574 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:38:16.574 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:38:16.574 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:38:16.574 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-25T01:38:16.574 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:38:16.574 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr" 2024-04-25T01:38:16.574 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:38:16.574 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "2/2 daemons upgraded", 2024-04-25T01:38:16.575 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-25T01:38:16.575 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:38:16.575 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f7de406db70 msgr2=0x7f7de4070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f7de406db70 0x7f7de4070020 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7df000b970 tx=0x7f7df0009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 msgr2=0x7f7dfc10fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 0x7f7dfc10fb10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7dec002a80 tx=0x7f7dec004600 comp rx=0 tx=0).stop 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 shutdown_connections 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f7de406db70 0x7f7de4070020 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dfc070830 0x7f7dfc10fb10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 --2- 172.21.15.33:0/2567002333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dfc071de0 0x7f7dfc110050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 >> 172.21.15.33:0/2567002333 conn(0x7f7dfc0f8540 msgr2=0x7f7dfc0f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:16.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 shutdown_connections 2024-04-25T01:38:16.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:16.597+0000 7f7e009d8700 1 -- 172.21.15.33:0/2567002333 wait complete. 2024-04-25T01:38:17.245 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:17 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/3813666053' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:17.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:17 smithi033 ceph-mon[26353]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:17.416 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:17 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/3813666053' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:17.416 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:17 smithi138 ceph-mon[32977]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:18.409 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:18 smithi033 ceph-mon[26353]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:18.409 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:18 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:18.409 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:18 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:18.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:18 smithi138 ceph-mon[32977]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:18.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:18 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:18.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:18 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:20.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:20.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:20.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:20 smithi033 ceph-mon[26353]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:20.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:20.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:20.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:38:20.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:20.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:20.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:20 smithi138 ceph-mon[32977]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:20.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:20.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:20.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:38:22.219 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:21 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:22.219 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:21 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:22.219 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:21 smithi033 ceph-mon[26353]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:22.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:21 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:22.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:21 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:22.239 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:21 smithi138 ceph-mon[32977]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:23.817 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:23 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:23.818 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:23 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:24.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:23 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:24.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:23 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:24.996 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:24 smithi033 ceph-mon[26353]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:25.021 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:24 smithi138 ceph-mon[32977]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:26.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:26.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:26.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:26.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:26.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:26.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:26.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:26.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:28.674 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:28 smithi033 ceph-mon[26353]: Upgrade: Updating prometheus.smithi033 2024-04-25T01:38:28.674 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:28 smithi033 ceph-mon[26353]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:28.674 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:28 smithi033 ceph-mon[26353]: Deploying daemon prometheus.smithi033 on smithi033 2024-04-25T01:38:28.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:28 smithi138 ceph-mon[32977]: Upgrade: Updating prometheus.smithi033 2024-04-25T01:38:28.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:28 smithi138 ceph-mon[32977]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:28.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:28 smithi138 ceph-mon[32977]: Deploying daemon prometheus.smithi033 on smithi033 2024-04-25T01:38:30.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:30 smithi033 ceph-mon[26353]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:30.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:30 smithi138 ceph-mon[32977]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:32.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:32 smithi033 ceph-mon[26353]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:32.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:32 smithi138 ceph-mon[32977]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:34.246 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:34 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:34.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:34 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:34.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:34 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:34.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:34 smithi033 ceph-mon[26353]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:34.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:34 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:34.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:34 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:34.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:34 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:34.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:34 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:34.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:34 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:34.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:34 smithi138 ceph-mon[32977]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:38:34.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:34 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:34.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:34 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:35.433 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:35 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:38:35.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:35 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:38:36.388 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:36 smithi033 ceph-mon[26353]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T01:38:36.521 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:36 smithi138 ceph-mon[32977]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T01:38:38.568 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:38 smithi033 ceph-mon[26353]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:38.771 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:38 smithi138 ceph-mon[32977]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:40.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:40 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:40.496 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:40 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:40.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:40 smithi033 ceph-mon[26353]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:40.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:40 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:40.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:40 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:40.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:40 smithi138 ceph-mon[32977]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:41.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:41 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:41.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:41 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:41.774 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:41 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:41.774 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:41 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:42.681 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:42 smithi033 ceph-mon[26353]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:42.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:42 smithi138 ceph-mon[32977]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:44.725 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:44.725 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:44.725 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:44.725 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:44.725 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:44.725 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:44.725 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:38:44.725 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.726 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:44 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:44.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:44.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:44.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:44.773 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:44 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:45.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:45 smithi033 ceph-mon[26353]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T01:38:45.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:45 smithi033 ceph-mon[26353]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T01:38:45.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:45 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:38:45.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:45 smithi138 ceph-mon[32977]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T01:38:45.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:45 smithi138 ceph-mon[32977]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T01:38:45.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:45 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T01:38:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.712+0000 7f373068a700 1 -- 172.21.15.33:0/3507527164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fcff0 msgr2=0x7f37280fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.712+0000 7f373068a700 1 --2- 172.21.15.33:0/3507527164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fcff0 0x7f37280fd460 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f371c004940 tx=0x7f371c009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.712+0000 7f373068a700 1 -- 172.21.15.33:0/3507527164 shutdown_connections 2024-04-25T01:38:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.712+0000 7f373068a700 1 --2- 172.21.15.33:0/3507527164 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fcff0 0x7f37280fd460 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.712+0000 7f373068a700 1 --2- 172.21.15.33:0/3507527164 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37280fbd40 0x7f37280fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.712+0000 7f373068a700 1 -- 172.21.15.33:0/3507527164 >> 172.21.15.33:0/3507527164 conn(0x7f37280f74c0 msgr2=0x7f37280f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.712+0000 7f373068a700 1 -- 172.21.15.33:0/3507527164 shutdown_connections 2024-04-25T01:38:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.712+0000 7f373068a700 1 -- 172.21.15.33:0/3507527164 wait complete. 2024-04-25T01:38:46.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f373068a700 1 Processor -- start 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f373068a700 1 -- start start 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f373068a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fbd40 0x7f37280ffe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f373068a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37280fcff0 0x7f37280fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f373068a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37280fe970 con 0x7f37280fbd40 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f373068a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37280feae0 con 0x7f37280fcff0 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f372e426700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fbd40 0x7f37280ffe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f372e426700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fbd40 0x7f37280ffe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51668/0 (socket says 172.21.15.33:51668) 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f372e426700 1 -- 172.21.15.33:0/3840011741 learned_addr learned my addr 172.21.15.33:0/3840011741 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f372dc25700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37280fcff0 0x7f37280fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f372e426700 1 -- 172.21.15.33:0/3840011741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37280fcff0 msgr2=0x7f37280fe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f372e426700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37280fcff0 0x7f37280fe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.713+0000 7f372e426700 1 -- 172.21.15.33:0/3840011741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c005040 con 0x7f37280fbd40 2024-04-25T01:38:46.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.714+0000 7f372e426700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fbd40 0x7f37280ffe60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f372000bbb0 tx=0x7f3720009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:46.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.714+0000 7f3726ffd700 1 -- 172.21.15.33:0/3840011741 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37200092e0 con 0x7f37280fbd40 2024-04-25T01:38:46.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.714+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37280fedc0 con 0x7f37280fbd40 2024-04-25T01:38:46.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.714+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3728115ab0 con 0x7f37280fbd40 2024-04-25T01:38:46.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.714+0000 7f3726ffd700 1 -- 172.21.15.33:0/3840011741 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f372000cb20 con 0x7f37280fbd40 2024-04-25T01:38:46.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.714+0000 7f3726ffd700 1 -- 172.21.15.33:0/3840011741 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3720015d70 con 0x7f37280fbd40 2024-04-25T01:38:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.715+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37140052e0 con 0x7f37280fbd40 2024-04-25T01:38:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.715+0000 7f3726ffd700 1 -- 172.21.15.33:0/3840011741 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f3720012070 con 0x7f37280fbd40 2024-04-25T01:38:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.716+0000 7f3726ffd700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f371006db60 0x7f3710070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.716+0000 7f3726ffd700 1 -- 172.21.15.33:0/3840011741 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f372008cd60 con 0x7f37280fbd40 2024-04-25T01:38:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.716+0000 7f372dc25700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f371006db60 0x7f3710070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:46.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.716+0000 7f372dc25700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f371006db60 0x7f3710070010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f371c00b3b0 tx=0x7f371c009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:46.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.721+0000 7f3726ffd700 1 -- 172.21.15.33:0/3840011741 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3720056900 con 0x7f37280fbd40 2024-04-25T01:38:46.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.946+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3714000bf0 con 0x7f371006db60 2024-04-25T01:38:46.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.948+0000 7f3726ffd700 1 -- 172.21.15.33:0/3840011741 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f3714000bf0 con 0x7f371006db60 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.949+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f371006db60 msgr2=0x7f3710070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.949+0000 7f373068a700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f371006db60 0x7f3710070010 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f371c00b3b0 tx=0x7f371c009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.949+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fbd40 msgr2=0x7f37280ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.949+0000 7f373068a700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fbd40 0x7f37280ffe60 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f372000bbb0 tx=0x7f3720009840 comp rx=0 tx=0).stop 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.950+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 shutdown_connections 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.950+0000 7f373068a700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f371006db60 0x7f3710070010 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.950+0000 7f373068a700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f37280fbd40 0x7f37280ffe60 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.950+0000 7f373068a700 1 --2- 172.21.15.33:0/3840011741 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f37280fcff0 0x7f37280fe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.950+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 >> 172.21.15.33:0/3840011741 conn(0x7f37280f74c0 msgr2=0x7f37280663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.950+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 shutdown_connections 2024-04-25T01:38:46.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:46.950+0000 7f373068a700 1 -- 172.21.15.33:0/3840011741 wait complete. 2024-04-25T01:38:46.941 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.057+0000 7f8561183700 1 -- 172.21.15.33:0/2062107986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbcb0 msgr2=0x7f855c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.057+0000 7f8561183700 1 --2- 172.21.15.33:0/2062107986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbcb0 0x7f855c0fc0c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f854c004940 tx=0x7f854c009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.057+0000 7f8561183700 1 -- 172.21.15.33:0/2062107986 shutdown_connections 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.057+0000 7f8561183700 1 --2- 172.21.15.33:0/2062107986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c0fcf60 0x7f855c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.057+0000 7f8561183700 1 --2- 172.21.15.33:0/2062107986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbcb0 0x7f855c0fc0c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.057+0000 7f8561183700 1 -- 172.21.15.33:0/2062107986 >> 172.21.15.33:0/2062107986 conn(0x7f855c0f7430 msgr2=0x7f855c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.057+0000 7f8561183700 1 -- 172.21.15.33:0/2062107986 shutdown_connections 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.057+0000 7f8561183700 1 -- 172.21.15.33:0/2062107986 wait complete. 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.057+0000 7f8561183700 1 Processor -- start 2024-04-25T01:38:47.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f8561183700 1 -- start start 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f8561183700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c0fbcb0 0x7f855c10da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f8561183700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fcf60 0x7f855c10dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f8561183700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c10e5b0 con 0x7f855c0fcf60 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f8561183700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c10edb0 con 0x7f855c0fbcb0 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f855a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fcf60 0x7f855c10dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f855a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fcf60 0x7f855c10dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51698/0 (socket says 172.21.15.33:51698) 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f855a59c700 1 -- 172.21.15.33:0/3974611109 learned_addr learned my addr 172.21.15.33:0/3974611109 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f855ad9d700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c0fbcb0 0x7f855c10da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f855a59c700 1 -- 172.21.15.33:0/3974611109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c0fbcb0 msgr2=0x7f855c10da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f855a59c700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c0fbcb0 0x7f855c10da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f855a59c700 1 -- 172.21.15.33:0/3974611109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c005040 con 0x7f855c0fcf60 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f855a59c700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fcf60 0x7f855c10dfb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8544002a50 tx=0x7f85440046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:47.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.058+0000 7f85537fe700 1 -- 172.21.15.33:0/3974611109 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8544007cc0 con 0x7f855c0fcf60 2024-04-25T01:38:47.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.059+0000 7f85537fe700 1 -- 172.21.15.33:0/3974611109 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f854400f040 con 0x7f855c0fcf60 2024-04-25T01:38:47.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.059+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c10f090 con 0x7f855c0fcf60 2024-04-25T01:38:47.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.059+0000 7f85537fe700 1 -- 172.21.15.33:0/3974611109 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f854400da70 con 0x7f855c0fcf60 2024-04-25T01:38:47.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.059+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c10f5d0 con 0x7f855c0fcf60 2024-04-25T01:38:47.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.059+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c101af0 con 0x7f855c0fcf60 2024-04-25T01:38:47.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.060+0000 7f85537fe700 1 -- 172.21.15.33:0/3974611109 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f854400dc10 con 0x7f855c0fcf60 2024-04-25T01:38:47.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.061+0000 7f85537fe700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f854806db60 0x7f8548070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.061+0000 7f85537fe700 1 -- 172.21.15.33:0/3974611109 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f854408ca40 con 0x7f855c0fcf60 2024-04-25T01:38:47.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.061+0000 7f855ad9d700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f854806db60 0x7f8548070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.061+0000 7f855ad9d700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f854806db60 0x7f8548070010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f854c00b970 tx=0x7f854c009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.065+0000 7f85537fe700 1 -- 172.21.15.33:0/3974611109 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8544059f80 con 0x7f855c0fcf60 2024-04-25T01:38:47.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.303+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f855c10f7f0 con 0x7f854806db60 2024-04-25T01:38:47.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.304+0000 7f85537fe700 1 -- 172.21.15.33:0/3974611109 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f855c10f7f0 con 0x7f854806db60 2024-04-25T01:38:47.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f854806db60 msgr2=0x7f8548070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f854806db60 0x7f8548070010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f854c00b970 tx=0x7f854c009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:47.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fcf60 msgr2=0x7f855c10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fcf60 0x7f855c10dfb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8544002a50 tx=0x7f85440046b0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 shutdown_connections 2024-04-25T01:38:47.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f854806db60 0x7f8548070010 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c0fbcb0 0x7f855c10da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 --2- 172.21.15.33:0/3974611109 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fcf60 0x7f855c10dfb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 >> 172.21.15.33:0/3974611109 conn(0x7f855c0f7430 msgr2=0x7f855c065b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:47.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 shutdown_connections 2024-04-25T01:38:47.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.306+0000 7f8561183700 1 -- 172.21.15.33:0/3974611109 wait complete. 2024-04-25T01:38:47.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.410+0000 7fdfc072a700 1 -- 172.21.15.33:0/2619762128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80662c0 msgr2=0x7fdfb80ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.410+0000 7fdfc072a700 1 --2- 172.21.15.33:0/2619762128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80662c0 0x7fdfb80ff9a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009ce0 tx=0x7fdfb0004d10 comp rx=0 tx=0).stop 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.410+0000 7fdfc072a700 1 -- 172.21.15.33:0/2619762128 shutdown_connections 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.410+0000 7fdfc072a700 1 --2- 172.21.15.33:0/2619762128 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80ffee0 0x7fdfb8102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.410+0000 7fdfc072a700 1 --2- 172.21.15.33:0/2619762128 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80662c0 0x7fdfb80ff9a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.410+0000 7fdfc072a700 1 -- 172.21.15.33:0/2619762128 >> 172.21.15.33:0/2619762128 conn(0x7fdfb80f7430 msgr2=0x7fdfb80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.410+0000 7fdfc072a700 1 -- 172.21.15.33:0/2619762128 shutdown_connections 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.410+0000 7fdfc072a700 1 -- 172.21.15.33:0/2619762128 wait complete. 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.410+0000 7fdfc072a700 1 Processor -- start 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfc072a700 1 -- start start 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfc072a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80662c0 0x7fdfb810db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfc072a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ffee0 0x7fdfb810e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfc072a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb810e6d0 con 0x7fdfb80ffee0 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfc072a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb810ef20 con 0x7fdfb80662c0 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfbdcc5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ffee0 0x7fdfb810e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfbdcc5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ffee0 0x7fdfb810e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51730/0 (socket says 172.21.15.33:51730) 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfbdcc5700 1 -- 172.21.15.33:0/2176010689 learned_addr learned my addr 172.21.15.33:0/2176010689 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfbe4c6700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80662c0 0x7fdfb810db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfbdcc5700 1 -- 172.21.15.33:0/2176010689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80662c0 msgr2=0x7fdfb810db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfbdcc5700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80662c0 0x7fdfb810db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfbdcc5700 1 -- 172.21.15.33:0/2176010689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb0009940 con 0x7fdfb80ffee0 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfbdcc5700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ffee0 0x7fdfb810e0d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c830 tx=0x7fdfac004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.411+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/2176010689 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfac0106f0 con 0x7fdfb80ffee0 2024-04-25T01:38:47.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.412+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/2176010689 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfac010d20 con 0x7fdfb80ffee0 2024-04-25T01:38:47.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.412+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfb810f200 con 0x7fdfb80ffee0 2024-04-25T01:38:47.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.412+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/2176010689 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfac0113a0 con 0x7fdfb80ffee0 2024-04-25T01:38:47.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.412+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb810f740 con 0x7fdfb80ffee0 2024-04-25T01:38:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.413+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb805f5c0 con 0x7fdfb80ffee0 2024-04-25T01:38:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.413+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/2176010689 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fdfac012400 con 0x7fdfb80ffee0 2024-04-25T01:38:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.413+0000 7fdfb6ffd700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdfa006db60 0x7fdfa0070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.414+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/2176010689 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fdfac08d250 con 0x7fdfb80ffee0 2024-04-25T01:38:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.414+0000 7fdfbe4c6700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdfa006db60 0x7fdfa0070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.414+0000 7fdfbe4c6700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdfa006db60 0x7fdfa0070010 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009470 tx=0x7fdfb0004d10 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:47.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.418+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/2176010689 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fdfac05a920 con 0x7fdfb80ffee0 2024-04-25T01:38:47.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.639+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfb810fac0 con 0x7fdfa006db60 2024-04-25T01:38:47.621 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:47 smithi033 ceph-mon[26353]: Upgrade: Updating alertmanager.smithi033 2024-04-25T01:38:47.621 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:47 smithi033 ceph-mon[26353]: Deploying daemon alertmanager.smithi033 on smithi033 2024-04-25T01:38:47.621 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:47 smithi033 ceph-mon[26353]: pgmap v43: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.650+0000 7fdfb6ffd700 1 -- 172.21.15.33:0/2176010689 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdfb810fac0 con 0x7fdfa006db60 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (2m) 8s ago 11m 25.7M - 0.25.0 c8568f914cd2 c74b90b8b6ca 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (11m) 8s ago 11m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (10m) 29s ago 10m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 unknown 8s ago 10m - - 954c08fa6188 c7ab93999a32 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (5m) 8s ago 5m 61.7M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (5m) 8s ago 5m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (5m) 29s ago 5m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (5m) 29s ago 5m 60.9M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:38:47.631 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (85s) 8s ago 12m 498M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (50s) 29s ago 10m 448M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (12m) 8s ago 12m 55.6M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (10m) 29s ago 10m 47.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (37s) 8s ago 11m 8702k - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (32s) 29s ago 10m 3653k - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (9m) 8s ago 9m 223M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (8m) 8s ago 8m 235M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (8m) 8s ago 8m 187M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (8m) 29s ago 8m 262M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (7m) 29s ago 7m 229M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (7m) 29s ago 7m 217M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:38:47.632 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (15s) 8s ago 10m 40.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:38:47.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdfa006db60 msgr2=0x7fdfa0070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdfa006db60 0x7fdfa0070010 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0009470 tx=0x7fdfb0004d10 comp rx=0 tx=0).stop 2024-04-25T01:38:47.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ffee0 msgr2=0x7fdfb810e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ffee0 0x7fdfb810e0d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fdfac00c830 tx=0x7fdfac004470 comp rx=0 tx=0).stop 2024-04-25T01:38:47.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 shutdown_connections 2024-04-25T01:38:47.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdfa006db60 0x7fdfa0070010 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfb80662c0 0x7fdfb810db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 --2- 172.21.15.33:0/2176010689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfb80ffee0 0x7fdfb810e0d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 >> 172.21.15.33:0/2176010689 conn(0x7fdfb80f7430 msgr2=0x7fdfb80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:47.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 shutdown_connections 2024-04-25T01:38:47.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.652+0000 7fdfc072a700 1 -- 172.21.15.33:0/2176010689 wait complete. 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 -- 172.21.15.33:0/1585481714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 msgr2=0x7fbe300fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 --2- 172.21.15.33:0/1585481714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 0x7fbe300fe440 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c004940 tx=0x7fbe1c009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 -- 172.21.15.33:0/1585481714 shutdown_connections 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 --2- 172.21.15.33:0/1585481714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 0x7fbe300fe440 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 --2- 172.21.15.33:0/1585481714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe300fce60 0x7fbe300fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 -- 172.21.15.33:0/1585481714 >> 172.21.15.33:0/1585481714 conn(0x7fbe300f8540 msgr2=0x7fbe300fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 -- 172.21.15.33:0/1585481714 shutdown_connections 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 -- 172.21.15.33:0/1585481714 wait complete. 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 Processor -- start 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.756+0000 7fbe36d72700 1 -- start start 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe36d72700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe300fce60 0x7fbe3010fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe36d72700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 0x7fbe30110070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe36d72700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe30110670 con 0x7fbe300fdfd0 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe36d72700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe301107e0 con 0x7fbe300fce60 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 0x7fbe30110070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe2ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 0x7fbe30110070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51746/0 (socket says 172.21.15.33:51746) 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe2ffff700 1 -- 172.21.15.33:0/2741369519 learned_addr learned my addr 172.21.15.33:0/2741369519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe34b0e700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe300fce60 0x7fbe3010fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe2ffff700 1 -- 172.21.15.33:0/2741369519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe300fce60 msgr2=0x7fbe3010fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe2ffff700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe300fce60 0x7fbe3010fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe2ffff700 1 -- 172.21.15.33:0/2741369519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c005040 con 0x7fbe300fdfd0 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe34b0e700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe300fce60 0x7fbe3010fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.757+0000 7fbe2ffff700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 0x7fbe30110070 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00b3b0 tx=0x7fbe1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:47.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.758+0000 7fbe2d7fa700 1 -- 172.21.15.33:0/2741369519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c00e040 con 0x7fbe300fdfd0 2024-04-25T01:38:47.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.758+0000 7fbe2d7fa700 1 -- 172.21.15.33:0/2741369519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe1c00bcc0 con 0x7fbe300fdfd0 2024-04-25T01:38:47.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.758+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe301151e0 con 0x7fbe300fdfd0 2024-04-25T01:38:47.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.758+0000 7fbe2d7fa700 1 -- 172.21.15.33:0/2741369519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1c01c630 con 0x7fbe300fdfd0 2024-04-25T01:38:47.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.758+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe301157a0 con 0x7fbe300fdfd0 2024-04-25T01:38:47.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.758+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe30101960 con 0x7fbe300fdfd0 2024-04-25T01:38:47.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.759+0000 7fbe2d7fa700 1 -- 172.21.15.33:0/2741369519 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7fbe1c013980 con 0x7fbe300fdfd0 2024-04-25T01:38:47.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.760+0000 7fbe2d7fa700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbe2006db70 0x7fbe20070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:47.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.760+0000 7fbe2d7fa700 1 -- 172.21.15.33:0/2741369519 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fbe1c08e030 con 0x7fbe300fdfd0 2024-04-25T01:38:47.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.760+0000 7fbe34b0e700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbe2006db70 0x7fbe20070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:47.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.760+0000 7fbe34b0e700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbe2006db70 0x7fbe20070020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009fd0 tx=0x7fbe2400b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:47.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:47.764+0000 7fbe2d7fa700 1 -- 172.21.15.33:0/2741369519 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fbe1c05b5f0 con 0x7fbe300fdfd0 2024-04-25T01:38:47.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:47 smithi138 ceph-mon[32977]: Upgrade: Updating alertmanager.smithi033 2024-04-25T01:38:47.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:47 smithi138 ceph-mon[32977]: Deploying daemon alertmanager.smithi033 on smithi033 2024-04-25T01:38:47.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:47 smithi138 ceph-mon[32977]: pgmap v43: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T01:38:48.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.058+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbe3004c710 con 0x7fbe300fdfd0 2024-04-25T01:38:48.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.058+0000 7fbe2d7fa700 1 -- 172.21.15.33:0/2741369519 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fbe1c05b180 con 0x7fbe300fdfd0 2024-04-25T01:38:48.039 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:38:48.039 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:38:48.040 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-25T01:38:48.041 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:38:48.041 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:38:48.041 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:38:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbe2006db70 msgr2=0x7fbe20070020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbe2006db70 0x7fbe20070020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbe24009fd0 tx=0x7fbe2400b040 comp rx=0 tx=0).stop 2024-04-25T01:38:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 msgr2=0x7fbe30110070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 0x7fbe30110070 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c00b3b0 tx=0x7fbe1c009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 shutdown_connections 2024-04-25T01:38:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fbe2006db70 0x7fbe20070020 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbe300fce60 0x7fbe3010fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 --2- 172.21.15.33:0/2741369519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbe300fdfd0 0x7fbe30110070 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 >> 172.21.15.33:0/2741369519 conn(0x7fbe300f8540 msgr2=0x7fbe300f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 shutdown_connections 2024-04-25T01:38:48.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.060+0000 7fbe36d72700 1 -- 172.21.15.33:0/2741369519 wait complete. 2024-04-25T01:38:48.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.165+0000 7f3d16d52700 1 -- 172.21.15.33:0/4242248715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fbcb0 msgr2=0x7f3d100fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.165+0000 7f3d16d52700 1 --2- 172.21.15.33:0/4242248715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fbcb0 0x7f3d100fc0c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004940 tx=0x7f3d04009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.165+0000 7f3d16d52700 1 -- 172.21.15.33:0/4242248715 shutdown_connections 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.165+0000 7f3d16d52700 1 --2- 172.21.15.33:0/4242248715 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d100fcf60 0x7f3d100fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.165+0000 7f3d16d52700 1 --2- 172.21.15.33:0/4242248715 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fbcb0 0x7f3d100fc0c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.165+0000 7f3d16d52700 1 -- 172.21.15.33:0/4242248715 >> 172.21.15.33:0/4242248715 conn(0x7f3d100f7430 msgr2=0x7f3d100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.165+0000 7f3d16d52700 1 -- 172.21.15.33:0/4242248715 shutdown_connections 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.165+0000 7f3d16d52700 1 -- 172.21.15.33:0/4242248715 wait complete. 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.165+0000 7f3d16d52700 1 Processor -- start 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d16d52700 1 -- start start 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d16d52700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d100fbcb0 0x7f3d10110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d16d52700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fcf60 0x7f3d10112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d16d52700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d10066330 con 0x7f3d100fcf60 2024-04-25T01:38:48.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d16d52700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d10066470 con 0x7f3d100fbcb0 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d0ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fcf60 0x7f3d10112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d0ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fcf60 0x7f3d10112c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51758/0 (socket says 172.21.15.33:51758) 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d0ffff700 1 -- 172.21.15.33:0/1063849376 learned_addr learned my addr 172.21.15.33:0/1063849376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d14aee700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d100fbcb0 0x7f3d10110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d0ffff700 1 -- 172.21.15.33:0/1063849376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d100fbcb0 msgr2=0x7f3d10110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d0ffff700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d100fbcb0 0x7f3d10110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.166+0000 7f3d0ffff700 1 -- 172.21.15.33:0/1063849376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d04005040 con 0x7f3d100fcf60 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.167+0000 7f3d0ffff700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fcf60 0x7f3d10112c70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc002a50 tx=0x7f3cfc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.167+0000 7f3d0d7fa700 1 -- 172.21.15.33:0/1063849376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cfc007cc0 con 0x7f3d100fcf60 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.167+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d10113210 con 0x7f3d100fcf60 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.167+0000 7f3d0d7fa700 1 -- 172.21.15.33:0/1063849376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cfc00f040 con 0x7f3d100fcf60 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.167+0000 7f3d0d7fa700 1 -- 172.21.15.33:0/1063849376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cfc00da30 con 0x7f3d100fcf60 2024-04-25T01:38:48.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.167+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d101137d0 con 0x7f3d100fcf60 2024-04-25T01:38:48.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.168+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d1005f5c0 con 0x7f3d100fcf60 2024-04-25T01:38:48.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.168+0000 7f3d0d7fa700 1 -- 172.21.15.33:0/1063849376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 33) v1 ==== 92027+0+0 (secure 0 0 0) 0x7f3cfc00dbb0 con 0x7f3d100fcf60 2024-04-25T01:38:48.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.169+0000 7f3d0d7fa700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f3d0006db60 0x7f3d00070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:38:48.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.169+0000 7f3d0d7fa700 1 -- 172.21.15.33:0/1063849376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f3cfc08d180 con 0x7f3d100fcf60 2024-04-25T01:38:48.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.169+0000 7f3d14aee700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f3d0006db60 0x7f3d00070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:38:48.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.169+0000 7f3d14aee700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f3d0006db60 0x7f3d00070010 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004890 tx=0x7f3d04009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:38:48.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.173+0000 7f3d0d7fa700 1 -- 172.21.15.33:0/1063849376 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f3cfc056b90 con 0x7f3d100fcf60 2024-04-25T01:38:48.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.402+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d101012c0 con 0x7f3d0006db60 2024-04-25T01:38:48.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.403+0000 7f3d0d7fa700 1 -- 172.21.15.33:0/1063849376 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f3d101012c0 con 0x7f3d0006db60 2024-04-25T01:38:48.384 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:38:48.384 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:38:48.384 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:38:48.384 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-25T01:38:48.384 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:38:48.384 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr" 2024-04-25T01:38:48.385 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:38:48.385 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "2/2 daemons upgraded", 2024-04-25T01:38:48.385 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-25T01:38:48.385 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:38:48.385 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.405+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f3d0006db60 msgr2=0x7f3d00070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.405+0000 7f3d16d52700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f3d0006db60 0x7f3d00070010 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3d04004890 tx=0x7f3d04009e30 comp rx=0 tx=0).stop 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.405+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fcf60 msgr2=0x7f3d10112c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.405+0000 7f3d16d52700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fcf60 0x7f3d10112c70 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc002a50 tx=0x7f3cfc0046b0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.406+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 shutdown_connections 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.406+0000 7f3d16d52700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f3d0006db60 0x7f3d00070010 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.406+0000 7f3d16d52700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d100fbcb0 0x7f3d10110720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.406+0000 7f3d16d52700 1 --2- 172.21.15.33:0/1063849376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d100fcf60 0x7f3d10112c70 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.406+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 >> 172.21.15.33:0/1063849376 conn(0x7f3d100f7430 msgr2=0x7f3d100f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.406+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 shutdown_connections 2024-04-25T01:38:48.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:38:48.406+0000 7f3d16d52700 1 -- 172.21.15.33:0/1063849376 wait complete. 2024-04-25T01:38:48.641 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:48 smithi033 ceph-mon[26353]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:48.642 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:48 smithi033 ceph-mon[26353]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:48.642 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:48 smithi033 ceph-mon[26353]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:48.642 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:48 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2741369519' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:48.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:48 smithi138 ceph-mon[32977]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:48.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:48 smithi138 ceph-mon[32977]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:48.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:48 smithi138 ceph-mon[32977]: from='client.14764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:48.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:48 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2741369519' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:38:49.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:49 smithi033 ceph-mon[26353]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T01:38:49.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:49 smithi033 ceph-mon[26353]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:49.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:49 smithi138 ceph-mon[32977]: pgmap v44: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T01:38:49.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:49 smithi138 ceph-mon[32977]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:38:50.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:50 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:38:50.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:50 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:38:51.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:51 smithi033 ceph-mon[26353]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T01:38:51.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:51 smithi138 ceph-mon[32977]: pgmap v45: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-04-25T01:38:53.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:53 smithi033 ceph-mon[26353]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T01:38:53.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:53 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:53.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:53 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:53.747 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:53 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:53.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:53 smithi138 ceph-mon[32977]: pgmap v46: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-04-25T01:38:53.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:53 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:53.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:53 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:53.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:53 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:38:55.744 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:55 smithi033 ceph-mon[26353]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T01:38:55.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:55 smithi138 ceph-mon[32977]: pgmap v47: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T01:38:57.681 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:57 smithi033 ceph-mon[26353]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:57.772 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:57 smithi138 ceph-mon[32977]: pgmap v48: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:38:58.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:58 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:58.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:58 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:58.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:38:58 smithi033 ceph-mon[26353]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:38:59.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:58 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:59.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:58 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:38:59.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:38:58 smithi138 ceph-mon[32977]: pgmap v49: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:00.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:00 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:00.248 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:00 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:00.522 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:00 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:00.523 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:00 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:01.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:01 smithi033 ceph-mon[26353]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:01.523 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:01 smithi138 ceph-mon[32977]: pgmap v50: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:03.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:03.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:03.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:03.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:03.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:03.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T01:39:03.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T01:39:03.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033:9093"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.987 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.988 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:03.988 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:03 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: pgmap v51: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:04.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:04.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:04.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:04.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: pgmap v52: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: pgmap v53: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033:9093"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:03 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:04.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:04 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:39:04.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:04 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033:9093"}]: dispatch 2024-04-25T01:39:05.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:04 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T01:39:05.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:04 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi033:9093"}]: dispatch 2024-04-25T01:39:05.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:05 smithi033 ceph-mon[26353]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:39:05.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:05 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:39:06.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:05 smithi138 ceph-mon[32977]: pgmap v54: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:39:06.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:05 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:39:06.984 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:06 smithi033 ceph-mon[26353]: Upgrade: Updating grafana.smithi033 2024-04-25T01:39:06.984 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:06 smithi033 ceph-mon[26353]: Deploying daemon grafana.smithi033 on smithi033 2024-04-25T01:39:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:06 smithi138 ceph-mon[32977]: Upgrade: Updating grafana.smithi033 2024-04-25T01:39:07.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:06 smithi138 ceph-mon[32977]: Deploying daemon grafana.smithi033 on smithi033 2024-04-25T01:39:07.997 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:07 smithi033 ceph-mon[26353]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 723 B/s wr, 5 op/s 2024-04-25T01:39:07.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:07 smithi033 ceph-mon[26353]: mgrmap e34: smithi033.asemba(active, since 91s), standbys: smithi138.mmfhyi 2024-04-25T01:39:08.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:07 smithi138 ceph-mon[32977]: pgmap v55: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 723 B/s wr, 5 op/s 2024-04-25T01:39:08.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:07 smithi138 ceph-mon[32977]: mgrmap e34: smithi033.asemba(active, since 91s), standbys: smithi138.mmfhyi 2024-04-25T01:39:09.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:09 smithi033 ceph-mon[26353]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 723 B/s wr, 5 op/s 2024-04-25T01:39:09.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:09 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:09.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:09 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:09.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:09 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:10.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:09 smithi138 ceph-mon[32977]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 723 B/s wr, 5 op/s 2024-04-25T01:39:10.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:09 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:10.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:09 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:10.026 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:09 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:11.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:11 smithi033 ceph-mon[26353]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 723 B/s wr, 3 op/s 2024-04-25T01:39:12.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:11 smithi138 ceph-mon[32977]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.4 KiB/s rd, 723 B/s wr, 3 op/s 2024-04-25T01:39:13.993 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:13 smithi033 ceph-mon[26353]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 614 B/s wr, 4 op/s 2024-04-25T01:39:14.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:13 smithi138 ceph-mon[32977]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 614 B/s wr, 4 op/s 2024-04-25T01:39:15.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:15 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:15.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:15 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:15.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:15 smithi033 ceph-mon[26353]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T01:39:16.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:15 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:16.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:15 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:16.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:15 smithi138 ceph-mon[32977]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-25T01:39:17.248 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:16 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:17.248 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:16 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:17.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:16 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:17.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:16 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:18.072 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:17 smithi033 ceph-mon[26353]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T01:39:18.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:17 smithi138 ceph-mon[32977]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T01:39:18.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.518+0000 7f68d29b1700 1 -- 172.21.15.33:0/684777528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 msgr2=0x7f68cc0fcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:18.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.518+0000 7f68d29b1700 1 --2- 172.21.15.33:0/684777528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 0x7f68cc0fcb40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f68c0004940 tx=0x7f68c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:18.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.518+0000 7f68d29b1700 1 -- 172.21.15.33:0/684777528 shutdown_connections 2024-04-25T01:39:18.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.518+0000 7f68d29b1700 1 --2- 172.21.15.33:0/684777528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 0x7f68cc0fcb40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.518+0000 7f68d29b1700 1 --2- 172.21.15.33:0/684777528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc0fb420 0x7f68cc0fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.518+0000 7f68d29b1700 1 -- 172.21.15.33:0/684777528 >> 172.21.15.33:0/684777528 conn(0x7f68cc0f6b80 msgr2=0x7f68cc0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:18.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.518+0000 7f68d29b1700 1 -- 172.21.15.33:0/684777528 shutdown_connections 2024-04-25T01:39:18.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.518+0000 7f68d29b1700 1 -- 172.21.15.33:0/684777528 wait complete. 2024-04-25T01:39:18.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.519+0000 7f68d29b1700 1 Processor -- start 2024-04-25T01:39:18.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.519+0000 7f68d29b1700 1 -- start start 2024-04-25T01:39:18.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68d29b1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc0fb420 0x7f68cc0ff540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:18.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68d29b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 0x7f68cc0fda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68d29b1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0fe050 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68d29b1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0fe1c0 con 0x7f68cc0fb420 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 0x7f68cc0fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68cb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 0x7f68cc0fda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33158/0 (socket says 172.21.15.33:33158) 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68cb7fe700 1 -- 172.21.15.33:0/2357687674 learned_addr learned my addr 172.21.15.33:0/2357687674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68cbfff700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc0fb420 0x7f68cc0ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68cb7fe700 1 -- 172.21.15.33:0/2357687674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc0fb420 msgr2=0x7f68cc0ff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68cb7fe700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc0fb420 0x7f68cc0ff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.520+0000 7f68cb7fe700 1 -- 172.21.15.33:0/2357687674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0005040 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.521+0000 7f68cb7fe700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 0x7f68cc0fda10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009150 tx=0x7f68c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.521+0000 7f68c8ff9700 1 -- 172.21.15.33:0/2357687674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68c000e040 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.521+0000 7f68c8ff9700 1 -- 172.21.15.33:0/2357687674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68c0004290 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.521+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68cc0fe440 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.521+0000 7f68c8ff9700 1 -- 172.21.15.33:0/2357687674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68c001c630 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.521+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc115a60 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.522+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc05f5c0 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.523+0000 7f68c8ff9700 1 -- 172.21.15.33:0/2357687674 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f68c000bbf0 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.523+0000 7f68c8ff9700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f68b406da40 0x7f68b406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:18.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.523+0000 7f68c8ff9700 1 -- 172.21.15.33:0/2357687674 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f68c008dc40 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.523+0000 7f68cbfff700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f68b406da40 0x7f68b406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:18.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.524+0000 7f68cbfff700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f68b406da40 0x7f68b406fef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0ff2d0 tx=0x7f68bc0091a0 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:18.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.529+0000 7f68c8ff9700 1 -- 172.21.15.33:0/2357687674 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f68c0057990 con 0x7f68cc0fc6d0 2024-04-25T01:39:18.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.806+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68cc0f82c0 con 0x7f68b406da40 2024-04-25T01:39:18.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.807+0000 7f68c8ff9700 1 -- 172.21.15.33:0/2357687674 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f68cc0f82c0 con 0x7f68b406da40 2024-04-25T01:39:18.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.809+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f68b406da40 msgr2=0x7f68b406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:18.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.809+0000 7f68d29b1700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f68b406da40 0x7f68b406fef0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0ff2d0 tx=0x7f68bc0091a0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.810+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 msgr2=0x7f68cc0fda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:18.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.810+0000 7f68d29b1700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 0x7f68cc0fda10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009150 tx=0x7f68c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.810+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 shutdown_connections 2024-04-25T01:39:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.810+0000 7f68d29b1700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f68b406da40 0x7f68b406fef0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.810+0000 7f68d29b1700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68cc0fb420 0x7f68cc0ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.810+0000 7f68d29b1700 1 --2- 172.21.15.33:0/2357687674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68cc0fc6d0 0x7f68cc0fda10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.810+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 >> 172.21.15.33:0/2357687674 conn(0x7f68cc0f6b80 msgr2=0x7f68cc0f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.810+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 shutdown_connections 2024-04-25T01:39:18.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.810+0000 7f68d29b1700 1 -- 172.21.15.33:0/2357687674 wait complete. 2024-04-25T01:39:18.811 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:39:18.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.926+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2473740889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fbcb0 msgr2=0x7f41c40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.926+0000 7f41c8e6e700 1 --2- 172.21.15.33:0/2473740889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fbcb0 0x7f41c40fc0c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f41b8004940 tx=0x7f41b8009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.926+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2473740889 shutdown_connections 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.926+0000 7f41c8e6e700 1 --2- 172.21.15.33:0/2473740889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c40fcf60 0x7f41c40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.926+0000 7f41c8e6e700 1 --2- 172.21.15.33:0/2473740889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fbcb0 0x7f41c40fc0c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.926+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2473740889 >> 172.21.15.33:0/2473740889 conn(0x7f41c40f7430 msgr2=0x7f41c40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.926+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2473740889 shutdown_connections 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.926+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2473740889 wait complete. 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c8e6e700 1 Processor -- start 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c8e6e700 1 -- start start 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c8e6e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c40fbcb0 0x7f41c40ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c8e6e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fcf60 0x7f41c40fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c8e6e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40fe830 con 0x7f41c40fcf60 2024-04-25T01:39:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c8e6e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40fe970 con 0x7f41c40fbcb0 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fcf60 0x7f41c40fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fcf60 0x7f41c40fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33164/0 (socket says 172.21.15.33:33164) 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c1d9b700 1 -- 172.21.15.33:0/2562070323 learned_addr learned my addr 172.21.15.33:0/2562070323 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.927+0000 7f41c259c700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c40fbcb0 0x7f41c40ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.928+0000 7f41c1d9b700 1 -- 172.21.15.33:0/2562070323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c40fbcb0 msgr2=0x7f41c40ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.928+0000 7f41c1d9b700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c40fbcb0 0x7f41c40ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.928+0000 7f41c1d9b700 1 -- 172.21.15.33:0/2562070323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b8005040 con 0x7f41c40fcf60 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.928+0000 7f41c1d9b700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fcf60 0x7f41c40fe2a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f41b4002a50 tx=0x7f41b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.928+0000 7f41b2ffd700 1 -- 172.21.15.33:0/2562070323 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b4007cc0 con 0x7f41c40fcf60 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.928+0000 7f41b2ffd700 1 -- 172.21.15.33:0/2562070323 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41b400f040 con 0x7f41c40fcf60 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.928+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c40fec50 con 0x7f41c40fcf60 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.928+0000 7f41b2ffd700 1 -- 172.21.15.33:0/2562070323 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41b400da30 con 0x7f41c40fcf60 2024-04-25T01:39:18.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.928+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c4115ab0 con 0x7f41c40fcf60 2024-04-25T01:39:18.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.929+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c4101af0 con 0x7f41c40fcf60 2024-04-25T01:39:18.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.930+0000 7f41b2ffd700 1 -- 172.21.15.33:0/2562070323 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f41b400db90 con 0x7f41c40fcf60 2024-04-25T01:39:18.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.931+0000 7f41b2ffd700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f41ac06da40 0x7f41ac06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:18.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.931+0000 7f41b2ffd700 1 -- 172.21.15.33:0/2562070323 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f41b408cf00 con 0x7f41c40fcf60 2024-04-25T01:39:18.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.931+0000 7f41c259c700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f41ac06da40 0x7f41ac06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:18.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.931+0000 7f41c259c700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f41ac06da40 0x7f41ac06fef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41c40ffb10 tx=0x7f41b8009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:18.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:18.936+0000 7f41b2ffd700 1 -- 172.21.15.33:0/2562070323 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f41b4056b30 con 0x7f41c40fcf60 2024-04-25T01:39:19.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.168+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41c405e3a0 con 0x7f41ac06da40 2024-04-25T01:39:19.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.171+0000 7f41b2ffd700 1 -- 172.21.15.33:0/2562070323 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f41c405e3a0 con 0x7f41ac06da40 2024-04-25T01:39:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.174+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f41ac06da40 msgr2=0x7f41ac06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.174+0000 7f41c8e6e700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f41ac06da40 0x7f41ac06fef0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41c40ffb10 tx=0x7f41b8009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.174+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fcf60 msgr2=0x7f41c40fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.174+0000 7f41c8e6e700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fcf60 0x7f41c40fe2a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f41b4002a50 tx=0x7f41b40046b0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.174+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 shutdown_connections 2024-04-25T01:39:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.174+0000 7f41c8e6e700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f41ac06da40 0x7f41ac06fef0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.174+0000 7f41c8e6e700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41c40fbcb0 0x7f41c40ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.174+0000 7f41c8e6e700 1 --2- 172.21.15.33:0/2562070323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41c40fcf60 0x7f41c40fe2a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.174+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 >> 172.21.15.33:0/2562070323 conn(0x7f41c40f7430 msgr2=0x7f41c4065b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.175+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 shutdown_connections 2024-04-25T01:39:19.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.175+0000 7f41c8e6e700 1 -- 172.21.15.33:0/2562070323 wait complete. 2024-04-25T01:39:19.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.280+0000 7ff131b00700 1 -- 172.21.15.33:0/856573439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fbcb0 msgr2=0x7ff12c0fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.280+0000 7ff131b00700 1 --2- 172.21.15.33:0/856573439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fbcb0 0x7ff12c0fc0c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff11c004940 tx=0x7ff11c009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.280+0000 7ff131b00700 1 -- 172.21.15.33:0/856573439 shutdown_connections 2024-04-25T01:39:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.280+0000 7ff131b00700 1 --2- 172.21.15.33:0/856573439 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff12c0fcf60 0x7ff12c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.280+0000 7ff131b00700 1 --2- 172.21.15.33:0/856573439 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fbcb0 0x7ff12c0fc0c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.280+0000 7ff131b00700 1 -- 172.21.15.33:0/856573439 >> 172.21.15.33:0/856573439 conn(0x7ff12c0f7430 msgr2=0x7ff12c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.280+0000 7ff131b00700 1 -- 172.21.15.33:0/856573439 shutdown_connections 2024-04-25T01:39:19.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.280+0000 7ff131b00700 1 -- 172.21.15.33:0/856573439 wait complete. 2024-04-25T01:39:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.280+0000 7ff131b00700 1 Processor -- start 2024-04-25T01:39:19.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff131b00700 1 -- start start 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff131b00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff12c0fbcb0 0x7ff12c10dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff131b00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fcf60 0x7ff12c10dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff131b00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c10e5f0 con 0x7ff12c0fcf60 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff131b00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c10eda0 con 0x7ff12c0fbcb0 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff12affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fcf60 0x7ff12c10dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff12affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fcf60 0x7ff12c10dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33176/0 (socket says 172.21.15.33:33176) 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff12affd700 1 -- 172.21.15.33:0/1396364590 learned_addr learned my addr 172.21.15.33:0/1396364590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff12b7fe700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff12c0fbcb0 0x7ff12c10dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff12affd700 1 -- 172.21.15.33:0/1396364590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff12c0fbcb0 msgr2=0x7ff12c10dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff12affd700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff12c0fbcb0 0x7ff12c10dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.281+0000 7ff12affd700 1 -- 172.21.15.33:0/1396364590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c005040 con 0x7ff12c0fcf60 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.282+0000 7ff12affd700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fcf60 0x7ff12c10dff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff114002a50 tx=0x7ff1140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.282+0000 7ff123fff700 1 -- 172.21.15.33:0/1396364590 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff114007cc0 con 0x7ff12c0fcf60 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.282+0000 7ff123fff700 1 -- 172.21.15.33:0/1396364590 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff11400f040 con 0x7ff12c0fcf60 2024-04-25T01:39:19.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.282+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c10f080 con 0x7ff12c0fcf60 2024-04-25T01:39:19.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.282+0000 7ff123fff700 1 -- 172.21.15.33:0/1396364590 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11400da30 con 0x7ff12c0fcf60 2024-04-25T01:39:19.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.282+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c10f5c0 con 0x7ff12c0fcf60 2024-04-25T01:39:19.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.283+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c101af0 con 0x7ff12c0fcf60 2024-04-25T01:39:19.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.284+0000 7ff123fff700 1 -- 172.21.15.33:0/1396364590 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7ff11400db90 con 0x7ff12c0fcf60 2024-04-25T01:39:19.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.284+0000 7ff123fff700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff11806da40 0x7ff11806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.284+0000 7ff123fff700 1 -- 172.21.15.33:0/1396364590 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7ff11408ce30 con 0x7ff12c0fcf60 2024-04-25T01:39:19.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.284+0000 7ff12b7fe700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff11806da40 0x7ff11806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.285+0000 7ff12b7fe700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff11806da40 0x7ff11806fef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00b970 tx=0x7ff11c009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:19.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.289+0000 7ff123fff700 1 -- 172.21.15.33:0/1396364590 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff11405a610 con 0x7ff12c0fcf60 2024-04-25T01:39:19.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.504+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff12c10f830 con 0x7ff11806da40 2024-04-25T01:39:19.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.515+0000 7ff123fff700 1 -- 172.21.15.33:0/1396364590 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ff12c10f830 con 0x7ff11806da40 2024-04-25T01:39:19.497 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (26s) 4s ago 11m 14.9M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (11m) 4s ago 11m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (10m) 61s ago 10m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (10s) 4s ago 11m 52.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (6m) 4s ago 6m 61.9M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (6m) 4s ago 6m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (6m) 61s ago 6m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (6m) 61s ago 6m 60.9M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (117s) 4s ago 12m 505M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (82s) 61s ago 10m 448M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (12m) 4s ago 13m 56.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (10m) 61s ago 10m 47.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (69s) 4s ago 11m 15.9M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:39:19.498 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (64s) 61s ago 10m 3653k - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:39:19.499 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (9m) 4s ago 9m 223M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:39:19.499 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (9m) 4s ago 9m 236M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:39:19.499 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (9m) 4s ago 9m 188M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:39:19.499 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (8m) 61s ago 8m 262M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:39:19.499 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (8m) 61s ago 8m 229M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:39:19.499 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (7m) 61s ago 7m 217M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:39:19.499 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (46s) 4s ago 11m 48.4M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff11806da40 msgr2=0x7ff11806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff11806da40 0x7ff11806fef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00b970 tx=0x7ff11c009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fcf60 msgr2=0x7ff12c10dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fcf60 0x7ff12c10dff0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff114002a50 tx=0x7ff1140046b0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 shutdown_connections 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff11806da40 0x7ff11806fef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff12c0fbcb0 0x7ff12c10dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 --2- 172.21.15.33:0/1396364590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff12c0fcf60 0x7ff12c10dff0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 >> 172.21.15.33:0/1396364590 conn(0x7ff12c0f7430 msgr2=0x7ff12c065ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 shutdown_connections 2024-04-25T01:39:19.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.517+0000 7ff131b00700 1 -- 172.21.15.33:0/1396364590 wait complete. 2024-04-25T01:39:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.623+0000 7fa77718f700 1 -- 172.21.15.33:0/997026895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 msgr2=0x7fa7700fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.623+0000 7fa77718f700 1 --2- 172.21.15.33:0/997026895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 0x7fa7700fd3d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa760004940 tx=0x7fa760009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.623+0000 7fa77718f700 1 -- 172.21.15.33:0/997026895 shutdown_connections 2024-04-25T01:39:19.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.623+0000 7fa77718f700 1 --2- 172.21.15.33:0/997026895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 0x7fa7700fd3d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.623+0000 7fa77718f700 1 --2- 172.21.15.33:0/997026895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700fbcb0 0x7fa7700fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.623+0000 7fa77718f700 1 -- 172.21.15.33:0/997026895 >> 172.21.15.33:0/997026895 conn(0x7fa7700f7430 msgr2=0x7fa7700f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.623+0000 7fa77718f700 1 -- 172.21.15.33:0/997026895 shutdown_connections 2024-04-25T01:39:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.623+0000 7fa77718f700 1 -- 172.21.15.33:0/997026895 wait complete. 2024-04-25T01:39:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.624+0000 7fa77718f700 1 Processor -- start 2024-04-25T01:39:19.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.624+0000 7fa77718f700 1 -- start start 2024-04-25T01:39:19.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa77718f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700fbcb0 0x7fa77010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa77718f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 0x7fa77010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa77718f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77010e430 con 0x7fa7700fcf60 2024-04-25T01:39:19.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa77718f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77010e5a0 con 0x7fa7700fbcb0 2024-04-25T01:39:19.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 0x7fa77010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa76ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 0x7fa77010de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33184/0 (socket says 172.21.15.33:33184) 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa76ffff700 1 -- 172.21.15.33:0/3125795501 learned_addr learned my addr 172.21.15.33:0/3125795501 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa774f2b700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700fbcb0 0x7fa77010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa76ffff700 1 -- 172.21.15.33:0/3125795501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700fbcb0 msgr2=0x7fa77010d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa76ffff700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700fbcb0 0x7fa77010d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa76ffff700 1 -- 172.21.15.33:0/3125795501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa760005040 con 0x7fa7700fcf60 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa76ffff700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 0x7fa77010de30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa76000b3b0 tx=0x7fa760009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa76d7fa700 1 -- 172.21.15.33:0/3125795501 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa76000e040 con 0x7fa7700fcf60 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa76d7fa700 1 -- 172.21.15.33:0/3125795501 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa76000bbf0 con 0x7fa7700fcf60 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.625+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa770112fa0 con 0x7fa7700fcf60 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.626+0000 7fa76d7fa700 1 -- 172.21.15.33:0/3125795501 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa76001c630 con 0x7fa7700fcf60 2024-04-25T01:39:19.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.626+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa770113530 con 0x7fa7700fcf60 2024-04-25T01:39:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.627+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa770101af0 con 0x7fa7700fcf60 2024-04-25T01:39:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.627+0000 7fa76d7fa700 1 -- 172.21.15.33:0/3125795501 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fa76000bd60 con 0x7fa7700fcf60 2024-04-25T01:39:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.628+0000 7fa76d7fa700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fa75c06da40 0x7fa75c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.628+0000 7fa76d7fa700 1 -- 172.21.15.33:0/3125795501 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fa76008dc20 con 0x7fa7700fcf60 2024-04-25T01:39:19.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.628+0000 7fa774f2b700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fa75c06da40 0x7fa75c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:19.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.628+0000 7fa774f2b700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fa75c06da40 0x7fa75c06fef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa758004f40 tx=0x7fa75800b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:19.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.633+0000 7fa76d7fa700 1 -- 172.21.15.33:0/3125795501 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fa76005b4a0 con 0x7fa7700fcf60 2024-04-25T01:39:19.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.760+0000 7fa76d7fa700 1 -- 172.21.15.33:0/3125795501 <== mon.0 v2:172.21.15.33:3300/0 7 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa76005b4a0 con 0x7fa7700fcf60 2024-04-25T01:39:19.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.937+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa754005120 con 0x7fa7700fcf60 2024-04-25T01:39:19.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.937+0000 7fa76d7fa700 1 -- 172.21.15.33:0/3125795501 <== mon.0 v2:172.21.15.33:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7fa76005b030 con 0x7fa7700fcf60 2024-04-25T01:39:19.920 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:19.920 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:19.920 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:19.920 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:19.920 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:19.920 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:19.920 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:19.920 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:39:19.920 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033:3000"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.921 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:19.922 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:39:19.922 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:39:19.923 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:39:19.923 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:39:19.923 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:39:19.923 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:39:19.923 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-25T01:39:19.923 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:39:19.923 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:39:19.923 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:39:19.923 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-25T01:39:19.924 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-25T01:39:19.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:19 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.939+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fa75c06da40 msgr2=0x7fa75c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.939+0000 7fa77718f700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fa75c06da40 0x7fa75c06fef0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa758004f40 tx=0x7fa75800b040 comp rx=0 tx=0).stop 2024-04-25T01:39:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.939+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 msgr2=0x7fa77010de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.939+0000 7fa77718f700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 0x7fa77010de30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa76000b3b0 tx=0x7fa760009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.940+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 shutdown_connections 2024-04-25T01:39:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.940+0000 7fa77718f700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fa75c06da40 0x7fa75c06fef0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.940+0000 7fa77718f700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7700fbcb0 0x7fa77010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.940+0000 7fa77718f700 1 --2- 172.21.15.33:0/3125795501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7700fcf60 0x7fa77010de30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.940+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 >> 172.21.15.33:0/3125795501 conn(0x7fa7700f7430 msgr2=0x7fa770066790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.940+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 shutdown_connections 2024-04-25T01:39:19.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:19.940+0000 7fa77718f700 1 -- 172.21.15.33:0/3125795501 wait complete. 2024-04-25T01:39:20.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 -- 172.21.15.33:0/1804944071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 msgr2=0x7fec380fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:20.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 --2- 172.21.15.33:0/1804944071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 0x7fec380fe8f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fec2c004940 tx=0x7fec2c009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:20.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 -- 172.21.15.33:0/1804944071 shutdown_connections 2024-04-25T01:39:20.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 --2- 172.21.15.33:0/1804944071 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec380fee30 0x7fec380faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 --2- 172.21.15.33:0/1804944071 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 0x7fec380fe8f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 -- 172.21.15.33:0/1804944071 >> 172.21.15.33:0/1804944071 conn(0x7fec380f6be0 msgr2=0x7fec380f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:20.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 -- 172.21.15.33:0/1804944071 shutdown_connections 2024-04-25T01:39:20.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 -- 172.21.15.33:0/1804944071 wait complete. 2024-04-25T01:39:20.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 Processor -- start 2024-04-25T01:39:20.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.045+0000 7fec3eb66700 1 -- start start 2024-04-25T01:39:20.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3eb66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 0x7fec380fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:20.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3eb66700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec380fee30 0x7fec380fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:20.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3eb66700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380fe0f0 con 0x7fec380fe4e0 2024-04-25T01:39:20.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3eb66700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec380fc6e0 con 0x7fec380fee30 2024-04-25T01:39:20.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3c902700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 0x7fec380fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:20.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3c902700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 0x7fec380fdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33210/0 (socket says 172.21.15.33:33210) 2024-04-25T01:39:20.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3c902700 1 -- 172.21.15.33:0/3770802418 learned_addr learned my addr 172.21.15.33:0/3770802418 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:39:20.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec37fff700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec380fee30 0x7fec380fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:20.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3c902700 1 -- 172.21.15.33:0/3770802418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec380fee30 msgr2=0x7fec380fc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:20.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3c902700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec380fee30 0x7fec380fc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3c902700 1 -- 172.21.15.33:0/3770802418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c005040 con 0x7fec380fe4e0 2024-04-25T01:39:20.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.046+0000 7fec3c902700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 0x7fec380fdaf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009150 tx=0x7fec2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:20.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.047+0000 7fec357fa700 1 -- 172.21.15.33:0/3770802418 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c00e040 con 0x7fec380fe4e0 2024-04-25T01:39:20.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.047+0000 7fec357fa700 1 -- 172.21.15.33:0/3770802418 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec2c004290 con 0x7fec380fe4e0 2024-04-25T01:39:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.047+0000 7fec357fa700 1 -- 172.21.15.33:0/3770802418 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c01c630 con 0x7fec380fe4e0 2024-04-25T01:39:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.047+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec380fc930 con 0x7fec380fe4e0 2024-04-25T01:39:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.047+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec380fcef0 con 0x7fec380fe4e0 2024-04-25T01:39:20.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.048+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3804c710 con 0x7fec380fe4e0 2024-04-25T01:39:20.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.048+0000 7fec357fa700 1 -- 172.21.15.33:0/3770802418 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fec2c00bbf0 con 0x7fec380fe4e0 2024-04-25T01:39:20.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.049+0000 7fec357fa700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fec2806da40 0x7fec2806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:20.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.049+0000 7fec357fa700 1 -- 172.21.15.33:0/3770802418 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fec2c08dc30 con 0x7fec380fe4e0 2024-04-25T01:39:20.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.049+0000 7fec37fff700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fec2806da40 0x7fec2806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:20.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.049+0000 7fec37fff700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fec2806da40 0x7fec2806fef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fec24009fd0 tx=0x7fec2400b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:20.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.053+0000 7fec357fa700 1 -- 172.21.15.33:0/3770802418 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fec2c05b4b0 con 0x7fec380fe4e0 2024-04-25T01:39:20.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.284+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec381025b0 con 0x7fec2806da40 2024-04-25T01:39:20.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.285+0000 7fec357fa700 1 -- 172.21.15.33:0/3770802418 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fec381025b0 con 0x7fec2806da40 2024-04-25T01:39:20.268 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:39:20.268 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": null, 2024-04-25T01:39:20.268 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": false, 2024-04-25T01:39:20.268 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "", 2024-04-25T01:39:20.268 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [], 2024-04-25T01:39:20.268 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": null, 2024-04-25T01:39:20.268 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "", 2024-04-25T01:39:20.268 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:39:20.268 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.287+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fec2806da40 msgr2=0x7fec2806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.287+0000 7fec3eb66700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fec2806da40 0x7fec2806fef0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fec24009fd0 tx=0x7fec2400b040 comp rx=0 tx=0).stop 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.287+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 msgr2=0x7fec380fdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.287+0000 7fec3eb66700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 0x7fec380fdaf0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009150 tx=0x7fec2c009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.288+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 shutdown_connections 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.288+0000 7fec3eb66700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fec2806da40 0x7fec2806fef0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.288+0000 7fec3eb66700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec380fe4e0 0x7fec380fdaf0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.288+0000 7fec3eb66700 1 --2- 172.21.15.33:0/3770802418 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec380fee30 0x7fec380fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.288+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 >> 172.21.15.33:0/3770802418 conn(0x7fec380f6be0 msgr2=0x7fec380f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.288+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 shutdown_connections 2024-04-25T01:39:20.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:20.288+0000 7fec3eb66700 1 -- 172.21.15.33:0/3770802418 wait complete. 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='client.14776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033:3000"}]: dispatch 2024-04-25T01:39:20.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-25T01:39:20.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T01:39:20.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-25T01:39:20.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-25T01:39:20.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:19 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:21.248 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033:3000"}]: dispatch 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: Upgrade: Finalizing container_image settings 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: Upgrade: Complete! 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/3125795501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:21.249 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:21.250 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:21.250 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:21.250 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:21.250 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-25T01:39:21.250 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:20 smithi033 ceph-mon[26353]: Cluster is now healthy 2024-04-25T01:39:21.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='client.14780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi033:3000"}]: dispatch 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: Upgrade: Finalizing container_image settings 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: Upgrade: Complete! 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/3125795501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:21.274 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:21.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:21.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:21.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:39:21.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:39:21.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:39:21.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:21.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-04-25T01:39:21.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:20 smithi138 ceph-mon[32977]: Cluster is now healthy 2024-04-25T01:39:22.248 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:21 smithi033 ceph-mon[26353]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T01:39:22.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:21 smithi138 ceph-mon[32977]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T01:39:24.977 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:24 smithi138 ceph-mon[32977]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:24.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:24 smithi033 ceph-mon[26353]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:26.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:26 smithi033 ceph-mon[26353]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:27.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:26 smithi138 ceph-mon[32977]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:28.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:28 smithi033 ceph-mon[26353]: pgmap v66: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:29.023 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:28 smithi138 ceph-mon[32977]: pgmap v66: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:30.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:30 smithi033 ceph-mon[26353]: pgmap v67: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:31.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:30 smithi138 ceph-mon[32977]: pgmap v67: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:39:32.998 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:32 smithi033 ceph-mon[26353]: pgmap v68: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:33.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:32 smithi138 ceph-mon[32977]: pgmap v68: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:34.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:34 smithi033 ceph-mon[26353]: pgmap v69: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:35.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:34 smithi138 ceph-mon[32977]: pgmap v69: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:36.499 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:36 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:36.499 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:36 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:39:36.499 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:36 smithi033 ceph-mon[26353]: pgmap v70: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:36.524 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:36 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:39:36.524 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:36 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:39:36.524 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:36 smithi138 ceph-mon[32977]: pgmap v70: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:38.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:38 smithi033 ceph-mon[26353]: pgmap v71: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:39.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:38 smithi138 ceph-mon[32977]: pgmap v71: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:40.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:40 smithi033 ceph-mon[26353]: pgmap v72: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:41.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:40 smithi138 ceph-mon[32977]: pgmap v72: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:42.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:42 smithi033 ceph-mon[26353]: pgmap v73: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:43.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:42 smithi138 ceph-mon[32977]: pgmap v73: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:44.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:44 smithi033 ceph-mon[26353]: pgmap v74: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:45.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:44 smithi138 ceph-mon[32977]: pgmap v74: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:46.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:46 smithi033 ceph-mon[26353]: pgmap v75: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:47.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:46 smithi138 ceph-mon[32977]: pgmap v75: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:48.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:48 smithi033 ceph-mon[26353]: pgmap v76: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:49.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:48 smithi138 ceph-mon[32977]: pgmap v76: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:50.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.407+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/3262161062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 msgr2=0x7ff2cc0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:50.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.407+0000 7ff2d3f5f700 1 --2- 172.21.15.33:0/3262161062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 0x7ff2cc0fd270 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8004940 tx=0x7ff2c8009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:50.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.407+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/3262161062 shutdown_connections 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.407+0000 7ff2d3f5f700 1 --2- 172.21.15.33:0/3262161062 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc0fdfd0 0x7ff2cc0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.407+0000 7ff2d3f5f700 1 --2- 172.21.15.33:0/3262161062 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 0x7ff2cc0fd270 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.407+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/3262161062 >> 172.21.15.33:0/3262161062 conn(0x7ff2cc0f8540 msgr2=0x7ff2cc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/3262161062 shutdown_connections 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/3262161062 wait complete. 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d3f5f700 1 Processor -- start 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d3f5f700 1 -- start start 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d3f5f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 0x7ff2cc071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d3f5f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc0fdfd0 0x7ff2cc0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d3f5f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc072200 con 0x7ff2cc0fce60 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d3f5f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2cc0707f0 con 0x7ff2cc0fdfd0 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d1cfb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 0x7ff2cc071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:50.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d1cfb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 0x7ff2cc071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33588/0 (socket says 172.21.15.33:33588) 2024-04-25T01:39:50.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.408+0000 7ff2d1cfb700 1 -- 172.21.15.33:0/4231663637 learned_addr learned my addr 172.21.15.33:0/4231663637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:39:50.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2d14fa700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc0fdfd0 0x7ff2cc0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:50.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2d1cfb700 1 -- 172.21.15.33:0/4231663637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc0fdfd0 msgr2=0x7ff2cc0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:50.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2d1cfb700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc0fdfd0 0x7ff2cc0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:50.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2d1cfb700 1 -- 172.21.15.33:0/4231663637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c8005040 con 0x7ff2cc0fce60 2024-04-25T01:39:50.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2d1cfb700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 0x7ff2cc071c00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800b3b0 tx=0x7ff2c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:50.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2c27fc700 1 -- 172.21.15.33:0/4231663637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2c800e040 con 0x7ff2cc0fce60 2024-04-25T01:39:50.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2c27fc700 1 -- 172.21.15.33:0/4231663637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2c800bbf0 con 0x7ff2cc0fce60 2024-04-25T01:39:50.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2cc070a40 con 0x7ff2cc0fce60 2024-04-25T01:39:50.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2c27fc700 1 -- 172.21.15.33:0/4231663637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2c801c630 con 0x7ff2cc0fce60 2024-04-25T01:39:50.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.409+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2cc071000 con 0x7ff2cc0fce60 2024-04-25T01:39:50.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.410+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2cc04c710 con 0x7ff2cc0fce60 2024-04-25T01:39:50.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.411+0000 7ff2c27fc700 1 -- 172.21.15.33:0/4231663637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7ff2c800bd60 con 0x7ff2cc0fce60 2024-04-25T01:39:50.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.411+0000 7ff2c27fc700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff2b806da40 0x7ff2b806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:50.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.412+0000 7ff2c27fc700 1 -- 172.21.15.33:0/4231663637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7ff2c808dc40 con 0x7ff2cc0fce60 2024-04-25T01:39:50.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.412+0000 7ff2d14fa700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff2b806da40 0x7ff2b806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:50.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.412+0000 7ff2d14fa700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff2b806da40 0x7ff2b806fef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc0041d0 tx=0x7ff2bc0043c0 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:50.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.416+0000 7ff2c27fc700 1 -- 172.21.15.33:0/4231663637 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff2c805b4c0 con 0x7ff2cc0fce60 2024-04-25T01:39:50.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.642+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2cc102330 con 0x7ff2b806da40 2024-04-25T01:39:50.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.643+0000 7ff2c27fc700 1 -- 172.21.15.33:0/4231663637 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7ff2cc102330 con 0x7ff2b806da40 2024-04-25T01:39:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff2b806da40 msgr2=0x7ff2b806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff2b806da40 0x7ff2b806fef0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff2bc0041d0 tx=0x7ff2bc0043c0 comp rx=0 tx=0).stop 2024-04-25T01:39:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 msgr2=0x7ff2cc071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 0x7ff2cc071c00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800b3b0 tx=0x7ff2c8009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 shutdown_connections 2024-04-25T01:39:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff2b806da40 0x7ff2b806fef0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:50.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2cc0fce60 0x7ff2cc071c00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 --2- 172.21.15.33:0/4231663637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2cc0fdfd0 0x7ff2cc0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 >> 172.21.15.33:0/4231663637 conn(0x7ff2cc0f8540 msgr2=0x7ff2cc0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 shutdown_connections 2024-04-25T01:39:50.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:50.645+0000 7ff2d3f5f700 1 -- 172.21.15.33:0/4231663637 wait complete. 2024-04-25T01:39:50.641 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:50 smithi033 ceph-mon[26353]: pgmap v77: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:50.641 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:50 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:39:51.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:50 smithi138 ceph-mon[32977]: pgmap v77: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:51.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:50 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:39:51.497 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-25T01:39:51.749 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:51 smithi033 ceph-mon[26353]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:51.816 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:39:52.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:51 smithi138 ceph-mon[32977]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:39:52.966 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:52 smithi033 ceph-mon[26353]: pgmap v78: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:53.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:52 smithi138 ceph-mon[32977]: pgmap v78: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 -- 172.21.15.33:0/1380137937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 msgr2=0x7ff56c0fb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 --2- 172.21.15.33:0/1380137937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 0x7ff56c0fb7c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff558004940 tx=0x7ff558009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 -- 172.21.15.33:0/1380137937 shutdown_connections 2024-04-25T01:39:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 --2- 172.21.15.33:0/1380137937 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 0x7ff56c0fb7c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 --2- 172.21.15.33:0/1380137937 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c0fdc10 0x7ff56c0fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 -- 172.21.15.33:0/1380137937 >> 172.21.15.33:0/1380137937 conn(0x7ff56c073350 msgr2=0x7ff56c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 -- 172.21.15.33:0/1380137937 shutdown_connections 2024-04-25T01:39:53.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 -- 172.21.15.33:0/1380137937 wait complete. 2024-04-25T01:39:53.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 Processor -- start 2024-04-25T01:39:53.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.537+0000 7ff572f73700 1 -- start start 2024-04-25T01:39:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff572f73700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 0x7ff56c115c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff572f73700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c0fdc10 0x7ff56c116180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff572f73700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c1167d0 con 0x7ff56c0fb350 2024-04-25T01:39:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff572f73700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c10fdb0 con 0x7ff56c0fdc10 2024-04-25T01:39:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff570d0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 0x7ff56c115c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff570d0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 0x7ff56c115c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33606/0 (socket says 172.21.15.33:33606) 2024-04-25T01:39:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff570d0f700 1 -- 172.21.15.33:0/534325990 learned_addr learned my addr 172.21.15.33:0/534325990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:39:53.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff56bfff700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c0fdc10 0x7ff56c116180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff570d0f700 1 -- 172.21.15.33:0/534325990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c0fdc10 msgr2=0x7ff56c116180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff570d0f700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c0fdc10 0x7ff56c116180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.538+0000 7ff570d0f700 1 -- 172.21.15.33:0/534325990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff558005040 con 0x7ff56c0fb350 2024-04-25T01:39:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.539+0000 7ff570d0f700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 0x7ff56c115c40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff560002a50 tx=0x7ff5600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.539+0000 7ff5697fa700 1 -- 172.21.15.33:0/534325990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff560007cc0 con 0x7ff56c0fb350 2024-04-25T01:39:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.539+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56c110090 con 0x7ff56c0fb350 2024-04-25T01:39:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.539+0000 7ff5697fa700 1 -- 172.21.15.33:0/534325990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff56000f040 con 0x7ff56c0fb350 2024-04-25T01:39:53.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.539+0000 7ff5697fa700 1 -- 172.21.15.33:0/534325990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff56000da70 con 0x7ff56c0fb350 2024-04-25T01:39:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.539+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56c1105d0 con 0x7ff56c0fb350 2024-04-25T01:39:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.540+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56c0fc360 con 0x7ff56c0fb350 2024-04-25T01:39:53.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.540+0000 7ff5697fa700 1 -- 172.21.15.33:0/534325990 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7ff56000dc10 con 0x7ff56c0fb350 2024-04-25T01:39:53.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.541+0000 7ff5697fa700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff55c06da40 0x7ff55c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:53.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.541+0000 7ff5697fa700 1 -- 172.21.15.33:0/534325990 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7ff56008c700 con 0x7ff56c0fb350 2024-04-25T01:39:53.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.541+0000 7ff56bfff700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff55c06da40 0x7ff55c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:53.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.541+0000 7ff56bfff700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff55c06da40 0x7ff55c06fef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff558004f90 tx=0x7ff558009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:53.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.546+0000 7ff5697fa700 1 -- 172.21.15.33:0/534325990 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7ff560059ee0 con 0x7ff56c0fb350 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.844+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff56c111290 con 0x7ff56c0fb350 2024-04-25T01:39:53.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.844+0000 7ff5697fa700 1 -- 172.21.15.33:0/534325990 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7ff560059a70 con 0x7ff56c0fb350 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff55c06da40 msgr2=0x7ff55c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff55c06da40 0x7ff55c06fef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff558004f90 tx=0x7ff558009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 msgr2=0x7ff56c115c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:53.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 0x7ff56c115c40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff560002a50 tx=0x7ff5600046b0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 shutdown_connections 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7ff55c06da40 0x7ff55c06fef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c0fb350 0x7ff56c115c40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 --2- 172.21.15.33:0/534325990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c0fdc10 0x7ff56c116180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 >> 172.21.15.33:0/534325990 conn(0x7ff56c073350 msgr2=0x7ff56c0f8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 shutdown_connections 2024-04-25T01:39:53.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:53.846+0000 7ff572f73700 1 -- 172.21.15.33:0/534325990 wait complete. 2024-04-25T01:39:53.840 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:39:54.470 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-25T01:39:54.743 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:39:54.748 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:54 smithi033 ceph-mon[26353]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:54.748 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:54 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/534325990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:55.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:54 smithi138 ceph-mon[32977]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:55.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:54 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/534325990' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:56.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.612+0000 7f614e559700 1 -- 172.21.15.33:0/3048673485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fde10 msgr2=0x7f61480fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:56.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.612+0000 7f614e559700 1 --2- 172.21.15.33:0/3048673485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fde10 0x7f61480fe1e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6138004940 tx=0x7f6138009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:56.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.612+0000 7f614e559700 1 -- 172.21.15.33:0/3048673485 shutdown_connections 2024-04-25T01:39:56.596 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.612+0000 7f614e559700 1 --2- 172.21.15.33:0/3048673485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61480fe7b0 0x7f6148106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:56.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.612+0000 7f614e559700 1 --2- 172.21.15.33:0/3048673485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fde10 0x7f61480fe1e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:56.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.612+0000 7f614e559700 1 -- 172.21.15.33:0/3048673485 >> 172.21.15.33:0/3048673485 conn(0x7f61480f7820 msgr2=0x7f61480f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:56.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.612+0000 7f614e559700 1 -- 172.21.15.33:0/3048673485 shutdown_connections 2024-04-25T01:39:56.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.612+0000 7f614e559700 1 -- 172.21.15.33:0/3048673485 wait complete. 2024-04-25T01:39:56.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.612+0000 7f614e559700 1 Processor -- start 2024-04-25T01:39:56.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f614e559700 1 -- start start 2024-04-25T01:39:56.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f614e559700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61480fde10 0x7f6148113c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f614e559700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fe7b0 0x7f6148114150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f614e559700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61481147a0 con 0x7f61480fe7b0 2024-04-25T01:39:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f614e559700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614810ddc0 con 0x7f61480fde10 2024-04-25T01:39:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f61477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fe7b0 0x7f6148114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f61477fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fe7b0 0x7f6148114150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48542/0 (socket says 172.21.15.33:48542) 2024-04-25T01:39:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f61477fe700 1 -- 172.21.15.33:0/4033958947 learned_addr learned my addr 172.21.15.33:0/4033958947 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:39:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f6147fff700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61480fde10 0x7f6148113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f61477fe700 1 -- 172.21.15.33:0/4033958947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61480fde10 msgr2=0x7f6148113c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:56.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f61477fe700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61480fde10 0x7f6148113c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:56.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f61477fe700 1 -- 172.21.15.33:0/4033958947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6138005040 con 0x7f61480fe7b0 2024-04-25T01:39:56.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.613+0000 7f61477fe700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fe7b0 0x7f6148114150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f613c002a50 tx=0x7f613c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:56.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.614+0000 7f6144ff9700 1 -- 172.21.15.33:0/4033958947 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f613c007cc0 con 0x7f61480fe7b0 2024-04-25T01:39:56.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.614+0000 7f6144ff9700 1 -- 172.21.15.33:0/4033958947 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f613c00f040 con 0x7f61480fe7b0 2024-04-25T01:39:56.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.614+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614810e0a0 con 0x7f61480fe7b0 2024-04-25T01:39:56.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.614+0000 7f6144ff9700 1 -- 172.21.15.33:0/4033958947 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f613c00da30 con 0x7f61480fe7b0 2024-04-25T01:39:56.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.614+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614810e5e0 con 0x7f61480fe7b0 2024-04-25T01:39:56.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.615+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6148066b30 con 0x7f61480fe7b0 2024-04-25T01:39:56.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.615+0000 7f6144ff9700 1 -- 172.21.15.33:0/4033958947 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f613c00db90 con 0x7f61480fe7b0 2024-04-25T01:39:56.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.616+0000 7f6144ff9700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f613006da40 0x7f613006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:56.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.616+0000 7f6144ff9700 1 -- 172.21.15.33:0/4033958947 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f613c08cf00 con 0x7f61480fe7b0 2024-04-25T01:39:56.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.616+0000 7f6147fff700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f613006da40 0x7f613006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:56.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.616+0000 7f6147fff700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f613006da40 0x7f613006fef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6138004f90 tx=0x7f6138009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:56.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.620+0000 7f6144ff9700 1 -- 172.21.15.33:0/4033958947 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f613c056b30 con 0x7f61480fe7b0 2024-04-25T01:39:56.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.917+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6148109ea0 con 0x7f61480fe7b0 2024-04-25T01:39:56.901 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:56 smithi033 ceph-mon[26353]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:56.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.918+0000 7f6144ff9700 1 -- 172.21.15.33:0/4033958947 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f613c05a1f0 con 0x7f61480fe7b0 2024-04-25T01:39:56.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f613006da40 msgr2=0x7f613006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f613006da40 0x7f613006fef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6138004f90 tx=0x7f6138009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fe7b0 msgr2=0x7f6148114150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fe7b0 0x7f6148114150 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f613c002a50 tx=0x7f613c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 shutdown_connections 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f613006da40 0x7f613006fef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f61480fde10 0x7f6148113c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 --2- 172.21.15.33:0/4033958947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f61480fe7b0 0x7f6148114150 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 >> 172.21.15.33:0/4033958947 conn(0x7f61480f7820 msgr2=0x7f61480f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 shutdown_connections 2024-04-25T01:39:56.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:56.920+0000 7f614e559700 1 -- 172.21.15.33:0/4033958947 wait complete. 2024-04-25T01:39:56.914 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)" 2024-04-25T01:39:57.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:56 smithi138 ceph-mon[32977]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:39:57.545 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-25T01:39:57.816 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:39:57.938 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:57 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/4033958947' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:58.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:57 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/4033958947' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:39:58.881 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:39:58 smithi033 ceph-mon[26353]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:59.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:39:58 smithi138 ceph-mon[32977]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:39:59.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.412+0000 7f0ff356d700 1 -- 172.21.15.33:0/3973328053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 msgr2=0x7f0fec0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.412+0000 7f0ff356d700 1 --2- 172.21.15.33:0/3973328053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 0x7f0fec0fb990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc004940 tx=0x7f0fdc009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.412+0000 7f0ff356d700 1 -- 172.21.15.33:0/3973328053 shutdown_connections 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.412+0000 7f0ff356d700 1 --2- 172.21.15.33:0/3973328053 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 0x7f0fec0fb990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.412+0000 7f0ff356d700 1 --2- 172.21.15.33:0/3973328053 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fec102f60 0x7f0fec103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.412+0000 7f0ff356d700 1 -- 172.21.15.33:0/3973328053 >> 172.21.15.33:0/3973328053 conn(0x7f0fec0714d0 msgr2=0x7f0fec0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.412+0000 7f0ff356d700 1 -- 172.21.15.33:0/3973328053 shutdown_connections 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.412+0000 7f0ff356d700 1 -- 172.21.15.33:0/3973328053 wait complete. 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff356d700 1 Processor -- start 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff356d700 1 -- start start 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff356d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 0x7f0fec113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff356d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fec102f60 0x7f0fec113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:59.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff356d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec114510 con 0x7f0fec0fb520 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff356d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec10db10 con 0x7f0fec102f60 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff1309700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 0x7f0fec113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff1309700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 0x7f0fec113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48566/0 (socket says 172.21.15.33:48566) 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff1309700 1 -- 172.21.15.33:0/1398458043 learned_addr learned my addr 172.21.15.33:0/1398458043 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff0b08700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fec102f60 0x7f0fec113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff1309700 1 -- 172.21.15.33:0/1398458043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fec102f60 msgr2=0x7f0fec113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff1309700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fec102f60 0x7f0fec113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.413+0000 7f0ff1309700 1 -- 172.21.15.33:0/1398458043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fdc005040 con 0x7f0fec0fb520 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.414+0000 7f0ff1309700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 0x7f0fec113980 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c830 tx=0x7f0fe8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.414+0000 7f0fe1ffb700 1 -- 172.21.15.33:0/1398458043 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe80106f0 con 0x7f0fec0fb520 2024-04-25T01:39:59.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.414+0000 7f0fe1ffb700 1 -- 172.21.15.33:0/1398458043 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fe8010d20 con 0x7f0fec0fb520 2024-04-25T01:39:59.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.414+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec10ddf0 con 0x7f0fec0fb520 2024-04-25T01:39:59.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.414+0000 7f0fe1ffb700 1 -- 172.21.15.33:0/1398458043 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe80113a0 con 0x7f0fec0fb520 2024-04-25T01:39:59.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.414+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec10e300 con 0x7f0fec0fb520 2024-04-25T01:39:59.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.415+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec100250 con 0x7f0fec0fb520 2024-04-25T01:39:59.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.415+0000 7f0fe1ffb700 1 -- 172.21.15.33:0/1398458043 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f0fe8012400 con 0x7f0fec0fb520 2024-04-25T01:39:59.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.416+0000 7f0fe1ffb700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f0fd806da40 0x7f0fd806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:39:59.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.416+0000 7f0fe1ffb700 1 -- 172.21.15.33:0/1398458043 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f0fe808d150 con 0x7f0fec0fb520 2024-04-25T01:39:59.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.416+0000 7f0ff0b08700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f0fd806da40 0x7f0fd806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:39:59.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.416+0000 7f0ff0b08700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f0fd806da40 0x7f0fd806fef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0fec10edf0 tx=0x7f0fdc009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:39:59.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.420+0000 7f0fe1ffb700 1 -- 172.21.15.33:0/1398458043 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f0fe805a890 con 0x7f0fec0fb520 2024-04-25T01:39:59.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.716+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0fec10efd0 con 0x7f0fec0fb520 2024-04-25T01:39:59.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.717+0000 7f0fe1ffb700 1 -- 172.21.15.33:0/1398458043 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f0fe805a420 con 0x7f0fec0fb520 2024-04-25T01:39:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f0fd806da40 msgr2=0x7f0fd806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f0fd806da40 0x7f0fd806fef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0fec10edf0 tx=0x7f0fdc009e30 comp rx=0 tx=0).stop 2024-04-25T01:39:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 msgr2=0x7f0fec113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:39:59.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 0x7f0fec113980 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800c830 tx=0x7f0fe8004470 comp rx=0 tx=0).stop 2024-04-25T01:39:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 shutdown_connections 2024-04-25T01:39:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f0fd806da40 0x7f0fd806fef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fec0fb520 0x7f0fec113980 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 --2- 172.21.15.33:0/1398458043 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fec102f60 0x7f0fec113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:39:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 >> 172.21.15.33:0/1398458043 conn(0x7f0fec0714d0 msgr2=0x7f0fec0fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:39:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 shutdown_connections 2024-04-25T01:39:59.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:39:59.719+0000 7f0ff356d700 1 -- 172.21.15.33:0/1398458043 wait complete. 2024-04-25T01:39:59.713 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:40:00.323 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-25T01:40:00.591 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:40:00.999 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:00 smithi033 ceph-mon[26353]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:01.000 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:00 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1398458043' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:01.000 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:00 smithi033 ceph-mon[26353]: overall HEALTH_OK 2024-04-25T01:40:01.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:00 smithi138 ceph-mon[32977]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:01.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:00 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1398458043' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:01.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:00 smithi138 ceph-mon[32977]: overall HEALTH_OK 2024-04-25T01:40:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.476+0000 7f1e2974b700 1 -- 172.21.15.33:0/2393147710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e240fd0a0 msgr2=0x7f1e24109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.476+0000 7f1e2974b700 1 --2- 172.21.15.33:0/2393147710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e240fd0a0 0x7f1e24109350 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c004940 tx=0x7f1e0c009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.476+0000 7f1e2974b700 1 -- 172.21.15.33:0/2393147710 shutdown_connections 2024-04-25T01:40:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.476+0000 7f1e2974b700 1 --2- 172.21.15.33:0/2393147710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e240fd0a0 0x7f1e24109350 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.476+0000 7f1e2974b700 1 --2- 172.21.15.33:0/2393147710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e240fc790 0x7f1e240fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.476+0000 7f1e2974b700 1 -- 172.21.15.33:0/2393147710 >> 172.21.15.33:0/2393147710 conn(0x7f1e24071d00 msgr2=0x7f1e24072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:02.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.476+0000 7f1e2974b700 1 -- 172.21.15.33:0/2393147710 shutdown_connections 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.476+0000 7f1e2974b700 1 -- 172.21.15.33:0/2393147710 wait complete. 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.477+0000 7f1e2974b700 1 Processor -- start 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.477+0000 7f1e2974b700 1 -- start start 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.477+0000 7f1e2974b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e240fc790 0x7f1e24115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.477+0000 7f1e2974b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e240fd0a0 0x7f1e241162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.477+0000 7f1e2974b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e24116930 con 0x7f1e240fd0a0 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.477+0000 7f1e2974b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2410fd00 con 0x7f1e240fc790 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e240fc790 0x7f1e24115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e227fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e240fd0a0 0x7f1e241162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e22ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e240fc790 0x7f1e24115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:56740/0 (socket says 172.21.15.33:56740) 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e22ffd700 1 -- 172.21.15.33:0/446762457 learned_addr learned my addr 172.21.15.33:0/446762457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e227fc700 1 -- 172.21.15.33:0/446762457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e240fc790 msgr2=0x7f1e24115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e227fc700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e240fc790 0x7f1e24115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:02.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e227fc700 1 -- 172.21.15.33:0/446762457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c005040 con 0x7f1e240fd0a0 2024-04-25T01:40:02.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e22ffd700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e240fc790 0x7f1e24115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:40:02.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e227fc700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e240fd0a0 0x7f1e241162e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000c00 tx=0x7f1e0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:02.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e1b7fe700 1 -- 172.21.15.33:0/446762457 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e0c021070 con 0x7f1e240fd0a0 2024-04-25T01:40:02.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2410ff80 con 0x7f1e240fd0a0 2024-04-25T01:40:02.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e1b7fe700 1 -- 172.21.15.33:0/446762457 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e0c013c00 con 0x7f1e240fd0a0 2024-04-25T01:40:02.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.478+0000 7f1e1b7fe700 1 -- 172.21.15.33:0/446762457 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e0c00e070 con 0x7f1e240fd0a0 2024-04-25T01:40:02.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.479+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e24110460 con 0x7f1e240fd0a0 2024-04-25T01:40:02.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.480+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e240fc330 con 0x7f1e240fd0a0 2024-04-25T01:40:02.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.480+0000 7f1e1b7fe700 1 -- 172.21.15.33:0/446762457 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f1e0c01c9d0 con 0x7f1e240fd0a0 2024-04-25T01:40:02.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.480+0000 7f1e1b7fe700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f1e1006da40 0x7f1e1006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:02.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.481+0000 7f1e1b7fe700 1 -- 172.21.15.33:0/446762457 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f1e0c08e480 con 0x7f1e240fd0a0 2024-04-25T01:40:02.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.481+0000 7f1e22ffd700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f1e1006da40 0x7f1e1006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:02.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.481+0000 7f1e22ffd700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f1e1006da40 0x7f1e1006fef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c004fd0 tx=0x7f1e1c00b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.485+0000 7f1e1b7fe700 1 -- 172.21.15.33:0/446762457 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f1e0c05bc60 con 0x7f1e240fd0a0 2024-04-25T01:40:02.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:02.713+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e2404bed0 con 0x7f1e1006da40 2024-04-25T01:40:02.749 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:02 smithi033 ceph-mon[26353]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:03.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:02 smithi138 ceph-mon[32977]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:03.944 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:03 smithi033 ceph-mon[26353]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:04.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:03 smithi138 ceph-mon[32977]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:04.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.833+0000 7f1e1b7fe700 1 -- 172.21.15.33:0/446762457 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5092 (secure 0 0 0) 0x7f1e2404bed0 con 0x7f1e1006da40 2024-04-25T01:40:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f1e1006da40 msgr2=0x7f1e1006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f1e1006da40 0x7f1e1006fef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c004fd0 tx=0x7f1e1c00b040 comp rx=0 tx=0).stop 2024-04-25T01:40:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e240fd0a0 msgr2=0x7f1e241162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:04.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e240fd0a0 0x7f1e241162e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c000c00 tx=0x7f1e0c009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 shutdown_connections 2024-04-25T01:40:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f1e1006da40 0x7f1e1006fef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e240fc790 0x7f1e24115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 --2- 172.21.15.33:0/446762457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e240fd0a0 0x7f1e241162e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 >> 172.21.15.33:0/446762457 conn(0x7f1e24071d00 msgr2=0x7f1e240f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 shutdown_connections 2024-04-25T01:40:04.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:04.836+0000 7f1e2974b700 1 -- 172.21.15.33:0/446762457 wait complete. 2024-04-25T01:40:04.832 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:40:05.000 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:04 smithi033 ceph-mon[26353]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:05.000 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:04 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:40:05.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:04 smithi138 ceph-mon[32977]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:05.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:04 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:40:05.529 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch ps' 2024-04-25T01:40:05.749 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:05 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:40:05.801 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:40:06.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:05 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:40:07.000 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:06 smithi033 ceph-mon[26353]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:07.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:06 smithi138 ceph-mon[32977]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/1567340524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 msgr2=0x7f8cbc0fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 --2- 172.21.15.33:0/1567340524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 0x7f8cbc0fe230 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0004940 tx=0x7f8cb0009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/1567340524 shutdown_connections 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 --2- 172.21.15.33:0/1567340524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cbc0fe800 0x7f8cbc106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 --2- 172.21.15.33:0/1567340524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 0x7f8cbc0fe230 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/1567340524 >> 172.21.15.33:0/1567340524 conn(0x7f8cbc0f7870 msgr2=0x7f8cbc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/1567340524 shutdown_connections 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/1567340524 wait complete. 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 Processor -- start 2024-04-25T01:40:07.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.626+0000 7f8cc2ddf700 1 -- start start 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc2ddf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 0x7f8cbc0fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc2ddf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cbc0fe800 0x7f8cbc0fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc2ddf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0fdc70 con 0x7f8cbc0fde60 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc2ddf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0fc710 con 0x7f8cbc0fe800 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc0b7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 0x7f8cbc0fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc0b7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 0x7f8cbc0fbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43790/0 (socket says 172.21.15.33:43790) 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc0b7b700 1 -- 172.21.15.33:0/315087143 learned_addr learned my addr 172.21.15.33:0/315087143 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cbbfff700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cbc0fe800 0x7f8cbc0fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc0b7b700 1 -- 172.21.15.33:0/315087143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cbc0fe800 msgr2=0x7f8cbc0fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:07.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc0b7b700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cbc0fe800 0x7f8cbc0fc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc0b7b700 1 -- 172.21.15.33:0/315087143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb0005040 con 0x7f8cbc0fde60 2024-04-25T01:40:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc0b7b700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 0x7f8cbc0fbc90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009180 tx=0x7f8cb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cb97fa700 1 -- 172.21.15.33:0/315087143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cb000e040 con 0x7f8cbc0fde60 2024-04-25T01:40:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc0fc9f0 con 0x7f8cbc0fde60 2024-04-25T01:40:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cb97fa700 1 -- 172.21.15.33:0/315087143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cb0013420 con 0x7f8cbc0fde60 2024-04-25T01:40:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.627+0000 7f8cb97fa700 1 -- 172.21.15.33:0/315087143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cb001c630 con 0x7f8cbc0fde60 2024-04-25T01:40:07.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.628+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc11a110 con 0x7f8cbc0fde60 2024-04-25T01:40:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.628+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc066ad0 con 0x7f8cbc0fde60 2024-04-25T01:40:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.629+0000 7f8cb97fa700 1 -- 172.21.15.33:0/315087143 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f8cb0013590 con 0x7f8cbc0fde60 2024-04-25T01:40:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.629+0000 7f8cb97fa700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8cac06da40 0x7f8cac06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.630+0000 7f8cb97fa700 1 -- 172.21.15.33:0/315087143 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f8cb008e330 con 0x7f8cbc0fde60 2024-04-25T01:40:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.630+0000 7f8cbbfff700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8cac06da40 0x7f8cac06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:07.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.630+0000 7f8cbbfff700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8cac06da40 0x7f8cac06fef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0fd3c0 tx=0x7f8ca80091a0 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:07.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.634+0000 7f8cb97fa700 1 -- 172.21.15.33:0/315087143 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8cb0057fe0 con 0x7f8cbc0fde60 2024-04-25T01:40:07.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.853+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cbc0fd1e0 con 0x7f8cac06da40 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.864+0000 7f8cb97fa700 1 -- 172.21.15.33:0/315087143 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8cbc0fd1e0 con 0x7f8cac06da40 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (75s) 53s ago 12m 14.9M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (12m) 53s ago 12m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (11m) 109s ago 11m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (59s) 53s ago 12m 52.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (7m) 53s ago 7m 61.9M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (7m) 53s ago 7m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (7m) 109s ago 7m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:40:07.848 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (6m) 109s ago 6m 60.9M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (2m) 53s ago 13m 505M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (2m) 109s ago 11m 448M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (13m) 53s ago 14m 56.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (11m) 109s ago 11m 47.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (117s) 53s ago 12m 15.9M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (113s) 109s ago 11m 3653k - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (10m) 53s ago 10m 223M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (10m) 53s ago 10m 236M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (9m) 53s ago 9m 188M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (9m) 109s ago 9m 262M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (9m) 109s ago 9m 229M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (8m) 109s ago 8m 217M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:40:07.849 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (95s) 53s ago 11m 48.4M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.866+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8cac06da40 msgr2=0x7f8cac06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.866+0000 7f8cc2ddf700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8cac06da40 0x7f8cac06fef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc0fd3c0 tx=0x7f8ca80091a0 comp rx=0 tx=0).stop 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.866+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 msgr2=0x7f8cbc0fbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.866+0000 7f8cc2ddf700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 0x7f8cbc0fbc90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009180 tx=0x7f8cb0009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.867+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 shutdown_connections 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.867+0000 7f8cc2ddf700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8cac06da40 0x7f8cac06fef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.867+0000 7f8cc2ddf700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8cbc0fde60 0x7f8cbc0fbc90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.867+0000 7f8cc2ddf700 1 --2- 172.21.15.33:0/315087143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8cbc0fe800 0x7f8cbc0fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.867+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 >> 172.21.15.33:0/315087143 conn(0x7f8cbc0f7870 msgr2=0x7f8cbc0f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.867+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 shutdown_connections 2024-04-25T01:40:07.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:07.867+0000 7f8cc2ddf700 1 -- 172.21.15.33:0/315087143 wait complete. 2024-04-25T01:40:08.522 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-25T01:40:08.523 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:40:08.523 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-25T01:40:08.792 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:40:08.915 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:08 smithi033 ceph-mon[26353]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:08.915 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:08 smithi033 ceph-mon[26353]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:09.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:08 smithi138 ceph-mon[32977]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:09.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:08 smithi138 ceph-mon[32977]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.560+0000 7f7313688700 1 -- 172.21.15.33:0/1954820958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c101d10 msgr2=0x7f730c1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.560+0000 7f7313688700 1 --2- 172.21.15.33:0/1954820958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c101d10 0x7f730c1020e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f7308004940 tx=0x7f7308009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.560+0000 7f7313688700 1 -- 172.21.15.33:0/1954820958 shutdown_connections 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.560+0000 7f7313688700 1 --2- 172.21.15.33:0/1954820958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c1026b0 0x7f730c0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.560+0000 7f7313688700 1 --2- 172.21.15.33:0/1954820958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c101d10 0x7f730c1020e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.560+0000 7f7313688700 1 -- 172.21.15.33:0/1954820958 >> 172.21.15.33:0/1954820958 conn(0x7f730c0f7820 msgr2=0x7f730c0f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.560+0000 7f7313688700 1 -- 172.21.15.33:0/1954820958 shutdown_connections 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.561+0000 7f7313688700 1 -- 172.21.15.33:0/1954820958 wait complete. 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.561+0000 7f7313688700 1 Processor -- start 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.561+0000 7f7313688700 1 -- start start 2024-04-25T01:40:10.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.561+0000 7f7313688700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c101d10 0x7f730c115e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.561+0000 7f7313688700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c1026b0 0x7f730c1163b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.561+0000 7f7313688700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c116a00 con 0x7f730c1026b0 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.561+0000 7f7313688700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c10ffe0 con 0x7f730c101d10 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7310c23700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c1026b0 0x7f730c1163b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7310c23700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c1026b0 0x7f730c1163b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43814/0 (socket says 172.21.15.33:43814) 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7310c23700 1 -- 172.21.15.33:0/2774512103 learned_addr learned my addr 172.21.15.33:0/2774512103 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7311424700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c101d10 0x7f730c115e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7310c23700 1 -- 172.21.15.33:0/2774512103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c101d10 msgr2=0x7f730c115e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7310c23700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c101d10 0x7f730c115e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7310c23700 1 -- 172.21.15.33:0/2774512103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7308005040 con 0x7f730c1026b0 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7310c23700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c1026b0 0x7f730c1163b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f72fc002a50 tx=0x7f72fc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7301ffb700 1 -- 172.21.15.33:0/2774512103 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72fc007cc0 con 0x7f730c1026b0 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7301ffb700 1 -- 172.21.15.33:0/2774512103 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72fc00f040 con 0x7f730c1026b0 2024-04-25T01:40:10.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.562+0000 7f7301ffb700 1 -- 172.21.15.33:0/2774512103 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72fc00d9c0 con 0x7f730c1026b0 2024-04-25T01:40:10.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.563+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730c1102c0 con 0x7f730c1026b0 2024-04-25T01:40:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.564+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730c110880 con 0x7f730c1026b0 2024-04-25T01:40:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.565+0000 7f7301ffb700 1 -- 172.21.15.33:0/2774512103 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f72fc009730 con 0x7f730c1026b0 2024-04-25T01:40:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.565+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730c0fd460 con 0x7f730c1026b0 2024-04-25T01:40:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.565+0000 7f7301ffb700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f72f806da40 0x7f72f806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.565+0000 7f7301ffb700 1 -- 172.21.15.33:0/2774512103 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f72fc08c670 con 0x7f730c1026b0 2024-04-25T01:40:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.565+0000 7f7311424700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f72f806da40 0x7f72f806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:10.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.566+0000 7f7311424700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f72f806da40 0x7f72f806fef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7308004f90 tx=0x7f7308009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:10.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.571+0000 7f7301ffb700 1 -- 172.21.15.33:0/2774512103 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f72fc059dc0 con 0x7f730c1026b0 2024-04-25T01:40:10.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.774+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f730c04c710 con 0x7f730c1026b0 2024-04-25T01:40:10.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.779+0000 7f7301ffb700 1 -- 172.21.15.33:0/2774512103 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v52)=0 v52) v1 ==== 125+0+0 (secure 0 0 0) 0x7f72fc059950 con 0x7f730c1026b0 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f72f806da40 msgr2=0x7f72f806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f72f806da40 0x7f72f806fef0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7308004f90 tx=0x7f7308009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c1026b0 msgr2=0x7f730c1163b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c1026b0 0x7f730c1163b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f72fc002a50 tx=0x7f72fc004600 comp rx=0 tx=0).stop 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 shutdown_connections 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f72f806da40 0x7f72f806fef0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f730c101d10 0x7f730c115e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 --2- 172.21.15.33:0/2774512103 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f730c1026b0 0x7f730c1163b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 >> 172.21.15.33:0/2774512103 conn(0x7f730c0f7820 msgr2=0x7f730c0f87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 shutdown_connections 2024-04-25T01:40:10.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:10.781+0000 7f7313688700 1 -- 172.21.15.33:0/2774512103 wait complete. 2024-04-25T01:40:10.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:10 smithi033 ceph-mon[26353]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:11.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:10 smithi138 ceph-mon[32977]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:11.464 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-25T01:40:11.464 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:40:11.464 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-25T01:40:11.730 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:40:12.025 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:11 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2774512103' entity='client.admin' 2024-04-25T01:40:12.026 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:11 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:12.026 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:11 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:12.026 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:11 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:12.026 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:11 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:12.064 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:11 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2774512103' entity='client.admin' 2024-04-25T01:40:12.064 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:11 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:12.064 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:11 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:12.064 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:11 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:12.064 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:11 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:13.053 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:12 smithi033 ceph-mon[26353]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:13.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:12 smithi138 ceph-mon[32977]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 -- 172.21.15.33:0/3034721314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 msgr2=0x7f60380fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:13.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 --2- 172.21.15.33:0/3034721314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 0x7f60380fe1d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f6034004940 tx=0x7f6034009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 -- 172.21.15.33:0/3034721314 shutdown_connections 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 --2- 172.21.15.33:0/3034721314 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60380fe7a0 0x7f6038106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 --2- 172.21.15.33:0/3034721314 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 0x7f60380fe1d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 -- 172.21.15.33:0/3034721314 >> 172.21.15.33:0/3034721314 conn(0x7f60380f7870 msgr2=0x7f60380f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 -- 172.21.15.33:0/3034721314 shutdown_connections 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 -- 172.21.15.33:0/3034721314 wait complete. 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 Processor -- start 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.338+0000 7f603fc7f700 1 -- start start 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603fc7f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 0x7f6038113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603fc7f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60380fe7a0 0x7f6038113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603fc7f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6038114630 con 0x7f60380fde00 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603fc7f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603810dba0 con 0x7f60380fe7a0 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603da1b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 0x7f6038113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:13.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603da1b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 0x7f6038113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43838/0 (socket says 172.21.15.33:43838) 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603da1b700 1 -- 172.21.15.33:0/2035529628 learned_addr learned my addr 172.21.15.33:0/2035529628 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603d21a700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60380fe7a0 0x7f6038113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603da1b700 1 -- 172.21.15.33:0/2035529628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60380fe7a0 msgr2=0x7f6038113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603da1b700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60380fe7a0 0x7f6038113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603da1b700 1 -- 172.21.15.33:0/2035529628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034005040 con 0x7f60380fde00 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603d21a700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60380fe7a0 0x7f6038113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.339+0000 7f603da1b700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 0x7f6038113a10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6034004890 tx=0x7f6034009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.340+0000 7f602e7fc700 1 -- 172.21.15.33:0/2035529628 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f603400e040 con 0x7f60380fde00 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.340+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603810de20 con 0x7f60380fde00 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.340+0000 7f602e7fc700 1 -- 172.21.15.33:0/2035529628 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6034009af0 con 0x7f60380fde00 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.340+0000 7f602e7fc700 1 -- 172.21.15.33:0/2035529628 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f603401c630 con 0x7f60380fde00 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.340+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603810e360 con 0x7f60380fde00 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.341+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60381017f0 con 0x7f60380fde00 2024-04-25T01:40:13.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.341+0000 7f602e7fc700 1 -- 172.21.15.33:0/2035529628 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f603400bbc0 con 0x7f60380fde00 2024-04-25T01:40:13.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.341+0000 7f602e7fc700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f602406da40 0x7f602406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:13.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.342+0000 7f602e7fc700 1 -- 172.21.15.33:0/2035529628 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f603408e450 con 0x7f60380fde00 2024-04-25T01:40:13.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.342+0000 7f603d21a700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f602406da40 0x7f602406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:13.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.342+0000 7f603d21a700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f602406da40 0x7f602406fef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6028006fd0 tx=0x7f6028008040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:13.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.346+0000 7f602e7fc700 1 -- 172.21.15.33:0/2035529628 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f603405bc30 con 0x7f60380fde00 2024-04-25T01:40:13.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.552+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f6038109f10 con 0x7f60380fde00 2024-04-25T01:40:13.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.552+0000 7f602e7fc700 1 -- 172.21.15.33:0/2035529628 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v52)=0 v52) v1 ==== 155+0+0 (secure 0 0 0) 0x7f603405b7c0 con 0x7f60380fde00 2024-04-25T01:40:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.554+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f602406da40 msgr2=0x7f602406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.554+0000 7f603fc7f700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f602406da40 0x7f602406fef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6028006fd0 tx=0x7f6028008040 comp rx=0 tx=0).stop 2024-04-25T01:40:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.554+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 msgr2=0x7f6038113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.554+0000 7f603fc7f700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 0x7f6038113a10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f6034004890 tx=0x7f6034009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.554+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 shutdown_connections 2024-04-25T01:40:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.554+0000 7f603fc7f700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f602406da40 0x7f602406fef0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.554+0000 7f603fc7f700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f60380fde00 0x7f6038113a10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.554+0000 7f603fc7f700 1 --2- 172.21.15.33:0/2035529628 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f60380fe7a0 0x7f6038113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.554+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 >> 172.21.15.33:0/2035529628 conn(0x7f60380f7870 msgr2=0x7f60380ff210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.555+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 shutdown_connections 2024-04-25T01:40:13.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:13.555+0000 7f603fc7f700 1 -- 172.21.15.33:0/2035529628 wait complete. 2024-04-25T01:40:14.216 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-25T01:40:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:40:14.937 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:14 smithi033 ceph-mon[26353]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:15.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:14 smithi138 ceph-mon[32977]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.059+0000 7f9d7314e700 1 -- 172.21.15.33:0/2152671644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 msgr2=0x7f9d6c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:16.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.059+0000 7f9d7314e700 1 --2- 172.21.15.33:0/2152671644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 0x7f9d6c0fcdf0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9d58004940 tx=0x7f9d58009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.059+0000 7f9d7314e700 1 -- 172.21.15.33:0/2152671644 shutdown_connections 2024-04-25T01:40:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.059+0000 7f9d7314e700 1 --2- 172.21.15.33:0/2152671644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 0x7f9d6c0fcdf0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.059+0000 7f9d7314e700 1 --2- 172.21.15.33:0/2152671644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d6c102de0 0x7f9d6c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.059+0000 7f9d7314e700 1 -- 172.21.15.33:0/2152671644 >> 172.21.15.33:0/2152671644 conn(0x7f9d6c0f8990 msgr2=0x7f9d6c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.059+0000 7f9d7314e700 1 -- 172.21.15.33:0/2152671644 shutdown_connections 2024-04-25T01:40:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.059+0000 7f9d7314e700 1 -- 172.21.15.33:0/2152671644 wait complete. 2024-04-25T01:40:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d7314e700 1 Processor -- start 2024-04-25T01:40:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d7314e700 1 -- start start 2024-04-25T01:40:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d7314e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d6c102de0 0x7f9d6c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d7314e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 0x7f9d6c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d7314e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c1145b0 con 0x7f9d6c103780 2024-04-25T01:40:16.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d7314e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c10db20 con 0x7f9d6c102de0 2024-04-25T01:40:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 0x7f9d6c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 0x7f9d6c113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53298/0 (socket says 172.21.15.33:53298) 2024-04-25T01:40:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d6bfff700 1 -- 172.21.15.33:0/3507960118 learned_addr learned my addr 172.21.15.33:0/3507960118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.060+0000 7f9d70eea700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d6c102de0 0x7f9d6c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.061+0000 7f9d6bfff700 1 -- 172.21.15.33:0/3507960118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d6c102de0 msgr2=0x7f9d6c113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:16.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.061+0000 7f9d6bfff700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d6c102de0 0x7f9d6c113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.061+0000 7f9d6bfff700 1 -- 172.21.15.33:0/3507960118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d58005040 con 0x7f9d6c103780 2024-04-25T01:40:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.061+0000 7f9d6bfff700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 0x7f9d6c113ed0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009100 tx=0x7f9d58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:16.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.061+0000 7f9d697fa700 1 -- 172.21.15.33:0/3507960118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5800e040 con 0x7f9d6c103780 2024-04-25T01:40:16.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.061+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c10dda0 con 0x7f9d6c103780 2024-04-25T01:40:16.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.061+0000 7f9d697fa700 1 -- 172.21.15.33:0/3507960118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d58013420 con 0x7f9d6c103780 2024-04-25T01:40:16.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.061+0000 7f9d697fa700 1 -- 172.21.15.33:0/3507960118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d5801c610 con 0x7f9d6c103780 2024-04-25T01:40:16.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.061+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c10e360 con 0x7f9d6c103780 2024-04-25T01:40:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.062+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c1000f0 con 0x7f9d6c103780 2024-04-25T01:40:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.063+0000 7f9d697fa700 1 -- 172.21.15.33:0/3507960118 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f9d58013590 con 0x7f9d6c103780 2024-04-25T01:40:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.063+0000 7f9d697fa700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f9d5c06da40 0x7f9d5c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.063+0000 7f9d697fa700 1 -- 172.21.15.33:0/3507960118 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f9d5808e2d0 con 0x7f9d6c103780 2024-04-25T01:40:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.063+0000 7f9d70eea700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f9d5c06da40 0x7f9d5c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:16.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.064+0000 7f9d70eea700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f9d5c06da40 0x7f9d5c06fef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d60006fd0 tx=0x7f9d60008040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:16.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.068+0000 7f9d697fa700 1 -- 172.21.15.33:0/3507960118 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f9d5805bb50 con 0x7f9d6c103780 2024-04-25T01:40:16.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.273+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f9d6c10efe0 con 0x7f9d6c103780 2024-04-25T01:40:16.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.274+0000 7f9d697fa700 1 -- 172.21.15.33:0/3507960118 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v52)=0 v52) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9d5805b6e0 con 0x7f9d6c103780 2024-04-25T01:40:16.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f9d5c06da40 msgr2=0x7f9d5c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:16.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f9d5c06da40 0x7f9d5c06fef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d60006fd0 tx=0x7f9d60008040 comp rx=0 tx=0).stop 2024-04-25T01:40:16.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 msgr2=0x7f9d6c113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:16.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 0x7f9d6c113ed0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009100 tx=0x7f9d58009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 shutdown_connections 2024-04-25T01:40:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f9d5c06da40 0x7f9d5c06fef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d6c102de0 0x7f9d6c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 --2- 172.21.15.33:0/3507960118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d6c103780 0x7f9d6c113ed0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 >> 172.21.15.33:0/3507960118 conn(0x7f9d6c0f8990 msgr2=0x7f9d6c0fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 shutdown_connections 2024-04-25T01:40:16.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:16.276+0000 7f9d7314e700 1 -- 172.21.15.33:0/3507960118 wait complete. 2024-04-25T01:40:16.865 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:16 smithi033 ceph-mon[26353]: pgmap v90: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:16.893 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-25T01:40:17.161 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:40:17.275 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:16 smithi138 ceph-mon[32977]: pgmap v90: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 -- 172.21.15.33:0/1540729540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fbf40 msgr2=0x7f9700101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 --2- 172.21.15.33:0/1540729540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fbf40 0x7f9700101a30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f96f0004940 tx=0x7f96f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 -- 172.21.15.33:0/1540729540 shutdown_connections 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 --2- 172.21.15.33:0/1540729540 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fbf40 0x7f9700101a30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 --2- 172.21.15.33:0/1540729540 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97000fb5a0 0x7f97000fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 -- 172.21.15.33:0/1540729540 >> 172.21.15.33:0/1540729540 conn(0x7f9700073350 msgr2=0x7f9700073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 -- 172.21.15.33:0/1540729540 shutdown_connections 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 -- 172.21.15.33:0/1540729540 wait complete. 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 Processor -- start 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.875+0000 7f97078fd700 1 -- start start 2024-04-25T01:40:18.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f97078fd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fb5a0 0x7f97000fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f97078fd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97000fbf40 0x7f97000fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f97078fd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000fe480 con 0x7f97000fb5a0 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f97078fd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000fcfb0 con 0x7f97000fbf40 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f9705699700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fb5a0 0x7f97000fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f9705699700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fb5a0 0x7f97000fc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53314/0 (socket says 172.21.15.33:53314) 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f9705699700 1 -- 172.21.15.33:0/2064346166 learned_addr learned my addr 172.21.15.33:0/2064346166 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f9704e98700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97000fbf40 0x7f97000fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f9705699700 1 -- 172.21.15.33:0/2064346166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97000fbf40 msgr2=0x7f97000fca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f9705699700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97000fbf40 0x7f97000fca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f9705699700 1 -- 172.21.15.33:0/2064346166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0005040 con 0x7f97000fb5a0 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f9704e98700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97000fbf40 0x7f97000fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f9705699700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fb5a0 0x7f97000fc530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96fc002a50 tx=0x7f96fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f96f5ffb700 1 -- 172.21.15.33:0/2064346166 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96fc007cc0 con 0x7f97000fb5a0 2024-04-25T01:40:18.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f96f5ffb700 1 -- 172.21.15.33:0/2064346166 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96fc00f040 con 0x7f97000fb5a0 2024-04-25T01:40:18.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97000fd290 con 0x7f97000fb5a0 2024-04-25T01:40:18.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.876+0000 7f96f5ffb700 1 -- 172.21.15.33:0/2064346166 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96fc009640 con 0x7f97000fb5a0 2024-04-25T01:40:18.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.877+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970011a150 con 0x7f97000fb5a0 2024-04-25T01:40:18.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.878+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97000fe670 con 0x7f97000fb5a0 2024-04-25T01:40:18.864 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:18 smithi033 ceph-mon[26353]: pgmap v91: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.878+0000 7f96f5ffb700 1 -- 172.21.15.33:0/2064346166 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f96fc0097a0 con 0x7f97000fb5a0 2024-04-25T01:40:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.878+0000 7f96f5ffb700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f96ec06da40 0x7f96ec06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.879+0000 7f96f5ffb700 1 -- 172.21.15.33:0/2064346166 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f96fc08ce40 con 0x7f97000fb5a0 2024-04-25T01:40:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.879+0000 7f9704e98700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f96ec06da40 0x7f96ec06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:18.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.879+0000 7f9704e98700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f96ec06da40 0x7f96ec06fef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97000fdc60 tx=0x7f96f0009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:18.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:18.883+0000 7f96f5ffb700 1 -- 172.21.15.33:0/2064346166 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f96fc056af0 con 0x7f97000fb5a0 2024-04-25T01:40:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.090+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f970011a3a0 con 0x7f97000fb5a0 2024-04-25T01:40:19.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.090+0000 7f96f5ffb700 1 -- 172.21.15.33:0/2064346166 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v52)=0 v52) v1 ==== 135+0+0 (secure 0 0 0) 0x7f96fc05a1b0 con 0x7f97000fb5a0 2024-04-25T01:40:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.092+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f96ec06da40 msgr2=0x7f96ec06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.092+0000 7f97078fd700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f96ec06da40 0x7f96ec06fef0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f97000fdc60 tx=0x7f96f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.092+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fb5a0 msgr2=0x7f97000fc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:19.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.092+0000 7f97078fd700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fb5a0 0x7f97000fc530 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96fc002a50 tx=0x7f96fc0046b0 comp rx=0 tx=0).stop 2024-04-25T01:40:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.093+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 shutdown_connections 2024-04-25T01:40:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.093+0000 7f97078fd700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f96ec06da40 0x7f96ec06fef0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.093+0000 7f97078fd700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f97000fb5a0 0x7f97000fc530 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.093+0000 7f97078fd700 1 --2- 172.21.15.33:0/2064346166 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f97000fbf40 0x7f97000fca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.093+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 >> 172.21.15.33:0/2064346166 conn(0x7f9700073350 msgr2=0x7f9700100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.093+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 shutdown_connections 2024-04-25T01:40:19.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:19.093+0000 7f97078fd700 1 -- 172.21.15.33:0/2064346166 wait complete. 2024-04-25T01:40:19.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:18 smithi138 ceph-mon[32977]: pgmap v91: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:19.707 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-25T01:40:19.973 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:40:21.112 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:20 smithi033 ceph-mon[26353]: pgmap v92: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:21.112 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:20 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:40:21.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:20 smithi138 ceph-mon[32977]: pgmap v92: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:21.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:20 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:40:21.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 -- 172.21.15.33:0/3880581199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 msgr2=0x7fdf801031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 --2- 172.21.15.33:0/3880581199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 0x7fdf801031f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fdf74004940 tx=0x7fdf74009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 -- 172.21.15.33:0/3880581199 shutdown_connections 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 --2- 172.21.15.33:0/3880581199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf801037c0 0x7fdf800fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 --2- 172.21.15.33:0/3880581199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 0x7fdf801031f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 -- 172.21.15.33:0/3880581199 >> 172.21.15.33:0/3880581199 conn(0x7fdf80072380 msgr2=0x7fdf80072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 -- 172.21.15.33:0/3880581199 shutdown_connections 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 -- 172.21.15.33:0/3880581199 wait complete. 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 Processor -- start 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.680+0000 7fdf84d96700 1 -- start start 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf84d96700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 0x7fdf80113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf84d96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf801037c0 0x7fdf80113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:21.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf84d96700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf80114500 con 0x7fdf80102e20 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf84d96700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8010db20 con 0x7fdf801037c0 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf7e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 0x7fdf80113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf7e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 0x7fdf80113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53322/0 (socket says 172.21.15.33:53322) 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf7e59c700 1 -- 172.21.15.33:0/4096469770 learned_addr learned my addr 172.21.15.33:0/4096469770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf7dd9b700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf801037c0 0x7fdf80113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf7e59c700 1 -- 172.21.15.33:0/4096469770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf801037c0 msgr2=0x7fdf80113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf7e59c700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf801037c0 0x7fdf80113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf7e59c700 1 -- 172.21.15.33:0/4096469770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf74005040 con 0x7fdf80102e20 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.681+0000 7fdf7e59c700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 0x7fdf80113970 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009180 tx=0x7fdf74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.682+0000 7fdf6effd700 1 -- 172.21.15.33:0/4096469770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf7400e040 con 0x7fdf80102e20 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.682+0000 7fdf6effd700 1 -- 172.21.15.33:0/4096469770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf74013420 con 0x7fdf80102e20 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.682+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8010dda0 con 0x7fdf80102e20 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.682+0000 7fdf6effd700 1 -- 172.21.15.33:0/4096469770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf7401c610 con 0x7fdf80102e20 2024-04-25T01:40:21.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.682+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8010e2e0 con 0x7fdf80102e20 2024-04-25T01:40:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.683+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf80100160 con 0x7fdf80102e20 2024-04-25T01:40:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.683+0000 7fdf6effd700 1 -- 172.21.15.33:0/4096469770 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7fdf74013a00 con 0x7fdf80102e20 2024-04-25T01:40:21.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.684+0000 7fdf6effd700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdf6806da40 0x7fdf6806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.684+0000 7fdf6effd700 1 -- 172.21.15.33:0/4096469770 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fdf7408e230 con 0x7fdf80102e20 2024-04-25T01:40:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.684+0000 7fdf7dd9b700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdf6806da40 0x7fdf6806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:21.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.684+0000 7fdf7dd9b700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdf6806da40 0x7fdf6806fef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf8010f210 tx=0x7fdf7000b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:21.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.688+0000 7fdf6effd700 1 -- 172.21.15.33:0/4096469770 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7fdf74057f90 con 0x7fdf80102e20 2024-04-25T01:40:21.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.916+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf80109fc0 con 0x7fdf6806da40 2024-04-25T01:40:21.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.921+0000 7fdf6effd700 1 -- 172.21.15.33:0/4096469770 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fdf80109fc0 con 0x7fdf6806da40 2024-04-25T01:40:21.907 INFO:teuthology.orchestra.run.smithi033.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:40:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdf6806da40 msgr2=0x7fdf6806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdf6806da40 0x7fdf6806fef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdf8010f210 tx=0x7fdf7000b040 comp rx=0 tx=0).stop 2024-04-25T01:40:21.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 msgr2=0x7fdf80113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 0x7fdf80113970 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fdf74009180 tx=0x7fdf74009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 shutdown_connections 2024-04-25T01:40:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7fdf6806da40 0x7fdf6806fef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf80102e20 0x7fdf80113970 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 --2- 172.21.15.33:0/4096469770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf801037c0 0x7fdf80113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 >> 172.21.15.33:0/4096469770 conn(0x7fdf80072380 msgr2=0x7fdf800face0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 shutdown_connections 2024-04-25T01:40:21.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:21.924+0000 7fdf84d96700 1 -- 172.21.15.33:0/4096469770 wait complete. 2024-04-25T01:40:22.595 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-25T01:40:22.595 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T01:40:22.596 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-25T01:40:22.870 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T01:40:23.113 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:22 smithi033 ceph-mon[26353]: pgmap v93: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:23.113 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:22 smithi033 ceph-mon[26353]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:40:23.113 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:22 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:23.113 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:22 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:23.113 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:22 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:23.113 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:22 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:23.113 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:22 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:23.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:22 smithi138 ceph-mon[32977]: pgmap v93: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:23.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:22 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:40:23.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:22 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:23.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:22 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:23.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:22 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:23.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:22 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:23.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:22 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:24.143 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:23 smithi033 ceph-mon[26353]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:24.143 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:23 smithi033 ceph-mon[26353]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:40:24.177 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:23 smithi138 ceph-mon[32977]: from='client.14836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:24.177 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:23 smithi138 ceph-mon[32977]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:40:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.559+0000 7f21e299f700 1 -- 172.21.15.33:0/252550467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 msgr2=0x7f21dc1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.559+0000 7f21e299f700 1 --2- 172.21.15.33:0/252550467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 0x7f21dc1020e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f21cc004940 tx=0x7f21cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.560+0000 7f21e299f700 1 -- 172.21.15.33:0/252550467 shutdown_connections 2024-04-25T01:40:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.560+0000 7f21e299f700 1 --2- 172.21.15.33:0/252550467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21dc1026b0 0x7f21dc0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.560+0000 7f21e299f700 1 --2- 172.21.15.33:0/252550467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 0x7f21dc1020e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.560+0000 7f21e299f700 1 -- 172.21.15.33:0/252550467 >> 172.21.15.33:0/252550467 conn(0x7f21dc0f7860 msgr2=0x7f21dc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.560+0000 7f21e299f700 1 -- 172.21.15.33:0/252550467 shutdown_connections 2024-04-25T01:40:24.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.560+0000 7f21e299f700 1 -- 172.21.15.33:0/252550467 wait complete. 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.560+0000 7f21e299f700 1 Processor -- start 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.560+0000 7f21e299f700 1 -- start start 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21e299f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 0x7f21dc113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21e299f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21dc1026b0 0x7f21dc113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21e299f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc1145e0 con 0x7f21dc101d10 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21e299f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc10dbc0 con 0x7f21dc1026b0 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21dbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 0x7f21dc113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21dbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 0x7f21dc113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53332/0 (socket says 172.21.15.33:53332) 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21dbfff700 1 -- 172.21.15.33:0/3171469281 learned_addr learned my addr 172.21.15.33:0/3171469281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21db7fe700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21dc1026b0 0x7f21dc113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21dbfff700 1 -- 172.21.15.33:0/3171469281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21dc1026b0 msgr2=0x7f21dc113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21dbfff700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21dc1026b0 0x7f21dc113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21dbfff700 1 -- 172.21.15.33:0/3171469281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc005040 con 0x7f21dc101d10 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21db7fe700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21dc1026b0 0x7f21dc113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21dbfff700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 0x7f21dc113a50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00b8b0 tx=0x7f21cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21d8ff9700 1 -- 172.21.15.33:0/3171469281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21cc00e040 con 0x7f21dc101d10 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21dc10de40 con 0x7f21dc101d10 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21d8ff9700 1 -- 172.21.15.33:0/3171469281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21cc004290 con 0x7f21dc101d10 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21d8ff9700 1 -- 172.21.15.33:0/3171469281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21cc01c3f0 con 0x7f21dc101d10 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.561+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc10e380 con 0x7f21dc101d10 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.562+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc0fd460 con 0x7f21dc101d10 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.563+0000 7f21d8ff9700 1 -- 172.21.15.33:0/3171469281 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f21cc0137b0 con 0x7f21dc101d10 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.563+0000 7f21d8ff9700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f21c406da40 0x7f21c406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.563+0000 7f21d8ff9700 1 -- 172.21.15.33:0/3171469281 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f21cc08e490 con 0x7f21dc101d10 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.563+0000 7f21db7fe700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f21c406da40 0x7f21c406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.564+0000 7f21db7fe700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f21c406da40 0x7f21c406fef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009fd0 tx=0x7f21d0009ee0 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:24.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.568+0000 7f21d8ff9700 1 -- 172.21.15.33:0/3171469281 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f21cc05bc70 con 0x7f21dc101d10 2024-04-25T01:40:24.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.809+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21dc10f0a0 con 0x7f21c406da40 2024-04-25T01:40:24.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.811+0000 7f21d8ff9700 1 -- 172.21.15.33:0/3171469281 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f21dc10f0a0 con 0x7f21c406da40 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f21c406da40 msgr2=0x7f21c406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f21c406da40 0x7f21c406fef0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f21d0009fd0 tx=0x7f21d0009ee0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 msgr2=0x7f21dc113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 0x7f21dc113a50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00b8b0 tx=0x7f21cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 shutdown_connections 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f21c406da40 0x7f21c406fef0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21dc101d10 0x7f21dc113a50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 --2- 172.21.15.33:0/3171469281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21dc1026b0 0x7f21dc113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 >> 172.21.15.33:0/3171469281 conn(0x7f21dc0f7860 msgr2=0x7f21dc0f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 shutdown_connections 2024-04-25T01:40:24.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.813+0000 7f21e299f700 1 -- 172.21.15.33:0/3171469281 wait complete. 2024-04-25T01:40:24.808 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:40:24.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.919+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2762483167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 msgr2=0x7f73ac065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:24.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.919+0000 7f73b3e9d700 1 --2- 172.21.15.33:0/2762483167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 0x7f73ac065ab0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f739c004940 tx=0x7f739c009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:24.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.920+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2762483167 shutdown_connections 2024-04-25T01:40:24.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.920+0000 7f73b3e9d700 1 --2- 172.21.15.33:0/2762483167 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 0x7f73ac065ab0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.920+0000 7f73b3e9d700 1 --2- 172.21.15.33:0/2762483167 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ac102e50 0x7f73ac103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.920+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2762483167 >> 172.21.15.33:0/2762483167 conn(0x7f73ac071d00 msgr2=0x7f73ac072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:24.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.920+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2762483167 shutdown_connections 2024-04-25T01:40:24.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.920+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2762483167 wait complete. 2024-04-25T01:40:24.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.920+0000 7f73b3e9d700 1 Processor -- start 2024-04-25T01:40:24.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.920+0000 7f73b3e9d700 1 -- start start 2024-04-25T01:40:24.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b3e9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 0x7f73ac113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:24.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b3e9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ac102e50 0x7f73ac1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b3e9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac114700 con 0x7f73ac065640 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b3e9d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73ac10daf0 con 0x7f73ac102e50 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b1c39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 0x7f73ac113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b1c39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 0x7f73ac113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53348/0 (socket says 172.21.15.33:53348) 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b1c39700 1 -- 172.21.15.33:0/2416586862 learned_addr learned my addr 172.21.15.33:0/2416586862 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b1438700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ac102e50 0x7f73ac1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b1c39700 1 -- 172.21.15.33:0/2416586862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ac102e50 msgr2=0x7f73ac1140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b1c39700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ac102e50 0x7f73ac1140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b1c39700 1 -- 172.21.15.33:0/2416586862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c005040 con 0x7f73ac065640 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.921+0000 7f73b1c39700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 0x7f73ac113b70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f73a8002a50 tx=0x7f73a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.922+0000 7f73a27fc700 1 -- 172.21.15.33:0/2416586862 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73a8007cc0 con 0x7f73ac065640 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.922+0000 7f73a27fc700 1 -- 172.21.15.33:0/2416586862 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73a800f040 con 0x7f73ac065640 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.922+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73ac10ddd0 con 0x7f73ac065640 2024-04-25T01:40:24.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.922+0000 7f73a27fc700 1 -- 172.21.15.33:0/2416586862 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73a800da30 con 0x7f73ac065640 2024-04-25T01:40:24.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.922+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73ac10e390 con 0x7f73ac065640 2024-04-25T01:40:24.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.923+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73ac100100 con 0x7f73ac065640 2024-04-25T01:40:24.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.923+0000 7f73a27fc700 1 -- 172.21.15.33:0/2416586862 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f73a800db90 con 0x7f73ac065640 2024-04-25T01:40:24.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.924+0000 7f73a27fc700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f739806da50 0x7f739806ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:24.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.924+0000 7f73a27fc700 1 -- 172.21.15.33:0/2416586862 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f73a808cfa0 con 0x7f73ac065640 2024-04-25T01:40:24.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.924+0000 7f73b1438700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f739806da50 0x7f739806ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:24.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.924+0000 7f73b1438700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f739806da50 0x7f739806ff00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73ac10ecb0 tx=0x7f739c009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:24.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:24.928+0000 7f73a27fc700 1 -- 172.21.15.33:0/2416586862 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f73a805a660 con 0x7f73ac065640 2024-04-25T01:40:25.000 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:24 smithi033 ceph-mon[26353]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:40:25.000 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:24 smithi033 ceph-mon[26353]: pgmap v94: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:25.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:24 smithi033 ceph-mon[26353]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:25.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.160+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73ac10f1c0 con 0x7f739806da50 2024-04-25T01:40:25.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.162+0000 7f73a27fc700 1 -- 172.21.15.33:0/2416586862 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f73ac10f1c0 con 0x7f739806da50 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.164+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f739806da50 msgr2=0x7f739806ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.164+0000 7f73b3e9d700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f739806da50 0x7f739806ff00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f73ac10ecb0 tx=0x7f739c009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.164+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 msgr2=0x7f73ac113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.164+0000 7f73b3e9d700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 0x7f73ac113b70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f73a8002a50 tx=0x7f73a80046b0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.165+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 shutdown_connections 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.165+0000 7f73b3e9d700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f739806da50 0x7f739806ff00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.165+0000 7f73b3e9d700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f73ac065640 0x7f73ac113b70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.165+0000 7f73b3e9d700 1 --2- 172.21.15.33:0/2416586862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f73ac102e50 0x7f73ac1140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.165+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 >> 172.21.15.33:0/2416586862 conn(0x7f73ac071d00 msgr2=0x7f73ac0fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.165+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 shutdown_connections 2024-04-25T01:40:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.165+0000 7f73b3e9d700 1 -- 172.21.15.33:0/2416586862 wait complete. 2024-04-25T01:40:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.276+0000 7f8f1d981700 1 -- 172.21.15.33:0/1031273941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 msgr2=0x7f8f181031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.276+0000 7f8f1d981700 1 --2- 172.21.15.33:0/1031273941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 0x7f8f181031b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8f08004940 tx=0x7f8f08009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.277+0000 7f8f1d981700 1 -- 172.21.15.33:0/1031273941 shutdown_connections 2024-04-25T01:40:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.277+0000 7f8f1d981700 1 --2- 172.21.15.33:0/1031273941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f18103780 0x7f8f180fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.277+0000 7f8f1d981700 1 --2- 172.21.15.33:0/1031273941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 0x7f8f181031b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.277+0000 7f8f1d981700 1 -- 172.21.15.33:0/1031273941 >> 172.21.15.33:0/1031273941 conn(0x7f8f180f8950 msgr2=0x7f8f180fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.277+0000 7f8f1d981700 1 -- 172.21.15.33:0/1031273941 shutdown_connections 2024-04-25T01:40:25.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.277+0000 7f8f1d981700 1 -- 172.21.15.33:0/1031273941 wait complete. 2024-04-25T01:40:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f1d981700 1 Processor -- start 2024-04-25T01:40:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f1d981700 1 -- start start 2024-04-25T01:40:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f1d981700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 0x7f8f18070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f1d981700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f18103780 0x7f8f180707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f1d981700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f18070ce0 con 0x7f8f18102de0 2024-04-25T01:40:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f1d981700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f18070e20 con 0x7f8f18103780 2024-04-25T01:40:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f16ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 0x7f8f18070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f16ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 0x7f8f18070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53362/0 (socket says 172.21.15.33:53362) 2024-04-25T01:40:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f16ffd700 1 -- 172.21.15.33:0/1065572623 learned_addr learned my addr 172.21.15.33:0/1065572623 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.278+0000 7f8f167fc700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f18103780 0x7f8f180707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.279+0000 7f8f16ffd700 1 -- 172.21.15.33:0/1065572623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f18103780 msgr2=0x7f8f180707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.279+0000 7f8f16ffd700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f18103780 0x7f8f180707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.279+0000 7f8f16ffd700 1 -- 172.21.15.33:0/1065572623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f08005040 con 0x7f8f18102de0 2024-04-25T01:40:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.279+0000 7f8f16ffd700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 0x7f8f18070260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800b3b0 tx=0x7f8f08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.279+0000 7f8eff7fe700 1 -- 172.21.15.33:0/1065572623 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f0800e040 con 0x7f8f18102de0 2024-04-25T01:40:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.279+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f18071070 con 0x7f8f18102de0 2024-04-25T01:40:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.279+0000 7f8eff7fe700 1 -- 172.21.15.33:0/1065572623 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f08013420 con 0x7f8f18102de0 2024-04-25T01:40:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.279+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1811a2a0 con 0x7f8f18102de0 2024-04-25T01:40:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.279+0000 7f8eff7fe700 1 -- 172.21.15.33:0/1065572623 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f0801c630 con 0x7f8f18102de0 2024-04-25T01:40:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.280+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f181000f0 con 0x7f8f18102de0 2024-04-25T01:40:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.282+0000 7f8eff7fe700 1 -- 172.21.15.33:0/1065572623 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f8f08013590 con 0x7f8f18102de0 2024-04-25T01:40:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.282+0000 7f8eff7fe700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8f0006da40 0x7f8f0006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.282+0000 7f8eff7fe700 1 -- 172.21.15.33:0/1065572623 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f8f0808e260 con 0x7f8f18102de0 2024-04-25T01:40:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.282+0000 7f8f167fc700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8f0006da40 0x7f8f0006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.283+0000 7f8f167fc700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8f0006da40 0x7f8f0006fef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f18071b50 tx=0x7f8f0c00b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:25.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.287+0000 7f8eff7fe700 1 -- 172.21.15.33:0/1065572623 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8f08057f30 con 0x7f8f18102de0 2024-04-25T01:40:25.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:24 smithi138 ceph-mon[32977]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T01:40:25.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:24 smithi138 ceph-mon[32977]: pgmap v94: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:25.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:24 smithi138 ceph-mon[32977]: from='client.14840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:25.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.512+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f1804bf40 con 0x7f8f0006da40 2024-04-25T01:40:25.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.522+0000 7f8eff7fe700 1 -- 172.21.15.33:0/1065572623 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f8f1804bf40 con 0x7f8f0006da40 2024-04-25T01:40:25.507 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:40:25.507 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (92s) 70s ago 12m 14.9M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:40:25.507 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (12m) 70s ago 12m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (12m) 2m ago 11m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (76s) 70s ago 12m 52.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (7m) 70s ago 7m 61.9M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (7m) 70s ago 7m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (7m) 2m ago 7m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (7m) 2m ago 7m 60.9M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (3m) 70s ago 14m 505M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (2m) 2m ago 11m 448M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (14m) 70s ago 14m 56.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af dd064971bee1 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (11m) 2m ago 11m 47.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 20de19b099cc 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (2m) 70s ago 12m 15.9M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (2m) 2m ago 11m 3653k - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (11m) 70s ago 11m 223M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (10m) 70s ago 10m 236M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:40:25.508 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (10m) 70s ago 10m 188M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:40:25.509 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (9m) 2m ago 9m 262M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:40:25.509 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (9m) 2m ago 9m 229M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:40:25.509 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (8m) 2m ago 8m 217M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:40:25.509 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (112s) 70s ago 12m 48.4M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8f0006da40 msgr2=0x7f8f0006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8f0006da40 0x7f8f0006fef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8f18071b50 tx=0x7f8f0c00b040 comp rx=0 tx=0).stop 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 msgr2=0x7f8f18070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 0x7f8f18070260 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8f0800b3b0 tx=0x7f8f08009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 shutdown_connections 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8f0006da40 0x7f8f0006fef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f18102de0 0x7f8f18070260 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 --2- 172.21.15.33:0/1065572623 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f18103780 0x7f8f180707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 >> 172.21.15.33:0/1065572623 conn(0x7f8f180f8950 msgr2=0x7f8f181073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 shutdown_connections 2024-04-25T01:40:25.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.525+0000 7f8f1d981700 1 -- 172.21.15.33:0/1065572623 wait complete. 2024-04-25T01:40:25.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.636+0000 7f8dacafe700 1 -- 172.21.15.33:0/3285561141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 msgr2=0x7f8da81020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.636+0000 7f8dacafe700 1 --2- 172.21.15.33:0/3285561141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 0x7f8da81020b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8d98004940 tx=0x7f8d98009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:25.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.636+0000 7f8dacafe700 1 -- 172.21.15.33:0/3285561141 shutdown_connections 2024-04-25T01:40:25.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.636+0000 7f8dacafe700 1 --2- 172.21.15.33:0/3285561141 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da8102680 0x7f8da80fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.636+0000 7f8dacafe700 1 --2- 172.21.15.33:0/3285561141 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 0x7f8da81020b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.636+0000 7f8dacafe700 1 -- 172.21.15.33:0/3285561141 >> 172.21.15.33:0/3285561141 conn(0x7f8da80f7850 msgr2=0x7f8da80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:25.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.636+0000 7f8dacafe700 1 -- 172.21.15.33:0/3285561141 shutdown_connections 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.636+0000 7f8dacafe700 1 -- 172.21.15.33:0/3285561141 wait complete. 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8dacafe700 1 Processor -- start 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8dacafe700 1 -- start start 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8dacafe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 0x7f8da810eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8dacafe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da8102680 0x7f8da810f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8dacafe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da810f720 con 0x7f8da8101ce0 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8dacafe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da810f860 con 0x7f8da8102680 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8da659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 0x7f8da810eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8da659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 0x7f8da810eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53390/0 (socket says 172.21.15.33:53390) 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8da659c700 1 -- 172.21.15.33:0/2057958255 learned_addr learned my addr 172.21.15.33:0/2057958255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:25.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8da5d9b700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da8102680 0x7f8da810f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:25.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8da659c700 1 -- 172.21.15.33:0/2057958255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da8102680 msgr2=0x7f8da810f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8da659c700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da8102680 0x7f8da810f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8da659c700 1 -- 172.21.15.33:0/2057958255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d98005040 con 0x7f8da8101ce0 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.637+0000 7f8da5d9b700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da8102680 0x7f8da810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.638+0000 7f8da659c700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 0x7f8da810eca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800b3b0 tx=0x7f8d98009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.638+0000 7f8d96ffd700 1 -- 172.21.15.33:0/2057958255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9800e040 con 0x7f8da8101ce0 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.638+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da8117b50 con 0x7f8da8101ce0 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.638+0000 7f8d96ffd700 1 -- 172.21.15.33:0/2057958255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d98013420 con 0x7f8da8101ce0 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.638+0000 7f8d96ffd700 1 -- 172.21.15.33:0/2057958255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d980255e0 con 0x7f8da8101ce0 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.638+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da81180b0 con 0x7f8da8101ce0 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.639+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da80fd430 con 0x7f8da8101ce0 2024-04-25T01:40:25.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.640+0000 7f8d96ffd700 1 -- 172.21.15.33:0/2057958255 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f8d98013590 con 0x7f8da8101ce0 2024-04-25T01:40:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.640+0000 7f8d96ffd700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8d9006da40 0x7f8d9006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.640+0000 7f8d96ffd700 1 -- 172.21.15.33:0/2057958255 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f8d9808ed50 con 0x7f8da8101ce0 2024-04-25T01:40:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.640+0000 7f8da5d9b700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8d9006da40 0x7f8d9006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.641+0000 7f8da5d9b700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8d9006da40 0x7f8d9006fef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8da80668d0 tx=0x7f8d9c00b040 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:25.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.644+0000 7f8d96ffd700 1 -- 172.21.15.33:0/2057958255 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f8d98058aa0 con 0x7f8da8101ce0 2024-04-25T01:40:25.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.941+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8da8066680 con 0x7f8da8101ce0 2024-04-25T01:40:25.926 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:25 smithi033 ceph-mon[26353]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:25.927 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:25 smithi033 ceph-mon[26353]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:25.927 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:25 smithi033 ceph-mon[26353]: pgmap v95: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:25.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.941+0000 7f8d96ffd700 1 -- 172.21.15.33:0/2057958255 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f8d9805c160 con 0x7f8da8101ce0 2024-04-25T01:40:25.927 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:40:25.927 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:40:25.927 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T01:40:25.927 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:40:25.928 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:40:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8d9006da40 msgr2=0x7f8d9006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8d9006da40 0x7f8d9006fef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8da80668d0 tx=0x7f8d9c00b040 comp rx=0 tx=0).stop 2024-04-25T01:40:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 msgr2=0x7f8da810eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 0x7f8da810eca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800b3b0 tx=0x7f8d98009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:25.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 shutdown_connections 2024-04-25T01:40:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f8d9006da40 0x7f8d9006fef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8da8101ce0 0x7f8da810eca0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 --2- 172.21.15.33:0/2057958255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8da8102680 0x7f8da810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 >> 172.21.15.33:0/2057958255 conn(0x7f8da80f7850 msgr2=0x7f8da80f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 shutdown_connections 2024-04-25T01:40:25.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:25.944+0000 7f8dacafe700 1 -- 172.21.15.33:0/2057958255 wait complete. 2024-04-25T01:40:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.057+0000 7f650edd9700 1 -- 172.21.15.33:0/1027438476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65081026a0 msgr2=0x7f65080fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.058+0000 7f650edd9700 1 --2- 172.21.15.33:0/1027438476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65081026a0 0x7f65080fbd10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f64f4004940 tx=0x7f64f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.058+0000 7f650edd9700 1 -- 172.21.15.33:0/1027438476 shutdown_connections 2024-04-25T01:40:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.058+0000 7f650edd9700 1 --2- 172.21.15.33:0/1027438476 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65081026a0 0x7f65080fbd10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.058+0000 7f650edd9700 1 --2- 172.21.15.33:0/1027438476 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6508101d00 0x7f65081020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.058+0000 7f650edd9700 1 -- 172.21.15.33:0/1027438476 >> 172.21.15.33:0/1027438476 conn(0x7f65080f7810 msgr2=0x7f65080f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.058+0000 7f650edd9700 1 -- 172.21.15.33:0/1027438476 shutdown_connections 2024-04-25T01:40:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.058+0000 7f650edd9700 1 -- 172.21.15.33:0/1027438476 wait complete. 2024-04-25T01:40:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.058+0000 7f650edd9700 1 Processor -- start 2024-04-25T01:40:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.058+0000 7f650edd9700 1 -- start start 2024-04-25T01:40:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650edd9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101d00 0x7f650810ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650edd9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65081026a0 0x7f650810f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650edd9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650810f550 con 0x7f6508101d00 2024-04-25T01:40:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650edd9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650810f690 con 0x7f65081026a0 2024-04-25T01:40:26.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650cb75700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101d00 0x7f650810ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650cb75700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101d00 0x7f650810ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55666/0 (socket says 172.21.15.33:55666) 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650cb75700 1 -- 172.21.15.33:0/820114070 learned_addr learned my addr 172.21.15.33:0/820114070 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f6507fff700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65081026a0 0x7f650810f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650cb75700 1 -- 172.21.15.33:0/820114070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65081026a0 msgr2=0x7f650810f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650cb75700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65081026a0 0x7f650810f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.059+0000 7f650cb75700 1 -- 172.21.15.33:0/820114070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f4005040 con 0x7f6508101d00 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.060+0000 7f650cb75700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101d00 0x7f650810ead0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f64fc002a80 tx=0x7f64fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.060+0000 7f65057fa700 1 -- 172.21.15.33:0/820114070 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64fc007cc0 con 0x7f6508101d00 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.060+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6508117b40 con 0x7f6508101d00 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.060+0000 7f65057fa700 1 -- 172.21.15.33:0/820114070 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64fc00f040 con 0x7f6508101d00 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.060+0000 7f65057fa700 1 -- 172.21.15.33:0/820114070 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64fc00dac0 con 0x7f6508101d00 2024-04-25T01:40:26.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.060+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6508117e90 con 0x7f6508101d00 2024-04-25T01:40:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.061+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f650810f8e0 con 0x7f6508101d00 2024-04-25T01:40:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.062+0000 7f65057fa700 1 -- 172.21.15.33:0/820114070 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f64fc00dc20 con 0x7f6508101d00 2024-04-25T01:40:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.062+0000 7f65057fa700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f64f806da40 0x7f64f806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:26.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.062+0000 7f65057fa700 1 -- 172.21.15.33:0/820114070 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f64fc08cdc0 con 0x7f6508101d00 2024-04-25T01:40:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.063+0000 7f6507fff700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f64f806da40 0x7f64f806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.063+0000 7f6507fff700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f64f806da40 0x7f64f806fef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64f4004890 tx=0x7f64f4009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.068+0000 7f65057fa700 1 -- 172.21.15.33:0/820114070 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f650810f8e0 con 0x7f6508101d00 2024-04-25T01:40:26.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:25 smithi138 ceph-mon[32977]: from='client.14844 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:26.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:25 smithi138 ceph-mon[32977]: from='client.14848 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:26.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:25 smithi138 ceph-mon[32977]: pgmap v95: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:40:26.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.385+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f65080fd450 con 0x7f6508101d00 2024-04-25T01:40:26.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.385+0000 7f65057fa700 1 -- 172.21.15.33:0/820114070 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f64fc0569f0 con 0x7f6508101d00 2024-04-25T01:40:26.370 INFO:teuthology.orchestra.run.smithi033.stdout:e19 2024-04-25T01:40:26.370 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:40:26.370 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:40:26.370 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 19 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:40:26.371 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:26.372 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:26.373 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:26.373 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:40:26.373 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:40:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.387+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f64f806da40 msgr2=0x7f64f806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.387+0000 7f650edd9700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f64f806da40 0x7f64f806fef0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f64f4004890 tx=0x7f64f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.387+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101d00 msgr2=0x7f650810ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.387+0000 7f650edd9700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101d00 0x7f650810ead0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f64fc002a80 tx=0x7f64fc0046b0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.387+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 shutdown_connections 2024-04-25T01:40:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.387+0000 7f650edd9700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f64f806da40 0x7f64f806fef0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.387+0000 7f650edd9700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6508101d00 0x7f650810ead0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.387+0000 7f650edd9700 1 --2- 172.21.15.33:0/820114070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65081026a0 0x7f650810f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.387+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 >> 172.21.15.33:0/820114070 conn(0x7f65080f7810 msgr2=0x7f65080f86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.388+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 shutdown_connections 2024-04-25T01:40:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.388+0000 7f650edd9700 1 -- 172.21.15.33:0/820114070 wait complete. 2024-04-25T01:40:26.375 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:40:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.496+0000 7f8337a24700 1 -- 172.21.15.33:0/1853314144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102670 msgr2=0x7f83300fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.496+0000 7f8337a24700 1 --2- 172.21.15.33:0/1853314144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102670 0x7f83300fb600 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8320004940 tx=0x7f8320009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.497+0000 7f8337a24700 1 -- 172.21.15.33:0/1853314144 shutdown_connections 2024-04-25T01:40:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.497+0000 7f8337a24700 1 --2- 172.21.15.33:0/1853314144 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102670 0x7f83300fb600 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.497+0000 7f8337a24700 1 --2- 172.21.15.33:0/1853314144 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330101cd0 0x7f83301020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.497+0000 7f8337a24700 1 -- 172.21.15.33:0/1853314144 >> 172.21.15.33:0/1853314144 conn(0x7f8330071460 msgr2=0x7f8330071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.497+0000 7f8337a24700 1 -- 172.21.15.33:0/1853314144 shutdown_connections 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.497+0000 7f8337a24700 1 -- 172.21.15.33:0/1853314144 wait complete. 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.497+0000 7f8337a24700 1 Processor -- start 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.497+0000 7f8337a24700 1 -- start start 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f8337a24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330101cd0 0x7f83300fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f8337a24700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330102670 0x7f83300fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f8337a24700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83300fd2b0 con 0x7f8330101cd0 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f8337a24700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83300fd3f0 con 0x7f8330102670 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f83357c0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330101cd0 0x7f83300fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:26.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f83357c0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330101cd0 0x7f83300fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55678/0 (socket says 172.21.15.33:55678) 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f83357c0700 1 -- 172.21.15.33:0/3519545839 learned_addr learned my addr 172.21.15.33:0/3519545839 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f8334fbf700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330102670 0x7f83300fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f83357c0700 1 -- 172.21.15.33:0/3519545839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330102670 msgr2=0x7f83300fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f83357c0700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330102670 0x7f83300fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f83357c0700 1 -- 172.21.15.33:0/3519545839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8320005040 con 0x7f8330101cd0 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f83357c0700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330101cd0 0x7f83300fc7e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f832c00c8c0 tx=0x7f832c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.498+0000 7f8325ffb700 1 -- 172.21.15.33:0/3519545839 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f832c00a980 con 0x7f8330101cd0 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.499+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8330119eb0 con 0x7f8330101cd0 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.499+0000 7f8325ffb700 1 -- 172.21.15.33:0/3519545839 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f832c00ce60 con 0x7f8330101cd0 2024-04-25T01:40:26.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.499+0000 7f8325ffb700 1 -- 172.21.15.33:0/3519545839 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f832c0115e0 con 0x7f8330101cd0 2024-04-25T01:40:26.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.499+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833011a2d0 con 0x7f8330101cd0 2024-04-25T01:40:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.500+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8330065200 con 0x7f8330101cd0 2024-04-25T01:40:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.500+0000 7f8325ffb700 1 -- 172.21.15.33:0/3519545839 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f832c00aae0 con 0x7f8330101cd0 2024-04-25T01:40:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.501+0000 7f8325ffb700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f831c06da50 0x7f831c06ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.501+0000 7f8325ffb700 1 -- 172.21.15.33:0/3519545839 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f832c08d1f0 con 0x7f8330101cd0 2024-04-25T01:40:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.501+0000 7f8334fbf700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f831c06da50 0x7f831c06ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:26.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.502+0000 7f8334fbf700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f831c06da50 0x7f831c06ff00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f832000b3b0 tx=0x7f8320009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:26.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.505+0000 7f8325ffb700 1 -- 172.21.15.33:0/3519545839 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f832c056ea0 con 0x7f8330101cd0 2024-04-25T01:40:26.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.754+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 --> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8330109f40 con 0x7f831c06da50 2024-04-25T01:40:26.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.755+0000 7f8325ffb700 1 -- 172.21.15.33:0/3519545839 <== mgr.14680 v2:172.21.15.33:6800/1731533099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f8330109f40 con 0x7f831c06da50 2024-04-25T01:40:26.740 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:40:26.740 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:40:26.740 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:40:26.740 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:40:26.740 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:40:26.740 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr" 2024-04-25T01:40:26.740 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:40:26.741 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "2/21 daemons upgraded", 2024-04-25T01:40:26.741 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mon daemons", 2024-04-25T01:40:26.741 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:40:26.741 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:40:26.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f831c06da50 msgr2=0x7f831c06ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f831c06da50 0x7f831c06ff00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f832000b3b0 tx=0x7f8320009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330101cd0 msgr2=0x7f83300fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330101cd0 0x7f83300fc7e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f832c00c8c0 tx=0x7f832c0044c0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 shutdown_connections 2024-04-25T01:40:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f831c06da50 0x7f831c06ff00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330101cd0 0x7f83300fc7e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 --2- 172.21.15.33:0/3519545839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330102670 0x7f83300fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 >> 172.21.15.33:0/3519545839 conn(0x7f8330071460 msgr2=0x7f83301009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:26.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.757+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 shutdown_connections 2024-04-25T01:40:26.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.758+0000 7f8337a24700 1 -- 172.21.15.33:0/3519545839 wait complete. 2024-04-25T01:40:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.865+0000 7f1456666700 1 -- 172.21.15.33:0/95029458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450103780 msgr2=0x7f14500fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.865+0000 7f1456666700 1 --2- 172.21.15.33:0/95029458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450103780 0x7f14500fcdf0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f1444004940 tx=0x7f1444009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.865+0000 7f1456666700 1 -- 172.21.15.33:0/95029458 shutdown_connections 2024-04-25T01:40:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.865+0000 7f1456666700 1 --2- 172.21.15.33:0/95029458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450103780 0x7f14500fcdf0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.865+0000 7f1456666700 1 --2- 172.21.15.33:0/95029458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1450102de0 0x7f14501031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.865+0000 7f1456666700 1 -- 172.21.15.33:0/95029458 >> 172.21.15.33:0/95029458 conn(0x7f14500f8990 msgr2=0x7f14500fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:26.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.865+0000 7f1456666700 1 -- 172.21.15.33:0/95029458 shutdown_connections 2024-04-25T01:40:26.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.865+0000 7f1456666700 1 -- 172.21.15.33:0/95029458 wait complete. 2024-04-25T01:40:26.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.865+0000 7f1456666700 1 Processor -- start 2024-04-25T01:40:26.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f1456666700 1 -- start start 2024-04-25T01:40:26.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f1456666700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450102de0 0x7f1450070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:26.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f1456666700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1450103780 0x7f14500707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:26.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f1456666700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14500721f0 con 0x7f1450102de0 2024-04-25T01:40:26.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f1456666700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1450070ce0 con 0x7f1450103780 2024-04-25T01:40:26.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f144ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450102de0 0x7f1450070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:26.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f144ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450102de0 0x7f1450070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55694/0 (socket says 172.21.15.33:55694) 2024-04-25T01:40:26.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f144ffff700 1 -- 172.21.15.33:0/1399375232 learned_addr learned my addr 172.21.15.33:0/1399375232 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:26.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f144f7fe700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1450103780 0x7f14500707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:26.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f144ffff700 1 -- 172.21.15.33:0/1399375232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1450103780 msgr2=0x7f14500707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:26.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f144ffff700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1450103780 0x7f14500707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:26.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.866+0000 7f144ffff700 1 -- 172.21.15.33:0/1399375232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444005040 con 0x7f1450102de0 2024-04-25T01:40:26.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.867+0000 7f144ffff700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450102de0 0x7f1450070260 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f144000b790 tx=0x7f1440009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:26.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.868+0000 7f144cff9700 1 -- 172.21.15.33:0/1399375232 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f144000f6f0 con 0x7f1450102de0 2024-04-25T01:40:26.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.868+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1450070fc0 con 0x7f1450102de0 2024-04-25T01:40:26.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.868+0000 7f144cff9700 1 -- 172.21.15.33:0/1399375232 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f144000fd20 con 0x7f1450102de0 2024-04-25T01:40:26.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.869+0000 7f144cff9700 1 -- 172.21.15.33:0/1399375232 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14400103a0 con 0x7f1450102de0 2024-04-25T01:40:26.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.869+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145011a1d0 con 0x7f1450102de0 2024-04-25T01:40:26.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.870+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14501000f0 con 0x7f1450102de0 2024-04-25T01:40:26.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.870+0000 7f144cff9700 1 -- 172.21.15.33:0/1399375232 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 34) v1 ==== 91925+0+0 (secure 0 0 0) 0x7f1440010500 con 0x7f1450102de0 2024-04-25T01:40:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.870+0000 7f144cff9700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f143806da40 0x7f143806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.871+0000 7f144cff9700 1 -- 172.21.15.33:0/1399375232 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f144008cf60 con 0x7f1450102de0 2024-04-25T01:40:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.871+0000 7f144f7fe700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f143806da40 0x7f143806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:26.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.871+0000 7f144f7fe700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f143806da40 0x7f143806fef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1450071940 tx=0x7f1444009e30 comp rx=0 tx=0).ready entity=mgr.14680 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:26.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:26.875+0000 7f144cff9700 1 -- 172.21.15.33:0/1399375232 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+184553 (secure 0 0 0) 0x7f14400c1100 con 0x7f1450102de0 2024-04-25T01:40:26.947 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/2057958255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (squid) 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: Upgrade: Setting container_image for all mgr 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/820114070' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:40:26.948 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:26 smithi033 ceph-mon[26353]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:27.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.170+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f14500fe530 con 0x7f1450102de0 2024-04-25T01:40:27.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.170+0000 7f144cff9700 1 -- 172.21.15.33:0/1399375232 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1440056c10 con 0x7f1450102de0 2024-04-25T01:40:27.155 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_OK 2024-04-25T01:40:27.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f143806da40 msgr2=0x7f143806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:27.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f143806da40 0x7f143806fef0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1450071940 tx=0x7f1444009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450102de0 msgr2=0x7f1450070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450102de0 0x7f1450070260 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f144000b790 tx=0x7f1440009250 comp rx=0 tx=0).stop 2024-04-25T01:40:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 shutdown_connections 2024-04-25T01:40:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:6800/1731533099,v1:172.21.15.33:6801/1731533099] conn(0x7f143806da40 0x7f143806fef0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1450102de0 0x7f1450070260 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 --2- 172.21.15.33:0/1399375232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1450103780 0x7f14500707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 >> 172.21.15.33:0/1399375232 conn(0x7f14500f8990 msgr2=0x7f1450107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 shutdown_connections 2024-04-25T01:40:27.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:27.172+0000 7f1456666700 1 -- 172.21.15.33:0/1399375232 wait complete. 2024-04-25T01:40:27.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/2057958255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:27.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:27.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: Upgrade: Target is version 19.0.0-2445-gd2c45d97 (squid) 2024-04-25T01:40:27.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, digests ['quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901'] 2024-04-25T01:40:27.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:27.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:27.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: Upgrade: Setting container_image for all mgr 2024-04-25T01:40:27.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:27.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-25T01:40:27.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/820114070' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:40:27.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:26 smithi138 ceph-mon[32977]: from='client.14860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:28.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:27 smithi033 ceph-mon[26353]: from='client.? 172.21.15.33:0/1399375232' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:40:28.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:27 smithi033 ceph-mon[26353]: pgmap v96: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:28.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:27 smithi033 ceph-mon[26353]: Upgrade: Updating mon.smithi033 2024-04-25T01:40:28.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:27 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:28.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:27 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:40:28.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:27 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:40:28.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:27 smithi033 ceph-mon[26353]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:28.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:27 smithi033 ceph-mon[26353]: Deploying daemon mon.smithi033 on smithi033 2024-04-25T01:40:28.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:27 smithi138 ceph-mon[32977]: from='client.? 172.21.15.33:0/1399375232' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:40:28.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:27 smithi138 ceph-mon[32977]: pgmap v96: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:28.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:27 smithi138 ceph-mon[32977]: Upgrade: Updating mon.smithi033 2024-04-25T01:40:28.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:27 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' 2024-04-25T01:40:28.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:27 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:40:28.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:27 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:40:28.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:27 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:28.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:27 smithi138 ceph-mon[32977]: Deploying daemon mon.smithi033 on smithi033 2024-04-25T01:40:29.889 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:29 smithi033 systemd[1]: Stopping Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:40:30.251 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:29 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[26349]: 2024-04-25T01:40:29.903+0000 7f5d2f298700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi033 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:40:30.251 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:29 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[26349]: 2024-04-25T01:40:29.903+0000 7f5d2f298700 -1 mon.smithi033@0(leader) e2 *** Got Signal Terminated *** 2024-04-25T01:40:30.251 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:30 smithi033 podman[111133]: 2024-04-25 01:40:30.067702404 +0000 UTC m=+0.289001140 container died dd064971bee1519abf6c530333899fc769b74fb8b61a585bffe3cc3b2858da22 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T01:40:30.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:30 smithi033 podman[111133]: 2024-04-25 01:40:30.445981893 +0000 UTC m=+0.667280638 container cleanup dd064971bee1519abf6c530333899fc769b74fb8b61a585bffe3cc3b2858da22 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, RELEASE=quincy-c61472c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T01:40:30.690 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:30 smithi033 bash[111133]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033 2024-04-25T01:40:31.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:30 smithi033 podman[111158]: 2024-04-25 01:40:30.705800213 +0000 UTC m=+0.630477725 container remove dd064971bee1519abf6c530333899fc769b74fb8b61a585bffe3cc3b2858da22 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, ceph=True, RELEASE=quincy-c61472c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:40:31.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:30 smithi033 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033.service: Deactivated successfully. 2024-04-25T01:40:31.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:30 smithi033 systemd[1]: Stopped Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:40:31.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:30 smithi033 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033.service: Consumed 14.268s CPU time. 2024-04-25T01:40:31.424 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 systemd[1]: Starting Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:40:31.424 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 podman[111288]: 2024-04-25 01:40:31.216923639 +0000 UTC m=+0.019089769 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:40:31.424 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 podman[111288]: 2024-04-25 01:40:31.323585875 +0000 UTC m=+0.125752047 container create f072ec3fc94f7ebd13ffe37e042436dffd1a5a39f2d8ec4e5030e2c7f1da2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T01:40:32.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 podman[111288]: 2024-04-25 01:40:31.551770497 +0000 UTC m=+0.353936633 container init f072ec3fc94f7ebd13ffe37e042436dffd1a5a39f2d8ec4e5030e2c7f1da2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:40:32.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 podman[111288]: 2024-04-25 01:40:31.555427718 +0000 UTC m=+0.357593848 container start f072ec3fc94f7ebd13ffe37e042436dffd1a5a39f2d8ec4e5030e2c7f1da2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: pidfile_write: ignore empty --pid-file 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: load: jerasure load: lrc 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: RocksDB version: 7.9.2 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Git sha 0 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: DB SUMMARY 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: DB Session ID: TEGO8MAO4S0L2A5I10JA 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: CURRENT file: CURRENT 2024-04-25T01:40:32.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1159 Bytes 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi033/store.db dir, Total Num: 1, files: 000033.sst 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi033/store.db: 000031.log size: 2052793 ; 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.error_if_exists: 0 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.create_if_missing: 0 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.paranoid_checks: 1 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.env: 0x55c43d39fc60 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T01:40:32.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.info_log: 0x55c43dd2f960 2024-04-25T01:40:32.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T01:40:32.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.statistics: (nil) 2024-04-25T01:40:32.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.use_fsync: 0 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_log_file_size: 0 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.allow_fallocate: 1 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.use_direct_reads: 0 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T01:40:32.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.db_log_dir: 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.wal_dir: 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.write_buffer_manager: 0x55c43dd021e0 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T01:40:32.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.unordered_write: 0 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.row_cache: None 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.wal_filter: None 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T01:40:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.two_write_queues: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.wal_compression: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.atomic_flush: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.log_readahead_size: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T01:40:32.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_background_jobs: 2 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_background_compactions: -1 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_subcompactions: 1 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_open_files: -1 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T01:40:32.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_background_flushes: -1 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Compression algorithms supported: 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: kZSTD supported: 0 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: kXpressCompression supported: 0 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: kZlibCompression supported: 1 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: kSnappyCompression supported: 1 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: kLZ4Compression supported: 1 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: kBZip2Compression supported: 0 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T01:40:32.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi033/store.db/MANIFEST-000015 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.merge_operator: 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_filter: None 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c43dd2fa60) 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: cache_index_and_filter_blocks: 1 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: pin_top_level_index_and_filter: 1 2024-04-25T01:40:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_type: 0 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_block_index_type: 0 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_shortening: 1 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: checksum: 4 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: no_block_cache: 0 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache: 0x55c43dd33090 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_name: BinnedLRUCache 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_options: 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: capacity : 536870912 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: num_shard_bits : 4 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: strict_capacity_limit : 0 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: high_pri_pool_ratio: 0.000 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_cache_compressed: (nil) 2024-04-25T01:40:32.013 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: persistent_cache: (nil) 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_size: 4096 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_size_deviation: 10 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_restart_interval: 16 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: index_block_restart_interval: 1 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: metadata_block_size: 4096 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: partition_filters: 0 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: use_delta_encoding: 1 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: filter_policy: bloomfilter 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: whole_key_filtering: 1 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: verify_compression: 0 2024-04-25T01:40:32.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: read_amp_bytes_per_bit: 0 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: format_version: 5 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: enable_index_compression: 1 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: block_align: 0 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: max_auto_readahead_size: 262144 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: prepopulate_block_cache: 0 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: initial_auto_readahead_size: 8192 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression: NoCompression 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.num_levels: 7 2024-04-25T01:40:32.015 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T01:40:32.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T01:40:32.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T01:40:32.018 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.table_properties_collectors: 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.inplace_update_support: 0 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T01:40:32.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.bloom_locality: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.max_successive_merges: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.ttl: 2592000 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.enable_blob_files: false 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.min_blob_size: 0 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T01:40:32.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 33.sst 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi033/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 35, last_sequence is 11995, log_number is 31,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 31 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 5ff92dd9-4b6c-435b-add9-8fb8380852f6 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714009231585171, "job": 1, "event": "recovery_started", "wal_files": [31]} 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #31 mode 2 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714009231592152, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 36, "file_size": 1840379, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11997, "largest_seqno": 13720, "table_properties": {"data_size": 1832437, "index_size": 4942, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1989, "raw_key_size": 18519, "raw_average_key_size": 23, "raw_value_size": 1815857, "raw_average_value_size": 2346, "num_data_blocks": 225, "num_entries": 774, "num_filter_entries": 774, "num_deletions": 7, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714009231, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "5ff92dd9-4b6c-435b-add9-8fb8380852f6", "db_session_id": "TEGO8MAO4S0L2A5I10JA", "orig_file_number": 36, "seqno_to_time_mapping": "N/A"}} 2024-04-25T01:40:32.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714009231592258, "job": 1, "event": "recovery_finished"} 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/version_set.cc:5047] Creating manifest 38 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi033/store.db/000031.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c43de24000 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: DB pointer 0x55c43de0e000 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** DB Stats ** 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T01:40:32.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: L0 1/0 1.76 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 528.8 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: L6 1/0 6.89 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Sum 2/0 8.64 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 528.8 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 528.8 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** Compaction Stats [default] ** 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T01:40:32.023 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 528.8 0.00 0.00 1 0.003 0 0 0.0 0.0 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Cumulative compaction: 0.00 GB write, 155.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Interval compaction: 0.00 GB write, 155.16 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Block cache BinnedLRUCache@0x55c43dd33090#2 capacity: 512.00 MB usage: 7.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,2.03 KB,0.00038743%) IndexBlock(1,5.08 KB,0.000968575%) Misc(1,0.00 KB,0%) 2024-04-25T01:40:32.024 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: starting mon.smithi033 rank 0 at public addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] at bind addrs [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi033 fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: mon.smithi033@-1(???) e2 preinit fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: mon.smithi033@-1(???).mds e19 new map 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: mon.smithi033@-1(???).mds e19 print_map 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: e19 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: legacy client fscid: 1 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: Filesystem 'cephfs' (1) 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: fs_name cephfs 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: epoch 19 2024-04-25T01:40:32.025 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: tableserver 0 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: root 0 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: session_timeout 60 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: session_autoclose 300 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: max_file_size 1099511627776 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: max_xattr_size 65536 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: required_client_features {} 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: last_failure 0 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: last_failure_osd_epoch 0 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: max_mds 2 2024-04-25T01:40:32.026 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: in 0,1 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: up {0=14446,1=24279} 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: failed 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: damaged 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: stopped 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: data_pools [3] 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: metadata_pool 2 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: inline_data disabled 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: balancer 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: bal_rank_mask -1 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: standby_count_wanted 1 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: qdb_cluster leader: 0 members: 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: [mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: [mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: [mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:32.027 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: [mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:32.028 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:40:32.028 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout: 2024-04-25T01:40:32.028 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: mon.smithi033@-1(???).osd e47 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-25T01:40:32.028 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: mon.smithi033@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T01:40:32.028 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: mon.smithi033@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T01:40:32.028 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: mon.smithi033@-1(???).osd e47 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T01:40:32.028 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 ceph-mon[111306]: mon.smithi033@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-04-25T01:40:32.028 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 bash[111288]: f072ec3fc94f7ebd13ffe37e042436dffd1a5a39f2d8ec4e5030e2c7f1da2863 2024-04-25T01:40:32.028 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:31 smithi033 systemd[1]: Started Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: pgmap v98: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: mon.smithi033 calling monitor election 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: mon.smithi033 is new leader, mons smithi033,smithi138 in quorum (ranks 0,1) 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: monmap epoch 2 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: last_changed 2024-04-25T01:28:36.545717+0000 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: created 2024-04-25T01:26:02.814821+0000 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: min_mon_release 17 (quincy) 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: election_strategy: 1 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.smithi033 2024-04-25T01:40:33.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-04-25T01:40:33.378 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:40:33.378 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: osdmap e47: 6 total, 6 up, 6 in 2024-04-25T01:40:33.378 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: mgrmap e34: smithi033.asemba(active, since 2m), standbys: smithi138.mmfhyi 2024-04-25T01:40:33.378 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: overall HEALTH_OK 2024-04-25T01:40:33.378 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: from='mgr.14680 ' entity='' 2024-04-25T01:40:33.378 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:33 smithi138 ceph-mon[32977]: mgrmap e35: smithi033.asemba(active, since 2m), standbys: smithi138.mmfhyi 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: from='mgr.14680 172.21.15.33:0/1250156547' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: pgmap v98: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: mon.smithi033 calling monitor election 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: mon.smithi033 is new leader, mons smithi033,smithi138 in quorum (ranks 0,1) 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: monmap epoch 2 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: last_changed 2024-04-25T01:28:36.545717+0000 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: created 2024-04-25T01:26:02.814821+0000 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: min_mon_release 17 (quincy) 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: election_strategy: 1 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.smithi033 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-04-25T01:40:33.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:40:33.394 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: osdmap e47: 6 total, 6 up, 6 in 2024-04-25T01:40:33.394 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: mgrmap e34: smithi033.asemba(active, since 2m), standbys: smithi138.mmfhyi 2024-04-25T01:40:33.394 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: overall HEALTH_OK 2024-04-25T01:40:33.394 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: from='mgr.14680 ' entity='' 2024-04-25T01:40:33.394 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:33 smithi033 ceph-mon[111306]: mgrmap e35: smithi033.asemba(active, since 2m), standbys: smithi138.mmfhyi 2024-04-25T01:40:40.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:39 smithi033 ceph-mon[111306]: Standby manager daemon smithi138.mmfhyi restarted 2024-04-25T01:40:40.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:39 smithi033 ceph-mon[111306]: Standby manager daemon smithi138.mmfhyi started 2024-04-25T01:40:40.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:39 smithi033 ceph-mon[111306]: from='mgr.? 172.21.15.138:0/2822463076' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/crt"}]: dispatch 2024-04-25T01:40:40.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:39 smithi033 ceph-mon[111306]: from='mgr.? 172.21.15.138:0/2822463076' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:40:40.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:39 smithi033 ceph-mon[111306]: from='mgr.? 172.21.15.138:0/2822463076' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/key"}]: dispatch 2024-04-25T01:40:40.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:39 smithi033 ceph-mon[111306]: from='mgr.? 172.21.15.138:0/2822463076' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:40:40.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:39 smithi138 ceph-mon[32977]: Standby manager daemon smithi138.mmfhyi restarted 2024-04-25T01:40:40.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:39 smithi138 ceph-mon[32977]: Standby manager daemon smithi138.mmfhyi started 2024-04-25T01:40:40.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:39 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/2822463076' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/crt"}]: dispatch 2024-04-25T01:40:40.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:39 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/2822463076' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T01:40:40.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:39 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/2822463076' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi138.mmfhyi/key"}]: dispatch 2024-04-25T01:40:40.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:39 smithi138 ceph-mon[32977]: from='mgr.? 172.21.15.138:0/2822463076' entity='mgr.smithi138.mmfhyi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T01:40:41.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: mgrmap e36: smithi033.asemba(active, since 3m), standbys: smithi138.mmfhyi 2024-04-25T01:40:41.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: Active manager daemon smithi033.asemba restarted 2024-04-25T01:40:41.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: Activating manager daemon smithi033.asemba 2024-04-25T01:40:41.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: osdmap e48: 6 total, 6 up, 6 in 2024-04-25T01:40:41.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: mgrmap e37: smithi033.asemba(active, starting, since 0.00598434s), standbys: smithi138.mmfhyi 2024-04-25T01:40:41.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:40:41.186 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:40:41.186 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:40:41.186 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:40:41.186 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:40:41.186 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:40:41.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:40:41.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi138.mmfhyi", "id": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:40:41.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:40:41.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:40:41.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:40:41.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:40:41.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:40:41.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:40:41.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:40:41.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:40:41.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:40:41.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: Manager daemon smithi033.asemba is now available 2024-04-25T01:40:41.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:41.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:40:41.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:40:41.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/trash_purge_schedule"}]: dispatch 2024-04-25T01:40:41.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: mgrmap e36: smithi033.asemba(active, since 3m), standbys: smithi138.mmfhyi 2024-04-25T01:40:41.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: Active manager daemon smithi033.asemba restarted 2024-04-25T01:40:41.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: Activating manager daemon smithi033.asemba 2024-04-25T01:40:41.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: osdmap e48: 6 total, 6 up, 6 in 2024-04-25T01:40:41.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: mgrmap e37: smithi033.asemba(active, starting, since 0.00598434s), standbys: smithi138.mmfhyi 2024-04-25T01:40:41.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:40:41.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:40:41.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi033.asemba", "id": "smithi033.asemba"}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr metadata", "who": "smithi138.mmfhyi", "id": "smithi138.mmfhyi"}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:40:41.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:40:41.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:40:41.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T01:40:41.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T01:40:41.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T01:40:41.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: Manager daemon smithi033.asemba is now available 2024-04-25T01:40:41.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:41.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:40:41.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/mirror_snapshot_schedule"}]: dispatch 2024-04-25T01:40:41.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:40 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi033.asemba/trash_purge_schedule"}]: dispatch 2024-04-25T01:40:42.220 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:41 smithi033 ceph-mon[111306]: [25/Apr/2024:01:40:40] ENGINE Bus STARTING 2024-04-25T01:40:42.220 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:41 smithi033 ceph-mon[111306]: [25/Apr/2024:01:40:40] ENGINE Serving on https://172.21.15.33:7150 2024-04-25T01:40:42.221 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:41 smithi033 ceph-mon[111306]: [25/Apr/2024:01:40:40] ENGINE Serving on http://172.21.15.33:8765 2024-04-25T01:40:42.221 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:41 smithi033 ceph-mon[111306]: [25/Apr/2024:01:40:40] ENGINE Bus STARTED 2024-04-25T01:40:42.221 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:41 smithi033 ceph-mon[111306]: mgrmap e38: smithi033.asemba(active, since 1.00963s), standbys: smithi138.mmfhyi 2024-04-25T01:40:42.264 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:41 smithi138 ceph-mon[32977]: [25/Apr/2024:01:40:40] ENGINE Bus STARTING 2024-04-25T01:40:42.264 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:41 smithi138 ceph-mon[32977]: [25/Apr/2024:01:40:40] ENGINE Serving on https://172.21.15.33:7150 2024-04-25T01:40:42.264 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:41 smithi138 ceph-mon[32977]: [25/Apr/2024:01:40:40] ENGINE Serving on http://172.21.15.33:8765 2024-04-25T01:40:42.264 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:41 smithi138 ceph-mon[32977]: [25/Apr/2024:01:40:40] ENGINE Bus STARTED 2024-04-25T01:40:42.265 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:41 smithi138 ceph-mon[32977]: mgrmap e38: smithi033.asemba(active, since 1.00963s), standbys: smithi138.mmfhyi 2024-04-25T01:40:43.026 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:42 smithi138 ceph-mon[32977]: pgmap v3: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:40:43.251 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:42 smithi033 ceph-mon[111306]: pgmap v3: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:40:44.194 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:43 smithi033 ceph-mon[111306]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:40:44.194 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:43 smithi033 ceph-mon[111306]: mgrmap e39: smithi033.asemba(active, since 2s), standbys: smithi138.mmfhyi 2024-04-25T01:40:44.194 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:43 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:44.194 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:43 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:44.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:43 smithi138 ceph-mon[32977]: pgmap v4: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:40:44.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:43 smithi138 ceph-mon[32977]: mgrmap e39: smithi033.asemba(active, since 2s), standbys: smithi138.mmfhyi 2024-04-25T01:40:44.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:43 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:44.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:43 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:45.751 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:45 smithi033 ceph-mon[111306]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:40:45.751 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:45 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:45.751 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:45 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:45.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:45 smithi033 ceph-mon[111306]: mgrmap e40: smithi033.asemba(active, since 4s), standbys: smithi138.mmfhyi 2024-04-25T01:40:45.776 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:45 smithi138 ceph-mon[32977]: pgmap v5: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:40:45.776 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:45 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:45.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:45 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:45.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:45 smithi138 ceph-mon[32977]: mgrmap e40: smithi033.asemba(active, since 4s), standbys: smithi138.mmfhyi 2024-04-25T01:40:47.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:46 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:47.001 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:46 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:47.026 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:46 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:47.026 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:46 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:48.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:47 smithi033 ceph-mon[111306]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:40:48.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:47 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:48.255 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:47 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:48.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:47 smithi138 ceph-mon[32977]: pgmap v6: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T01:40:48.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:47 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:48.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:47 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:49.958 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:49 smithi033 ceph-mon[111306]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-25T01:40:50.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:49 smithi138 ceph-mon[32977]: pgmap v7: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-25T01:40:51.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-04-25T01:40:51.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: Detected new or changed devices on smithi033 2024-04-25T01:40:51.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:51.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:51.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:51.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:51.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-25T01:40:51.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T01:40:51.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:40:51.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:40:51.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:40:51.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:40:51.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:40:51.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:40:51.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:51.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:51 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:51.751 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: pgmap v8: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: Detected new or changed devices on smithi033 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: Updating smithi033:/etc/ceph/ceph.conf 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: Updating smithi138:/etc/ceph/ceph.conf 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: Updating smithi138:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: Updating smithi033:/etc/ceph/ceph.client.admin.keyring 2024-04-25T01:40:51.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: Updating smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:40:51.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: Updating smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T01:40:51.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:51.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:51 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:52.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:52 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:52.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:52 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:52.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:52 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:52.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:52 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:52.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:52 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:52.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:52 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-25T01:40:52.751 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:52.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:52.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:52.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:52.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:52.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-25T01:40:54.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:54 smithi138 ceph-mon[32977]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T01:40:54.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:54 smithi138 ceph-mon[32977]: Upgrade: Updating mon.smithi138 2024-04-25T01:40:54.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:54 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:54.225 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:54 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:40:54.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:54 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:40:54.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:54 smithi138 ceph-mon[32977]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:54.226 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:54 smithi138 ceph-mon[32977]: Deploying daemon mon.smithi138 on smithi138 2024-04-25T01:40:54.502 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:54 smithi033 ceph-mon[111306]: pgmap v9: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T01:40:54.502 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:54 smithi033 ceph-mon[111306]: Upgrade: Updating mon.smithi138 2024-04-25T01:40:54.502 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:54 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:54.502 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:54 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:40:54.502 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:54 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:40:54.502 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:54 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:40:54.502 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:54 smithi033 ceph-mon[111306]: Deploying daemon mon.smithi138 on smithi138 2024-04-25T01:40:55.131 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:54 smithi138 systemd[1]: Stopping Ceph mon.smithi138 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:40:55.415 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:55 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138[32971]: 2024-04-25T01:40:55.145+0000 7f110b25a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi138 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:40:55.415 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:55 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138[32971]: 2024-04-25T01:40:55.145+0000 7f110b25a700 -1 mon.smithi138@1(peon) e2 *** Got Signal Terminated *** 2024-04-25T01:40:55.415 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:55 smithi138 podman[81133]: 2024-04-25 01:40:55.186606116 +0000 UTC m=+0.158685392 container died 20de19b099ccdf0f47b8c3b19439fe6c3eb9998f99797b08877f22a79489f90f (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=quincy-c61472c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T01:40:55.706 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:55 smithi138 podman[81133]: 2024-04-25 01:40:55.528752693 +0000 UTC m=+0.500831967 container cleanup 20de19b099ccdf0f47b8c3b19439fe6c3eb9998f99797b08877f22a79489f90f (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:40:55.707 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:55 smithi138 bash[81133]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138 2024-04-25T01:40:55.961 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:55 smithi138 podman[81147]: 2024-04-25 01:40:55.721866406 +0000 UTC m=+0.532874674 container remove 20de19b099ccdf0f47b8c3b19439fe6c3eb9998f99797b08877f22a79489f90f (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-25T01:40:55.961 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:55 smithi138 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi138.service: Deactivated successfully. 2024-04-25T01:40:55.961 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:55 smithi138 systemd[1]: Stopped Ceph mon.smithi138 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:40:55.961 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:55 smithi138 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi138.service: Consumed 7.222s CPU time. 2024-04-25T01:40:56.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 systemd[1]: Starting Ceph mon.smithi138 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:40:56.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 podman[81284]: 2024-04-25 01:40:56.287595581 +0000 UTC m=+0.018342368 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:40:56.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 podman[81284]: 2024-04-25 01:40:56.422769039 +0000 UTC m=+0.153515815 container create e2b197ed2121ac88e0a817c814a82aaa1d0129236c0654364be06717b6bd54eb (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T01:40:56.635 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 podman[81284]: 2024-04-25 01:40:56.650548724 +0000 UTC m=+0.381295496 container init e2b197ed2121ac88e0a817c814a82aaa1d0129236c0654364be06717b6bd54eb (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T01:40:57.027 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 podman[81284]: 2024-04-25 01:40:56.654096354 +0000 UTC m=+0.384843130 container start e2b197ed2121ac88e0a817c814a82aaa1d0129236c0654364be06717b6bd54eb (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T01:40:57.027 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T01:40:57.027 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: pidfile_write: ignore empty --pid-file 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: load: jerasure load: lrc 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: RocksDB version: 7.9.2 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Git sha 0 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: DB SUMMARY 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: DB Session ID: SUVL4VB8JMR0EUF2Z6X8 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: CURRENT file: CURRENT 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi138/store.db dir, Total Num: 1, files: 000027.sst 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi138/store.db: 000025.log size: 5953961 ; 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.error_if_exists: 0 2024-04-25T01:40:57.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.create_if_missing: 0 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.paranoid_checks: 1 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.env: 0x562d821f3c60 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.info_log: 0x562d846a1960 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.statistics: (nil) 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.use_fsync: 0 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_log_file_size: 0 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T01:40:57.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T01:40:57.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.allow_fallocate: 1 2024-04-25T01:40:57.031 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T01:40:57.031 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T01:40:57.031 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.use_direct_reads: 0 2024-04-25T01:40:57.031 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T01:40:57.031 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T01:40:57.031 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.db_log_dir: 2024-04-25T01:40:57.031 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.wal_dir: 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.write_buffer_manager: 0x562d846741e0 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T01:40:57.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.unordered_write: 0 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.row_cache: None 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.wal_filter: None 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.two_write_queues: 0 2024-04-25T01:40:57.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.wal_compression: 0 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.atomic_flush: 0 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.log_readahead_size: 0 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T01:40:57.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_background_jobs: 2 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_background_compactions: -1 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_subcompactions: 1 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_open_files: -1 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T01:40:57.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_background_flushes: -1 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Compression algorithms supported: 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: kZSTD supported: 0 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: kXpressCompression supported: 0 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: kZlibCompression supported: 1 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: kSnappyCompression supported: 1 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: kLZ4Compression supported: 1 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: kBZip2Compression supported: 0 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi138/store.db/MANIFEST-000009 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T01:40:57.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.merge_operator: 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_filter: None 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562d846a1a60) 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: cache_index_and_filter_blocks: 1 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: pin_top_level_index_and_filter: 1 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_type: 0 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_block_index_type: 0 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_shortening: 1 2024-04-25T01:40:57.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: checksum: 4 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: no_block_cache: 0 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache: 0x562d846a5090 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_name: BinnedLRUCache 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_options: 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: capacity : 536870912 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: num_shard_bits : 4 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: strict_capacity_limit : 0 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: high_pri_pool_ratio: 0.000 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_cache_compressed: (nil) 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: persistent_cache: (nil) 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_size: 4096 2024-04-25T01:40:57.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_size_deviation: 10 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_restart_interval: 16 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: index_block_restart_interval: 1 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: metadata_block_size: 4096 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: partition_filters: 0 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: use_delta_encoding: 1 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: filter_policy: bloomfilter 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: whole_key_filtering: 1 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: verify_compression: 0 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: read_amp_bytes_per_bit: 0 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: format_version: 5 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: enable_index_compression: 1 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: block_align: 0 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_auto_readahead_size: 262144 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: prepopulate_block_cache: 0 2024-04-25T01:40:57.039 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: initial_auto_readahead_size: 8192 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression: NoCompression 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.num_levels: 7 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T01:40:57.040 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T01:40:57.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T01:40:57.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T01:40:57.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.table_properties_collectors: 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.inplace_update_support: 0 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.bloom_locality: 0 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.max_successive_merges: 0 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T01:40:57.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.ttl: 2592000 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.enable_blob_files: false 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.min_blob_size: 0 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T01:40:57.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi138/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12290, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6f706d39-2004-40a1-9c43-67cd76a7f0f8 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714009256683801, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714009256701998, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 3657292, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12295, "largest_seqno": 14958, "table_properties": {"data_size": 3646670, "index_size": 6465, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3141, "raw_key_size": 27814, "raw_average_key_size": 22, "raw_value_size": 3622266, "raw_average_value_size": 2923, "num_data_blocks": 292, "num_entries": 1239, "num_filter_entries": 1239, "num_deletions": 258, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714009256, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6f706d39-2004-40a1-9c43-67cd76a7f0f8", "db_session_id": "SUVL4VB8JMR0EUF2Z6X8", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714009256702128, "job": 1, "event": "recovery_finished"} 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi138/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T01:40:57.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562d84796000 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: DB pointer 0x562d84780000 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** DB Stats ** 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-04-25T01:40:57.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** Compaction Stats [default] ** 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: L0 1/0 3.49 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 497.4 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: L6 1/0 6.89 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Sum 2/0 10.37 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 497.4 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 497.4 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** Compaction Stats [default] ** 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 497.4 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T01:40:57.048 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Cumulative compaction: 0.00 GB write, 166.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Interval compaction: 0.00 GB write, 166.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Block cache BinnedLRUCache@0x562d846a5090#2 capacity: 512.00 MB usage: 9.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,3.28 KB,0.000625849%) IndexBlock(1,6.58 KB,0.00125468%) Misc(1,0.00 KB,0%) 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T01:40:57.049 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: starting mon.smithi138 rank 1 at public addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] at bind addrs [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi138 fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: mon.smithi138@-1(???) e2 preinit fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: mon.smithi138@-1(???).mds e19 new map 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: mon.smithi138@-1(???).mds e19 print_map 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: e19 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: legacy client fscid: 1 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: Filesystem 'cephfs' (1) 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: fs_name cephfs 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: epoch 19 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:40:57.050 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: tableserver 0 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: root 0 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: session_timeout 60 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: session_autoclose 300 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_file_size 1099511627776 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_xattr_size 65536 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: required_client_features {} 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: last_failure 0 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: last_failure_osd_epoch 0 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: max_mds 2 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: in 0,1 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: up {0=14446,1=24279} 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: failed 2024-04-25T01:40:57.051 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: damaged 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: stopped 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: data_pools [3] 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: metadata_pool 2 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: inline_data disabled 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: balancer 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: bal_rank_mask -1 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: standby_count_wanted 1 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: qdb_cluster leader: 0 members: 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: [mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-04-25T01:40:57.052 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout: 2024-04-25T01:40:57.053 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: mon.smithi138@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-25T01:40:57.053 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: mon.smithi138@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T01:40:57.053 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: mon.smithi138@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T01:40:57.053 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: mon.smithi138@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T01:40:57.053 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 ceph-mon[81304]: mon.smithi138@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-25T01:40:57.053 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 bash[81284]: e2b197ed2121ac88e0a817c814a82aaa1d0129236c0654364be06717b6bd54eb 2024-04-25T01:40:57.053 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:56 smithi138 systemd[1]: Started Ceph mon.smithi138 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.292+0000 7fedb7ed5700 1 -- 172.21.15.33:0/838581864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 msgr2=0x7fedb00fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.292+0000 7fedb7ed5700 1 --2- 172.21.15.33:0/838581864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 0x7fedb00fcb60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fedac004940 tx=0x7fedac009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.292+0000 7fedb7ed5700 1 -- 172.21.15.33:0/838581864 shutdown_connections 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.292+0000 7fedb7ed5700 1 --2- 172.21.15.33:0/838581864 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 0x7fedb0109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.292+0000 7fedb7ed5700 1 --2- 172.21.15.33:0/838581864 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 0x7fedb00fcb60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.292+0000 7fedb7ed5700 1 -- 172.21.15.33:0/838581864 >> 172.21.15.33:0/838581864 conn(0x7fedb0071d00 msgr2=0x7fedb0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.292+0000 7fedb7ed5700 1 -- 172.21.15.33:0/838581864 shutdown_connections 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.292+0000 7fedb7ed5700 1 -- 172.21.15.33:0/838581864 wait complete. 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb7ed5700 1 Processor -- start 2024-04-25T01:40:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb7ed5700 1 -- start start 2024-04-25T01:40:57.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb7ed5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 0x7fedb0115e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb7ed5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 0x7fedb0116350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb7ed5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb0116a30 con 0x7fedb00fc790 2024-04-25T01:40:57.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb7ed5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb010fd70 con 0x7fedb00fd0a0 2024-04-25T01:40:57.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb5c71700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 0x7fedb0115e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb5c71700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 0x7fedb0115e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55690/0 (socket says 172.21.15.33:55690) 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb5c71700 1 -- 172.21.15.33:0/2197779633 learned_addr learned my addr 172.21.15.33:0/2197779633 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.293+0000 7fedb5470700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 0x7fedb0116350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb5470700 1 -- 172.21.15.33:0/2197779633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 msgr2=0x7fedb0116350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb5470700 1 -- 172.21.15.33:0/2197779633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 msgr2=0x7fedb0116350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb5c71700 1 -- 172.21.15.33:0/2197779633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 msgr2=0x7fedb0116350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb5470700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 0x7fedb0116350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb5470700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 0x7fedb0116350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb5c71700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 0x7fedb0116350 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb5c71700 1 -- 172.21.15.33:0/2197779633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac005040 con 0x7fedb00fc790 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb5c71700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 0x7fedb0115e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedac00b3b0 tx=0x7fedac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7feda67fc700 1 -- 172.21.15.33:0/2197779633 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedac00e040 con 0x7fedb00fc790 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb010ff90 con 0x7fedb00fc790 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7feda67fc700 1 -- 172.21.15.33:0/2197779633 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedac00bbc0 con 0x7fedb00fc790 2024-04-25T01:40:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7feda67fc700 1 -- 172.21.15.33:0/2197779633 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedac01c3f0 con 0x7fedb00fc790 2024-04-25T01:40:57.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.294+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb01104d0 con 0x7fedb00fc790 2024-04-25T01:40:57.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.295+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb00fc330 con 0x7fedb00fc790 2024-04-25T01:40:57.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.296+0000 7feda67fc700 1 -- 172.21.15.33:0/2197779633 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fedac00bd30 con 0x7fedb00fc790 2024-04-25T01:40:57.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.296+0000 7feda67fc700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed9c06e260 0x7fed9c070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.296+0000 7feda67fc700 1 -- 172.21.15.33:0/2197779633 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fedac08eeb0 con 0x7fedb00fc790 2024-04-25T01:40:57.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.296+0000 7fedb5470700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed9c06e260 0x7fed9c070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.297+0000 7fedb5470700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed9c06e260 0x7fed9c070710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feda0006fd0 tx=0x7feda0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.301+0000 7feda67fc700 1 -- 172.21.15.33:0/2197779633 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fedac05bc70 con 0x7fedb00fc790 2024-04-25T01:40:57.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.433+0000 7feda67fc700 1 -- 172.21.15.33:0/2197779633 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedac05bc70 con 0x7fedb00fc790 2024-04-25T01:40:57.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.532+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fedb010c1f0 con 0x7fed9c06e260 2024-04-25T01:40:57.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.533+0000 7feda67fc700 1 -- 172.21.15.33:0/2197779633 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fedb010c1f0 con 0x7fed9c06e260 2024-04-25T01:40:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed9c06e260 msgr2=0x7fed9c070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed9c06e260 0x7fed9c070710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7feda0006fd0 tx=0x7feda0008040 comp rx=0 tx=0).stop 2024-04-25T01:40:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 msgr2=0x7fedb0115e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 0x7fedb0115e10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fedac00b3b0 tx=0x7fedac009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 shutdown_connections 2024-04-25T01:40:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed9c06e260 0x7fed9c070710 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fedb00fc790 0x7fedb0115e10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 --2- 172.21.15.33:0/2197779633 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fedb00fd0a0 0x7fedb0116350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 >> 172.21.15.33:0/2197779633 conn(0x7fedb0071d00 msgr2=0x7fedb00f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 shutdown_connections 2024-04-25T01:40:57.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.535+0000 7fedb7ed5700 1 -- 172.21.15.33:0/2197779633 wait complete. 2024-04-25T01:40:57.531 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:40:57.532 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:57 smithi033 ceph-mon[111306]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T01:40:57.532 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:57 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:40:57.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 -- 172.21.15.33:0/3668848347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fc190 msgr2=0x7f6600101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 --2- 172.21.15.33:0/3668848347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fc190 0x7f6600101a70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f65ec004940 tx=0x7f65ec009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:57.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 -- 172.21.15.33:0/3668848347 shutdown_connections 2024-04-25T01:40:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 --2- 172.21.15.33:0/3668848347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fc190 0x7f6600101a70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 --2- 172.21.15.33:0/3668848347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fb7f0 0x7f66000fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 -- 172.21.15.33:0/3668848347 >> 172.21.15.33:0/3668848347 conn(0x7f6600071460 msgr2=0x7f6600071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 -- 172.21.15.33:0/3668848347 shutdown_connections 2024-04-25T01:40:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 -- 172.21.15.33:0/3668848347 wait complete. 2024-04-25T01:40:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 Processor -- start 2024-04-25T01:40:57.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.641+0000 7f66071c6700 1 -- start start 2024-04-25T01:40:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f66071c6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fb7f0 0x7f66000fc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f66071c6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fc190 0x7f66000fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f66071c6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000fe6d0 con 0x7f66000fb7f0 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f66071c6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000fd200 con 0x7f66000fc190 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f6604f62700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fb7f0 0x7f66000fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f6604f62700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fb7f0 0x7f66000fc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55708/0 (socket says 172.21.15.33:55708) 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f6604f62700 1 -- 172.21.15.33:0/2641329541 learned_addr learned my addr 172.21.15.33:0/2641329541 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f65fffff700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fc190 0x7f66000fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f65fffff700 1 -- 172.21.15.33:0/2641329541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fc190 msgr2=0x7f66000fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f65fffff700 1 -- 172.21.15.33:0/2641329541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fc190 msgr2=0x7f66000fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f65fffff700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fc190 0x7f66000fccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f6604f62700 1 -- 172.21.15.33:0/2641329541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fc190 msgr2=0x7f66000fccc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f65fffff700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fc190 0x7f66000fccc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f6604f62700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fc190 0x7f66000fccc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f6604f62700 1 -- 172.21.15.33:0/2641329541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ec005040 con 0x7f66000fb7f0 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f6604f62700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fb7f0 0x7f66000fc780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65f4002a50 tx=0x7f65f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f65fd7fa700 1 -- 172.21.15.33:0/2641329541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65f4007cc0 con 0x7f66000fb7f0 2024-04-25T01:40:57.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.642+0000 7f65fd7fa700 1 -- 172.21.15.33:0/2641329541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65f400f040 con 0x7f66000fb7f0 2024-04-25T01:40:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.643+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66000fd4e0 con 0x7f66000fb7f0 2024-04-25T01:40:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.643+0000 7f65fd7fa700 1 -- 172.21.15.33:0/2641329541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65f4009640 con 0x7f66000fb7f0 2024-04-25T01:40:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.643+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660011a110 con 0x7f66000fb7f0 2024-04-25T01:40:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.643+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6600065f30 con 0x7f66000fb7f0 2024-04-25T01:40:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.644+0000 7f65fd7fa700 1 -- 172.21.15.33:0/2641329541 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f65f40097a0 con 0x7f66000fb7f0 2024-04-25T01:40:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.644+0000 7f65fd7fa700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e340 0x7f65f00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.645+0000 7f65fd7fa700 1 -- 172.21.15.33:0/2641329541 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f65f408dac0 con 0x7f66000fb7f0 2024-04-25T01:40:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.645+0000 7f65fffff700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e340 0x7f65f00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.645+0000 7f65fffff700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e340 0x7f65f00707f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f66000fdeb0 tx=0x7f65ec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.649+0000 7f65fd7fa700 1 -- 172.21.15.33:0/2641329541 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f65f4056cb0 con 0x7f66000fb7f0 2024-04-25T01:40:57.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:57 smithi138 ceph-mon[81304]: pgmap v10: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-25T01:40:57.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:57 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:40:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.877+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6600101160 con 0x7f65f006e340 2024-04-25T01:40:57.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.879+0000 7f65fd7fa700 1 -- 172.21.15.33:0/2641329541 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f6600101160 con 0x7f65f006e340 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e340 msgr2=0x7f65f00707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e340 0x7f65f00707f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f66000fdeb0 tx=0x7f65ec009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fb7f0 msgr2=0x7f66000fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fb7f0 0x7f66000fc780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f65f4002a50 tx=0x7f65f40046b0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 shutdown_connections 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e340 0x7f65f00707f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66000fb7f0 0x7f66000fc780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 --2- 172.21.15.33:0/2641329541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66000fc190 0x7f66000fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 >> 172.21.15.33:0/2641329541 conn(0x7f6600071460 msgr2=0x7f6600100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 shutdown_connections 2024-04-25T01:40:57.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.881+0000 7f66071c6700 1 -- 172.21.15.33:0/2641329541 wait complete. 2024-04-25T01:40:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.985+0000 7f0b8b453700 1 -- 172.21.15.33:0/2421091186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 msgr2=0x7f0b84103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.985+0000 7f0b8b453700 1 --2- 172.21.15.33:0/2421091186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 0x7f0b84103220 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004940 tx=0x7f0b78009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.985+0000 7f0b8b453700 1 -- 172.21.15.33:0/2421091186 shutdown_connections 2024-04-25T01:40:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.985+0000 7f0b8b453700 1 --2- 172.21.15.33:0/2421091186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 0x7f0b84065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.985+0000 7f0b8b453700 1 --2- 172.21.15.33:0/2421091186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 0x7f0b84103220 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.985+0000 7f0b8b453700 1 -- 172.21.15.33:0/2421091186 >> 172.21.15.33:0/2421091186 conn(0x7f0b84071d00 msgr2=0x7f0b84072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.985+0000 7f0b8b453700 1 -- 172.21.15.33:0/2421091186 shutdown_connections 2024-04-25T01:40:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.985+0000 7f0b8b453700 1 -- 172.21.15.33:0/2421091186 wait complete. 2024-04-25T01:40:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b8b453700 1 Processor -- start 2024-04-25T01:40:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b8b453700 1 -- start start 2024-04-25T01:40:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b8b453700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 0x7f0b840fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b8b453700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 0x7f0b840fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b8b453700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840fc880 con 0x7f0b84102e50 2024-04-25T01:40:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b8b453700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b840fc9c0 con 0x7f0b84065640 2024-04-25T01:40:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b889ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 0x7f0b840fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b889ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 0x7f0b840fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55726/0 (socket says 172.21.15.33:55726) 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b889ee700 1 -- 172.21.15.33:0/3484588304 learned_addr learned my addr 172.21.15.33:0/3484588304 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.986+0000 7f0b891ef700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 0x7f0b840fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b891ef700 1 -- 172.21.15.33:0/3484588304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 msgr2=0x7f0b840fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b891ef700 1 -- 172.21.15.33:0/3484588304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 msgr2=0x7f0b840fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b891ef700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 0x7f0b840fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b891ef700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 0x7f0b840fbdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b889ee700 1 -- 172.21.15.33:0/3484588304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 msgr2=0x7f0b840fbdb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b889ee700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 0x7f0b840fbdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b889ee700 1 -- 172.21.15.33:0/3484588304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b78005040 con 0x7f0b84102e50 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b889ee700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 0x7f0b840fc2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b800041d0 tx=0x7f0b800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b75ffb700 1 -- 172.21.15.33:0/3484588304 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b80007cc0 con 0x7f0b84102e50 2024-04-25T01:40:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b75ffb700 1 -- 172.21.15.33:0/3484588304 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b8000f040 con 0x7f0b84102e50 2024-04-25T01:40:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b75ffb700 1 -- 172.21.15.33:0/3484588304 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b8000dbe0 con 0x7f0b84102e50 2024-04-25T01:40:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b84119e50 con 0x7f0b84102e50 2024-04-25T01:40:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.987+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8411a270 con 0x7f0b84102e50 2024-04-25T01:40:57.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.988+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b84100100 con 0x7f0b84102e50 2024-04-25T01:40:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.989+0000 7f0b75ffb700 1 -- 172.21.15.33:0/3484588304 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0b8000dd40 con 0x7f0b84102e50 2024-04-25T01:40:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.989+0000 7f0b75ffb700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0b7006e260 0x7f0b70070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.989+0000 7f0b75ffb700 1 -- 172.21.15.33:0/3484588304 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f0b8008d470 con 0x7f0b84102e50 2024-04-25T01:40:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.989+0000 7f0b891ef700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0b7006e260 0x7f0b70070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:57.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.990+0000 7f0b891ef700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0b7006e260 0x7f0b70070710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004f90 tx=0x7f0b78009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:57.994+0000 7f0b75ffb700 1 -- 172.21.15.33:0/3484588304 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0b80056680 con 0x7f0b84102e50 2024-04-25T01:40:58.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.210+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b84109f20 con 0x7f0b7006e260 2024-04-25T01:40:58.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.221+0000 7f0b75ffb700 1 -- 172.21.15.33:0/3484588304 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0b84109f20 con 0x7f0b7006e260 2024-04-25T01:40:58.207 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:40:58.207 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (2m) 12s ago 13m 25.1M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (13m) 12s ago 13m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (12m) 14s ago 12m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (109s) 12s ago 12m 76.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (7m) 12s ago 7m 62.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (7m) 12s ago 7m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (7m) 14s ago 7m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (7m) 14s ago 7m 60.9M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (3m) 12s ago 14m 474M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (3m) 14s ago 12m 447M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (26s) 12s ago 14m 27.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 starting - - - 2048M 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (2m) 12s ago 13m 15.7M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (2m) 14s ago 12m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (11m) 12s ago 11m 224M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:40:58.208 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (11m) 12s ago 11m 237M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:40:58.209 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (10m) 12s ago 10m 189M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:40:58.209 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (10m) 14s ago 10m 263M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:40:58.209 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (9m) 14s ago 9m 231M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:40:58.209 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (9m) 14s ago 9m 217M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:40:58.209 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (2m) 12s ago 12m 60.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:40:58.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0b7006e260 msgr2=0x7f0b70070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0b7006e260 0x7f0b70070710 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b78004f90 tx=0x7f0b78009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:58.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 msgr2=0x7f0b840fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 0x7f0b840fc2f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0b800041d0 tx=0x7f0b800046b0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 shutdown_connections 2024-04-25T01:40:58.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0b7006e260 0x7f0b70070710 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0b84065640 0x7f0b840fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 --2- 172.21.15.33:0/3484588304 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0b84102e50 0x7f0b840fc2f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 >> 172.21.15.33:0/3484588304 conn(0x7f0b84071d00 msgr2=0x7f0b840fafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:58.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 shutdown_connections 2024-04-25T01:40:58.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.224+0000 7f0b8b453700 1 -- 172.21.15.33:0/3484588304 wait complete. 2024-04-25T01:40:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 -- 172.21.15.33:0/3960503676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 msgr2=0x7f5338103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 --2- 172.21.15.33:0/3960503676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 0x7f5338103220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5328004940 tx=0x7f5328009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 -- 172.21.15.33:0/3960503676 shutdown_connections 2024-04-25T01:40:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 --2- 172.21.15.33:0/3960503676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 0x7f5338065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 --2- 172.21.15.33:0/3960503676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 0x7f5338103220 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 -- 172.21.15.33:0/3960503676 >> 172.21.15.33:0/3960503676 conn(0x7f5338071d00 msgr2=0x7f5338072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 -- 172.21.15.33:0/3960503676 shutdown_connections 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 -- 172.21.15.33:0/3960503676 wait complete. 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 Processor -- start 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.331+0000 7f533dadd700 1 -- start start 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f533dadd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 0x7f53380fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f533dadd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 0x7f53380fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f533dadd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53380fc880 con 0x7f5338102e50 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f533dadd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53380fc9c0 con 0x7f5338065640 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f5336ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 0x7f53380fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f5336ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 0x7f53380fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55750/0 (socket says 172.21.15.33:55750) 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f53377fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 0x7f53380fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f5336ffd700 1 -- 172.21.15.33:0/3208104926 learned_addr learned my addr 172.21.15.33:0/3208104926 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f53377fe700 1 -- 172.21.15.33:0/3208104926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 msgr2=0x7f53380fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f53377fe700 1 -- 172.21.15.33:0/3208104926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 msgr2=0x7f53380fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:40:58.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f53377fe700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 0x7f53380fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T01:40:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f53377fe700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 0x7f53380fbdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f5336ffd700 1 -- 172.21.15.33:0/3208104926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 msgr2=0x7f53380fbdb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f5336ffd700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 0x7f53380fbdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f5336ffd700 1 -- 172.21.15.33:0/3208104926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5328005040 con 0x7f5338102e50 2024-04-25T01:40:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.332+0000 7f5336ffd700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 0x7f53380fc2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f532c002a50 tx=0x7f532c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.333+0000 7f531ffff700 1 -- 172.21.15.33:0/3208104926 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f532c007cc0 con 0x7f5338102e50 2024-04-25T01:40:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.333+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5338119e80 con 0x7f5338102e50 2024-04-25T01:40:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.333+0000 7f531ffff700 1 -- 172.21.15.33:0/3208104926 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f532c00f040 con 0x7f5338102e50 2024-04-25T01:40:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.333+0000 7f531ffff700 1 -- 172.21.15.33:0/3208104926 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f532c009640 con 0x7f5338102e50 2024-04-25T01:40:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.333+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f533811a300 con 0x7f5338102e50 2024-04-25T01:40:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.334+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5338100100 con 0x7f5338102e50 2024-04-25T01:40:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.334+0000 7f531ffff700 1 -- 172.21.15.33:0/3208104926 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f532c0097a0 con 0x7f5338102e50 2024-04-25T01:40:58.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.335+0000 7f531ffff700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f532006e260 0x7f5320070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.335+0000 7f531ffff700 1 -- 172.21.15.33:0/3208104926 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f532c08db00 con 0x7f5338102e50 2024-04-25T01:40:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.335+0000 7f53377fe700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f532006e260 0x7f5320070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.335+0000 7f53377fe700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f532006e260 0x7f5320070710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5328004f90 tx=0x7f5328009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.340+0000 7f531ffff700 1 -- 172.21.15.33:0/3208104926 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f532c056d10 con 0x7f5338102e50 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: mon.smithi138 calling monitor election 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: mon.smithi033 calling monitor election 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: mon.smithi033 is new leader, mons smithi033,smithi138 in quorum (ranks 0,1) 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: monmap epoch 3 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: last_changed 2024-04-25T01:40:57.431510+0000 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: created 2024-04-25T01:26:02.814821+0000 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: min_mon_release 19 (squid) 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: election_strategy: 1 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.smithi033 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-04-25T01:40:58.472 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:40:58.473 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: osdmap e48: 6 total, 6 up, 6 in 2024-04-25T01:40:58.473 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: mgrmap e40: smithi033.asemba(active, since 17s), standbys: smithi138.mmfhyi 2024-04-25T01:40:58.473 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: overall HEALTH_OK 2024-04-25T01:40:58.473 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:58.473 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:58.473 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:58.473 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:58.473 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:58 smithi033 ceph-mon[111306]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:58.514 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi033"}]: dispatch 2024-04-25T01:40:58.514 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: mon.smithi138 calling monitor election 2024-04-25T01:40:58.514 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: mon.smithi033 calling monitor election 2024-04-25T01:40:58.514 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mon metadata", "id": "smithi138"}]: dispatch 2024-04-25T01:40:58.514 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: mon.smithi033 is new leader, mons smithi033,smithi138 in quorum (ranks 0,1) 2024-04-25T01:40:58.514 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: monmap epoch 3 2024-04-25T01:40:58.514 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 2024-04-25T01:40:58.514 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: last_changed 2024-04-25T01:40:57.431510+0000 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: created 2024-04-25T01:26:02.814821+0000 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: min_mon_release 19 (squid) 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: election_strategy: 1 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: 0: [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] mon.smithi033 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: 1: [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] mon.smithi138 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2 up:standby-replay 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: osdmap e48: 6 total, 6 up, 6 in 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: mgrmap e40: smithi033.asemba(active, since 17s), standbys: smithi138.mmfhyi 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: overall HEALTH_OK 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:58.515 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:58 smithi138 ceph-mon[81304]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:58.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.641+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5338109ef0 con 0x7f5338102e50 2024-04-25T01:40:58.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.641+0000 7f531ffff700 1 -- 172.21.15.33:0/3208104926 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f532c05a3d0 con 0x7f5338102e50 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:40:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 10, 2024-04-25T01:40:58.629 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T01:40:58.629 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:40:58.629 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:40:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f532006e260 msgr2=0x7f5320070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f532006e260 0x7f5320070710 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5328004f90 tx=0x7f5328009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 msgr2=0x7f53380fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 0x7f53380fc2f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f532c002a50 tx=0x7f532c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 shutdown_connections 2024-04-25T01:40:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f532006e260 0x7f5320070710 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5338065640 0x7f53380fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 --2- 172.21.15.33:0/3208104926 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5338102e50 0x7f53380fc2f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 >> 172.21.15.33:0/3208104926 conn(0x7f5338071d00 msgr2=0x7f53380fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 shutdown_connections 2024-04-25T01:40:58.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.644+0000 7f533dadd700 1 -- 172.21.15.33:0/3208104926 wait complete. 2024-04-25T01:40:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.751+0000 7fccccaff700 1 -- 172.21.15.33:0/979592076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 msgr2=0x7fccc81020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.751+0000 7fccccaff700 1 --2- 172.21.15.33:0/979592076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 0x7fccc81020a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fccb8004940 tx=0x7fccb8009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.751+0000 7fccccaff700 1 -- 172.21.15.33:0/979592076 shutdown_connections 2024-04-25T01:40:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.751+0000 7fccccaff700 1 --2- 172.21.15.33:0/979592076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 0x7fccc80fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.751+0000 7fccccaff700 1 --2- 172.21.15.33:0/979592076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 0x7fccc81020a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.751+0000 7fccccaff700 1 -- 172.21.15.33:0/979592076 >> 172.21.15.33:0/979592076 conn(0x7fccc8071460 msgr2=0x7fccc8071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.751+0000 7fccccaff700 1 -- 172.21.15.33:0/979592076 shutdown_connections 2024-04-25T01:40:58.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.751+0000 7fccccaff700 1 -- 172.21.15.33:0/979592076 wait complete. 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccccaff700 1 Processor -- start 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccccaff700 1 -- start start 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccccaff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 0x7fccc80fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccccaff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 0x7fccc80fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccccaff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80fd2b0 con 0x7fccc8101cd0 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccccaff700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80fd3f0 con 0x7fccc8102670 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccc659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 0x7fccc80fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccc659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 0x7fccc80fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55770/0 (socket says 172.21.15.33:55770) 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccc659c700 1 -- 172.21.15.33:0/3948728903 learned_addr learned my addr 172.21.15.33:0/3948728903 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccc5d9b700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 0x7fccc80fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccc5d9b700 1 -- 172.21.15.33:0/3948728903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 msgr2=0x7fccc80fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccc5d9b700 1 -- 172.21.15.33:0/3948728903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 msgr2=0x7fccc80fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:40:58.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccc5d9b700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 0x7fccc80fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.752+0000 7fccc5d9b700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 0x7fccc80fcd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.753+0000 7fccc659c700 1 -- 172.21.15.33:0/3948728903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 msgr2=0x7fccc80fcd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.753+0000 7fccc659c700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 0x7fccc80fcd20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.753+0000 7fccc659c700 1 -- 172.21.15.33:0/3948728903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb8005040 con 0x7fccc8101cd0 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.753+0000 7fccc659c700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 0x7fccc80fc7e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fccb800b3b0 tx=0x7fccb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.753+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3948728903 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb800e040 con 0x7fccc8101cd0 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.753+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3948728903 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccb800bbc0 con 0x7fccc8101cd0 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.753+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc80fd640 con 0x7fccc8101cd0 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.753+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3948728903 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb801c3f0 con 0x7fccc8101cd0 2024-04-25T01:40:58.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.753+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc811a2d0 con 0x7fccc8101cd0 2024-04-25T01:40:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.754+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc8065200 con 0x7fccc8101cd0 2024-04-25T01:40:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.755+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3948728903 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fccb800bd30 con 0x7fccc8101cd0 2024-04-25T01:40:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.756+0000 7fccb6ffd700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccb006e260 0x7fccb0070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.756+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3948728903 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fccb808eed0 con 0x7fccc8101cd0 2024-04-25T01:40:58.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.756+0000 7fccc5d9b700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccb006e260 0x7fccb0070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:58.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.757+0000 7fccc5d9b700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccb006e260 0x7fccb0070710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fccc80fe120 tx=0x7fccbc009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:58.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:58.761+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3948728903 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fccb8058160 con 0x7fccc8101cd0 2024-04-25T01:40:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.012+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fccc805f5c0 con 0x7fccc8101cd0 2024-04-25T01:40:58.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.012+0000 7fccb6ffd700 1 -- 172.21.15.33:0/3948728903 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1903 (secure 0 0 0) 0x7fccb805b820 con 0x7fccc8101cd0 2024-04-25T01:40:58.998 INFO:teuthology.orchestra.run.smithi033.stdout:e19 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 19 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:40:58.999 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:40:59.000 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:40:59.001 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:40:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.015+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccb006e260 msgr2=0x7fccb0070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.015+0000 7fccccaff700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccb006e260 0x7fccb0070710 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fccc80fe120 tx=0x7fccbc009ee0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.015+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 msgr2=0x7fccc80fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.015+0000 7fccccaff700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 0x7fccc80fc7e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fccb800b3b0 tx=0x7fccb8009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.015+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 shutdown_connections 2024-04-25T01:40:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.015+0000 7fccccaff700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccb006e260 0x7fccb0070710 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.015+0000 7fccccaff700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fccc8101cd0 0x7fccc80fc7e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.015+0000 7fccccaff700 1 --2- 172.21.15.33:0/3948728903 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fccc8102670 0x7fccc80fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.015+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 >> 172.21.15.33:0/3948728903 conn(0x7fccc8071460 msgr2=0x7fccc80f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.016+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 shutdown_connections 2024-04-25T01:40:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.016+0000 7fccccaff700 1 -- 172.21.15.33:0/3948728903 wait complete. 2024-04-25T01:40:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:40:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.120+0000 7f66be2f1700 1 -- 172.21.15.33:0/1507066100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8102e50 msgr2=0x7f66b8103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.120+0000 7f66be2f1700 1 --2- 172.21.15.33:0/1507066100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8102e50 0x7f66b8103220 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f66a8004940 tx=0x7f66a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.120+0000 7f66be2f1700 1 -- 172.21.15.33:0/1507066100 shutdown_connections 2024-04-25T01:40:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.120+0000 7f66be2f1700 1 --2- 172.21.15.33:0/1507066100 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8065640 0x7f66b8065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.120+0000 7f66be2f1700 1 --2- 172.21.15.33:0/1507066100 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8102e50 0x7f66b8103220 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.120+0000 7f66be2f1700 1 -- 172.21.15.33:0/1507066100 >> 172.21.15.33:0/1507066100 conn(0x7f66b8071d00 msgr2=0x7f66b8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:59.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.121+0000 7f66be2f1700 1 -- 172.21.15.33:0/1507066100 shutdown_connections 2024-04-25T01:40:59.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.121+0000 7f66be2f1700 1 -- 172.21.15.33:0/1507066100 wait complete. 2024-04-25T01:40:59.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.121+0000 7f66be2f1700 1 Processor -- start 2024-04-25T01:40:59.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.121+0000 7f66be2f1700 1 -- start start 2024-04-25T01:40:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.121+0000 7f66be2f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8065640 0x7f66b8113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:59.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.121+0000 7f66be2f1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8102e50 0x7f66b81140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.121+0000 7f66be2f1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b8114710 con 0x7f66b8065640 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.121+0000 7f66be2f1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b810db00 con 0x7f66b8102e50 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8065640 0x7f66b8113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8065640 0x7f66b8113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55788/0 (socket says 172.21.15.33:55788) 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b7fff700 1 -- 172.21.15.33:0/1081599353 learned_addr learned my addr 172.21.15.33:0/1081599353 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b77fe700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8102e50 0x7f66b81140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b77fe700 1 -- 172.21.15.33:0/1081599353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8102e50 msgr2=0x7f66b81140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b77fe700 1 -- 172.21.15.33:0/1081599353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8102e50 msgr2=0x7f66b81140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b77fe700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8102e50 0x7f66b81140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b7fff700 1 -- 172.21.15.33:0/1081599353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8102e50 msgr2=0x7f66b81140c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b77fe700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8102e50 0x7f66b81140c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b7fff700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8102e50 0x7f66b81140c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b7fff700 1 -- 172.21.15.33:0/1081599353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a8005040 con 0x7f66b8065640 2024-04-25T01:40:59.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b7fff700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8065640 0x7f66b8113b80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009150 tx=0x7f66a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:59.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b4ff9700 1 -- 172.21.15.33:0/1081599353 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66a800e040 con 0x7f66b8065640 2024-04-25T01:40:59.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b4ff9700 1 -- 172.21.15.33:0/1081599353 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66a8013420 con 0x7f66b8065640 2024-04-25T01:40:59.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b810dd80 con 0x7f66b8065640 2024-04-25T01:40:59.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66b4ff9700 1 -- 172.21.15.33:0/1081599353 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66a801c630 con 0x7f66b8065640 2024-04-25T01:40:59.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.122+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b810e340 con 0x7f66b8065640 2024-04-25T01:40:59.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.123+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b8100100 con 0x7f66b8065640 2024-04-25T01:40:59.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.124+0000 7f66b4ff9700 1 -- 172.21.15.33:0/1081599353 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f66a8013590 con 0x7f66b8065640 2024-04-25T01:40:59.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.125+0000 7f66b4ff9700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66a406e260 0x7f66a4070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:59.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.125+0000 7f66b4ff9700 1 -- 172.21.15.33:0/1081599353 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f66a808ee70 con 0x7f66b8065640 2024-04-25T01:40:59.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.125+0000 7f66b77fe700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66a406e260 0x7f66a4070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:59.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.125+0000 7f66b77fe700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66a406e260 0x7f66a4070710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f66b810ecc0 tx=0x7f66a0009f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:59.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.133+0000 7f66b4ff9700 1 -- 172.21.15.33:0/1081599353 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f66a805bcd0 con 0x7f66b8065640 2024-04-25T01:40:59.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.361+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66b8109ef0 con 0x7f66a406e260 2024-04-25T01:40:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.363+0000 7f66b4ff9700 1 -- 172.21.15.33:0/1081599353 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f66b8109ef0 con 0x7f66a406e260 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr" 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "3/21 daemons upgraded", 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mon daemons", 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:40:59.349 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:40:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66a406e260 msgr2=0x7f66a4070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66a406e260 0x7f66a4070710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f66b810ecc0 tx=0x7f66a0009f10 comp rx=0 tx=0).stop 2024-04-25T01:40:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8065640 msgr2=0x7f66b8113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8065640 0x7f66b8113b80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f66a8009150 tx=0x7f66a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 shutdown_connections 2024-04-25T01:40:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66a406e260 0x7f66a4070710 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66b8065640 0x7f66b8113b80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 --2- 172.21.15.33:0/1081599353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66b8102e50 0x7f66b81140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 >> 172.21.15.33:0/1081599353 conn(0x7f66b8071d00 msgr2=0x7f66b80fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:59.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 shutdown_connections 2024-04-25T01:40:59.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.365+0000 7f66be2f1700 1 -- 172.21.15.33:0/1081599353 wait complete. 2024-04-25T01:40:59.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.471+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3399842977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 msgr2=0x7fa8c81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.471+0000 7fa8cde5e700 1 --2- 172.21.15.33:0/3399842977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 0x7fa8c81031b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8004940 tx=0x7fa8b8009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.471+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3399842977 shutdown_connections 2024-04-25T01:40:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.471+0000 7fa8cde5e700 1 --2- 172.21.15.33:0/3399842977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 0x7fa8c80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.471+0000 7fa8cde5e700 1 --2- 172.21.15.33:0/3399842977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 0x7fa8c81031b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.471+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3399842977 >> 172.21.15.33:0/3399842977 conn(0x7fa8c80f8990 msgr2=0x7fa8c80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.471+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3399842977 shutdown_connections 2024-04-25T01:40:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.471+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3399842977 wait complete. 2024-04-25T01:40:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8cde5e700 1 Processor -- start 2024-04-25T01:40:59.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8cde5e700 1 -- start start 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8cde5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 0x7fa8c8113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8cde5e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 0x7fa8c8113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8cde5e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81145b0 con 0x7fa8c8102de0 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8cde5e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c810db90 con 0x7fa8c8103780 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 0x7fa8c8113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 0x7fa8c8113a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55802/0 (socket says 172.21.15.33:55802) 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8c77fe700 1 -- 172.21.15.33:0/3730066147 learned_addr learned my addr 172.21.15.33:0/3730066147 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.472+0000 7fa8c6ffd700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 0x7fa8c8113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8c6ffd700 1 -- 172.21.15.33:0/3730066147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 msgr2=0x7fa8c8113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8c77fe700 1 -- 172.21.15.33:0/3730066147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 msgr2=0x7fa8c8113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8c6ffd700 1 -- 172.21.15.33:0/3730066147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 msgr2=0x7fa8c8113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8c6ffd700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 0x7fa8c8113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8c6ffd700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 0x7fa8c8113f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T01:40:59.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8c77fe700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 0x7fa8c8113f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8c77fe700 1 -- 172.21.15.33:0/3730066147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b8005040 con 0x7fa8c8102de0 2024-04-25T01:40:59.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8c77fe700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 0x7fa8c8113a20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800b3b0 tx=0x7fa8b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:59.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8affff700 1 -- 172.21.15.33:0/3730066147 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8b800e040 con 0x7fa8c8102de0 2024-04-25T01:40:59.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c810de10 con 0x7fa8c8102de0 2024-04-25T01:40:59.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c810e350 con 0x7fa8c8102de0 2024-04-25T01:40:59.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8affff700 1 -- 172.21.15.33:0/3730066147 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8b800bbc0 con 0x7fa8c8102de0 2024-04-25T01:40:59.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.473+0000 7fa8affff700 1 -- 172.21.15.33:0/3730066147 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8b8025400 con 0x7fa8c8102de0 2024-04-25T01:40:59.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.474+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c81000f0 con 0x7fa8c8102de0 2024-04-25T01:40:59.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.475+0000 7fa8affff700 1 -- 172.21.15.33:0/3730066147 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa8b800bd30 con 0x7fa8c8102de0 2024-04-25T01:40:59.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.475+0000 7fa8affff700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa8b006e270 0x7fa8b0070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:40:59.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.475+0000 7fa8affff700 1 -- 172.21.15.33:0/3730066147 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fa8b808ef60 con 0x7fa8c8102de0 2024-04-25T01:40:59.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.475+0000 7fa8c6ffd700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa8b006e270 0x7fa8b0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:40:59.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.476+0000 7fa8c6ffd700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa8b006e270 0x7fa8b0070720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8c810f2c0 tx=0x7fa8bc009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:40:59.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.480+0000 7fa8affff700 1 -- 172.21.15.33:0/3730066147 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa8b80c3740 con 0x7fa8c8102de0 2024-04-25T01:40:59.751 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:59 smithi033 ceph-mon[111306]: pgmap v12: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T01:40:59.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:59 smithi033 ceph-mon[111306]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:59.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:59 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3208104926' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:59.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:40:59 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3948728903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:40:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.774+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa8c810f070 con 0x7fa8c8102de0 2024-04-25T01:40:59.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.774+0000 7fa8affff700 1 -- 172.21.15.33:0/3730066147 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa8b805bca0 con 0x7fa8c8102de0 2024-04-25T01:40:59.760 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_OK 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.776+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa8b006e270 msgr2=0x7fa8b0070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.776+0000 7fa8cde5e700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa8b006e270 0x7fa8b0070720 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa8c810f2c0 tx=0x7fa8bc009f70 comp rx=0 tx=0).stop 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.776+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 msgr2=0x7fa8c8113a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.776+0000 7fa8cde5e700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 0x7fa8c8113a20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800b3b0 tx=0x7fa8b8009e30 comp rx=0 tx=0).stop 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.777+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 shutdown_connections 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.777+0000 7fa8cde5e700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa8b006e270 0x7fa8b0070720 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.777+0000 7fa8cde5e700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8c8102de0 0x7fa8c8113a20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.777+0000 7fa8cde5e700 1 --2- 172.21.15.33:0/3730066147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8c8103780 0x7fa8c8113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.777+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 >> 172.21.15.33:0/3730066147 conn(0x7fa8c80f8990 msgr2=0x7fa8c8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.777+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 shutdown_connections 2024-04-25T01:40:59.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:40:59.777+0000 7fa8cde5e700 1 -- 172.21.15.33:0/3730066147 wait complete. 2024-04-25T01:40:59.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:59 smithi138 ceph-mon[81304]: pgmap v12: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-04-25T01:40:59.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:59 smithi138 ceph-mon[81304]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:40:59.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:59 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3208104926' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:40:59.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:40:59 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3948728903' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:41:00.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:00 smithi033 ceph-mon[111306]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:00.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:00 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3730066147' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:41:00.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:00 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:00.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:00 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:00.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:00 smithi138 ceph-mon[81304]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:00.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:00 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3730066147' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:41:00.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:00 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:00.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:00 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:01.696 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:01 smithi033 ceph-mon[111306]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:01.696 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:01.696 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:01.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:01 smithi138 ceph-mon[81304]: pgmap v13: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:01.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:01.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:04.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:03 smithi033 ceph-mon[111306]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:04.024 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:03 smithi138 ceph-mon[81304]: pgmap v14: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:06.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: pgmap v15: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:06.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: Detected new or changed devices on smithi138 2024-04-25T01:41:06.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:06.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:06.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:06.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:06.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:06.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: Reconfiguring mon.smithi033 (monmap changed)... 2024-04-25T01:41:06.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:41:06.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:41:06.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:06.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:05 smithi033 ceph-mon[111306]: Reconfiguring daemon mon.smithi033 on smithi033 2024-04-25T01:41:06.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: pgmap v15: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:06.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: Detected new or changed devices on smithi138 2024-04-25T01:41:06.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:06.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:06.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:06.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:06.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:06.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: Reconfiguring mon.smithi033 (monmap changed)... 2024-04-25T01:41:06.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:41:06.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:41:06.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:06.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:05 smithi138 ceph-mon[81304]: Reconfiguring daemon mon.smithi033 on smithi033 2024-04-25T01:41:08.140 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:07 smithi033 ceph-mon[111306]: pgmap v16: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:08.140 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:08.140 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:08.140 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:07 smithi033 ceph-mon[111306]: Reconfiguring mgr.smithi033.asemba (monmap changed)... 2024-04-25T01:41:08.140 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.asemba", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:41:08.140 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:41:08.141 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:08.141 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:07 smithi033 ceph-mon[111306]: Reconfiguring daemon mgr.smithi033.asemba on smithi033 2024-04-25T01:41:08.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:07 smithi138 ceph-mon[81304]: pgmap v16: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:08.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:08.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:08.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:07 smithi138 ceph-mon[81304]: Reconfiguring mgr.smithi033.asemba (monmap changed)... 2024-04-25T01:41:08.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi033.asemba", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:41:08.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:41:08.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:08.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:07 smithi138 ceph-mon[81304]: Reconfiguring daemon mgr.smithi033.asemba on smithi033 2024-04-25T01:41:10.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:09 smithi033 ceph-mon[111306]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:41:10.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:10.002 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:10.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:09 smithi033 ceph-mon[111306]: Reconfiguring crash.smithi033 (monmap changed)... 2024-04-25T01:41:10.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:41:10.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:10.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:09 smithi033 ceph-mon[111306]: Reconfiguring daemon crash.smithi033 on smithi033 2024-04-25T01:41:10.027 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:09 smithi138 ceph-mon[81304]: pgmap v17: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:41:10.027 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:10.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:10.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:09 smithi138 ceph-mon[81304]: Reconfiguring crash.smithi033 (monmap changed)... 2024-04-25T01:41:10.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:41:10.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:10.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:09 smithi138 ceph-mon[81304]: Reconfiguring daemon crash.smithi033 on smithi033 2024-04-25T01:41:11.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:11.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:11.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:11 smithi033 ceph-mon[111306]: Reconfiguring osd.0 (monmap changed)... 2024-04-25T01:41:11.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:41:11.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:11.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:11 smithi033 ceph-mon[111306]: Reconfiguring daemon osd.0 on smithi033 2024-04-25T01:41:11.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:11 smithi033 ceph-mon[111306]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:11.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:41:11.527 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:11.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:11.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:11 smithi138 ceph-mon[81304]: Reconfiguring osd.0 (monmap changed)... 2024-04-25T01:41:11.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:41:11.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:11.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:11 smithi138 ceph-mon[81304]: Reconfiguring daemon osd.0 on smithi033 2024-04-25T01:41:11.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:11 smithi138 ceph-mon[81304]: pgmap v18: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:11.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:41:13.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:12 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:13.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:12 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:13.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:12 smithi033 ceph-mon[111306]: Reconfiguring osd.1 (monmap changed)... 2024-04-25T01:41:13.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:12 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:41:13.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:12 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:13.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:12 smithi033 ceph-mon[111306]: Reconfiguring daemon osd.1 on smithi033 2024-04-25T01:41:13.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:12 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:13.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:12 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:13.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:12 smithi138 ceph-mon[81304]: Reconfiguring osd.1 (monmap changed)... 2024-04-25T01:41:13.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:12 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:41:13.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:12 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:13.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:12 smithi138 ceph-mon[81304]: Reconfiguring daemon osd.1 on smithi033 2024-04-25T01:41:14.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:13 smithi033 ceph-mon[111306]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:14.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:13 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:14.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:13 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:14.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:13 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:41:14.183 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:13 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:14.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:13 smithi138 ceph-mon[81304]: pgmap v19: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:14.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:13 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:14.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:13 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:14.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:13 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:41:14.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:13 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:15.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:14 smithi033 ceph-mon[111306]: Reconfiguring osd.2 (monmap changed)... 2024-04-25T01:41:15.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:14 smithi033 ceph-mon[111306]: Reconfiguring daemon osd.2 on smithi033 2024-04-25T01:41:15.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:14 smithi138 ceph-mon[81304]: Reconfiguring osd.2 (monmap changed)... 2024-04-25T01:41:15.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:14 smithi138 ceph-mon[81304]: Reconfiguring daemon osd.2 on smithi033 2024-04-25T01:41:16.128 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:15 smithi033 ceph-mon[111306]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:16.128 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:16.128 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:16.128 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.qqhhcz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:41:16.129 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:16.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:15 smithi138 ceph-mon[81304]: pgmap v20: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:16.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:16.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:16.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.qqhhcz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:41:16.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:17.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:16 smithi033 ceph-mon[111306]: Reconfiguring mds.cephfs.smithi033.qqhhcz (monmap changed)... 2024-04-25T01:41:17.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:16 smithi033 ceph-mon[111306]: Reconfiguring daemon mds.cephfs.smithi033.qqhhcz on smithi033 2024-04-25T01:41:17.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:16 smithi138 ceph-mon[81304]: Reconfiguring mds.cephfs.smithi033.qqhhcz (monmap changed)... 2024-04-25T01:41:17.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:16 smithi138 ceph-mon[81304]: Reconfiguring daemon mds.cephfs.smithi033.qqhhcz on smithi033 2024-04-25T01:41:18.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:18 smithi033 ceph-mon[111306]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:18.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:18 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:18.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:18 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:18.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:18 smithi033 ceph-mon[111306]: Reconfiguring mds.cephfs.smithi033.hobkqr (monmap changed)... 2024-04-25T01:41:18.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:18 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hobkqr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:41:18.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:18 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:18.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:18 smithi033 ceph-mon[111306]: Reconfiguring daemon mds.cephfs.smithi033.hobkqr on smithi033 2024-04-25T01:41:18.277 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:18 smithi138 ceph-mon[81304]: pgmap v21: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:18.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:18 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:18.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:18 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:18.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:18 smithi138 ceph-mon[81304]: Reconfiguring mds.cephfs.smithi033.hobkqr (monmap changed)... 2024-04-25T01:41:18.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:18 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hobkqr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:41:18.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:18 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:18.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:18 smithi138 ceph-mon[81304]: Reconfiguring daemon mds.cephfs.smithi033.hobkqr on smithi033 2024-04-25T01:41:20.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:19 smithi138 ceph-mon[81304]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:20.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:19 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:20.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:19 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:20.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:19 smithi138 ceph-mon[81304]: Reconfiguring crash.smithi138 (monmap changed)... 2024-04-25T01:41:20.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:19 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:41:20.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:19 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:20.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:19 smithi138 ceph-mon[81304]: Reconfiguring daemon crash.smithi138 on smithi138 2024-04-25T01:41:20.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:19 smithi033 ceph-mon[111306]: pgmap v22: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:20.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:19 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:20.252 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:19 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:20.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:19 smithi033 ceph-mon[111306]: Reconfiguring crash.smithi138 (monmap changed)... 2024-04-25T01:41:20.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:19 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:41:20.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:19 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:20.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:19 smithi033 ceph-mon[111306]: Reconfiguring daemon crash.smithi138 on smithi138 2024-04-25T01:41:21.752 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:21 smithi033 ceph-mon[111306]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:21.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:21 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:21.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:21 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:21.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:21 smithi033 ceph-mon[111306]: Reconfiguring mgr.smithi138.mmfhyi (monmap changed)... 2024-04-25T01:41:21.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:21 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:41:21.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:21 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:41:21.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:21 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:21.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:21 smithi033 ceph-mon[111306]: Reconfiguring daemon mgr.smithi138.mmfhyi on smithi138 2024-04-25T01:41:21.777 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:21 smithi138 ceph-mon[81304]: pgmap v23: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:21.778 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:21 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:21.778 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:21 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:21.778 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:21 smithi138 ceph-mon[81304]: Reconfiguring mgr.smithi138.mmfhyi (monmap changed)... 2024-04-25T01:41:21.778 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:21 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi138.mmfhyi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T01:41:21.778 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:21 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T01:41:21.778 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:21 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:21.778 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:21 smithi138 ceph-mon[81304]: Reconfiguring daemon mgr.smithi138.mmfhyi on smithi138 2024-04-25T01:41:23.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:23 smithi138 ceph-mon[81304]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:23.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:23.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:23.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:23 smithi138 ceph-mon[81304]: Reconfiguring mon.smithi138 (monmap changed)... 2024-04-25T01:41:23.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:41:23.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:41:23.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:23.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:23 smithi138 ceph-mon[81304]: Reconfiguring daemon mon.smithi138 on smithi138 2024-04-25T01:41:23.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:23 smithi033 ceph-mon[111306]: pgmap v24: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:23.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:23.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:23.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:23 smithi033 ceph-mon[111306]: Reconfiguring mon.smithi138 (monmap changed)... 2024-04-25T01:41:23.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T01:41:23.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T01:41:23.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:23.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:23 smithi033 ceph-mon[111306]: Reconfiguring daemon mon.smithi138 on smithi138 2024-04-25T01:41:24.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:24.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:24.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:24 smithi138 ceph-mon[81304]: Reconfiguring osd.3 (monmap changed)... 2024-04-25T01:41:24.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:41:24.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:24.983 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:24 smithi138 ceph-mon[81304]: Reconfiguring daemon osd.3 on smithi138 2024-04-25T01:41:25.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:25.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:25.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:24 smithi033 ceph-mon[111306]: Reconfiguring osd.3 (monmap changed)... 2024-04-25T01:41:25.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:41:25.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:25.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:24 smithi033 ceph-mon[111306]: Reconfiguring daemon osd.3 on smithi138 2024-04-25T01:41:25.976 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:25 smithi138 ceph-mon[81304]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:25.977 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:41:25.977 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:25.977 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:25.977 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:41:25.977 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:26.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:25 smithi033 ceph-mon[111306]: pgmap v25: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:26.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:41:26.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:26.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:26.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:41:26.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:26.923 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:26 smithi138 ceph-mon[81304]: Reconfiguring osd.4 (monmap changed)... 2024-04-25T01:41:26.924 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:26 smithi138 ceph-mon[81304]: Reconfiguring daemon osd.4 on smithi138 2024-04-25T01:41:27.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:26 smithi033 ceph-mon[111306]: Reconfiguring osd.4 (monmap changed)... 2024-04-25T01:41:27.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:26 smithi033 ceph-mon[111306]: Reconfiguring daemon osd.4 on smithi138 2024-04-25T01:41:28.272 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:28 smithi138 ceph-mon[81304]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:28.272 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:28.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:28.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:28 smithi138 ceph-mon[81304]: Reconfiguring osd.5 (monmap changed)... 2024-04-25T01:41:28.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:41:28.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:28.273 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:28 smithi138 ceph-mon[81304]: Reconfiguring daemon osd.5 on smithi138 2024-04-25T01:41:28.442 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:28 smithi033 ceph-mon[111306]: pgmap v26: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:28.442 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:28.442 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:28.442 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:28 smithi033 ceph-mon[111306]: Reconfiguring osd.5 (monmap changed)... 2024-04-25T01:41:28.442 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:41:28.442 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:28.442 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:28 smithi033 ceph-mon[111306]: Reconfiguring daemon osd.5 on smithi138 2024-04-25T01:41:29.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.885+0000 7f111564d700 1 -- 172.21.15.33:0/3388413298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fe790 msgr2=0x7f1110106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.885+0000 7f111564d700 1 --2- 172.21.15.33:0/3388413298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fe790 0x7f1110106e60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1104004940 tx=0x7f1104009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.886+0000 7f111564d700 1 -- 172.21.15.33:0/3388413298 shutdown_connections 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.886+0000 7f111564d700 1 --2- 172.21.15.33:0/3388413298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fe790 0x7f1110106e60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.886+0000 7f111564d700 1 --2- 172.21.15.33:0/3388413298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11100fddf0 0x7f11100fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.886+0000 7f111564d700 1 -- 172.21.15.33:0/3388413298 >> 172.21.15.33:0/3388413298 conn(0x7f11100f7870 msgr2=0x7f11100f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.886+0000 7f111564d700 1 -- 172.21.15.33:0/3388413298 shutdown_connections 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.886+0000 7f111564d700 1 -- 172.21.15.33:0/3388413298 wait complete. 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.886+0000 7f111564d700 1 Processor -- start 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.886+0000 7f111564d700 1 -- start start 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f111564d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fddf0 0x7f11101139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f111564d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11100fe790 0x7f1110113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:29.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f111564d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11101145c0 con 0x7f11100fddf0 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f111564d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111010db30 con 0x7f11100fe790 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f110effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fddf0 0x7f11101139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f110effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fddf0 0x7f11101139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50192/0 (socket says 172.21.15.33:50192) 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f110effd700 1 -- 172.21.15.33:0/3217809199 learned_addr learned my addr 172.21.15.33:0/3217809199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f110e7fc700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11100fe790 0x7f1110113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f110effd700 1 -- 172.21.15.33:0/3217809199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11100fe790 msgr2=0x7f1110113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f110effd700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11100fe790 0x7f1110113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f110effd700 1 -- 172.21.15.33:0/3217809199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1104005040 con 0x7f11100fddf0 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f110e7fc700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11100fe790 0x7f1110113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f110effd700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fddf0 0x7f11101139a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f110000b790 tx=0x7f1100009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.887+0000 7f10f77fe700 1 -- 172.21.15.33:0/3217809199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110000f770 con 0x7f11100fddf0 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.888+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111010de10 con 0x7f11100fddf0 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.888+0000 7f10f77fe700 1 -- 172.21.15.33:0/3217809199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f110000fda0 con 0x7f11100fddf0 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.888+0000 7f10f77fe700 1 -- 172.21.15.33:0/3217809199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f110000d310 con 0x7f11100fddf0 2024-04-25T01:41:29.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.888+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111010e320 con 0x7f11100fddf0 2024-04-25T01:41:29.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.889+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1110066ab0 con 0x7f11100fddf0 2024-04-25T01:41:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.889+0000 7f10f77fe700 1 -- 172.21.15.33:0/3217809199 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f110000f8d0 con 0x7f11100fddf0 2024-04-25T01:41:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.889+0000 7f10f77fe700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f10f806e270 0x7f10f8070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.889+0000 7f10f77fe700 1 -- 172.21.15.33:0/3217809199 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f110008dc30 con 0x7f11100fddf0 2024-04-25T01:41:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.890+0000 7f110e7fc700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f10f806e270 0x7f10f8070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:29.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.890+0000 7f110e7fc700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f10f806e270 0x7f10f8070720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f111010ee10 tx=0x7f1104009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:29.894+0000 7f10f77fe700 1 -- 172.21.15.33:0/3217809199 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f110005ab00 con 0x7f11100fddf0 2024-04-25T01:41:30.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:29 smithi033 ceph-mon[111306]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:30.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:29 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:30.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:29 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:30.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:29 smithi033 ceph-mon[111306]: Reconfiguring mds.cephfs.smithi138.jwzxlz (monmap changed)... 2024-04-25T01:41:30.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:29 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.jwzxlz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:41:30.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:29 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:30.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:29 smithi033 ceph-mon[111306]: Reconfiguring daemon mds.cephfs.smithi138.jwzxlz on smithi138 2024-04-25T01:41:30.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:29 smithi138 ceph-mon[81304]: pgmap v27: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:30.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:29 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:30.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:29 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:30.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:29 smithi138 ceph-mon[81304]: Reconfiguring mds.cephfs.smithi138.jwzxlz (monmap changed)... 2024-04-25T01:41:30.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:29 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.jwzxlz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:41:30.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:29 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:30.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:29 smithi138 ceph-mon[81304]: Reconfiguring daemon mds.cephfs.smithi138.jwzxlz on smithi138 2024-04-25T01:41:30.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.121+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1110109f70 con 0x7f10f806e270 2024-04-25T01:41:30.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.122+0000 7f10f77fe700 1 -- 172.21.15.33:0/3217809199 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f1110109f70 con 0x7f10f806e270 2024-04-25T01:41:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f10f806e270 msgr2=0x7f10f8070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f10f806e270 0x7f10f8070720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f111010ee10 tx=0x7f1104009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fddf0 msgr2=0x7f11101139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fddf0 0x7f11101139a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f110000b790 tx=0x7f1100009250 comp rx=0 tx=0).stop 2024-04-25T01:41:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 shutdown_connections 2024-04-25T01:41:30.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f10f806e270 0x7f10f8070720 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11100fddf0 0x7f11101139a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 --2- 172.21.15.33:0/3217809199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11100fe790 0x7f1110113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 >> 172.21.15.33:0/3217809199 conn(0x7f11100f7870 msgr2=0x7f1110103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 shutdown_connections 2024-04-25T01:41:30.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.124+0000 7f111564d700 1 -- 172.21.15.33:0/3217809199 wait complete. 2024-04-25T01:41:30.121 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.229+0000 7f59c7119700 1 -- 172.21.15.33:0/2659789253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 msgr2=0x7f59c01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.229+0000 7f59c7119700 1 --2- 172.21.15.33:0/2659789253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 0x7f59c01031b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f59a8004940 tx=0x7f59a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.229+0000 7f59c7119700 1 -- 172.21.15.33:0/2659789253 shutdown_connections 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.229+0000 7f59c7119700 1 --2- 172.21.15.33:0/2659789253 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c0103780 0x7f59c00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.229+0000 7f59c7119700 1 --2- 172.21.15.33:0/2659789253 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 0x7f59c01031b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.229+0000 7f59c7119700 1 -- 172.21.15.33:0/2659789253 >> 172.21.15.33:0/2659789253 conn(0x7f59c00f8990 msgr2=0x7f59c00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.229+0000 7f59c7119700 1 -- 172.21.15.33:0/2659789253 shutdown_connections 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.230+0000 7f59c7119700 1 -- 172.21.15.33:0/2659789253 wait complete. 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.230+0000 7f59c7119700 1 Processor -- start 2024-04-25T01:41:30.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.230+0000 7f59c7119700 1 -- start start 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.230+0000 7f59c7119700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 0x7f59c01138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.230+0000 7f59c7119700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c0103780 0x7f59c0113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.230+0000 7f59c7119700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0114440 con 0x7f59c0102de0 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.230+0000 7f59c7119700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c010da90 con 0x7f59c0103780 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59c4eb5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 0x7f59c01138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59c4eb5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 0x7f59c01138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50210/0 (socket says 172.21.15.33:50210) 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59c4eb5700 1 -- 172.21.15.33:0/2907376650 learned_addr learned my addr 172.21.15.33:0/2907376650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59bffff700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c0103780 0x7f59c0113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59c4eb5700 1 -- 172.21.15.33:0/2907376650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c0103780 msgr2=0x7f59c0113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59c4eb5700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c0103780 0x7f59c0113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59c4eb5700 1 -- 172.21.15.33:0/2907376650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a8005040 con 0x7f59c0102de0 2024-04-25T01:41:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59c4eb5700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 0x7f59c01138b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59a800b8b0 tx=0x7f59a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59bd7fa700 1 -- 172.21.15.33:0/2907376650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59a800e040 con 0x7f59c0102de0 2024-04-25T01:41:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59bd7fa700 1 -- 172.21.15.33:0/2907376650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59a8004290 con 0x7f59c0102de0 2024-04-25T01:41:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59c7119700 1 -- 172.21.15.33:0/2907376650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c010dd10 con 0x7f59c0102de0 2024-04-25T01:41:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59bd7fa700 1 -- 172.21.15.33:0/2907376650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59a801c3f0 con 0x7f59c0102de0 2024-04-25T01:41:30.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.231+0000 7f59c7119700 1 -- 172.21.15.33:0/2907376650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c010e2d0 con 0x7f59c0102de0 2024-04-25T01:41:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.232+0000 7f59b6ffd700 1 -- 172.21.15.33:0/2907376650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a40052b0 con 0x7f59c0102de0 2024-04-25T01:41:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.233+0000 7f59bd7fa700 1 -- 172.21.15.33:0/2907376650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f59a8013420 con 0x7f59c0102de0 2024-04-25T01:41:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.233+0000 7f59bd7fa700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f59ac06e270 0x7f59ac070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.233+0000 7f59bd7fa700 1 -- 172.21.15.33:0/2907376650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f59a808ef00 con 0x7f59c0102de0 2024-04-25T01:41:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.233+0000 7f59bffff700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f59ac06e270 0x7f59ac070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:30.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.234+0000 7f59bffff700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f59ac06e270 0x7f59ac070720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59c010ec00 tx=0x7f59b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:30.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.238+0000 7f59bd7fa700 1 -- 172.21.15.33:0/2907376650 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f59a805bcc0 con 0x7f59c0102de0 2024-04-25T01:41:30.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.469+0000 7f59b6ffd700 1 -- 172.21.15.33:0/2907376650 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59a4000bc0 con 0x7f59ac06e270 2024-04-25T01:41:30.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.470+0000 7f59bd7fa700 1 -- 172.21.15.33:0/2907376650 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f59a4000bc0 con 0x7f59ac06e270 2024-04-25T01:41:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 -- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f59ac06e270 msgr2=0x7f59ac070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f59ac06e270 0x7f59ac070720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f59c010ec00 tx=0x7f59b000b040 comp rx=0 tx=0).stop 2024-04-25T01:41:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 -- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 msgr2=0x7f59c01138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 0x7f59c01138b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59a800b8b0 tx=0x7f59a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:30.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 -- 172.21.15.33:0/2907376650 shutdown_connections 2024-04-25T01:41:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f59ac06e270 0x7f59ac070720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59c0102de0 0x7f59c01138b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 --2- 172.21.15.33:0/2907376650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59c0103780 0x7f59c0113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 -- 172.21.15.33:0/2907376650 >> 172.21.15.33:0/2907376650 conn(0x7f59c00f8990 msgr2=0x7f59c00fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 -- 172.21.15.33:0/2907376650 shutdown_connections 2024-04-25T01:41:30.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.472+0000 7f59b6ffd700 1 -- 172.21.15.33:0/2907376650 wait complete. 2024-04-25T01:41:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.576+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/3336817880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 msgr2=0x7f9fa00fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.576+0000 7f9fa7eaf700 1 --2- 172.21.15.33:0/3336817880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 0x7f9fa00fbd00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c004940 tx=0x7f9f9c009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.576+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/3336817880 shutdown_connections 2024-04-25T01:41:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.576+0000 7f9fa7eaf700 1 --2- 172.21.15.33:0/3336817880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 0x7f9fa00fbd00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.576+0000 7f9fa7eaf700 1 --2- 172.21.15.33:0/3336817880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0101cf0 0x7f9fa01020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.576+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/3336817880 >> 172.21.15.33:0/3336817880 conn(0x7f9fa00f7860 msgr2=0x7f9fa00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.576+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/3336817880 shutdown_connections 2024-04-25T01:41:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.576+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/3336817880 wait complete. 2024-04-25T01:41:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa7eaf700 1 Processor -- start 2024-04-25T01:41:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa7eaf700 1 -- start start 2024-04-25T01:41:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa7eaf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0101cf0 0x7f9fa0115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa7eaf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 0x7f9fa01160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa7eaf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa0116720 con 0x7f9fa0102690 2024-04-25T01:41:30.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa7eaf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa010fd00 con 0x7f9fa0101cf0 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa544a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 0x7f9fa01160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa544a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 0x7f9fa01160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50236/0 (socket says 172.21.15.33:50236) 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa544a700 1 -- 172.21.15.33:0/598514740 learned_addr learned my addr 172.21.15.33:0/598514740 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa5c4b700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0101cf0 0x7f9fa0115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa544a700 1 -- 172.21.15.33:0/598514740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0101cf0 msgr2=0x7f9fa0115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.577+0000 7f9fa544a700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0101cf0 0x7f9fa0115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.578+0000 7f9fa544a700 1 -- 172.21.15.33:0/598514740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c005040 con 0x7f9fa0102690 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.578+0000 7f9fa544a700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 0x7f9fa01160d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009180 tx=0x7f9f9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.578+0000 7f9f967fc700 1 -- 172.21.15.33:0/598514740 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f9c00e040 con 0x7f9fa0102690 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.578+0000 7f9f967fc700 1 -- 172.21.15.33:0/598514740 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f9c013420 con 0x7f9fa0102690 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.578+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa010ff80 con 0x7f9fa0102690 2024-04-25T01:41:30.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.578+0000 7f9f967fc700 1 -- 172.21.15.33:0/598514740 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f9c01c610 con 0x7f9fa0102690 2024-04-25T01:41:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.578+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa0110540 con 0x7f9fa0102690 2024-04-25T01:41:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.579+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa00fd440 con 0x7f9fa0102690 2024-04-25T01:41:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.579+0000 7f9f967fc700 1 -- 172.21.15.33:0/598514740 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f9f9c013a00 con 0x7f9fa0102690 2024-04-25T01:41:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.580+0000 7f9f967fc700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f8c06e270 0x7f9f8c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.580+0000 7f9f967fc700 1 -- 172.21.15.33:0/598514740 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f9f9c08edf0 con 0x7f9fa0102690 2024-04-25T01:41:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.580+0000 7f9fa5c4b700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f8c06e270 0x7f9f8c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.580+0000 7f9fa5c4b700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f8c06e270 0x7f9f8c070720 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9f90004f40 tx=0x7f9f9000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:30.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.585+0000 7f9f967fc700 1 -- 172.21.15.33:0/598514740 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9f9c05bc60 con 0x7f9fa0102690 2024-04-25T01:41:30.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.799+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fa0108e40 con 0x7f9f8c06e270 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.809+0000 7f9f967fc700 1 -- 172.21.15.33:0/598514740 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9fa0108e40 con 0x7f9f8c06e270 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (2m) 45s ago 13m 25.1M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (14m) 45s ago 14m 6669k - 17.2.7-902-gc61472c8 0de407c1b3af 1c1bc1f27a95 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (13m) 30s ago 13m 6660k - 17.2.7-902-gc61472c8 0de407c1b3af cd4507db564e 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (2m) 45s ago 13m 76.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (8m) 45s ago 8m 62.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (8m) 45s ago 8m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (8m) 30s ago 8m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:41:30.796 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (8m) 30s ago 8m 61.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (4m) 45s ago 15m 474M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (3m) 30s ago 12m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (59s) 45s ago 15m 27.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (34s) 30s ago 12m 23.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (3m) 45s ago 13m 15.7M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (3m) 30s ago 13m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (12m) 45s ago 12m 224M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (11m) 45s ago 11m 237M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (11m) 45s ago 11m 189M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (10m) 30s ago 10m 263M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (10m) 30s ago 10m 231M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (9m) 30s ago 9m 218M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:41:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (2m) 45s ago 13m 60.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:41:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f8c06e270 msgr2=0x7f9f8c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f8c06e270 0x7f9f8c070720 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9f90004f40 tx=0x7f9f9000b040 comp rx=0 tx=0).stop 2024-04-25T01:41:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 msgr2=0x7f9fa01160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 0x7f9fa01160d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009180 tx=0x7f9f9c009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 shutdown_connections 2024-04-25T01:41:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f8c06e270 0x7f9f8c070720 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9fa0101cf0 0x7f9fa0115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 --2- 172.21.15.33:0/598514740 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9fa0102690 0x7f9fa01160d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 >> 172.21.15.33:0/598514740 conn(0x7f9fa00f7860 msgr2=0x7f9fa00f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 shutdown_connections 2024-04-25T01:41:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.811+0000 7f9fa7eaf700 1 -- 172.21.15.33:0/598514740 wait complete. 2024-04-25T01:41:30.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.913+0000 7f39050d2700 1 -- 172.21.15.33:0/3948489229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 msgr2=0x7f39001031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.913+0000 7f39050d2700 1 --2- 172.21.15.33:0/3948489229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 0x7f39001031b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f38f0004940 tx=0x7f38f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 -- 172.21.15.33:0/3948489229 shutdown_connections 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 --2- 172.21.15.33:0/3948489229 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900103780 0x7f39000fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 --2- 172.21.15.33:0/3948489229 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 0x7f39001031b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 -- 172.21.15.33:0/3948489229 >> 172.21.15.33:0/3948489229 conn(0x7f39000f8990 msgr2=0x7f39000fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 -- 172.21.15.33:0/3948489229 shutdown_connections 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 -- 172.21.15.33:0/3948489229 wait complete. 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 Processor -- start 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 -- start start 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 0x7f39001138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900103780 0x7f3900113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:30.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900114470 con 0x7f3900102de0 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.914+0000 7f39050d2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390010da70 con 0x7f3900103780 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38fed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 0x7f39001138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38fed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 0x7f39001138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50246/0 (socket says 172.21.15.33:50246) 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38fed9d700 1 -- 172.21.15.33:0/3772410093 learned_addr learned my addr 172.21.15.33:0/3772410093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38fe59c700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900103780 0x7f3900113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38fed9d700 1 -- 172.21.15.33:0/3772410093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900103780 msgr2=0x7f3900113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38fed9d700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900103780 0x7f3900113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38fed9d700 1 -- 172.21.15.33:0/3772410093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f0005040 con 0x7f3900102de0 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38fed9d700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 0x7f39001138e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009b30 tx=0x7f38f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38e77fe700 1 -- 172.21.15.33:0/3772410093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f0021070 con 0x7f3900102de0 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390010dcf0 con 0x7f3900102de0 2024-04-25T01:41:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38e77fe700 1 -- 172.21.15.33:0/3772410093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38f0013c00 con 0x7f3900102de0 2024-04-25T01:41:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f38e77fe700 1 -- 172.21.15.33:0/3772410093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38f000e070 con 0x7f3900102de0 2024-04-25T01:41:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.915+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390010e1d0 con 0x7f3900102de0 2024-04-25T01:41:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.916+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39001000f0 con 0x7f3900102de0 2024-04-25T01:41:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.917+0000 7f38e77fe700 1 -- 172.21.15.33:0/3772410093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f38f001cad0 con 0x7f3900102de0 2024-04-25T01:41:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.917+0000 7f38e77fe700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38e806e270 0x7f38e8070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.917+0000 7f38e77fe700 1 -- 172.21.15.33:0/3772410093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f38f008f120 con 0x7f3900102de0 2024-04-25T01:41:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.918+0000 7f38fe59c700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38e806e270 0x7f38e8070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:30.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.918+0000 7f38fe59c700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38e806e270 0x7f38e8070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f390010ec30 tx=0x7f38f400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:30.922+0000 7f38e77fe700 1 -- 172.21.15.33:0/3772410093 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f38f005bee0 con 0x7f3900102de0 2024-04-25T01:41:31.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.228+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f390010ef30 con 0x7f3900102de0 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.228+0000 7f38e77fe700 1 -- 172.21.15.33:0/3772410093 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f38f0004490 con 0x7f3900102de0 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:41:31.216 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T01:41:31.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:41:31.217 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:41:31.217 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:41:31.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:41:31.217 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:41:31.217 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 10, 2024-04-25T01:41:31.217 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T01:41:31.217 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:41:31.217 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:41:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38e806e270 msgr2=0x7f38e8070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38e806e270 0x7f38e8070720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f390010ec30 tx=0x7f38f400b040 comp rx=0 tx=0).stop 2024-04-25T01:41:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 msgr2=0x7f39001138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 0x7f39001138e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f38f0009b30 tx=0x7f38f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 shutdown_connections 2024-04-25T01:41:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38e806e270 0x7f38e8070720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3900102de0 0x7f39001138e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 --2- 172.21.15.33:0/3772410093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3900103780 0x7f3900113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 >> 172.21.15.33:0/3772410093 conn(0x7f39000f8990 msgr2=0x7f39000fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 shutdown_connections 2024-04-25T01:41:31.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.231+0000 7f39050d2700 1 -- 172.21.15.33:0/3772410093 wait complete. 2024-04-25T01:41:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.337+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2660134624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 msgr2=0x7fcd941031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.337+0000 7fcd99cf0700 1 --2- 172.21.15.33:0/2660134624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 0x7fcd941031b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcd84004940 tx=0x7fcd84009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.337+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2660134624 shutdown_connections 2024-04-25T01:41:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.337+0000 7fcd99cf0700 1 --2- 172.21.15.33:0/2660134624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94103780 0x7fcd940fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.337+0000 7fcd99cf0700 1 --2- 172.21.15.33:0/2660134624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 0x7fcd941031b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.337+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2660134624 >> 172.21.15.33:0/2660134624 conn(0x7fcd940f8990 msgr2=0x7fcd940fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.337+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2660134624 shutdown_connections 2024-04-25T01:41:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.337+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2660134624 wait complete. 2024-04-25T01:41:31.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.337+0000 7fcd99cf0700 1 Processor -- start 2024-04-25T01:41:31.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd99cf0700 1 -- start start 2024-04-25T01:41:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd99cf0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 0x7fcd94113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd99cf0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94103780 0x7fcd94113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd99cf0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd94114500 con 0x7fcd94102de0 2024-04-25T01:41:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd99cf0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9410db50 con 0x7fcd94103780 2024-04-25T01:41:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 0x7fcd94113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 0x7fcd94113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50254/0 (socket says 172.21.15.33:50254) 2024-04-25T01:41:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd937fe700 1 -- 172.21.15.33:0/2825237514 learned_addr learned my addr 172.21.15.33:0/2825237514 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:41:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd92ffd700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94103780 0x7fcd94113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:31.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd937fe700 1 -- 172.21.15.33:0/2825237514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94103780 msgr2=0x7fcd94113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd937fe700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94103780 0x7fcd94113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd937fe700 1 -- 172.21.15.33:0/2825237514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd84005040 con 0x7fcd94102de0 2024-04-25T01:41:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.338+0000 7fcd937fe700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 0x7fcd94113970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009150 tx=0x7fcd84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.339+0000 7fcd7bfff700 1 -- 172.21.15.33:0/2825237514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd8400e040 con 0x7fcd94102de0 2024-04-25T01:41:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.339+0000 7fcd7bfff700 1 -- 172.21.15.33:0/2825237514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd84013420 con 0x7fcd94102de0 2024-04-25T01:41:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.339+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9410ddd0 con 0x7fcd94102de0 2024-04-25T01:41:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.339+0000 7fcd7bfff700 1 -- 172.21.15.33:0/2825237514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd8401c630 con 0x7fcd94102de0 2024-04-25T01:41:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.339+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9410e390 con 0x7fcd94102de0 2024-04-25T01:41:31.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.340+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd941000f0 con 0x7fcd94102de0 2024-04-25T01:41:31.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.340+0000 7fcd7bfff700 1 -- 172.21.15.33:0/2825237514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fcd84013590 con 0x7fcd94102de0 2024-04-25T01:41:31.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.341+0000 7fcd7bfff700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd7c06e270 0x7fcd7c070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:31.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.341+0000 7fcd7bfff700 1 -- 172.21.15.33:0/2825237514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fcd8408efd0 con 0x7fcd94102de0 2024-04-25T01:41:31.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.341+0000 7fcd92ffd700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd7c06e270 0x7fcd7c070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:31.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.341+0000 7fcd92ffd700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd7c06e270 0x7fcd7c070720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcd88007fd0 tx=0x7fcd88009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:31.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.345+0000 7fcd7bfff700 1 -- 172.21.15.33:0/2825237514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcd840c3740 con 0x7fcd94102de0 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: Reconfiguring mds.cephfs.smithi138.rvxnby (monmap changed)... 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.rvxnby", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: Reconfiguring daemon mds.cephfs.smithi138.rvxnby on smithi138 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:31.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:31 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3772410093' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:31.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:31.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: pgmap v28: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:31.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:31.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:31.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: Reconfiguring mds.cephfs.smithi138.rvxnby (monmap changed)... 2024-04-25T01:41:31.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.rvxnby", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:41:31.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:31.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: Reconfiguring daemon mds.cephfs.smithi138.rvxnby on smithi138 2024-04-25T01:41:31.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:31.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:31.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:31 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3772410093' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:31.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.600+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcd9410efc0 con 0x7fcd94102de0 2024-04-25T01:41:31.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.601+0000 7fcd7bfff700 1 -- 172.21.15.33:0/2825237514 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1903 (secure 0 0 0) 0x7fcd8405bcf0 con 0x7fcd94102de0 2024-04-25T01:41:31.588 INFO:teuthology.orchestra.run.smithi033.stdout:e19 2024-04-25T01:41:31.588 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:41:31.588 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:41:31.588 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:41:31.588 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:41:31.588 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:41:31.588 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 19 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:41:31.589 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:41:31.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:41:31.591 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:41:31.591 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:41:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:41:31.591 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:41:31.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd7c06e270 msgr2=0x7fcd7c070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd7c06e270 0x7fcd7c070720 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcd88007fd0 tx=0x7fcd88009040 comp rx=0 tx=0).stop 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 msgr2=0x7fcd94113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 0x7fcd94113970 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcd84009150 tx=0x7fcd84009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 shutdown_connections 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd7c06e270 0x7fcd7c070720 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd94102de0 0x7fcd94113970 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 --2- 172.21.15.33:0/2825237514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd94103780 0x7fcd94113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 >> 172.21.15.33:0/2825237514 conn(0x7fcd940f8990 msgr2=0x7fcd94107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 shutdown_connections 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.603+0000 7fcd99cf0700 1 -- 172.21.15.33:0/2825237514 wait complete. 2024-04-25T01:41:31.592 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:41:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.707+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/1985637290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fde10 msgr2=0x7f8bcc0fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.707+0000 7f8bd1a6c700 1 --2- 172.21.15.33:0/1985637290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fe1e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc004940 tx=0x7f8bbc009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.707+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/1985637290 shutdown_connections 2024-04-25T01:41:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.707+0000 7f8bd1a6c700 1 --2- 172.21.15.33:0/1985637290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.707+0000 7f8bd1a6c700 1 --2- 172.21.15.33:0/1985637290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fe1e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.707+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/1985637290 >> 172.21.15.33:0/1985637290 conn(0x7f8bcc0f7860 msgr2=0x7f8bcc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.708+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/1985637290 shutdown_connections 2024-04-25T01:41:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.708+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/1985637290 wait complete. 2024-04-25T01:41:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.708+0000 7f8bd1a6c700 1 Processor -- start 2024-04-25T01:41:31.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.708+0000 7f8bd1a6c700 1 -- start start 2024-04-25T01:41:31.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.708+0000 7f8bd1a6c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:31.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.708+0000 7f8bd1a6c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:31.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.708+0000 7f8bd1a6c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0fdb90 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.708+0000 7f8bd1a6c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0fc6c0 con 0x7f8bcc0fde10 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bcaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bcaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50276/0 (socket says 172.21.15.33:50276) 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bcaffd700 1 -- 172.21.15.33:0/2016149553 learned_addr learned my addr 172.21.15.33:0/2016149553 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bcb7fe700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bcaffd700 1 -- 172.21.15.33:0/2016149553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 msgr2=0x7f8bcc0fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bcaffd700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bcaffd700 1 -- 172.21.15.33:0/2016149553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc005040 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bcaffd700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000c830 tx=0x7f8bc0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bb3fff700 1 -- 172.21.15.33:0/2016149553 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc00106f0 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc0fc9a0 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bb3fff700 1 -- 172.21.15.33:0/2016149553 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bc0010d20 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bb3fff700 1 -- 172.21.15.33:0/2016149553 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc00113a0 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.709+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc11a0d0 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.710+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc066b50 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.711+0000 7f8bb3fff700 1 -- 172.21.15.33:0/2016149553 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f8bc0012400 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.711+0000 7f8bb3fff700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e270 0x7f8bb4070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.711+0000 7f8bb3fff700 1 -- 172.21.15.33:0/2016149553 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f8bc008dc10 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.712+0000 7f8bcb7fe700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e270 0x7f8bb4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:31.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.712+0000 7f8bcb7fe700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e270 0x7f8bb4070720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009fd0 tx=0x7f8bbc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:31.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.716+0000 7f8bb3fff700 1 -- 172.21.15.33:0/2016149553 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8bc0056f10 con 0x7f8bcc0fe7b0 2024-04-25T01:41:31.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.944+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bcc0f94a0 con 0x7f8bb406e270 2024-04-25T01:41:31.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.945+0000 7f8bb3fff700 1 -- 172.21.15.33:0/2016149553 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f8bcc0f94a0 con 0x7f8bb406e270 2024-04-25T01:41:31.932 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:41:31.932 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:41:31.932 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:41:31.932 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:41:31.932 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:41:31.932 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:41:31.932 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T01:41:31.933 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:41:31.933 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "4/21 daemons upgraded", 2024-04-25T01:41:31.933 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mon daemons", 2024-04-25T01:41:31.933 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:41:31.933 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:41:31.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e270 msgr2=0x7f8bb4070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e270 0x7f8bb4070720 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009fd0 tx=0x7f8bbc009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:31.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 msgr2=0x7f8bcc0fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:31.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000c830 tx=0x7f8bc0004470 comp rx=0 tx=0).stop 2024-04-25T01:41:31.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 shutdown_connections 2024-04-25T01:41:31.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e270 0x7f8bb4070720 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 --2- 172.21.15.33:0/2016149553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:31.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 >> 172.21.15.33:0/2016149553 conn(0x7f8bcc0f7860 msgr2=0x7f8bcc0f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:31.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 shutdown_connections 2024-04-25T01:41:31.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:31.947+0000 7f8bd1a6c700 1 -- 172.21.15.33:0/2016149553 wait complete. 2024-04-25T01:41:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.057+0000 7f9741904700 1 -- 172.21.15.33:0/450295564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c102de0 msgr2=0x7f973c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.057+0000 7f9741904700 1 --2- 172.21.15.33:0/450295564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c102de0 0x7f973c1031b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9730004940 tx=0x7f9730009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:32.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.057+0000 7f9741904700 1 -- 172.21.15.33:0/450295564 shutdown_connections 2024-04-25T01:41:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.057+0000 7f9741904700 1 --2- 172.21.15.33:0/450295564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c103780 0x7f973c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.057+0000 7f9741904700 1 --2- 172.21.15.33:0/450295564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c102de0 0x7f973c1031b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.057+0000 7f9741904700 1 -- 172.21.15.33:0/450295564 >> 172.21.15.33:0/450295564 conn(0x7f973c0f8950 msgr2=0x7f973c0fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.057+0000 7f9741904700 1 -- 172.21.15.33:0/450295564 shutdown_connections 2024-04-25T01:41:32.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.057+0000 7f9741904700 1 -- 172.21.15.33:0/450295564 wait complete. 2024-04-25T01:41:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f9741904700 1 Processor -- start 2024-04-25T01:41:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f9741904700 1 -- start start 2024-04-25T01:41:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f9741904700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c102de0 0x7f973c113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f9741904700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c103780 0x7f973c1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:32.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f9741904700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c114700 con 0x7f973c103780 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f9741904700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973c10daf0 con 0x7f973c102de0 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f973a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c103780 0x7f973c1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f973a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c103780 0x7f973c1140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50288/0 (socket says 172.21.15.33:50288) 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f973a7fc700 1 -- 172.21.15.33:0/350385544 learned_addr learned my addr 172.21.15.33:0/350385544 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.058+0000 7f973affd700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c102de0 0x7f973c113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f973a7fc700 1 -- 172.21.15.33:0/350385544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c102de0 msgr2=0x7f973c113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f973a7fc700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c102de0 0x7f973c113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f973a7fc700 1 -- 172.21.15.33:0/350385544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730005040 con 0x7f973c103780 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f973affd700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c102de0 0x7f973c113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f973a7fc700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c103780 0x7f973c1140b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f972c002a50 tx=0x7f972c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f97237fe700 1 -- 172.21.15.33:0/350385544 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f972c007cc0 con 0x7f973c103780 2024-04-25T01:41:32.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f973c10ddd0 con 0x7f973c103780 2024-04-25T01:41:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f97237fe700 1 -- 172.21.15.33:0/350385544 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f972c00f040 con 0x7f973c103780 2024-04-25T01:41:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f97237fe700 1 -- 172.21.15.33:0/350385544 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f972c00d9c0 con 0x7f973c103780 2024-04-25T01:41:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.059+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f973c10e390 con 0x7f973c103780 2024-04-25T01:41:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.060+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f973c1000f0 con 0x7f973c103780 2024-04-25T01:41:32.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.061+0000 7f97237fe700 1 -- 172.21.15.33:0/350385544 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f972c009730 con 0x7f973c103780 2024-04-25T01:41:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.062+0000 7f97237fe700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f972406e260 0x7f9724070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:41:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.062+0000 7f97237fe700 1 -- 172.21.15.33:0/350385544 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f972c08d9d0 con 0x7f973c103780 2024-04-25T01:41:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.062+0000 7f973affd700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f972406e260 0x7f9724070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:41:32.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.062+0000 7f973affd700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f972406e260 0x7f9724070710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9730004560 tx=0x7f9730009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:41:32.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.067+0000 7f97237fe700 1 -- 172.21.15.33:0/350385544 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f972c05a710 con 0x7f973c103780 2024-04-25T01:41:32.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.372+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f973c10f210 con 0x7f973c103780 2024-04-25T01:41:32.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.373+0000 7f97237fe700 1 -- 172.21.15.33:0/350385544 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f972c05a2a0 con 0x7f973c103780 2024-04-25T01:41:32.360 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_OK 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.376+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f972406e260 msgr2=0x7f9724070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.376+0000 7f9741904700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f972406e260 0x7f9724070710 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9730004560 tx=0x7f9730009e30 comp rx=0 tx=0).stop 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.377+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c103780 msgr2=0x7f973c1140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.377+0000 7f9741904700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c103780 0x7f973c1140b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f972c002a50 tx=0x7f972c004600 comp rx=0 tx=0).stop 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.377+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 shutdown_connections 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.377+0000 7f9741904700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f972406e260 0x7f9724070710 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.377+0000 7f9741904700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f973c102de0 0x7f973c113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.377+0000 7f9741904700 1 --2- 172.21.15.33:0/350385544 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f973c103780 0x7f973c1140b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.377+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 >> 172.21.15.33:0/350385544 conn(0x7f973c0f8950 msgr2=0x7f973c1073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:41:32.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.377+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 shutdown_connections 2024-04-25T01:41:32.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:41:32.377+0000 7f9741904700 1 -- 172.21.15.33:0/350385544 wait complete. 2024-04-25T01:41:32.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2825237514' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:41:32.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:32.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:32.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:32.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:32.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:32.504 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:32.504 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all mon 2024-04-25T01:41:32.504 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:32.504 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi033"}]: dispatch 2024-04-25T01:41:32.504 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi033"}]': finished 2024-04-25T01:41:32.504 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]: dispatch 2024-04-25T01:41:32.504 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]': finished 2024-04-25T01:41:32.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2825237514' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:41:32.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:41:32.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:32.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:32.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:32.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:32.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:32.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all mon 2024-04-25T01:41:32.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:32.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi033"}]: dispatch 2024-04-25T01:41:32.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi033"}]': finished 2024-04-25T01:41:32.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]: dispatch 2024-04-25T01:41:32.529 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi138"}]': finished 2024-04-25T01:41:33.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:33 smithi138 ceph-mon[81304]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:33.528 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:33 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/350385544' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:41:33.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:33 smithi033 ceph-mon[111306]: pgmap v29: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:33.583 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:33 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/350385544' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:41:35.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:34 smithi033 ceph-mon[111306]: Upgrade: Updating crash.smithi033 (1/2) 2024-04-25T01:41:35.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:34 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:35.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:34 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:41:35.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:34 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:35.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:34 smithi033 ceph-mon[111306]: Deploying daemon crash.smithi033 on smithi033 2024-04-25T01:41:35.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:34 smithi138 ceph-mon[81304]: Upgrade: Updating crash.smithi033 (1/2) 2024-04-25T01:41:35.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:34 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:35.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:34 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi033", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:41:35.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:34 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:35.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:34 smithi138 ceph-mon[81304]: Deploying daemon crash.smithi033 on smithi033 2024-04-25T01:41:35.896 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:35 smithi033 ceph-mon[111306]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:36.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:35 smithi138 ceph-mon[81304]: pgmap v30: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:37.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:37 smithi033 ceph-mon[111306]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:37.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:37 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:37.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:37 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:38.028 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:37 smithi138 ceph-mon[81304]: pgmap v31: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:38.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:37 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:38.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:37 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:40.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:40 smithi138 ceph-mon[81304]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:40.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:40 smithi138 ceph-mon[81304]: Upgrade: Updating crash.smithi138 (2/2) 2024-04-25T01:41:40.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:40.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:41:40.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:40.276 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:40 smithi138 ceph-mon[81304]: Deploying daemon crash.smithi138 on smithi138 2024-04-25T01:41:40.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:40 smithi033 ceph-mon[111306]: pgmap v32: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:40.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:40 smithi033 ceph-mon[111306]: Upgrade: Updating crash.smithi138 (2/2) 2024-04-25T01:41:40.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:40.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi138", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T01:41:40.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:40.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:40 smithi033 ceph-mon[111306]: Deploying daemon crash.smithi138 on smithi138 2024-04-25T01:41:41.114 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:41:41.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:41:42.377 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:42 smithi138 ceph-mon[81304]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:42.503 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:42 smithi033 ceph-mon[111306]: pgmap v33: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:43.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:43 smithi033 ceph-mon[111306]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:43.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:43 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:43.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:43 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:43.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:43 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:43.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:43 smithi138 ceph-mon[81304]: pgmap v34: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:43.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:43 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:43.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:43 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:43.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:43 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:45.892 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:45 smithi033 ceph-mon[111306]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:45.892 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:45 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:45.892 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:45 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:45.920 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:45 smithi138 ceph-mon[81304]: pgmap v35: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:45.920 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:45 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:45.920 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:45 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:47.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:47 smithi138 ceph-mon[81304]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:47.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:47 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:47.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:47 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:48.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:47 smithi033 ceph-mon[111306]: pgmap v36: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:48.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:47 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:48.003 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:47 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:49.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:48 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:49.245 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:48 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:49.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:48 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:49.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:48 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:50.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:50 smithi033 ceph-mon[111306]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:50.253 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:50 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:50.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:50 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:50.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:50 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:50.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:50 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:50.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:50 smithi138 ceph-mon[81304]: pgmap v37: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:50.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:50 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:50.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:50 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:50.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:50 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:50.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:50 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:52.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:52 smithi033 ceph-mon[111306]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:52.278 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:52 smithi138 ceph-mon[81304]: pgmap v38: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:53.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:53.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:53.838 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all crash 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi033"}]: dispatch 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi033"}]': finished 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]: dispatch 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]': finished 2024-04-25T01:41:53.839 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-25T01:41:53.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-25T01:41:53.840 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:53 smithi033 ceph-mon[111306]: Upgrade: osd.0 is safe to restart 2024-04-25T01:41:54.081 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all crash 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:54.082 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi033"}]: dispatch 2024-04-25T01:41:54.083 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi033"}]': finished 2024-04-25T01:41:54.083 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]: dispatch 2024-04-25T01:41:54.083 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi138"}]': finished 2024-04-25T01:41:54.083 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-25T01:41:54.083 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-25T01:41:54.083 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:53 smithi138 ceph-mon[81304]: Upgrade: osd.0 is safe to restart 2024-04-25T01:41:55.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:55 smithi033 ceph-mon[111306]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:55.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:55 smithi033 ceph-mon[111306]: Upgrade: Updating osd.0 2024-04-25T01:41:55.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:55.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:41:55.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:55.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:55 smithi033 ceph-mon[111306]: Deploying daemon osd.0 on smithi033 2024-04-25T01:41:55.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:41:55.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:55 smithi138 ceph-mon[81304]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:55.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:55 smithi138 ceph-mon[81304]: Upgrade: Updating osd.0 2024-04-25T01:41:55.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:41:55.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T01:41:55.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:41:55.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:55 smithi138 ceph-mon[81304]: Deploying daemon osd.0 on smithi033 2024-04-25T01:41:55.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:41:57.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:57 smithi033 ceph-mon[111306]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:57.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:57 smithi138 ceph-mon[81304]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:41:58.063 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:57 smithi033 systemd[1]: Stopping Ceph osd.0 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:41:58.500 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:58 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[45018]: 2024-04-25T01:41:58.075+0000 7fced3f94700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:41:58.501 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:58 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[45018]: 2024-04-25T01:41:58.075+0000 7fced3f94700 -1 osd.0 48 *** Got signal Terminated *** 2024-04-25T01:41:58.501 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:58 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[45018]: 2024-04-25T01:41:58.075+0000 7fced3f94700 -1 osd.0 48 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:41:58.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:58 smithi033 ceph-mon[111306]: osd.0 marked itself down and dead 2024-04-25T01:41:58.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:58 smithi138 ceph-mon[81304]: osd.0 marked itself down and dead 2024-04-25T01:41:59.220 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:58 smithi033 podman[122190]: 2024-04-25 01:41:58.90365315 +0000 UTC m=+0.931844522 container died 7b84b6673e5122c7a15aeb65cbc00caa70d6ee595b93f217531e8175d1833f25 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:41:59.220 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:59 smithi033 podman[122190]: 2024-04-25 01:41:59.233276587 +0000 UTC m=+1.261467957 container cleanup 7b84b6673e5122c7a15aeb65cbc00caa70d6ee595b93f217531e8175d1833f25 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T01:41:59.480 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:59 smithi033 bash[122190]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0 2024-04-25T01:41:59.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:59 smithi033 podman[122208]: 2024-04-25 01:41:59.493180423 +0000 UTC m=+0.587122998 container remove 7b84b6673e5122c7a15aeb65cbc00caa70d6ee595b93f217531e8175d1833f25 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T01:41:59.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:59 smithi033 ceph-mon[111306]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:59.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:59 smithi033 ceph-mon[111306]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:41:59.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:41:59 smithi033 ceph-mon[111306]: osdmap e49: 6 total, 5 up, 6 in 2024-04-25T01:41:59.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:59 smithi138 ceph-mon[81304]: pgmap v42: 65 pgs: 65 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:41:59.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:59 smithi138 ceph-mon[81304]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:41:59.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:41:59 smithi138 ceph-mon[81304]: osdmap e49: 6 total, 5 up, 6 in 2024-04-25T01:42:00.209 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:59 smithi033 podman[122305]: 2024-04-25 01:41:59.822639796 +0000 UTC m=+0.021220356 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:00.209 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:41:59 smithi033 podman[122305]: 2024-04-25 01:41:59.927461363 +0000 UTC m=+0.126041912 container create 786bf710877cc4d1cd7632b964a763333c3eab924f215f79fb310328edc18698 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T01:42:00.209 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:00 smithi033 podman[122305]: 2024-04-25 01:42:00.155683242 +0000 UTC m=+0.354263792 container init 786bf710877cc4d1cd7632b964a763333c3eab924f215f79fb310328edc18698 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-25T01:42:00.210 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:00 smithi033 podman[122305]: 2024-04-25 01:42:00.159937796 +0000 UTC m=+0.358518351 container start 786bf710877cc4d1cd7632b964a763333c3eab924f215f79fb310328edc18698 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T01:42:00.210 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:00 smithi033 podman[122305]: 2024-04-25 01:42:00.222060514 +0000 UTC m=+0.420641073 container attach 786bf710877cc4d1cd7632b964a763333c3eab924f215f79fb310328edc18698 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T01:42:00.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:00 smithi033 ceph-mon[111306]: osdmap e50: 6 total, 5 up, 6 in 2024-04-25T01:42:00.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:00 smithi033 podman[122305]: 2024-04-25 01:42:00.475926342 +0000 UTC m=+0.674506901 container died 786bf710877cc4d1cd7632b964a763333c3eab924f215f79fb310328edc18698 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:42:00.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:00 smithi138 ceph-mon[81304]: osdmap e50: 6 total, 5 up, 6 in 2024-04-25T01:42:01.462 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:01 smithi033 podman[122332]: 2024-04-25 01:42:01.218554905 +0000 UTC m=+0.735350188 container remove 786bf710877cc4d1cd7632b964a763333c3eab924f215f79fb310328edc18698 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-25T01:42:01.462 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:01 smithi033 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-25T01:42:01.462 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:01 smithi033 systemd[1]: Stopped Ceph osd.0 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:42:01.462 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:01 smithi033 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.0.service: Consumed 1min 6.058s CPU time. 2024-04-25T01:42:01.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:01 smithi033 ceph-mon[111306]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:42:01.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:01 smithi033 systemd[1]: Starting Ceph osd.0 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:42:01.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:01 smithi033 podman[122436]: 2024-04-25 01:42:01.598736292 +0000 UTC m=+0.018467164 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:01.754 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:01 smithi033 podman[122436]: 2024-04-25 01:42:01.711905326 +0000 UTC m=+0.131636181 container create 1a1cdef993fced78eae97bfec89b98c7ec2421dacff5499bceeaf1f9a9ad0300 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T01:42:01.779 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:01 smithi138 ceph-mon[81304]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:42:02.254 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:01 smithi033 podman[122436]: 2024-04-25 01:42:01.973089205 +0000 UTC m=+0.392820062 container init 1a1cdef993fced78eae97bfec89b98c7ec2421dacff5499bceeaf1f9a9ad0300 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T01:42:02.254 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:01 smithi033 podman[122436]: 2024-04-25 01:42:01.976639407 +0000 UTC m=+0.396370264 container start 1a1cdef993fced78eae97bfec89b98c7ec2421dacff5499bceeaf1f9a9ad0300 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T01:42:02.254 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 podman[122436]: 2024-04-25 01:42:02.056289998 +0000 UTC m=+0.476020855 container attach 1a1cdef993fced78eae97bfec89b98c7ec2421dacff5499bceeaf1f9a9ad0300 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:42:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.490+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/2613529711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 msgr2=0x7fa4e8106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.490+0000 7fa4ee0b0700 1 --2- 172.21.15.33:0/2613529711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 0x7fa4e8106e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0004940 tx=0x7fa4d0009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.490+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/2613529711 shutdown_connections 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.490+0000 7fa4ee0b0700 1 --2- 172.21.15.33:0/2613529711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 0x7fa4e8106e10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.490+0000 7fa4ee0b0700 1 --2- 172.21.15.33:0/2613529711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e80fde30 0x7fa4e80fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.490+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/2613529711 >> 172.21.15.33:0/2613529711 conn(0x7fa4e80f7860 msgr2=0x7fa4e80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.490+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/2613529711 shutdown_connections 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.490+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/2613529711 wait complete. 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.491+0000 7fa4ee0b0700 1 Processor -- start 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.491+0000 7fa4ee0b0700 1 -- start start 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.491+0000 7fa4ee0b0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e80fde30 0x7fa4e80fbc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.491+0000 7fa4ee0b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 0x7fa4e80fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.491+0000 7fa4ee0b0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80fdbb0 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.491+0000 7fa4ee0b0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e80fc6e0 con 0x7fa4e80fde30 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.491+0000 7fa4e6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 0x7fa4e80fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4e6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 0x7fa4e80fc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59156/0 (socket says 172.21.15.33:59156) 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4e6ffd700 1 -- 172.21.15.33:0/4058399492 learned_addr learned my addr 172.21.15.33:0/4058399492 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4e77fe700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e80fde30 0x7fa4e80fbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4e6ffd700 1 -- 172.21.15.33:0/4058399492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e80fde30 msgr2=0x7fa4e80fbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4e6ffd700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e80fde30 0x7fa4e80fbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4e6ffd700 1 -- 172.21.15.33:0/4058399492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0005040 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4e77fe700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e80fde30 0x7fa4e80fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4e6ffd700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 0x7fa4e80fc1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009380 tx=0x7fa4d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4dffff700 1 -- 172.21.15.33:0/4058399492 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d000e040 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4dffff700 1 -- 172.21.15.33:0/4058399492 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4d0009af0 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e80fc9c0 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4dffff700 1 -- 172.21.15.33:0/4058399492 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4d001c630 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.492+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e8119ff0 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.493+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e8066b10 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.494+0000 7fa4dffff700 1 -- 172.21.15.33:0/4058399492 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa4d000bbc0 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.494+0000 7fa4dffff700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa4d406e270 0x7fa4d4070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.494+0000 7fa4dffff700 1 -- 172.21.15.33:0/4058399492 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fa4d008eed0 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.494+0000 7fa4e77fe700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa4d406e270 0x7fa4d4070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.495+0000 7fa4e77fe700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa4d406e270 0x7fa4d4070720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009fd0 tx=0x7fa4d800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.499+0000 7fa4dffff700 1 -- 172.21.15.33:0/4058399492 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa4d0058200 con 0x7fa4e80fe7d0 2024-04-25T01:42:02.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.733+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4e8109e60 con 0x7fa4d406e270 2024-04-25T01:42:02.722 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:02 smithi033 ceph-mon[111306]: Health check failed: Degraded data redundancy: 61/321 objects degraded (19.003%), 19 pgs degraded (PG_DEGRADED) 2024-04-25T01:42:02.722 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate[122451]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T01:42:02.722 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate[122451]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-25T01:42:02.722 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 bash[122436]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T01:42:02.722 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 bash[122436]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-25T01:42:02.722 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 bash[122436]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-25T01:42:02.722 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 bash[122436]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-25T01:42:02.722 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 bash[122436]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-25T01:42:02.722 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 bash[122436]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T01:42:02.722 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 bash[122436]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-25T01:42:02.723 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate[122451]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-25T01:42:02.723 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate[122451]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-25T01:42:02.723 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate[122451]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-25T01:42:02.723 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate[122451]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T01:42:02.723 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate[122451]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-25T01:42:02.723 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:02 smithi033 podman[122436]: 2024-04-25 01:42:02.465758976 +0000 UTC m=+0.885489824 container died 1a1cdef993fced78eae97bfec89b98c7ec2421dacff5499bceeaf1f9a9ad0300 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T01:42:02.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.735+0000 7fa4dffff700 1 -- 172.21.15.33:0/4058399492 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa4e8109e60 con 0x7fa4d406e270 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa4d406e270 msgr2=0x7fa4d4070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa4d406e270 0x7fa4d4070720 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8009fd0 tx=0x7fa4d800b040 comp rx=0 tx=0).stop 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 msgr2=0x7fa4e80fc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 0x7fa4e80fc1a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0009380 tx=0x7fa4d0009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 shutdown_connections 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa4d406e270 0x7fa4d4070720 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa4e80fde30 0x7fa4e80fbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 --2- 172.21.15.33:0/4058399492 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4e80fe7d0 0x7fa4e80fc1a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 >> 172.21.15.33:0/4058399492 conn(0x7fa4e80f7860 msgr2=0x7fa4e80f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 shutdown_connections 2024-04-25T01:42:02.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.737+0000 7fa4ee0b0700 1 -- 172.21.15.33:0/4058399492 wait complete. 2024-04-25T01:42:02.735 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:42:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/716647169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 msgr2=0x7fa2b00fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 --2- 172.21.15.33:0/716647169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 0x7fa2b00fbd10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa29c004940 tx=0x7fa29c009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/716647169 shutdown_connections 2024-04-25T01:42:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 --2- 172.21.15.33:0/716647169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 0x7fa2b00fbd10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 --2- 172.21.15.33:0/716647169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b0101d00 0x7fa2b01020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/716647169 >> 172.21.15.33:0/716647169 conn(0x7fa2b00f7870 msgr2=0x7fa2b00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/716647169 shutdown_connections 2024-04-25T01:42:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/716647169 wait complete. 2024-04-25T01:42:02.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 Processor -- start 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.843+0000 7fa2b6fd9700 1 -- start start 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2b6fd9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b0101d00 0x7fa2b00fd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2b6fd9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 0x7fa2b00fdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2b6fd9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00ff8d0 con 0x7fa2b01026a0 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2b6fd9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b00fe400 con 0x7fa2b0101d00 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 0x7fa2b00fdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2affff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 0x7fa2b00fdec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59172/0 (socket says 172.21.15.33:59172) 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2affff700 1 -- 172.21.15.33:0/1921170485 learned_addr learned my addr 172.21.15.33:0/1921170485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2b4d75700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b0101d00 0x7fa2b00fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2affff700 1 -- 172.21.15.33:0/1921170485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b0101d00 msgr2=0x7fa2b00fd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2affff700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b0101d00 0x7fa2b00fd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2affff700 1 -- 172.21.15.33:0/1921170485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c005040 con 0x7fa2b01026a0 2024-04-25T01:42:02.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2affff700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 0x7fa2b00fdec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009100 tx=0x7fa29c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2ad7fa700 1 -- 172.21.15.33:0/1921170485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29c00e040 con 0x7fa2b01026a0 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b00fe680 con 0x7fa2b01026a0 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2ad7fa700 1 -- 172.21.15.33:0/1921170485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa29c009af0 con 0x7fa2b01026a0 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.844+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b011a100 con 0x7fa2b01026a0 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.845+0000 7fa2ad7fa700 1 -- 172.21.15.33:0/1921170485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29c025400 con 0x7fa2b01026a0 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.845+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b00fd450 con 0x7fa2b01026a0 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.846+0000 7fa2ad7fa700 1 -- 172.21.15.33:0/1921170485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa29c00baf0 con 0x7fa2b01026a0 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.846+0000 7fa2ad7fa700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa2a006e270 0x7fa2a0070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.847+0000 7fa2ad7fa700 1 -- 172.21.15.33:0/1921170485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fa29c08ef10 con 0x7fa2b01026a0 2024-04-25T01:42:02.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.847+0000 7fa2b4d75700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa2a006e270 0x7fa2a0070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:02.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.847+0000 7fa2b4d75700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa2a006e270 0x7fa2a0070720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4004eb0 tx=0x7fa2a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:02.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:02.852+0000 7fa2ad7fa700 1 -- 172.21.15.33:0/1921170485 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa29c0581a0 con 0x7fa2b01026a0 2024-04-25T01:42:03.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:02 smithi138 ceph-mon[81304]: Health check failed: Degraded data redundancy: 61/321 objects degraded (19.003%), 19 pgs degraded (PG_DEGRADED) 2024-04-25T01:42:03.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.079+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2b0108de0 con 0x7fa2a006e270 2024-04-25T01:42:03.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.080+0000 7fa2ad7fa700 1 -- 172.21.15.33:0/1921170485 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fa2b0108de0 con 0x7fa2a006e270 2024-04-25T01:42:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa2a006e270 msgr2=0x7fa2a0070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa2a006e270 0x7fa2a0070720 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4004eb0 tx=0x7fa2a400b040 comp rx=0 tx=0).stop 2024-04-25T01:42:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 msgr2=0x7fa2b00fdec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 0x7fa2b00fdec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009100 tx=0x7fa29c009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 shutdown_connections 2024-04-25T01:42:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa2a006e270 0x7fa2a0070720 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa2b0101d00 0x7fa2b00fd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 --2- 172.21.15.33:0/1921170485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa2b01026a0 0x7fa2b00fdec0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 >> 172.21.15.33:0/1921170485 conn(0x7fa2b00f7870 msgr2=0x7fa2b00f8f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 shutdown_connections 2024-04-25T01:42:03.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.082+0000 7fa2b6fd9700 1 -- 172.21.15.33:0/1921170485 wait complete. 2024-04-25T01:42:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.190+0000 7f549d2c0700 1 -- 172.21.15.33:0/1841831615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498102de0 msgr2=0x7f54981031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.190+0000 7f549d2c0700 1 --2- 172.21.15.33:0/1841831615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498102de0 0x7f54981031b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5488004940 tx=0x7f5488009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.190+0000 7f549d2c0700 1 -- 172.21.15.33:0/1841831615 shutdown_connections 2024-04-25T01:42:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.190+0000 7f549d2c0700 1 --2- 172.21.15.33:0/1841831615 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5498103780 0x7f54980fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.190+0000 7f549d2c0700 1 --2- 172.21.15.33:0/1841831615 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498102de0 0x7f54981031b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.190+0000 7f549d2c0700 1 -- 172.21.15.33:0/1841831615 >> 172.21.15.33:0/1841831615 conn(0x7f54980f8990 msgr2=0x7f54980fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.190+0000 7f549d2c0700 1 -- 172.21.15.33:0/1841831615 shutdown_connections 2024-04-25T01:42:03.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.190+0000 7f549d2c0700 1 -- 172.21.15.33:0/1841831615 wait complete. 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.191+0000 7f549d2c0700 1 Processor -- start 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.191+0000 7f549d2c0700 1 -- start start 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f549d2c0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5498102de0 0x7f5498113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f549d2c0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498103780 0x7f5498113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f549d2c0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498114500 con 0x7f5498103780 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f549d2c0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549810db50 con 0x7f5498102de0 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f54967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498103780 0x7f5498113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f54967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498103780 0x7f5498113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59176/0 (socket says 172.21.15.33:59176) 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f54967fc700 1 -- 172.21.15.33:0/2893608516 learned_addr learned my addr 172.21.15.33:0/2893608516 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f5496ffd700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5498102de0 0x7f5498113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f54967fc700 1 -- 172.21.15.33:0/2893608516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5498102de0 msgr2=0x7f5498113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f54967fc700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5498102de0 0x7f5498113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f54967fc700 1 -- 172.21.15.33:0/2893608516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488005040 con 0x7f5498103780 2024-04-25T01:42:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f5496ffd700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5498102de0 0x7f5498113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:42:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.192+0000 7f54967fc700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498103780 0x7f5498113eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5480002a50 tx=0x7f5480004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.193+0000 7f548f7fe700 1 -- 172.21.15.33:0/2893608516 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5480007cc0 con 0x7f5498103780 2024-04-25T01:42:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.193+0000 7f548f7fe700 1 -- 172.21.15.33:0/2893608516 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f548000f040 con 0x7f5498103780 2024-04-25T01:42:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.193+0000 7f548f7fe700 1 -- 172.21.15.33:0/2893608516 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548000daf0 con 0x7f5498103780 2024-04-25T01:42:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.193+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549810de30 con 0x7f5498103780 2024-04-25T01:42:03.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.193+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549810e3a0 con 0x7f5498103780 2024-04-25T01:42:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.194+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54981000f0 con 0x7f5498103780 2024-04-25T01:42:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.194+0000 7f548f7fe700 1 -- 172.21.15.33:0/2893608516 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f5480009810 con 0x7f5498103780 2024-04-25T01:42:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.195+0000 7f548f7fe700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e270 0x7f5484070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.195+0000 7f548f7fe700 1 -- 172.21.15.33:0/2893608516 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f548008dad0 con 0x7f5498103780 2024-04-25T01:42:03.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.195+0000 7f5496ffd700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e270 0x7f5484070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.196+0000 7f5496ffd700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e270 0x7f5484070720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f548800b3b0 tx=0x7f5488009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.200+0000 7f548f7fe700 1 -- 172.21.15.33:0/2893608516 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f548005a810 con 0x7f5498103780 2024-04-25T01:42:03.366 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:03 smithi033 podman[122588]: 2024-04-25 01:42:03.160843682 +0000 UTC m=+0.689905099 container remove 1a1cdef993fced78eae97bfec89b98c7ec2421dacff5499bceeaf1f9a9ad0300 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-activate, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:42:03.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.425+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f549810f030 con 0x7f548406e270 2024-04-25T01:42:03.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.435+0000 7f548f7fe700 1 -- 172.21.15.33:0/2893608516 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f549810f030 con 0x7f548406e270 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (3m) 15s ago 14m 27.2M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (26s) 15s ago 14m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (21s) 17s ago 13m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (2m) 15s ago 14m 79.2M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (8m) 15s ago 8m 62.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (9m) 15s ago 9m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (9m) 17s ago 8m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (8m) 17s ago 8m 61.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (4m) 15s ago 15m 495M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (4m) 17s ago 13m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (92s) 15s ago 15m 42.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:42:03.424 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (67s) 17s ago 13m 33.9M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:42:03.425 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (3m) 15s ago 14m 16.0M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:42:03.425 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (3m) 17s ago 13m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:42:03.425 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (12m) 15s ago 12m 225M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 7b84b6673e51 2024-04-25T01:42:03.425 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (12m) 15s ago 12m 237M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:42:03.425 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (11m) 15s ago 11m 190M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:42:03.425 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (11m) 17s ago 11m 264M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:42:03.425 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (10m) 17s ago 10m 231M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:42:03.425 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (10m) 17s ago 10m 219M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:42:03.425 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (3m) 15s ago 13m 60.8M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e270 msgr2=0x7f5484070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e270 0x7f5484070720 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f548800b3b0 tx=0x7f5488009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498103780 msgr2=0x7f5498113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498103780 0x7f5498113eb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5480002a50 tx=0x7f5480004600 comp rx=0 tx=0).stop 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 shutdown_connections 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e270 0x7f5484070720 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5498102de0 0x7f5498113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 --2- 172.21.15.33:0/2893608516 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5498103780 0x7f5498113eb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 >> 172.21.15.33:0/2893608516 conn(0x7f54980f8990 msgr2=0x7f5498107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 shutdown_connections 2024-04-25T01:42:03.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.438+0000 7f549d2c0700 1 -- 172.21.15.33:0/2893608516 wait complete. 2024-04-25T01:42:03.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.543+0000 7f83c2b92700 1 -- 172.21.15.33:0/4186419743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fd0a0 msgr2=0x7f83bc109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.543+0000 7f83c2b92700 1 --2- 172.21.15.33:0/4186419743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fd0a0 0x7f83bc109350 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f83a8004940 tx=0x7f83a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:03.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.543+0000 7f83c2b92700 1 -- 172.21.15.33:0/4186419743 shutdown_connections 2024-04-25T01:42:03.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.543+0000 7f83c2b92700 1 --2- 172.21.15.33:0/4186419743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fd0a0 0x7f83bc109350 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.543+0000 7f83c2b92700 1 --2- 172.21.15.33:0/4186419743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83bc0fc790 0x7f83bc0fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.543+0000 7f83c2b92700 1 -- 172.21.15.33:0/4186419743 >> 172.21.15.33:0/4186419743 conn(0x7f83bc071d00 msgr2=0x7f83bc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.543+0000 7f83c2b92700 1 -- 172.21.15.33:0/4186419743 shutdown_connections 2024-04-25T01:42:03.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.543+0000 7f83c2b92700 1 -- 172.21.15.33:0/4186419743 wait complete. 2024-04-25T01:42:03.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.544+0000 7f83c2b92700 1 Processor -- start 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.544+0000 7f83c2b92700 1 -- start start 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.544+0000 7f83c2b92700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fc790 0x7f83bc115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c2b92700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83bc0fd0a0 0x7f83bc116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c2b92700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc116990 con 0x7f83bc0fc790 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c2b92700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc10fdb0 con 0x7f83bc0fd0a0 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c092e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fc790 0x7f83bc115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c092e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fc790 0x7f83bc115e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59192/0 (socket says 172.21.15.33:59192) 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c092e700 1 -- 172.21.15.33:0/3620441254 learned_addr learned my addr 172.21.15.33:0/3620441254 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83bbfff700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83bc0fd0a0 0x7f83bc116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c092e700 1 -- 172.21.15.33:0/3620441254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83bc0fd0a0 msgr2=0x7f83bc116340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c092e700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83bc0fd0a0 0x7f83bc116340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c092e700 1 -- 172.21.15.33:0/3620441254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a8005040 con 0x7f83bc0fc790 2024-04-25T01:42:03.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83c092e700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fc790 0x7f83bc115e00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f83b0002a50 tx=0x7f83b00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.545+0000 7f83b97fa700 1 -- 172.21.15.33:0/3620441254 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b0007cc0 con 0x7f83bc0fc790 2024-04-25T01:42:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.546+0000 7f83b97fa700 1 -- 172.21.15.33:0/3620441254 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83b000f040 con 0x7f83bc0fc790 2024-04-25T01:42:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.546+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc110090 con 0x7f83bc0fc790 2024-04-25T01:42:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.546+0000 7f83b97fa700 1 -- 172.21.15.33:0/3620441254 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b000da70 con 0x7f83bc0fc790 2024-04-25T01:42:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.546+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc1105d0 con 0x7f83bc0fc790 2024-04-25T01:42:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.547+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc0fc330 con 0x7f83bc0fc790 2024-04-25T01:42:03.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.548+0000 7f83b97fa700 1 -- 172.21.15.33:0/3620441254 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f83b000dc10 con 0x7f83bc0fc790 2024-04-25T01:42:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.548+0000 7f83b97fa700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83ac06e260 0x7f83ac070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.548+0000 7f83b97fa700 1 -- 172.21.15.33:0/3620441254 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f83b008d2a0 con 0x7f83bc0fc790 2024-04-25T01:42:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.548+0000 7f83bbfff700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83ac06e260 0x7f83ac070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.549+0000 7f83bbfff700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83ac06e260 0x7f83ac070710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f83bc1116a0 tx=0x7f83a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.553+0000 7f83b97fa700 1 -- 172.21.15.33:0/3620441254 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f83b005a060 con 0x7f83bc0fc790 2024-04-25T01:42:03.635 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:03 smithi033 ceph-mon[111306]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:03.635 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:03 smithi033 ceph-mon[111306]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:03.635 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:03 smithi033 ceph-mon[111306]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:03.635 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:03 smithi033 podman[122724]: 2024-04-25 01:42:03.283168079 +0000 UTC m=+0.019436618 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:03.635 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:03 smithi033 podman[122724]: 2024-04-25 01:42:03.416461088 +0000 UTC m=+0.152729628 container create 1da61cf4b50bba3c83539faf2ef9269d5baeb2d2db1b383c435eef01ad729796 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T01:42:03.635 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:03 smithi033 podman[122724]: 2024-04-25 01:42:03.648237417 +0000 UTC m=+0.384505957 container init 1da61cf4b50bba3c83539faf2ef9269d5baeb2d2db1b383c435eef01ad729796 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T01:42:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.878+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83bc04bed0 con 0x7f83bc0fc790 2024-04-25T01:42:03.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.878+0000 7f83b97fa700 1 -- 172.21.15.33:0/3620441254 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+735 (secure 0 0 0) 0x7f83b0059bf0 con 0x7f83bc0fc790 2024-04-25T01:42:03.866 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 9, 2024-04-25T01:42:03.867 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T01:42:03.868 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:42:03.868 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:42:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.880+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83ac06e260 msgr2=0x7f83ac070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.880+0000 7f83c2b92700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83ac06e260 0x7f83ac070710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f83bc1116a0 tx=0x7f83a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:03.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.880+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fc790 msgr2=0x7f83bc115e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.880+0000 7f83c2b92700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fc790 0x7f83bc115e00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f83b0002a50 tx=0x7f83b00046b0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.881+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 shutdown_connections 2024-04-25T01:42:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.881+0000 7f83c2b92700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83ac06e260 0x7f83ac070710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.881+0000 7f83c2b92700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83bc0fc790 0x7f83bc115e00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.881+0000 7f83c2b92700 1 --2- 172.21.15.33:0/3620441254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83bc0fd0a0 0x7f83bc116340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.881+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 >> 172.21.15.33:0/3620441254 conn(0x7f83bc071d00 msgr2=0x7f83bc0f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.881+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 shutdown_connections 2024-04-25T01:42:03.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.881+0000 7f83c2b92700 1 -- 172.21.15.33:0/3620441254 wait complete. 2024-04-25T01:42:03.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.991+0000 7f934394c700 1 -- 172.21.15.33:0/1307990559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 msgr2=0x7f933c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.991+0000 7f934394c700 1 --2- 172.21.15.33:0/1307990559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 0x7f933c103220 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9338004940 tx=0x7f9338009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:03.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.991+0000 7f934394c700 1 -- 172.21.15.33:0/1307990559 shutdown_connections 2024-04-25T01:42:03.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.991+0000 7f934394c700 1 --2- 172.21.15.33:0/1307990559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c065640 0x7f933c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.991+0000 7f934394c700 1 --2- 172.21.15.33:0/1307990559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 0x7f933c103220 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.991+0000 7f934394c700 1 -- 172.21.15.33:0/1307990559 >> 172.21.15.33:0/1307990559 conn(0x7f933c071d00 msgr2=0x7f933c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.991+0000 7f934394c700 1 -- 172.21.15.33:0/1307990559 shutdown_connections 2024-04-25T01:42:03.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.991+0000 7f934394c700 1 -- 172.21.15.33:0/1307990559 wait complete. 2024-04-25T01:42:03.980 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:03 smithi033 podman[122724]: 2024-04-25 01:42:03.651917965 +0000 UTC m=+0.388186511 container start 1da61cf4b50bba3c83539faf2ef9269d5baeb2d2db1b383c435eef01ad729796 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux ) 2024-04-25T01:42:03.980 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:03 smithi033 ceph-osd[122766]: -- 172.21.15.33:0/2545014591 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5638a38e9680 con 0x5638a38b2c00 2024-04-25T01:42:03.980 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:03 smithi033 bash[122724]: 1da61cf4b50bba3c83539faf2ef9269d5baeb2d2db1b383c435eef01ad729796 2024-04-25T01:42:03.980 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:03 smithi033 systemd[1]: Started Ceph osd.0 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.991+0000 7f934394c700 1 Processor -- start 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f934394c700 1 -- start start 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f934394c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c065640 0x7f933c115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f934394c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 0x7f933c1162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f934394c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c116900 con 0x7f933c102e50 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f934394c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933c10fcd0 con 0x7f933c065640 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f9340ee7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 0x7f933c1162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f93416e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c065640 0x7f933c115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f93416e8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c065640 0x7f933c115d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:54350/0 (socket says 172.21.15.33:54350) 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f93416e8700 1 -- 172.21.15.33:0/1477727247 learned_addr learned my addr 172.21.15.33:0/1477727247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.992+0000 7f9340ee7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 0x7f933c1162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59220/0 (socket says 172.21.15.33:59220) 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f9340ee7700 1 -- 172.21.15.33:0/1477727247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c065640 msgr2=0x7f933c115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:03.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f9340ee7700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c065640 0x7f933c115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f9340ee7700 1 -- 172.21.15.33:0/1477727247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338005040 con 0x7f933c102e50 2024-04-25T01:42:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f93416e8700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c065640 0x7f933c115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T01:42:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f9340ee7700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 0x7f933c1162b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f932c002820 tx=0x7f932c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f9331ffb700 1 -- 172.21.15.33:0/1477727247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f932c007cc0 con 0x7f933c102e50 2024-04-25T01:42:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f9331ffb700 1 -- 172.21.15.33:0/1477727247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f932c00f040 con 0x7f933c102e50 2024-04-25T01:42:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c10ffb0 con 0x7f933c102e50 2024-04-25T01:42:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f9331ffb700 1 -- 172.21.15.33:0/1477727247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f932c013920 con 0x7f933c102e50 2024-04-25T01:42:03.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.993+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933c110570 con 0x7f933c102e50 2024-04-25T01:42:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.994+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f933c106980 con 0x7f933c102e50 2024-04-25T01:42:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.995+0000 7f9331ffb700 1 -- 172.21.15.33:0/1477727247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f932c01c400 con 0x7f933c102e50 2024-04-25T01:42:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.996+0000 7f9331ffb700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932806e260 0x7f9328070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.996+0000 7f9331ffb700 1 -- 172.21.15.33:0/1477727247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f932c08d9e0 con 0x7f933c102e50 2024-04-25T01:42:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.996+0000 7f93416e8700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932806e260 0x7f9328070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:03.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:03.996+0000 7f93416e8700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932806e260 0x7f9328070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9338004f90 tx=0x7f9338009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:03.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.001+0000 7f9331ffb700 1 -- 172.21.15.33:0/1477727247 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f932c05a720 con 0x7f933c102e50 2024-04-25T01:42:04.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:03 smithi138 ceph-mon[81304]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:04.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:03 smithi138 ceph-mon[81304]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:04.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:03 smithi138 ceph-mon[81304]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:04.253 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.265+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f933c1113c0 con 0x7f933c102e50 2024-04-25T01:42:04.254 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:04 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[122751]: 2024-04-25T01:42:04.190+0000 7f8773f0b7c0 -1 Falling back to public interface 2024-04-25T01:42:04.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.265+0000 7f9331ffb700 1 -- 172.21.15.33:0/1477727247 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f932c05a2b0 con 0x7f933c102e50 2024-04-25T01:42:04.254 INFO:teuthology.orchestra.run.smithi033.stdout:e19 2024-04-25T01:42:04.254 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:42:04.254 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:42:04.254 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 19 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:42:04.255 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:42:04.256 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:42:04.257 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:42:04.257 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:42:04.257 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:42:04.257 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:42:04.257 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:42:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:42:04.257 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932806e260 msgr2=0x7f9328070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932806e260 0x7f9328070710 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9338004f90 tx=0x7f9338009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 msgr2=0x7f933c1162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 0x7f933c1162b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f932c002820 tx=0x7f932c004570 comp rx=0 tx=0).stop 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 shutdown_connections 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932806e260 0x7f9328070710 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f933c065640 0x7f933c115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 --2- 172.21.15.33:0/1477727247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f933c102e50 0x7f933c1162b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 >> 172.21.15.33:0/1477727247 conn(0x7f933c071d00 msgr2=0x7f933c0faea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 shutdown_connections 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.268+0000 7f934394c700 1 -- 172.21.15.33:0/1477727247 wait complete. 2024-04-25T01:42:04.258 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:42:04.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.381+0000 7f53d2afc700 1 -- 172.21.15.33:0/1850830577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 msgr2=0x7f53cc102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.381+0000 7f53d2afc700 1 --2- 172.21.15.33:0/1850830577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 0x7f53cc102160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f53c0004940 tx=0x7f53c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.381+0000 7f53d2afc700 1 -- 172.21.15.33:0/1850830577 shutdown_connections 2024-04-25T01:42:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.381+0000 7f53d2afc700 1 --2- 172.21.15.33:0/1850830577 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc0fbd40 0x7f53cc0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.381+0000 7f53d2afc700 1 --2- 172.21.15.33:0/1850830577 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 0x7f53cc102160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.381+0000 7f53d2afc700 1 -- 172.21.15.33:0/1850830577 >> 172.21.15.33:0/1850830577 conn(0x7f53cc071d00 msgr2=0x7f53cc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.381+0000 7f53d2afc700 1 -- 172.21.15.33:0/1850830577 shutdown_connections 2024-04-25T01:42:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.381+0000 7f53d2afc700 1 -- 172.21.15.33:0/1850830577 wait complete. 2024-04-25T01:42:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.382+0000 7f53d2afc700 1 Processor -- start 2024-04-25T01:42:04.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.382+0000 7f53d2afc700 1 -- start start 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.382+0000 7f53d2afc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc0fbd40 0x7f53cc102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.382+0000 7f53d2afc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 0x7f53cc103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.382+0000 7f53d2afc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc1035f0 con 0x7f53cc101d90 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.382+0000 7f53d2afc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc103730 con 0x7f53cc0fbd40 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.382+0000 7f53cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 0x7f53cc103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.382+0000 7f53cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 0x7f53cc103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59234/0 (socket says 172.21.15.33:59234) 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.382+0000 7f53cbfff700 1 -- 172.21.15.33:0/2640669863 learned_addr learned my addr 172.21.15.33:0/2640669863 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53d0898700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc0fbd40 0x7f53cc102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53cbfff700 1 -- 172.21.15.33:0/2640669863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc0fbd40 msgr2=0x7f53cc102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53cbfff700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc0fbd40 0x7f53cc102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53cbfff700 1 -- 172.21.15.33:0/2640669863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0005040 con 0x7f53cc101d90 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53cbfff700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 0x7f53cc103060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f53b800eb40 tx=0x7f53b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53c97fa700 1 -- 172.21.15.33:0/2640669863 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53b8007cc0 con 0x7f53cc101d90 2024-04-25T01:42:04.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53c97fa700 1 -- 172.21.15.33:0/2640669863 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53b800f040 con 0x7f53cc101d90 2024-04-25T01:42:04.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc103980 con 0x7f53cc101d90 2024-04-25T01:42:04.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53c97fa700 1 -- 172.21.15.33:0/2640669863 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53b800ca10 con 0x7f53cc101d90 2024-04-25T01:42:04.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.383+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc11a170 con 0x7f53cc101d90 2024-04-25T01:42:04.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.384+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc1063c0 con 0x7f53cc101d90 2024-04-25T01:42:04.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.385+0000 7f53c97fa700 1 -- 172.21.15.33:0/2640669863 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f53b800cb70 con 0x7f53cc101d90 2024-04-25T01:42:04.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.385+0000 7f53c97fa700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53bc06e260 0x7f53bc070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.385+0000 7f53c97fa700 1 -- 172.21.15.33:0/2640669863 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f53b808d7f0 con 0x7f53cc101d90 2024-04-25T01:42:04.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.386+0000 7f53d0898700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53bc06e260 0x7f53bc070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.386+0000 7f53d0898700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53bc06e260 0x7f53bc070710 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53c000b0d0 tx=0x7f53c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:04.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.390+0000 7f53c97fa700 1 -- 172.21.15.33:0/2640669863 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f53b8056b30 con 0x7f53cc101d90 2024-04-25T01:42:04.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.630+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53cc104280 con 0x7f53bc06e260 2024-04-25T01:42:04.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.632+0000 7f53c97fa700 1 -- 172.21.15.33:0/2640669863 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f53cc104280 con 0x7f53bc06e260 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "6/21 daemons upgraded", 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T01:42:04.620 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:42:04.621 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:42:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53bc06e260 msgr2=0x7f53bc070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53bc06e260 0x7f53bc070710 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53c000b0d0 tx=0x7f53c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 msgr2=0x7f53cc103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 0x7f53cc103060 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f53b800eb40 tx=0x7f53b800b040 comp rx=0 tx=0).stop 2024-04-25T01:42:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 shutdown_connections 2024-04-25T01:42:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53bc06e260 0x7f53bc070710 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53cc0fbd40 0x7f53cc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 --2- 172.21.15.33:0/2640669863 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53cc101d90 0x7f53cc103060 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 >> 172.21.15.33:0/2640669863 conn(0x7f53cc071d00 msgr2=0x7f53cc0fb100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 shutdown_connections 2024-04-25T01:42:04.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.634+0000 7f53d2afc700 1 -- 172.21.15.33:0/2640669863 wait complete. 2024-04-25T01:42:04.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.739+0000 7f0fe82da700 1 -- 172.21.15.33:0/3649687360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0065640 msgr2=0x7f0fe0065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.739+0000 7f0fe82da700 1 --2- 172.21.15.33:0/3649687360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0065640 0x7f0fe0065ab0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0004940 tx=0x7f0fd0009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:04.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.740+0000 7f0fe82da700 1 -- 172.21.15.33:0/3649687360 shutdown_connections 2024-04-25T01:42:04.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.740+0000 7f0fe82da700 1 --2- 172.21.15.33:0/3649687360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0065640 0x7f0fe0065ab0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.740+0000 7f0fe82da700 1 --2- 172.21.15.33:0/3649687360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe0102e50 0x7f0fe0103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.740+0000 7f0fe82da700 1 -- 172.21.15.33:0/3649687360 >> 172.21.15.33:0/3649687360 conn(0x7f0fe0071d00 msgr2=0x7f0fe0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:04.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.740+0000 7f0fe82da700 1 -- 172.21.15.33:0/3649687360 shutdown_connections 2024-04-25T01:42:04.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.740+0000 7f0fe82da700 1 -- 172.21.15.33:0/3649687360 wait complete. 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.740+0000 7f0fe82da700 1 Processor -- start 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.740+0000 7f0fe82da700 1 -- start start 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.741+0000 7f0fe82da700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0065640 0x7f0fe0113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.741+0000 7f0fe82da700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe0102e50 0x7f0fe0114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.741+0000 7f0fe82da700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe0114660 con 0x7f0fe0065640 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.741+0000 7f0fe82da700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe010da50 con 0x7f0fe0102e50 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.741+0000 7f0fe5875700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe0102e50 0x7f0fe0114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.741+0000 7f0fe5875700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe0102e50 0x7f0fe0114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:54388/0 (socket says 172.21.15.33:54388) 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.741+0000 7f0fe5875700 1 -- 172.21.15.33:0/2641105081 learned_addr learned my addr 172.21.15.33:0/2641105081 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.741+0000 7f0fe6076700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0065640 0x7f0fe0113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.742+0000 7f0fe5875700 1 -- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0065640 msgr2=0x7f0fe0113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.742+0000 7f0fe5875700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0065640 0x7f0fe0113ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.742+0000 7f0fe5875700 1 -- 172.21.15.33:0/2641105081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd0005040 con 0x7f0fe0102e50 2024-04-25T01:42:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.742+0000 7f0fe5875700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe0102e50 0x7f0fe0114010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b8b0 tx=0x7f0fd0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.742+0000 7f0fd6ffd700 1 -- 172.21.15.33:0/2641105081 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fd000e040 con 0x7f0fe0102e50 2024-04-25T01:42:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.742+0000 7f0fd6ffd700 1 -- 172.21.15.33:0/2641105081 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fd0004290 con 0x7f0fe0102e50 2024-04-25T01:42:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.742+0000 7f0fd6ffd700 1 -- 172.21.15.33:0/2641105081 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fd001c3f0 con 0x7f0fe0102e50 2024-04-25T01:42:04.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.742+0000 7f0fe82da700 1 -- 172.21.15.33:0/2641105081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe010dcd0 con 0x7f0fe0102e50 2024-04-25T01:42:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.742+0000 7f0fe82da700 1 -- 172.21.15.33:0/2641105081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe010e1d0 con 0x7f0fe0102e50 2024-04-25T01:42:04.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.744+0000 7f0fd4ff9700 1 -- 172.21.15.33:0/2641105081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe00fe540 con 0x7f0fe0102e50 2024-04-25T01:42:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.745+0000 7f0fd6ffd700 1 -- 172.21.15.33:0/2641105081 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 40) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0fd000baf0 con 0x7f0fe0102e50 2024-04-25T01:42:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.745+0000 7f0fd6ffd700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fcc06e260 0x7f0fcc070710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:04.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.745+0000 7f0fd6ffd700 1 -- 172.21.15.33:0/2641105081 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f0fd008efd0 con 0x7f0fe0102e50 2024-04-25T01:42:04.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.746+0000 7f0fe6076700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fcc06e260 0x7f0fcc070710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:04.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.746+0000 7f0fe6076700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fcc06e260 0x7f0fcc070710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009fd0 tx=0x7f0fdc009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:04.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:04.750+0000 7f0fd6ffd700 1 -- 172.21.15.33:0/2641105081 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0fd005bdb0 con 0x7f0fe0102e50 2024-04-25T01:42:05.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:04 smithi033 ceph-mon[111306]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:05.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:04 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:05.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:04 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:05.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:04 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:05.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:04 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3620441254' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:05.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:04 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1477727247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:42:05.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.061+0000 7f0fd4ff9700 1 -- 172.21.15.33:0/2641105081 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0fe010f120 con 0x7f0fe0102e50 2024-04-25T01:42:05.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.062+0000 7f0fd6ffd700 1 -- 172.21.15.33:0/2641105081 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1328 (secure 0 0 0) 0x7f0fd005b940 con 0x7f0fe0102e50 2024-04-25T01:42:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 61/321 objects degraded (19.003%), 19 pgs degraded 2024-04-25T01:42:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-25T01:42:05.050 INFO:teuthology.orchestra.run.smithi033.stdout: osd.0 (root=default,host=smithi033) is down 2024-04-25T01:42:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 61/321 objects degraded (19.003%), 19 pgs degraded 2024-04-25T01:42:05.050 INFO:teuthology.orchestra.run.smithi033.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-25T01:42:05.051 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-04-25T01:42:05.052 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-25T01:42:05.052 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-25T01:42:05.052 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-25T01:42:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.064+0000 7f0fd4ff9700 1 -- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fcc06e260 msgr2=0x7f0fcc070710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.064+0000 7f0fd4ff9700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fcc06e260 0x7f0fcc070710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc009fd0 tx=0x7f0fdc009f60 comp rx=0 tx=0).stop 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.065+0000 7f0fd4ff9700 1 -- 172.21.15.33:0/2641105081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe0102e50 msgr2=0x7f0fe0114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.065+0000 7f0fd4ff9700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe0102e50 0x7f0fe0114010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0fd000b8b0 tx=0x7f0fd0009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.065+0000 7f0fd4ff9700 1 -- 172.21.15.33:0/2641105081 shutdown_connections 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.065+0000 7f0fd4ff9700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fcc06e260 0x7f0fcc070710 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.065+0000 7f0fd4ff9700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0fe0065640 0x7f0fe0113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.065+0000 7f0fd4ff9700 1 --2- 172.21.15.33:0/2641105081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0fe0102e50 0x7f0fe0114010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.065+0000 7f0fd4ff9700 1 -- 172.21.15.33:0/2641105081 >> 172.21.15.33:0/2641105081 conn(0x7f0fe0071d00 msgr2=0x7f0fe00fae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.065+0000 7f0fd4ff9700 1 -- 172.21.15.33:0/2641105081 shutdown_connections 2024-04-25T01:42:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:05.065+0000 7f0fd4ff9700 1 -- 172.21.15.33:0/2641105081 wait complete. 2024-04-25T01:42:05.133 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:04 smithi138 ceph-mon[81304]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:05.133 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:04 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:05.133 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:04 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:05.133 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:04 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:05.133 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:04 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3620441254' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:05.133 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:04 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1477727247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:42:06.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:05 smithi033 ceph-mon[111306]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:06.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:05 smithi033 ceph-mon[111306]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:06.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:05 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2641105081' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:42:06.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:05 smithi138 ceph-mon[81304]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:06.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:05 smithi138 ceph-mon[81304]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:06.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:05 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2641105081' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:42:08.100 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:07 smithi033 ceph-mon[111306]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:08.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:07 smithi138 ceph-mon[81304]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:09.504 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:09 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[122751]: 2024-04-25T01:42:09.106+0000 7f8773f0b7c0 -1 osd.0 48 log_to_monitors true 2024-04-25T01:42:10.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:09 smithi033 ceph-mon[111306]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:10.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:09 smithi033 ceph-mon[111306]: from='osd.0 [v2:172.21.15.33:6802/3103417098,v1:172.21.15.33:6803/3103417098]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:42:10.255 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:10.255 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:10.255 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 01:42:09 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[122751]: 2024-04-25T01:42:09.865+0000 7f8765363700 -1 osd.0 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:42:10.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:09 smithi138 ceph-mon[81304]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:10.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:09 smithi138 ceph-mon[81304]: from='osd.0 [v2:172.21.15.33:6802/3103417098,v1:172.21.15.33:6803/3103417098]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T01:42:10.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:10.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:11.192 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:10 smithi033 ceph-mon[111306]: from='osd.0 [v2:172.21.15.33:6802/3103417098,v1:172.21.15.33:6803/3103417098]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:42:11.192 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:10 smithi033 ceph-mon[111306]: osdmap e51: 6 total, 5 up, 6 in 2024-04-25T01:42:11.193 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:10 smithi033 ceph-mon[111306]: from='osd.0 [v2:172.21.15.33:6802/3103417098,v1:172.21.15.33:6803/3103417098]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:42:11.193 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:11.193 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:42:11.193 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:11.193 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:11.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:10 smithi138 ceph-mon[81304]: from='osd.0 [v2:172.21.15.33:6802/3103417098,v1:172.21.15.33:6803/3103417098]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T01:42:11.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:10 smithi138 ceph-mon[81304]: osdmap e51: 6 total, 5 up, 6 in 2024-04-25T01:42:11.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:10 smithi138 ceph-mon[81304]: from='osd.0 [v2:172.21.15.33:6802/3103417098,v1:172.21.15.33:6803/3103417098]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:42:11.280 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:11.280 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:42:11.280 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:11.280 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:12.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:11 smithi033 ceph-mon[111306]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:12.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:11 smithi033 ceph-mon[111306]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:42:12.254 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:11 smithi033 ceph-mon[111306]: osd.0 [v2:172.21.15.33:6802/3103417098,v1:172.21.15.33:6803/3103417098] boot 2024-04-25T01:42:12.255 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:11 smithi033 ceph-mon[111306]: osdmap e52: 6 total, 6 up, 6 in 2024-04-25T01:42:12.255 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:42:12.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:11 smithi138 ceph-mon[81304]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2 op/s; 61/321 objects degraded (19.003%) 2024-04-25T01:42:12.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:11 smithi138 ceph-mon[81304]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:42:12.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:11 smithi138 ceph-mon[81304]: osd.0 [v2:172.21.15.33:6802/3103417098,v1:172.21.15.33:6803/3103417098] boot 2024-04-25T01:42:12.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:11 smithi138 ceph-mon[81304]: osdmap e52: 6 total, 6 up, 6 in 2024-04-25T01:42:12.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T01:42:13.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:12 smithi033 ceph-mon[111306]: osdmap e53: 6 total, 6 up, 6 in 2024-04-25T01:42:13.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:12 smithi138 ceph-mon[81304]: osdmap e53: 6 total, 6 up, 6 in 2024-04-25T01:42:14.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:13 smithi033 ceph-mon[111306]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s; 11/321 objects degraded (3.427%) 2024-04-25T01:42:14.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:13 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 11/321 objects degraded (3.427%), 3 pgs degraded (PG_DEGRADED) 2024-04-25T01:42:14.170 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:13 smithi033 ceph-mon[111306]: mgrmap e41: smithi033.asemba(active, since 92s), standbys: smithi138.mmfhyi 2024-04-25T01:42:14.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:13 smithi138 ceph-mon[81304]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s; 11/321 objects degraded (3.427%) 2024-04-25T01:42:14.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:13 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 11/321 objects degraded (3.427%), 3 pgs degraded (PG_DEGRADED) 2024-04-25T01:42:14.279 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:13 smithi138 ceph-mon[81304]: mgrmap e41: smithi033.asemba(active, since 92s), standbys: smithi138.mmfhyi 2024-04-25T01:42:16.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: pgmap v55: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s; 11/321 objects degraded (3.427%) 2024-04-25T01:42:16.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:16.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:16.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:16.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:16.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:16.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:16.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:16.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:16.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:16.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T01:42:16.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T01:42:16.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:15 smithi033 ceph-mon[111306]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-25T01:42:16.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: pgmap v55: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s; 11/321 objects degraded (3.427%) 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T01:42:16.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:15 smithi138 ceph-mon[81304]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-25T01:42:17.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:16 smithi033 ceph-mon[111306]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/321 objects degraded (3.427%), 3 pgs degraded) 2024-04-25T01:42:17.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:16 smithi033 ceph-mon[111306]: Cluster is now healthy 2024-04-25T01:42:17.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:16 smithi138 ceph-mon[81304]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 11/321 objects degraded (3.427%), 3 pgs degraded) 2024-04-25T01:42:17.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:16 smithi138 ceph-mon[81304]: Cluster is now healthy 2024-04-25T01:42:18.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:17 smithi033 ceph-mon[111306]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.5 KiB/s rd, 963 B/s wr, 7 op/s 2024-04-25T01:42:18.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:17 smithi138 ceph-mon[81304]: pgmap v56: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 3.5 KiB/s rd, 963 B/s wr, 7 op/s 2024-04-25T01:42:20.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:19 smithi033 ceph-mon[111306]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T01:42:20.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:19 smithi138 ceph-mon[81304]: pgmap v57: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 5 op/s 2024-04-25T01:42:22.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:21 smithi033 ceph-mon[111306]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 655 B/s wr, 5 op/s 2024-04-25T01:42:22.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:21 smithi138 ceph-mon[81304]: pgmap v58: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.3 KiB/s rd, 655 B/s wr, 5 op/s 2024-04-25T01:42:24.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:23 smithi033 ceph-mon[111306]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T01:42:24.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:23 smithi138 ceph-mon[81304]: pgmap v59: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T01:42:26.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:25 smithi033 ceph-mon[111306]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:42:26.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:26.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:42:26.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:26.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:25 smithi138 ceph-mon[81304]: pgmap v60: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:42:26.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:26.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:42:26.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:28.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:27 smithi033 ceph-mon[111306]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:42:28.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:27 smithi138 ceph-mon[81304]: pgmap v61: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:42:30.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:29 smithi033 ceph-mon[111306]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:42:30.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:29 smithi138 ceph-mon[81304]: pgmap v62: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:42:31.004 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:30 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T01:42:31.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:30 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T01:42:31.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:30 smithi033 ceph-mon[111306]: Upgrade: osd.1 is safe to restart 2024-04-25T01:42:31.029 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:30 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T01:42:31.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:30 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T01:42:31.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:30 smithi138 ceph-mon[81304]: Upgrade: osd.1 is safe to restart 2024-04-25T01:42:31.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:31 smithi033 ceph-mon[111306]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:42:31.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:31.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:42:31.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:32.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:31 smithi138 ceph-mon[81304]: pgmap v63: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:42:32.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:32.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T01:42:32.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:32.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:32 smithi033 ceph-mon[111306]: Upgrade: Updating osd.1 2024-04-25T01:42:32.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:32 smithi033 ceph-mon[111306]: Deploying daemon osd.1 on smithi033 2024-04-25T01:42:33.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:32 smithi138 ceph-mon[81304]: Upgrade: Updating osd.1 2024-04-25T01:42:33.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:32 smithi138 ceph-mon[81304]: Deploying daemon osd.1 on smithi033 2024-04-25T01:42:34.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:33 smithi033 ceph-mon[111306]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:42:34.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:33 smithi138 ceph-mon[81304]: pgmap v64: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:42:35.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.174+0000 7f25bd94c700 1 -- 172.21.15.33:0/2113608393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 msgr2=0x7f25b8101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.174+0000 7f25bd94c700 1 --2- 172.21.15.33:0/2113608393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 0x7f25b8101a70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f25ac004940 tx=0x7f25ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.174+0000 7f25bd94c700 1 -- 172.21.15.33:0/2113608393 shutdown_connections 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.174+0000 7f25bd94c700 1 --2- 172.21.15.33:0/2113608393 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 0x7f25b8101a70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.174+0000 7f25bd94c700 1 --2- 172.21.15.33:0/2113608393 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fb7f0 0x7f25b80fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.174+0000 7f25bd94c700 1 -- 172.21.15.33:0/2113608393 >> 172.21.15.33:0/2113608393 conn(0x7f25b8071460 msgr2=0x7f25b8071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.174+0000 7f25bd94c700 1 -- 172.21.15.33:0/2113608393 shutdown_connections 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.174+0000 7f25bd94c700 1 -- 172.21.15.33:0/2113608393 wait complete. 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.175+0000 7f25bd94c700 1 Processor -- start 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.175+0000 7f25bd94c700 1 -- start start 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.175+0000 7f25bd94c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fb7f0 0x7f25b81139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.175+0000 7f25bd94c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 0x7f25b8113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.175+0000 7f25bd94c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b8114530 con 0x7f25b80fc190 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.175+0000 7f25bd94c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b810db50 con 0x7f25b80fb7f0 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.175+0000 7f25b67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 0x7f25b8113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.175+0000 7f25b67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 0x7f25b8113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59460/0 (socket says 172.21.15.33:59460) 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.175+0000 7f25b67fc700 1 -- 172.21.15.33:0/1535191669 learned_addr learned my addr 172.21.15.33:0/1535191669 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f25b6ffd700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fb7f0 0x7f25b81139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f25b67fc700 1 -- 172.21.15.33:0/1535191669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fb7f0 msgr2=0x7f25b81139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f25b67fc700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fb7f0 0x7f25b81139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f25b67fc700 1 -- 172.21.15.33:0/1535191669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac005040 con 0x7f25b80fc190 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f25b6ffd700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fb7f0 0x7f25b81139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f25b67fc700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 0x7f25b8113ee0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009030 tx=0x7f25ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f259f7fe700 1 -- 172.21.15.33:0/1535191669 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25ac00e040 con 0x7f25b80fc190 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f259f7fe700 1 -- 172.21.15.33:0/1535191669 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25ac004290 con 0x7f25b80fc190 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b810ddd0 con 0x7f25b80fc190 2024-04-25T01:42:35.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f259f7fe700 1 -- 172.21.15.33:0/1535191669 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25ac01c3f0 con 0x7f25b80fc190 2024-04-25T01:42:35.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.176+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b810e310 con 0x7f25b80fc190 2024-04-25T01:42:35.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.177+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b8065f30 con 0x7f25b80fc190 2024-04-25T01:42:35.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.178+0000 7f259f7fe700 1 -- 172.21.15.33:0/1535191669 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f25ac013420 con 0x7f25b80fc190 2024-04-25T01:42:35.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.178+0000 7f259f7fe700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f25a006e140 0x7f25a00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:35.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.178+0000 7f25b6ffd700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f25a006e140 0x7f25a00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:35.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.178+0000 7f259f7fe700 1 -- 172.21.15.33:0/1535191669 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f25ac08ec20 con 0x7f25b80fc190 2024-04-25T01:42:35.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.179+0000 7f25b6ffd700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f25a006e140 0x7f25a00705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009fd0 tx=0x7f25a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:35.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.183+0000 7f259f7fe700 1 -- 172.21.15.33:0/1535191669 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f25ac058170 con 0x7f25b80fc190 2024-04-25T01:42:35.173 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:34 smithi033 systemd[1]: Stopping Ceph osd.1 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:42:35.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.413+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25b8109f90 con 0x7f25a006e140 2024-04-25T01:42:35.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.414+0000 7f259f7fe700 1 -- 172.21.15.33:0/1535191669 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f25b8109f90 con 0x7f25a006e140 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.416+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f25a006e140 msgr2=0x7f25a00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.416+0000 7f25bd94c700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f25a006e140 0x7f25a00705f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25a8009fd0 tx=0x7f25a800b040 comp rx=0 tx=0).stop 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.416+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 msgr2=0x7f25b8113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.416+0000 7f25bd94c700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 0x7f25b8113ee0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f25ac009030 tx=0x7f25ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.417+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 shutdown_connections 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.417+0000 7f25bd94c700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f25a006e140 0x7f25a00705f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.417+0000 7f25bd94c700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b80fb7f0 0x7f25b81139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.417+0000 7f25bd94c700 1 --2- 172.21.15.33:0/1535191669 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b80fc190 0x7f25b8113ee0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.417+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 >> 172.21.15.33:0/1535191669 conn(0x7f25b8071460 msgr2=0x7f25b8100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.417+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 shutdown_connections 2024-04-25T01:42:35.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.417+0000 7f25bd94c700 1 -- 172.21.15.33:0/1535191669 wait complete. 2024-04-25T01:42:35.416 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:42:35.505 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:35 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[53014]: 2024-04-25T01:42:35.203+0000 7fd36e986700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:42:35.505 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:35 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[53014]: 2024-04-25T01:42:35.203+0000 7fd36e986700 -1 osd.1 53 *** Got signal Terminated *** 2024-04-25T01:42:35.505 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:35 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[53014]: 2024-04-25T01:42:35.203+0000 7fd36e986700 -1 osd.1 53 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:42:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 -- 172.21.15.33:0/1648429425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fde70 msgr2=0x7f52dc0fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 --2- 172.21.15.33:0/1648429425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fde70 0x7f52dc0fe240 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f52cc004940 tx=0x7f52cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 -- 172.21.15.33:0/1648429425 shutdown_connections 2024-04-25T01:42:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 --2- 172.21.15.33:0/1648429425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52dc0fe810 0x7f52dc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 --2- 172.21.15.33:0/1648429425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fde70 0x7f52dc0fe240 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 -- 172.21.15.33:0/1648429425 >> 172.21.15.33:0/1648429425 conn(0x7f52dc0f7860 msgr2=0x7f52dc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 -- 172.21.15.33:0/1648429425 shutdown_connections 2024-04-25T01:42:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 -- 172.21.15.33:0/1648429425 wait complete. 2024-04-25T01:42:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 Processor -- start 2024-04-25T01:42:35.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.522+0000 7f52e23ab700 1 -- start start 2024-04-25T01:42:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52e23ab700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52dc0fde70 0x7f52dc113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52e23ab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fe810 0x7f52dc113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52e23ab700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc1144e0 con 0x7f52dc0fe810 2024-04-25T01:42:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52e23ab700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52dc10dae0 con 0x7f52dc0fde70 2024-04-25T01:42:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fe810 0x7f52dc113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fe810 0x7f52dc113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59476/0 (socket says 172.21.15.33:59476) 2024-04-25T01:42:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52db7fe700 1 -- 172.21.15.33:0/1033454637 learned_addr learned my addr 172.21.15.33:0/1033454637 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52dbfff700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52dc0fde70 0x7f52dc113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:35.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52db7fe700 1 -- 172.21.15.33:0/1033454637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52dc0fde70 msgr2=0x7f52dc113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52db7fe700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52dc0fde70 0x7f52dc113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52db7fe700 1 -- 172.21.15.33:0/1033454637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc005040 con 0x7f52dc0fe810 2024-04-25T01:42:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52db7fe700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fe810 0x7f52dc113e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f52d000c830 tx=0x7f52d0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.523+0000 7f52d8ff9700 1 -- 172.21.15.33:0/1033454637 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d00106f0 con 0x7f52dc0fe810 2024-04-25T01:42:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.524+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52dc10ddc0 con 0x7f52dc0fe810 2024-04-25T01:42:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.524+0000 7f52d8ff9700 1 -- 172.21.15.33:0/1033454637 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52d0010d20 con 0x7f52dc0fe810 2024-04-25T01:42:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.524+0000 7f52d8ff9700 1 -- 172.21.15.33:0/1033454637 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d00113a0 con 0x7f52dc0fe810 2024-04-25T01:42:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.524+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52dc10e300 con 0x7f52dc0fe810 2024-04-25T01:42:35.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.525+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52dc066af0 con 0x7f52dc0fe810 2024-04-25T01:42:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.525+0000 7f52d8ff9700 1 -- 172.21.15.33:0/1033454637 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f52d0012400 con 0x7f52dc0fe810 2024-04-25T01:42:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.526+0000 7f52d8ff9700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f52c406e140 0x7f52c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.526+0000 7f52d8ff9700 1 -- 172.21.15.33:0/1033454637 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f52d008da60 con 0x7f52dc0fe810 2024-04-25T01:42:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.526+0000 7f52dbfff700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f52c406e140 0x7f52c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:35.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.526+0000 7f52dbfff700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f52c406e140 0x7f52c40705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f52cc004f90 tx=0x7f52cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:35.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.530+0000 7f52d8ff9700 1 -- 172.21.15.33:0/1033454637 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f52d005a9a0 con 0x7f52dc0fe810 2024-04-25T01:42:35.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.773+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52dc109e30 con 0x7f52c406e140 2024-04-25T01:42:35.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:35 smithi033 ceph-mon[111306]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:42:35.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:35 smithi033 ceph-mon[111306]: osd.1 marked itself down and dead 2024-04-25T01:42:35.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.774+0000 7f52d8ff9700 1 -- 172.21.15.33:0/1033454637 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f52dc109e30 con 0x7f52c406e140 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f52c406e140 msgr2=0x7f52c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f52c406e140 0x7f52c40705f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f52cc004f90 tx=0x7f52cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fe810 msgr2=0x7f52dc113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fe810 0x7f52dc113e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f52d000c830 tx=0x7f52d0004470 comp rx=0 tx=0).stop 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 shutdown_connections 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f52c406e140 0x7f52c40705f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52dc0fde70 0x7f52dc113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 --2- 172.21.15.33:0/1033454637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52dc0fe810 0x7f52dc113e90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 >> 172.21.15.33:0/1033454637 conn(0x7f52dc0f7860 msgr2=0x7f52dc0f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 shutdown_connections 2024-04-25T01:42:35.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.777+0000 7f52e23ab700 1 -- 172.21.15.33:0/1033454637 wait complete. 2024-04-25T01:42:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.883+0000 7efeb23f5700 1 -- 172.21.15.33:0/3427858200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac102e50 msgr2=0x7efeac103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.883+0000 7efeb23f5700 1 --2- 172.21.15.33:0/3427858200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac102e50 0x7efeac103220 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efe9c004940 tx=0x7efe9c009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.883+0000 7efeb23f5700 1 -- 172.21.15.33:0/3427858200 shutdown_connections 2024-04-25T01:42:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.883+0000 7efeb23f5700 1 --2- 172.21.15.33:0/3427858200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeac065640 0x7efeac065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.883+0000 7efeb23f5700 1 --2- 172.21.15.33:0/3427858200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac102e50 0x7efeac103220 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.883+0000 7efeb23f5700 1 -- 172.21.15.33:0/3427858200 >> 172.21.15.33:0/3427858200 conn(0x7efeac071d00 msgr2=0x7efeac072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.883+0000 7efeb23f5700 1 -- 172.21.15.33:0/3427858200 shutdown_connections 2024-04-25T01:42:35.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.883+0000 7efeb23f5700 1 -- 172.21.15.33:0/3427858200 wait complete. 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.883+0000 7efeb23f5700 1 Processor -- start 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeb23f5700 1 -- start start 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeb23f5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac065640 0x7efeac113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeb23f5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeac102e50 0x7efeac1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeb23f5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac114710 con 0x7efeac065640 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeb23f5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeac10db00 con 0x7efeac102e50 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac065640 0x7efeac113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac065640 0x7efeac113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40696/0 (socket says 172.21.15.33:40696) 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeabfff700 1 -- 172.21.15.33:0/2560902480 learned_addr learned my addr 172.21.15.33:0/2560902480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeab7fe700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeac102e50 0x7efeac1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeabfff700 1 -- 172.21.15.33:0/2560902480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeac102e50 msgr2=0x7efeac1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:35.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeabfff700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeac102e50 0x7efeac1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.884+0000 7efeabfff700 1 -- 172.21.15.33:0/2560902480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c005040 con 0x7efeac065640 2024-04-25T01:42:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.885+0000 7efeabfff700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac065640 0x7efeac113b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009150 tx=0x7efe9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.885+0000 7efea8ff9700 1 -- 172.21.15.33:0/2560902480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9c00e040 con 0x7efeac065640 2024-04-25T01:42:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.885+0000 7efea8ff9700 1 -- 172.21.15.33:0/2560902480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe9c013420 con 0x7efeac065640 2024-04-25T01:42:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.885+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeac10dd80 con 0x7efeac065640 2024-04-25T01:42:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.885+0000 7efea8ff9700 1 -- 172.21.15.33:0/2560902480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe9c01c610 con 0x7efeac065640 2024-04-25T01:42:35.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.885+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeac10e340 con 0x7efeac065640 2024-04-25T01:42:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.886+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeac100100 con 0x7efeac065640 2024-04-25T01:42:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.886+0000 7efea8ff9700 1 -- 172.21.15.33:0/2560902480 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efe9c00baf0 con 0x7efeac065640 2024-04-25T01:42:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.887+0000 7efea8ff9700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe9406e150 0x7efe94070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.887+0000 7efea8ff9700 1 -- 172.21.15.33:0/2560902480 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6238+0+0 (secure 0 0 0) 0x7efe9c08ec20 con 0x7efeac065640 2024-04-25T01:42:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.887+0000 7efeab7fe700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe9406e150 0x7efe94070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:35.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.887+0000 7efeab7fe700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe9406e150 0x7efe94070600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efeac10ecc0 tx=0x7efea000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:35.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:35.892+0000 7efea8ff9700 1 -- 172.21.15.33:0/2560902480 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efe9c05bc00 con 0x7efeac065640 2024-04-25T01:42:36.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:35 smithi138 ceph-mon[81304]: pgmap v65: 65 pgs: 65 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:42:36.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:35 smithi138 ceph-mon[81304]: osd.1 marked itself down and dead 2024-04-25T01:42:36.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.112+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efeac10f1d0 con 0x7efe9406e150 2024-04-25T01:42:36.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.123+0000 7efea8ff9700 1 -- 172.21.15.33:0/2560902480 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7efeac10f1d0 con 0x7efe9406e150 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (3m) 26s ago 14m 27.2M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (59s) 26s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (53s) 50s ago 14m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (3m) 26s ago 14m 79.2M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (9m) 26s ago 9m 62.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (9m) 26s ago 9m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (9m) 50s ago 9m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (9m) 50s ago 9m 61.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (5m) 26s ago 16m 498M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (4m) 50s ago 14m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (2m) 26s ago 16m 43.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:42:36.113 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (99s) 50s ago 13m 33.9M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:42:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (4m) 26s ago 15m 16.0M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:42:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (4m) 50s ago 14m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:42:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (32s) 26s ago 13m 11.9M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:42:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (12m) 26s ago 12m 235M 1559M 17.2.7-902-gc61472c8 0de407c1b3af a354c284ed70 2024-04-25T01:42:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (12m) 26s ago 12m 190M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:42:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (11m) 50s ago 11m 264M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:42:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (11m) 50s ago 11m 231M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:42:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (11m) 50s ago 11m 219M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:42:36.114 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (4m) 26s ago 14m 60.8M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:42:36.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.125+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe9406e150 msgr2=0x7efe94070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.125+0000 7efeb23f5700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe9406e150 0x7efe94070600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7efeac10ecc0 tx=0x7efea000b040 comp rx=0 tx=0).stop 2024-04-25T01:42:36.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.125+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac065640 msgr2=0x7efeac113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.125+0000 7efeb23f5700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac065640 0x7efeac113b80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009150 tx=0x7efe9c009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:36.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.126+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 shutdown_connections 2024-04-25T01:42:36.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.126+0000 7efeb23f5700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe9406e150 0x7efe94070600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.126+0000 7efeb23f5700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efeac065640 0x7efeac113b80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.126+0000 7efeb23f5700 1 --2- 172.21.15.33:0/2560902480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efeac102e50 0x7efeac1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.126+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 >> 172.21.15.33:0/2560902480 conn(0x7efeac071d00 msgr2=0x7efeac0fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:36.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.126+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 shutdown_connections 2024-04-25T01:42:36.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.126+0000 7efeb23f5700 1 -- 172.21.15.33:0/2560902480 wait complete. 2024-04-25T01:42:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.238+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/973536586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4065640 msgr2=0x7f5dd4065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.238+0000 7f5dd9f4f700 1 --2- 172.21.15.33:0/973536586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4065640 0x7f5dd4065ab0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8004940 tx=0x7f5dc8009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.238+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/973536586 shutdown_connections 2024-04-25T01:42:36.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.238+0000 7f5dd9f4f700 1 --2- 172.21.15.33:0/973536586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4065640 0x7f5dd4065ab0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.238+0000 7f5dd9f4f700 1 --2- 172.21.15.33:0/973536586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd4102e50 0x7f5dd4103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.238+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/973536586 >> 172.21.15.33:0/973536586 conn(0x7f5dd4071d00 msgr2=0x7f5dd4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.238+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/973536586 shutdown_connections 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.238+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/973536586 wait complete. 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.238+0000 7f5dd9f4f700 1 Processor -- start 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd9f4f700 1 -- start start 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd9f4f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd4065640 0x7f5dd4113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd9f4f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4102e50 0x7f5dd41140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd9f4f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd4114700 con 0x7f5dd4102e50 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd9f4f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd410daf0 con 0x7f5dd4065640 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4102e50 0x7f5dd41140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4102e50 0x7f5dd41140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40710/0 (socket says 172.21.15.33:40710) 2024-04-25T01:42:36.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd2ffd700 1 -- 172.21.15.33:0/3004550483 learned_addr learned my addr 172.21.15.33:0/3004550483 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd37fe700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd4065640 0x7f5dd4113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd2ffd700 1 -- 172.21.15.33:0/3004550483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd4065640 msgr2=0x7f5dd4113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd2ffd700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd4065640 0x7f5dd4113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.239+0000 7f5dd2ffd700 1 -- 172.21.15.33:0/3004550483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc8005040 con 0x7f5dd4102e50 2024-04-25T01:42:36.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.240+0000 7f5dd2ffd700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4102e50 0x7f5dd41140b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009180 tx=0x7f5dc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.240+0000 7f5dbbfff700 1 -- 172.21.15.33:0/3004550483 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dc800e040 con 0x7f5dd4102e50 2024-04-25T01:42:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.240+0000 7f5dbbfff700 1 -- 172.21.15.33:0/3004550483 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5dc800bbc0 con 0x7f5dd4102e50 2024-04-25T01:42:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.240+0000 7f5dbbfff700 1 -- 172.21.15.33:0/3004550483 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dc801c630 con 0x7f5dd4102e50 2024-04-25T01:42:36.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.240+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd410dd70 con 0x7f5dd4102e50 2024-04-25T01:42:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.240+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd410e270 con 0x7f5dd4102e50 2024-04-25T01:42:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.241+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd4100100 con 0x7f5dd4102e50 2024-04-25T01:42:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.242+0000 7f5dbbfff700 1 -- 172.21.15.33:0/3004550483 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5dc800bd30 con 0x7f5dd4102e50 2024-04-25T01:42:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.242+0000 7f5dbbfff700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5dbc06e150 0x7f5dbc070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.242+0000 7f5dbbfff700 1 -- 172.21.15.33:0/3004550483 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f5dc808ed60 con 0x7f5dd4102e50 2024-04-25T01:42:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.242+0000 7f5dd37fe700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5dbc06e150 0x7f5dbc070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.243+0000 7f5dd37fe700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5dbc06e150 0x7f5dbc070600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4004990 tx=0x7f5dc4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:36.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.247+0000 7f5dbbfff700 1 -- 172.21.15.33:0/3004550483 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5dc805bde0 con 0x7f5dd4102e50 2024-04-25T01:42:36.505 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:36 smithi033 podman[127883]: 2024-04-25 01:42:36.230141446 +0000 UTC m=+1.152240966 container died a354c284ed70fb6aa4e8636e698e31bc9358f030ad985d495877c3b87cf0331d (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T01:42:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.545+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5dd410f1c0 con 0x7f5dd4102e50 2024-04-25T01:42:36.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.546+0000 7f5dbbfff700 1 -- 172.21.15.33:0/3004550483 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f5dc805b970 con 0x7f5dd4102e50 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4, 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 1 2024-04-25T01:42:36.535 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:42:36.536 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:42:36.536 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:42:36.536 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:42:36.536 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:42:36.536 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 8, 2024-04-25T01:42:36.536 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 5 2024-04-25T01:42:36.536 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:42:36.536 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5dbc06e150 msgr2=0x7f5dbc070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5dbc06e150 0x7f5dbc070600 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5dc4004990 tx=0x7f5dc4009ee0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4102e50 msgr2=0x7f5dd41140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4102e50 0x7f5dd41140b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009180 tx=0x7f5dc8009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 shutdown_connections 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5dbc06e150 0x7f5dbc070600 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dd4065640 0x7f5dd4113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 --2- 172.21.15.33:0/3004550483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dd4102e50 0x7f5dd41140b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 >> 172.21.15.33:0/3004550483 conn(0x7f5dd4071d00 msgr2=0x7f5dd40fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 shutdown_connections 2024-04-25T01:42:36.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.548+0000 7f5dd9f4f700 1 -- 172.21.15.33:0/3004550483 wait complete. 2024-04-25T01:42:36.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.653+0000 7f86aaa9f700 1 -- 172.21.15.33:0/4062310800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 msgr2=0x7f86a40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.653+0000 7f86aaa9f700 1 --2- 172.21.15.33:0/4062310800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 0x7f86a40fcdf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8690004940 tx=0x7f8690009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:36.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.653+0000 7f86aaa9f700 1 -- 172.21.15.33:0/4062310800 shutdown_connections 2024-04-25T01:42:36.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.653+0000 7f86aaa9f700 1 --2- 172.21.15.33:0/4062310800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 0x7f86a40fcdf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.653+0000 7f86aaa9f700 1 --2- 172.21.15.33:0/4062310800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4102de0 0x7f86a41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.653+0000 7f86aaa9f700 1 -- 172.21.15.33:0/4062310800 >> 172.21.15.33:0/4062310800 conn(0x7f86a40f8990 msgr2=0x7f86a40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.654+0000 7f86aaa9f700 1 -- 172.21.15.33:0/4062310800 shutdown_connections 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.654+0000 7f86aaa9f700 1 -- 172.21.15.33:0/4062310800 wait complete. 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.654+0000 7f86aaa9f700 1 Processor -- start 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.654+0000 7f86aaa9f700 1 -- start start 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.654+0000 7f86aaa9f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4102de0 0x7f86a4115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.654+0000 7f86aaa9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 0x7f86a41160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.654+0000 7f86aaa9f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a4116720 con 0x7f86a4103780 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.654+0000 7f86aaa9f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a410fd00 con 0x7f86a4102de0 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 0x7f86a41160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 0x7f86a41160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40726/0 (socket says 172.21.15.33:40726) 2024-04-25T01:42:36.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a3fff700 1 -- 172.21.15.33:0/1339057461 learned_addr learned my addr 172.21.15.33:0/1339057461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a883b700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4102de0 0x7f86a4115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a3fff700 1 -- 172.21.15.33:0/1339057461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4102de0 msgr2=0x7f86a4115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a3fff700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4102de0 0x7f86a4115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a3fff700 1 -- 172.21.15.33:0/1339057461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8690005040 con 0x7f86a4103780 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a3fff700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 0x7f86a41160d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8690009100 tx=0x7f8690009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a17fa700 1 -- 172.21.15.33:0/1339057461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f869000e040 con 0x7f86a4103780 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a410ff80 con 0x7f86a4103780 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a17fa700 1 -- 172.21.15.33:0/1339057461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8690009af0 con 0x7f86a4103780 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86a17fa700 1 -- 172.21.15.33:0/1339057461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f869001c3f0 con 0x7f86a4103780 2024-04-25T01:42:36.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.655+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a4110540 con 0x7f86a4103780 2024-04-25T01:42:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.656+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a40fe530 con 0x7f86a4103780 2024-04-25T01:42:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.657+0000 7f86a17fa700 1 -- 172.21.15.33:0/1339057461 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f869000baf0 con 0x7f86a4103780 2024-04-25T01:42:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.657+0000 7f86a17fa700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f869406e140 0x7f86940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.657+0000 7f86a17fa700 1 -- 172.21.15.33:0/1339057461 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f869008ecb0 con 0x7f86a4103780 2024-04-25T01:42:36.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.657+0000 7f86a883b700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f869406e140 0x7f86940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:36.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.658+0000 7f86a883b700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f869406e140 0x7f86940705f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8698004eb0 tx=0x7f869800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:36.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.662+0000 7f86a17fa700 1 -- 172.21.15.33:0/1339057461 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f869005bc90 con 0x7f86a4103780 2024-04-25T01:42:36.894 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:36 smithi033 podman[127883]: 2024-04-25 01:42:36.670811572 +0000 UTC m=+1.592911088 container cleanup a354c284ed70fb6aa4e8636e698e31bc9358f030ad985d495877c3b87cf0331d (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:42:36.894 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:36 smithi033 bash[127883]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1 2024-04-25T01:42:36.894 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:36 smithi033 ceph-mon[111306]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:36.894 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:36 smithi033 ceph-mon[111306]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:42:36.894 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:36 smithi033 ceph-mon[111306]: osdmap e54: 6 total, 5 up, 6 in 2024-04-25T01:42:36.894 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:36 smithi033 ceph-mon[111306]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:36.894 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:36 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3004550483' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:36.894 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:36 smithi033 ceph-mon[111306]: osdmap e55: 6 total, 5 up, 6 in 2024-04-25T01:42:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.943+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f86a4110110 con 0x7f86a4103780 2024-04-25T01:42:36.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.943+0000 7f86a17fa700 1 -- 172.21.15.33:0/1339057461 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f86a4110110 con 0x7f86a4103780 2024-04-25T01:42:36.932 INFO:teuthology.orchestra.run.smithi033.stdout:e19 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 19 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:42:36.933 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:42:36.934 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:42:36.935 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:42:36.936 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:42:36.936 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:42:36.936 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f869406e140 msgr2=0x7f86940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f869406e140 0x7f86940705f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8698004eb0 tx=0x7f869800b040 comp rx=0 tx=0).stop 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 msgr2=0x7f86a41160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 0x7f86a41160d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8690009100 tx=0x7f8690009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 shutdown_connections 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f869406e140 0x7f86940705f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f86a4102de0 0x7f86a4115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 --2- 172.21.15.33:0/1339057461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f86a4103780 0x7f86a41160d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 >> 172.21.15.33:0/1339057461 conn(0x7f86a40f8990 msgr2=0x7f86a4071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.945+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 shutdown_connections 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:36.946+0000 7f86aaa9f700 1 -- 172.21.15.33:0/1339057461 wait complete. 2024-04-25T01:42:36.937 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:42:37.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:36 smithi138 ceph-mon[81304]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:37.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:36 smithi138 ceph-mon[81304]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:42:37.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:36 smithi138 ceph-mon[81304]: osdmap e54: 6 total, 5 up, 6 in 2024-04-25T01:42:37.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:36 smithi138 ceph-mon[81304]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:37.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:36 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3004550483' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:37.031 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:36 smithi138 ceph-mon[81304]: osdmap e55: 6 total, 5 up, 6 in 2024-04-25T01:42:37.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.067+0000 7f946fdc9700 1 -- 172.21.15.33:0/2830091142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fe780 msgr2=0x7f9468106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.067+0000 7f946fdc9700 1 --2- 172.21.15.33:0/2830091142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fe780 0x7f9468106dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9464004940 tx=0x7f9464009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.067+0000 7f946fdc9700 1 -- 172.21.15.33:0/2830091142 shutdown_connections 2024-04-25T01:42:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.067+0000 7f946fdc9700 1 --2- 172.21.15.33:0/2830091142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fe780 0x7f9468106dc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.067+0000 7f946fdc9700 1 --2- 172.21.15.33:0/2830091142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94680fdde0 0x7f94680fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.067+0000 7f946fdc9700 1 -- 172.21.15.33:0/2830091142 >> 172.21.15.33:0/2830091142 conn(0x7f94680f7850 msgr2=0x7f94680f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.067+0000 7f946fdc9700 1 -- 172.21.15.33:0/2830091142 shutdown_connections 2024-04-25T01:42:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.067+0000 7f946fdc9700 1 -- 172.21.15.33:0/2830091142 wait complete. 2024-04-25T01:42:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946fdc9700 1 Processor -- start 2024-04-25T01:42:37.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946fdc9700 1 -- start start 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946fdc9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fdde0 0x7f94681138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946fdc9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94680fe780 0x7f9468113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946fdc9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9468114470 con 0x7f94680fdde0 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946fdc9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946810d860 con 0x7f94680fe780 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946db65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fdde0 0x7f94681138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946db65700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fdde0 0x7f94681138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40738/0 (socket says 172.21.15.33:40738) 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946db65700 1 -- 172.21.15.33:0/1318007412 learned_addr learned my addr 172.21.15.33:0/1318007412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.068+0000 7f946d364700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94680fe780 0x7f9468113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.069+0000 7f946db65700 1 -- 172.21.15.33:0/1318007412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94680fe780 msgr2=0x7f9468113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.069+0000 7f946db65700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94680fe780 0x7f9468113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.069+0000 7f946db65700 1 -- 172.21.15.33:0/1318007412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464005040 con 0x7f94680fdde0 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.069+0000 7f946db65700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fdde0 0x7f94681138e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9458002a50 tx=0x7f94580046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:37.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.069+0000 7f945e7fc700 1 -- 172.21.15.33:0/1318007412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9458007cc0 con 0x7f94680fdde0 2024-04-25T01:42:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.069+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946810db40 con 0x7f94680fdde0 2024-04-25T01:42:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.069+0000 7f945e7fc700 1 -- 172.21.15.33:0/1318007412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f945800f040 con 0x7f94680fdde0 2024-04-25T01:42:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.069+0000 7f945e7fc700 1 -- 172.21.15.33:0/1318007412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f945800da30 con 0x7f94680fdde0 2024-04-25T01:42:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.069+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946810e050 con 0x7f94680fdde0 2024-04-25T01:42:37.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.070+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9468066750 con 0x7f94680fdde0 2024-04-25T01:42:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.071+0000 7f945e7fc700 1 -- 172.21.15.33:0/1318007412 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f945800dbb0 con 0x7f94680fdde0 2024-04-25T01:42:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.072+0000 7f945e7fc700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f945406e140 0x7f94540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.072+0000 7f945e7fc700 1 -- 172.21.15.33:0/1318007412 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f945808d8b0 con 0x7f94680fdde0 2024-04-25T01:42:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.072+0000 7f946d364700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f945406e140 0x7f94540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.072+0000 7f946d364700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f945406e140 0x7f94540705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f946810eb40 tx=0x7f9464009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:37.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.077+0000 7f945e7fc700 1 -- 172.21.15.33:0/1318007412 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f945805a770 con 0x7f94680fdde0 2024-04-25T01:42:37.254 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:36 smithi033 podman[127976]: 2024-04-25 01:42:36.905796884 +0000 UTC m=+0.678305816 container remove a354c284ed70fb6aa4e8636e698e31bc9358f030ad985d495877c3b87cf0331d (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:42:37.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.320+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9468109e70 con 0x7f945406e140 2024-04-25T01:42:37.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.321+0000 7f945e7fc700 1 -- 172.21.15.33:0/1318007412 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9468109e70 con 0x7f945406e140 2024-04-25T01:42:37.310 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "7/21 daemons upgraded", 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:42:37.311 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:42:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f945406e140 msgr2=0x7f94540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f945406e140 0x7f94540705f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f946810eb40 tx=0x7f9464009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fdde0 msgr2=0x7f94681138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fdde0 0x7f94681138e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9458002a50 tx=0x7f94580046b0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 shutdown_connections 2024-04-25T01:42:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f945406e140 0x7f94540705f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94680fdde0 0x7f94681138e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 --2- 172.21.15.33:0/1318007412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94680fe780 0x7f9468113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 >> 172.21.15.33:0/1318007412 conn(0x7f94680f7850 msgr2=0x7f94680f8be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 shutdown_connections 2024-04-25T01:42:37.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.324+0000 7f946fdc9700 1 -- 172.21.15.33:0/1318007412 wait complete. 2024-04-25T01:42:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.429+0000 7fa9c6809700 1 -- 172.21.15.33:0/1405843113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070c10 msgr2=0x7fa9c0109060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.429+0000 7fa9c6809700 1 --2- 172.21.15.33:0/1405843113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070c10 0x7fa9c0109060 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0004970 tx=0x7fa9b0009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.429+0000 7fa9c6809700 1 -- 172.21.15.33:0/1405843113 shutdown_connections 2024-04-25T01:42:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.429+0000 7fa9c6809700 1 --2- 172.21.15.33:0/1405843113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070c10 0x7fa9c0109060 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.429+0000 7fa9c6809700 1 --2- 172.21.15.33:0/1405843113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c0070300 0x7fa9c00706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.429+0000 7fa9c6809700 1 -- 172.21.15.33:0/1405843113 >> 172.21.15.33:0/1405843113 conn(0x7fa9c00f8900 msgr2=0x7fa9c00fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.429+0000 7fa9c6809700 1 -- 172.21.15.33:0/1405843113 shutdown_connections 2024-04-25T01:42:37.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.429+0000 7fa9c6809700 1 -- 172.21.15.33:0/1405843113 wait complete. 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.429+0000 7fa9c6809700 1 Processor -- start 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9c6809700 1 -- start start 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9c6809700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070300 0x7fa9c0115c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9c6809700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c0070c10 0x7fa9c01161d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9c6809700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0116820 con 0x7fa9c0070300 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9c6809700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c010fc30 con 0x7fa9c0070c10 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070300 0x7fa9c0115c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9bffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070300 0x7fa9c0115c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40760/0 (socket says 172.21.15.33:40760) 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9bffff700 1 -- 172.21.15.33:0/42850198 learned_addr learned my addr 172.21.15.33:0/42850198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9bf7fe700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c0070c10 0x7fa9c01161d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9bffff700 1 -- 172.21.15.33:0/42850198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c0070c10 msgr2=0x7fa9c01161d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9bffff700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c0070c10 0x7fa9c01161d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.430+0000 7fa9bffff700 1 -- 172.21.15.33:0/42850198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b0005040 con 0x7fa9c0070300 2024-04-25T01:42:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.431+0000 7fa9bffff700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070300 0x7fa9c0115c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8002a50 tx=0x7fa9a80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.431+0000 7fa9bcff9700 1 -- 172.21.15.33:0/42850198 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9a8007cc0 con 0x7fa9c0070300 2024-04-25T01:42:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.431+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c010ff10 con 0x7fa9c0070300 2024-04-25T01:42:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.431+0000 7fa9bcff9700 1 -- 172.21.15.33:0/42850198 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9a800f040 con 0x7fa9c0070300 2024-04-25T01:42:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.431+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c0110420 con 0x7fa9c0070300 2024-04-25T01:42:37.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.431+0000 7fa9bcff9700 1 -- 172.21.15.33:0/42850198 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9a800da30 con 0x7fa9c0070300 2024-04-25T01:42:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.432+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c0071a40 con 0x7fa9c0070300 2024-04-25T01:42:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.432+0000 7fa9bcff9700 1 -- 172.21.15.33:0/42850198 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa9a800dc70 con 0x7fa9c0070300 2024-04-25T01:42:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.433+0000 7fa9bcff9700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9ac06e140 0x7fa9ac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:42:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.433+0000 7fa9bcff9700 1 -- 172.21.15.33:0/42850198 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fa9a808d790 con 0x7fa9c0070300 2024-04-25T01:42:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.433+0000 7fa9bf7fe700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9ac06e140 0x7fa9ac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:42:37.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.433+0000 7fa9bf7fe700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9ac06e140 0x7fa9ac0705f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0004f50 tx=0x7fa9b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:42:37.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.437+0000 7fa9bcff9700 1 -- 172.21.15.33:0/42850198 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa9a8056c40 con 0x7fa9c0070300 2024-04-25T01:42:37.589 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:37 smithi033 podman[128130]: 2024-04-25 01:42:37.282325904 +0000 UTC m=+0.018970794 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:37.589 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:37 smithi033 podman[128130]: 2024-04-25 01:42:37.382024673 +0000 UTC m=+0.118669554 container create 0acc61d30d0b92ea91f48eddcee9bc4a8abe8dcecb4555d8146dee9202cd65dd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:42:37.589 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:37 smithi033 podman[128130]: 2024-04-25 01:42:37.601658279 +0000 UTC m=+0.338303165 container init 0acc61d30d0b92ea91f48eddcee9bc4a8abe8dcecb4555d8146dee9202cd65dd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T01:42:37.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.754+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa9c00fc500 con 0x7fa9c0070300 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.755+0000 7fa9bcff9700 1 -- 172.21.15.33:0/42850198 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+901 (secure 0 0 0) 0x7fa9c00fc500 con 0x7fa9c0070300 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 9 pgs peering 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: osd.1 (root=default,host=smithi033) is down 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 9 pgs peering 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.6 is stuck peering for 5m, current state peering, last acting [3,4] 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.9 is stuck peering for 5m, current state peering, last acting [4,0] 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.a is stuck peering for 5m, current state peering, last acting [4,3] 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.d is stuck peering for 5m, current state peering, last acting [3,2] 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: pg 2.15 is stuck peering for 5m, current state peering, last acting [3,0] 2024-04-25T01:42:37.744 INFO:teuthology.orchestra.run.smithi033.stdout: pg 3.0 is stuck peering for 5m, current state peering, last acting [2,4] 2024-04-25T01:42:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: pg 3.2 is stuck peering for 5m, current state peering, last acting [5,3] 2024-04-25T01:42:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: pg 3.4 is stuck peering for 5m, current state peering, last acting [2,3] 2024-04-25T01:42:37.745 INFO:teuthology.orchestra.run.smithi033.stdout: pg 3.19 is stuck peering for 5m, current state peering, last acting [4,3] 2024-04-25T01:42:37.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.757+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9ac06e140 msgr2=0x7fa9ac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.757+0000 7fa9c6809700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9ac06e140 0x7fa9ac0705f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0004f50 tx=0x7fa9b0009e30 comp rx=0 tx=0).stop 2024-04-25T01:42:37.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.757+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070300 msgr2=0x7fa9c0115c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:42:37.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.757+0000 7fa9c6809700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070300 0x7fa9c0115c90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa9a8002a50 tx=0x7fa9a80046b0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.757+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 shutdown_connections 2024-04-25T01:42:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.757+0000 7fa9c6809700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9ac06e140 0x7fa9ac0705f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.757+0000 7fa9c6809700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9c0070300 0x7fa9c0115c90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.757+0000 7fa9c6809700 1 --2- 172.21.15.33:0/42850198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9c0070c10 0x7fa9c01161d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:42:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.757+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 >> 172.21.15.33:0/42850198 conn(0x7fa9c00f8900 msgr2=0x7fa9c00f9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:42:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.758+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 shutdown_connections 2024-04-25T01:42:37.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:42:37.758+0000 7fa9c6809700 1 -- 172.21.15.33:0/42850198 wait complete. 2024-04-25T01:42:37.900 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:37 smithi033 podman[128130]: 2024-04-25 01:42:37.605425186 +0000 UTC m=+0.342070061 container start 0acc61d30d0b92ea91f48eddcee9bc4a8abe8dcecb4555d8146dee9202cd65dd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:42:37.900 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:37 smithi033 podman[128130]: 2024-04-25 01:42:37.718535324 +0000 UTC m=+0.455180211 container attach 0acc61d30d0b92ea91f48eddcee9bc4a8abe8dcecb4555d8146dee9202cd65dd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T01:42:37.901 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:37 smithi033 ceph-mon[111306]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:37.901 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:37 smithi033 ceph-mon[111306]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3 op/s 2024-04-25T01:42:37.901 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:37 smithi033 ceph-mon[111306]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-25T01:42:37.901 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:37 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1339057461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:42:38.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:37 smithi138 ceph-mon[81304]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:38.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:37 smithi138 ceph-mon[81304]: pgmap v67: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3 op/s 2024-04-25T01:42:38.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:37 smithi138 ceph-mon[81304]: Health check failed: Reduced data availability: 9 pgs peering (PG_AVAILABILITY) 2024-04-25T01:42:38.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:37 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1339057461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:42:38.254 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:37 smithi033 podman[128130]: 2024-04-25 01:42:37.912379523 +0000 UTC m=+0.649024407 container died 0acc61d30d0b92ea91f48eddcee9bc4a8abe8dcecb4555d8146dee9202cd65dd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:42:39.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:38 smithi033 ceph-mon[111306]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:39.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:38 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/42850198' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:42:39.005 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:38 smithi033 podman[128182]: 2024-04-25 01:42:38.664328941 +0000 UTC m=+0.740600574 container remove 0acc61d30d0b92ea91f48eddcee9bc4a8abe8dcecb4555d8146dee9202cd65dd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T01:42:39.005 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:38 smithi033 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-25T01:42:39.005 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:38 smithi033 systemd[1]: Stopped Ceph osd.1 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:42:39.005 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:38 smithi033 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.1.service: Consumed 1min 11.946s CPU time. 2024-04-25T01:42:39.005 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:38 smithi033 systemd[1]: Starting Ceph osd.1 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:42:39.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:38 smithi138 ceph-mon[81304]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:42:39.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:38 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/42850198' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:42:39.415 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 podman[128287]: 2024-04-25 01:42:39.024913582 +0000 UTC m=+0.019117085 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:39.415 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 podman[128287]: 2024-04-25 01:42:39.13253929 +0000 UTC m=+0.126742809 container create 64176a4c0d0da95cd87010dd6decbf993e105ffd2226f19e9a32996e195c26e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True) 2024-04-25T01:42:39.415 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 podman[128287]: 2024-04-25 01:42:39.34364873 +0000 UTC m=+0.337852207 container init 64176a4c0d0da95cd87010dd6decbf993e105ffd2226f19e9a32996e195c26e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T01:42:39.415 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 podman[128287]: 2024-04-25 01:42:39.34694765 +0000 UTC m=+0.341151135 container start 64176a4c0d0da95cd87010dd6decbf993e105ffd2226f19e9a32996e195c26e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T01:42:39.415 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 podman[128287]: 2024-04-25 01:42:39.427038577 +0000 UTC m=+0.421242072 container attach 64176a4c0d0da95cd87010dd6decbf993e105ffd2226f19e9a32996e195c26e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424) 2024-04-25T01:42:40.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:39 smithi033 ceph-mon[111306]: pgmap v69: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3 op/s 2024-04-25T01:42:40.005 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate[128302]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T01:42:40.005 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate[128302]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 bash[128287]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 bash[128287]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 bash[128287]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 bash[128287]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 bash[128287]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 bash[128287]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 bash[128287]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate[128302]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate[128302]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate[128302]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate[128302]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate[128302]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-25T01:42:40.006 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:39 smithi033 podman[128287]: 2024-04-25 01:42:39.83056592 +0000 UTC m=+0.824769409 container died 64176a4c0d0da95cd87010dd6decbf993e105ffd2226f19e9a32996e195c26e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T01:42:40.030 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:39 smithi138 ceph-mon[81304]: pgmap v69: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3 op/s 2024-04-25T01:42:40.755 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:40 smithi033 podman[128425]: 2024-04-25 01:42:40.439652159 +0000 UTC m=+0.600150454 container remove 64176a4c0d0da95cd87010dd6decbf993e105ffd2226f19e9a32996e195c26e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-activate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T01:42:40.755 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:40 smithi033 podman[128489]: 2024-04-25 01:42:40.657289022 +0000 UTC m=+0.113746745 container create 4ab9ac022ca9ab34e180f8acf33e80ecbf86827b6dd437b2ab1312150f6ea971 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T01:42:40.755 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:40 smithi033 podman[128489]: 2024-04-25 01:42:40.562977885 +0000 UTC m=+0.019435609 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:42:41.145 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:40 smithi033 podman[128489]: 2024-04-25 01:42:40.874000743 +0000 UTC m=+0.330458467 container init 4ab9ac022ca9ab34e180f8acf33e80ecbf86827b6dd437b2ab1312150f6ea971 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T01:42:41.146 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:40 smithi033 podman[128489]: 2024-04-25 01:42:40.877185199 +0000 UTC m=+0.333642922 container start 4ab9ac022ca9ab34e180f8acf33e80ecbf86827b6dd437b2ab1312150f6ea971 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:42:41.146 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:40 smithi033 ceph-osd[128509]: -- 172.21.15.33:0/3339583260 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c646491680 con 0x55c64645ac00 2024-04-25T01:42:41.146 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:40 smithi033 bash[128489]: 4ab9ac022ca9ab34e180f8acf33e80ecbf86827b6dd437b2ab1312150f6ea971 2024-04-25T01:42:41.146 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:40 smithi033 systemd[1]: Started Ceph osd.1 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:42:41.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:41 smithi033 ceph-mon[111306]: pgmap v70: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 6/321 objects degraded (1.869%) 2024-04-25T01:42:41.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:41.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:42:41.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:41 smithi033 ceph-mon[111306]: Health check failed: Degraded data redundancy: 6/321 objects degraded (1.869%), 2 pgs degraded (PG_DEGRADED) 2024-04-25T01:42:41.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:41.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:41.416 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:41.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:41 smithi138 ceph-mon[81304]: pgmap v70: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 6/321 objects degraded (1.869%) 2024-04-25T01:42:41.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:41.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:42:41.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:41 smithi138 ceph-mon[81304]: Health check failed: Degraded data redundancy: 6/321 objects degraded (1.869%), 2 pgs degraded (PG_DEGRADED) 2024-04-25T01:42:41.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:41.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:41.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:41.755 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:41 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[128505]: 2024-04-25T01:42:41.426+0000 7fa8d12a77c0 -1 Falling back to public interface 2024-04-25T01:42:42.505 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:42 smithi033 ceph-mon[111306]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-25T01:42:42.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:42 smithi138 ceph-mon[81304]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs peering) 2024-04-25T01:42:43.570 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:43 smithi033 ceph-mon[111306]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:43.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:43 smithi138 ceph-mon[81304]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:45.653 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:45 smithi033 ceph-mon[111306]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:45.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:45 smithi138 ceph-mon[81304]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:47.005 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:46 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[128505]: 2024-04-25T01:42:46.723+0000 7fa8d12a77c0 -1 osd.1 53 log_to_monitors true 2024-04-25T01:42:47.505 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 01:42:47 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[128505]: 2024-04-25T01:42:47.214+0000 7fa8c26ff700 -1 osd.1 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:42:47.505 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:47 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:47.505 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:47 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:47.505 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:47 smithi033 ceph-mon[111306]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:47.505 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:47 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 48/321 objects degraded (14.953%), 16 pgs degraded (PG_DEGRADED) 2024-04-25T01:42:47.505 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:47 smithi033 ceph-mon[111306]: from='osd.1 [v2:172.21.15.33:6810/166114729,v1:172.21.15.33:6811/166114729]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:42:47.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:47 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:47.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:47 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:47.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:47 smithi138 ceph-mon[81304]: pgmap v73: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:47.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:47 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 48/321 objects degraded (14.953%), 16 pgs degraded (PG_DEGRADED) 2024-04-25T01:42:47.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:47 smithi138 ceph-mon[81304]: from='osd.1 [v2:172.21.15.33:6810/166114729,v1:172.21.15.33:6811/166114729]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T01:42:48.476 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:48 smithi033 ceph-mon[111306]: from='osd.1 [v2:172.21.15.33:6810/166114729,v1:172.21.15.33:6811/166114729]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:42:48.476 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:48 smithi033 ceph-mon[111306]: osdmap e56: 6 total, 5 up, 6 in 2024-04-25T01:42:48.477 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:48 smithi033 ceph-mon[111306]: from='osd.1 [v2:172.21.15.33:6810/166114729,v1:172.21.15.33:6811/166114729]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:42:48.477 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:48 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:48.477 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:48 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:48.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:48 smithi138 ceph-mon[81304]: from='osd.1 [v2:172.21.15.33:6810/166114729,v1:172.21.15.33:6811/166114729]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T01:42:48.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:48 smithi138 ceph-mon[81304]: osdmap e56: 6 total, 5 up, 6 in 2024-04-25T01:42:48.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:48 smithi138 ceph-mon[81304]: from='osd.1 [v2:172.21.15.33:6810/166114729,v1:172.21.15.33:6811/166114729]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:42:48.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:48 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:48.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:48 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:49.466 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:49 smithi033 ceph-mon[111306]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:42:49.466 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:49 smithi033 ceph-mon[111306]: osd.1 [v2:172.21.15.33:6810/166114729,v1:172.21.15.33:6811/166114729] boot 2024-04-25T01:42:49.466 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:49 smithi033 ceph-mon[111306]: osdmap e57: 6 total, 6 up, 6 in 2024-04-25T01:42:49.467 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:42:49.467 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:49 smithi033 ceph-mon[111306]: pgmap v76: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:49.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:49 smithi138 ceph-mon[81304]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:42:49.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:49 smithi138 ceph-mon[81304]: osd.1 [v2:172.21.15.33:6810/166114729,v1:172.21.15.33:6811/166114729] boot 2024-04-25T01:42:49.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:49 smithi138 ceph-mon[81304]: osdmap e57: 6 total, 6 up, 6 in 2024-04-25T01:42:49.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T01:42:49.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:49 smithi138 ceph-mon[81304]: pgmap v76: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:50.487 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:50 smithi033 ceph-mon[111306]: osdmap e58: 6 total, 6 up, 6 in 2024-04-25T01:42:50.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:50 smithi138 ceph-mon[81304]: osdmap e58: 6 total, 6 up, 6 in 2024-04-25T01:42:51.477 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:51 smithi033 ceph-mon[111306]: pgmap v78: 65 pgs: 16 active+undersized+degraded, 16 active+undersized, 33 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:51.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:51 smithi138 ceph-mon[81304]: pgmap v78: 65 pgs: 16 active+undersized+degraded, 16 active+undersized, 33 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 48/321 objects degraded (14.953%) 2024-04-25T01:42:52.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T01:42:52.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:52 smithi033 ceph-mon[111306]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-25T01:42:52.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T01:42:52.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:52 smithi138 ceph-mon[81304]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-25T01:42:53.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:53 smithi138 ceph-mon[81304]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:42:53.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:53 smithi138 ceph-mon[81304]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 48/321 objects degraded (14.953%), 16 pgs degraded) 2024-04-25T01:42:53.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:53 smithi138 ceph-mon[81304]: Cluster is now healthy 2024-04-25T01:42:54.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:53 smithi033 ceph-mon[111306]: pgmap v79: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:42:54.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:53 smithi033 ceph-mon[111306]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 48/321 objects degraded (14.953%), 16 pgs degraded) 2024-04-25T01:42:54.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:53 smithi033 ceph-mon[111306]: Cluster is now healthy 2024-04-25T01:42:55.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:55 smithi138 ceph-mon[81304]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1020 B/s rd, 2 op/s 2024-04-25T01:42:55.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:55.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:42:56.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:55 smithi033 ceph-mon[111306]: pgmap v80: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1020 B/s rd, 2 op/s 2024-04-25T01:42:56.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:42:56.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:42:57.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:57 smithi138 ceph-mon[81304]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:42:58.005 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:57 smithi033 ceph-mon[111306]: pgmap v81: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:42:59.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:42:59 smithi138 ceph-mon[81304]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:42:59.808 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:42:59 smithi033 ceph-mon[111306]: pgmap v82: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:01.530 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:01 smithi138 ceph-mon[81304]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-04-25T01:43:01.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:01.552 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:01 smithi033 ceph-mon[111306]: pgmap v83: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-04-25T01:43:01.552 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:03.755 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:03 smithi033 ceph-mon[111306]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T01:43:03.780 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:03 smithi138 ceph-mon[81304]: pgmap v84: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T01:43:05.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:05 smithi033 ceph-mon[111306]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:05.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:05 smithi138 ceph-mon[81304]: pgmap v85: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:07.534 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:07 smithi033 ceph-mon[111306]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:43:07.534 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T01:43:07.534 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:07 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T01:43:07.534 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:07 smithi033 ceph-mon[111306]: Upgrade: osd.2 is safe to restart 2024-04-25T01:43:07.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:07 smithi138 ceph-mon[81304]: pgmap v86: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:43:07.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T01:43:07.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:07 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T01:43:07.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:07 smithi138 ceph-mon[81304]: Upgrade: osd.2 is safe to restart 2024-04-25T01:43:07.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.867+0000 7f1610a29700 1 -- 172.21.15.33:0/3701427889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c101d90 msgr2=0x7f160c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:07.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.867+0000 7f1610a29700 1 --2- 172.21.15.33:0/3701427889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c101d90 0x7f160c102160 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f15fc004940 tx=0x7f15fc009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:07.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.867+0000 7f1610a29700 1 -- 172.21.15.33:0/3701427889 shutdown_connections 2024-04-25T01:43:07.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.867+0000 7f1610a29700 1 --2- 172.21.15.33:0/3701427889 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f160c0fbd40 0x7f160c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:07.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.867+0000 7f1610a29700 1 --2- 172.21.15.33:0/3701427889 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c101d90 0x7f160c102160 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:07.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.867+0000 7f1610a29700 1 -- 172.21.15.33:0/3701427889 >> 172.21.15.33:0/3701427889 conn(0x7f160c071d00 msgr2=0x7f160c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:07.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.867+0000 7f1610a29700 1 -- 172.21.15.33:0/3701427889 shutdown_connections 2024-04-25T01:43:07.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.867+0000 7f1610a29700 1 -- 172.21.15.33:0/3701427889 wait complete. 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f1610a29700 1 Processor -- start 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f1610a29700 1 -- start start 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f1610a29700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0fbd40 0x7f160c10eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f1610a29700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f160c101d90 0x7f160c10f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f1610a29700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c10f720 con 0x7f160c0fbd40 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f1610a29700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160c10f860 con 0x7f160c101d90 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f160a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0fbd40 0x7f160c10eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f160a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0fbd40 0x7f160c10eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59754/0 (socket says 172.21.15.33:59754) 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f160a59c700 1 -- 172.21.15.33:0/677416962 learned_addr learned my addr 172.21.15.33:0/677416962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:07.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.868+0000 7f1609d9b700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f160c101d90 0x7f160c10f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.869+0000 7f160a59c700 1 -- 172.21.15.33:0/677416962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f160c101d90 msgr2=0x7f160c10f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.869+0000 7f160a59c700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f160c101d90 0x7f160c10f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.869+0000 7f160a59c700 1 -- 172.21.15.33:0/677416962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15fc005040 con 0x7f160c0fbd40 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.869+0000 7f160a59c700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0fbd40 0x7f160c10eca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00b3b0 tx=0x7f15fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.869+0000 7f15faffd700 1 -- 172.21.15.33:0/677416962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15fc00e040 con 0x7f160c0fbd40 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.869+0000 7f15faffd700 1 -- 172.21.15.33:0/677416962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15fc00bbc0 con 0x7f160c0fbd40 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.869+0000 7f15faffd700 1 -- 172.21.15.33:0/677416962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15fc01c3f0 con 0x7f160c0fbd40 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.869+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f160c117b50 con 0x7f160c0fbd40 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.869+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160c1180e0 con 0x7f160c0fbd40 2024-04-25T01:43:07.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.870+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f160c0fd430 con 0x7f160c0fbd40 2024-04-25T01:43:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.871+0000 7f15faffd700 1 -- 172.21.15.33:0/677416962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f15fc00bd30 con 0x7f160c0fbd40 2024-04-25T01:43:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.871+0000 7f15faffd700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15f406e140 0x7f15f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.871+0000 7f15faffd700 1 -- 172.21.15.33:0/677416962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f15fc08ed80 con 0x7f160c0fbd40 2024-04-25T01:43:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.871+0000 7f1609d9b700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15f406e140 0x7f15f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:07.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.872+0000 7f1609d9b700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15f406e140 0x7f15f40705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1600007fd0 tx=0x7f1600009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:07.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:07.876+0000 7f15faffd700 1 -- 172.21.15.33:0/677416962 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f15fc0581b0 con 0x7f160c0fbd40 2024-04-25T01:43:08.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.106+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f160c04bf40 con 0x7f15f406e140 2024-04-25T01:43:08.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.107+0000 7f15faffd700 1 -- 172.21.15.33:0/677416962 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f160c04bf40 con 0x7f15f406e140 2024-04-25T01:43:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15f406e140 msgr2=0x7f15f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15f406e140 0x7f15f40705f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1600007fd0 tx=0x7f1600009040 comp rx=0 tx=0).stop 2024-04-25T01:43:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0fbd40 msgr2=0x7f160c10eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0fbd40 0x7f160c10eca0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00b3b0 tx=0x7f15fc009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 shutdown_connections 2024-04-25T01:43:08.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15f406e140 0x7f15f40705f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f160c0fbd40 0x7f160c10eca0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 --2- 172.21.15.33:0/677416962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f160c101d90 0x7f160c10f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 >> 172.21.15.33:0/677416962 conn(0x7f160c071d00 msgr2=0x7f160c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:08.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 shutdown_connections 2024-04-25T01:43:08.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.109+0000 7f1610a29700 1 -- 172.21.15.33:0/677416962 wait complete. 2024-04-25T01:43:08.110 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:43:08.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.215+0000 7f133ea07700 1 -- 172.21.15.33:0/638323838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 msgr2=0x7f13381020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.215+0000 7f133ea07700 1 --2- 172.21.15.33:0/638323838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 0x7f13381020e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1328004940 tx=0x7f1328009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.215+0000 7f133ea07700 1 -- 172.21.15.33:0/638323838 shutdown_connections 2024-04-25T01:43:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.215+0000 7f133ea07700 1 --2- 172.21.15.33:0/638323838 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13381026b0 0x7f13380fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.215+0000 7f133ea07700 1 --2- 172.21.15.33:0/638323838 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 0x7f13381020e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.215+0000 7f133ea07700 1 -- 172.21.15.33:0/638323838 >> 172.21.15.33:0/638323838 conn(0x7f13380f7860 msgr2=0x7f13380f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.215+0000 7f133ea07700 1 -- 172.21.15.33:0/638323838 shutdown_connections 2024-04-25T01:43:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.215+0000 7f133ea07700 1 -- 172.21.15.33:0/638323838 wait complete. 2024-04-25T01:43:08.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.216+0000 7f133ea07700 1 Processor -- start 2024-04-25T01:43:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.216+0000 7f133ea07700 1 -- start start 2024-04-25T01:43:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.216+0000 7f133ea07700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 0x7f1338102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:08.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.216+0000 7f133ea07700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13381026b0 0x7f13381030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.216+0000 7f133ea07700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1338104ac0 con 0x7f1338101d10 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.216+0000 7f133ea07700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13381035f0 con 0x7f13381026b0 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.216+0000 7f1337fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 0x7f1338102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.216+0000 7f1337fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 0x7f1338102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59772/0 (socket says 172.21.15.33:59772) 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f1337fff700 1 -- 172.21.15.33:0/3647040748 learned_addr learned my addr 172.21.15.33:0/3647040748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f13377fe700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13381026b0 0x7f13381030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f1337fff700 1 -- 172.21.15.33:0/3647040748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13381026b0 msgr2=0x7f13381030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f1337fff700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13381026b0 0x7f13381030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f1337fff700 1 -- 172.21.15.33:0/3647040748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1328005040 con 0x7f1338101d10 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f1337fff700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 0x7f1338102b70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f132800b8b0 tx=0x7f1328009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f1334ff9700 1 -- 172.21.15.33:0/3647040748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f132800e040 con 0x7f1338101d10 2024-04-25T01:43:08.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f1334ff9700 1 -- 172.21.15.33:0/3647040748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1328004290 con 0x7f1338101d10 2024-04-25T01:43:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1338103870 con 0x7f1338101d10 2024-04-25T01:43:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f1334ff9700 1 -- 172.21.15.33:0/3647040748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f132801c3f0 con 0x7f1338101d10 2024-04-25T01:43:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.217+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133811a0e0 con 0x7f1338101d10 2024-04-25T01:43:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.218+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13380fd460 con 0x7f1338101d10 2024-04-25T01:43:08.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.219+0000 7f1334ff9700 1 -- 172.21.15.33:0/3647040748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1328013420 con 0x7f1338101d10 2024-04-25T01:43:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.219+0000 7f1334ff9700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132006e140 0x7f13200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.219+0000 7f1334ff9700 1 -- 172.21.15.33:0/3647040748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f132808ec70 con 0x7f1338101d10 2024-04-25T01:43:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.219+0000 7f13377fe700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132006e140 0x7f13200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:08.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.220+0000 7f13377fe700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132006e140 0x7f13200705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f13381042a0 tx=0x7f132c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:08.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.224+0000 7f1334ff9700 1 -- 172.21.15.33:0/3647040748 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1328058120 con 0x7f1338101d10 2024-04-25T01:43:08.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.468+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1338066980 con 0x7f132006e140 2024-04-25T01:43:08.458 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:08 smithi033 ceph-mon[111306]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:08.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.470+0000 7f1334ff9700 1 -- 172.21.15.33:0/3647040748 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1338066980 con 0x7f132006e140 2024-04-25T01:43:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.472+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132006e140 msgr2=0x7f13200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.472+0000 7f133ea07700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132006e140 0x7f13200705f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f13381042a0 tx=0x7f132c009040 comp rx=0 tx=0).stop 2024-04-25T01:43:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.472+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 msgr2=0x7f1338102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.472+0000 7f133ea07700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 0x7f1338102b70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f132800b8b0 tx=0x7f1328009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.472+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 shutdown_connections 2024-04-25T01:43:08.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.472+0000 7f133ea07700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132006e140 0x7f13200705f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.472+0000 7f133ea07700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1338101d10 0x7f1338102b70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.472+0000 7f133ea07700 1 --2- 172.21.15.33:0/3647040748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f13381026b0 0x7f13381030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.472+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 >> 172.21.15.33:0/3647040748 conn(0x7f13380f7860 msgr2=0x7f1338066270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.473+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 shutdown_connections 2024-04-25T01:43:08.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.473+0000 7f133ea07700 1 -- 172.21.15.33:0/3647040748 wait complete. 2024-04-25T01:43:08.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 -- 172.21.15.33:0/4175035376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 msgr2=0x7ff6b80fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 --2- 172.21.15.33:0/4175035376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 0x7ff6b80fbcf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac004940 tx=0x7ff6ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:08.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 -- 172.21.15.33:0/4175035376 shutdown_connections 2024-04-25T01:43:08.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 --2- 172.21.15.33:0/4175035376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 0x7ff6b80fbcf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 --2- 172.21.15.33:0/4175035376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8101ce0 0x7ff6b81020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 -- 172.21.15.33:0/4175035376 >> 172.21.15.33:0/4175035376 conn(0x7ff6b80f7850 msgr2=0x7ff6b80f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:08.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 -- 172.21.15.33:0/4175035376 shutdown_connections 2024-04-25T01:43:08.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 -- 172.21.15.33:0/4175035376 wait complete. 2024-04-25T01:43:08.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 Processor -- start 2024-04-25T01:43:08.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.581+0000 7ff6bcf66700 1 -- start start 2024-04-25T01:43:08.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6bcf66700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8101ce0 0x7ff6b8113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:08.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6bcf66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 0x7ff6b8114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:08.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6bcf66700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b81146e0 con 0x7ff6b8102680 2024-04-25T01:43:08.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6bcf66700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6b810dad0 con 0x7ff6b8101ce0 2024-04-25T01:43:08.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6b5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 0x7ff6b8114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:08.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6b5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 0x7ff6b8114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59800/0 (socket says 172.21.15.33:59800) 2024-04-25T01:43:08.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6b5d9b700 1 -- 172.21.15.33:0/266420826 learned_addr learned my addr 172.21.15.33:0/266420826 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:08.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6b659c700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8101ce0 0x7ff6b8113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:08.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6b5d9b700 1 -- 172.21.15.33:0/266420826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8101ce0 msgr2=0x7ff6b8113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6b5d9b700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8101ce0 0x7ff6b8113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6b5d9b700 1 -- 172.21.15.33:0/266420826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ac005040 con 0x7ff6b8102680 2024-04-25T01:43:08.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.582+0000 7ff6b5d9b700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 0x7ff6b8114090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009100 tx=0x7ff6ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:08.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.583+0000 7ff6a6ffd700 1 -- 172.21.15.33:0/266420826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6ac00e040 con 0x7ff6b8102680 2024-04-25T01:43:08.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.583+0000 7ff6a6ffd700 1 -- 172.21.15.33:0/266420826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6ac004290 con 0x7ff6b8102680 2024-04-25T01:43:08.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.583+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6b810dd50 con 0x7ff6b8102680 2024-04-25T01:43:08.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.583+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6b810e310 con 0x7ff6b8102680 2024-04-25T01:43:08.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.583+0000 7ff6a6ffd700 1 -- 172.21.15.33:0/266420826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6ac01c3f0 con 0x7ff6b8102680 2024-04-25T01:43:08.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.584+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6b80fd430 con 0x7ff6b8102680 2024-04-25T01:43:08.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.584+0000 7ff6a6ffd700 1 -- 172.21.15.33:0/266420826 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff6ac01c660 con 0x7ff6b8102680 2024-04-25T01:43:08.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.585+0000 7ff6a6ffd700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6a006e150 0x7ff6a0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:08.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.585+0000 7ff6a6ffd700 1 -- 172.21.15.33:0/266420826 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7ff6ac08ecf0 con 0x7ff6b8102680 2024-04-25T01:43:08.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.585+0000 7ff6b659c700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6a006e150 0x7ff6a0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:08.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.585+0000 7ff6b659c700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6a006e150 0x7ff6a0070600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8004990 tx=0x7ff6a8009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:08.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.589+0000 7ff6a6ffd700 1 -- 172.21.15.33:0/266420826 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff6ac05bcb0 con 0x7ff6b8102680 2024-04-25T01:43:08.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:08 smithi138 ceph-mon[81304]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:08.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.809+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6b810f210 con 0x7ff6a006e150 2024-04-25T01:43:08.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.820+0000 7ff6a6ffd700 1 -- 172.21.15.33:0/266420826 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff6b810f210 con 0x7ff6a006e150 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (4m) 22s ago 15m 27.2M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (91s) 22s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (86s) 83s ago 14m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (4m) 22s ago 15m 79.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (10m) 22s ago 10m 62.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (10m) 22s ago 10m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (10m) 83s ago 10m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (10m) 83s ago 10m 61.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (5m) 22s ago 16m 503M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (5m) 83s ago 14m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (2m) 22s ago 17m 46.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (2m) 83s ago 14m 33.9M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:43:08.810 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (4m) 22s ago 15m 15.8M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:43:08.811 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (4m) 83s ago 14m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:43:08.811 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (65s) 22s ago 13m 87.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:43:08.811 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (28s) 22s ago 13m 14.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:43:08.811 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (12m) 22s ago 12m 188M 1559M 17.2.7-902-gc61472c8 0de407c1b3af 72521f5b99fc 2024-04-25T01:43:08.811 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (12m) 83s ago 12m 264M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:43:08.811 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (12m) 83s ago 12m 231M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:43:08.811 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (11m) 83s ago 11m 219M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:43:08.811 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (4m) 22s ago 15m 60.9M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:43:08.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6a006e150 msgr2=0x7ff6a0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6a006e150 0x7ff6a0070600 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff6a8004990 tx=0x7ff6a8009ee0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 msgr2=0x7ff6b8114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 0x7ff6b8114090 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009100 tx=0x7ff6ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:08.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 shutdown_connections 2024-04-25T01:43:08.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6a006e150 0x7ff6a0070600 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6b8101ce0 0x7ff6b8113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 --2- 172.21.15.33:0/266420826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6b8102680 0x7ff6b8114090 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 >> 172.21.15.33:0/266420826 conn(0x7ff6b80f7850 msgr2=0x7ff6b8065e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:08.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 shutdown_connections 2024-04-25T01:43:08.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.822+0000 7ff6bcf66700 1 -- 172.21.15.33:0/266420826 wait complete. 2024-04-25T01:43:08.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.937+0000 7f1a3f85c700 1 -- 172.21.15.33:0/2508468292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070300 msgr2=0x7f1a380706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.937+0000 7f1a3f85c700 1 --2- 172.21.15.33:0/2508468292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070300 0x7f1a380706d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1a34004970 tx=0x7f1a34009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:08.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.937+0000 7f1a3f85c700 1 -- 172.21.15.33:0/2508468292 shutdown_connections 2024-04-25T01:43:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.937+0000 7f1a3f85c700 1 --2- 172.21.15.33:0/2508468292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a38070c10 0x7f1a381090f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.937+0000 7f1a3f85c700 1 --2- 172.21.15.33:0/2508468292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070300 0x7f1a380706d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.937+0000 7f1a3f85c700 1 -- 172.21.15.33:0/2508468292 >> 172.21.15.33:0/2508468292 conn(0x7f1a380f89b0 msgr2=0x7f1a380fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3f85c700 1 -- 172.21.15.33:0/2508468292 shutdown_connections 2024-04-25T01:43:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3f85c700 1 -- 172.21.15.33:0/2508468292 wait complete. 2024-04-25T01:43:08.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3f85c700 1 Processor -- start 2024-04-25T01:43:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3f85c700 1 -- start start 2024-04-25T01:43:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3f85c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a38070300 0x7f1a38115cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3f85c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070c10 0x7f1a38116230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3f85c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a38116880 con 0x7f1a38070c10 2024-04-25T01:43:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3f85c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3810fc50 con 0x7f1a38070300 2024-04-25T01:43:08.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3cdf7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070c10 0x7f1a38116230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:08.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3cdf7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070c10 0x7f1a38116230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59808/0 (socket says 172.21.15.33:59808) 2024-04-25T01:43:08.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.938+0000 7f1a3cdf7700 1 -- 172.21.15.33:0/3730587659 learned_addr learned my addr 172.21.15.33:0/3730587659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:08.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a3d5f8700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a38070300 0x7f1a38115cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:08.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a3cdf7700 1 -- 172.21.15.33:0/3730587659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a38070300 msgr2=0x7f1a38115cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:08.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a3cdf7700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a38070300 0x7f1a38115cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a3cdf7700 1 -- 172.21.15.33:0/3730587659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34005040 con 0x7f1a38070c10 2024-04-25T01:43:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a3d5f8700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a38070300 0x7f1a38115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a3cdf7700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070c10 0x7f1a38116230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c002a80 tx=0x7f1a2c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a29ffb700 1 -- 172.21.15.33:0/3730587659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2c007cc0 con 0x7f1a38070c10 2024-04-25T01:43:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a29ffb700 1 -- 172.21.15.33:0/3730587659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a2c00f040 con 0x7f1a38070c10 2024-04-25T01:43:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a29ffb700 1 -- 172.21.15.33:0/3730587659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2c0096f0 con 0x7f1a38070c10 2024-04-25T01:43:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3810ff30 con 0x7f1a38070c10 2024-04-25T01:43:08.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.939+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a38110470 con 0x7f1a38070c10 2024-04-25T01:43:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.940+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a38071a40 con 0x7f1a38070c10 2024-04-25T01:43:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.941+0000 7f1a29ffb700 1 -- 172.21.15.33:0/3730587659 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1a2c00d690 con 0x7f1a38070c10 2024-04-25T01:43:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.941+0000 7f1a29ffb700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1a2406e140 0x7f1a240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.941+0000 7f1a29ffb700 1 -- 172.21.15.33:0/3730587659 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f1a2c08d140 con 0x7f1a38070c10 2024-04-25T01:43:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.941+0000 7f1a3d5f8700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1a2406e140 0x7f1a240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.942+0000 7f1a3d5f8700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1a2406e140 0x7f1a240705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a34004f50 tx=0x7f1a34009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:08.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:08.946+0000 7f1a29ffb700 1 -- 172.21.15.33:0/3730587659 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1a2c05a120 con 0x7f1a38070c10 2024-04-25T01:43:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.246+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1a3804bed0 con 0x7f1a38070c10 2024-04-25T01:43:09.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.247+0000 7f1a29ffb700 1 -- 172.21.15.33:0/3730587659 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f1a2c059cb0 con 0x7f1a38070c10 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4, 2024-04-25T01:43:09.237 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:43:09.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:43:09.238 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:43:09.238 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:43:09.238 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:43:09.238 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:43:09.238 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 8, 2024-04-25T01:43:09.238 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:43:09.238 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:43:09.238 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:43:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1a2406e140 msgr2=0x7f1a240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1a2406e140 0x7f1a240705f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1a34004f50 tx=0x7f1a34009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070c10 msgr2=0x7f1a38116230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070c10 0x7f1a38116230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c002a80 tx=0x7f1a2c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 shutdown_connections 2024-04-25T01:43:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1a2406e140 0x7f1a240705f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1a38070300 0x7f1a38115cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 --2- 172.21.15.33:0/3730587659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1a38070c10 0x7f1a38116230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 >> 172.21.15.33:0/3730587659 conn(0x7f1a380f89b0 msgr2=0x7f1a380f9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 shutdown_connections 2024-04-25T01:43:09.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.249+0000 7f1a3f85c700 1 -- 172.21.15.33:0/3730587659 wait complete. 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.356+0000 7f45d7ce6700 1 -- 172.21.15.33:0/2656489370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0102670 msgr2=0x7f45d00fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.356+0000 7f45d7ce6700 1 --2- 172.21.15.33:0/2656489370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0102670 0x7f45d00fb600 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f45c0004940 tx=0x7f45c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.356+0000 7f45d7ce6700 1 -- 172.21.15.33:0/2656489370 shutdown_connections 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.356+0000 7f45d7ce6700 1 --2- 172.21.15.33:0/2656489370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0102670 0x7f45d00fb600 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.356+0000 7f45d7ce6700 1 --2- 172.21.15.33:0/2656489370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45d0101cd0 0x7f45d01020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.356+0000 7f45d7ce6700 1 -- 172.21.15.33:0/2656489370 >> 172.21.15.33:0/2656489370 conn(0x7f45d0071460 msgr2=0x7f45d0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.357+0000 7f45d7ce6700 1 -- 172.21.15.33:0/2656489370 shutdown_connections 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.357+0000 7f45d7ce6700 1 -- 172.21.15.33:0/2656489370 wait complete. 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.357+0000 7f45d7ce6700 1 Processor -- start 2024-04-25T01:43:09.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.357+0000 7f45d7ce6700 1 -- start start 2024-04-25T01:43:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.357+0000 7f45d7ce6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0101cd0 0x7f45d0113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.357+0000 7f45d7ce6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45d0102670 0x7f45d01140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.357+0000 7f45d7ce6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d0114700 con 0x7f45d0101cd0 2024-04-25T01:43:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d7ce6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d010daf0 con 0x7f45d0102670 2024-04-25T01:43:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d5a82700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0101cd0 0x7f45d0113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d5a82700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0101cd0 0x7f45d0113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59812/0 (socket says 172.21.15.33:59812) 2024-04-25T01:43:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d5a82700 1 -- 172.21.15.33:0/80087361 learned_addr learned my addr 172.21.15.33:0/80087361 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:09.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d5281700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45d0102670 0x7f45d01140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d5a82700 1 -- 172.21.15.33:0/80087361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45d0102670 msgr2=0x7f45d01140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d5a82700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45d0102670 0x7f45d01140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d5a82700 1 -- 172.21.15.33:0/80087361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c0005040 con 0x7f45d0101cd0 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d5a82700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0101cd0 0x7f45d0113b70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45cc002a50 tx=0x7f45cc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45c67fc700 1 -- 172.21.15.33:0/80087361 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45cc007cc0 con 0x7f45d0101cd0 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d010ddd0 con 0x7f45d0101cd0 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45c67fc700 1 -- 172.21.15.33:0/80087361 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45cc00f040 con 0x7f45d0101cd0 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45c67fc700 1 -- 172.21.15.33:0/80087361 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45cc00da30 con 0x7f45d0101cd0 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.358+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d010e360 con 0x7f45d0101cd0 2024-04-25T01:43:09.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.359+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d0065200 con 0x7f45d0101cd0 2024-04-25T01:43:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.360+0000 7f45c67fc700 1 -- 172.21.15.33:0/80087361 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f45cc00db90 con 0x7f45d0101cd0 2024-04-25T01:43:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.360+0000 7f45c67fc700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45bc06e140 0x7f45bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.360+0000 7f45d5281700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45bc06e140 0x7f45bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.360+0000 7f45c67fc700 1 -- 172.21.15.33:0/80087361 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f45cc08d740 con 0x7f45d0101cd0 2024-04-25T01:43:09.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.361+0000 7f45d5281700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45bc06e140 0x7f45bc0705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45c000b3b0 tx=0x7f45c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:09.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.365+0000 7f45c67fc700 1 -- 172.21.15.33:0/80087361 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f45cc05a720 con 0x7f45d0101cd0 2024-04-25T01:43:09.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.633+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f45d0109ed0 con 0x7f45d0101cd0 2024-04-25T01:43:09.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.633+0000 7f45c67fc700 1 -- 172.21.15.33:0/80087361 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f45cc05a2b0 con 0x7f45d0101cd0 2024-04-25T01:43:09.623 INFO:teuthology.orchestra.run.smithi033.stdout:e19 2024-04-25T01:43:09.623 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:43:09.623 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:43:09.623 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:43:09.623 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:43:09.623 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 19 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:43:09.624 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:43:09.625 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:43:09.626 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:43:09.626 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:43:09.626 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:43:09.626 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:43:09.626 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:43:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:43:09.626 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:43:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45bc06e140 msgr2=0x7f45bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45bc06e140 0x7f45bc0705f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45c000b3b0 tx=0x7f45c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0101cd0 msgr2=0x7f45d0113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0101cd0 0x7f45d0113b70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f45cc002a50 tx=0x7f45cc0046b0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 shutdown_connections 2024-04-25T01:43:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45bc06e140 0x7f45bc0705f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45d0101cd0 0x7f45d0113b70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 --2- 172.21.15.33:0/80087361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45d0102670 0x7f45d01140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 >> 172.21.15.33:0/80087361 conn(0x7f45d0071460 msgr2=0x7f45d01009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 shutdown_connections 2024-04-25T01:43:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.635+0000 7f45d7ce6700 1 -- 172.21.15.33:0/80087361 wait complete. 2024-04-25T01:43:09.628 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:43:09.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:09 smithi033 ceph-mon[111306]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:09.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:09 smithi033 ceph-mon[111306]: Upgrade: Updating osd.2 2024-04-25T01:43:09.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:09.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:43:09.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:09.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:09 smithi033 ceph-mon[111306]: Deploying daemon osd.2 on smithi033 2024-04-25T01:43:09.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:09 smithi033 ceph-mon[111306]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:09.679 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:09 smithi033 ceph-mon[111306]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:09.679 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:09 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3730587659' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:09.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.741+0000 7efe18005700 1 -- 172.21.15.33:0/1514675761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe100fbd40 msgr2=0x7efe100fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.741+0000 7efe18005700 1 --2- 172.21.15.33:0/1514675761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe100fbd40 0x7efe100fc1b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efe0c004940 tx=0x7efe0c009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:09.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.741+0000 7efe18005700 1 -- 172.21.15.33:0/1514675761 shutdown_connections 2024-04-25T01:43:09.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.741+0000 7efe18005700 1 --2- 172.21.15.33:0/1514675761 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe100fbd40 0x7efe100fc1b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.741+0000 7efe18005700 1 --2- 172.21.15.33:0/1514675761 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe10101d90 0x7efe10102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.741+0000 7efe18005700 1 -- 172.21.15.33:0/1514675761 >> 172.21.15.33:0/1514675761 conn(0x7efe10071d00 msgr2=0x7efe10072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:09.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.741+0000 7efe18005700 1 -- 172.21.15.33:0/1514675761 shutdown_connections 2024-04-25T01:43:09.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.741+0000 7efe18005700 1 -- 172.21.15.33:0/1514675761 wait complete. 2024-04-25T01:43:09.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe18005700 1 Processor -- start 2024-04-25T01:43:09.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe18005700 1 -- start start 2024-04-25T01:43:09.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe18005700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe100fbd40 0x7efe10113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe18005700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe10101d90 0x7efe101140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe18005700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe10114710 con 0x7efe10101d90 2024-04-25T01:43:09.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe18005700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1010db00 con 0x7efe100fbd40 2024-04-25T01:43:09.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe155a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe10101d90 0x7efe101140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe155a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe10101d90 0x7efe101140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59834/0 (socket says 172.21.15.33:59834) 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe155a0700 1 -- 172.21.15.33:0/4032502307 learned_addr learned my addr 172.21.15.33:0/4032502307 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.742+0000 7efe15da1700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe100fbd40 0x7efe10113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.743+0000 7efe155a0700 1 -- 172.21.15.33:0/4032502307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe100fbd40 msgr2=0x7efe10113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.743+0000 7efe155a0700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe100fbd40 0x7efe10113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.743+0000 7efe155a0700 1 -- 172.21.15.33:0/4032502307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c005040 con 0x7efe10101d90 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.743+0000 7efe155a0700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe10101d90 0x7efe101140c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009150 tx=0x7efe0c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.743+0000 7efe027fc700 1 -- 172.21.15.33:0/4032502307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0c00e040 con 0x7efe10101d90 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.743+0000 7efe027fc700 1 -- 172.21.15.33:0/4032502307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe0c013420 con 0x7efe10101d90 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.743+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1010dd80 con 0x7efe10101d90 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.743+0000 7efe027fc700 1 -- 172.21.15.33:0/4032502307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe0c01c610 con 0x7efe10101d90 2024-04-25T01:43:09.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.743+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1010e340 con 0x7efe10101d90 2024-04-25T01:43:09.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.744+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe100fd430 con 0x7efe10101d90 2024-04-25T01:43:09.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.744+0000 7efe027fc700 1 -- 172.21.15.33:0/4032502307 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efe0c01c8d0 con 0x7efe10101d90 2024-04-25T01:43:09.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.745+0000 7efe027fc700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdfc06e150 0x7efdfc070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:09.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.745+0000 7efe027fc700 1 -- 172.21.15.33:0/4032502307 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7efe0c08ece0 con 0x7efe10101d90 2024-04-25T01:43:09.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.745+0000 7efe15da1700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdfc06e150 0x7efdfc070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:09.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.745+0000 7efe15da1700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdfc06e150 0x7efdfc070600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe04004f40 tx=0x7efe0400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:09.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.750+0000 7efe027fc700 1 -- 172.21.15.33:0/4032502307 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efe0c05bcc0 con 0x7efe10101d90 2024-04-25T01:43:09.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:09 smithi138 ceph-mon[81304]: pgmap v87: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:09.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:09 smithi138 ceph-mon[81304]: Upgrade: Updating osd.2 2024-04-25T01:43:09.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:09.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T01:43:09.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:09.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:09 smithi138 ceph-mon[81304]: Deploying daemon osd.2 on smithi033 2024-04-25T01:43:09.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:09 smithi138 ceph-mon[81304]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:09.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:09 smithi138 ceph-mon[81304]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:09.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:09 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3730587659' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:09.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.979+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe10108df0 con 0x7efdfc06e150 2024-04-25T01:43:09.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.980+0000 7efe027fc700 1 -- 172.21.15.33:0/4032502307 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7efe10108df0 con 0x7efdfc06e150 2024-04-25T01:43:09.970 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:43:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:43:09.970 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:43:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:43:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:43:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:43:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:43:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:43:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:43:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "8/21 daemons upgraded", 2024-04-25T01:43:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T01:43:09.971 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:43:09.972 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:43:09.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdfc06e150 msgr2=0x7efdfc070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdfc06e150 0x7efdfc070600 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efe04004f40 tx=0x7efe0400b040 comp rx=0 tx=0).stop 2024-04-25T01:43:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe10101d90 msgr2=0x7efe101140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe10101d90 0x7efe101140c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009150 tx=0x7efe0c009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 shutdown_connections 2024-04-25T01:43:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdfc06e150 0x7efdfc070600 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe100fbd40 0x7efe10113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 --2- 172.21.15.33:0/4032502307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe10101d90 0x7efe101140c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:09.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 >> 172.21.15.33:0/4032502307 conn(0x7efe10071d00 msgr2=0x7efe100655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 shutdown_connections 2024-04-25T01:43:09.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:09.982+0000 7efe18005700 1 -- 172.21.15.33:0/4032502307 wait complete. 2024-04-25T01:43:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.089+0000 7f1ed2359700 1 -- 172.21.15.33:0/924612805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc101ce0 msgr2=0x7f1ecc1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.089+0000 7f1ed2359700 1 --2- 172.21.15.33:0/924612805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc101ce0 0x7f1ecc1020b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc004940 tx=0x7f1ebc009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.089+0000 7f1ed2359700 1 -- 172.21.15.33:0/924612805 shutdown_connections 2024-04-25T01:43:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.089+0000 7f1ed2359700 1 --2- 172.21.15.33:0/924612805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc102680 0x7f1ecc0fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.089+0000 7f1ed2359700 1 --2- 172.21.15.33:0/924612805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc101ce0 0x7f1ecc1020b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.089+0000 7f1ed2359700 1 -- 172.21.15.33:0/924612805 >> 172.21.15.33:0/924612805 conn(0x7f1ecc0f7850 msgr2=0x7f1ecc0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.089+0000 7f1ed2359700 1 -- 172.21.15.33:0/924612805 shutdown_connections 2024-04-25T01:43:10.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.089+0000 7f1ed2359700 1 -- 172.21.15.33:0/924612805 wait complete. 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.089+0000 7f1ed2359700 1 Processor -- start 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ed2359700 1 -- start start 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ed2359700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc101ce0 0x7f1ecc102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ed2359700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc102680 0x7f1ecc103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ed2359700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc1036a0 con 0x7f1ecc102680 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ed2359700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc1037e0 con 0x7f1ecc101ce0 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ecb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc102680 0x7f1ecc103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ecb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc102680 0x7f1ecc103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59846/0 (socket says 172.21.15.33:59846) 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ecb7fe700 1 -- 172.21.15.33:0/1867865467 learned_addr learned my addr 172.21.15.33:0/1867865467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ecbfff700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc101ce0 0x7f1ecc102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:10.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ecb7fe700 1 -- 172.21.15.33:0/1867865467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc101ce0 msgr2=0x7f1ecc102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ecb7fe700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc101ce0 0x7f1ecc102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ecb7fe700 1 -- 172.21.15.33:0/1867865467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc005040 con 0x7f1ecc102680 2024-04-25T01:43:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.090+0000 7f1ecbfff700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc101ce0 0x7f1ecc102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:43:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.091+0000 7f1ecb7fe700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc102680 0x7f1ecc103080 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400b790 tx=0x7f1eb4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.091+0000 7f1ec8ff9700 1 -- 172.21.15.33:0/1867865467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1eb400f6f0 con 0x7f1ecc102680 2024-04-25T01:43:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.091+0000 7f1ec8ff9700 1 -- 172.21.15.33:0/1867865467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1eb400fd20 con 0x7f1ecc102680 2024-04-25T01:43:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.091+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ecc119e40 con 0x7f1ecc102680 2024-04-25T01:43:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.091+0000 7f1ec8ff9700 1 -- 172.21.15.33:0/1867865467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1eb40103a0 con 0x7f1ecc102680 2024-04-25T01:43:10.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.091+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc11a2c0 con 0x7f1ecc102680 2024-04-25T01:43:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.092+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ecc0fd430 con 0x7f1ecc102680 2024-04-25T01:43:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.092+0000 7f1ec8ff9700 1 -- 172.21.15.33:0/1867865467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1eb4011400 con 0x7f1ecc102680 2024-04-25T01:43:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.093+0000 7f1ec8ff9700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb806e140 0x7f1eb80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.093+0000 7f1ec8ff9700 1 -- 172.21.15.33:0/1867865467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f1eb408d800 con 0x7f1ecc102680 2024-04-25T01:43:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.093+0000 7f1ecbfff700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb806e140 0x7f1eb80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:10.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.093+0000 7f1ecbfff700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb806e140 0x7f1eb80705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc004560 tx=0x7f1ebc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:10.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.098+0000 7f1ec8ff9700 1 -- 172.21.15.33:0/1867865467 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1eb4056cb0 con 0x7f1ecc102680 2024-04-25T01:43:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.414+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1ecc05f5c0 con 0x7f1ecc102680 2024-04-25T01:43:10.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.415+0000 7f1ec8ff9700 1 -- 172.21.15.33:0/1867865467 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1eb405a370 con 0x7f1ecc102680 2024-04-25T01:43:10.405 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_OK 2024-04-25T01:43:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb806e140 msgr2=0x7f1eb80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb806e140 0x7f1eb80705f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc004560 tx=0x7f1ebc009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc102680 msgr2=0x7f1ecc103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc102680 0x7f1ecc103080 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400b790 tx=0x7f1eb4009220 comp rx=0 tx=0).stop 2024-04-25T01:43:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 shutdown_connections 2024-04-25T01:43:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb806e140 0x7f1eb80705f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:10.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc101ce0 0x7f1ecc102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:10.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 --2- 172.21.15.33:0/1867865467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc102680 0x7f1ecc103080 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:10.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 >> 172.21.15.33:0/1867865467 conn(0x7f1ecc0f7850 msgr2=0x7f1ecc0f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:10.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 shutdown_connections 2024-04-25T01:43:10.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:10.417+0000 7f1ed2359700 1 -- 172.21.15.33:0/1867865467 wait complete. 2024-04-25T01:43:10.505 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:10 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/80087361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:43:10.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:10 smithi033 ceph-mon[111306]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:10.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:43:10.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:10 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/80087361' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:43:10.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:10 smithi138 ceph-mon[81304]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:10.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:43:11.505 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:11 smithi033 ceph-mon[111306]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:11.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:11 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1867865467' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:43:11.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:11 smithi138 ceph-mon[81304]: pgmap v88: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:11.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:11 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1867865467' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:43:12.335 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:12 smithi033 systemd[1]: Stopping Ceph osd.2 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:43:12.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:12 smithi033 ceph-mon[111306]: osd.2 marked itself down and dead 2024-04-25T01:43:12.756 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:12 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[60541]: 2024-04-25T01:43:12.345+0000 7efe4fa7d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:43:12.756 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:12 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[60541]: 2024-04-25T01:43:12.345+0000 7efe4fa7d700 -1 osd.2 58 *** Got signal Terminated *** 2024-04-25T01:43:12.756 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:12 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[60541]: 2024-04-25T01:43:12.345+0000 7efe4fa7d700 -1 osd.2 58 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:43:12.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:12 smithi138 ceph-mon[81304]: osd.2 marked itself down and dead 2024-04-25T01:43:13.169 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:12 smithi033 podman[133872]: 2024-04-25 01:43:12.88816562 +0000 UTC m=+0.675984854 container died 72521f5b99fced8d2f1366d8296d7aa4a4a547acd64585dbc8bd579df440bd39 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T01:43:13.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:13 smithi033 ceph-mon[111306]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:43:13.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:13 smithi033 ceph-mon[111306]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:43:13.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:13 smithi033 ceph-mon[111306]: osdmap e59: 6 total, 5 up, 6 in 2024-04-25T01:43:13.506 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:13 smithi033 podman[133872]: 2024-04-25 01:43:13.261777911 +0000 UTC m=+1.049597137 container cleanup 72521f5b99fced8d2f1366d8296d7aa4a4a547acd64585dbc8bd579df440bd39 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=quincy-c61472c) 2024-04-25T01:43:13.506 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:13 smithi033 bash[133872]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2 2024-04-25T01:43:13.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:13 smithi138 ceph-mon[81304]: pgmap v89: 65 pgs: 65 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:43:13.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:13 smithi138 ceph-mon[81304]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:43:13.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:13 smithi138 ceph-mon[81304]: osdmap e59: 6 total, 5 up, 6 in 2024-04-25T01:43:13.860 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:13 smithi033 podman[133890]: 2024-04-25 01:43:13.551395957 +0000 UTC m=+0.660052394 container remove 72521f5b99fced8d2f1366d8296d7aa4a4a547acd64585dbc8bd579df440bd39 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T01:43:14.214 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:13 smithi033 podman[133983]: 2024-04-25 01:43:13.775090127 +0000 UTC m=+0.019004623 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:14.214 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:13 smithi033 podman[133983]: 2024-04-25 01:43:13.88097346 +0000 UTC m=+0.124887928 container create a263208aa1a7c23f9e89693d5d01a3ab8b7fcec0a37a9b32d53fc61559d508ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T01:43:14.215 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:14 smithi033 podman[133983]: 2024-04-25 01:43:14.083585845 +0000 UTC m=+0.327500314 container init a263208aa1a7c23f9e89693d5d01a3ab8b7fcec0a37a9b32d53fc61559d508ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T01:43:14.215 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:14 smithi033 podman[133983]: 2024-04-25 01:43:14.08693621 +0000 UTC m=+0.330850687 container start a263208aa1a7c23f9e89693d5d01a3ab8b7fcec0a37a9b32d53fc61559d508ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T01:43:14.215 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:14 smithi033 podman[133983]: 2024-04-25 01:43:14.225380914 +0000 UTC m=+0.469295392 container attach a263208aa1a7c23f9e89693d5d01a3ab8b7fcec0a37a9b32d53fc61559d508ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T01:43:14.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:14 smithi033 ceph-mon[111306]: osdmap e60: 6 total, 5 up, 6 in 2024-04-25T01:43:14.506 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:14 smithi033 podman[133983]: 2024-04-25 01:43:14.387017505 +0000 UTC m=+0.630931986 container died a263208aa1a7c23f9e89693d5d01a3ab8b7fcec0a37a9b32d53fc61559d508ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T01:43:14.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:14 smithi138 ceph-mon[81304]: osdmap e60: 6 total, 5 up, 6 in 2024-04-25T01:43:15.202 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:14 smithi033 podman[134009]: 2024-04-25 01:43:14.978812594 +0000 UTC m=+0.585497696 container remove a263208aa1a7c23f9e89693d5d01a3ab8b7fcec0a37a9b32d53fc61559d508ac (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T01:43:15.202 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:14 smithi033 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-25T01:43:15.202 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:14 smithi033 systemd[1]: Stopped Ceph osd.2 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:43:15.203 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:14 smithi033 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.2.service: Consumed 57.025s CPU time. 2024-04-25T01:43:15.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:15 smithi033 ceph-mon[111306]: pgmap v92: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:43:15.506 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:15 smithi033 systemd[1]: Starting Ceph osd.2 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:43:15.506 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:15 smithi033 podman[134115]: 2024-04-25 01:43:15.334944175 +0000 UTC m=+0.019019838 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:15.506 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:15 smithi033 podman[134115]: 2024-04-25 01:43:15.438242598 +0000 UTC m=+0.122318248 container create 0f5cd6d589ab714d2f1c3320665b72d858b993ff954989d914bbf79f59caf929 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T01:43:15.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:15 smithi138 ceph-mon[81304]: pgmap v92: 65 pgs: 9 stale+active+clean, 56 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:43:16.006 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:15 smithi033 podman[134115]: 2024-04-25 01:43:15.666712367 +0000 UTC m=+0.350788026 container init 0f5cd6d589ab714d2f1c3320665b72d858b993ff954989d914bbf79f59caf929 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:43:16.006 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:15 smithi033 podman[134115]: 2024-04-25 01:43:15.670416278 +0000 UTC m=+0.354491937 container start 0f5cd6d589ab714d2f1c3320665b72d858b993ff954989d914bbf79f59caf929 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T01:43:16.006 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:15 smithi033 podman[134115]: 2024-04-25 01:43:15.749516415 +0000 UTC m=+0.433592086 container attach 0f5cd6d589ab714d2f1c3320665b72d858b993ff954989d914bbf79f59caf929 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate[134130]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate[134130]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 bash[134115]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 bash[134115]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 bash[134115]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 bash[134115]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 bash[134115]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 bash[134115]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 bash[134115]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate[134130]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate[134130]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate[134130]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-25T01:43:16.467 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate[134130]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T01:43:16.468 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate[134130]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-25T01:43:16.468 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 podman[134115]: 2024-04-25 01:43:16.154554092 +0000 UTC m=+0.838629752 container died 0f5cd6d589ab714d2f1c3320665b72d858b993ff954989d914bbf79f59caf929 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T01:43:16.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:16 smithi033 ceph-mon[111306]: Health check failed: Degraded data redundancy: 27/321 objects degraded (8.411%), 9 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:16.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:16 smithi138 ceph-mon[81304]: Health check failed: Degraded data redundancy: 27/321 objects degraded (8.411%), 9 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:17.132 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:16 smithi033 podman[134269]: 2024-04-25 01:43:16.820192163 +0000 UTC m=+0.660343914 container remove 0f5cd6d589ab714d2f1c3320665b72d858b993ff954989d914bbf79f59caf929 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-activate, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T01:43:17.133 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:17 smithi033 podman[134333]: 2024-04-25 01:43:16.943047666 +0000 UTC m=+0.018795289 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:17.133 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:17 smithi033 podman[134333]: 2024-04-25 01:43:17.054310143 +0000 UTC m=+0.130057757 container create 2cb795d14a3f502d5070b862f000aeade35e180caa54cf4af5f85c7acb886068 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T01:43:17.469 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:17 smithi033 podman[134333]: 2024-04-25 01:43:17.249032193 +0000 UTC m=+0.324779820 container init 2cb795d14a3f502d5070b862f000aeade35e180caa54cf4af5f85c7acb886068 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:43:17.469 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:17 smithi033 podman[134333]: 2024-04-25 01:43:17.252444378 +0000 UTC m=+0.328191993 container start 2cb795d14a3f502d5070b862f000aeade35e180caa54cf4af5f85c7acb886068 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T01:43:17.469 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:17 smithi033 ceph-osd[134353]: -- 172.21.15.33:0/1692859371 <== mon.0 v2:172.21.15.33:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5571164af680 con 0x557116479000 2024-04-25T01:43:17.469 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:17 smithi033 bash[134333]: 2cb795d14a3f502d5070b862f000aeade35e180caa54cf4af5f85c7acb886068 2024-04-25T01:43:17.470 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:17 smithi033 systemd[1]: Started Ceph osd.2 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:43:17.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:17 smithi033 ceph-mon[111306]: pgmap v93: 65 pgs: 9 active+undersized+degraded, 4 stale+active+clean, 11 active+undersized, 41 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/321 objects degraded (8.411%) 2024-04-25T01:43:17.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:17 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:17.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:17 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:17.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:17 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:17.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:17 smithi138 ceph-mon[81304]: pgmap v93: 65 pgs: 9 active+undersized+degraded, 4 stale+active+clean, 11 active+undersized, 41 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 27/321 objects degraded (8.411%) 2024-04-25T01:43:17.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:17 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:17.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:17 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:17.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:17 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:18.235 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:17 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[134349]: 2024-04-25T01:43:17.797+0000 7f57879857c0 -1 Falling back to public interface 2024-04-25T01:43:19.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:19 smithi033 ceph-mon[111306]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 38/321 objects degraded (11.838%) 2024-04-25T01:43:19.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:19 smithi138 ceph-mon[81304]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 38/321 objects degraded (11.838%) 2024-04-25T01:43:21.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:21 smithi033 ceph-mon[111306]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 38/321 objects degraded (11.838%) 2024-04-25T01:43:21.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:21 smithi138 ceph-mon[81304]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 38/321 objects degraded (11.838%) 2024-04-25T01:43:22.479 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:22 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[134349]: 2024-04-25T01:43:22.164+0000 7f57879857c0 -1 osd.2 58 log_to_monitors true 2024-04-25T01:43:22.756 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 01:43:22 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[134349]: 2024-04-25T01:43:22.511+0000 7f5778ddd700 -1 osd.2 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:43:22.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:22 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 38/321 objects degraded (11.838%), 12 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:22.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:22 smithi033 ceph-mon[111306]: from='osd.2 [v2:172.21.15.33:6818/4078679342,v1:172.21.15.33:6819/4078679342]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:43:22.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:22 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 38/321 objects degraded (11.838%), 12 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:22.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:22 smithi138 ceph-mon[81304]: from='osd.2 [v2:172.21.15.33:6818/4078679342,v1:172.21.15.33:6819/4078679342]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T01:43:23.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:23 smithi033 ceph-mon[111306]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 38/321 objects degraded (11.838%) 2024-04-25T01:43:23.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:23 smithi033 ceph-mon[111306]: from='osd.2 [v2:172.21.15.33:6818/4078679342,v1:172.21.15.33:6819/4078679342]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:43:23.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:23 smithi033 ceph-mon[111306]: osdmap e61: 6 total, 5 up, 6 in 2024-04-25T01:43:23.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:23 smithi033 ceph-mon[111306]: from='osd.2 [v2:172.21.15.33:6818/4078679342,v1:172.21.15.33:6819/4078679342]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:43:23.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:23.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:23.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:23 smithi138 ceph-mon[81304]: pgmap v96: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 38/321 objects degraded (11.838%) 2024-04-25T01:43:23.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:23 smithi138 ceph-mon[81304]: from='osd.2 [v2:172.21.15.33:6818/4078679342,v1:172.21.15.33:6819/4078679342]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T01:43:23.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:23 smithi138 ceph-mon[81304]: osdmap e61: 6 total, 5 up, 6 in 2024-04-25T01:43:23.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:23 smithi138 ceph-mon[81304]: from='osd.2 [v2:172.21.15.33:6818/4078679342,v1:172.21.15.33:6819/4078679342]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi033", "root=default"]}]: dispatch 2024-04-25T01:43:23.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:23.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:24.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:24 smithi033 ceph-mon[111306]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:43:24.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:24 smithi033 ceph-mon[111306]: osd.2 [v2:172.21.15.33:6818/4078679342,v1:172.21.15.33:6819/4078679342] boot 2024-04-25T01:43:24.753 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:24 smithi033 ceph-mon[111306]: osdmap e62: 6 total, 6 up, 6 in 2024-04-25T01:43:24.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:43:24.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:24.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:24.754 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:24 smithi033 ceph-mon[111306]: osdmap e63: 6 total, 6 up, 6 in 2024-04-25T01:43:24.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:24 smithi138 ceph-mon[81304]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:43:24.781 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:24 smithi138 ceph-mon[81304]: osd.2 [v2:172.21.15.33:6818/4078679342,v1:172.21.15.33:6819/4078679342] boot 2024-04-25T01:43:24.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:24 smithi138 ceph-mon[81304]: osdmap e62: 6 total, 6 up, 6 in 2024-04-25T01:43:24.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T01:43:24.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:24.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:24.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:24 smithi138 ceph-mon[81304]: osdmap e63: 6 total, 6 up, 6 in 2024-04-25T01:43:26.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:26 smithi033 ceph-mon[111306]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s; 38/321 objects degraded (11.838%) 2024-04-25T01:43:26.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:26.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:43:26.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:26 smithi138 ceph-mon[81304]: pgmap v99: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s rd, 2 op/s; 38/321 objects degraded (11.838%) 2024-04-25T01:43:26.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:26.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:43:27.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:27 smithi033 ceph-mon[111306]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 7 active+undersized, 53 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 6 op/s; 16/321 objects degraded (4.984%) 2024-04-25T01:43:27.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:27 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 16/321 objects degraded (4.984%), 5 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:27.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:27 smithi138 ceph-mon[81304]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 7 active+undersized, 53 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 6 op/s; 16/321 objects degraded (4.984%) 2024-04-25T01:43:27.531 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:27 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 16/321 objects degraded (4.984%), 5 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T01:43:29.158 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T01:43:29.159 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:28 smithi033 ceph-mon[111306]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-04-25T01:43:29.281 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T01:43:29.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:28 smithi138 ceph-mon[81304]: Upgrade: unsafe to stop osd(s) at this time (5 PGs are or would become offline) 2024-04-25T01:43:30.256 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:29 smithi033 ceph-mon[111306]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-04-25T01:43:30.256 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:29 smithi033 ceph-mon[111306]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/321 objects degraded (4.984%), 5 pgs degraded) 2024-04-25T01:43:30.256 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:29 smithi033 ceph-mon[111306]: Cluster is now healthy 2024-04-25T01:43:30.281 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:29 smithi138 ceph-mon[81304]: pgmap v102: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-04-25T01:43:30.281 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:29 smithi138 ceph-mon[81304]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/321 objects degraded (4.984%), 5 pgs degraded) 2024-04-25T01:43:30.281 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:29 smithi138 ceph-mon[81304]: Cluster is now healthy 2024-04-25T01:43:32.256 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:31 smithi033 ceph-mon[111306]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:43:32.281 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:31 smithi138 ceph-mon[81304]: pgmap v103: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:43:34.256 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:33 smithi033 ceph-mon[111306]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T01:43:34.281 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:33 smithi138 ceph-mon[81304]: pgmap v104: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T01:43:36.256 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:35 smithi033 ceph-mon[111306]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T01:43:36.256 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:36.281 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:35 smithi138 ceph-mon[81304]: pgmap v105: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-04-25T01:43:36.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:38.256 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:37 smithi033 ceph-mon[111306]: pgmap v106: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:43:38.281 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:37 smithi138 ceph-mon[81304]: pgmap v106: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T01:43:40.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:39 smithi033 ceph-mon[111306]: pgmap v107: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:40.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:39 smithi138 ceph-mon[81304]: pgmap v107: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 -- 172.21.15.33:0/975299098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc102dc0 msgr2=0x7f55bc103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 --2- 172.21.15.33:0/975299098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc102dc0 0x7f55bc103190 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f55b0004940 tx=0x7f55b0009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 -- 172.21.15.33:0/975299098 shutdown_connections 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 --2- 172.21.15.33:0/975299098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55bc103760 0x7f55bc0fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 --2- 172.21.15.33:0/975299098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc102dc0 0x7f55bc103190 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 -- 172.21.15.33:0/975299098 >> 172.21.15.33:0/975299098 conn(0x7f55bc0f8930 msgr2=0x7f55bc0fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 -- 172.21.15.33:0/975299098 shutdown_connections 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 -- 172.21.15.33:0/975299098 wait complete. 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 Processor -- start 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.541+0000 7f55c2d54700 1 -- start start 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55c2d54700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55bc102dc0 0x7f55bc070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55c2d54700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc103760 0x7f55bc0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55c2d54700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0721b0 con 0x7f55bc103760 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55c2d54700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc070ce0 con 0x7f55bc102dc0 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc103760 0x7f55bc0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc103760 0x7f55bc0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35164/0 (socket says 172.21.15.33:35164) 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55bbfff700 1 -- 172.21.15.33:0/1341666331 learned_addr learned my addr 172.21.15.33:0/1341666331 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55c0af0700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55bc102dc0 0x7f55bc070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55bbfff700 1 -- 172.21.15.33:0/1341666331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55bc102dc0 msgr2=0x7f55bc070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55bbfff700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55bc102dc0 0x7f55bc070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55bbfff700 1 -- 172.21.15.33:0/1341666331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b0005040 con 0x7f55bc103760 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.542+0000 7f55bbfff700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc103760 0x7f55bc0707a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f55a8002a50 tx=0x7f55a8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.543+0000 7f55b97fa700 1 -- 172.21.15.33:0/1341666331 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55a8007cc0 con 0x7f55bc103760 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.543+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc070fc0 con 0x7f55bc103760 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.543+0000 7f55b97fa700 1 -- 172.21.15.33:0/1341666331 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f55a800f040 con 0x7f55bc103760 2024-04-25T01:43:40.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.543+0000 7f55b97fa700 1 -- 172.21.15.33:0/1341666331 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55a800d9c0 con 0x7f55bc103760 2024-04-25T01:43:40.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.543+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc11a110 con 0x7f55bc103760 2024-04-25T01:43:40.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.544+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc0fe510 con 0x7f55bc103760 2024-04-25T01:43:40.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.544+0000 7f55b97fa700 1 -- 172.21.15.33:0/1341666331 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f55a8009730 con 0x7f55bc103760 2024-04-25T01:43:40.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.545+0000 7f55b97fa700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f55ac06e140 0x7f55ac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.545+0000 7f55b97fa700 1 -- 172.21.15.33:0/1341666331 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f55a808d7c0 con 0x7f55bc103760 2024-04-25T01:43:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.545+0000 7f55c0af0700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f55ac06e140 0x7f55ac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.545+0000 7f55c0af0700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f55ac06e140 0x7f55ac0705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55b0004f90 tx=0x7f55b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:40.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.550+0000 7f55b97fa700 1 -- 172.21.15.33:0/1341666331 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f55a8056bf0 con 0x7f55bc103760 2024-04-25T01:43:40.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.777+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55bc0f9fd0 con 0x7f55ac06e140 2024-04-25T01:43:40.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.778+0000 7f55b97fa700 1 -- 172.21.15.33:0/1341666331 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f55bc0f9fd0 con 0x7f55ac06e140 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f55ac06e140 msgr2=0x7f55ac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f55ac06e140 0x7f55ac0705f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f55b0004f90 tx=0x7f55b0009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc103760 msgr2=0x7f55bc0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc103760 0x7f55bc0707a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f55a8002a50 tx=0x7f55a8004600 comp rx=0 tx=0).stop 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 shutdown_connections 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f55ac06e140 0x7f55ac0705f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f55bc102dc0 0x7f55bc070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 --2- 172.21.15.33:0/1341666331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f55bc103760 0x7f55bc0707a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 >> 172.21.15.33:0/1341666331 conn(0x7f55bc0f8930 msgr2=0x7f55bc0f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:40.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 shutdown_connections 2024-04-25T01:43:40.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.780+0000 7f55c2d54700 1 -- 172.21.15.33:0/1341666331 wait complete. 2024-04-25T01:43:40.782 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:43:40.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.887+0000 7f575a489700 1 -- 172.21.15.33:0/1870012168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fbf40 msgr2=0x7f5754101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.887+0000 7f575a489700 1 --2- 172.21.15.33:0/1870012168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fbf40 0x7f5754101a30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5744004940 tx=0x7f5744009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:40.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.887+0000 7f575a489700 1 -- 172.21.15.33:0/1870012168 shutdown_connections 2024-04-25T01:43:40.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.887+0000 7f575a489700 1 --2- 172.21.15.33:0/1870012168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fbf40 0x7f5754101a30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.887+0000 7f575a489700 1 --2- 172.21.15.33:0/1870012168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fb5a0 0x7f57540fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.887+0000 7f575a489700 1 -- 172.21.15.33:0/1870012168 >> 172.21.15.33:0/1870012168 conn(0x7f5754073350 msgr2=0x7f5754073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:40.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.887+0000 7f575a489700 1 -- 172.21.15.33:0/1870012168 shutdown_connections 2024-04-25T01:43:40.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.887+0000 7f575a489700 1 -- 172.21.15.33:0/1870012168 wait complete. 2024-04-25T01:43:40.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f575a489700 1 Processor -- start 2024-04-25T01:43:40.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f575a489700 1 -- start start 2024-04-25T01:43:40.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f575a489700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fb5a0 0x7f5754113a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f575a489700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fbf40 0x7f5754113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f575a489700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57541145d0 con 0x7f57540fb5a0 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f575a489700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575410dbb0 con 0x7f57540fbf40 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f5753fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fb5a0 0x7f5754113a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f5753fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fb5a0 0x7f5754113a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35184/0 (socket says 172.21.15.33:35184) 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f5753fff700 1 -- 172.21.15.33:0/3450934933 learned_addr learned my addr 172.21.15.33:0/3450934933 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.888+0000 7f57537fe700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fbf40 0x7f5754113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.889+0000 7f5753fff700 1 -- 172.21.15.33:0/3450934933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fbf40 msgr2=0x7f5754113f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.889+0000 7f5753fff700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fbf40 0x7f5754113f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.889+0000 7f5753fff700 1 -- 172.21.15.33:0/3450934933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5744005040 con 0x7f57540fb5a0 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.889+0000 7f5753fff700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fb5a0 0x7f5754113a40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f573c00c8c0 tx=0x7f573c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.889+0000 7f5750ff9700 1 -- 172.21.15.33:0/3450934933 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573c00a980 con 0x7f57540fb5a0 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.889+0000 7f5750ff9700 1 -- 172.21.15.33:0/3450934933 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f573c00ce60 con 0x7f57540fb5a0 2024-04-25T01:43:40.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.889+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575410de90 con 0x7f57540fb5a0 2024-04-25T01:43:40.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.889+0000 7f5750ff9700 1 -- 172.21.15.33:0/3450934933 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573c0115e0 con 0x7f57540fb5a0 2024-04-25T01:43:40.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.889+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575410e3d0 con 0x7f57540fb5a0 2024-04-25T01:43:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.890+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57540fe670 con 0x7f57540fb5a0 2024-04-25T01:43:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.891+0000 7f5750ff9700 1 -- 172.21.15.33:0/3450934933 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f573c012400 con 0x7f57540fb5a0 2024-04-25T01:43:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.891+0000 7f5750ff9700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f574006e140 0x7f57400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.891+0000 7f5750ff9700 1 -- 172.21.15.33:0/3450934933 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f573c08db40 con 0x7f57540fb5a0 2024-04-25T01:43:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.891+0000 7f57537fe700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f574006e140 0x7f57400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:40.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.892+0000 7f57537fe700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f574006e140 0x7f57400705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f575410f2e0 tx=0x7f5744009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:40.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:40.896+0000 7f5750ff9700 1 -- 172.21.15.33:0/3450934933 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f573c05ab20 con 0x7f57540fb5a0 2024-04-25T01:43:41.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.126+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f575410f090 con 0x7f574006e140 2024-04-25T01:43:41.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.127+0000 7f5750ff9700 1 -- 172.21.15.33:0/3450934933 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f575410f090 con 0x7f574006e140 2024-04-25T01:43:41.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f574006e140 msgr2=0x7f57400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f574006e140 0x7f57400705f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f575410f2e0 tx=0x7f5744009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fb5a0 msgr2=0x7f5754113a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fb5a0 0x7f5754113a40 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f573c00c8c0 tx=0x7f573c0044c0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 shutdown_connections 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f574006e140 0x7f57400705f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fb5a0 0x7f5754113a40 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 --2- 172.21.15.33:0/3450934933 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fbf40 0x7f5754113f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 >> 172.21.15.33:0/3450934933 conn(0x7f5754073350 msgr2=0x7f5754100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 shutdown_connections 2024-04-25T01:43:41.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.130+0000 7f575a489700 1 -- 172.21.15.33:0/3450934933 wait complete. 2024-04-25T01:43:41.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.235+0000 7fecb285d700 1 -- 172.21.15.33:0/2340278246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 msgr2=0x7fecac1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.235+0000 7fecb285d700 1 --2- 172.21.15.33:0/2340278246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 0x7fecac1031b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feca0004940 tx=0x7feca0009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:41.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.235+0000 7fecb285d700 1 -- 172.21.15.33:0/2340278246 shutdown_connections 2024-04-25T01:43:41.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.235+0000 7fecb285d700 1 --2- 172.21.15.33:0/2340278246 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecac103780 0x7fecac0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.235+0000 7fecb285d700 1 --2- 172.21.15.33:0/2340278246 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 0x7fecac1031b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.235+0000 7fecb285d700 1 -- 172.21.15.33:0/2340278246 >> 172.21.15.33:0/2340278246 conn(0x7fecac0f89b0 msgr2=0x7fecac0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:41.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.235+0000 7fecb285d700 1 -- 172.21.15.33:0/2340278246 shutdown_connections 2024-04-25T01:43:41.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.235+0000 7fecb285d700 1 -- 172.21.15.33:0/2340278246 wait complete. 2024-04-25T01:43:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.236+0000 7fecb285d700 1 Processor -- start 2024-04-25T01:43:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.236+0000 7fecb285d700 1 -- start start 2024-04-25T01:43:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.236+0000 7fecb285d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 0x7fecac113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.236+0000 7fecb285d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecac103780 0x7fecac114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.236+0000 7fecb285d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac1146e0 con 0x7fecac102de0 2024-04-25T01:43:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.236+0000 7fecb285d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac10dad0 con 0x7fecac103780 2024-04-25T01:43:41.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.236+0000 7fecabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 0x7fecac113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.236+0000 7fecabfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 0x7fecac113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35202/0 (socket says 172.21.15.33:35202) 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.236+0000 7fecabfff700 1 -- 172.21.15.33:0/1567667974 learned_addr learned my addr 172.21.15.33:0/1567667974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7fecab7fe700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecac103780 0x7fecac114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7fecabfff700 1 -- 172.21.15.33:0/1567667974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecac103780 msgr2=0x7fecac114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7fecabfff700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecac103780 0x7fecac114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7fecabfff700 1 -- 172.21.15.33:0/1567667974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca0005040 con 0x7fecac102de0 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7fecabfff700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 0x7fecac113b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feca0009180 tx=0x7feca0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7feca8ff9700 1 -- 172.21.15.33:0/1567667974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca000e040 con 0x7fecac102de0 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7feca8ff9700 1 -- 172.21.15.33:0/1567667974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feca000bbc0 con 0x7fecac102de0 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7feca8ff9700 1 -- 172.21.15.33:0/1567667974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feca001c630 con 0x7fecac102de0 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac10dd50 con 0x7fecac102de0 2024-04-25T01:43:41.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.237+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac10e310 con 0x7fecac102de0 2024-04-25T01:43:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.238+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac1000f0 con 0x7fecac102de0 2024-04-25T01:43:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.239+0000 7feca8ff9700 1 -- 172.21.15.33:0/1567667974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feca000bd30 con 0x7fecac102de0 2024-04-25T01:43:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.239+0000 7feca8ff9700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec9406e150 0x7fec94070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.239+0000 7feca8ff9700 1 -- 172.21.15.33:0/1567667974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7feca008ed60 con 0x7fecac102de0 2024-04-25T01:43:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.239+0000 7fecab7fe700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec9406e150 0x7fec94070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.240+0000 7fecab7fe700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec9406e150 0x7fec94070600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec9c007fd0 tx=0x7fec9c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:41.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.244+0000 7feca8ff9700 1 -- 172.21.15.33:0/1567667974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feca005bde0 con 0x7fecac102de0 2024-04-25T01:43:41.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.460+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fecac10f1f0 con 0x7fec9406e150 2024-04-25T01:43:41.452 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:41 smithi033 ceph-mon[111306]: pgmap v108: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:41.452 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:41.452 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:43:41.452 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:41 smithi033 ceph-mon[111306]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:41.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.471+0000 7feca8ff9700 1 -- 172.21.15.33:0/1567667974 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fecac10f1f0 con 0x7fec9406e150 2024-04-25T01:43:41.462 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:43:41.462 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (4m) 18s ago 15m 26.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (2m) 18s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (119s) 115s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (4m) 18s ago 15m 77.2M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (10m) 18s ago 10m 62.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (10m) 18s ago 10m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (10m) 115s ago 10m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (10m) 115s ago 10m 61.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (6m) 18s ago 17m 504M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (5m) 115s ago 15m 449M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (3m) 18s ago 17m 46.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (2m) 115s ago 15m 33.9M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (5m) 18s ago 16m 16.0M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 115s ago 15m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:43:41.463 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (98s) 18s ago 14m 88.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:43:41.464 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (60s) 18s ago 13m 93.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:43:41.464 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (24s) 18s ago 13m 12.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:43:41.464 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (12m) 115s ago 12m 264M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 55ddc8bb31e0 2024-04-25T01:43:41.464 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (12m) 115s ago 12m 231M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:43:41.464 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (12m) 115s ago 12m 219M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:43:41.464 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (5m) 18s ago 15m 60.9M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.473+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec9406e150 msgr2=0x7fec94070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.473+0000 7fecb285d700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec9406e150 0x7fec94070600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec9c007fd0 tx=0x7fec9c009040 comp rx=0 tx=0).stop 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.473+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 msgr2=0x7fecac113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.473+0000 7fecb285d700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 0x7fecac113b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feca0009180 tx=0x7feca0009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.473+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 shutdown_connections 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.473+0000 7fecb285d700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec9406e150 0x7fec94070600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.473+0000 7fecb285d700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fecac102de0 0x7fecac113b50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.473+0000 7fecb285d700 1 --2- 172.21.15.33:0/1567667974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fecac103780 0x7fecac114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.474+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 >> 172.21.15.33:0/1567667974 conn(0x7fecac0f89b0 msgr2=0x7fecac0fa730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.474+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 shutdown_connections 2024-04-25T01:43:41.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.474+0000 7fecb285d700 1 -- 172.21.15.33:0/1567667974 wait complete. 2024-04-25T01:43:41.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:41 smithi138 ceph-mon[81304]: pgmap v108: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:41.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:41.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:43:41.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:41 smithi138 ceph-mon[81304]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 -- 172.21.15.33:0/2864991542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 msgr2=0x7f35781020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 --2- 172.21.15.33:0/2864991542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 0x7f35781020d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3574004940 tx=0x7f3574009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 -- 172.21.15.33:0/2864991542 shutdown_connections 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 --2- 172.21.15.33:0/2864991542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35781026a0 0x7f35780fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 --2- 172.21.15.33:0/2864991542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 0x7f35781020d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 -- 172.21.15.33:0/2864991542 >> 172.21.15.33:0/2864991542 conn(0x7f35780f7870 msgr2=0x7f35780f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 -- 172.21.15.33:0/2864991542 shutdown_connections 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 -- 172.21.15.33:0/2864991542 wait complete. 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 Processor -- start 2024-04-25T01:43:41.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.577+0000 7f358006d700 1 -- start start 2024-04-25T01:43:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f358006d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 0x7f35781139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f358006d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35781026a0 0x7f3578113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f358006d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3578114530 con 0x7f3578101d00 2024-04-25T01:43:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f358006d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357810db30 con 0x7f35781026a0 2024-04-25T01:43:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f357de09700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 0x7f35781139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f357de09700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 0x7f35781139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35226/0 (socket says 172.21.15.33:35226) 2024-04-25T01:43:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f357de09700 1 -- 172.21.15.33:0/4009988762 learned_addr learned my addr 172.21.15.33:0/4009988762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f357de09700 1 -- 172.21.15.33:0/4009988762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35781026a0 msgr2=0x7f3578113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f357de09700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35781026a0 0x7f3578113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f357de09700 1 -- 172.21.15.33:0/4009988762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574005040 con 0x7f3578101d00 2024-04-25T01:43:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f357de09700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 0x7f35781139a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3574009180 tx=0x7f3574009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f356e7fc700 1 -- 172.21.15.33:0/4009988762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f357400e040 con 0x7f3578101d00 2024-04-25T01:43:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357810ddb0 con 0x7f3578101d00 2024-04-25T01:43:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f356e7fc700 1 -- 172.21.15.33:0/4009988762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f357400bbc0 con 0x7f3578101d00 2024-04-25T01:43:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f356e7fc700 1 -- 172.21.15.33:0/4009988762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f357401c630 con 0x7f3578101d00 2024-04-25T01:43:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.578+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357810e370 con 0x7f3578101d00 2024-04-25T01:43:41.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.579+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35780fd450 con 0x7f3578101d00 2024-04-25T01:43:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.580+0000 7f356e7fc700 1 -- 172.21.15.33:0/4009988762 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f357400bd30 con 0x7f3578101d00 2024-04-25T01:43:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.580+0000 7f356e7fc700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f356406e140 0x7f35640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.581+0000 7f356e7fc700 1 -- 172.21.15.33:0/4009988762 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f357408ee90 con 0x7f3578101d00 2024-04-25T01:43:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.581+0000 7f357d608700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f356406e140 0x7f35640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.581+0000 7f357d608700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f356406e140 0x7f35640705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f357810ecf0 tx=0x7f3568007d40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:41.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.585+0000 7f356e7fc700 1 -- 172.21.15.33:0/4009988762 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f357405be70 con 0x7f3578101d00 2024-04-25T01:43:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.878+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f357810e590 con 0x7f3578101d00 2024-04-25T01:43:41.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.879+0000 7f356e7fc700 1 -- 172.21.15.33:0/4009988762 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f357405ba00 con 0x7f3578101d00 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:43:41.870 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:43:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:43:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:43:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:43:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:43:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 7, 2024-04-25T01:43:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 7 2024-04-25T01:43:41.871 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:43:41.871 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f356406e140 msgr2=0x7f35640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f356406e140 0x7f35640705f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f357810ecf0 tx=0x7f3568007d40 comp rx=0 tx=0).stop 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 msgr2=0x7f35781139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 0x7f35781139a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3574009180 tx=0x7f3574009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 shutdown_connections 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f356406e140 0x7f35640705f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3578101d00 0x7f35781139a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 --2- 172.21.15.33:0/4009988762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35781026a0 0x7f3578113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 >> 172.21.15.33:0/4009988762 conn(0x7f35780f7870 msgr2=0x7f3578066280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 shutdown_connections 2024-04-25T01:43:41.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.881+0000 7f358006d700 1 -- 172.21.15.33:0/4009988762 wait complete. 2024-04-25T01:43:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 -- 172.21.15.33:0/2772704953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6580102f60 msgr2=0x7f6580103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 --2- 172.21.15.33:0/2772704953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6580102f60 0x7f6580103330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f657c004940 tx=0x7f657c009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:41.976 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 -- 172.21.15.33:0/2772704953 shutdown_connections 2024-04-25T01:43:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 --2- 172.21.15.33:0/2772704953 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65800fb520 0x7f65800fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 --2- 172.21.15.33:0/2772704953 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6580102f60 0x7f6580103330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 -- 172.21.15.33:0/2772704953 >> 172.21.15.33:0/2772704953 conn(0x7f65800714d0 msgr2=0x7f65800718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 -- 172.21.15.33:0/2772704953 shutdown_connections 2024-04-25T01:43:41.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 -- 172.21.15.33:0/2772704953 wait complete. 2024-04-25T01:43:41.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 Processor -- start 2024-04-25T01:43:41.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.986+0000 7f65881af700 1 -- start start 2024-04-25T01:43:41.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f65881af700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65800fb520 0x7f65800fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f65881af700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6580102f60 0x7f65800fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f65881af700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800fe650 con 0x7f65800fb520 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f65881af700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65800fd180 con 0x7f6580102f60 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f6585f4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65800fb520 0x7f65800fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f6585f4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65800fb520 0x7f65800fc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35240/0 (socket says 172.21.15.33:35240) 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f6585f4b700 1 -- 172.21.15.33:0/2098821543 learned_addr learned my addr 172.21.15.33:0/2098821543 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f6585f4b700 1 -- 172.21.15.33:0/2098821543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6580102f60 msgr2=0x7f65800fcc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f6585f4b700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6580102f60 0x7f65800fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f6585f4b700 1 -- 172.21.15.33:0/2098821543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c005040 con 0x7f65800fb520 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f6585f4b700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65800fb520 0x7f65800fc700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f657c009150 tx=0x7f657c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f65767fc700 1 -- 172.21.15.33:0/2098821543 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657c021070 con 0x7f65800fb520 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f65767fc700 1 -- 172.21.15.33:0/2098821543 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f657c013c00 con 0x7f65800fb520 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f65767fc700 1 -- 172.21.15.33:0/2098821543 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f657c00e070 con 0x7f65800fb520 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65800fd400 con 0x7f65800fb520 2024-04-25T01:43:41.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.987+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658011a070 con 0x7f65800fb520 2024-04-25T01:43:41.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.988+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6580100250 con 0x7f65800fb520 2024-04-25T01:43:41.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.989+0000 7f65767fc700 1 -- 172.21.15.33:0/2098821543 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f657c01c400 con 0x7f65800fb520 2024-04-25T01:43:41.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.989+0000 7f65767fc700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f656c06e140 0x7f656c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:41.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.989+0000 7f65767fc700 1 -- 172.21.15.33:0/2098821543 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f657c08edd0 con 0x7f65800fb520 2024-04-25T01:43:41.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.989+0000 7f658574a700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f656c06e140 0x7f656c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:41.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.990+0000 7f658574a700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f656c06e140 0x7f656c0705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65800fde30 tx=0x7f657000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:41.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:41.994+0000 7f65767fc700 1 -- 172.21.15.33:0/2098821543 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f657c058320 con 0x7f65800fb520 2024-04-25T01:43:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.249+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f65800fe7e0 con 0x7f65800fb520 2024-04-25T01:43:42.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.249+0000 7f65767fc700 1 -- 172.21.15.33:0/2098821543 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f657c05b9e0 con 0x7f65800fb520 2024-04-25T01:43:42.240 INFO:teuthology.orchestra.run.smithi033.stdout:e19 2024-04-25T01:43:42.240 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:43:42.240 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 19 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:43:42.241 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:43:42.242 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:43:42.243 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:43:42.243 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:43:42.243 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:43:42.243 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:43:42.243 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:43:42.243 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:43:42.243 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:43:42.243 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.252+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f656c06e140 msgr2=0x7f656c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.252+0000 7f65881af700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f656c06e140 0x7f656c0705f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f65800fde30 tx=0x7f657000b040 comp rx=0 tx=0).stop 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.252+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65800fb520 msgr2=0x7f65800fc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.252+0000 7f65881af700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65800fb520 0x7f65800fc700 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f657c009150 tx=0x7f657c009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.253+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 shutdown_connections 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.253+0000 7f65881af700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f656c06e140 0x7f656c0705f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.253+0000 7f65881af700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65800fb520 0x7f65800fc700 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.253+0000 7f65881af700 1 --2- 172.21.15.33:0/2098821543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6580102f60 0x7f65800fcc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.253+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 >> 172.21.15.33:0/2098821543 conn(0x7f65800714d0 msgr2=0x7f65800fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.253+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 shutdown_connections 2024-04-25T01:43:42.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.253+0000 7f65881af700 1 -- 172.21.15.33:0/2098821543 wait complete. 2024-04-25T01:43:42.245 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:43:42.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.361+0000 7f6fbbd78700 1 -- 172.21.15.33:0/471127603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 msgr2=0x7f6fb40fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.361+0000 7f6fbbd78700 1 --2- 172.21.15.33:0/471127603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 0x7f6fb40fcb80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4004940 tx=0x7f6fa4009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.361+0000 7f6fbbd78700 1 -- 172.21.15.33:0/471127603 shutdown_connections 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.361+0000 7f6fbbd78700 1 --2- 172.21.15.33:0/471127603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 0x7f6fb40fcb80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.361+0000 7f6fbbd78700 1 --2- 172.21.15.33:0/471127603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb4101e20 0x7f6fb41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.361+0000 7f6fbbd78700 1 -- 172.21.15.33:0/471127603 >> 172.21.15.33:0/471127603 conn(0x7f6fb4071460 msgr2=0x7f6fb4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.361+0000 7f6fbbd78700 1 -- 172.21.15.33:0/471127603 shutdown_connections 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.361+0000 7f6fbbd78700 1 -- 172.21.15.33:0/471127603 wait complete. 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fbbd78700 1 Processor -- start 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fbbd78700 1 -- start start 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fbbd78700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 0x7f6fb410c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fbbd78700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb4101e20 0x7f6fb410ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fbbd78700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb410d390 con 0x7f6fb40fc710 2024-04-25T01:43:42.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fbbd78700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb410d500 con 0x7f6fb4101e20 2024-04-25T01:43:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fb9b14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 0x7f6fb410c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fb9b14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 0x7f6fb410c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35246/0 (socket says 172.21.15.33:35246) 2024-04-25T01:43:42.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fb9b14700 1 -- 172.21.15.33:0/1511260452 learned_addr learned my addr 172.21.15.33:0/1511260452 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fb9313700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb4101e20 0x7f6fb410ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fb9b14700 1 -- 172.21.15.33:0/1511260452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb4101e20 msgr2=0x7f6fb410ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fb9b14700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb4101e20 0x7f6fb410ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.362+0000 7f6fb9b14700 1 -- 172.21.15.33:0/1511260452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4005040 con 0x7f6fb40fc710 2024-04-25T01:43:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.363+0000 7f6fb9b14700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 0x7f6fb410c910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0002a50 tx=0x7f6fb00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.363+0000 7f6faa7fc700 1 -- 172.21.15.33:0/1511260452 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fb0007cc0 con 0x7f6fb40fc710 2024-04-25T01:43:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.363+0000 7f6faa7fc700 1 -- 172.21.15.33:0/1511260452 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fb000f040 con 0x7f6fb40fc710 2024-04-25T01:43:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.363+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb4119f80 con 0x7f6fb40fc710 2024-04-25T01:43:42.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.363+0000 7f6faa7fc700 1 -- 172.21.15.33:0/1511260452 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fb0009640 con 0x7f6fb40fc710 2024-04-25T01:43:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.363+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb411a350 con 0x7f6fb40fc710 2024-04-25T01:43:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.364+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4104200 con 0x7f6fb40fc710 2024-04-25T01:43:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.365+0000 7f6faa7fc700 1 -- 172.21.15.33:0/1511260452 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6fb00097a0 con 0x7f6fb40fc710 2024-04-25T01:43:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.365+0000 7f6faa7fc700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6fa006e200 0x7f6fa00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.365+0000 7f6faa7fc700 1 -- 172.21.15.33:0/1511260452 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f6fb008d630 con 0x7f6fb40fc710 2024-04-25T01:43:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.365+0000 7f6fb9313700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6fa006e200 0x7f6fa00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.365+0000 7f6fb9313700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6fa006e200 0x7f6fa00706b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400b3b0 tx=0x7f6fa4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:42.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.370+0000 7f6faa7fc700 1 -- 172.21.15.33:0/1511260452 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6fb005a6b0 con 0x7f6fb40fc710 2024-04-25T01:43:42.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:42 smithi033 ceph-mon[111306]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:42.506 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:42 smithi033 ceph-mon[111306]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:42.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:42 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4009988762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:42.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:42 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2098821543' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:43:42.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:42 smithi138 ceph-mon[81304]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:42.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:42 smithi138 ceph-mon[81304]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:42.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:42 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4009988762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:43:42.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:42 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2098821543' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:43:42.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.595+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fb40fbe50 con 0x7f6fa006e200 2024-04-25T01:43:42.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.596+0000 7f6faa7fc700 1 -- 172.21.15.33:0/1511260452 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f6fb40fbe50 con 0x7f6fa006e200 2024-04-25T01:43:42.587 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:43:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:43:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:43:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:43:42.587 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:43:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:43:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:43:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:43:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:43:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "9/21 daemons upgraded", 2024-04-25T01:43:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T01:43:42.588 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:43:42.588 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:43:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.598+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6fa006e200 msgr2=0x7f6fa00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.598+0000 7f6fbbd78700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6fa006e200 0x7f6fa00706b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6fa400b3b0 tx=0x7f6fa4009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.598+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 msgr2=0x7f6fb410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.598+0000 7f6fbbd78700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 0x7f6fb410c910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0002a50 tx=0x7f6fb00046b0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.598+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 shutdown_connections 2024-04-25T01:43:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.598+0000 7f6fbbd78700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6fa006e200 0x7f6fa00706b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.599+0000 7f6fbbd78700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6fb40fc710 0x7f6fb410c910 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.599+0000 7f6fbbd78700 1 --2- 172.21.15.33:0/1511260452 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6fb4101e20 0x7f6fb410ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.599+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 >> 172.21.15.33:0/1511260452 conn(0x7f6fb4071460 msgr2=0x7f6fb40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.599+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 shutdown_connections 2024-04-25T01:43:42.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.599+0000 7f6fbbd78700 1 -- 172.21.15.33:0/1511260452 wait complete. 2024-04-25T01:43:42.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.704+0000 7f5072a3f700 1 -- 172.21.15.33:0/609797515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 msgr2=0x7f506c070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.704+0000 7f5072a3f700 1 --2- 172.21.15.33:0/609797515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 0x7f506c070cb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5060004970 tx=0x7f5060009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:42.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.704+0000 7f5072a3f700 1 -- 172.21.15.33:0/609797515 shutdown_connections 2024-04-25T01:43:42.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.704+0000 7f5072a3f700 1 --2- 172.21.15.33:0/609797515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 0x7f506c070cb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.704+0000 7f5072a3f700 1 --2- 172.21.15.33:0/609797515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f506c071f30 0x7f506c070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.704+0000 7f5072a3f700 1 -- 172.21.15.33:0/609797515 >> 172.21.15.33:0/609797515 conn(0x7f506c0f89d0 msgr2=0x7f506c0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:42.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.704+0000 7f5072a3f700 1 -- 172.21.15.33:0/609797515 shutdown_connections 2024-04-25T01:43:42.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.704+0000 7f5072a3f700 1 -- 172.21.15.33:0/609797515 wait complete. 2024-04-25T01:43:42.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.704+0000 7f5072a3f700 1 Processor -- start 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f5072a3f700 1 -- start start 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f5072a3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 0x7f506c119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f5072a3f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f506c071f30 0x7f506c11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f5072a3f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c11aad0 con 0x7f506c070840 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f5072a3f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c1140b0 con 0x7f506c071f30 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f506bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 0x7f506c119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f506bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 0x7f506c119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35268/0 (socket says 172.21.15.33:35268) 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f506bfff700 1 -- 172.21.15.33:0/465641244 learned_addr learned my addr 172.21.15.33:0/465641244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f506b7fe700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f506c071f30 0x7f506c11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f506bfff700 1 -- 172.21.15.33:0/465641244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f506c071f30 msgr2=0x7f506c11a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f506bfff700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f506c071f30 0x7f506c11a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f506bfff700 1 -- 172.21.15.33:0/465641244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5060005040 con 0x7f506c070840 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.705+0000 7f506bfff700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 0x7f506c119f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f505c00c8c0 tx=0x7f505c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.706+0000 7f5068ff9700 1 -- 172.21.15.33:0/465641244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f505c00a760 con 0x7f506c070840 2024-04-25T01:43:42.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.706+0000 7f5068ff9700 1 -- 172.21.15.33:0/465641244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f505c00ad90 con 0x7f506c070840 2024-04-25T01:43:42.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.706+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f506c114390 con 0x7f506c070840 2024-04-25T01:43:42.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.706+0000 7f5068ff9700 1 -- 172.21.15.33:0/465641244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f505c0113a0 con 0x7f506c070840 2024-04-25T01:43:42.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.706+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c1148a0 con 0x7f506c070840 2024-04-25T01:43:42.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.707+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c071a40 con 0x7f506c070840 2024-04-25T01:43:42.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.707+0000 7f5068ff9700 1 -- 172.21.15.33:0/465641244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f505c012400 con 0x7f506c070840 2024-04-25T01:43:42.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.708+0000 7f5068ff9700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f505406e140 0x7f50540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:43:42.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.708+0000 7f5068ff9700 1 -- 172.21.15.33:0/465641244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f505c08dab0 con 0x7f506c070840 2024-04-25T01:43:42.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.708+0000 7f506b7fe700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f505406e140 0x7f50540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:43:42.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.708+0000 7f506b7fe700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f505406e140 0x7f50540705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5060004f50 tx=0x7f5060009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:43:42.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:42.712+0000 7f5068ff9700 1 -- 172.21.15.33:0/465641244 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f505c05aa90 con 0x7f506c070840 2024-04-25T01:43:42.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.005+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f506c04c710 con 0x7f506c070840 2024-04-25T01:43:42.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.005+0000 7f5068ff9700 1 -- 172.21.15.33:0/465641244 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f505c05a620 con 0x7f506c070840 2024-04-25T01:43:42.996 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_OK 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f505406e140 msgr2=0x7f50540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f505406e140 0x7f50540705f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5060004f50 tx=0x7f5060009e30 comp rx=0 tx=0).stop 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 msgr2=0x7f506c119f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 0x7f506c119f40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f505c00c8c0 tx=0x7f505c0044c0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 shutdown_connections 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f505406e140 0x7f50540705f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f506c070840 0x7f506c119f40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 --2- 172.21.15.33:0/465641244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f506c071f30 0x7f506c11a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 >> 172.21.15.33:0/465641244 conn(0x7f506c0f89d0 msgr2=0x7f506c0f99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 shutdown_connections 2024-04-25T01:43:42.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:43:43.008+0000 7f5072a3f700 1 -- 172.21.15.33:0/465641244 wait complete. 2024-04-25T01:43:43.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:43 smithi138 ceph-mon[81304]: pgmap v109: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:43:43.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:43 smithi138 ceph-mon[81304]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:43.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:43 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/465641244' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:43:43.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:43 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T01:43:43.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:43 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T01:43:43.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:43 smithi138 ceph-mon[81304]: Upgrade: osd.3 is safe to restart 2024-04-25T01:43:43.756 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:43 smithi033 ceph-mon[111306]: pgmap v109: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:43:43.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:43 smithi033 ceph-mon[111306]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:43:43.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:43 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/465641244' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:43:43.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:43 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T01:43:43.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:43 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T01:43:43.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:43 smithi033 ceph-mon[111306]: Upgrade: osd.3 is safe to restart 2024-04-25T01:43:45.934 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:45 smithi138 ceph-mon[81304]: pgmap v110: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:45.934 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:45 smithi138 ceph-mon[81304]: Upgrade: Updating osd.3 2024-04-25T01:43:45.934 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:45 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:45.934 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:45 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:43:45.934 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:45 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:45.934 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:45 smithi138 ceph-mon[81304]: Deploying daemon osd.3 on smithi138 2024-04-25T01:43:46.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:45 smithi033 ceph-mon[111306]: pgmap v110: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:46.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:45 smithi033 ceph-mon[111306]: Upgrade: Updating osd.3 2024-04-25T01:43:46.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:45 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:46.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:45 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T01:43:46.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:45 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:43:46.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:45 smithi033 ceph-mon[111306]: Deploying daemon osd.3 on smithi138 2024-04-25T01:43:47.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:47 smithi138 ceph-mon[81304]: pgmap v111: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:43:48.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:47 smithi033 ceph-mon[111306]: pgmap v111: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:43:48.281 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:48 smithi138 systemd[1]: Stopping Ceph osd.3 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:43:48.532 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:48 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[38469]: 2024-04-25T01:43:48.314+0000 7f476d5ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:43:48.532 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:48 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[38469]: 2024-04-25T01:43:48.314+0000 7f476d5ea700 -1 osd.3 63 *** Got signal Terminated *** 2024-04-25T01:43:48.532 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:48 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[38469]: 2024-04-25T01:43:48.314+0000 7f476d5ea700 -1 osd.3 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:43:49.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:48 smithi033 ceph-mon[111306]: osd.3 marked itself down and dead 2024-04-25T01:43:49.031 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:48 smithi138 ceph-mon[81304]: osd.3 marked itself down and dead 2024-04-25T01:43:49.481 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:49 smithi138 podman[90352]: 2024-04-25 01:43:49.169310296 +0000 UTC m=+0.984188837 container died 55ddc8bb31e03a15e2248069913d24f09c3814fab26a2b9e7654347748ad7380 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=quincy-c61472c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T01:43:49.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:49 smithi138 ceph-mon[81304]: pgmap v112: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:49.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:49 smithi138 ceph-mon[81304]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:43:49.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:49 smithi138 ceph-mon[81304]: osdmap e64: 6 total, 5 up, 6 in 2024-04-25T01:43:49.782 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:49 smithi138 podman[90352]: 2024-04-25 01:43:49.581784607 +0000 UTC m=+1.396663154 container cleanup 55ddc8bb31e03a15e2248069913d24f09c3814fab26a2b9e7654347748ad7380 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T01:43:49.782 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:49 smithi138 bash[90352]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3 2024-04-25T01:43:50.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:49 smithi033 ceph-mon[111306]: pgmap v112: 65 pgs: 65 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:43:50.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:49 smithi033 ceph-mon[111306]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:43:50.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:49 smithi033 ceph-mon[111306]: osdmap e64: 6 total, 5 up, 6 in 2024-04-25T01:43:50.124 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:49 smithi138 podman[90381]: 2024-04-25 01:43:49.808309599 +0000 UTC m=+0.638538967 container remove 55ddc8bb31e03a15e2248069913d24f09c3814fab26a2b9e7654347748ad7380 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=quincy-c61472c) 2024-04-25T01:43:50.427 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:50 smithi138 podman[90474]: 2024-04-25 01:43:50.037147646 +0000 UTC m=+0.018260321 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:50.427 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:50 smithi138 podman[90474]: 2024-04-25 01:43:50.142637706 +0000 UTC m=+0.123750376 container create ecc2d0ccca9b5596b5ba99a9868c2fe6949e6ad57956d6a54f7cb57f98ce9f70 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9) 2024-04-25T01:43:50.427 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:50 smithi138 podman[90474]: 2024-04-25 01:43:50.353831504 +0000 UTC m=+0.334944179 container init ecc2d0ccca9b5596b5ba99a9868c2fe6949e6ad57956d6a54f7cb57f98ce9f70 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T01:43:50.427 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:50 smithi138 podman[90474]: 2024-04-25 01:43:50.357006721 +0000 UTC m=+0.338119388 container start ecc2d0ccca9b5596b5ba99a9868c2fe6949e6ad57956d6a54f7cb57f98ce9f70 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:43:50.682 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:50 smithi138 podman[90474]: 2024-04-25 01:43:50.436976888 +0000 UTC m=+0.418089566 container attach ecc2d0ccca9b5596b5ba99a9868c2fe6949e6ad57956d6a54f7cb57f98ce9f70 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T01:43:50.682 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:50 smithi138 podman[90474]: 2024-04-25 01:43:50.646870064 +0000 UTC m=+0.627982735 container died ecc2d0ccca9b5596b5ba99a9868c2fe6949e6ad57956d6a54f7cb57f98ce9f70 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T01:43:50.980 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:50 smithi138 ceph-mon[81304]: osdmap e65: 6 total, 5 up, 6 in 2024-04-25T01:43:51.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:50 smithi033 ceph-mon[111306]: osdmap e65: 6 total, 5 up, 6 in 2024-04-25T01:43:51.532 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:51 smithi138 podman[90500]: 2024-04-25 01:43:51.283003995 +0000 UTC m=+0.631304118 container remove ecc2d0ccca9b5596b5ba99a9868c2fe6949e6ad57956d6a54f7cb57f98ce9f70 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-25T01:43:51.532 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:51 smithi138 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-25T01:43:51.532 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:51 smithi138 systemd[1]: Stopped Ceph osd.3 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:43:51.532 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:51 smithi138 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 16.889s CPU time. 2024-04-25T01:43:51.532 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:51 smithi138 systemd[1]: Starting Ceph osd.3 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:43:51.960 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:51 smithi138 ceph-mon[81304]: pgmap v115: 65 pgs: 3 active+undersized+degraded, 13 stale+active+clean, 4 active+undersized, 45 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7/321 objects degraded (2.181%) 2024-04-25T01:43:51.960 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:51 smithi138 ceph-mon[81304]: Health check failed: Degraded data redundancy: 7/321 objects degraded (2.181%), 3 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:51.961 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:51 smithi138 podman[90605]: 2024-04-25 01:43:51.709142142 +0000 UTC m=+0.096409796 container create 2739b551dfe29738583e63a4376b860f78b482fa59b5707f2130c2e7d644482a (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:43:51.961 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:51 smithi138 podman[90605]: 2024-04-25 01:43:51.630698313 +0000 UTC m=+0.017965987 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:51.961 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:51 smithi138 podman[90605]: 2024-04-25 01:43:51.970097819 +0000 UTC m=+0.357365477 container init 2739b551dfe29738583e63a4376b860f78b482fa59b5707f2130c2e7d644482a (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-25T01:43:52.006 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:51 smithi033 ceph-mon[111306]: pgmap v115: 65 pgs: 3 active+undersized+degraded, 13 stale+active+clean, 4 active+undersized, 45 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 7/321 objects degraded (2.181%) 2024-04-25T01:43:52.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:51 smithi033 ceph-mon[111306]: Health check failed: Degraded data redundancy: 7/321 objects degraded (2.181%), 3 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:52.282 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:51 smithi138 podman[90605]: 2024-04-25 01:43:51.974029181 +0000 UTC m=+0.361296842 container start 2739b551dfe29738583e63a4376b860f78b482fa59b5707f2130c2e7d644482a (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T01:43:52.282 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 podman[90605]: 2024-04-25 01:43:52.07874472 +0000 UTC m=+0.466012374 container attach 2739b551dfe29738583e63a4376b860f78b482fa59b5707f2130c2e7d644482a (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T01:43:52.782 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate[90620]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate[90620]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 bash[90605]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 bash[90605]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 bash[90605]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 bash[90605]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 bash[90605]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 bash[90605]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 bash[90605]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate[90620]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate[90620]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate[90620]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate[90620]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate[90620]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-25T01:43:52.783 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:52 smithi138 podman[90605]: 2024-04-25 01:43:52.440421633 +0000 UTC m=+0.827689299 container died 2739b551dfe29738583e63a4376b860f78b482fa59b5707f2130c2e7d644482a (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T01:43:53.507 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:53 smithi138 podman[90755]: 2024-04-25 01:43:53.07457433 +0000 UTC m=+0.629091416 container remove 2739b551dfe29738583e63a4376b860f78b482fa59b5707f2130c2e7d644482a (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-activate, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:43:53.507 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:53 smithi138 podman[90822]: 2024-04-25 01:43:53.192827796 +0000 UTC m=+0.018180368 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:43:53.507 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:53 smithi138 podman[90822]: 2024-04-25 01:43:53.517521747 +0000 UTC m=+0.342874314 container create d00965714a8d750bb5a15f73a0baaac55b3b751a67fe5b4639c3705756f964f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T01:43:53.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:53 smithi138 ceph-mon[81304]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 1 stale+active+clean, 16 active+undersized, 30 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/321 objects degraded (20.249%) 2024-04-25T01:43:53.782 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:53 smithi138 podman[90822]: 2024-04-25 01:43:53.728588682 +0000 UTC m=+0.553941252 container init d00965714a8d750bb5a15f73a0baaac55b3b751a67fe5b4639c3705756f964f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T01:43:53.782 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:53 smithi138 podman[90822]: 2024-04-25 01:43:53.731678225 +0000 UTC m=+0.557030801 container start d00965714a8d750bb5a15f73a0baaac55b3b751a67fe5b4639c3705756f964f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T01:43:53.782 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:53 smithi138 ceph-osd[90840]: -- 172.21.15.138:0/26603832 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f476913680 con 0x55f4768dcc00 2024-04-25T01:43:54.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:53 smithi033 ceph-mon[111306]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 1 stale+active+clean, 16 active+undersized, 30 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 65/321 objects degraded (20.249%) 2024-04-25T01:43:54.267 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:53 smithi138 bash[90822]: d00965714a8d750bb5a15f73a0baaac55b3b751a67fe5b4639c3705756f964f5 2024-04-25T01:43:54.267 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:53 smithi138 systemd[1]: Started Ceph osd.3 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:43:54.532 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:54 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[90836]: 2024-04-25T01:43:54.275+0000 7fb5158987c0 -1 Falling back to public interface 2024-04-25T01:43:55.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:54 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:55.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:54 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:55.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:54 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:55.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:54 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:55.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:54 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:55.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:54 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:43:56.153 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:55 smithi138 ceph-mon[81304]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/321 objects degraded (21.184%) 2024-04-25T01:43:56.153 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:56.153 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:43:56.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:55 smithi033 ceph-mon[111306]: pgmap v117: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/321 objects degraded (21.184%) 2024-04-25T01:43:56.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:56.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:43:57.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:56 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 68/321 objects degraded (21.184%), 19 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:57.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:57.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:57.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:56 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 68/321 objects degraded (21.184%), 19 pgs degraded (PG_DEGRADED) 2024-04-25T01:43:57.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:57.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:58.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:57 smithi033 ceph-mon[111306]: pgmap v118: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/321 objects degraded (21.184%) 2024-04-25T01:43:58.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:57 smithi138 ceph-mon[81304]: pgmap v118: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/321 objects degraded (21.184%) 2024-04-25T01:43:59.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:59.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:59.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:59.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:43:59.638 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:59 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[90836]: 2024-04-25T01:43:59.380+0000 7fb5158987c0 -1 osd.3 63 log_to_monitors true 2024-04-25T01:44:00.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:59 smithi033 ceph-mon[111306]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/321 objects degraded (21.184%) 2024-04-25T01:44:00.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:59 smithi033 ceph-mon[111306]: from='osd.3 [v2:172.21.15.138:6800/1157003839,v1:172.21.15.138:6801/1157003839]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:44:00.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:43:59 smithi033 ceph-mon[111306]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:44:00.282 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 01:43:59 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[90836]: 2024-04-25T01:43:59.962+0000 7fb506cf0700 -1 osd.3 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:44:00.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:59 smithi138 ceph-mon[81304]: pgmap v119: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68/321 objects degraded (21.184%) 2024-04-25T01:44:00.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:59 smithi138 ceph-mon[81304]: from='osd.3 [v2:172.21.15.138:6800/1157003839,v1:172.21.15.138:6801/1157003839]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:44:00.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:43:59 smithi138 ceph-mon[81304]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T01:44:01.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:00 smithi138 ceph-mon[81304]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:44:01.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:00 smithi138 ceph-mon[81304]: osdmap e66: 6 total, 5 up, 6 in 2024-04-25T01:44:01.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:00 smithi138 ceph-mon[81304]: from='osd.3 [v2:172.21.15.138:6800/1157003839,v1:172.21.15.138:6801/1157003839]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:44:01.213 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:00 smithi138 ceph-mon[81304]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:44:01.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:00 smithi033 ceph-mon[111306]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T01:44:01.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:00 smithi033 ceph-mon[111306]: osdmap e66: 6 total, 5 up, 6 in 2024-04-25T01:44:01.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:00 smithi033 ceph-mon[111306]: from='osd.3 [v2:172.21.15.138:6800/1157003839,v1:172.21.15.138:6801/1157003839]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:44:01.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:00 smithi033 ceph-mon[111306]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:44:02.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: pgmap v121: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/321 objects degraded (21.184%) 2024-04-25T01:44:02.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:44:02.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: osd.3 [v2:172.21.15.138:6800/1157003839,v1:172.21.15.138:6801/1157003839] boot 2024-04-25T01:44:02.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: osdmap e67: 6 total, 6 up, 6 in 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T01:44:02.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: pgmap v121: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/321 objects degraded (21.184%) 2024-04-25T01:44:02.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:44:02.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: osd.3 [v2:172.21.15.138:6800/1157003839,v1:172.21.15.138:6801/1157003839] boot 2024-04-25T01:44:02.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: osdmap e67: 6 total, 6 up, 6 in 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:02.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T01:44:03.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:02 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T01:44:03.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:02 smithi033 ceph-mon[111306]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-25T01:44:03.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:02 smithi033 ceph-mon[111306]: osdmap e68: 6 total, 6 up, 6 in 2024-04-25T01:44:03.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:02 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T01:44:03.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:02 smithi138 ceph-mon[81304]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-25T01:44:03.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:02 smithi138 ceph-mon[81304]: osdmap e68: 6 total, 6 up, 6 in 2024-04-25T01:44:04.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:03 smithi033 ceph-mon[111306]: pgmap v124: 65 pgs: 8 peering, 12 active+undersized+degraded, 11 active+undersized, 34 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 47/321 objects degraded (14.642%) 2024-04-25T01:44:04.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:03 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 47/321 objects degraded (14.642%), 12 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:04.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:03 smithi138 ceph-mon[81304]: pgmap v124: 65 pgs: 8 peering, 12 active+undersized+degraded, 11 active+undersized, 34 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 47/321 objects degraded (14.642%) 2024-04-25T01:44:04.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:03 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 47/321 objects degraded (14.642%), 12 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:06.215 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:05 smithi138 ceph-mon[81304]: pgmap v125: 65 pgs: 8 peering, 11 active+undersized+degraded, 9 active+undersized, 37 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 44/321 objects degraded (13.707%) 2024-04-25T01:44:06.257 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:05 smithi033 ceph-mon[111306]: pgmap v125: 65 pgs: 8 peering, 11 active+undersized+degraded, 9 active+undersized, 37 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 0 op/s; 44/321 objects degraded (13.707%) 2024-04-25T01:44:07.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:07 smithi138 ceph-mon[81304]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/321 objects degraded (13.707%), 11 pgs degraded) 2024-04-25T01:44:07.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:07 smithi138 ceph-mon[81304]: Cluster is now healthy 2024-04-25T01:44:07.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:07 smithi033 ceph-mon[111306]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/321 objects degraded (13.707%), 11 pgs degraded) 2024-04-25T01:44:07.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:07 smithi033 ceph-mon[111306]: Cluster is now healthy 2024-04-25T01:44:08.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:08 smithi138 ceph-mon[81304]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 4 op/s 2024-04-25T01:44:08.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:08 smithi033 ceph-mon[111306]: pgmap v126: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 4 op/s 2024-04-25T01:44:10.282 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:10 smithi138 ceph-mon[81304]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:44:10.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:10 smithi033 ceph-mon[111306]: pgmap v127: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:44:11.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:11 smithi033 ceph-mon[111306]: pgmap v128: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T01:44:11.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:11.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:44:11.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:11 smithi138 ceph-mon[81304]: pgmap v128: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T01:44:11.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:11.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:44:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.127+0000 7fd820e73700 1 -- 172.21.15.33:0/976912247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 msgr2=0x7fd81c0fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.127+0000 7fd820e73700 1 --2- 172.21.15.33:0/976912247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 0x7fd81c0fb970 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd80c004940 tx=0x7fd80c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.127+0000 7fd820e73700 1 -- 172.21.15.33:0/976912247 shutdown_connections 2024-04-25T01:44:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.127+0000 7fd820e73700 1 --2- 172.21.15.33:0/976912247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c0fbf40 0x7fd81c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.127+0000 7fd820e73700 1 --2- 172.21.15.33:0/976912247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 0x7fd81c0fb970 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.127+0000 7fd820e73700 1 -- 172.21.15.33:0/976912247 >> 172.21.15.33:0/976912247 conn(0x7fd81c073350 msgr2=0x7fd81c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.127+0000 7fd820e73700 1 -- 172.21.15.33:0/976912247 shutdown_connections 2024-04-25T01:44:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.127+0000 7fd820e73700 1 -- 172.21.15.33:0/976912247 wait complete. 2024-04-25T01:44:13.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd820e73700 1 Processor -- start 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd820e73700 1 -- start start 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd820e73700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 0x7fd81c1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd820e73700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c0fbf40 0x7fd81c113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd820e73700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c1145c0 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd820e73700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c10db50 con 0x7fd81c0fbf40 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd81a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 0x7fd81c1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd81a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 0x7fd81c1139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47124/0 (socket says 172.21.15.33:47124) 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd81a59c700 1 -- 172.21.15.33:0/3376347612 learned_addr learned my addr 172.21.15.33:0/3376347612 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd819d9b700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c0fbf40 0x7fd81c113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd81a59c700 1 -- 172.21.15.33:0/3376347612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c0fbf40 msgr2=0x7fd81c113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd81a59c700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c0fbf40 0x7fd81c113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd81a59c700 1 -- 172.21.15.33:0/3376347612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c005040 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.128+0000 7fd819d9b700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c0fbf40 0x7fd81c113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:13.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.129+0000 7fd81a59c700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 0x7fd81c1139a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00b3b0 tx=0x7fd80c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.129+0000 7fd80affd700 1 -- 172.21.15.33:0/3376347612 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd80c00e030 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.129+0000 7fd80affd700 1 -- 172.21.15.33:0/3376347612 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd80c00bde0 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.129+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c10ddd0 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.129+0000 7fd80affd700 1 -- 172.21.15.33:0/3376347612 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd80c013e50 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.129+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c10e310 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.130+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c0fe670 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.130+0000 7fd80affd700 1 -- 172.21.15.33:0/3376347612 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd80c0136b0 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.131+0000 7fd80affd700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd80406e140 0x7fd8040705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.131+0000 7fd80affd700 1 -- 172.21.15.33:0/3376347612 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fd80c08f3b0 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.131+0000 7fd819d9b700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd80406e140 0x7fd8040705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.131+0000 7fd819d9b700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd80406e140 0x7fd8040705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd81c10f240 tx=0x7fd810008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.136+0000 7fd80affd700 1 -- 172.21.15.33:0/3376347612 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd80c058900 con 0x7fd81c0fb5a0 2024-04-25T01:44:13.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.363+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd81c10eff0 con 0x7fd80406e140 2024-04-25T01:44:13.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.364+0000 7fd80affd700 1 -- 172.21.15.33:0/3376347612 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fd81c10eff0 con 0x7fd80406e140 2024-04-25T01:44:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.366+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd80406e140 msgr2=0x7fd8040705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.366+0000 7fd820e73700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd80406e140 0x7fd8040705f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd81c10f240 tx=0x7fd810008040 comp rx=0 tx=0).stop 2024-04-25T01:44:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.366+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 msgr2=0x7fd81c1139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.366+0000 7fd820e73700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 0x7fd81c1139a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00b3b0 tx=0x7fd80c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.366+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 shutdown_connections 2024-04-25T01:44:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.366+0000 7fd820e73700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd80406e140 0x7fd8040705f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.366+0000 7fd820e73700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd81c0fb5a0 0x7fd81c1139a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.366+0000 7fd820e73700 1 --2- 172.21.15.33:0/3376347612 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd81c0fbf40 0x7fd81c113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.366+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 >> 172.21.15.33:0/3376347612 conn(0x7fd81c073350 msgr2=0x7fd81c0f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.367+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 shutdown_connections 2024-04-25T01:44:13.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.367+0000 7fd820e73700 1 -- 172.21.15.33:0/3376347612 wait complete. 2024-04-25T01:44:13.369 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:44:13.369 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:13 smithi033 ceph-mon[111306]: pgmap v129: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:44:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.472+0000 7f5081416700 1 -- 172.21.15.33:0/2913398737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 msgr2=0x7f507c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.472+0000 7f5081416700 1 --2- 172.21.15.33:0/2913398737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 0x7f507c1020a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f506c004940 tx=0x7f506c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.472+0000 7f5081416700 1 -- 172.21.15.33:0/2913398737 shutdown_connections 2024-04-25T01:44:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.472+0000 7f5081416700 1 --2- 172.21.15.33:0/2913398737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f507c102670 0x7f507c0fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.472+0000 7f5081416700 1 --2- 172.21.15.33:0/2913398737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 0x7f507c1020a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.472+0000 7f5081416700 1 -- 172.21.15.33:0/2913398737 >> 172.21.15.33:0/2913398737 conn(0x7f507c071460 msgr2=0x7f507c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.472+0000 7f5081416700 1 -- 172.21.15.33:0/2913398737 shutdown_connections 2024-04-25T01:44:13.464 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.472+0000 7f5081416700 1 -- 172.21.15.33:0/2913398737 wait complete. 2024-04-25T01:44:13.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.473+0000 7f5081416700 1 Processor -- start 2024-04-25T01:44:13.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.473+0000 7f5081416700 1 -- start start 2024-04-25T01:44:13.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.473+0000 7f5081416700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 0x7f507c0fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.473+0000 7f5081416700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f507c102670 0x7f507c0fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.473+0000 7f5081416700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0fd2b0 con 0x7f507c101cd0 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.473+0000 7f5081416700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c0fd3f0 con 0x7f507c102670 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.473+0000 7f507affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 0x7f507c0fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.473+0000 7f507affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 0x7f507c0fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47144/0 (socket says 172.21.15.33:47144) 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.473+0000 7f507affd700 1 -- 172.21.15.33:0/1760627223 learned_addr learned my addr 172.21.15.33:0/1760627223 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f507a7fc700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f507c102670 0x7f507c0fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f507affd700 1 -- 172.21.15.33:0/1760627223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f507c102670 msgr2=0x7f507c0fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f507affd700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f507c102670 0x7f507c0fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f507affd700 1 -- 172.21.15.33:0/1760627223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c005040 con 0x7f507c101cd0 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f507affd700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 0x7f507c0fc7e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f506c00b3b0 tx=0x7f506c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f50637fe700 1 -- 172.21.15.33:0/1760627223 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f506c00e040 con 0x7f507c101cd0 2024-04-25T01:44:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c0fd640 con 0x7f507c101cd0 2024-04-25T01:44:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f50637fe700 1 -- 172.21.15.33:0/1760627223 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f506c00bbc0 con 0x7f507c101cd0 2024-04-25T01:44:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f50637fe700 1 -- 172.21.15.33:0/1760627223 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f506c01c3f0 con 0x7f507c101cd0 2024-04-25T01:44:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.474+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c11a2b0 con 0x7f507c101cd0 2024-04-25T01:44:13.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.475+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c065200 con 0x7f507c101cd0 2024-04-25T01:44:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.476+0000 7f50637fe700 1 -- 172.21.15.33:0/1760627223 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f506c00bd30 con 0x7f507c101cd0 2024-04-25T01:44:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.476+0000 7f50637fe700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f506406e140 0x7f50640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.476+0000 7f50637fe700 1 -- 172.21.15.33:0/1760627223 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f506c08ec40 con 0x7f507c101cd0 2024-04-25T01:44:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.476+0000 7f507a7fc700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f506406e140 0x7f50640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.477+0000 7f507a7fc700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f506406e140 0x7f50640705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f507c0fe120 tx=0x7f5070009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.481+0000 7f50637fe700 1 -- 172.21.15.33:0/1760627223 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f506c0580f0 con 0x7f507c101cd0 2024-04-25T01:44:13.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:13 smithi138 ceph-mon[81304]: pgmap v129: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:44:13.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.710+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f507c109f80 con 0x7f506406e140 2024-04-25T01:44:13.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.711+0000 7f50637fe700 1 -- 172.21.15.33:0/1760627223 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f507c109f80 con 0x7f506406e140 2024-04-25T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.713+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f506406e140 msgr2=0x7f50640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.713+0000 7f5081416700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f506406e140 0x7f50640705f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f507c0fe120 tx=0x7f5070009ee0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.713+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 msgr2=0x7f507c0fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.713+0000 7f5081416700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 0x7f507c0fc7e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f506c00b3b0 tx=0x7f506c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.713+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 shutdown_connections 2024-04-25T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.713+0000 7f5081416700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f506406e140 0x7f50640705f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.713+0000 7f5081416700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f507c101cd0 0x7f507c0fc7e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.713+0000 7f5081416700 1 --2- 172.21.15.33:0/1760627223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f507c102670 0x7f507c0fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.713+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 >> 172.21.15.33:0/1760627223 conn(0x7f507c071460 msgr2=0x7f507c0fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.714+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 shutdown_connections 2024-04-25T01:44:13.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.714+0000 7f5081416700 1 -- 172.21.15.33:0/1760627223 wait complete. 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.818+0000 7f5451736700 1 -- 172.21.15.33:0/348855957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c102de0 msgr2=0x7f544c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.818+0000 7f5451736700 1 --2- 172.21.15.33:0/348855957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c102de0 0x7f544c1031b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f543c004940 tx=0x7f543c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.818+0000 7f5451736700 1 -- 172.21.15.33:0/348855957 shutdown_connections 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.818+0000 7f5451736700 1 --2- 172.21.15.33:0/348855957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f544c103780 0x7f544c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.818+0000 7f5451736700 1 --2- 172.21.15.33:0/348855957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c102de0 0x7f544c1031b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.818+0000 7f5451736700 1 -- 172.21.15.33:0/348855957 >> 172.21.15.33:0/348855957 conn(0x7f544c0f8990 msgr2=0x7f544c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.818+0000 7f5451736700 1 -- 172.21.15.33:0/348855957 shutdown_connections 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.818+0000 7f5451736700 1 -- 172.21.15.33:0/348855957 wait complete. 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.819+0000 7f5451736700 1 Processor -- start 2024-04-25T01:44:13.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.819+0000 7f5451736700 1 -- start start 2024-04-25T01:44:13.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.819+0000 7f5451736700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f544c102de0 0x7f544c071cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.819+0000 7f5451736700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c103780 0x7f544c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.819+0000 7f5451736700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544c070830 con 0x7f544c103780 2024-04-25T01:44:13.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.819+0000 7f5451736700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544c0709a0 con 0x7f544c102de0 2024-04-25T01:44:13.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.819+0000 7f544a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c103780 0x7f544c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f544a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c103780 0x7f544c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47156/0 (socket says 172.21.15.33:47156) 2024-04-25T01:44:13.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f544a7fc700 1 -- 172.21.15.33:0/4221464233 learned_addr learned my addr 172.21.15.33:0/4221464233 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:13.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f544affd700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f544c102de0 0x7f544c071cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f544a7fc700 1 -- 172.21.15.33:0/4221464233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f544c102de0 msgr2=0x7f544c071cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f544a7fc700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f544c102de0 0x7f544c071cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f544a7fc700 1 -- 172.21.15.33:0/4221464233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c005040 con 0x7f544c103780 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f544a7fc700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c103780 0x7f544c070260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f544000b790 tx=0x7f5440009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f54337fe700 1 -- 172.21.15.33:0/4221464233 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f544000f6f0 con 0x7f544c103780 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f54337fe700 1 -- 172.21.15.33:0/4221464233 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f544000fd20 con 0x7f544c103780 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544c070c80 con 0x7f544c103780 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f54337fe700 1 -- 172.21.15.33:0/4221464233 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54400103a0 con 0x7f544c103780 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.820+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544c11e610 con 0x7f544c103780 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.821+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544c1000f0 con 0x7f544c103780 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.822+0000 7f54337fe700 1 -- 172.21.15.33:0/4221464233 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f544000f850 con 0x7f544c103780 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.822+0000 7f54337fe700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f543406e140 0x7f54340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.822+0000 7f54337fe700 1 -- 172.21.15.33:0/4221464233 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f544008d8b0 con 0x7f544c103780 2024-04-25T01:44:13.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.822+0000 7f544affd700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f543406e140 0x7f54340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:13.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.823+0000 7f544affd700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f543406e140 0x7f54340705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f543c004f90 tx=0x7f543c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:13.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:13.827+0000 7f54337fe700 1 -- 172.21.15.33:0/4221464233 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f544005a810 con 0x7f544c103780 2024-04-25T01:44:14.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.047+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f544c0717b0 con 0x7f543406e140 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.057+0000 7f54337fe700 1 -- 172.21.15.33:0/4221464233 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f544c0717b0 con 0x7f543406e140 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (5m) 51s ago 16m 26.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (2m) 51s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (2m) 17s ago 15m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (5m) 51s ago 16m 77.2M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (11m) 51s ago 11m 62.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (11m) 51s ago 11m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (11m) 17s ago 11m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (11m) 17s ago 11m 61.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:44:14.049 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (6m) 51s ago 17m 504M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (6m) 17s ago 15m 450M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (3m) 51s ago 18m 46.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (3m) 17s ago 15m 37.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (6m) 51s ago 16m 16.0M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (5m) 17s ago 15m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (2m) 51s ago 14m 88.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (93s) 51s ago 14m 93.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (57s) 51s ago 13m 12.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (20s) 17s ago 13m 11.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (13m) 17s ago 13m 231M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 2b9115ab7674 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (12m) 17s ago 12m 219M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:44:14.050 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (5m) 51s ago 16m 60.9M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:44:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.059+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f543406e140 msgr2=0x7f54340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.059+0000 7f5451736700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f543406e140 0x7f54340705f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f543c004f90 tx=0x7f543c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:14.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.059+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c103780 msgr2=0x7f544c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.059+0000 7f5451736700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c103780 0x7f544c070260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f544000b790 tx=0x7f5440009220 comp rx=0 tx=0).stop 2024-04-25T01:44:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.059+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 shutdown_connections 2024-04-25T01:44:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.059+0000 7f5451736700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f543406e140 0x7f54340705f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.059+0000 7f5451736700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f544c102de0 0x7f544c071cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.059+0000 7f5451736700 1 --2- 172.21.15.33:0/4221464233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f544c103780 0x7f544c070260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.059+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 >> 172.21.15.33:0/4221464233 conn(0x7f544c0f8990 msgr2=0x7f544c0fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.060+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 shutdown_connections 2024-04-25T01:44:14.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.060+0000 7f5451736700 1 -- 172.21.15.33:0/4221464233 wait complete. 2024-04-25T01:44:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.162+0000 7f21eb296700 1 -- 172.21.15.33:0/1786979596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 msgr2=0x7f21e40fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.162+0000 7f21eb296700 1 --2- 172.21.15.33:0/1786979596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 0x7f21e40fe1d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f21d4009d10 tx=0x7f21d4004d10 comp rx=0 tx=0).stop 2024-04-25T01:44:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.162+0000 7f21eb296700 1 -- 172.21.15.33:0/1786979596 shutdown_connections 2024-04-25T01:44:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.162+0000 7f21eb296700 1 --2- 172.21.15.33:0/1786979596 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e40fe7a0 0x7f21e4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.162+0000 7f21eb296700 1 --2- 172.21.15.33:0/1786979596 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 0x7f21e40fe1d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.162+0000 7f21eb296700 1 -- 172.21.15.33:0/1786979596 >> 172.21.15.33:0/1786979596 conn(0x7f21e40f7860 msgr2=0x7f21e40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.162+0000 7f21eb296700 1 -- 172.21.15.33:0/1786979596 shutdown_connections 2024-04-25T01:44:14.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.162+0000 7f21eb296700 1 -- 172.21.15.33:0/1786979596 wait complete. 2024-04-25T01:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21eb296700 1 Processor -- start 2024-04-25T01:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21eb296700 1 -- start start 2024-04-25T01:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21eb296700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 0x7f21e4113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21eb296700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e40fe7a0 0x7f21e41140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21eb296700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41147a0 con 0x7f21e40fde00 2024-04-25T01:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21eb296700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e410dcf0 con 0x7f21e40fe7a0 2024-04-25T01:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21e9032700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 0x7f21e4113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21e9032700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 0x7f21e4113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47176/0 (socket says 172.21.15.33:47176) 2024-04-25T01:44:14.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21e9032700 1 -- 172.21.15.33:0/3941657045 learned_addr learned my addr 172.21.15.33:0/3941657045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21e8831700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e40fe7a0 0x7f21e41140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21e9032700 1 -- 172.21.15.33:0/3941657045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e40fe7a0 msgr2=0x7f21e41140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21e9032700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e40fe7a0 0x7f21e41140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.163+0000 7f21e9032700 1 -- 172.21.15.33:0/3941657045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4009940 con 0x7f21e40fde00 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.164+0000 7f21e9032700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 0x7f21e4113b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21d4008df0 tx=0x7f21d4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.164+0000 7f21e1ffb700 1 -- 172.21.15.33:0/3941657045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d400e040 con 0x7f21e40fde00 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.164+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e410df10 con 0x7f21e40fde00 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.164+0000 7f21e1ffb700 1 -- 172.21.15.33:0/3941657045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21d4004610 con 0x7f21e40fde00 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.164+0000 7f21e1ffb700 1 -- 172.21.15.33:0/3941657045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d401c8a0 con 0x7f21e40fde00 2024-04-25T01:44:14.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.164+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e410e450 con 0x7f21e40fde00 2024-04-25T01:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.165+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e4066b10 con 0x7f21e40fde00 2024-04-25T01:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.166+0000 7f21e1ffb700 1 -- 172.21.15.33:0/3941657045 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f21d4012410 con 0x7f21e40fde00 2024-04-25T01:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.166+0000 7f21e1ffb700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f21cc06e140 0x7f21cc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.166+0000 7f21e1ffb700 1 -- 172.21.15.33:0/3941657045 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f21d408f790 con 0x7f21e40fde00 2024-04-25T01:44:14.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.166+0000 7f21e8831700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f21cc06e140 0x7f21cc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:14.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.167+0000 7f21e8831700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f21cc06e140 0x7f21cc0705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21d8006fd0 tx=0x7f21d8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:14.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.171+0000 7f21e1ffb700 1 -- 172.21.15.33:0/3941657045 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f21d405c6f0 con 0x7f21e40fde00 2024-04-25T01:44:14.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.465+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f21e404c710 con 0x7f21e40fde00 2024-04-25T01:44:14.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:14 smithi033 ceph-mon[111306]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:14.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:14 smithi033 ceph-mon[111306]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:14.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:14 smithi033 ceph-mon[111306]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:14.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.465+0000 7f21e1ffb700 1 -- 172.21.15.33:0/3941657045 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f21d4061e60 con 0x7f21e40fde00 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:44:14.458 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:44:14.459 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:44:14.459 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:44:14.459 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:44:14.459 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6, 2024-04-25T01:44:14.459 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 8 2024-04-25T01:44:14.459 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:44:14.459 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f21cc06e140 msgr2=0x7f21cc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f21cc06e140 0x7f21cc0705f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f21d8006fd0 tx=0x7f21d8008040 comp rx=0 tx=0).stop 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 msgr2=0x7f21e4113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 0x7f21e4113b80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f21d4008df0 tx=0x7f21d4004d10 comp rx=0 tx=0).stop 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 shutdown_connections 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f21cc06e140 0x7f21cc0705f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21e40fde00 0x7f21e4113b80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 --2- 172.21.15.33:0/3941657045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21e40fe7a0 0x7f21e41140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 >> 172.21.15.33:0/3941657045 conn(0x7f21e40f7860 msgr2=0x7f21e40f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 shutdown_connections 2024-04-25T01:44:14.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.468+0000 7f21eb296700 1 -- 172.21.15.33:0/3941657045 wait complete. 2024-04-25T01:44:14.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:14 smithi138 ceph-mon[81304]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:14.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:14 smithi138 ceph-mon[81304]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:14.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:14 smithi138 ceph-mon[81304]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.572+0000 7fdfa8f91700 1 -- 172.21.15.33:0/1348631691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa40fc630 msgr2=0x7fdfa40fcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.572+0000 7fdfa8f91700 1 --2- 172.21.15.33:0/1348631691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa40fc630 0x7fdfa40fcaa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdf94004940 tx=0x7fdf94009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.572+0000 7fdfa8f91700 1 -- 172.21.15.33:0/1348631691 shutdown_connections 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.572+0000 7fdfa8f91700 1 --2- 172.21.15.33:0/1348631691 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa40fc630 0x7fdfa40fcaa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.572+0000 7fdfa8f91700 1 --2- 172.21.15.33:0/1348631691 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa4101e10 0x7fdfa41021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.572+0000 7fdfa8f91700 1 -- 172.21.15.33:0/1348631691 >> 172.21.15.33:0/1348631691 conn(0x7fdfa40714d0 msgr2=0x7fdfa40718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.573+0000 7fdfa8f91700 1 -- 172.21.15.33:0/1348631691 shutdown_connections 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.573+0000 7fdfa8f91700 1 -- 172.21.15.33:0/1348631691 wait complete. 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.573+0000 7fdfa8f91700 1 Processor -- start 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.573+0000 7fdfa8f91700 1 -- start start 2024-04-25T01:44:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.573+0000 7fdfa8f91700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa40fc630 0x7fdfa410eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.573+0000 7fdfa8f91700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa4101e10 0x7fdfa410f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.573+0000 7fdfa8f91700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa410f590 con 0x7fdfa4101e10 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.573+0000 7fdfa8f91700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa410f6d0 con 0x7fdfa40fc630 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.573+0000 7fdfa1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa4101e10 0x7fdfa410f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdfa1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa4101e10 0x7fdfa410f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47194/0 (socket says 172.21.15.33:47194) 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdfa1d9b700 1 -- 172.21.15.33:0/142609377 learned_addr learned my addr 172.21.15.33:0/142609377 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdfa259c700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa40fc630 0x7fdfa410eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdfa1d9b700 1 -- 172.21.15.33:0/142609377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa40fc630 msgr2=0x7fdfa410eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdfa1d9b700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa40fc630 0x7fdfa410eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdfa1d9b700 1 -- 172.21.15.33:0/142609377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94005040 con 0x7fdfa4101e10 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdfa1d9b700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa4101e10 0x7fdfa410f050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400b8b0 tx=0x7fdf94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdf9affd700 1 -- 172.21.15.33:0/142609377 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf9400e040 con 0x7fdfa4101e10 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdf9affd700 1 -- 172.21.15.33:0/142609377 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf94004290 con 0x7fdfa4101e10 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa410f950 con 0x7fdfa4101e10 2024-04-25T01:44:14.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdf9affd700 1 -- 172.21.15.33:0/142609377 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf9401c3f0 con 0x7fdfa4101e10 2024-04-25T01:44:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.574+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa4117e00 con 0x7fdfa4101e10 2024-04-25T01:44:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.575+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa40ff130 con 0x7fdfa4101e10 2024-04-25T01:44:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.576+0000 7fdf9affd700 1 -- 172.21.15.33:0/142609377 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdf940137b0 con 0x7fdfa4101e10 2024-04-25T01:44:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.576+0000 7fdf9affd700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf9006e140 0x7fdf900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.576+0000 7fdf9affd700 1 -- 172.21.15.33:0/142609377 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fdf9408ecb0 con 0x7fdfa4101e10 2024-04-25T01:44:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.576+0000 7fdfa259c700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf9006e140 0x7fdf900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.577+0000 7fdfa259c700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf9006e140 0x7fdf900705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c007fd0 tx=0x7fdf8c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:14.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.581+0000 7fdf9affd700 1 -- 172.21.15.33:0/142609377 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdf94058200 con 0x7fdfa4101e10 2024-04-25T01:44:14.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.832+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdfa40fbac0 con 0x7fdfa4101e10 2024-04-25T01:44:14.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.832+0000 7fdf9affd700 1 -- 172.21.15.33:0/142609377 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1903 (secure 0 0 0) 0x7fdf9408e020 con 0x7fdfa4101e10 2024-04-25T01:44:14.824 INFO:teuthology.orchestra.run.smithi033.stdout:e19 2024-04-25T01:44:14.824 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:44:14.824 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:44:14.824 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:44:14.824 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 19 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:44:14.825 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:44:14.826 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:44:14.827 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:44:14.827 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:44:14.827 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:44:14.827 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:44:14.827 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:44:14.827 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:44:14.827 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:44:14.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.834+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf9006e140 msgr2=0x7fdf900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.834+0000 7fdfa8f91700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf9006e140 0x7fdf900705f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c007fd0 tx=0x7fdf8c009040 comp rx=0 tx=0).stop 2024-04-25T01:44:14.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.834+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa4101e10 msgr2=0x7fdfa410f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.834+0000 7fdfa8f91700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa4101e10 0x7fdfa410f050 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdf9400b8b0 tx=0x7fdf94009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:14.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.835+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 shutdown_connections 2024-04-25T01:44:14.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.835+0000 7fdfa8f91700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf9006e140 0x7fdf900705f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.835+0000 7fdfa8f91700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfa40fc630 0x7fdfa410eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.835+0000 7fdfa8f91700 1 --2- 172.21.15.33:0/142609377 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfa4101e10 0x7fdfa410f050 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.835+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 >> 172.21.15.33:0/142609377 conn(0x7fdfa40714d0 msgr2=0x7fdfa40fa7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:14.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.835+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 shutdown_connections 2024-04-25T01:44:14.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.835+0000 7fdfa8f91700 1 -- 172.21.15.33:0/142609377 wait complete. 2024-04-25T01:44:14.829 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:44:14.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.940+0000 7fb3955d5700 1 -- 172.21.15.33:0/73630441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 msgr2=0x7fb3900fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.940+0000 7fb3955d5700 1 --2- 172.21.15.33:0/73630441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 0x7fb3900fb600 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb378004940 tx=0x7fb378009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:14.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.940+0000 7fb3955d5700 1 -- 172.21.15.33:0/73630441 shutdown_connections 2024-04-25T01:44:14.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.940+0000 7fb3955d5700 1 --2- 172.21.15.33:0/73630441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 0x7fb3900fb600 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.940+0000 7fb3955d5700 1 --2- 172.21.15.33:0/73630441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb390101cd0 0x7fb3901020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.940+0000 7fb3955d5700 1 -- 172.21.15.33:0/73630441 >> 172.21.15.33:0/73630441 conn(0x7fb390071460 msgr2=0x7fb390071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:14.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.940+0000 7fb3955d5700 1 -- 172.21.15.33:0/73630441 shutdown_connections 2024-04-25T01:44:14.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.940+0000 7fb3955d5700 1 -- 172.21.15.33:0/73630441 wait complete. 2024-04-25T01:44:14.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb3955d5700 1 Processor -- start 2024-04-25T01:44:14.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb3955d5700 1 -- start start 2024-04-25T01:44:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb3955d5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb390101cd0 0x7fb3900fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb3955d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 0x7fb3900fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb3955d5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900fd340 con 0x7fb390102670 2024-04-25T01:44:14.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb3955d5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900fd480 con 0x7fb390101cd0 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb38e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 0x7fb3900fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb38e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 0x7fb3900fcd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47214/0 (socket says 172.21.15.33:47214) 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb38e7fc700 1 -- 172.21.15.33:0/2732239586 learned_addr learned my addr 172.21.15.33:0/2732239586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb38effd700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb390101cd0 0x7fb3900fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb38e7fc700 1 -- 172.21.15.33:0/2732239586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb390101cd0 msgr2=0x7fb3900fc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb38e7fc700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb390101cd0 0x7fb3900fc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.941+0000 7fb38e7fc700 1 -- 172.21.15.33:0/2732239586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb378005040 con 0x7fb390102670 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.942+0000 7fb38effd700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb390101cd0 0x7fb3900fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.942+0000 7fb38e7fc700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 0x7fb3900fcd20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb37800b3b0 tx=0x7fb378009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.942+0000 7fb3877fe700 1 -- 172.21.15.33:0/2732239586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb37800e040 con 0x7fb390102670 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.942+0000 7fb3877fe700 1 -- 172.21.15.33:0/2732239586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb378013420 con 0x7fb390102670 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.942+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb390119e70 con 0x7fb390102670 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.942+0000 7fb3877fe700 1 -- 172.21.15.33:0/2732239586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb37801c630 con 0x7fb390102670 2024-04-25T01:44:14.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.942+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb39011a350 con 0x7fb390102670 2024-04-25T01:44:14.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.943+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3901052e0 con 0x7fb390102670 2024-04-25T01:44:14.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.943+0000 7fb3877fe700 1 -- 172.21.15.33:0/2732239586 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb378013590 con 0x7fb390102670 2024-04-25T01:44:14.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.944+0000 7fb3877fe700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb37c06e140 0x7fb37c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:14.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.944+0000 7fb3877fe700 1 -- 172.21.15.33:0/2732239586 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fb37808eb90 con 0x7fb390102670 2024-04-25T01:44:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.944+0000 7fb38effd700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb37c06e140 0x7fb37c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:14.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.944+0000 7fb38effd700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb37c06e140 0x7fb37c0705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb380001370 tx=0x7fb3800096a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:14.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:14.949+0000 7fb3877fe700 1 -- 172.21.15.33:0/2732239586 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb3780580e0 con 0x7fb390102670 2024-04-25T01:44:15.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.175+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3900fdf40 con 0x7fb37c06e140 2024-04-25T01:44:15.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.176+0000 7fb3877fe700 1 -- 172.21.15.33:0/2732239586 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb3900fdf40 con 0x7fb37c06e140 2024-04-25T01:44:15.168 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:44:15.168 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:44:15.168 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:44:15.168 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:44:15.168 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:44:15.168 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:44:15.169 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:44:15.169 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:44:15.169 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:44:15.169 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "10/21 daemons upgraded", 2024-04-25T01:44:15.169 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T01:44:15.169 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:44:15.169 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:44:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb37c06e140 msgr2=0x7fb37c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb37c06e140 0x7fb37c0705f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb380001370 tx=0x7fb3800096a0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 msgr2=0x7fb3900fcd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 0x7fb3900fcd20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb37800b3b0 tx=0x7fb378009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:15.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 shutdown_connections 2024-04-25T01:44:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb37c06e140 0x7fb37c0705f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb390101cd0 0x7fb3900fc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 --2- 172.21.15.33:0/2732239586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb390102670 0x7fb3900fcd20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 >> 172.21.15.33:0/2732239586 conn(0x7fb390071460 msgr2=0x7fb3900f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 shutdown_connections 2024-04-25T01:44:15.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.179+0000 7fb3955d5700 1 -- 172.21.15.33:0/2732239586 wait complete. 2024-04-25T01:44:15.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 -- 172.21.15.33:0/969623311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fd0a0 msgr2=0x7ff6d8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 --2- 172.21.15.33:0/969623311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fd0a0 0x7ff6d8109350 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc004940 tx=0x7ff6cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 -- 172.21.15.33:0/969623311 shutdown_connections 2024-04-25T01:44:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 --2- 172.21.15.33:0/969623311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fd0a0 0x7ff6d8109350 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 --2- 172.21.15.33:0/969623311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d80fc790 0x7ff6d80fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 -- 172.21.15.33:0/969623311 >> 172.21.15.33:0/969623311 conn(0x7ff6d8071d00 msgr2=0x7ff6d8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 -- 172.21.15.33:0/969623311 shutdown_connections 2024-04-25T01:44:15.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 -- 172.21.15.33:0/969623311 wait complete. 2024-04-25T01:44:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 Processor -- start 2024-04-25T01:44:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.284+0000 7ff6e036b700 1 -- start start 2024-04-25T01:44:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6e036b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fc790 0x7ff6d8115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:15.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6e036b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d80fd0a0 0x7ff6d8116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6e036b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d8116a00 con 0x7ff6d80fc790 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6e036b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d810fd40 con 0x7ff6d80fd0a0 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6de107700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fc790 0x7ff6d8115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6de107700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fc790 0x7ff6d8115de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47224/0 (socket says 172.21.15.33:47224) 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6de107700 1 -- 172.21.15.33:0/1680058011 learned_addr learned my addr 172.21.15.33:0/1680058011 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6de107700 1 -- 172.21.15.33:0/1680058011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d80fd0a0 msgr2=0x7ff6d8116320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6de107700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d80fd0a0 0x7ff6d8116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6de107700 1 -- 172.21.15.33:0/1680058011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc005040 con 0x7ff6d80fc790 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6de107700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fc790 0x7ff6d8115de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000bbb0 tx=0x7ff6d0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6d6ffd700 1 -- 172.21.15.33:0/1680058011 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6d00092e0 con 0x7ff6d80fc790 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6d6ffd700 1 -- 172.21.15.33:0/1680058011 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6d000cbd0 con 0x7ff6d80fc790 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.285+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d810ffc0 con 0x7ff6d80fc790 2024-04-25T01:44:15.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.286+0000 7ff6d6ffd700 1 -- 172.21.15.33:0/1680058011 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6d0007400 con 0x7ff6d80fc790 2024-04-25T01:44:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.286+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d8110500 con 0x7ff6d80fc790 2024-04-25T01:44:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.287+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d80fc330 con 0x7ff6d80fc790 2024-04-25T01:44:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.287+0000 7ff6d6ffd700 1 -- 172.21.15.33:0/1680058011 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff6d0012070 con 0x7ff6d80fc790 2024-04-25T01:44:15.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.288+0000 7ff6d6ffd700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6c006e150 0x7ff6c0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.288+0000 7ff6d6ffd700 1 -- 172.21.15.33:0/1680058011 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6238+0+0 (secure 0 0 0) 0x7ff6d008d770 con 0x7ff6d80fc790 2024-04-25T01:44:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.288+0000 7ff6dd906700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6c006e150 0x7ff6c0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:15.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.288+0000 7ff6dd906700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6c006e150 0x7ff6c0070600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc004f90 tx=0x7ff6cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:15.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.292+0000 7ff6d6ffd700 1 -- 172.21.15.33:0/1680058011 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff6d005a6d0 con 0x7ff6d80fc790 2024-04-25T01:44:15.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:15 smithi033 ceph-mon[111306]: pgmap v130: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:44:15.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:15 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3941657045' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:15.507 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:15 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/142609377' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:44:15.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:15 smithi138 ceph-mon[81304]: pgmap v130: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:44:15.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:15 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3941657045' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:15.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:15 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/142609377' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:44:15.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.590+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff6d805f5c0 con 0x7ff6d80fc790 2024-04-25T01:44:15.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.591+0000 7ff6d6ffd700 1 -- 172.21.15.33:0/1680058011 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7ff6d005a260 con 0x7ff6d80fc790 2024-04-25T01:44:15.583 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_OK 2024-04-25T01:44:15.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6c006e150 msgr2=0x7ff6c0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6c006e150 0x7ff6c0070600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc004f90 tx=0x7ff6cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:15.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fc790 msgr2=0x7ff6d8115de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:15.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fc790 0x7ff6d8115de0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff6d000bbb0 tx=0x7ff6d0009840 comp rx=0 tx=0).stop 2024-04-25T01:44:15.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 shutdown_connections 2024-04-25T01:44:15.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6c006e150 0x7ff6c0070600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6d80fc790 0x7ff6d8115de0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 --2- 172.21.15.33:0/1680058011 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6d80fd0a0 0x7ff6d8116320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:15.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 >> 172.21.15.33:0/1680058011 conn(0x7ff6d8071d00 msgr2=0x7ff6d80f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:15.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 shutdown_connections 2024-04-25T01:44:15.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:15.593+0000 7ff6e036b700 1 -- 172.21.15.33:0/1680058011 wait complete. 2024-04-25T01:44:16.782 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:16 smithi138 ceph-mon[81304]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:16.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:16 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:16.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:16 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1680058011' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:44:17.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:16 smithi033 ceph-mon[111306]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:17.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:16 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:17.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:16 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1680058011' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:44:17.642 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:17 smithi138 ceph-mon[81304]: pgmap v131: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T01:44:17.643 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:17 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T01:44:17.643 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:17 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T01:44:17.643 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:17 smithi138 ceph-mon[81304]: Upgrade: osd.4 is safe to restart 2024-04-25T01:44:18.007 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:17 smithi033 ceph-mon[111306]: pgmap v131: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T01:44:18.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:17 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T01:44:18.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:17 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T01:44:18.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:17 smithi033 ceph-mon[111306]: Upgrade: osd.4 is safe to restart 2024-04-25T01:44:19.621 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:19 smithi138 ceph-mon[81304]: pgmap v132: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:44:19.621 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:19 smithi138 ceph-mon[81304]: Upgrade: Updating osd.4 2024-04-25T01:44:19.621 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:19 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:19.621 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:19 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:44:19.621 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:19 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:19.621 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:19 smithi138 ceph-mon[81304]: Deploying daemon osd.4 on smithi138 2024-04-25T01:44:19.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:19 smithi033 ceph-mon[111306]: pgmap v132: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:44:19.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:19 smithi033 ceph-mon[111306]: Upgrade: Updating osd.4 2024-04-25T01:44:19.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:19 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:19.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:19 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T01:44:19.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:19 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:19.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:19 smithi033 ceph-mon[111306]: Deploying daemon osd.4 on smithi138 2024-04-25T01:44:21.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:21 smithi138 ceph-mon[81304]: pgmap v133: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:44:21.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:21 smithi033 ceph-mon[111306]: pgmap v133: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T01:44:21.817 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:21 smithi138 systemd[1]: Stopping Ceph osd.4 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:44:22.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:21 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[45353]: 2024-04-25T01:44:21.835+0000 7fed7a2a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:44:22.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:21 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[45353]: 2024-04-25T01:44:21.835+0000 7fed7a2a4700 -1 osd.4 68 *** Got signal Terminated *** 2024-04-25T01:44:22.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:21 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[45353]: 2024-04-25T01:44:21.835+0000 7fed7a2a4700 -1 osd.4 68 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:44:22.744 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:22 smithi138 ceph-mon[81304]: osd.4 marked itself down and dead 2024-04-25T01:44:22.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:22 smithi033 ceph-mon[111306]: osd.4 marked itself down and dead 2024-04-25T01:44:23.033 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:22 smithi138 podman[95675]: 2024-04-25 01:44:22.872922548 +0000 UTC m=+1.149464806 container died 2b9115ab7674fbcb0b8ee103cd27d559ab582b9cda6fd6ac7c355f0f32ca7bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:44:23.511 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:23 smithi138 ceph-mon[81304]: pgmap v134: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:44:23.511 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:23 smithi138 ceph-mon[81304]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:44:23.511 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:23 smithi138 ceph-mon[81304]: osdmap e69: 6 total, 5 up, 6 in 2024-04-25T01:44:23.511 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:23 smithi138 podman[95675]: 2024-04-25 01:44:23.268223504 +0000 UTC m=+1.544765759 container cleanup 2b9115ab7674fbcb0b8ee103cd27d559ab582b9cda6fd6ac7c355f0f32ca7bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T01:44:23.511 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:23 smithi138 bash[95675]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4 2024-04-25T01:44:23.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:23 smithi033 ceph-mon[111306]: pgmap v134: 65 pgs: 65 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:44:23.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:23 smithi033 ceph-mon[111306]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:44:23.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:23 smithi033 ceph-mon[111306]: osdmap e69: 6 total, 5 up, 6 in 2024-04-25T01:44:23.782 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:23 smithi138 podman[95703]: 2024-04-25 01:44:23.520144064 +0000 UTC m=+0.651668520 container remove 2b9115ab7674fbcb0b8ee103cd27d559ab582b9cda6fd6ac7c355f0f32ca7bb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=quincy-c61472c) 2024-04-25T01:44:24.131 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:23 smithi138 podman[95792]: 2024-04-25 01:44:23.879480075 +0000 UTC m=+0.113008262 container create 56a781a31c091f837f90470bd9b706e69bfeee8aae58548eb49b0f07d1d38882 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T01:44:24.131 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:23 smithi138 podman[95792]: 2024-04-25 01:44:23.784594056 +0000 UTC m=+0.018122272 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:44:24.131 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:24 smithi138 podman[95792]: 2024-04-25 01:44:24.140514149 +0000 UTC m=+0.374042338 container init 56a781a31c091f837f90470bd9b706e69bfeee8aae58548eb49b0f07d1d38882 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T01:44:24.426 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:24 smithi138 podman[95792]: 2024-04-25 01:44:24.144661335 +0000 UTC m=+0.378189524 container start 56a781a31c091f837f90470bd9b706e69bfeee8aae58548eb49b0f07d1d38882 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T01:44:24.426 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:24 smithi138 podman[95792]: 2024-04-25 01:44:24.23216593 +0000 UTC m=+0.465694127 container attach 56a781a31c091f837f90470bd9b706e69bfeee8aae58548eb49b0f07d1d38882 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T01:44:24.709 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:24 smithi138 ceph-mon[81304]: osdmap e70: 6 total, 5 up, 6 in 2024-04-25T01:44:24.709 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:24 smithi138 podman[95792]: 2024-04-25 01:44:24.436534789 +0000 UTC m=+0.670062995 container died 56a781a31c091f837f90470bd9b706e69bfeee8aae58548eb49b0f07d1d38882 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T01:44:24.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:24 smithi033 ceph-mon[111306]: osdmap e70: 6 total, 5 up, 6 in 2024-04-25T01:44:25.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:24 smithi138 podman[95818]: 2024-04-25 01:44:24.969173376 +0000 UTC m=+0.525580297 container remove 56a781a31c091f837f90470bd9b706e69bfeee8aae58548eb49b0f07d1d38882 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T01:44:25.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:24 smithi138 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-25T01:44:25.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:24 smithi138 systemd[1]: Stopped Ceph osd.4 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:44:25.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:24 smithi138 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.4.service: Consumed 1min 3.778s CPU time. 2024-04-25T01:44:25.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:25 smithi138 systemd[1]: Starting Ceph osd.4 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:44:25.659 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:25 smithi138 ceph-mon[81304]: pgmap v137: 65 pgs: 2 peering, 5 stale+active+clean, 58 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:44:25.659 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:25 smithi138 ceph-mon[81304]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T01:44:25.659 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:25.659 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:44:25.659 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:25 smithi138 podman[95924]: 2024-04-25 01:44:25.319995356 +0000 UTC m=+0.017866850 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:44:25.659 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:25 smithi138 podman[95924]: 2024-04-25 01:44:25.437494744 +0000 UTC m=+0.135366245 container create c7a4465a75a6040e73426eff6ad82bc86aa2c119452d2d9f9b48b07f35b1adce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-25T01:44:25.659 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:25 smithi138 podman[95924]: 2024-04-25 01:44:25.665266566 +0000 UTC m=+0.363138059 container init c7a4465a75a6040e73426eff6ad82bc86aa2c119452d2d9f9b48b07f35b1adce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T01:44:25.660 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:25 smithi138 podman[95924]: 2024-04-25 01:44:25.668307381 +0000 UTC m=+0.366178878 container start c7a4465a75a6040e73426eff6ad82bc86aa2c119452d2d9f9b48b07f35b1adce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T01:44:25.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:25 smithi033 ceph-mon[111306]: pgmap v137: 65 pgs: 2 peering, 5 stale+active+clean, 58 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-25T01:44:25.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:25 smithi033 ceph-mon[111306]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T01:44:25.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:25.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:44:26.033 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:25 smithi138 podman[95924]: 2024-04-25 01:44:25.782134201 +0000 UTC m=+0.480005711 container attach c7a4465a75a6040e73426eff6ad82bc86aa2c119452d2d9f9b48b07f35b1adce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate[95939]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate[95939]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 bash[95924]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 bash[95924]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 bash[95924]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 bash[95924]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 bash[95924]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 bash[95924]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 bash[95924]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-25T01:44:26.427 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate[95939]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-25T01:44:26.428 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate[95939]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-25T01:44:26.428 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate[95939]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-25T01:44:26.428 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate[95939]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T01:44:26.428 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate[95939]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-25T01:44:26.428 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 podman[95924]: 2024-04-25 01:44:26.13543426 +0000 UTC m=+0.833305761 container died c7a4465a75a6040e73426eff6ad82bc86aa2c119452d2d9f9b48b07f35b1adce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T01:44:26.694 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:26 smithi138 ceph-mon[81304]: Health check failed: Degraded data redundancy: 44/321 objects degraded (13.707%), 12 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:26.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:26 smithi033 ceph-mon[111306]: Health check failed: Degraded data redundancy: 44/321 objects degraded (13.707%), 12 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:27.019 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 podman[96074]: 2024-04-25 01:44:26.703317635 +0000 UTC m=+0.562842758 container remove c7a4465a75a6040e73426eff6ad82bc86aa2c119452d2d9f9b48b07f35b1adce (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True) 2024-04-25T01:44:27.019 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 podman[96137]: 2024-04-25 01:44:26.820372538 +0000 UTC m=+0.017977788 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:44:27.019 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:26 smithi138 podman[96137]: 2024-04-25 01:44:26.929430719 +0000 UTC m=+0.127035961 container create 5adf4eaefb0b8ece675527bac237972a1bb9310baaf2399377c75fdf6b3fc653 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T01:44:27.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:27 smithi138 podman[96137]: 2024-04-25 01:44:27.132074618 +0000 UTC m=+0.329679859 container init 5adf4eaefb0b8ece675527bac237972a1bb9310baaf2399377c75fdf6b3fc653 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T01:44:27.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:27 smithi138 podman[96137]: 2024-04-25 01:44:27.135807341 +0000 UTC m=+0.333412583 container start 5adf4eaefb0b8ece675527bac237972a1bb9310baaf2399377c75fdf6b3fc653 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T01:44:27.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:27 smithi138 ceph-osd[96155]: -- 172.21.15.138:0/1001170873 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0b9401680 con 0x55f0b93cac00 2024-04-25T01:44:27.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:27 smithi138 bash[96137]: 5adf4eaefb0b8ece675527bac237972a1bb9310baaf2399377c75fdf6b3fc653 2024-04-25T01:44:27.283 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:27 smithi138 systemd[1]: Started Ceph osd.4 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:44:27.757 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:27 smithi033 ceph-mon[111306]: pgmap v138: 65 pgs: 12 active+undersized+degraded, 2 peering, 1 stale+active+clean, 12 active+undersized, 38 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44/321 objects degraded (13.707%) 2024-04-25T01:44:27.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:27 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:27.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:27 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:27.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:27 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:44:27.783 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:27 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T01:44:27.686+0000 7f736d4897c0 -1 Falling back to public interface 2024-04-25T01:44:27.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:27 smithi138 ceph-mon[81304]: pgmap v138: 65 pgs: 12 active+undersized+degraded, 2 peering, 1 stale+active+clean, 12 active+undersized, 38 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44/321 objects degraded (13.707%) 2024-04-25T01:44:27.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:27 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:27.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:27 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:27.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:27 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:44:28.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:28 smithi138 ceph-mon[81304]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T01:44:29.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:28 smithi033 ceph-mon[111306]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T01:44:29.909 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:29 smithi033 ceph-mon[111306]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/321 objects degraded (16.822%) 2024-04-25T01:44:30.032 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:29 smithi138 ceph-mon[81304]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/321 objects degraded (16.822%) 2024-04-25T01:44:31.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:31 smithi138 ceph-mon[81304]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/321 objects degraded (16.822%) 2024-04-25T01:44:31.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:31.783 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:31 smithi033 ceph-mon[111306]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/321 objects degraded (16.822%) 2024-04-25T01:44:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:32.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:32.705 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:32 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 54/321 objects degraded (16.822%), 16 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:32.705 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:32.705 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:32 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:33.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:32 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 54/321 objects degraded (16.822%), 16 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:33.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:33.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:32 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:33.033 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:32 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T01:44:32.800+0000 7f736d4897c0 -1 osd.4 68 log_to_monitors true 2024-04-25T01:44:34.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:33 smithi033 ceph-mon[111306]: pgmap v141: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/321 objects degraded (16.822%) 2024-04-25T01:44:34.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:33 smithi033 ceph-mon[111306]: from='osd.4 [v2:172.21.15.138:6808/4024012885,v1:172.21.15.138:6809/4024012885]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:44:34.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:33 smithi033 ceph-mon[111306]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:44:34.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:33 smithi138 ceph-mon[81304]: pgmap v141: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/321 objects degraded (16.822%) 2024-04-25T01:44:34.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:33 smithi138 ceph-mon[81304]: from='osd.4 [v2:172.21.15.138:6808/4024012885,v1:172.21.15.138:6809/4024012885]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:44:34.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:33 smithi138 ceph-mon[81304]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T01:44:34.967 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:34 smithi138 ceph-mon[81304]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:44:34.967 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:34 smithi138 ceph-mon[81304]: from='osd.4 [v2:172.21.15.138:6808/4024012885,v1:172.21.15.138:6809/4024012885]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:44:34.967 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:34 smithi138 ceph-mon[81304]: osdmap e71: 6 total, 5 up, 6 in 2024-04-25T01:44:34.967 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:34 smithi138 ceph-mon[81304]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:44:34.967 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:34 smithi138 ceph-mon[81304]: from='osd.4 ' entity='osd.4' 2024-04-25T01:44:34.967 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 01:44:34 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T01:44:34.574+0000 7f735a8d9700 -1 osd.4 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:44:35.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:34 smithi033 ceph-mon[111306]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T01:44:35.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:34 smithi033 ceph-mon[111306]: from='osd.4 [v2:172.21.15.138:6808/4024012885,v1:172.21.15.138:6809/4024012885]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:44:35.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:34 smithi033 ceph-mon[111306]: osdmap e71: 6 total, 5 up, 6 in 2024-04-25T01:44:35.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:34 smithi033 ceph-mon[111306]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:44:35.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:34 smithi033 ceph-mon[111306]: from='osd.4 ' entity='osd.4' 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: pgmap v143: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/321 objects degraded (16.822%) 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:36.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:36.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T01:44:36.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:44:36.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: osd.4 [v2:172.21.15.138:6808/4024012885,v1:172.21.15.138:6809/4024012885] boot 2024-04-25T01:44:36.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: osdmap e72: 6 total, 6 up, 6 in 2024-04-25T01:44:36.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:35 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: pgmap v143: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/321 objects degraded (16.822%) 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T01:44:36.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:44:36.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: osd.4 [v2:172.21.15.138:6808/4024012885,v1:172.21.15.138:6809/4024012885] boot 2024-04-25T01:44:36.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: osdmap e72: 6 total, 6 up, 6 in 2024-04-25T01:44:36.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:35 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T01:44:37.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:36 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T01:44:37.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:36 smithi033 ceph-mon[111306]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-25T01:44:37.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:36 smithi033 ceph-mon[111306]: osdmap e73: 6 total, 6 up, 6 in 2024-04-25T01:44:37.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:36 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T01:44:37.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:36 smithi138 ceph-mon[81304]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-25T01:44:37.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:36 smithi138 ceph-mon[81304]: osdmap e73: 6 total, 6 up, 6 in 2024-04-25T01:44:38.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:37 smithi033 ceph-mon[111306]: pgmap v145: 65 pgs: 4 peering, 15 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 51/321 objects degraded (15.888%) 2024-04-25T01:44:38.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:37 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 51/321 objects degraded (15.888%), 15 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:38.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:37 smithi138 ceph-mon[81304]: pgmap v145: 65 pgs: 4 peering, 15 active+undersized+degraded, 15 active+undersized, 31 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 51/321 objects degraded (15.888%) 2024-04-25T01:44:38.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:37 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 51/321 objects degraded (15.888%), 15 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:40.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:39 smithi033 ceph-mon[111306]: pgmap v147: 65 pgs: 4 peering, 11 active+undersized+degraded, 11 active+undersized, 39 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 41/321 objects degraded (12.773%) 2024-04-25T01:44:40.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:39 smithi138 ceph-mon[81304]: pgmap v147: 65 pgs: 4 peering, 11 active+undersized+degraded, 11 active+undersized, 39 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 41/321 objects degraded (12.773%) 2024-04-25T01:44:41.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:41 smithi033 ceph-mon[111306]: pgmap v148: 65 pgs: 4 peering, 61 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-25T01:44:41.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:41.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:44:41.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:41 smithi033 ceph-mon[111306]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/321 objects degraded (12.773%), 11 pgs degraded) 2024-04-25T01:44:41.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:41 smithi033 ceph-mon[111306]: Cluster is now healthy 2024-04-25T01:44:41.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:41 smithi138 ceph-mon[81304]: pgmap v148: 65 pgs: 4 peering, 61 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-25T01:44:41.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:41.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:44:41.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:41 smithi138 ceph-mon[81304]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 41/321 objects degraded (12.773%), 11 pgs degraded) 2024-04-25T01:44:41.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:41 smithi138 ceph-mon[81304]: Cluster is now healthy 2024-04-25T01:44:43.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:43 smithi138 ceph-mon[81304]: pgmap v149: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T01:44:43.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:43 smithi033 ceph-mon[111306]: pgmap v149: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-04-25T01:44:45.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:45 smithi138 ceph-mon[81304]: pgmap v150: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T01:44:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.713+0000 7fecee143700 1 -- 172.21.15.33:0/2544382900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 msgr2=0x7fece81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.713+0000 7fecee143700 1 --2- 172.21.15.33:0/2544382900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 0x7fece81031b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fecd8004940 tx=0x7fecd8009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.713+0000 7fecee143700 1 -- 172.21.15.33:0/2544382900 shutdown_connections 2024-04-25T01:44:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.713+0000 7fecee143700 1 --2- 172.21.15.33:0/2544382900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fece8103780 0x7fece80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.713+0000 7fecee143700 1 --2- 172.21.15.33:0/2544382900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 0x7fece81031b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.713+0000 7fecee143700 1 -- 172.21.15.33:0/2544382900 >> 172.21.15.33:0/2544382900 conn(0x7fece80f8990 msgr2=0x7fece80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.713+0000 7fecee143700 1 -- 172.21.15.33:0/2544382900 shutdown_connections 2024-04-25T01:44:45.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.713+0000 7fecee143700 1 -- 172.21.15.33:0/2544382900 wait complete. 2024-04-25T01:44:45.710 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:45 smithi033 ceph-mon[111306]: pgmap v150: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T01:44:45.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fecee143700 1 Processor -- start 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fecee143700 1 -- start start 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fecee143700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 0x7fece81139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fecee143700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fece8103780 0x7fece8113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fecee143700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece8114530 con 0x7fece8102de0 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fecee143700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece810db30 con 0x7fece8103780 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fece77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 0x7fece81139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fece77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 0x7fece81139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56096/0 (socket says 172.21.15.33:56096) 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fece77fe700 1 -- 172.21.15.33:0/1305569508 learned_addr learned my addr 172.21.15.33:0/1305569508 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.714+0000 7fece6ffd700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fece8103780 0x7fece8113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.715+0000 7fece77fe700 1 -- 172.21.15.33:0/1305569508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fece8103780 msgr2=0x7fece8113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.715+0000 7fece77fe700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fece8103780 0x7fece8113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.715+0000 7fece77fe700 1 -- 172.21.15.33:0/1305569508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd8005040 con 0x7fece8102de0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.715+0000 7fece77fe700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 0x7fece81139a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fecd800b8b0 tx=0x7fecd8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.715+0000 7feccffff700 1 -- 172.21.15.33:0/1305569508 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd800e040 con 0x7fece8102de0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.715+0000 7feccffff700 1 -- 172.21.15.33:0/1305569508 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecd8004290 con 0x7fece8102de0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.715+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece810ddb0 con 0x7fece8102de0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.715+0000 7feccffff700 1 -- 172.21.15.33:0/1305569508 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecd801c3f0 con 0x7fece8102de0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.715+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece810e370 con 0x7fece8102de0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.716+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece81000f0 con 0x7fece8102de0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.717+0000 7feccffff700 1 -- 172.21.15.33:0/1305569508 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fecd8013420 con 0x7fece8102de0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.717+0000 7feccffff700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecd006e140 0x7fecd00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.717+0000 7feccffff700 1 -- 172.21.15.33:0/1305569508 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fecd808ec70 con 0x7fece8102de0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.717+0000 7fece6ffd700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecd006e140 0x7fecd00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:45.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.718+0000 7fece6ffd700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecd006e140 0x7fecd00705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fece810ecf0 tx=0x7fecdc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:45.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.722+0000 7feccffff700 1 -- 172.21.15.33:0/1305569508 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fecd805bc50 con 0x7fece8102de0 2024-04-25T01:44:45.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.947+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fece810f060 con 0x7fecd006e140 2024-04-25T01:44:45.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.948+0000 7feccffff700 1 -- 172.21.15.33:0/1305569508 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fece810f060 con 0x7fecd006e140 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecd006e140 msgr2=0x7fecd00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecd006e140 0x7fecd00705f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fece810ecf0 tx=0x7fecdc00b040 comp rx=0 tx=0).stop 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 msgr2=0x7fece81139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 0x7fece81139a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fecd800b8b0 tx=0x7fecd8009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 shutdown_connections 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecd006e140 0x7fecd00705f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fece8102de0 0x7fece81139a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 --2- 172.21.15.33:0/1305569508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fece8103780 0x7fece8113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 >> 172.21.15.33:0/1305569508 conn(0x7fece80f8990 msgr2=0x7fece81073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 shutdown_connections 2024-04-25T01:44:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:45.950+0000 7fecee143700 1 -- 172.21.15.33:0/1305569508 wait complete. 2024-04-25T01:44:45.953 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.056+0000 7f0c6c704700 1 -- 172.21.15.33:0/1157788506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fdde0 msgr2=0x7f0c640fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.056+0000 7f0c6c704700 1 --2- 172.21.15.33:0/1157788506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fdde0 0x7f0c640fe1b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009ce0 tx=0x7f0c5c004d10 comp rx=0 tx=0).stop 2024-04-25T01:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.056+0000 7f0c6c704700 1 -- 172.21.15.33:0/1157788506 shutdown_connections 2024-04-25T01:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.056+0000 7f0c6c704700 1 --2- 172.21.15.33:0/1157788506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c640fe780 0x7f0c64106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.056+0000 7f0c6c704700 1 --2- 172.21.15.33:0/1157788506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fdde0 0x7f0c640fe1b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.056+0000 7f0c6c704700 1 -- 172.21.15.33:0/1157788506 >> 172.21.15.33:0/1157788506 conn(0x7f0c640f7890 msgr2=0x7f0c640f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.056+0000 7f0c6c704700 1 -- 172.21.15.33:0/1157788506 shutdown_connections 2024-04-25T01:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.056+0000 7f0c6c704700 1 -- 172.21.15.33:0/1157788506 wait complete. 2024-04-25T01:44:46.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c6c704700 1 Processor -- start 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c6c704700 1 -- start start 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c6c704700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c640fdde0 0x7f0c64113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c6c704700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fe780 0x7f0c641140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c6c704700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c64114740 con 0x7f0c640fe780 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c6c704700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6410db30 con 0x7f0c640fdde0 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c69c9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fe780 0x7f0c641140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c69c9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fe780 0x7f0c641140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56104/0 (socket says 172.21.15.33:56104) 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c69c9f700 1 -- 172.21.15.33:0/1189239811 learned_addr learned my addr 172.21.15.33:0/1189239811 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c6a4a0700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c640fdde0 0x7f0c64113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c69c9f700 1 -- 172.21.15.33:0/1189239811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c640fdde0 msgr2=0x7f0c64113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c69c9f700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c640fdde0 0x7f0c64113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c69c9f700 1 -- 172.21.15.33:0/1189239811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c009940 con 0x7f0c640fe780 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.057+0000 7f0c69c9f700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fe780 0x7f0c641140f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800eb40 tx=0x7f0c5800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:46.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.058+0000 7f0c62ffd700 1 -- 172.21.15.33:0/1189239811 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c58007cc0 con 0x7f0c640fe780 2024-04-25T01:44:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.058+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6410ddb0 con 0x7f0c640fe780 2024-04-25T01:44:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.058+0000 7f0c62ffd700 1 -- 172.21.15.33:0/1189239811 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c5800f040 con 0x7f0c640fe780 2024-04-25T01:44:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.058+0000 7f0c62ffd700 1 -- 172.21.15.33:0/1189239811 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c5800ca10 con 0x7f0c640fe780 2024-04-25T01:44:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.058+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6410e2f0 con 0x7f0c640fe780 2024-04-25T01:44:46.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.059+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c64066750 con 0x7f0c640fe780 2024-04-25T01:44:46.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.059+0000 7f0c62ffd700 1 -- 172.21.15.33:0/1189239811 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0c5800cc80 con 0x7f0c640fe780 2024-04-25T01:44:46.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.060+0000 7f0c62ffd700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c4c06e150 0x7f0c4c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:46.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.060+0000 7f0c62ffd700 1 -- 172.21.15.33:0/1189239811 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f0c5808d640 con 0x7f0c640fe780 2024-04-25T01:44:46.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.060+0000 7f0c6a4a0700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c4c06e150 0x7f0c4c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:46.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.060+0000 7f0c6a4a0700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c4c06e150 0x7f0c4c070600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005010 tx=0x7f0c5c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:46.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.064+0000 7f0c62ffd700 1 -- 172.21.15.33:0/1189239811 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0c5805a650 con 0x7f0c640fe780 2024-04-25T01:44:46.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.292+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c64109e70 con 0x7f0c4c06e150 2024-04-25T01:44:46.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.293+0000 7f0c62ffd700 1 -- 172.21.15.33:0/1189239811 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f0c64109e70 con 0x7f0c4c06e150 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c4c06e150 msgr2=0x7f0c4c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c4c06e150 0x7f0c4c070600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005010 tx=0x7f0c5c004d10 comp rx=0 tx=0).stop 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fe780 msgr2=0x7f0c641140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fe780 0x7f0c641140f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800eb40 tx=0x7f0c5800b040 comp rx=0 tx=0).stop 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 shutdown_connections 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c4c06e150 0x7f0c4c070600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c640fdde0 0x7f0c64113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 --2- 172.21.15.33:0/1189239811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c640fe780 0x7f0c641140f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 >> 172.21.15.33:0/1189239811 conn(0x7f0c640f7890 msgr2=0x7f0c640f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 shutdown_connections 2024-04-25T01:44:46.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.295+0000 7f0c6c704700 1 -- 172.21.15.33:0/1189239811 wait complete. 2024-04-25T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 -- 172.21.15.33:0/41136156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 msgr2=0x7fcbc01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 --2- 172.21.15.33:0/41136156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 0x7fcbc01031b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8009ce0 tx=0x7fcbb8004d10 comp rx=0 tx=0).stop 2024-04-25T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 -- 172.21.15.33:0/41136156 shutdown_connections 2024-04-25T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 --2- 172.21.15.33:0/41136156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc0103780 0x7fcbc00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 --2- 172.21.15.33:0/41136156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 0x7fcbc01031b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 -- 172.21.15.33:0/41136156 >> 172.21.15.33:0/41136156 conn(0x7fcbc00f8990 msgr2=0x7fcbc00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 -- 172.21.15.33:0/41136156 shutdown_connections 2024-04-25T01:44:46.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 -- 172.21.15.33:0/41136156 wait complete. 2024-04-25T01:44:46.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:46 smithi033 ceph-mon[111306]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 Processor -- start 2024-04-25T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.400+0000 7fcbc5312700 1 -- start start 2024-04-25T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbc5312700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 0x7fcbc0113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:46.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbc5312700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc0103780 0x7fcbc0113fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbc5312700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc01145f0 con 0x7fcbc0102de0 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbc5312700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc010dbd0 con 0x7fcbc0103780 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbbeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 0x7fcbc0113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbbeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 0x7fcbc0113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56116/0 (socket says 172.21.15.33:56116) 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbbeffd700 1 -- 172.21.15.33:0/1570099835 learned_addr learned my addr 172.21.15.33:0/1570099835 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbbe7fc700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc0103780 0x7fcbc0113fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbbeffd700 1 -- 172.21.15.33:0/1570099835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc0103780 msgr2=0x7fcbc0113fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbbeffd700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc0103780 0x7fcbc0113fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbbeffd700 1 -- 172.21.15.33:0/1570099835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb8009940 con 0x7fcbc0102de0 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.401+0000 7fcbbeffd700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 0x7fcbc0113a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8003400 tx=0x7fcbb8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.402+0000 7fcbb77fe700 1 -- 172.21.15.33:0/1570099835 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbb800e040 con 0x7fcbc0102de0 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.402+0000 7fcbb77fe700 1 -- 172.21.15.33:0/1570099835 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbb8004610 con 0x7fcbc0102de0 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.402+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc010de50 con 0x7fcbc0102de0 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.402+0000 7fcbb77fe700 1 -- 172.21.15.33:0/1570099835 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbb801c640 con 0x7fcbc0102de0 2024-04-25T01:44:46.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.402+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc010e410 con 0x7fcbc0102de0 2024-04-25T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.403+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc01000f0 con 0x7fcbc0102de0 2024-04-25T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.403+0000 7fcbb77fe700 1 -- 172.21.15.33:0/1570099835 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcbb8012a70 con 0x7fcbc0102de0 2024-04-25T01:44:46.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.404+0000 7fcbb77fe700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbac06e140 0x7fcbac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.404+0000 7fcbb77fe700 1 -- 172.21.15.33:0/1570099835 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7fcbb808f4a0 con 0x7fcbc0102de0 2024-04-25T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.404+0000 7fcbbe7fc700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbac06e140 0x7fcbac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:46.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.404+0000 7fcbbe7fc700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbac06e140 0x7fcbac0705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcbc010f300 tx=0x7fcba800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:46.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.409+0000 7fcbb77fe700 1 -- 172.21.15.33:0/1570099835 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcbb805c400 con 0x7fcbc0102de0 2024-04-25T01:44:46.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:46 smithi138 ceph-mon[81304]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:46.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.627+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbc0109f50 con 0x7fcbac06e140 2024-04-25T01:44:46.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.637+0000 7fcbb77fe700 1 -- 172.21.15.33:0/1570099835 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcbc0109f50 con 0x7fcbac06e140 2024-04-25T01:44:46.629 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (5m) 83s ago 17m 26.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (3m) 83s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (3m) 16s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (5m) 83s ago 16m 77.2M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (11m) 83s ago 11m 62.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (11m) 83s ago 11m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (11m) 16s ago 11m 120M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (11m) 16s ago 11m 61.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (7m) 83s ago 18m 504M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (6m) 16s ago 16m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (4m) 83s ago 18m 46.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (3m) 16s ago 16m 38.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (6m) 83s ago 17m 16.0M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:44:46.630 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (6m) 16s ago 16m 15.9M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:44:46.631 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (2m) 83s ago 15m 88.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:44:46.631 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (2m) 83s ago 14m 93.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:44:46.631 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (89s) 83s ago 14m 12.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:44:46.631 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (53s) 16s ago 14m 98.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:44:46.631 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (19s) 16s ago 13m 15.0M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:44:46.631 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (13m) 16s ago 13m 220M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 4d03083a2d76 2024-04-25T01:44:46.631 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (6m) 83s ago 16m 60.9M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:44:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbac06e140 msgr2=0x7fcbac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbac06e140 0x7fcbac0705f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcbc010f300 tx=0x7fcba800b040 comp rx=0 tx=0).stop 2024-04-25T01:44:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 msgr2=0x7fcbc0113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 0x7fcbc0113a60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8003400 tx=0x7fcbb8004d10 comp rx=0 tx=0).stop 2024-04-25T01:44:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 shutdown_connections 2024-04-25T01:44:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbac06e140 0x7fcbac0705f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbc0102de0 0x7fcbc0113a60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 --2- 172.21.15.33:0/1570099835 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbc0103780 0x7fcbc0113fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 >> 172.21.15.33:0/1570099835 conn(0x7fcbc00f8990 msgr2=0x7fcbc00fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 shutdown_connections 2024-04-25T01:44:46.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.640+0000 7fcbc5312700 1 -- 172.21.15.33:0/1570099835 wait complete. 2024-04-25T01:44:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.744+0000 7f5d68700700 1 -- 172.21.15.33:0/2187473793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 msgr2=0x7f5d601031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 --2- 172.21.15.33:0/2187473793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 0x7f5d601031b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c004940 tx=0x7f5d5c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:46.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 -- 172.21.15.33:0/2187473793 shutdown_connections 2024-04-25T01:44:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 --2- 172.21.15.33:0/2187473793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d60103780 0x7f5d600fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 --2- 172.21.15.33:0/2187473793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 0x7f5d601031b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 -- 172.21.15.33:0/2187473793 >> 172.21.15.33:0/2187473793 conn(0x7f5d600f8990 msgr2=0x7f5d600fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 -- 172.21.15.33:0/2187473793 shutdown_connections 2024-04-25T01:44:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 -- 172.21.15.33:0/2187473793 wait complete. 2024-04-25T01:44:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 Processor -- start 2024-04-25T01:44:46.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 -- start start 2024-04-25T01:44:46.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 0x7f5d60115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:46.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.745+0000 7f5d68700700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d60103780 0x7f5d60116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:46.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d68700700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d601166e0 con 0x7f5d60102de0 2024-04-25T01:44:46.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d68700700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6010fcc0 con 0x7f5d60103780 2024-04-25T01:44:46.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d6649c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 0x7f5d60115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d6649c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 0x7f5d60115b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56134/0 (socket says 172.21.15.33:56134) 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d6649c700 1 -- 172.21.15.33:0/3906007733 learned_addr learned my addr 172.21.15.33:0/3906007733 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d6649c700 1 -- 172.21.15.33:0/3906007733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d60103780 msgr2=0x7f5d60116090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d6649c700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d60103780 0x7f5d60116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d6649c700 1 -- 172.21.15.33:0/3906007733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c005040 con 0x7f5d60102de0 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d6649c700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 0x7f5d60115b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00b3b0 tx=0x7f5d5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d56ffd700 1 -- 172.21.15.33:0/3906007733 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d5c00e040 con 0x7f5d60102de0 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6010ff40 con 0x7f5d60102de0 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d56ffd700 1 -- 172.21.15.33:0/3906007733 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d5c00bbc0 con 0x7f5d60102de0 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d56ffd700 1 -- 172.21.15.33:0/3906007733 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d5c01c3f0 con 0x7f5d60102de0 2024-04-25T01:44:46.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.746+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d60110500 con 0x7f5d60102de0 2024-04-25T01:44:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.748+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d601072e0 con 0x7f5d60102de0 2024-04-25T01:44:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.748+0000 7f5d56ffd700 1 -- 172.21.15.33:0/3906007733 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5d5c00bd30 con 0x7f5d60102de0 2024-04-25T01:44:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.748+0000 7f5d56ffd700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d4c06e140 0x7f5d4c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.748+0000 7f5d56ffd700 1 -- 172.21.15.33:0/3906007733 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f5d5c08ed00 con 0x7f5d60102de0 2024-04-25T01:44:46.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.748+0000 7f5d65c9b700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d4c06e140 0x7f5d4c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:46.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.749+0000 7f5d65c9b700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d4c06e140 0x7f5d4c0705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d601113f0 tx=0x7f5d50009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:46.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:46.753+0000 7f5d56ffd700 1 -- 172.21.15.33:0/3906007733 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5d5c092030 con 0x7f5d60102de0 2024-04-25T01:44:47.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.054+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5d60109ee0 con 0x7f5d60102de0 2024-04-25T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.055+0000 7f5d56ffd700 1 -- 172.21.15.33:0/3906007733 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f5d5c05bc60 con 0x7f5d60102de0 2024-04-25T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:44:47.047 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 5 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5, 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 9 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:44:47.048 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d4c06e140 msgr2=0x7f5d4c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d4c06e140 0x7f5d4c0705f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5d601113f0 tx=0x7f5d50009ce0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 msgr2=0x7f5d60115b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 0x7f5d60115b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c00b3b0 tx=0x7f5d5c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 shutdown_connections 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d4c06e140 0x7f5d4c0705f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d60102de0 0x7f5d60115b50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 --2- 172.21.15.33:0/3906007733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d60103780 0x7f5d60116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 >> 172.21.15.33:0/3906007733 conn(0x7f5d600f8990 msgr2=0x7f5d600fa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 shutdown_connections 2024-04-25T01:44:47.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.057+0000 7f5d68700700 1 -- 172.21.15.33:0/3906007733 wait complete. 2024-04-25T01:44:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.161+0000 7f9c8a542700 1 -- 172.21.15.33:0/3354433960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 msgr2=0x7f9c84109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.161+0000 7f9c8a542700 1 --2- 172.21.15.33:0/3354433960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 0x7f9c84109350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9c74004940 tx=0x7f9c74009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.161+0000 7f9c8a542700 1 -- 172.21.15.33:0/3354433960 shutdown_connections 2024-04-25T01:44:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.161+0000 7f9c8a542700 1 --2- 172.21.15.33:0/3354433960 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 0x7f9c84109350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.161+0000 7f9c8a542700 1 --2- 172.21.15.33:0/3354433960 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c840fc790 0x7f9c840fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.161+0000 7f9c8a542700 1 -- 172.21.15.33:0/3354433960 >> 172.21.15.33:0/3354433960 conn(0x7f9c84071d00 msgr2=0x7f9c84072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:47.157 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.161+0000 7f9c8a542700 1 -- 172.21.15.33:0/3354433960 shutdown_connections 2024-04-25T01:44:47.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.161+0000 7f9c8a542700 1 -- 172.21.15.33:0/3354433960 wait complete. 2024-04-25T01:44:47.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.161+0000 7f9c8a542700 1 Processor -- start 2024-04-25T01:44:47.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c8a542700 1 -- start start 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c8a542700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c840fc790 0x7f9c84115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c8a542700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 0x7f9c84116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c8a542700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c841169c0 con 0x7f9c840fd0a0 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c8a542700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8410fd90 con 0x7f9c840fc790 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 0x7f9c84116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 0x7f9c84116370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56158/0 (socket says 172.21.15.33:56158) 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c837fe700 1 -- 172.21.15.33:0/3851901412 learned_addr learned my addr 172.21.15.33:0/3851901412 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c83fff700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c840fc790 0x7f9c84115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c837fe700 1 -- 172.21.15.33:0/3851901412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c840fc790 msgr2=0x7f9c84115e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c837fe700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c840fc790 0x7f9c84115e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c837fe700 1 -- 172.21.15.33:0/3851901412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c74005040 con 0x7f9c840fd0a0 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.162+0000 7f9c837fe700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 0x7f9c84116370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400b3b0 tx=0x7f9c74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.163+0000 7f9c80ff9700 1 -- 172.21.15.33:0/3851901412 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7400e040 con 0x7f9c840fd0a0 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.163+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c84110010 con 0x7f9c840fd0a0 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.163+0000 7f9c80ff9700 1 -- 172.21.15.33:0/3851901412 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c7400bbc0 con 0x7f9c840fd0a0 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.163+0000 7f9c80ff9700 1 -- 172.21.15.33:0/3851901412 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c7401c3f0 con 0x7f9c840fd0a0 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.163+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c84110550 con 0x7f9c840fd0a0 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.164+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c840fc330 con 0x7f9c840fd0a0 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.164+0000 7f9c80ff9700 1 -- 172.21.15.33:0/3851901412 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9c7400bd30 con 0x7f9c840fd0a0 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.165+0000 7f9c80ff9700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c7006e140 0x7f9c700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:47.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.165+0000 7f9c80ff9700 1 -- 172.21.15.33:0/3851901412 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f9c7408ed10 con 0x7f9c840fd0a0 2024-04-25T01:44:47.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.165+0000 7f9c83fff700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c7006e140 0x7f9c700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:47.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.165+0000 7f9c83fff700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c7006e140 0x7f9c700705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c004d60 tx=0x7f9c6c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:47.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.169+0000 7f9c80ff9700 1 -- 172.21.15.33:0/3851901412 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9c7405bc70 con 0x7f9c840fd0a0 2024-04-25T01:44:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.422+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9c841114b0 con 0x7f9c840fd0a0 2024-04-25T01:44:47.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.422+0000 7f9c80ff9700 1 -- 172.21.15.33:0/3851901412 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1903 (secure 0 0 0) 0x7f9c7405b800 con 0x7f9c840fd0a0 2024-04-25T01:44:47.417 INFO:teuthology.orchestra.run.smithi033.stdout:e19 2024-04-25T01:44:47.417 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:44:47.417 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:44:47.417 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:44:47.417 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:44:47.418 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:44:47.418 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:44:47.421 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 19 2024-04-25T01:44:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T01:44:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:44:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:40:21.849082+0000 2024-04-25T01:44:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:44:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:44:47.422 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:44:47.423 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:44:47.423 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:44:47.423 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:44:47.423 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:44:47.424 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:44:47.425 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 0 2024-04-25T01:44:47.426 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=14446,1=24279} 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:44:47.427 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:44:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:44:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:44:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:14446} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.33:6826/3303942324,v1:172.21.15.33:6827/3303942324] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:44:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{0:14458} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.33:6828/3968098059,v1:172.21.15.33:6829/3968098059] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:44:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{1:24279} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.138:6824/3887358693,v1:172.21.15.138:6825/3887358693] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:44:47.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:24289} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.138:6826/4019073381,v1:172.21.15.138:6827/4019073381] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:44:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:44:47.428 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:44:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.424+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c7006e140 msgr2=0x7f9c700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.424+0000 7f9c8a542700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c7006e140 0x7f9c700705f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c004d60 tx=0x7f9c6c00b040 comp rx=0 tx=0).stop 2024-04-25T01:44:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.424+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 msgr2=0x7f9c84116370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.424+0000 7f9c8a542700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 0x7f9c84116370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c7400b3b0 tx=0x7f9c74009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.425+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 shutdown_connections 2024-04-25T01:44:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.425+0000 7f9c8a542700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c7006e140 0x7f9c700705f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.425+0000 7f9c8a542700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c840fc790 0x7f9c84115e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.425+0000 7f9c8a542700 1 --2- 172.21.15.33:0/3851901412 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c840fd0a0 0x7f9c84116370 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.425+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 >> 172.21.15.33:0/3851901412 conn(0x7f9c84071d00 msgr2=0x7f9c840f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.425+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 shutdown_connections 2024-04-25T01:44:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.425+0000 7f9c8a542700 1 -- 172.21.15.33:0/3851901412 wait complete. 2024-04-25T01:44:47.430 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T01:44:47.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.531+0000 7f5eb20e2700 1 -- 172.21.15.33:0/2499358727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 msgr2=0x7f5eac0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.531+0000 7f5eb20e2700 1 --2- 172.21.15.33:0/2499358727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 0x7f5eac0fc1b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c004940 tx=0x7f5e9c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:47.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.531+0000 7f5eb20e2700 1 -- 172.21.15.33:0/2499358727 shutdown_connections 2024-04-25T01:44:47.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.531+0000 7f5eb20e2700 1 --2- 172.21.15.33:0/2499358727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 0x7f5eac0fc1b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.531+0000 7f5eb20e2700 1 --2- 172.21.15.33:0/2499358727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac101d90 0x7f5eac102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.531+0000 7f5eb20e2700 1 -- 172.21.15.33:0/2499358727 >> 172.21.15.33:0/2499358727 conn(0x7f5eac071d00 msgr2=0x7f5eac072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:47.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.531+0000 7f5eb20e2700 1 -- 172.21.15.33:0/2499358727 shutdown_connections 2024-04-25T01:44:47.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.531+0000 7f5eb20e2700 1 -- 172.21.15.33:0/2499358727 wait complete. 2024-04-25T01:44:47.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.531+0000 7f5eb20e2700 1 Processor -- start 2024-04-25T01:44:47.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eb20e2700 1 -- start start 2024-04-25T01:44:47.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eb20e2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 0x7f5eac112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:47.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eb20e2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac101d90 0x7f5eac112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:47.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eb20e2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac113640 con 0x7f5eac0fbd40 2024-04-25T01:44:47.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eb20e2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac10ca30 con 0x7f5eac101d90 2024-04-25T01:44:47.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 0x7f5eac112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:47.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eab7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 0x7f5eac112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56180/0 (socket says 172.21.15.33:56180) 2024-04-25T01:44:47.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eab7fe700 1 -- 172.21.15.33:0/3521336047 learned_addr learned my addr 172.21.15.33:0/3521336047 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eaaffd700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac101d90 0x7f5eac112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eab7fe700 1 -- 172.21.15.33:0/3521336047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac101d90 msgr2=0x7f5eac112ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eab7fe700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac101d90 0x7f5eac112ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eab7fe700 1 -- 172.21.15.33:0/3521336047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9c005040 con 0x7f5eac0fbd40 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.532+0000 7f5eaaffd700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac101d90 0x7f5eac112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.533+0000 7f5eab7fe700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 0x7f5eac112ab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400ead0 tx=0x7f5e94004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.533+0000 7f5ea3fff700 1 -- 172.21.15.33:0/3521336047 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e9400cc60 con 0x7f5eac0fbd40 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.533+0000 7f5ea3fff700 1 -- 172.21.15.33:0/3521336047 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e9400cdc0 con 0x7f5eac0fbd40 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.533+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac10ccb0 con 0x7f5eac0fbd40 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.533+0000 7f5ea3fff700 1 -- 172.21.15.33:0/3521336047 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e940109f0 con 0x7f5eac0fbd40 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.533+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac10d270 con 0x7f5eac0fbd40 2024-04-25T01:44:47.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.534+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac0fd430 con 0x7f5eac0fbd40 2024-04-25T01:44:47.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.534+0000 7f5ea3fff700 1 -- 172.21.15.33:0/3521336047 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5e94010b50 con 0x7f5eac0fbd40 2024-04-25T01:44:47.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.535+0000 7f5ea3fff700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e9806e140 0x7f5e980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:47.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.535+0000 7f5ea3fff700 1 -- 172.21.15.33:0/3521336047 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f5e94090250 con 0x7f5eac0fbd40 2024-04-25T01:44:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.535+0000 7f5eaaffd700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e9806e140 0x7f5e980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:47.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.535+0000 7f5eaaffd700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e9806e140 0x7f5e980705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00b3b0 tx=0x7f5e9c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:47.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.540+0000 7f5ea3fff700 1 -- 172.21.15.33:0/3521336047 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5e9405d2d0 con 0x7f5eac0fbd40 2024-04-25T01:44:47.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:47 smithi033 ceph-mon[111306]: pgmap v151: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:44:47.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:47 smithi033 ceph-mon[111306]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:47.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:47 smithi033 ceph-mon[111306]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:47.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:47 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3906007733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:47.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:47 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3851901412' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:44:47.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.767+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5eac04bed0 con 0x7f5e9806e140 2024-04-25T01:44:47.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.769+0000 7f5ea3fff700 1 -- 172.21.15.33:0/3521336047 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f5eac04bed0 con 0x7f5e9806e140 2024-04-25T01:44:47.761 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:44:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:44:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:44:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:44:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:44:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:44:47.761 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:44:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:44:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:44:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "11/21 daemons upgraded", 2024-04-25T01:44:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T01:44:47.762 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:44:47.762 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:44:47.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e9806e140 msgr2=0x7f5e980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e9806e140 0x7f5e980705f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00b3b0 tx=0x7f5e9c009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:47.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 msgr2=0x7f5eac112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 0x7f5eac112ab0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5e9400ead0 tx=0x7f5e94004500 comp rx=0 tx=0).stop 2024-04-25T01:44:47.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 shutdown_connections 2024-04-25T01:44:47.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e9806e140 0x7f5e980705f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5eac0fbd40 0x7f5eac112ab0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 --2- 172.21.15.33:0/3521336047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5eac101d90 0x7f5eac112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 >> 172.21.15.33:0/3521336047 conn(0x7f5eac071d00 msgr2=0x7f5eac0fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:47.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 shutdown_connections 2024-04-25T01:44:47.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.771+0000 7f5eb20e2700 1 -- 172.21.15.33:0/3521336047 wait complete. 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.876+0000 7f655c273700 1 -- 172.21.15.33:0/1743668727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65540fbd40 msgr2=0x7f65540fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.876+0000 7f655c273700 1 --2- 172.21.15.33:0/1743668727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65540fbd40 0x7f65540fc1b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6550004940 tx=0x7f6550009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.876+0000 7f655c273700 1 -- 172.21.15.33:0/1743668727 shutdown_connections 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.876+0000 7f655c273700 1 --2- 172.21.15.33:0/1743668727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65540fbd40 0x7f65540fc1b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.876+0000 7f655c273700 1 --2- 172.21.15.33:0/1743668727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6554101d90 0x7f6554102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.876+0000 7f655c273700 1 -- 172.21.15.33:0/1743668727 >> 172.21.15.33:0/1743668727 conn(0x7f6554071d00 msgr2=0x7f6554072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.876+0000 7f655c273700 1 -- 172.21.15.33:0/1743668727 shutdown_connections 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.876+0000 7f655c273700 1 -- 172.21.15.33:0/1743668727 wait complete. 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655c273700 1 Processor -- start 2024-04-25T01:44:47.869 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655c273700 1 -- start start 2024-04-25T01:44:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655c273700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65540fbd40 0x7f6554113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655c273700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6554101d90 0x7f65541140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655c273700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6554114720 con 0x7f6554101d90 2024-04-25T01:44:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655c273700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655410db30 con 0x7f65540fbd40 2024-04-25T01:44:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655980e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6554101d90 0x7f65541140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655980e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6554101d90 0x7f65541140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56204/0 (socket says 172.21.15.33:56204) 2024-04-25T01:44:47.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655980e700 1 -- 172.21.15.33:0/3720744137 learned_addr learned my addr 172.21.15.33:0/3720744137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655a00f700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65540fbd40 0x7f6554113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655980e700 1 -- 172.21.15.33:0/3720744137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65540fbd40 msgr2=0x7f6554113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655980e700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65540fbd40 0x7f6554113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.877+0000 7f655980e700 1 -- 172.21.15.33:0/3720744137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550005040 con 0x7f6554101d90 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.878+0000 7f655980e700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6554101d90 0x7f65541140d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f655000b3b0 tx=0x7f6550009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.878+0000 7f654affd700 1 -- 172.21.15.33:0/3720744137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f655000e040 con 0x7f6554101d90 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.878+0000 7f654affd700 1 -- 172.21.15.33:0/3720744137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6550013420 con 0x7f6554101d90 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.878+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655410ddb0 con 0x7f6554101d90 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.878+0000 7f654affd700 1 -- 172.21.15.33:0/3720744137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f655001c630 con 0x7f6554101d90 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.878+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655410e370 con 0x7f6554101d90 2024-04-25T01:44:47.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.879+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65540fd430 con 0x7f6554101d90 2024-04-25T01:44:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.879+0000 7f654affd700 1 -- 172.21.15.33:0/3720744137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6550013590 con 0x7f6554101d90 2024-04-25T01:44:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.880+0000 7f654affd700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f654006e140 0x7f65400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:44:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.880+0000 7f654affd700 1 -- 172.21.15.33:0/3720744137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6238+0+0 (secure 0 0 0) 0x7f655008ecd0 con 0x7f6554101d90 2024-04-25T01:44:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.880+0000 7f655a00f700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f654006e140 0x7f65400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:44:47.873 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.880+0000 7f655a00f700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f654006e140 0x7f65400705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6544004f60 tx=0x7f654400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:44:47.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:47.885+0000 7f654affd700 1 -- 172.21.15.33:0/3720744137 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6550058180 con 0x7f6554101d90 2024-04-25T01:44:48.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:47 smithi138 ceph-mon[81304]: pgmap v151: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-04-25T01:44:48.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:47 smithi138 ceph-mon[81304]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:48.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:47 smithi138 ceph-mon[81304]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:48.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:47 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3906007733' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:44:48.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:47 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3851901412' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:44:48.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.177+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f655410f1e0 con 0x7f6554101d90 2024-04-25T01:44:48.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.177+0000 7f654affd700 1 -- 172.21.15.33:0/3720744137 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f655005b840 con 0x7f6554101d90 2024-04-25T01:44:48.170 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_OK 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f654006e140 msgr2=0x7f65400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f654006e140 0x7f65400705f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6544004f60 tx=0x7f654400b040 comp rx=0 tx=0).stop 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6554101d90 msgr2=0x7f65541140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6554101d90 0x7f65541140d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f655000b3b0 tx=0x7f6550009e30 comp rx=0 tx=0).stop 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 shutdown_connections 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f654006e140 0x7f65400705f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65540fbd40 0x7f6554113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 --2- 172.21.15.33:0/3720744137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6554101d90 0x7f65541140d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 >> 172.21.15.33:0/3720744137 conn(0x7f6554071d00 msgr2=0x7f65540655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 shutdown_connections 2024-04-25T01:44:48.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:44:48.180+0000 7f655c273700 1 -- 172.21.15.33:0/3720744137 wait complete. 2024-04-25T01:44:49.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:48 smithi033 ceph-mon[111306]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:49.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:48 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3720744137' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:44:49.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:48 smithi138 ceph-mon[81304]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:44:49.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:48 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3720744137' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:44:50.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:49 smithi033 ceph-mon[111306]: pgmap v152: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T01:44:50.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:49 smithi138 ceph-mon[81304]: pgmap v152: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-04-25T01:44:51.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:50 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T01:44:51.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:50 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:51.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:50 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T01:44:51.033 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:50 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:51.975 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:51 smithi138 ceph-mon[81304]: pgmap v153: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-04-25T01:44:51.975 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:51 smithi138 ceph-mon[81304]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T01:44:51.975 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:51 smithi138 ceph-mon[81304]: Upgrade: osd.5 is safe to restart 2024-04-25T01:44:52.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:51 smithi033 ceph-mon[111306]: pgmap v153: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-04-25T01:44:52.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:51 smithi033 ceph-mon[111306]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T01:44:52.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:51 smithi033 ceph-mon[111306]: Upgrade: osd.5 is safe to restart 2024-04-25T01:44:53.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:53 smithi138 ceph-mon[81304]: Upgrade: Updating osd.5 2024-04-25T01:44:53.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:53.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:44:53.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:53 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:53.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:53 smithi138 ceph-mon[81304]: Deploying daemon osd.5 on smithi138 2024-04-25T01:44:53.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:53 smithi033 ceph-mon[111306]: Upgrade: Updating osd.5 2024-04-25T01:44:53.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:53.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T01:44:53.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:53 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:44:53.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:53 smithi033 ceph-mon[111306]: Deploying daemon osd.5 on smithi138 2024-04-25T01:44:54.128 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:54 smithi138 ceph-mon[81304]: pgmap v154: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:44:54.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:54 smithi033 ceph-mon[111306]: pgmap v154: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T01:44:55.520 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:55 smithi138 systemd[1]: Stopping Ceph osd.5 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:44:55.783 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:55 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[51830]: 2024-04-25T01:44:55.527+0000 7fbd85519700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T01:44:55.784 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:55 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[51830]: 2024-04-25T01:44:55.527+0000 7fbd85519700 -1 osd.5 73 *** Got signal Terminated *** 2024-04-25T01:44:55.784 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:55 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[51830]: 2024-04-25T01:44:55.527+0000 7fbd85519700 -1 osd.5 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T01:44:56.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:56 smithi033 ceph-mon[111306]: pgmap v155: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:44:56.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:56.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:44:56.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:56 smithi033 ceph-mon[111306]: osd.5 marked itself down and dead 2024-04-25T01:44:56.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:56 smithi138 ceph-mon[81304]: pgmap v155: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T01:44:56.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:44:56.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:44:56.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:56 smithi138 ceph-mon[81304]: osd.5 marked itself down and dead 2024-04-25T01:44:56.929 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:56 smithi138 podman[100846]: 2024-04-25 01:44:56.631558792 +0000 UTC m=+1.217782436 container died 4d03083a2d7659c86ac0965d6cea3cc130fdfb7e0f5175ee597ceeff7c212536 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, org.label-schema.vendor=CentOS) 2024-04-25T01:44:57.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:57 smithi138 ceph-mon[81304]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:44:57.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:57 smithi138 ceph-mon[81304]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-25T01:44:57.238 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:57 smithi138 ceph-mon[81304]: osdmap e74: 6 total, 5 up, 6 in 2024-04-25T01:44:57.239 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:57 smithi138 podman[100846]: 2024-04-25 01:44:57.037473614 +0000 UTC m=+1.623697259 container cleanup 4d03083a2d7659c86ac0965d6cea3cc130fdfb7e0f5175ee597ceeff7c212536 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=quincy-c61472c, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T01:44:57.239 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:57 smithi138 bash[100846]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5 2024-04-25T01:44:57.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:57 smithi033 ceph-mon[111306]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T01:44:57.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:57 smithi033 ceph-mon[111306]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-25T01:44:57.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:57 smithi033 ceph-mon[111306]: osdmap e74: 6 total, 5 up, 6 in 2024-04-25T01:44:57.533 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:57 smithi138 podman[100866]: 2024-04-25 01:44:57.246992755 +0000 UTC m=+0.616109491 container remove 4d03083a2d7659c86ac0965d6cea3cc130fdfb7e0f5175ee597ceeff7c212536 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T01:44:57.850 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:57 smithi138 podman[100957]: 2024-04-25 01:44:57.57301315 +0000 UTC m=+0.113667032 container create 38a98f100ec38c918c4a04f609c229a226f5d070e7dce1430e0b722c4ad6a76e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T01:44:57.851 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:57 smithi138 podman[100957]: 2024-04-25 01:44:57.477394697 +0000 UTC m=+0.018048607 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:44:57.851 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:57 smithi138 podman[100957]: 2024-04-25 01:44:57.784331392 +0000 UTC m=+0.324985274 container init 38a98f100ec38c918c4a04f609c229a226f5d070e7dce1430e0b722c4ad6a76e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T01:44:57.851 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:57 smithi138 podman[100957]: 2024-04-25 01:44:57.787974486 +0000 UTC m=+0.328628374 container start 38a98f100ec38c918c4a04f609c229a226f5d070e7dce1430e0b722c4ad6a76e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T01:44:57.851 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:57 smithi138 podman[100957]: 2024-04-25 01:44:57.859083615 +0000 UTC m=+0.399737499 container attach 38a98f100ec38c918c4a04f609c229a226f5d070e7dce1430e0b722c4ad6a76e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T01:44:58.283 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:58 smithi138 ceph-mon[81304]: pgmap v156: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:44:58.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:58 smithi138 ceph-mon[81304]: osdmap e75: 6 total, 5 up, 6 in 2024-04-25T01:44:58.284 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:58 smithi138 podman[100957]: 2024-04-25 01:44:58.079725452 +0000 UTC m=+0.620379346 container died 38a98f100ec38c918c4a04f609c229a226f5d070e7dce1430e0b722c4ad6a76e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T01:44:58.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:58 smithi033 ceph-mon[111306]: pgmap v156: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T01:44:58.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:58 smithi033 ceph-mon[111306]: osdmap e75: 6 total, 5 up, 6 in 2024-04-25T01:44:59.034 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:58 smithi138 podman[100983]: 2024-04-25 01:44:58.688224169 +0000 UTC m=+0.603533512 container remove 38a98f100ec38c918c4a04f609c229a226f5d070e7dce1430e0b722c4ad6a76e (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:44:59.034 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:58 smithi138 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-25T01:44:59.034 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:58 smithi138 systemd[1]: Stopped Ceph osd.5 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:44:59.034 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:58 smithi138 systemd[1]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.5.service: Consumed 1min 6.150s CPU time. 2024-04-25T01:44:59.034 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:58 smithi138 systemd[1]: Starting Ceph osd.5 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T01:44:59.354 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:59 smithi138 ceph-mon[81304]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T01:44:59.354 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:44:59 smithi138 ceph-mon[81304]: Health check failed: Degraded data redundancy: 7/321 objects degraded (2.181%), 3 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:59.355 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 podman[101088]: 2024-04-25 01:44:59.026488477 +0000 UTC m=+0.018118839 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:44:59.355 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 podman[101088]: 2024-04-25 01:44:59.131574632 +0000 UTC m=+0.123205015 container create e62117526527aec7724eaf4d89a8df7f447536dc487b0daf46f572ae9f97d661 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T01:44:59.355 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 podman[101088]: 2024-04-25 01:44:59.362829448 +0000 UTC m=+0.354459787 container init e62117526527aec7724eaf4d89a8df7f447536dc487b0daf46f572ae9f97d661 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T01:44:59.409 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:59 smithi033 ceph-mon[111306]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T01:44:59.410 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:44:59 smithi033 ceph-mon[111306]: Health check failed: Degraded data redundancy: 7/321 objects degraded (2.181%), 3 pgs degraded (PG_DEGRADED) 2024-04-25T01:44:59.783 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 podman[101088]: 2024-04-25 01:44:59.365978121 +0000 UTC m=+0.357608459 container start e62117526527aec7724eaf4d89a8df7f447536dc487b0daf46f572ae9f97d661 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T01:44:59.784 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 podman[101088]: 2024-04-25 01:44:59.451019654 +0000 UTC m=+0.442650005 container attach e62117526527aec7724eaf4d89a8df7f447536dc487b0daf46f572ae9f97d661 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate[101103]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate[101103]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 bash[101088]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 bash[101088]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 bash[101088]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 bash[101088]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 bash[101088]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 bash[101088]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 bash[101088]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-25T01:45:00.105 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate[101103]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-25T01:45:00.106 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate[101103]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-25T01:45:00.106 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate[101103]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-25T01:45:00.106 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate[101103]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T01:45:00.106 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate[101103]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-25T01:45:00.106 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:44:59 smithi138 podman[101088]: 2024-04-25 01:44:59.834796544 +0000 UTC m=+0.826426881 container died e62117526527aec7724eaf4d89a8df7f447536dc487b0daf46f572ae9f97d661 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:45:00.446 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:00 smithi138 ceph-mon[81304]: pgmap v159: 65 pgs: 3 active+undersized+degraded, 3 peering, 9 stale+active+clean, 2 active+undersized, 48 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7/321 objects degraded (2.181%) 2024-04-25T01:45:00.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:00 smithi033 ceph-mon[111306]: pgmap v159: 65 pgs: 3 active+undersized+degraded, 3 peering, 9 stale+active+clean, 2 active+undersized, 48 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 7/321 objects degraded (2.181%) 2024-04-25T01:45:00.770 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:00 smithi138 podman[101227]: 2024-04-25 01:45:00.45512898 +0000 UTC m=+0.615225110 container remove e62117526527aec7724eaf4d89a8df7f447536dc487b0daf46f572ae9f97d661 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T01:45:00.770 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:00 smithi138 podman[101292]: 2024-04-25 01:45:00.571487846 +0000 UTC m=+0.018242929 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T01:45:00.770 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:00 smithi138 podman[101292]: 2024-04-25 01:45:00.681258873 +0000 UTC m=+0.128013985 container create ab01b5d50ec749143885e7882d30df52c7b6b2273f4f1e3f5d91813b32c6a134 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T01:45:01.283 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:00 smithi138 podman[101292]: 2024-04-25 01:45:00.929022146 +0000 UTC m=+0.375777215 container init ab01b5d50ec749143885e7882d30df52c7b6b2273f4f1e3f5d91813b32c6a134 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T01:45:01.284 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:00 smithi138 podman[101292]: 2024-04-25 01:45:00.932198826 +0000 UTC m=+0.378953897 container start ab01b5d50ec749143885e7882d30df52c7b6b2273f4f1e3f5d91813b32c6a134 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T01:45:01.284 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:00 smithi138 ceph-osd[101312]: -- 172.21.15.138:0/2995917306 <== mon.1 v2:172.21.15.138:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e8b1b5680 con 0x564e8b17f000 2024-04-25T01:45:01.284 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:01 smithi138 bash[101292]: ab01b5d50ec749143885e7882d30df52c7b6b2273f4f1e3f5d91813b32c6a134 2024-04-25T01:45:01.284 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:01 smithi138 systemd[1]: Started Ceph osd.5 for b17b7f0e-02a2-11ef-bc93-c7b262605968. 2024-04-25T01:45:01.783 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:01 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T01:45:01.480+0000 7f29a7daa7c0 -1 Falling back to public interface 2024-04-25T01:45:02.155 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:02 smithi138 ceph-mon[81304]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/321 objects degraded (14.019%) 2024-04-25T01:45:02.155 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:02 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:02.155 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:02 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:02.155 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:02 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:02.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:02 smithi033 ceph-mon[111306]: pgmap v160: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 45/321 objects degraded (14.019%) 2024-04-25T01:45:03.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:02 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:03.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:02 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:03.103 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:02 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:03.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:03 smithi033 ceph-mon[111306]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-04-25T01:45:03.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:03 smithi033 ceph-mon[111306]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T01:45:03.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:03 smithi138 ceph-mon[81304]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-04-25T01:45:03.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:03 smithi138 ceph-mon[81304]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T01:45:05.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:05.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:05.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:05 smithi033 ceph-mon[111306]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-04-25T01:45:05.533 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:05.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:05.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:05 smithi138 ceph-mon[81304]: pgmap v162: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-04-25T01:45:06.215 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:06 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T01:45:06.111+0000 7f29a7daa7c0 -1 osd.5 73 log_to_monitors true 2024-04-25T01:45:06.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:06 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:06.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:06 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:06.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:06 smithi138 ceph-mon[81304]: from='osd.5 [v2:172.21.15.138:6816/2521547597,v1:172.21.15.138:6817/2521547597]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:45:06.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:06 smithi138 ceph-mon[81304]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:45:06.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:06 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:06.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:06 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:06.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:06 smithi033 ceph-mon[111306]: from='osd.5 [v2:172.21.15.138:6816/2521547597,v1:172.21.15.138:6817/2521547597]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:45:06.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:06 smithi033 ceph-mon[111306]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T01:45:07.243 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 01:45:07 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T01:45:07.146+0000 7f29951fa700 -1 osd.5 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T01:45:07.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:07 smithi138 ceph-mon[81304]: pgmap v163: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-04-25T01:45:07.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:07 smithi138 ceph-mon[81304]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:45:07.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:07 smithi138 ceph-mon[81304]: osdmap e76: 6 total, 5 up, 6 in 2024-04-25T01:45:07.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:07 smithi138 ceph-mon[81304]: from='osd.5 [v2:172.21.15.138:6816/2521547597,v1:172.21.15.138:6817/2521547597]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:45:07.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:07 smithi138 ceph-mon[81304]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:45:07.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:07 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 52/321 objects degraded (16.199%), 14 pgs degraded (PG_DEGRADED) 2024-04-25T01:45:07.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:07 smithi138 ceph-mon[81304]: from='osd.5 ' entity='osd.5' 2024-04-25T01:45:07.758 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:07 smithi033 ceph-mon[111306]: pgmap v163: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 52/321 objects degraded (16.199%) 2024-04-25T01:45:07.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:07 smithi033 ceph-mon[111306]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T01:45:07.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:07 smithi033 ceph-mon[111306]: osdmap e76: 6 total, 5 up, 6 in 2024-04-25T01:45:07.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:07 smithi033 ceph-mon[111306]: from='osd.5 [v2:172.21.15.138:6816/2521547597,v1:172.21.15.138:6817/2521547597]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:45:07.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:07 smithi033 ceph-mon[111306]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi138", "root=default"]}]: dispatch 2024-04-25T01:45:07.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:07 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 52/321 objects degraded (16.199%), 14 pgs degraded (PG_DEGRADED) 2024-04-25T01:45:07.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:07 smithi033 ceph-mon[111306]: from='osd.5 ' entity='osd.5' 2024-04-25T01:45:08.428 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:08 smithi138 ceph-mon[81304]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:45:08.428 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:08 smithi138 ceph-mon[81304]: osd.5 [v2:172.21.15.138:6816/2521547597,v1:172.21.15.138:6817/2521547597] boot 2024-04-25T01:45:08.428 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:08 smithi138 ceph-mon[81304]: osdmap e77: 6 total, 6 up, 6 in 2024-04-25T01:45:08.428 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:08 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:45:08.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:08 smithi033 ceph-mon[111306]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T01:45:08.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:08 smithi033 ceph-mon[111306]: osd.5 [v2:172.21.15.138:6816/2521547597,v1:172.21.15.138:6817/2521547597] boot 2024-04-25T01:45:08.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:08 smithi033 ceph-mon[111306]: osdmap e77: 6 total, 6 up, 6 in 2024-04-25T01:45:08.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:08 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T01:45:10.196 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: pgmap v166: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1017 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 52/321 objects degraded (16.199%) 2024-04-25T01:45:10.196 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.196 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all osd 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-25T01:45:10.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: Upgrade: Setting require_osd_release to 19 squid 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: osdmap e78: 6 total, 6 up, 6 in 2024-04-25T01:45:10.198 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.199 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-25T01:45:10.199 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: pgmap v166: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 257 MiB data, 1017 MiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 52/321 objects degraded (16.199%) 2024-04-25T01:45:10.199 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.199 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.199 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:10.199 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:10.199 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all osd 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-25T01:45:10.200 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: Upgrade: Setting require_osd_release to 19 squid 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: osdmap e78: 6 total, 6 up, 6 in 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:10.201 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-04-25T01:45:10.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T01:45:10.322+0000 7f05e3512700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:45:11.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-25T01:45:11.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: osdmap e79: 6 total, 6 up, 6 in 2024-04-25T01:45:11.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-25T01:45:11.187 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:45:11.188 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T01:45:11.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:45:11.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-25T01:45:11.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T01:45:11.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: osdmap e80: 6 total, 6 up, 6 in 2024-04-25T01:45:11.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] up:boot 2024-04-25T01:45:11.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6828/2956426668,v1:172.21.15.33:6829/2956426668] up:boot 2024-04-25T01:45:11.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-25T01:45:11.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:45:11.189 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:45:11.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-04-25T01:45:11.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: osdmap e79: 6 total, 6 up, 6 in 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi138.jwzxlz=up:active} 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:11.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: osdmap e80: 6 total, 6 up, 6 in 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] up:boot 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6828/2956426668,v1:172.21.15.33:6829/2956426668] up:boot 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:45:11.286 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T01:45:11.982 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:11 smithi033 ceph-mon[111306]: pgmap v169: 65 pgs: 3 peering, 7 active+undersized+degraded, 10 active+undersized, 45 active+clean; 257 MiB data, 1017 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 21/321 objects degraded (6.542%) 2024-04-25T01:45:11.983 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:11 smithi033 ceph-mon[111306]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-25T01:45:11.983 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:11 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.138:6824/2355068967,v1:172.21.15.138:6825/2355068967] up:boot 2024-04-25T01:45:11.983 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:11 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6826/4046804099,v1:172.21.15.33:6827/4046804099] up:boot 2024-04-25T01:45:11.983 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:11 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T01:45:11.983 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:45:11.983 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:45:11.983 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:11 smithi033 ceph-mon[111306]: Health check update: Degraded data redundancy: 21/321 objects degraded (6.542%), 7 pgs degraded (PG_DEGRADED) 2024-04-25T01:45:12.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:11 smithi138 ceph-mon[81304]: pgmap v169: 65 pgs: 3 peering, 7 active+undersized+degraded, 10 active+undersized, 45 active+clean; 257 MiB data, 1017 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 21/321 objects degraded (6.542%) 2024-04-25T01:45:12.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:11 smithi138 ceph-mon[81304]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-25T01:45:12.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:11 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.138:6824/2355068967,v1:172.21.15.138:6825/2355068967] up:boot 2024-04-25T01:45:12.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:11 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6826/4046804099,v1:172.21.15.33:6827/4046804099] up:boot 2024-04-25T01:45:12.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:11 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T01:45:12.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:45:12.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:45:12.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:11 smithi138 ceph-mon[81304]: Health check update: Degraded data redundancy: 21/321 objects degraded (6.542%), 7 pgs degraded (PG_DEGRADED) 2024-04-25T01:45:13.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:13 smithi033 ceph-mon[111306]: Upgrade: Updating mds.cephfs.smithi033.qqhhcz 2024-04-25T01:45:13.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:13 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:13.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:13 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.qqhhcz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:45:13.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:13 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:13.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:13 smithi033 ceph-mon[111306]: Deploying daemon mds.cephfs.smithi033.qqhhcz on smithi033 2024-04-25T01:45:13.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:13 smithi138 ceph-mon[81304]: Upgrade: Updating mds.cephfs.smithi033.qqhhcz 2024-04-25T01:45:13.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:13 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:13.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:13 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.qqhhcz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:45:13.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:13 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:13.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:13 smithi138 ceph-mon[81304]: Deploying daemon mds.cephfs.smithi033.qqhhcz on smithi033 2024-04-25T01:45:14.378 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:14 smithi033 ceph-mon[111306]: pgmap v171: 65 pgs: 3 peering, 1 active+undersized+degraded, 1 active+undersized, 60 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 2/321 objects degraded (0.623%) 2024-04-25T01:45:14.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:14 smithi138 ceph-mon[81304]: pgmap v171: 65 pgs: 3 peering, 1 active+undersized+degraded, 1 active+undersized, 60 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 2/321 objects degraded (0.623%) 2024-04-25T01:45:15.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:15 smithi033 ceph-mon[111306]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/321 objects degraded (0.623%), 1 pg degraded) 2024-04-25T01:45:15.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:15 smithi033 ceph-mon[111306]: osdmap e81: 6 total, 6 up, 6 in 2024-04-25T01:45:15.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:15 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-25T01:45:15.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:15 smithi138 ceph-mon[81304]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/321 objects degraded (0.623%), 1 pg degraded) 2024-04-25T01:45:15.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:15 smithi138 ceph-mon[81304]: osdmap e81: 6 total, 6 up, 6 in 2024-04-25T01:45:15.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:15 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-25T01:45:16.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:16 smithi033 ceph-mon[111306]: pgmap v172: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:16.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:16 smithi138 ceph-mon[81304]: pgmap v172: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:17.508 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:17 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:45:17.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:17 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.301+0000 7ff8174c3700 1 -- 172.21.15.33:0/1562802981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8100fbd40 msgr2=0x7ff8100fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.301+0000 7ff8174c3700 1 --2- 172.21.15.33:0/1562802981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8100fbd40 0x7ff8100fc1b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff800004940 tx=0x7ff800009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.301+0000 7ff8174c3700 1 -- 172.21.15.33:0/1562802981 shutdown_connections 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.301+0000 7ff8174c3700 1 --2- 172.21.15.33:0/1562802981 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8100fbd40 0x7ff8100fc1b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.301+0000 7ff8174c3700 1 --2- 172.21.15.33:0/1562802981 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff810101d90 0x7ff810102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.301+0000 7ff8174c3700 1 -- 172.21.15.33:0/1562802981 >> 172.21.15.33:0/1562802981 conn(0x7ff810071d00 msgr2=0x7ff810072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.302+0000 7ff8174c3700 1 -- 172.21.15.33:0/1562802981 shutdown_connections 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.302+0000 7ff8174c3700 1 -- 172.21.15.33:0/1562802981 wait complete. 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.302+0000 7ff8174c3700 1 Processor -- start 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.302+0000 7ff8174c3700 1 -- start start 2024-04-25T01:45:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.302+0000 7ff8174c3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8100fbd40 0x7ff810112ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.302+0000 7ff8174c3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810101d90 0x7ff810113010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff8174c3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff810113660 con 0x7ff810101d90 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff8174c3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81010ca50 con 0x7ff8100fbd40 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff814a5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810101d90 0x7ff810113010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff814a5e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810101d90 0x7ff810113010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55268/0 (socket says 172.21.15.33:55268) 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff814a5e700 1 -- 172.21.15.33:0/494475447 learned_addr learned my addr 172.21.15.33:0/494475447 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff81525f700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8100fbd40 0x7ff810112ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff814a5e700 1 -- 172.21.15.33:0/494475447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8100fbd40 msgr2=0x7ff810112ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff814a5e700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8100fbd40 0x7ff810112ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff814a5e700 1 -- 172.21.15.33:0/494475447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800005040 con 0x7ff810101d90 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff814a5e700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810101d90 0x7ff810113010 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff80000b3b0 tx=0x7ff800009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff805ffb700 1 -- 172.21.15.33:0/494475447 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff80000e040 con 0x7ff810101d90 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81010ccd0 con 0x7ff810101d90 2024-04-25T01:45:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff805ffb700 1 -- 172.21.15.33:0/494475447 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff800013420 con 0x7ff810101d90 2024-04-25T01:45:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.303+0000 7ff805ffb700 1 -- 172.21.15.33:0/494475447 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff80001c630 con 0x7ff810101d90 2024-04-25T01:45:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.304+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81010d290 con 0x7ff810101d90 2024-04-25T01:45:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.305+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8100fd430 con 0x7ff810101d90 2024-04-25T01:45:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.305+0000 7ff805ffb700 1 -- 172.21.15.33:0/494475447 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff800013590 con 0x7ff810101d90 2024-04-25T01:45:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.306+0000 7ff805ffb700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff7fc06e140 0x7ff7fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.306+0000 7ff805ffb700 1 -- 172.21.15.33:0/494475447 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff80008ed80 con 0x7ff810101d90 2024-04-25T01:45:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.306+0000 7ff81525f700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff7fc06e140 0x7ff7fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.306+0000 7ff81525f700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff7fc06e140 0x7ff7fc0705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff80c004f60 tx=0x7ff80c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.311+0000 7ff805ffb700 1 -- 172.21.15.33:0/494475447 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff80005bcb0 con 0x7ff810101d90 2024-04-25T01:45:18.377 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:18 smithi033 ceph-mon[111306]: pgmap v174: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:18.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.540+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff810108e10 con 0x7ff7fc06e140 2024-04-25T01:45:18.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:18 smithi138 ceph-mon[81304]: pgmap v174: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:18.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.542+0000 7ff805ffb700 1 -- 172.21.15.33:0/494475447 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff810108e10 con 0x7ff7fc06e140 2024-04-25T01:45:18.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff7fc06e140 msgr2=0x7ff7fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff7fc06e140 0x7ff7fc0705f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff80c004f60 tx=0x7ff80c00b040 comp rx=0 tx=0).stop 2024-04-25T01:45:18.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810101d90 msgr2=0x7ff810113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810101d90 0x7ff810113010 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff80000b3b0 tx=0x7ff800009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 shutdown_connections 2024-04-25T01:45:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff7fc06e140 0x7ff7fc0705f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8100fbd40 0x7ff810112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 --2- 172.21.15.33:0/494475447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff810101d90 0x7ff810113010 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 >> 172.21.15.33:0/494475447 conn(0x7ff810071d00 msgr2=0x7ff8100fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 shutdown_connections 2024-04-25T01:45:18.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.544+0000 7ff8174c3700 1 -- 172.21.15.33:0/494475447 wait complete. 2024-04-25T01:45:18.547 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:45:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.649+0000 7f58e0fae700 1 -- 172.21.15.33:0/2392974884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 msgr2=0x7f58dc0fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.649+0000 7f58e0fae700 1 --2- 172.21.15.33:0/2392974884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 0x7f58dc0fe1d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004940 tx=0x7f58cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.650+0000 7f58e0fae700 1 -- 172.21.15.33:0/2392974884 shutdown_connections 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.650+0000 7f58e0fae700 1 --2- 172.21.15.33:0/2392974884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58dc0fe7a0 0x7f58dc106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.650+0000 7f58e0fae700 1 --2- 172.21.15.33:0/2392974884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 0x7f58dc0fe1d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.650+0000 7f58e0fae700 1 -- 172.21.15.33:0/2392974884 >> 172.21.15.33:0/2392974884 conn(0x7f58dc0f7860 msgr2=0x7f58dc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.650+0000 7f58e0fae700 1 -- 172.21.15.33:0/2392974884 shutdown_connections 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.650+0000 7f58e0fae700 1 -- 172.21.15.33:0/2392974884 wait complete. 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.650+0000 7f58e0fae700 1 Processor -- start 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.650+0000 7f58e0fae700 1 -- start start 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58e0fae700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 0x7f58dc0fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58e0fae700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58dc0fe7a0 0x7f58dc0fc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58e0fae700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0fdc10 con 0x7f58dc0fde00 2024-04-25T01:45:18.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58e0fae700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0fc740 con 0x7f58dc0fe7a0 2024-04-25T01:45:18.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 0x7f58dc0fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 0x7f58dc0fbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55282/0 (socket says 172.21.15.33:55282) 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58da59c700 1 -- 172.21.15.33:0/3593575601 learned_addr learned my addr 172.21.15.33:0/3593575601 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58d9d9b700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58dc0fe7a0 0x7f58dc0fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58da59c700 1 -- 172.21.15.33:0/3593575601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58dc0fe7a0 msgr2=0x7f58dc0fc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58da59c700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58dc0fe7a0 0x7f58dc0fc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58da59c700 1 -- 172.21.15.33:0/3593575601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc005040 con 0x7f58dc0fde00 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.651+0000 7f58da59c700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 0x7f58dc0fbcc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004890 tx=0x7f58cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.652+0000 7f58caffd700 1 -- 172.21.15.33:0/3593575601 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58cc00e040 con 0x7f58dc0fde00 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.652+0000 7f58caffd700 1 -- 172.21.15.33:0/3593575601 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58cc009af0 con 0x7f58dc0fde00 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.652+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc0fc9c0 con 0x7f58dc0fde00 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.652+0000 7f58caffd700 1 -- 172.21.15.33:0/3593575601 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58cc01c3f0 con 0x7f58dc0fde00 2024-04-25T01:45:18.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.652+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc11a0c0 con 0x7f58dc0fde00 2024-04-25T01:45:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.653+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc066ad0 con 0x7f58dc0fde00 2024-04-25T01:45:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.653+0000 7f58caffd700 1 -- 172.21.15.33:0/3593575601 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f58cc01c6b0 con 0x7f58dc0fde00 2024-04-25T01:45:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.654+0000 7f58caffd700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f58c406e140 0x7f58c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.654+0000 7f58caffd700 1 -- 172.21.15.33:0/3593575601 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f58cc08edb0 con 0x7f58dc0fde00 2024-04-25T01:45:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.654+0000 7f58d9d9b700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f58c406e140 0x7f58c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.654+0000 7f58d9d9b700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f58c406e140 0x7f58c40705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0fd3f0 tx=0x7f58d0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:18.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.658+0000 7f58caffd700 1 -- 172.21.15.33:0/3593575601 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f58cc059110 con 0x7f58dc0fde00 2024-04-25T01:45:18.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.890+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58dc0fd1a0 con 0x7f58c406e140 2024-04-25T01:45:18.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.891+0000 7f58caffd700 1 -- 172.21.15.33:0/3593575601 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f58dc0fd1a0 con 0x7f58c406e140 2024-04-25T01:45:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f58c406e140 msgr2=0x7f58c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f58c406e140 0x7f58c40705f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f58dc0fd3f0 tx=0x7f58d0009040 comp rx=0 tx=0).stop 2024-04-25T01:45:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 msgr2=0x7f58dc0fbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 0x7f58dc0fbcc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f58cc004890 tx=0x7f58cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 shutdown_connections 2024-04-25T01:45:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f58c406e140 0x7f58c40705f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f58dc0fde00 0x7f58dc0fbcc0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 --2- 172.21.15.33:0/3593575601 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f58dc0fe7a0 0x7f58dc0fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 >> 172.21.15.33:0/3593575601 conn(0x7f58dc0f7860 msgr2=0x7f58dc0f8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.893+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 shutdown_connections 2024-04-25T01:45:18.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:18.894+0000 7f58e0fae700 1 -- 172.21.15.33:0/3593575601 wait complete. 2024-04-25T01:45:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.003+0000 7fa9f8928700 1 -- 172.21.15.33:0/3465797733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 msgr2=0x7fa9f4106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.003+0000 7fa9f8928700 1 --2- 172.21.15.33:0/3465797733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 0x7fa9f4106de0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc004940 tx=0x7fa9dc009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.003+0000 7fa9f8928700 1 -- 172.21.15.33:0/3465797733 shutdown_connections 2024-04-25T01:45:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.003+0000 7fa9f8928700 1 --2- 172.21.15.33:0/3465797733 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 0x7fa9f4106de0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.003+0000 7fa9f8928700 1 --2- 172.21.15.33:0/3465797733 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f40fde00 0x7fa9f40fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.003+0000 7fa9f8928700 1 -- 172.21.15.33:0/3465797733 >> 172.21.15.33:0/3465797733 conn(0x7fa9f40f7870 msgr2=0x7fa9f40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.003+0000 7fa9f8928700 1 -- 172.21.15.33:0/3465797733 shutdown_connections 2024-04-25T01:45:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.004+0000 7fa9f8928700 1 -- 172.21.15.33:0/3465797733 wait complete. 2024-04-25T01:45:18.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.004+0000 7fa9f8928700 1 Processor -- start 2024-04-25T01:45:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.004+0000 7fa9f8928700 1 -- start start 2024-04-25T01:45:18.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.004+0000 7fa9f8928700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f40fde00 0x7fa9f40fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.004+0000 7fa9f8928700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 0x7fa9f40fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.004+0000 7fa9f8928700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40fdb80 con 0x7fa9f40fe7a0 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.004+0000 7fa9f8928700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40fc6b0 con 0x7fa9f40fde00 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.004+0000 7fa9f1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 0x7fa9f40fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9f1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 0x7fa9f40fc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55308/0 (socket says 172.21.15.33:55308) 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9f1d9b700 1 -- 172.21.15.33:0/3043077730 learned_addr learned my addr 172.21.15.33:0/3043077730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9f259c700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f40fde00 0x7fa9f40fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9f1d9b700 1 -- 172.21.15.33:0/3043077730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f40fde00 msgr2=0x7fa9f40fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9f1d9b700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f40fde00 0x7fa9f40fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9f1d9b700 1 -- 172.21.15.33:0/3043077730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9dc005040 con 0x7fa9f40fe7a0 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9f1d9b700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 0x7fa9f40fc170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009180 tx=0x7fa9dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9eaffd700 1 -- 172.21.15.33:0/3043077730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9dc00e040 con 0x7fa9f40fe7a0 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9eaffd700 1 -- 172.21.15.33:0/3043077730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9dc013420 con 0x7fa9f40fe7a0 2024-04-25T01:45:18.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f40fc930 con 0x7fa9f40fe7a0 2024-04-25T01:45:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9eaffd700 1 -- 172.21.15.33:0/3043077730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9dc01c630 con 0x7fa9f40fe7a0 2024-04-25T01:45:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.005+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f411a040 con 0x7fa9f40fe7a0 2024-04-25T01:45:19.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.006+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f4066b30 con 0x7fa9f40fe7a0 2024-04-25T01:45:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.007+0000 7fa9eaffd700 1 -- 172.21.15.33:0/3043077730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa9dc013590 con 0x7fa9f40fe7a0 2024-04-25T01:45:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.007+0000 7fa9eaffd700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9e006e140 0x7fa9e00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.007+0000 7fa9eaffd700 1 -- 172.21.15.33:0/3043077730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa9dc08ee90 con 0x7fa9f40fe7a0 2024-04-25T01:45:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.008+0000 7fa9f259c700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9e006e140 0x7fa9e00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:19.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.008+0000 7fa9f259c700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9e006e140 0x7fa9e00705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4004990 tx=0x7fa9e4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:19.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.016+0000 7fa9eaffd700 1 -- 172.21.15.33:0/3043077730 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa9dc058290 con 0x7fa9f40fe7a0 2024-04-25T01:45:19.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.239+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9f40f9490 con 0x7fa9e006e140 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.250+0000 7fa9eaffd700 1 -- 172.21.15.33:0/3043077730 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa9f40f9490 con 0x7fa9e006e140 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (6m) 116s ago 17m 26.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (3m) 116s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (3m) 15s ago 16m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (6m) 116s ago 17m 77.2M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (12m) 116s ago 12m 62.1M - 17.2.7-902-gc61472c8 0de407c1b3af 1e72c18a04aa 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (12m) 116s ago 12m 113M - 17.2.7-902-gc61472c8 0de407c1b3af 80ad24666d9e 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (12m) 15s ago 12m 119M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (12m) 15s ago 12m 61.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:45:19.243 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (7m) 116s ago 18m 504M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (7m) 15s ago 16m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (4m) 116s ago 19m 46.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (4m) 15s ago 16m 39.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (7m) 116s ago 17m 16.0M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 15s ago 16m 15.2M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (3m) 116s ago 15m 88.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (2m) 116s ago 15m 93.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (2m) 116s ago 15m 12.1M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (86s) 15s ago 14m 100M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (52s) 15s ago 14m 100M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:45:19.244 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (18s) 15s ago 13m 12.6M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:45:19.245 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (6m) 116s ago 17m 60.9M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:45:19.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9e006e140 msgr2=0x7fa9e00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9e006e140 0x7fa9e00705f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4004990 tx=0x7fa9e4009ee0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 msgr2=0x7fa9f40fc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 0x7fa9f40fc170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009180 tx=0x7fa9dc009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 shutdown_connections 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9e006e140 0x7fa9e00705f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9f40fde00 0x7fa9f40fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 --2- 172.21.15.33:0/3043077730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9f40fe7a0 0x7fa9f40fc170 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 >> 172.21.15.33:0/3043077730 conn(0x7fa9f40f7870 msgr2=0x7fa9f40f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 shutdown_connections 2024-04-25T01:45:19.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.252+0000 7fa9f8928700 1 -- 172.21.15.33:0/3043077730 wait complete. 2024-04-25T01:45:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.356+0000 7faed4217700 1 -- 172.21.15.33:0/1767930146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 msgr2=0x7faecc106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.356+0000 7faed4217700 1 --2- 172.21.15.33:0/1767930146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 0x7faecc106df0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7faebc004940 tx=0x7faebc009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:19.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.356+0000 7faed4217700 1 -- 172.21.15.33:0/1767930146 shutdown_connections 2024-04-25T01:45:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.356+0000 7faed4217700 1 --2- 172.21.15.33:0/1767930146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 0x7faecc106df0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.356+0000 7faed4217700 1 --2- 172.21.15.33:0/1767930146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc0fde10 0x7faecc0fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.356+0000 7faed4217700 1 -- 172.21.15.33:0/1767930146 >> 172.21.15.33:0/1767930146 conn(0x7faecc0f7860 msgr2=0x7faecc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.356+0000 7faed4217700 1 -- 172.21.15.33:0/1767930146 shutdown_connections 2024-04-25T01:45:19.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.356+0000 7faed4217700 1 -- 172.21.15.33:0/1767930146 wait complete. 2024-04-25T01:45:19.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed4217700 1 Processor -- start 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed4217700 1 -- start start 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed4217700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc0fde10 0x7faecc113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed4217700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 0x7faecc113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed4217700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc1145a0 con 0x7faecc0fe7b0 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed4217700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc10db10 con 0x7faecc0fde10 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed17b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 0x7faecc113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed17b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 0x7faecc113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55322/0 (socket says 172.21.15.33:55322) 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed17b2700 1 -- 172.21.15.33:0/2689857073 learned_addr learned my addr 172.21.15.33:0/2689857073 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.357+0000 7faed1fb3700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc0fde10 0x7faecc113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.358+0000 7faed17b2700 1 -- 172.21.15.33:0/2689857073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc0fde10 msgr2=0x7faecc113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.358+0000 7faed17b2700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc0fde10 0x7faecc113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.358+0000 7faed17b2700 1 -- 172.21.15.33:0/2689857073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc005040 con 0x7faecc0fe7b0 2024-04-25T01:45:19.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.358+0000 7faed17b2700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 0x7faecc113ec0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faebc009380 tx=0x7faebc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:19.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.358+0000 7faec27fc700 1 -- 172.21.15.33:0/2689857073 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faebc00e040 con 0x7faecc0fe7b0 2024-04-25T01:45:19.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.358+0000 7faec27fc700 1 -- 172.21.15.33:0/2689857073 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faebc009af0 con 0x7faecc0fe7b0 2024-04-25T01:45:19.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.358+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc10dd90 con 0x7faecc0fe7b0 2024-04-25T01:45:19.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.358+0000 7faec27fc700 1 -- 172.21.15.33:0/2689857073 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faebc01c3f0 con 0x7faecc0fe7b0 2024-04-25T01:45:19.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.358+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc10e2d0 con 0x7faecc0fe7b0 2024-04-25T01:45:19.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.359+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faecc066b50 con 0x7faecc0fe7b0 2024-04-25T01:45:19.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.360+0000 7faec27fc700 1 -- 172.21.15.33:0/2689857073 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faebc013420 con 0x7faecc0fe7b0 2024-04-25T01:45:19.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.360+0000 7faec27fc700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faeb806e140 0x7faeb80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:19.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.360+0000 7faec27fc700 1 -- 172.21.15.33:0/2689857073 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7faebc08ee80 con 0x7faecc0fe7b0 2024-04-25T01:45:19.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.360+0000 7faed1fb3700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faeb806e140 0x7faeb80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:19.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.360+0000 7faed1fb3700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faeb806e140 0x7faeb80705f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faec8009bf0 tx=0x7faec80091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:19.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.365+0000 7faec27fc700 1 -- 172.21.15.33:0/2689857073 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faebc05bd30 con 0x7faecc0fe7b0 2024-04-25T01:45:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.681+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faecc109f20 con 0x7faecc0fe7b0 2024-04-25T01:45:19.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.682+0000 7faec27fc700 1 -- 172.21.15.33:0/2689857073 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+733 (secure 0 0 0) 0x7faebc05b8c0 con 0x7faecc0fe7b0 2024-04-25T01:45:19.675 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:45:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:45:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:45:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:45:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:45:19.675 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 10 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:45:19.676 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:45:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faeb806e140 msgr2=0x7faeb80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faeb806e140 0x7faeb80705f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7faec8009bf0 tx=0x7faec80091a0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 msgr2=0x7faecc113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 0x7faecc113ec0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faebc009380 tx=0x7faebc009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 shutdown_connections 2024-04-25T01:45:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faeb806e140 0x7faeb80705f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faecc0fde10 0x7faecc113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 --2- 172.21.15.33:0/2689857073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faecc0fe7b0 0x7faecc113ec0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 >> 172.21.15.33:0/2689857073 conn(0x7faecc0f7860 msgr2=0x7faecc103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 shutdown_connections 2024-04-25T01:45:19.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.684+0000 7faed4217700 1 -- 172.21.15.33:0/2689857073 wait complete. 2024-04-25T01:45:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.798+0000 7fadac870700 1 -- 172.21.15.33:0/4111818353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fdd80 msgr2=0x7fada80fe150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.798+0000 7fadac870700 1 --2- 172.21.15.33:0/4111818353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fdd80 0x7fada80fe150 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fad98004940 tx=0x7fad98009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 -- 172.21.15.33:0/4111818353 shutdown_connections 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 --2- 172.21.15.33:0/4111818353 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada80fe720 0x7fada8106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 --2- 172.21.15.33:0/4111818353 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fdd80 0x7fada80fe150 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 -- 172.21.15.33:0/4111818353 >> 172.21.15.33:0/4111818353 conn(0x7fada80f7830 msgr2=0x7fada80f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 -- 172.21.15.33:0/4111818353 shutdown_connections 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 -- 172.21.15.33:0/4111818353 wait complete. 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 Processor -- start 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 -- start start 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada80fdd80 0x7fada8113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fe720 0x7fada8114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada81146e0 con 0x7fada80fe720 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fadac870700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada810dad0 con 0x7fada80fdd80 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fada5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fe720 0x7fada8114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:19.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.799+0000 7fada5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fe720 0x7fada8114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55340/0 (socket says 172.21.15.33:55340) 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fada5d9b700 1 -- 172.21.15.33:0/770414064 learned_addr learned my addr 172.21.15.33:0/770414064 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fada659c700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada80fdd80 0x7fada8113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fada5d9b700 1 -- 172.21.15.33:0/770414064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada80fdd80 msgr2=0x7fada8113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fada5d9b700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada80fdd80 0x7fada8113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fada5d9b700 1 -- 172.21.15.33:0/770414064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad98005040 con 0x7fada80fe720 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fada5d9b700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fe720 0x7fada8114090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad90002a50 tx=0x7fad900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fad9effd700 1 -- 172.21.15.33:0/770414064 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad90007cc0 con 0x7fada80fe720 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fad9effd700 1 -- 172.21.15.33:0/770414064 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad9000f040 con 0x7fada80fe720 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fad9effd700 1 -- 172.21.15.33:0/770414064 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad90009670 con 0x7fada80fe720 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.800+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada810ddb0 con 0x7fada80fe720 2024-04-25T01:45:19.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.801+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada810e2a0 con 0x7fada80fe720 2024-04-25T01:45:19.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.802+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada8066750 con 0x7fada80fe720 2024-04-25T01:45:19.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.802+0000 7fad9effd700 1 -- 172.21.15.33:0/770414064 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fad900097d0 con 0x7fada80fe720 2024-04-25T01:45:19.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.803+0000 7fad9effd700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad9406e140 0x7fad940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:19.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.803+0000 7fad9effd700 1 -- 172.21.15.33:0/770414064 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fad9008d8d0 con 0x7fada80fe720 2024-04-25T01:45:19.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.803+0000 7fada659c700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad9406e140 0x7fad940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:19.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.804+0000 7fada659c700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad9406e140 0x7fad940705f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad98004f90 tx=0x7fad98009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:19.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:19.808+0000 7fad9effd700 1 -- 172.21.15.33:0/770414064 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fad9005a780 con 0x7fada80fe720 2024-04-25T01:45:20.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.074+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fada805f5c0 con 0x7fada80fe720 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.074+0000 7fad9effd700 1 -- 172.21.15.33:0/770414064 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1697 (secure 0 0 0) 0x7fad9005a310 con 0x7fada80fe720 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:e23 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:45:20.068 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:45:20.069 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34378} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2956426668,v1:172.21.15.33:6829/2956426668] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:45:20.070 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:45:20.071 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/2355068967,v1:172.21.15.138:6825/2355068967] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:45:20.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.076+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad9406e140 msgr2=0x7fad940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.076+0000 7fadac870700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad9406e140 0x7fad940705f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad98004f90 tx=0x7fad98009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:20.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.076+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fe720 msgr2=0x7fada8114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.076+0000 7fadac870700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fe720 0x7fada8114090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fad90002a50 tx=0x7fad900046b0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.077+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 shutdown_connections 2024-04-25T01:45:20.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.077+0000 7fadac870700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad9406e140 0x7fad940705f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.077+0000 7fadac870700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fada80fdd80 0x7fada8113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.077+0000 7fadac870700 1 --2- 172.21.15.33:0/770414064 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fada80fe720 0x7fada8114090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.077+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 >> 172.21.15.33:0/770414064 conn(0x7fada80f7830 msgr2=0x7fada80f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:20.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.077+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 shutdown_connections 2024-04-25T01:45:20.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.077+0000 7fadac870700 1 -- 172.21.15.33:0/770414064 wait complete. 2024-04-25T01:45:20.072 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 23 2024-04-25T01:45:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.188+0000 7fdad8297700 1 -- 172.21.15.33:0/2773104967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0101cd0 msgr2=0x7fdad01020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.188+0000 7fdad8297700 1 --2- 172.21.15.33:0/2773104967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0101cd0 0x7fdad01020a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fdacc004940 tx=0x7fdacc009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.189+0000 7fdad8297700 1 -- 172.21.15.33:0/2773104967 shutdown_connections 2024-04-25T01:45:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.189+0000 7fdad8297700 1 --2- 172.21.15.33:0/2773104967 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad0102670 0x7fdad00fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.189+0000 7fdad8297700 1 --2- 172.21.15.33:0/2773104967 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0101cd0 0x7fdad01020a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.189+0000 7fdad8297700 1 -- 172.21.15.33:0/2773104967 >> 172.21.15.33:0/2773104967 conn(0x7fdad0071460 msgr2=0x7fdad0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.189+0000 7fdad8297700 1 -- 172.21.15.33:0/2773104967 shutdown_connections 2024-04-25T01:45:20.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.189+0000 7fdad8297700 1 -- 172.21.15.33:0/2773104967 wait complete. 2024-04-25T01:45:20.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.189+0000 7fdad8297700 1 Processor -- start 2024-04-25T01:45:20.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.189+0000 7fdad8297700 1 -- start start 2024-04-25T01:45:20.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad8297700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad0101cd0 0x7fdad0113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:20.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad8297700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0102670 0x7fdad01140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:20.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad8297700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad0114710 con 0x7fdad0102670 2024-04-25T01:45:20.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad8297700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad010db00 con 0x7fdad0101cd0 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad5832700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0102670 0x7fdad01140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad5832700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0102670 0x7fdad01140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55358/0 (socket says 172.21.15.33:55358) 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad5832700 1 -- 172.21.15.33:0/3702805093 learned_addr learned my addr 172.21.15.33:0/3702805093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad6033700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad0101cd0 0x7fdad0113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad5832700 1 -- 172.21.15.33:0/3702805093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad0101cd0 msgr2=0x7fdad0113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad5832700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad0101cd0 0x7fdad0113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad5832700 1 -- 172.21.15.33:0/3702805093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc005040 con 0x7fdad0102670 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad6033700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad0101cd0 0x7fdad0113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad5832700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0102670 0x7fdad01140c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c860 tx=0x7fdac00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdac6ffd700 1 -- 172.21.15.33:0/3702805093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdac000a7a0 con 0x7fdad0102670 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdac6ffd700 1 -- 172.21.15.33:0/3702805093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdac000add0 con 0x7fdad0102670 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.190+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad010dde0 con 0x7fdad0102670 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.191+0000 7fdac6ffd700 1 -- 172.21.15.33:0/3702805093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdac00113a0 con 0x7fdad0102670 2024-04-25T01:45:20.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.191+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad010e3a0 con 0x7fdad0102670 2024-04-25T01:45:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.192+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad0065200 con 0x7fdad0102670 2024-04-25T01:45:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.192+0000 7fdac6ffd700 1 -- 172.21.15.33:0/3702805093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdac0012400 con 0x7fdad0102670 2024-04-25T01:45:20.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.193+0000 7fdac6ffd700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdabc06e140 0x7fdabc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:20.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.193+0000 7fdac6ffd700 1 -- 172.21.15.33:0/3702805093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdac008daa0 con 0x7fdad0102670 2024-04-25T01:45:20.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.193+0000 7fdad6033700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdabc06e140 0x7fdabc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:20.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.193+0000 7fdad6033700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdabc06e140 0x7fdabc0705f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdacc004560 tx=0x7fdacc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:20.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.197+0000 7fdac6ffd700 1 -- 172.21.15.33:0/3702805093 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdac005aa70 con 0x7fdad0102670 2024-04-25T01:45:20.343 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:20 smithi033 ceph-mon[111306]: pgmap v175: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:20.343 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:20 smithi033 ceph-mon[111306]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:20.343 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:20 smithi033 ceph-mon[111306]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:20.343 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:20 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2689857073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:20.343 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:20 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/770414064' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:45:20.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.436+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdad0109f10 con 0x7fdabc06e140 2024-04-25T01:45:20.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.437+0000 7fdac6ffd700 1 -- 172.21.15.33:0/3702805093 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdad0109f10 con 0x7fdabc06e140 2024-04-25T01:45:20.430 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:45:20.430 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:45:20.430 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "12/21 daemons upgraded", 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:45:20.431 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:45:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdabc06e140 msgr2=0x7fdabc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdabc06e140 0x7fdabc0705f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdacc004560 tx=0x7fdacc009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0102670 msgr2=0x7fdad01140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0102670 0x7fdad01140c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdac000c860 tx=0x7fdac00044c0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 shutdown_connections 2024-04-25T01:45:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdabc06e140 0x7fdabc0705f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdad0101cd0 0x7fdad0113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 --2- 172.21.15.33:0/3702805093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdad0102670 0x7fdad01140c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 >> 172.21.15.33:0/3702805093 conn(0x7fdad0071460 msgr2=0x7fdad00fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 shutdown_connections 2024-04-25T01:45:20.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.440+0000 7fdad8297700 1 -- 172.21.15.33:0/3702805093 wait complete. 2024-04-25T01:45:20.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:20 smithi138 ceph-mon[81304]: pgmap v175: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:20.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:20 smithi138 ceph-mon[81304]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:20.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:20 smithi138 ceph-mon[81304]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:20.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:20 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2689857073' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:20.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:20 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/770414064' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:45:20.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.548+0000 7f16993f1700 1 -- 172.21.15.33:0/265038906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d00 msgr2=0x7f16941020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.548+0000 7f16993f1700 1 --2- 172.21.15.33:0/265038906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d00 0x7f16941020d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f167c004940 tx=0x7f167c009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:20.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.548+0000 7f16993f1700 1 -- 172.21.15.33:0/265038906 shutdown_connections 2024-04-25T01:45:20.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.548+0000 7f16993f1700 1 --2- 172.21.15.33:0/265038906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16941026a0 0x7f16940fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.548+0000 7f16993f1700 1 --2- 172.21.15.33:0/265038906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d00 0x7f16941020d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.548+0000 7f16993f1700 1 -- 172.21.15.33:0/265038906 >> 172.21.15.33:0/265038906 conn(0x7f16940f7870 msgr2=0x7f16940f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:20.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.548+0000 7f16993f1700 1 -- 172.21.15.33:0/265038906 shutdown_connections 2024-04-25T01:45:20.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.548+0000 7f16993f1700 1 -- 172.21.15.33:0/265038906 wait complete. 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f16993f1700 1 Processor -- start 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f16993f1700 1 -- start start 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f16993f1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1694101d00 0x7f1694115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f16993f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16941026a0 0x7f16941160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f16993f1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1694116730 con 0x7f16941026a0 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f16993f1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169410fd10 con 0x7f1694101d00 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f16927fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16941026a0 0x7f16941160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f16927fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16941026a0 0x7f16941160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55396/0 (socket says 172.21.15.33:55396) 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f16927fc700 1 -- 172.21.15.33:0/2447014541 learned_addr learned my addr 172.21.15.33:0/2447014541 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.549+0000 7f1692ffd700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1694101d00 0x7f1694115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.550+0000 7f16927fc700 1 -- 172.21.15.33:0/2447014541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1694101d00 msgr2=0x7f1694115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.550+0000 7f16927fc700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1694101d00 0x7f1694115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.550+0000 7f16927fc700 1 -- 172.21.15.33:0/2447014541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c005040 con 0x7f16941026a0 2024-04-25T01:45:20.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.550+0000 7f16927fc700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16941026a0 0x7f16941160e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f168400c890 tx=0x7f16840044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:20.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.550+0000 7f168b7fe700 1 -- 172.21.15.33:0/2447014541 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f168400a7d0 con 0x7f16941026a0 2024-04-25T01:45:20.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.550+0000 7f168b7fe700 1 -- 172.21.15.33:0/2447014541 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f168400ae00 con 0x7f16941026a0 2024-04-25T01:45:20.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.550+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169410fff0 con 0x7f16941026a0 2024-04-25T01:45:20.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.550+0000 7f168b7fe700 1 -- 172.21.15.33:0/2447014541 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16840113a0 con 0x7f16941026a0 2024-04-25T01:45:20.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.550+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16941105b0 con 0x7f16941026a0 2024-04-25T01:45:20.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.551+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16940fd450 con 0x7f16941026a0 2024-04-25T01:45:20.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.552+0000 7f168b7fe700 1 -- 172.21.15.33:0/2447014541 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1684012400 con 0x7f16941026a0 2024-04-25T01:45:20.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.552+0000 7f168b7fe700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f168006e140 0x7f16800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:20.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.552+0000 7f168b7fe700 1 -- 172.21.15.33:0/2447014541 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f168408dc00 con 0x7f16941026a0 2024-04-25T01:45:20.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.552+0000 7f1692ffd700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f168006e140 0x7f16800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:20.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.553+0000 7f1692ffd700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f168006e140 0x7f16800705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f167c00b020 tx=0x7f167c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:20.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.557+0000 7f168b7fe700 1 -- 172.21.15.33:0/2447014541 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1684093020 con 0x7f16941026a0 2024-04-25T01:45:20.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.856+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1694108e00 con 0x7f16941026a0 2024-04-25T01:45:20.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.856+0000 7f168b7fe700 1 -- 172.21.15.33:0/2447014541 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f168405ab30 con 0x7f16941026a0 2024-04-25T01:45:20.849 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:45:20.849 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:45:20.849 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:45:20.850 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:45:20.850 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:45:20.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.858+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f168006e140 msgr2=0x7f16800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.858+0000 7f16993f1700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f168006e140 0x7f16800705f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f167c00b020 tx=0x7f167c009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.858+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16941026a0 msgr2=0x7f16941160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.858+0000 7f16993f1700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16941026a0 0x7f16941160e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f168400c890 tx=0x7f16840044c0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.858+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 shutdown_connections 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.859+0000 7f16993f1700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f168006e140 0x7f16800705f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.859+0000 7f16993f1700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1694101d00 0x7f1694115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.859+0000 7f16993f1700 1 --2- 172.21.15.33:0/2447014541 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16941026a0 0x7f16941160e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.859+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 >> 172.21.15.33:0/2447014541 conn(0x7f16940f7870 msgr2=0x7f16940f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.859+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 shutdown_connections 2024-04-25T01:45:20.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:20.859+0000 7f16993f1700 1 -- 172.21.15.33:0/2447014541 wait complete. 2024-04-25T01:45:21.392 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:21 smithi033 ceph-mon[111306]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:21.392 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:21 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:21.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:21 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:21.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:21 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:21.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:21 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:21.393 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:21 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2447014541' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:45:21.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:21 smithi138 ceph-mon[81304]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:21.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:21 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:21.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:21 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:21.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:21 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:21.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:21 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:21.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:21 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2447014541' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:45:22.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:22 smithi033 ceph-mon[111306]: pgmap v176: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:22.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:22 smithi033 ceph-mon[111306]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:22.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:22 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:boot 2024-04-25T01:45:22.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:22 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T01:45:22.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:22 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:45:22.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:22 smithi138 ceph-mon[81304]: pgmap v176: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:22.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:22 smithi138 ceph-mon[81304]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:22.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:22 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:boot 2024-04-25T01:45:22.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:22 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T01:45:22.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:22 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T01:45:24.385 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:24 smithi033 ceph-mon[111306]: pgmap v177: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:24.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:24 smithi138 ceph-mon[81304]: pgmap v177: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:26.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:26 smithi033 ceph-mon[111306]: pgmap v178: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:26.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:26.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:45:26.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:26.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:26.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:26 smithi138 ceph-mon[81304]: pgmap v178: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:26.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:26.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:45:26.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:26.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:28.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:27 smithi033 ceph-mon[111306]: pgmap v179: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:28.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:27 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:28.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:27 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:28.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:27 smithi138 ceph-mon[81304]: pgmap v179: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:28.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:27 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:28.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:27 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:29.955 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:29 smithi033 ceph-mon[111306]: pgmap v180: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:30.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:29 smithi138 ceph-mon[81304]: pgmap v180: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:31.401 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: pgmap v181: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:31.401 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:31.401 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:31.401 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:31.402 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:31.402 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:31.402 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:31.402 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:31.402 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:31.402 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:31.402 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:31 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:31.784 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: pgmap v181: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:31 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:33.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:33 smithi033 ceph-mon[111306]: Upgrade: Updating mds.cephfs.smithi033.hobkqr 2024-04-25T01:45:33.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:33 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:33.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:33 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hobkqr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:45:33.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:33 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:33.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:33 smithi033 ceph-mon[111306]: Deploying daemon mds.cephfs.smithi033.hobkqr on smithi033 2024-04-25T01:45:33.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:33 smithi138 ceph-mon[81304]: Upgrade: Updating mds.cephfs.smithi033.hobkqr 2024-04-25T01:45:33.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:33 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:33.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:33 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi033.hobkqr", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:45:33.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:33 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:33.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:33 smithi138 ceph-mon[81304]: Deploying daemon mds.cephfs.smithi033.hobkqr on smithi033 2024-04-25T01:45:34.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:34 smithi033 ceph-mon[111306]: pgmap v182: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:34.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:34 smithi138 ceph-mon[81304]: pgmap v182: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:36.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:36 smithi033 ceph-mon[111306]: pgmap v183: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:36.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:36 smithi033 ceph-mon[111306]: osdmap e82: 6 total, 6 up, 6 in 2024-04-25T01:45:36.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:36 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-25T01:45:36.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:36 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:45:36.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:36 smithi138 ceph-mon[81304]: pgmap v183: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:36.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:36 smithi138 ceph-mon[81304]: osdmap e82: 6 total, 6 up, 6 in 2024-04-25T01:45:36.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:36 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-25T01:45:36.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:36 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:45:38.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:38 smithi138 ceph-mon[81304]: pgmap v185: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:38.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:38 smithi033 ceph-mon[111306]: pgmap v185: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:40.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:40 smithi033 ceph-mon[111306]: pgmap v186: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:40.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:40.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:40.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:40.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:40 smithi138 ceph-mon[81304]: pgmap v186: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:40.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:40.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:40.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:41.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:41 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:boot 2024-04-25T01:45:41.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:41 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T01:45:41.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:45:41.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:45:41.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:41 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:boot 2024-04-25T01:45:41.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:41 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T01:45:41.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T01:45:41.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:45:42.435 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:42 smithi033 ceph-mon[111306]: pgmap v187: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:42.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:42 smithi138 ceph-mon[81304]: pgmap v187: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:44.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:44 smithi033 ceph-mon[111306]: pgmap v188: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:44.534 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:44 smithi138 ceph-mon[81304]: pgmap v188: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:45.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:45 smithi033 ceph-mon[111306]: pgmap v189: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:45.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:45 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:45.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:45 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:45.784 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:45 smithi138 ceph-mon[81304]: pgmap v189: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:45.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:45 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:45.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:45 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:46.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:46 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:46.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:46 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:47.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:46 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:47.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:46 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:47.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:47 smithi033 ceph-mon[111306]: pgmap v190: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:48.034 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:47 smithi138 ceph-mon[81304]: pgmap v190: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: pgmap v191: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:49.907 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:49 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:50.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: pgmap v191: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:50.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:50.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:50.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:50.010 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T01:45:50.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:50.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:50.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:50.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:50.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:50.011 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:49 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.981+0000 7f9519ae6700 1 -- 172.21.15.33:0/3298194924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 msgr2=0x7f9514106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.981+0000 7f9519ae6700 1 --2- 172.21.15.33:0/3298194924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 0x7f9514106df0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9508004940 tx=0x7f9508009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.981+0000 7f9519ae6700 1 -- 172.21.15.33:0/3298194924 shutdown_connections 2024-04-25T01:45:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.981+0000 7f9519ae6700 1 --2- 172.21.15.33:0/3298194924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 0x7f9514106df0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.981+0000 7f9519ae6700 1 --2- 172.21.15.33:0/3298194924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95140fde10 0x7f95140fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.981+0000 7f9519ae6700 1 -- 172.21.15.33:0/3298194924 >> 172.21.15.33:0/3298194924 conn(0x7f95140f7860 msgr2=0x7f95140f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:50.977 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.981+0000 7f9519ae6700 1 -- 172.21.15.33:0/3298194924 shutdown_connections 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.981+0000 7f9519ae6700 1 -- 172.21.15.33:0/3298194924 wait complete. 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f9519ae6700 1 Processor -- start 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f9519ae6700 1 -- start start 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f9519ae6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95140fde10 0x7f9514113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f9519ae6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 0x7f9514113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f9519ae6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95141144b0 con 0x7f95140fe7b0 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f9519ae6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951410dab0 con 0x7f95140fde10 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f9512ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 0x7f9514113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f9512ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 0x7f9514113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56460/0 (socket says 172.21.15.33:56460) 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f9512ffd700 1 -- 172.21.15.33:0/2675506519 learned_addr learned my addr 172.21.15.33:0/2675506519 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.982+0000 7f95137fe700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95140fde10 0x7f9514113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.983+0000 7f9512ffd700 1 -- 172.21.15.33:0/2675506519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95140fde10 msgr2=0x7f9514113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:50.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.983+0000 7f9512ffd700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95140fde10 0x7f9514113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:50.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.983+0000 7f9512ffd700 1 -- 172.21.15.33:0/2675506519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9508005040 con 0x7f95140fe7b0 2024-04-25T01:45:50.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.983+0000 7f9512ffd700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 0x7f9514113e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f9508009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:50.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.983+0000 7f94fbfff700 1 -- 172.21.15.33:0/2675506519 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9508021070 con 0x7f95140fe7b0 2024-04-25T01:45:50.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.983+0000 7f94fbfff700 1 -- 172.21.15.33:0/2675506519 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9508013c00 con 0x7f95140fe7b0 2024-04-25T01:45:50.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.983+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951410dd30 con 0x7f95140fe7b0 2024-04-25T01:45:50.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.983+0000 7f94fbfff700 1 -- 172.21.15.33:0/2675506519 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f950800e070 con 0x7f95140fe7b0 2024-04-25T01:45:50.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.983+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951410e270 con 0x7f95140fe7b0 2024-04-25T01:45:50.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.984+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9514066b50 con 0x7f95140fe7b0 2024-04-25T01:45:50.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.985+0000 7f94fbfff700 1 -- 172.21.15.33:0/2675506519 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f950801c400 con 0x7f95140fe7b0 2024-04-25T01:45:50.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.985+0000 7f94fbfff700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94fc06e140 0x7f94fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:50.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.986+0000 7f94fbfff700 1 -- 172.21.15.33:0/2675506519 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f950808efb0 con 0x7f95140fe7b0 2024-04-25T01:45:50.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.986+0000 7f95137fe700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94fc06e140 0x7f94fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:50.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.986+0000 7f95137fe700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94fc06e140 0x7f94fc0705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9504009b80 tx=0x7f950400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:50.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:50.993+0000 7f94fbfff700 1 -- 172.21.15.33:0/2675506519 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f950805be40 con 0x7f95140fe7b0 2024-04-25T01:45:51.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.221+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9514109f10 con 0x7f94fc06e140 2024-04-25T01:45:51.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.223+0000 7f94fbfff700 1 -- 172.21.15.33:0/2675506519 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9514109f10 con 0x7f94fc06e140 2024-04-25T01:45:51.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94fc06e140 msgr2=0x7f94fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94fc06e140 0x7f94fc0705f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9504009b80 tx=0x7f950400b040 comp rx=0 tx=0).stop 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 msgr2=0x7f9514113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 0x7f9514113e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9508000c00 tx=0x7f9508009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 shutdown_connections 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94fc06e140 0x7f94fc0705f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95140fde10 0x7f9514113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 --2- 172.21.15.33:0/2675506519 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95140fe7b0 0x7f9514113e60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 >> 172.21.15.33:0/2675506519 conn(0x7f95140f7860 msgr2=0x7f95140f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 shutdown_connections 2024-04-25T01:45:51.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.225+0000 7f9519ae6700 1 -- 172.21.15.33:0/2675506519 wait complete. 2024-04-25T01:45:51.229 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:45:51.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.330+0000 7f128bbdd700 1 -- 172.21.15.33:0/2965429099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 msgr2=0x7f12841020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.330+0000 7f128bbdd700 1 --2- 172.21.15.33:0/2965429099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 0x7f12841020d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1280004940 tx=0x7f1280009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.330+0000 7f128bbdd700 1 -- 172.21.15.33:0/2965429099 shutdown_connections 2024-04-25T01:45:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.330+0000 7f128bbdd700 1 --2- 172.21.15.33:0/2965429099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12841026a0 0x7f12840fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.330+0000 7f128bbdd700 1 --2- 172.21.15.33:0/2965429099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 0x7f12841020d0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.330+0000 7f128bbdd700 1 -- 172.21.15.33:0/2965429099 >> 172.21.15.33:0/2965429099 conn(0x7f12840f7870 msgr2=0x7f12840f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.331+0000 7f128bbdd700 1 -- 172.21.15.33:0/2965429099 shutdown_connections 2024-04-25T01:45:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.331+0000 7f128bbdd700 1 -- 172.21.15.33:0/2965429099 wait complete. 2024-04-25T01:45:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.331+0000 7f128bbdd700 1 Processor -- start 2024-04-25T01:45:51.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.331+0000 7f128bbdd700 1 -- start start 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.331+0000 7f128bbdd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 0x7f1284113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.331+0000 7f128bbdd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12841026a0 0x7f1284113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.331+0000 7f128bbdd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12841144a0 con 0x7f1284101d00 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.331+0000 7f128bbdd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128410daa0 con 0x7f12841026a0 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f1289979700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 0x7f1284113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f1289979700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 0x7f1284113910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56474/0 (socket says 172.21.15.33:56474) 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f1289979700 1 -- 172.21.15.33:0/4199293581 learned_addr learned my addr 172.21.15.33:0/4199293581 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f1289178700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12841026a0 0x7f1284113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f1289979700 1 -- 172.21.15.33:0/4199293581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12841026a0 msgr2=0x7f1284113e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f1289979700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12841026a0 0x7f1284113e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f1289979700 1 -- 172.21.15.33:0/4199293581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1280005040 con 0x7f1284101d00 2024-04-25T01:45:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f1289979700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 0x7f1284113910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f128000b8b0 tx=0x7f1280009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f127a7fc700 1 -- 172.21.15.33:0/4199293581 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f128000e040 con 0x7f1284101d00 2024-04-25T01:45:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128410dd20 con 0x7f1284101d00 2024-04-25T01:45:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f127a7fc700 1 -- 172.21.15.33:0/4199293581 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1280004290 con 0x7f1284101d00 2024-04-25T01:45:51.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f127a7fc700 1 -- 172.21.15.33:0/4199293581 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f128001c3f0 con 0x7f1284101d00 2024-04-25T01:45:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.332+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128410e2e0 con 0x7f1284101d00 2024-04-25T01:45:51.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.333+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12840fd450 con 0x7f1284101d00 2024-04-25T01:45:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.334+0000 7f127a7fc700 1 -- 172.21.15.33:0/4199293581 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1280013420 con 0x7f1284101d00 2024-04-25T01:45:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.334+0000 7f127a7fc700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f127006e140 0x7f12700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.334+0000 7f127a7fc700 1 -- 172.21.15.33:0/4199293581 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f128008eb20 con 0x7f1284101d00 2024-04-25T01:45:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.334+0000 7f1289178700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f127006e140 0x7f12700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:51.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.335+0000 7f1289178700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f127006e140 0x7f12700705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f128410ec60 tx=0x7f127400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:51.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.339+0000 7f127a7fc700 1 -- 172.21.15.33:0/4199293581 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f128005ba50 con 0x7f1284101d00 2024-04-25T01:45:51.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.566+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1284108e00 con 0x7f127006e140 2024-04-25T01:45:51.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.567+0000 7f127a7fc700 1 -- 172.21.15.33:0/4199293581 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1284108e00 con 0x7f127006e140 2024-04-25T01:45:51.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.569+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f127006e140 msgr2=0x7f12700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.569+0000 7f128bbdd700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f127006e140 0x7f12700705f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f128410ec60 tx=0x7f127400b040 comp rx=0 tx=0).stop 2024-04-25T01:45:51.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.569+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 msgr2=0x7f1284113910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.569+0000 7f128bbdd700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 0x7f1284113910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f128000b8b0 tx=0x7f1280009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:51.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.569+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 shutdown_connections 2024-04-25T01:45:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.569+0000 7f128bbdd700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f127006e140 0x7f12700705f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.569+0000 7f128bbdd700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1284101d00 0x7f1284113910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.569+0000 7f128bbdd700 1 --2- 172.21.15.33:0/4199293581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f12841026a0 0x7f1284113e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.569+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 >> 172.21.15.33:0/4199293581 conn(0x7f12840f7870 msgr2=0x7f12840f8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.570+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 shutdown_connections 2024-04-25T01:45:51.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.570+0000 7f128bbdd700 1 -- 172.21.15.33:0/4199293581 wait complete. 2024-04-25T01:45:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 -- 172.21.15.33:0/2496585437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 msgr2=0x7f84c80fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 --2- 172.21.15.33:0/2496585437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 0x7f84c80fbd20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f84b8004940 tx=0x7f84b8009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 -- 172.21.15.33:0/2496585437 shutdown_connections 2024-04-25T01:45:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 --2- 172.21.15.33:0/2496585437 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 0x7f84c80fbd20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 --2- 172.21.15.33:0/2496585437 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c8101d10 0x7f84c81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 -- 172.21.15.33:0/2496585437 >> 172.21.15.33:0/2496585437 conn(0x7f84c80f7860 msgr2=0x7f84c80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 -- 172.21.15.33:0/2496585437 shutdown_connections 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 -- 172.21.15.33:0/2496585437 wait complete. 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 Processor -- start 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.674+0000 7f84d06a4700 1 -- start start 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84d06a4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c8101d10 0x7f84c810eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84d06a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 0x7f84c810f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84d06a4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c810f560 con 0x7f84c81026b0 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84d06a4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c810f6a0 con 0x7f84c8101d10 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84cdc3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 0x7f84c810f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84cdc3f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 0x7f84c810f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56486/0 (socket says 172.21.15.33:56486) 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84cdc3f700 1 -- 172.21.15.33:0/4151812434 learned_addr learned my addr 172.21.15.33:0/4151812434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:51.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84ce440700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c8101d10 0x7f84c810eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84cdc3f700 1 -- 172.21.15.33:0/4151812434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c8101d10 msgr2=0x7f84c810eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84cdc3f700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c8101d10 0x7f84c810eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84cdc3f700 1 -- 172.21.15.33:0/4151812434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8005040 con 0x7f84c81026b0 2024-04-25T01:45:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84cdc3f700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 0x7f84c810f020 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f84b8000c00 tx=0x7f84b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84beffd700 1 -- 172.21.15.33:0/4151812434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b8021070 con 0x7f84c81026b0 2024-04-25T01:45:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84beffd700 1 -- 172.21.15.33:0/4151812434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84b8013c00 con 0x7f84c81026b0 2024-04-25T01:45:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c810f920 con 0x7f84c81026b0 2024-04-25T01:45:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.675+0000 7f84beffd700 1 -- 172.21.15.33:0/4151812434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84b800e070 con 0x7f84c81026b0 2024-04-25T01:45:51.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.676+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c8117dc0 con 0x7f84c81026b0 2024-04-25T01:45:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.677+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c80fd460 con 0x7f84c81026b0 2024-04-25T01:45:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.677+0000 7f84beffd700 1 -- 172.21.15.33:0/4151812434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f84b801ca60 con 0x7f84c81026b0 2024-04-25T01:45:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.677+0000 7f84beffd700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84b406e140 0x7f84b40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.677+0000 7f84beffd700 1 -- 172.21.15.33:0/4151812434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f84b808eef0 con 0x7f84c81026b0 2024-04-25T01:45:51.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.678+0000 7f84ce440700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84b406e140 0x7f84b40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:51.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.678+0000 7f84ce440700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84b406e140 0x7f84b40705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009b80 tx=0x7f84c400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:51.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.682+0000 7f84beffd700 1 -- 172.21.15.33:0/4151812434 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f84b80582f0 con 0x7f84c81026b0 2024-04-25T01:45:51.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.897+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84c8108df0 con 0x7f84b406e140 2024-04-25T01:45:51.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.908+0000 7f84beffd700 1 -- 172.21.15.33:0/4151812434 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f84c8108df0 con 0x7f84b406e140 2024-04-25T01:45:51.902 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:45:51.902 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (6m) 7s ago 18m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:45:51.902 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (4m) 7s ago 18m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (4m) 47s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (6m) 7s ago 17m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (12s) 7s ago 12m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (32s) 7s ago 12m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (12m) 47s ago 12m 119M - 17.2.7-902-gc61472c8 0de407c1b3af 32ed44c64dc7 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (12m) 47s ago 12m 61.0M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (8m) 7s ago 19m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (7m) 47s ago 17m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (5m) 7s ago 19m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (4m) 47s ago 17m 39.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (7m) 7s ago 18m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (7m) 47s ago 17m 15.2M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (3m) 7s ago 16m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (3m) 7s ago 16m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:45:51.903 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (2m) 7s ago 15m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:45:51.904 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (118s) 47s ago 15m 100M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:45:51.904 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (85s) 47s ago 14m 100M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:45:51.904 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (51s) 47s ago 14m 12.6M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:45:51.904 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (7m) 7s ago 17m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84b406e140 msgr2=0x7f84b40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84b406e140 0x7f84b40705f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009b80 tx=0x7f84c400b040 comp rx=0 tx=0).stop 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 msgr2=0x7f84c810f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 0x7f84c810f020 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f84b8000c00 tx=0x7f84b8009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 shutdown_connections 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84b406e140 0x7f84b40705f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84c8101d10 0x7f84c810eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 --2- 172.21.15.33:0/4151812434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84c81026b0 0x7f84c810f020 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 >> 172.21.15.33:0/4151812434 conn(0x7f84c80f7860 msgr2=0x7f84c80f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 shutdown_connections 2024-04-25T01:45:51.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:51.911+0000 7f84d06a4700 1 -- 172.21.15.33:0/4151812434 wait complete. 2024-04-25T01:45:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.016+0000 7f3afcca9700 1 -- 172.21.15.33:0/1215711015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fe7a0 msgr2=0x7f3af8106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.016+0000 7f3afcca9700 1 --2- 172.21.15.33:0/1215711015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fe7a0 0x7f3af8106e70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3aec004940 tx=0x7f3aec009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 -- 172.21.15.33:0/1215711015 shutdown_connections 2024-04-25T01:45:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 --2- 172.21.15.33:0/1215711015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fe7a0 0x7f3af8106e70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 --2- 172.21.15.33:0/1215711015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af80fde00 0x7f3af80fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 -- 172.21.15.33:0/1215711015 >> 172.21.15.33:0/1215711015 conn(0x7f3af80f7860 msgr2=0x7f3af80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 -- 172.21.15.33:0/1215711015 shutdown_connections 2024-04-25T01:45:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 -- 172.21.15.33:0/1215711015 wait complete. 2024-04-25T01:45:52.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 Processor -- start 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 -- start start 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fde00 0x7f3af8113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af80fe7a0 0x7f3af8113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af8114490 con 0x7f3af80fde00 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.017+0000 7f3afcca9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af810da90 con 0x7f3af80fe7a0 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3af659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fde00 0x7f3af8113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3af659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fde00 0x7f3af8113900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56492/0 (socket says 172.21.15.33:56492) 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3af659c700 1 -- 172.21.15.33:0/1061614754 learned_addr learned my addr 172.21.15.33:0/1061614754 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3af5d9b700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af80fe7a0 0x7f3af8113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3af659c700 1 -- 172.21.15.33:0/1061614754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af80fe7a0 msgr2=0x7f3af8113e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3af659c700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af80fe7a0 0x7f3af8113e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3af659c700 1 -- 172.21.15.33:0/1061614754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec005040 con 0x7f3af80fde00 2024-04-25T01:45:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3af659c700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fde00 0x7f3af8113900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800eb40 tx=0x7f3ae800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3ae6ffd700 1 -- 172.21.15.33:0/1061614754 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ae8007cc0 con 0x7f3af80fde00 2024-04-25T01:45:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3ae6ffd700 1 -- 172.21.15.33:0/1061614754 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ae800f040 con 0x7f3af80fde00 2024-04-25T01:45:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af810dd10 con 0x7f3af80fde00 2024-04-25T01:45:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3ae6ffd700 1 -- 172.21.15.33:0/1061614754 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ae800ca10 con 0x7f3af80fde00 2024-04-25T01:45:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.018+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af810e250 con 0x7f3af80fde00 2024-04-25T01:45:52.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.019+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af8066ad0 con 0x7f3af80fde00 2024-04-25T01:45:52.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.020+0000 7f3ae6ffd700 1 -- 172.21.15.33:0/1061614754 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3ae800cc80 con 0x7f3af80fde00 2024-04-25T01:45:52.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.020+0000 7f3ae6ffd700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ae006e140 0x7f3ae00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:52.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.020+0000 7f3ae6ffd700 1 -- 172.21.15.33:0/1061614754 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3ae808d6f0 con 0x7f3af80fde00 2024-04-25T01:45:52.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.020+0000 7f3af5d9b700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ae006e140 0x7f3ae00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:52.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.021+0000 7f3af5d9b700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ae006e140 0x7f3ae00705f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3af810ec50 tx=0x7f3aec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:52.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.025+0000 7f3ae6ffd700 1 -- 172.21.15.33:0/1061614754 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3ae805a650 con 0x7f3af80fde00 2024-04-25T01:45:52.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:51 smithi138 ceph-mon[81304]: pgmap v192: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:52.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:51 smithi138 ceph-mon[81304]: Upgrade: Updating mds.cephfs.smithi138.jwzxlz 2024-04-25T01:45:52.035 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:51 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:52.036 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:51 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.jwzxlz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:45:52.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:51 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:52.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:51 smithi138 ceph-mon[81304]: Deploying daemon mds.cephfs.smithi138.jwzxlz on smithi138 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:51 smithi033 ceph-mon[111306]: pgmap v192: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:51 smithi033 ceph-mon[111306]: Upgrade: Updating mds.cephfs.smithi138.jwzxlz 2024-04-25T01:45:52.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:51 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:52.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:51 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.jwzxlz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T01:45:52.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:51 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T01:45:52.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:51 smithi033 ceph-mon[111306]: Deploying daemon mds.cephfs.smithi138.jwzxlz on smithi138 2024-04-25T01:45:52.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.322+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3af8109ef0 con 0x7f3af80fde00 2024-04-25T01:45:52.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.323+0000 7f3ae6ffd700 1 -- 172.21.15.33:0/1061614754 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f3ae805a1e0 con 0x7f3af80fde00 2024-04-25T01:45:52.316 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:45:52.316 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-25T01:45:52.317 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:45:52.318 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:45:52.318 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:45:52.318 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-25T01:45:52.318 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 12 2024-04-25T01:45:52.318 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:45:52.318 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:45:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ae006e140 msgr2=0x7f3ae00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ae006e140 0x7f3ae00705f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3af810ec50 tx=0x7f3aec009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fde00 msgr2=0x7f3af8113900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fde00 0x7f3af8113900 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800eb40 tx=0x7f3ae800b040 comp rx=0 tx=0).stop 2024-04-25T01:45:52.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 shutdown_connections 2024-04-25T01:45:52.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ae006e140 0x7f3ae00705f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3af80fde00 0x7f3af8113900 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 --2- 172.21.15.33:0/1061614754 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3af80fe7a0 0x7f3af8113e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 >> 172.21.15.33:0/1061614754 conn(0x7f3af80f7860 msgr2=0x7f3af80f8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:52.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 shutdown_connections 2024-04-25T01:45:52.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.325+0000 7f3afcca9700 1 -- 172.21.15.33:0/1061614754 wait complete. 2024-04-25T01:45:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.430+0000 7f2874568700 1 -- 172.21.15.33:0/1815521024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c0fbd40 msgr2=0x7f286c0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.430+0000 7f2874568700 1 --2- 172.21.15.33:0/1815521024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c0fbd40 0x7f286c0fc1b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f285c004940 tx=0x7f285c009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.430+0000 7f2874568700 1 -- 172.21.15.33:0/1815521024 shutdown_connections 2024-04-25T01:45:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.430+0000 7f2874568700 1 --2- 172.21.15.33:0/1815521024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c0fbd40 0x7f286c0fc1b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.430+0000 7f2874568700 1 --2- 172.21.15.33:0/1815521024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f286c101d90 0x7f286c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.430+0000 7f2874568700 1 -- 172.21.15.33:0/1815521024 >> 172.21.15.33:0/1815521024 conn(0x7f286c071d00 msgr2=0x7f286c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.430+0000 7f2874568700 1 -- 172.21.15.33:0/1815521024 shutdown_connections 2024-04-25T01:45:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.430+0000 7f2874568700 1 -- 172.21.15.33:0/1815521024 wait complete. 2024-04-25T01:45:52.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2874568700 1 Processor -- start 2024-04-25T01:45:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2874568700 1 -- start start 2024-04-25T01:45:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2874568700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f286c0fbd40 0x7f286c102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2874568700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c101d90 0x7f286c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2874568700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c1035f0 con 0x7f286c101d90 2024-04-25T01:45:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2874568700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286c103730 con 0x7f286c0fbd40 2024-04-25T01:45:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2871b03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c101d90 0x7f286c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2871b03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c101d90 0x7f286c103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56510/0 (socket says 172.21.15.33:56510) 2024-04-25T01:45:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2871b03700 1 -- 172.21.15.33:0/2152077969 learned_addr learned my addr 172.21.15.33:0/2152077969 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:52.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.431+0000 7f2872304700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f286c0fbd40 0x7f286c102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.432+0000 7f2871b03700 1 -- 172.21.15.33:0/2152077969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f286c0fbd40 msgr2=0x7f286c102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.432+0000 7f2871b03700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f286c0fbd40 0x7f286c102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.432+0000 7f2871b03700 1 -- 172.21.15.33:0/2152077969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c005040 con 0x7f286c101d90 2024-04-25T01:45:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.432+0000 7f2871b03700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c101d90 0x7f286c103060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f285c00b8b0 tx=0x7f285c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.432+0000 7f2862ffd700 1 -- 172.21.15.33:0/2152077969 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f285c00e040 con 0x7f286c101d90 2024-04-25T01:45:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.432+0000 7f2862ffd700 1 -- 172.21.15.33:0/2152077969 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f285c013420 con 0x7f286c101d90 2024-04-25T01:45:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.432+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f286c103980 con 0x7f286c101d90 2024-04-25T01:45:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.432+0000 7f2862ffd700 1 -- 172.21.15.33:0/2152077969 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f285c01c630 con 0x7f286c101d90 2024-04-25T01:45:52.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.432+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286c11a2e0 con 0x7f286c101d90 2024-04-25T01:45:52.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.433+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286c0fd430 con 0x7f286c101d90 2024-04-25T01:45:52.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.434+0000 7f2862ffd700 1 -- 172.21.15.33:0/2152077969 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f285c013a00 con 0x7f286c101d90 2024-04-25T01:45:52.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.434+0000 7f2862ffd700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f285806e140 0x7f28580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:52.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.434+0000 7f2862ffd700 1 -- 172.21.15.33:0/2152077969 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f285c08ecb0 con 0x7f286c101d90 2024-04-25T01:45:52.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.434+0000 7f2872304700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f285806e140 0x7f28580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:52.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.435+0000 7f2872304700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f285806e140 0x7f28580705f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2868007fd0 tx=0x7f2868009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:52.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.439+0000 7f2862ffd700 1 -- 172.21.15.33:0/2152077969 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f285c058150 con 0x7f286c101d90 2024-04-25T01:45:52.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.692+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f286c119eb0 con 0x7f286c101d90 2024-04-25T01:45:52.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.692+0000 7f2862ffd700 1 -- 172.21.15.33:0/2152077969 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f286c119eb0 con 0x7f286c101d90 2024-04-25T01:45:52.686 INFO:teuthology.orchestra.run.smithi033.stdout:e26 2024-04-25T01:45:52.686 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:45:52.686 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:45:52.687 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:45:52.688 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:45:52.689 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44275} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/2355068967,v1:172.21.15.138:6825/2355068967] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:45:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.694+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f285806e140 msgr2=0x7f28580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.694+0000 7f2874568700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f285806e140 0x7f28580705f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2868007fd0 tx=0x7f2868009040 comp rx=0 tx=0).stop 2024-04-25T01:45:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.694+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c101d90 msgr2=0x7f286c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.694+0000 7f2874568700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c101d90 0x7f286c103060 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f285c00b8b0 tx=0x7f285c009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.695+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 shutdown_connections 2024-04-25T01:45:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.695+0000 7f2874568700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f285806e140 0x7f28580705f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.695+0000 7f2874568700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f286c0fbd40 0x7f286c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.695+0000 7f2874568700 1 --2- 172.21.15.33:0/2152077969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f286c101d90 0x7f286c103060 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.695+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 >> 172.21.15.33:0/2152077969 conn(0x7f286c071d00 msgr2=0x7f286c0655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.695+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 shutdown_connections 2024-04-25T01:45:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.695+0000 7f2874568700 1 -- 172.21.15.33:0/2152077969 wait complete. 2024-04-25T01:45:52.691 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 26 2024-04-25T01:45:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.800+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/1862654324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a40fb520 msgr2=0x7fa9a40fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.800+0000 7fa9a8dfc700 1 --2- 172.21.15.33:0/1862654324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a40fb520 0x7fa9a40fb990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa98c004940 tx=0x7fa98c009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.800+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/1862654324 shutdown_connections 2024-04-25T01:45:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.800+0000 7fa9a8dfc700 1 --2- 172.21.15.33:0/1862654324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a40fb520 0x7fa9a40fb990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.800+0000 7fa9a8dfc700 1 --2- 172.21.15.33:0/1862654324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9a4102f60 0x7fa9a4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.800+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/1862654324 >> 172.21.15.33:0/1862654324 conn(0x7fa9a40714d0 msgr2=0x7fa9a40718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.801+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/1862654324 shutdown_connections 2024-04-25T01:45:52.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.801+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/1862654324 wait complete. 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.801+0000 7fa9a8dfc700 1 Processor -- start 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.801+0000 7fa9a8dfc700 1 -- start start 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.801+0000 7fa9a8dfc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9a40fb520 0x7fa9a4113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.801+0000 7fa9a8dfc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a4102f60 0x7fa9a4113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.801+0000 7fa9a8dfc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a41144f0 con 0x7fa9a4102f60 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.801+0000 7fa9a8dfc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a410daf0 con 0x7fa9a40fb520 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a4102f60 0x7fa9a4113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a4102f60 0x7fa9a4113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56522/0 (socket says 172.21.15.33:56522) 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a1d9b700 1 -- 172.21.15.33:0/3416371524 learned_addr learned my addr 172.21.15.33:0/3416371524 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a259c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9a40fb520 0x7fa9a4113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a1d9b700 1 -- 172.21.15.33:0/3416371524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9a40fb520 msgr2=0x7fa9a4113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:52.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a1d9b700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9a40fb520 0x7fa9a4113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a1d9b700 1 -- 172.21.15.33:0/3416371524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c005040 con 0x7fa9a4102f60 2024-04-25T01:45:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a1d9b700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a4102f60 0x7fa9a4113ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00b8b0 tx=0x7fa98c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa99affd700 1 -- 172.21.15.33:0/3416371524 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa98c00e040 con 0x7fa9a4102f60 2024-04-25T01:45:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a410dd70 con 0x7fa9a4102f60 2024-04-25T01:45:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa99affd700 1 -- 172.21.15.33:0/3416371524 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa98c004290 con 0x7fa9a4102f60 2024-04-25T01:45:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa99affd700 1 -- 172.21.15.33:0/3416371524 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa98c01c3f0 con 0x7fa9a4102f60 2024-04-25T01:45:52.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.802+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a410e2b0 con 0x7fa9a4102f60 2024-04-25T01:45:52.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.803+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a4100250 con 0x7fa9a4102f60 2024-04-25T01:45:52.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.804+0000 7fa99affd700 1 -- 172.21.15.33:0/3416371524 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa98c013420 con 0x7fa9a4102f60 2024-04-25T01:45:52.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.804+0000 7fa99affd700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa99006e140 0x7fa9900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:52.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.804+0000 7fa99affd700 1 -- 172.21.15.33:0/3416371524 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa98c08ed20 con 0x7fa9a4102f60 2024-04-25T01:45:52.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.804+0000 7fa9a259c700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa99006e140 0x7fa9900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:52.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.805+0000 7fa9a259c700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa99006e140 0x7fa9900705f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa994009fd0 tx=0x7fa99400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:52.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:52.809+0000 7fa99affd700 1 -- 172.21.15.33:0/3416371524 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa98c05bc50 con 0x7fa9a4102f60 2024-04-25T01:45:53.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.039+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9a410efb0 con 0x7fa99006e140 2024-04-25T01:45:53.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.040+0000 7fa99affd700 1 -- 172.21.15.33:0/3416371524 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa9a410efb0 con 0x7fa99006e140 2024-04-25T01:45:53.034 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:45:53.034 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:45:53.034 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:45:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:45:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:45:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:45:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:45:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:45:53.035 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:45:53.036 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:45:53.036 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "14/21 daemons upgraded", 2024-04-25T01:45:53.036 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:45:53.036 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:45:53.036 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:45:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa99006e140 msgr2=0x7fa9900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa99006e140 0x7fa9900705f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa994009fd0 tx=0x7fa99400b040 comp rx=0 tx=0).stop 2024-04-25T01:45:53.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a4102f60 msgr2=0x7fa9a4113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a4102f60 0x7fa9a4113ea0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00b8b0 tx=0x7fa98c009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 shutdown_connections 2024-04-25T01:45:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa99006e140 0x7fa9900705f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9a40fb520 0x7fa9a4113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 --2- 172.21.15.33:0/3416371524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9a4102f60 0x7fa9a4113ea0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 >> 172.21.15.33:0/3416371524 conn(0x7fa9a40714d0 msgr2=0x7fa9a40fa7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 shutdown_connections 2024-04-25T01:45:53.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.043+0000 7fa9a8dfc700 1 -- 172.21.15.33:0/3416371524 wait complete. 2024-04-25T01:45:53.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.148+0000 7f3fca863700 1 -- 172.21.15.33:0/3062324059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fde40 msgr2=0x7f3fc40fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:53.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.148+0000 7f3fca863700 1 --2- 172.21.15.33:0/3062324059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fde40 0x7f3fc40fe210 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4004940 tx=0x7f3fb4009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:53.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.148+0000 7f3fca863700 1 -- 172.21.15.33:0/3062324059 shutdown_connections 2024-04-25T01:45:53.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.148+0000 7f3fca863700 1 --2- 172.21.15.33:0/3062324059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc40fe7e0 0x7f3fc4106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.148+0000 7f3fca863700 1 --2- 172.21.15.33:0/3062324059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fde40 0x7f3fc40fe210 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.148+0000 7f3fca863700 1 -- 172.21.15.33:0/3062324059 >> 172.21.15.33:0/3062324059 conn(0x7f3fc40f7860 msgr2=0x7f3fc40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:53.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.148+0000 7f3fca863700 1 -- 172.21.15.33:0/3062324059 shutdown_connections 2024-04-25T01:45:53.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.148+0000 7f3fca863700 1 -- 172.21.15.33:0/3062324059 wait complete. 2024-04-25T01:45:53.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fca863700 1 Processor -- start 2024-04-25T01:45:53.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fca863700 1 -- start start 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fca863700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc40fde40 0x7f3fc4113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fca863700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fe7e0 0x7f3fc4113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fca863700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc4114490 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fca863700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc410dad0 con 0x7f3fc40fde40 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fc37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fe7e0 0x7f3fc4113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fc37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fe7e0 0x7f3fc4113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56542/0 (socket says 172.21.15.33:56542) 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fc3fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc40fde40 0x7f3fc4113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.149+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3113797408 learned_addr learned my addr 172.21.15.33:0/3113797408 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.150+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3113797408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc40fde40 msgr2=0x7f3fc4113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.150+0000 7f3fc37fe700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc40fde40 0x7f3fc4113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.150+0000 7f3fc37fe700 1 -- 172.21.15.33:0/3113797408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb4005040 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.150+0000 7f3fc37fe700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fe7e0 0x7f3fc4113e40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3fac002a80 tx=0x7f3fac0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:53.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.150+0000 7f3fc0ff9700 1 -- 172.21.15.33:0/3113797408 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fac007cc0 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.150+0000 7f3fc0ff9700 1 -- 172.21.15.33:0/3113797408 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fac00f040 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.150+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc410ddb0 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.150+0000 7f3fc0ff9700 1 -- 172.21.15.33:0/3113797408 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fac00dac0 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.150+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc410e2f0 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.151+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc4066a90 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.152+0000 7f3fc0ff9700 1 -- 172.21.15.33:0/3113797408 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3fac00dc20 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.152+0000 7f3fc0ff9700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fb006e140 0x7f3fb00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:45:53.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.152+0000 7f3fc0ff9700 1 -- 172.21.15.33:0/3113797408 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3fac08d7a0 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.152+0000 7f3fc3fff700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fb006e140 0x7f3fb00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:45:53.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.153+0000 7f3fc3fff700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fb006e140 0x7f3fb00705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400b3b0 tx=0x7f3fb4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:45:53.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.157+0000 7f3fc0ff9700 1 -- 172.21.15.33:0/3113797408 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3fac05a650 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.166 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:52 smithi138 ceph-mon[81304]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:53.167 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:52 smithi138 ceph-mon[81304]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:53.167 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:52 smithi138 ceph-mon[81304]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:53.167 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:52 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1061614754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:53.167 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:52 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2152077969' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:45:53.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:52 smithi033 ceph-mon[111306]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:53.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:52 smithi033 ceph-mon[111306]: from='client.34430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:53.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:52 smithi033 ceph-mon[111306]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:53.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:52 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1061614754' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:45:53.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:52 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2152077969' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:45:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.450+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3fc4109eb0 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.451+0000 7f3fc0ff9700 1 -- 172.21.15.33:0/3113797408 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f3fac05a1e0 con 0x7f3fc40fe7e0 2024-04-25T01:45:53.445 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:45:53.445 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:45:53.445 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:45:53.445 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:45:53.445 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:45:53.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fb006e140 msgr2=0x7f3fb00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fb006e140 0x7f3fb00705f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f3fb400b3b0 tx=0x7f3fb4009e30 comp rx=0 tx=0).stop 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fe7e0 msgr2=0x7f3fc4113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fe7e0 0x7f3fc4113e40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3fac002a80 tx=0x7f3fac0046b0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 shutdown_connections 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fb006e140 0x7f3fb00705f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fc40fde40 0x7f3fc4113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 --2- 172.21.15.33:0/3113797408 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fc40fe7e0 0x7f3fc4113e40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 >> 172.21.15.33:0/3113797408 conn(0x7f3fc40f7860 msgr2=0x7f3fc40f8cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 shutdown_connections 2024-04-25T01:45:53.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:45:53.453+0000 7f3fca863700 1 -- 172.21.15.33:0/3113797408 wait complete. 2024-04-25T01:45:54.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:53 smithi033 ceph-mon[111306]: pgmap v193: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:54.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:53 smithi033 ceph-mon[111306]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:54.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:53 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3113797408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:45:54.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:53 smithi138 ceph-mon[81304]: pgmap v193: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:54.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:53 smithi138 ceph-mon[81304]: from='client.34446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:45:54.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:53 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3113797408' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:45:55.253 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:54 smithi138 ceph-mon[81304]: osdmap e83: 6 total, 6 up, 6 in 2024-04-25T01:45:55.253 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:54 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-25T01:45:55.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:54 smithi033 ceph-mon[111306]: osdmap e83: 6 total, 6 up, 6 in 2024-04-25T01:45:55.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:54 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-25T01:45:56.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:55 smithi033 ceph-mon[111306]: pgmap v195: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:56.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:45:56.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:55 smithi138 ceph-mon[81304]: pgmap v195: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:56.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:45:57.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:56 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] up:boot 2024-04-25T01:45:57.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:56 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T01:45:57.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:45:57.137 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:57.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:57.138 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:57.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:56 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] up:boot 2024-04-25T01:45:57.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:56 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T01:45:57.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T01:45:57.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:57.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:45:57.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T01:45:58.385 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:45:58 smithi138 ceph-mon[81304]: pgmap v196: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:45:58.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:45:58 smithi033 ceph-mon[111306]: pgmap v196: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:00.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:00 smithi033 ceph-mon[111306]: pgmap v197: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:00.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:00 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:46:00.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:00 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:46:00.284 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:00 smithi138 ceph-mon[81304]: pgmap v197: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:00.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:00 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:46:00.285 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:00 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:46:01.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:46:01.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:01 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:46:01.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:46:01.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:01 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T01:46:02.380 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:02 smithi138 ceph-mon[81304]: pgmap v198: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:02.509 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:02 smithi033 ceph-mon[111306]: pgmap v198: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:04.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:04 smithi033 ceph-mon[111306]: pgmap v199: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:04.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:04 smithi138 ceph-mon[81304]: pgmap v199: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:06.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:06 smithi033 ceph-mon[111306]: pgmap v200: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:06.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:06 smithi138 ceph-mon[81304]: pgmap v200: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:08.456 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:08 smithi138 ceph-mon[81304]: pgmap v201: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:08.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:08 smithi033 ceph-mon[111306]: pgmap v201: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:10.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:10 smithi033 ceph-mon[111306]: pgmap v202: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:10.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:10 smithi138 ceph-mon[81304]: pgmap v202: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:11.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:46:11.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:46:12.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:12 smithi033 ceph-mon[111306]: pgmap v203: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:12.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:12 smithi138 ceph-mon[81304]: pgmap v203: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:14.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:14 smithi033 ceph-mon[111306]: pgmap v204: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:14.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:14 smithi138 ceph-mon[81304]: pgmap v204: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:16.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:16 smithi033 ceph-mon[111306]: pgmap v205: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:16.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:16 smithi138 ceph-mon[81304]: pgmap v205: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:18.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:18 smithi033 ceph-mon[111306]: pgmap v206: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:18.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:18 smithi138 ceph-mon[81304]: pgmap v206: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:20.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:20 smithi033 ceph-mon[111306]: pgmap v207: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:20.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:20 smithi138 ceph-mon[81304]: pgmap v207: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:22.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:22 smithi033 ceph-mon[111306]: pgmap v208: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:22.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:22 smithi138 ceph-mon[81304]: pgmap v208: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 -- 172.21.15.33:0/1191123480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 msgr2=0x7f061c0fcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 --2- 172.21.15.33:0/1191123480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 0x7f061c0fcc20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f060c004940 tx=0x7f060c009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 -- 172.21.15.33:0/1191123480 shutdown_connections 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 --2- 172.21.15.33:0/1191123480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 0x7f061c0fcc20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 --2- 172.21.15.33:0/1191123480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061c102c10 0x7f061c102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 -- 172.21.15.33:0/1191123480 >> 172.21.15.33:0/1191123480 conn(0x7f061c0f8740 msgr2=0x7f061c0fab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 -- 172.21.15.33:0/1191123480 shutdown_connections 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 -- 172.21.15.33:0/1191123480 wait complete. 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 Processor -- start 2024-04-25T01:46:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.574+0000 7f06225a2700 1 -- start start 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f06225a2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061c102c10 0x7f061c1139c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f06225a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 0x7f061c113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f06225a2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c114550 con 0x7f061c1035b0 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f06225a2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c10d940 con 0x7f061c102c10 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f0620d9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 0x7f061c113f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f0620d9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 0x7f061c113f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53076/0 (socket says 172.21.15.33:53076) 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f0620d9f700 1 -- 172.21.15.33:0/121547648 learned_addr learned my addr 172.21.15.33:0/121547648 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f06215a0700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061c102c10 0x7f061c1139c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f0620d9f700 1 -- 172.21.15.33:0/121547648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061c102c10 msgr2=0x7f061c1139c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f0620d9f700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061c102c10 0x7f061c1139c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f0620d9f700 1 -- 172.21.15.33:0/121547648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f060c005040 con 0x7f061c1035b0 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.575+0000 7f0620d9f700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 0x7f061c113f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f060c009100 tx=0x7f060c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.576+0000 7f0611ffb700 1 -- 172.21.15.33:0/121547648 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c00e040 con 0x7f061c1035b0 2024-04-25T01:46:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.576+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c10dbc0 con 0x7f061c1035b0 2024-04-25T01:46:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.576+0000 7f0611ffb700 1 -- 172.21.15.33:0/121547648 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f060c009af0 con 0x7f061c1035b0 2024-04-25T01:46:23.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.576+0000 7f0611ffb700 1 -- 172.21.15.33:0/121547648 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060c01c3f0 con 0x7f061c1035b0 2024-04-25T01:46:23.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.576+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c10e180 con 0x7f061c1035b0 2024-04-25T01:46:23.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.577+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c0fe360 con 0x7f061c1035b0 2024-04-25T01:46:23.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.578+0000 7f0611ffb700 1 -- 172.21.15.33:0/121547648 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f060c00baf0 con 0x7f061c1035b0 2024-04-25T01:46:23.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.578+0000 7f0611ffb700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f060806e140 0x7f06080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:23.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.578+0000 7f0611ffb700 1 -- 172.21.15.33:0/121547648 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f060c08ed60 con 0x7f061c1035b0 2024-04-25T01:46:23.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.578+0000 7f06215a0700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f060806e140 0x7f06080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:23.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.579+0000 7f06215a0700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f060806e140 0x7f06080705f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0618004eb0 tx=0x7f061800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:23.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.584+0000 7f0611ffb700 1 -- 172.21.15.33:0/121547648 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f060c05bc90 con 0x7f061c1035b0 2024-04-25T01:46:23.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.810+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f061c10f080 con 0x7f060806e140 2024-04-25T01:46:23.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.811+0000 7f0611ffb700 1 -- 172.21.15.33:0/121547648 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f061c10f080 con 0x7f060806e140 2024-04-25T01:46:23.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f060806e140 msgr2=0x7f06080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:23.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f060806e140 0x7f06080705f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0618004eb0 tx=0x7f061800b040 comp rx=0 tx=0).stop 2024-04-25T01:46:23.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 msgr2=0x7f061c113f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:23.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 0x7f061c113f00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f060c009100 tx=0x7f060c009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:23.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 shutdown_connections 2024-04-25T01:46:23.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f060806e140 0x7f06080705f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:23.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f061c102c10 0x7f061c1139c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:23.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 --2- 172.21.15.33:0/121547648 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f061c1035b0 0x7f061c113f00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:23.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 >> 172.21.15.33:0/121547648 conn(0x7f061c0f8740 msgr2=0x7f061c0f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:23.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 shutdown_connections 2024-04-25T01:46:23.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.814+0000 7f06225a2700 1 -- 172.21.15.33:0/121547648 wait complete. 2024-04-25T01:46:23.819 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 -- 172.21.15.33:0/2559606453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 msgr2=0x7f41000fb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 --2- 172.21.15.33:0/2559606453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 0x7f41000fb9a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f40f4004a50 tx=0x7f40f400af40 comp rx=0 tx=0).stop 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 -- 172.21.15.33:0/2559606453 shutdown_connections 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 --2- 172.21.15.33:0/2559606453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 0x7f41000fb9a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 --2- 172.21.15.33:0/2559606453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4100101490 0x7f4100101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 -- 172.21.15.33:0/2559606453 >> 172.21.15.33:0/2559606453 conn(0x7f4100071380 msgr2=0x7f4100071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 -- 172.21.15.33:0/2559606453 shutdown_connections 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 -- 172.21.15.33:0/2559606453 wait complete. 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 Processor -- start 2024-04-25T01:46:23.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.920+0000 7f41071c0700 1 -- start start 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f41071c0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 0x7f4100113b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f41071c0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4100101490 0x7f4100114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f41071c0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4100114690 con 0x7f41000fb530 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f41071c0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410010da80 con 0x7f4100101490 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f4104f5c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 0x7f4100113b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f4104f5c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 0x7f4100113b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53096/0 (socket says 172.21.15.33:53096) 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f4104f5c700 1 -- 172.21.15.33:0/215479650 learned_addr learned my addr 172.21.15.33:0/215479650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f40fffff700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4100101490 0x7f4100114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f4104f5c700 1 -- 172.21.15.33:0/215479650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4100101490 msgr2=0x7f4100114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f4104f5c700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4100101490 0x7f4100114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:23.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f4104f5c700 1 -- 172.21.15.33:0/215479650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f4005040 con 0x7f41000fb530 2024-04-25T01:46:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f4104f5c700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 0x7f4100113b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f40f000eb40 tx=0x7f40f000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.921+0000 7f40fd7fa700 1 -- 172.21.15.33:0/215479650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40f0007cc0 con 0x7f41000fb530 2024-04-25T01:46:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.922+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410010dd00 con 0x7f41000fb530 2024-04-25T01:46:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.922+0000 7f40fd7fa700 1 -- 172.21.15.33:0/215479650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40f000f040 con 0x7f41000fb530 2024-04-25T01:46:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.922+0000 7f40fd7fa700 1 -- 172.21.15.33:0/215479650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40f000ca10 con 0x7f41000fb530 2024-04-25T01:46:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.922+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410010e290 con 0x7f41000fb530 2024-04-25T01:46:23.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.922+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41000fcb70 con 0x7f41000fb530 2024-04-25T01:46:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.923+0000 7f40fd7fa700 1 -- 172.21.15.33:0/215479650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f40f000cc80 con 0x7f41000fb530 2024-04-25T01:46:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.923+0000 7f40fd7fa700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f40e806e140 0x7f40e80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.924+0000 7f40fd7fa700 1 -- 172.21.15.33:0/215479650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f40f008d6f0 con 0x7f41000fb530 2024-04-25T01:46:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.924+0000 7f40fffff700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f40e806e140 0x7f40e80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:23.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.924+0000 7f40fffff700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f40e806e140 0x7f40e80705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f40f400ac40 tx=0x7f40f400af40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:23.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:23.928+0000 7f40fd7fa700 1 -- 172.21.15.33:0/215479650 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f40f005a650 con 0x7f41000fb530 2024-04-25T01:46:24.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.156+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4100107ce0 con 0x7f40e806e140 2024-04-25T01:46:24.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.157+0000 7f40fd7fa700 1 -- 172.21.15.33:0/215479650 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4100107ce0 con 0x7f40e806e140 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.159+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f40e806e140 msgr2=0x7f40e80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.159+0000 7f41071c0700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f40e806e140 0x7f40e80705f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f40f400ac40 tx=0x7f40f400af40 comp rx=0 tx=0).stop 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.159+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 msgr2=0x7f4100113b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.159+0000 7f41071c0700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 0x7f4100113b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f40f000eb40 tx=0x7f40f000b040 comp rx=0 tx=0).stop 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.160+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 shutdown_connections 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.160+0000 7f41071c0700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f40e806e140 0x7f40e80705f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.160+0000 7f41071c0700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41000fb530 0x7f4100113b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.160+0000 7f41071c0700 1 --2- 172.21.15.33:0/215479650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4100101490 0x7f4100114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.160+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 >> 172.21.15.33:0/215479650 conn(0x7f4100071380 msgr2=0x7f4100100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:24.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.160+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 shutdown_connections 2024-04-25T01:46:24.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.160+0000 7f41071c0700 1 -- 172.21.15.33:0/215479650 wait complete. 2024-04-25T01:46:24.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.264+0000 7fb796e3c700 1 -- 172.21.15.33:0/3365427174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 msgr2=0x7fb790106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.264+0000 7fb796e3c700 1 --2- 172.21.15.33:0/3365427174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 0x7fb790106e50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb77c004940 tx=0x7fb77c009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.265+0000 7fb796e3c700 1 -- 172.21.15.33:0/3365427174 shutdown_connections 2024-04-25T01:46:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.265+0000 7fb796e3c700 1 --2- 172.21.15.33:0/3365427174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 0x7fb790106e50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.265+0000 7fb796e3c700 1 --2- 172.21.15.33:0/3365427174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7900fde70 0x7fb7900fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.265+0000 7fb796e3c700 1 -- 172.21.15.33:0/3365427174 >> 172.21.15.33:0/3365427174 conn(0x7fb7900f7860 msgr2=0x7fb7900f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.265+0000 7fb796e3c700 1 -- 172.21.15.33:0/3365427174 shutdown_connections 2024-04-25T01:46:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.265+0000 7fb796e3c700 1 -- 172.21.15.33:0/3365427174 wait complete. 2024-04-25T01:46:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.265+0000 7fb796e3c700 1 Processor -- start 2024-04-25T01:46:24.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.265+0000 7fb796e3c700 1 -- start start 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb796e3c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7900fde70 0x7fb790113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb796e3c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 0x7fb790113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb796e3c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb790114560 con 0x7fb7900fe810 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb796e3c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79010dad0 con 0x7fb7900fde70 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb78ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 0x7fb790113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb78ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 0x7fb790113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53126/0 (socket says 172.21.15.33:53126) 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb78ffff700 1 -- 172.21.15.33:0/2339356102 learned_addr learned my addr 172.21.15.33:0/2339356102 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb794bd8700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7900fde70 0x7fb790113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb78ffff700 1 -- 172.21.15.33:0/2339356102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7900fde70 msgr2=0x7fb790113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb78ffff700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7900fde70 0x7fb790113940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb78ffff700 1 -- 172.21.15.33:0/2339356102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb77c005040 con 0x7fb7900fe810 2024-04-25T01:46:24.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb78ffff700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 0x7fb790113e80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009380 tx=0x7fb77c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:24.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.266+0000 7fb78d7fa700 1 -- 172.21.15.33:0/2339356102 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb77c00e040 con 0x7fb7900fe810 2024-04-25T01:46:24.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.267+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79010ddb0 con 0x7fb7900fe810 2024-04-25T01:46:24.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.267+0000 7fb78d7fa700 1 -- 172.21.15.33:0/2339356102 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb77c009af0 con 0x7fb7900fe810 2024-04-25T01:46:24.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.267+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79010e2f0 con 0x7fb7900fe810 2024-04-25T01:46:24.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.267+0000 7fb78d7fa700 1 -- 172.21.15.33:0/2339356102 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb77c0255e0 con 0x7fb7900fe810 2024-04-25T01:46:24.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.268+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb790066af0 con 0x7fb7900fe810 2024-04-25T01:46:24.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.268+0000 7fb78d7fa700 1 -- 172.21.15.33:0/2339356102 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb77c00bbc0 con 0x7fb7900fe810 2024-04-25T01:46:24.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.269+0000 7fb78d7fa700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb78006e140 0x7fb7800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:24.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.269+0000 7fb794bd8700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb78006e140 0x7fb7800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:24.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.269+0000 7fb78d7fa700 1 -- 172.21.15.33:0/2339356102 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb77c08ee60 con 0x7fb7900fe810 2024-04-25T01:46:24.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.269+0000 7fb794bd8700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb78006e140 0x7fb7800705f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb784006fd0 tx=0x7fb784008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:24.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.274+0000 7fb78d7fa700 1 -- 172.21.15.33:0/2339356102 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb77c05bcf0 con 0x7fb7900fe810 2024-04-25T01:46:24.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.490+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb790109eb0 con 0x7fb78006e140 2024-04-25T01:46:24.484 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:24 smithi033 ceph-mon[111306]: pgmap v209: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:24.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.501+0000 7fb78d7fa700 1 -- 172.21.15.33:0/2339356102 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fb790109eb0 con 0x7fb78006e140 2024-04-25T01:46:24.495 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:46:24.495 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (7m) 40s ago 18m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:46:24.495 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (4m) 40s ago 18m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:46:24.495 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (4m) 25s ago 17m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:46:24.495 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (7m) 40s ago 18m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:46:24.495 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (45s) 40s ago 13m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (64s) 40s ago 13m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (28s) 25s ago 13m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (13m) 25s ago 13m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (9m) 40s ago 20m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (8m) 25s ago 17m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (5m) 40s ago 20m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (5m) 25s ago 17m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (8m) 40s ago 18m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (8m) 25s ago 17m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (4m) 40s ago 17m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (3m) 40s ago 16m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (3m) 40s ago 16m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (2m) 25s ago 15m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (117s) 25s ago 15m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:46:24.496 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (83s) 25s ago 14m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:46:24.497 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (7m) 40s ago 18m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:46:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb78006e140 msgr2=0x7fb7800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb78006e140 0x7fb7800705f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb784006fd0 tx=0x7fb784008040 comp rx=0 tx=0).stop 2024-04-25T01:46:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 msgr2=0x7fb790113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 0x7fb790113e80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb77c009380 tx=0x7fb77c009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 shutdown_connections 2024-04-25T01:46:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb78006e140 0x7fb7800705f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7900fde70 0x7fb790113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 --2- 172.21.15.33:0/2339356102 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7900fe810 0x7fb790113e80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 >> 172.21.15.33:0/2339356102 conn(0x7fb7900f7860 msgr2=0x7fb7900f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 shutdown_connections 2024-04-25T01:46:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.504+0000 7fb796e3c700 1 -- 172.21.15.33:0/2339356102 wait complete. 2024-04-25T01:46:24.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:24 smithi138 ceph-mon[81304]: pgmap v209: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:24.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.607+0000 7fcdf5eba700 1 -- 172.21.15.33:0/671137448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf01026b0 msgr2=0x7fcdf00fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.607+0000 7fcdf5eba700 1 --2- 172.21.15.33:0/671137448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf01026b0 0x7fcdf00fbd20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8004940 tx=0x7fcdd8009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:24.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.607+0000 7fcdf5eba700 1 -- 172.21.15.33:0/671137448 shutdown_connections 2024-04-25T01:46:24.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.607+0000 7fcdf5eba700 1 --2- 172.21.15.33:0/671137448 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf01026b0 0x7fcdf00fbd20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.607+0000 7fcdf5eba700 1 --2- 172.21.15.33:0/671137448 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf0101d10 0x7fcdf01020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.607+0000 7fcdf5eba700 1 -- 172.21.15.33:0/671137448 >> 172.21.15.33:0/671137448 conn(0x7fcdf00f7860 msgr2=0x7fcdf00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:24.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.607+0000 7fcdf5eba700 1 -- 172.21.15.33:0/671137448 shutdown_connections 2024-04-25T01:46:24.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.607+0000 7fcdf5eba700 1 -- 172.21.15.33:0/671137448 wait complete. 2024-04-25T01:46:24.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.608+0000 7fcdf5eba700 1 Processor -- start 2024-04-25T01:46:24.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.608+0000 7fcdf5eba700 1 -- start start 2024-04-25T01:46:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.608+0000 7fcdf5eba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf0101d10 0x7fcdf0113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.608+0000 7fcdf5eba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf01026b0 0x7fcdf0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.608+0000 7fcdf5eba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0114510 con 0x7fcdf0101d10 2024-04-25T01:46:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.608+0000 7fcdf5eba700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf010db30 con 0x7fcdf01026b0 2024-04-25T01:46:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.608+0000 7fcdef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf0101d10 0x7fcdf0113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.608+0000 7fcdef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf0101d10 0x7fcdf0113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53148/0 (socket says 172.21.15.33:53148) 2024-04-25T01:46:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.608+0000 7fcdef7fe700 1 -- 172.21.15.33:0/2264989333 learned_addr learned my addr 172.21.15.33:0/2264989333 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:24.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcdef7fe700 1 -- 172.21.15.33:0/2264989333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf01026b0 msgr2=0x7fcdf0113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcdeeffd700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf01026b0 0x7fcdf0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcdef7fe700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf01026b0 0x7fcdf0113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcdef7fe700 1 -- 172.21.15.33:0/2264989333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8005040 con 0x7fcdf0101d10 2024-04-25T01:46:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcdef7fe700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf0101d10 0x7fcdf0113980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001d80 tx=0x7fcde0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcde7fff700 1 -- 172.21.15.33:0/2264989333 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcde0007d10 con 0x7fcdf0101d10 2024-04-25T01:46:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcde7fff700 1 -- 172.21.15.33:0/2264989333 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcde0007e70 con 0x7fcdf0101d10 2024-04-25T01:46:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf010de10 con 0x7fcdf0101d10 2024-04-25T01:46:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcde7fff700 1 -- 172.21.15.33:0/2264989333 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcde0014640 con 0x7fcdf0101d10 2024-04-25T01:46:24.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.609+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf010e3d0 con 0x7fcdf0101d10 2024-04-25T01:46:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.610+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf00fd460 con 0x7fcdf0101d10 2024-04-25T01:46:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.611+0000 7fcde7fff700 1 -- 172.21.15.33:0/2264989333 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcde000e960 con 0x7fcdf0101d10 2024-04-25T01:46:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.611+0000 7fcde7fff700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcddc06e140 0x7fcddc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.611+0000 7fcde7fff700 1 -- 172.21.15.33:0/2264989333 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcde008d940 con 0x7fcdf0101d10 2024-04-25T01:46:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.611+0000 7fcdeeffd700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcddc06e140 0x7fcddc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:24.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.612+0000 7fcdeeffd700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcddc06e140 0x7fcddc0705f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800b3b0 tx=0x7fcdd8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:24.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.616+0000 7fcde7fff700 1 -- 172.21.15.33:0/2264989333 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcde0056d70 con 0x7fcdf0101d10 2024-04-25T01:46:24.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.915+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcdf004c710 con 0x7fcdf0101d10 2024-04-25T01:46:24.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.916+0000 7fcde7fff700 1 -- 172.21.15.33:0/2264989333 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fcde005a430 con 0x7fcdf0101d10 2024-04-25T01:46:24.909 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:46:24.909 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:46:24.909 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:46:24.909 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:46:24.909 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:46:24.910 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:46:24.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcddc06e140 msgr2=0x7fcddc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcddc06e140 0x7fcddc0705f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcdd800b3b0 tx=0x7fcdd8009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:24.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf0101d10 msgr2=0x7fcdf0113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:24.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf0101d10 0x7fcdf0113980 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcde0001d80 tx=0x7fcde0009ec0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 shutdown_connections 2024-04-25T01:46:24.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcddc06e140 0x7fcddc0705f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcdf0101d10 0x7fcdf0113980 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 --2- 172.21.15.33:0/2264989333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcdf01026b0 0x7fcdf0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:24.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 >> 172.21.15.33:0/2264989333 conn(0x7fcdf00f7860 msgr2=0x7fcdf00662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:24.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 shutdown_connections 2024-04-25T01:46:24.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:24.918+0000 7fcdf5eba700 1 -- 172.21.15.33:0/2264989333 wait complete. 2024-04-25T01:46:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.022+0000 7f83db833700 1 -- 172.21.15.33:0/2713468901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 msgr2=0x7f83d40fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.022+0000 7f83db833700 1 --2- 172.21.15.33:0/2713468901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 0x7f83d40fb990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f83c4004940 tx=0x7f83c4009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.023+0000 7f83db833700 1 -- 172.21.15.33:0/2713468901 shutdown_connections 2024-04-25T01:46:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.023+0000 7f83db833700 1 --2- 172.21.15.33:0/2713468901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 0x7f83d40fb990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.023+0000 7f83db833700 1 --2- 172.21.15.33:0/2713468901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d4102f60 0x7f83d4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.023+0000 7f83db833700 1 -- 172.21.15.33:0/2713468901 >> 172.21.15.33:0/2713468901 conn(0x7f83d40714d0 msgr2=0x7f83d40718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.023+0000 7f83db833700 1 -- 172.21.15.33:0/2713468901 shutdown_connections 2024-04-25T01:46:25.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.023+0000 7f83db833700 1 -- 172.21.15.33:0/2713468901 wait complete. 2024-04-25T01:46:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.023+0000 7f83db833700 1 Processor -- start 2024-04-25T01:46:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.023+0000 7f83db833700 1 -- start start 2024-04-25T01:46:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83db833700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 0x7f83d4113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83db833700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d4102f60 0x7f83d4113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83db833700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d4114510 con 0x7f83d40fb520 2024-04-25T01:46:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83db833700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d410db30 con 0x7f83d4102f60 2024-04-25T01:46:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83d95cf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 0x7f83d4113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83d95cf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 0x7f83d4113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53156/0 (socket says 172.21.15.33:53156) 2024-04-25T01:46:25.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83d95cf700 1 -- 172.21.15.33:0/1492709602 learned_addr learned my addr 172.21.15.33:0/1492709602 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83d8dce700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d4102f60 0x7f83d4113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83d95cf700 1 -- 172.21.15.33:0/1492709602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d4102f60 msgr2=0x7f83d4113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83d95cf700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d4102f60 0x7f83d4113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83d95cf700 1 -- 172.21.15.33:0/1492709602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c4005040 con 0x7f83d40fb520 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83d95cf700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 0x7f83d4113980 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c8e0 tx=0x7f83d00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83c9ffb700 1 -- 172.21.15.33:0/1492709602 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d000a760 con 0x7f83d40fb520 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83c9ffb700 1 -- 172.21.15.33:0/1492709602 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83d000ad90 con 0x7f83d40fb520 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.024+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d410de10 con 0x7f83d40fb520 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.025+0000 7f83c9ffb700 1 -- 172.21.15.33:0/1492709602 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83d00113a0 con 0x7f83d40fb520 2024-04-25T01:46:25.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.025+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d410e350 con 0x7f83d40fb520 2024-04-25T01:46:25.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.026+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d4100250 con 0x7f83d40fb520 2024-04-25T01:46:25.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.026+0000 7f83c9ffb700 1 -- 172.21.15.33:0/1492709602 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f83d0012400 con 0x7f83d40fb520 2024-04-25T01:46:25.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.027+0000 7f83c9ffb700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83c006e140 0x7f83c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:25.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.027+0000 7f83c9ffb700 1 -- 172.21.15.33:0/1492709602 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f83d008daf0 con 0x7f83d40fb520 2024-04-25T01:46:25.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.027+0000 7f83d8dce700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83c006e140 0x7f83c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:25.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.027+0000 7f83d8dce700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83c006e140 0x7f83c00705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83d410f220 tx=0x7f83c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:25.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.031+0000 7f83c9ffb700 1 -- 172.21.15.33:0/1492709602 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f83d0056ef0 con 0x7f83d40fb520 2024-04-25T01:46:25.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:25 smithi033 ceph-mon[111306]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:25.260 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:25 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2264989333' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:46:25.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.290+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f83d40fe840 con 0x7f83d40fb520 2024-04-25T01:46:25.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.290+0000 7f83c9ffb700 1 -- 172.21.15.33:0/1492709602 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f83d008d020 con 0x7f83d40fb520 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:46:25.284 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:46:25.285 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:46:25.286 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:46:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.292+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83c006e140 msgr2=0x7f83c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.292+0000 7f83db833700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83c006e140 0x7f83c00705f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f83d410f220 tx=0x7f83c4009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.292+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 msgr2=0x7f83d4113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.292+0000 7f83db833700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 0x7f83d4113980 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f83d000c8e0 tx=0x7f83d00044c0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.293+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 shutdown_connections 2024-04-25T01:46:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.293+0000 7f83db833700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83c006e140 0x7f83c00705f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.293+0000 7f83db833700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d40fb520 0x7f83d4113980 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.293+0000 7f83db833700 1 --2- 172.21.15.33:0/1492709602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d4102f60 0x7f83d4113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.293+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 >> 172.21.15.33:0/1492709602 conn(0x7f83d40714d0 msgr2=0x7f83d40fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.293+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 shutdown_connections 2024-04-25T01:46:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.293+0000 7f83db833700 1 -- 172.21.15.33:0/1492709602 wait complete. 2024-04-25T01:46:25.288 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.398+0000 7fb93dc82700 1 -- 172.21.15.33:0/576661702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 msgr2=0x7fb938101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.398+0000 7fb93dc82700 1 --2- 172.21.15.33:0/576661702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 0x7fb938101a30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb920004940 tx=0x7fb920009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.398+0000 7fb93dc82700 1 -- 172.21.15.33:0/576661702 shutdown_connections 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.398+0000 7fb93dc82700 1 --2- 172.21.15.33:0/576661702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 0x7fb938101a30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.398+0000 7fb93dc82700 1 --2- 172.21.15.33:0/576661702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9380fb5a0 0x7fb9380fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.398+0000 7fb93dc82700 1 -- 172.21.15.33:0/576661702 >> 172.21.15.33:0/576661702 conn(0x7fb938073350 msgr2=0x7fb938073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.398+0000 7fb93dc82700 1 -- 172.21.15.33:0/576661702 shutdown_connections 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.398+0000 7fb93dc82700 1 -- 172.21.15.33:0/576661702 wait complete. 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb93dc82700 1 Processor -- start 2024-04-25T01:46:25.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb93dc82700 1 -- start start 2024-04-25T01:46:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb93dc82700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9380fb5a0 0x7fb9380fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb93dc82700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 0x7fb9380fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb93dc82700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9380fe430 con 0x7fb9380fbf40 2024-04-25T01:46:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb93dc82700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9380fcfb0 con 0x7fb9380fb5a0 2024-04-25T01:46:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb936ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 0x7fb9380fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb936ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 0x7fb9380fca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53172/0 (socket says 172.21.15.33:53172) 2024-04-25T01:46:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb936ffd700 1 -- 172.21.15.33:0/1248102505 learned_addr learned my addr 172.21.15.33:0/1248102505 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb9377fe700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9380fb5a0 0x7fb9380fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:25.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb936ffd700 1 -- 172.21.15.33:0/1248102505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9380fb5a0 msgr2=0x7fb9380fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb936ffd700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9380fb5a0 0x7fb9380fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.399+0000 7fb936ffd700 1 -- 172.21.15.33:0/1248102505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb920005040 con 0x7fb9380fbf40 2024-04-25T01:46:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.400+0000 7fb936ffd700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 0x7fb9380fca70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb920000c00 tx=0x7fb920009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.400+0000 7fb92ffff700 1 -- 172.21.15.33:0/1248102505 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb92000e040 con 0x7fb9380fbf40 2024-04-25T01:46:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.400+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9380fd230 con 0x7fb9380fbf40 2024-04-25T01:46:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.400+0000 7fb92ffff700 1 -- 172.21.15.33:0/1248102505 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb920009400 con 0x7fb9380fbf40 2024-04-25T01:46:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.400+0000 7fb92ffff700 1 -- 172.21.15.33:0/1248102505 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb92001c3f0 con 0x7fb9380fbf40 2024-04-25T01:46:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.400+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93811a150 con 0x7fb9380fbf40 2024-04-25T01:46:25.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.401+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93804c710 con 0x7fb9380fbf40 2024-04-25T01:46:25.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.401+0000 7fb92ffff700 1 -- 172.21.15.33:0/1248102505 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb92000baf0 con 0x7fb9380fbf40 2024-04-25T01:46:25.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.402+0000 7fb92ffff700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb92406e140 0x7fb9240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:25.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.402+0000 7fb92ffff700 1 -- 172.21.15.33:0/1248102505 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb92008ecd0 con 0x7fb9380fbf40 2024-04-25T01:46:25.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.402+0000 7fb9377fe700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb92406e140 0x7fb9240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:25.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.402+0000 7fb9377fe700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb92406e140 0x7fb9240705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb928004550 tx=0x7fb928007eb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:25.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.407+0000 7fb92ffff700 1 -- 172.21.15.33:0/1248102505 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb9200580d0 con 0x7fb9380fbf40 2024-04-25T01:46:25.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:25 smithi138 ceph-mon[81304]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:25.535 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:25 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2264989333' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:46:25.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.633+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb938101120 con 0x7fb92406e140 2024-04-25T01:46:25.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.634+0000 7fb92ffff700 1 -- 172.21.15.33:0/1248102505 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb938101120 con 0x7fb92406e140 2024-04-25T01:46:25.627 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:46:25.628 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:46:25.629 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:46:25.629 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:46:25.629 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:46:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb92406e140 msgr2=0x7fb9240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb92406e140 0x7fb9240705f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb928004550 tx=0x7fb928007eb0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 msgr2=0x7fb9380fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 0x7fb9380fca70 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb920000c00 tx=0x7fb920009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 shutdown_connections 2024-04-25T01:46:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb92406e140 0x7fb9240705f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9380fb5a0 0x7fb9380fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 --2- 172.21.15.33:0/1248102505 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9380fbf40 0x7fb9380fca70 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 >> 172.21.15.33:0/1248102505 conn(0x7fb938073350 msgr2=0x7fb938100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 shutdown_connections 2024-04-25T01:46:25.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.637+0000 7fb93dc82700 1 -- 172.21.15.33:0/1248102505 wait complete. 2024-04-25T01:46:25.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.742+0000 7f795b6c1700 1 -- 172.21.15.33:0/3540496459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 msgr2=0x7f79541020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.742+0000 7f795b6c1700 1 --2- 172.21.15.33:0/3540496459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 0x7f79541020d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7944004940 tx=0x7f7944009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:25.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.742+0000 7f795b6c1700 1 -- 172.21.15.33:0/3540496459 shutdown_connections 2024-04-25T01:46:25.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.742+0000 7f795b6c1700 1 --2- 172.21.15.33:0/3540496459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79541026a0 0x7f79540fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.742+0000 7f795b6c1700 1 --2- 172.21.15.33:0/3540496459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 0x7f79541020d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.742+0000 7f795b6c1700 1 -- 172.21.15.33:0/3540496459 >> 172.21.15.33:0/3540496459 conn(0x7f79540f7870 msgr2=0x7f79540f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795b6c1700 1 -- 172.21.15.33:0/3540496459 shutdown_connections 2024-04-25T01:46:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795b6c1700 1 -- 172.21.15.33:0/3540496459 wait complete. 2024-04-25T01:46:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795b6c1700 1 Processor -- start 2024-04-25T01:46:25.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795b6c1700 1 -- start start 2024-04-25T01:46:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795b6c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 0x7f7954102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:25.738 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795b6c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79541026a0 0x7f79541030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795b6c1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7954104ab0 con 0x7f7954101d00 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795b6c1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79541035e0 con 0x7f79541026a0 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795945d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 0x7f7954102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795945d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 0x7f7954102b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38170/0 (socket says 172.21.15.33:38170) 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.743+0000 7f795945d700 1 -- 172.21.15.33:0/1215057810 learned_addr learned my addr 172.21.15.33:0/1215057810 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f7958c5c700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79541026a0 0x7f79541030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f795945d700 1 -- 172.21.15.33:0/1215057810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79541026a0 msgr2=0x7f79541030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f795945d700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79541026a0 0x7f79541030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f795945d700 1 -- 172.21.15.33:0/1215057810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7944005040 con 0x7f7954101d00 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f795945d700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 0x7f7954102b60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f794400b8b0 tx=0x7f7944009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f7949ffb700 1 -- 172.21.15.33:0/1215057810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f794400e040 con 0x7f7954101d00 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7954103860 con 0x7f7954101d00 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f7949ffb700 1 -- 172.21.15.33:0/1215057810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7944004290 con 0x7f7954101d00 2024-04-25T01:46:25.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f7949ffb700 1 -- 172.21.15.33:0/1215057810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f794401c3f0 con 0x7f7954101d00 2024-04-25T01:46:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.744+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795411a0e0 con 0x7f7954101d00 2024-04-25T01:46:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.745+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79540fd450 con 0x7f7954101d00 2024-04-25T01:46:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.746+0000 7f7949ffb700 1 -- 172.21.15.33:0/1215057810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f79440137b0 con 0x7f7954101d00 2024-04-25T01:46:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.746+0000 7f7949ffb700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f794006e140 0x7f79400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:25.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.746+0000 7f7949ffb700 1 -- 172.21.15.33:0/1215057810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f794408ee50 con 0x7f7954101d00 2024-04-25T01:46:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.746+0000 7f7958c5c700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f794006e140 0x7f79400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:25.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.747+0000 7f7958c5c700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f794006e140 0x7f79400705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7950007fd0 tx=0x7f7950009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:25.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:25.751+0000 7f7949ffb700 1 -- 172.21.15.33:0/1215057810 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7944058250 con 0x7f7954101d00 2024-04-25T01:46:26.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.047+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f795404c710 con 0x7f7954101d00 2024-04-25T01:46:26.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.047+0000 7f7949ffb700 1 -- 172.21.15.33:0/1215057810 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f794405b910 con 0x7f7954101d00 2024-04-25T01:46:26.041 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:46:26.041 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:46:26.041 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:46:26.041 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:46:26.041 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:46:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.049+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f794006e140 msgr2=0x7f79400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.049+0000 7f795b6c1700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f794006e140 0x7f79400705f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7950007fd0 tx=0x7f7950009040 comp rx=0 tx=0).stop 2024-04-25T01:46:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.049+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 msgr2=0x7f7954102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.049+0000 7f795b6c1700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 0x7f7954102b60 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f794400b8b0 tx=0x7f7944009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.049+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 shutdown_connections 2024-04-25T01:46:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.049+0000 7f795b6c1700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f794006e140 0x7f79400705f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.049+0000 7f795b6c1700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7954101d00 0x7f7954102b60 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.049+0000 7f795b6c1700 1 --2- 172.21.15.33:0/1215057810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f79541026a0 0x7f79541030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:26.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.049+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 >> 172.21.15.33:0/1215057810 conn(0x7f79540f7870 msgr2=0x7f79540f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.050+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 shutdown_connections 2024-04-25T01:46:26.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:26.050+0000 7f795b6c1700 1 -- 172.21.15.33:0/1215057810 wait complete. 2024-04-25T01:46:26.437 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:26 smithi138 ceph-mon[81304]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:26.437 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:26 smithi138 ceph-mon[81304]: pgmap v210: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:26.437 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:26 smithi138 ceph-mon[81304]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:26.437 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:46:26.437 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:26 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1492709602' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:46:26.437 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:26 smithi138 ceph-mon[81304]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:26.437 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:26 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1215057810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:46:26.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:26 smithi033 ceph-mon[111306]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:26.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:26 smithi033 ceph-mon[111306]: pgmap v210: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:26.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:26 smithi033 ceph-mon[111306]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:26.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:46:26.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:26 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1492709602' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:46:26.511 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:26 smithi033 ceph-mon[111306]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:26.511 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:26 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1215057810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:46:27.453 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:27 smithi138 ceph-mon[81304]: pgmap v211: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:27.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:27 smithi033 ceph-mon[111306]: pgmap v211: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:29.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:29 smithi033 ceph-mon[111306]: pgmap v212: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:29.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:29 smithi138 ceph-mon[81304]: pgmap v212: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:31.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:31 smithi033 ceph-mon[111306]: pgmap v213: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:31.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:31 smithi138 ceph-mon[81304]: pgmap v213: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:33.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:33 smithi033 ceph-mon[111306]: pgmap v214: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:33.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:33 smithi138 ceph-mon[81304]: pgmap v214: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:35.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:35 smithi033 ceph-mon[111306]: pgmap v215: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:35.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:35 smithi138 ceph-mon[81304]: pgmap v215: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:37.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:37 smithi033 ceph-mon[111306]: pgmap v216: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:37.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:37 smithi138 ceph-mon[81304]: pgmap v216: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:39.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:39 smithi033 ceph-mon[111306]: pgmap v217: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:39.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:39 smithi138 ceph-mon[81304]: pgmap v217: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:40.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:46:40.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:46:41.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:41 smithi033 ceph-mon[111306]: pgmap v218: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:41.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:41 smithi138 ceph-mon[81304]: pgmap v218: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:43.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:43 smithi033 ceph-mon[111306]: pgmap v219: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:43.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:43 smithi138 ceph-mon[81304]: pgmap v219: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:45.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:45 smithi033 ceph-mon[111306]: pgmap v220: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:45.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:45 smithi138 ceph-mon[81304]: pgmap v220: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:47.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:47 smithi033 ceph-mon[111306]: pgmap v221: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:47.785 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:47 smithi138 ceph-mon[81304]: pgmap v221: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:49.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:49 smithi033 ceph-mon[111306]: pgmap v222: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:49.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:49 smithi138 ceph-mon[81304]: pgmap v222: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:51.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:51 smithi033 ceph-mon[111306]: pgmap v223: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:51.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:51 smithi138 ceph-mon[81304]: pgmap v223: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:53.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:53 smithi033 ceph-mon[111306]: pgmap v224: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:53.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:53 smithi138 ceph-mon[81304]: pgmap v224: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:55.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:55 smithi033 ceph-mon[111306]: pgmap v225: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:55.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:46:55.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:55 smithi138 ceph-mon[81304]: pgmap v225: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:55.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.168+0000 7f39ba7a2700 1 -- 172.21.15.33:0/1768291127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b40fbb40 msgr2=0x7f39b40fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.168+0000 7f39ba7a2700 1 --2- 172.21.15.33:0/1768291127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b40fbb40 0x7f39b40fbfb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f39a4004940 tx=0x7f39a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.168+0000 7f39ba7a2700 1 -- 172.21.15.33:0/1768291127 shutdown_connections 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.168+0000 7f39ba7a2700 1 --2- 172.21.15.33:0/1768291127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b40fbb40 0x7f39b40fbfb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.168+0000 7f39ba7a2700 1 --2- 172.21.15.33:0/1768291127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39b4101b00 0x7f39b4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.168+0000 7f39ba7a2700 1 -- 172.21.15.33:0/1768291127 >> 172.21.15.33:0/1768291127 conn(0x7f39b40f7640 msgr2=0x7f39b40f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.169+0000 7f39ba7a2700 1 -- 172.21.15.33:0/1768291127 shutdown_connections 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.169+0000 7f39ba7a2700 1 -- 172.21.15.33:0/1768291127 wait complete. 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.169+0000 7f39ba7a2700 1 Processor -- start 2024-04-25T01:46:56.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.169+0000 7f39ba7a2700 1 -- start start 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.169+0000 7f39ba7a2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39b40fbb40 0x7f39b41139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.169+0000 7f39ba7a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b4101b00 0x7f39b4113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.169+0000 7f39ba7a2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4114540 con 0x7f39b4101b00 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.169+0000 7f39ba7a2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b410d930 con 0x7f39b40fbb40 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39b8f9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b4101b00 0x7f39b4113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39b8f9f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b4101b00 0x7f39b4113ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41312/0 (socket says 172.21.15.33:41312) 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39b8f9f700 1 -- 172.21.15.33:0/515917432 learned_addr learned my addr 172.21.15.33:0/515917432 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39b97a0700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39b40fbb40 0x7f39b41139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39b8f9f700 1 -- 172.21.15.33:0/515917432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39b40fbb40 msgr2=0x7f39b41139b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39b8f9f700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39b40fbb40 0x7f39b41139b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39b8f9f700 1 -- 172.21.15.33:0/515917432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a4005040 con 0x7f39b4101b00 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39b97a0700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39b40fbb40 0x7f39b41139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:46:56.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39b8f9f700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b4101b00 0x7f39b4113ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39a400b8b0 tx=0x7f39a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39a9ffb700 1 -- 172.21.15.33:0/515917432 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39a400e040 con 0x7f39b4101b00 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39a9ffb700 1 -- 172.21.15.33:0/515917432 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39a4004290 con 0x7f39b4101b00 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b410dbb0 con 0x7f39b4101b00 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.170+0000 7f39a9ffb700 1 -- 172.21.15.33:0/515917432 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39a401c3f0 con 0x7f39b4101b00 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.171+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b410e170 con 0x7f39b4101b00 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.172+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b40fd1f0 con 0x7f39b4101b00 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.172+0000 7f39a9ffb700 1 -- 172.21.15.33:0/515917432 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f39a40137b0 con 0x7f39b4101b00 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.173+0000 7f39a9ffb700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39a006e140 0x7f39a00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.173+0000 7f39a9ffb700 1 -- 172.21.15.33:0/515917432 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f39a408ed70 con 0x7f39b4101b00 2024-04-25T01:46:56.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.173+0000 7f39b97a0700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39a006e140 0x7f39a00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:56.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.173+0000 7f39b97a0700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39a006e140 0x7f39a00705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009fd0 tx=0x7f39b0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:56.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.178+0000 7f39a9ffb700 1 -- 172.21.15.33:0/515917432 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f39a405bd40 con 0x7f39b4101b00 2024-04-25T01:46:56.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.407+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f39b4108bf0 con 0x7f39a006e140 2024-04-25T01:46:56.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.408+0000 7f39a9ffb700 1 -- 172.21.15.33:0/515917432 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f39b4108bf0 con 0x7f39a006e140 2024-04-25T01:46:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39a006e140 msgr2=0x7f39a00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39a006e140 0x7f39a00705f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009fd0 tx=0x7f39b0009ee0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b4101b00 msgr2=0x7f39b4113ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b4101b00 0x7f39b4113ef0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f39a400b8b0 tx=0x7f39a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 shutdown_connections 2024-04-25T01:46:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39a006e140 0x7f39a00705f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39b40fbb40 0x7f39b41139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 --2- 172.21.15.33:0/515917432 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39b4101b00 0x7f39b4113ef0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 >> 172.21.15.33:0/515917432 conn(0x7f39b40f7640 msgr2=0x7f39b40f8570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 shutdown_connections 2024-04-25T01:46:56.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.410+0000 7f39ba7a2700 1 -- 172.21.15.33:0/515917432 wait complete. 2024-04-25T01:46:56.416 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:46:56.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 -- 172.21.15.33:0/693004409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 msgr2=0x7f4748103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 --2- 172.21.15.33:0/693004409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 0x7f4748103190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f4740009ce0 tx=0x7f4740004d10 comp rx=0 tx=0).stop 2024-04-25T01:46:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 -- 172.21.15.33:0/693004409 shutdown_connections 2024-04-25T01:46:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 --2- 172.21.15.33:0/693004409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4748103760 0x7f47480fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 --2- 172.21.15.33:0/693004409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 0x7f4748103190 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 -- 172.21.15.33:0/693004409 >> 172.21.15.33:0/693004409 conn(0x7f47480f8930 msgr2=0x7f47480fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 -- 172.21.15.33:0/693004409 shutdown_connections 2024-04-25T01:46:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 -- 172.21.15.33:0/693004409 wait complete. 2024-04-25T01:46:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 Processor -- start 2024-04-25T01:46:56.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.517+0000 7f474ef9a700 1 -- start start 2024-04-25T01:46:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474ef9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 0x7f4748070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474ef9a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4748103760 0x7f47480707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474ef9a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47480721b0 con 0x7f4748102dc0 2024-04-25T01:46:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474ef9a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4748070ce0 con 0x7f4748103760 2024-04-25T01:46:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474cd36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 0x7f4748070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474cd36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 0x7f4748070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41340/0 (socket says 172.21.15.33:41340) 2024-04-25T01:46:56.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474cd36700 1 -- 172.21.15.33:0/4127795135 learned_addr learned my addr 172.21.15.33:0/4127795135 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f4747fff700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4748103760 0x7f47480707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:56.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474cd36700 1 -- 172.21.15.33:0/4127795135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4748103760 msgr2=0x7f47480707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474cd36700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4748103760 0x7f47480707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.518+0000 7f474cd36700 1 -- 172.21.15.33:0/4127795135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4740009940 con 0x7f4748102dc0 2024-04-25T01:46:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.519+0000 7f474cd36700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 0x7f4748070260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4740005010 tx=0x7f4740004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.519+0000 7f47457fa700 1 -- 172.21.15.33:0/4127795135 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f474000e040 con 0x7f4748102dc0 2024-04-25T01:46:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.519+0000 7f47457fa700 1 -- 172.21.15.33:0/4127795135 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4740008e20 con 0x7f4748102dc0 2024-04-25T01:46:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.519+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4748070f60 con 0x7f4748102dc0 2024-04-25T01:46:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.519+0000 7f47457fa700 1 -- 172.21.15.33:0/4127795135 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f474001c660 con 0x7f4748102dc0 2024-04-25T01:46:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.519+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474811a270 con 0x7f4748102dc0 2024-04-25T01:46:56.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.520+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47480fe510 con 0x7f4748102dc0 2024-04-25T01:46:56.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.520+0000 7f47457fa700 1 -- 172.21.15.33:0/4127795135 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4740012a70 con 0x7f4748102dc0 2024-04-25T01:46:56.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.521+0000 7f47457fa700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f473006e140 0x7f47300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:56.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.521+0000 7f47457fa700 1 -- 172.21.15.33:0/4127795135 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f474008f660 con 0x7f4748102dc0 2024-04-25T01:46:56.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.521+0000 7f4747fff700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f473006e140 0x7f47300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:56.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.521+0000 7f4747fff700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f473006e140 0x7f47300705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f473c007fd0 tx=0x7f473c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:56.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.526+0000 7f47457fa700 1 -- 172.21.15.33:0/4127795135 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4740058a60 con 0x7f4748102dc0 2024-04-25T01:46:56.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.752+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47480fa010 con 0x7f473006e140 2024-04-25T01:46:56.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.753+0000 7f47457fa700 1 -- 172.21.15.33:0/4127795135 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f47480fa010 con 0x7f473006e140 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.755+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f473006e140 msgr2=0x7f47300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.755+0000 7f474ef9a700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f473006e140 0x7f47300705f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f473c007fd0 tx=0x7f473c009040 comp rx=0 tx=0).stop 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.755+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 msgr2=0x7f4748070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.755+0000 7f474ef9a700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 0x7f4748070260 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4740005010 tx=0x7f4740004d10 comp rx=0 tx=0).stop 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.756+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 shutdown_connections 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.756+0000 7f474ef9a700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f473006e140 0x7f47300705f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.756+0000 7f474ef9a700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4748102dc0 0x7f4748070260 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.756+0000 7f474ef9a700 1 --2- 172.21.15.33:0/4127795135 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4748103760 0x7f47480707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.756+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 >> 172.21.15.33:0/4127795135 conn(0x7f47480f8930 msgr2=0x7f47480f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.756+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 shutdown_connections 2024-04-25T01:46:56.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.756+0000 7f474ef9a700 1 -- 172.21.15.33:0/4127795135 wait complete. 2024-04-25T01:46:56.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 -- 172.21.15.33:0/3435002345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 msgr2=0x7f32301091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 --2- 172.21.15.33:0/3435002345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 0x7f32301091e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3224004940 tx=0x7f3224009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:56.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 -- 172.21.15.33:0/3435002345 shutdown_connections 2024-04-25T01:46:56.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 --2- 172.21.15.33:0/3435002345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 0x7f32301091e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 --2- 172.21.15.33:0/3435002345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32300fbb70 0x7f32300fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 -- 172.21.15.33:0/3435002345 >> 172.21.15.33:0/3435002345 conn(0x7f32300f7820 msgr2=0x7f32300f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:56.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 -- 172.21.15.33:0/3435002345 shutdown_connections 2024-04-25T01:46:56.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 -- 172.21.15.33:0/3435002345 wait complete. 2024-04-25T01:46:56.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 Processor -- start 2024-04-25T01:46:56.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.860+0000 7f3234e92700 1 -- start start 2024-04-25T01:46:56.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f3234e92700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32300fbb70 0x7f3230115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:56.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f3234e92700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 0x7f32301163d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f3234e92700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3230116a20 con 0x7f32300fc480 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f3234e92700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3230110000 con 0x7f32300fbb70 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f322dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 0x7f32301163d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f322dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 0x7f32301163d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41348/0 (socket says 172.21.15.33:41348) 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f322dd9b700 1 -- 172.21.15.33:0/2888757277 learned_addr learned my addr 172.21.15.33:0/2888757277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f322e59c700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32300fbb70 0x7f3230115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f322dd9b700 1 -- 172.21.15.33:0/2888757277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32300fbb70 msgr2=0x7f3230115e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f322dd9b700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32300fbb70 0x7f3230115e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f322dd9b700 1 -- 172.21.15.33:0/2888757277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3224005040 con 0x7f32300fc480 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f322e59c700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32300fbb70 0x7f3230115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f322dd9b700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 0x7f32301163d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3224009180 tx=0x7f3224009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.861+0000 7f321effd700 1 -- 172.21.15.33:0/2888757277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f322400e040 con 0x7f32300fc480 2024-04-25T01:46:56.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.862+0000 7f321effd700 1 -- 172.21.15.33:0/2888757277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f322400bbc0 con 0x7f32300fc480 2024-04-25T01:46:56.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.862+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3230110280 con 0x7f32300fc480 2024-04-25T01:46:56.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.862+0000 7f321effd700 1 -- 172.21.15.33:0/2888757277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f322401c630 con 0x7f32300fc480 2024-04-25T01:46:56.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.862+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3230110840 con 0x7f32300fc480 2024-04-25T01:46:56.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.862+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3230106910 con 0x7f32300fc480 2024-04-25T01:46:56.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.863+0000 7f321effd700 1 -- 172.21.15.33:0/2888757277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f322400bd30 con 0x7f32300fc480 2024-04-25T01:46:56.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.863+0000 7f321effd700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f321806e140 0x7f32180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:56.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.864+0000 7f321effd700 1 -- 172.21.15.33:0/2888757277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f322408f050 con 0x7f32300fc480 2024-04-25T01:46:56.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.864+0000 7f322e59c700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f321806e140 0x7f32180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:56.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.864+0000 7f322e59c700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f321806e140 0x7f32180705f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3220009fd0 tx=0x7f322000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:56.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:56.868+0000 7f321effd700 1 -- 172.21.15.33:0/2888757277 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f322405bf80 con 0x7f32300fc480 2024-04-25T01:46:57.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.085+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3230111530 con 0x7f321806e140 2024-04-25T01:46:57.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.095+0000 7f321effd700 1 -- 172.21.15.33:0/2888757277 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f3230111530 con 0x7f321806e140 2024-04-25T01:46:57.090 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:46:57.090 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (8m) 72s ago 19m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:46:57.090 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (5m) 72s ago 19m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:46:57.090 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (5m) 58s ago 18m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:46:57.090 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (7m) 72s ago 18m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:46:57.090 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (77s) 72s ago 13m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:46:57.090 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (97s) 72s ago 13m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:46:57.090 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (61s) 58s ago 13m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (13m) 58s ago 13m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (9m) 72s ago 20m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (9m) 58s ago 18m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (6m) 72s ago 20m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (6m) 58s ago 18m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (8m) 72s ago 19m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (8m) 58s ago 18m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (4m) 72s ago 17m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (4m) 72s ago 17m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (3m) 72s ago 16m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (3m) 58s ago 16m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (2m) 58s ago 15m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (116s) 58s ago 15m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:46:57.091 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (8m) 72s ago 18m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f321806e140 msgr2=0x7f32180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f321806e140 0x7f32180705f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3220009fd0 tx=0x7f322000b040 comp rx=0 tx=0).stop 2024-04-25T01:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 msgr2=0x7f32301163d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 0x7f32301163d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3224009180 tx=0x7f3224009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 shutdown_connections 2024-04-25T01:46:57.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f321806e140 0x7f32180705f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32300fbb70 0x7f3230115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 --2- 172.21.15.33:0/2888757277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32300fc480 0x7f32301163d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 >> 172.21.15.33:0/2888757277 conn(0x7f32300f7820 msgr2=0x7f32300f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:57.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 shutdown_connections 2024-04-25T01:46:57.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.097+0000 7f3234e92700 1 -- 172.21.15.33:0/2888757277 wait complete. 2024-04-25T01:46:57.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.201+0000 7f0f87e68700 1 -- 172.21.15.33:0/3323646821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 msgr2=0x7f0f801031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.201+0000 7f0f87e68700 1 --2- 172.21.15.33:0/3323646821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 0x7f0f801031b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c004940 tx=0x7f0f7c009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.201+0000 7f0f87e68700 1 -- 172.21.15.33:0/3323646821 shutdown_connections 2024-04-25T01:46:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.201+0000 7f0f87e68700 1 --2- 172.21.15.33:0/3323646821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f80103780 0x7f0f800fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.201+0000 7f0f87e68700 1 --2- 172.21.15.33:0/3323646821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 0x7f0f801031b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.201+0000 7f0f87e68700 1 -- 172.21.15.33:0/3323646821 >> 172.21.15.33:0/3323646821 conn(0x7f0f800f8990 msgr2=0x7f0f800fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.201+0000 7f0f87e68700 1 -- 172.21.15.33:0/3323646821 shutdown_connections 2024-04-25T01:46:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.201+0000 7f0f87e68700 1 -- 172.21.15.33:0/3323646821 wait complete. 2024-04-25T01:46:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f87e68700 1 Processor -- start 2024-04-25T01:46:57.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f87e68700 1 -- start start 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f87e68700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 0x7f0f801138e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f87e68700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f80103780 0x7f0f80113e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f87e68700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80114470 con 0x7f0f80102de0 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f87e68700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8010da70 con 0x7f0f80103780 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f85c04700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 0x7f0f801138e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f85c04700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 0x7f0f801138e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41366/0 (socket says 172.21.15.33:41366) 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f85c04700 1 -- 172.21.15.33:0/4293593668 learned_addr learned my addr 172.21.15.33:0/4293593668 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f85403700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f80103780 0x7f0f80113e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f85c04700 1 -- 172.21.15.33:0/4293593668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f80103780 msgr2=0x7f0f80113e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f85c04700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f80103780 0x7f0f80113e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.202+0000 7f0f85c04700 1 -- 172.21.15.33:0/4293593668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c005040 con 0x7f0f80102de0 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.203+0000 7f0f85c04700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 0x7f0f801138e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009b30 tx=0x7f0f7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:57.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.203+0000 7f0f767fc700 1 -- 172.21.15.33:0/4293593668 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7c021070 con 0x7f0f80102de0 2024-04-25T01:46:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.203+0000 7f0f767fc700 1 -- 172.21.15.33:0/4293593668 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f7c013c00 con 0x7f0f80102de0 2024-04-25T01:46:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.203+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8010dcf0 con 0x7f0f80102de0 2024-04-25T01:46:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.203+0000 7f0f767fc700 1 -- 172.21.15.33:0/4293593668 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7c00e070 con 0x7f0f80102de0 2024-04-25T01:46:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.203+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8010e1d0 con 0x7f0f80102de0 2024-04-25T01:46:57.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.204+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f801000f0 con 0x7f0f80102de0 2024-04-25T01:46:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.204+0000 7f0f767fc700 1 -- 172.21.15.33:0/4293593668 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0f7c01cad0 con 0x7f0f80102de0 2024-04-25T01:46:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.205+0000 7f0f767fc700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f6c06e140 0x7f0f6c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.205+0000 7f0f767fc700 1 -- 172.21.15.33:0/4293593668 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0f7c08ef50 con 0x7f0f80102de0 2024-04-25T01:46:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.205+0000 7f0f85403700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f6c06e140 0x7f0f6c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:57.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.205+0000 7f0f85403700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f6c06e140 0x7f0f6c0705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0f8010ec30 tx=0x7f0f7000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:57.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.210+0000 7f0f767fc700 1 -- 172.21.15.33:0/4293593668 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0f7c0c3740 con 0x7f0f80102de0 2024-04-25T01:46:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.506+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f8010ef30 con 0x7f0f80102de0 2024-04-25T01:46:57.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.507+0000 7f0f767fc700 1 -- 172.21.15.33:0/4293593668 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f0f7c05be80 con 0x7f0f80102de0 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:46:57.502 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:46:57.503 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:46:57.503 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:46:57.503 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:46:57.503 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:46:57.503 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:46:57.503 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:46:57.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f6c06e140 msgr2=0x7f0f6c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f6c06e140 0x7f0f6c0705f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0f8010ec30 tx=0x7f0f7000b040 comp rx=0 tx=0).stop 2024-04-25T01:46:57.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 msgr2=0x7f0f801138e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 0x7f0f801138e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009b30 tx=0x7f0f7c009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 shutdown_connections 2024-04-25T01:46:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f6c06e140 0x7f0f6c0705f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f80102de0 0x7f0f801138e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 --2- 172.21.15.33:0/4293593668 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f80103780 0x7f0f80113e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 >> 172.21.15.33:0/4293593668 conn(0x7f0f800f8990 msgr2=0x7f0f800fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 shutdown_connections 2024-04-25T01:46:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.509+0000 7f0f87e68700 1 -- 172.21.15.33:0/4293593668 wait complete. 2024-04-25T01:46:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.614+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/2370052611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e80fbd40 msgr2=0x7fb6e80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.614+0000 7fb6ee2dd700 1 --2- 172.21.15.33:0/2370052611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e80fbd40 0x7fb6e80fc1b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0004940 tx=0x7fb6d0009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.614+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/2370052611 shutdown_connections 2024-04-25T01:46:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.614+0000 7fb6ee2dd700 1 --2- 172.21.15.33:0/2370052611 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e80fbd40 0x7fb6e80fc1b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.614+0000 7fb6ee2dd700 1 --2- 172.21.15.33:0/2370052611 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e8101d90 0x7fb6e8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.614+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/2370052611 >> 172.21.15.33:0/2370052611 conn(0x7fb6e8071d00 msgr2=0x7fb6e8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.614+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/2370052611 shutdown_connections 2024-04-25T01:46:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.614+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/2370052611 wait complete. 2024-04-25T01:46:57.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.614+0000 7fb6ee2dd700 1 Processor -- start 2024-04-25T01:46:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6ee2dd700 1 -- start start 2024-04-25T01:46:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6ee2dd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e80fbd40 0x7fb6e810c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6ee2dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e8101d90 0x7fb6e810ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6ee2dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e810d3a0 con 0x7fb6e8101d90 2024-04-25T01:46:57.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6ee2dd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e810d510 con 0x7fb6e80fbd40 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e8101d90 0x7fb6e810ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6e77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e8101d90 0x7fb6e810ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41390/0 (socket says 172.21.15.33:41390) 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6e77fe700 1 -- 172.21.15.33:0/3771617320 learned_addr learned my addr 172.21.15.33:0/3771617320 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6e7fff700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e80fbd40 0x7fb6e810c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6e77fe700 1 -- 172.21.15.33:0/3771617320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e80fbd40 msgr2=0x7fb6e810c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6e77fe700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e80fbd40 0x7fb6e810c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.615+0000 7fb6e77fe700 1 -- 172.21.15.33:0/3771617320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0005040 con 0x7fb6e8101d90 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.616+0000 7fb6e77fe700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e8101d90 0x7fb6e810ce60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000b3b0 tx=0x7fb6d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.616+0000 7fb6e4ff9700 1 -- 172.21.15.33:0/3771617320 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d000e040 con 0x7fb6e8101d90 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.616+0000 7fb6e4ff9700 1 -- 172.21.15.33:0/3771617320 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6d0013420 con 0x7fb6e8101d90 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.616+0000 7fb6e4ff9700 1 -- 172.21.15.33:0/3771617320 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6d001c630 con 0x7fb6e8101d90 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.616+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e810d790 con 0x7fb6e8101d90 2024-04-25T01:46:57.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.616+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e811a2e0 con 0x7fb6e8101d90 2024-04-25T01:46:57.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.617+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e80fd430 con 0x7fb6e8101d90 2024-04-25T01:46:57.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.617+0000 7fb6e4ff9700 1 -- 172.21.15.33:0/3771617320 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb6d0013590 con 0x7fb6e8101d90 2024-04-25T01:46:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.618+0000 7fb6e4ff9700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6d406e140 0x7fb6d40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.618+0000 7fb6e4ff9700 1 -- 172.21.15.33:0/3771617320 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb6d008ebb0 con 0x7fb6e8101d90 2024-04-25T01:46:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.618+0000 7fb6e7fff700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6d406e140 0x7fb6d40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:57.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.618+0000 7fb6e7fff700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6d406e140 0x7fb6d40705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8004f40 tx=0x7fb6d800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:57.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.623+0000 7fb6e4ff9700 1 -- 172.21.15.33:0/3771617320 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb6d0095080 con 0x7fb6e8101d90 2024-04-25T01:46:57.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:57 smithi033 ceph-mon[111306]: pgmap v226: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:57.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:57 smithi033 ceph-mon[111306]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:57.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:57 smithi033 ceph-mon[111306]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:57.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:57 smithi033 ceph-mon[111306]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:57.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:57 smithi138 ceph-mon[81304]: pgmap v226: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:57.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:57 smithi138 ceph-mon[81304]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:57.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:57 smithi138 ceph-mon[81304]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:57.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:57 smithi138 ceph-mon[81304]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.876+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb6e80fac30 con 0x7fb6e8101d90 2024-04-25T01:46:57.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.876+0000 7fb6e4ff9700 1 -- 172.21.15.33:0/3771617320 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fb6d005bae0 con 0x7fb6e8101d90 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:46:57.872 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:46:57.873 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:46:57.874 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:46:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.878+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6d406e140 msgr2=0x7fb6d40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.878+0000 7fb6ee2dd700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6d406e140 0x7fb6d40705f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb6d8004f40 tx=0x7fb6d800b040 comp rx=0 tx=0).stop 2024-04-25T01:46:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.878+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e8101d90 msgr2=0x7fb6e810ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.878+0000 7fb6ee2dd700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e8101d90 0x7fb6e810ce60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6d000b3b0 tx=0x7fb6d0009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.878+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 shutdown_connections 2024-04-25T01:46:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.879+0000 7fb6ee2dd700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6d406e140 0x7fb6d40705f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.879+0000 7fb6ee2dd700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e80fbd40 0x7fb6e810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.879+0000 7fb6ee2dd700 1 --2- 172.21.15.33:0/3771617320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e8101d90 0x7fb6e810ce60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.879+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 >> 172.21.15.33:0/3771617320 conn(0x7fb6e8071d00 msgr2=0x7fb6e8104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.879+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 shutdown_connections 2024-04-25T01:46:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.879+0000 7fb6ee2dd700 1 -- 172.21.15.33:0/3771617320 wait complete. 2024-04-25T01:46:57.876 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:46:57.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.984+0000 7eff7a17b700 1 -- 172.21.15.33:0/3204346300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74103790 msgr2=0x7eff740fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.984+0000 7eff7a17b700 1 --2- 172.21.15.33:0/3204346300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74103790 0x7eff740fce00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7eff68004940 tx=0x7eff68009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.984+0000 7eff7a17b700 1 -- 172.21.15.33:0/3204346300 shutdown_connections 2024-04-25T01:46:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.984+0000 7eff7a17b700 1 --2- 172.21.15.33:0/3204346300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74103790 0x7eff740fce00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.984+0000 7eff7a17b700 1 --2- 172.21.15.33:0/3204346300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74102df0 0x7eff741031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.984+0000 7eff7a17b700 1 -- 172.21.15.33:0/3204346300 >> 172.21.15.33:0/3204346300 conn(0x7eff740f8980 msgr2=0x7eff740fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.984+0000 7eff7a17b700 1 -- 172.21.15.33:0/3204346300 shutdown_connections 2024-04-25T01:46:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.984+0000 7eff7a17b700 1 -- 172.21.15.33:0/3204346300 wait complete. 2024-04-25T01:46:57.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff7a17b700 1 Processor -- start 2024-04-25T01:46:57.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff7a17b700 1 -- start start 2024-04-25T01:46:57.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff7a17b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74102df0 0x7eff74070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:57.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff7a17b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74103790 0x7eff740707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:57.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff7a17b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff74070d30 con 0x7eff74102df0 2024-04-25T01:46:57.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff7a17b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff74070e70 con 0x7eff74103790 2024-04-25T01:46:57.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74102df0 0x7eff74070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:57.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74102df0 0x7eff74070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41400/0 (socket says 172.21.15.33:41400) 2024-04-25T01:46:57.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff72ffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74103790 0x7eff740707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.985+0000 7eff737fe700 1 -- 172.21.15.33:0/2568689763 learned_addr learned my addr 172.21.15.33:0/2568689763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.986+0000 7eff737fe700 1 -- 172.21.15.33:0/2568689763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74103790 msgr2=0x7eff740707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.986+0000 7eff737fe700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74103790 0x7eff740707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.986+0000 7eff737fe700 1 -- 172.21.15.33:0/2568689763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff68005040 con 0x7eff74102df0 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.986+0000 7eff737fe700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74102df0 0x7eff74070260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c8c0 tx=0x7eff640044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.986+0000 7eff5bfff700 1 -- 172.21.15.33:0/2568689763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff6400a760 con 0x7eff74102df0 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.986+0000 7eff5bfff700 1 -- 172.21.15.33:0/2568689763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff6400ad90 con 0x7eff74102df0 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.986+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff74119e80 con 0x7eff74102df0 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.986+0000 7eff5bfff700 1 -- 172.21.15.33:0/2568689763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff640113a0 con 0x7eff74102df0 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.986+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff7411a300 con 0x7eff74102df0 2024-04-25T01:46:57.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.987+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff740fe540 con 0x7eff74102df0 2024-04-25T01:46:57.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.988+0000 7eff5bfff700 1 -- 172.21.15.33:0/2568689763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7eff64012400 con 0x7eff74102df0 2024-04-25T01:46:57.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.988+0000 7eff5bfff700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff5c06e140 0x7eff5c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:57.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.988+0000 7eff5bfff700 1 -- 172.21.15.33:0/2568689763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7eff6408db70 con 0x7eff74102df0 2024-04-25T01:46:57.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.988+0000 7eff72ffd700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff5c06e140 0x7eff5c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:57.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.989+0000 7eff72ffd700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff5c06e140 0x7eff5c0705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff74071ba0 tx=0x7eff68009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:57.988 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:57.993+0000 7eff5bfff700 1 -- 172.21.15.33:0/2568689763 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7eff64056f70 con 0x7eff74102df0 2024-04-25T01:46:58.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.222+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff7404bf40 con 0x7eff5c06e140 2024-04-25T01:46:58.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.223+0000 7eff5bfff700 1 -- 172.21.15.33:0/2568689763 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7eff7404bf40 con 0x7eff5c06e140 2024-04-25T01:46:58.218 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:46:58.218 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:46:58.218 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:46:58.218 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:46:58.218 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:46:58.219 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:46:58.219 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:46:58.219 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:46:58.219 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:46:58.219 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:46:58.219 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:46:58.219 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:46:58.219 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:46:58.219 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:46:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff5c06e140 msgr2=0x7eff5c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff5c06e140 0x7eff5c0705f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7eff74071ba0 tx=0x7eff68009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74102df0 msgr2=0x7eff74070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74102df0 0x7eff74070260 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7eff6400c8c0 tx=0x7eff640044c0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 shutdown_connections 2024-04-25T01:46:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff5c06e140 0x7eff5c0705f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff74102df0 0x7eff74070260 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 --2- 172.21.15.33:0/2568689763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff74103790 0x7eff740707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 >> 172.21.15.33:0/2568689763 conn(0x7eff740f8980 msgr2=0x7eff740f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 shutdown_connections 2024-04-25T01:46:58.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.226+0000 7eff7a17b700 1 -- 172.21.15.33:0/2568689763 wait complete. 2024-04-25T01:46:58.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.331+0000 7feb25e3a700 1 -- 172.21.15.33:0/951270056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb200fbd40 msgr2=0x7feb200fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:58.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.331+0000 7feb25e3a700 1 --2- 172.21.15.33:0/951270056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb200fbd40 0x7feb200fc1b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb14004940 tx=0x7feb14009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.331+0000 7feb25e3a700 1 -- 172.21.15.33:0/951270056 shutdown_connections 2024-04-25T01:46:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.331+0000 7feb25e3a700 1 --2- 172.21.15.33:0/951270056 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb200fbd40 0x7feb200fc1b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.331+0000 7feb25e3a700 1 --2- 172.21.15.33:0/951270056 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb20101d90 0x7feb20102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.331+0000 7feb25e3a700 1 -- 172.21.15.33:0/951270056 >> 172.21.15.33:0/951270056 conn(0x7feb20071d00 msgr2=0x7feb20072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.331+0000 7feb25e3a700 1 -- 172.21.15.33:0/951270056 shutdown_connections 2024-04-25T01:46:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.331+0000 7feb25e3a700 1 -- 172.21.15.33:0/951270056 wait complete. 2024-04-25T01:46:58.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb25e3a700 1 Processor -- start 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb25e3a700 1 -- start start 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb25e3a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb200fbd40 0x7feb20115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb25e3a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb20101d90 0x7feb201162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb25e3a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb201169b0 con 0x7feb20101d90 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb25e3a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2010fcf0 con 0x7feb200fbd40 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb1effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb20101d90 0x7feb201162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb1effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb20101d90 0x7feb201162d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41424/0 (socket says 172.21.15.33:41424) 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb1effd700 1 -- 172.21.15.33:0/1245310298 learned_addr learned my addr 172.21.15.33:0/1245310298 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.332+0000 7feb1f7fe700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb200fbd40 0x7feb20115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.333+0000 7feb1effd700 1 -- 172.21.15.33:0/1245310298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb200fbd40 msgr2=0x7feb20115d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.333+0000 7feb1effd700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb200fbd40 0x7feb20115d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.333+0000 7feb1effd700 1 -- 172.21.15.33:0/1245310298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb14005040 con 0x7feb20101d90 2024-04-25T01:46:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.333+0000 7feb1effd700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb20101d90 0x7feb201162d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb1400b8b0 tx=0x7feb14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.333+0000 7feb07fff700 1 -- 172.21.15.33:0/1245310298 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb1400e040 con 0x7feb20101d90 2024-04-25T01:46:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.333+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2010ff10 con 0x7feb20101d90 2024-04-25T01:46:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.333+0000 7feb07fff700 1 -- 172.21.15.33:0/1245310298 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb14004290 con 0x7feb20101d90 2024-04-25T01:46:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.333+0000 7feb07fff700 1 -- 172.21.15.33:0/1245310298 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb1401c3f0 con 0x7feb20101d90 2024-04-25T01:46:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.333+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb201104d0 con 0x7feb20101d90 2024-04-25T01:46:58.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.334+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb200fd430 con 0x7feb20101d90 2024-04-25T01:46:58.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.335+0000 7feb07fff700 1 -- 172.21.15.33:0/1245310298 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feb14013820 con 0x7feb20101d90 2024-04-25T01:46:58.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.335+0000 7feb07fff700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb0806e140 0x7feb080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:46:58.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.335+0000 7feb07fff700 1 -- 172.21.15.33:0/1245310298 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7feb1408ed90 con 0x7feb20101d90 2024-04-25T01:46:58.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.335+0000 7feb1f7fe700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb0806e140 0x7feb080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:46:58.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.336+0000 7feb1f7fe700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb0806e140 0x7feb080705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb10009bf0 tx=0x7feb100091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:46:58.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.340+0000 7feb07fff700 1 -- 172.21.15.33:0/1245310298 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feb1405bcc0 con 0x7feb20101d90 2024-04-25T01:46:58.510 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:58 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4293593668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:46:58.511 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:58 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3771617320' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:46:58.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.632+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feb2005f5c0 con 0x7feb20101d90 2024-04-25T01:46:58.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.632+0000 7feb07fff700 1 -- 172.21.15.33:0/1245310298 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7feb1405b850 con 0x7feb20101d90 2024-04-25T01:46:58.628 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:46:58.628 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:46:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:46:58.628 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:46:58.628 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:46:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.634+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb0806e140 msgr2=0x7feb080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:58.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.634+0000 7feb25e3a700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb0806e140 0x7feb080705f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7feb10009bf0 tx=0x7feb100091a0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.634+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb20101d90 msgr2=0x7feb201162d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:46:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.634+0000 7feb25e3a700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb20101d90 0x7feb201162d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7feb1400b8b0 tx=0x7feb14009e30 comp rx=0 tx=0).stop 2024-04-25T01:46:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.635+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 shutdown_connections 2024-04-25T01:46:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.635+0000 7feb25e3a700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb0806e140 0x7feb080705f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.635+0000 7feb25e3a700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb200fbd40 0x7feb20115d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.635+0000 7feb25e3a700 1 --2- 172.21.15.33:0/1245310298 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb20101d90 0x7feb201162d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:46:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.635+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 >> 172.21.15.33:0/1245310298 conn(0x7feb20071d00 msgr2=0x7feb200fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:46:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.635+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 shutdown_connections 2024-04-25T01:46:58.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:46:58.635+0000 7feb25e3a700 1 -- 172.21.15.33:0/1245310298 wait complete. 2024-04-25T01:46:58.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:58 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4293593668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:46:58.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:58 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3771617320' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:46:59.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:59 smithi033 ceph-mon[111306]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:59.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:59 smithi033 ceph-mon[111306]: pgmap v227: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:59.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:46:59 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1245310298' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:46:59.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:59 smithi138 ceph-mon[81304]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:46:59.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:59 smithi138 ceph-mon[81304]: pgmap v227: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:46:59.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:46:59 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1245310298' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:47:01.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:01 smithi033 ceph-mon[111306]: pgmap v228: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:01.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:01 smithi138 ceph-mon[81304]: pgmap v228: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:03.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:03 smithi033 ceph-mon[111306]: pgmap v229: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:03.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:03 smithi138 ceph-mon[81304]: pgmap v229: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:05.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:05 smithi033 ceph-mon[111306]: pgmap v230: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:05.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:05 smithi138 ceph-mon[81304]: pgmap v230: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:07.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:07 smithi033 ceph-mon[111306]: pgmap v231: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:07.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:07 smithi138 ceph-mon[81304]: pgmap v231: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:09.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:09 smithi033 ceph-mon[111306]: pgmap v232: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:09.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:09 smithi138 ceph-mon[81304]: pgmap v232: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:10.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:47:10.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:47:11.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:11 smithi033 ceph-mon[111306]: pgmap v233: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:11.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:11 smithi138 ceph-mon[81304]: pgmap v233: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:13.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:13 smithi033 ceph-mon[111306]: pgmap v234: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:13.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:13 smithi138 ceph-mon[81304]: pgmap v234: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:15.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:15 smithi033 ceph-mon[111306]: pgmap v235: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:15.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:15 smithi138 ceph-mon[81304]: pgmap v235: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:17.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:17 smithi033 ceph-mon[111306]: pgmap v236: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:17.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:17 smithi138 ceph-mon[81304]: pgmap v236: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:19.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:19 smithi033 ceph-mon[111306]: pgmap v237: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:19.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:19 smithi138 ceph-mon[81304]: pgmap v237: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:21.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:21 smithi033 ceph-mon[111306]: pgmap v238: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:21.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:21 smithi138 ceph-mon[81304]: pgmap v238: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:23.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:23 smithi033 ceph-mon[111306]: pgmap v239: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:23.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:23 smithi138 ceph-mon[81304]: pgmap v239: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:25.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:25 smithi033 ceph-mon[111306]: pgmap v240: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:25.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:47:25.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:25 smithi138 ceph-mon[81304]: pgmap v240: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:25.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:47:27.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:27 smithi033 ceph-mon[111306]: pgmap v241: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:27.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:27 smithi138 ceph-mon[81304]: pgmap v241: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:28.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.754+0000 7fb57084f700 1 -- 172.21.15.33:0/680695455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 msgr2=0x7fb56c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.754+0000 7fb57084f700 1 --2- 172.21.15.33:0/680695455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 0x7fb56c106e40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb560004940 tx=0x7fb560009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.754+0000 7fb57084f700 1 -- 172.21.15.33:0/680695455 shutdown_connections 2024-04-25T01:47:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.754+0000 7fb57084f700 1 --2- 172.21.15.33:0/680695455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 0x7fb56c106e40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.754+0000 7fb57084f700 1 --2- 172.21.15.33:0/680695455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb56c0fde60 0x7fb56c0fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.754+0000 7fb57084f700 1 -- 172.21.15.33:0/680695455 >> 172.21.15.33:0/680695455 conn(0x7fb56c0f7870 msgr2=0x7fb56c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.754+0000 7fb57084f700 1 -- 172.21.15.33:0/680695455 shutdown_connections 2024-04-25T01:47:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.754+0000 7fb57084f700 1 -- 172.21.15.33:0/680695455 wait complete. 2024-04-25T01:47:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb57084f700 1 Processor -- start 2024-04-25T01:47:28.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb57084f700 1 -- start start 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb57084f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb56c0fde60 0x7fb56c0fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb57084f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 0x7fb56c0fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb57084f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c0fdbe0 con 0x7fb56c0fe800 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb57084f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56c0fc710 con 0x7fb56c0fde60 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb569d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 0x7fb56c0fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb569d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 0x7fb56c0fc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32880/0 (socket says 172.21.15.33:32880) 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb569d9b700 1 -- 172.21.15.33:0/2336163690 learned_addr learned my addr 172.21.15.33:0/2336163690 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.755+0000 7fb56a59c700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb56c0fde60 0x7fb56c0fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.756+0000 7fb569d9b700 1 -- 172.21.15.33:0/2336163690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb56c0fde60 msgr2=0x7fb56c0fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.756+0000 7fb569d9b700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb56c0fde60 0x7fb56c0fbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:28.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.756+0000 7fb569d9b700 1 -- 172.21.15.33:0/2336163690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb560005040 con 0x7fb56c0fe800 2024-04-25T01:47:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.756+0000 7fb569d9b700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 0x7fb56c0fc1d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb560009380 tx=0x7fb560009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.756+0000 7fb55affd700 1 -- 172.21.15.33:0/2336163690 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56000e040 con 0x7fb56c0fe800 2024-04-25T01:47:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.756+0000 7fb55affd700 1 -- 172.21.15.33:0/2336163690 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb560009af0 con 0x7fb56c0fe800 2024-04-25T01:47:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.756+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56c0fc9f0 con 0x7fb56c0fe800 2024-04-25T01:47:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.756+0000 7fb55affd700 1 -- 172.21.15.33:0/2336163690 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb56001c630 con 0x7fb56c0fe800 2024-04-25T01:47:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.756+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56c11a070 con 0x7fb56c0fe800 2024-04-25T01:47:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.757+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56c066ad0 con 0x7fb56c0fe800 2024-04-25T01:47:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.758+0000 7fb55affd700 1 -- 172.21.15.33:0/2336163690 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb56000bbc0 con 0x7fb56c0fe800 2024-04-25T01:47:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.758+0000 7fb55affd700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb55406e140 0x7fb5540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.758+0000 7fb55affd700 1 -- 172.21.15.33:0/2336163690 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb56008ee00 con 0x7fb56c0fe800 2024-04-25T01:47:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.758+0000 7fb56a59c700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb55406e140 0x7fb5540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.759+0000 7fb56a59c700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb55406e140 0x7fb5540705f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb55c007fd0 tx=0x7fb55c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:28.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.763+0000 7fb55affd700 1 -- 172.21.15.33:0/2336163690 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb560058200 con 0x7fb56c0fe800 2024-04-25T01:47:28.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.988+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb56c101dd0 con 0x7fb55406e140 2024-04-25T01:47:28.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.989+0000 7fb55affd700 1 -- 172.21.15.33:0/2336163690 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb56c101dd0 con 0x7fb55406e140 2024-04-25T01:47:28.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb55406e140 msgr2=0x7fb5540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb55406e140 0x7fb5540705f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb55c007fd0 tx=0x7fb55c009040 comp rx=0 tx=0).stop 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 msgr2=0x7fb56c0fc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 0x7fb56c0fc1d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb560009380 tx=0x7fb560009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 shutdown_connections 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb55406e140 0x7fb5540705f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb56c0fde60 0x7fb56c0fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 --2- 172.21.15.33:0/2336163690 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb56c0fe800 0x7fb56c0fc1d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 >> 172.21.15.33:0/2336163690 conn(0x7fb56c0f7870 msgr2=0x7fb56c1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 shutdown_connections 2024-04-25T01:47:28.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:28.991+0000 7fb57084f700 1 -- 172.21.15.33:0/2336163690 wait complete. 2024-04-25T01:47:28.997 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:47:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.096+0000 7f086c10f700 1 -- 172.21.15.33:0/1928153490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fde60 msgr2=0x7f08640fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.096+0000 7f086c10f700 1 --2- 172.21.15.33:0/1928153490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fde60 0x7f08640fe230 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0860004940 tx=0x7f0860009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.096+0000 7f086c10f700 1 -- 172.21.15.33:0/1928153490 shutdown_connections 2024-04-25T01:47:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.096+0000 7f086c10f700 1 --2- 172.21.15.33:0/1928153490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08640fe800 0x7f0864106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.096+0000 7f086c10f700 1 --2- 172.21.15.33:0/1928153490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fde60 0x7f08640fe230 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.096+0000 7f086c10f700 1 -- 172.21.15.33:0/1928153490 >> 172.21.15.33:0/1928153490 conn(0x7f08640f7870 msgr2=0x7f08640f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f086c10f700 1 -- 172.21.15.33:0/1928153490 shutdown_connections 2024-04-25T01:47:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f086c10f700 1 -- 172.21.15.33:0/1928153490 wait complete. 2024-04-25T01:47:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f086c10f700 1 Processor -- start 2024-04-25T01:47:29.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f086c10f700 1 -- start start 2024-04-25T01:47:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f086c10f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08640fde60 0x7f08641138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:29.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f086c10f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fe800 0x7f0864113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f086c10f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0864114480 con 0x7f08640fe800 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f086c10f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086410da80 con 0x7f08640fde60 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f08696aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fe800 0x7f0864113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.097+0000 7f08696aa700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fe800 0x7f0864113e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32890/0 (socket says 172.21.15.33:32890) 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f08696aa700 1 -- 172.21.15.33:0/2689680063 learned_addr learned my addr 172.21.15.33:0/2689680063 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f0869eab700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08640fde60 0x7f08641138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f08696aa700 1 -- 172.21.15.33:0/2689680063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08640fde60 msgr2=0x7f08641138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f08696aa700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08640fde60 0x7f08641138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f08696aa700 1 -- 172.21.15.33:0/2689680063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0860005040 con 0x7f08640fe800 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f08696aa700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fe800 0x7f0864113e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f085400c830 tx=0x7f0854004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f085a7fc700 1 -- 172.21.15.33:0/2689680063 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08540106f0 con 0x7f08640fe800 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f085a7fc700 1 -- 172.21.15.33:0/2689680063 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0854010d20 con 0x7f08640fe800 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086410dd60 con 0x7f08640fe800 2024-04-25T01:47:29.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f085a7fc700 1 -- 172.21.15.33:0/2689680063 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08540113a0 con 0x7f08640fe800 2024-04-25T01:47:29.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.098+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086410e2a0 con 0x7f08640fe800 2024-04-25T01:47:29.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.099+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0864066ad0 con 0x7f08640fe800 2024-04-25T01:47:29.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.100+0000 7f085a7fc700 1 -- 172.21.15.33:0/2689680063 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0854012400 con 0x7f08640fe800 2024-04-25T01:47:29.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.100+0000 7f085a7fc700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f085006e140 0x7f08500705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:29.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.100+0000 7f085a7fc700 1 -- 172.21.15.33:0/2689680063 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f085408d9b0 con 0x7f08640fe800 2024-04-25T01:47:29.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.100+0000 7f0869eab700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f085006e140 0x7f08500705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:29.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.101+0000 7f0869eab700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f085006e140 0x7f08500705f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0860004f90 tx=0x7f0860009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:29.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.105+0000 7f085a7fc700 1 -- 172.21.15.33:0/2689680063 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f085405a980 con 0x7f08640fe800 2024-04-25T01:47:29.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.330+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0864109e20 con 0x7f085006e140 2024-04-25T01:47:29.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.331+0000 7f085a7fc700 1 -- 172.21.15.33:0/2689680063 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0864109e20 con 0x7f085006e140 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f085006e140 msgr2=0x7f08500705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f085006e140 0x7f08500705f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0860004f90 tx=0x7f0860009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fe800 msgr2=0x7f0864113e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fe800 0x7f0864113e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f085400c830 tx=0x7f0854004470 comp rx=0 tx=0).stop 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 shutdown_connections 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f085006e140 0x7f08500705f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08640fde60 0x7f08641138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 --2- 172.21.15.33:0/2689680063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08640fe800 0x7f0864113e30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 >> 172.21.15.33:0/2689680063 conn(0x7f08640f7870 msgr2=0x7f08640f8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:29.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 shutdown_connections 2024-04-25T01:47:29.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.333+0000 7f086c10f700 1 -- 172.21.15.33:0/2689680063 wait complete. 2024-04-25T01:47:29.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 -- 172.21.15.33:0/1855740182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 msgr2=0x7fabc01020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 --2- 172.21.15.33:0/1855740182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 0x7fabc01020b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fabb0004940 tx=0x7fabb0009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 -- 172.21.15.33:0/1855740182 shutdown_connections 2024-04-25T01:47:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 --2- 172.21.15.33:0/1855740182 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc0102680 0x7fabc00fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 --2- 172.21.15.33:0/1855740182 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 0x7fabc01020b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 -- 172.21.15.33:0/1855740182 >> 172.21.15.33:0/1855740182 conn(0x7fabc00f7850 msgr2=0x7fabc00f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 -- 172.21.15.33:0/1855740182 shutdown_connections 2024-04-25T01:47:29.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 -- 172.21.15.33:0/1855740182 wait complete. 2024-04-25T01:47:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 Processor -- start 2024-04-25T01:47:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.438+0000 7fabc5725700 1 -- start start 2024-04-25T01:47:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabc5725700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 0x7fabc010ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabc5725700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc0102680 0x7fabc010efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabc5725700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc010f620 con 0x7fabc0101ce0 2024-04-25T01:47:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabc5725700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc010f760 con 0x7fabc0102680 2024-04-25T01:47:29.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabbeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 0x7fabc010ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabbeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 0x7fabc010ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32908/0 (socket says 172.21.15.33:32908) 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabbeffd700 1 -- 172.21.15.33:0/2262076919 learned_addr learned my addr 172.21.15.33:0/2262076919 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabbe7fc700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc0102680 0x7fabc010efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabbeffd700 1 -- 172.21.15.33:0/2262076919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc0102680 msgr2=0x7fabc010efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabbeffd700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc0102680 0x7fabc010efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabbeffd700 1 -- 172.21.15.33:0/2262076919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0005040 con 0x7fabc0101ce0 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.439+0000 7fabbeffd700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 0x7fabc010ea90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009b30 tx=0x7fabb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.440+0000 7faba77fe700 1 -- 172.21.15.33:0/2262076919 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb0021070 con 0x7fabc0101ce0 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.440+0000 7faba77fe700 1 -- 172.21.15.33:0/2262076919 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabb0013c00 con 0x7fabc0101ce0 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.440+0000 7faba77fe700 1 -- 172.21.15.33:0/2262076919 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabb000e070 con 0x7fabc0101ce0 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.440+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc0117b00 con 0x7fabc0101ce0 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.440+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc0117f60 con 0x7fabc0101ce0 2024-04-25T01:47:29.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.441+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc00fd430 con 0x7fabc0101ce0 2024-04-25T01:47:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.441+0000 7faba77fe700 1 -- 172.21.15.33:0/2262076919 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fabb001cad0 con 0x7fabc0101ce0 2024-04-25T01:47:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.442+0000 7faba77fe700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faba806e140 0x7faba80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.442+0000 7faba77fe700 1 -- 172.21.15.33:0/2262076919 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fabb008ee50 con 0x7fabc0101ce0 2024-04-25T01:47:29.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.442+0000 7fabbe7fc700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faba806e140 0x7faba80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:29.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.442+0000 7fabbe7fc700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faba806e140 0x7faba80705f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fabc0066870 tx=0x7fabb400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:29.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.446+0000 7faba77fe700 1 -- 172.21.15.33:0/2262076919 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fabb00582f0 con 0x7fabc0101ce0 2024-04-25T01:47:29.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.663+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fabc0066620 con 0x7faba806e140 2024-04-25T01:47:29.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.673+0000 7faba77fe700 1 -- 172.21.15.33:0/2262076919 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fabc0066620 con 0x7faba806e140 2024-04-25T01:47:29.668 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (8m) 105s ago 19m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (5m) 105s ago 20m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (5m) 90s ago 19m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (8m) 105s ago 19m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (110s) 105s ago 14m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (2m) 105s ago 14m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (94s) 90s ago 14m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (14m) 90s ago 14m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (10m) 105s ago 21m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (9m) 90s ago 18m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (6m) 105s ago 21m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (6m) 90s ago 18m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (9m) 105s ago 19m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (9m) 90s ago 18m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:47:29.669 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (5m) 105s ago 18m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:47:29.670 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (4m) 105s ago 17m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:47:29.670 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (4m) 105s ago 17m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:47:29.670 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (3m) 90s ago 16m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:47:29.670 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (3m) 90s ago 16m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:47:29.670 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (2m) 90s ago 15m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:47:29.670 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (8m) 105s ago 19m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:47:29.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faba806e140 msgr2=0x7faba80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faba806e140 0x7faba80705f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fabc0066870 tx=0x7fabb400b040 comp rx=0 tx=0).stop 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 msgr2=0x7fabc010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 0x7fabc010ea90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fabb0009b30 tx=0x7fabb0009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 shutdown_connections 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faba806e140 0x7faba80705f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fabc0101ce0 0x7fabc010ea90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 --2- 172.21.15.33:0/2262076919 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fabc0102680 0x7fabc010efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 >> 172.21.15.33:0/2262076919 conn(0x7fabc00f7850 msgr2=0x7fabc00f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 shutdown_connections 2024-04-25T01:47:29.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.675+0000 7fabc5725700 1 -- 172.21.15.33:0/2262076919 wait complete. 2024-04-25T01:47:29.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:29 smithi033 ceph-mon[111306]: pgmap v242: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:29.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:29 smithi033 ceph-mon[111306]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:47:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.778+0000 7fb9b917b700 1 -- 172.21.15.33:0/2351390055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b4102f60 msgr2=0x7fb9b4103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.778+0000 7fb9b917b700 1 --2- 172.21.15.33:0/2351390055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b4102f60 0x7fb9b4103330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4004940 tx=0x7fb9a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.778+0000 7fb9b917b700 1 -- 172.21.15.33:0/2351390055 shutdown_connections 2024-04-25T01:47:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.778+0000 7fb9b917b700 1 --2- 172.21.15.33:0/2351390055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b40fb390 0x7fb9b40fb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.778+0000 7fb9b917b700 1 --2- 172.21.15.33:0/2351390055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b4102f60 0x7fb9b4103330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.778+0000 7fb9b917b700 1 -- 172.21.15.33:0/2351390055 >> 172.21.15.33:0/2351390055 conn(0x7fb9b4073350 msgr2=0x7fb9b4073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:29.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.778+0000 7fb9b917b700 1 -- 172.21.15.33:0/2351390055 shutdown_connections 2024-04-25T01:47:29.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.778+0000 7fb9b917b700 1 -- 172.21.15.33:0/2351390055 wait complete. 2024-04-25T01:47:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b917b700 1 Processor -- start 2024-04-25T01:47:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b917b700 1 -- start start 2024-04-25T01:47:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b917b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b40fb390 0x7fb9b40fc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b917b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102f60 0x7fb9b40fcab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b917b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40fe550 con 0x7fb9b40fb390 2024-04-25T01:47:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b917b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b40fcff0 con 0x7fb9b4102f60 2024-04-25T01:47:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b40fb390 0x7fb9b40fc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b2d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b40fb390 0x7fb9b40fc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32928/0 (socket says 172.21.15.33:32928) 2024-04-25T01:47:29.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b2d9d700 1 -- 172.21.15.33:0/1675879215 learned_addr learned my addr 172.21.15.33:0/1675879215 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.779+0000 7fb9b259c700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102f60 0x7fb9b40fcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb9b2d9d700 1 -- 172.21.15.33:0/1675879215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102f60 msgr2=0x7fb9b40fcab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb9b2d9d700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102f60 0x7fb9b40fcab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb9b2d9d700 1 -- 172.21.15.33:0/1675879215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a4005040 con 0x7fb9b40fb390 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb9b259c700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102f60 0x7fb9b40fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb9b2d9d700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b40fb390 0x7fb9b40fc570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b3b0 tx=0x7fb9a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb99b7fe700 1 -- 172.21.15.33:0/1675879215 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a400e040 con 0x7fb9b40fb390 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb99b7fe700 1 -- 172.21.15.33:0/1675879215 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9a400bbc0 con 0x7fb9b40fb390 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b40fd270 con 0x7fb9b40fb390 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb99b7fe700 1 -- 172.21.15.33:0/1675879215 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9a401c630 con 0x7fb9b40fb390 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.780+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b411a1b0 con 0x7fb9b40fb390 2024-04-25T01:47:29.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.781+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b40fe6b0 con 0x7fb9b40fb390 2024-04-25T01:47:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.782+0000 7fb99b7fe700 1 -- 172.21.15.33:0/1675879215 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb9a400bd30 con 0x7fb9b40fb390 2024-04-25T01:47:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.782+0000 7fb99b7fe700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb99c06e140 0x7fb99c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.782+0000 7fb99b7fe700 1 -- 172.21.15.33:0/1675879215 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb9a408ed70 con 0x7fb9b40fb390 2024-04-25T01:47:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.782+0000 7fb9b259c700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb99c06e140 0x7fb99c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:29.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.783+0000 7fb9b259c700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb99c06e140 0x7fb99c0705f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8006fd0 tx=0x7fb9a8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:29.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:29.787+0000 7fb99b7fe700 1 -- 172.21.15.33:0/1675879215 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb9a4058170 con 0x7fb9b40fb390 2024-04-25T01:47:29.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:29 smithi138 ceph-mon[81304]: pgmap v242: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:29.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:29 smithi138 ceph-mon[81304]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:47:30.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.086+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb9b40fda50 con 0x7fb9b40fb390 2024-04-25T01:47:30.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.087+0000 7fb99b7fe700 1 -- 172.21.15.33:0/1675879215 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fb9a405b830 con 0x7fb9b40fb390 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:47:30.083 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:47:30.084 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:47:30.084 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:47:30.084 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:47:30.084 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:47:30.084 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:47:30.084 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:47:30.084 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:47:30.084 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:47:30.084 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:47:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb99c06e140 msgr2=0x7fb99c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb99c06e140 0x7fb99c0705f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8006fd0 tx=0x7fb9a8008040 comp rx=0 tx=0).stop 2024-04-25T01:47:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b40fb390 msgr2=0x7fb9b40fc570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b40fb390 0x7fb9b40fc570 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb9a400b3b0 tx=0x7fb9a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 shutdown_connections 2024-04-25T01:47:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb99c06e140 0x7fb99c0705f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb9b40fb390 0x7fb9b40fc570 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 --2- 172.21.15.33:0/1675879215 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb9b4102f60 0x7fb9b40fcab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 >> 172.21.15.33:0/1675879215 conn(0x7fb9b4073350 msgr2=0x7fb9b40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:30.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 shutdown_connections 2024-04-25T01:47:30.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.089+0000 7fb9b917b700 1 -- 172.21.15.33:0/1675879215 wait complete. 2024-04-25T01:47:30.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.194+0000 7ff8d95e1700 1 -- 172.21.15.33:0/4206863318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 msgr2=0x7ff8d40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.194+0000 7ff8d95e1700 1 --2- 172.21.15.33:0/4206863318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 0x7ff8d40fcdf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc004940 tx=0x7ff8bc009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.194+0000 7ff8d95e1700 1 -- 172.21.15.33:0/4206863318 shutdown_connections 2024-04-25T01:47:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.194+0000 7ff8d95e1700 1 --2- 172.21.15.33:0/4206863318 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 0x7ff8d40fcdf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.194+0000 7ff8d95e1700 1 --2- 172.21.15.33:0/4206863318 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8d4102de0 0x7ff8d41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.194+0000 7ff8d95e1700 1 -- 172.21.15.33:0/4206863318 >> 172.21.15.33:0/4206863318 conn(0x7ff8d40f8990 msgr2=0x7ff8d40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.194+0000 7ff8d95e1700 1 -- 172.21.15.33:0/4206863318 shutdown_connections 2024-04-25T01:47:30.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.194+0000 7ff8d95e1700 1 -- 172.21.15.33:0/4206863318 wait complete. 2024-04-25T01:47:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.194+0000 7ff8d95e1700 1 Processor -- start 2024-04-25T01:47:30.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d95e1700 1 -- start start 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d95e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8d4102de0 0x7ff8d4113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d95e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 0x7ff8d4113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d95e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41145a0 con 0x7ff8d4103780 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d95e1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d410db60 con 0x7ff8d4102de0 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 0x7ff8d4113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 0x7ff8d4113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32948/0 (socket says 172.21.15.33:32948) 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d27fc700 1 -- 172.21.15.33:0/1681509662 learned_addr learned my addr 172.21.15.33:0/1681509662 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d2ffd700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8d4102de0 0x7ff8d4113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d27fc700 1 -- 172.21.15.33:0/1681509662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8d4102de0 msgr2=0x7ff8d4113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d27fc700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8d4102de0 0x7ff8d4113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d27fc700 1 -- 172.21.15.33:0/1681509662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc005040 con 0x7ff8d4103780 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.195+0000 7ff8d27fc700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 0x7ff8d4113ec0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009100 tx=0x7ff8bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:30.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.196+0000 7ff8cb7fe700 1 -- 172.21.15.33:0/1681509662 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8bc00e040 con 0x7ff8d4103780 2024-04-25T01:47:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.196+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d410dde0 con 0x7ff8d4103780 2024-04-25T01:47:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.196+0000 7ff8cb7fe700 1 -- 172.21.15.33:0/1681509662 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8bc009af0 con 0x7ff8d4103780 2024-04-25T01:47:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.196+0000 7ff8cb7fe700 1 -- 172.21.15.33:0/1681509662 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8bc01c3f0 con 0x7ff8d4103780 2024-04-25T01:47:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.196+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d410e3a0 con 0x7ff8d4103780 2024-04-25T01:47:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.197+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d41000f0 con 0x7ff8d4103780 2024-04-25T01:47:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.197+0000 7ff8cb7fe700 1 -- 172.21.15.33:0/1681509662 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff8bc00baf0 con 0x7ff8d4103780 2024-04-25T01:47:30.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.198+0000 7ff8cb7fe700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8c006e140 0x7ff8c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.198+0000 7ff8cb7fe700 1 -- 172.21.15.33:0/1681509662 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff8bc08ed60 con 0x7ff8d4103780 2024-04-25T01:47:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.198+0000 7ff8d2ffd700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8c006e140 0x7ff8c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:30.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.198+0000 7ff8d2ffd700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8c006e140 0x7ff8c00705f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4007540 tx=0x7ff8c4009780 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:30.198 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.202+0000 7ff8cb7fe700 1 -- 172.21.15.33:0/1681509662 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff8bc05bc90 con 0x7ff8d4103780 2024-04-25T01:47:30.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.454+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff8d410efd0 con 0x7ff8d4103780 2024-04-25T01:47:30.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.454+0000 7ff8cb7fe700 1 -- 172.21.15.33:0/1681509662 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7ff8bc05b820 con 0x7ff8d4103780 2024-04-25T01:47:30.450 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:47:30.450 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:47:30.450 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:47:30.451 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:47:30.452 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:47:30.453 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:47:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.456+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8c006e140 msgr2=0x7ff8c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.456+0000 7ff8d95e1700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8c006e140 0x7ff8c00705f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4007540 tx=0x7ff8c4009780 comp rx=0 tx=0).stop 2024-04-25T01:47:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.456+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 msgr2=0x7ff8d4113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.456+0000 7ff8d95e1700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 0x7ff8d4113ec0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009100 tx=0x7ff8bc009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.457+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 shutdown_connections 2024-04-25T01:47:30.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.457+0000 7ff8d95e1700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8c006e140 0x7ff8c00705f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.457+0000 7ff8d95e1700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8d4102de0 0x7ff8d4113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.457+0000 7ff8d95e1700 1 --2- 172.21.15.33:0/1681509662 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8d4103780 0x7ff8d4113ec0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.457+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 >> 172.21.15.33:0/1681509662 conn(0x7ff8d40f8990 msgr2=0x7ff8d4107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.457+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 shutdown_connections 2024-04-25T01:47:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.457+0000 7ff8d95e1700 1 -- 172.21.15.33:0/1681509662 wait complete. 2024-04-25T01:47:30.455 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:47:30.455 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:30 smithi033 ceph-mon[111306]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:47:30.455 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:30 smithi033 ceph-mon[111306]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:47:30.455 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:30 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1675879215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:47:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.563+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/2648183688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 msgr2=0x7f2b980fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.563+0000 7f2b9f0ff700 1 --2- 172.21.15.33:0/2648183688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 0x7f2b980fb600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2b84004940 tx=0x7f2b84009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.563+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/2648183688 shutdown_connections 2024-04-25T01:47:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.563+0000 7f2b9f0ff700 1 --2- 172.21.15.33:0/2648183688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 0x7f2b980fb600 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.563+0000 7f2b9f0ff700 1 --2- 172.21.15.33:0/2648183688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b98101cd0 0x7f2b981020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.563+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/2648183688 >> 172.21.15.33:0/2648183688 conn(0x7f2b98071460 msgr2=0x7f2b98071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.563+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/2648183688 shutdown_connections 2024-04-25T01:47:30.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.563+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/2648183688 wait complete. 2024-04-25T01:47:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b9f0ff700 1 Processor -- start 2024-04-25T01:47:30.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b9f0ff700 1 -- start start 2024-04-25T01:47:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b9f0ff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b98101cd0 0x7f2b98113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b9f0ff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 0x7f2b981140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b9f0ff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b98114740 con 0x7f2b98102670 2024-04-25T01:47:30.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b9f0ff700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9810db30 con 0x7f2b98101cd0 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b97fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 0x7f2b981140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b97fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 0x7f2b981140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32968/0 (socket says 172.21.15.33:32968) 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b97fff700 1 -- 172.21.15.33:0/3188285044 learned_addr learned my addr 172.21.15.33:0/3188285044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.564+0000 7f2b9ce9b700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b98101cd0 0x7f2b98113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b97fff700 1 -- 172.21.15.33:0/3188285044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b98101cd0 msgr2=0x7f2b98113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b97fff700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b98101cd0 0x7f2b98113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b97fff700 1 -- 172.21.15.33:0/3188285044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84005040 con 0x7f2b98102670 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b9ce9b700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b98101cd0 0x7f2b98113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b97fff700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 0x7f2b981140f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2b84000c00 tx=0x7f2b84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b957fa700 1 -- 172.21.15.33:0/3188285044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b8400e040 con 0x7f2b98102670 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9810ddb0 con 0x7f2b98102670 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b957fa700 1 -- 172.21.15.33:0/3188285044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b84013420 con 0x7f2b98102670 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b957fa700 1 -- 172.21.15.33:0/3188285044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b8401c630 con 0x7f2b98102670 2024-04-25T01:47:30.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.565+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9810e370 con 0x7f2b98102670 2024-04-25T01:47:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.566+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b98065200 con 0x7f2b98102670 2024-04-25T01:47:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.566+0000 7f2b957fa700 1 -- 172.21.15.33:0/3188285044 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2b84013590 con 0x7f2b98102670 2024-04-25T01:47:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.567+0000 7f2b957fa700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b8806e190 0x7f2b88070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:30.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.567+0000 7f2b957fa700 1 -- 172.21.15.33:0/3188285044 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2b8408ede0 con 0x7f2b98102670 2024-04-25T01:47:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.567+0000 7f2b9ce9b700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b8806e190 0x7f2b88070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:30.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.567+0000 7f2b9ce9b700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b8806e190 0x7f2b88070640 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0041d0 tx=0x7f2b8c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:30.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.571+0000 7f2b957fa700 1 -- 172.21.15.33:0/3188285044 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2b8405bc70 con 0x7f2b98102670 2024-04-25T01:47:30.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:30 smithi138 ceph-mon[81304]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:47:30.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:30 smithi138 ceph-mon[81304]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:47:30.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:30 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1675879215' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:47:30.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.800+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b98109ed0 con 0x7f2b8806e190 2024-04-25T01:47:30.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.801+0000 7f2b957fa700 1 -- 172.21.15.33:0/3188285044 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2b98109ed0 con 0x7f2b8806e190 2024-04-25T01:47:30.797 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:47:30.797 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:47:30.797 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:47:30.797 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:47:30.797 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:47:30.797 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:47:30.797 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:47:30.797 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:47:30.798 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:47:30.798 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:47:30.798 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:47:30.798 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:47:30.798 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:47:30.798 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:47:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.803+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b8806e190 msgr2=0x7f2b88070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.803+0000 7f2b9f0ff700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b8806e190 0x7f2b88070640 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0041d0 tx=0x7f2b8c009040 comp rx=0 tx=0).stop 2024-04-25T01:47:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.804+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 msgr2=0x7f2b981140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.804+0000 7f2b9f0ff700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 0x7f2b981140f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f2b84000c00 tx=0x7f2b84009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.804+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 shutdown_connections 2024-04-25T01:47:30.799 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.804+0000 7f2b9f0ff700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b8806e190 0x7f2b88070640 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.804+0000 7f2b9f0ff700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b98101cd0 0x7f2b98113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.804+0000 7f2b9f0ff700 1 --2- 172.21.15.33:0/3188285044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b98102670 0x7f2b981140f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.804+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 >> 172.21.15.33:0/3188285044 conn(0x7f2b98071460 msgr2=0x7f2b981009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.804+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 shutdown_connections 2024-04-25T01:47:30.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.804+0000 7f2b9f0ff700 1 -- 172.21.15.33:0/3188285044 wait complete. 2024-04-25T01:47:30.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.908+0000 7f766266e700 1 -- 172.21.15.33:0/837935660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 msgr2=0x7f765c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.908+0000 7f766266e700 1 --2- 172.21.15.33:0/837935660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 0x7f765c1031b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f764c004940 tx=0x7f764c009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.908+0000 7f766266e700 1 -- 172.21.15.33:0/837935660 shutdown_connections 2024-04-25T01:47:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.908+0000 7f766266e700 1 --2- 172.21.15.33:0/837935660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765c103780 0x7f765c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.908+0000 7f766266e700 1 --2- 172.21.15.33:0/837935660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 0x7f765c1031b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.908+0000 7f766266e700 1 -- 172.21.15.33:0/837935660 >> 172.21.15.33:0/837935660 conn(0x7f765c0f8990 msgr2=0x7f765c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.908+0000 7f766266e700 1 -- 172.21.15.33:0/837935660 shutdown_connections 2024-04-25T01:47:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.908+0000 7f766266e700 1 -- 172.21.15.33:0/837935660 wait complete. 2024-04-25T01:47:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f766266e700 1 Processor -- start 2024-04-25T01:47:30.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f766266e700 1 -- start start 2024-04-25T01:47:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f766266e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 0x7f765c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f766266e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765c103780 0x7f765c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:30.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f766266e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c072160 con 0x7f765c102de0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f766266e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c070ce0 con 0x7f765c103780 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f765bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 0x7f765c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f765bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 0x7f765c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32992/0 (socket says 172.21.15.33:32992) 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f765bfff700 1 -- 172.21.15.33:0/3810781270 learned_addr learned my addr 172.21.15.33:0/3810781270 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.909+0000 7f765b7fe700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765c103780 0x7f765c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.910+0000 7f765bfff700 1 -- 172.21.15.33:0/3810781270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765c103780 msgr2=0x7f765c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.910+0000 7f765bfff700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765c103780 0x7f765c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.910+0000 7f765bfff700 1 -- 172.21.15.33:0/3810781270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c005040 con 0x7f765c102de0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.910+0000 7f765bfff700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 0x7f765c070260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f764c00b3b0 tx=0x7f764c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.910+0000 7f7658ff9700 1 -- 172.21.15.33:0/3810781270 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764c00e040 con 0x7f765c102de0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.910+0000 7f7658ff9700 1 -- 172.21.15.33:0/3810781270 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f764c013420 con 0x7f765c102de0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.910+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c070f60 con 0x7f765c102de0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.910+0000 7f7658ff9700 1 -- 172.21.15.33:0/3810781270 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f764c01c630 con 0x7f765c102de0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.910+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c11a110 con 0x7f765c102de0 2024-04-25T01:47:30.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.911+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c1000f0 con 0x7f765c102de0 2024-04-25T01:47:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.911+0000 7f7658ff9700 1 -- 172.21.15.33:0/3810781270 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f764c013590 con 0x7f765c102de0 2024-04-25T01:47:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.912+0000 7f7658ff9700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f764806e140 0x7f76480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:47:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.912+0000 7f7658ff9700 1 -- 172.21.15.33:0/3810781270 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f764c08ec00 con 0x7f765c102de0 2024-04-25T01:47:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.912+0000 7f765b7fe700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f764806e140 0x7f76480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:47:30.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.912+0000 7f765b7fe700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f764806e140 0x7f76480705f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f765c071940 tx=0x7f764400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:47:30.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:30.917+0000 7f7658ff9700 1 -- 172.21.15.33:0/3810781270 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f764c094020 con 0x7f765c102de0 2024-04-25T01:47:31.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.207+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f765c0717b0 con 0x7f765c102de0 2024-04-25T01:47:31.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.208+0000 7f7658ff9700 1 -- 172.21.15.33:0/3810781270 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f764c0580a0 con 0x7f765c102de0 2024-04-25T01:47:31.204 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:47:31.204 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:47:31.204 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:47:31.205 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:47:31.205 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f764806e140 msgr2=0x7f76480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f764806e140 0x7f76480705f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f765c071940 tx=0x7f764400b040 comp rx=0 tx=0).stop 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 msgr2=0x7f765c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 0x7f765c070260 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f764c00b3b0 tx=0x7f764c009e30 comp rx=0 tx=0).stop 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 shutdown_connections 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f764806e140 0x7f76480705f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f765c102de0 0x7f765c070260 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 --2- 172.21.15.33:0/3810781270 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f765c103780 0x7f765c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 >> 172.21.15.33:0/3810781270 conn(0x7f765c0f8990 msgr2=0x7f765c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:47:31.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 shutdown_connections 2024-04-25T01:47:31.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:47:31.211+0000 7f766266e700 1 -- 172.21.15.33:0/3810781270 wait complete. 2024-04-25T01:47:31.511 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:31 smithi033 ceph-mon[111306]: pgmap v243: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:31.512 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:31 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1681509662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:47:31.512 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:31 smithi033 ceph-mon[111306]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:47:31.512 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:31 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3810781270' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:47:31.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:31 smithi138 ceph-mon[81304]: pgmap v243: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:31.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:31 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1681509662' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:47:31.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:31 smithi138 ceph-mon[81304]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:47:31.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:31 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3810781270' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:47:33.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:33 smithi033 ceph-mon[111306]: pgmap v244: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:33.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:33 smithi138 ceph-mon[81304]: pgmap v244: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:35.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:35 smithi033 ceph-mon[111306]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:35.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:35 smithi138 ceph-mon[81304]: pgmap v245: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:37.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:37 smithi033 ceph-mon[111306]: pgmap v246: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:37.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:37 smithi138 ceph-mon[81304]: pgmap v246: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:39.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:39 smithi033 ceph-mon[111306]: pgmap v247: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:39.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:39 smithi138 ceph-mon[81304]: pgmap v247: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:40.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:47:40.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:47:41.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:41 smithi033 ceph-mon[111306]: pgmap v248: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:41.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:41 smithi138 ceph-mon[81304]: pgmap v248: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:43.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:43 smithi033 ceph-mon[111306]: pgmap v249: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:43.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:43 smithi138 ceph-mon[81304]: pgmap v249: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:45.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:45 smithi033 ceph-mon[111306]: pgmap v250: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:45.786 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:45 smithi138 ceph-mon[81304]: pgmap v250: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:47.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:47 smithi033 ceph-mon[111306]: pgmap v251: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:47.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:47 smithi138 ceph-mon[81304]: pgmap v251: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:49.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:49 smithi033 ceph-mon[111306]: pgmap v252: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:49.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:49 smithi138 ceph-mon[81304]: pgmap v252: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:51.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:51 smithi033 ceph-mon[111306]: pgmap v253: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:51.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:51 smithi138 ceph-mon[81304]: pgmap v253: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:53.761 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:53 smithi033 ceph-mon[111306]: pgmap v254: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:53.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:53 smithi138 ceph-mon[81304]: pgmap v254: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:55.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:55 smithi033 ceph-mon[111306]: pgmap v255: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:55.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:47:55.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:55 smithi138 ceph-mon[81304]: pgmap v255: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:55.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:47:57.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:57 smithi033 ceph-mon[111306]: pgmap v256: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:57.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:57 smithi138 ceph-mon[81304]: pgmap v256: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:59.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:47:59 smithi033 ceph-mon[111306]: pgmap v257: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:47:59.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:47:59 smithi138 ceph-mon[81304]: pgmap v257: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.329+0000 7ff3012f8700 1 -- 172.21.15.33:0/1918912636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc101cd0 msgr2=0x7ff2fc1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.329+0000 7ff3012f8700 1 --2- 172.21.15.33:0/1918912636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc101cd0 0x7ff2fc1020a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009ce0 tx=0x7ff2f4004d10 comp rx=0 tx=0).stop 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.329+0000 7ff3012f8700 1 -- 172.21.15.33:0/1918912636 shutdown_connections 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.329+0000 7ff3012f8700 1 --2- 172.21.15.33:0/1918912636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2fc102670 0x7ff2fc0fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.329+0000 7ff3012f8700 1 --2- 172.21.15.33:0/1918912636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc101cd0 0x7ff2fc1020a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.329+0000 7ff3012f8700 1 -- 172.21.15.33:0/1918912636 >> 172.21.15.33:0/1918912636 conn(0x7ff2fc071460 msgr2=0x7ff2fc071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.329+0000 7ff3012f8700 1 -- 172.21.15.33:0/1918912636 shutdown_connections 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.329+0000 7ff3012f8700 1 -- 172.21.15.33:0/1918912636 wait complete. 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff3012f8700 1 Processor -- start 2024-04-25T01:48:01.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff3012f8700 1 -- start start 2024-04-25T01:48:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff3012f8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2fc101cd0 0x7ff2fc113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff3012f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc102670 0x7ff2fc114230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff3012f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc114880 con 0x7ff2fc102670 2024-04-25T01:48:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff3012f8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2fc10dc50 con 0x7ff2fc101cd0 2024-04-25T01:48:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff2fa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc102670 0x7ff2fc114230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff2fa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc102670 0x7ff2fc114230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:47982/0 (socket says 172.21.15.33:47982) 2024-04-25T01:48:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff2fa7fc700 1 -- 172.21.15.33:0/2831933391 learned_addr learned my addr 172.21.15.33:0/2831933391 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:01.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.330+0000 7ff2faffd700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2fc101cd0 0x7ff2fc113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.331+0000 7ff2fa7fc700 1 -- 172.21.15.33:0/2831933391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2fc101cd0 msgr2=0x7ff2fc113cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.331+0000 7ff2fa7fc700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2fc101cd0 0x7ff2fc113cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.331+0000 7ff2fa7fc700 1 -- 172.21.15.33:0/2831933391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f4009940 con 0x7ff2fc102670 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.331+0000 7ff2fa7fc700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc102670 0x7ff2fc114230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009b80 tx=0x7ff2e4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.331+0000 7ff2f37fe700 1 -- 172.21.15.33:0/2831933391 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e4004e70 con 0x7ff2fc102670 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.331+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2fc10df30 con 0x7ff2fc102670 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.331+0000 7ff2f37fe700 1 -- 172.21.15.33:0/2831933391 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2e400f040 con 0x7ff2fc102670 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.331+0000 7ff2f37fe700 1 -- 172.21.15.33:0/2831933391 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e400da60 con 0x7ff2fc102670 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.331+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2fc10e4f0 con 0x7ff2fc102670 2024-04-25T01:48:01.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.332+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2fc065200 con 0x7ff2fc102670 2024-04-25T01:48:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.333+0000 7ff2f37fe700 1 -- 172.21.15.33:0/2831933391 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff2e400dce0 con 0x7ff2fc102670 2024-04-25T01:48:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.333+0000 7ff2f37fe700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff2e806e150 0x7ff2e8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.333+0000 7ff2f37fe700 1 -- 172.21.15.33:0/2831933391 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff2e408d880 con 0x7ff2fc102670 2024-04-25T01:48:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.333+0000 7ff2faffd700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff2e806e150 0x7ff2e8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:01.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.334+0000 7ff2faffd700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff2e806e150 0x7ff2e8070600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400b740 tx=0x7ff2f4004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:01.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.338+0000 7ff2f37fe700 1 -- 172.21.15.33:0/2831933391 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff2e405a730 con 0x7ff2fc102670 2024-04-25T01:48:01.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.565+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2fc105330 con 0x7ff2e806e150 2024-04-25T01:48:01.562 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:01 smithi033 ceph-mon[111306]: pgmap v258: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:01.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.567+0000 7ff2f37fe700 1 -- 172.21.15.33:0/2831933391 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff2fc105330 con 0x7ff2e806e150 2024-04-25T01:48:01.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff2e806e150 msgr2=0x7ff2e8070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:01.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff2e806e150 0x7ff2e8070600 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400b740 tx=0x7ff2f4004d10 comp rx=0 tx=0).stop 2024-04-25T01:48:01.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc102670 msgr2=0x7ff2fc114230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:01.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc102670 0x7ff2fc114230 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4009b80 tx=0x7ff2e4004600 comp rx=0 tx=0).stop 2024-04-25T01:48:01.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 shutdown_connections 2024-04-25T01:48:01.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff2e806e150 0x7ff2e8070600 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff2fc101cd0 0x7ff2fc113cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 --2- 172.21.15.33:0/2831933391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff2fc102670 0x7ff2fc114230 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 >> 172.21.15.33:0/2831933391 conn(0x7ff2fc071460 msgr2=0x7ff2fc0f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:01.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 shutdown_connections 2024-04-25T01:48:01.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.570+0000 7ff3012f8700 1 -- 172.21.15.33:0/2831933391 wait complete. 2024-04-25T01:48:01.577 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:48:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 -- 172.21.15.33:0/3946750284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 msgr2=0x7fbcc80fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 --2- 172.21.15.33:0/3946750284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 0x7fbcc80fbd20 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc004940 tx=0x7fbcbc009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 -- 172.21.15.33:0/3946750284 shutdown_connections 2024-04-25T01:48:01.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 --2- 172.21.15.33:0/3946750284 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 0x7fbcc80fbd20 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 --2- 172.21.15.33:0/3946750284 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc8101d10 0x7fbcc81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 -- 172.21.15.33:0/3946750284 >> 172.21.15.33:0/3946750284 conn(0x7fbcc80f7860 msgr2=0x7fbcc80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 -- 172.21.15.33:0/3946750284 shutdown_connections 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 -- 172.21.15.33:0/3946750284 wait complete. 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 Processor -- start 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.676+0000 7fbccd313700 1 -- start start 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbccd313700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc8101d10 0x7fbcc810eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbccd313700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 0x7fbcc810f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbccd313700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc810f570 con 0x7fbcc81026b0 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbccd313700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc810f6b0 con 0x7fbcc8101d10 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbcc67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 0x7fbcc810f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbcc67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 0x7fbcc810f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48000/0 (socket says 172.21.15.33:48000) 2024-04-25T01:48:01.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbcc67fc700 1 -- 172.21.15.33:0/3548674345 learned_addr learned my addr 172.21.15.33:0/3548674345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbcc6ffd700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc8101d10 0x7fbcc810eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbcc67fc700 1 -- 172.21.15.33:0/3548674345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc8101d10 msgr2=0x7fbcc810eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbcc67fc700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc8101d10 0x7fbcc810eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbcc67fc700 1 -- 172.21.15.33:0/3548674345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc005040 con 0x7fbcc81026b0 2024-04-25T01:48:01.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.677+0000 7fbcc67fc700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 0x7fbcc810f030 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009180 tx=0x7fbcbc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:01.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.678+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/3548674345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcbc00e040 con 0x7fbcc81026b0 2024-04-25T01:48:01.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.678+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/3548674345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcbc00bbc0 con 0x7fbcc81026b0 2024-04-25T01:48:01.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.678+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc810f930 con 0x7fbcc81026b0 2024-04-25T01:48:01.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.678+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/3548674345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcbc01c3f0 con 0x7fbcc81026b0 2024-04-25T01:48:01.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.678+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc8117f00 con 0x7fbcc81026b0 2024-04-25T01:48:01.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.679+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc80fd460 con 0x7fbcc81026b0 2024-04-25T01:48:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.679+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/3548674345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbcbc00bd30 con 0x7fbcc81026b0 2024-04-25T01:48:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.680+0000 7fbcaf7fe700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcb006e140 0x7fbcb00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.680+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/3548674345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbcbc08ee50 con 0x7fbcc81026b0 2024-04-25T01:48:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.680+0000 7fbcc6ffd700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcb006e140 0x7fbcb00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:01.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.680+0000 7fbcc6ffd700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcb006e140 0x7fbcb00705f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80049b0 tx=0x7fbcb8009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:01.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.685+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/3548674345 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbcbc058250 con 0x7fbcc81026b0 2024-04-25T01:48:01.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:01 smithi138 ceph-mon[81304]: pgmap v258: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:01.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.912+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcc8108df0 con 0x7fbcb006e140 2024-04-25T01:48:01.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.914+0000 7fbcaf7fe700 1 -- 172.21.15.33:0/3548674345 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbcc8108df0 con 0x7fbcb006e140 2024-04-25T01:48:01.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcb006e140 msgr2=0x7fbcb00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcb006e140 0x7fbcb00705f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80049b0 tx=0x7fbcb8009ee0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 msgr2=0x7fbcc810f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 0x7fbcc810f030 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009180 tx=0x7fbcbc009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 shutdown_connections 2024-04-25T01:48:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcb006e140 0x7fbcb00705f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcc8101d10 0x7fbcc810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 --2- 172.21.15.33:0/3548674345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcc81026b0 0x7fbcc810f030 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 >> 172.21.15.33:0/3548674345 conn(0x7fbcc80f7860 msgr2=0x7fbcc80f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 shutdown_connections 2024-04-25T01:48:01.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:01.916+0000 7fbccd313700 1 -- 172.21.15.33:0/3548674345 wait complete. 2024-04-25T01:48:02.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.021+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1629523538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 msgr2=0x7f3fa40fe1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.021+0000 7f3fab6d6700 1 --2- 172.21.15.33:0/1629523538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 0x7f3fa40fe1a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0004940 tx=0x7f3fa0009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:02.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.021+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1629523538 shutdown_connections 2024-04-25T01:48:02.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.021+0000 7f3fab6d6700 1 --2- 172.21.15.33:0/1629523538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa40fe770 0x7f3fa4106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.021+0000 7f3fab6d6700 1 --2- 172.21.15.33:0/1629523538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 0x7f3fa40fe1a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.021+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1629523538 >> 172.21.15.33:0/1629523538 conn(0x7f3fa40f7850 msgr2=0x7f3fa40f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:02.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.021+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1629523538 shutdown_connections 2024-04-25T01:48:02.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.021+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1629523538 wait complete. 2024-04-25T01:48:02.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fab6d6700 1 Processor -- start 2024-04-25T01:48:02.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fab6d6700 1 -- start start 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fab6d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 0x7f3fa4113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fab6d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa40fe770 0x7f3fa41140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fab6d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa4114700 con 0x7f3fa40fddd0 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fab6d6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa410daf0 con 0x7f3fa40fe770 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fa9472700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 0x7f3fa4113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fa9472700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 0x7f3fa4113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48024/0 (socket says 172.21.15.33:48024) 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fa9472700 1 -- 172.21.15.33:0/1381308575 learned_addr learned my addr 172.21.15.33:0/1381308575 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fa9472700 1 -- 172.21.15.33:0/1381308575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa40fe770 msgr2=0x7f3fa41140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fa9472700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa40fe770 0x7f3fa41140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.022+0000 7f3fa9472700 1 -- 172.21.15.33:0/1381308575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa0005040 con 0x7f3fa40fddd0 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.023+0000 7f3fa9472700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 0x7f3fa4113b70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009380 tx=0x7f3fa0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.023+0000 7f3f95ffb700 1 -- 172.21.15.33:0/1381308575 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fa000e040 con 0x7f3fa40fddd0 2024-04-25T01:48:02.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.023+0000 7f3f95ffb700 1 -- 172.21.15.33:0/1381308575 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fa0009af0 con 0x7f3fa40fddd0 2024-04-25T01:48:02.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.023+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa410dd70 con 0x7f3fa40fddd0 2024-04-25T01:48:02.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.023+0000 7f3f95ffb700 1 -- 172.21.15.33:0/1381308575 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fa001c630 con 0x7f3fa40fddd0 2024-04-25T01:48:02.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.023+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa410e2b0 con 0x7f3fa40fddd0 2024-04-25T01:48:02.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.024+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa40666d0 con 0x7f3fa40fddd0 2024-04-25T01:48:02.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.024+0000 7f3f95ffb700 1 -- 172.21.15.33:0/1381308575 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3fa000bbc0 con 0x7f3fa40fddd0 2024-04-25T01:48:02.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.025+0000 7f3f95ffb700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f9006e140 0x7f3f900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:02.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.025+0000 7f3f95ffb700 1 -- 172.21.15.33:0/1381308575 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3fa008ee60 con 0x7f3fa40fddd0 2024-04-25T01:48:02.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.025+0000 7f3fa8c71700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f9006e140 0x7f3f900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:02.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.025+0000 7f3fa8c71700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f9006e140 0x7f3f900705f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fa410ecb0 tx=0x7f3f98007d40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:02.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.029+0000 7f3f95ffb700 1 -- 172.21.15.33:0/1381308575 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3fa005bd90 con 0x7f3fa40fddd0 2024-04-25T01:48:02.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.245+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fa4109ed0 con 0x7f3f9006e140 2024-04-25T01:48:02.251 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.255+0000 7f3f95ffb700 1 -- 172.21.15.33:0/1381308575 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3fa4109ed0 con 0x7f3f9006e140 2024-04-25T01:48:02.251 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:48:02.251 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (9m) 2m ago 20m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:48:02.251 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (6m) 2m ago 20m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (6m) 2m ago 19m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (8m) 2m ago 20m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (2m) 2m ago 14m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (2m) 2m ago 15m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (2m) 2m ago 14m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (14m) 2m ago 14m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (10m) 2m ago 21m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (10m) 2m ago 19m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (7m) 2m ago 21m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (7m) 2m ago 19m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (9m) 2m ago 20m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (9m) 2m ago 19m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (5m) 2m ago 18m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (5m) 2m ago 18m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:48:02.252 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (4m) 2m ago 17m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:48:02.253 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (4m) 2m ago 17m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:48:02.253 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (3m) 2m ago 16m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:48:02.253 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (3m) 2m ago 16m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:48:02.253 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (9m) 2m ago 19m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:48:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f9006e140 msgr2=0x7f3f900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f9006e140 0x7f3f900705f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3fa410ecb0 tx=0x7f3f98007d40 comp rx=0 tx=0).stop 2024-04-25T01:48:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 msgr2=0x7f3fa4113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 0x7f3fa4113b70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009380 tx=0x7f3fa0009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:02.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 shutdown_connections 2024-04-25T01:48:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f9006e140 0x7f3f900705f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa40fddd0 0x7f3fa4113b70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 --2- 172.21.15.33:0/1381308575 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa40fe770 0x7f3fa41140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 >> 172.21.15.33:0/1381308575 conn(0x7f3fa40f7850 msgr2=0x7f3fa40f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 shutdown_connections 2024-04-25T01:48:02.255 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.258+0000 7f3fab6d6700 1 -- 172.21.15.33:0/1381308575 wait complete. 2024-04-25T01:48:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.362+0000 7ff85f886700 1 -- 172.21.15.33:0/1826445904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 msgr2=0x7ff858102140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.362+0000 7ff85f886700 1 --2- 172.21.15.33:0/1826445904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 0x7ff858102140 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff854004940 tx=0x7ff854009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.362+0000 7ff85f886700 1 -- 172.21.15.33:0/1826445904 shutdown_connections 2024-04-25T01:48:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.362+0000 7ff85f886700 1 --2- 172.21.15.33:0/1826445904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858102710 0x7ff8580fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.362+0000 7ff85f886700 1 --2- 172.21.15.33:0/1826445904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 0x7ff858102140 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.362+0000 7ff85f886700 1 -- 172.21.15.33:0/1826445904 >> 172.21.15.33:0/1826445904 conn(0x7ff8580f78e0 msgr2=0x7ff8580f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.362+0000 7ff85f886700 1 -- 172.21.15.33:0/1826445904 shutdown_connections 2024-04-25T01:48:02.359 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.362+0000 7ff85f886700 1 -- 172.21.15.33:0/1826445904 wait complete. 2024-04-25T01:48:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.362+0000 7ff85f886700 1 Processor -- start 2024-04-25T01:48:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85f886700 1 -- start start 2024-04-25T01:48:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85f886700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 0x7ff85810eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85f886700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858102710 0x7ff85810f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85f886700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff858067160 con 0x7ff858101d70 2024-04-25T01:48:02.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85f886700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85810f5e0 con 0x7ff858102710 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85d622700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 0x7ff85810eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85d622700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 0x7ff85810eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48040/0 (socket says 172.21.15.33:48040) 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85d622700 1 -- 172.21.15.33:0/2641591280 learned_addr learned my addr 172.21.15.33:0/2641591280 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85ce21700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858102710 0x7ff85810f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85d622700 1 -- 172.21.15.33:0/2641591280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858102710 msgr2=0x7ff85810f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85d622700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858102710 0x7ff85810f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.363+0000 7ff85d622700 1 -- 172.21.15.33:0/2641591280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff854005040 con 0x7ff858101d70 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.364+0000 7ff85d622700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 0x7ff85810eb60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff85400b3b0 tx=0x7ff854009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.364+0000 7ff84dffb700 1 -- 172.21.15.33:0/2641591280 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff85400e040 con 0x7ff858101d70 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.364+0000 7ff84dffb700 1 -- 172.21.15.33:0/2641591280 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff854013420 con 0x7ff858101d70 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.364+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85810f800 con 0x7ff858101d70 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.364+0000 7ff84dffb700 1 -- 172.21.15.33:0/2641591280 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff85401c630 con 0x7ff858101d70 2024-04-25T01:48:02.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.364+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff858117e60 con 0x7ff858101d70 2024-04-25T01:48:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.365+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8580ff080 con 0x7ff858101d70 2024-04-25T01:48:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.366+0000 7ff84dffb700 1 -- 172.21.15.33:0/2641591280 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff85401c8b0 con 0x7ff858101d70 2024-04-25T01:48:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.366+0000 7ff84dffb700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff84406e140 0x7ff8440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:02.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.366+0000 7ff84dffb700 1 -- 172.21.15.33:0/2641591280 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff85408ed30 con 0x7ff858101d70 2024-04-25T01:48:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.367+0000 7ff85ce21700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff84406e140 0x7ff8440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:02.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.367+0000 7ff85ce21700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff84406e140 0x7ff8440705f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff858066940 tx=0x7ff8480043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:02.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.371+0000 7ff84dffb700 1 -- 172.21.15.33:0/2641591280 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff85405bc60 con 0x7ff858101d70 2024-04-25T01:48:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.672+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff858108f00 con 0x7ff858101d70 2024-04-25T01:48:02.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.673+0000 7ff84dffb700 1 -- 172.21.15.33:0/2641591280 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff85405b7f0 con 0x7ff858101d70 2024-04-25T01:48:02.669 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:48:02.669 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:48:02.669 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:48:02.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:48:02.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:48:02.671 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:48:02.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.675+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff84406e140 msgr2=0x7ff8440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.675+0000 7ff85f886700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff84406e140 0x7ff8440705f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff858066940 tx=0x7ff8480043c0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.675+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 msgr2=0x7ff85810eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.675+0000 7ff85f886700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 0x7ff85810eb60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff85400b3b0 tx=0x7ff854009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.675+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 shutdown_connections 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.675+0000 7ff85f886700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff84406e140 0x7ff8440705f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.675+0000 7ff85f886700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff858101d70 0x7ff85810eb60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.675+0000 7ff85f886700 1 --2- 172.21.15.33:0/2641591280 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff858102710 0x7ff85810f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.675+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 >> 172.21.15.33:0/2641591280 conn(0x7ff8580f78e0 msgr2=0x7ff858106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.676+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 shutdown_connections 2024-04-25T01:48:02.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.676+0000 7ff85f886700 1 -- 172.21.15.33:0/2641591280 wait complete. 2024-04-25T01:48:02.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:02 smithi033 ceph-mon[111306]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:02.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:02 smithi033 ceph-mon[111306]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.780+0000 7fcda9578700 1 -- 172.21.15.33:0/1566332385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4101cd0 msgr2=0x7fcda41020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.780+0000 7fcda9578700 1 --2- 172.21.15.33:0/1566332385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4101cd0 0x7fcda41020a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c009ce0 tx=0x7fcd9c004d10 comp rx=0 tx=0).stop 2024-04-25T01:48:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.780+0000 7fcda9578700 1 -- 172.21.15.33:0/1566332385 shutdown_connections 2024-04-25T01:48:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.780+0000 7fcda9578700 1 --2- 172.21.15.33:0/1566332385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcda4102670 0x7fcda40fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.780+0000 7fcda9578700 1 --2- 172.21.15.33:0/1566332385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4101cd0 0x7fcda41020a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.780+0000 7fcda9578700 1 -- 172.21.15.33:0/1566332385 >> 172.21.15.33:0/1566332385 conn(0x7fcda4071460 msgr2=0x7fcda4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.780+0000 7fcda9578700 1 -- 172.21.15.33:0/1566332385 shutdown_connections 2024-04-25T01:48:02.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.780+0000 7fcda9578700 1 -- 172.21.15.33:0/1566332385 wait complete. 2024-04-25T01:48:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda9578700 1 Processor -- start 2024-04-25T01:48:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda9578700 1 -- start start 2024-04-25T01:48:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda9578700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcda4101cd0 0x7fcda4113d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda9578700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4102670 0x7fcda4114240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda9578700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda4114890 con 0x7fcda4102670 2024-04-25T01:48:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda9578700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda410dc60 con 0x7fcda4101cd0 2024-04-25T01:48:02.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4102670 0x7fcda4114240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4102670 0x7fcda4114240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48062/0 (socket says 172.21.15.33:48062) 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda27fc700 1 -- 172.21.15.33:0/3504675993 learned_addr learned my addr 172.21.15.33:0/3504675993 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda2ffd700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcda4101cd0 0x7fcda4113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda27fc700 1 -- 172.21.15.33:0/3504675993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcda4101cd0 msgr2=0x7fcda4113d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda27fc700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcda4101cd0 0x7fcda4113d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.781+0000 7fcda27fc700 1 -- 172.21.15.33:0/3504675993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c009940 con 0x7fcda4102670 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.782+0000 7fcda27fc700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4102670 0x7fcda4114240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009b80 tx=0x7fcd8c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.782+0000 7fcd9b7fe700 1 -- 172.21.15.33:0/3504675993 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd8c004e70 con 0x7fcda4102670 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.782+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda410df40 con 0x7fcda4102670 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.782+0000 7fcd9b7fe700 1 -- 172.21.15.33:0/3504675993 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd8c00f040 con 0x7fcda4102670 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.782+0000 7fcd9b7fe700 1 -- 172.21.15.33:0/3504675993 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd8c00da60 con 0x7fcda4102670 2024-04-25T01:48:02.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.782+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda410e500 con 0x7fcda4102670 2024-04-25T01:48:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.783+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda4065200 con 0x7fcda4102670 2024-04-25T01:48:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.783+0000 7fcd9b7fe700 1 -- 172.21.15.33:0/3504675993 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcd8c00dce0 con 0x7fcda4102670 2024-04-25T01:48:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.784+0000 7fcd9b7fe700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd9006e140 0x7fcd900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.784+0000 7fcd9b7fe700 1 -- 172.21.15.33:0/3504675993 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcd8c08d8b0 con 0x7fcda4102670 2024-04-25T01:48:02.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.784+0000 7fcda2ffd700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd9006e140 0x7fcd900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:02.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.785+0000 7fcda2ffd700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd9006e140 0x7fcd900705f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00b740 tx=0x7fcd9c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:02.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:02.789+0000 7fcd9b7fe700 1 -- 172.21.15.33:0/3504675993 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcd8c05a760 con 0x7fcda4102670 2024-04-25T01:48:02.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:02 smithi138 ceph-mon[81304]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:02.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:02 smithi138 ceph-mon[81304]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:03.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.044+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcda41052c0 con 0x7fcda4102670 2024-04-25T01:48:03.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.045+0000 7fcd9b7fe700 1 -- 172.21.15.33:0/3504675993 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fcd8c05a2f0 con 0x7fcda4102670 2024-04-25T01:48:03.041 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:48:03.041 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:48:03.041 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:48:03.041 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:48:03.041 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:48:03.041 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:48:03.041 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:48:03.042 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:48:03.043 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:48:03.044 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:48:03.044 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:48:03.044 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:48:03.044 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:48:03.044 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:48:03.044 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd9006e140 msgr2=0x7fcd900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd9006e140 0x7fcd900705f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00b740 tx=0x7fcd9c004d10 comp rx=0 tx=0).stop 2024-04-25T01:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4102670 msgr2=0x7fcda4114240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4102670 0x7fcda4114240 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009b80 tx=0x7fcd8c004600 comp rx=0 tx=0).stop 2024-04-25T01:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 shutdown_connections 2024-04-25T01:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcd9006e140 0x7fcd900705f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcda4101cd0 0x7fcda4113d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 --2- 172.21.15.33:0/3504675993 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcda4102670 0x7fcda4114240 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 >> 172.21.15.33:0/3504675993 conn(0x7fcda4071460 msgr2=0x7fcda40f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 shutdown_connections 2024-04-25T01:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.048+0000 7fcda9578700 1 -- 172.21.15.33:0/3504675993 wait complete. 2024-04-25T01:48:03.046 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:48:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.153+0000 7f3b612b5700 1 -- 172.21.15.33:0/3409192554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 msgr2=0x7f3b5c101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.153+0000 7f3b612b5700 1 --2- 172.21.15.33:0/3409192554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 0x7f3b5c101a30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c004940 tx=0x7f3b4c009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.154+0000 7f3b612b5700 1 -- 172.21.15.33:0/3409192554 shutdown_connections 2024-04-25T01:48:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.154+0000 7f3b612b5700 1 --2- 172.21.15.33:0/3409192554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 0x7f3b5c101a30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.154+0000 7f3b612b5700 1 --2- 172.21.15.33:0/3409192554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b5c0fb5a0 0x7f3b5c0fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.154+0000 7f3b612b5700 1 -- 172.21.15.33:0/3409192554 >> 172.21.15.33:0/3409192554 conn(0x7f3b5c073350 msgr2=0x7f3b5c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.154+0000 7f3b612b5700 1 -- 172.21.15.33:0/3409192554 shutdown_connections 2024-04-25T01:48:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.154+0000 7f3b612b5700 1 -- 172.21.15.33:0/3409192554 wait complete. 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.154+0000 7f3b612b5700 1 Processor -- start 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.154+0000 7f3b612b5700 1 -- start start 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b612b5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b5c0fb5a0 0x7f3b5c115c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b612b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 0x7f3b5c116170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b612b5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c1167c0 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b612b5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5c10fda0 con 0x7f3b5c0fb5a0 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b5a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 0x7f3b5c116170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b5a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 0x7f3b5c116170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48078/0 (socket says 172.21.15.33:48078) 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b5a7fc700 1 -- 172.21.15.33:0/1838242532 learned_addr learned my addr 172.21.15.33:0/1838242532 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:03.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b5affd700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b5c0fb5a0 0x7f3b5c115c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b5a7fc700 1 -- 172.21.15.33:0/1838242532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b5c0fb5a0 msgr2=0x7f3b5c115c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b5a7fc700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b5c0fb5a0 0x7f3b5c115c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b5a7fc700 1 -- 172.21.15.33:0/1838242532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b4c005040 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b5affd700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b5c0fb5a0 0x7f3b5c115c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b5a7fc700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 0x7f3b5c116170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00b3b0 tx=0x7f3b4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b437fe700 1 -- 172.21.15.33:0/1838242532 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b4c00e040 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b437fe700 1 -- 172.21.15.33:0/1838242532 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b4c013420 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b5c110020 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b437fe700 1 -- 172.21.15.33:0/1838242532 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b4c01c630 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.155+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5c110560 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.156+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b5c101170 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.157+0000 7f3b437fe700 1 -- 172.21.15.33:0/1838242532 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3b4c013590 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.157+0000 7f3b437fe700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b4406e140 0x7f3b440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.157+0000 7f3b437fe700 1 -- 172.21.15.33:0/1838242532 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3b4c08ec50 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.157+0000 7f3b5affd700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b4406e140 0x7f3b440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.158+0000 7f3b5affd700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b4406e140 0x7f3b440705f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009fd0 tx=0x7f3b5000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:03.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.162+0000 7f3b437fe700 1 -- 172.21.15.33:0/1838242532 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3b4c05bc20 con 0x7f3b5c0fbf40 2024-04-25T01:48:03.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.389+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b5c111280 con 0x7f3b4406e140 2024-04-25T01:48:03.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.391+0000 7f3b437fe700 1 -- 172.21.15.33:0/1838242532 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3b5c111280 con 0x7f3b4406e140 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:48:03.387 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:48:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b4406e140 msgr2=0x7f3b440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b4406e140 0x7f3b440705f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3b50009fd0 tx=0x7f3b5000b040 comp rx=0 tx=0).stop 2024-04-25T01:48:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 msgr2=0x7f3b5c116170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 0x7f3b5c116170 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3b4c00b3b0 tx=0x7f3b4c009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 shutdown_connections 2024-04-25T01:48:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b4406e140 0x7f3b440705f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b5c0fb5a0 0x7f3b5c115c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 --2- 172.21.15.33:0/1838242532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b5c0fbf40 0x7f3b5c116170 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 >> 172.21.15.33:0/1838242532 conn(0x7f3b5c073350 msgr2=0x7f3b5c0f9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 shutdown_connections 2024-04-25T01:48:03.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.393+0000 7f3b612b5700 1 -- 172.21.15.33:0/1838242532 wait complete. 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.498+0000 7f06a08b5700 1 -- 172.21.15.33:0/4119912458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 msgr2=0x7f069c0fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.498+0000 7f06a08b5700 1 --2- 172.21.15.33:0/4119912458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 0x7f069c0fcb80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0690004940 tx=0x7f0690009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.498+0000 7f06a08b5700 1 -- 172.21.15.33:0/4119912458 shutdown_connections 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.498+0000 7f06a08b5700 1 --2- 172.21.15.33:0/4119912458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 0x7f069c0fcb80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.498+0000 7f06a08b5700 1 --2- 172.21.15.33:0/4119912458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f069c101e20 0x7f069c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.498+0000 7f06a08b5700 1 -- 172.21.15.33:0/4119912458 >> 172.21.15.33:0/4119912458 conn(0x7f069c071460 msgr2=0x7f069c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.498+0000 7f06a08b5700 1 -- 172.21.15.33:0/4119912458 shutdown_connections 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.498+0000 7f06a08b5700 1 -- 172.21.15.33:0/4119912458 wait complete. 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f06a08b5700 1 Processor -- start 2024-04-25T01:48:03.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f06a08b5700 1 -- start start 2024-04-25T01:48:03.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f06a08b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 0x7f069c10c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f06a08b5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f069c101e20 0x7f069c10ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f06a08b5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c10d350 con 0x7f069c0fc710 2024-04-25T01:48:03.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f06a08b5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069c10d4c0 con 0x7f069c101e20 2024-04-25T01:48:03.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f069a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 0x7f069c10c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f069a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 0x7f069c10c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48088/0 (socket says 172.21.15.33:48088) 2024-04-25T01:48:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f069a59c700 1 -- 172.21.15.33:0/2212633888 learned_addr learned my addr 172.21.15.33:0/2212633888 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f0699d9b700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f069c101e20 0x7f069c10ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f069a59c700 1 -- 172.21.15.33:0/2212633888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f069c101e20 msgr2=0x7f069c10ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f069a59c700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f069c101e20 0x7f069c10ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.499+0000 7f069a59c700 1 -- 172.21.15.33:0/2212633888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0690005040 con 0x7f069c0fc710 2024-04-25T01:48:03.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.500+0000 7f069a59c700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 0x7f069c10c8d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c8c0 tx=0x7f068c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:03.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.502+0000 7f068affd700 1 -- 172.21.15.33:0/2212633888 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068c00a760 con 0x7f069c0fc710 2024-04-25T01:48:03.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.502+0000 7f068affd700 1 -- 172.21.15.33:0/2212633888 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f068c00ad90 con 0x7f069c0fc710 2024-04-25T01:48:03.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.502+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f069c119f40 con 0x7f069c0fc710 2024-04-25T01:48:03.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.502+0000 7f068affd700 1 -- 172.21.15.33:0/2212633888 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068c0113a0 con 0x7f069c0fc710 2024-04-25T01:48:03.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.502+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f069c11a310 con 0x7f069c0fc710 2024-04-25T01:48:03.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.503+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069c104200 con 0x7f069c0fc710 2024-04-25T01:48:03.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.504+0000 7f068affd700 1 -- 172.21.15.33:0/2212633888 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f068c012400 con 0x7f069c0fc710 2024-04-25T01:48:03.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.504+0000 7f068affd700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f068406e140 0x7f06840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:03.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.504+0000 7f068affd700 1 -- 172.21.15.33:0/2212633888 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f068c08da00 con 0x7f069c0fc710 2024-04-25T01:48:03.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.505+0000 7f0699d9b700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f068406e140 0x7f06840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:03.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.505+0000 7f0699d9b700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f068406e140 0x7f06840705f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f069c0fc000 tx=0x7f0690009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:03.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.511+0000 7f068affd700 1 -- 172.21.15.33:0/2212633888 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f068c05a9d0 con 0x7f069c0fc710 2024-04-25T01:48:03.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:03 smithi033 ceph-mon[111306]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:03.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:03 smithi033 ceph-mon[111306]: pgmap v259: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:03.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:03 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2641591280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:48:03.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:03 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3504675993' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:48:03.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:03 smithi138 ceph-mon[81304]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:03.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:03 smithi138 ceph-mon[81304]: pgmap v259: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:03.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:03 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2641591280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:48:03.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:03 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3504675993' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:48:03.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.807+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f069c10d680 con 0x7f069c0fc710 2024-04-25T01:48:03.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.807+0000 7f068affd700 1 -- 172.21.15.33:0/2212633888 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f068c05a560 con 0x7f069c0fc710 2024-04-25T01:48:03.803 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:48:03.803 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:48:03.803 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:48:03.803 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:48:03.803 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:48:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f068406e140 msgr2=0x7f06840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f068406e140 0x7f06840705f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f069c0fc000 tx=0x7f0690009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 msgr2=0x7f069c10c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 0x7f069c10c8d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f068c00c8c0 tx=0x7f068c0044c0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 shutdown_connections 2024-04-25T01:48:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f068406e140 0x7f06840705f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f069c0fc710 0x7f069c10c8d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 --2- 172.21.15.33:0/2212633888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f069c101e20 0x7f069c10ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 >> 172.21.15.33:0/2212633888 conn(0x7f069c071460 msgr2=0x7f069c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 shutdown_connections 2024-04-25T01:48:03.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:03.809+0000 7f06a08b5700 1 -- 172.21.15.33:0/2212633888 wait complete. 2024-04-25T01:48:04.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:04 smithi033 ceph-mon[111306]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:04.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:04 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2212633888' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:48:04.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:04 smithi138 ceph-mon[81304]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:04.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:04 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2212633888' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:48:05.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:05 smithi033 ceph-mon[111306]: pgmap v260: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:05.787 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:05 smithi138 ceph-mon[81304]: pgmap v260: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:08.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:07 smithi033 ceph-mon[111306]: pgmap v261: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:08.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:07 smithi138 ceph-mon[81304]: pgmap v261: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:10.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:09 smithi033 ceph-mon[111306]: pgmap v262: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:10.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:09 smithi138 ceph-mon[81304]: pgmap v262: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:11.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:48:11.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:48:12.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:11 smithi033 ceph-mon[111306]: pgmap v263: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:12.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:11 smithi138 ceph-mon[81304]: pgmap v263: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:14.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:13 smithi033 ceph-mon[111306]: pgmap v264: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:14.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:13 smithi138 ceph-mon[81304]: pgmap v264: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:16.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:15 smithi033 ceph-mon[111306]: pgmap v265: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:16.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:15 smithi138 ceph-mon[81304]: pgmap v265: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:18.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:17 smithi033 ceph-mon[111306]: pgmap v266: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:18.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:17 smithi138 ceph-mon[81304]: pgmap v266: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:20.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:19 smithi033 ceph-mon[111306]: pgmap v267: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:20.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:19 smithi138 ceph-mon[81304]: pgmap v267: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:22.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:21 smithi033 ceph-mon[111306]: pgmap v268: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:22.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:21 smithi138 ceph-mon[81304]: pgmap v268: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:24.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:23 smithi033 ceph-mon[111306]: pgmap v269: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:24.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:23 smithi138 ceph-mon[81304]: pgmap v269: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:26.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:25 smithi033 ceph-mon[111306]: pgmap v270: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:26.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:48:26.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:25 smithi138 ceph-mon[81304]: pgmap v270: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:26.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:48:28.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:27 smithi033 ceph-mon[111306]: pgmap v271: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:28.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:27 smithi138 ceph-mon[81304]: pgmap v271: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:30.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:29 smithi033 ceph-mon[111306]: pgmap v272: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:30.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:29 smithi138 ceph-mon[81304]: pgmap v272: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:32.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:31 smithi033 ceph-mon[111306]: pgmap v273: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:32.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:31 smithi138 ceph-mon[81304]: pgmap v273: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.927+0000 7f1ddd159700 1 -- 172.21.15.33:0/2770358225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 msgr2=0x7f1dd81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.927+0000 7f1ddd159700 1 --2- 172.21.15.33:0/2770358225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 0x7f1dd81031b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8004940 tx=0x7f1dc8009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.928+0000 7f1ddd159700 1 -- 172.21.15.33:0/2770358225 shutdown_connections 2024-04-25T01:48:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.928+0000 7f1ddd159700 1 --2- 172.21.15.33:0/2770358225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8103780 0x7f1dd80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.928+0000 7f1ddd159700 1 --2- 172.21.15.33:0/2770358225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 0x7f1dd81031b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.928+0000 7f1ddd159700 1 -- 172.21.15.33:0/2770358225 >> 172.21.15.33:0/2770358225 conn(0x7f1dd80f8990 msgr2=0x7f1dd80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:33.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.928+0000 7f1ddd159700 1 -- 172.21.15.33:0/2770358225 shutdown_connections 2024-04-25T01:48:33.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.928+0000 7f1ddd159700 1 -- 172.21.15.33:0/2770358225 wait complete. 2024-04-25T01:48:33.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.928+0000 7f1ddd159700 1 Processor -- start 2024-04-25T01:48:33.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.928+0000 7f1ddd159700 1 -- start start 2024-04-25T01:48:33.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1ddd159700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 0x7f1dd8113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:33.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1ddd159700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8103780 0x7f1dd8113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:33.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1ddd159700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd8114500 con 0x7f1dd8102de0 2024-04-25T01:48:33.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1ddd159700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd810db50 con 0x7f1dd8103780 2024-04-25T01:48:33.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1dd6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 0x7f1dd8113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:33.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1dd6d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 0x7f1dd8113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59010/0 (socket says 172.21.15.33:59010) 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1dd6d9d700 1 -- 172.21.15.33:0/394755072 learned_addr learned my addr 172.21.15.33:0/394755072 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1dd659c700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8103780 0x7f1dd8113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1dd6d9d700 1 -- 172.21.15.33:0/394755072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8103780 msgr2=0x7f1dd8113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1dd6d9d700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8103780 0x7f1dd8113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1dd6d9d700 1 -- 172.21.15.33:0/394755072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc8005040 con 0x7f1dd8102de0 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.929+0000 7f1dd6d9d700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 0x7f1dd8113970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800b3b0 tx=0x7f1dc8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.930+0000 7f1dbf7fe700 1 -- 172.21.15.33:0/394755072 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dc800e040 con 0x7f1dd8102de0 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.930+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd810ddd0 con 0x7f1dd8102de0 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.930+0000 7f1dbf7fe700 1 -- 172.21.15.33:0/394755072 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dc800bbc0 con 0x7f1dd8102de0 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.930+0000 7f1dbf7fe700 1 -- 172.21.15.33:0/394755072 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dc801c3f0 con 0x7f1dd8102de0 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.930+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd810e390 con 0x7f1dd8102de0 2024-04-25T01:48:33.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.931+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd81000f0 con 0x7f1dd8102de0 2024-04-25T01:48:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.931+0000 7f1dbf7fe700 1 -- 172.21.15.33:0/394755072 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1dc800bd30 con 0x7f1dd8102de0 2024-04-25T01:48:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.932+0000 7f1dbf7fe700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1dc006e140 0x7f1dc00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.932+0000 7f1dbf7fe700 1 -- 172.21.15.33:0/394755072 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1dc808edc0 con 0x7f1dd8102de0 2024-04-25T01:48:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.932+0000 7f1dd659c700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1dc006e140 0x7f1dc00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:33.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.932+0000 7f1dd659c700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1dc006e140 0x7f1dc00705f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc007fd0 tx=0x7f1dcc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:33.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:33.937+0000 7f1dbf7fe700 1 -- 172.21.15.33:0/394755072 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1dc805bc70 con 0x7f1dd8102de0 2024-04-25T01:48:34.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:33 smithi033 ceph-mon[111306]: pgmap v274: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:34.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:33 smithi138 ceph-mon[81304]: pgmap v274: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:34.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.167+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dd810f030 con 0x7f1dc006e140 2024-04-25T01:48:34.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.169+0000 7f1dbf7fe700 1 -- 172.21.15.33:0/394755072 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1dd810f030 con 0x7f1dc006e140 2024-04-25T01:48:34.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.171+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1dc006e140 msgr2=0x7f1dc00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.171+0000 7f1ddd159700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1dc006e140 0x7f1dc00705f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc007fd0 tx=0x7f1dcc009040 comp rx=0 tx=0).stop 2024-04-25T01:48:34.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.171+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 msgr2=0x7f1dd8113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.171+0000 7f1ddd159700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 0x7f1dd8113970 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800b3b0 tx=0x7f1dc8009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:34.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.171+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 shutdown_connections 2024-04-25T01:48:34.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.171+0000 7f1ddd159700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1dc006e140 0x7f1dc00705f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.171+0000 7f1ddd159700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1dd8102de0 0x7f1dd8113970 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.171+0000 7f1ddd159700 1 --2- 172.21.15.33:0/394755072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1dd8103780 0x7f1dd8113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.171+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 >> 172.21.15.33:0/394755072 conn(0x7f1dd80f8990 msgr2=0x7f1dd8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:34.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.172+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 shutdown_connections 2024-04-25T01:48:34.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.172+0000 7f1ddd159700 1 -- 172.21.15.33:0/394755072 wait complete. 2024-04-25T01:48:34.179 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:48:34.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 -- 172.21.15.33:0/790927498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4103780 msgr2=0x7f19d40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 --2- 172.21.15.33:0/790927498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4103780 0x7f19d40fcdf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f19c4004940 tx=0x7f19c4009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:34.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 -- 172.21.15.33:0/790927498 shutdown_connections 2024-04-25T01:48:34.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 --2- 172.21.15.33:0/790927498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4103780 0x7f19d40fcdf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 --2- 172.21.15.33:0/790927498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d4102de0 0x7f19d41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 -- 172.21.15.33:0/790927498 >> 172.21.15.33:0/790927498 conn(0x7f19d40f8990 msgr2=0x7f19d40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:34.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 -- 172.21.15.33:0/790927498 shutdown_connections 2024-04-25T01:48:34.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 -- 172.21.15.33:0/790927498 wait complete. 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 Processor -- start 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.277+0000 7f19d9f6e700 1 -- start start 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d9f6e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4102de0 0x7f19d4113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d9f6e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d4103780 0x7f19d4113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d9f6e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d41144e0 con 0x7f19d4102de0 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d9f6e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d410dae0 con 0x7f19d4103780 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4102de0 0x7f19d4113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4102de0 0x7f19d4113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59026/0 (socket says 172.21.15.33:59026) 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d37fe700 1 -- 172.21.15.33:0/3013237989 learned_addr learned my addr 172.21.15.33:0/3013237989 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d2ffd700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d4103780 0x7f19d4113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:34.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d37fe700 1 -- 172.21.15.33:0/3013237989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d4103780 msgr2=0x7f19d4113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d37fe700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d4103780 0x7f19d4113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d37fe700 1 -- 172.21.15.33:0/3013237989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c4005040 con 0x7f19d4102de0 2024-04-25T01:48:34.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d37fe700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4102de0 0x7f19d4113950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f19c8002a50 tx=0x7f19c80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:34.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19bbfff700 1 -- 172.21.15.33:0/3013237989 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c8007cc0 con 0x7f19d4102de0 2024-04-25T01:48:34.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d410ddc0 con 0x7f19d4102de0 2024-04-25T01:48:34.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19bbfff700 1 -- 172.21.15.33:0/3013237989 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19c800f040 con 0x7f19d4102de0 2024-04-25T01:48:34.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.278+0000 7f19bbfff700 1 -- 172.21.15.33:0/3013237989 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19c8009640 con 0x7f19d4102de0 2024-04-25T01:48:34.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.279+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d410e350 con 0x7f19d4102de0 2024-04-25T01:48:34.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.279+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d40fe530 con 0x7f19d4102de0 2024-04-25T01:48:34.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.280+0000 7f19bbfff700 1 -- 172.21.15.33:0/3013237989 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f19c80097c0 con 0x7f19d4102de0 2024-04-25T01:48:34.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.280+0000 7f19bbfff700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f19bc06e140 0x7f19bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:34.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.280+0000 7f19bbfff700 1 -- 172.21.15.33:0/3013237989 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f19c808d8c0 con 0x7f19d4102de0 2024-04-25T01:48:34.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.281+0000 7f19d2ffd700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f19bc06e140 0x7f19bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:34.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.281+0000 7f19d2ffd700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f19bc06e140 0x7f19bc0705f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f19d410edc0 tx=0x7f19c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:34.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.285+0000 7f19bbfff700 1 -- 172.21.15.33:0/3013237989 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f19c805a770 con 0x7f19d4102de0 2024-04-25T01:48:34.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.513+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19d4109ec0 con 0x7f19bc06e140 2024-04-25T01:48:34.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.514+0000 7f19bbfff700 1 -- 172.21.15.33:0/3013237989 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f19d4109ec0 con 0x7f19bc06e140 2024-04-25T01:48:34.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f19bc06e140 msgr2=0x7f19bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f19bc06e140 0x7f19bc0705f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f19d410edc0 tx=0x7f19c4009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:34.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4102de0 msgr2=0x7f19d4113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4102de0 0x7f19d4113950 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f19c8002a50 tx=0x7f19c80046b0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 shutdown_connections 2024-04-25T01:48:34.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f19bc06e140 0x7f19bc0705f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19d4102de0 0x7f19d4113950 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 --2- 172.21.15.33:0/3013237989 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19d4103780 0x7f19d4113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 >> 172.21.15.33:0/3013237989 conn(0x7f19d40f8990 msgr2=0x7f19d40f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 shutdown_connections 2024-04-25T01:48:34.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.516+0000 7f19d9f6e700 1 -- 172.21.15.33:0/3013237989 wait complete. 2024-04-25T01:48:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.620+0000 7f7d33af3700 1 -- 172.21.15.33:0/4203341466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c0fbd40 msgr2=0x7f7d2c0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.620+0000 7f7d33af3700 1 --2- 172.21.15.33:0/4203341466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c0fbd40 0x7f7d2c0fc1b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7d28004940 tx=0x7f7d28009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.620+0000 7f7d33af3700 1 -- 172.21.15.33:0/4203341466 shutdown_connections 2024-04-25T01:48:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.620+0000 7f7d33af3700 1 --2- 172.21.15.33:0/4203341466 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c0fbd40 0x7f7d2c0fc1b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.620+0000 7f7d33af3700 1 --2- 172.21.15.33:0/4203341466 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d2c101d90 0x7f7d2c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.620+0000 7f7d33af3700 1 -- 172.21.15.33:0/4203341466 >> 172.21.15.33:0/4203341466 conn(0x7f7d2c071d00 msgr2=0x7f7d2c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.620+0000 7f7d33af3700 1 -- 172.21.15.33:0/4203341466 shutdown_connections 2024-04-25T01:48:34.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.620+0000 7f7d33af3700 1 -- 172.21.15.33:0/4203341466 wait complete. 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d33af3700 1 Processor -- start 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d33af3700 1 -- start start 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d33af3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d2c0fbd40 0x7f7d2c102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d33af3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c101d90 0x7f7d2c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d33af3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1035f0 con 0x7f7d2c101d90 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d33af3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c103730 con 0x7f7d2c0fbd40 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d3108e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c101d90 0x7f7d2c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d3108e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c101d90 0x7f7d2c103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59036/0 (socket says 172.21.15.33:59036) 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d3108e700 1 -- 172.21.15.33:0/3597685684 learned_addr learned my addr 172.21.15.33:0/3597685684 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d3188f700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d2c0fbd40 0x7f7d2c102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:34.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d3108e700 1 -- 172.21.15.33:0/3597685684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d2c0fbd40 msgr2=0x7f7d2c102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d3108e700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d2c0fbd40 0x7f7d2c102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.621+0000 7f7d3108e700 1 -- 172.21.15.33:0/3597685684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d28005040 con 0x7f7d2c101d90 2024-04-25T01:48:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.622+0000 7f7d3108e700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c101d90 0x7f7d2c103060 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800b3b0 tx=0x7f7d28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.622+0000 7f7d1e7fc700 1 -- 172.21.15.33:0/3597685684 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d2800e040 con 0x7f7d2c101d90 2024-04-25T01:48:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.622+0000 7f7d1e7fc700 1 -- 172.21.15.33:0/3597685684 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d2800bbc0 con 0x7f7d2c101d90 2024-04-25T01:48:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.622+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c103980 con 0x7f7d2c101d90 2024-04-25T01:48:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.622+0000 7f7d1e7fc700 1 -- 172.21.15.33:0/3597685684 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d2801c3f0 con 0x7f7d2c101d90 2024-04-25T01:48:34.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.622+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c11a2e0 con 0x7f7d2c101d90 2024-04-25T01:48:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.623+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c0fd430 con 0x7f7d2c101d90 2024-04-25T01:48:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.623+0000 7f7d1e7fc700 1 -- 172.21.15.33:0/3597685684 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7d2800bd30 con 0x7f7d2c101d90 2024-04-25T01:48:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.624+0000 7f7d1e7fc700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d1806e140 0x7f7d180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.624+0000 7f7d1e7fc700 1 -- 172.21.15.33:0/3597685684 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7d2808ee10 con 0x7f7d2c101d90 2024-04-25T01:48:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.624+0000 7f7d3188f700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d1806e140 0x7f7d180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:34.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.624+0000 7f7d3188f700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d1806e140 0x7f7d180705f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d20004990 tx=0x7f7d20009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:34.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.629+0000 7f7d1e7fc700 1 -- 172.21.15.33:0/3597685684 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7d28058190 con 0x7f7d2c101d90 2024-04-25T01:48:34.762 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:34 smithi033 ceph-mon[111306]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:34.845 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.848+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d2c04bed0 con 0x7f7d1806e140 2024-04-25T01:48:34.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.858+0000 7f7d1e7fc700 1 -- 172.21.15.33:0/3597685684 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7d2c04bed0 con 0x7f7d1806e140 2024-04-25T01:48:34.855 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:48:34.855 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (9m) 2m ago 20m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:48:34.855 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (6m) 2m ago 21m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:48:34.855 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (6m) 2m ago 20m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:48:34.855 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (9m) 2m ago 20m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (2m) 2m ago 15m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (3m) 2m ago 15m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (2m) 2m ago 15m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (15m) 2m ago 15m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (11m) 2m ago 22m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (10m) 2m ago 20m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (8m) 2m ago 22m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (7m) 2m ago 19m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (10m) 2m ago 21m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (10m) 2m ago 20m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (6m) 2m ago 19m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (5m) 2m ago 18m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (5m) 2m ago 18m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (4m) 2m ago 17m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:48:34.856 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (4m) 2m ago 17m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:48:34.857 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (3m) 2m ago 17m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:48:34.857 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (10m) 2m ago 20m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:48:34.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d1806e140 msgr2=0x7f7d180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d1806e140 0x7f7d180705f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7d20004990 tx=0x7f7d20009ee0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c101d90 msgr2=0x7f7d2c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c101d90 0x7f7d2c103060 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7d2800b3b0 tx=0x7f7d28009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 shutdown_connections 2024-04-25T01:48:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d1806e140 0x7f7d180705f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d2c0fbd40 0x7f7d2c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 --2- 172.21.15.33:0/3597685684 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d2c101d90 0x7f7d2c103060 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 >> 172.21.15.33:0/3597685684 conn(0x7f7d2c071d00 msgr2=0x7f7d2c0655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 shutdown_connections 2024-04-25T01:48:34.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.861+0000 7f7d33af3700 1 -- 172.21.15.33:0/3597685684 wait complete. 2024-04-25T01:48:34.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.964+0000 7f4c54752700 1 -- 172.21.15.33:0/2397984602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 msgr2=0x7f4c4c0fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.964+0000 7f4c54752700 1 --2- 172.21.15.33:0/2397984602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 0x7f4c4c0fe1e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4c48004940 tx=0x7f4c48009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.964+0000 7f4c54752700 1 -- 172.21.15.33:0/2397984602 shutdown_connections 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.964+0000 7f4c54752700 1 --2- 172.21.15.33:0/2397984602 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4c0fe7b0 0x7f4c4c106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.964+0000 7f4c54752700 1 --2- 172.21.15.33:0/2397984602 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 0x7f4c4c0fe1e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.964+0000 7f4c54752700 1 -- 172.21.15.33:0/2397984602 >> 172.21.15.33:0/2397984602 conn(0x7f4c4c0f7860 msgr2=0x7f4c4c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.964+0000 7f4c54752700 1 -- 172.21.15.33:0/2397984602 shutdown_connections 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.964+0000 7f4c54752700 1 -- 172.21.15.33:0/2397984602 wait complete. 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c54752700 1 Processor -- start 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c54752700 1 -- start start 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c54752700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 0x7f4c4c1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c54752700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4c0fe7b0 0x7f4c4c113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:34.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c54752700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c114530 con 0x7f4c4c0fde10 2024-04-25T01:48:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c54752700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c10db50 con 0x7f4c4c0fe7b0 2024-04-25T01:48:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c524ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 0x7f4c4c1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c524ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 0x7f4c4c1139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59046/0 (socket says 172.21.15.33:59046) 2024-04-25T01:48:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c524ee700 1 -- 172.21.15.33:0/3272123562 learned_addr learned my addr 172.21.15.33:0/3272123562 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c51ced700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4c0fe7b0 0x7f4c4c113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c524ee700 1 -- 172.21.15.33:0/3272123562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4c0fe7b0 msgr2=0x7f4c4c113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c524ee700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4c0fe7b0 0x7f4c4c113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:34.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c524ee700 1 -- 172.21.15.33:0/3272123562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48005040 con 0x7f4c4c0fde10 2024-04-25T01:48:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.965+0000 7f4c524ee700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 0x7f4c4c1139a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c48004890 tx=0x7f4c48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.966+0000 7f4c42ffd700 1 -- 172.21.15.33:0/3272123562 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4800e040 con 0x7f4c4c0fde10 2024-04-25T01:48:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.966+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c10ddd0 con 0x7f4c4c0fde10 2024-04-25T01:48:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.966+0000 7f4c42ffd700 1 -- 172.21.15.33:0/3272123562 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c48009af0 con 0x7f4c4c0fde10 2024-04-25T01:48:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.966+0000 7f4c42ffd700 1 -- 172.21.15.33:0/3272123562 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c4801c3f0 con 0x7f4c4c0fde10 2024-04-25T01:48:34.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.966+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c10e310 con 0x7f4c4c0fde10 2024-04-25T01:48:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.967+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c066b50 con 0x7f4c4c0fde10 2024-04-25T01:48:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.967+0000 7f4c42ffd700 1 -- 172.21.15.33:0/3272123562 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4c4800baf0 con 0x7f4c4c0fde10 2024-04-25T01:48:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.968+0000 7f4c42ffd700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c3806e140 0x7f4c380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.968+0000 7f4c42ffd700 1 -- 172.21.15.33:0/3272123562 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4c4808edc0 con 0x7f4c4c0fde10 2024-04-25T01:48:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.968+0000 7f4c51ced700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c3806e140 0x7f4c380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:34.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.968+0000 7f4c51ced700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c3806e140 0x7f4c380705f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c10f240 tx=0x7f4c3c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:34.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:34.972+0000 7f4c42ffd700 1 -- 172.21.15.33:0/3272123562 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4c48058140 con 0x7f4c4c0fde10 2024-04-25T01:48:35.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:34 smithi138 ceph-mon[81304]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:35.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.275+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4c4c10eff0 con 0x7f4c4c0fde10 2024-04-25T01:48:35.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.276+0000 7f4c42ffd700 1 -- 172.21.15.33:0/3272123562 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f4c4805b800 con 0x7f4c4c0fde10 2024-04-25T01:48:35.272 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:48:35.272 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:48:35.272 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:48:35.272 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:48:35.273 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:48:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c3806e140 msgr2=0x7f4c380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c3806e140 0x7f4c380705f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c10f240 tx=0x7f4c3c00b040 comp rx=0 tx=0).stop 2024-04-25T01:48:35.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 msgr2=0x7f4c4c1139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 0x7f4c4c1139a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4c48004890 tx=0x7f4c48009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:35.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 shutdown_connections 2024-04-25T01:48:35.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c3806e140 0x7f4c380705f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c4c0fde10 0x7f4c4c1139a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 --2- 172.21.15.33:0/3272123562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c4c0fe7b0 0x7f4c4c113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 >> 172.21.15.33:0/3272123562 conn(0x7f4c4c0f7860 msgr2=0x7f4c4c0f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:35.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 shutdown_connections 2024-04-25T01:48:35.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.278+0000 7f4c54752700 1 -- 172.21.15.33:0/3272123562 wait complete. 2024-04-25T01:48:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.381+0000 7ff641e25700 1 -- 172.21.15.33:0/3174043524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c0fb520 msgr2=0x7ff63c0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.381+0000 7ff641e25700 1 --2- 172.21.15.33:0/3174043524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c0fb520 0x7ff63c0fb990 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff62c004940 tx=0x7ff62c009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:35.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.382+0000 7ff641e25700 1 -- 172.21.15.33:0/3174043524 shutdown_connections 2024-04-25T01:48:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.382+0000 7ff641e25700 1 --2- 172.21.15.33:0/3174043524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c0fb520 0x7ff63c0fb990 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.382+0000 7ff641e25700 1 --2- 172.21.15.33:0/3174043524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c102f60 0x7ff63c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.382+0000 7ff641e25700 1 -- 172.21.15.33:0/3174043524 >> 172.21.15.33:0/3174043524 conn(0x7ff63c0714d0 msgr2=0x7ff63c0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.382+0000 7ff641e25700 1 -- 172.21.15.33:0/3174043524 shutdown_connections 2024-04-25T01:48:35.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.382+0000 7ff641e25700 1 -- 172.21.15.33:0/3174043524 wait complete. 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.382+0000 7ff641e25700 1 Processor -- start 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.382+0000 7ff641e25700 1 -- start start 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff641e25700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0fb520 0x7ff63c113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff641e25700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c102f60 0x7ff63c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff641e25700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c114510 con 0x7ff63c102f60 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff641e25700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c10db10 con 0x7ff63c0fb520 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff63affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c102f60 0x7ff63c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff63affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c102f60 0x7ff63c113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59058/0 (socket says 172.21.15.33:59058) 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff63affd700 1 -- 172.21.15.33:0/2318094242 learned_addr learned my addr 172.21.15.33:0/2318094242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff63b7fe700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0fb520 0x7ff63c113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff63affd700 1 -- 172.21.15.33:0/2318094242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0fb520 msgr2=0x7ff63c113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff63affd700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0fb520 0x7ff63c113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff63affd700 1 -- 172.21.15.33:0/2318094242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62c005040 con 0x7ff63c102f60 2024-04-25T01:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff63b7fe700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0fb520 0x7ff63c113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.383+0000 7ff63affd700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c102f60 0x7ff63c113ec0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00b3b0 tx=0x7ff62c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.384+0000 7ff623fff700 1 -- 172.21.15.33:0/2318094242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff62c00e040 con 0x7ff63c102f60 2024-04-25T01:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.384+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c10dd90 con 0x7ff63c102f60 2024-04-25T01:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.384+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c10e2d0 con 0x7ff63c102f60 2024-04-25T01:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.384+0000 7ff623fff700 1 -- 172.21.15.33:0/2318094242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff62c00bbc0 con 0x7ff63c102f60 2024-04-25T01:48:35.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.384+0000 7ff623fff700 1 -- 172.21.15.33:0/2318094242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff62c025400 con 0x7ff63c102f60 2024-04-25T01:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.385+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6280052e0 con 0x7ff63c102f60 2024-04-25T01:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.385+0000 7ff623fff700 1 -- 172.21.15.33:0/2318094242 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff62c00bd30 con 0x7ff63c102f60 2024-04-25T01:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.385+0000 7ff623fff700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff62406e140 0x7ff6240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.386+0000 7ff623fff700 1 -- 172.21.15.33:0/2318094242 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff62c08eca0 con 0x7ff63c102f60 2024-04-25T01:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.386+0000 7ff63b7fe700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff62406e140 0x7ff6240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:35.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.386+0000 7ff63b7fe700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff62406e140 0x7ff6240705f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff630007fd0 tx=0x7ff630009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:35.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.390+0000 7ff623fff700 1 -- 172.21.15.33:0/2318094242 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff62c093070 con 0x7ff63c102f60 2024-04-25T01:48:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.644+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff628005da0 con 0x7ff63c102f60 2024-04-25T01:48:35.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.644+0000 7ff623fff700 1 -- 172.21.15.33:0/2318094242 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7ff62c05bb50 con 0x7ff63c102f60 2024-04-25T01:48:35.641 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:48:35.642 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:48:35.643 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:48:35.644 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:48:35.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff62406e140 msgr2=0x7ff6240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff62406e140 0x7ff6240705f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff630007fd0 tx=0x7ff630009040 comp rx=0 tx=0).stop 2024-04-25T01:48:35.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c102f60 msgr2=0x7ff63c113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c102f60 0x7ff63c113ec0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00b3b0 tx=0x7ff62c009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:35.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 shutdown_connections 2024-04-25T01:48:35.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff62406e140 0x7ff6240705f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff63c0fb520 0x7ff63c113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 --2- 172.21.15.33:0/2318094242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff63c102f60 0x7ff63c113ec0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 >> 172.21.15.33:0/2318094242 conn(0x7ff63c0714d0 msgr2=0x7ff63c0fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 shutdown_connections 2024-04-25T01:48:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.648+0000 7ff641e25700 1 -- 172.21.15.33:0/2318094242 wait complete. 2024-04-25T01:48:35.646 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:48:35.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.753+0000 7f0ff93b5700 1 -- 172.21.15.33:0/2264024819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4101cd0 msgr2=0x7f0ff41020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.753+0000 7f0ff93b5700 1 --2- 172.21.15.33:0/2264024819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4101cd0 0x7f0ff41020a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009ce0 tx=0x7f0fe4004d10 comp rx=0 tx=0).stop 2024-04-25T01:48:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.753+0000 7f0ff93b5700 1 -- 172.21.15.33:0/2264024819 shutdown_connections 2024-04-25T01:48:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.753+0000 7f0ff93b5700 1 --2- 172.21.15.33:0/2264024819 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ff4102670 0x7f0ff40fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.753+0000 7f0ff93b5700 1 --2- 172.21.15.33:0/2264024819 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4101cd0 0x7f0ff41020a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.753+0000 7f0ff93b5700 1 -- 172.21.15.33:0/2264024819 >> 172.21.15.33:0/2264024819 conn(0x7f0ff4071460 msgr2=0x7f0ff4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.753+0000 7f0ff93b5700 1 -- 172.21.15.33:0/2264024819 shutdown_connections 2024-04-25T01:48:35.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.753+0000 7f0ff93b5700 1 -- 172.21.15.33:0/2264024819 wait complete. 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff93b5700 1 Processor -- start 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff93b5700 1 -- start start 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff93b5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ff4101cd0 0x7f0ff4113cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff93b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4102670 0x7f0ff4114230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff93b5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff4114880 con 0x7f0ff4102670 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff93b5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff410dc50 con 0x7f0ff4101cd0 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4102670 0x7f0ff4114230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4102670 0x7f0ff4114230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33362/0 (socket says 172.21.15.33:33362) 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff27fc700 1 -- 172.21.15.33:0/3234944116 learned_addr learned my addr 172.21.15.33:0/3234944116 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff2ffd700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ff4101cd0 0x7f0ff4113cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff27fc700 1 -- 172.21.15.33:0/3234944116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ff4101cd0 msgr2=0x7f0ff4113cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff27fc700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ff4101cd0 0x7f0ff4113cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff27fc700 1 -- 172.21.15.33:0/3234944116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe4009940 con 0x7f0ff4102670 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.754+0000 7f0ff27fc700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4102670 0x7f0ff4114230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00c860 tx=0x7f0fdc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:35.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.755+0000 7f0feb7fe700 1 -- 172.21.15.33:0/3234944116 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fdc00a7a0 con 0x7f0ff4102670 2024-04-25T01:48:35.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.755+0000 7f0feb7fe700 1 -- 172.21.15.33:0/3234944116 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fdc00add0 con 0x7f0ff4102670 2024-04-25T01:48:35.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.755+0000 7f0feb7fe700 1 -- 172.21.15.33:0/3234944116 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fdc0113a0 con 0x7f0ff4102670 2024-04-25T01:48:35.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.755+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff410df30 con 0x7f0ff4102670 2024-04-25T01:48:35.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.755+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff410e4f0 con 0x7f0ff4102670 2024-04-25T01:48:35.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.756+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff4065200 con 0x7f0ff4102670 2024-04-25T01:48:35.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.756+0000 7f0feb7fe700 1 -- 172.21.15.33:0/3234944116 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0fdc012400 con 0x7f0ff4102670 2024-04-25T01:48:35.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.757+0000 7f0feb7fe700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fe006e150 0x7f0fe0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:35.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.757+0000 7f0feb7fe700 1 -- 172.21.15.33:0/3234944116 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0fdc08dbe0 con 0x7f0ff4102670 2024-04-25T01:48:35.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.757+0000 7f0ff2ffd700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fe006e150 0x7f0fe0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:35.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.757+0000 7f0ff2ffd700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fe006e150 0x7f0fe0070600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400b740 tx=0x7f0fe4004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:35.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.761+0000 7f0feb7fe700 1 -- 172.21.15.33:0/3234944116 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0fdc05ab10 con 0x7f0ff4102670 2024-04-25T01:48:35.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.989+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ff4109f10 con 0x7f0fe006e150 2024-04-25T01:48:35.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:35 smithi033 ceph-mon[111306]: pgmap v275: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:35.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:35 smithi033 ceph-mon[111306]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:35.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:35 smithi033 ceph-mon[111306]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:35.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:35 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3272123562' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:48:35.986 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:35 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2318094242' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:48:35.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.990+0000 7f0feb7fe700 1 -- 172.21.15.33:0/3234944116 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0ff4109f10 con 0x7f0fe006e150 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:48:35.987 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fe006e150 msgr2=0x7f0fe0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fe006e150 0x7f0fe0070600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400b740 tx=0x7f0fe4004d10 comp rx=0 tx=0).stop 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4102670 msgr2=0x7f0ff4114230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4102670 0x7f0ff4114230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00c860 tx=0x7f0fdc0044c0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 shutdown_connections 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0fe006e150 0x7f0fe0070600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0ff4101cd0 0x7f0ff4113cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 --2- 172.21.15.33:0/3234944116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0ff4102670 0x7f0ff4114230 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 >> 172.21.15.33:0/3234944116 conn(0x7f0ff4071460 msgr2=0x7f0ff40f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 shutdown_connections 2024-04-25T01:48:35.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:35.992+0000 7f0ff93b5700 1 -- 172.21.15.33:0/3234944116 wait complete. 2024-04-25T01:48:36.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:35 smithi138 ceph-mon[81304]: pgmap v275: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:36.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:35 smithi138 ceph-mon[81304]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:36.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:35 smithi138 ceph-mon[81304]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:36.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:35 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3272123562' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:48:36.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:35 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2318094242' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:48:36.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.096+0000 7f2e88485700 1 -- 172.21.15.33:0/1729610292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fb5a0 msgr2=0x7f2e800fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:36.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.096+0000 7f2e88485700 1 --2- 172.21.15.33:0/1729610292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fb5a0 0x7f2e800fb970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c004940 tx=0x7f2e7c009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:36.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.096+0000 7f2e88485700 1 -- 172.21.15.33:0/1729610292 shutdown_connections 2024-04-25T01:48:36.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.096+0000 7f2e88485700 1 --2- 172.21.15.33:0/1729610292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e800fbf40 0x7f2e80101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:36.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.096+0000 7f2e88485700 1 --2- 172.21.15.33:0/1729610292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fb5a0 0x7f2e800fb970 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:36.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.096+0000 7f2e88485700 1 -- 172.21.15.33:0/1729610292 >> 172.21.15.33:0/1729610292 conn(0x7f2e80073350 msgr2=0x7f2e80073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:36.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e88485700 1 -- 172.21.15.33:0/1729610292 shutdown_connections 2024-04-25T01:48:36.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e88485700 1 -- 172.21.15.33:0/1729610292 wait complete. 2024-04-25T01:48:36.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e88485700 1 Processor -- start 2024-04-25T01:48:36.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e88485700 1 -- start start 2024-04-25T01:48:36.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e88485700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e800fb5a0 0x7f2e80113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:36.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e88485700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fbf40 0x7f2e80113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:36.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e88485700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e80114510 con 0x7f2e800fbf40 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e88485700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e8010db30 con 0x7f2e800fb5a0 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e85a20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fbf40 0x7f2e80113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e85a20700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fbf40 0x7f2e80113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33376/0 (socket says 172.21.15.33:33376) 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.097+0000 7f2e85a20700 1 -- 172.21.15.33:0/3536576925 learned_addr learned my addr 172.21.15.33:0/3536576925 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e86221700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e800fb5a0 0x7f2e80113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e85a20700 1 -- 172.21.15.33:0/3536576925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e800fb5a0 msgr2=0x7f2e80113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e85a20700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e800fb5a0 0x7f2e80113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e85a20700 1 -- 172.21.15.33:0/3536576925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c005040 con 0x7f2e800fbf40 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e86221700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e800fb5a0 0x7f2e80113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e85a20700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fbf40 0x7f2e80113ec0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400c860 tx=0x7f2e740044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e72ffd700 1 -- 172.21.15.33:0/3536576925 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e7400a7a0 con 0x7f2e800fbf40 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e72ffd700 1 -- 172.21.15.33:0/3536576925 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e7400add0 con 0x7f2e800fbf40 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8010de10 con 0x7f2e800fbf40 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e72ffd700 1 -- 172.21.15.33:0/3536576925 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e740113a0 con 0x7f2e800fbf40 2024-04-25T01:48:36.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.098+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8010e350 con 0x7f2e800fbf40 2024-04-25T01:48:36.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.099+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e800fe670 con 0x7f2e800fbf40 2024-04-25T01:48:36.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.100+0000 7f2e72ffd700 1 -- 172.21.15.33:0/3536576925 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2e74012400 con 0x7f2e800fbf40 2024-04-25T01:48:36.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.100+0000 7f2e72ffd700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2e6c06e140 0x7f2e6c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:48:36.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.100+0000 7f2e72ffd700 1 -- 172.21.15.33:0/3536576925 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2e7408dbc0 con 0x7f2e800fbf40 2024-04-25T01:48:36.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.100+0000 7f2e86221700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2e6c06e140 0x7f2e6c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:48:36.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.101+0000 7f2e86221700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2e6c06e140 0x7f2e6c0705f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c004560 tx=0x7f2e7c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:48:36.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.105+0000 7f2e72ffd700 1 -- 172.21.15.33:0/3536576925 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2e74056fc0 con 0x7f2e800fbf40 2024-04-25T01:48:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.395+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2e80109f40 con 0x7f2e800fbf40 2024-04-25T01:48:36.392 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.396+0000 7f2e72ffd700 1 -- 172.21.15.33:0/3536576925 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2e7405a680 con 0x7f2e800fbf40 2024-04-25T01:48:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:48:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:48:36.393 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:48:36.393 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:48:36.393 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:48:36.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2e6c06e140 msgr2=0x7f2e6c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:36.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2e6c06e140 0x7f2e6c0705f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c004560 tx=0x7f2e7c009e30 comp rx=0 tx=0).stop 2024-04-25T01:48:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fbf40 msgr2=0x7f2e80113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:48:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fbf40 0x7f2e80113ec0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2e7400c860 tx=0x7f2e740044c0 comp rx=0 tx=0).stop 2024-04-25T01:48:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 shutdown_connections 2024-04-25T01:48:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2e6c06e140 0x7f2e6c0705f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2e800fb5a0 0x7f2e80113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 --2- 172.21.15.33:0/3536576925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2e800fbf40 0x7f2e80113ec0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:48:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 >> 172.21.15.33:0/3536576925 conn(0x7f2e80073350 msgr2=0x7f2e80100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:48:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 shutdown_connections 2024-04-25T01:48:36.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:48:36.398+0000 7f2e88485700 1 -- 172.21.15.33:0/3536576925 wait complete. 2024-04-25T01:48:37.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:36 smithi033 ceph-mon[111306]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:37.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:36 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3536576925' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:48:37.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:36 smithi138 ceph-mon[81304]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:48:37.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:36 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3536576925' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:48:38.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:37 smithi033 ceph-mon[111306]: pgmap v276: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:38.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:37 smithi138 ceph-mon[81304]: pgmap v276: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:40.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:39 smithi033 ceph-mon[111306]: pgmap v277: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:40.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:39 smithi138 ceph-mon[81304]: pgmap v277: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:41.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:48:41.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:48:42.012 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:41 smithi033 ceph-mon[111306]: pgmap v278: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:42.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:41 smithi138 ceph-mon[81304]: pgmap v278: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:44.037 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:43 smithi138 ceph-mon[81304]: pgmap v279: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:44.262 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:43 smithi033 ceph-mon[111306]: pgmap v279: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:46.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:45 smithi138 ceph-mon[81304]: pgmap v280: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:46.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:45 smithi033 ceph-mon[111306]: pgmap v280: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:48.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:47 smithi138 ceph-mon[81304]: pgmap v281: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:48.262 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:47 smithi033 ceph-mon[111306]: pgmap v281: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:50.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:49 smithi138 ceph-mon[81304]: pgmap v282: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:50.262 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:49 smithi033 ceph-mon[111306]: pgmap v282: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:52.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:51 smithi138 ceph-mon[81304]: pgmap v283: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:52.262 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:51 smithi033 ceph-mon[111306]: pgmap v283: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:54.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:53 smithi138 ceph-mon[81304]: pgmap v284: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:54.262 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:53 smithi033 ceph-mon[111306]: pgmap v284: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:56.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:55 smithi138 ceph-mon[81304]: pgmap v285: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:56.038 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:48:56.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:55 smithi033 ceph-mon[111306]: pgmap v285: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:56.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:48:58.262 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:57 smithi033 ceph-mon[111306]: pgmap v286: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:48:58.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:57 smithi138 ceph-mon[81304]: pgmap v286: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:00.043 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:48:59 smithi138 ceph-mon[81304]: pgmap v287: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:00.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:48:59 smithi033 ceph-mon[111306]: pgmap v287: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:02.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:01 smithi033 ceph-mon[111306]: pgmap v288: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:02.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:01 smithi138 ceph-mon[81304]: pgmap v288: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:04.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:03 smithi033 ceph-mon[111306]: pgmap v289: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:04.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:03 smithi138 ceph-mon[81304]: pgmap v289: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:06.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:05 smithi033 ceph-mon[111306]: pgmap v290: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:06.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:05 smithi138 ceph-mon[81304]: pgmap v290: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.517+0000 7f11a99a0700 1 -- 172.21.15.33:0/1822254454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 msgr2=0x7f11a40fcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.517+0000 7f11a99a0700 1 --2- 172.21.15.33:0/1822254454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 0x7f11a40fcc20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1198004940 tx=0x7f1198009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.517+0000 7f11a99a0700 1 -- 172.21.15.33:0/1822254454 shutdown_connections 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.517+0000 7f11a99a0700 1 --2- 172.21.15.33:0/1822254454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 0x7f11a40fcc20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.517+0000 7f11a99a0700 1 --2- 172.21.15.33:0/1822254454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a4102c10 0x7f11a4102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.517+0000 7f11a99a0700 1 -- 172.21.15.33:0/1822254454 >> 172.21.15.33:0/1822254454 conn(0x7f11a40f8740 msgr2=0x7f11a40fab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.517+0000 7f11a99a0700 1 -- 172.21.15.33:0/1822254454 shutdown_connections 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.517+0000 7f11a99a0700 1 -- 172.21.15.33:0/1822254454 wait complete. 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a99a0700 1 Processor -- start 2024-04-25T01:49:06.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a99a0700 1 -- start start 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a99a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a4102c10 0x7f11a4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a99a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 0x7f11a40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a99a0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4070d30 con 0x7f11a41035b0 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a99a0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4070e70 con 0x7f11a4102c10 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 0x7f11a40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a3fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 0x7f11a40707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43050/0 (socket says 172.21.15.33:43050) 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a3fff700 1 -- 172.21.15.33:0/1222322757 learned_addr learned my addr 172.21.15.33:0/1222322757 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.518+0000 7f11a899e700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a4102c10 0x7f11a4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.519+0000 7f11a3fff700 1 -- 172.21.15.33:0/1222322757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a4102c10 msgr2=0x7f11a4070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.519+0000 7f11a3fff700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a4102c10 0x7f11a4070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.519+0000 7f11a3fff700 1 -- 172.21.15.33:0/1222322757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1198005040 con 0x7f11a41035b0 2024-04-25T01:49:06.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.519+0000 7f11a3fff700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 0x7f11a40707a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1198009180 tx=0x7f1198009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.519+0000 7f11a17fa700 1 -- 172.21.15.33:0/1222322757 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f119800e040 con 0x7f11a41035b0 2024-04-25T01:49:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.519+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a40710c0 con 0x7f11a41035b0 2024-04-25T01:49:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.519+0000 7f11a17fa700 1 -- 172.21.15.33:0/1222322757 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1198013420 con 0x7f11a41035b0 2024-04-25T01:49:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.519+0000 7f11a17fa700 1 -- 172.21.15.33:0/1222322757 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f119801c610 con 0x7f11a41035b0 2024-04-25T01:49:06.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.519+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a411a0f0 con 0x7f11a41035b0 2024-04-25T01:49:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.520+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a40fe360 con 0x7f11a41035b0 2024-04-25T01:49:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.521+0000 7f11a17fa700 1 -- 172.21.15.33:0/1222322757 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1198013a00 con 0x7f11a41035b0 2024-04-25T01:49:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.521+0000 7f11a17fa700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f118c06e140 0x7f118c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.521+0000 7f11a17fa700 1 -- 172.21.15.33:0/1222322757 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f119808ebe0 con 0x7f11a41035b0 2024-04-25T01:49:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.521+0000 7f11a899e700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f118c06e140 0x7f118c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:06.519 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.522+0000 7f11a899e700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f118c06e140 0x7f118c0705f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f119c007a40 tx=0x7f119c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:06.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.526+0000 7f11a17fa700 1 -- 172.21.15.33:0/1222322757 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1198058090 con 0x7f11a41035b0 2024-04-25T01:49:06.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.753+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11a4109ce0 con 0x7f118c06e140 2024-04-25T01:49:06.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.754+0000 7f11a17fa700 1 -- 172.21.15.33:0/1222322757 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f11a4109ce0 con 0x7f118c06e140 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.756+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f118c06e140 msgr2=0x7f118c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.756+0000 7f11a99a0700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f118c06e140 0x7f118c0705f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f119c007a40 tx=0x7f119c009da0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.757+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 msgr2=0x7f11a40707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.757+0000 7f11a99a0700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 0x7f11a40707a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1198009180 tx=0x7f1198009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.757+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 shutdown_connections 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.757+0000 7f11a99a0700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f118c06e140 0x7f118c0705f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.757+0000 7f11a99a0700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11a4102c10 0x7f11a4070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.757+0000 7f11a99a0700 1 --2- 172.21.15.33:0/1222322757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f11a41035b0 0x7f11a40707a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.757+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 >> 172.21.15.33:0/1222322757 conn(0x7f11a40f8740 msgr2=0x7f11a40f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.757+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 shutdown_connections 2024-04-25T01:49:06.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.757+0000 7f11a99a0700 1 -- 172.21.15.33:0/1222322757 wait complete. 2024-04-25T01:49:06.764 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:49:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.864+0000 7fd92c801700 1 -- 172.21.15.33:0/2354187782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 msgr2=0x7fd9280fcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.864+0000 7fd92c801700 1 --2- 172.21.15.33:0/2354187782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 0x7fd9280fcc30 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fd91c004940 tx=0x7fd91c009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.864+0000 7fd92c801700 1 -- 172.21.15.33:0/2354187782 shutdown_connections 2024-04-25T01:49:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.864+0000 7fd92c801700 1 --2- 172.21.15.33:0/2354187782 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 0x7fd9280fcc30 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.864+0000 7fd92c801700 1 --2- 172.21.15.33:0/2354187782 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd928102c20 0x7fd928102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.864+0000 7fd92c801700 1 -- 172.21.15.33:0/2354187782 >> 172.21.15.33:0/2354187782 conn(0x7fd9280f87d0 msgr2=0x7fd9280fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.864+0000 7fd92c801700 1 -- 172.21.15.33:0/2354187782 shutdown_connections 2024-04-25T01:49:06.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.864+0000 7fd92c801700 1 -- 172.21.15.33:0/2354187782 wait complete. 2024-04-25T01:49:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.865+0000 7fd92c801700 1 Processor -- start 2024-04-25T01:49:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.865+0000 7fd92c801700 1 -- start start 2024-04-25T01:49:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.865+0000 7fd92c801700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd928102c20 0x7fd928115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.865+0000 7fd92c801700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 0x7fd9281160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:06.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.865+0000 7fd92c801700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd928116730 con 0x7fd9281035c0 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.865+0000 7fd92c801700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92810fb20 con 0x7fd928102c20 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.865+0000 7fd92659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 0x7fd9281160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.865+0000 7fd92659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 0x7fd9281160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43064/0 (socket says 172.21.15.33:43064) 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.865+0000 7fd92659c700 1 -- 172.21.15.33:0/2034195584 learned_addr learned my addr 172.21.15.33:0/2034195584 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd926d9d700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd928102c20 0x7fd928115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd92659c700 1 -- 172.21.15.33:0/2034195584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd928102c20 msgr2=0x7fd928115ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd92659c700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd928102c20 0x7fd928115ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd92659c700 1 -- 172.21.15.33:0/2034195584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c005040 con 0x7fd9281035c0 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd92659c700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 0x7fd9281160e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009100 tx=0x7fd91c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd90f7fe700 1 -- 172.21.15.33:0/2034195584 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd91c00e040 con 0x7fd9281035c0 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd90f7fe700 1 -- 172.21.15.33:0/2034195584 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd91c013420 con 0x7fd9281035c0 2024-04-25T01:49:06.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd90f7fe700 1 -- 172.21.15.33:0/2034195584 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd91c01c610 con 0x7fd9281035c0 2024-04-25T01:49:06.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92810fda0 con 0x7fd9281035c0 2024-04-25T01:49:06.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.866+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd928110310 con 0x7fd9281035c0 2024-04-25T01:49:06.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.867+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9280fe370 con 0x7fd9281035c0 2024-04-25T01:49:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.868+0000 7fd90f7fe700 1 -- 172.21.15.33:0/2034195584 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd91c013590 con 0x7fd9281035c0 2024-04-25T01:49:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.868+0000 7fd90f7fe700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd91006e140 0x7fd9100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.868+0000 7fd90f7fe700 1 -- 172.21.15.33:0/2034195584 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd91c08edc0 con 0x7fd9281035c0 2024-04-25T01:49:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.868+0000 7fd926d9d700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd91006e140 0x7fd9100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:06.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.869+0000 7fd926d9d700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd91006e140 0x7fd9100705f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd918004c40 tx=0x7fd918009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:06.870 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:06.873+0000 7fd90f7fe700 1 -- 172.21.15.33:0/2034195584 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd91c05bcf0 con 0x7fd9281035c0 2024-04-25T01:49:07.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.098+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd928115870 con 0x7fd91006e140 2024-04-25T01:49:07.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.099+0000 7fd90f7fe700 1 -- 172.21.15.33:0/2034195584 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd928115870 con 0x7fd91006e140 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.101+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd91006e140 msgr2=0x7fd9100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.101+0000 7fd92c801700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd91006e140 0x7fd9100705f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd918004c40 tx=0x7fd918009040 comp rx=0 tx=0).stop 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.101+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 msgr2=0x7fd9281160e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.101+0000 7fd92c801700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 0x7fd9281160e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009100 tx=0x7fd91c009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.101+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 shutdown_connections 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.101+0000 7fd92c801700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd91006e140 0x7fd9100705f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.101+0000 7fd92c801700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd928102c20 0x7fd928115ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.101+0000 7fd92c801700 1 --2- 172.21.15.33:0/2034195584 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd9281035c0 0x7fd9281160e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.102+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 >> 172.21.15.33:0/2034195584 conn(0x7fd9280f87d0 msgr2=0x7fd928071a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.102+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 shutdown_connections 2024-04-25T01:49:07.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.102+0000 7fd92c801700 1 -- 172.21.15.33:0/2034195584 wait complete. 2024-04-25T01:49:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.209+0000 7fd687b14700 1 -- 172.21.15.33:0/1394915356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680103780 msgr2=0x7fd6800fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.209+0000 7fd687b14700 1 --2- 172.21.15.33:0/1394915356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680103780 0x7fd6800fcdf0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd670004940 tx=0x7fd670009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.209+0000 7fd687b14700 1 -- 172.21.15.33:0/1394915356 shutdown_connections 2024-04-25T01:49:07.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.209+0000 7fd687b14700 1 --2- 172.21.15.33:0/1394915356 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680103780 0x7fd6800fcdf0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.209+0000 7fd687b14700 1 --2- 172.21.15.33:0/1394915356 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd680102de0 0x7fd6801031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.209+0000 7fd687b14700 1 -- 172.21.15.33:0/1394915356 >> 172.21.15.33:0/1394915356 conn(0x7fd6800f8990 msgr2=0x7fd6800fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.209+0000 7fd687b14700 1 -- 172.21.15.33:0/1394915356 shutdown_connections 2024-04-25T01:49:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.209+0000 7fd687b14700 1 -- 172.21.15.33:0/1394915356 wait complete. 2024-04-25T01:49:07.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.209+0000 7fd687b14700 1 Processor -- start 2024-04-25T01:49:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd687b14700 1 -- start start 2024-04-25T01:49:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd687b14700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680102de0 0x7fd680070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd687b14700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd680103780 0x7fd6800707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd687b14700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6800721b0 con 0x7fd680102de0 2024-04-25T01:49:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd687b14700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd680070ce0 con 0x7fd680103780 2024-04-25T01:49:07.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd6858b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680102de0 0x7fd680070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:07.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd6858b0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680102de0 0x7fd680070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43082/0 (socket says 172.21.15.33:43082) 2024-04-25T01:49:07.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd6858b0700 1 -- 172.21.15.33:0/1441175511 learned_addr learned my addr 172.21.15.33:0/1441175511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:07.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd6850af700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd680103780 0x7fd6800707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:07.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd6858b0700 1 -- 172.21.15.33:0/1441175511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd680103780 msgr2=0x7fd6800707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd6858b0700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd680103780 0x7fd6800707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd6858b0700 1 -- 172.21.15.33:0/1441175511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd670005040 con 0x7fd680102de0 2024-04-25T01:49:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.210+0000 7fd6858b0700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680102de0 0x7fd680070260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00eb40 tx=0x7fd67c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.211+0000 7fd6767fc700 1 -- 172.21.15.33:0/1441175511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd67c007cc0 con 0x7fd680102de0 2024-04-25T01:49:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.211+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd680070f60 con 0x7fd680102de0 2024-04-25T01:49:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.211+0000 7fd6767fc700 1 -- 172.21.15.33:0/1441175511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd67c00f040 con 0x7fd680102de0 2024-04-25T01:49:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.211+0000 7fd6767fc700 1 -- 172.21.15.33:0/1441175511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd67c00ca10 con 0x7fd680102de0 2024-04-25T01:49:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.211+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd680119fd0 con 0x7fd680102de0 2024-04-25T01:49:07.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.212+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6640052b0 con 0x7fd680102de0 2024-04-25T01:49:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.212+0000 7fd6767fc700 1 -- 172.21.15.33:0/1441175511 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd67c00cc80 con 0x7fd680102de0 2024-04-25T01:49:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.213+0000 7fd6767fc700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd66c06e140 0x7fd66c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.213+0000 7fd6767fc700 1 -- 172.21.15.33:0/1441175511 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd67c08d880 con 0x7fd680102de0 2024-04-25T01:49:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.213+0000 7fd6850af700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd66c06e140 0x7fd66c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.213+0000 7fd6850af700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd66c06e140 0x7fd66c0705f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd670004890 tx=0x7fd670009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:07.215 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.217+0000 7fd6767fc700 1 -- 172.21.15.33:0/1441175511 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd67c056d30 con 0x7fd680102de0 2024-04-25T01:49:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.433+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd664000bc0 con 0x7fd66c06e140 2024-04-25T01:49:07.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.443+0000 7fd6767fc700 1 -- 172.21.15.33:0/1441175511 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd664000bc0 con 0x7fd66c06e140 2024-04-25T01:49:07.440 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (10m) 3m ago 21m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (7m) 3m ago 21m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (7m) 3m ago 20m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (9m) 3m ago 21m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (3m) 3m ago 16m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (3m) 3m ago 16m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (3m) 3m ago 16m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (15m) 3m ago 15m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (11m) 3m ago 22m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (11m) 3m ago 20m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (8m) 3m ago 23m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (8m) 3m ago 20m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:49:07.441 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (10m) 3m ago 21m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:49:07.442 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (10m) 3m ago 20m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:49:07.442 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (7m) 3m ago 19m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:49:07.442 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (6m) 3m ago 19m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:49:07.442 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (5m) 3m ago 18m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:49:07.442 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (5m) 3m ago 18m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:49:07.442 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (4m) 3m ago 18m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:49:07.442 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (4m) 3m ago 17m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:49:07.442 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (10m) 3m ago 20m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:49:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd66c06e140 msgr2=0x7fd66c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd66c06e140 0x7fd66c0705f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd670004890 tx=0x7fd670009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680102de0 msgr2=0x7fd680070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680102de0 0x7fd680070260 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00eb40 tx=0x7fd67c00b040 comp rx=0 tx=0).stop 2024-04-25T01:49:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 shutdown_connections 2024-04-25T01:49:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd66c06e140 0x7fd66c0705f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd680102de0 0x7fd680070260 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 --2- 172.21.15.33:0/1441175511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd680103780 0x7fd6800707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 >> 172.21.15.33:0/1441175511 conn(0x7fd6800f8990 msgr2=0x7fd6800f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:07.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 shutdown_connections 2024-04-25T01:49:07.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.445+0000 7fd687b14700 1 -- 172.21.15.33:0/1441175511 wait complete. 2024-04-25T01:49:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.549+0000 7ff97dd8b700 1 -- 172.21.15.33:0/1796620827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 msgr2=0x7ff9781020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.549+0000 7ff97dd8b700 1 --2- 172.21.15.33:0/1796620827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 0x7ff9781020e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff968004940 tx=0x7ff968009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.549+0000 7ff97dd8b700 1 -- 172.21.15.33:0/1796620827 shutdown_connections 2024-04-25T01:49:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.549+0000 7ff97dd8b700 1 --2- 172.21.15.33:0/1796620827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9781026b0 0x7ff9780fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.549+0000 7ff97dd8b700 1 --2- 172.21.15.33:0/1796620827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 0x7ff9781020e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.549+0000 7ff97dd8b700 1 -- 172.21.15.33:0/1796620827 >> 172.21.15.33:0/1796620827 conn(0x7ff9780f7860 msgr2=0x7ff9780f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.549+0000 7ff97dd8b700 1 -- 172.21.15.33:0/1796620827 shutdown_connections 2024-04-25T01:49:07.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.549+0000 7ff97dd8b700 1 -- 172.21.15.33:0/1796620827 wait complete. 2024-04-25T01:49:07.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.550+0000 7ff97dd8b700 1 Processor -- start 2024-04-25T01:49:07.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.550+0000 7ff97dd8b700 1 -- start start 2024-04-25T01:49:07.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff97dd8b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 0x7ff97810c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:07.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff97dd8b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9781026b0 0x7ff97810cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff97dd8b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff978067100 con 0x7ff978101d10 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff97dd8b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff978067240 con 0x7ff9781026b0 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff9777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 0x7ff97810c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff9777fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 0x7ff97810c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43096/0 (socket says 172.21.15.33:43096) 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff9777fe700 1 -- 172.21.15.33:0/2821815244 learned_addr learned my addr 172.21.15.33:0/2821815244 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff976ffd700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9781026b0 0x7ff97810cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff9777fe700 1 -- 172.21.15.33:0/2821815244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9781026b0 msgr2=0x7ff97810cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff9777fe700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9781026b0 0x7ff97810cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff9777fe700 1 -- 172.21.15.33:0/2821815244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968005040 con 0x7ff978101d10 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff9777fe700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 0x7ff97810c980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff96800b3b0 tx=0x7ff968009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff95ffff700 1 -- 172.21.15.33:0/2821815244 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96800e040 con 0x7ff978101d10 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff97810d4f0 con 0x7ff978101d10 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff95ffff700 1 -- 172.21.15.33:0/2821815244 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff968013420 con 0x7ff978101d10 2024-04-25T01:49:07.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.551+0000 7ff95ffff700 1 -- 172.21.15.33:0/2821815244 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96801c630 con 0x7ff978101d10 2024-04-25T01:49:07.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.552+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97811a1b0 con 0x7ff978101d10 2024-04-25T01:49:07.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.552+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9780fd460 con 0x7ff978101d10 2024-04-25T01:49:07.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.553+0000 7ff95ffff700 1 -- 172.21.15.33:0/2821815244 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff968013590 con 0x7ff978101d10 2024-04-25T01:49:07.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.554+0000 7ff95ffff700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff96006e140 0x7ff9600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:07.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.554+0000 7ff95ffff700 1 -- 172.21.15.33:0/2821815244 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff96808f740 con 0x7ff978101d10 2024-04-25T01:49:07.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.554+0000 7ff976ffd700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff96006e140 0x7ff9600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:07.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.554+0000 7ff976ffd700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff96006e140 0x7ff9600705f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff97810eae0 tx=0x7ff96c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:07.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.558+0000 7ff95ffff700 1 -- 172.21.15.33:0/2821815244 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff968058be0 con 0x7ff978101d10 2024-04-25T01:49:07.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.855+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff97804c710 con 0x7ff978101d10 2024-04-25T01:49:07.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.855+0000 7ff95ffff700 1 -- 172.21.15.33:0/2821815244 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff96805c2a0 con 0x7ff978101d10 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:49:07.853 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:49:07.854 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:49:07.854 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:49:07.854 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:49:07.854 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:49:07.854 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:49:07.854 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:49:07.854 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:49:07.854 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.857+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff96006e140 msgr2=0x7ff9600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.857+0000 7ff97dd8b700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff96006e140 0x7ff9600705f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff97810eae0 tx=0x7ff96c00b040 comp rx=0 tx=0).stop 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.857+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 msgr2=0x7ff97810c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.857+0000 7ff97dd8b700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 0x7ff97810c980 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff96800b3b0 tx=0x7ff968009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.858+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 shutdown_connections 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.858+0000 7ff97dd8b700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff96006e140 0x7ff9600705f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.858+0000 7ff97dd8b700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff978101d10 0x7ff97810c980 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.858+0000 7ff97dd8b700 1 --2- 172.21.15.33:0/2821815244 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff9781026b0 0x7ff97810cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.858+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 >> 172.21.15.33:0/2821815244 conn(0x7ff9780f7860 msgr2=0x7ff9780f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.858+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 shutdown_connections 2024-04-25T01:49:07.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.858+0000 7ff97dd8b700 1 -- 172.21.15.33:0/2821815244 wait complete. 2024-04-25T01:49:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.961+0000 7fb453635700 1 -- 172.21.15.33:0/308096671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c103780 msgr2=0x7fb44c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.961+0000 7fb453635700 1 --2- 172.21.15.33:0/308096671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c103780 0x7fb44c0fcdf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb43c004940 tx=0x7fb43c009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.962+0000 7fb453635700 1 -- 172.21.15.33:0/308096671 shutdown_connections 2024-04-25T01:49:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.962+0000 7fb453635700 1 --2- 172.21.15.33:0/308096671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c103780 0x7fb44c0fcdf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.962+0000 7fb453635700 1 --2- 172.21.15.33:0/308096671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb44c102de0 0x7fb44c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.962+0000 7fb453635700 1 -- 172.21.15.33:0/308096671 >> 172.21.15.33:0/308096671 conn(0x7fb44c0f8990 msgr2=0x7fb44c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.962+0000 7fb453635700 1 -- 172.21.15.33:0/308096671 shutdown_connections 2024-04-25T01:49:07.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.962+0000 7fb453635700 1 -- 172.21.15.33:0/308096671 wait complete. 2024-04-25T01:49:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.962+0000 7fb453635700 1 Processor -- start 2024-04-25T01:49:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.962+0000 7fb453635700 1 -- start start 2024-04-25T01:49:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb453635700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c102de0 0x7fb44c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb453635700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb44c103780 0x7fb44c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb453635700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c114520 con 0x7fb44c102de0 2024-04-25T01:49:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb453635700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44c10db20 con 0x7fb44c103780 2024-04-25T01:49:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb4513d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c102de0 0x7fb44c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:07.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb4513d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c102de0 0x7fb44c113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43124/0 (socket says 172.21.15.33:43124) 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb4513d1700 1 -- 172.21.15.33:0/3821871589 learned_addr learned my addr 172.21.15.33:0/3821871589 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb450bd0700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb44c103780 0x7fb44c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb4513d1700 1 -- 172.21.15.33:0/3821871589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb44c103780 msgr2=0x7fb44c113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb4513d1700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb44c103780 0x7fb44c113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb4513d1700 1 -- 172.21.15.33:0/3821871589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43c005040 con 0x7fb44c102de0 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb450bd0700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb44c103780 0x7fb44c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb4513d1700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c102de0 0x7fb44c113990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb448002a50 tx=0x7fb4480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb441ffb700 1 -- 172.21.15.33:0/3821871589 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb448007cc0 con 0x7fb44c102de0 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c10de00 con 0x7fb44c102de0 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb441ffb700 1 -- 172.21.15.33:0/3821871589 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb44800f040 con 0x7fb44c102de0 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.963+0000 7fb441ffb700 1 -- 172.21.15.33:0/3821871589 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb448009640 con 0x7fb44c102de0 2024-04-25T01:49:07.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.964+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44c10e3c0 con 0x7fb44c102de0 2024-04-25T01:49:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.964+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44c0fe530 con 0x7fb44c102de0 2024-04-25T01:49:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.965+0000 7fb441ffb700 1 -- 172.21.15.33:0/3821871589 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb4480097a0 con 0x7fb44c102de0 2024-04-25T01:49:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.965+0000 7fb441ffb700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb43806e140 0x7fb4380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.966+0000 7fb441ffb700 1 -- 172.21.15.33:0/3821871589 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb44808d940 con 0x7fb44c102de0 2024-04-25T01:49:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.966+0000 7fb450bd0700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb43806e140 0x7fb4380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:07.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.966+0000 7fb450bd0700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb43806e140 0x7fb4380705f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb44c10ece0 tx=0x7fb43c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:07.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:07.970+0000 7fb441ffb700 1 -- 172.21.15.33:0/3821871589 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb44805a7f0 con 0x7fb44c102de0 2024-04-25T01:49:08.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.219+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb44c109ec0 con 0x7fb44c102de0 2024-04-25T01:49:08.216 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:07 smithi033 ceph-mon[111306]: pgmap v291: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:08.216 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:07 smithi033 ceph-mon[111306]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:08.217 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:07 smithi033 ceph-mon[111306]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:08.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.219+0000 7fb441ffb700 1 -- 172.21.15.33:0/3821871589 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fb44805a380 con 0x7fb44c102de0 2024-04-25T01:49:08.217 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:49:08.217 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:49:08.217 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:49:08.217 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:49:08.217 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:49:08.217 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:49:08.218 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:49:08.219 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:49:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:49:08.220 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:49:08.220 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:49:08.220 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:49:08.220 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:49:08.220 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:49:08.220 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.221+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb43806e140 msgr2=0x7fb4380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.221+0000 7fb453635700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb43806e140 0x7fb4380705f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb44c10ece0 tx=0x7fb43c009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.221+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c102de0 msgr2=0x7fb44c113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.221+0000 7fb453635700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c102de0 0x7fb44c113990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb448002a50 tx=0x7fb4480046b0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.221+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 shutdown_connections 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.221+0000 7fb453635700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb43806e140 0x7fb4380705f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.221+0000 7fb453635700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb44c102de0 0x7fb44c113990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.221+0000 7fb453635700 1 --2- 172.21.15.33:0/3821871589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb44c103780 0x7fb44c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.221+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 >> 172.21.15.33:0/3821871589 conn(0x7fb44c0f8990 msgr2=0x7fb44c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.222+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 shutdown_connections 2024-04-25T01:49:08.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.222+0000 7fb453635700 1 -- 172.21.15.33:0/3821871589 wait complete. 2024-04-25T01:49:08.222 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:49:08.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:07 smithi138 ceph-mon[81304]: pgmap v291: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:08.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:07 smithi138 ceph-mon[81304]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:08.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:07 smithi138 ceph-mon[81304]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 -- 172.21.15.33:0/1794611829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0298102e00 msgr2=0x7f02981031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 --2- 172.21.15.33:0/1794611829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0298102e00 0x7f02981031d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f028c004940 tx=0x7f028c009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 -- 172.21.15.33:0/1794611829 shutdown_connections 2024-04-25T01:49:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 --2- 172.21.15.33:0/1794611829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02981037a0 0x7f02980fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 --2- 172.21.15.33:0/1794611829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0298102e00 0x7f02981031d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 -- 172.21.15.33:0/1794611829 >> 172.21.15.33:0/1794611829 conn(0x7f02980f89d0 msgr2=0x7f02980fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 -- 172.21.15.33:0/1794611829 shutdown_connections 2024-04-25T01:49:08.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 -- 172.21.15.33:0/1794611829 wait complete. 2024-04-25T01:49:08.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 Processor -- start 2024-04-25T01:49:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.327+0000 7f029e67c700 1 -- start start 2024-04-25T01:49:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f029e67c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102e00 0x7f0298113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f029e67c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02981037a0 0x7f0298113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f029e67c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02981144d0 con 0x7f02981037a0 2024-04-25T01:49:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f029e67c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029810dad0 con 0x7f0298102e00 2024-04-25T01:49:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f02977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02981037a0 0x7f0298113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f02977fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02981037a0 0x7f0298113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43140/0 (socket says 172.21.15.33:43140) 2024-04-25T01:49:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f02977fe700 1 -- 172.21.15.33:0/2645251514 learned_addr learned my addr 172.21.15.33:0/2645251514 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:08.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f0297fff700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102e00 0x7f0298113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f02977fe700 1 -- 172.21.15.33:0/2645251514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102e00 msgr2=0x7f0298113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f02977fe700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102e00 0x7f0298113940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f02977fe700 1 -- 172.21.15.33:0/2645251514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f028c005040 con 0x7f02981037a0 2024-04-25T01:49:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f02977fe700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02981037a0 0x7f0298113e80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f028800c800 tx=0x7f0288004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.328+0000 7f0294ff9700 1 -- 172.21.15.33:0/2645251514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f028800a760 con 0x7f02981037a0 2024-04-25T01:49:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.329+0000 7f0294ff9700 1 -- 172.21.15.33:0/2645251514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f028800ad90 con 0x7f02981037a0 2024-04-25T01:49:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.329+0000 7f0294ff9700 1 -- 172.21.15.33:0/2645251514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02880113a0 con 0x7f02981037a0 2024-04-25T01:49:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.329+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029810ddb0 con 0x7f02981037a0 2024-04-25T01:49:08.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.329+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029810e370 con 0x7f02981037a0 2024-04-25T01:49:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.330+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0298100110 con 0x7f02981037a0 2024-04-25T01:49:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.330+0000 7f0294ff9700 1 -- 172.21.15.33:0/2645251514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0288012400 con 0x7f02981037a0 2024-04-25T01:49:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.331+0000 7f0294ff9700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028006e140 0x7f02800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.331+0000 7f0294ff9700 1 -- 172.21.15.33:0/2645251514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f028808db40 con 0x7f02981037a0 2024-04-25T01:49:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.331+0000 7f0297fff700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028006e140 0x7f02800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:08.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.331+0000 7f0297fff700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028006e140 0x7f02800705f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f028c004f90 tx=0x7f028c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:08.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.335+0000 7f0294ff9700 1 -- 172.21.15.33:0/2645251514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f028805aa70 con 0x7f02981037a0 2024-04-25T01:49:08.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.561+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f029810f000 con 0x7f028006e140 2024-04-25T01:49:08.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.562+0000 7f0294ff9700 1 -- 172.21.15.33:0/2645251514 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f029810f000 con 0x7f028006e140 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:49:08.560 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:49:08.561 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.564+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028006e140 msgr2=0x7f02800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028006e140 0x7f02800705f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f028c004f90 tx=0x7f028c009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02981037a0 msgr2=0x7f0298113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02981037a0 0x7f0298113e80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f028800c800 tx=0x7f0288004470 comp rx=0 tx=0).stop 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 shutdown_connections 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028006e140 0x7f02800705f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102e00 0x7f0298113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 --2- 172.21.15.33:0/2645251514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02981037a0 0x7f0298113e80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 >> 172.21.15.33:0/2645251514 conn(0x7f02980f89d0 msgr2=0x7f02980fa770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:08.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 shutdown_connections 2024-04-25T01:49:08.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.565+0000 7f029e67c700 1 -- 172.21.15.33:0/2645251514 wait complete. 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.670+0000 7f165683e700 1 -- 172.21.15.33:0/2094204987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 msgr2=0x7f16500fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.670+0000 7f165683e700 1 --2- 172.21.15.33:0/2094204987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 0x7f16500fcdf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1644004940 tx=0x7f1644009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.670+0000 7f165683e700 1 -- 172.21.15.33:0/2094204987 shutdown_connections 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.670+0000 7f165683e700 1 --2- 172.21.15.33:0/2094204987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 0x7f16500fcdf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.670+0000 7f165683e700 1 --2- 172.21.15.33:0/2094204987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1650102de0 0x7f16501031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.670+0000 7f165683e700 1 -- 172.21.15.33:0/2094204987 >> 172.21.15.33:0/2094204987 conn(0x7f16500f8990 msgr2=0x7f16500fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.671+0000 7f165683e700 1 -- 172.21.15.33:0/2094204987 shutdown_connections 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.671+0000 7f165683e700 1 -- 172.21.15.33:0/2094204987 wait complete. 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.671+0000 7f165683e700 1 Processor -- start 2024-04-25T01:49:08.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.671+0000 7f165683e700 1 -- start start 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f165683e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1650102de0 0x7f1650113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f165683e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 0x7f1650113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f165683e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1650114490 con 0x7f1650103780 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f165683e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f165010da90 con 0x7f1650102de0 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f164f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 0x7f1650113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f164f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 0x7f1650113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43162/0 (socket says 172.21.15.33:43162) 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f164f7fe700 1 -- 172.21.15.33:0/2796398087 learned_addr learned my addr 172.21.15.33:0/2796398087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f164ffff700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1650102de0 0x7f1650113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f164f7fe700 1 -- 172.21.15.33:0/2796398087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1650102de0 msgr2=0x7f1650113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f164f7fe700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1650102de0 0x7f1650113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f164f7fe700 1 -- 172.21.15.33:0/2796398087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1644005040 con 0x7f1650103780 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f164f7fe700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 0x7f1650113e40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1644000c00 tx=0x7f1644009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.672+0000 7f164cff9700 1 -- 172.21.15.33:0/2796398087 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1644021070 con 0x7f1650103780 2024-04-25T01:49:08.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.673+0000 7f164cff9700 1 -- 172.21.15.33:0/2796398087 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1644013c00 con 0x7f1650103780 2024-04-25T01:49:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.673+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f165010dd10 con 0x7f1650103780 2024-04-25T01:49:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.673+0000 7f164cff9700 1 -- 172.21.15.33:0/2796398087 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f164400e070 con 0x7f1650103780 2024-04-25T01:49:08.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.673+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f165010e1f0 con 0x7f1650103780 2024-04-25T01:49:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.674+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16500fe530 con 0x7f1650103780 2024-04-25T01:49:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.674+0000 7f164cff9700 1 -- 172.21.15.33:0/2796398087 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f164401ca60 con 0x7f1650103780 2024-04-25T01:49:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.675+0000 7f164cff9700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f163806e140 0x7f16380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.675+0000 7f164cff9700 1 -- 172.21.15.33:0/2796398087 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f164408ee50 con 0x7f1650103780 2024-04-25T01:49:08.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.675+0000 7f164ffff700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f163806e140 0x7f16380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:08.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.675+0000 7f164ffff700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f163806e140 0x7f16380705f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1640009b80 tx=0x7f164000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:08.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.679+0000 7f164cff9700 1 -- 172.21.15.33:0/2796398087 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f164405be20 con 0x7f1650103780 2024-04-25T01:49:08.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.982+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1650109ec0 con 0x7f1650103780 2024-04-25T01:49:08.979 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:08 smithi033 ceph-mon[111306]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:08.979 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:08 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2821815244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:49:08.979 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:08 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3821871589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:49:08.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.982+0000 7f164cff9700 1 -- 172.21.15.33:0/2796398087 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f1644004490 con 0x7f1650103780 2024-04-25T01:49:08.980 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:49:08.980 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:49:08.980 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:49:08.980 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:49:08.980 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.984+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f163806e140 msgr2=0x7f16380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.984+0000 7f165683e700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f163806e140 0x7f16380705f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1640009b80 tx=0x7f164000b040 comp rx=0 tx=0).stop 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.984+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 msgr2=0x7f1650113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.984+0000 7f165683e700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 0x7f1650113e40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1644000c00 tx=0x7f1644009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.984+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 shutdown_connections 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.984+0000 7f165683e700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f163806e140 0x7f16380705f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.984+0000 7f165683e700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1650102de0 0x7f1650113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.984+0000 7f165683e700 1 --2- 172.21.15.33:0/2796398087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1650103780 0x7f1650113e40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.984+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 >> 172.21.15.33:0/2796398087 conn(0x7f16500f8990 msgr2=0x7f16500f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.985+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 shutdown_connections 2024-04-25T01:49:08.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:08.985+0000 7f165683e700 1 -- 172.21.15.33:0/2796398087 wait complete. 2024-04-25T01:49:09.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:08 smithi138 ceph-mon[81304]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:09.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:08 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2821815244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:49:09.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:08 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3821871589' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:49:10.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:09 smithi033 ceph-mon[111306]: pgmap v292: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:10.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:09 smithi033 ceph-mon[111306]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:10.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:09 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2796398087' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:49:10.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:09 smithi138 ceph-mon[81304]: pgmap v292: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:10.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:09 smithi138 ceph-mon[81304]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:10.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:09 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2796398087' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:49:11.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:49:11.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:49:12.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:11 smithi033 ceph-mon[111306]: pgmap v293: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:12.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:11 smithi138 ceph-mon[81304]: pgmap v293: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:14.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:13 smithi033 ceph-mon[111306]: pgmap v294: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:14.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:13 smithi138 ceph-mon[81304]: pgmap v294: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:16.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:15 smithi033 ceph-mon[111306]: pgmap v295: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:16.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:15 smithi138 ceph-mon[81304]: pgmap v295: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:18.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:17 smithi033 ceph-mon[111306]: pgmap v296: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:18.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:17 smithi138 ceph-mon[81304]: pgmap v296: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:20.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:19 smithi033 ceph-mon[111306]: pgmap v297: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:20.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:19 smithi138 ceph-mon[81304]: pgmap v297: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:22.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:21 smithi033 ceph-mon[111306]: pgmap v298: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:22.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:21 smithi138 ceph-mon[81304]: pgmap v298: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:24.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:23 smithi033 ceph-mon[111306]: pgmap v299: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:24.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:23 smithi138 ceph-mon[81304]: pgmap v299: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:26.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:25 smithi033 ceph-mon[111306]: pgmap v300: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:26.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:49:26.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:25 smithi138 ceph-mon[81304]: pgmap v300: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:26.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:49:28.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:27 smithi033 ceph-mon[111306]: pgmap v301: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:28.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:27 smithi138 ceph-mon[81304]: pgmap v301: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:30.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:29 smithi033 ceph-mon[111306]: pgmap v302: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:30.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:29 smithi138 ceph-mon[81304]: pgmap v302: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:32.164 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:31 smithi033 ceph-mon[111306]: pgmap v303: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:32.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:31 smithi138 ceph-mon[81304]: pgmap v303: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:34.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:33 smithi033 ceph-mon[111306]: pgmap v304: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:34.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:33 smithi138 ceph-mon[81304]: pgmap v304: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:36.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:35 smithi033 ceph-mon[111306]: pgmap v305: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:36.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:35 smithi138 ceph-mon[81304]: pgmap v305: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:38.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:37 smithi033 ceph-mon[111306]: pgmap v306: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:38.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:37 smithi138 ceph-mon[81304]: pgmap v306: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:39.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.103+0000 7f20b8868700 1 -- 172.21.15.33:0/2057304793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b41026b0 msgr2=0x7f20b40fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.103+0000 7f20b8868700 1 --2- 172.21.15.33:0/2057304793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b41026b0 0x7f20b40fbd20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f20a8004940 tx=0x7f20a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:39.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.103+0000 7f20b8868700 1 -- 172.21.15.33:0/2057304793 shutdown_connections 2024-04-25T01:49:39.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.103+0000 7f20b8868700 1 --2- 172.21.15.33:0/2057304793 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b41026b0 0x7f20b40fbd20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.103+0000 7f20b8868700 1 --2- 172.21.15.33:0/2057304793 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b4101d10 0x7f20b41020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.103+0000 7f20b8868700 1 -- 172.21.15.33:0/2057304793 >> 172.21.15.33:0/2057304793 conn(0x7f20b40f78a0 msgr2=0x7f20b40f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:39.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.103+0000 7f20b8868700 1 -- 172.21.15.33:0/2057304793 shutdown_connections 2024-04-25T01:49:39.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.103+0000 7f20b8868700 1 -- 172.21.15.33:0/2057304793 wait complete. 2024-04-25T01:49:39.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b8868700 1 Processor -- start 2024-04-25T01:49:39.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b8868700 1 -- start start 2024-04-25T01:49:39.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b8868700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b4101d10 0x7f20b4112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:39.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b8868700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b41026b0 0x7f20b4112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:39.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b8868700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4113410 con 0x7f20b4101d10 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b8868700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b410ca10 con 0x7f20b41026b0 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b4101d10 0x7f20b4112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b4101d10 0x7f20b4112880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53242/0 (socket says 172.21.15.33:53242) 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b259c700 1 -- 172.21.15.33:0/3653589467 learned_addr learned my addr 172.21.15.33:0/3653589467 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.104+0000 7f20b1d9b700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b41026b0 0x7f20b4112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.105+0000 7f20b259c700 1 -- 172.21.15.33:0/3653589467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b41026b0 msgr2=0x7f20b4112dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.105+0000 7f20b259c700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b41026b0 0x7f20b4112dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.105+0000 7f20b259c700 1 -- 172.21.15.33:0/3653589467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a8005040 con 0x7f20b4101d10 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.105+0000 7f20b259c700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b4101d10 0x7f20b4112880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20a4002a50 tx=0x7f20a40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.105+0000 7f20a2ffd700 1 -- 172.21.15.33:0/3653589467 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a4007cc0 con 0x7f20b4101d10 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.105+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b410ccf0 con 0x7f20b4101d10 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.105+0000 7f20a2ffd700 1 -- 172.21.15.33:0/3653589467 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20a400f040 con 0x7f20b4101d10 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.105+0000 7f20a2ffd700 1 -- 172.21.15.33:0/3653589467 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a400da30 con 0x7f20b4101d10 2024-04-25T01:49:39.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.105+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b410d280 con 0x7f20b4101d10 2024-04-25T01:49:39.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.106+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b40fd460 con 0x7f20b4101d10 2024-04-25T01:49:39.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.107+0000 7f20a2ffd700 1 -- 172.21.15.33:0/3653589467 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f20a400db90 con 0x7f20b4101d10 2024-04-25T01:49:39.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.107+0000 7f20a2ffd700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f209c06e140 0x7f209c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:39.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.107+0000 7f20a2ffd700 1 -- 172.21.15.33:0/3653589467 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f20a408d8c0 con 0x7f20b4101d10 2024-04-25T01:49:39.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.107+0000 7f20b1d9b700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f209c06e140 0x7f209c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:39.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.108+0000 7f20b1d9b700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f209c06e140 0x7f209c0705f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f20a8004890 tx=0x7f20a8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:39.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.112+0000 7f20a2ffd700 1 -- 172.21.15.33:0/3653589467 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f20a405a770 con 0x7f20b4101d10 2024-04-25T01:49:39.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.338+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20b4108e90 con 0x7f209c06e140 2024-04-25T01:49:39.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.340+0000 7f20a2ffd700 1 -- 172.21.15.33:0/3653589467 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f20a800b5c0 con 0x7f209c06e140 2024-04-25T01:49:39.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f209c06e140 msgr2=0x7f209c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f209c06e140 0x7f209c0705f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f20a8004890 tx=0x7f20a8009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:39.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b4101d10 msgr2=0x7f20b4112880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b4101d10 0x7f20b4112880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f20a4002a50 tx=0x7f20a40046b0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 shutdown_connections 2024-04-25T01:49:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f209c06e140 0x7f209c0705f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20b4101d10 0x7f20b4112880 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 --2- 172.21.15.33:0/3653589467 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20b41026b0 0x7f20b4112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 >> 172.21.15.33:0/3653589467 conn(0x7f20b40f78a0 msgr2=0x7f20b40f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 shutdown_connections 2024-04-25T01:49:39.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.342+0000 7f20b8868700 1 -- 172.21.15.33:0/3653589467 wait complete. 2024-04-25T01:49:39.350 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:49:39.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.447+0000 7f6803dbe700 1 -- 172.21.15.33:0/3556097745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc103780 msgr2=0x7f67fc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.447+0000 7f6803dbe700 1 --2- 172.21.15.33:0/3556097745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc103780 0x7f67fc0fcdf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f67f0004940 tx=0x7f67f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:39.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.447+0000 7f6803dbe700 1 -- 172.21.15.33:0/3556097745 shutdown_connections 2024-04-25T01:49:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.447+0000 7f6803dbe700 1 --2- 172.21.15.33:0/3556097745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc103780 0x7f67fc0fcdf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.447+0000 7f6803dbe700 1 --2- 172.21.15.33:0/3556097745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc102de0 0x7f67fc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.447+0000 7f6803dbe700 1 -- 172.21.15.33:0/3556097745 >> 172.21.15.33:0/3556097745 conn(0x7f67fc0f8990 msgr2=0x7f67fc0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.447+0000 7f6803dbe700 1 -- 172.21.15.33:0/3556097745 shutdown_connections 2024-04-25T01:49:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.447+0000 7f6803dbe700 1 -- 172.21.15.33:0/3556097745 wait complete. 2024-04-25T01:49:39.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.447+0000 7f6803dbe700 1 Processor -- start 2024-04-25T01:49:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6803dbe700 1 -- start start 2024-04-25T01:49:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6803dbe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc102de0 0x7f67fc1138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6803dbe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc103780 0x7f67fc113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6803dbe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc114460 con 0x7f67fc102de0 2024-04-25T01:49:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6803dbe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc10da60 con 0x7f67fc103780 2024-04-25T01:49:39.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6801b5a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc102de0 0x7f67fc1138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6801b5a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc102de0 0x7f67fc1138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53268/0 (socket says 172.21.15.33:53268) 2024-04-25T01:49:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6801b5a700 1 -- 172.21.15.33:0/2316101616 learned_addr learned my addr 172.21.15.33:0/2316101616 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6801359700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc103780 0x7f67fc113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6801b5a700 1 -- 172.21.15.33:0/2316101616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc103780 msgr2=0x7f67fc113e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6801b5a700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc103780 0x7f67fc113e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6801b5a700 1 -- 172.21.15.33:0/2316101616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0005040 con 0x7f67fc102de0 2024-04-25T01:49:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.448+0000 7f6801b5a700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc102de0 0x7f67fc1138d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f67f400ec90 tx=0x7f67f400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.449+0000 7f67fa7fc700 1 -- 172.21.15.33:0/2316101616 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f4007b30 con 0x7f67fc102de0 2024-04-25T01:49:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.449+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc10dd40 con 0x7f67fc102de0 2024-04-25T01:49:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.449+0000 7f67fa7fc700 1 -- 172.21.15.33:0/2316101616 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67f400f040 con 0x7f67fc102de0 2024-04-25T01:49:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.449+0000 7f67fa7fc700 1 -- 172.21.15.33:0/2316101616 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f400cc80 con 0x7f67fc102de0 2024-04-25T01:49:39.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.449+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc10e250 con 0x7f67fc102de0 2024-04-25T01:49:39.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.450+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc0fe530 con 0x7f67fc102de0 2024-04-25T01:49:39.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.450+0000 7f67fa7fc700 1 -- 172.21.15.33:0/2316101616 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f67f4007c90 con 0x7f67fc102de0 2024-04-25T01:49:39.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.451+0000 7f67fa7fc700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f67e406e140 0x7f67e40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:39.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.451+0000 7f67fa7fc700 1 -- 172.21.15.33:0/2316101616 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f67f408df50 con 0x7f67fc102de0 2024-04-25T01:49:39.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.451+0000 7f6801359700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f67e406e140 0x7f67e40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:39.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.451+0000 7f6801359700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f67e406e140 0x7f67e40705f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67f0004890 tx=0x7f67f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:39.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.456+0000 7f67fa7fc700 1 -- 172.21.15.33:0/2316101616 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f67f40c2740 con 0x7f67fc102de0 2024-04-25T01:49:39.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.681+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67fc109f10 con 0x7f67e406e140 2024-04-25T01:49:39.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.682+0000 7f67fa7fc700 1 -- 172.21.15.33:0/2316101616 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f67fc109f10 con 0x7f67e406e140 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f67e406e140 msgr2=0x7f67e40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f67e406e140 0x7f67e40705f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f67f0004890 tx=0x7f67f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc102de0 msgr2=0x7f67fc1138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc102de0 0x7f67fc1138d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f67f400ec90 tx=0x7f67f400b040 comp rx=0 tx=0).stop 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 shutdown_connections 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f67e406e140 0x7f67e40705f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f67fc102de0 0x7f67fc1138d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 --2- 172.21.15.33:0/2316101616 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f67fc103780 0x7f67fc113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 >> 172.21.15.33:0/2316101616 conn(0x7f67fc0f8990 msgr2=0x7f67fc0f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 shutdown_connections 2024-04-25T01:49:39.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.684+0000 7f6803dbe700 1 -- 172.21.15.33:0/2316101616 wait complete. 2024-04-25T01:49:39.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.788+0000 7fe37347f700 1 -- 172.21.15.33:0/4033610320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 msgr2=0x7fe36c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.788+0000 7fe37347f700 1 --2- 172.21.15.33:0/4033610320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 0x7fe36c1031d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe368004940 tx=0x7fe368009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.788+0000 7fe37347f700 1 -- 172.21.15.33:0/4033610320 shutdown_connections 2024-04-25T01:49:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 --2- 172.21.15.33:0/4033610320 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c1037a0 0x7fe36c0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 --2- 172.21.15.33:0/4033610320 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 0x7fe36c1031d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 -- 172.21.15.33:0/4033610320 >> 172.21.15.33:0/4033610320 conn(0x7fe36c0f89d0 msgr2=0x7fe36c0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 -- 172.21.15.33:0/4033610320 shutdown_connections 2024-04-25T01:49:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 -- 172.21.15.33:0/4033610320 wait complete. 2024-04-25T01:49:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 Processor -- start 2024-04-25T01:49:39.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 -- start start 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 0x7fe36c115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c1037a0 0x7fe36c116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c1166e0 con 0x7fe36c102e00 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.789+0000 7fe37347f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36c10fcc0 con 0x7fe36c1037a0 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe37121b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 0x7fe36c115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe37121b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 0x7fe36c115b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53294/0 (socket says 172.21.15.33:53294) 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe37121b700 1 -- 172.21.15.33:0/2546905118 learned_addr learned my addr 172.21.15.33:0/2546905118 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe370a1a700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c1037a0 0x7fe36c116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe37121b700 1 -- 172.21.15.33:0/2546905118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c1037a0 msgr2=0x7fe36c116090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe37121b700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c1037a0 0x7fe36c116090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe37121b700 1 -- 172.21.15.33:0/2546905118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368005040 con 0x7fe36c102e00 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe37121b700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 0x7fe36c115b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe368009150 tx=0x7fe368009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:39.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe361ffb700 1 -- 172.21.15.33:0/2546905118 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe36800e040 con 0x7fe36c102e00 2024-04-25T01:49:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36c10ff40 con 0x7fe36c102e00 2024-04-25T01:49:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe361ffb700 1 -- 172.21.15.33:0/2546905118 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe368013420 con 0x7fe36c102e00 2024-04-25T01:49:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.790+0000 7fe361ffb700 1 -- 172.21.15.33:0/2546905118 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe36801c610 con 0x7fe36c102e00 2024-04-25T01:49:39.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.791+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36c110500 con 0x7fe36c102e00 2024-04-25T01:49:39.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.792+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36c107300 con 0x7fe36c102e00 2024-04-25T01:49:39.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.792+0000 7fe361ffb700 1 -- 172.21.15.33:0/2546905118 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe36800baf0 con 0x7fe36c102e00 2024-04-25T01:49:39.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.792+0000 7fe361ffb700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35806e140 0x7fe3580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:39.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.792+0000 7fe361ffb700 1 -- 172.21.15.33:0/2546905118 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe36808ed40 con 0x7fe36c102e00 2024-04-25T01:49:39.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.793+0000 7fe370a1a700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35806e140 0x7fe3580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:39.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.793+0000 7fe370a1a700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35806e140 0x7fe3580705f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe36c1113f0 tx=0x7fe35c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:39.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:39.797+0000 7fe361ffb700 1 -- 172.21.15.33:0/2546905118 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe36805bc70 con 0x7fe36c102e00 2024-04-25T01:49:40.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.013+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe36c109f00 con 0x7fe35806e140 2024-04-25T01:49:40.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.024+0000 7fe361ffb700 1 -- 172.21.15.33:0/2546905118 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe36c109f00 con 0x7fe35806e140 2024-04-25T01:49:40.021 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:49:40.021 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (10m) 3m ago 21m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (8m) 3m ago 22m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (7m) 3m ago 21m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (10m) 3m ago 21m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (4m) 3m ago 16m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (4m) 3m ago 16m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (3m) 3m ago 16m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (16m) 3m ago 16m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (12m) 3m ago 23m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (11m) 3m ago 21m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (9m) 3m ago 23m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (8m) 3m ago 21m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (11m) 3m ago 22m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:49:40.022 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (11m) 3m ago 21m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:49:40.023 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (7m) 3m ago 20m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:49:40.023 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (6m) 3m ago 19m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:49:40.023 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (6m) 3m ago 19m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:49:40.023 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (5m) 3m ago 18m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:49:40.023 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (5m) 3m ago 18m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:49:40.023 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (4m) 3m ago 18m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:49:40.023 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (11m) 3m ago 21m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:49:40.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35806e140 msgr2=0x7fe3580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35806e140 0x7fe3580705f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe36c1113f0 tx=0x7fe35c00b040 comp rx=0 tx=0).stop 2024-04-25T01:49:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 msgr2=0x7fe36c115b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 0x7fe36c115b50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe368009150 tx=0x7fe368009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 shutdown_connections 2024-04-25T01:49:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35806e140 0x7fe3580705f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe36c102e00 0x7fe36c115b50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 --2- 172.21.15.33:0/2546905118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe36c1037a0 0x7fe36c116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 >> 172.21.15.33:0/2546905118 conn(0x7fe36c0f89d0 msgr2=0x7fe36c0fa560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 shutdown_connections 2024-04-25T01:49:40.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.026+0000 7fe37347f700 1 -- 172.21.15.33:0/2546905118 wait complete. 2024-04-25T01:49:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.129+0000 7fae5f28c700 1 -- 172.21.15.33:0/1612497679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58103780 msgr2=0x7fae580fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.129+0000 7fae5f28c700 1 --2- 172.21.15.33:0/1612497679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58103780 0x7fae580fcdf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fae48004940 tx=0x7fae48009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.129+0000 7fae5f28c700 1 -- 172.21.15.33:0/1612497679 shutdown_connections 2024-04-25T01:49:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.129+0000 7fae5f28c700 1 --2- 172.21.15.33:0/1612497679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58103780 0x7fae580fcdf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.129+0000 7fae5f28c700 1 --2- 172.21.15.33:0/1612497679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58102de0 0x7fae581031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.129+0000 7fae5f28c700 1 -- 172.21.15.33:0/1612497679 >> 172.21.15.33:0/1612497679 conn(0x7fae580f8990 msgr2=0x7fae580fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.129+0000 7fae5f28c700 1 -- 172.21.15.33:0/1612497679 shutdown_connections 2024-04-25T01:49:40.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.129+0000 7fae5f28c700 1 -- 172.21.15.33:0/1612497679 wait complete. 2024-04-25T01:49:40.128 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:39 smithi033 ceph-mon[111306]: pgmap v307: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:40.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5f28c700 1 Processor -- start 2024-04-25T01:49:40.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5f28c700 1 -- start start 2024-04-25T01:49:40.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5f28c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58102de0 0x7fae58070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:40.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5f28c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58103780 0x7fae580707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:40.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5f28c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580721b0 con 0x7fae58102de0 2024-04-25T01:49:40.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5f28c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae58070ce0 con 0x7fae58103780 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5d028700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58102de0 0x7fae58070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5d028700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58102de0 0x7fae58070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53308/0 (socket says 172.21.15.33:53308) 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5d028700 1 -- 172.21.15.33:0/3653006603 learned_addr learned my addr 172.21.15.33:0/3653006603 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5c827700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58103780 0x7fae580707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5d028700 1 -- 172.21.15.33:0/3653006603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58103780 msgr2=0x7fae580707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5d028700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58103780 0x7fae580707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.130+0000 7fae5d028700 1 -- 172.21.15.33:0/3653006603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae48005040 con 0x7fae58102de0 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.131+0000 7fae5d028700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58102de0 0x7fae58070260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae5400c830 tx=0x7fae54004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.131+0000 7fae4dffb700 1 -- 172.21.15.33:0/3653006603 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae540106f0 con 0x7fae58102de0 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.131+0000 7fae4dffb700 1 -- 172.21.15.33:0/3653006603 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae54010d20 con 0x7fae58102de0 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.131+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae58070fc0 con 0x7fae58102de0 2024-04-25T01:49:40.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.131+0000 7fae4dffb700 1 -- 172.21.15.33:0/3653006603 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae540113a0 con 0x7fae58102de0 2024-04-25T01:49:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.131+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5811a0a0 con 0x7fae58102de0 2024-04-25T01:49:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.132+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae580fe530 con 0x7fae58102de0 2024-04-25T01:49:40.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.133+0000 7fae4dffb700 1 -- 172.21.15.33:0/3653006603 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fae54012400 con 0x7fae58102de0 2024-04-25T01:49:40.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.133+0000 7fae4dffb700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fae4406e140 0x7fae440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:40.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.133+0000 7fae4dffb700 1 -- 172.21.15.33:0/3653006603 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fae5408da70 con 0x7fae58102de0 2024-04-25T01:49:40.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.133+0000 7fae5c827700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fae4406e140 0x7fae440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:40.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.134+0000 7fae5c827700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fae4406e140 0x7fae440705f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae48004890 tx=0x7fae48009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:40.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.138+0000 7fae4dffb700 1 -- 172.21.15.33:0/3653006603 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fae54056e70 con 0x7fae58102de0 2024-04-25T01:49:40.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:39 smithi138 ceph-mon[81304]: pgmap v307: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.436+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fae5811a2f0 con 0x7fae58102de0 2024-04-25T01:49:40.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.436+0000 7fae4dffb700 1 -- 172.21.15.33:0/3653006603 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fae5405a530 con 0x7fae58102de0 2024-04-25T01:49:40.434 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:49:40.434 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:49:40.434 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:49:40.434 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:49:40.434 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:49:40.434 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:49:40.435 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:49:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fae4406e140 msgr2=0x7fae440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fae4406e140 0x7fae440705f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fae48004890 tx=0x7fae48009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58102de0 msgr2=0x7fae58070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58102de0 0x7fae58070260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fae5400c830 tx=0x7fae54004470 comp rx=0 tx=0).stop 2024-04-25T01:49:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 shutdown_connections 2024-04-25T01:49:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fae4406e140 0x7fae440705f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fae58102de0 0x7fae58070260 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 --2- 172.21.15.33:0/3653006603 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fae58103780 0x7fae580707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 >> 172.21.15.33:0/3653006603 conn(0x7fae580f8990 msgr2=0x7fae580f9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 shutdown_connections 2024-04-25T01:49:40.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.438+0000 7fae5f28c700 1 -- 172.21.15.33:0/3653006603 wait complete. 2024-04-25T01:49:40.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 -- 172.21.15.33:0/4165919446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8103780 msgr2=0x7f7be80fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 --2- 172.21.15.33:0/4165919446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8103780 0x7f7be80fcdf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc004940 tx=0x7f7bdc009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 -- 172.21.15.33:0/4165919446 shutdown_connections 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 --2- 172.21.15.33:0/4165919446 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8103780 0x7f7be80fcdf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 --2- 172.21.15.33:0/4165919446 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8102de0 0x7f7be81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 -- 172.21.15.33:0/4165919446 >> 172.21.15.33:0/4165919446 conn(0x7f7be80f8990 msgr2=0x7f7be80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 -- 172.21.15.33:0/4165919446 shutdown_connections 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 -- 172.21.15.33:0/4165919446 wait complete. 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 Processor -- start 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.543+0000 7f7bf0076700 1 -- start start 2024-04-25T01:49:40.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bf0076700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8102de0 0x7f7be8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bf0076700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8103780 0x7f7be80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bf0076700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be80721b0 con 0x7f7be8102de0 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bf0076700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be8070ce0 con 0x7f7be8103780 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bede12700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8102de0 0x7f7be8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bede12700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8102de0 0x7f7be8070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53328/0 (socket says 172.21.15.33:53328) 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bede12700 1 -- 172.21.15.33:0/3698339821 learned_addr learned my addr 172.21.15.33:0/3698339821 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bed611700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8103780 0x7f7be80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bede12700 1 -- 172.21.15.33:0/3698339821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8103780 msgr2=0x7f7be80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bede12700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8103780 0x7f7be80707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bede12700 1 -- 172.21.15.33:0/3698339821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc005040 con 0x7f7be8102de0 2024-04-25T01:49:40.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7bede12700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8102de0 0x7f7be8070260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7be000cac0 tx=0x7f7be0007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:40.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.544+0000 7f7be67fc700 1 -- 172.21.15.33:0/3698339821 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7be000a980 con 0x7f7be8102de0 2024-04-25T01:49:40.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.545+0000 7f7be67fc700 1 -- 172.21.15.33:0/3698339821 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7be0012440 con 0x7f7be8102de0 2024-04-25T01:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.545+0000 7f7be67fc700 1 -- 172.21.15.33:0/3698339821 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7be00115e0 con 0x7f7be8102de0 2024-04-25T01:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.545+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be8070fc0 con 0x7f7be8102de0 2024-04-25T01:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.545+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be811a0e0 con 0x7f7be8102de0 2024-04-25T01:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.546+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be80fe530 con 0x7f7be8102de0 2024-04-25T01:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.546+0000 7f7be67fc700 1 -- 172.21.15.33:0/3698339821 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7be00125b0 con 0x7f7be8102de0 2024-04-25T01:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.547+0000 7f7be67fc700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7bd006e140 0x7f7bd00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.547+0000 7f7be67fc700 1 -- 172.21.15.33:0/3698339821 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7be008e220 con 0x7f7be8102de0 2024-04-25T01:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.547+0000 7f7bed611700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7bd006e140 0x7f7bd00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:40.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.547+0000 7f7bed611700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7bd006e140 0x7f7bd00705f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7be8071990 tx=0x7f7bdc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:40.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.551+0000 7f7be67fc700 1 -- 172.21.15.33:0/3698339821 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7be00575a0 con 0x7f7be8102de0 2024-04-25T01:49:40.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.808+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7be811a380 con 0x7f7be8102de0 2024-04-25T01:49:40.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.809+0000 7f7be67fc700 1 -- 172.21.15.33:0/3698339821 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f7be0012860 con 0x7f7be8102de0 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:49:40.807 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:49:40.808 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:49:40.809 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:49:40.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7bd006e140 msgr2=0x7f7bd00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7bd006e140 0x7f7bd00705f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7be8071990 tx=0x7f7bdc009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8102de0 msgr2=0x7f7be8070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8102de0 0x7f7be8070260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7be000cac0 tx=0x7f7be0007ee0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 shutdown_connections 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7bd006e140 0x7f7bd00705f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7be8102de0 0x7f7be8070260 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 --2- 172.21.15.33:0/3698339821 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7be8103780 0x7f7be80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 >> 172.21.15.33:0/3698339821 conn(0x7f7be80f8990 msgr2=0x7f7be80f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 shutdown_connections 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.811+0000 7f7bf0076700 1 -- 172.21.15.33:0/3698339821 wait complete. 2024-04-25T01:49:40.811 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:49:40.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 -- 172.21.15.33:0/1819801606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 msgr2=0x7fac1c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 --2- 172.21.15.33:0/1819801606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 0x7fac1c1031b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fac18004940 tx=0x7fac18009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:40.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 -- 172.21.15.33:0/1819801606 shutdown_connections 2024-04-25T01:49:40.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 --2- 172.21.15.33:0/1819801606 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c103780 0x7fac1c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 --2- 172.21.15.33:0/1819801606 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 0x7fac1c1031b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 -- 172.21.15.33:0/1819801606 >> 172.21.15.33:0/1819801606 conn(0x7fac1c0f8990 msgr2=0x7fac1c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 -- 172.21.15.33:0/1819801606 shutdown_connections 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 -- 172.21.15.33:0/1819801606 wait complete. 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 Processor -- start 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.917+0000 7fac238c9700 1 -- start start 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac238c9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 0x7fac1c1138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac238c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c103780 0x7fac1c113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac238c9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c114440 con 0x7fac1c102de0 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac238c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c10da90 con 0x7fac1c103780 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac21665700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 0x7fac1c1138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac21665700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 0x7fac1c1138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53356/0 (socket says 172.21.15.33:53356) 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac21665700 1 -- 172.21.15.33:0/3266344898 learned_addr learned my addr 172.21.15.33:0/3266344898 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac20e64700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c103780 0x7fac1c113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac21665700 1 -- 172.21.15.33:0/3266344898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c103780 msgr2=0x7fac1c113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac21665700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c103780 0x7fac1c113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:40.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac21665700 1 -- 172.21.15.33:0/3266344898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18005040 con 0x7fac1c102de0 2024-04-25T01:49:40.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac20e64700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c103780 0x7fac1c113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:49:40.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac21665700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 0x7fac1c1138b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fac180090d0 tx=0x7fac18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:40.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac11ffb700 1 -- 172.21.15.33:0/3266344898 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac18021070 con 0x7fac1c102de0 2024-04-25T01:49:40.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c10dd10 con 0x7fac1c102de0 2024-04-25T01:49:40.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac11ffb700 1 -- 172.21.15.33:0/3266344898 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac18013c00 con 0x7fac1c102de0 2024-04-25T01:49:40.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.918+0000 7fac11ffb700 1 -- 172.21.15.33:0/3266344898 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac1800e070 con 0x7fac1c102de0 2024-04-25T01:49:40.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.919+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c10e270 con 0x7fac1c102de0 2024-04-25T01:49:40.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.920+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac1c1000f0 con 0x7fac1c102de0 2024-04-25T01:49:40.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.920+0000 7fac11ffb700 1 -- 172.21.15.33:0/3266344898 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fac1801c400 con 0x7fac1c102de0 2024-04-25T01:49:40.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.920+0000 7fac11ffb700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac0806e140 0x7fac080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:40.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.921+0000 7fac11ffb700 1 -- 172.21.15.33:0/3266344898 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fac1808f040 con 0x7fac1c102de0 2024-04-25T01:49:40.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.921+0000 7fac20e64700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac0806e140 0x7fac080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:40.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.921+0000 7fac20e64700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac0806e140 0x7fac080705f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac1c10ec00 tx=0x7fac0c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:40.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:40.925+0000 7fac11ffb700 1 -- 172.21.15.33:0/3266344898 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fac1805bf70 con 0x7fac1c102de0 2024-04-25T01:49:41.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.150+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac1c10efa0 con 0x7fac0806e140 2024-04-25T01:49:41.148 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:40 smithi033 ceph-mon[111306]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:41.148 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:40 smithi033 ceph-mon[111306]: from='client.34628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:41.148 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:40 smithi033 ceph-mon[111306]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:41.148 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:49:41.149 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:40 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3653006603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:49:41.149 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:40 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3698339821' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:49:41.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.151+0000 7fac11ffb700 1 -- 172.21.15.33:0/3266344898 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fac1c10efa0 con 0x7fac0806e140 2024-04-25T01:49:41.149 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:49:41.149 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:49:41.149 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:49:41.149 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:49:41.150 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:49:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac0806e140 msgr2=0x7fac080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:41.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac0806e140 0x7fac080705f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fac1c10ec00 tx=0x7fac0c00b040 comp rx=0 tx=0).stop 2024-04-25T01:49:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 msgr2=0x7fac1c1138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 0x7fac1c1138b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fac180090d0 tx=0x7fac18009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 shutdown_connections 2024-04-25T01:49:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac0806e140 0x7fac080705f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac1c102de0 0x7fac1c1138b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 --2- 172.21.15.33:0/3266344898 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac1c103780 0x7fac1c113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 >> 172.21.15.33:0/3266344898 conn(0x7fac1c0f8990 msgr2=0x7fac1c0fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 shutdown_connections 2024-04-25T01:49:41.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.154+0000 7fac238c9700 1 -- 172.21.15.33:0/3266344898 wait complete. 2024-04-25T01:49:41.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 -- 172.21.15.33:0/1113251766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fe7e0 msgr2=0x7fc084106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 --2- 172.21.15.33:0/1113251766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fe7e0 0x7fc084106eb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc074004940 tx=0x7fc074009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 -- 172.21.15.33:0/1113251766 shutdown_connections 2024-04-25T01:49:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 --2- 172.21.15.33:0/1113251766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fe7e0 0x7fc084106eb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 --2- 172.21.15.33:0/1113251766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0840fde40 0x7fc0840fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 -- 172.21.15.33:0/1113251766 >> 172.21.15.33:0/1113251766 conn(0x7fc0840f7860 msgr2=0x7fc0840f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 -- 172.21.15.33:0/1113251766 shutdown_connections 2024-04-25T01:49:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 -- 172.21.15.33:0/1113251766 wait complete. 2024-04-25T01:49:41.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 Processor -- start 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.260+0000 7fc089f5d700 1 -- start start 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc089f5d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fde40 0x7fc0840fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc089f5d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0840fe7e0 0x7fc0840fc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc089f5d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840fdce0 con 0x7fc0840fde40 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc089f5d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0840fc780 con 0x7fc0840fe7e0 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc0837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fde40 0x7fc0840fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc0837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fde40 0x7fc0840fbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53374/0 (socket says 172.21.15.33:53374) 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc0837fe700 1 -- 172.21.15.33:0/1619514554 learned_addr learned my addr 172.21.15.33:0/1619514554 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc082ffd700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0840fe7e0 0x7fc0840fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc0837fe700 1 -- 172.21.15.33:0/1619514554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0840fe7e0 msgr2=0x7fc0840fc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc0837fe700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0840fe7e0 0x7fc0840fc240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc0837fe700 1 -- 172.21.15.33:0/1619514554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc074005040 con 0x7fc0840fde40 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc0837fe700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fde40 0x7fc0840fbd00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b790 tx=0x7fc06c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:41.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.261+0000 7fc07bfff700 1 -- 172.21.15.33:0/1619514554 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc06c00f6f0 con 0x7fc0840fde40 2024-04-25T01:49:41.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.262+0000 7fc07bfff700 1 -- 172.21.15.33:0/1619514554 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc06c00fd20 con 0x7fc0840fde40 2024-04-25T01:49:41.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.262+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0840fca60 con 0x7fc0840fde40 2024-04-25T01:49:41.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.262+0000 7fc07bfff700 1 -- 172.21.15.33:0/1619514554 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc06c0103a0 con 0x7fc0840fde40 2024-04-25T01:49:41.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.262+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08411a190 con 0x7fc0840fde40 2024-04-25T01:49:41.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.263+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc084066a90 con 0x7fc0840fde40 2024-04-25T01:49:41.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.263+0000 7fc07bfff700 1 -- 172.21.15.33:0/1619514554 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc06c011400 con 0x7fc0840fde40 2024-04-25T01:49:41.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.263+0000 7fc07bfff700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc07006e140 0x7fc0700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:49:41.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.264+0000 7fc07bfff700 1 -- 172.21.15.33:0/1619514554 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc06c08d910 con 0x7fc0840fde40 2024-04-25T01:49:41.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.264+0000 7fc082ffd700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc07006e140 0x7fc0700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:49:41.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.264+0000 7fc082ffd700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc07006e140 0x7fc0700705f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc0840fd430 tx=0x7fc074009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:49:41.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.268+0000 7fc07bfff700 1 -- 172.21.15.33:0/1619514554 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc06c056d10 con 0x7fc0840fde40 2024-04-25T01:49:41.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:40 smithi138 ceph-mon[81304]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:41.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:40 smithi138 ceph-mon[81304]: from='client.34628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:41.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:40 smithi138 ceph-mon[81304]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:41.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:49:41.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:40 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3653006603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:49:41.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:40 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3698339821' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:49:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.561+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc08404c710 con 0x7fc0840fde40 2024-04-25T01:49:41.559 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.561+0000 7fc07bfff700 1 -- 172.21.15.33:0/1619514554 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc06c05a3d0 con 0x7fc0840fde40 2024-04-25T01:49:41.559 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:49:41.559 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:49:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:49:41.559 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:49:41.559 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:49:41.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.563+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc07006e140 msgr2=0x7fc0700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:41.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.563+0000 7fc089f5d700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc07006e140 0x7fc0700705f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc0840fd430 tx=0x7fc074009e30 comp rx=0 tx=0).stop 2024-04-25T01:49:41.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.563+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fde40 msgr2=0x7fc0840fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:49:41.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.563+0000 7fc089f5d700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fde40 0x7fc0840fbd00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00b790 tx=0x7fc06c009250 comp rx=0 tx=0).stop 2024-04-25T01:49:41.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.563+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 shutdown_connections 2024-04-25T01:49:41.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.563+0000 7fc089f5d700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc07006e140 0x7fc0700705f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:41.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.563+0000 7fc089f5d700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0840fde40 0x7fc0840fbd00 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:41.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.563+0000 7fc089f5d700 1 --2- 172.21.15.33:0/1619514554 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0840fe7e0 0x7fc0840fc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:49:41.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.563+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 >> 172.21.15.33:0/1619514554 conn(0x7fc0840f7860 msgr2=0x7fc084103770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:49:41.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.564+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 shutdown_connections 2024-04-25T01:49:41.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:49:41.564+0000 7fc089f5d700 1 -- 172.21.15.33:0/1619514554 wait complete. 2024-04-25T01:49:42.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:41 smithi033 ceph-mon[111306]: pgmap v308: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:42.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:41 smithi033 ceph-mon[111306]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:42.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:41 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1619514554' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:49:42.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:41 smithi138 ceph-mon[81304]: pgmap v308: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:42.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:41 smithi138 ceph-mon[81304]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:49:42.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:41 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1619514554' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:49:44.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:43 smithi033 ceph-mon[111306]: pgmap v309: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:44.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:43 smithi138 ceph-mon[81304]: pgmap v309: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:46.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:45 smithi033 ceph-mon[111306]: pgmap v310: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:46.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:45 smithi138 ceph-mon[81304]: pgmap v310: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:48.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:47 smithi033 ceph-mon[111306]: pgmap v311: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:48.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:47 smithi138 ceph-mon[81304]: pgmap v311: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:50.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:49 smithi033 ceph-mon[111306]: pgmap v312: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:50.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:49 smithi138 ceph-mon[81304]: pgmap v312: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:52.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:51 smithi033 ceph-mon[111306]: pgmap v313: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:52.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:51 smithi138 ceph-mon[81304]: pgmap v313: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:54.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:53 smithi033 ceph-mon[111306]: pgmap v314: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:54.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:53 smithi138 ceph-mon[81304]: pgmap v314: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:56.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:55 smithi033 ceph-mon[111306]: pgmap v315: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:56.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:49:56.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:55 smithi138 ceph-mon[81304]: pgmap v315: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:56.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:49:58.263 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:57 smithi033 ceph-mon[111306]: pgmap v316: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:49:58.288 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:57 smithi138 ceph-mon[81304]: pgmap v316: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:00.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:49:59 smithi033 ceph-mon[111306]: pgmap v317: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:00.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T01:49:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:50:00.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T01:49:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:50:00.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T01:49:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-25T01:50:00.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T01:49:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:50:00.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T01:49:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-25T01:50:00.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:49:59 smithi138 ceph-mon[81304]: pgmap v317: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:01.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-mon[111306]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:50:01.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-mon[111306]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:50:01.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-mon[111306]: fs cephfs is degraded 2024-04-25T01:50:01.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-mon[111306]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:50:01.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:00 smithi033 ceph-mon[111306]: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:50:01.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:00 smithi138 ceph-mon[81304]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:50:01.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:00 smithi138 ceph-mon[81304]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:50:01.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:00 smithi138 ceph-mon[81304]: fs cephfs is degraded 2024-04-25T01:50:01.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:00 smithi138 ceph-mon[81304]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:50:01.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:00 smithi138 ceph-mon[81304]: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:50:02.165 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:01 smithi033 ceph-mon[111306]: pgmap v318: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:02.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:01 smithi138 ceph-mon[81304]: pgmap v318: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:04.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:03 smithi033 ceph-mon[111306]: pgmap v319: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:04.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:03 smithi138 ceph-mon[81304]: pgmap v319: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:06.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:05 smithi033 ceph-mon[111306]: pgmap v320: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:06.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:05 smithi138 ceph-mon[81304]: pgmap v320: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:08.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:07 smithi033 ceph-mon[111306]: pgmap v321: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:08.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:07 smithi138 ceph-mon[81304]: pgmap v321: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:10.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:09 smithi033 ceph-mon[111306]: pgmap v322: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:10.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:09 smithi138 ceph-mon[81304]: pgmap v322: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:11.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:50:11.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:50:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.682+0000 7fafecd2b700 1 -- 172.21.15.33:0/1416107907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8102de0 msgr2=0x7fafe81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.682+0000 7fafecd2b700 1 --2- 172.21.15.33:0/1416107907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8102de0 0x7fafe81031b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fafd8004940 tx=0x7fafd8009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:11.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.682+0000 7fafecd2b700 1 -- 172.21.15.33:0/1416107907 shutdown_connections 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.682+0000 7fafecd2b700 1 --2- 172.21.15.33:0/1416107907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe8103780 0x7fafe80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.682+0000 7fafecd2b700 1 --2- 172.21.15.33:0/1416107907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8102de0 0x7fafe81031b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.682+0000 7fafecd2b700 1 -- 172.21.15.33:0/1416107907 >> 172.21.15.33:0/1416107907 conn(0x7fafe80f8990 msgr2=0x7fafe80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.682+0000 7fafecd2b700 1 -- 172.21.15.33:0/1416107907 shutdown_connections 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.682+0000 7fafecd2b700 1 -- 172.21.15.33:0/1416107907 wait complete. 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafecd2b700 1 Processor -- start 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafecd2b700 1 -- start start 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafecd2b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe8102de0 0x7fafe8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafecd2b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8103780 0x7fafe80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafecd2b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe8072160 con 0x7fafe8103780 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafecd2b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe8070ce0 con 0x7fafe8102de0 2024-04-25T01:50:11.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafe5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8103780 0x7fafe80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafe5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8103780 0x7fafe80707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46568/0 (socket says 172.21.15.33:46568) 2024-04-25T01:50:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafe5d9b700 1 -- 172.21.15.33:0/66953670 learned_addr learned my addr 172.21.15.33:0/66953670 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.683+0000 7fafe659c700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe8102de0 0x7fafe8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.684+0000 7fafe5d9b700 1 -- 172.21.15.33:0/66953670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe8102de0 msgr2=0x7fafe8070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.684+0000 7fafe5d9b700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe8102de0 0x7fafe8070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:11.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.684+0000 7fafe5d9b700 1 -- 172.21.15.33:0/66953670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd8005040 con 0x7fafe8103780 2024-04-25T01:50:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.684+0000 7fafe5d9b700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8103780 0x7fafe80707a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00c800 tx=0x7fafdc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.684+0000 7fafd6ffd700 1 -- 172.21.15.33:0/66953670 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafdc0106f0 con 0x7fafe8103780 2024-04-25T01:50:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.684+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe8070fc0 con 0x7fafe8103780 2024-04-25T01:50:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.684+0000 7fafd6ffd700 1 -- 172.21.15.33:0/66953670 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafdc010d20 con 0x7fafe8103780 2024-04-25T01:50:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.684+0000 7fafd6ffd700 1 -- 172.21.15.33:0/66953670 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafdc0113a0 con 0x7fafe8103780 2024-04-25T01:50:11.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.684+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe811a160 con 0x7fafe8103780 2024-04-25T01:50:11.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.685+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe81000f0 con 0x7fafe8103780 2024-04-25T01:50:11.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.686+0000 7fafd6ffd700 1 -- 172.21.15.33:0/66953670 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fafdc012400 con 0x7fafe8103780 2024-04-25T01:50:11.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.686+0000 7fafd6ffd700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fafd006e140 0x7fafd00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:11.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.686+0000 7fafd6ffd700 1 -- 172.21.15.33:0/66953670 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fafdc08da90 con 0x7fafe8103780 2024-04-25T01:50:11.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.686+0000 7fafe659c700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fafd006e140 0x7fafd00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:11.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.687+0000 7fafe659c700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fafd006e140 0x7fafd00705f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fafd8004f90 tx=0x7fafd8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:11.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.691+0000 7fafd6ffd700 1 -- 172.21.15.33:0/66953670 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fafdc056e90 con 0x7fafe8103780 2024-04-25T01:50:11.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.918+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafe8109ee0 con 0x7fafd006e140 2024-04-25T01:50:11.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.919+0000 7fafd6ffd700 1 -- 172.21.15.33:0/66953670 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fafe8109ee0 con 0x7fafd006e140 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.921+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fafd006e140 msgr2=0x7fafd00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.921+0000 7fafecd2b700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fafd006e140 0x7fafd00705f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fafd8004f90 tx=0x7fafd8009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.921+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8103780 msgr2=0x7fafe80707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.921+0000 7fafecd2b700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8103780 0x7fafe80707a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fafdc00c800 tx=0x7fafdc004470 comp rx=0 tx=0).stop 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.922+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 shutdown_connections 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.922+0000 7fafecd2b700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fafd006e140 0x7fafd00705f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.922+0000 7fafecd2b700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fafe8102de0 0x7fafe8070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.922+0000 7fafecd2b700 1 --2- 172.21.15.33:0/66953670 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fafe8103780 0x7fafe80707a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.922+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 >> 172.21.15.33:0/66953670 conn(0x7fafe80f8990 msgr2=0x7fafe8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.922+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 shutdown_connections 2024-04-25T01:50:11.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:11.922+0000 7fafecd2b700 1 -- 172.21.15.33:0/66953670 wait complete. 2024-04-25T01:50:11.930 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:50:11.931 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:11 smithi033 ceph-mon[111306]: pgmap v323: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:12.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.027+0000 7faad3589700 1 -- 172.21.15.33:0/464380424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc102730 msgr2=0x7faacc0fbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.027+0000 7faad3589700 1 --2- 172.21.15.33:0/464380424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc102730 0x7faacc0fbda0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7faac8004940 tx=0x7faac8009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:12.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.027+0000 7faad3589700 1 -- 172.21.15.33:0/464380424 shutdown_connections 2024-04-25T01:50:12.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.027+0000 7faad3589700 1 --2- 172.21.15.33:0/464380424 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc102730 0x7faacc0fbda0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.027+0000 7faad3589700 1 --2- 172.21.15.33:0/464380424 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc101d90 0x7faacc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.027+0000 7faad3589700 1 -- 172.21.15.33:0/464380424 >> 172.21.15.33:0/464380424 conn(0x7faacc0f7940 msgr2=0x7faacc0f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:12.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.028+0000 7faad3589700 1 -- 172.21.15.33:0/464380424 shutdown_connections 2024-04-25T01:50:12.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.028+0000 7faad3589700 1 -- 172.21.15.33:0/464380424 wait complete. 2024-04-25T01:50:12.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.028+0000 7faad3589700 1 Processor -- start 2024-04-25T01:50:12.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.028+0000 7faad3589700 1 -- start start 2024-04-25T01:50:12.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.028+0000 7faad3589700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc101d90 0x7faacc10eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:12.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.028+0000 7faad3589700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc102730 0x7faacc10f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:12.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.028+0000 7faad3589700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc10f5e0 con 0x7faacc101d90 2024-04-25T01:50:12.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad3589700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc10f720 con 0x7faacc102730 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad1325700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc101d90 0x7faacc10eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad1325700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc101d90 0x7faacc10eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46588/0 (socket says 172.21.15.33:46588) 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad1325700 1 -- 172.21.15.33:0/304901488 learned_addr learned my addr 172.21.15.33:0/304901488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad0b24700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc102730 0x7faacc10f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad1325700 1 -- 172.21.15.33:0/304901488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc102730 msgr2=0x7faacc10f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad1325700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc102730 0x7faacc10f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad1325700 1 -- 172.21.15.33:0/304901488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac8005040 con 0x7faacc101d90 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad0b24700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc102730 0x7faacc10f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad1325700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc101d90 0x7faacc10eb60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c8c0 tx=0x7faabc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faac1ffb700 1 -- 172.21.15.33:0/304901488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faabc00a980 con 0x7faacc101d90 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faacc117bd0 con 0x7faacc101d90 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faac1ffb700 1 -- 172.21.15.33:0/304901488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faabc00ce60 con 0x7faacc101d90 2024-04-25T01:50:12.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.029+0000 7faac1ffb700 1 -- 172.21.15.33:0/304901488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faabc0115e0 con 0x7faacc101d90 2024-04-25T01:50:12.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.030+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc117fa0 con 0x7faacc101d90 2024-04-25T01:50:12.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.030+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faacc104260 con 0x7faacc101d90 2024-04-25T01:50:12.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.031+0000 7faac1ffb700 1 -- 172.21.15.33:0/304901488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faabc012400 con 0x7faacc101d90 2024-04-25T01:50:12.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.031+0000 7faac1ffb700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faab806e140 0x7faab80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:12.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.032+0000 7faac1ffb700 1 -- 172.21.15.33:0/304901488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7faabc08dbf0 con 0x7faacc101d90 2024-04-25T01:50:12.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.032+0000 7faad0b24700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faab806e140 0x7faab80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:12.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.032+0000 7faad0b24700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faab806e140 0x7faab80705f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faac800b3b0 tx=0x7faac8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:12.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.036+0000 7faac1ffb700 1 -- 172.21.15.33:0/304901488 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faabc056ff0 con 0x7faacc101d90 2024-04-25T01:50:12.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.264+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faacc108e70 con 0x7faab806e140 2024-04-25T01:50:12.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.265+0000 7faac1ffb700 1 -- 172.21.15.33:0/304901488 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faacc108e70 con 0x7faab806e140 2024-04-25T01:50:12.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faab806e140 msgr2=0x7faab80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faab806e140 0x7faab80705f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7faac800b3b0 tx=0x7faac8009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:12.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc101d90 msgr2=0x7faacc10eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc101d90 0x7faacc10eb60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c8c0 tx=0x7faabc0044c0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 shutdown_connections 2024-04-25T01:50:12.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faab806e140 0x7faab80705f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faacc101d90 0x7faacc10eb60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 --2- 172.21.15.33:0/304901488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faacc102730 0x7faacc10f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 >> 172.21.15.33:0/304901488 conn(0x7faacc0f7940 msgr2=0x7faacc1011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:12.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 shutdown_connections 2024-04-25T01:50:12.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.267+0000 7faad3589700 1 -- 172.21.15.33:0/304901488 wait complete. 2024-04-25T01:50:12.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:11 smithi138 ceph-mon[81304]: pgmap v323: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:12.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.372+0000 7f8336db7700 1 -- 172.21.15.33:0/978773495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330103780 msgr2=0x7f83300fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.372+0000 7f8336db7700 1 --2- 172.21.15.33:0/978773495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330103780 0x7f83300fcdf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f831c004940 tx=0x7f831c009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.372+0000 7f8336db7700 1 -- 172.21.15.33:0/978773495 shutdown_connections 2024-04-25T01:50:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.372+0000 7f8336db7700 1 --2- 172.21.15.33:0/978773495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330103780 0x7f83300fcdf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.372+0000 7f8336db7700 1 --2- 172.21.15.33:0/978773495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330102de0 0x7f83301031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.372+0000 7f8336db7700 1 -- 172.21.15.33:0/978773495 >> 172.21.15.33:0/978773495 conn(0x7f83300f8950 msgr2=0x7f83300fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.372+0000 7f8336db7700 1 -- 172.21.15.33:0/978773495 shutdown_connections 2024-04-25T01:50:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.372+0000 7f8336db7700 1 -- 172.21.15.33:0/978773495 wait complete. 2024-04-25T01:50:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8336db7700 1 Processor -- start 2024-04-25T01:50:12.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8336db7700 1 -- start start 2024-04-25T01:50:12.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8336db7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102de0 0x7f8330113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:12.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8336db7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330103780 0x7f83301140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:12.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8336db7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8330114700 con 0x7f8330102de0 2024-04-25T01:50:12.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8336db7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833010daf0 con 0x7f8330103780 2024-04-25T01:50:12.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8334b53700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102de0 0x7f8330113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:12.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8334b53700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102de0 0x7f8330113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46592/0 (socket says 172.21.15.33:46592) 2024-04-25T01:50:12.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8334b53700 1 -- 172.21.15.33:0/2769885457 learned_addr learned my addr 172.21.15.33:0/2769885457 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:12.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f832ffff700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330103780 0x7f83301140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:12.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8334b53700 1 -- 172.21.15.33:0/2769885457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330103780 msgr2=0x7f83301140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8334b53700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330103780 0x7f83301140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.373+0000 7f8334b53700 1 -- 172.21.15.33:0/2769885457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c005040 con 0x7f8330102de0 2024-04-25T01:50:12.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.374+0000 7f8334b53700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102de0 0x7f8330113b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8324002a50 tx=0x7f83240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:12.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.374+0000 7f832d7fa700 1 -- 172.21.15.33:0/2769885457 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8324007cc0 con 0x7f8330102de0 2024-04-25T01:50:12.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.374+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833010ddd0 con 0x7f8330102de0 2024-04-25T01:50:12.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.374+0000 7f832d7fa700 1 -- 172.21.15.33:0/2769885457 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f832400f040 con 0x7f8330102de0 2024-04-25T01:50:12.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.374+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833010e390 con 0x7f8330102de0 2024-04-25T01:50:12.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.374+0000 7f832d7fa700 1 -- 172.21.15.33:0/2769885457 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8324009600 con 0x7f8330102de0 2024-04-25T01:50:12.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.375+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83300fe530 con 0x7f8330102de0 2024-04-25T01:50:12.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.375+0000 7f832d7fa700 1 -- 172.21.15.33:0/2769885457 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8324009760 con 0x7f8330102de0 2024-04-25T01:50:12.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.376+0000 7f832d7fa700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f832006e140 0x7f83200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:12.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.376+0000 7f832d7fa700 1 -- 172.21.15.33:0/2769885457 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f832408d880 con 0x7f8330102de0 2024-04-25T01:50:12.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.376+0000 7f832ffff700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f832006e140 0x7f83200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:12.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.376+0000 7f832ffff700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f832006e140 0x7f83200705f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f831c009fd0 tx=0x7f831c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:12.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.381+0000 7f832d7fa700 1 -- 172.21.15.33:0/2769885457 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f832405a730 con 0x7f8330102de0 2024-04-25T01:50:12.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.600+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8330109ec0 con 0x7f832006e140 2024-04-25T01:50:12.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.610+0000 7f832d7fa700 1 -- 172.21.15.33:0/2769885457 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8330109ec0 con 0x7f832006e140 2024-04-25T01:50:12.608 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (11m) 4m ago 22m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (8m) 4m ago 22m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (8m) 4m ago 21m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (11m) 4m ago 22m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (4m) 4m ago 17m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (4m) 4m ago 17m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (4m) 4m ago 17m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (17m) 4m ago 17m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (12m) 4m ago 23m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (12m) 4m ago 21m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (9m) 4m ago 24m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (9m) 4m ago 21m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (12m) 4m ago 22m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:50:12.609 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (11m) 4m ago 21m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:50:12.610 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (8m) 4m ago 20m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:50:12.610 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (7m) 4m ago 20m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:50:12.610 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (6m) 4m ago 19m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:50:12.610 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (6m) 4m ago 19m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:50:12.610 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (5m) 4m ago 19m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:50:12.610 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (5m) 4m ago 18m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:50:12.610 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (11m) 4m ago 22m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.612+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f832006e140 msgr2=0x7f83200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f832006e140 0x7f83200705f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f831c009fd0 tx=0x7f831c009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102de0 msgr2=0x7f8330113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102de0 0x7f8330113b70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8324002a50 tx=0x7f83240046b0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 shutdown_connections 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f832006e140 0x7f83200705f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8330102de0 0x7f8330113b70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 --2- 172.21.15.33:0/2769885457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8330103780 0x7f83301140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 >> 172.21.15.33:0/2769885457 conn(0x7f83300f8950 msgr2=0x7f8330100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 shutdown_connections 2024-04-25T01:50:12.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.613+0000 7f8336db7700 1 -- 172.21.15.33:0/2769885457 wait complete. 2024-04-25T01:50:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.716+0000 7f68c6645700 1 -- 172.21.15.33:0/3297068589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c00fbd40 msgr2=0x7f68c00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.716+0000 7f68c6645700 1 --2- 172.21.15.33:0/3297068589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c00fbd40 0x7f68c00fc1b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f68b4004940 tx=0x7f68b4009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.716+0000 7f68c6645700 1 -- 172.21.15.33:0/3297068589 shutdown_connections 2024-04-25T01:50:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.716+0000 7f68c6645700 1 --2- 172.21.15.33:0/3297068589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c00fbd40 0x7f68c00fc1b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.716+0000 7f68c6645700 1 --2- 172.21.15.33:0/3297068589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c0101d90 0x7f68c0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.716+0000 7f68c6645700 1 -- 172.21.15.33:0/3297068589 >> 172.21.15.33:0/3297068589 conn(0x7f68c0071d00 msgr2=0x7f68c0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.716+0000 7f68c6645700 1 -- 172.21.15.33:0/3297068589 shutdown_connections 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.716+0000 7f68c6645700 1 -- 172.21.15.33:0/3297068589 wait complete. 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.717+0000 7f68c6645700 1 Processor -- start 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.717+0000 7f68c6645700 1 -- start start 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.717+0000 7f68c6645700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c00fbd40 0x7f68c0113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.717+0000 7f68c6645700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c0101d90 0x7f68c01140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.717+0000 7f68c6645700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0114740 con 0x7f68c0101d90 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.717+0000 7f68c6645700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c010db30 con 0x7f68c00fbd40 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.717+0000 7f68bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c0101d90 0x7f68c01140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:12.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bf7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c0101d90 0x7f68c01140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46606/0 (socket says 172.21.15.33:46606) 2024-04-25T01:50:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bf7fe700 1 -- 172.21.15.33:0/3744605179 learned_addr learned my addr 172.21.15.33:0/3744605179 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bffff700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c00fbd40 0x7f68c0113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bf7fe700 1 -- 172.21.15.33:0/3744605179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c00fbd40 msgr2=0x7f68c0113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bf7fe700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c00fbd40 0x7f68c0113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bf7fe700 1 -- 172.21.15.33:0/3744605179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4005040 con 0x7f68c0101d90 2024-04-25T01:50:12.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bf7fe700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c0101d90 0x7f68c01140f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68b400b3b0 tx=0x7f68b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bcff9700 1 -- 172.21.15.33:0/3744605179 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b400e040 con 0x7f68c0101d90 2024-04-25T01:50:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bcff9700 1 -- 172.21.15.33:0/3744605179 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b4013420 con 0x7f68c0101d90 2024-04-25T01:50:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68bcff9700 1 -- 172.21.15.33:0/3744605179 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68b401c630 con 0x7f68c0101d90 2024-04-25T01:50:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c010ddb0 con 0x7f68c0101d90 2024-04-25T01:50:12.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.718+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c010e370 con 0x7f68c0101d90 2024-04-25T01:50:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.719+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c00fd430 con 0x7f68c0101d90 2024-04-25T01:50:12.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.720+0000 7f68bcff9700 1 -- 172.21.15.33:0/3744605179 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f68b4013590 con 0x7f68c0101d90 2024-04-25T01:50:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.720+0000 7f68bcff9700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f68a806e140 0x7f68a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.720+0000 7f68bcff9700 1 -- 172.21.15.33:0/3744605179 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f68b408ed80 con 0x7f68c0101d90 2024-04-25T01:50:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.720+0000 7f68bffff700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f68a806e140 0x7f68a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:12.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.721+0000 7f68bffff700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f68a806e140 0x7f68a80705f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004f40 tx=0x7f68b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:12.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:12.725+0000 7f68bcff9700 1 -- 172.21.15.33:0/3744605179 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f68b405bcb0 con 0x7f68c0101d90 2024-04-25T01:50:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.023+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f68c0108df0 con 0x7f68c0101d90 2024-04-25T01:50:13.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.023+0000 7f68bcff9700 1 -- 172.21.15.33:0/3744605179 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f68b405b840 con 0x7f68c0101d90 2024-04-25T01:50:13.021 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:50:13.022 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:50:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:50:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:50:13.023 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:50:13.023 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:50:13.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.025+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f68a806e140 msgr2=0x7f68a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.025+0000 7f68c6645700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f68a806e140 0x7f68a80705f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f68b0004f40 tx=0x7f68b000b040 comp rx=0 tx=0).stop 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.025+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c0101d90 msgr2=0x7f68c01140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.025+0000 7f68c6645700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c0101d90 0x7f68c01140f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f68b400b3b0 tx=0x7f68b4009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.025+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 shutdown_connections 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.025+0000 7f68c6645700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f68a806e140 0x7f68a80705f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.025+0000 7f68c6645700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68c00fbd40 0x7f68c0113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.025+0000 7f68c6645700 1 --2- 172.21.15.33:0/3744605179 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68c0101d90 0x7f68c01140f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.025+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 >> 172.21.15.33:0/3744605179 conn(0x7f68c0071d00 msgr2=0x7f68c00655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.026+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 shutdown_connections 2024-04-25T01:50:13.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.026+0000 7f68c6645700 1 -- 172.21.15.33:0/3744605179 wait complete. 2024-04-25T01:50:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.129+0000 7f112fefe700 1 -- 172.21.15.33:0/1843532818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 msgr2=0x7f11281020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.129+0000 7f112fefe700 1 --2- 172.21.15.33:0/1843532818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 0x7f11281020d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f1124004940 tx=0x7f1124009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 -- 172.21.15.33:0/1843532818 shutdown_connections 2024-04-25T01:50:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 --2- 172.21.15.33:0/1843532818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11281026a0 0x7f11280fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 --2- 172.21.15.33:0/1843532818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 0x7f11281020d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 -- 172.21.15.33:0/1843532818 >> 172.21.15.33:0/1843532818 conn(0x7f11280f7870 msgr2=0x7f11280f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 -- 172.21.15.33:0/1843532818 shutdown_connections 2024-04-25T01:50:13.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 -- 172.21.15.33:0/1843532818 wait complete. 2024-04-25T01:50:13.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 Processor -- start 2024-04-25T01:50:13.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 -- start start 2024-04-25T01:50:13.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 0x7f1128102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:13.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11281026a0 0x7f11281030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:13.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128104ab0 con 0x7f1128101d00 2024-04-25T01:50:13.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.130+0000 7f112fefe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11281035e0 con 0x7f11281026a0 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112dc9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 0x7f1128102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112dc9a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 0x7f1128102b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46620/0 (socket says 172.21.15.33:46620) 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112dc9a700 1 -- 172.21.15.33:0/1407013219 learned_addr learned my addr 172.21.15.33:0/1407013219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112dc9a700 1 -- 172.21.15.33:0/1407013219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11281026a0 msgr2=0x7f11281030a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112d499700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11281026a0 0x7f11281030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112dc9a700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11281026a0 0x7f11281030a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112dc9a700 1 -- 172.21.15.33:0/1407013219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1124005040 con 0x7f1128101d00 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112d499700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11281026a0 0x7f11281030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112dc9a700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 0x7f1128102b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f112400b3b0 tx=0x7f1124009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f111e7fc700 1 -- 172.21.15.33:0/1407013219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f112400e040 con 0x7f1128101d00 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f111e7fc700 1 -- 172.21.15.33:0/1407013219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f112400bbc0 con 0x7f1128101d00 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1128103860 con 0x7f1128101d00 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f111e7fc700 1 -- 172.21.15.33:0/1407013219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f112401c3f0 con 0x7f1128101d00 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.131+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112811a0d0 con 0x7f1128101d00 2024-04-25T01:50:13.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.132+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11280fd450 con 0x7f1128101d00 2024-04-25T01:50:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.133+0000 7f111e7fc700 1 -- 172.21.15.33:0/1407013219 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f112400bd30 con 0x7f1128101d00 2024-04-25T01:50:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.133+0000 7f111e7fc700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f111406e140 0x7f11140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.133+0000 7f111e7fc700 1 -- 172.21.15.33:0/1407013219 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f112408ecf0 con 0x7f1128101d00 2024-04-25T01:50:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.133+0000 7f112d499700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f111406e140 0x7f11140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:13.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.134+0000 7f112d499700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f111406e140 0x7f11140705f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1118009fd0 tx=0x7f111800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:13.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.138+0000 7f111e7fc700 1 -- 172.21.15.33:0/1407013219 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f11240580f0 con 0x7f1128101d00 2024-04-25T01:50:13.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:12 smithi033 ceph-mon[111306]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:13.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:12 smithi138 ceph-mon[81304]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.395+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f11280fd450 con 0x7f1128101d00 2024-04-25T01:50:13.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.395+0000 7f111e7fc700 1 -- 172.21.15.33:0/1407013219 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f112405b7b0 con 0x7f1128101d00 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:50:13.394 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:50:13.397 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:50:13.397 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:50:13.398 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:50:13.399 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:50:13.400 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:50:13.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.397+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f111406e140 msgr2=0x7f11140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.400 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.397+0000 7f112fefe700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f111406e140 0x7f11140705f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1118009fd0 tx=0x7f111800b040 comp rx=0 tx=0).stop 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.397+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 msgr2=0x7f1128102b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.397+0000 7f112fefe700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 0x7f1128102b60 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f112400b3b0 tx=0x7f1124009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.397+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 shutdown_connections 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.397+0000 7f112fefe700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f111406e140 0x7f11140705f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.397+0000 7f112fefe700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1128101d00 0x7f1128102b60 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.397+0000 7f112fefe700 1 --2- 172.21.15.33:0/1407013219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f11281026a0 0x7f11281030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.397+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 >> 172.21.15.33:0/1407013219 conn(0x7f11280f7870 msgr2=0x7f1128066280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.398+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 shutdown_connections 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.398+0000 7f112fefe700 1 -- 172.21.15.33:0/1407013219 wait complete. 2024-04-25T01:50:13.401 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:50:13.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.503+0000 7f198cec8700 1 -- 172.21.15.33:0/2206299841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1988103780 msgr2=0x7f19880fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.503+0000 7f198cec8700 1 --2- 172.21.15.33:0/2206299841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1988103780 0x7f19880fcdf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f1970004940 tx=0x7f1970009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.503+0000 7f198cec8700 1 -- 172.21.15.33:0/2206299841 shutdown_connections 2024-04-25T01:50:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.503+0000 7f198cec8700 1 --2- 172.21.15.33:0/2206299841 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1988103780 0x7f19880fcdf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.503+0000 7f198cec8700 1 --2- 172.21.15.33:0/2206299841 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1988102de0 0x7f19881031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.503+0000 7f198cec8700 1 -- 172.21.15.33:0/2206299841 >> 172.21.15.33:0/2206299841 conn(0x7f19880f8990 msgr2=0x7f19880fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.503+0000 7f198cec8700 1 -- 172.21.15.33:0/2206299841 shutdown_connections 2024-04-25T01:50:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.503+0000 7f198cec8700 1 -- 172.21.15.33:0/2206299841 wait complete. 2024-04-25T01:50:13.502 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.504+0000 7f198cec8700 1 Processor -- start 2024-04-25T01:50:13.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.507+0000 7f198cec8700 1 -- start start 2024-04-25T01:50:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.507+0000 7f198cec8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1988102de0 0x7f1988113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.507+0000 7f198cec8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1988103780 0x7f1988113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.507+0000 7f198cec8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19881144e0 con 0x7f1988103780 2024-04-25T01:50:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.507+0000 7f198cec8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198810dae0 con 0x7f1988102de0 2024-04-25T01:50:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.507+0000 7f198659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1988102de0 0x7f1988113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.507+0000 7f1985d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1988103780 0x7f1988113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.507+0000 7f198659c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1988102de0 0x7f1988113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.138:3300/0 says I am v2:172.21.15.33:54850/0 (socket says 172.21.15.33:54850) 2024-04-25T01:50:13.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.507+0000 7f198659c700 1 -- 172.21.15.33:0/2041912522 learned_addr learned my addr 172.21.15.33:0/2041912522 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:13.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.508+0000 7f198659c700 1 -- 172.21.15.33:0/2041912522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1988103780 msgr2=0x7f1988113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.508+0000 7f198659c700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1988103780 0x7f1988113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.508+0000 7f198659c700 1 -- 172.21.15.33:0/2041912522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1970005040 con 0x7f1988102de0 2024-04-25T01:50:13.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.508+0000 7f198659c700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1988102de0 0x7f1988113950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1980001d80 tx=0x7f1980009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:13.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.508+0000 7f197effd700 1 -- 172.21.15.33:0/2041912522 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1980007d10 con 0x7f1988102de0 2024-04-25T01:50:13.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.508+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198810ddc0 con 0x7f1988102de0 2024-04-25T01:50:13.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.508+0000 7f197effd700 1 -- 172.21.15.33:0/2041912522 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1980001e80 con 0x7f1988102de0 2024-04-25T01:50:13.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.508+0000 7f197effd700 1 -- 172.21.15.33:0/2041912522 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1980014640 con 0x7f1988102de0 2024-04-25T01:50:13.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.508+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198810e350 con 0x7f1988102de0 2024-04-25T01:50:13.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.509+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19880fe530 con 0x7f1988102de0 2024-04-25T01:50:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.510+0000 7f197effd700 1 -- 172.21.15.33:0/2041912522 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1980039a40 con 0x7f1988102de0 2024-04-25T01:50:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.510+0000 7f197effd700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f197406e140 0x7f19740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.510+0000 7f197effd700 1 -- 172.21.15.33:0/2041912522 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f198008d790 con 0x7f1988102de0 2024-04-25T01:50:13.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.511+0000 7f1985d9b700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f197406e140 0x7f19740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:13.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.511+0000 7f1985d9b700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f197406e140 0x7f19740705f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f198810edc0 tx=0x7f1970009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:13.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.515+0000 7f197effd700 1 -- 172.21.15.33:0/2041912522 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f198005a810 con 0x7f1988102de0 2024-04-25T01:50:13.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.741+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1988109f60 con 0x7f197406e140 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.743+0000 7f197effd700 1 -- 172.21.15.33:0/2041912522 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f197000b670 con 0x7f197406e140 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:50:13.741 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:50:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:50:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:50:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:50:13.742 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:50:13.742 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:50:13.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f197406e140 msgr2=0x7f19740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f197406e140 0x7f19740705f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f198810edc0 tx=0x7f1970009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:13.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1988102de0 msgr2=0x7f1988113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1988102de0 0x7f1988113950 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1980001d80 tx=0x7f1980009ec0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 shutdown_connections 2024-04-25T01:50:13.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f197406e140 0x7f19740705f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1988102de0 0x7f1988113950 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 --2- 172.21.15.33:0/2041912522 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1988103780 0x7f1988113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 >> 172.21.15.33:0/2041912522 conn(0x7f19880f8990 msgr2=0x7f19880f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:13.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 shutdown_connections 2024-04-25T01:50:13.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.745+0000 7f198cec8700 1 -- 172.21.15.33:0/2041912522 wait complete. 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.850+0000 7f1c3be1a700 1 -- 172.21.15.33:0/835327003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34101cd0 msgr2=0x7f1c341020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.850+0000 7f1c3be1a700 1 --2- 172.21.15.33:0/835327003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34101cd0 0x7f1c341020a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f1c24004940 tx=0x7f1c24009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.850+0000 7f1c3be1a700 1 -- 172.21.15.33:0/835327003 shutdown_connections 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.850+0000 7f1c3be1a700 1 --2- 172.21.15.33:0/835327003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c34102670 0x7f1c340fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.850+0000 7f1c3be1a700 1 --2- 172.21.15.33:0/835327003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34101cd0 0x7f1c341020a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.850+0000 7f1c3be1a700 1 -- 172.21.15.33:0/835327003 >> 172.21.15.33:0/835327003 conn(0x7f1c340f7860 msgr2=0x7f1c340f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.850+0000 7f1c3be1a700 1 -- 172.21.15.33:0/835327003 shutdown_connections 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.850+0000 7f1c3be1a700 1 -- 172.21.15.33:0/835327003 wait complete. 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.851+0000 7f1c3be1a700 1 Processor -- start 2024-04-25T01:50:13.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.851+0000 7f1c3be1a700 1 -- start start 2024-04-25T01:50:13.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.851+0000 7f1c3be1a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c34101cd0 0x7f1c341139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:13.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.851+0000 7f1c3be1a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34102670 0x7f1c34113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:13.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.851+0000 7f1c3be1a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c34114530 con 0x7f1c34102670 2024-04-25T01:50:13.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.851+0000 7f1c3be1a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3410db30 con 0x7f1c34101cd0 2024-04-25T01:50:13.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.851+0000 7f1c393b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34102670 0x7f1c34113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:13.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.851+0000 7f1c393b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34102670 0x7f1c34113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46656/0 (socket says 172.21.15.33:46656) 2024-04-25T01:50:13.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.851+0000 7f1c393b5700 1 -- 172.21.15.33:0/1565120114 learned_addr learned my addr 172.21.15.33:0/1565120114 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:13.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c39bb6700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c34101cd0 0x7f1c341139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c393b5700 1 -- 172.21.15.33:0/1565120114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c34101cd0 msgr2=0x7f1c341139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c393b5700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c34101cd0 0x7f1c341139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c393b5700 1 -- 172.21.15.33:0/1565120114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c24005040 con 0x7f1c34102670 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c393b5700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34102670 0x7f1c34113ee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c800 tx=0x7f1c30004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c2a7fc700 1 -- 172.21.15.33:0/1565120114 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c3000a760 con 0x7f1c34102670 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c2a7fc700 1 -- 172.21.15.33:0/1565120114 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c3000ad90 con 0x7f1c34102670 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3410de10 con 0x7f1c34102670 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c2a7fc700 1 -- 172.21.15.33:0/1565120114 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c300113a0 con 0x7f1c34102670 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.852+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3410e350 con 0x7f1c34102670 2024-04-25T01:50:13.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.853+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c340fd420 con 0x7f1c34102670 2024-04-25T01:50:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.854+0000 7f1c2a7fc700 1 -- 172.21.15.33:0/1565120114 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1c30012400 con 0x7f1c34102670 2024-04-25T01:50:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.854+0000 7f1c2a7fc700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1c2006e140 0x7f1c200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.854+0000 7f1c2a7fc700 1 -- 172.21.15.33:0/1565120114 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1c3008db40 con 0x7f1c34102670 2024-04-25T01:50:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.854+0000 7f1c39bb6700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1c2006e140 0x7f1c200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:13.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.855+0000 7f1c39bb6700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1c2006e140 0x7f1c200705f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400b020 tx=0x7f1c24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:13.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:13.859+0000 7f1c2a7fc700 1 -- 172.21.15.33:0/1565120114 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1c3005aa70 con 0x7f1c34102670 2024-04-25T01:50:14.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:13 smithi033 ceph-mon[111306]: from='client.34656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:14.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:13 smithi033 ceph-mon[111306]: pgmap v324: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:14.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:13 smithi033 ceph-mon[111306]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:14.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:13 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3744605179' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:50:14.014 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:13 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1407013219' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:50:14.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.152+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1c3404c710 con 0x7f1c34102670 2024-04-25T01:50:14.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.152+0000 7f1c2a7fc700 1 -- 172.21.15.33:0/1565120114 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f1c3005a600 con 0x7f1c34102670 2024-04-25T01:50:14.151 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:50:14.151 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:50:14.151 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:50:14.151 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:50:14.151 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:50:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1c2006e140 msgr2=0x7f1c200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1c2006e140 0x7f1c200705f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400b020 tx=0x7f1c24009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34102670 msgr2=0x7f1c34113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:14.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34102670 0x7f1c34113ee0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1c3000c800 tx=0x7f1c30004470 comp rx=0 tx=0).stop 2024-04-25T01:50:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 shutdown_connections 2024-04-25T01:50:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1c2006e140 0x7f1c200705f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1c34101cd0 0x7f1c341139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 --2- 172.21.15.33:0/1565120114 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1c34102670 0x7f1c34113ee0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 >> 172.21.15.33:0/1565120114 conn(0x7f1c340f7860 msgr2=0x7f1c340662c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 shutdown_connections 2024-04-25T01:50:14.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:14.154+0000 7f1c3be1a700 1 -- 172.21.15.33:0/1565120114 wait complete. 2024-04-25T01:50:14.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:13 smithi138 ceph-mon[81304]: from='client.34656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:14.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:13 smithi138 ceph-mon[81304]: pgmap v324: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:14.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:13 smithi138 ceph-mon[81304]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:14.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:13 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3744605179' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:50:14.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:13 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1407013219' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:50:15.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:14 smithi033 ceph-mon[111306]: from='client.44475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:15.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:14 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1565120114' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:50:15.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:14 smithi138 ceph-mon[81304]: from='client.44475 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:15.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:14 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1565120114' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:50:16.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:15 smithi033 ceph-mon[111306]: pgmap v325: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:16.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:15 smithi138 ceph-mon[81304]: pgmap v325: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:18.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:17 smithi033 ceph-mon[111306]: pgmap v326: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:18.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:17 smithi138 ceph-mon[81304]: pgmap v326: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:20.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:19 smithi033 ceph-mon[111306]: pgmap v327: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:20.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:19 smithi138 ceph-mon[81304]: pgmap v327: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:22.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:21 smithi033 ceph-mon[111306]: pgmap v328: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:22.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:21 smithi138 ceph-mon[81304]: pgmap v328: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:24.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:23 smithi033 ceph-mon[111306]: pgmap v329: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:24.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:23 smithi138 ceph-mon[81304]: pgmap v329: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:26.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:25 smithi033 ceph-mon[111306]: pgmap v330: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:26.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:50:26.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:25 smithi138 ceph-mon[81304]: pgmap v330: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:26.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:50:28.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:27 smithi033 ceph-mon[111306]: pgmap v331: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:28.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:27 smithi138 ceph-mon[81304]: pgmap v331: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:30.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:29 smithi033 ceph-mon[111306]: pgmap v332: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:30.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:29 smithi138 ceph-mon[81304]: pgmap v332: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:32.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:31 smithi033 ceph-mon[111306]: pgmap v333: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:32.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:31 smithi138 ceph-mon[81304]: pgmap v333: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:34.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:33 smithi033 ceph-mon[111306]: pgmap v334: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:34.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:33 smithi138 ceph-mon[81304]: pgmap v334: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:36.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:35 smithi033 ceph-mon[111306]: pgmap v335: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:36.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:35 smithi138 ceph-mon[81304]: pgmap v335: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:38.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:37 smithi033 ceph-mon[111306]: pgmap v336: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:38.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:37 smithi138 ceph-mon[81304]: pgmap v336: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:40.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:39 smithi033 ceph-mon[111306]: pgmap v337: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:40.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:39 smithi138 ceph-mon[81304]: pgmap v337: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:41.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:50:41.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:50:42.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:41 smithi033 ceph-mon[111306]: pgmap v338: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:42.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:41 smithi138 ceph-mon[81304]: pgmap v338: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:44.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:43 smithi033 ceph-mon[111306]: pgmap v339: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:44.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.273+0000 7f460e0d1700 1 -- 172.21.15.33:0/1030008017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 msgr2=0x7f4608103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.273+0000 7f460e0d1700 1 --2- 172.21.15.33:0/1030008017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 0x7f4608103190 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f45f8004940 tx=0x7f45f8009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.273+0000 7f460e0d1700 1 -- 172.21.15.33:0/1030008017 shutdown_connections 2024-04-25T01:50:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.273+0000 7f460e0d1700 1 --2- 172.21.15.33:0/1030008017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608103760 0x7f46080fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.273+0000 7f460e0d1700 1 --2- 172.21.15.33:0/1030008017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 0x7f4608103190 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.273+0000 7f460e0d1700 1 -- 172.21.15.33:0/1030008017 >> 172.21.15.33:0/1030008017 conn(0x7f46080f8930 msgr2=0x7f46080fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.273+0000 7f460e0d1700 1 -- 172.21.15.33:0/1030008017 shutdown_connections 2024-04-25T01:50:44.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.273+0000 7f460e0d1700 1 -- 172.21.15.33:0/1030008017 wait complete. 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f460e0d1700 1 Processor -- start 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f460e0d1700 1 -- start start 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f460e0d1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 0x7f4608113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f460e0d1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608103760 0x7f4608113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f460e0d1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46081144f0 con 0x7f4608102dc0 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f460e0d1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460810daf0 con 0x7f4608103760 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f46077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 0x7f4608113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f46077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 0x7f4608113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54452/0 (socket says 172.21.15.33:54452) 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f46077fe700 1 -- 172.21.15.33:0/3010351725 learned_addr learned my addr 172.21.15.33:0/3010351725 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.274+0000 7f4606ffd700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608103760 0x7f4608113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.275+0000 7f46077fe700 1 -- 172.21.15.33:0/3010351725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608103760 msgr2=0x7f4608113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.275+0000 7f46077fe700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608103760 0x7f4608113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.275+0000 7f46077fe700 1 -- 172.21.15.33:0/3010351725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8005040 con 0x7f4608102dc0 2024-04-25T01:50:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.275+0000 7f46077fe700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 0x7f4608113960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009180 tx=0x7f45f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.275+0000 7f45effff700 1 -- 172.21.15.33:0/3010351725 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45f800e040 con 0x7f4608102dc0 2024-04-25T01:50:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.275+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460810dd70 con 0x7f4608102dc0 2024-04-25T01:50:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.275+0000 7f45effff700 1 -- 172.21.15.33:0/3010351725 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45f8013420 con 0x7f4608102dc0 2024-04-25T01:50:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.275+0000 7f45effff700 1 -- 172.21.15.33:0/3010351725 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45f801c630 con 0x7f4608102dc0 2024-04-25T01:50:44.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.275+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460810e330 con 0x7f4608102dc0 2024-04-25T01:50:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.276+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46080fe510 con 0x7f4608102dc0 2024-04-25T01:50:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.277+0000 7f45effff700 1 -- 172.21.15.33:0/3010351725 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f45f8013590 con 0x7f4608102dc0 2024-04-25T01:50:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.277+0000 7f45effff700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45f006e140 0x7f45f00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.277+0000 7f45effff700 1 -- 172.21.15.33:0/3010351725 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f45f808ebe0 con 0x7f4608102dc0 2024-04-25T01:50:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.277+0000 7f4606ffd700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45f006e140 0x7f45f00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:44.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.278+0000 7f4606ffd700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45f006e140 0x7f45f00705f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f460810ecb0 tx=0x7f45fc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:44.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.282+0000 7f45effff700 1 -- 172.21.15.33:0/3010351725 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f45f805bbb0 con 0x7f4608102dc0 2024-04-25T01:50:44.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:43 smithi138 ceph-mon[81304]: pgmap v339: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:44.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.510+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4608109f30 con 0x7f45f006e140 2024-04-25T01:50:44.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.511+0000 7f45effff700 1 -- 172.21.15.33:0/3010351725 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4608109f30 con 0x7f45f006e140 2024-04-25T01:50:44.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45f006e140 msgr2=0x7f45f00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45f006e140 0x7f45f00705f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f460810ecb0 tx=0x7f45fc00b040 comp rx=0 tx=0).stop 2024-04-25T01:50:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 msgr2=0x7f4608113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 0x7f4608113960 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f45f8009180 tx=0x7f45f8009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 shutdown_connections 2024-04-25T01:50:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f45f006e140 0x7f45f00705f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4608102dc0 0x7f4608113960 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 --2- 172.21.15.33:0/3010351725 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4608103760 0x7f4608113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 >> 172.21.15.33:0/3010351725 conn(0x7f46080f8930 msgr2=0x7f46080f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 shutdown_connections 2024-04-25T01:50:44.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.513+0000 7f460e0d1700 1 -- 172.21.15.33:0/3010351725 wait complete. 2024-04-25T01:50:44.522 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:50:44.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.620+0000 7fd8b479e700 1 -- 172.21.15.33:0/3321415242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 msgr2=0x7fd8ac0fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.620+0000 7fd8b479e700 1 --2- 172.21.15.33:0/3321415242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 0x7fd8ac0fcdd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd89c004940 tx=0x7fd89c009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:44.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 -- 172.21.15.33:0/3321415242 shutdown_connections 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 --2- 172.21.15.33:0/3321415242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 0x7fd8ac0fcdd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 --2- 172.21.15.33:0/3321415242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8ac102dc0 0x7fd8ac103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 -- 172.21.15.33:0/3321415242 >> 172.21.15.33:0/3321415242 conn(0x7fd8ac0f8930 msgr2=0x7fd8ac0fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 -- 172.21.15.33:0/3321415242 shutdown_connections 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 -- 172.21.15.33:0/3321415242 wait complete. 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 Processor -- start 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 -- start start 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8ac102dc0 0x7fd8ac070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 0x7fd8ac0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac0721b0 con 0x7fd8ac103760 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b479e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac070ce0 con 0x7fd8ac102dc0 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.621+0000 7fd8b1d39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 0x7fd8ac0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b1d39700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 0x7fd8ac0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54464/0 (socket says 172.21.15.33:54464) 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b1d39700 1 -- 172.21.15.33:0/3786541347 learned_addr learned my addr 172.21.15.33:0/3786541347 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:44.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b253a700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8ac102dc0 0x7fd8ac070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b1d39700 1 -- 172.21.15.33:0/3786541347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8ac102dc0 msgr2=0x7fd8ac070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b1d39700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8ac102dc0 0x7fd8ac070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b1d39700 1 -- 172.21.15.33:0/3786541347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c005040 con 0x7fd8ac103760 2024-04-25T01:50:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b253a700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8ac102dc0 0x7fd8ac070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:50:44.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b1d39700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 0x7fd8ac0707a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009100 tx=0x7fd89c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8a2ffd700 1 -- 172.21.15.33:0/3786541347 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd89c00e040 con 0x7fd8ac103760 2024-04-25T01:50:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ac070f60 con 0x7fd8ac103760 2024-04-25T01:50:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8a2ffd700 1 -- 172.21.15.33:0/3786541347 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd89c013420 con 0x7fd8ac103760 2024-04-25T01:50:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8a2ffd700 1 -- 172.21.15.33:0/3786541347 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd89c01c610 con 0x7fd8ac103760 2024-04-25T01:50:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.622+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac11a0a0 con 0x7fd8ac103760 2024-04-25T01:50:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.623+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac0fe510 con 0x7fd8ac103760 2024-04-25T01:50:44.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.624+0000 7fd8a2ffd700 1 -- 172.21.15.33:0/3786541347 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd89c013590 con 0x7fd8ac103760 2024-04-25T01:50:44.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.624+0000 7fd8a2ffd700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd89806e140 0x7fd8980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:44.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.624+0000 7fd8a2ffd700 1 -- 172.21.15.33:0/3786541347 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd89c08ed90 con 0x7fd8ac103760 2024-04-25T01:50:44.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.624+0000 7fd8b253a700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd89806e140 0x7fd8980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:44.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.625+0000 7fd8b253a700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd89806e140 0x7fd8980705f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8004c60 tx=0x7fd8a8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:44.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.629+0000 7fd8a2ffd700 1 -- 172.21.15.33:0/3786541347 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd89c058190 con 0x7fd8ac103760 2024-04-25T01:50:44.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.853+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8ac0fa000 con 0x7fd89806e140 2024-04-25T01:50:44.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.855+0000 7fd8a2ffd700 1 -- 172.21.15.33:0/3786541347 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd8ac0fa000 con 0x7fd89806e140 2024-04-25T01:50:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.857+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd89806e140 msgr2=0x7fd8980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.857+0000 7fd8b479e700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd89806e140 0x7fd8980705f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd8a8004c60 tx=0x7fd8a8009040 comp rx=0 tx=0).stop 2024-04-25T01:50:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.857+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 msgr2=0x7fd8ac0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.857+0000 7fd8b479e700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 0x7fd8ac0707a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd89c009100 tx=0x7fd89c009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.858+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 shutdown_connections 2024-04-25T01:50:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.858+0000 7fd8b479e700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd89806e140 0x7fd8980705f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.858+0000 7fd8b479e700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8ac102dc0 0x7fd8ac070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.858+0000 7fd8b479e700 1 --2- 172.21.15.33:0/3786541347 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8ac103760 0x7fd8ac0707a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.858+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 >> 172.21.15.33:0/3786541347 conn(0x7fd8ac0f8930 msgr2=0x7fd8ac0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.858+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 shutdown_connections 2024-04-25T01:50:44.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.858+0000 7fd8b479e700 1 -- 172.21.15.33:0/3786541347 wait complete. 2024-04-25T01:50:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.962+0000 7fe35596f700 1 -- 172.21.15.33:0/2631973302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 msgr2=0x7fe3501020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.962+0000 7fe35596f700 1 --2- 172.21.15.33:0/2631973302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 0x7fe3501020a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe340004940 tx=0x7fe340009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.962+0000 7fe35596f700 1 -- 172.21.15.33:0/2631973302 shutdown_connections 2024-04-25T01:50:44.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.963+0000 7fe35596f700 1 --2- 172.21.15.33:0/2631973302 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350102670 0x7fe3500fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.963+0000 7fe35596f700 1 --2- 172.21.15.33:0/2631973302 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 0x7fe3501020a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.963+0000 7fe35596f700 1 -- 172.21.15.33:0/2631973302 >> 172.21.15.33:0/2631973302 conn(0x7fe350071460 msgr2=0x7fe350071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:44.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.963+0000 7fe35596f700 1 -- 172.21.15.33:0/2631973302 shutdown_connections 2024-04-25T01:50:44.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.963+0000 7fe35596f700 1 -- 172.21.15.33:0/2631973302 wait complete. 2024-04-25T01:50:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.963+0000 7fe35596f700 1 Processor -- start 2024-04-25T01:50:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.963+0000 7fe35596f700 1 -- start start 2024-04-25T01:50:44.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe35596f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 0x7fe3500fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe35596f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350102670 0x7fe3500fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe35596f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500fd2b0 con 0x7fe350101cd0 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe35596f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500fd3f0 con 0x7fe350102670 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe34effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 0x7fe3500fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe34effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 0x7fe3500fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54486/0 (socket says 172.21.15.33:54486) 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe34effd700 1 -- 172.21.15.33:0/3290708729 learned_addr learned my addr 172.21.15.33:0/3290708729 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe34e7fc700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350102670 0x7fe3500fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe34effd700 1 -- 172.21.15.33:0/3290708729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350102670 msgr2=0x7fe3500fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe34effd700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350102670 0x7fe3500fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe34effd700 1 -- 172.21.15.33:0/3290708729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe340005040 con 0x7fe350101cd0 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe34effd700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 0x7fe3500fc7e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe34000b8b0 tx=0x7fe340009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe3377fe700 1 -- 172.21.15.33:0/3290708729 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe34000e040 con 0x7fe350101cd0 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3500fd640 con 0x7fe350101cd0 2024-04-25T01:50:44.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe3377fe700 1 -- 172.21.15.33:0/3290708729 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe340004290 con 0x7fe350101cd0 2024-04-25T01:50:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe3377fe700 1 -- 172.21.15.33:0/3290708729 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe34001c3f0 con 0x7fe350101cd0 2024-04-25T01:50:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.964+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35011a2d0 con 0x7fe350101cd0 2024-04-25T01:50:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.966+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe350065200 con 0x7fe350101cd0 2024-04-25T01:50:44.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.966+0000 7fe3377fe700 1 -- 172.21.15.33:0/3290708729 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe3400137b0 con 0x7fe350101cd0 2024-04-25T01:50:44.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.966+0000 7fe3377fe700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe33806e140 0x7fe3380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:44.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.967+0000 7fe3377fe700 1 -- 172.21.15.33:0/3290708729 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe34008ee50 con 0x7fe350101cd0 2024-04-25T01:50:44.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.967+0000 7fe34e7fc700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe33806e140 0x7fe3380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:44.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.967+0000 7fe34e7fc700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe33806e140 0x7fe3380705f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe3500fe120 tx=0x7fe344009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:44.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:44.971+0000 7fe3377fe700 1 -- 172.21.15.33:0/3290708729 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe340058250 con 0x7fe350101cd0 2024-04-25T01:50:45.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.190+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe35004bed0 con 0x7fe33806e140 2024-04-25T01:50:45.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.201+0000 7fe3377fe700 1 -- 172.21.15.33:0/3290708729 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe35004bed0 con 0x7fe33806e140 2024-04-25T01:50:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:50:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (11m) 5m ago 23m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:50:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (9m) 5m ago 23m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:50:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (9m) 4m ago 22m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:50:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (11m) 5m ago 22m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:50:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (5m) 5m ago 17m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:50:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (5m) 5m ago 17m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:50:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (4m) 4m ago 17m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:50:45.200 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (17m) 4m ago 17m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (13m) 5m ago 24m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (12m) 4m ago 22m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (10m) 5m ago 24m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (9m) 4m ago 22m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (12m) 5m ago 23m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (12m) 4m ago 22m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (8m) 5m ago 21m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (8m) 5m ago 20m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (7m) 5m ago 20m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:50:45.201 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (6m) 4m ago 20m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:50:45.202 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (6m) 4m ago 19m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:50:45.202 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (5m) 4m ago 19m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:50:45.202 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (12m) 5m ago 22m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:50:45.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe33806e140 msgr2=0x7fe3380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe33806e140 0x7fe3380705f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe3500fe120 tx=0x7fe344009ee0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 msgr2=0x7fe3500fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 0x7fe3500fc7e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe34000b8b0 tx=0x7fe340009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:45.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 shutdown_connections 2024-04-25T01:50:45.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe33806e140 0x7fe3380705f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe350101cd0 0x7fe3500fc7e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 --2- 172.21.15.33:0/3290708729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe350102670 0x7fe3500fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 >> 172.21.15.33:0/3290708729 conn(0x7fe350071460 msgr2=0x7fe3500f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:45.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 shutdown_connections 2024-04-25T01:50:45.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.203+0000 7fe35596f700 1 -- 172.21.15.33:0/3290708729 wait complete. 2024-04-25T01:50:45.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.306+0000 7fd397b02700 1 -- 172.21.15.33:0/2997072181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd390070bb0 msgr2=0x7fd390109160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.306+0000 7fd397b02700 1 --2- 172.21.15.33:0/2997072181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd390070bb0 0x7fd390109160 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fd38c004970 tx=0x7fd38c009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.306+0000 7fd397b02700 1 -- 172.21.15.33:0/2997072181 shutdown_connections 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.306+0000 7fd397b02700 1 --2- 172.21.15.33:0/2997072181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd390070bb0 0x7fd390109160 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.306+0000 7fd397b02700 1 --2- 172.21.15.33:0/2997072181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd3900702a0 0x7fd390070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.306+0000 7fd397b02700 1 -- 172.21.15.33:0/2997072181 >> 172.21.15.33:0/2997072181 conn(0x7fd3900f8980 msgr2=0x7fd3900fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.306+0000 7fd397b02700 1 -- 172.21.15.33:0/2997072181 shutdown_connections 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.306+0000 7fd397b02700 1 -- 172.21.15.33:0/2997072181 wait complete. 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd397b02700 1 Processor -- start 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd397b02700 1 -- start start 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd397b02700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3900702a0 0x7fd390115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:45.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd397b02700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd390070bb0 0x7fd3901162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd397b02700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3901169c0 con 0x7fd3900702a0 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd397b02700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39010fd00 con 0x7fd390070bb0 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd39589e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3900702a0 0x7fd390115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd39589e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3900702a0 0x7fd390115da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54504/0 (socket says 172.21.15.33:54504) 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd39589e700 1 -- 172.21.15.33:0/3649562521 learned_addr learned my addr 172.21.15.33:0/3649562521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd39509d700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd390070bb0 0x7fd3901162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd39589e700 1 -- 172.21.15.33:0/3649562521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd390070bb0 msgr2=0x7fd3901162e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd39589e700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd390070bb0 0x7fd3901162e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd39589e700 1 -- 172.21.15.33:0/3649562521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c005040 con 0x7fd3900702a0 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.307+0000 7fd39509d700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd390070bb0 0x7fd3901162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:50:45.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.308+0000 7fd39589e700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3900702a0 0x7fd390115da0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd38000b790 tx=0x7fd380009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:45.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.308+0000 7fd3867fc700 1 -- 172.21.15.33:0/3649562521 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd38000f770 con 0x7fd3900702a0 2024-04-25T01:50:45.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.308+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39010ff80 con 0x7fd3900702a0 2024-04-25T01:50:45.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.308+0000 7fd3867fc700 1 -- 172.21.15.33:0/3649562521 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd38000fda0 con 0x7fd3900702a0 2024-04-25T01:50:45.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.308+0000 7fd3867fc700 1 -- 172.21.15.33:0/3649562521 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd38000d310 con 0x7fd3900702a0 2024-04-25T01:50:45.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.308+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd390110490 con 0x7fd3900702a0 2024-04-25T01:50:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.309+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3900719e0 con 0x7fd3900702a0 2024-04-25T01:50:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.309+0000 7fd3867fc700 1 -- 172.21.15.33:0/3649562521 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd380010420 con 0x7fd3900702a0 2024-04-25T01:50:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.310+0000 7fd3867fc700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd37c06e140 0x7fd37c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.310+0000 7fd3867fc700 1 -- 172.21.15.33:0/3649562521 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd38008db40 con 0x7fd3900702a0 2024-04-25T01:50:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.310+0000 7fd39509d700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd37c06e140 0x7fd37c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:45.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.310+0000 7fd39509d700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd37c06e140 0x7fd37c0705f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd38c004f50 tx=0x7fd38c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:45.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.315+0000 7fd3867fc700 1 -- 172.21.15.33:0/3649562521 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd38005aa70 con 0x7fd3900702a0 2024-04-25T01:50:45.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.613+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd3901113f0 con 0x7fd3900702a0 2024-04-25T01:50:45.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.614+0000 7fd3867fc700 1 -- 172.21.15.33:0/3649562521 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fd38005a600 con 0x7fd3900702a0 2024-04-25T01:50:45.612 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:50:45.612 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:50:45.613 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:50:45.614 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:50:45.614 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:50:45.614 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:50:45.614 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd37c06e140 msgr2=0x7fd37c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd37c06e140 0x7fd37c0705f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd38c004f50 tx=0x7fd38c009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3900702a0 msgr2=0x7fd390115da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3900702a0 0x7fd390115da0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd38000b790 tx=0x7fd380009250 comp rx=0 tx=0).stop 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 shutdown_connections 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd37c06e140 0x7fd37c0705f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd3900702a0 0x7fd390115da0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 --2- 172.21.15.33:0/3649562521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd390070bb0 0x7fd3901162e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 >> 172.21.15.33:0/3649562521 conn(0x7fd3900f8980 msgr2=0x7fd3900f9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 shutdown_connections 2024-04-25T01:50:45.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.616+0000 7fd397b02700 1 -- 172.21.15.33:0/3649562521 wait complete. 2024-04-25T01:50:45.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.720+0000 7f66106a4700 1 -- 172.21.15.33:0/3170827543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 msgr2=0x7f66081020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.720+0000 7f66106a4700 1 --2- 172.21.15.33:0/3170827543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 0x7f66081020a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f6600009ce0 tx=0x7f6600004d10 comp rx=0 tx=0).stop 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.720+0000 7f66106a4700 1 -- 172.21.15.33:0/3170827543 shutdown_connections 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.720+0000 7f66106a4700 1 --2- 172.21.15.33:0/3170827543 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608102670 0x7f66080fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.720+0000 7f66106a4700 1 --2- 172.21.15.33:0/3170827543 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 0x7f66081020a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.720+0000 7f66106a4700 1 -- 172.21.15.33:0/3170827543 >> 172.21.15.33:0/3170827543 conn(0x7f6608071460 msgr2=0x7f6608071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.720+0000 7f66106a4700 1 -- 172.21.15.33:0/3170827543 shutdown_connections 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.720+0000 7f66106a4700 1 -- 172.21.15.33:0/3170827543 wait complete. 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f66106a4700 1 Processor -- start 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f66106a4700 1 -- start start 2024-04-25T01:50:45.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f66106a4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 0x7f6608113d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f66106a4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608102670 0x7f6608114240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f66106a4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6608114890 con 0x7f6608101cd0 2024-04-25T01:50:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f66106a4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660810dc60 con 0x7f6608102670 2024-04-25T01:50:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f660e440700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 0x7f6608113d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f660e440700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 0x7f6608113d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46218/0 (socket says 172.21.15.33:46218) 2024-04-25T01:50:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f660e440700 1 -- 172.21.15.33:0/4157066394 learned_addr learned my addr 172.21.15.33:0/4157066394 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f660dc3f700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608102670 0x7f6608114240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:45.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f660e440700 1 -- 172.21.15.33:0/4157066394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608102670 msgr2=0x7f6608114240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f660e440700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608102670 0x7f6608114240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f660e440700 1 -- 172.21.15.33:0/4157066394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6600009940 con 0x7f6608101cd0 2024-04-25T01:50:45.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.721+0000 7f660e440700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 0x7f6608113d00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6600003400 tx=0x7f6600004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:45.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.722+0000 7f6606ffd700 1 -- 172.21.15.33:0/4157066394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f660000e040 con 0x7f6608101cd0 2024-04-25T01:50:45.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.722+0000 7f6606ffd700 1 -- 172.21.15.33:0/4157066394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6600004610 con 0x7f6608101cd0 2024-04-25T01:50:45.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.722+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f660810dee0 con 0x7f6608101cd0 2024-04-25T01:50:45.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.722+0000 7f6606ffd700 1 -- 172.21.15.33:0/4157066394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f660001c640 con 0x7f6608101cd0 2024-04-25T01:50:45.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.722+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f660810e4a0 con 0x7f6608101cd0 2024-04-25T01:50:45.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.723+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66081052e0 con 0x7f6608101cd0 2024-04-25T01:50:45.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.723+0000 7f6606ffd700 1 -- 172.21.15.33:0/4157066394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6600012a70 con 0x7f6608101cd0 2024-04-25T01:50:45.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.724+0000 7f6606ffd700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e140 0x7f65f00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:45.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.724+0000 7f6606ffd700 1 -- 172.21.15.33:0/4157066394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f660008f550 con 0x7f6608101cd0 2024-04-25T01:50:45.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.724+0000 7f660dc3f700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e140 0x7f65f00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:45.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.724+0000 7f660dc3f700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e140 0x7f65f00705f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f660810f5a0 tx=0x7f65fc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:45.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.729+0000 7f6606ffd700 1 -- 172.21.15.33:0/4157066394 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f660005c400 con 0x7f6608101cd0 2024-04-25T01:50:45.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.980+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f660804bed0 con 0x7f6608101cd0 2024-04-25T01:50:45.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.981+0000 7f6606ffd700 1 -- 172.21.15.33:0/4157066394 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f6600004e10 con 0x7f6608101cd0 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:50:45.980 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:50:45.981 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:50:45.982 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:50:45.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e140 msgr2=0x7f65f00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e140 0x7f65f00705f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f660810f5a0 tx=0x7f65fc00b040 comp rx=0 tx=0).stop 2024-04-25T01:50:45.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 msgr2=0x7f6608113d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:45.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 0x7f6608113d00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6600003400 tx=0x7f6600004d10 comp rx=0 tx=0).stop 2024-04-25T01:50:45.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 shutdown_connections 2024-04-25T01:50:45.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f65f006e140 0x7f65f00705f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6608101cd0 0x7f6608113d00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 --2- 172.21.15.33:0/4157066394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6608102670 0x7f6608114240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:45.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 >> 172.21.15.33:0/4157066394 conn(0x7f6608071460 msgr2=0x7f66080f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:45.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 shutdown_connections 2024-04-25T01:50:45.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:45.983+0000 7f66106a4700 1 -- 172.21.15.33:0/4157066394 wait complete. 2024-04-25T01:50:45.984 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:50:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.088+0000 7fcc014c8700 1 -- 172.21.15.33:0/1350012275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 msgr2=0x7fcbfc1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.088+0000 7fcc014c8700 1 --2- 172.21.15.33:0/1350012275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 0x7fcbfc1020a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4009ce0 tx=0x7fcbf4004d10 comp rx=0 tx=0).stop 2024-04-25T01:50:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.088+0000 7fcc014c8700 1 -- 172.21.15.33:0/1350012275 shutdown_connections 2024-04-25T01:50:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.088+0000 7fcc014c8700 1 --2- 172.21.15.33:0/1350012275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc102670 0x7fcbfc0fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.088+0000 7fcc014c8700 1 --2- 172.21.15.33:0/1350012275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 0x7fcbfc1020a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.088+0000 7fcc014c8700 1 -- 172.21.15.33:0/1350012275 >> 172.21.15.33:0/1350012275 conn(0x7fcbfc071460 msgr2=0x7fcbfc071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.088+0000 7fcc014c8700 1 -- 172.21.15.33:0/1350012275 shutdown_connections 2024-04-25T01:50:46.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.088+0000 7fcc014c8700 1 -- 172.21.15.33:0/1350012275 wait complete. 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcc014c8700 1 Processor -- start 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcc014c8700 1 -- start start 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcc014c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 0x7fcbfc113cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcc014c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc102670 0x7fcbfc114200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcc014c8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc114850 con 0x7fcbfc101cd0 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcc014c8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbfc10dc20 con 0x7fcbfc102670 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcbfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 0x7fcbfc113cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcbfaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 0x7fcbfc113cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46238/0 (socket says 172.21.15.33:46238) 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcbfaffd700 1 -- 172.21.15.33:0/468531341 learned_addr learned my addr 172.21.15.33:0/468531341 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcbfa7fc700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc102670 0x7fcbfc114200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:46.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcbfaffd700 1 -- 172.21.15.33:0/468531341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc102670 msgr2=0x7fcbfc114200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcbfaffd700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc102670 0x7fcbfc114200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.089+0000 7fcbfaffd700 1 -- 172.21.15.33:0/468531341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf4009940 con 0x7fcbfc101cd0 2024-04-25T01:50:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.090+0000 7fcbfaffd700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 0x7fcbfc113cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4003400 tx=0x7fcbf4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.090+0000 7fcbf37fe700 1 -- 172.21.15.33:0/468531341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf400e040 con 0x7fcbfc101cd0 2024-04-25T01:50:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.090+0000 7fcbf37fe700 1 -- 172.21.15.33:0/468531341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbf4004610 con 0x7fcbfc101cd0 2024-04-25T01:50:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.090+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbfc10dea0 con 0x7fcbfc101cd0 2024-04-25T01:50:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.090+0000 7fcbf37fe700 1 -- 172.21.15.33:0/468531341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbf401c6d0 con 0x7fcbfc101cd0 2024-04-25T01:50:46.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.090+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbfc10e460 con 0x7fcbfc101cd0 2024-04-25T01:50:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.091+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbfc1052e0 con 0x7fcbfc101cd0 2024-04-25T01:50:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.092+0000 7fcbf37fe700 1 -- 172.21.15.33:0/468531341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcbf4012b00 con 0x7fcbfc101cd0 2024-04-25T01:50:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.092+0000 7fcbf37fe700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbe806e140 0x7fcbe80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.092+0000 7fcbf37fe700 1 -- 172.21.15.33:0/468531341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcbf408f4e0 con 0x7fcbfc101cd0 2024-04-25T01:50:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.092+0000 7fcbfa7fc700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbe806e140 0x7fcbe80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:46.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.092+0000 7fcbfa7fc700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbe806e140 0x7fcbe80705f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc10f560 tx=0x7fcbe4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:46.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.097+0000 7fcbf37fe700 1 -- 172.21.15.33:0/468531341 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcbf405c410 con 0x7fcbfc101cd0 2024-04-25T01:50:46.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:45 smithi033 ceph-mon[111306]: pgmap v340: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:46.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:45 smithi033 ceph-mon[111306]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:46.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:45 smithi033 ceph-mon[111306]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:46.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:45 smithi033 ceph-mon[111306]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:46.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:45 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3649562521' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:50:46.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:45 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4157066394' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:50:46.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:45 smithi138 ceph-mon[81304]: pgmap v340: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:46.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:45 smithi138 ceph-mon[81304]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:46.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:45 smithi138 ceph-mon[81304]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:46.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:45 smithi138 ceph-mon[81304]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:46.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:45 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3649562521' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:50:46.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:45 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4157066394' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:50:46.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.324+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbfc109f10 con 0x7fcbe806e140 2024-04-25T01:50:46.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.325+0000 7fcbf37fe700 1 -- 172.21.15.33:0/468531341 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcbfc109f10 con 0x7fcbe806e140 2024-04-25T01:50:46.324 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:50:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:50:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:50:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:50:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:50:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:50:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:50:46.324 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:50:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:50:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:50:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:50:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:50:46.325 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:50:46.325 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:50:46.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbe806e140 msgr2=0x7fcbe80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:46.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbe806e140 0x7fcbe80705f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc10f560 tx=0x7fcbe4009ee0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 msgr2=0x7fcbfc113cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 0x7fcbfc113cc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4003400 tx=0x7fcbf4004d10 comp rx=0 tx=0).stop 2024-04-25T01:50:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 shutdown_connections 2024-04-25T01:50:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcbe806e140 0x7fcbe80705f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcbfc101cd0 0x7fcbfc113cc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 --2- 172.21.15.33:0/468531341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcbfc102670 0x7fcbfc114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 >> 172.21.15.33:0/468531341 conn(0x7fcbfc071460 msgr2=0x7fcbfc0fabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 shutdown_connections 2024-04-25T01:50:46.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.328+0000 7fcc014c8700 1 -- 172.21.15.33:0/468531341 wait complete. 2024-04-25T01:50:46.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.432+0000 7fa81d763700 1 -- 172.21.15.33:0/1716841070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 msgr2=0x7fa8180fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.432+0000 7fa81d763700 1 --2- 172.21.15.33:0/1716841070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 0x7fa8180fce00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa808004940 tx=0x7fa808009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.432+0000 7fa81d763700 1 -- 172.21.15.33:0/1716841070 shutdown_connections 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.432+0000 7fa81d763700 1 --2- 172.21.15.33:0/1716841070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 0x7fa8180fce00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.432+0000 7fa81d763700 1 --2- 172.21.15.33:0/1716841070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa818102df0 0x7fa8181031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.432+0000 7fa81d763700 1 -- 172.21.15.33:0/1716841070 >> 172.21.15.33:0/1716841070 conn(0x7fa8180f8980 msgr2=0x7fa8180fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.432+0000 7fa81d763700 1 -- 172.21.15.33:0/1716841070 shutdown_connections 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.432+0000 7fa81d763700 1 -- 172.21.15.33:0/1716841070 wait complete. 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa81d763700 1 Processor -- start 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa81d763700 1 -- start start 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa81d763700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa818102df0 0x7fa818070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa81d763700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 0x7fa8180707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa81d763700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818070d30 con 0x7fa818103790 2024-04-25T01:50:46.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa81d763700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa818070e70 con 0x7fa818102df0 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa8167fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 0x7fa8180707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa8167fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 0x7fa8180707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46242/0 (socket says 172.21.15.33:46242) 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa8167fc700 1 -- 172.21.15.33:0/1166850385 learned_addr learned my addr 172.21.15.33:0/1166850385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.433+0000 7fa816ffd700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa818102df0 0x7fa818070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.434+0000 7fa8167fc700 1 -- 172.21.15.33:0/1166850385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa818102df0 msgr2=0x7fa818070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.434+0000 7fa8167fc700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa818102df0 0x7fa818070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.434+0000 7fa8167fc700 1 -- 172.21.15.33:0/1166850385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa808005040 con 0x7fa818103790 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.434+0000 7fa8167fc700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 0x7fa8180707a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa808009100 tx=0x7fa808009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.434+0000 7fa7ff7fe700 1 -- 172.21.15.33:0/1166850385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80800e040 con 0x7fa818103790 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.434+0000 7fa7ff7fe700 1 -- 172.21.15.33:0/1166850385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa808009af0 con 0x7fa818103790 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.434+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8180710c0 con 0x7fa818103790 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.434+0000 7fa7ff7fe700 1 -- 172.21.15.33:0/1166850385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80801c3f0 con 0x7fa818103790 2024-04-25T01:50:46.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.434+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81811a310 con 0x7fa818103790 2024-04-25T01:50:46.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.435+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8181052c0 con 0x7fa818103790 2024-04-25T01:50:46.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.435+0000 7fa7ff7fe700 1 -- 172.21.15.33:0/1166850385 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa80800baf0 con 0x7fa818103790 2024-04-25T01:50:46.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.436+0000 7fa7ff7fe700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa80006e140 0x7fa8000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:50:46.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.436+0000 7fa7ff7fe700 1 -- 172.21.15.33:0/1166850385 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa80808ec90 con 0x7fa818103790 2024-04-25T01:50:46.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.436+0000 7fa816ffd700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa80006e140 0x7fa8000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:50:46.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.437+0000 7fa816ffd700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa80006e140 0x7fa8000705f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa80c004ed0 tx=0x7fa80c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:50:46.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.441+0000 7fa7ff7fe700 1 -- 172.21.15.33:0/1166850385 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa808058130 con 0x7fa818103790 2024-04-25T01:50:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.732+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa8181052c0 con 0x7fa818103790 2024-04-25T01:50:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.732+0000 7fa7ff7fe700 1 -- 172.21.15.33:0/1166850385 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa80805b7f0 con 0x7fa818103790 2024-04-25T01:50:46.731 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:50:46.731 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:50:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:50:46.731 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:50:46.731 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa80006e140 msgr2=0x7fa8000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa80006e140 0x7fa8000705f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa80c004ed0 tx=0x7fa80c00b040 comp rx=0 tx=0).stop 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 msgr2=0x7fa8180707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 0x7fa8180707a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa808009100 tx=0x7fa808009e30 comp rx=0 tx=0).stop 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 shutdown_connections 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa80006e140 0x7fa8000705f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa818102df0 0x7fa818070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 --2- 172.21.15.33:0/1166850385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa818103790 0x7fa8180707a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 >> 172.21.15.33:0/1166850385 conn(0x7fa8180f8980 msgr2=0x7fa818102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 shutdown_connections 2024-04-25T01:50:46.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:50:46.734+0000 7fa81d763700 1 -- 172.21.15.33:0/1166850385 wait complete. 2024-04-25T01:50:47.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:47 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1166850385' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:50:47.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:47 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1166850385' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:50:48.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:48 smithi033 ceph-mon[111306]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:48.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:48 smithi033 ceph-mon[111306]: pgmap v341: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:48.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:48 smithi138 ceph-mon[81304]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:50:48.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:48 smithi138 ceph-mon[81304]: pgmap v341: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:50.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:50 smithi033 ceph-mon[111306]: pgmap v342: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:50.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:50 smithi138 ceph-mon[81304]: pgmap v342: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:52.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:52 smithi033 ceph-mon[111306]: pgmap v343: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:52.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:52 smithi138 ceph-mon[81304]: pgmap v343: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:54.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:54 smithi033 ceph-mon[111306]: pgmap v344: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:54.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:54 smithi138 ceph-mon[81304]: pgmap v344: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:56.264 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:56 smithi033 ceph-mon[111306]: pgmap v345: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:56.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:50:56.289 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:56 smithi138 ceph-mon[81304]: pgmap v345: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:56.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:50:58.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:50:58 smithi138 ceph-mon[81304]: pgmap v346: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:50:58.514 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:50:58 smithi033 ceph-mon[111306]: pgmap v346: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:00.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:00 smithi138 ceph-mon[81304]: pgmap v347: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:00.514 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:00 smithi033 ceph-mon[111306]: pgmap v347: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:02.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:02 smithi138 ceph-mon[81304]: pgmap v348: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:02.514 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:02 smithi033 ceph-mon[111306]: pgmap v348: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:04.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:04 smithi138 ceph-mon[81304]: pgmap v349: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:04.514 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:04 smithi033 ceph-mon[111306]: pgmap v349: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:06.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:06 smithi138 ceph-mon[81304]: pgmap v350: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:06.514 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:06 smithi033 ceph-mon[111306]: pgmap v350: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:08.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:08 smithi138 ceph-mon[81304]: pgmap v351: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:08.514 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:08 smithi033 ceph-mon[111306]: pgmap v351: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:10.290 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:10 smithi138 ceph-mon[81304]: pgmap v352: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:10.514 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:10 smithi033 ceph-mon[111306]: pgmap v352: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:11.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:51:11.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:51:12.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:12 smithi033 ceph-mon[111306]: pgmap v353: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:12.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:12 smithi138 ceph-mon[81304]: pgmap v353: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:14.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:14 smithi033 ceph-mon[111306]: pgmap v354: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:14.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:14 smithi138 ceph-mon[81304]: pgmap v354: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:16.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:16 smithi033 ceph-mon[111306]: pgmap v355: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:16.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:16 smithi138 ceph-mon[81304]: pgmap v355: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.854+0000 7f169b080700 1 -- 172.21.15.33:0/977565817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16940fbd40 msgr2=0x7f16940fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:16.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.854+0000 7f169b080700 1 --2- 172.21.15.33:0/977565817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16940fbd40 0x7f16940fc1b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1688004940 tx=0x7f1688009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.854+0000 7f169b080700 1 -- 172.21.15.33:0/977565817 shutdown_connections 2024-04-25T01:51:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.854+0000 7f169b080700 1 --2- 172.21.15.33:0/977565817 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16940fbd40 0x7f16940fc1b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.854+0000 7f169b080700 1 --2- 172.21.15.33:0/977565817 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1694101d90 0x7f1694102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.854+0000 7f169b080700 1 -- 172.21.15.33:0/977565817 >> 172.21.15.33:0/977565817 conn(0x7f1694071d00 msgr2=0x7f1694072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.854+0000 7f169b080700 1 -- 172.21.15.33:0/977565817 shutdown_connections 2024-04-25T01:51:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.854+0000 7f169b080700 1 -- 172.21.15.33:0/977565817 wait complete. 2024-04-25T01:51:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f169b080700 1 Processor -- start 2024-04-25T01:51:16.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f169b080700 1 -- start start 2024-04-25T01:51:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f169b080700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16940fbd40 0x7f1694102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f169b080700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d90 0x7f1694103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:16.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f169b080700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1694103680 con 0x7f1694101d90 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f169b080700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16941037c0 con 0x7f16940fbd40 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f1693fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d90 0x7f1694103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f1693fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d90 0x7f1694103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50990/0 (socket says 172.21.15.33:50990) 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f1693fff700 1 -- 172.21.15.33:0/1261655150 learned_addr learned my addr 172.21.15.33:0/1261655150 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.855+0000 7f1698e1c700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16940fbd40 0x7f1694102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.856+0000 7f1693fff700 1 -- 172.21.15.33:0/1261655150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16940fbd40 msgr2=0x7f1694102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.856+0000 7f1693fff700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16940fbd40 0x7f1694102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.856+0000 7f1693fff700 1 -- 172.21.15.33:0/1261655150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688005040 con 0x7f1694101d90 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.856+0000 7f1693fff700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d90 0x7f1694103060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f168800b3b0 tx=0x7f1688009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.856+0000 7f16917fa700 1 -- 172.21.15.33:0/1261655150 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f168800e040 con 0x7f1694101d90 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.856+0000 7f16917fa700 1 -- 172.21.15.33:0/1261655150 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1688013420 con 0x7f1694101d90 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.856+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1694119ea0 con 0x7f1694101d90 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.856+0000 7f16917fa700 1 -- 172.21.15.33:0/1261655150 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f168801c630 con 0x7f1694101d90 2024-04-25T01:51:16.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.856+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169411a380 con 0x7f1694101d90 2024-04-25T01:51:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.857+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16940fd430 con 0x7f1694101d90 2024-04-25T01:51:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.858+0000 7f16917fa700 1 -- 172.21.15.33:0/1261655150 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1688013590 con 0x7f1694101d90 2024-04-25T01:51:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.858+0000 7f16917fa700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f167c06e140 0x7f167c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:16.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.859+0000 7f16917fa700 1 -- 172.21.15.33:0/1261655150 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f168808ed80 con 0x7f1694101d90 2024-04-25T01:51:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.859+0000 7f1698e1c700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f167c06e140 0x7f167c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:16.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.859+0000 7f1698e1c700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f167c06e140 0x7f167c0705f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1684009bf0 tx=0x7f16840091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:16.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:16.863+0000 7f16917fa700 1 -- 172.21.15.33:0/1261655150 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1688058180 con 0x7f1694101d90 2024-04-25T01:51:17.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.092+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1694108e70 con 0x7f167c06e140 2024-04-25T01:51:17.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.094+0000 7f16917fa700 1 -- 172.21.15.33:0/1261655150 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1694108e70 con 0x7f167c06e140 2024-04-25T01:51:17.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f167c06e140 msgr2=0x7f167c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f167c06e140 0x7f167c0705f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1684009bf0 tx=0x7f16840091a0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d90 msgr2=0x7f1694103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d90 0x7f1694103060 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f168800b3b0 tx=0x7f1688009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 shutdown_connections 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f167c06e140 0x7f167c0705f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16940fbd40 0x7f1694102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 --2- 172.21.15.33:0/1261655150 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1694101d90 0x7f1694103060 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 >> 172.21.15.33:0/1261655150 conn(0x7f1694071d00 msgr2=0x7f1694065610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 shutdown_connections 2024-04-25T01:51:17.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.096+0000 7f169b080700 1 -- 172.21.15.33:0/1261655150 wait complete. 2024-04-25T01:51:17.105 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.203+0000 7f3fe89e7700 1 -- 172.21.15.33:0/221433213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4102e50 msgr2=0x7f3fe4103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.203+0000 7f3fe89e7700 1 --2- 172.21.15.33:0/221433213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4102e50 0x7f3fe4103220 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4004940 tx=0x7f3fd4009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.203+0000 7f3fe89e7700 1 -- 172.21.15.33:0/221433213 shutdown_connections 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.203+0000 7f3fe89e7700 1 --2- 172.21.15.33:0/221433213 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4065640 0x7f3fe4065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.203+0000 7f3fe89e7700 1 --2- 172.21.15.33:0/221433213 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4102e50 0x7f3fe4103220 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.203+0000 7f3fe89e7700 1 -- 172.21.15.33:0/221433213 >> 172.21.15.33:0/221433213 conn(0x7f3fe4071d00 msgr2=0x7f3fe4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.203+0000 7f3fe89e7700 1 -- 172.21.15.33:0/221433213 shutdown_connections 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.203+0000 7f3fe89e7700 1 -- 172.21.15.33:0/221433213 wait complete. 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe89e7700 1 Processor -- start 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe89e7700 1 -- start start 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe89e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4065640 0x7f3fe40fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe89e7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4102e50 0x7f3fe40fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:17.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe89e7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40fc880 con 0x7f3fe4065640 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe89e7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40fc9c0 con 0x7f3fe4102e50 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4065640 0x7f3fe40fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4065640 0x7f3fe40fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51000/0 (socket says 172.21.15.33:51000) 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe259c700 1 -- 172.21.15.33:0/924872322 learned_addr learned my addr 172.21.15.33:0/924872322 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe1d9b700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4102e50 0x7f3fe40fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe259c700 1 -- 172.21.15.33:0/924872322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4102e50 msgr2=0x7f3fe40fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe259c700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4102e50 0x7f3fe40fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.204+0000 7f3fe259c700 1 -- 172.21.15.33:0/924872322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4005040 con 0x7f3fe4065640 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.205+0000 7f3fe259c700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4065640 0x7f3fe40fbdb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009b30 tx=0x7f3fd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.205+0000 7f3fd2ffd700 1 -- 172.21.15.33:0/924872322 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fd4021070 con 0x7f3fe4065640 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.205+0000 7f3fd2ffd700 1 -- 172.21.15.33:0/924872322 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fd4013c00 con 0x7f3fe4065640 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.205+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe40fcc10 con 0x7f3fe4065640 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.205+0000 7f3fd2ffd700 1 -- 172.21.15.33:0/924872322 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fd400e070 con 0x7f3fe4065640 2024-04-25T01:51:17.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.205+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe411a130 con 0x7f3fe4065640 2024-04-25T01:51:17.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.206+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe4100100 con 0x7f3fe4065640 2024-04-25T01:51:17.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.206+0000 7f3fd2ffd700 1 -- 172.21.15.33:0/924872322 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3fd401cad0 con 0x7f3fe4065640 2024-04-25T01:51:17.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.207+0000 7f3fd2ffd700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fcc06e140 0x7f3fcc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:17.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.207+0000 7f3fd2ffd700 1 -- 172.21.15.33:0/924872322 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3fd408ee50 con 0x7f3fe4065640 2024-04-25T01:51:17.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.207+0000 7f3fe1d9b700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fcc06e140 0x7f3fcc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:17.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.207+0000 7f3fe1d9b700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fcc06e140 0x7f3fcc0705f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40fd6f0 tx=0x7f3fd800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:17.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.212+0000 7f3fd2ffd700 1 -- 172.21.15.33:0/924872322 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3fd40582f0 con 0x7f3fe4065640 2024-04-25T01:51:17.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.438+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fe4109f90 con 0x7f3fcc06e140 2024-04-25T01:51:17.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.439+0000 7f3fd2ffd700 1 -- 172.21.15.33:0/924872322 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3fd800f040 con 0x7f3fcc06e140 2024-04-25T01:51:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fcc06e140 msgr2=0x7f3fcc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fcc06e140 0x7f3fcc0705f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40fd6f0 tx=0x7f3fd800b040 comp rx=0 tx=0).stop 2024-04-25T01:51:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4065640 msgr2=0x7f3fe40fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4065640 0x7f3fe40fbdb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009b30 tx=0x7f3fd4009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 shutdown_connections 2024-04-25T01:51:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fcc06e140 0x7f3fcc0705f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fe4065640 0x7f3fe40fbdb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 --2- 172.21.15.33:0/924872322 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fe4102e50 0x7f3fe40fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 >> 172.21.15.33:0/924872322 conn(0x7f3fe4071d00 msgr2=0x7f3fe40faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.441+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 shutdown_connections 2024-04-25T01:51:17.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.442+0000 7f3fe89e7700 1 -- 172.21.15.33:0/924872322 wait complete. 2024-04-25T01:51:17.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 -- 172.21.15.33:0/3356347523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 msgr2=0x7f30d0106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 --2- 172.21.15.33:0/3356347523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 0x7f30d0106de0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30c0004940 tx=0x7f30c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 -- 172.21.15.33:0/3356347523 shutdown_connections 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 --2- 172.21.15.33:0/3356347523 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 0x7f30d0106de0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 --2- 172.21.15.33:0/3356347523 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d00fde00 0x7f30d00fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 -- 172.21.15.33:0/3356347523 >> 172.21.15.33:0/3356347523 conn(0x7f30d00f7870 msgr2=0x7f30d00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 -- 172.21.15.33:0/3356347523 shutdown_connections 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 -- 172.21.15.33:0/3356347523 wait complete. 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 Processor -- start 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.546+0000 7f30d4d1b700 1 -- start start 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30d4d1b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d00fde00 0x7f30d00fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30d4d1b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 0x7f30d00fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30d4d1b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00fdb80 con 0x7f30d00fe7a0 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30d4d1b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d00fc6b0 con 0x7f30d00fde00 2024-04-25T01:51:17.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30cdd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 0x7f30d00fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30cdd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 0x7f30d00fc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51016/0 (socket says 172.21.15.33:51016) 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30cdd9b700 1 -- 172.21.15.33:0/2361397281 learned_addr learned my addr 172.21.15.33:0/2361397281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30ce59c700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d00fde00 0x7f30d00fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30cdd9b700 1 -- 172.21.15.33:0/2361397281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d00fde00 msgr2=0x7f30d00fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30cdd9b700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d00fde00 0x7f30d00fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30cdd9b700 1 -- 172.21.15.33:0/2361397281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c0005040 con 0x7f30d00fe7a0 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30cdd9b700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 0x7f30d00fc170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009380 tx=0x7f30c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30beffd700 1 -- 172.21.15.33:0/2361397281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30c000e040 con 0x7f30d00fe7a0 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.547+0000 7f30beffd700 1 -- 172.21.15.33:0/2361397281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f30c0009af0 con 0x7f30d00fe7a0 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.548+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d00fc930 con 0x7f30d00fe7a0 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.548+0000 7f30beffd700 1 -- 172.21.15.33:0/2361397281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30c001c3f0 con 0x7f30d00fe7a0 2024-04-25T01:51:17.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.548+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d011a040 con 0x7f30d00fe7a0 2024-04-25T01:51:17.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.548+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d0066b30 con 0x7f30d00fe7a0 2024-04-25T01:51:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.549+0000 7f30beffd700 1 -- 172.21.15.33:0/2361397281 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f30c0013420 con 0x7f30d00fe7a0 2024-04-25T01:51:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.550+0000 7f30beffd700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f30b806e140 0x7f30b80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.550+0000 7f30beffd700 1 -- 172.21.15.33:0/2361397281 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f30c008ee10 con 0x7f30d00fe7a0 2024-04-25T01:51:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.550+0000 7f30ce59c700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f30b806e140 0x7f30b80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:17.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.550+0000 7f30ce59c700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f30b806e140 0x7f30b80705f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30c4004f40 tx=0x7f30c400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:17.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.554+0000 7f30beffd700 1 -- 172.21.15.33:0/2361397281 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f30c0058190 con 0x7f30d00fe7a0 2024-04-25T01:51:17.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.770+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f30d00f9490 con 0x7f30b806e140 2024-04-25T01:51:17.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.780+0000 7f30beffd700 1 -- 172.21.15.33:0/2361397281 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f30d00f9490 con 0x7f30b806e140 2024-04-25T01:51:17.779 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (12m) 5m ago 23m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (9m) 5m ago 23m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (9m) 5m ago 22m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (12m) 5m ago 23m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (5m) 5m ago 18m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (5m) 5m ago 18m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (5m) 5m ago 18m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (18m) 5m ago 18m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (13m) 5m ago 24m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (13m) 5m ago 22m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (10m) 5m ago 25m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (10m) 5m ago 22m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (13m) 5m ago 23m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:51:17.780 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (13m) 5m ago 22m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:51:17.781 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (9m) 5m ago 21m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:51:17.781 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (8m) 5m ago 21m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:51:17.781 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (8m) 5m ago 20m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:51:17.781 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (7m) 5m ago 20m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:51:17.781 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (6m) 5m ago 20m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:51:17.781 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (6m) 5m ago 19m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:51:17.781 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (12m) 5m ago 23m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:51:17.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.782+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f30b806e140 msgr2=0x7f30b80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.782+0000 7f30d4d1b700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f30b806e140 0x7f30b80705f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f30c4004f40 tx=0x7f30c400b040 comp rx=0 tx=0).stop 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.782+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 msgr2=0x7f30d00fc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.782+0000 7f30d4d1b700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 0x7f30d00fc170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009380 tx=0x7f30c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.782+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 shutdown_connections 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.782+0000 7f30d4d1b700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f30b806e140 0x7f30b80705f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.782+0000 7f30d4d1b700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30d00fde00 0x7f30d00fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.782+0000 7f30d4d1b700 1 --2- 172.21.15.33:0/2361397281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30d00fe7a0 0x7f30d00fc170 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.782+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 >> 172.21.15.33:0/2361397281 conn(0x7f30d00f7870 msgr2=0x7f30d00f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.783+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 shutdown_connections 2024-04-25T01:51:17.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.783+0000 7f30d4d1b700 1 -- 172.21.15.33:0/2361397281 wait complete. 2024-04-25T01:51:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.886+0000 7f03d2c7b700 1 -- 172.21.15.33:0/4099965471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fc790 msgr2=0x7f03cc0fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.886+0000 7f03d2c7b700 1 --2- 172.21.15.33:0/4099965471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fc790 0x7f03cc0fcb60 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f03c0004940 tx=0x7f03c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.886+0000 7f03d2c7b700 1 -- 172.21.15.33:0/4099965471 shutdown_connections 2024-04-25T01:51:17.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.886+0000 7f03d2c7b700 1 --2- 172.21.15.33:0/4099965471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03cc0fd0a0 0x7f03cc109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.886+0000 7f03d2c7b700 1 --2- 172.21.15.33:0/4099965471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fc790 0x7f03cc0fcb60 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.886+0000 7f03d2c7b700 1 -- 172.21.15.33:0/4099965471 >> 172.21.15.33:0/4099965471 conn(0x7f03cc071d00 msgr2=0x7f03cc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.886+0000 7f03d2c7b700 1 -- 172.21.15.33:0/4099965471 shutdown_connections 2024-04-25T01:51:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.886+0000 7f03d2c7b700 1 -- 172.21.15.33:0/4099965471 wait complete. 2024-04-25T01:51:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.886+0000 7f03d2c7b700 1 Processor -- start 2024-04-25T01:51:17.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03d2c7b700 1 -- start start 2024-04-25T01:51:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03d2c7b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03cc0fc790 0x7f03cc115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03d2c7b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fd0a0 0x7f03cc1162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03d2c7b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc116930 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03d2c7b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc10fd40 con 0x7f03cc0fc790 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fd0a0 0x7f03cc1162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03cbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fd0a0 0x7f03cc1162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51042/0 (socket says 172.21.15.33:51042) 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03cbfff700 1 -- 172.21.15.33:0/2940734748 learned_addr learned my addr 172.21.15.33:0/2940734748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03d0a17700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03cc0fc790 0x7f03cc115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03cbfff700 1 -- 172.21.15.33:0/2940734748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03cc0fc790 msgr2=0x7f03cc115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03cbfff700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03cc0fc790 0x7f03cc115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.887+0000 7f03cbfff700 1 -- 172.21.15.33:0/2940734748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c0005040 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.888+0000 7f03cbfff700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fd0a0 0x7f03cc1162e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f03b8002a50 tx=0x7f03b80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.888+0000 7f03c97fa700 1 -- 172.21.15.33:0/2940734748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b8007cc0 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.888+0000 7f03c97fa700 1 -- 172.21.15.33:0/2940734748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f03b800f040 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.888+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc110020 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.888+0000 7f03c97fa700 1 -- 172.21.15.33:0/2940734748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03b8009640 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.888+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc110560 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.889+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc0fc330 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.889+0000 7f03c97fa700 1 -- 172.21.15.33:0/2940734748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f03b80097a0 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.890+0000 7f03c97fa700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f03bc06e140 0x7f03bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:17.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.890+0000 7f03c97fa700 1 -- 172.21.15.33:0/2940734748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f03b808d940 con 0x7f03cc0fd0a0 2024-04-25T01:51:17.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.890+0000 7f03d0a17700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f03bc06e140 0x7f03bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:17.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.890+0000 7f03d0a17700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f03bc06e140 0x7f03bc0705f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f03c0004f90 tx=0x7f03c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:17.893 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:17.895+0000 7f03c97fa700 1 -- 172.21.15.33:0/2940734748 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f03b8056cc0 con 0x7f03cc0fd0a0 2024-04-25T01:51:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.194+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f03cc04bed0 con 0x7f03cc0fd0a0 2024-04-25T01:51:18.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.194+0000 7f03c97fa700 1 -- 172.21.15.33:0/2940734748 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f03b805a380 con 0x7f03cc0fd0a0 2024-04-25T01:51:18.193 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:51:18.194 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:51:18.195 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:51:18.195 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:51:18.195 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:51:18.195 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f03bc06e140 msgr2=0x7f03bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f03bc06e140 0x7f03bc0705f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f03c0004f90 tx=0x7f03c0009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fd0a0 msgr2=0x7f03cc1162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fd0a0 0x7f03cc1162e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f03b8002a50 tx=0x7f03b80046b0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 shutdown_connections 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f03bc06e140 0x7f03bc0705f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03cc0fc790 0x7f03cc115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 --2- 172.21.15.33:0/2940734748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f03cc0fd0a0 0x7f03cc1162e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 >> 172.21.15.33:0/2940734748 conn(0x7f03cc071d00 msgr2=0x7f03cc0f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 shutdown_connections 2024-04-25T01:51:18.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.197+0000 7f03d2c7b700 1 -- 172.21.15.33:0/2940734748 wait complete. 2024-04-25T01:51:18.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.301+0000 7fced4556700 1 -- 172.21.15.33:0/3418154515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc1014d0 msgr2=0x7fcecc1018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.301+0000 7fced4556700 1 --2- 172.21.15.33:0/3418154515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc1014d0 0x7fcecc1018a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fcebc004940 tx=0x7fcebc009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.301+0000 7fced4556700 1 -- 172.21.15.33:0/3418154515 shutdown_connections 2024-04-25T01:51:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.301+0000 7fced4556700 1 --2- 172.21.15.33:0/3418154515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcecc0fb600 0x7fcecc0fba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.301+0000 7fced4556700 1 --2- 172.21.15.33:0/3418154515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc1014d0 0x7fcecc1018a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.301+0000 7fced4556700 1 -- 172.21.15.33:0/3418154515 >> 172.21.15.33:0/3418154515 conn(0x7fcecc071460 msgr2=0x7fcecc071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.301+0000 7fced4556700 1 -- 172.21.15.33:0/3418154515 shutdown_connections 2024-04-25T01:51:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.301+0000 7fced4556700 1 -- 172.21.15.33:0/3418154515 wait complete. 2024-04-25T01:51:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced4556700 1 Processor -- start 2024-04-25T01:51:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced4556700 1 -- start start 2024-04-25T01:51:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced4556700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc0fb600 0x7fcecc115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced4556700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcecc1014d0 0x7fcecc116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:18.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced4556700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc1168d0 con 0x7fcecc0fb600 2024-04-25T01:51:18.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced4556700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcecc10fca0 con 0x7fcecc1014d0 2024-04-25T01:51:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced22f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc0fb600 0x7fcecc115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced22f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc0fb600 0x7fcecc115d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51070/0 (socket says 172.21.15.33:51070) 2024-04-25T01:51:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced22f2700 1 -- 172.21.15.33:0/756271955 learned_addr learned my addr 172.21.15.33:0/756271955 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:18.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced1af1700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcecc1014d0 0x7fcecc116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced22f2700 1 -- 172.21.15.33:0/756271955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcecc1014d0 msgr2=0x7fcecc116280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced22f2700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcecc1014d0 0x7fcecc116280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.302+0000 7fced22f2700 1 -- 172.21.15.33:0/756271955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc005040 con 0x7fcecc0fb600 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.303+0000 7fced22f2700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc0fb600 0x7fcecc115d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00b3b0 tx=0x7fcebc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.303+0000 7fcec2ffd700 1 -- 172.21.15.33:0/756271955 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcebc00e040 con 0x7fcecc0fb600 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.303+0000 7fcec2ffd700 1 -- 172.21.15.33:0/756271955 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcebc013420 con 0x7fcecc0fb600 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.303+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcecc10ff20 con 0x7fcecc0fb600 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.303+0000 7fcec2ffd700 1 -- 172.21.15.33:0/756271955 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcebc01c630 con 0x7fcecc0fb600 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.303+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcecc1104e0 con 0x7fcecc0fb600 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.304+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcecc0fcbb0 con 0x7fcecc0fb600 2024-04-25T01:51:18.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.304+0000 7fcec2ffd700 1 -- 172.21.15.33:0/756271955 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcebc013590 con 0x7fcecc0fb600 2024-04-25T01:51:18.306 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:18 smithi033 ceph-mon[111306]: pgmap v356: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:18.306 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:18 smithi033 ceph-mon[111306]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:18.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.305+0000 7fcec2ffd700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fceb806e140 0x7fceb80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.305+0000 7fcec2ffd700 1 -- 172.21.15.33:0/756271955 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcebc08edc0 con 0x7fcecc0fb600 2024-04-25T01:51:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.305+0000 7fced1af1700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fceb806e140 0x7fceb80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:18.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.305+0000 7fced1af1700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fceb806e140 0x7fceb80705f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcecc1115e0 tx=0x7fcec800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:18.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.310+0000 7fcec2ffd700 1 -- 172.21.15.33:0/756271955 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcebc092030 con 0x7fcecc0fb600 2024-04-25T01:51:18.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:18 smithi138 ceph-mon[81304]: pgmap v356: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:18.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:18 smithi138 ceph-mon[81304]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.563+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcecc04bed0 con 0x7fcecc0fb600 2024-04-25T01:51:18.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.563+0000 7fcec2ffd700 1 -- 172.21.15.33:0/756271955 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fcebc05bc50 con 0x7fcecc0fb600 2024-04-25T01:51:18.562 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:51:18.562 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:51:18.563 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:51:18.564 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:51:18.565 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:51:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fceb806e140 msgr2=0x7fceb80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fceb806e140 0x7fceb80705f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcecc1115e0 tx=0x7fcec800b040 comp rx=0 tx=0).stop 2024-04-25T01:51:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc0fb600 msgr2=0x7fcecc115d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc0fb600 0x7fcecc115d40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00b3b0 tx=0x7fcebc009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 shutdown_connections 2024-04-25T01:51:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fceb806e140 0x7fceb80705f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcecc0fb600 0x7fcecc115d40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 --2- 172.21.15.33:0/756271955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcecc1014d0 0x7fcecc116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 >> 172.21.15.33:0/756271955 conn(0x7fcecc071460 msgr2=0x7fcecc0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:18.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 shutdown_connections 2024-04-25T01:51:18.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.565+0000 7fced4556700 1 -- 172.21.15.33:0/756271955 wait complete. 2024-04-25T01:51:18.567 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:51:18.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.671+0000 7f458e72d700 1 -- 172.21.15.33:0/3374558335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45880fc630 msgr2=0x7f45880fcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.671+0000 7f458e72d700 1 --2- 172.21.15.33:0/3374558335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45880fc630 0x7f45880fcaa0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f457c004940 tx=0x7f457c009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:18.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.671+0000 7f458e72d700 1 -- 172.21.15.33:0/3374558335 shutdown_connections 2024-04-25T01:51:18.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.671+0000 7f458e72d700 1 --2- 172.21.15.33:0/3374558335 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45880fc630 0x7f45880fcaa0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.671+0000 7f458e72d700 1 --2- 172.21.15.33:0/3374558335 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4588101e10 0x7f45881021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.671+0000 7f458e72d700 1 -- 172.21.15.33:0/3374558335 >> 172.21.15.33:0/3374558335 conn(0x7f45880714d0 msgr2=0x7f45880718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.671+0000 7f458e72d700 1 -- 172.21.15.33:0/3374558335 shutdown_connections 2024-04-25T01:51:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.671+0000 7f458e72d700 1 -- 172.21.15.33:0/3374558335 wait complete. 2024-04-25T01:51:18.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.671+0000 7f458e72d700 1 Processor -- start 2024-04-25T01:51:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f458e72d700 1 -- start start 2024-04-25T01:51:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f458e72d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45880fc630 0x7f4588113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:18.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f458e72d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588101e10 0x7f4588113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f458e72d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4588114520 con 0x7f4588101e10 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f458e72d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458810db20 con 0x7f45880fc630 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f45877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588101e10 0x7f4588113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f45877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588101e10 0x7f4588113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51086/0 (socket says 172.21.15.33:51086) 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f45877fe700 1 -- 172.21.15.33:0/3065654572 learned_addr learned my addr 172.21.15.33:0/3065654572 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f4587fff700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45880fc630 0x7f4588113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f45877fe700 1 -- 172.21.15.33:0/3065654572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45880fc630 msgr2=0x7f4588113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f45877fe700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45880fc630 0x7f4588113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.672+0000 7f45877fe700 1 -- 172.21.15.33:0/3065654572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c005040 con 0x7f4588101e10 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.673+0000 7f45877fe700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588101e10 0x7f4588113ed0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b8b0 tx=0x7f457c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.673+0000 7f4584ff9700 1 -- 172.21.15.33:0/3065654572 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457c00e040 con 0x7f4588101e10 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.673+0000 7f4584ff9700 1 -- 172.21.15.33:0/3065654572 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f457c004290 con 0x7f4588101e10 2024-04-25T01:51:18.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.673+0000 7f4584ff9700 1 -- 172.21.15.33:0/3065654572 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457c01c3f0 con 0x7f4588101e10 2024-04-25T01:51:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.673+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458810dda0 con 0x7f4588101e10 2024-04-25T01:51:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.673+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458810e2e0 con 0x7f4588101e10 2024-04-25T01:51:18.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.674+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4588100140 con 0x7f4588101e10 2024-04-25T01:51:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.675+0000 7f4584ff9700 1 -- 172.21.15.33:0/3065654572 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f457c013420 con 0x7f4588101e10 2024-04-25T01:51:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.675+0000 7f4584ff9700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f457006e140 0x7f45700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.675+0000 7f4584ff9700 1 -- 172.21.15.33:0/3065654572 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f457c08ed20 con 0x7f4588101e10 2024-04-25T01:51:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.675+0000 7f4587fff700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f457006e140 0x7f45700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:18.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.675+0000 7f4587fff700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f457006e140 0x7f45700705f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4578009fd0 tx=0x7f457800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:18.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.680+0000 7f4584ff9700 1 -- 172.21.15.33:0/3065654572 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f457c05bc50 con 0x7f4588101e10 2024-04-25T01:51:18.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.907+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f458810efe0 con 0x7f457006e140 2024-04-25T01:51:18.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.908+0000 7f4584ff9700 1 -- 172.21.15.33:0/3065654572 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f458810efe0 con 0x7f457006e140 2024-04-25T01:51:18.907 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:51:18.908 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:51:18.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f457006e140 msgr2=0x7f45700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f457006e140 0x7f45700705f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4578009fd0 tx=0x7f457800b040 comp rx=0 tx=0).stop 2024-04-25T01:51:18.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588101e10 msgr2=0x7f4588113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:18.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588101e10 0x7f4588113ed0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f457c00b8b0 tx=0x7f457c009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 shutdown_connections 2024-04-25T01:51:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f457006e140 0x7f45700705f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45880fc630 0x7f4588113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 --2- 172.21.15.33:0/3065654572 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4588101e10 0x7f4588113ed0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 >> 172.21.15.33:0/3065654572 conn(0x7f45880714d0 msgr2=0x7f45881052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 shutdown_connections 2024-04-25T01:51:18.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:18.911+0000 7f458e72d700 1 -- 172.21.15.33:0/3065654572 wait complete. 2024-04-25T01:51:19.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.016+0000 7fdb67b58700 1 -- 172.21.15.33:0/840314671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 msgr2=0x7fdb601020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:19.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.016+0000 7fdb67b58700 1 --2- 172.21.15.33:0/840314671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 0x7fdb601020a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009ce0 tx=0x7fdb50004d10 comp rx=0 tx=0).stop 2024-04-25T01:51:19.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.016+0000 7fdb67b58700 1 -- 172.21.15.33:0/840314671 shutdown_connections 2024-04-25T01:51:19.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.016+0000 7fdb67b58700 1 --2- 172.21.15.33:0/840314671 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb60102670 0x7fdb600fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:19.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.016+0000 7fdb67b58700 1 --2- 172.21.15.33:0/840314671 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 0x7fdb601020a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:19.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.016+0000 7fdb67b58700 1 -- 172.21.15.33:0/840314671 >> 172.21.15.33:0/840314671 conn(0x7fdb60071460 msgr2=0x7fdb60071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:19.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.016+0000 7fdb67b58700 1 -- 172.21.15.33:0/840314671 shutdown_connections 2024-04-25T01:51:19.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.016+0000 7fdb67b58700 1 -- 172.21.15.33:0/840314671 wait complete. 2024-04-25T01:51:19.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb67b58700 1 Processor -- start 2024-04-25T01:51:19.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb67b58700 1 -- start start 2024-04-25T01:51:19.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb67b58700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 0x7fdb600fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:19.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb67b58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb60102670 0x7fdb600fcd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:19.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb67b58700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600fd2b0 con 0x7fdb60101cd0 2024-04-25T01:51:19.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb67b58700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb600fd3f0 con 0x7fdb60102670 2024-04-25T01:51:19.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb658f4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 0x7fdb600fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb658f4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 0x7fdb600fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51106/0 (socket says 172.21.15.33:51106) 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb658f4700 1 -- 172.21.15.33:0/2036118878 learned_addr learned my addr 172.21.15.33:0/2036118878 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb650f3700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb60102670 0x7fdb600fcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb658f4700 1 -- 172.21.15.33:0/2036118878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb60102670 msgr2=0x7fdb600fcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb658f4700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb60102670 0x7fdb600fcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb658f4700 1 -- 172.21.15.33:0/2036118878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb50009940 con 0x7fdb60101cd0 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.017+0000 7fdb658f4700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 0x7fdb600fc7e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000be90 tx=0x7fdb50004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.018+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/2036118878 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5000e040 con 0x7fdb60101cd0 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.018+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/2036118878 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb50004610 con 0x7fdb60101cd0 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.018+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb600fd640 con 0x7fdb60101cd0 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.018+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/2036118878 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5001c6d0 con 0x7fdb60101cd0 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.018+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6011a430 con 0x7fdb60101cd0 2024-04-25T01:51:19.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.019+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb601052e0 con 0x7fdb60101cd0 2024-04-25T01:51:19.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.019+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/2036118878 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdb50012b00 con 0x7fdb60101cd0 2024-04-25T01:51:19.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.020+0000 7fdb5e7fc700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdb4806e140 0x7fdb480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:19.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.020+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/2036118878 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdb5008f620 con 0x7fdb60101cd0 2024-04-25T01:51:19.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.020+0000 7fdb650f3700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdb4806e140 0x7fdb480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:19.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.020+0000 7fdb650f3700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdb4806e140 0x7fdb480705f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdb600fe120 tx=0x7fdb54009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:19.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.025+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/2036118878 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdb500589a0 con 0x7fdb60101cd0 2024-04-25T01:51:19.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:19 smithi033 ceph-mon[111306]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:19.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:19 smithi033 ceph-mon[111306]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:19.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:19 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2940734748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:51:19.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:19 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/756271955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:51:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.320+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdb6004bed0 con 0x7fdb60101cd0 2024-04-25T01:51:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.320+0000 7fdb5e7fc700 1 -- 172.21.15.33:0/2036118878 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fdb50004e10 con 0x7fdb60101cd0 2024-04-25T01:51:19.319 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:51:19.319 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:51:19.319 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:51:19.319 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:51:19.320 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:51:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdb4806e140 msgr2=0x7fdb480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdb4806e140 0x7fdb480705f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdb600fe120 tx=0x7fdb54009040 comp rx=0 tx=0).stop 2024-04-25T01:51:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 msgr2=0x7fdb600fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 0x7fdb600fc7e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000be90 tx=0x7fdb50004d10 comp rx=0 tx=0).stop 2024-04-25T01:51:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 shutdown_connections 2024-04-25T01:51:19.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdb4806e140 0x7fdb480705f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdb60101cd0 0x7fdb600fc7e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 --2- 172.21.15.33:0/2036118878 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdb60102670 0x7fdb600fcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 >> 172.21.15.33:0/2036118878 conn(0x7fdb60071460 msgr2=0x7fdb600f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 shutdown_connections 2024-04-25T01:51:19.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:19.322+0000 7fdb67b58700 1 -- 172.21.15.33:0/2036118878 wait complete. 2024-04-25T01:51:19.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:19 smithi138 ceph-mon[81304]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:19.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:19 smithi138 ceph-mon[81304]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:19.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:19 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2940734748' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:51:19.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:19 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/756271955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:51:20.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:20 smithi033 ceph-mon[111306]: pgmap v357: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:20.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:20 smithi033 ceph-mon[111306]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:20.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:20 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2036118878' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:51:20.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:20 smithi138 ceph-mon[81304]: pgmap v357: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:20.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:20 smithi138 ceph-mon[81304]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:20.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:20 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2036118878' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:51:22.514 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:22 smithi033 ceph-mon[111306]: pgmap v358: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:22.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:22 smithi138 ceph-mon[81304]: pgmap v358: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:24.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:24 smithi033 ceph-mon[111306]: pgmap v359: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:24.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:24 smithi138 ceph-mon[81304]: pgmap v359: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:26.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:26 smithi033 ceph-mon[111306]: pgmap v360: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:26.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:51:26.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:26 smithi138 ceph-mon[81304]: pgmap v360: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:26.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:51:28.341 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:28 smithi138 ceph-mon[81304]: pgmap v361: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:28.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:28 smithi033 ceph-mon[111306]: pgmap v361: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:30.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:30 smithi033 ceph-mon[111306]: pgmap v362: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:30.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:30 smithi138 ceph-mon[81304]: pgmap v362: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:32.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:32 smithi033 ceph-mon[111306]: pgmap v363: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:32.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:32 smithi138 ceph-mon[81304]: pgmap v363: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:34.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:34 smithi033 ceph-mon[111306]: pgmap v364: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:34.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:34 smithi138 ceph-mon[81304]: pgmap v364: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:36.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:36 smithi033 ceph-mon[111306]: pgmap v365: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:36.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:36 smithi138 ceph-mon[81304]: pgmap v365: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:38.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:38 smithi033 ceph-mon[111306]: pgmap v366: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:38.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:38 smithi138 ceph-mon[81304]: pgmap v366: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:40.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:40 smithi033 ceph-mon[111306]: pgmap v367: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:40.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:40 smithi138 ceph-mon[81304]: pgmap v367: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:41.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:51:41.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:51:42.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:42 smithi033 ceph-mon[111306]: pgmap v368: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:42.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:42 smithi138 ceph-mon[81304]: pgmap v368: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:44.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:44 smithi033 ceph-mon[111306]: pgmap v369: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:44.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:44 smithi138 ceph-mon[81304]: pgmap v369: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:46.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:46 smithi033 ceph-mon[111306]: pgmap v370: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:46.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:46 smithi138 ceph-mon[81304]: pgmap v370: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:48.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:48 smithi033 ceph-mon[111306]: pgmap v371: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:48.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:48 smithi138 ceph-mon[81304]: pgmap v371: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.441+0000 7f53f0c0d700 1 -- 172.21.15.33:0/3362157375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec102e50 msgr2=0x7f53ec103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.441+0000 7f53f0c0d700 1 --2- 172.21.15.33:0/3362157375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec102e50 0x7f53ec103220 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f53dc004940 tx=0x7f53dc009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.441+0000 7f53f0c0d700 1 -- 172.21.15.33:0/3362157375 shutdown_connections 2024-04-25T01:51:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.441+0000 7f53f0c0d700 1 --2- 172.21.15.33:0/3362157375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53ec065640 0x7f53ec065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.441+0000 7f53f0c0d700 1 --2- 172.21.15.33:0/3362157375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec102e50 0x7f53ec103220 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:49.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.441+0000 7f53f0c0d700 1 -- 172.21.15.33:0/3362157375 >> 172.21.15.33:0/3362157375 conn(0x7f53ec071d00 msgr2=0x7f53ec072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.441+0000 7f53f0c0d700 1 -- 172.21.15.33:0/3362157375 shutdown_connections 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.441+0000 7f53f0c0d700 1 -- 172.21.15.33:0/3362157375 wait complete. 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53f0c0d700 1 Processor -- start 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53f0c0d700 1 -- start start 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53f0c0d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec065640 0x7f53ec113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53f0c0d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53ec102e50 0x7f53ec1140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53f0c0d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec114720 con 0x7f53ec065640 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53f0c0d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec10db10 con 0x7f53ec102e50 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53ea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec065640 0x7f53ec113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53ea59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec065640 0x7f53ec113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40978/0 (socket says 172.21.15.33:40978) 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53ea59c700 1 -- 172.21.15.33:0/634846880 learned_addr learned my addr 172.21.15.33:0/634846880 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:49.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.442+0000 7f53e9d9b700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53ec102e50 0x7f53ec1140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.443+0000 7f53ea59c700 1 -- 172.21.15.33:0/634846880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53ec102e50 msgr2=0x7f53ec1140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.443+0000 7f53ea59c700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53ec102e50 0x7f53ec1140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.443+0000 7f53ea59c700 1 -- 172.21.15.33:0/634846880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc005040 con 0x7f53ec065640 2024-04-25T01:51:49.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.443+0000 7f53ea59c700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec065640 0x7f53ec113b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00b3b0 tx=0x7f53dc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:49.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.443+0000 7f53daffd700 1 -- 172.21.15.33:0/634846880 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53dc00e040 con 0x7f53ec065640 2024-04-25T01:51:49.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.443+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec10dd90 con 0x7f53ec065640 2024-04-25T01:51:49.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.443+0000 7f53daffd700 1 -- 172.21.15.33:0/634846880 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53dc00bbc0 con 0x7f53ec065640 2024-04-25T01:51:49.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.443+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec10e350 con 0x7f53ec065640 2024-04-25T01:51:49.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.443+0000 7f53daffd700 1 -- 172.21.15.33:0/634846880 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53dc01c3f0 con 0x7f53ec065640 2024-04-25T01:51:49.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.444+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ec100100 con 0x7f53ec065640 2024-04-25T01:51:49.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.444+0000 7f53daffd700 1 -- 172.21.15.33:0/634846880 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f53dc00bd30 con 0x7f53ec065640 2024-04-25T01:51:49.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.445+0000 7f53daffd700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53d406e140 0x7f53d40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:49.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.445+0000 7f53daffd700 1 -- 172.21.15.33:0/634846880 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f53dc08ecd0 con 0x7f53ec065640 2024-04-25T01:51:49.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.445+0000 7f53e9d9b700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53d406e140 0x7f53d40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:49.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.446+0000 7f53e9d9b700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53d406e140 0x7f53d40705f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009fd0 tx=0x7f53e0009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:49.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.450+0000 7f53daffd700 1 -- 172.21.15.33:0/634846880 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f53dc05bc00 con 0x7f53ec065640 2024-04-25T01:51:49.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.675+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53ec10f1e0 con 0x7f53d406e140 2024-04-25T01:51:49.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.677+0000 7f53daffd700 1 -- 172.21.15.33:0/634846880 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f53ec10f1e0 con 0x7f53d406e140 2024-04-25T01:51:49.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53d406e140 msgr2=0x7f53d40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:49.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53d406e140 0x7f53d40705f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009fd0 tx=0x7f53e0009f20 comp rx=0 tx=0).stop 2024-04-25T01:51:49.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec065640 msgr2=0x7f53ec113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:49.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec065640 0x7f53ec113b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00b3b0 tx=0x7f53dc009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:49.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 shutdown_connections 2024-04-25T01:51:49.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f53d406e140 0x7f53d40705f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:49.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f53ec065640 0x7f53ec113b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:49.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 --2- 172.21.15.33:0/634846880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f53ec102e50 0x7f53ec1140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:49.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 >> 172.21.15.33:0/634846880 conn(0x7f53ec071d00 msgr2=0x7f53ec0fb010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:49.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 shutdown_connections 2024-04-25T01:51:49.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.679+0000 7f53f0c0d700 1 -- 172.21.15.33:0/634846880 wait complete. 2024-04-25T01:51:49.689 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:51:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.784+0000 7f44a4f64700 1 -- 172.21.15.33:0/3432875910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a00fbd40 msgr2=0x7f44a00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.784+0000 7f44a4f64700 1 --2- 172.21.15.33:0/3432875910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a00fbd40 0x7f44a00fc1b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4488004940 tx=0x7f4488009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.784+0000 7f44a4f64700 1 -- 172.21.15.33:0/3432875910 shutdown_connections 2024-04-25T01:51:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.784+0000 7f44a4f64700 1 --2- 172.21.15.33:0/3432875910 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a00fbd40 0x7f44a00fc1b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.784+0000 7f44a4f64700 1 --2- 172.21.15.33:0/3432875910 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a0101d90 0x7f44a0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:49.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.784+0000 7f44a4f64700 1 -- 172.21.15.33:0/3432875910 >> 172.21.15.33:0/3432875910 conn(0x7f44a0071d00 msgr2=0x7f44a0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.784+0000 7f44a4f64700 1 -- 172.21.15.33:0/3432875910 shutdown_connections 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.784+0000 7f44a4f64700 1 -- 172.21.15.33:0/3432875910 wait complete. 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f44a4f64700 1 Processor -- start 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f44a4f64700 1 -- start start 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f44a4f64700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a00fbd40 0x7f44a010c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f44a4f64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0101d90 0x7f44a010ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f44a4f64700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a010d3c0 con 0x7f44a0101d90 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f44a4f64700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a010d530 con 0x7f44a00fbd40 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f449dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0101d90 0x7f44a010ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f449dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0101d90 0x7f44a010ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40998/0 (socket says 172.21.15.33:40998) 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f449dd9b700 1 -- 172.21.15.33:0/3644361473 learned_addr learned my addr 172.21.15.33:0/3644361473 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f449e59c700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a00fbd40 0x7f44a010c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f449dd9b700 1 -- 172.21.15.33:0/3644361473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a00fbd40 msgr2=0x7f44a010c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:49.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f449dd9b700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a00fbd40 0x7f44a010c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.785+0000 7f449dd9b700 1 -- 172.21.15.33:0/3644361473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4488005040 con 0x7f44a0101d90 2024-04-25T01:51:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.786+0000 7f449dd9b700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0101d90 0x7f44a010ce80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4488009180 tx=0x7f4488009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.786+0000 7f4496ffd700 1 -- 172.21.15.33:0/3644361473 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f448800e040 con 0x7f44a0101d90 2024-04-25T01:51:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.786+0000 7f4496ffd700 1 -- 172.21.15.33:0/3644361473 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f448800bbc0 con 0x7f44a0101d90 2024-04-25T01:51:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.786+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a010d7b0 con 0x7f44a0101d90 2024-04-25T01:51:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.786+0000 7f4496ffd700 1 -- 172.21.15.33:0/3644361473 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f448801c3f0 con 0x7f44a0101d90 2024-04-25T01:51:49.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.786+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a011a330 con 0x7f44a0101d90 2024-04-25T01:51:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.787+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a00fd430 con 0x7f44a0101d90 2024-04-25T01:51:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.787+0000 7f4496ffd700 1 -- 172.21.15.33:0/3644361473 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f448800bd30 con 0x7f44a0101d90 2024-04-25T01:51:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.788+0000 7f4496ffd700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f448c06e150 0x7f448c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.788+0000 7f4496ffd700 1 -- 172.21.15.33:0/3644361473 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f448808eb40 con 0x7f44a0101d90 2024-04-25T01:51:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.788+0000 7f449e59c700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f448c06e150 0x7f448c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:49.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.788+0000 7f449e59c700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f448c06e150 0x7f448c070600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4490004fd0 tx=0x7f4490009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:49.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:49.793+0000 7f4496ffd700 1 -- 172.21.15.33:0/3644361473 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f448805ba70 con 0x7f44a0101d90 2024-04-25T01:51:50.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.019+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44a010d7b0 con 0x7f448c06e150 2024-04-25T01:51:50.019 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.020+0000 7f4496ffd700 1 -- 172.21.15.33:0/3644361473 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f44a010d7b0 con 0x7f448c06e150 2024-04-25T01:51:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f448c06e150 msgr2=0x7f448c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f448c06e150 0x7f448c070600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4490004fd0 tx=0x7f4490009040 comp rx=0 tx=0).stop 2024-04-25T01:51:50.021 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0101d90 msgr2=0x7f44a010ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0101d90 0x7f44a010ce80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4488009180 tx=0x7f4488009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 shutdown_connections 2024-04-25T01:51:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f448c06e150 0x7f448c070600 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44a00fbd40 0x7f44a010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 --2- 172.21.15.33:0/3644361473 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44a0101d90 0x7f44a010ce80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 >> 172.21.15.33:0/3644361473 conn(0x7f44a0071d00 msgr2=0x7f44a00fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 shutdown_connections 2024-04-25T01:51:50.022 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.022+0000 7f44a4f64700 1 -- 172.21.15.33:0/3644361473 wait complete. 2024-04-25T01:51:50.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.127+0000 7fd6de29a700 1 -- 172.21.15.33:0/18739773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 msgr2=0x7fd6d8103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.127+0000 7fd6de29a700 1 --2- 172.21.15.33:0/18739773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 0x7fd6d8103330 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc004940 tx=0x7fd6cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.127+0000 7fd6de29a700 1 -- 172.21.15.33:0/18739773 shutdown_connections 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.127+0000 7fd6de29a700 1 --2- 172.21.15.33:0/18739773 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80fb520 0x7fd6d80fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.127+0000 7fd6de29a700 1 --2- 172.21.15.33:0/18739773 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 0x7fd6d8103330 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.127+0000 7fd6de29a700 1 -- 172.21.15.33:0/18739773 >> 172.21.15.33:0/18739773 conn(0x7fd6d80714d0 msgr2=0x7fd6d80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.128+0000 7fd6de29a700 1 -- 172.21.15.33:0/18739773 shutdown_connections 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.128+0000 7fd6de29a700 1 -- 172.21.15.33:0/18739773 wait complete. 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.128+0000 7fd6de29a700 1 Processor -- start 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.128+0000 7fd6de29a700 1 -- start start 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.128+0000 7fd6de29a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80fb520 0x7fd6d81139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.128+0000 7fd6de29a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 0x7fd6d8113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.128+0000 7fd6de29a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d8114530 con 0x7fd6d8102f60 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.128+0000 7fd6de29a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d810db30 con 0x7fd6d80fb520 2024-04-25T01:51:50.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 0x7fd6d8113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 0x7fd6d8113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41020/0 (socket says 172.21.15.33:41020) 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d77fe700 1 -- 172.21.15.33:0/987673828 learned_addr learned my addr 172.21.15.33:0/987673828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80fb520 0x7fd6d81139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d77fe700 1 -- 172.21.15.33:0/987673828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80fb520 msgr2=0x7fd6d81139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d77fe700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80fb520 0x7fd6d81139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d77fe700 1 -- 172.21.15.33:0/987673828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc005040 con 0x7fd6d8102f60 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d77fe700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 0x7fd6d8113ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8002a50 tx=0x7fd6c80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/987673828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c8007cc0 con 0x7fd6d8102f60 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/987673828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6c800f040 con 0x7fd6d8102f60 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d810de10 con 0x7fd6d8102f60 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/987673828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c800da30 con 0x7fd6d8102f60 2024-04-25T01:51:50.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.129+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d810e350 con 0x7fd6d8102f60 2024-04-25T01:51:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.130+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d8100250 con 0x7fd6d8102f60 2024-04-25T01:51:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.131+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/987673828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd6c800db90 con 0x7fd6d8102f60 2024-04-25T01:51:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.131+0000 7fd6d4ff9700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd6c006e140 0x7fd6c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:50.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.131+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/987673828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd6c808d8c0 con 0x7fd6d8102f60 2024-04-25T01:51:50.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.131+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd6c006e140 0x7fd6c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:50.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.132+0000 7fd6d7fff700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd6c006e140 0x7fd6c00705f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc004f90 tx=0x7fd6cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:50.135 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.136+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/987673828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd6c805a770 con 0x7fd6d8102f60 2024-04-25T01:51:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.354+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6d8113670 con 0x7fd6c006e140 2024-04-25T01:51:50.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.365+0000 7fd6d4ff9700 1 -- 172.21.15.33:0/987673828 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd6d8113670 con 0x7fd6c006e140 2024-04-25T01:51:50.364 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:50 smithi033 ceph-mon[111306]: pgmap v372: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:50.364 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (12m) 6m ago 24m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (10m) 6m ago 24m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (10m) 5m ago 23m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (12m) 6m ago 23m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (6m) 6m ago 18m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (6m) 6m ago 18m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (5m) 5m ago 18m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (18m) 5m ago 18m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (14m) 6m ago 25m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (13m) 5m ago 23m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (11m) 6m ago 25m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (10m) 5m ago 23m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (13m) 6m ago 24m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (13m) 5m ago 23m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:51:50.365 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (9m) 6m ago 22m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:51:50.366 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (9m) 6m ago 21m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:51:50.366 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (8m) 6m ago 21m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:51:50.366 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (7m) 5m ago 21m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:51:50.366 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (7m) 5m ago 20m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:51:50.366 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (6m) 5m ago 20m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:51:50.366 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (13m) 6m ago 23m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:51:50.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd6c006e140 msgr2=0x7fd6c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd6c006e140 0x7fd6c00705f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc004f90 tx=0x7fd6cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 msgr2=0x7fd6d8113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 0x7fd6d8113ee0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8002a50 tx=0x7fd6c80046b0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 shutdown_connections 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd6c006e140 0x7fd6c00705f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6d80fb520 0x7fd6d81139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 --2- 172.21.15.33:0/987673828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd6d8102f60 0x7fd6d8113ee0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 >> 172.21.15.33:0/987673828 conn(0x7fd6d80714d0 msgr2=0x7fd6d80fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 shutdown_connections 2024-04-25T01:51:50.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.367+0000 7fd6de29a700 1 -- 172.21.15.33:0/987673828 wait complete. 2024-04-25T01:51:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.471+0000 7ff102ba2700 1 -- 172.21.15.33:0/333752317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 msgr2=0x7ff0fc0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.471+0000 7ff102ba2700 1 --2- 172.21.15.33:0/333752317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 0x7ff0fc0fc1b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8004940 tx=0x7ff0e8009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.471+0000 7ff102ba2700 1 -- 172.21.15.33:0/333752317 shutdown_connections 2024-04-25T01:51:50.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.471+0000 7ff102ba2700 1 --2- 172.21.15.33:0/333752317 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 0x7ff0fc0fc1b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.471+0000 7ff102ba2700 1 --2- 172.21.15.33:0/333752317 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc101d90 0x7ff0fc102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.471+0000 7ff102ba2700 1 -- 172.21.15.33:0/333752317 >> 172.21.15.33:0/333752317 conn(0x7ff0fc071d00 msgr2=0x7ff0fc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.471+0000 7ff102ba2700 1 -- 172.21.15.33:0/333752317 shutdown_connections 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.471+0000 7ff102ba2700 1 -- 172.21.15.33:0/333752317 wait complete. 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff102ba2700 1 Processor -- start 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff102ba2700 1 -- start start 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff102ba2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 0x7ff0fc102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff102ba2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc101d90 0x7ff0fc103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff102ba2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc1036a0 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff102ba2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc103810 con 0x7ff0fc101d90 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff10093e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 0x7ff0fc102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff10093e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 0x7ff0fc102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41036/0 (socket says 172.21.15.33:41036) 2024-04-25T01:51:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff10093e700 1 -- 172.21.15.33:0/423601206 learned_addr learned my addr 172.21.15.33:0/423601206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff0fbfff700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc101d90 0x7ff0fc103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.472+0000 7ff10093e700 1 -- 172.21.15.33:0/423601206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc101d90 msgr2=0x7ff0fc103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.473+0000 7ff10093e700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc101d90 0x7ff0fc103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.473+0000 7ff10093e700 1 -- 172.21.15.33:0/423601206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0e8005040 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.473+0000 7ff10093e700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 0x7ff0fc102b20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000b790 tx=0x7ff0f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.473+0000 7ff0f97fa700 1 -- 172.21.15.33:0/423601206 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0f000f6f0 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.473+0000 7ff0f97fa700 1 -- 172.21.15.33:0/423601206 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0f000fd20 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.473+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0fc119e80 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.473+0000 7ff0f97fa700 1 -- 172.21.15.33:0/423601206 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0f00103a0 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.473+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0fc11a390 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.474+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc0fd430 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.475+0000 7ff0f97fa700 1 -- 172.21.15.33:0/423601206 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff0f0011400 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.475+0000 7ff0f97fa700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0ec06e140 0x7ff0ec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.475+0000 7ff0f97fa700 1 -- 172.21.15.33:0/423601206 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff0f008d900 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.475+0000 7ff0fbfff700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0ec06e140 0x7ff0ec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:50.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.476+0000 7ff0fbfff700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0ec06e140 0x7ff0ec0705f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc104460 tx=0x7ff0e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:50.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.480+0000 7ff0f97fa700 1 -- 172.21.15.33:0/423601206 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff0f0056d00 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:50 smithi138 ceph-mon[81304]: pgmap v372: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:50.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.775+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff0fc04bf60 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.775+0000 7ff0f97fa700 1 -- 172.21.15.33:0/423601206 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff0f005a3c0 con 0x7ff0fc0fbd40 2024-04-25T01:51:50.775 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:51:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:51:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:51:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:51:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:51:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:51:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:51:50.775 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:51:50.776 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.777+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0ec06e140 msgr2=0x7ff0ec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.777+0000 7ff102ba2700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0ec06e140 0x7ff0ec0705f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc104460 tx=0x7ff0e8009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.777+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 msgr2=0x7ff0fc102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.777+0000 7ff102ba2700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 0x7ff0fc102b20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff0f000b790 tx=0x7ff0f0009250 comp rx=0 tx=0).stop 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.778+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 shutdown_connections 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.778+0000 7ff102ba2700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0ec06e140 0x7ff0ec0705f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.778+0000 7ff102ba2700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0fc0fbd40 0x7ff0fc102b20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.778+0000 7ff102ba2700 1 --2- 172.21.15.33:0/423601206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0fc101d90 0x7ff0fc103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.778+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 >> 172.21.15.33:0/423601206 conn(0x7ff0fc071d00 msgr2=0x7ff0fc065620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:50.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.778+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 shutdown_connections 2024-04-25T01:51:50.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.778+0000 7ff102ba2700 1 -- 172.21.15.33:0/423601206 wait complete. 2024-04-25T01:51:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.882+0000 7f14c6ac3700 1 -- 172.21.15.33:0/1376214743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0103780 msgr2=0x7f14c00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.882+0000 7f14c6ac3700 1 --2- 172.21.15.33:0/1376214743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0103780 0x7f14c00fcdf0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f14ac004940 tx=0x7f14ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.882+0000 7f14c6ac3700 1 -- 172.21.15.33:0/1376214743 shutdown_connections 2024-04-25T01:51:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.882+0000 7f14c6ac3700 1 --2- 172.21.15.33:0/1376214743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0103780 0x7f14c00fcdf0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.882+0000 7f14c6ac3700 1 --2- 172.21.15.33:0/1376214743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14c0102de0 0x7f14c01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.882+0000 7f14c6ac3700 1 -- 172.21.15.33:0/1376214743 >> 172.21.15.33:0/1376214743 conn(0x7f14c00f8990 msgr2=0x7f14c00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.882+0000 7f14c6ac3700 1 -- 172.21.15.33:0/1376214743 shutdown_connections 2024-04-25T01:51:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.882+0000 7f14c6ac3700 1 -- 172.21.15.33:0/1376214743 wait complete. 2024-04-25T01:51:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.883+0000 7f14c6ac3700 1 Processor -- start 2024-04-25T01:51:50.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.883+0000 7f14c6ac3700 1 -- start start 2024-04-25T01:51:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.883+0000 7f14c6ac3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0102de0 0x7f14c0070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.883+0000 7f14c6ac3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14c0103780 0x7f14c00707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.883+0000 7f14c6ac3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c00721b0 con 0x7f14c0102de0 2024-04-25T01:51:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.883+0000 7f14c6ac3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c0070ce0 con 0x7f14c0103780 2024-04-25T01:51:50.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.883+0000 7f14c485f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0102de0 0x7f14c0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14c485f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0102de0 0x7f14c0070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41064/0 (socket says 172.21.15.33:41064) 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14c485f700 1 -- 172.21.15.33:0/58527267 learned_addr learned my addr 172.21.15.33:0/58527267 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14bffff700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14c0103780 0x7f14c00707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14c485f700 1 -- 172.21.15.33:0/58527267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14c0103780 msgr2=0x7f14c00707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14c485f700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14c0103780 0x7f14c00707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14c485f700 1 -- 172.21.15.33:0/58527267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac005040 con 0x7f14c0102de0 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14c485f700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0102de0 0x7f14c0070260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f14b400c830 tx=0x7f14b4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14bd7fa700 1 -- 172.21.15.33:0/58527267 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14b400a760 con 0x7f14c0102de0 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14bd7fa700 1 -- 172.21.15.33:0/58527267 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14b400ad90 con 0x7f14c0102de0 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c0070fc0 con 0x7f14c0102de0 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14bd7fa700 1 -- 172.21.15.33:0/58527267 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14b40113a0 con 0x7f14c0102de0 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.884+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c011a140 con 0x7f14c0102de0 2024-04-25T01:51:50.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.885+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c00fe530 con 0x7f14c0102de0 2024-04-25T01:51:50.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.886+0000 7f14bd7fa700 1 -- 172.21.15.33:0/58527267 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f14b4012400 con 0x7f14c0102de0 2024-04-25T01:51:50.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.886+0000 7f14bd7fa700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f14b006e140 0x7f14b00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:50.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.886+0000 7f14bd7fa700 1 -- 172.21.15.33:0/58527267 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f14b408db40 con 0x7f14c0102de0 2024-04-25T01:51:50.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.886+0000 7f14bffff700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f14b006e140 0x7f14b00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:50.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.887+0000 7f14bffff700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f14b006e140 0x7f14b00705f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00b3b0 tx=0x7f14ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:50.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:50.891+0000 7f14bd7fa700 1 -- 172.21.15.33:0/58527267 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f14b4056f40 con 0x7f14c0102de0 2024-04-25T01:51:51.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.150+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f14c011a390 con 0x7f14c0102de0 2024-04-25T01:51:51.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.150+0000 7f14bd7fa700 1 -- 172.21.15.33:0/58527267 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f14b405a600 con 0x7f14c0102de0 2024-04-25T01:51:51.151 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:51:51.151 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:51:51.151 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:51:51.151 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:51:51.151 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:51:51.151 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:51:51.151 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:51:51.151 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:51:51.152 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:51:51.153 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:51:51.154 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:51:51.154 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:51:51.154 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:51:51.154 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:51:51.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.152+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f14b006e140 msgr2=0x7f14b00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.152+0000 7f14c6ac3700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f14b006e140 0x7f14b00705f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00b3b0 tx=0x7f14ac009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.152+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0102de0 msgr2=0x7f14c0070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.152+0000 7f14c6ac3700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0102de0 0x7f14c0070260 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f14b400c830 tx=0x7f14b4004470 comp rx=0 tx=0).stop 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.153+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 shutdown_connections 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.153+0000 7f14c6ac3700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f14b006e140 0x7f14b00705f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.153+0000 7f14c6ac3700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14c0102de0 0x7f14c0070260 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.153+0000 7f14c6ac3700 1 --2- 172.21.15.33:0/58527267 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14c0103780 0x7f14c00707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.153+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 >> 172.21.15.33:0/58527267 conn(0x7f14c00f8990 msgr2=0x7f14c00f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.153+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 shutdown_connections 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.153+0000 7f14c6ac3700 1 -- 172.21.15.33:0/58527267 wait complete. 2024-04-25T01:51:51.155 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:51:51.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 -- 172.21.15.33:0/1647150491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 msgr2=0x7f3c3c065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 --2- 172.21.15.33:0/1647150491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 0x7f3c3c065ab0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004940 tx=0x7f3c24009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 -- 172.21.15.33:0/1647150491 shutdown_connections 2024-04-25T01:51:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 --2- 172.21.15.33:0/1647150491 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 0x7f3c3c065ab0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 --2- 172.21.15.33:0/1647150491 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c3c102e50 0x7f3c3c103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 -- 172.21.15.33:0/1647150491 >> 172.21.15.33:0/1647150491 conn(0x7f3c3c071d00 msgr2=0x7f3c3c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 -- 172.21.15.33:0/1647150491 shutdown_connections 2024-04-25T01:51:51.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 -- 172.21.15.33:0/1647150491 wait complete. 2024-04-25T01:51:51.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 Processor -- start 2024-04-25T01:51:51.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.258+0000 7f3c4140c700 1 -- start start 2024-04-25T01:51:51.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c4140c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 0x7f3c3c113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:51.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c4140c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c3c102e50 0x7f3c3c1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:51.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c4140c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c114700 con 0x7f3c3c065640 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c4140c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3c10daf0 con 0x7f3c3c102e50 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 0x7f3c3c113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 0x7f3c3c113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41082/0 (socket says 172.21.15.33:41082) 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c3affd700 1 -- 172.21.15.33:0/4079593928 learned_addr learned my addr 172.21.15.33:0/4079593928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c3a7fc700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c3c102e50 0x7f3c3c1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c3affd700 1 -- 172.21.15.33:0/4079593928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c3c102e50 msgr2=0x7f3c3c1140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c3affd700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c3c102e50 0x7f3c3c1140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c3affd700 1 -- 172.21.15.33:0/4079593928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c24005040 con 0x7f3c3c065640 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c3affd700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 0x7f3c3c113b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400cb10 tx=0x7f3c34007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.259+0000 7f3c337fe700 1 -- 172.21.15.33:0/4079593928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c34004200 con 0x7f3c3c065640 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.260+0000 7f3c337fe700 1 -- 172.21.15.33:0/4079593928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c3400acc0 con 0x7f3c3c065640 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.260+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c3c10ddd0 con 0x7f3c3c065640 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.260+0000 7f3c337fe700 1 -- 172.21.15.33:0/4079593928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c34003d90 con 0x7f3c3c065640 2024-04-25T01:51:51.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.260+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3c10e390 con 0x7f3c3c065640 2024-04-25T01:51:51.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.260+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c3c100100 con 0x7f3c3c065640 2024-04-25T01:51:51.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.261+0000 7f3c337fe700 1 -- 172.21.15.33:0/4079593928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3c34003430 con 0x7f3c3c065640 2024-04-25T01:51:51.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.262+0000 7f3c337fe700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c2806e150 0x7f3c28070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:51.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.262+0000 7f3c337fe700 1 -- 172.21.15.33:0/4079593928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3c3408e120 con 0x7f3c3c065640 2024-04-25T01:51:51.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.262+0000 7f3c3a7fc700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c2806e150 0x7f3c28070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:51.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.262+0000 7f3c3a7fc700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c2806e150 0x7f3c28070600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c10ecb0 tx=0x7f3c24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:51.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.266+0000 7f3c337fe700 1 -- 172.21.15.33:0/4079593928 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3c3405afd0 con 0x7f3c3c065640 2024-04-25T01:51:51.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.492+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c3c10f230 con 0x7f3c2806e150 2024-04-25T01:51:51.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:51 smithi033 ceph-mon[111306]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:51.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:51 smithi033 ceph-mon[111306]: from='client.34740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:51.492 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:51 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/423601206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:51:51.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.494+0000 7f3c337fe700 1 -- 172.21.15.33:0/4079593928 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3c3c10f230 con 0x7f3c2806e150 2024-04-25T01:51:51.493 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:51:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:51:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:51:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:51:51.493 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:51:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:51:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:51:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:51:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:51:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:51:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:51:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:51:51.494 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:51:51.494 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:51:51.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c2806e150 msgr2=0x7f3c28070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c2806e150 0x7f3c28070600 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c10ecb0 tx=0x7f3c24009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 msgr2=0x7f3c3c113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 0x7f3c3c113b70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f3c3400cb10 tx=0x7f3c34007f90 comp rx=0 tx=0).stop 2024-04-25T01:51:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 shutdown_connections 2024-04-25T01:51:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c2806e150 0x7f3c28070600 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c3c065640 0x7f3c3c113b70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 --2- 172.21.15.33:0/4079593928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c3c102e50 0x7f3c3c1140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 >> 172.21.15.33:0/4079593928 conn(0x7f3c3c071d00 msgr2=0x7f3c3c0fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 shutdown_connections 2024-04-25T01:51:51.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.496+0000 7f3c4140c700 1 -- 172.21.15.33:0/4079593928 wait complete. 2024-04-25T01:51:51.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:51 smithi138 ceph-mon[81304]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:51.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:51 smithi138 ceph-mon[81304]: from='client.34740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:51.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:51 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/423601206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:51:51.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.601+0000 7efe287f9700 1 -- 172.21.15.33:0/1706837626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 msgr2=0x7efe201020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.601+0000 7efe287f9700 1 --2- 172.21.15.33:0/1706837626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 0x7efe201020b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efe1c004940 tx=0x7efe1c009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:51.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.601+0000 7efe287f9700 1 -- 172.21.15.33:0/1706837626 shutdown_connections 2024-04-25T01:51:51.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.601+0000 7efe287f9700 1 --2- 172.21.15.33:0/1706837626 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe20102680 0x7efe200fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.601+0000 7efe287f9700 1 --2- 172.21.15.33:0/1706837626 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 0x7efe201020b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.601+0000 7efe287f9700 1 -- 172.21.15.33:0/1706837626 >> 172.21.15.33:0/1706837626 conn(0x7efe200f7850 msgr2=0x7efe200f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.601+0000 7efe287f9700 1 -- 172.21.15.33:0/1706837626 shutdown_connections 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.601+0000 7efe287f9700 1 -- 172.21.15.33:0/1706837626 wait complete. 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe287f9700 1 Processor -- start 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe287f9700 1 -- start start 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe287f9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 0x7efe2010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe287f9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe20102680 0x7efe2010ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe287f9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2010d3b0 con 0x7efe20101ce0 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe287f9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2010d520 con 0x7efe20102680 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe26595700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 0x7efe2010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe26595700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 0x7efe2010c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41102/0 (socket says 172.21.15.33:41102) 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe26595700 1 -- 172.21.15.33:0/3757798133 learned_addr learned my addr 172.21.15.33:0/3757798133 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:51:51.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe25d94700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe20102680 0x7efe2010ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:51.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe26595700 1 -- 172.21.15.33:0/3757798133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe20102680 msgr2=0x7efe2010ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe26595700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe20102680 0x7efe2010ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.602+0000 7efe26595700 1 -- 172.21.15.33:0/3757798133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c005040 con 0x7efe20101ce0 2024-04-25T01:51:51.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.603+0000 7efe26595700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 0x7efe2010c930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b3b0 tx=0x7efe1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.603+0000 7efe16ffd700 1 -- 172.21.15.33:0/3757798133 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1c00e040 con 0x7efe20101ce0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.603+0000 7efe16ffd700 1 -- 172.21.15.33:0/3757798133 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe1c013420 con 0x7efe20101ce0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.603+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2010d7a0 con 0x7efe20101ce0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.603+0000 7efe16ffd700 1 -- 172.21.15.33:0/3757798133 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1c01c630 con 0x7efe20101ce0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.603+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2011a320 con 0x7efe20101ce0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.604+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe200fd430 con 0x7efe20101ce0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.604+0000 7efe16ffd700 1 -- 172.21.15.33:0/3757798133 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efe1c013590 con 0x7efe20101ce0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.605+0000 7efe16ffd700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe0c06e140 0x7efe0c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.605+0000 7efe16ffd700 1 -- 172.21.15.33:0/3757798133 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efe1c08eb90 con 0x7efe20101ce0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.605+0000 7efe25d94700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe0c06e140 0x7efe0c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:51:51.606 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.605+0000 7efe25d94700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe0c06e140 0x7efe0c0705f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe20066870 tx=0x7efe1000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:51:51.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.609+0000 7efe16ffd700 1 -- 172.21.15.33:0/3757798133 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efe1c05ba20 con 0x7efe20101ce0 2024-04-25T01:51:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.904+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efe2005f5c0 con 0x7efe20101ce0 2024-04-25T01:51:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.905+0000 7efe16ffd700 1 -- 172.21.15.33:0/3757798133 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7efe1c08e020 con 0x7efe20101ce0 2024-04-25T01:51:51.905 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:51:51.905 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:51:51.905 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:51:51.905 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:51:51.905 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:51:51.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe0c06e140 msgr2=0x7efe0c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe0c06e140 0x7efe0c0705f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe20066870 tx=0x7efe1000b040 comp rx=0 tx=0).stop 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 msgr2=0x7efe2010c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 0x7efe2010c930 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efe1c00b3b0 tx=0x7efe1c009e30 comp rx=0 tx=0).stop 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 shutdown_connections 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe0c06e140 0x7efe0c0705f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe20101ce0 0x7efe2010c930 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 --2- 172.21.15.33:0/3757798133 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe20102680 0x7efe2010ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 >> 172.21.15.33:0/3757798133 conn(0x7efe200f7850 msgr2=0x7efe200f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 shutdown_connections 2024-04-25T01:51:51.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:51:51.907+0000 7efe287f9700 1 -- 172.21.15.33:0/3757798133 wait complete. 2024-04-25T01:51:52.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:52 smithi033 ceph-mon[111306]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:52.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:52 smithi033 ceph-mon[111306]: pgmap v373: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:52.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:52 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/58527267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:51:52.265 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:52 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3757798133' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:51:52.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:52 smithi138 ceph-mon[81304]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:52.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:52 smithi138 ceph-mon[81304]: pgmap v373: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:52.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:52 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/58527267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:51:52.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:52 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3757798133' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:51:53.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:53 smithi033 ceph-mon[111306]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:53.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:53 smithi138 ceph-mon[81304]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:51:54.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:54 smithi033 ceph-mon[111306]: pgmap v374: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:54.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:54 smithi138 ceph-mon[81304]: pgmap v374: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:56.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:56 smithi033 ceph-mon[111306]: pgmap v375: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:56.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:51:56.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:56 smithi138 ceph-mon[81304]: pgmap v375: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:56.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:51:58.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:51:58 smithi033 ceph-mon[111306]: pgmap v376: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:51:58.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:51:58 smithi138 ceph-mon[81304]: pgmap v376: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:00.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:00 smithi033 ceph-mon[111306]: pgmap v377: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:00.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:00 smithi138 ceph-mon[81304]: pgmap v377: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:02.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:02 smithi033 ceph-mon[111306]: pgmap v378: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:02.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:02 smithi138 ceph-mon[81304]: pgmap v378: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:04.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:04 smithi033 ceph-mon[111306]: pgmap v379: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:04.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:04 smithi138 ceph-mon[81304]: pgmap v379: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:06.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:06 smithi033 ceph-mon[111306]: pgmap v380: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:06.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:06 smithi138 ceph-mon[81304]: pgmap v380: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:08.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:08 smithi033 ceph-mon[111306]: pgmap v381: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:08.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:08 smithi138 ceph-mon[81304]: pgmap v381: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:10.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:10 smithi033 ceph-mon[111306]: pgmap v382: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:10.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:10 smithi138 ceph-mon[81304]: pgmap v382: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:11.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:52:11.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:52:12.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:12 smithi033 ceph-mon[111306]: pgmap v383: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:12.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:12 smithi138 ceph-mon[81304]: pgmap v383: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:14.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:14 smithi033 ceph-mon[111306]: pgmap v384: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:14.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:14 smithi138 ceph-mon[81304]: pgmap v384: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:16.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:16 smithi033 ceph-mon[111306]: pgmap v385: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:16.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:16 smithi138 ceph-mon[81304]: pgmap v385: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:18.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:18 smithi033 ceph-mon[111306]: pgmap v386: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:18.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:18 smithi138 ceph-mon[81304]: pgmap v386: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:20.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:20 smithi033 ceph-mon[111306]: pgmap v387: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:20.540 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:20 smithi138 ceph-mon[81304]: pgmap v387: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:22.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.025+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/4043369136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0103780 msgr2=0x7fc2f00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.025+0000 7fc2f4ef4700 1 --2- 172.21.15.33:0/4043369136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0103780 0x7fc2f00fcdf0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8004940 tx=0x7fc2d8009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:22.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.025+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/4043369136 shutdown_connections 2024-04-25T01:52:22.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.025+0000 7fc2f4ef4700 1 --2- 172.21.15.33:0/4043369136 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0103780 0x7fc2f00fcdf0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.025+0000 7fc2f4ef4700 1 --2- 172.21.15.33:0/4043369136 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0102de0 0x7fc2f01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.025+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/4043369136 >> 172.21.15.33:0/4043369136 conn(0x7fc2f00f8950 msgr2=0x7fc2f00fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:22.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.025+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/4043369136 shutdown_connections 2024-04-25T01:52:22.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.025+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/4043369136 wait complete. 2024-04-25T01:52:22.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.026+0000 7fc2f4ef4700 1 Processor -- start 2024-04-25T01:52:22.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.026+0000 7fc2f4ef4700 1 -- start start 2024-04-25T01:52:22.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.026+0000 7fc2f4ef4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0102de0 0x7fc2f0113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.026+0000 7fc2f4ef4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0103780 0x7fc2f01140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.026+0000 7fc2f4ef4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0114700 con 0x7fc2f0102de0 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.026+0000 7fc2f4ef4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f010daf0 con 0x7fc2f0103780 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.026+0000 7fc2ee59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0102de0 0x7fc2f0113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.026+0000 7fc2ee59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0102de0 0x7fc2f0113b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53210/0 (socket says 172.21.15.33:53210) 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.026+0000 7fc2ee59c700 1 -- 172.21.15.33:0/1012529680 learned_addr learned my addr 172.21.15.33:0/1012529680 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2edd9b700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0103780 0x7fc2f01140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2ee59c700 1 -- 172.21.15.33:0/1012529680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0103780 msgr2=0x7fc2f01140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2ee59c700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0103780 0x7fc2f01140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2ee59c700 1 -- 172.21.15.33:0/1012529680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d8005040 con 0x7fc2f0102de0 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2ee59c700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0102de0 0x7fc2f0113b70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800cb10 tx=0x7fc2e8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2e6ffd700 1 -- 172.21.15.33:0/1012529680 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2e8004200 con 0x7fc2f0102de0 2024-04-25T01:52:22.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f010ddd0 con 0x7fc2f0102de0 2024-04-25T01:52:22.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2e6ffd700 1 -- 172.21.15.33:0/1012529680 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2e800acc0 con 0x7fc2f0102de0 2024-04-25T01:52:22.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2e6ffd700 1 -- 172.21.15.33:0/1012529680 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2e8003d90 con 0x7fc2f0102de0 2024-04-25T01:52:22.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.027+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f010e390 con 0x7fc2f0102de0 2024-04-25T01:52:22.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.028+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f00fe530 con 0x7fc2f0102de0 2024-04-25T01:52:22.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.029+0000 7fc2e6ffd700 1 -- 172.21.15.33:0/1012529680 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc2e8003430 con 0x7fc2f0102de0 2024-04-25T01:52:22.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.029+0000 7fc2e6ffd700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc2dc06e140 0x7fc2dc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:22.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.030+0000 7fc2e6ffd700 1 -- 172.21.15.33:0/1012529680 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc2e808e120 con 0x7fc2f0102de0 2024-04-25T01:52:22.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.030+0000 7fc2edd9b700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc2dc06e140 0x7fc2dc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:22.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.030+0000 7fc2edd9b700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc2dc06e140 0x7fc2dc0705f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc2f010ecb0 tx=0x7fc2d8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:22.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.034+0000 7fc2e6ffd700 1 -- 172.21.15.33:0/1012529680 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc2e805afd0 con 0x7fc2f0102de0 2024-04-25T01:52:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.265+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2f0109f30 con 0x7fc2dc06e140 2024-04-25T01:52:22.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.266+0000 7fc2e6ffd700 1 -- 172.21.15.33:0/1012529680 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc2f0109f30 con 0x7fc2dc06e140 2024-04-25T01:52:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc2dc06e140 msgr2=0x7fc2dc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc2dc06e140 0x7fc2dc0705f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc2f010ecb0 tx=0x7fc2d8009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0102de0 msgr2=0x7fc2f0113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0102de0 0x7fc2f0113b70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800cb10 tx=0x7fc2e8007f90 comp rx=0 tx=0).stop 2024-04-25T01:52:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 shutdown_connections 2024-04-25T01:52:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc2dc06e140 0x7fc2dc0705f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc2f0102de0 0x7fc2f0113b70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 --2- 172.21.15.33:0/1012529680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc2f0103780 0x7fc2f01140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 >> 172.21.15.33:0/1012529680 conn(0x7fc2f00f8950 msgr2=0x7fc2f0100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 shutdown_connections 2024-04-25T01:52:22.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.268+0000 7fc2f4ef4700 1 -- 172.21.15.33:0/1012529680 wait complete. 2024-04-25T01:52:22.279 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:52:22.279 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:22 smithi033 ceph-mon[111306]: pgmap v388: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:22.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 -- 172.21.15.33:0/514582121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c102e50 msgr2=0x7fc20c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 --2- 172.21.15.33:0/514582121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c102e50 0x7fc20c103220 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc200004940 tx=0x7fc200009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 -- 172.21.15.33:0/514582121 shutdown_connections 2024-04-25T01:52:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 --2- 172.21.15.33:0/514582121 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c065640 0x7fc20c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 --2- 172.21.15.33:0/514582121 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c102e50 0x7fc20c103220 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 -- 172.21.15.33:0/514582121 >> 172.21.15.33:0/514582121 conn(0x7fc20c071d00 msgr2=0x7fc20c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 -- 172.21.15.33:0/514582121 shutdown_connections 2024-04-25T01:52:22.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 -- 172.21.15.33:0/514582121 wait complete. 2024-04-25T01:52:22.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 Processor -- start 2024-04-25T01:52:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.376+0000 7fc211ab4700 1 -- start start 2024-04-25T01:52:22.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc211ab4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c065640 0x7fc20c113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc211ab4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c102e50 0x7fc20c1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc211ab4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c114710 con 0x7fc20c065640 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc211ab4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c10db00 con 0x7fc20c102e50 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc20b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c065640 0x7fc20c113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc20b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c065640 0x7fc20c113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53220/0 (socket says 172.21.15.33:53220) 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc20b7fe700 1 -- 172.21.15.33:0/1622497059 learned_addr learned my addr 172.21.15.33:0/1622497059 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc20affd700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c102e50 0x7fc20c1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc20b7fe700 1 -- 172.21.15.33:0/1622497059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c102e50 msgr2=0x7fc20c1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc20b7fe700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c102e50 0x7fc20c1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc20b7fe700 1 -- 172.21.15.33:0/1622497059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc200005040 con 0x7fc20c065640 2024-04-25T01:52:22.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc20b7fe700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c065640 0x7fc20c113b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc20000b3b0 tx=0x7fc200009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc1f3fff700 1 -- 172.21.15.33:0/1622497059 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc20000e040 con 0x7fc20c065640 2024-04-25T01:52:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc1f3fff700 1 -- 172.21.15.33:0/1622497059 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc200013420 con 0x7fc20c065640 2024-04-25T01:52:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20c10dd80 con 0x7fc20c065640 2024-04-25T01:52:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc1f3fff700 1 -- 172.21.15.33:0/1622497059 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc20001c630 con 0x7fc20c065640 2024-04-25T01:52:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.377+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20c10e340 con 0x7fc20c065640 2024-04-25T01:52:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.378+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20c100100 con 0x7fc20c065640 2024-04-25T01:52:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.379+0000 7fc1f3fff700 1 -- 172.21.15.33:0/1622497059 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc200013590 con 0x7fc20c065640 2024-04-25T01:52:22.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.379+0000 7fc1f3fff700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc1f406e140 0x7fc1f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.380+0000 7fc1f3fff700 1 -- 172.21.15.33:0/1622497059 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc20008ec20 con 0x7fc20c065640 2024-04-25T01:52:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.380+0000 7fc20affd700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc1f406e140 0x7fc1f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:22.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.380+0000 7fc20affd700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc1f406e140 0x7fc1f40705f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc20c10ecc0 tx=0x7fc1fc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:22.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.384+0000 7fc1f3fff700 1 -- 172.21.15.33:0/1622497059 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc20005bb50 con 0x7fc20c065640 2024-04-25T01:52:22.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:22 smithi138 ceph-mon[81304]: pgmap v388: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.611+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc20c10f1d0 con 0x7fc1f406e140 2024-04-25T01:52:22.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.612+0000 7fc1f3fff700 1 -- 172.21.15.33:0/1622497059 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc20c10f1d0 con 0x7fc1f406e140 2024-04-25T01:52:22.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.614+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc1f406e140 msgr2=0x7fc1f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.614+0000 7fc211ab4700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc1f406e140 0x7fc1f40705f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc20c10ecc0 tx=0x7fc1fc00b040 comp rx=0 tx=0).stop 2024-04-25T01:52:22.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.614+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c065640 msgr2=0x7fc20c113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.614+0000 7fc211ab4700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c065640 0x7fc20c113b80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc20000b3b0 tx=0x7fc200009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:22.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.614+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 shutdown_connections 2024-04-25T01:52:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.614+0000 7fc211ab4700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc1f406e140 0x7fc1f40705f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.614+0000 7fc211ab4700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc20c065640 0x7fc20c113b80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.614+0000 7fc211ab4700 1 --2- 172.21.15.33:0/1622497059 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc20c102e50 0x7fc20c1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.614+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 >> 172.21.15.33:0/1622497059 conn(0x7fc20c071d00 msgr2=0x7fc20c0fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.615+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 shutdown_connections 2024-04-25T01:52:22.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.615+0000 7fc211ab4700 1 -- 172.21.15.33:0/1622497059 wait complete. 2024-04-25T01:52:22.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.719+0000 7f8f9393d700 1 -- 172.21.15.33:0/455217020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fe7a0 msgr2=0x7f8f8c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.719+0000 7f8f9393d700 1 --2- 172.21.15.33:0/455217020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fe7a0 0x7f8f8c106e70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8f88004940 tx=0x7f8f88009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.719+0000 7f8f9393d700 1 -- 172.21.15.33:0/455217020 shutdown_connections 2024-04-25T01:52:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.719+0000 7f8f9393d700 1 --2- 172.21.15.33:0/455217020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fe7a0 0x7f8f8c106e70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.719+0000 7f8f9393d700 1 --2- 172.21.15.33:0/455217020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c0fde00 0x7f8f8c0fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.719+0000 7f8f9393d700 1 -- 172.21.15.33:0/455217020 >> 172.21.15.33:0/455217020 conn(0x7f8f8c0f7860 msgr2=0x7f8f8c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.719+0000 7f8f9393d700 1 -- 172.21.15.33:0/455217020 shutdown_connections 2024-04-25T01:52:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.719+0000 7f8f9393d700 1 -- 172.21.15.33:0/455217020 wait complete. 2024-04-25T01:52:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.720+0000 7f8f9393d700 1 Processor -- start 2024-04-25T01:52:22.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.720+0000 7f8f9393d700 1 -- start start 2024-04-25T01:52:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.720+0000 7f8f9393d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fde00 0x7f8f8c0fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.720+0000 7f8f9393d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c0fe7a0 0x7f8f8c0fc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.720+0000 7f8f9393d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0fdca0 con 0x7f8f8c0fde00 2024-04-25T01:52:22.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.720+0000 7f8f9393d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c0fc740 con 0x7f8f8c0fe7a0 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.720+0000 7f8f916d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fde00 0x7f8f8c0fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.720+0000 7f8f916d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fde00 0x7f8f8c0fbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53236/0 (socket says 172.21.15.33:53236) 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.720+0000 7f8f916d9700 1 -- 172.21.15.33:0/3090292374 learned_addr learned my addr 172.21.15.33:0/3090292374 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f90ed8700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c0fe7a0 0x7f8f8c0fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f916d9700 1 -- 172.21.15.33:0/3090292374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c0fe7a0 msgr2=0x7f8f8c0fc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f916d9700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c0fe7a0 0x7f8f8c0fc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f916d9700 1 -- 172.21.15.33:0/3090292374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f88005040 con 0x7f8f8c0fde00 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f916d9700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fde00 0x7f8f8c0fbcc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00b790 tx=0x7f8f7c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f81ffb700 1 -- 172.21.15.33:0/3090292374 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f7c00f770 con 0x7f8f8c0fde00 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f81ffb700 1 -- 172.21.15.33:0/3090292374 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f7c00fda0 con 0x7f8f8c0fde00 2024-04-25T01:52:22.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c0fca20 con 0x7f8f8c0fde00 2024-04-25T01:52:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f81ffb700 1 -- 172.21.15.33:0/3090292374 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f7c00d310 con 0x7f8f8c0fde00 2024-04-25T01:52:22.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.721+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c11a170 con 0x7f8f8c0fde00 2024-04-25T01:52:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.722+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8c066ad0 con 0x7f8f8c0fde00 2024-04-25T01:52:22.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.723+0000 7f8f81ffb700 1 -- 172.21.15.33:0/3090292374 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8f7c010420 con 0x7f8f8c0fde00 2024-04-25T01:52:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.723+0000 7f8f81ffb700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f7806e140 0x7f8f780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.723+0000 7f8f81ffb700 1 -- 172.21.15.33:0/3090292374 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8f7c08db50 con 0x7f8f8c0fde00 2024-04-25T01:52:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.723+0000 7f8f90ed8700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f7806e140 0x7f8f780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:22.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.724+0000 7f8f90ed8700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f7806e140 0x7f8f780705f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800b3b0 tx=0x7f8f88009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:22.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.728+0000 7f8f81ffb700 1 -- 172.21.15.33:0/3090292374 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8f7c056f50 con 0x7f8f8c0fde00 2024-04-25T01:52:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.947+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f8c0fd240 con 0x7f8f7806e140 2024-04-25T01:52:22.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.959+0000 7f8f81ffb700 1 -- 172.21.15.33:0/3090292374 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8f8c0fd240 con 0x7f8f7806e140 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (13m) 6m ago 24m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (10m) 6m ago 24m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (10m) 6m ago 23m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (13m) 6m ago 24m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (6m) 6m ago 19m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (7m) 6m ago 19m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (6m) 6m ago 19m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (19m) 6m ago 19m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (15m) 6m ago 26m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (14m) 6m ago 23m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:52:22.959 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (11m) 6m ago 26m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (11m) 6m ago 23m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (14m) 6m ago 24m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (14m) 6m ago 23m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (10m) 6m ago 23m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (9m) 6m ago 22m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (9m) 6m ago 22m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (8m) 6m ago 21m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (7m) 6m ago 21m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (7m) 6m ago 20m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:52:22.960 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (13m) 6m ago 24m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f7806e140 msgr2=0x7f8f780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f7806e140 0x7f8f780705f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800b3b0 tx=0x7f8f88009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fde00 msgr2=0x7f8f8c0fbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fde00 0x7f8f8c0fbcc0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00b790 tx=0x7f8f7c009250 comp rx=0 tx=0).stop 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 shutdown_connections 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f7806e140 0x7f8f780705f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f8c0fde00 0x7f8f8c0fbcc0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 --2- 172.21.15.33:0/3090292374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f8c0fe7a0 0x7f8f8c0fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 >> 172.21.15.33:0/3090292374 conn(0x7f8f8c0f7860 msgr2=0x7f8f8c1036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:22.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 shutdown_connections 2024-04-25T01:52:22.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:22.961+0000 7f8f9393d700 1 -- 172.21.15.33:0/3090292374 wait complete. 2024-04-25T01:52:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.064+0000 7fe589c8d700 1 -- 172.21.15.33:0/901609186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 msgr2=0x7fe5841031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.064+0000 7fe589c8d700 1 --2- 172.21.15.33:0/901609186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 0x7fe5841031b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe574004940 tx=0x7fe574009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.064+0000 7fe589c8d700 1 -- 172.21.15.33:0/901609186 shutdown_connections 2024-04-25T01:52:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.064+0000 7fe589c8d700 1 --2- 172.21.15.33:0/901609186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe584103780 0x7fe5840fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.064+0000 7fe589c8d700 1 --2- 172.21.15.33:0/901609186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 0x7fe5841031b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.064+0000 7fe589c8d700 1 -- 172.21.15.33:0/901609186 >> 172.21.15.33:0/901609186 conn(0x7fe5840f8990 msgr2=0x7fe5840fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.064+0000 7fe589c8d700 1 -- 172.21.15.33:0/901609186 shutdown_connections 2024-04-25T01:52:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.064+0000 7fe589c8d700 1 -- 172.21.15.33:0/901609186 wait complete. 2024-04-25T01:52:23.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.064+0000 7fe589c8d700 1 Processor -- start 2024-04-25T01:52:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe589c8d700 1 -- start start 2024-04-25T01:52:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe589c8d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 0x7fe584113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:23.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe589c8d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe584103780 0x7fe584113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe589c8d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe584114500 con 0x7fe584102de0 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe589c8d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58410db50 con 0x7fe584103780 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe5837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 0x7fe584113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe5837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 0x7fe584113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53260/0 (socket says 172.21.15.33:53260) 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe5837fe700 1 -- 172.21.15.33:0/1939300422 learned_addr learned my addr 172.21.15.33:0/1939300422 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe582ffd700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe584103780 0x7fe584113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe5837fe700 1 -- 172.21.15.33:0/1939300422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe584103780 msgr2=0x7fe584113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe5837fe700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe584103780 0x7fe584113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe5837fe700 1 -- 172.21.15.33:0/1939300422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe574005040 con 0x7fe584102de0 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.065+0000 7fe5837fe700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 0x7fe584113970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe57400b3b0 tx=0x7fe574009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.066+0000 7fe56bfff700 1 -- 172.21.15.33:0/1939300422 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe57400e040 con 0x7fe584102de0 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.066+0000 7fe56bfff700 1 -- 172.21.15.33:0/1939300422 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe574013420 con 0x7fe584102de0 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.066+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58410ddd0 con 0x7fe584102de0 2024-04-25T01:52:23.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.066+0000 7fe56bfff700 1 -- 172.21.15.33:0/1939300422 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe57401c630 con 0x7fe584102de0 2024-04-25T01:52:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.066+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe58410e390 con 0x7fe584102de0 2024-04-25T01:52:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.067+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5841000f0 con 0x7fe584102de0 2024-04-25T01:52:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.067+0000 7fe56bfff700 1 -- 172.21.15.33:0/1939300422 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe574013590 con 0x7fe584102de0 2024-04-25T01:52:23.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.068+0000 7fe56bfff700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe56c06e140 0x7fe56c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.068+0000 7fe56bfff700 1 -- 172.21.15.33:0/1939300422 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe57408ed80 con 0x7fe584102de0 2024-04-25T01:52:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.068+0000 7fe582ffd700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe56c06e140 0x7fe56c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:23.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.068+0000 7fe582ffd700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe56c06e140 0x7fe56c0705f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe58410ecc0 tx=0x7fe578009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:23.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.072+0000 7fe56bfff700 1 -- 172.21.15.33:0/1939300422 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe574092030 con 0x7fe584102de0 2024-04-25T01:52:23.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.371+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe58410efc0 con 0x7fe584102de0 2024-04-25T01:52:23.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.372+0000 7fe56bfff700 1 -- 172.21.15.33:0/1939300422 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fe57405bc50 con 0x7fe584102de0 2024-04-25T01:52:23.372 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:52:23.372 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:52:23.372 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:52:23.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:52:23.372 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:52:23.372 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:52:23.372 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:52:23.373 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:52:23.374 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:52:23.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe56c06e140 msgr2=0x7fe56c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe56c06e140 0x7fe56c0705f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe58410ecc0 tx=0x7fe578009040 comp rx=0 tx=0).stop 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 msgr2=0x7fe584113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 0x7fe584113970 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe57400b3b0 tx=0x7fe574009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 shutdown_connections 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe56c06e140 0x7fe56c0705f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe584102de0 0x7fe584113970 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 --2- 172.21.15.33:0/1939300422 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe584103780 0x7fe584113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 >> 172.21.15.33:0/1939300422 conn(0x7fe5840f8990 msgr2=0x7fe584107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 shutdown_connections 2024-04-25T01:52:23.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.374+0000 7fe589c8d700 1 -- 172.21.15.33:0/1939300422 wait complete. 2024-04-25T01:52:23.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.478+0000 7f9444047700 1 -- 172.21.15.33:0/3016298453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fe7b0 msgr2=0x7f943c106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.478+0000 7f9444047700 1 --2- 172.21.15.33:0/3016298453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fe7b0 0x7f943c106df0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f942c004940 tx=0x7f942c009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:23.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.478+0000 7f9444047700 1 -- 172.21.15.33:0/3016298453 shutdown_connections 2024-04-25T01:52:23.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.478+0000 7f9444047700 1 --2- 172.21.15.33:0/3016298453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fe7b0 0x7f943c106df0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.478+0000 7f9444047700 1 --2- 172.21.15.33:0/3016298453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f943c0fde10 0x7f943c0fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.478+0000 7f9444047700 1 -- 172.21.15.33:0/3016298453 >> 172.21.15.33:0/3016298453 conn(0x7f943c0f7860 msgr2=0x7f943c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:23.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.478+0000 7f9444047700 1 -- 172.21.15.33:0/3016298453 shutdown_connections 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.478+0000 7f9444047700 1 -- 172.21.15.33:0/3016298453 wait complete. 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f9444047700 1 Processor -- start 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f9444047700 1 -- start start 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f9444047700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fde10 0x7f943c115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f9444047700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f943c0fe7b0 0x7f943c1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f9444047700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c116710 con 0x7f943c0fde10 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f9444047700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c10fcf0 con 0x7f943c0fe7b0 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f9441de3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fde10 0x7f943c115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f9441de3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fde10 0x7f943c115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53286/0 (socket says 172.21.15.33:53286) 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f9441de3700 1 -- 172.21.15.33:0/1271830965 learned_addr learned my addr 172.21.15.33:0/1271830965 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:23.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.479+0000 7f94415e2700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f943c0fe7b0 0x7f943c1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:23.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.480+0000 7f9441de3700 1 -- 172.21.15.33:0/1271830965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f943c0fe7b0 msgr2=0x7f943c1160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.480+0000 7f9441de3700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f943c0fe7b0 0x7f943c1160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.480+0000 7f9441de3700 1 -- 172.21.15.33:0/1271830965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c005040 con 0x7f943c0fde10 2024-04-25T01:52:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.480+0000 7f9441de3700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fde10 0x7f943c115b80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f943800c830 tx=0x7f9438004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.480+0000 7f94327fc700 1 -- 172.21.15.33:0/1271830965 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94380106f0 con 0x7f943c0fde10 2024-04-25T01:52:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.480+0000 7f94327fc700 1 -- 172.21.15.33:0/1271830965 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9438010d20 con 0x7f943c0fde10 2024-04-25T01:52:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.480+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c10ffd0 con 0x7f943c0fde10 2024-04-25T01:52:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.480+0000 7f94327fc700 1 -- 172.21.15.33:0/1271830965 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94380113a0 con 0x7f943c0fde10 2024-04-25T01:52:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.480+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c110510 con 0x7f943c0fde10 2024-04-25T01:52:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.481+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c066b50 con 0x7f943c0fde10 2024-04-25T01:52:23.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.482+0000 7f94327fc700 1 -- 172.21.15.33:0/1271830965 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9438012400 con 0x7f943c0fde10 2024-04-25T01:52:23.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.482+0000 7f94327fc700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f942806e140 0x7f94280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:23.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.482+0000 7f94327fc700 1 -- 172.21.15.33:0/1271830965 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f943808d9b0 con 0x7f943c0fde10 2024-04-25T01:52:23.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.482+0000 7f94415e2700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f942806e140 0x7f94280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:23.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.483+0000 7f94415e2700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f942806e140 0x7f94280705f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f943c111420 tx=0x7f942c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:23.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.487+0000 7f94327fc700 1 -- 172.21.15.33:0/1271830965 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f943805a980 con 0x7f943c0fde10 2024-04-25T01:52:23.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.736+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f943c04c710 con 0x7f943c0fde10 2024-04-25T01:52:23.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.737+0000 7f94327fc700 1 -- 172.21.15.33:0/1271830965 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f943805a510 con 0x7f943c0fde10 2024-04-25T01:52:23.737 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:52:23.737 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:52:23.737 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:52:23.737 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:52:23.737 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:52:23.737 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:52:23.737 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:52:23.738 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:52:23.739 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:52:23.740 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:52:23.740 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:52:23.740 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:52:23.740 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:52:23.740 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:52:23.740 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:52:23.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f942806e140 msgr2=0x7f94280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f942806e140 0x7f94280705f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f943c111420 tx=0x7f942c009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fde10 msgr2=0x7f943c115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fde10 0x7f943c115b80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f943800c830 tx=0x7f9438004470 comp rx=0 tx=0).stop 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 shutdown_connections 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f942806e140 0x7f94280705f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f943c0fde10 0x7f943c115b80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 --2- 172.21.15.33:0/1271830965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f943c0fe7b0 0x7f943c1160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 >> 172.21.15.33:0/1271830965 conn(0x7f943c0f7860 msgr2=0x7f943c0f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 shutdown_connections 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.739+0000 7f9444047700 1 -- 172.21.15.33:0/1271830965 wait complete. 2024-04-25T01:52:23.741 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:52:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.845+0000 7f96fcab4700 1 -- 172.21.15.33:0/3020864870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f80fbd40 msgr2=0x7f96f80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.845+0000 7f96fcab4700 1 --2- 172.21.15.33:0/3020864870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f80fbd40 0x7f96f80fc1b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f96e8004940 tx=0x7f96e8009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.845+0000 7f96fcab4700 1 -- 172.21.15.33:0/3020864870 shutdown_connections 2024-04-25T01:52:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.845+0000 7f96fcab4700 1 --2- 172.21.15.33:0/3020864870 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f80fbd40 0x7f96f80fc1b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.845+0000 7f96fcab4700 1 --2- 172.21.15.33:0/3020864870 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f8101d90 0x7f96f8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.845+0000 7f96fcab4700 1 -- 172.21.15.33:0/3020864870 >> 172.21.15.33:0/3020864870 conn(0x7f96f8071d00 msgr2=0x7f96f8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.845+0000 7f96fcab4700 1 -- 172.21.15.33:0/3020864870 shutdown_connections 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.845+0000 7f96fcab4700 1 -- 172.21.15.33:0/3020864870 wait complete. 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96fcab4700 1 Processor -- start 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96fcab4700 1 -- start start 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96fcab4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f80fbd40 0x7f96f810ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96fcab4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8101d90 0x7f96f810f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96fcab4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f810f700 con 0x7f96f8101d90 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96fcab4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f810f840 con 0x7f96f80fbd40 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96f5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8101d90 0x7f96f810f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96f5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8101d90 0x7f96f810f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53312/0 (socket says 172.21.15.33:53312) 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96f5d9b700 1 -- 172.21.15.33:0/1408751527 learned_addr learned my addr 172.21.15.33:0/1408751527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96f659c700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f80fbd40 0x7f96f810ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:23.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96f5d9b700 1 -- 172.21.15.33:0/1408751527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f80fbd40 msgr2=0x7f96f810ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96f5d9b700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f80fbd40 0x7f96f810ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.846+0000 7f96f5d9b700 1 -- 172.21.15.33:0/1408751527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e8005040 con 0x7f96f8101d90 2024-04-25T01:52:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.847+0000 7f96f5d9b700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8101d90 0x7f96f810f1c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009150 tx=0x7f96e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.847+0000 7f96eeffd700 1 -- 172.21.15.33:0/1408751527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96e8021070 con 0x7f96f8101d90 2024-04-25T01:52:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.847+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f8117b30 con 0x7f96f8101d90 2024-04-25T01:52:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.847+0000 7f96eeffd700 1 -- 172.21.15.33:0/1408751527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96e8013c00 con 0x7f96f8101d90 2024-04-25T01:52:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.847+0000 7f96eeffd700 1 -- 172.21.15.33:0/1408751527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96e800e070 con 0x7f96f8101d90 2024-04-25T01:52:23.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.847+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f8117fc0 con 0x7f96f8101d90 2024-04-25T01:52:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.848+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f80fd430 con 0x7f96f8101d90 2024-04-25T01:52:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.848+0000 7f96eeffd700 1 -- 172.21.15.33:0/1408751527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f96e801c400 con 0x7f96f8101d90 2024-04-25T01:52:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.849+0000 7f96eeffd700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96e406e150 0x7f96e4070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.849+0000 7f96eeffd700 1 -- 172.21.15.33:0/1408751527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f96e808eff0 con 0x7f96f8101d90 2024-04-25T01:52:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.849+0000 7f96f659c700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96e406e150 0x7f96e4070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:23.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.849+0000 7f96f659c700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96e406e150 0x7f96e4070600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009b80 tx=0x7f96e000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:23.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:23.854+0000 7f96eeffd700 1 -- 172.21.15.33:0/1408751527 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f96e80c3740 con 0x7f96f8101d90 2024-04-25T01:52:24.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.085+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96f8108e90 con 0x7f96e406e150 2024-04-25T01:52:24.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.086+0000 7f96eeffd700 1 -- 172.21.15.33:0/1408751527 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f96e000f040 con 0x7f96e406e150 2024-04-25T01:52:24.086 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:52:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:52:24.086 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:52:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:52:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:52:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:52:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:52:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:52:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:52:24.087 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:52:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:52:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:52:24.088 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:52:24.088 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:52:24.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96e406e150 msgr2=0x7f96e4070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:24.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96e406e150 0x7f96e4070600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f96e0009b80 tx=0x7f96e000b040 comp rx=0 tx=0).stop 2024-04-25T01:52:24.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8101d90 msgr2=0x7f96f810f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:24.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8101d90 0x7f96f810f1c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009150 tx=0x7f96e8009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:24.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 shutdown_connections 2024-04-25T01:52:24.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96e406e150 0x7f96e4070600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96f80fbd40 0x7f96f810ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 --2- 172.21.15.33:0/1408751527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96f8101d90 0x7f96f810f1c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 >> 172.21.15.33:0/1408751527 conn(0x7f96f8071d00 msgr2=0x7f96f80f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:24.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 shutdown_connections 2024-04-25T01:52:24.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.088+0000 7f96fcab4700 1 -- 172.21.15.33:0/1408751527 wait complete. 2024-04-25T01:52:24.192 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.192+0000 7f303289e700 1 -- 172.21.15.33:0/1627889211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fe800 msgr2=0x7f302c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:24.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.192+0000 7f303289e700 1 --2- 172.21.15.33:0/1627889211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fe800 0x7f302c106e40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3020004940 tx=0x7f3020009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:24.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.192+0000 7f303289e700 1 -- 172.21.15.33:0/1627889211 shutdown_connections 2024-04-25T01:52:24.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.192+0000 7f303289e700 1 --2- 172.21.15.33:0/1627889211 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fe800 0x7f302c106e40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.192+0000 7f303289e700 1 --2- 172.21.15.33:0/1627889211 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c0fde60 0x7f302c0fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.192+0000 7f303289e700 1 -- 172.21.15.33:0/1627889211 >> 172.21.15.33:0/1627889211 conn(0x7f302c0f7870 msgr2=0x7f302c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:24.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.192+0000 7f303289e700 1 -- 172.21.15.33:0/1627889211 shutdown_connections 2024-04-25T01:52:24.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.193+0000 7f303289e700 1 -- 172.21.15.33:0/1627889211 wait complete. 2024-04-25T01:52:24.193 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.193+0000 7f303289e700 1 Processor -- start 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.193+0000 7f303289e700 1 -- start start 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.193+0000 7f303289e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fde60 0x7f302c113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.193+0000 7f303289e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c0fe800 0x7f302c113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.193+0000 7f303289e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1144d0 con 0x7f302c0fde60 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.193+0000 7f303289e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c10dad0 con 0x7f302c0fe800 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f302bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fde60 0x7f302c113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f302bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fde60 0x7f302c113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53318/0 (socket says 172.21.15.33:53318) 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f302bfff700 1 -- 172.21.15.33:0/2489457810 learned_addr learned my addr 172.21.15.33:0/2489457810 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f302b7fe700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c0fe800 0x7f302c113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f302bfff700 1 -- 172.21.15.33:0/2489457810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c0fe800 msgr2=0x7f302c113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:24.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f302bfff700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c0fe800 0x7f302c113e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f302bfff700 1 -- 172.21.15.33:0/2489457810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020005040 con 0x7f302c0fde60 2024-04-25T01:52:24.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f302b7fe700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c0fe800 0x7f302c113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:52:24.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f302bfff700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fde60 0x7f302c113940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c8c0 tx=0x7f301c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:24.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f3028ff9700 1 -- 172.21.15.33:0/2489457810 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c00a980 con 0x7f302c0fde60 2024-04-25T01:52:24.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f3028ff9700 1 -- 172.21.15.33:0/2489457810 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f301c00ce60 con 0x7f302c0fde60 2024-04-25T01:52:24.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c10ddb0 con 0x7f302c0fde60 2024-04-25T01:52:24.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f3028ff9700 1 -- 172.21.15.33:0/2489457810 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f301c0115e0 con 0x7f302c0fde60 2024-04-25T01:52:24.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.194+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c10e2f0 con 0x7f302c0fde60 2024-04-25T01:52:24.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.195+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c066ad0 con 0x7f302c0fde60 2024-04-25T01:52:24.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.196+0000 7f3028ff9700 1 -- 172.21.15.33:0/2489457810 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f301c012400 con 0x7f302c0fde60 2024-04-25T01:52:24.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.196+0000 7f3028ff9700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f301406e140 0x7f30140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:24.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.196+0000 7f3028ff9700 1 -- 172.21.15.33:0/2489457810 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f301c08dbf0 con 0x7f302c0fde60 2024-04-25T01:52:24.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.196+0000 7f302b7fe700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f301406e140 0x7f30140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:24.197 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.197+0000 7f302b7fe700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f301406e140 0x7f30140705f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f302c10ec90 tx=0x7f3020009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:24.201 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.202+0000 7f3028ff9700 1 -- 172.21.15.33:0/2489457810 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f301c05ab20 con 0x7f302c0fde60 2024-04-25T01:52:24.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.497+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f302c109e20 con 0x7f302c0fde60 2024-04-25T01:52:24.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:24 smithi033 ceph-mon[111306]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:24.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:24 smithi033 ceph-mon[111306]: pgmap v389: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:24.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:24 smithi033 ceph-mon[111306]: from='client.34768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:24.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:24 smithi033 ceph-mon[111306]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:24.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:24 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1939300422' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:52:24.497 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:24 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1271830965' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:52:24.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.497+0000 7f3028ff9700 1 -- 172.21.15.33:0/2489457810 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f301c0126b0 con 0x7f302c0fde60 2024-04-25T01:52:24.498 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:52:24.498 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:52:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:52:24.498 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:52:24.498 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:52:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.499+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f301406e140 msgr2=0x7f30140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.499+0000 7f303289e700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f301406e140 0x7f30140705f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f302c10ec90 tx=0x7f3020009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.499+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fde60 msgr2=0x7f302c113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.499+0000 7f303289e700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fde60 0x7f302c113940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f301c00c8c0 tx=0x7f301c0044c0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.499+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 shutdown_connections 2024-04-25T01:52:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.499+0000 7f303289e700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f301406e140 0x7f30140705f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.499+0000 7f303289e700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f302c0fde60 0x7f302c113940 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.499+0000 7f303289e700 1 --2- 172.21.15.33:0/2489457810 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f302c0fe800 0x7f302c113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.499+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 >> 172.21.15.33:0/2489457810 conn(0x7f302c0f7870 msgr2=0x7f302c1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.500+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 shutdown_connections 2024-04-25T01:52:24.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:24.500+0000 7f303289e700 1 -- 172.21.15.33:0/2489457810 wait complete. 2024-04-25T01:52:24.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:24 smithi138 ceph-mon[81304]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:24.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:24 smithi138 ceph-mon[81304]: pgmap v389: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:24.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:24 smithi138 ceph-mon[81304]: from='client.34768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:24.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:24 smithi138 ceph-mon[81304]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:24.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:24 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1939300422' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:52:24.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:24 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1271830965' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:52:25.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:25 smithi033 ceph-mon[111306]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:25.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:25 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2489457810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:52:25.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:25 smithi138 ceph-mon[81304]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:25.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:25 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2489457810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:52:26.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:26 smithi033 ceph-mon[111306]: pgmap v390: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:26.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:52:26.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:26 smithi138 ceph-mon[81304]: pgmap v390: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:26.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:52:28.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:28 smithi033 ceph-mon[111306]: pgmap v391: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:28.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:28 smithi138 ceph-mon[81304]: pgmap v391: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:30.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:30 smithi033 ceph-mon[111306]: pgmap v392: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:30.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:30 smithi138 ceph-mon[81304]: pgmap v392: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:32.515 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:32 smithi033 ceph-mon[111306]: pgmap v393: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:32.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:32 smithi138 ceph-mon[81304]: pgmap v393: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:34.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:34 smithi033 ceph-mon[111306]: pgmap v394: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:34.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:34 smithi138 ceph-mon[81304]: pgmap v394: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:36.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:36 smithi033 ceph-mon[111306]: pgmap v395: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:36.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:36 smithi138 ceph-mon[81304]: pgmap v395: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:38.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:38 smithi033 ceph-mon[111306]: pgmap v396: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:38.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:38 smithi138 ceph-mon[81304]: pgmap v396: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:40.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:40 smithi033 ceph-mon[111306]: pgmap v397: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:40.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:40 smithi138 ceph-mon[81304]: pgmap v397: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:41.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:52:41.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:52:42.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:42 smithi033 ceph-mon[111306]: pgmap v398: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:42.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:42 smithi138 ceph-mon[81304]: pgmap v398: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:44.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:44 smithi033 ceph-mon[111306]: pgmap v399: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:44.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:44 smithi138 ceph-mon[81304]: pgmap v399: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:46.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:46 smithi033 ceph-mon[111306]: pgmap v400: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:46.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:46 smithi138 ceph-mon[81304]: pgmap v400: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:48.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:48 smithi033 ceph-mon[111306]: pgmap v401: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:48.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:48 smithi138 ceph-mon[81304]: pgmap v401: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:50.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:50 smithi033 ceph-mon[111306]: pgmap v402: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:50.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:50 smithi138 ceph-mon[81304]: pgmap v402: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:51.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:51 smithi033 ceph-mon[111306]: pgmap v403: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:51.541 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:51 smithi138 ceph-mon[81304]: pgmap v403: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:53.766 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:53 smithi033 ceph-mon[111306]: pgmap v404: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:53.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:53 smithi138 ceph-mon[81304]: pgmap v404: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:54.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.619+0000 7effc30e1700 1 -- 172.21.15.33:0/4205551595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 msgr2=0x7effbc1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.619+0000 7effc30e1700 1 --2- 172.21.15.33:0/4205551595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 0x7effbc1031c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7effb0004940 tx=0x7effb0009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.619+0000 7effc30e1700 1 -- 172.21.15.33:0/4205551595 shutdown_connections 2024-04-25T01:52:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.619+0000 7effc30e1700 1 --2- 172.21.15.33:0/4205551595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc103790 0x7effbc0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.619+0000 7effc30e1700 1 --2- 172.21.15.33:0/4205551595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 0x7effbc1031c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.619+0000 7effc30e1700 1 -- 172.21.15.33:0/4205551595 >> 172.21.15.33:0/4205551595 conn(0x7effbc0f8980 msgr2=0x7effbc0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.619+0000 7effc30e1700 1 -- 172.21.15.33:0/4205551595 shutdown_connections 2024-04-25T01:52:54.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.619+0000 7effc30e1700 1 -- 172.21.15.33:0/4205551595 wait complete. 2024-04-25T01:52:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.620+0000 7effc30e1700 1 Processor -- start 2024-04-25T01:52:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.620+0000 7effc30e1700 1 -- start start 2024-04-25T01:52:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc30e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 0x7effbc113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc30e1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc103790 0x7effbc1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc30e1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc114730 con 0x7effbc102df0 2024-04-25T01:52:54.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc30e1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effbc10db40 con 0x7effbc103790 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc0e7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 0x7effbc113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc0e7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 0x7effbc113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56920/0 (socket says 172.21.15.33:56920) 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc0e7d700 1 -- 172.21.15.33:0/1317977409 learned_addr learned my addr 172.21.15.33:0/1317977409 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effbbfff700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc103790 0x7effbc1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc0e7d700 1 -- 172.21.15.33:0/1317977409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc103790 msgr2=0x7effbc1140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc0e7d700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc103790 0x7effbc1140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc0e7d700 1 -- 172.21.15.33:0/1317977409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0005040 con 0x7effbc102df0 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effbbfff700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc103790 0x7effbc1140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc0e7d700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 0x7effbc113ba0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7effb0009150 tx=0x7effb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effb97fa700 1 -- 172.21.15.33:0/1317977409 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effb000e040 con 0x7effbc102df0 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc10ddc0 con 0x7effbc102df0 2024-04-25T01:52:54.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effb97fa700 1 -- 172.21.15.33:0/1317977409 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7effb0013420 con 0x7effbc102df0 2024-04-25T01:52:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.621+0000 7effb97fa700 1 -- 172.21.15.33:0/1317977409 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effb001c630 con 0x7effbc102df0 2024-04-25T01:52:54.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.622+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effbc10e380 con 0x7effbc102df0 2024-04-25T01:52:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.623+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effbc100100 con 0x7effbc102df0 2024-04-25T01:52:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.623+0000 7effb97fa700 1 -- 172.21.15.33:0/1317977409 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7effb0013590 con 0x7effbc102df0 2024-04-25T01:52:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.624+0000 7effb97fa700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7effac06e150 0x7effac070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.624+0000 7effb97fa700 1 -- 172.21.15.33:0/1317977409 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7effb008eda0 con 0x7effbc102df0 2024-04-25T01:52:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.624+0000 7effbbfff700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7effac06e150 0x7effac070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:54.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.624+0000 7effbbfff700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7effac06e150 0x7effac070600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7effa8009fd0 tx=0x7effa8009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:54.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.629+0000 7effb97fa700 1 -- 172.21.15.33:0/1317977409 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7effb00581a0 con 0x7effbc102df0 2024-04-25T01:52:54.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.856+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7effbc109ef0 con 0x7effac06e150 2024-04-25T01:52:54.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.857+0000 7effb97fa700 1 -- 172.21.15.33:0/1317977409 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7effbc109ef0 con 0x7effac06e150 2024-04-25T01:52:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7effac06e150 msgr2=0x7effac070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7effac06e150 0x7effac070600 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7effa8009fd0 tx=0x7effa8009ee0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 msgr2=0x7effbc113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 0x7effbc113ba0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7effb0009150 tx=0x7effb0009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 shutdown_connections 2024-04-25T01:52:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7effac06e150 0x7effac070600 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7effbc102df0 0x7effbc113ba0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 --2- 172.21.15.33:0/1317977409 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7effbc103790 0x7effbc1140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 >> 172.21.15.33:0/1317977409 conn(0x7effbc0f8980 msgr2=0x7effbc0facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 shutdown_connections 2024-04-25T01:52:54.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.859+0000 7effc30e1700 1 -- 172.21.15.33:0/1317977409 wait complete. 2024-04-25T01:52:54.870 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:52:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.964+0000 7f3ac347c700 1 -- 172.21.15.33:0/1021822900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc101d00 msgr2=0x7f3abc1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.964+0000 7f3ac347c700 1 --2- 172.21.15.33:0/1021822900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc101d00 0x7f3abc1020d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8004940 tx=0x7f3ab8009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.964+0000 7f3ac347c700 1 -- 172.21.15.33:0/1021822900 shutdown_connections 2024-04-25T01:52:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.964+0000 7f3ac347c700 1 --2- 172.21.15.33:0/1021822900 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc1026a0 0x7f3abc0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.964+0000 7f3ac347c700 1 --2- 172.21.15.33:0/1021822900 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc101d00 0x7f3abc1020d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.964+0000 7f3ac347c700 1 -- 172.21.15.33:0/1021822900 >> 172.21.15.33:0/1021822900 conn(0x7f3abc0f7870 msgr2=0x7f3abc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:54.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.965+0000 7f3ac347c700 1 -- 172.21.15.33:0/1021822900 shutdown_connections 2024-04-25T01:52:54.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.965+0000 7f3ac347c700 1 -- 172.21.15.33:0/1021822900 wait complete. 2024-04-25T01:52:54.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.965+0000 7f3ac347c700 1 Processor -- start 2024-04-25T01:52:54.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.965+0000 7f3ac347c700 1 -- start start 2024-04-25T01:52:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.965+0000 7f3ac347c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc101d00 0x7f3abc113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.965+0000 7f3ac347c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc1026a0 0x7f3abc113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.965+0000 7f3ac347c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc1145a0 con 0x7f3abc1026a0 2024-04-25T01:52:54.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.965+0000 7f3ac347c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc10db10 con 0x7f3abc101d00 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac0a17700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc1026a0 0x7f3abc113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac0a17700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc1026a0 0x7f3abc113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56950/0 (socket says 172.21.15.33:56950) 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac0a17700 1 -- 172.21.15.33:0/493532281 learned_addr learned my addr 172.21.15.33:0/493532281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac1218700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc101d00 0x7f3abc113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac0a17700 1 -- 172.21.15.33:0/493532281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc101d00 msgr2=0x7f3abc113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac0a17700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc101d00 0x7f3abc113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac0a17700 1 -- 172.21.15.33:0/493532281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8005040 con 0x7f3abc1026a0 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac1218700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc101d00 0x7f3abc113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac0a17700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc1026a0 0x7f3abc113ec0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00b790 tx=0x7f3aac009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ab1ffb700 1 -- 172.21.15.33:0/493532281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aac00f6f0 con 0x7f3abc1026a0 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc10ddf0 con 0x7f3abc1026a0 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ab1ffb700 1 -- 172.21.15.33:0/493532281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3aac00fd20 con 0x7f3abc1026a0 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ab1ffb700 1 -- 172.21.15.33:0/493532281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aac0103a0 con 0x7f3abc1026a0 2024-04-25T01:52:54.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.966+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc10e3b0 con 0x7f3abc1026a0 2024-04-25T01:52:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.967+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3abc0fd450 con 0x7f3abc1026a0 2024-04-25T01:52:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.968+0000 7f3ab1ffb700 1 -- 172.21.15.33:0/493532281 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3aac011400 con 0x7f3abc1026a0 2024-04-25T01:52:54.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.968+0000 7f3ab1ffb700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3aa806e140 0x7f3aa80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.968+0000 7f3ab1ffb700 1 -- 172.21.15.33:0/493532281 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3aac08d8f0 con 0x7f3abc1026a0 2024-04-25T01:52:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.968+0000 7f3ac1218700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3aa806e140 0x7f3aa80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:54.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.969+0000 7f3ac1218700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3aa806e140 0x7f3aa80705f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8004560 tx=0x7f3ab8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:54.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:54.973+0000 7f3ab1ffb700 1 -- 172.21.15.33:0/493532281 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3aac05a820 con 0x7f3abc1026a0 2024-04-25T01:52:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.198+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3abc108e70 con 0x7f3aa806e140 2024-04-25T01:52:55.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.200+0000 7f3ab1ffb700 1 -- 172.21.15.33:0/493532281 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3abc108e70 con 0x7f3aa806e140 2024-04-25T01:52:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3aa806e140 msgr2=0x7f3aa80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3aa806e140 0x7f3aa80705f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8004560 tx=0x7f3ab8009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc1026a0 msgr2=0x7f3abc113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc1026a0 0x7f3abc113ec0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3aac00b790 tx=0x7f3aac009220 comp rx=0 tx=0).stop 2024-04-25T01:52:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 shutdown_connections 2024-04-25T01:52:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3aa806e140 0x7f3aa80705f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3abc101d00 0x7f3abc113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 --2- 172.21.15.33:0/493532281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3abc1026a0 0x7f3abc113ec0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 >> 172.21.15.33:0/493532281 conn(0x7f3abc0f7870 msgr2=0x7f3abc066250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 shutdown_connections 2024-04-25T01:52:55.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.202+0000 7f3ac347c700 1 -- 172.21.15.33:0/493532281 wait complete. 2024-04-25T01:52:55.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.307+0000 7f6c13df9700 1 -- 172.21.15.33:0/1781536947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c103780 msgr2=0x7f6c0c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.307+0000 7f6c13df9700 1 --2- 172.21.15.33:0/1781536947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c103780 0x7f6c0c0fcdf0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6c00004940 tx=0x7f6c00009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:55.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.307+0000 7f6c13df9700 1 -- 172.21.15.33:0/1781536947 shutdown_connections 2024-04-25T01:52:55.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.307+0000 7f6c13df9700 1 --2- 172.21.15.33:0/1781536947 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c103780 0x7f6c0c0fcdf0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.307+0000 7f6c13df9700 1 --2- 172.21.15.33:0/1781536947 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c0c102de0 0x7f6c0c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.307+0000 7f6c13df9700 1 -- 172.21.15.33:0/1781536947 >> 172.21.15.33:0/1781536947 conn(0x7f6c0c0f89b0 msgr2=0x7f6c0c0fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:55.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.307+0000 7f6c13df9700 1 -- 172.21.15.33:0/1781536947 shutdown_connections 2024-04-25T01:52:55.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.307+0000 7f6c13df9700 1 -- 172.21.15.33:0/1781536947 wait complete. 2024-04-25T01:52:55.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c13df9700 1 Processor -- start 2024-04-25T01:52:55.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c13df9700 1 -- start start 2024-04-25T01:52:55.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c13df9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c102de0 0x7f6c0c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:55.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c13df9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c0c103780 0x7f6c0c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:55.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c13df9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c070d30 con 0x7f6c0c102de0 2024-04-25T01:52:55.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c13df9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c070e70 con 0x7f6c0c103780 2024-04-25T01:52:55.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c11b95700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c102de0 0x7f6c0c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:55.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c11b95700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c102de0 0x7f6c0c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56978/0 (socket says 172.21.15.33:56978) 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c11b95700 1 -- 172.21.15.33:0/1135229204 learned_addr learned my addr 172.21.15.33:0/1135229204 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c11394700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c0c103780 0x7f6c0c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c11b95700 1 -- 172.21.15.33:0/1135229204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c0c103780 msgr2=0x7f6c0c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c11b95700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c0c103780 0x7f6c0c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c11b95700 1 -- 172.21.15.33:0/1135229204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c00005040 con 0x7f6c0c102de0 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.308+0000 7f6c11b95700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c102de0 0x7f6c0c070260 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400ec90 tx=0x7f6c0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.309+0000 7f6c0a7fc700 1 -- 172.21.15.33:0/1135229204 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c04007b30 con 0x7f6c0c102de0 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.309+0000 7f6c0a7fc700 1 -- 172.21.15.33:0/1135229204 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c0400f040 con 0x7f6c0c102de0 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.309+0000 7f6c0a7fc700 1 -- 172.21.15.33:0/1135229204 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c0400cc80 con 0x7f6c0c102de0 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.309+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c119db0 con 0x7f6c0c102de0 2024-04-25T01:52:55.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.309+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c11a1b0 con 0x7f6c0c102de0 2024-04-25T01:52:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.310+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c0fe530 con 0x7f6c0c102de0 2024-04-25T01:52:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.310+0000 7f6c0a7fc700 1 -- 172.21.15.33:0/1135229204 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6c04007c90 con 0x7f6c0c102de0 2024-04-25T01:52:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.311+0000 7f6c0a7fc700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6bf406e140 0x7f6bf40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.311+0000 7f6c0a7fc700 1 -- 172.21.15.33:0/1135229204 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6c0408df90 con 0x7f6c0c102de0 2024-04-25T01:52:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.311+0000 7f6c11394700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6bf406e140 0x7f6bf40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:55.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.311+0000 7f6c11394700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6bf406e140 0x7f6bf40705f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c00004890 tx=0x7f6c00009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:55.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.315+0000 7f6c0a7fc700 1 -- 172.21.15.33:0/1135229204 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6c04057310 con 0x7f6c0c102de0 2024-04-25T01:52:55.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:55 smithi033 ceph-mon[111306]: pgmap v405: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:55.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:55 smithi033 ceph-mon[111306]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:55.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:55 smithi033 ceph-mon[111306]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:55.516 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:52:55.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.535+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c0c0719c0 con 0x7f6bf406e140 2024-04-25T01:52:55.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.545+0000 7f6c0a7fc700 1 -- 172.21.15.33:0/1135229204 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6c0c0719c0 con 0x7f6bf406e140 2024-04-25T01:52:55.545 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:52:55.545 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (14m) 7m ago 25m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:52:55.545 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (11m) 7m ago 25m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:52:55.545 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (11m) 6m ago 24m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (13m) 7m ago 24m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (7m) 7m ago 19m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (7m) 7m ago 19m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (6m) 6m ago 19m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (19m) 6m ago 19m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (15m) 7m ago 26m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (14m) 6m ago 24m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (12m) 7m ago 26m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (11m) 6m ago 24m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (14m) 7m ago 25m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (14m) 6m ago 24m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (10m) 7m ago 23m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (10m) 7m ago 23m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:52:55.546 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (9m) 7m ago 22m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:52:55.547 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (9m) 6m ago 22m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:52:55.547 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (8m) 6m ago 21m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:52:55.547 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (7m) 6m ago 21m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:52:55.547 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (14m) 7m ago 24m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6bf406e140 msgr2=0x7f6bf40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6bf406e140 0x7f6bf40705f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c00004890 tx=0x7f6c00009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c102de0 msgr2=0x7f6c0c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c102de0 0x7f6c0c070260 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400ec90 tx=0x7f6c0400b040 comp rx=0 tx=0).stop 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 shutdown_connections 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6bf406e140 0x7f6bf40705f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c0c102de0 0x7f6c0c070260 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 --2- 172.21.15.33:0/1135229204 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c0c103780 0x7f6c0c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 >> 172.21.15.33:0/1135229204 conn(0x7f6c0c0f89b0 msgr2=0x7f6c0c0f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 shutdown_connections 2024-04-25T01:52:55.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.547+0000 7f6c13df9700 1 -- 172.21.15.33:0/1135229204 wait complete. 2024-04-25T01:52:55.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/1271722224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fddf0 msgr2=0x7fbd480fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 --2- 172.21.15.33:0/1271722224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fddf0 0x7fbd480fe1c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbd38004940 tx=0x7fbd38009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:55.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/1271722224 shutdown_connections 2024-04-25T01:52:55.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 --2- 172.21.15.33:0/1271722224 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd480fe790 0x7fbd48106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 --2- 172.21.15.33:0/1271722224 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fddf0 0x7fbd480fe1c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/1271722224 >> 172.21.15.33:0/1271722224 conn(0x7fbd480f7860 msgr2=0x7fbd480f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:55.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/1271722224 shutdown_connections 2024-04-25T01:52:55.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/1271722224 wait complete. 2024-04-25T01:52:55.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 Processor -- start 2024-04-25T01:52:55.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.650+0000 7fbd4cfd0700 1 -- start start 2024-04-25T01:52:55.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd4cfd0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd480fddf0 0x7fbd481139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:55.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd4cfd0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fe790 0x7fbd48113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:55.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd4cfd0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48114530 con 0x7fbd480fe790 2024-04-25T01:52:55.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd4cfd0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4810db50 con 0x7fbd480fddf0 2024-04-25T01:52:55.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd45d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fe790 0x7fbd48113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:55.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd45d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fe790 0x7fbd48113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45042/0 (socket says 172.21.15.33:45042) 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd45d9b700 1 -- 172.21.15.33:0/16441826 learned_addr learned my addr 172.21.15.33:0/16441826 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd4659c700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd480fddf0 0x7fbd481139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd45d9b700 1 -- 172.21.15.33:0/16441826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd480fddf0 msgr2=0x7fbd481139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd45d9b700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd480fddf0 0x7fbd481139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd45d9b700 1 -- 172.21.15.33:0/16441826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd38005040 con 0x7fbd480fe790 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd4659c700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd480fddf0 0x7fbd481139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd45d9b700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fe790 0x7fbd48113ee0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00c8c0 tx=0x7fbd3c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd36ffd700 1 -- 172.21.15.33:0/16441826 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd3c00a980 con 0x7fbd480fe790 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd36ffd700 1 -- 172.21.15.33:0/16441826 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd3c00ce60 con 0x7fbd480fe790 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4810de30 con 0x7fbd480fe790 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd36ffd700 1 -- 172.21.15.33:0/16441826 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd3c0115e0 con 0x7fbd480fe790 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.651+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4810e370 con 0x7fbd480fe790 2024-04-25T01:52:55.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.652+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd48066bb0 con 0x7fbd480fe790 2024-04-25T01:52:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.653+0000 7fbd36ffd700 1 -- 172.21.15.33:0/16441826 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbd3c012400 con 0x7fbd480fe790 2024-04-25T01:52:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.653+0000 7fbd36ffd700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbd3006e140 0x7fbd300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.653+0000 7fbd36ffd700 1 -- 172.21.15.33:0/16441826 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbd3c08db20 con 0x7fbd480fe790 2024-04-25T01:52:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.653+0000 7fbd4659c700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbd3006e140 0x7fbd300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:55.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.654+0000 7fbd4659c700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbd3006e140 0x7fbd300705f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b3b0 tx=0x7fbd38009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:55.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.658+0000 7fbd36ffd700 1 -- 172.21.15.33:0/16441826 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbd3c056fc0 con 0x7fbd480fe790 2024-04-25T01:52:55.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:55 smithi138 ceph-mon[81304]: pgmap v405: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:55.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:55 smithi138 ceph-mon[81304]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:55.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:55 smithi138 ceph-mon[81304]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:55.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:52:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.953+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbd4810f050 con 0x7fbd480fe790 2024-04-25T01:52:55.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.953+0000 7fbd36ffd700 1 -- 172.21.15.33:0/16441826 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fbd3c0126b0 con 0x7fbd480fe790 2024-04-25T01:52:55.953 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:52:55.954 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:52:55.955 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:52:55.955 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:52:55.955 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:52:55.955 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:52:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbd3006e140 msgr2=0x7fbd300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbd3006e140 0x7fbd300705f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b3b0 tx=0x7fbd38009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fe790 msgr2=0x7fbd48113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fe790 0x7fbd48113ee0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c00c8c0 tx=0x7fbd3c0044c0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 shutdown_connections 2024-04-25T01:52:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbd3006e140 0x7fbd300705f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbd480fddf0 0x7fbd481139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 --2- 172.21.15.33:0/16441826 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbd480fe790 0x7fbd48113ee0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 >> 172.21.15.33:0/16441826 conn(0x7fbd480f7860 msgr2=0x7fbd480f8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.955+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 shutdown_connections 2024-04-25T01:52:55.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:55.956+0000 7fbd4cfd0700 1 -- 172.21.15.33:0/16441826 wait complete. 2024-04-25T01:52:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.059+0000 7f4430e40700 1 -- 172.21.15.33:0/2659087454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fbf40 msgr2=0x7f442c101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.059+0000 7f4430e40700 1 --2- 172.21.15.33:0/2659087454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fbf40 0x7f442c101a30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4420004940 tx=0x7f4420009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.059+0000 7f4430e40700 1 -- 172.21.15.33:0/2659087454 shutdown_connections 2024-04-25T01:52:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.059+0000 7f4430e40700 1 --2- 172.21.15.33:0/2659087454 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fbf40 0x7f442c101a30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.059+0000 7f4430e40700 1 --2- 172.21.15.33:0/2659087454 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f442c0fb5a0 0x7f442c0fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.059+0000 7f4430e40700 1 -- 172.21.15.33:0/2659087454 >> 172.21.15.33:0/2659087454 conn(0x7f442c073350 msgr2=0x7f442c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.059+0000 7f4430e40700 1 -- 172.21.15.33:0/2659087454 shutdown_connections 2024-04-25T01:52:56.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.060+0000 7f4430e40700 1 -- 172.21.15.33:0/2659087454 wait complete. 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.060+0000 7f4430e40700 1 Processor -- start 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.060+0000 7f4430e40700 1 -- start start 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.060+0000 7f4430e40700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fb5a0 0x7f442c113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.060+0000 7f4430e40700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f442c0fbf40 0x7f442c113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.060+0000 7f4430e40700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c1144f0 con 0x7f442c0fb5a0 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.060+0000 7f4430e40700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442c10daf0 con 0x7f442c0fbf40 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f442a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fb5a0 0x7f442c113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f442a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fb5a0 0x7f442c113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45046/0 (socket says 172.21.15.33:45046) 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f442a59c700 1 -- 172.21.15.33:0/290346968 learned_addr learned my addr 172.21.15.33:0/290346968 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:56.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f4429d9b700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f442c0fbf40 0x7f442c113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:56.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f442a59c700 1 -- 172.21.15.33:0/290346968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f442c0fbf40 msgr2=0x7f442c113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f442a59c700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f442c0fbf40 0x7f442c113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f442a59c700 1 -- 172.21.15.33:0/290346968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420005040 con 0x7f442c0fb5a0 2024-04-25T01:52:56.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f442a59c700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fb5a0 0x7f442c113960 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f441c002a80 tx=0x7f441c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:56.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f441affd700 1 -- 172.21.15.33:0/290346968 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f441c007cc0 con 0x7f442c0fb5a0 2024-04-25T01:52:56.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f441affd700 1 -- 172.21.15.33:0/290346968 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f441c00f040 con 0x7f442c0fb5a0 2024-04-25T01:52:56.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442c10ddd0 con 0x7f442c0fb5a0 2024-04-25T01:52:56.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f441affd700 1 -- 172.21.15.33:0/290346968 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f441c00dac0 con 0x7f442c0fb5a0 2024-04-25T01:52:56.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.061+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442c10e2e0 con 0x7f442c0fb5a0 2024-04-25T01:52:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.062+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442c0fe670 con 0x7f442c0fb5a0 2024-04-25T01:52:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.063+0000 7f441affd700 1 -- 172.21.15.33:0/290346968 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f441c00dc20 con 0x7f442c0fb5a0 2024-04-25T01:52:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.063+0000 7f441affd700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441406e140 0x7f44140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.063+0000 7f441affd700 1 -- 172.21.15.33:0/290346968 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f441c08d830 con 0x7f442c0fb5a0 2024-04-25T01:52:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.063+0000 7f4429d9b700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441406e140 0x7f44140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:56.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.064+0000 7f4429d9b700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441406e140 0x7f44140705f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4420004890 tx=0x7f4420009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:56.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.068+0000 7f441affd700 1 -- 172.21.15.33:0/290346968 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f441c05a760 con 0x7f442c0fb5a0 2024-04-25T01:52:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.324+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f442c109f00 con 0x7f442c0fb5a0 2024-04-25T01:52:56.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.324+0000 7f441affd700 1 -- 172.21.15.33:0/290346968 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f441c05a2f0 con 0x7f442c0fb5a0 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:52:56.325 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:52:56.326 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:52:56.327 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:52:56.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441406e140 msgr2=0x7f44140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441406e140 0x7f44140705f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4420004890 tx=0x7f4420009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:56.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fb5a0 msgr2=0x7f442c113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fb5a0 0x7f442c113960 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f441c002a80 tx=0x7f441c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 shutdown_connections 2024-04-25T01:52:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441406e140 0x7f44140705f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f442c0fb5a0 0x7f442c113960 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 --2- 172.21.15.33:0/290346968 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f442c0fbf40 0x7f442c113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 >> 172.21.15.33:0/290346968 conn(0x7f442c073350 msgr2=0x7f442c100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 shutdown_connections 2024-04-25T01:52:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.327+0000 7f4430e40700 1 -- 172.21.15.33:0/290346968 wait complete. 2024-04-25T01:52:56.329 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:52:56.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.433+0000 7fe32a07f700 1 -- 172.21.15.33:0/3778274163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3240fbd40 msgr2=0x7fe3240fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.433 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.433+0000 7fe32a07f700 1 --2- 172.21.15.33:0/3778274163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3240fbd40 0x7fe3240fc1b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe318004940 tx=0x7fe318009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:56.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.433+0000 7fe32a07f700 1 -- 172.21.15.33:0/3778274163 shutdown_connections 2024-04-25T01:52:56.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.433+0000 7fe32a07f700 1 --2- 172.21.15.33:0/3778274163 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3240fbd40 0x7fe3240fc1b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.433+0000 7fe32a07f700 1 --2- 172.21.15.33:0/3778274163 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe324101d90 0x7fe324102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.433+0000 7fe32a07f700 1 -- 172.21.15.33:0/3778274163 >> 172.21.15.33:0/3778274163 conn(0x7fe324071d00 msgr2=0x7fe324072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:56.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.433+0000 7fe32a07f700 1 -- 172.21.15.33:0/3778274163 shutdown_connections 2024-04-25T01:52:56.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.433+0000 7fe32a07f700 1 -- 172.21.15.33:0/3778274163 wait complete. 2024-04-25T01:52:56.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.434+0000 7fe32a07f700 1 Processor -- start 2024-04-25T01:52:56.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.434+0000 7fe32a07f700 1 -- start start 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.434+0000 7fe32a07f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3240fbd40 0x7fe324112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.434+0000 7fe32a07f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324101d90 0x7fe324112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.434+0000 7fe32a07f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe324113640 con 0x7fe324101d90 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.434+0000 7fe32a07f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32410ca30 con 0x7fe3240fbd40 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.434+0000 7fe322ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324101d90 0x7fe324112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.434+0000 7fe322ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324101d90 0x7fe324112ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45070/0 (socket says 172.21.15.33:45070) 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.434+0000 7fe322ffd700 1 -- 172.21.15.33:0/2689429497 learned_addr learned my addr 172.21.15.33:0/2689429497 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe3237fe700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3240fbd40 0x7fe324112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe322ffd700 1 -- 172.21.15.33:0/2689429497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3240fbd40 msgr2=0x7fe324112ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe322ffd700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3240fbd40 0x7fe324112ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe322ffd700 1 -- 172.21.15.33:0/2689429497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe318005040 con 0x7fe324101d90 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe322ffd700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324101d90 0x7fe324112ff0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe31800b8b0 tx=0x7fe318009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe30bfff700 1 -- 172.21.15.33:0/2689429497 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe31800e040 con 0x7fe324101d90 2024-04-25T01:52:56.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe30bfff700 1 -- 172.21.15.33:0/2689429497 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe318013420 con 0x7fe324101d90 2024-04-25T01:52:56.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32410ccb0 con 0x7fe324101d90 2024-04-25T01:52:56.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe30bfff700 1 -- 172.21.15.33:0/2689429497 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe31801c630 con 0x7fe324101d90 2024-04-25T01:52:56.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.435+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32410d270 con 0x7fe324101d90 2024-04-25T01:52:56.437 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.436+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3240fd430 con 0x7fe324101d90 2024-04-25T01:52:56.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.439+0000 7fe30bfff700 1 -- 172.21.15.33:0/2689429497 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe318013a00 con 0x7fe324101d90 2024-04-25T01:52:56.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.439+0000 7fe30bfff700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe30c06e190 0x7fe30c070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:56.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.439+0000 7fe30bfff700 1 -- 172.21.15.33:0/2689429497 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe31808f900 con 0x7fe324101d90 2024-04-25T01:52:56.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.439+0000 7fe3237fe700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe30c06e190 0x7fe30c070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:56.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.440+0000 7fe3237fe700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe30c06e190 0x7fe30c070640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe314007fd0 tx=0x7fe314009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:56.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.443+0000 7fe30bfff700 1 -- 172.21.15.33:0/2689429497 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe31805c810 con 0x7fe324101d90 2024-04-25T01:52:56.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.669+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe324112780 con 0x7fe30c06e190 2024-04-25T01:52:56.670 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:56 smithi033 ceph-mon[111306]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:56.670 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:56 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/16441826' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:52:56.670 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:56 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/290346968' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.671+0000 7fe30bfff700 1 -- 172.21.15.33:0/2689429497 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe324112780 con 0x7fe30c06e190 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:52:56.671 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:52:56.672 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:52:56.672 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:52:56.672 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:52:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe30c06e190 msgr2=0x7fe30c070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe30c06e190 0x7fe30c070640 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe314007fd0 tx=0x7fe314009040 comp rx=0 tx=0).stop 2024-04-25T01:52:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324101d90 msgr2=0x7fe324112ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324101d90 0x7fe324112ff0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe31800b8b0 tx=0x7fe318009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 shutdown_connections 2024-04-25T01:52:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe30c06e190 0x7fe30c070640 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3240fbd40 0x7fe324112ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 --2- 172.21.15.33:0/2689429497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe324101d90 0x7fe324112ff0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 >> 172.21.15.33:0/2689429497 conn(0x7fe324071d00 msgr2=0x7fe3240fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:56.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 shutdown_connections 2024-04-25T01:52:56.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.673+0000 7fe32a07f700 1 -- 172.21.15.33:0/2689429497 wait complete. 2024-04-25T01:52:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.777+0000 7f04347fc700 1 -- 172.21.15.33:0/3561177636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c103780 msgr2=0x7f042c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.777+0000 7f04347fc700 1 --2- 172.21.15.33:0/3561177636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c103780 0x7f042c0fcdf0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0428004940 tx=0x7f0428009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.777+0000 7f04347fc700 1 -- 172.21.15.33:0/3561177636 shutdown_connections 2024-04-25T01:52:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.777+0000 7f04347fc700 1 --2- 172.21.15.33:0/3561177636 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c103780 0x7f042c0fcdf0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.777+0000 7f04347fc700 1 --2- 172.21.15.33:0/3561177636 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c102de0 0x7f042c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.777+0000 7f04347fc700 1 -- 172.21.15.33:0/3561177636 >> 172.21.15.33:0/3561177636 conn(0x7f042c0f8990 msgr2=0x7f042c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f04347fc700 1 -- 172.21.15.33:0/3561177636 shutdown_connections 2024-04-25T01:52:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f04347fc700 1 -- 172.21.15.33:0/3561177636 wait complete. 2024-04-25T01:52:56.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f04347fc700 1 Processor -- start 2024-04-25T01:52:56.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f04347fc700 1 -- start start 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f04347fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c102de0 0x7f042c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f04347fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c103780 0x7f042c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f04347fc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c114520 con 0x7f042c102de0 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f04347fc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042c10db20 con 0x7f042c103780 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f0432598700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c102de0 0x7f042c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f0432598700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c102de0 0x7f042c113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45088/0 (socket says 172.21.15.33:45088) 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.778+0000 7f0432598700 1 -- 172.21.15.33:0/1619824380 learned_addr learned my addr 172.21.15.33:0/1619824380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.779+0000 7f0432598700 1 -- 172.21.15.33:0/1619824380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c103780 msgr2=0x7f042c113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.779+0000 7f0432598700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c103780 0x7f042c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.779+0000 7f0432598700 1 -- 172.21.15.33:0/1619824380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0428005040 con 0x7f042c102de0 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.779+0000 7f0432598700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c102de0 0x7f042c113990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f041c002a50 tx=0x7f041c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:56.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.779+0000 7f0422ffd700 1 -- 172.21.15.33:0/1619824380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f041c007cc0 con 0x7f042c102de0 2024-04-25T01:52:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.779+0000 7f0422ffd700 1 -- 172.21.15.33:0/1619824380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f041c00f040 con 0x7f042c102de0 2024-04-25T01:52:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.779+0000 7f0422ffd700 1 -- 172.21.15.33:0/1619824380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f041c009640 con 0x7f042c102de0 2024-04-25T01:52:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.779+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042c10de00 con 0x7f042c102de0 2024-04-25T01:52:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.779+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042c10e370 con 0x7f042c102de0 2024-04-25T01:52:56.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.780+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f042c0fe530 con 0x7f042c102de0 2024-04-25T01:52:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.781+0000 7f0422ffd700 1 -- 172.21.15.33:0/1619824380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f041c0097a0 con 0x7f042c102de0 2024-04-25T01:52:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.781+0000 7f0422ffd700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f041806e140 0x7f04180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:52:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.781+0000 7f0422ffd700 1 -- 172.21.15.33:0/1619824380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f041c08d940 con 0x7f042c102de0 2024-04-25T01:52:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.781+0000 7f0431d97700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f041806e140 0x7f04180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:52:56.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.782+0000 7f0431d97700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f041806e140 0x7f04180705f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f042800b3b0 tx=0x7f0428009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:52:56.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:56.786+0000 7f0422ffd700 1 -- 172.21.15.33:0/1619824380 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f041c05a7f0 con 0x7f042c102de0 2024-04-25T01:52:56.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:56 smithi138 ceph-mon[81304]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:56.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:56 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/16441826' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:52:56.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:56 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/290346968' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:52:57.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.086+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f042c109ec0 con 0x7f042c102de0 2024-04-25T01:52:57.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.087+0000 7f0422ffd700 1 -- 172.21.15.33:0/1619824380 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f041c05a380 con 0x7f042c102de0 2024-04-25T01:52:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:52:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:52:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:52:57.087 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:52:57.087 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.088+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f041806e140 msgr2=0x7f04180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f041806e140 0x7f04180705f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f042800b3b0 tx=0x7f0428009e30 comp rx=0 tx=0).stop 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c102de0 msgr2=0x7f042c113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c102de0 0x7f042c113990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f041c002a50 tx=0x7f041c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 shutdown_connections 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f041806e140 0x7f04180705f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f042c102de0 0x7f042c113990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 --2- 172.21.15.33:0/1619824380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f042c103780 0x7f042c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 >> 172.21.15.33:0/1619824380 conn(0x7f042c0f8990 msgr2=0x7f042c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:52:57.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 shutdown_connections 2024-04-25T01:52:57.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:52:57.089+0000 7f04347fc700 1 -- 172.21.15.33:0/1619824380 wait complete. 2024-04-25T01:52:57.766 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:57 smithi033 ceph-mon[111306]: pgmap v406: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:57.766 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:57 smithi033 ceph-mon[111306]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:57.766 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:57 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1619824380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:52:57.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:57 smithi138 ceph-mon[81304]: pgmap v406: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:57.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:57 smithi138 ceph-mon[81304]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:52:57.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:57 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1619824380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:52:59.766 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:52:59 smithi033 ceph-mon[111306]: pgmap v407: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:52:59.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:52:59 smithi138 ceph-mon[81304]: pgmap v407: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:01.766 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:01 smithi033 ceph-mon[111306]: pgmap v408: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:01.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:01 smithi138 ceph-mon[81304]: pgmap v408: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:03.766 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:03 smithi033 ceph-mon[111306]: pgmap v409: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:03.791 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:03 smithi138 ceph-mon[81304]: pgmap v409: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:05.740 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:05 smithi138 ceph-mon[81304]: pgmap v410: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:05.766 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:05 smithi033 ceph-mon[111306]: pgmap v410: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:08.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:07 smithi033 ceph-mon[111306]: pgmap v411: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:08.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:07 smithi138 ceph-mon[81304]: pgmap v411: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:10.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:09 smithi033 ceph-mon[111306]: pgmap v412: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:10.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:09 smithi138 ceph-mon[81304]: pgmap v412: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:11.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:53:11.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:53:12.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:11 smithi033 ceph-mon[111306]: pgmap v413: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:12.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:11 smithi138 ceph-mon[81304]: pgmap v413: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:14.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:13 smithi033 ceph-mon[111306]: pgmap v414: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:14.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:13 smithi138 ceph-mon[81304]: pgmap v414: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:16.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:15 smithi033 ceph-mon[111306]: pgmap v415: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:16.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:15 smithi138 ceph-mon[81304]: pgmap v415: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:18.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:17 smithi033 ceph-mon[111306]: pgmap v416: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:18.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:17 smithi138 ceph-mon[81304]: pgmap v416: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:20.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:19 smithi033 ceph-mon[111306]: pgmap v417: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:20.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:19 smithi138 ceph-mon[81304]: pgmap v417: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:22.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:21 smithi033 ceph-mon[111306]: pgmap v418: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:22.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:21 smithi138 ceph-mon[81304]: pgmap v418: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:24.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:23 smithi033 ceph-mon[111306]: pgmap v419: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:24.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:23 smithi138 ceph-mon[81304]: pgmap v419: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:26.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:25 smithi138 ceph-mon[81304]: pgmap v420: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:26.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:53:26.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:25 smithi033 ceph-mon[111306]: pgmap v420: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:26.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:53:27.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.209+0000 7f04b355d700 1 -- 172.21.15.33:0/2211916812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac070840 msgr2=0x7f04ac070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.209+0000 7f04b355d700 1 --2- 172.21.15.33:0/2211916812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac070840 0x7f04ac070cb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f049c004970 tx=0x7f049c009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:27.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.209+0000 7f04b355d700 1 -- 172.21.15.33:0/2211916812 shutdown_connections 2024-04-25T01:53:27.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.209+0000 7f04b355d700 1 --2- 172.21.15.33:0/2211916812 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac070840 0x7f04ac070cb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.209+0000 7f04b355d700 1 --2- 172.21.15.33:0/2211916812 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac071f30 0x7f04ac070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.209+0000 7f04b355d700 1 -- 172.21.15.33:0/2211916812 >> 172.21.15.33:0/2211916812 conn(0x7f04ac0f8990 msgr2=0x7f04ac0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:27.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.209+0000 7f04b355d700 1 -- 172.21.15.33:0/2211916812 shutdown_connections 2024-04-25T01:53:27.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.209+0000 7f04b355d700 1 -- 172.21.15.33:0/2211916812 wait complete. 2024-04-25T01:53:27.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.209+0000 7f04b355d700 1 Processor -- start 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b355d700 1 -- start start 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b355d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac070840 0x7f04ac119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b355d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac071f30 0x7f04ac11a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b355d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac11aad0 con 0x7f04ac071f30 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b355d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ac1140b0 con 0x7f04ac070840 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b0af8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac071f30 0x7f04ac11a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b0af8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac071f30 0x7f04ac11a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54958/0 (socket says 172.21.15.33:54958) 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b0af8700 1 -- 172.21.15.33:0/3959567976 learned_addr learned my addr 172.21.15.33:0/3959567976 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b12f9700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac070840 0x7f04ac119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b0af8700 1 -- 172.21.15.33:0/3959567976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac070840 msgr2=0x7f04ac119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b0af8700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac070840 0x7f04ac119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b0af8700 1 -- 172.21.15.33:0/3959567976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f049c005040 con 0x7f04ac071f30 2024-04-25T01:53:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.210+0000 7f04b0af8700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac071f30 0x7f04ac11a480 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f049c005010 tx=0x7f049c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.211+0000 7f04a1ffb700 1 -- 172.21.15.33:0/3959567976 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f049c00e040 con 0x7f04ac071f30 2024-04-25T01:53:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.211+0000 7f04a1ffb700 1 -- 172.21.15.33:0/3959567976 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f049c013420 con 0x7f04ac071f30 2024-04-25T01:53:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.211+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ac114330 con 0x7f04ac071f30 2024-04-25T01:53:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.211+0000 7f04a1ffb700 1 -- 172.21.15.33:0/3959567976 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f049c009920 con 0x7f04ac071f30 2024-04-25T01:53:27.213 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.211+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ac114870 con 0x7f04ac071f30 2024-04-25T01:53:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.212+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ac071a40 con 0x7f04ac071f30 2024-04-25T01:53:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.212+0000 7f04a1ffb700 1 -- 172.21.15.33:0/3959567976 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f049c013590 con 0x7f04ac071f30 2024-04-25T01:53:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.213+0000 7f04a1ffb700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f049806e140 0x7f04980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.213+0000 7f04a1ffb700 1 -- 172.21.15.33:0/3959567976 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f049c08ed60 con 0x7f04ac071f30 2024-04-25T01:53:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.213+0000 7f04b12f9700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f049806e140 0x7f04980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:27.214 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.213+0000 7f04b12f9700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f049806e140 0x7f04980705f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009b80 tx=0x7f04a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:27.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.218+0000 7f04a1ffb700 1 -- 172.21.15.33:0/3959567976 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f049c05bc90 con 0x7f04ac071f30 2024-04-25T01:53:27.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.448+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04ac110570 con 0x7f049806e140 2024-04-25T01:53:27.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.449+0000 7f04a1ffb700 1 -- 172.21.15.33:0/3959567976 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f04ac110570 con 0x7f049806e140 2024-04-25T01:53:27.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f049806e140 msgr2=0x7f04980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f049806e140 0x7f04980705f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009b80 tx=0x7f04a800b040 comp rx=0 tx=0).stop 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac071f30 msgr2=0x7f04ac11a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac071f30 0x7f04ac11a480 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f049c005010 tx=0x7f049c009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 shutdown_connections 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f049806e140 0x7f04980705f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f04ac070840 0x7f04ac119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 --2- 172.21.15.33:0/3959567976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f04ac071f30 0x7f04ac11a480 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 >> 172.21.15.33:0/3959567976 conn(0x7f04ac0f8990 msgr2=0x7f04ac0f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 shutdown_connections 2024-04-25T01:53:27.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.452+0000 7f04b355d700 1 -- 172.21.15.33:0/3959567976 wait complete. 2024-04-25T01:53:27.462 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:53:27.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.556+0000 7fc88adb4700 1 -- 172.21.15.33:0/118423254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884103760 msgr2=0x7fc8840fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.556+0000 7fc88adb4700 1 --2- 172.21.15.33:0/118423254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884103760 0x7fc8840fcdd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fc874009ce0 tx=0x7fc874004d10 comp rx=0 tx=0).stop 2024-04-25T01:53:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 -- 172.21.15.33:0/118423254 shutdown_connections 2024-04-25T01:53:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 --2- 172.21.15.33:0/118423254 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884103760 0x7fc8840fcdd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 --2- 172.21.15.33:0/118423254 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc884102dc0 0x7fc884103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 -- 172.21.15.33:0/118423254 >> 172.21.15.33:0/118423254 conn(0x7fc8840f8930 msgr2=0x7fc8840fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 -- 172.21.15.33:0/118423254 shutdown_connections 2024-04-25T01:53:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 -- 172.21.15.33:0/118423254 wait complete. 2024-04-25T01:53:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 Processor -- start 2024-04-25T01:53:27.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 -- start start 2024-04-25T01:53:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884102dc0 0x7fc884070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc884103760 0x7fc8840707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8840721b0 con 0x7fc884102dc0 2024-04-25T01:53:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.557+0000 7fc88adb4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884070ce0 con 0x7fc884103760 2024-04-25T01:53:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc888b50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884102dc0 0x7fc884070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc888b50700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884102dc0 0x7fc884070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54984/0 (socket says 172.21.15.33:54984) 2024-04-25T01:53:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc888b50700 1 -- 172.21.15.33:0/2611399488 learned_addr learned my addr 172.21.15.33:0/2611399488 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:53:27.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc888b50700 1 -- 172.21.15.33:0/2611399488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc884103760 msgr2=0x7fc8840707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc883fff700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc884103760 0x7fc8840707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc888b50700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc884103760 0x7fc8840707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc888b50700 1 -- 172.21.15.33:0/2611399488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874009940 con 0x7fc884102dc0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc883fff700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc884103760 0x7fc8840707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc888b50700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884102dc0 0x7fc884070260 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc86c002a50 tx=0x7fc86c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc8817fa700 1 -- 172.21.15.33:0/2611399488 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc86c007cc0 con 0x7fc884102dc0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc884070fc0 con 0x7fc884102dc0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc8817fa700 1 -- 172.21.15.33:0/2611399488 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc86c00f040 con 0x7fc884102dc0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc8817fa700 1 -- 172.21.15.33:0/2611399488 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc86c009640 con 0x7fc884102dc0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.558+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88411a2f0 con 0x7fc884102dc0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.559+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8840fe510 con 0x7fc884102dc0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.560+0000 7fc8817fa700 1 -- 172.21.15.33:0/2611399488 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc86c0097a0 con 0x7fc884102dc0 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.560+0000 7fc8817fa700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc87006e140 0x7fc8700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:27.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.560+0000 7fc8817fa700 1 -- 172.21.15.33:0/2611399488 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc86c08d920 con 0x7fc884102dc0 2024-04-25T01:53:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.560+0000 7fc883fff700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc87006e140 0x7fc8700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:27.562 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.561+0000 7fc883fff700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc87006e140 0x7fc8700705f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b740 tx=0x7fc874004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:27.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.565+0000 7fc8817fa700 1 -- 172.21.15.33:0/2611399488 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc86c056ca0 con 0x7fc884102dc0 2024-04-25T01:53:27.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.793+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8840fa070 con 0x7fc87006e140 2024-04-25T01:53:27.794 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.794+0000 7fc8817fa700 1 -- 172.21.15.33:0/2611399488 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc8840fa070 con 0x7fc87006e140 2024-04-25T01:53:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc87006e140 msgr2=0x7fc8700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc87006e140 0x7fc8700705f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc87400b740 tx=0x7fc874004d10 comp rx=0 tx=0).stop 2024-04-25T01:53:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884102dc0 msgr2=0x7fc884070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884102dc0 0x7fc884070260 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc86c002a50 tx=0x7fc86c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 shutdown_connections 2024-04-25T01:53:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc87006e140 0x7fc8700705f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc884102dc0 0x7fc884070260 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 --2- 172.21.15.33:0/2611399488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc884103760 0x7fc8840707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 >> 172.21.15.33:0/2611399488 conn(0x7fc8840f8930 msgr2=0x7fc8840f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 shutdown_connections 2024-04-25T01:53:27.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.797+0000 7fc88adb4700 1 -- 172.21.15.33:0/2611399488 wait complete. 2024-04-25T01:53:27.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.901+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/994222862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c102720 msgr2=0x7fdf9c0fbd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.901+0000 7fdfa0ea7700 1 --2- 172.21.15.33:0/994222862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c102720 0x7fdf9c0fbd90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdf84004940 tx=0x7fdf84009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/994222862 shutdown_connections 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 --2- 172.21.15.33:0/994222862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c102720 0x7fdf9c0fbd90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 --2- 172.21.15.33:0/994222862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf9c101d80 0x7fdf9c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/994222862 >> 172.21.15.33:0/994222862 conn(0x7fdf9c0f7890 msgr2=0x7fdf9c0f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/994222862 shutdown_connections 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/994222862 wait complete. 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 Processor -- start 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 -- start start 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c101d80 0x7fdf9c10ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf9c102720 0x7fdf9c10efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c10f510 con 0x7fdf9c101d80 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdfa0ea7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c10f650 con 0x7fdf9c102720 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdf9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c101d80 0x7fdf9c10ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:27.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.902+0000 7fdf9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c101d80 0x7fdf9c10ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:54996/0 (socket says 172.21.15.33:54996) 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdf9a59c700 1 -- 172.21.15.33:0/3367795099 learned_addr learned my addr 172.21.15.33:0/3367795099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdf99d9b700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf9c102720 0x7fdf9c10efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdf9a59c700 1 -- 172.21.15.33:0/3367795099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf9c102720 msgr2=0x7fdf9c10efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdf9a59c700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf9c102720 0x7fdf9c10efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdf9a59c700 1 -- 172.21.15.33:0/3367795099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84005040 con 0x7fdf9c101d80 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdf9a59c700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c101d80 0x7fdf9c10ea90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00eb40 tx=0x7fdf8c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdf92ffd700 1 -- 172.21.15.33:0/3367795099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf8c007cc0 con 0x7fdf9c101d80 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c10f8d0 con 0x7fdf9c101d80 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdf92ffd700 1 -- 172.21.15.33:0/3367795099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf8c00f040 con 0x7fdf9c101d80 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdf92ffd700 1 -- 172.21.15.33:0/3367795099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf8c00ca10 con 0x7fdf9c101d80 2024-04-25T01:53:27.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.903+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c117dd0 con 0x7fdf9c101d80 2024-04-25T01:53:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.904+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c104250 con 0x7fdf9c101d80 2024-04-25T01:53:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.905+0000 7fdf92ffd700 1 -- 172.21.15.33:0/3367795099 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdf8c00cc80 con 0x7fdf9c101d80 2024-04-25T01:53:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.905+0000 7fdf92ffd700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf8806e140 0x7fdf880705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.905+0000 7fdf92ffd700 1 -- 172.21.15.33:0/3367795099 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdf8c08d6f0 con 0x7fdf9c101d80 2024-04-25T01:53:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.905+0000 7fdf99d9b700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf8806e140 0x7fdf880705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:27.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.906+0000 7fdf99d9b700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf8806e140 0x7fdf880705f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdf84004890 tx=0x7fdf84009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:27.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:27.910+0000 7fdf92ffd700 1 -- 172.21.15.33:0/3367795099 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdf8c056b20 con 0x7fdf9c101d80 2024-04-25T01:53:28.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:27 smithi138 ceph-mon[81304]: pgmap v421: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.128+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf9c108ed0 con 0x7fdf8806e140 2024-04-25T01:53:28.128 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:27 smithi033 ceph-mon[111306]: pgmap v421: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:28.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.139+0000 7fdf92ffd700 1 -- 172.21.15.33:0/3367795099 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdf9c108ed0 con 0x7fdf8806e140 2024-04-25T01:53:28.138 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:53:28.138 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (14m) 7m ago 25m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:53:28.138 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (11m) 7m ago 26m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:53:28.138 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (11m) 7m ago 25m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:53:28.138 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (14m) 7m ago 25m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:53:28.138 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (7m) 7m ago 20m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (8m) 7m ago 20m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (7m) 7m ago 20m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (20m) 7m ago 20m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (16m) 7m ago 27m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (15m) 7m ago 24m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (12m) 7m ago 27m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (12m) 7m ago 24m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (15m) 7m ago 25m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (15m) 7m ago 24m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (11m) 7m ago 24m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (10m) 7m ago 23m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (10m) 7m ago 23m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (9m) 7m ago 22m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (9m) 7m ago 22m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (8m) 7m ago 21m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:53:28.139 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (14m) 7m ago 25m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:53:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf8806e140 msgr2=0x7fdf880705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf8806e140 0x7fdf880705f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdf84004890 tx=0x7fdf84009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c101d80 msgr2=0x7fdf9c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c101d80 0x7fdf9c10ea90 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c00eb40 tx=0x7fdf8c00b040 comp rx=0 tx=0).stop 2024-04-25T01:53:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 shutdown_connections 2024-04-25T01:53:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf8806e140 0x7fdf880705f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf9c101d80 0x7fdf9c10ea90 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 --2- 172.21.15.33:0/3367795099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf9c102720 0x7fdf9c10efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 >> 172.21.15.33:0/3367795099 conn(0x7fdf9c0f7890 msgr2=0x7fdf9c0f95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 shutdown_connections 2024-04-25T01:53:28.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.141+0000 7fdfa0ea7700 1 -- 172.21.15.33:0/3367795099 wait complete. 2024-04-25T01:53:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.244+0000 7f1445193700 1 -- 172.21.15.33:0/698677956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 msgr2=0x7f1440109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.244+0000 7f1445193700 1 --2- 172.21.15.33:0/698677956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 0x7f1440109350 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1434004940 tx=0x7f1434009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.244+0000 7f1445193700 1 -- 172.21.15.33:0/698677956 shutdown_connections 2024-04-25T01:53:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.244+0000 7f1445193700 1 --2- 172.21.15.33:0/698677956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 0x7f1440109350 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.244+0000 7f1445193700 1 --2- 172.21.15.33:0/698677956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14400fc790 0x7f14400fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.244+0000 7f1445193700 1 -- 172.21.15.33:0/698677956 >> 172.21.15.33:0/698677956 conn(0x7f1440071d00 msgr2=0x7f1440072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:28.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.244+0000 7f1445193700 1 -- 172.21.15.33:0/698677956 shutdown_connections 2024-04-25T01:53:28.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.244+0000 7f1445193700 1 -- 172.21.15.33:0/698677956 wait complete. 2024-04-25T01:53:28.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.245+0000 7f1445193700 1 Processor -- start 2024-04-25T01:53:28.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.245+0000 7f1445193700 1 -- start start 2024-04-25T01:53:28.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.245+0000 7f1445193700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14400fc790 0x7f1440115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.245+0000 7f1445193700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 0x7f14401162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.245+0000 7f1445193700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1440116930 con 0x7f14400fd0a0 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.245+0000 7f1445193700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144010fd00 con 0x7f14400fc790 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.245+0000 7f143e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 0x7f14401162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f143e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 0x7f14401162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55012/0 (socket says 172.21.15.33:55012) 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f143e59c700 1 -- 172.21.15.33:0/3216702943 learned_addr learned my addr 172.21.15.33:0/3216702943 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f143ed9d700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14400fc790 0x7f1440115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f143e59c700 1 -- 172.21.15.33:0/3216702943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14400fc790 msgr2=0x7f1440115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f143e59c700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14400fc790 0x7f1440115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f143e59c700 1 -- 172.21.15.33:0/3216702943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434005040 con 0x7f14400fd0a0 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f143ed9d700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14400fc790 0x7f1440115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f143e59c700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 0x7f14401162e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1434009b30 tx=0x7f1434009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f14277fe700 1 -- 172.21.15.33:0/3216702943 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f143400e070 con 0x7f14400fd0a0 2024-04-25T01:53:28.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f14277fe700 1 -- 172.21.15.33:0/3216702943 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1434013cd0 con 0x7f14400fd0a0 2024-04-25T01:53:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144010ff80 con 0x7f14400fd0a0 2024-04-25T01:53:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f14277fe700 1 -- 172.21.15.33:0/3216702943 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f143400b720 con 0x7f14400fd0a0 2024-04-25T01:53:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.246+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1440110460 con 0x7f14400fd0a0 2024-04-25T01:53:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.247+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14400fc330 con 0x7f14400fd0a0 2024-04-25T01:53:28.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.248+0000 7f14277fe700 1 -- 172.21.15.33:0/3216702943 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f143401cab0 con 0x7f14400fd0a0 2024-04-25T01:53:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.248+0000 7f14277fe700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f142806e140 0x7f14280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.248+0000 7f14277fe700 1 -- 172.21.15.33:0/3216702943 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f143408f430 con 0x7f14400fd0a0 2024-04-25T01:53:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.248+0000 7f143ed9d700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f142806e140 0x7f14280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:28.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.249+0000 7f143ed9d700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f142806e140 0x7f14280705f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1430004600 tx=0x7f143000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:28.252 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.253+0000 7f14277fe700 1 -- 172.21.15.33:0/3216702943 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f143405c360 con 0x7f14400fd0a0 2024-04-25T01:53:28.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.552+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f144004bed0 con 0x7f14400fd0a0 2024-04-25T01:53:28.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.553+0000 7f14277fe700 1 -- 172.21.15.33:0/3216702943 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f143405bef0 con 0x7f14400fd0a0 2024-04-25T01:53:28.553 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:53:28.553 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:53:28.554 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:53:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:53:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:53:28.555 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:53:28.555 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:53:28.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f142806e140 msgr2=0x7f14280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f142806e140 0x7f14280705f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1430004600 tx=0x7f143000b040 comp rx=0 tx=0).stop 2024-04-25T01:53:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 msgr2=0x7f14401162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 0x7f14401162e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1434009b30 tx=0x7f1434009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 shutdown_connections 2024-04-25T01:53:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f142806e140 0x7f14280705f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f14400fc790 0x7f1440115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 --2- 172.21.15.33:0/3216702943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f14400fd0a0 0x7f14401162e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 >> 172.21.15.33:0/3216702943 conn(0x7f1440071d00 msgr2=0x7f14400f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 shutdown_connections 2024-04-25T01:53:28.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.555+0000 7f1445193700 1 -- 172.21.15.33:0/3216702943 wait complete. 2024-04-25T01:53:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.660+0000 7f84e1876700 1 -- 172.21.15.33:0/1356254361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 msgr2=0x7f84dc0fe1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.660+0000 7f84e1876700 1 --2- 172.21.15.33:0/1356254361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 0x7f84dc0fe1c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f84cc004940 tx=0x7f84cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.660+0000 7f84e1876700 1 -- 172.21.15.33:0/1356254361 shutdown_connections 2024-04-25T01:53:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.660+0000 7f84e1876700 1 --2- 172.21.15.33:0/1356254361 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84dc0fe790 0x7f84dc106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.660+0000 7f84e1876700 1 --2- 172.21.15.33:0/1356254361 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 0x7f84dc0fe1c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.660+0000 7f84e1876700 1 -- 172.21.15.33:0/1356254361 >> 172.21.15.33:0/1356254361 conn(0x7f84dc0f78a0 msgr2=0x7f84dc0f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.660+0000 7f84e1876700 1 -- 172.21.15.33:0/1356254361 shutdown_connections 2024-04-25T01:53:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.660+0000 7f84e1876700 1 -- 172.21.15.33:0/1356254361 wait complete. 2024-04-25T01:53:28.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84e1876700 1 Processor -- start 2024-04-25T01:53:28.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84e1876700 1 -- start start 2024-04-25T01:53:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84e1876700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 0x7f84dc113c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84e1876700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84dc0fe790 0x7f84dc114170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84e1876700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc1147c0 con 0x7f84dc0fddf0 2024-04-25T01:53:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84e1876700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84dc10db90 con 0x7f84dc0fe790 2024-04-25T01:53:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84daffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 0x7f84dc113c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84daffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 0x7f84dc113c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55038/0 (socket says 172.21.15.33:55038) 2024-04-25T01:53:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84daffd700 1 -- 172.21.15.33:0/1955527096 learned_addr learned my addr 172.21.15.33:0/1955527096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:53:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84da7fc700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84dc0fe790 0x7f84dc114170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:28.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84daffd700 1 -- 172.21.15.33:0/1955527096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84dc0fe790 msgr2=0x7f84dc114170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84daffd700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84dc0fe790 0x7f84dc114170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84daffd700 1 -- 172.21.15.33:0/1955527096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc005040 con 0x7f84dc0fddf0 2024-04-25T01:53:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.661+0000 7f84daffd700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 0x7f84dc113c30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009380 tx=0x7f84cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.662+0000 7f84c37fe700 1 -- 172.21.15.33:0/1955527096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84cc00e040 con 0x7f84dc0fddf0 2024-04-25T01:53:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.662+0000 7f84c37fe700 1 -- 172.21.15.33:0/1955527096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84cc009af0 con 0x7f84dc0fddf0 2024-04-25T01:53:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.662+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84dc10de10 con 0x7f84dc0fddf0 2024-04-25T01:53:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.662+0000 7f84c37fe700 1 -- 172.21.15.33:0/1955527096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84cc01c3f0 con 0x7f84dc0fddf0 2024-04-25T01:53:28.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.662+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84dc10e350 con 0x7f84dc0fddf0 2024-04-25T01:53:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.663+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84dc066750 con 0x7f84dc0fddf0 2024-04-25T01:53:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.663+0000 7f84c37fe700 1 -- 172.21.15.33:0/1955527096 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f84cc013420 con 0x7f84dc0fddf0 2024-04-25T01:53:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.664+0000 7f84c37fe700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84c4072550 0x7f84c4074a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.664+0000 7f84c37fe700 1 -- 172.21.15.33:0/1955527096 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f84cc08ee80 con 0x7f84dc0fddf0 2024-04-25T01:53:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.664+0000 7f84da7fc700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84c4072550 0x7f84c4074a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:28.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.664+0000 7f84da7fc700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84c4072550 0x7f84c4074a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f84dc10f4d0 tx=0x7f84d000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:28.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.669+0000 7f84c37fe700 1 -- 172.21.15.33:0/1955527096 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f84cc05bd30 con 0x7f84dc0fddf0 2024-04-25T01:53:28.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.924+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f84dc04bed0 con 0x7f84dc0fddf0 2024-04-25T01:53:28.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:28 smithi033 ceph-mon[111306]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:53:28.925 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:28 smithi033 ceph-mon[111306]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:53:28.926 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:28 smithi033 ceph-mon[111306]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:53:28.926 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:28 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3216702943' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:53:28.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.925+0000 7f84c37fe700 1 -- 172.21.15.33:0/1955527096 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f84cc05b8c0 con 0x7f84dc0fddf0 2024-04-25T01:53:28.926 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:53:28.926 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:53:28.926 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:53:28.927 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:53:28.928 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:53:28.929 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:53:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84c4072550 msgr2=0x7f84c4074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84c4072550 0x7f84c4074a00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f84dc10f4d0 tx=0x7f84d000b040 comp rx=0 tx=0).stop 2024-04-25T01:53:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 msgr2=0x7f84dc113c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 0x7f84dc113c30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009380 tx=0x7f84cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 shutdown_connections 2024-04-25T01:53:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f84c4072550 0x7f84c4074a00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f84dc0fddf0 0x7f84dc113c30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 --2- 172.21.15.33:0/1955527096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f84dc0fe790 0x7f84dc114170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 >> 172.21.15.33:0/1955527096 conn(0x7f84dc0f78a0 msgr2=0x7f84dc0f8ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 shutdown_connections 2024-04-25T01:53:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:28.928+0000 7f84e1876700 1 -- 172.21.15.33:0/1955527096 wait complete. 2024-04-25T01:53:28.931 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.033+0000 7f3055bad700 1 -- 172.21.15.33:0/672457913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050103790 msgr2=0x7f30500fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.033+0000 7f3055bad700 1 --2- 172.21.15.33:0/672457913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050103790 0x7f30500fce00 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3044004940 tx=0x7f3044009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.033+0000 7f3055bad700 1 -- 172.21.15.33:0/672457913 shutdown_connections 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.033+0000 7f3055bad700 1 --2- 172.21.15.33:0/672457913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050103790 0x7f30500fce00 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.033+0000 7f3055bad700 1 --2- 172.21.15.33:0/672457913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3050102df0 0x7f30501031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.033+0000 7f3055bad700 1 -- 172.21.15.33:0/672457913 >> 172.21.15.33:0/672457913 conn(0x7f30500f8980 msgr2=0x7f30500fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.033+0000 7f3055bad700 1 -- 172.21.15.33:0/672457913 shutdown_connections 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.033+0000 7f3055bad700 1 -- 172.21.15.33:0/672457913 wait complete. 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f3055bad700 1 Processor -- start 2024-04-25T01:53:29.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f3055bad700 1 -- start start 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f3055bad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050102df0 0x7f3050113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f3055bad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3050103790 0x7f3050114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f3055bad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3050114750 con 0x7f3050102df0 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f3055bad700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305010dab0 con 0x7f3050103790 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f304f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050102df0 0x7f3050113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f304f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050102df0 0x7f3050113b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55052/0 (socket says 172.21.15.33:55052) 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f304f7fe700 1 -- 172.21.15.33:0/1081803445 learned_addr learned my addr 172.21.15.33:0/1081803445 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.034+0000 7f304effd700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3050103790 0x7f3050114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.035+0000 7f304f7fe700 1 -- 172.21.15.33:0/1081803445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3050103790 msgr2=0x7f3050114070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.035+0000 7f304f7fe700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3050103790 0x7f3050114070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.035+0000 7f304f7fe700 1 -- 172.21.15.33:0/1081803445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3044005040 con 0x7f3050102df0 2024-04-25T01:53:29.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.035+0000 7f304f7fe700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050102df0 0x7f3050113b30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f304000b790 tx=0x7f3040009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.035+0000 7f3037fff700 1 -- 172.21.15.33:0/1081803445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304000f6f0 con 0x7f3050102df0 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.035+0000 7f3037fff700 1 -- 172.21.15.33:0/1081803445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f304000fd20 con 0x7f3050102df0 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.035+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305010dd90 con 0x7f3050102df0 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.035+0000 7f3037fff700 1 -- 172.21.15.33:0/1081803445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30400103a0 con 0x7f3050102df0 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.035+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f305010e320 con 0x7f3050102df0 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.036+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3050100100 con 0x7f3050102df0 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.037+0000 7f3037fff700 1 -- 172.21.15.33:0/1081803445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3040011400 con 0x7f3050102df0 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.037+0000 7f3037fff700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f303806e140 0x7f30380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.037+0000 7f3037fff700 1 -- 172.21.15.33:0/1081803445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f304008d960 con 0x7f3050102df0 2024-04-25T01:53:29.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.037+0000 7f304effd700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f303806e140 0x7f30380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:29.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.038+0000 7f304effd700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f303806e140 0x7f30380705f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f305010ed90 tx=0x7f3044009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:29.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:28 smithi138 ceph-mon[81304]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:53:29.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:28 smithi138 ceph-mon[81304]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:53:29.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:28 smithi138 ceph-mon[81304]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:53:29.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:28 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3216702943' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:53:29.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.042+0000 7f3037fff700 1 -- 172.21.15.33:0/1081803445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f304005a890 con 0x7f3050102df0 2024-04-25T01:53:29.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.270+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f305010f180 con 0x7f303806e140 2024-04-25T01:53:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.271+0000 7f3037fff700 1 -- 172.21.15.33:0/1081803445 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f305010f180 con 0x7f303806e140 2024-04-25T01:53:29.271 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:53:29.272 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f303806e140 msgr2=0x7f30380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f303806e140 0x7f30380705f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f305010ed90 tx=0x7f3044009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050102df0 msgr2=0x7f3050113b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050102df0 0x7f3050113b30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f304000b790 tx=0x7f3040009250 comp rx=0 tx=0).stop 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 shutdown_connections 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f303806e140 0x7f30380705f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3050102df0 0x7f3050113b30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 --2- 172.21.15.33:0/1081803445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3050103790 0x7f3050114070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 >> 172.21.15.33:0/1081803445 conn(0x7f30500f8980 msgr2=0x7f30501073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 shutdown_connections 2024-04-25T01:53:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.273+0000 7f3055bad700 1 -- 172.21.15.33:0/1081803445 wait complete. 2024-04-25T01:53:29.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.377+0000 7fc93722f700 1 -- 172.21.15.33:0/3305544876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc930101d90 msgr2=0x7fc930102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:29.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.377+0000 7fc93722f700 1 --2- 172.21.15.33:0/3305544876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc930101d90 0x7fc930102160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc924004940 tx=0x7fc924009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:29.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.377+0000 7fc93722f700 1 -- 172.21.15.33:0/3305544876 shutdown_connections 2024-04-25T01:53:29.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.377+0000 7fc93722f700 1 --2- 172.21.15.33:0/3305544876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc9300fbd40 0x7fc9300fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.377+0000 7fc93722f700 1 --2- 172.21.15.33:0/3305544876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc930101d90 0x7fc930102160 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.377+0000 7fc93722f700 1 -- 172.21.15.33:0/3305544876 >> 172.21.15.33:0/3305544876 conn(0x7fc930071d00 msgr2=0x7fc930072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:29.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.377+0000 7fc93722f700 1 -- 172.21.15.33:0/3305544876 shutdown_connections 2024-04-25T01:53:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.377+0000 7fc93722f700 1 -- 172.21.15.33:0/3305544876 wait complete. 2024-04-25T01:53:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.378+0000 7fc93722f700 1 Processor -- start 2024-04-25T01:53:29.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.378+0000 7fc93722f700 1 -- start start 2024-04-25T01:53:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.378+0000 7fc93722f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9300fbd40 0x7fc93010eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.378+0000 7fc93722f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930101d90 0x7fc93010f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.378+0000 7fc93722f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93010f6d0 con 0x7fc9300fbd40 2024-04-25T01:53:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.378+0000 7fc93722f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93010f810 con 0x7fc930101d90 2024-04-25T01:53:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.378+0000 7fc934fcb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9300fbd40 0x7fc93010eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.378+0000 7fc934fcb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9300fbd40 0x7fc93010eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55070/0 (socket says 172.21.15.33:55070) 2024-04-25T01:53:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.378+0000 7fc934fcb700 1 -- 172.21.15.33:0/1034328309 learned_addr learned my addr 172.21.15.33:0/1034328309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:53:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc92ffff700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930101d90 0x7fc93010f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:29.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc934fcb700 1 -- 172.21.15.33:0/1034328309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930101d90 msgr2=0x7fc93010f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:29.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc934fcb700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930101d90 0x7fc93010f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc934fcb700 1 -- 172.21.15.33:0/1034328309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc924005040 con 0x7fc9300fbd40 2024-04-25T01:53:29.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc934fcb700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9300fbd40 0x7fc93010eb40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc92400b3b0 tx=0x7fc924009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:29.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc92d7fa700 1 -- 172.21.15.33:0/1034328309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc92400e040 con 0x7fc9300fbd40 2024-04-25T01:53:29.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc930117bb0 con 0x7fc9300fbd40 2024-04-25T01:53:29.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc92d7fa700 1 -- 172.21.15.33:0/1034328309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc924013420 con 0x7fc9300fbd40 2024-04-25T01:53:29.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc92d7fa700 1 -- 172.21.15.33:0/1034328309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc92401c630 con 0x7fc9300fbd40 2024-04-25T01:53:29.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.379+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9301180c0 con 0x7fc9300fbd40 2024-04-25T01:53:29.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.380+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9300fd430 con 0x7fc9300fbd40 2024-04-25T01:53:29.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.381+0000 7fc92d7fa700 1 -- 172.21.15.33:0/1034328309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc924013590 con 0x7fc9300fbd40 2024-04-25T01:53:29.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.381+0000 7fc92d7fa700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc92006e140 0x7fc9200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:29.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.381+0000 7fc92d7fa700 1 -- 172.21.15.33:0/1034328309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc92408ed80 con 0x7fc9300fbd40 2024-04-25T01:53:29.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.381+0000 7fc92ffff700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc92006e140 0x7fc9200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:29.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.382+0000 7fc92ffff700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc92006e140 0x7fc9200705f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc930065fe0 tx=0x7fc91c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:29.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.386+0000 7fc92d7fa700 1 -- 172.21.15.33:0/1034328309 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc924058180 con 0x7fc9300fbd40 2024-04-25T01:53:29.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.677+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc93005f5c0 con 0x7fc9300fbd40 2024-04-25T01:53:29.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.678+0000 7fc92d7fa700 1 -- 172.21.15.33:0/1034328309 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fc92405b840 con 0x7fc9300fbd40 2024-04-25T01:53:29.678 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:53:29.679 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:53:29.679 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:53:29.679 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:53:29.679 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:53:29.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc92006e140 msgr2=0x7fc9200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:29.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc92006e140 0x7fc9200705f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc930065fe0 tx=0x7fc91c00b040 comp rx=0 tx=0).stop 2024-04-25T01:53:29.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9300fbd40 msgr2=0x7fc93010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9300fbd40 0x7fc93010eb40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc92400b3b0 tx=0x7fc924009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 shutdown_connections 2024-04-25T01:53:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc92006e140 0x7fc9200705f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc9300fbd40 0x7fc93010eb40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 --2- 172.21.15.33:0/1034328309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc930101d90 0x7fc93010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 >> 172.21.15.33:0/1034328309 conn(0x7fc930071d00 msgr2=0x7fc9300fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 shutdown_connections 2024-04-25T01:53:29.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:29.680+0000 7fc93722f700 1 -- 172.21.15.33:0/1034328309 wait complete. 2024-04-25T01:53:30.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:29 smithi033 ceph-mon[111306]: pgmap v422: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:30.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:29 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1955527096' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:53:30.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:29 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1034328309' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:53:30.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:29 smithi138 ceph-mon[81304]: pgmap v422: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:30.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:29 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1955527096' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:53:30.042 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:29 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1034328309' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:53:31.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:30 smithi138 ceph-mon[81304]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:53:31.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:30 smithi033 ceph-mon[111306]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:53:32.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:31 smithi138 ceph-mon[81304]: pgmap v423: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:32.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:31 smithi033 ceph-mon[111306]: pgmap v423: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:34.041 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:33 smithi138 ceph-mon[81304]: pgmap v424: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:34.068 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:33 smithi033 ceph-mon[111306]: pgmap v424: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:36.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:35 smithi033 ceph-mon[111306]: pgmap v425: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:36.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:35 smithi138 ceph-mon[81304]: pgmap v425: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:38.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:37 smithi033 ceph-mon[111306]: pgmap v426: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:38.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:37 smithi138 ceph-mon[81304]: pgmap v426: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:40.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:39 smithi033 ceph-mon[111306]: pgmap v427: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:40.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:39 smithi138 ceph-mon[81304]: pgmap v427: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:41.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:53:41.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:53:42.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:41 smithi033 ceph-mon[111306]: pgmap v428: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:42.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:41 smithi138 ceph-mon[81304]: pgmap v428: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:44.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:43 smithi033 ceph-mon[111306]: pgmap v429: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:44.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:43 smithi138 ceph-mon[81304]: pgmap v429: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:46.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:45 smithi033 ceph-mon[111306]: pgmap v430: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:46.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:45 smithi138 ceph-mon[81304]: pgmap v430: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:48.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:47 smithi033 ceph-mon[111306]: pgmap v431: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:48.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:47 smithi138 ceph-mon[81304]: pgmap v431: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:50.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:49 smithi033 ceph-mon[111306]: pgmap v432: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:50.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:49 smithi138 ceph-mon[81304]: pgmap v432: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:52.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:51 smithi033 ceph-mon[111306]: pgmap v433: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:52.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:51 smithi138 ceph-mon[81304]: pgmap v433: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:54.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:53 smithi033 ceph-mon[111306]: pgmap v434: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:54.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:53 smithi138 ceph-mon[81304]: pgmap v434: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:56.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:55 smithi033 ceph-mon[111306]: pgmap v435: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:56.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:53:56.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:55 smithi138 ceph-mon[81304]: pgmap v435: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:56.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:53:58.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:57 smithi033 ceph-mon[111306]: pgmap v436: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:58.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:57 smithi138 ceph-mon[81304]: pgmap v436: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:53:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.800+0000 7fb8a251b700 1 -- 172.21.15.33:0/3825192205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 msgr2=0x7fb89c109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.800+0000 7fb8a251b700 1 --2- 172.21.15.33:0/3825192205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 0x7fb89c109170 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fb890004940 tx=0x7fb890009e30 comp rx=0 tx=0).stop 2024-04-25T01:53:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.800+0000 7fb8a251b700 1 -- 172.21.15.33:0/3825192205 shutdown_connections 2024-04-25T01:53:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.800+0000 7fb8a251b700 1 --2- 172.21.15.33:0/3825192205 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 0x7fb89c109170 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.800+0000 7fb8a251b700 1 --2- 172.21.15.33:0/3825192205 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c070280 0x7fb89c070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.800+0000 7fb8a251b700 1 -- 172.21.15.33:0/3825192205 >> 172.21.15.33:0/3825192205 conn(0x7fb89c0f8990 msgr2=0x7fb89c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:53:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.800+0000 7fb8a251b700 1 -- 172.21.15.33:0/3825192205 shutdown_connections 2024-04-25T01:53:59.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.800+0000 7fb8a251b700 1 -- 172.21.15.33:0/3825192205 wait complete. 2024-04-25T01:53:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.800+0000 7fb8a251b700 1 Processor -- start 2024-04-25T01:53:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb8a251b700 1 -- start start 2024-04-25T01:53:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb8a251b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c070280 0x7fb89c115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb8a251b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 0x7fb89c1160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb8a251b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1166f0 con 0x7fb89c070b90 2024-04-25T01:53:59.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb8a251b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c10fcd0 con 0x7fb89c070280 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb89b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 0x7fb89c1160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb89b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 0x7fb89c1160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56836/0 (socket says 172.21.15.33:56836) 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb89b7fe700 1 -- 172.21.15.33:0/440869907 learned_addr learned my addr 172.21.15.33:0/440869907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb89bfff700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c070280 0x7fb89c115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb89b7fe700 1 -- 172.21.15.33:0/440869907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c070280 msgr2=0x7fb89c115b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb89b7fe700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c070280 0x7fb89c115b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb89b7fe700 1 -- 172.21.15.33:0/440869907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb890005040 con 0x7fb89c070b90 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.801+0000 7fb89bfff700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c070280 0x7fb89c115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.802+0000 7fb89b7fe700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 0x7fb89c1160a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb8900091a0 tx=0x7fb890009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.802+0000 7fb898ff9700 1 -- 172.21.15.33:0/440869907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89000e030 con 0x7fb89c070b90 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.802+0000 7fb898ff9700 1 -- 172.21.15.33:0/440869907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb890004290 con 0x7fb89c070b90 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.802+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c10ff50 con 0x7fb89c070b90 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.802+0000 7fb898ff9700 1 -- 172.21.15.33:0/440869907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89000b650 con 0x7fb89c070b90 2024-04-25T01:53:59.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.802+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c110490 con 0x7fb89c070b90 2024-04-25T01:53:59.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.803+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c0719c0 con 0x7fb89c070b90 2024-04-25T01:53:59.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.804+0000 7fb898ff9700 1 -- 172.21.15.33:0/440869907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb890013cd0 con 0x7fb89c070b90 2024-04-25T01:53:59.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.804+0000 7fb898ff9700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb88406e140 0x7fb8840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:53:59.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.804+0000 7fb898ff9700 1 -- 172.21.15.33:0/440869907 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb89008f790 con 0x7fb89c070b90 2024-04-25T01:53:59.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.804+0000 7fb89bfff700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb88406e140 0x7fb8840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:53:59.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.805+0000 7fb89bfff700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb88406e140 0x7fb8840705f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb88c004600 tx=0x7fb88c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:53:59.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:53:59.809+0000 7fb898ff9700 1 -- 172.21.15.33:0/440869907 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb89005c6c0 con 0x7fb89c070b90 2024-04-25T01:54:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.037+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb89c10c130 con 0x7fb88406e140 2024-04-25T01:54:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.038+0000 7fb898ff9700 1 -- 172.21.15.33:0/440869907 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb89c10c130 con 0x7fb88406e140 2024-04-25T01:54:00.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.040+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb88406e140 msgr2=0x7fb8840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.040+0000 7fb8a251b700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb88406e140 0x7fb8840705f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb88c004600 tx=0x7fb88c00b040 comp rx=0 tx=0).stop 2024-04-25T01:54:00.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.040+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 msgr2=0x7fb89c1160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.040+0000 7fb8a251b700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 0x7fb89c1160a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fb8900091a0 tx=0x7fb890009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.040+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 shutdown_connections 2024-04-25T01:54:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.040+0000 7fb8a251b700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb88406e140 0x7fb8840705f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.040+0000 7fb8a251b700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb89c070280 0x7fb89c115b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.040+0000 7fb8a251b700 1 --2- 172.21.15.33:0/440869907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb89c070b90 0x7fb89c1160a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.040+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 >> 172.21.15.33:0/440869907 conn(0x7fb89c0f8990 msgr2=0x7fb89c0f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.041+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 shutdown_connections 2024-04-25T01:54:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.041+0000 7fb8a251b700 1 -- 172.21.15.33:0/440869907 wait complete. 2024-04-25T01:54:00.052 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:54:00.052 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:53:59 smithi033 ceph-mon[111306]: pgmap v437: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:00.094 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:53:59 smithi138 ceph-mon[81304]: pgmap v437: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:00.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.146+0000 7f5d9925e700 1 -- 172.21.15.33:0/1505268113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 msgr2=0x7f5d941020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.146+0000 7f5d9925e700 1 --2- 172.21.15.33:0/1505268113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 0x7f5d941020e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5d84004940 tx=0x7f5d84009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.146+0000 7f5d9925e700 1 -- 172.21.15.33:0/1505268113 shutdown_connections 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.146+0000 7f5d9925e700 1 --2- 172.21.15.33:0/1505268113 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d941026b0 0x7f5d940fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.146+0000 7f5d9925e700 1 --2- 172.21.15.33:0/1505268113 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 0x7f5d941020e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.146+0000 7f5d9925e700 1 -- 172.21.15.33:0/1505268113 >> 172.21.15.33:0/1505268113 conn(0x7f5d940f7820 msgr2=0x7f5d940f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.146+0000 7f5d9925e700 1 -- 172.21.15.33:0/1505268113 shutdown_connections 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.146+0000 7f5d9925e700 1 -- 172.21.15.33:0/1505268113 wait complete. 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.146+0000 7f5d9925e700 1 Processor -- start 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d9925e700 1 -- start start 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d9925e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 0x7f5d94112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:00.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d9925e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d941026b0 0x7f5d94113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d9925e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d94113810 con 0x7f5d94101d10 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d9925e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9410cd60 con 0x7f5d941026b0 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d92d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 0x7f5d94112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d92d9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 0x7f5d94112bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56846/0 (socket says 172.21.15.33:56846) 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d92d9d700 1 -- 172.21.15.33:0/4251250445 learned_addr learned my addr 172.21.15.33:0/4251250445 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d9259c700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d941026b0 0x7f5d94113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d92d9d700 1 -- 172.21.15.33:0/4251250445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d941026b0 msgr2=0x7f5d94113130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d92d9d700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d941026b0 0x7f5d94113130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d92d9d700 1 -- 172.21.15.33:0/4251250445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d84005040 con 0x7f5d94101d10 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d92d9d700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 0x7f5d94112bf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b3b0 tx=0x7f5d84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.147+0000 7f5d7b7fe700 1 -- 172.21.15.33:0/4251250445 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8400e040 con 0x7f5d94101d10 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.148+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9410cf80 con 0x7f5d94101d10 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.148+0000 7f5d7b7fe700 1 -- 172.21.15.33:0/4251250445 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d8400bbc0 con 0x7f5d94101d10 2024-04-25T01:54:00.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.148+0000 7f5d7b7fe700 1 -- 172.21.15.33:0/4251250445 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d8401c3f0 con 0x7f5d94101d10 2024-04-25T01:54:00.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.148+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9410d4c0 con 0x7f5d94101d10 2024-04-25T01:54:00.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.149+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d940fd460 con 0x7f5d94101d10 2024-04-25T01:54:00.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.149+0000 7f5d7b7fe700 1 -- 172.21.15.33:0/4251250445 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5d8400bd30 con 0x7f5d94101d10 2024-04-25T01:54:00.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.150+0000 7f5d7b7fe700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d7c06e140 0x7f5d7c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:00.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.150+0000 7f5d7b7fe700 1 -- 172.21.15.33:0/4251250445 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5d8408edb0 con 0x7f5d94101d10 2024-04-25T01:54:00.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.150+0000 7f5d9259c700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d7c06e140 0x7f5d7c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:00.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.150+0000 7f5d9259c700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d7c06e140 0x7f5d7c0705f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d88006fd0 tx=0x7f5d88008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:00.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.154+0000 7f5d7b7fe700 1 -- 172.21.15.33:0/4251250445 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5d8405bc60 con 0x7f5d94101d10 2024-04-25T01:54:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.382+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d94065f30 con 0x7f5d7c06e140 2024-04-25T01:54:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.383+0000 7f5d7b7fe700 1 -- 172.21.15.33:0/4251250445 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5d94065f30 con 0x7f5d7c06e140 2024-04-25T01:54:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d7c06e140 msgr2=0x7f5d7c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d7c06e140 0x7f5d7c0705f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5d88006fd0 tx=0x7f5d88008040 comp rx=0 tx=0).stop 2024-04-25T01:54:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 msgr2=0x7f5d94112bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 0x7f5d94112bf0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5d8400b3b0 tx=0x7f5d84009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 shutdown_connections 2024-04-25T01:54:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5d7c06e140 0x7f5d7c0705f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5d94101d10 0x7f5d94112bf0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 --2- 172.21.15.33:0/4251250445 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5d941026b0 0x7f5d94113130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 >> 172.21.15.33:0/4251250445 conn(0x7f5d940f7820 msgr2=0x7f5d940f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 shutdown_connections 2024-04-25T01:54:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.385+0000 7f5d9925e700 1 -- 172.21.15.33:0/4251250445 wait complete. 2024-04-25T01:54:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.489+0000 7f385bc29700 1 -- 172.21.15.33:0/3316811677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854102670 msgr2=0x7f38540fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.489+0000 7f385bc29700 1 --2- 172.21.15.33:0/3316811677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854102670 0x7f38540fb600 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3848004940 tx=0x7f3848009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.489+0000 7f385bc29700 1 -- 172.21.15.33:0/3316811677 shutdown_connections 2024-04-25T01:54:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.489+0000 7f385bc29700 1 --2- 172.21.15.33:0/3316811677 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854102670 0x7f38540fb600 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.489+0000 7f385bc29700 1 --2- 172.21.15.33:0/3316811677 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3854101cd0 0x7f38541020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.489+0000 7f385bc29700 1 -- 172.21.15.33:0/3316811677 >> 172.21.15.33:0/3316811677 conn(0x7f3854071460 msgr2=0x7f3854071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.489+0000 7f385bc29700 1 -- 172.21.15.33:0/3316811677 shutdown_connections 2024-04-25T01:54:00.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.489+0000 7f385bc29700 1 -- 172.21.15.33:0/3316811677 wait complete. 2024-04-25T01:54:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f385bc29700 1 Processor -- start 2024-04-25T01:54:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f385bc29700 1 -- start start 2024-04-25T01:54:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f385bc29700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854101cd0 0x7f3854113af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f385bc29700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3854102670 0x7f3854114030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f385bc29700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854114680 con 0x7f3854101cd0 2024-04-25T01:54:00.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f385bc29700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385410da70 con 0x7f3854102670 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f38599c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854101cd0 0x7f3854113af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f38599c5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854101cd0 0x7f3854113af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56856/0 (socket says 172.21.15.33:56856) 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f38599c5700 1 -- 172.21.15.33:0/1702030290 learned_addr learned my addr 172.21.15.33:0/1702030290 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f38591c4700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3854102670 0x7f3854114030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f38599c5700 1 -- 172.21.15.33:0/1702030290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3854102670 msgr2=0x7f3854114030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f38599c5700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3854102670 0x7f3854114030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.490+0000 7f38599c5700 1 -- 172.21.15.33:0/1702030290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848005040 con 0x7f3854101cd0 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.491+0000 7f38599c5700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854101cd0 0x7f3854113af0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f384c00ec90 tx=0x7f384c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.491+0000 7f38527fc700 1 -- 172.21.15.33:0/1702030290 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f384c007b30 con 0x7f3854101cd0 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.491+0000 7f38527fc700 1 -- 172.21.15.33:0/1702030290 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f384c00f040 con 0x7f3854101cd0 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.491+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385410dd50 con 0x7f3854101cd0 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.491+0000 7f38527fc700 1 -- 172.21.15.33:0/1702030290 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f384c00cc80 con 0x7f3854101cd0 2024-04-25T01:54:00.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.491+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385410e210 con 0x7f3854101cd0 2024-04-25T01:54:00.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.492+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854065200 con 0x7f3854101cd0 2024-04-25T01:54:00.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.492+0000 7f38527fc700 1 -- 172.21.15.33:0/1702030290 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f384c007c90 con 0x7f3854101cd0 2024-04-25T01:54:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.493+0000 7f38527fc700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f383c06e140 0x7f383c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.493+0000 7f38527fc700 1 -- 172.21.15.33:0/1702030290 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f384c08df50 con 0x7f3854101cd0 2024-04-25T01:54:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.493+0000 7f38591c4700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f383c06e140 0x7f383c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:00.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.493+0000 7f38591c4700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f383c06e140 0x7f383c0705f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f384800b0d0 tx=0x7f3848009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:00.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.497+0000 7f38527fc700 1 -- 172.21.15.33:0/1702030290 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f384c0c2740 con 0x7f3854101cd0 2024-04-25T01:54:00.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.712+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3854109ed0 con 0x7f383c06e140 2024-04-25T01:54:00.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.722+0000 7f38527fc700 1 -- 172.21.15.33:0/1702030290 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3854109ed0 con 0x7f383c06e140 2024-04-25T01:54:00.723 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (15m) 8m ago 26m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (12m) 8m ago 26m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (12m) 8m ago 25m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (14m) 8m ago 26m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (8m) 8m ago 20m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (8m) 8m ago 20m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (8m) 8m ago 20m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (20m) 8m ago 20m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (16m) 8m ago 27m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (16m) 8m ago 25m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (13m) 8m ago 27m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (13m) 8m ago 25m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (15m) 8m ago 26m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:54:00.724 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (15m) 8m ago 25m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:54:00.725 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (11m) 8m ago 24m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:54:00.725 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (11m) 8m ago 24m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:54:00.725 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (10m) 8m ago 23m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:54:00.725 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (10m) 8m ago 23m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:54:00.725 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (9m) 8m ago 22m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:54:00.725 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (9m) 8m ago 22m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:54:00.725 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (15m) 8m ago 25m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:54:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f383c06e140 msgr2=0x7f383c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f383c06e140 0x7f383c0705f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f384800b0d0 tx=0x7f3848009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854101cd0 msgr2=0x7f3854113af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854101cd0 0x7f3854113af0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f384c00ec90 tx=0x7f384c00b040 comp rx=0 tx=0).stop 2024-04-25T01:54:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 shutdown_connections 2024-04-25T01:54:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f383c06e140 0x7f383c0705f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3854101cd0 0x7f3854113af0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 --2- 172.21.15.33:0/1702030290 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3854102670 0x7f3854114030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 >> 172.21.15.33:0/1702030290 conn(0x7f3854071460 msgr2=0x7f38541009d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 shutdown_connections 2024-04-25T01:54:00.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.725+0000 7f385bc29700 1 -- 172.21.15.33:0/1702030290 wait complete. 2024-04-25T01:54:00.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/2436268789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf41026a0 msgr2=0x7fdcf40fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 --2- 172.21.15.33:0/2436268789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf41026a0 0x7fdcf40fbd10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdce4004940 tx=0x7fdce4009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:00.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/2436268789 shutdown_connections 2024-04-25T01:54:00.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 --2- 172.21.15.33:0/2436268789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf41026a0 0x7fdcf40fbd10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 --2- 172.21.15.33:0/2436268789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf4101d00 0x7fdcf41020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/2436268789 >> 172.21.15.33:0/2436268789 conn(0x7fdcf40f7870 msgr2=0x7fdcf40f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/2436268789 shutdown_connections 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/2436268789 wait complete. 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 Processor -- start 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.828+0000 7fdcfb6dd700 1 -- start start 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcfb6dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf4101d00 0x7fdcf41139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcfb6dd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf41026a0 0x7fdcf4113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcfb6dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf4114570 con 0x7fdcf4101d00 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcfb6dd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf410db30 con 0x7fdcf41026a0 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcf9479700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf4101d00 0x7fdcf41139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcf9479700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf4101d00 0x7fdcf41139f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56866/0 (socket says 172.21.15.33:56866) 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcf9479700 1 -- 172.21.15.33:0/3110804856 learned_addr learned my addr 172.21.15.33:0/3110804856 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcf8c78700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf41026a0 0x7fdcf4113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcf9479700 1 -- 172.21.15.33:0/3110804856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf41026a0 msgr2=0x7fdcf4113f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:00.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcf9479700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf41026a0 0x7fdcf4113f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:00.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcf9479700 1 -- 172.21.15.33:0/3110804856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce4005040 con 0x7fdcf4101d00 2024-04-25T01:54:00.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcf8c78700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf41026a0 0x7fdcf4113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:54:00.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcf9479700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf4101d00 0x7fdcf41139f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000b790 tx=0x7fdcf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:00.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdce9ffb700 1 -- 172.21.15.33:0/3110804856 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf000f770 con 0x7fdcf4101d00 2024-04-25T01:54:00.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf410de10 con 0x7fdcf4101d00 2024-04-25T01:54:00.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdce9ffb700 1 -- 172.21.15.33:0/3110804856 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcf000fda0 con 0x7fdcf4101d00 2024-04-25T01:54:00.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.829+0000 7fdce9ffb700 1 -- 172.21.15.33:0/3110804856 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcf000d310 con 0x7fdcf4101d00 2024-04-25T01:54:00.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.830+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf410e3d0 con 0x7fdcf4101d00 2024-04-25T01:54:00.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.831+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf40fd450 con 0x7fdcf4101d00 2024-04-25T01:54:00.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.831+0000 7fdce9ffb700 1 -- 172.21.15.33:0/3110804856 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdcf0010420 con 0x7fdcf4101d00 2024-04-25T01:54:00.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.832+0000 7fdce9ffb700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdce006e140 0x7fdce00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:00.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.832+0000 7fdce9ffb700 1 -- 172.21.15.33:0/3110804856 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdcf008db80 con 0x7fdcf4101d00 2024-04-25T01:54:00.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.832+0000 7fdcf8c78700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdce006e140 0x7fdce00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:00.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.832+0000 7fdcf8c78700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdce006e140 0x7fdce00705f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40f9010 tx=0x7fdce4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:00.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:00.837+0000 7fdce9ffb700 1 -- 172.21.15.33:0/3110804856 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdcf0056f80 con 0x7fdcf4101d00 2024-04-25T01:54:01.016 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:00 smithi033 ceph-mon[111306]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.132+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdcf410efd0 con 0x7fdcf4101d00 2024-04-25T01:54:01.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.133+0000 7fdce9ffb700 1 -- 172.21.15.33:0/3110804856 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fdcf005a640 con 0x7fdcf4101d00 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:54:01.134 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:54:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:54:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:54:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:54:01.135 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:54:01.135 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:54:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdce006e140 msgr2=0x7fdce00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdce006e140 0x7fdce00705f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40f9010 tx=0x7fdce4009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf4101d00 msgr2=0x7fdcf41139f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf4101d00 0x7fdcf41139f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000b790 tx=0x7fdcf0009250 comp rx=0 tx=0).stop 2024-04-25T01:54:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 shutdown_connections 2024-04-25T01:54:01.136 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdce006e140 0x7fdce00705f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdcf4101d00 0x7fdcf41139f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 --2- 172.21.15.33:0/3110804856 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdcf41026a0 0x7fdcf4113f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 >> 172.21.15.33:0/3110804856 conn(0x7fdcf40f7870 msgr2=0x7fdcf4066250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 shutdown_connections 2024-04-25T01:54:01.137 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.135+0000 7fdcfb6dd700 1 -- 172.21.15.33:0/3110804856 wait complete. 2024-04-25T01:54:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 -- 172.21.15.33:0/3837985271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fbf40 msgr2=0x7f5e4c101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 --2- 172.21.15.33:0/3837985271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fbf40 0x7f5e4c101a30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5e40004940 tx=0x7f5e40009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 -- 172.21.15.33:0/3837985271 shutdown_connections 2024-04-25T01:54:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 --2- 172.21.15.33:0/3837985271 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fbf40 0x7f5e4c101a30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 --2- 172.21.15.33:0/3837985271 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c0fb5a0 0x7f5e4c0fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 -- 172.21.15.33:0/3837985271 >> 172.21.15.33:0/3837985271 conn(0x7f5e4c073350 msgr2=0x7f5e4c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 -- 172.21.15.33:0/3837985271 shutdown_connections 2024-04-25T01:54:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 -- 172.21.15.33:0/3837985271 wait complete. 2024-04-25T01:54:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 Processor -- start 2024-04-25T01:54:01.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.240+0000 7f5e52cc1700 1 -- start start 2024-04-25T01:54:01.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e52cc1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fb5a0 0x7f5e4c113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:01.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e52cc1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c0fbf40 0x7f5e4c113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e52cc1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c114670 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e52cc1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c10dbc0 con 0x7f5e4c0fbf40 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e50a5d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fb5a0 0x7f5e4c113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e50a5d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fb5a0 0x7f5e4c113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56888/0 (socket says 172.21.15.33:56888) 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e50a5d700 1 -- 172.21.15.33:0/2162367753 learned_addr learned my addr 172.21.15.33:0/2162367753 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e4bfff700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c0fbf40 0x7f5e4c113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e50a5d700 1 -- 172.21.15.33:0/2162367753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c0fbf40 msgr2=0x7f5e4c113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e50a5d700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c0fbf40 0x7f5e4c113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e50a5d700 1 -- 172.21.15.33:0/2162367753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e40005040 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e4bfff700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c0fbf40 0x7f5e4c113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.241+0000 7f5e50a5d700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fb5a0 0x7f5e4c113a50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00b790 tx=0x7f5e3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.242+0000 7f5e497fa700 1 -- 172.21.15.33:0/2162367753 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3c00f770 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.242+0000 7f5e497fa700 1 -- 172.21.15.33:0/2162367753 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e3c00fda0 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.242+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c10de40 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.242+0000 7f5e497fa700 1 -- 172.21.15.33:0/2162367753 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3c00d310 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.242+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c10e380 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.243+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c0fe670 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.243+0000 7f5e497fa700 1 -- 172.21.15.33:0/2162367753 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5e3c010420 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.244+0000 7f5e497fa700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e3406e140 0x7f5e340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.244+0000 7f5e497fa700 1 -- 172.21.15.33:0/2162367753 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5e3c08db40 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.244+0000 7f5e4bfff700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e3406e140 0x7f5e340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:01.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.244+0000 7f5e4bfff700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e3406e140 0x7f5e340705f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009fd0 tx=0x7f5e40009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:01.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.248+0000 7f5e497fa700 1 -- 172.21.15.33:0/2162367753 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5e3c05aa70 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:00 smithi138 ceph-mon[81304]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.500+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5e4c10f160 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.500+0000 7f5e497fa700 1 -- 172.21.15.33:0/2162367753 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f5e3c05a600 con 0x7f5e4c0fb5a0 2024-04-25T01:54:01.501 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:54:01.501 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:54:01.502 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:54:01.503 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:54:01.504 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:54:01.504 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:54:01.504 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:54:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:54:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:54:01.504 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:54:01.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:54:01.504 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:54:01.505 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:54:01.505 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:54:01.505 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:54:01.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.503+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e3406e140 msgr2=0x7f5e340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e3406e140 0x7f5e340705f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e40009fd0 tx=0x7f5e40009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fb5a0 msgr2=0x7f5e4c113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fb5a0 0x7f5e4c113a50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00b790 tx=0x7f5e3c009250 comp rx=0 tx=0).stop 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 shutdown_connections 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e3406e140 0x7f5e340705f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e4c0fb5a0 0x7f5e4c113a50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 --2- 172.21.15.33:0/2162367753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e4c0fbf40 0x7f5e4c113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 >> 172.21.15.33:0/2162367753 conn(0x7f5e4c073350 msgr2=0x7f5e4c0f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 shutdown_connections 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.504+0000 7f5e52cc1700 1 -- 172.21.15.33:0/2162367753 wait complete. 2024-04-25T01:54:01.506 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:54:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.610+0000 7f031eb1d700 1 -- 172.21.15.33:0/3721522077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 msgr2=0x7f0318103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.610+0000 7f031eb1d700 1 --2- 172.21.15.33:0/3721522077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 0x7f0318103330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f030c004940 tx=0x7f030c009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.611+0000 7f031eb1d700 1 -- 172.21.15.33:0/3721522077 shutdown_connections 2024-04-25T01:54:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.611+0000 7f031eb1d700 1 --2- 172.21.15.33:0/3721522077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03180fb520 0x7f03180fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.611+0000 7f031eb1d700 1 --2- 172.21.15.33:0/3721522077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 0x7f0318103330 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.611+0000 7f031eb1d700 1 -- 172.21.15.33:0/3721522077 >> 172.21.15.33:0/3721522077 conn(0x7f03180714d0 msgr2=0x7f03180718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.611+0000 7f031eb1d700 1 -- 172.21.15.33:0/3721522077 shutdown_connections 2024-04-25T01:54:01.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.611+0000 7f031eb1d700 1 -- 172.21.15.33:0/3721522077 wait complete. 2024-04-25T01:54:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.611+0000 7f031eb1d700 1 Processor -- start 2024-04-25T01:54:01.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.611+0000 7f031eb1d700 1 -- start start 2024-04-25T01:54:01.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f031eb1d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03180fb520 0x7f03181139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:01.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f031eb1d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 0x7f0318113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:01.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f031eb1d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318114530 con 0x7f0318102f60 2024-04-25T01:54:01.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f031eb1d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031810db50 con 0x7f03180fb520 2024-04-25T01:54:01.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f0317fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 0x7f0318113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f0317fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 0x7f0318113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56916/0 (socket says 172.21.15.33:56916) 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f0317fff700 1 -- 172.21.15.33:0/1532088769 learned_addr learned my addr 172.21.15.33:0/1532088769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f031c8b9700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03180fb520 0x7f03181139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f0317fff700 1 -- 172.21.15.33:0/1532088769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03180fb520 msgr2=0x7f03181139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f0317fff700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03180fb520 0x7f03181139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f0317fff700 1 -- 172.21.15.33:0/1532088769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030c005040 con 0x7f0318102f60 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f0317fff700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 0x7f0318113ee0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f030400c8c0 tx=0x7f03040044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f03157fa700 1 -- 172.21.15.33:0/1532088769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f030400a980 con 0x7f0318102f60 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031810de30 con 0x7f0318102f60 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f03157fa700 1 -- 172.21.15.33:0/1532088769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f030400ce60 con 0x7f0318102f60 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f03157fa700 1 -- 172.21.15.33:0/1532088769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03040115e0 con 0x7f0318102f60 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.612+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031810e370 con 0x7f0318102f60 2024-04-25T01:54:01.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.613+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0318100250 con 0x7f0318102f60 2024-04-25T01:54:01.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.614+0000 7f03157fa700 1 -- 172.21.15.33:0/1532088769 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0304012400 con 0x7f0318102f60 2024-04-25T01:54:01.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.614+0000 7f03157fa700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f030806e140 0x7f03080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:01.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.614+0000 7f03157fa700 1 -- 172.21.15.33:0/1532088769 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f030408dbe0 con 0x7f0318102f60 2024-04-25T01:54:01.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.614+0000 7f031c8b9700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f030806e140 0x7f03080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:01.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.615+0000 7f031c8b9700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f030806e140 0x7f03080705f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f030c000c00 tx=0x7f030c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:01.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.619+0000 7f03157fa700 1 -- 172.21.15.33:0/1532088769 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0304056fe0 con 0x7f0318102f60 2024-04-25T01:54:01.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.846+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f031810eff0 con 0x7f030806e140 2024-04-25T01:54:01.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.847+0000 7f03157fa700 1 -- 172.21.15.33:0/1532088769 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f031810eff0 con 0x7f030806e140 2024-04-25T01:54:01.848 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:54:01.848 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:54:01.848 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:54:01.848 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:54:01.848 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:54:01.848 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:54:01.848 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:54:01.848 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:54:01.849 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:54:01.849 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:54:01.849 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:54:01.849 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:54:01.849 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:54:01.849 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f030806e140 msgr2=0x7f03080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f030806e140 0x7f03080705f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f030c000c00 tx=0x7f030c009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 msgr2=0x7f0318113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 0x7f0318113ee0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f030400c8c0 tx=0x7f03040044c0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 shutdown_connections 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f030806e140 0x7f03080705f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f03180fb520 0x7f03181139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 --2- 172.21.15.33:0/1532088769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0318102f60 0x7f0318113ee0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 >> 172.21.15.33:0/1532088769 conn(0x7f03180714d0 msgr2=0x7f03180fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 shutdown_connections 2024-04-25T01:54:01.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.850+0000 7f031eb1d700 1 -- 172.21.15.33:0/1532088769 wait complete. 2024-04-25T01:54:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.954+0000 7f02b54f2700 1 -- 172.21.15.33:0/3052124038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 msgr2=0x7f02b01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.954+0000 7f02b54f2700 1 --2- 172.21.15.33:0/3052124038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 0x7f02b01031c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009ce0 tx=0x7f02a8004d10 comp rx=0 tx=0).stop 2024-04-25T01:54:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.954+0000 7f02b54f2700 1 -- 172.21.15.33:0/3052124038 shutdown_connections 2024-04-25T01:54:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.954+0000 7f02b54f2700 1 --2- 172.21.15.33:0/3052124038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b0103790 0x7f02b00fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.954+0000 7f02b54f2700 1 --2- 172.21.15.33:0/3052124038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 0x7f02b01031c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.954+0000 7f02b54f2700 1 -- 172.21.15.33:0/3052124038 >> 172.21.15.33:0/3052124038 conn(0x7f02b00f8980 msgr2=0x7f02b00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.954+0000 7f02b54f2700 1 -- 172.21.15.33:0/3052124038 shutdown_connections 2024-04-25T01:54:01.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.954+0000 7f02b54f2700 1 -- 172.21.15.33:0/3052124038 wait complete. 2024-04-25T01:54:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.954+0000 7f02b54f2700 1 Processor -- start 2024-04-25T01:54:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02b54f2700 1 -- start start 2024-04-25T01:54:01.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02b54f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 0x7f02b0113d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02b54f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b0103790 0x7f02b0114280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02b54f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b01148d0 con 0x7f02b0102df0 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02b54f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b010dca0 con 0x7f02b0103790 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 0x7f02b0113d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 0x7f02b0113d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56936/0 (socket says 172.21.15.33:56936) 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02aeffd700 1 -- 172.21.15.33:0/3438089527 learned_addr learned my addr 172.21.15.33:0/3438089527 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02ae7fc700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b0103790 0x7f02b0114280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02aeffd700 1 -- 172.21.15.33:0/3438089527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b0103790 msgr2=0x7f02b0114280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02aeffd700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b0103790 0x7f02b0114280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02aeffd700 1 -- 172.21.15.33:0/3438089527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a8009940 con 0x7f02b0102df0 2024-04-25T01:54:01.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.955+0000 7f02aeffd700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 0x7f02b0113d40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f02a800be90 tx=0x7f02a8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.956+0000 7f02a77fe700 1 -- 172.21.15.33:0/3438089527 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a800e040 con 0x7f02b0102df0 2024-04-25T01:54:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.956+0000 7f02a77fe700 1 -- 172.21.15.33:0/3438089527 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02a8008ea0 con 0x7f02b0102df0 2024-04-25T01:54:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.956+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b010df20 con 0x7f02b0102df0 2024-04-25T01:54:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.956+0000 7f02a77fe700 1 -- 172.21.15.33:0/3438089527 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a801c810 con 0x7f02b0102df0 2024-04-25T01:54:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.956+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b010e4e0 con 0x7f02b0102df0 2024-04-25T01:54:01.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.957+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02b0100100 con 0x7f02b0102df0 2024-04-25T01:54:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.957+0000 7f02a77fe700 1 -- 172.21.15.33:0/3438089527 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f02a8012410 con 0x7f02b0102df0 2024-04-25T01:54:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.957+0000 7f02a77fe700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f029c06e140 0x7f029c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.958+0000 7f02a77fe700 1 -- 172.21.15.33:0/3438089527 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f02a808f680 con 0x7f02b0102df0 2024-04-25T01:54:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.958+0000 7f02ae7fc700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f029c06e140 0x7f029c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:01.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.958+0000 7f02ae7fc700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f029c06e140 0x7f029c0705f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0298007fd0 tx=0x7f0298009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:01.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:01.962+0000 7f02a77fe700 1 -- 172.21.15.33:0/3438089527 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f02a805c5b0 con 0x7f02b0102df0 2024-04-25T01:54:02.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.257+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f02b00fe540 con 0x7f02b0102df0 2024-04-25T01:54:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:01 smithi033 ceph-mon[111306]: from='client.34852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:02.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:01 smithi033 ceph-mon[111306]: pgmap v438: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:02.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:01 smithi033 ceph-mon[111306]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:02.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:01 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3110804856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:54:02.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:01 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2162367753' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:54:02.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.257+0000 7f02a77fe700 1 -- 172.21.15.33:0/3438089527 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f02a8004e10 con 0x7f02b0102df0 2024-04-25T01:54:02.259 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:54:02.259 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:54:02.259 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:54:02.259 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:54:02.260 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:54:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f029c06e140 msgr2=0x7f029c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f029c06e140 0x7f029c0705f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0298007fd0 tx=0x7f0298009040 comp rx=0 tx=0).stop 2024-04-25T01:54:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 msgr2=0x7f02b0113d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 0x7f02b0113d40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f02a800be90 tx=0x7f02a8004d10 comp rx=0 tx=0).stop 2024-04-25T01:54:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 shutdown_connections 2024-04-25T01:54:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f029c06e140 0x7f029c0705f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02b0102df0 0x7f02b0113d40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 --2- 172.21.15.33:0/3438089527 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f02b0103790 0x7f02b0114280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:02.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 >> 172.21.15.33:0/3438089527 conn(0x7f02b00f8980 msgr2=0x7f02b01073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 shutdown_connections 2024-04-25T01:54:02.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:02.260+0000 7f02b54f2700 1 -- 172.21.15.33:0/3438089527 wait complete. 2024-04-25T01:54:02.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:01 smithi138 ceph-mon[81304]: from='client.34852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:02.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:01 smithi138 ceph-mon[81304]: pgmap v438: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:02.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:01 smithi138 ceph-mon[81304]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:02.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:01 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3110804856' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:54:02.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:01 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2162367753' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:54:03.266 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:02 smithi033 ceph-mon[111306]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:03.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:02 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3438089527' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:54:03.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:02 smithi138 ceph-mon[81304]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:03.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:02 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3438089527' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:54:04.168 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:03 smithi033 ceph-mon[111306]: pgmap v439: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:04.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:03 smithi138 ceph-mon[81304]: pgmap v439: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:06.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:05 smithi033 ceph-mon[111306]: pgmap v440: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:06.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:05 smithi138 ceph-mon[81304]: pgmap v440: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:08.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:07 smithi033 ceph-mon[111306]: pgmap v441: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:08.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:07 smithi138 ceph-mon[81304]: pgmap v441: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:10.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:09 smithi033 ceph-mon[111306]: pgmap v442: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:10.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:09 smithi138 ceph-mon[81304]: pgmap v442: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:11.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:54:11.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:54:12.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:11 smithi033 ceph-mon[111306]: pgmap v443: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:12.291 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:11 smithi138 ceph-mon[81304]: pgmap v443: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:14.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:13 smithi033 ceph-mon[111306]: pgmap v444: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:14.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:13 smithi138 ceph-mon[81304]: pgmap v444: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:16.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:15 smithi033 ceph-mon[111306]: pgmap v445: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:16.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:15 smithi138 ceph-mon[81304]: pgmap v445: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:18.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:17 smithi033 ceph-mon[111306]: pgmap v446: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:18.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:17 smithi138 ceph-mon[81304]: pgmap v446: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:20.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:19 smithi033 ceph-mon[111306]: pgmap v447: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:20.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:19 smithi138 ceph-mon[81304]: pgmap v447: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:22.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:21 smithi033 ceph-mon[111306]: pgmap v448: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:22.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:21 smithi138 ceph-mon[81304]: pgmap v448: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:24.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:23 smithi033 ceph-mon[111306]: pgmap v449: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:24.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:23 smithi138 ceph-mon[81304]: pgmap v449: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:26.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:25 smithi033 ceph-mon[111306]: pgmap v450: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:26.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:54:26.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:25 smithi138 ceph-mon[81304]: pgmap v450: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:26.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:54:28.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:27 smithi033 ceph-mon[111306]: pgmap v451: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:28.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:27 smithi138 ceph-mon[81304]: pgmap v451: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:30.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:29 smithi033 ceph-mon[111306]: pgmap v452: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:30.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:29 smithi138 ceph-mon[81304]: pgmap v452: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:32.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:31 smithi033 ceph-mon[111306]: pgmap v453: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:32.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:31 smithi138 ceph-mon[81304]: pgmap v453: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:32.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.381+0000 7f029c94d700 1 -- 172.21.15.33:0/951916895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 msgr2=0x7f02980fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.381+0000 7f029c94d700 1 --2- 172.21.15.33:0/951916895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 0x7f02980fb990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0280004940 tx=0x7f0280009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.381+0000 7f029c94d700 1 -- 172.21.15.33:0/951916895 shutdown_connections 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.381+0000 7f029c94d700 1 --2- 172.21.15.33:0/951916895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 0x7f02980fb990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.381+0000 7f029c94d700 1 --2- 172.21.15.33:0/951916895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102f60 0x7f0298103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.381+0000 7f029c94d700 1 -- 172.21.15.33:0/951916895 >> 172.21.15.33:0/951916895 conn(0x7f02980714d0 msgr2=0x7f02980718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.381+0000 7f029c94d700 1 -- 172.21.15.33:0/951916895 shutdown_connections 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.381+0000 7f029c94d700 1 -- 172.21.15.33:0/951916895 wait complete. 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f029c94d700 1 Processor -- start 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f029c94d700 1 -- start start 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f029c94d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 0x7f0298113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f029c94d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102f60 0x7f0298113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f029c94d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0298114590 con 0x7f02980fb520 2024-04-25T01:54:32.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f029c94d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029810dbb0 con 0x7f0298102f60 2024-04-25T01:54:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f029659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 0x7f0298113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f029659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 0x7f0298113a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51254/0 (socket says 172.21.15.33:51254) 2024-04-25T01:54:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f029659c700 1 -- 172.21.15.33:0/1274139402 learned_addr learned my addr 172.21.15.33:0/1274139402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.382+0000 7f0295d9b700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102f60 0x7f0298113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.383+0000 7f029659c700 1 -- 172.21.15.33:0/1274139402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102f60 msgr2=0x7f0298113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.383+0000 7f029659c700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102f60 0x7f0298113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.383+0000 7f029659c700 1 -- 172.21.15.33:0/1274139402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0280005040 con 0x7f02980fb520 2024-04-25T01:54:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.383+0000 7f029659c700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 0x7f0298113a00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f028800cac0 tx=0x7f0288007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.383+0000 7f028effd700 1 -- 172.21.15.33:0/1274139402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f028800aa50 con 0x7f02980fb520 2024-04-25T01:54:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.383+0000 7f028effd700 1 -- 172.21.15.33:0/1274139402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f028800abb0 con 0x7f02980fb520 2024-04-25T01:54:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.383+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029810de90 con 0x7f02980fb520 2024-04-25T01:54:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.383+0000 7f028effd700 1 -- 172.21.15.33:0/1274139402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02880115e0 con 0x7f02980fb520 2024-04-25T01:54:32.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.383+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029810e3d0 con 0x7f02980fb520 2024-04-25T01:54:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.384+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0298100250 con 0x7f02980fb520 2024-04-25T01:54:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.385+0000 7f028effd700 1 -- 172.21.15.33:0/1274139402 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f028800ad20 con 0x7f02980fb520 2024-04-25T01:54:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.385+0000 7f028effd700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028406e140 0x7f02840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.385+0000 7f028effd700 1 -- 172.21.15.33:0/1274139402 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f028808e260 con 0x7f02980fb520 2024-04-25T01:54:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.385+0000 7f0295d9b700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028406e140 0x7f02840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:32.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.386+0000 7f0295d9b700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028406e140 0x7f02840705f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f029810f2e0 tx=0x7f0280009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:32.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.390+0000 7f028effd700 1 -- 172.21.15.33:0/1274139402 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f02880575e0 con 0x7f02980fb520 2024-04-25T01:54:32.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.622+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0298109fe0 con 0x7f028406e140 2024-04-25T01:54:32.624 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.623+0000 7f028effd700 1 -- 172.21.15.33:0/1274139402 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0298109fe0 con 0x7f028406e140 2024-04-25T01:54:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028406e140 msgr2=0x7f02840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028406e140 0x7f02840705f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f029810f2e0 tx=0x7f0280009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 msgr2=0x7f0298113a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 0x7f0298113a00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f028800cac0 tx=0x7f0288007ee0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 shutdown_connections 2024-04-25T01:54:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f028406e140 0x7f02840705f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f02980fb520 0x7f0298113a00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 --2- 172.21.15.33:0/1274139402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0298102f60 0x7f0298113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 >> 172.21.15.33:0/1274139402 conn(0x7f02980714d0 msgr2=0x7f02980fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 shutdown_connections 2024-04-25T01:54:32.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.626+0000 7f029c94d700 1 -- 172.21.15.33:0/1274139402 wait complete. 2024-04-25T01:54:32.637 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:54:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.731+0000 7f4bdd829700 1 -- 172.21.15.33:0/537429566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8065640 msgr2=0x7f4bd8065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.731+0000 7f4bdd829700 1 --2- 172.21.15.33:0/537429566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8065640 0x7f4bd8065ab0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc004940 tx=0x7f4bcc009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.731+0000 7f4bdd829700 1 -- 172.21.15.33:0/537429566 shutdown_connections 2024-04-25T01:54:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.731+0000 7f4bdd829700 1 --2- 172.21.15.33:0/537429566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8065640 0x7f4bd8065ab0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.731+0000 7f4bdd829700 1 --2- 172.21.15.33:0/537429566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd8102e50 0x7f4bd8103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.731+0000 7f4bdd829700 1 -- 172.21.15.33:0/537429566 >> 172.21.15.33:0/537429566 conn(0x7f4bd8071d00 msgr2=0x7f4bd8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:32.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.731+0000 7f4bdd829700 1 -- 172.21.15.33:0/537429566 shutdown_connections 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.731+0000 7f4bdd829700 1 -- 172.21.15.33:0/537429566 wait complete. 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bdd829700 1 Processor -- start 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bdd829700 1 -- start start 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bdd829700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd8065640 0x7f4bd8113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bdd829700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8102e50 0x7f4bd81140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bdd829700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd8114780 con 0x7f4bd8102e50 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bdd829700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd810dae0 con 0x7f4bd8065640 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bd67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8102e50 0x7f4bd81140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bd67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8102e50 0x7f4bd81140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51264/0 (socket says 172.21.15.33:51264) 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bd67fc700 1 -- 172.21.15.33:0/1184761762 learned_addr learned my addr 172.21.15.33:0/1184761762 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bd6ffd700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd8065640 0x7f4bd8113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bd67fc700 1 -- 172.21.15.33:0/1184761762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd8065640 msgr2=0x7f4bd8113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:32.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bd67fc700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd8065640 0x7f4bd8113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bd67fc700 1 -- 172.21.15.33:0/1184761762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc005040 con 0x7f4bd8102e50 2024-04-25T01:54:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bd6ffd700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd8065640 0x7f4bd8113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:54:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.732+0000 7f4bd67fc700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8102e50 0x7f4bd81140a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009150 tx=0x7f4bcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.733+0000 7f4bbf7fe700 1 -- 172.21.15.33:0/1184761762 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bcc00e040 con 0x7f4bd8102e50 2024-04-25T01:54:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.733+0000 7f4bbf7fe700 1 -- 172.21.15.33:0/1184761762 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bcc013420 con 0x7f4bd8102e50 2024-04-25T01:54:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.733+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd810dd60 con 0x7f4bd8102e50 2024-04-25T01:54:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.733+0000 7f4bbf7fe700 1 -- 172.21.15.33:0/1184761762 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bcc01c630 con 0x7f4bd8102e50 2024-04-25T01:54:32.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.733+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd810e320 con 0x7f4bd8102e50 2024-04-25T01:54:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.734+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd80fe540 con 0x7f4bd8102e50 2024-04-25T01:54:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.735+0000 7f4bbf7fe700 1 -- 172.21.15.33:0/1184761762 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4bcc013590 con 0x7f4bd8102e50 2024-04-25T01:54:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.735+0000 7f4bbf7fe700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bc006e150 0x7f4bc0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.735+0000 7f4bbf7fe700 1 -- 172.21.15.33:0/1184761762 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4bcc08f850 con 0x7f4bd8102e50 2024-04-25T01:54:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.735+0000 7f4bd6ffd700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bc006e150 0x7f4bc0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:32.737 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.736+0000 7f4bd6ffd700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bc006e150 0x7f4bc0070600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8001370 tx=0x7f4bc80096a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:32.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.739+0000 7f4bbf7fe700 1 -- 172.21.15.33:0/1184761762 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4bcc05c820 con 0x7f4bd8102e50 2024-04-25T01:54:32.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.965+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bd8109f70 con 0x7f4bc006e150 2024-04-25T01:54:32.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.966+0000 7f4bbf7fe700 1 -- 172.21.15.33:0/1184761762 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4bd8109f70 con 0x7f4bc006e150 2024-04-25T01:54:32.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.968+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bc006e150 msgr2=0x7f4bc0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:32.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.968+0000 7f4bdd829700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bc006e150 0x7f4bc0070600 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8001370 tx=0x7f4bc80096a0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.969+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8102e50 msgr2=0x7f4bd81140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:32.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.969+0000 7f4bdd829700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8102e50 0x7f4bd81140a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc009150 tx=0x7f4bcc009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:32.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.969+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 shutdown_connections 2024-04-25T01:54:32.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.969+0000 7f4bdd829700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bc006e150 0x7f4bc0070600 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.969+0000 7f4bdd829700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bd8065640 0x7f4bd8113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.969+0000 7f4bdd829700 1 --2- 172.21.15.33:0/1184761762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bd8102e50 0x7f4bd81140a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:32.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.969+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 >> 172.21.15.33:0/1184761762 conn(0x7f4bd8071d00 msgr2=0x7f4bd80f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:32.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.969+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 shutdown_connections 2024-04-25T01:54:32.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:32.969+0000 7f4bdd829700 1 -- 172.21.15.33:0/1184761762 wait complete. 2024-04-25T01:54:33.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.073+0000 7f47062ba700 1 -- 172.21.15.33:0/2021994374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700103790 msgr2=0x7f47000fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.073+0000 7f47062ba700 1 --2- 172.21.15.33:0/2021994374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700103790 0x7f47000fce00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f46e8004940 tx=0x7f46e8009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:33.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.073+0000 7f47062ba700 1 -- 172.21.15.33:0/2021994374 shutdown_connections 2024-04-25T01:54:33.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.073+0000 7f47062ba700 1 --2- 172.21.15.33:0/2021994374 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700103790 0x7f47000fce00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.073+0000 7f47062ba700 1 --2- 172.21.15.33:0/2021994374 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4700102df0 0x7f47001031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.073+0000 7f47062ba700 1 -- 172.21.15.33:0/2021994374 >> 172.21.15.33:0/2021994374 conn(0x7f47000f8980 msgr2=0x7f47000fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:33.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.073+0000 7f47062ba700 1 -- 172.21.15.33:0/2021994374 shutdown_connections 2024-04-25T01:54:33.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.073+0000 7f47062ba700 1 -- 172.21.15.33:0/2021994374 wait complete. 2024-04-25T01:54:33.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f47062ba700 1 Processor -- start 2024-04-25T01:54:33.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f47062ba700 1 -- start start 2024-04-25T01:54:33.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f47062ba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700102df0 0x7f4700113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:33.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f47062ba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4700103790 0x7f47001140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:33.077 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f47062ba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47001147b0 con 0x7f4700102df0 2024-04-25T01:54:33.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f47062ba700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470010db30 con 0x7f4700103790 2024-04-25T01:54:33.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f46fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700102df0 0x7f4700113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:33.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f46fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700102df0 0x7f4700113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51288/0 (socket says 172.21.15.33:51288) 2024-04-25T01:54:33.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f46fffff700 1 -- 172.21.15.33:0/1952454743 learned_addr learned my addr 172.21.15.33:0/1952454743 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:33.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.074+0000 7f46ff7fe700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4700103790 0x7f47001140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:33.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.075+0000 7f46fffff700 1 -- 172.21.15.33:0/1952454743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4700103790 msgr2=0x7f47001140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.075+0000 7f46fffff700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4700103790 0x7f47001140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.075+0000 7f46fffff700 1 -- 172.21.15.33:0/1952454743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8005040 con 0x7f4700102df0 2024-04-25T01:54:33.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.075+0000 7f46fffff700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700102df0 0x7f4700113b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f46f000b790 tx=0x7f46f0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:33.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.075+0000 7f46fcff9700 1 -- 172.21.15.33:0/1952454743 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f000f770 con 0x7f4700102df0 2024-04-25T01:54:33.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.075+0000 7f46fcff9700 1 -- 172.21.15.33:0/1952454743 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46f000fda0 con 0x7f4700102df0 2024-04-25T01:54:33.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.075+0000 7f46fcff9700 1 -- 172.21.15.33:0/1952454743 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46f000d310 con 0x7f4700102df0 2024-04-25T01:54:33.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.075+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470010de10 con 0x7f4700102df0 2024-04-25T01:54:33.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.075+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470010e350 con 0x7f4700102df0 2024-04-25T01:54:33.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.076+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4700100100 con 0x7f4700102df0 2024-04-25T01:54:33.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.076+0000 7f46fcff9700 1 -- 172.21.15.33:0/1952454743 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f46f0010420 con 0x7f4700102df0 2024-04-25T01:54:33.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.077+0000 7f46fcff9700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46ec06e140 0x7f46ec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:33.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.077+0000 7f46fcff9700 1 -- 172.21.15.33:0/1952454743 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f46f008db40 con 0x7f4700102df0 2024-04-25T01:54:33.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.077+0000 7f46ff7fe700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46ec06e140 0x7f46ec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:33.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.077+0000 7f46ff7fe700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46ec06e140 0x7f46ec0705f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f46e800b3b0 tx=0x7f46e8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:33.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.081+0000 7f46fcff9700 1 -- 172.21.15.33:0/1952454743 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f46f0056f40 con 0x7f4700102df0 2024-04-25T01:54:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.297+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4700109f40 con 0x7f46ec06e140 2024-04-25T01:54:33.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.307+0000 7f46fcff9700 1 -- 172.21.15.33:0/1952454743 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4700109f40 con 0x7f46ec06e140 2024-04-25T01:54:33.308 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:54:33.308 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (15m) 8m ago 26m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (12m) 8m ago 27m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (12m) 8m ago 26m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (15m) 8m ago 26m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (8m) 8m ago 21m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (9m) 8m ago 21m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (8m) 8m ago 21m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (21m) 8m ago 21m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (17m) 8m ago 28m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (16m) 8m ago 26m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (14m) 8m ago 28m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (13m) 8m ago 25m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (16m) 8m ago 27m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (16m) 8m ago 26m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:54:33.309 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (12m) 8m ago 25m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:54:33.310 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (11m) 8m ago 24m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:54:33.310 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (11m) 8m ago 24m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:54:33.310 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (10m) 8m ago 23m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:54:33.310 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (10m) 8m ago 23m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:54:33.310 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (9m) 8m ago 23m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:54:33.310 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (16m) 8m ago 26m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46ec06e140 msgr2=0x7f46ec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46ec06e140 0x7f46ec0705f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f46e800b3b0 tx=0x7f46e8009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700102df0 msgr2=0x7f4700113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700102df0 0x7f4700113b90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f46f000b790 tx=0x7f46f0009250 comp rx=0 tx=0).stop 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 shutdown_connections 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46ec06e140 0x7f46ec0705f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4700102df0 0x7f4700113b90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 --2- 172.21.15.33:0/1952454743 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4700103790 0x7f47001140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 >> 172.21.15.33:0/1952454743 conn(0x7f47000f8980 msgr2=0x7f47001073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:33.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 shutdown_connections 2024-04-25T01:54:33.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.309+0000 7f47062ba700 1 -- 172.21.15.33:0/1952454743 wait complete. 2024-04-25T01:54:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 -- 172.21.15.33:0/3695680365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c103780 msgr2=0x7f324c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 --2- 172.21.15.33:0/3695680365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c103780 0x7f324c0fcdf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f323c004940 tx=0x7f323c009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 -- 172.21.15.33:0/3695680365 shutdown_connections 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 --2- 172.21.15.33:0/3695680365 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c103780 0x7f324c0fcdf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 --2- 172.21.15.33:0/3695680365 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c102de0 0x7f324c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 -- 172.21.15.33:0/3695680365 >> 172.21.15.33:0/3695680365 conn(0x7f324c0f8990 msgr2=0x7f324c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 -- 172.21.15.33:0/3695680365 shutdown_connections 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 -- 172.21.15.33:0/3695680365 wait complete. 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 Processor -- start 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 -- start start 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c102de0 0x7f324c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c103780 0x7f324c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c0721b0 con 0x7f324c102de0 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.413+0000 7f3253c74700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324c070ce0 con 0x7f324c103780 2024-04-25T01:54:33.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f3251a10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c102de0 0x7f324c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f3251a10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c102de0 0x7f324c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51310/0 (socket says 172.21.15.33:51310) 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f3251a10700 1 -- 172.21.15.33:0/3417941774 learned_addr learned my addr 172.21.15.33:0/3417941774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f325120f700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c103780 0x7f324c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f3251a10700 1 -- 172.21.15.33:0/3417941774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c103780 msgr2=0x7f324c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f3251a10700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c103780 0x7f324c0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f3251a10700 1 -- 172.21.15.33:0/3417941774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323c005040 con 0x7f324c102de0 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f3251a10700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c102de0 0x7f324c070260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3248002a50 tx=0x7f32480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f32427fc700 1 -- 172.21.15.33:0/3417941774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3248007cc0 con 0x7f324c102de0 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f32427fc700 1 -- 172.21.15.33:0/3417941774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f324800f040 con 0x7f324c102de0 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c070fc0 con 0x7f324c102de0 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f32427fc700 1 -- 172.21.15.33:0/3417941774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3248009640 con 0x7f324c102de0 2024-04-25T01:54:33.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.414+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c11a140 con 0x7f324c102de0 2024-04-25T01:54:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.415+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f324c0fe530 con 0x7f324c102de0 2024-04-25T01:54:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.416+0000 7f32427fc700 1 -- 172.21.15.33:0/3417941774 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f32480097a0 con 0x7f324c102de0 2024-04-25T01:54:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.416+0000 7f32427fc700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f323806e140 0x7f32380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:33.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.416+0000 7f32427fc700 1 -- 172.21.15.33:0/3417941774 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f324808d920 con 0x7f324c102de0 2024-04-25T01:54:33.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.417+0000 7f325120f700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f323806e140 0x7f32380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:33.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.417+0000 7f325120f700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f323806e140 0x7f32380705f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b3b0 tx=0x7f323c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:33.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.421+0000 7f32427fc700 1 -- 172.21.15.33:0/3417941774 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3248056ca0 con 0x7f324c102de0 2024-04-25T01:54:33.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.719+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f324c11a330 con 0x7f324c102de0 2024-04-25T01:54:33.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.719+0000 7f32427fc700 1 -- 172.21.15.33:0/3417941774 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f324805a360 con 0x7f324c102de0 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:54:33.721 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:54:33.722 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:54:33.722 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:54:33.722 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:54:33.722 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:54:33.722 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:54:33.722 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.721+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f323806e140 msgr2=0x7f32380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.721+0000 7f3253c74700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f323806e140 0x7f32380705f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f323c00b3b0 tx=0x7f323c009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.721+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c102de0 msgr2=0x7f324c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.721+0000 7f3253c74700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c102de0 0x7f324c070260 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3248002a50 tx=0x7f32480046b0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.721+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 shutdown_connections 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.721+0000 7f3253c74700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f323806e140 0x7f32380705f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.721+0000 7f3253c74700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f324c102de0 0x7f324c070260 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.721+0000 7f3253c74700 1 --2- 172.21.15.33:0/3417941774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f324c103780 0x7f324c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.721+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 >> 172.21.15.33:0/3417941774 conn(0x7f324c0f8990 msgr2=0x7f324c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.722+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 shutdown_connections 2024-04-25T01:54:33.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.722+0000 7f3253c74700 1 -- 172.21.15.33:0/3417941774 wait complete. 2024-04-25T01:54:33.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.826+0000 7fe640851700 1 -- 172.21.15.33:0/1784638536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c101d00 msgr2=0x7fe63c1020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.826+0000 7fe640851700 1 --2- 172.21.15.33:0/1784638536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c101d00 0x7fe63c1020d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe62c004940 tx=0x7fe62c009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:33.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.826+0000 7fe640851700 1 -- 172.21.15.33:0/1784638536 shutdown_connections 2024-04-25T01:54:33.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.826+0000 7fe640851700 1 --2- 172.21.15.33:0/1784638536 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe63c1026a0 0x7fe63c0fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.826+0000 7fe640851700 1 --2- 172.21.15.33:0/1784638536 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c101d00 0x7fe63c1020d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.826+0000 7fe640851700 1 -- 172.21.15.33:0/1784638536 >> 172.21.15.33:0/1784638536 conn(0x7fe63c0f7870 msgr2=0x7fe63c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:33.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.827+0000 7fe640851700 1 -- 172.21.15.33:0/1784638536 shutdown_connections 2024-04-25T01:54:33.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.827+0000 7fe640851700 1 -- 172.21.15.33:0/1784638536 wait complete. 2024-04-25T01:54:33.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.827+0000 7fe640851700 1 Processor -- start 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.827+0000 7fe640851700 1 -- start start 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.827+0000 7fe640851700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe63c101d00 0x7fe63c112890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.827+0000 7fe640851700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c1026a0 0x7fe63c112dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.827+0000 7fe640851700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c1134b0 con 0x7fe63c1026a0 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.827+0000 7fe640851700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c10ca20 con 0x7fe63c101d00 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.827+0000 7fe639d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c1026a0 0x7fe63c112dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe639d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c1026a0 0x7fe63c112dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51322/0 (socket says 172.21.15.33:51322) 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe639d9b700 1 -- 172.21.15.33:0/2300493729 learned_addr learned my addr 172.21.15.33:0/2300493729 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe63a59c700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe63c101d00 0x7fe63c112890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe639d9b700 1 -- 172.21.15.33:0/2300493729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe63c101d00 msgr2=0x7fe63c112890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe639d9b700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe63c101d00 0x7fe63c112890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe639d9b700 1 -- 172.21.15.33:0/2300493729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c005040 con 0x7fe63c1026a0 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe63a59c700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe63c101d00 0x7fe63c112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:54:33.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe639d9b700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c1026a0 0x7fe63c112dd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe63000b790 tx=0x7fe630009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:33.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe62affd700 1 -- 172.21.15.33:0/2300493729 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe63000f6f0 con 0x7fe63c1026a0 2024-04-25T01:54:33.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe62affd700 1 -- 172.21.15.33:0/2300493729 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe63000fd20 con 0x7fe63c1026a0 2024-04-25T01:54:33.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe62affd700 1 -- 172.21.15.33:0/2300493729 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6300103a0 con 0x7fe63c1026a0 2024-04-25T01:54:33.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c10cd00 con 0x7fe63c1026a0 2024-04-25T01:54:33.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.828+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c10d260 con 0x7fe63c1026a0 2024-04-25T01:54:33.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.829+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c0fd450 con 0x7fe63c1026a0 2024-04-25T01:54:33.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.830+0000 7fe62affd700 1 -- 172.21.15.33:0/2300493729 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe630011400 con 0x7fe63c1026a0 2024-04-25T01:54:33.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.830+0000 7fe62affd700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe62406e140 0x7fe6240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:33.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.830+0000 7fe62affd700 1 -- 172.21.15.33:0/2300493729 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe63008d990 con 0x7fe63c1026a0 2024-04-25T01:54:33.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.830+0000 7fe63a59c700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe62406e140 0x7fe6240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:33.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.831+0000 7fe63a59c700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe62406e140 0x7fe6240705f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe62c004560 tx=0x7fe62c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:33.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:33.835+0000 7fe62affd700 1 -- 172.21.15.33:0/2300493729 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe63005a820 con 0x7fe63c1026a0 2024-04-25T01:54:34.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:33 smithi033 ceph-mon[111306]: pgmap v454: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:34.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:33 smithi033 ceph-mon[111306]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:34.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:33 smithi033 ceph-mon[111306]: from='client.34880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:34.017 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:33 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3417941774' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:54:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.091+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe63c108e70 con 0x7fe63c1026a0 2024-04-25T01:54:34.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.091+0000 7fe62affd700 1 -- 172.21.15.33:0/2300493729 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fe63005a3b0 con 0x7fe63c1026a0 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:54:34.093 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:54:34.094 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:54:34.095 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:54:34.096 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:54:34.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe62406e140 msgr2=0x7fe6240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe62406e140 0x7fe6240705f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe62c004560 tx=0x7fe62c009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c1026a0 msgr2=0x7fe63c112dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c1026a0 0x7fe63c112dd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe63000b790 tx=0x7fe630009220 comp rx=0 tx=0).stop 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 shutdown_connections 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe62406e140 0x7fe6240705f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe63c101d00 0x7fe63c112890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 --2- 172.21.15.33:0/2300493729 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe63c1026a0 0x7fe63c112dd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 >> 172.21.15.33:0/2300493729 conn(0x7fe63c0f7870 msgr2=0x7fe63c0f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 shutdown_connections 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.093+0000 7fe640851700 1 -- 172.21.15.33:0/2300493729 wait complete. 2024-04-25T01:54:34.097 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:54:34.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.201+0000 7f63af8b1700 1 -- 172.21.15.33:0/1715736882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fd0a0 msgr2=0x7f63a8109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.201+0000 7f63af8b1700 1 --2- 172.21.15.33:0/1715736882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fd0a0 0x7f63a8109350 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f639c004940 tx=0x7f639c009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.201+0000 7f63af8b1700 1 -- 172.21.15.33:0/1715736882 shutdown_connections 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.201+0000 7f63af8b1700 1 --2- 172.21.15.33:0/1715736882 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fd0a0 0x7f63a8109350 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.201+0000 7f63af8b1700 1 --2- 172.21.15.33:0/1715736882 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a80fc790 0x7f63a80fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.201+0000 7f63af8b1700 1 -- 172.21.15.33:0/1715736882 >> 172.21.15.33:0/1715736882 conn(0x7f63a8071d00 msgr2=0x7f63a8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.201+0000 7f63af8b1700 1 -- 172.21.15.33:0/1715736882 shutdown_connections 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.201+0000 7f63af8b1700 1 -- 172.21.15.33:0/1715736882 wait complete. 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63af8b1700 1 Processor -- start 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63af8b1700 1 -- start start 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63af8b1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fc790 0x7f63a8115df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63af8b1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a80fd0a0 0x7f63a8116330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63af8b1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a8116a10 con 0x7f63a80fc790 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63af8b1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a810fd50 con 0x7f63a80fd0a0 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63ad64d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fc790 0x7f63a8115df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:34.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63ad64d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fc790 0x7f63a8115df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51338/0 (socket says 172.21.15.33:51338) 2024-04-25T01:54:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63ad64d700 1 -- 172.21.15.33:0/4181230539 learned_addr learned my addr 172.21.15.33:0/4181230539 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63ace4c700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a80fd0a0 0x7f63a8116330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63ad64d700 1 -- 172.21.15.33:0/4181230539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a80fd0a0 msgr2=0x7f63a8116330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63ad64d700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a80fd0a0 0x7f63a8116330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63ad64d700 1 -- 172.21.15.33:0/4181230539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c005040 con 0x7f63a80fc790 2024-04-25T01:54:34.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.202+0000 7f63ad64d700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fc790 0x7f63a8115df0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f639800bbb0 tx=0x7f6398009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.203+0000 7f63a5ffb700 1 -- 172.21.15.33:0/4181230539 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63980092e0 con 0x7f63a80fc790 2024-04-25T01:54:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.203+0000 7f63a5ffb700 1 -- 172.21.15.33:0/4181230539 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f639800cbd0 con 0x7f63a80fc790 2024-04-25T01:54:34.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.203+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a810ffd0 con 0x7f63a80fc790 2024-04-25T01:54:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.203+0000 7f63a5ffb700 1 -- 172.21.15.33:0/4181230539 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6398007400 con 0x7f63a80fc790 2024-04-25T01:54:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.203+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a8110510 con 0x7f63a80fc790 2024-04-25T01:54:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.204+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a80fc330 con 0x7f63a80fc790 2024-04-25T01:54:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.204+0000 7f63a5ffb700 1 -- 172.21.15.33:0/4181230539 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6398012070 con 0x7f63a80fc790 2024-04-25T01:54:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.205+0000 7f63a5ffb700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f639006e150 0x7f6390070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.205+0000 7f63a5ffb700 1 -- 172.21.15.33:0/4181230539 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f639808d710 con 0x7f63a80fc790 2024-04-25T01:54:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.205+0000 7f63ace4c700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f639006e150 0x7f6390070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:34.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.205+0000 7f63ace4c700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f639006e150 0x7f6390070600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f639c00b020 tx=0x7f639c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:34.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.209+0000 7f63a5ffb700 1 -- 172.21.15.33:0/4181230539 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f639805a640 con 0x7f63a80fc790 2024-04-25T01:54:34.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:33 smithi138 ceph-mon[81304]: pgmap v454: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:34.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:33 smithi138 ceph-mon[81304]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:34.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:33 smithi138 ceph-mon[81304]: from='client.34880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:34.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:33 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3417941774' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:54:34.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.434+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63a804bf40 con 0x7f639006e150 2024-04-25T01:54:34.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.435+0000 7f63a5ffb700 1 -- 172.21.15.33:0/4181230539 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f63a804bf40 con 0x7f639006e150 2024-04-25T01:54:34.436 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:54:34.436 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:54:34.436 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:54:34.436 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:54:34.436 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:54:34.436 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:54:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:54:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:54:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:54:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:54:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:54:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:54:34.437 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:54:34.437 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f639006e150 msgr2=0x7f6390070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f639006e150 0x7f6390070600 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f639c00b020 tx=0x7f639c009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fc790 msgr2=0x7f63a8115df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fc790 0x7f63a8115df0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f639800bbb0 tx=0x7f6398009840 comp rx=0 tx=0).stop 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 shutdown_connections 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f639006e150 0x7f6390070600 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f63a80fc790 0x7f63a8115df0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 --2- 172.21.15.33:0/4181230539 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f63a80fd0a0 0x7f63a8116330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 >> 172.21.15.33:0/4181230539 conn(0x7f63a8071d00 msgr2=0x7f63a80f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 shutdown_connections 2024-04-25T01:54:34.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.437+0000 7f63af8b1700 1 -- 172.21.15.33:0/4181230539 wait complete. 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.541+0000 7f78070f0700 1 -- 172.21.15.33:0/2215205483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 msgr2=0x7f78000fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.541+0000 7f78070f0700 1 --2- 172.21.15.33:0/2215205483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 0x7f78000fcdf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f77ec004940 tx=0x7f77ec009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.541+0000 7f78070f0700 1 -- 172.21.15.33:0/2215205483 shutdown_connections 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.541+0000 7f78070f0700 1 --2- 172.21.15.33:0/2215205483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 0x7f78000fcdf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.541+0000 7f78070f0700 1 --2- 172.21.15.33:0/2215205483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7800102de0 0x7f78001031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.541+0000 7f78070f0700 1 -- 172.21.15.33:0/2215205483 >> 172.21.15.33:0/2215205483 conn(0x7f78000f8990 msgr2=0x7f78000fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.541+0000 7f78070f0700 1 -- 172.21.15.33:0/2215205483 shutdown_connections 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.541+0000 7f78070f0700 1 -- 172.21.15.33:0/2215205483 wait complete. 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f78070f0700 1 Processor -- start 2024-04-25T01:54:34.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f78070f0700 1 -- start start 2024-04-25T01:54:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f78070f0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7800102de0 0x7f7800113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f78070f0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 0x7f7800113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f78070f0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7800114520 con 0x7f7800103780 2024-04-25T01:54:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f78070f0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780010db20 con 0x7f7800102de0 2024-04-25T01:54:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f77fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 0x7f7800113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:34.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f77fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 0x7f7800113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51342/0 (socket says 172.21.15.33:51342) 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f77fffff700 1 -- 172.21.15.33:0/2430832717 learned_addr learned my addr 172.21.15.33:0/2430832717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f7804e8c700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7800102de0 0x7f7800113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f77fffff700 1 -- 172.21.15.33:0/2430832717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7800102de0 msgr2=0x7f7800113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f77fffff700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7800102de0 0x7f7800113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.542+0000 7f77fffff700 1 -- 172.21.15.33:0/2430832717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ec005040 con 0x7f7800103780 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.543+0000 7f77fffff700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 0x7f7800113ed0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009180 tx=0x7f77ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.543+0000 7f77fd7fa700 1 -- 172.21.15.33:0/2430832717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77ec00e040 con 0x7f7800103780 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.543+0000 7f77fd7fa700 1 -- 172.21.15.33:0/2430832717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77ec00bbc0 con 0x7f7800103780 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.543+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780010dda0 con 0x7f7800103780 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.543+0000 7f77fd7fa700 1 -- 172.21.15.33:0/2430832717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77ec01c3f0 con 0x7f7800103780 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.543+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780010e360 con 0x7f7800103780 2024-04-25T01:54:34.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.544+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78000fe530 con 0x7f7800103780 2024-04-25T01:54:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.545+0000 7f77fd7fa700 1 -- 172.21.15.33:0/2430832717 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f77ec00bd30 con 0x7f7800103780 2024-04-25T01:54:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.545+0000 7f77fd7fa700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f77f006e140 0x7f77f00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:54:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.545+0000 7f77fd7fa700 1 -- 172.21.15.33:0/2430832717 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f77ec08ed80 con 0x7f7800103780 2024-04-25T01:54:34.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.545+0000 7f7804e8c700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f77f006e140 0x7f77f00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:54:34.548 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.545+0000 7f7804e8c700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f77f006e140 0x7f77f00705f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77f4007fd0 tx=0x7f77f4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:54:34.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.550+0000 7f77fd7fa700 1 -- 172.21.15.33:0/2430832717 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f77ec05bd50 con 0x7f7800103780 2024-04-25T01:54:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.847+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7800109ec0 con 0x7f7800103780 2024-04-25T01:54:34.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.847+0000 7f77fd7fa700 1 -- 172.21.15.33:0/2430832717 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f77ec05b8e0 con 0x7f7800103780 2024-04-25T01:54:34.849 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:54:34.849 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:54:34.849 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:54:34.849 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:54:34.849 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:54:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f77f006e140 msgr2=0x7f77f00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f77f006e140 0x7f77f00705f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77f4007fd0 tx=0x7f77f4009040 comp rx=0 tx=0).stop 2024-04-25T01:54:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 msgr2=0x7f7800113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:54:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 0x7f7800113ed0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009180 tx=0x7f77ec009e30 comp rx=0 tx=0).stop 2024-04-25T01:54:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 shutdown_connections 2024-04-25T01:54:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f77f006e140 0x7f77f00705f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7800102de0 0x7f7800113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 --2- 172.21.15.33:0/2430832717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7800103780 0x7f7800113ed0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:54:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 >> 172.21.15.33:0/2430832717 conn(0x7f78000f8990 msgr2=0x7f78000f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:54:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.849+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 shutdown_connections 2024-04-25T01:54:34.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:54:34.850+0000 7f78070f0700 1 -- 172.21.15.33:0/2430832717 wait complete. 2024-04-25T01:54:35.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:34 smithi033 ceph-mon[111306]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:35.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:34 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2300493729' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:54:35.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:34 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2430832717' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:54:35.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:34 smithi138 ceph-mon[81304]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:35.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:34 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2300493729' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:54:35.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:34 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2430832717' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:54:36.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:35 smithi033 ceph-mon[111306]: pgmap v455: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:36.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:35 smithi033 ceph-mon[111306]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:36.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:35 smithi138 ceph-mon[81304]: pgmap v455: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:36.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:35 smithi138 ceph-mon[81304]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:54:38.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:37 smithi033 ceph-mon[111306]: pgmap v456: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:38.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:37 smithi138 ceph-mon[81304]: pgmap v456: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:40.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:39 smithi033 ceph-mon[111306]: pgmap v457: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:40.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:39 smithi138 ceph-mon[81304]: pgmap v457: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:41.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:54:41.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:54:42.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:41 smithi033 ceph-mon[111306]: pgmap v458: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:42.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:41 smithi138 ceph-mon[81304]: pgmap v458: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:44.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:43 smithi033 ceph-mon[111306]: pgmap v459: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:44.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:43 smithi138 ceph-mon[81304]: pgmap v459: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:46.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:45 smithi033 ceph-mon[111306]: pgmap v460: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:46.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:45 smithi138 ceph-mon[81304]: pgmap v460: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:48.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:47 smithi033 ceph-mon[111306]: pgmap v461: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:48.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:47 smithi138 ceph-mon[81304]: pgmap v461: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:50.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:49 smithi033 ceph-mon[111306]: pgmap v462: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:50.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:49 smithi138 ceph-mon[81304]: pgmap v462: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:52.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:51 smithi033 ceph-mon[111306]: pgmap v463: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:52.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:51 smithi138 ceph-mon[81304]: pgmap v463: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:54.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:53 smithi033 ceph-mon[111306]: pgmap v464: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:54.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:53 smithi138 ceph-mon[81304]: pgmap v464: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:56.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:55 smithi033 ceph-mon[111306]: pgmap v465: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:56.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:54:56.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:55 smithi138 ceph-mon[81304]: pgmap v465: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:56.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:54:58.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:57 smithi033 ceph-mon[111306]: pgmap v466: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:54:58.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:57 smithi138 ceph-mon[81304]: pgmap v466: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:00.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:54:59 smithi033 ceph-mon[111306]: pgmap v467: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:00.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:54:59 smithi138 ceph-mon[81304]: pgmap v467: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:02.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:01 smithi033 ceph-mon[111306]: pgmap v468: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:02.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:01 smithi138 ceph-mon[81304]: pgmap v468: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:04.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:03 smithi033 ceph-mon[111306]: pgmap v469: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:04.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:03 smithi138 ceph-mon[81304]: pgmap v469: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.968+0000 7fbc04085700 1 -- 172.21.15.33:0/2342313387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 msgr2=0x7fbbfc103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.968+0000 7fbc04085700 1 --2- 172.21.15.33:0/2342313387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 0x7fbbfc103220 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8004940 tx=0x7fbbf8009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.968+0000 7fbc04085700 1 -- 172.21.15.33:0/2342313387 shutdown_connections 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.968+0000 7fbc04085700 1 --2- 172.21.15.33:0/2342313387 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbfc065640 0x7fbbfc065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.968+0000 7fbc04085700 1 --2- 172.21.15.33:0/2342313387 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 0x7fbbfc103220 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.968+0000 7fbc04085700 1 -- 172.21.15.33:0/2342313387 >> 172.21.15.33:0/2342313387 conn(0x7fbbfc071d00 msgr2=0x7fbbfc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.968+0000 7fbc04085700 1 -- 172.21.15.33:0/2342313387 shutdown_connections 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc04085700 1 -- 172.21.15.33:0/2342313387 wait complete. 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc04085700 1 Processor -- start 2024-04-25T01:55:04.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc04085700 1 -- start start 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc04085700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbfc065640 0x7fbbfc0fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc04085700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 0x7fbbfc0fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc04085700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0fc880 con 0x7fbbfc102e50 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc04085700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbfc0fc9c0 con 0x7fbbfc065640 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc01620700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 0x7fbbfc0fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc01620700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 0x7fbbfc0fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60066/0 (socket says 172.21.15.33:60066) 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.969+0000 7fbc01620700 1 -- 172.21.15.33:0/283027015 learned_addr learned my addr 172.21.15.33:0/283027015 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbc01e21700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbfc065640 0x7fbbfc0fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbc01620700 1 -- 172.21.15.33:0/283027015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbfc065640 msgr2=0x7fbbfc0fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbc01620700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbfc065640 0x7fbbfc0fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbc01620700 1 -- 172.21.15.33:0/283027015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf8005040 con 0x7fbbfc102e50 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbc01620700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 0x7fbbfc0fc2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbbec002a50 tx=0x7fbbec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:04.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbbf27fc700 1 -- 172.21.15.33:0/283027015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbec007cc0 con 0x7fbbfc102e50 2024-04-25T01:55:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbbf27fc700 1 -- 172.21.15.33:0/283027015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbec00f040 con 0x7fbbfc102e50 2024-04-25T01:55:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc119e80 con 0x7fbbfc102e50 2024-04-25T01:55:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbbf27fc700 1 -- 172.21.15.33:0/283027015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbec009640 con 0x7fbbfc102e50 2024-04-25T01:55:04.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.970+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbfc11a300 con 0x7fbbfc102e50 2024-04-25T01:55:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.971+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbfc100100 con 0x7fbbfc102e50 2024-04-25T01:55:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.972+0000 7fbbf27fc700 1 -- 172.21.15.33:0/283027015 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbbec0097c0 con 0x7fbbfc102e50 2024-04-25T01:55:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.972+0000 7fbbf27fc700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbbe806e140 0x7fbbe80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.972+0000 7fbbf27fc700 1 -- 172.21.15.33:0/283027015 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbbec08d890 con 0x7fbbfc102e50 2024-04-25T01:55:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.972+0000 7fbc01e21700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbbe806e140 0x7fbbe80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:04.975 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.973+0000 7fbc01e21700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbbe806e140 0x7fbbe80705f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8004f90 tx=0x7fbbf8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:04.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:04.977+0000 7fbbf27fc700 1 -- 172.21.15.33:0/283027015 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbbec056bf0 con 0x7fbbfc102e50 2024-04-25T01:55:05.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.205+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbfc0fcb50 con 0x7fbbe806e140 2024-04-25T01:55:05.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.206+0000 7fbbf27fc700 1 -- 172.21.15.33:0/283027015 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbbfc0fcb50 con 0x7fbbe806e140 2024-04-25T01:55:05.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbbe806e140 msgr2=0x7fbbe80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbbe806e140 0x7fbbe80705f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8004f90 tx=0x7fbbf8009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:05.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 msgr2=0x7fbbfc0fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 0x7fbbfc0fc2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbbec002a50 tx=0x7fbbec0046b0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 shutdown_connections 2024-04-25T01:55:05.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbbe806e140 0x7fbbe80705f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbbfc065640 0x7fbbfc0fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 --2- 172.21.15.33:0/283027015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbbfc102e50 0x7fbbfc0fc2f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 >> 172.21.15.33:0/283027015 conn(0x7fbbfc071d00 msgr2=0x7fbbfc0fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:05.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 shutdown_connections 2024-04-25T01:55:05.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.208+0000 7fbc04085700 1 -- 172.21.15.33:0/283027015 wait complete. 2024-04-25T01:55:05.220 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:55:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 -- 172.21.15.33:0/1779066839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf400fb520 msgr2=0x7fdf400fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 --2- 172.21.15.33:0/1779066839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf400fb520 0x7fdf400fb990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fdf34004940 tx=0x7fdf34009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 -- 172.21.15.33:0/1779066839 shutdown_connections 2024-04-25T01:55:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 --2- 172.21.15.33:0/1779066839 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf400fb520 0x7fdf400fb990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 --2- 172.21.15.33:0/1779066839 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf40102f60 0x7fdf40103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 -- 172.21.15.33:0/1779066839 >> 172.21.15.33:0/1779066839 conn(0x7fdf400714d0 msgr2=0x7fdf400718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 -- 172.21.15.33:0/1779066839 shutdown_connections 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 -- 172.21.15.33:0/1779066839 wait complete. 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 Processor -- start 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.314+0000 7fdf44928700 1 -- start start 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf44928700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf400fb520 0x7fdf40113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf44928700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40102f60 0x7fdf40113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf44928700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf40114590 con 0x7fdf40102f60 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf44928700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4010dbb0 con 0x7fdf400fb520 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf3dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40102f60 0x7fdf40113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf3dd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40102f60 0x7fdf40113f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60092/0 (socket says 172.21.15.33:60092) 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf3dd9b700 1 -- 172.21.15.33:0/1669530590 learned_addr learned my addr 172.21.15.33:0/1669530590 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf3e59c700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf400fb520 0x7fdf40113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf3dd9b700 1 -- 172.21.15.33:0/1669530590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf400fb520 msgr2=0x7fdf40113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf3dd9b700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf400fb520 0x7fdf40113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf3dd9b700 1 -- 172.21.15.33:0/1669530590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf34005040 con 0x7fdf40102f60 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf3dd9b700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40102f60 0x7fdf40113f40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400b3b0 tx=0x7fdf34009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf2effd700 1 -- 172.21.15.33:0/1669530590 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf3400e040 con 0x7fdf40102f60 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4010de30 con 0x7fdf40102f60 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf2effd700 1 -- 172.21.15.33:0/1669530590 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf3400bbc0 con 0x7fdf40102f60 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf2effd700 1 -- 172.21.15.33:0/1669530590 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf3401c3f0 con 0x7fdf40102f60 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.315+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4010e370 con 0x7fdf40102f60 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.316+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf40100250 con 0x7fdf40102f60 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.317+0000 7fdf2effd700 1 -- 172.21.15.33:0/1669530590 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdf3400bd30 con 0x7fdf40102f60 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.318+0000 7fdf2effd700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2806e140 0x7fdf280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.318+0000 7fdf2effd700 1 -- 172.21.15.33:0/1669530590 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdf3408edc0 con 0x7fdf40102f60 2024-04-25T01:55:05.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.318+0000 7fdf3e59c700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2806e140 0x7fdf280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:05.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.318+0000 7fdf3e59c700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2806e140 0x7fdf280705f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdf30007fd0 tx=0x7fdf30009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:05.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.322+0000 7fdf2effd700 1 -- 172.21.15.33:0/1669530590 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdf34058140 con 0x7fdf40102f60 2024-04-25T01:55:05.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.552+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf4010f090 con 0x7fdf2806e140 2024-04-25T01:55:05.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.553+0000 7fdf2effd700 1 -- 172.21.15.33:0/1669530590 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdf4010f090 con 0x7fdf2806e140 2024-04-25T01:55:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.555+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2806e140 msgr2=0x7fdf280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.555+0000 7fdf44928700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2806e140 0x7fdf280705f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdf30007fd0 tx=0x7fdf30009040 comp rx=0 tx=0).stop 2024-04-25T01:55:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.555+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40102f60 msgr2=0x7fdf40113f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.555+0000 7fdf44928700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40102f60 0x7fdf40113f40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdf3400b3b0 tx=0x7fdf34009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.555+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 shutdown_connections 2024-04-25T01:55:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.555+0000 7fdf44928700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2806e140 0x7fdf280705f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.555+0000 7fdf44928700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf400fb520 0x7fdf40113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.556+0000 7fdf44928700 1 --2- 172.21.15.33:0/1669530590 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf40102f60 0x7fdf40113f40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.556+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 >> 172.21.15.33:0/1669530590 conn(0x7fdf400714d0 msgr2=0x7fdf400fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.556+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 shutdown_connections 2024-04-25T01:55:05.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.556+0000 7fdf44928700 1 -- 172.21.15.33:0/1669530590 wait complete. 2024-04-25T01:55:05.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.660+0000 7f236b23f700 1 -- 172.21.15.33:0/2789556972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23641026a0 msgr2=0x7f23640fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.660+0000 7f236b23f700 1 --2- 172.21.15.33:0/2789556972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23641026a0 0x7f23640fbd10 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2350004940 tx=0x7f2350009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:05.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.660+0000 7f236b23f700 1 -- 172.21.15.33:0/2789556972 shutdown_connections 2024-04-25T01:55:05.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.660+0000 7f236b23f700 1 --2- 172.21.15.33:0/2789556972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23641026a0 0x7f23640fbd10 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.660+0000 7f236b23f700 1 --2- 172.21.15.33:0/2789556972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2364101d00 0x7f23641020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.660+0000 7f236b23f700 1 -- 172.21.15.33:0/2789556972 >> 172.21.15.33:0/2789556972 conn(0x7f23640f78d0 msgr2=0x7f23640f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.661+0000 7f236b23f700 1 -- 172.21.15.33:0/2789556972 shutdown_connections 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.661+0000 7f236b23f700 1 -- 172.21.15.33:0/2789556972 wait complete. 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.661+0000 7f236b23f700 1 Processor -- start 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.661+0000 7f236b23f700 1 -- start start 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.661+0000 7f236b23f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2364101d00 0x7f236410eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f236b23f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23641026a0 0x7f236410f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f236b23f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236410f770 con 0x7f2364101d00 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f236b23f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236410f8b0 con 0x7f23641026a0 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f2368fdb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2364101d00 0x7f236410eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f2368fdb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2364101d00 0x7f236410eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35710/0 (socket says 172.21.15.33:35710) 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f2368fdb700 1 -- 172.21.15.33:0/2787010174 learned_addr learned my addr 172.21.15.33:0/2787010174 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f2368fdb700 1 -- 172.21.15.33:0/2787010174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23641026a0 msgr2=0x7f236410f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f2368fdb700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23641026a0 0x7f236410f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f2368fdb700 1 -- 172.21.15.33:0/2787010174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2350005040 con 0x7f2364101d00 2024-04-25T01:55:05.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f2368fdb700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2364101d00 0x7f236410eb50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f235800b790 tx=0x7f2358009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:05.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f23617fa700 1 -- 172.21.15.33:0/2787010174 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f235800f770 con 0x7f2364101d00 2024-04-25T01:55:05.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f23617fa700 1 -- 172.21.15.33:0/2787010174 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f235800fda0 con 0x7f2364101d00 2024-04-25T01:55:05.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2364117c20 con 0x7f2364101d00 2024-04-25T01:55:05.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f23617fa700 1 -- 172.21.15.33:0/2787010174 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f235800d310 con 0x7f2364101d00 2024-04-25T01:55:05.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.662+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23641181b0 con 0x7f2364101d00 2024-04-25T01:55:05.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.663+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23640fd450 con 0x7f2364101d00 2024-04-25T01:55:05.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.664+0000 7f23617fa700 1 -- 172.21.15.33:0/2787010174 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2358010420 con 0x7f2364101d00 2024-04-25T01:55:05.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.664+0000 7f23617fa700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f235406e140 0x7f23540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:05.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.664+0000 7f23617fa700 1 -- 172.21.15.33:0/2787010174 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f235808db50 con 0x7f2364101d00 2024-04-25T01:55:05.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.664+0000 7f2363fff700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f235406e140 0x7f23540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:05.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.665+0000 7f2363fff700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f235406e140 0x7f23540705f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2350004f90 tx=0x7f2350009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:05.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.669+0000 7f23617fa700 1 -- 172.21.15.33:0/2787010174 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2358056f50 con 0x7f2364101d00 2024-04-25T01:55:05.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.885+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f236404bed0 con 0x7f235406e140 2024-04-25T01:55:05.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.896+0000 7f23617fa700 1 -- 172.21.15.33:0/2787010174 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f236404bed0 con 0x7f235406e140 2024-04-25T01:55:05.898 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:55:05.898 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (16m) 9m ago 27m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:55:05.898 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (13m) 9m ago 27m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:55:05.898 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (13m) 9m ago 26m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (15m) 9m ago 27m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (9m) 9m ago 21m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (9m) 9m ago 22m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (9m) 9m ago 22m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (21m) 9m ago 21m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (17m) 9m ago 28m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (17m) 9m ago 26m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (14m) 9m ago 29m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (14m) 9m ago 26m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:55:05.899 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (16m) 9m ago 27m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:55:05.900 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (16m) 9m ago 26m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:55:05.900 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (13m) 9m ago 25m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:55:05.900 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (12m) 9m ago 25m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:55:05.900 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (11m) 9m ago 24m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:55:05.900 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (11m) 9m ago 24m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:55:05.900 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (10m) 9m ago 23m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:55:05.900 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (10m) 9m ago 23m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:55:05.900 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (16m) 9m ago 26m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:55:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.898+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f235406e140 msgr2=0x7f23540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.898+0000 7f236b23f700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f235406e140 0x7f23540705f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2350004f90 tx=0x7f2350009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.898+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2364101d00 msgr2=0x7f236410eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.898+0000 7f236b23f700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2364101d00 0x7f236410eb50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f235800b790 tx=0x7f2358009250 comp rx=0 tx=0).stop 2024-04-25T01:55:05.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.898+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 shutdown_connections 2024-04-25T01:55:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.898+0000 7f236b23f700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f235406e140 0x7f23540705f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.898+0000 7f236b23f700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2364101d00 0x7f236410eb50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.898+0000 7f236b23f700 1 --2- 172.21.15.33:0/2787010174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23641026a0 0x7f236410f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.898+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 >> 172.21.15.33:0/2787010174 conn(0x7f23640f78d0 msgr2=0x7f23640f8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.899+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 shutdown_connections 2024-04-25T01:55:05.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:05.899+0000 7f236b23f700 1 -- 172.21.15.33:0/2787010174 wait complete. 2024-04-25T01:55:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.002+0000 7fe37739c700 1 -- 172.21.15.33:0/3890835333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 msgr2=0x7fe370103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.002+0000 7fe37739c700 1 --2- 172.21.15.33:0/3890835333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 0x7fe370103220 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe36c004940 tx=0x7fe36c009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.002+0000 7fe37739c700 1 -- 172.21.15.33:0/3890835333 shutdown_connections 2024-04-25T01:55:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.002+0000 7fe37739c700 1 --2- 172.21.15.33:0/3890835333 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe370065640 0x7fe370065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.002+0000 7fe37739c700 1 --2- 172.21.15.33:0/3890835333 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 0x7fe370103220 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.002+0000 7fe37739c700 1 -- 172.21.15.33:0/3890835333 >> 172.21.15.33:0/3890835333 conn(0x7fe370071d00 msgr2=0x7fe370072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.002+0000 7fe37739c700 1 -- 172.21.15.33:0/3890835333 shutdown_connections 2024-04-25T01:55:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.002+0000 7fe37739c700 1 -- 172.21.15.33:0/3890835333 wait complete. 2024-04-25T01:55:06.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.002+0000 7fe37739c700 1 Processor -- start 2024-04-25T01:55:06.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe37739c700 1 -- start start 2024-04-25T01:55:06.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe37739c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe370065640 0x7fe370113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:06.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe37739c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 0x7fe3701140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:06.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe37739c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe370114710 con 0x7fe370102e50 2024-04-25T01:55:06.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe37739c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37010db00 con 0x7fe370065640 2024-04-25T01:55:06.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe374937700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 0x7fe3701140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:06.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe374937700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 0x7fe3701140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35730/0 (socket says 172.21.15.33:35730) 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe374937700 1 -- 172.21.15.33:0/2383808096 learned_addr learned my addr 172.21.15.33:0/2383808096 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe375138700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe370065640 0x7fe370113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe374937700 1 -- 172.21.15.33:0/2383808096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe370065640 msgr2=0x7fe370113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe374937700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe370065640 0x7fe370113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe374937700 1 -- 172.21.15.33:0/2383808096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c005040 con 0x7fe370102e50 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe374937700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 0x7fe3701140c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe360002a50 tx=0x7fe3600046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.003+0000 7fe365ffb700 1 -- 172.21.15.33:0/2383808096 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe360007cc0 con 0x7fe370102e50 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.004+0000 7fe365ffb700 1 -- 172.21.15.33:0/2383808096 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe36000f040 con 0x7fe370102e50 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.004+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37010dde0 con 0x7fe370102e50 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.004+0000 7fe365ffb700 1 -- 172.21.15.33:0/2383808096 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe360009640 con 0x7fe370102e50 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.004+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37010e3a0 con 0x7fe370102e50 2024-04-25T01:55:06.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.005+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe370100100 con 0x7fe370102e50 2024-04-25T01:55:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.005+0000 7fe365ffb700 1 -- 172.21.15.33:0/2383808096 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe3600097a0 con 0x7fe370102e50 2024-04-25T01:55:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.006+0000 7fe365ffb700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35c06e150 0x7fe35c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.006+0000 7fe365ffb700 1 -- 172.21.15.33:0/2383808096 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe36008d850 con 0x7fe370102e50 2024-04-25T01:55:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.006+0000 7fe375138700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35c06e150 0x7fe35c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:06.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.006+0000 7fe375138700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35c06e150 0x7fe35c070600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00b3b0 tx=0x7fe36c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:06.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.010+0000 7fe365ffb700 1 -- 172.21.15.33:0/2383808096 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe36005a780 con 0x7fe370102e50 2024-04-25T01:55:06.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:05 smithi033 ceph-mon[111306]: pgmap v470: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:06.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:05 smithi033 ceph-mon[111306]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:06.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:05 smithi138 ceph-mon[81304]: pgmap v470: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:06.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:05 smithi138 ceph-mon[81304]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:06.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.302+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe37010f1d0 con 0x7fe370102e50 2024-04-25T01:55:06.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.303+0000 7fe365ffb700 1 -- 172.21.15.33:0/2383808096 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fe36005a310 con 0x7fe370102e50 2024-04-25T01:55:06.304 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:55:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:55:06.304 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:55:06.305 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:55:06.306 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35c06e150 msgr2=0x7fe35c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35c06e150 0x7fe35c070600 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe36c00b3b0 tx=0x7fe36c009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 msgr2=0x7fe3701140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 0x7fe3701140c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe360002a50 tx=0x7fe3600046b0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 shutdown_connections 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe35c06e150 0x7fe35c070600 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe370065640 0x7fe370113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 --2- 172.21.15.33:0/2383808096 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe370102e50 0x7fe3701140c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 >> 172.21.15.33:0/2383808096 conn(0x7fe370071d00 msgr2=0x7fe3700fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 shutdown_connections 2024-04-25T01:55:06.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.305+0000 7fe37739c700 1 -- 172.21.15.33:0/2383808096 wait complete. 2024-04-25T01:55:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.409+0000 7fce6df0b700 1 -- 172.21.15.33:0/990612009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68102de0 msgr2=0x7fce681031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.409+0000 7fce6df0b700 1 --2- 172.21.15.33:0/990612009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68102de0 0x7fce681031b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fce58004940 tx=0x7fce58009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.409+0000 7fce6df0b700 1 -- 172.21.15.33:0/990612009 shutdown_connections 2024-04-25T01:55:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.409+0000 7fce6df0b700 1 --2- 172.21.15.33:0/990612009 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce68103780 0x7fce680fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.409+0000 7fce6df0b700 1 --2- 172.21.15.33:0/990612009 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68102de0 0x7fce681031b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.409+0000 7fce6df0b700 1 -- 172.21.15.33:0/990612009 >> 172.21.15.33:0/990612009 conn(0x7fce680f8990 msgr2=0x7fce680fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.409+0000 7fce6df0b700 1 -- 172.21.15.33:0/990612009 shutdown_connections 2024-04-25T01:55:06.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.409+0000 7fce6df0b700 1 -- 172.21.15.33:0/990612009 wait complete. 2024-04-25T01:55:06.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce6df0b700 1 Processor -- start 2024-04-25T01:55:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce6df0b700 1 -- start start 2024-04-25T01:55:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce6df0b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce68102de0 0x7fce68113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:06.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce6df0b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68103780 0x7fce68113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce6df0b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce68114550 con 0x7fce68103780 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce6df0b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce6810dac0 con 0x7fce68102de0 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68103780 0x7fce68113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce66ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68103780 0x7fce68113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35742/0 (socket says 172.21.15.33:35742) 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce66ffd700 1 -- 172.21.15.33:0/3692605686 learned_addr learned my addr 172.21.15.33:0/3692605686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.410+0000 7fce677fe700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce68102de0 0x7fce68113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.411+0000 7fce66ffd700 1 -- 172.21.15.33:0/3692605686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce68102de0 msgr2=0x7fce68113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.411+0000 7fce66ffd700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce68102de0 0x7fce68113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.411+0000 7fce66ffd700 1 -- 172.21.15.33:0/3692605686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce58005040 con 0x7fce68103780 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.411+0000 7fce66ffd700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68103780 0x7fce68113e70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00b790 tx=0x7fce5c009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.411+0000 7fce4ffff700 1 -- 172.21.15.33:0/3692605686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce5c00f6f0 con 0x7fce68103780 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.411+0000 7fce4ffff700 1 -- 172.21.15.33:0/3692605686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce5c00fd20 con 0x7fce68103780 2024-04-25T01:55:06.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.411+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6810dda0 con 0x7fce68103780 2024-04-25T01:55:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.411+0000 7fce4ffff700 1 -- 172.21.15.33:0/3692605686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce5c0103a0 con 0x7fce68103780 2024-04-25T01:55:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.411+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6810e360 con 0x7fce68103780 2024-04-25T01:55:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.412+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce681000f0 con 0x7fce68103780 2024-04-25T01:55:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.413+0000 7fce4ffff700 1 -- 172.21.15.33:0/3692605686 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fce5c011400 con 0x7fce68103780 2024-04-25T01:55:06.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.413+0000 7fce4ffff700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fce5006e140 0x7fce500705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.413+0000 7fce4ffff700 1 -- 172.21.15.33:0/3692605686 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fce5c08d8e0 con 0x7fce68103780 2024-04-25T01:55:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.413+0000 7fce677fe700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fce5006e140 0x7fce500705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:06.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.413+0000 7fce677fe700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fce5006e140 0x7fce500705f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce58004f90 tx=0x7fce58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:06.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.418+0000 7fce4ffff700 1 -- 172.21.15.33:0/3692605686 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fce5c05a810 con 0x7fce68103780 2024-04-25T01:55:06.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.669+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce6810ef80 con 0x7fce68103780 2024-04-25T01:55:06.672 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.670+0000 7fce4ffff700 1 -- 172.21.15.33:0/3692605686 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fce5c05a3a0 con 0x7fce68103780 2024-04-25T01:55:06.672 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:55:06.672 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:55:06.672 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:55:06.672 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:55:06.673 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:55:06.674 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:55:06.674 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:55:06.674 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:55:06.674 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:55:06.674 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:55:06.674 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:55:06.674 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:55:06.674 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:55:06.675 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:55:06.676 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:55:06.676 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:55:06.676 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:55:06.676 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:55:06.676 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fce5006e140 msgr2=0x7fce500705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fce5006e140 0x7fce500705f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce58004f90 tx=0x7fce58009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68103780 msgr2=0x7fce68113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68103780 0x7fce68113e70 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fce5c00b790 tx=0x7fce5c009220 comp rx=0 tx=0).stop 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 shutdown_connections 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fce5006e140 0x7fce500705f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce68102de0 0x7fce68113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 --2- 172.21.15.33:0/3692605686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce68103780 0x7fce68113e70 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 >> 172.21.15.33:0/3692605686 conn(0x7fce680f8990 msgr2=0x7fce680fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 shutdown_connections 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.672+0000 7fce6df0b700 1 -- 172.21.15.33:0/3692605686 wait complete. 2024-04-25T01:55:06.677 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:55:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.777+0000 7f5fb4512700 1 -- 172.21.15.33:0/1831823478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 msgr2=0x7f5fac0fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.777+0000 7f5fb4512700 1 --2- 172.21.15.33:0/1831823478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 0x7f5fac0fce10 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8004940 tx=0x7f5fa8009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.777+0000 7f5fb4512700 1 -- 172.21.15.33:0/1831823478 shutdown_connections 2024-04-25T01:55:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.777+0000 7f5fb4512700 1 --2- 172.21.15.33:0/1831823478 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 0x7f5fac0fce10 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.777+0000 7f5fb4512700 1 --2- 172.21.15.33:0/1831823478 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fac102e00 0x7f5fac1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.777+0000 7f5fb4512700 1 -- 172.21.15.33:0/1831823478 >> 172.21.15.33:0/1831823478 conn(0x7f5fac0f89d0 msgr2=0x7f5fac0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.777+0000 7f5fb4512700 1 -- 172.21.15.33:0/1831823478 shutdown_connections 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.777+0000 7f5fb4512700 1 -- 172.21.15.33:0/1831823478 wait complete. 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb4512700 1 Processor -- start 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb4512700 1 -- start start 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb4512700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fac102e00 0x7f5fac070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb4512700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 0x7f5fac0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb4512700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac0721b0 con 0x7f5fac1037a0 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb4512700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac070ce0 con 0x7f5fac102e00 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb1aad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 0x7f5fac0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:06.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb1aad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 0x7f5fac0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35760/0 (socket says 172.21.15.33:35760) 2024-04-25T01:55:06.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb1aad700 1 -- 172.21.15.33:0/1690940901 learned_addr learned my addr 172.21.15.33:0/1690940901 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:06.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.778+0000 7f5fb22ae700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fac102e00 0x7f5fac070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:06.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fb1aad700 1 -- 172.21.15.33:0/1690940901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fac102e00 msgr2=0x7f5fac070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:06.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fb1aad700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fac102e00 0x7f5fac070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:06.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fb1aad700 1 -- 172.21.15.33:0/1690940901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fa8005040 con 0x7f5fac1037a0 2024-04-25T01:55:06.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fb22ae700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fac102e00 0x7f5fac070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:55:06.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fb1aad700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 0x7f5fac0707a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009100 tx=0x7f5fa8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:06.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fa2ffd700 1 -- 172.21.15.33:0/1690940901 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fa800e040 con 0x7f5fac1037a0 2024-04-25T01:55:06.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fa2ffd700 1 -- 172.21.15.33:0/1690940901 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fa8013420 con 0x7f5fac1037a0 2024-04-25T01:55:06.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fac070f60 con 0x7f5fac1037a0 2024-04-25T01:55:06.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fa2ffd700 1 -- 172.21.15.33:0/1690940901 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fa801c630 con 0x7f5fac1037a0 2024-04-25T01:55:06.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.779+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fac11a0b0 con 0x7f5fac1037a0 2024-04-25T01:55:06.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.780+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac1052d0 con 0x7f5fac1037a0 2024-04-25T01:55:06.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.781+0000 7f5fa2ffd700 1 -- 172.21.15.33:0/1690940901 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5fa8013590 con 0x7f5fac1037a0 2024-04-25T01:55:06.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.781+0000 7f5fa2ffd700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5f9806e140 0x7f5f980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:06.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.781+0000 7f5fa2ffd700 1 -- 172.21.15.33:0/1690940901 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5fa808ec60 con 0x7f5fac1037a0 2024-04-25T01:55:06.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.781+0000 7f5fb22ae700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5f9806e140 0x7f5f980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:06.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.782+0000 7f5fb22ae700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5f9806e140 0x7f5f980705f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009fd0 tx=0x7f5f9c009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:06.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:06.786+0000 7f5fa2ffd700 1 -- 172.21.15.33:0/1690940901 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5fa8058100 con 0x7f5fac1037a0 2024-04-25T01:55:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.015+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fac109f00 con 0x7f5f9806e140 2024-04-25T01:55:07.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.016+0000 7f5fa2ffd700 1 -- 172.21.15.33:0/1690940901 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5fac109f00 con 0x7f5f9806e140 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:55:07.018 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:55:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:55:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:55:07.019 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:55:07.019 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5f9806e140 msgr2=0x7f5f980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5f9806e140 0x7f5f980705f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009fd0 tx=0x7f5f9c009f20 comp rx=0 tx=0).stop 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 msgr2=0x7f5fac0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 0x7f5fac0707a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009100 tx=0x7f5fa8009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 shutdown_connections 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5f9806e140 0x7f5f980705f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fac102e00 0x7f5fac070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 --2- 172.21.15.33:0/1690940901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fac1037a0 0x7f5fac0707a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 >> 172.21.15.33:0/1690940901 conn(0x7f5fac0f89d0 msgr2=0x7f5fac102170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 shutdown_connections 2024-04-25T01:55:07.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.018+0000 7f5fb4512700 1 -- 172.21.15.33:0/1690940901 wait complete. 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 -- 172.21.15.33:0/3453329661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 msgr2=0x7f9dc41031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 --2- 172.21.15.33:0/3453329661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 0x7f9dc41031b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9dc0004940 tx=0x7f9dc0009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 -- 172.21.15.33:0/3453329661 shutdown_connections 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 --2- 172.21.15.33:0/3453329661 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc4103780 0x7f9dc40fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 --2- 172.21.15.33:0/3453329661 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 0x7f9dc41031b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 -- 172.21.15.33:0/3453329661 >> 172.21.15.33:0/3453329661 conn(0x7f9dc40f8990 msgr2=0x7f9dc40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 -- 172.21.15.33:0/3453329661 shutdown_connections 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 -- 172.21.15.33:0/3453329661 wait complete. 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 Processor -- start 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 -- start start 2024-04-25T01:55:07.126 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 0x7f9dc4113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc4103780 0x7f9dc4113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc41145b0 con 0x7f9dc4102de0 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.123+0000 7f9dcc05f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc410db20 con 0x7f9dc4103780 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dc9dfb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 0x7f9dc4113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dc9dfb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 0x7f9dc4113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35786/0 (socket says 172.21.15.33:35786) 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dc9dfb700 1 -- 172.21.15.33:0/800934721 learned_addr learned my addr 172.21.15.33:0/800934721 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dc95fa700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc4103780 0x7f9dc4113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dc9dfb700 1 -- 172.21.15.33:0/800934721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc4103780 msgr2=0x7f9dc4113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dc9dfb700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc4103780 0x7f9dc4113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dc9dfb700 1 -- 172.21.15.33:0/800934721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc0005040 con 0x7f9dc4102de0 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dc9dfb700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 0x7f9dc4113990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b3b0 tx=0x7f9dc0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:07.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dba7fc700 1 -- 172.21.15.33:0/800934721 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc000e040 con 0x7f9dc4102de0 2024-04-25T01:55:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dba7fc700 1 -- 172.21.15.33:0/800934721 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9dc000bbc0 con 0x7f9dc4102de0 2024-04-25T01:55:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dba7fc700 1 -- 172.21.15.33:0/800934721 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dc001c3f0 con 0x7f9dc4102de0 2024-04-25T01:55:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc410dda0 con 0x7f9dc4102de0 2024-04-25T01:55:07.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.124+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc410e360 con 0x7f9dc4102de0 2024-04-25T01:55:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.125+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc41000f0 con 0x7f9dc4102de0 2024-04-25T01:55:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.126+0000 7f9dba7fc700 1 -- 172.21.15.33:0/800934721 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9dc000bd30 con 0x7f9dc4102de0 2024-04-25T01:55:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.126+0000 7f9dba7fc700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9db006e140 0x7f9db00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.126+0000 7f9dba7fc700 1 -- 172.21.15.33:0/800934721 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9dc008eba0 con 0x7f9dc4102de0 2024-04-25T01:55:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.127+0000 7f9dc95fa700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9db006e140 0x7f9db00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:07.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.127+0000 7f9dc95fa700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9db006e140 0x7f9db00705f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9db4006fd0 tx=0x7f9db4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:07.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.131+0000 7f9dba7fc700 1 -- 172.21.15.33:0/800934721 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9dc005ba50 con 0x7f9dc4102de0 2024-04-25T01:55:07.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:06 smithi033 ceph-mon[111306]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:07.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:06 smithi033 ceph-mon[111306]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:07.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:06 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2383808096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:55:07.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:06 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3692605686' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:55:07.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:06 smithi138 ceph-mon[81304]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:07.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:06 smithi138 ceph-mon[81304]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:07.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:06 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2383808096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:55:07.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:06 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3692605686' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:55:07.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.426+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9dc410f010 con 0x7f9dc4102de0 2024-04-25T01:55:07.428 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.426+0000 7f9dba7fc700 1 -- 172.21.15.33:0/800934721 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f9dc005b5e0 con 0x7f9dc4102de0 2024-04-25T01:55:07.428 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:55:07.428 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:55:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:55:07.428 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:55:07.428 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9db006e140 msgr2=0x7f9db00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9db006e140 0x7f9db00705f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9db4006fd0 tx=0x7f9db4008040 comp rx=0 tx=0).stop 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 msgr2=0x7f9dc4113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 0x7f9dc4113990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000b3b0 tx=0x7f9dc0009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 shutdown_connections 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9db006e140 0x7f9db00705f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9dc4102de0 0x7f9dc4113990 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 --2- 172.21.15.33:0/800934721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9dc4103780 0x7f9dc4113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 >> 172.21.15.33:0/800934721 conn(0x7f9dc40f8990 msgr2=0x7f9dc4107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:07.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 shutdown_connections 2024-04-25T01:55:07.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:07.428+0000 7f9dcc05f700 1 -- 172.21.15.33:0/800934721 wait complete. 2024-04-25T01:55:08.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:07 smithi033 ceph-mon[111306]: pgmap v471: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:08.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:07 smithi033 ceph-mon[111306]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:08.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:07 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/800934721' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:55:08.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:07 smithi138 ceph-mon[81304]: pgmap v471: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:08.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:07 smithi138 ceph-mon[81304]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:08.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:07 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/800934721' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:55:10.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:09 smithi033 ceph-mon[111306]: pgmap v472: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:10.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:09 smithi138 ceph-mon[81304]: pgmap v472: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:11.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:55:11.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:55:12.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:11 smithi033 ceph-mon[111306]: pgmap v473: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:12.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:11 smithi138 ceph-mon[81304]: pgmap v473: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:14.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:13 smithi033 ceph-mon[111306]: pgmap v474: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:14.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:13 smithi138 ceph-mon[81304]: pgmap v474: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:16.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:15 smithi033 ceph-mon[111306]: pgmap v475: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:16.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:15 smithi138 ceph-mon[81304]: pgmap v475: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:18.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:17 smithi033 ceph-mon[111306]: pgmap v476: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:18.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:17 smithi138 ceph-mon[81304]: pgmap v476: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:20.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:19 smithi033 ceph-mon[111306]: pgmap v477: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:20.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:19 smithi138 ceph-mon[81304]: pgmap v477: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:22.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:21 smithi033 ceph-mon[111306]: pgmap v478: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:22.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:21 smithi138 ceph-mon[81304]: pgmap v478: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:24.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:23 smithi033 ceph-mon[111306]: pgmap v479: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:24.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:23 smithi138 ceph-mon[81304]: pgmap v479: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:26.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:25 smithi033 ceph-mon[111306]: pgmap v480: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:26.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:55:26.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:25 smithi138 ceph-mon[81304]: pgmap v480: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:26.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:55:28.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:27 smithi033 ceph-mon[111306]: pgmap v481: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:28.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:28 smithi138 ceph-mon[81304]: pgmap v481: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:30.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:30 smithi033 ceph-mon[111306]: pgmap v482: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:30.292 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:30 smithi138 ceph-mon[81304]: pgmap v482: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:32.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:32 smithi033 ceph-mon[111306]: pgmap v483: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:32.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:32 smithi138 ceph-mon[81304]: pgmap v483: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:34.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:34 smithi033 ceph-mon[111306]: pgmap v484: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:34.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:34 smithi138 ceph-mon[81304]: pgmap v484: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:36.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:36 smithi033 ceph-mon[111306]: pgmap v485: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:36.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:36 smithi138 ceph-mon[81304]: pgmap v485: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.547+0000 7f4843fff700 1 -- 172.21.15.33:0/37977996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 msgr2=0x7f4844102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.547+0000 7f4843fff700 1 --2- 172.21.15.33:0/37977996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 0x7f4844102ff0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f4834004940 tx=0x7f4834009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.547+0000 7f4843fff700 1 -- 172.21.15.33:0/37977996 shutdown_connections 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.547+0000 7f4843fff700 1 --2- 172.21.15.33:0/37977996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48441035c0 0x7f48440fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.547+0000 7f4843fff700 1 --2- 172.21.15.33:0/37977996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 0x7f4844102ff0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.547+0000 7f4843fff700 1 -- 172.21.15.33:0/37977996 >> 172.21.15.33:0/37977996 conn(0x7f48440f8790 msgr2=0x7f48440faba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.547+0000 7f4843fff700 1 -- 172.21.15.33:0/37977996 shutdown_connections 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.547+0000 7f4843fff700 1 -- 172.21.15.33:0/37977996 wait complete. 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4843fff700 1 Processor -- start 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4843fff700 1 -- start start 2024-04-25T01:55:37.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4843fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 0x7f4844070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4843fff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48441035c0 0x7f48440707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4843fff700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844070d30 con 0x7f4844102c20 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4843fff700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4844070e70 con 0x7f48441035c0 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4842ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 0x7f4844070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4842ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 0x7f4844070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59312/0 (socket says 172.21.15.33:59312) 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4842ffd700 1 -- 172.21.15.33:0/174015958 learned_addr learned my addr 172.21.15.33:0/174015958 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f48427fc700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48441035c0 0x7f48440707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4842ffd700 1 -- 172.21.15.33:0/174015958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48441035c0 msgr2=0x7f48440707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.548+0000 7f4842ffd700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48441035c0 0x7f48440707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.549+0000 7f4842ffd700 1 -- 172.21.15.33:0/174015958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834005040 con 0x7f4844102c20 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.549+0000 7f48427fc700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48441035c0 0x7f48440707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.549+0000 7f4842ffd700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 0x7f4844070260 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f483400b8b0 tx=0x7f4834009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.549+0000 7f482b7fe700 1 -- 172.21.15.33:0/174015958 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483400e040 con 0x7f4844102c20 2024-04-25T01:55:37.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.549+0000 7f482b7fe700 1 -- 172.21.15.33:0/174015958 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4834004290 con 0x7f4844102c20 2024-04-25T01:55:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.549+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48440710c0 con 0x7f4844102c20 2024-04-25T01:55:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.549+0000 7f482b7fe700 1 -- 172.21.15.33:0/174015958 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483401c3f0 con 0x7f4844102c20 2024-04-25T01:55:37.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.549+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484411a080 con 0x7f4844102c20 2024-04-25T01:55:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.550+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48440fe370 con 0x7f4844102c20 2024-04-25T01:55:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.551+0000 7f482b7fe700 1 -- 172.21.15.33:0/174015958 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f48340137b0 con 0x7f4844102c20 2024-04-25T01:55:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.551+0000 7f482b7fe700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f482c06e140 0x7f482c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.551+0000 7f482b7fe700 1 -- 172.21.15.33:0/174015958 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f483408ee50 con 0x7f4844102c20 2024-04-25T01:55:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.551+0000 7f48427fc700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f482c06e140 0x7f482c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:37.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.552+0000 7f48427fc700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f482c06e140 0x7f482c0705f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4838007fd0 tx=0x7f4838009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:37.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.556+0000 7f482b7fe700 1 -- 172.21.15.33:0/174015958 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4834058250 con 0x7f4844102c20 2024-04-25T01:55:37.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.781+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4844071950 con 0x7f482c06e140 2024-04-25T01:55:37.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.782+0000 7f482b7fe700 1 -- 172.21.15.33:0/174015958 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4844071950 con 0x7f482c06e140 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f482c06e140 msgr2=0x7f482c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f482c06e140 0x7f482c0705f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4838007fd0 tx=0x7f4838009040 comp rx=0 tx=0).stop 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 msgr2=0x7f4844070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 0x7f4844070260 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f483400b8b0 tx=0x7f4834009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 shutdown_connections 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f482c06e140 0x7f482c0705f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4844102c20 0x7f4844070260 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 --2- 172.21.15.33:0/174015958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f48441035c0 0x7f48440707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 >> 172.21.15.33:0/174015958 conn(0x7f48440f8790 msgr2=0x7f48440f9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 shutdown_connections 2024-04-25T01:55:37.786 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.784+0000 7f4843fff700 1 -- 172.21.15.33:0/174015958 wait complete. 2024-04-25T01:55:37.796 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:55:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 -- 172.21.15.33:0/1195063753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 msgr2=0x7f7d641031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 --2- 172.21.15.33:0/1195063753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 0x7f7d641031b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7d54004940 tx=0x7f7d54009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 -- 172.21.15.33:0/1195063753 shutdown_connections 2024-04-25T01:55:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 --2- 172.21.15.33:0/1195063753 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d64103780 0x7f7d640fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 --2- 172.21.15.33:0/1195063753 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 0x7f7d641031b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:37.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 -- 172.21.15.33:0/1195063753 >> 172.21.15.33:0/1195063753 conn(0x7f7d640f89b0 msgr2=0x7f7d640fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 -- 172.21.15.33:0/1195063753 shutdown_connections 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 -- 172.21.15.33:0/1195063753 wait complete. 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 Processor -- start 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.889+0000 7f7d68cbe700 1 -- start start 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d68cbe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 0x7f7d64070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d68cbe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d64103780 0x7f7d640707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d68cbe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64070ce0 con 0x7f7d64102de0 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d68cbe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64070e20 con 0x7f7d64103780 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d6259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 0x7f7d64070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d6259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 0x7f7d64070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59332/0 (socket says 172.21.15.33:59332) 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d6259c700 1 -- 172.21.15.33:0/3426393219 learned_addr learned my addr 172.21.15.33:0/3426393219 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d61d9b700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d64103780 0x7f7d640707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d6259c700 1 -- 172.21.15.33:0/3426393219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d64103780 msgr2=0x7f7d640707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d6259c700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d64103780 0x7f7d640707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:37.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d6259c700 1 -- 172.21.15.33:0/3426393219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d54005040 con 0x7f7d64102de0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d6259c700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 0x7f7d64070260 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009150 tx=0x7f7d54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d52ffd700 1 -- 172.21.15.33:0/3426393219 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5400e040 con 0x7f7d64102de0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d64071070 con 0x7f7d64102de0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.890+0000 7f7d52ffd700 1 -- 172.21.15.33:0/3426393219 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d54013420 con 0x7f7d64102de0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.891+0000 7f7d52ffd700 1 -- 172.21.15.33:0/3426393219 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d5401c630 con 0x7f7d64102de0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.891+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6411a260 con 0x7f7d64102de0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.892+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d641000f0 con 0x7f7d64102de0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.892+0000 7f7d52ffd700 1 -- 172.21.15.33:0/3426393219 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7d54013590 con 0x7f7d64102de0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.892+0000 7f7d52ffd700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d4c06e150 0x7f7d4c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.893+0000 7f7d52ffd700 1 -- 172.21.15.33:0/3426393219 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7d5408ece0 con 0x7f7d64102de0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.893+0000 7f7d61d9b700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d4c06e150 0x7f7d4c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:37.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.893+0000 7f7d61d9b700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d4c06e150 0x7f7d4c070600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7d64071b50 tx=0x7f7d58009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:37.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:37.899+0000 7f7d52ffd700 1 -- 172.21.15.33:0/3426393219 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7d540580e0 con 0x7f7d64102de0 2024-04-25T01:55:38.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.128+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d6404bed0 con 0x7f7d4c06e150 2024-04-25T01:55:38.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.129+0000 7f7d52ffd700 1 -- 172.21.15.33:0/3426393219 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7d6404bed0 con 0x7f7d4c06e150 2024-04-25T01:55:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d4c06e150 msgr2=0x7f7d4c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d4c06e150 0x7f7d4c070600 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f7d64071b50 tx=0x7f7d58009ee0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 msgr2=0x7f7d64070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 0x7f7d64070260 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009150 tx=0x7f7d54009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 shutdown_connections 2024-04-25T01:55:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7d4c06e150 0x7f7d4c070600 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d64102de0 0x7f7d64070260 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 --2- 172.21.15.33:0/3426393219 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d64103780 0x7f7d640707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 >> 172.21.15.33:0/3426393219 conn(0x7f7d640f89b0 msgr2=0x7f7d641073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:38.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 shutdown_connections 2024-04-25T01:55:38.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.131+0000 7f7d68cbe700 1 -- 172.21.15.33:0/3426393219 wait complete. 2024-04-25T01:55:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.235+0000 7f171154e700 1 -- 172.21.15.33:0/1041856689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c0fb520 msgr2=0x7f170c0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.235+0000 7f171154e700 1 --2- 172.21.15.33:0/1041856689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c0fb520 0x7f170c0fb990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f16f4004940 tx=0x7f16f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:38.237 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:38 smithi033 ceph-mon[111306]: pgmap v486: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.235+0000 7f171154e700 1 -- 172.21.15.33:0/1041856689 shutdown_connections 2024-04-25T01:55:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.235+0000 7f171154e700 1 --2- 172.21.15.33:0/1041856689 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c0fb520 0x7f170c0fb990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.235+0000 7f171154e700 1 --2- 172.21.15.33:0/1041856689 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f170c102f60 0x7f170c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.235+0000 7f171154e700 1 -- 172.21.15.33:0/1041856689 >> 172.21.15.33:0/1041856689 conn(0x7f170c0714d0 msgr2=0x7f170c0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.235+0000 7f171154e700 1 -- 172.21.15.33:0/1041856689 shutdown_connections 2024-04-25T01:55:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.235+0000 7f171154e700 1 -- 172.21.15.33:0/1041856689 wait complete. 2024-04-25T01:55:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.236+0000 7f171154e700 1 Processor -- start 2024-04-25T01:55:38.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.236+0000 7f171154e700 1 -- start start 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.236+0000 7f171154e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f170c0fb520 0x7f170c113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.236+0000 7f171154e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c102f60 0x7f170c113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.236+0000 7f171154e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c114510 con 0x7f170c102f60 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.236+0000 7f171154e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c10db30 con 0x7f170c0fb520 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f170a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c102f60 0x7f170c113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f170a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c102f60 0x7f170c113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59350/0 (socket says 172.21.15.33:59350) 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f170a7fc700 1 -- 172.21.15.33:0/3423993072 learned_addr learned my addr 172.21.15.33:0/3423993072 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f170affd700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f170c0fb520 0x7f170c113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f170a7fc700 1 -- 172.21.15.33:0/3423993072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f170c0fb520 msgr2=0x7f170c113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f170a7fc700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f170c0fb520 0x7f170c113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f170a7fc700 1 -- 172.21.15.33:0/3423993072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4005040 con 0x7f170c102f60 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f170affd700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f170c0fb520 0x7f170c113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f170a7fc700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c102f60 0x7f170c113ec0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009150 tx=0x7f16f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:38.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f17037fe700 1 -- 172.21.15.33:0/3423993072 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16f400e040 con 0x7f170c102f60 2024-04-25T01:55:38.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c10ddb0 con 0x7f170c102f60 2024-04-25T01:55:38.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f17037fe700 1 -- 172.21.15.33:0/3423993072 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16f4013420 con 0x7f170c102f60 2024-04-25T01:55:38.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f17037fe700 1 -- 172.21.15.33:0/3423993072 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16f401c630 con 0x7f170c102f60 2024-04-25T01:55:38.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.237+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c10e2f0 con 0x7f170c102f60 2024-04-25T01:55:38.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.238+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c100250 con 0x7f170c102f60 2024-04-25T01:55:38.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.239+0000 7f17037fe700 1 -- 172.21.15.33:0/3423993072 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f16f4013590 con 0x7f170c102f60 2024-04-25T01:55:38.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.239+0000 7f17037fe700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16f806e140 0x7f16f80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:38.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.240+0000 7f17037fe700 1 -- 172.21.15.33:0/3423993072 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f16f408eda0 con 0x7f170c102f60 2024-04-25T01:55:38.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.240+0000 7f170affd700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16f806e140 0x7f16f80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:38.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.240+0000 7f170affd700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16f806e140 0x7f16f80705f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009fd0 tx=0x7f16fc009f00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:38.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.244+0000 7f17037fe700 1 -- 172.21.15.33:0/3423993072 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f16f40581a0 con 0x7f170c102f60 2024-04-25T01:55:38.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:38 smithi138 ceph-mon[81304]: pgmap v486: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:38.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.460+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f170c10efd0 con 0x7f16f806e140 2024-04-25T01:55:38.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.470+0000 7f17037fe700 1 -- 172.21.15.33:0/3423993072 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f170c10efd0 con 0x7f16f806e140 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (16m) 9m ago 27m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (14m) 9m ago 28m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (13m) 9m ago 27m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (16m) 9m ago 27m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (9m) 9m ago 22m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (10m) 9m ago 22m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (9m) 9m ago 22m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (22m) 9m ago 22m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (18m) 9m ago 29m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (17m) 9m ago 27m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (15m) 9m ago 29m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:55:38.472 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (14m) 9m ago 27m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:55:38.473 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (17m) 9m ago 28m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:55:38.473 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (17m) 9m ago 27m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:55:38.473 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (13m) 9m ago 26m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:55:38.473 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (12m) 9m ago 25m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:55:38.473 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (12m) 9m ago 25m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:55:38.473 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (11m) 9m ago 24m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:55:38.473 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (11m) 9m ago 24m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:55:38.473 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (10m) 9m ago 24m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:55:38.473 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (17m) 9m ago 27m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:55:38.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16f806e140 msgr2=0x7f16f80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16f806e140 0x7f16f80705f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009fd0 tx=0x7f16fc009f00 comp rx=0 tx=0).stop 2024-04-25T01:55:38.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c102f60 msgr2=0x7f170c113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c102f60 0x7f170c113ec0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009150 tx=0x7f16f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:38.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 shutdown_connections 2024-04-25T01:55:38.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16f806e140 0x7f16f80705f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f170c0fb520 0x7f170c113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 --2- 172.21.15.33:0/3423993072 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f170c102f60 0x7f170c113ec0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 >> 172.21.15.33:0/3423993072 conn(0x7f170c0714d0 msgr2=0x7f170c0fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:38.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 shutdown_connections 2024-04-25T01:55:38.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.472+0000 7f171154e700 1 -- 172.21.15.33:0/3423993072 wait complete. 2024-04-25T01:55:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 -- 172.21.15.33:0/1145048694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 msgr2=0x7f981c0fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 --2- 172.21.15.33:0/1145048694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 0x7f981c0fe230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f980c009ce0 tx=0x7f980c004d10 comp rx=0 tx=0).stop 2024-04-25T01:55:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 -- 172.21.15.33:0/1145048694 shutdown_connections 2024-04-25T01:55:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 --2- 172.21.15.33:0/1145048694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f981c0fe800 0x7f981c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 --2- 172.21.15.33:0/1145048694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 0x7f981c0fe230 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 -- 172.21.15.33:0/1145048694 >> 172.21.15.33:0/1145048694 conn(0x7f981c0f7870 msgr2=0x7f981c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:38.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 -- 172.21.15.33:0/1145048694 shutdown_connections 2024-04-25T01:55:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 -- 172.21.15.33:0/1145048694 wait complete. 2024-04-25T01:55:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 Processor -- start 2024-04-25T01:55:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.576+0000 7f9823095700 1 -- start start 2024-04-25T01:55:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9823095700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 0x7f981c0fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9823095700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f981c0fe800 0x7f981c0fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9823095700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0fdbe0 con 0x7f981c0fde60 2024-04-25T01:55:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9823095700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c0fc710 con 0x7f981c0fe800 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9820e31700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 0x7f981c0fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9820e31700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 0x7f981c0fbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59366/0 (socket says 172.21.15.33:59366) 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9820e31700 1 -- 172.21.15.33:0/887520093 learned_addr learned my addr 172.21.15.33:0/887520093 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f981bfff700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f981c0fe800 0x7f981c0fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9820e31700 1 -- 172.21.15.33:0/887520093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f981c0fe800 msgr2=0x7f981c0fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9820e31700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f981c0fe800 0x7f981c0fc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9820e31700 1 -- 172.21.15.33:0/887520093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c009940 con 0x7f981c0fde60 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.577+0000 7f9820e31700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 0x7f981c0fbc90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f980c007c00 tx=0x7f980c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.578+0000 7f98197fa700 1 -- 172.21.15.33:0/887520093 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f980c00e070 con 0x7f981c0fde60 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.578+0000 7f98197fa700 1 -- 172.21.15.33:0/887520093 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f980c0128e0 con 0x7f981c0fde60 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.578+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c0fc9f0 con 0x7f981c0fde60 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.578+0000 7f98197fa700 1 -- 172.21.15.33:0/887520093 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f980c01b890 con 0x7f981c0fde60 2024-04-25T01:55:38.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.578+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c11a070 con 0x7f981c0fde60 2024-04-25T01:55:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.579+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c066ad0 con 0x7f981c0fde60 2024-04-25T01:55:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.579+0000 7f98197fa700 1 -- 172.21.15.33:0/887520093 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f980c019040 con 0x7f981c0fde60 2024-04-25T01:55:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.580+0000 7f98197fa700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f980806e140 0x7f98080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:38.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.580+0000 7f98197fa700 1 -- 172.21.15.33:0/887520093 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f980c08ef20 con 0x7f981c0fde60 2024-04-25T01:55:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.580+0000 7f981bfff700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f980806e140 0x7f98080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:38.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.580+0000 7f981bfff700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f980806e140 0x7f98080705f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f981c0fd3c0 tx=0x7f980400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:38.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.584+0000 7f98197fa700 1 -- 172.21.15.33:0/887520093 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f980c0c3740 con 0x7f981c0fde60 2024-04-25T01:55:38.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.885+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f981c11a2b0 con 0x7f981c0fde60 2024-04-25T01:55:38.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.886+0000 7f98197fa700 1 -- 172.21.15.33:0/887520093 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f980c058320 con 0x7f981c0fde60 2024-04-25T01:55:38.888 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:55:38.888 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:55:38.888 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:55:38.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:55:38.888 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:55:38.888 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:55:38.888 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:55:38.888 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:55:38.889 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:55:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f980806e140 msgr2=0x7f98080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f980806e140 0x7f98080705f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f981c0fd3c0 tx=0x7f980400b040 comp rx=0 tx=0).stop 2024-04-25T01:55:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 msgr2=0x7f981c0fbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 0x7f981c0fbc90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f980c007c00 tx=0x7f980c004d10 comp rx=0 tx=0).stop 2024-04-25T01:55:38.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 shutdown_connections 2024-04-25T01:55:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f980806e140 0x7f98080705f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f981c0fde60 0x7f981c0fbc90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 --2- 172.21.15.33:0/887520093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f981c0fe800 0x7f981c0fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 >> 172.21.15.33:0/887520093 conn(0x7f981c0f7870 msgr2=0x7f981c1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.888+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 shutdown_connections 2024-04-25T01:55:38.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.889+0000 7f9823095700 1 -- 172.21.15.33:0/887520093 wait complete. 2024-04-25T01:55:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.992+0000 7f91595b5700 1 -- 172.21.15.33:0/331099127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 msgr2=0x7f9154102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.992+0000 7f91595b5700 1 --2- 172.21.15.33:0/331099127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 0x7f9154102160 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9148004940 tx=0x7f9148009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.992+0000 7f91595b5700 1 -- 172.21.15.33:0/331099127 shutdown_connections 2024-04-25T01:55:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.992+0000 7f91595b5700 1 --2- 172.21.15.33:0/331099127 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91540fbd40 0x7f91540fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.992+0000 7f91595b5700 1 --2- 172.21.15.33:0/331099127 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 0x7f9154102160 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.992+0000 7f91595b5700 1 -- 172.21.15.33:0/331099127 >> 172.21.15.33:0/331099127 conn(0x7f9154071d00 msgr2=0x7f9154072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.992+0000 7f91595b5700 1 -- 172.21.15.33:0/331099127 shutdown_connections 2024-04-25T01:55:38.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.992+0000 7f91595b5700 1 -- 172.21.15.33:0/331099127 wait complete. 2024-04-25T01:55:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.992+0000 7f91595b5700 1 Processor -- start 2024-04-25T01:55:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91595b5700 1 -- start start 2024-04-25T01:55:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91595b5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91540fbd40 0x7f9154102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91595b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 0x7f9154103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91595b5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91541035f0 con 0x7f9154101d90 2024-04-25T01:55:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91595b5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9154103730 con 0x7f91540fbd40 2024-04-25T01:55:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91527fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 0x7f9154103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:38.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91527fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 0x7f9154103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59388/0 (socket says 172.21.15.33:59388) 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91527fc700 1 -- 172.21.15.33:0/2151128561 learned_addr learned my addr 172.21.15.33:0/2151128561 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f9152ffd700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91540fbd40 0x7f9154102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91527fc700 1 -- 172.21.15.33:0/2151128561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91540fbd40 msgr2=0x7f9154102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91527fc700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91540fbd40 0x7f9154102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91527fc700 1 -- 172.21.15.33:0/2151128561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148005040 con 0x7f9154101d90 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.993+0000 7f91527fc700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 0x7f9154103060 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f914400eb40 tx=0x7f914400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.994+0000 7f913b7fe700 1 -- 172.21.15.33:0/2151128561 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9144007cc0 con 0x7f9154101d90 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.994+0000 7f913b7fe700 1 -- 172.21.15.33:0/2151128561 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f914400f040 con 0x7f9154101d90 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.994+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9154103980 con 0x7f9154101d90 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.994+0000 7f913b7fe700 1 -- 172.21.15.33:0/2151128561 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f914400ca10 con 0x7f9154101d90 2024-04-25T01:55:38.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.994+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915411a1d0 con 0x7f9154101d90 2024-04-25T01:55:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.995+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91540fd430 con 0x7f9154101d90 2024-04-25T01:55:38.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.995+0000 7f913b7fe700 1 -- 172.21.15.33:0/2151128561 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f914400cb70 con 0x7f9154101d90 2024-04-25T01:55:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.996+0000 7f913b7fe700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f913c06e140 0x7f913c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.996+0000 7f913b7fe700 1 -- 172.21.15.33:0/2151128561 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f914408d610 con 0x7f9154101d90 2024-04-25T01:55:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.996+0000 7f9152ffd700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f913c06e140 0x7f913c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:38.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:38.996+0000 7f9152ffd700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f913c06e140 0x7f913c0705f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9148009b30 tx=0x7f9148009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:39.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.000+0000 7f913b7fe700 1 -- 172.21.15.33:0/2151128561 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9144056ac0 con 0x7f9154101d90 2024-04-25T01:55:39.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.256+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9154108df0 con 0x7f9154101d90 2024-04-25T01:55:39.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:39 smithi033 ceph-mon[111306]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:39.258 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:39 smithi033 ceph-mon[111306]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:39.259 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:39 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/887520093' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:55:39.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.257+0000 7f913b7fe700 1 -- 172.21.15.33:0/2151128561 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f914405a180 con 0x7f9154101d90 2024-04-25T01:55:39.259 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:55:39.259 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:55:39.259 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:55:39.259 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:55:39.259 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:55:39.259 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:55:39.259 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:55:39.260 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:55:39.261 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:55:39.262 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:55:39.262 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:55:39.262 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:55:39.262 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:55:39.262 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:55:39.262 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:55:39.262 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:55:39.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f913c06e140 msgr2=0x7f913c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:39.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f913c06e140 0x7f913c0705f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9148009b30 tx=0x7f9148009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:39.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 msgr2=0x7f9154103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:39.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 0x7f9154103060 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f914400eb40 tx=0x7f914400b040 comp rx=0 tx=0).stop 2024-04-25T01:55:39.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 shutdown_connections 2024-04-25T01:55:39.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f913c06e140 0x7f913c0705f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f91540fbd40 0x7f9154102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 --2- 172.21.15.33:0/2151128561 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9154101d90 0x7f9154103060 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 >> 172.21.15.33:0/2151128561 conn(0x7f9154071d00 msgr2=0x7f91540fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:39.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 shutdown_connections 2024-04-25T01:55:39.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.260+0000 7f91595b5700 1 -- 172.21.15.33:0/2151128561 wait complete. 2024-04-25T01:55:39.264 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:55:39.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:39 smithi138 ceph-mon[81304]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:39.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:39 smithi138 ceph-mon[81304]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:39.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:39 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/887520093' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:55:39.367 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.365+0000 7f16057c9700 1 -- 172.21.15.33:0/2574564833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 msgr2=0x7f16001091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.365+0000 7f16057c9700 1 --2- 172.21.15.33:0/2574564833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 0x7f16001091e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f15f4004940 tx=0x7f15f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.365+0000 7f16057c9700 1 -- 172.21.15.33:0/2574564833 shutdown_connections 2024-04-25T01:55:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.365+0000 7f16057c9700 1 --2- 172.21.15.33:0/2574564833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 0x7f16001091e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.365+0000 7f16057c9700 1 --2- 172.21.15.33:0/2574564833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16000fbb70 0x7f16000fbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.365+0000 7f16057c9700 1 -- 172.21.15.33:0/2574564833 >> 172.21.15.33:0/2574564833 conn(0x7f16000f7820 msgr2=0x7f16000f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.366+0000 7f16057c9700 1 -- 172.21.15.33:0/2574564833 shutdown_connections 2024-04-25T01:55:39.368 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.366+0000 7f16057c9700 1 -- 172.21.15.33:0/2574564833 wait complete. 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.366+0000 7f16057c9700 1 Processor -- start 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.366+0000 7f16057c9700 1 -- start start 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.366+0000 7f16057c9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16000fbb70 0x7f1600115e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f16057c9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 0x7f1600116390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f16057c9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16001169e0 con 0x7f16000fc480 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f16057c9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160010ffc0 con 0x7f16000fbb70 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15fe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 0x7f1600116390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15fe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 0x7f1600116390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59418/0 (socket says 172.21.15.33:59418) 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15fe7fc700 1 -- 172.21.15.33:0/1229418075 learned_addr learned my addr 172.21.15.33:0/1229418075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:39.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15feffd700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16000fbb70 0x7f1600115e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15fe7fc700 1 -- 172.21.15.33:0/1229418075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16000fbb70 msgr2=0x7f1600115e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15fe7fc700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16000fbb70 0x7f1600115e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15fe7fc700 1 -- 172.21.15.33:0/1229418075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4005040 con 0x7f16000fc480 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15feffd700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16000fbb70 0x7f1600115e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15fe7fc700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 0x7f1600116390 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009380 tx=0x7f15f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15e77fe700 1 -- 172.21.15.33:0/1229418075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f400e040 con 0x7f16000fc480 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15e77fe700 1 -- 172.21.15.33:0/1229418075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15f400bbc0 con 0x7f16000fc480 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1600110240 con 0x7f16000fc480 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f15e77fe700 1 -- 172.21.15.33:0/1229418075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15f401c630 con 0x7f16000fc480 2024-04-25T01:55:39.371 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.367+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1600110800 con 0x7f16000fc480 2024-04-25T01:55:39.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.368+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1600106910 con 0x7f16000fc480 2024-04-25T01:55:39.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.369+0000 7f15e77fe700 1 -- 172.21.15.33:0/1229418075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f15f400bd30 con 0x7f16000fc480 2024-04-25T01:55:39.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.369+0000 7f15e77fe700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15e806e140 0x7f15e80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:39.372 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.370+0000 7f15e77fe700 1 -- 172.21.15.33:0/1229418075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f15f408eef0 con 0x7f16000fc480 2024-04-25T01:55:39.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.370+0000 7f15feffd700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15e806e140 0x7f15e80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:39.373 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.370+0000 7f15feffd700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15e806e140 0x7f15e80705f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009fd0 tx=0x7f15f0009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:39.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.374+0000 7f15e77fe700 1 -- 172.21.15.33:0/1229418075 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f15f405be20 con 0x7f16000fc480 2024-04-25T01:55:39.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.602+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1600111510 con 0x7f15e806e140 2024-04-25T01:55:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.604+0000 7f15e77fe700 1 -- 172.21.15.33:0/1229418075 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1600111510 con 0x7f15e806e140 2024-04-25T01:55:39.605 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:55:39.606 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:55:39.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15e806e140 msgr2=0x7f15e80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15e806e140 0x7f15e80705f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009fd0 tx=0x7f15f0009ee0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 msgr2=0x7f1600116390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 0x7f1600116390 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f15f4009380 tx=0x7f15f4009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 shutdown_connections 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f15e806e140 0x7f15e80705f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16000fbb70 0x7f1600115e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 --2- 172.21.15.33:0/1229418075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f16000fc480 0x7f1600116390 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 >> 172.21.15.33:0/1229418075 conn(0x7f16000f7820 msgr2=0x7f16000f7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 shutdown_connections 2024-04-25T01:55:39.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.606+0000 7f16057c9700 1 -- 172.21.15.33:0/1229418075 wait complete. 2024-04-25T01:55:39.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 -- 172.21.15.33:0/3968644134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964102de0 msgr2=0x7f29641031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 --2- 172.21.15.33:0/3968644134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964102de0 0x7f29641031b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2958004940 tx=0x7f2958009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 -- 172.21.15.33:0/3968644134 shutdown_connections 2024-04-25T01:55:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 --2- 172.21.15.33:0/3968644134 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2964103780 0x7f29640fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 --2- 172.21.15.33:0/3968644134 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964102de0 0x7f29641031b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 -- 172.21.15.33:0/3968644134 >> 172.21.15.33:0/3968644134 conn(0x7f29640f8990 msgr2=0x7f29640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 -- 172.21.15.33:0/3968644134 shutdown_connections 2024-04-25T01:55:39.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 -- 172.21.15.33:0/3968644134 wait complete. 2024-04-25T01:55:39.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 Processor -- start 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.710+0000 7f296b243700 1 -- start start 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f296b243700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2964102de0 0x7f2964113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f296b243700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964103780 0x7f2964113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f296b243700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964114520 con 0x7f2964103780 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f296b243700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296410db70 con 0x7f2964102de0 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f2963fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964103780 0x7f2964113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f2963fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964103780 0x7f2964113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:59424/0 (socket says 172.21.15.33:59424) 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f2963fff700 1 -- 172.21.15.33:0/2751039990 learned_addr learned my addr 172.21.15.33:0/2751039990 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f2968fdf700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2964102de0 0x7f2964113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f2963fff700 1 -- 172.21.15.33:0/2751039990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2964102de0 msgr2=0x7f2964113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f2963fff700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2964102de0 0x7f2964113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f2963fff700 1 -- 172.21.15.33:0/2751039990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958005040 con 0x7f2964103780 2024-04-25T01:55:39.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.711+0000 7f2963fff700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964103780 0x7f2964113ed0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2950002a80 tx=0x7f2950004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.712+0000 7f29617fa700 1 -- 172.21.15.33:0/2751039990 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2950004e80 con 0x7f2964103780 2024-04-25T01:55:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.712+0000 7f29617fa700 1 -- 172.21.15.33:0/2751039990 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f295000f040 con 0x7f2964103780 2024-04-25T01:55:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.712+0000 7f29617fa700 1 -- 172.21.15.33:0/2751039990 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f295000daa0 con 0x7f2964103780 2024-04-25T01:55:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.712+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296410de50 con 0x7f2964103780 2024-04-25T01:55:39.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.712+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296410e410 con 0x7f2964103780 2024-04-25T01:55:39.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.713+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29641000f0 con 0x7f2964103780 2024-04-25T01:55:39.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.713+0000 7f29617fa700 1 -- 172.21.15.33:0/2751039990 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2950007cc0 con 0x7f2964103780 2024-04-25T01:55:39.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.714+0000 7f29617fa700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f295406e140 0x7f29540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:55:39.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.714+0000 7f29617fa700 1 -- 172.21.15.33:0/2751039990 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f295008d090 con 0x7f2964103780 2024-04-25T01:55:39.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.714+0000 7f2968fdf700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f295406e140 0x7f29540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:55:39.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.714+0000 7f2968fdf700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f295406e140 0x7f29540705f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2958009fd0 tx=0x7f2958009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:55:39.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:39.718+0000 7f29617fa700 1 -- 172.21.15.33:0/2751039990 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2950059fc0 con 0x7f2964103780 2024-04-25T01:55:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.013+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f29640fe530 con 0x7f2964103780 2024-04-25T01:55:40.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.013+0000 7f29617fa700 1 -- 172.21.15.33:0/2751039990 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2950059b50 con 0x7f2964103780 2024-04-25T01:55:40.015 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:55:40.015 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:55:40.016 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:55:40.016 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:55:40.016 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:55:40.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.015+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f295406e140 msgr2=0x7f29540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:40.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.015+0000 7f296b243700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f295406e140 0x7f29540705f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2958009fd0 tx=0x7f2958009e30 comp rx=0 tx=0).stop 2024-04-25T01:55:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.015+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964103780 msgr2=0x7f2964113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:55:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.015+0000 7f296b243700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964103780 0x7f2964113ed0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2950002a80 tx=0x7f2950004630 comp rx=0 tx=0).stop 2024-04-25T01:55:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.016+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 shutdown_connections 2024-04-25T01:55:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.016+0000 7f296b243700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f295406e140 0x7f29540705f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.016+0000 7f296b243700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2964102de0 0x7f2964113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.016+0000 7f296b243700 1 --2- 172.21.15.33:0/2751039990 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2964103780 0x7f2964113ed0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:55:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.016+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 >> 172.21.15.33:0/2751039990 conn(0x7f29640f8990 msgr2=0x7f2964107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:55:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.016+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 shutdown_connections 2024-04-25T01:55:40.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:55:40.016+0000 7f296b243700 1 -- 172.21.15.33:0/2751039990 wait complete. 2024-04-25T01:55:40.267 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:40 smithi033 ceph-mon[111306]: pgmap v487: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:40.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:40 smithi033 ceph-mon[111306]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:40.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:40 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2151128561' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:55:40.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:40 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2751039990' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:55:40.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:40 smithi138 ceph-mon[81304]: pgmap v487: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:40.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:40 smithi138 ceph-mon[81304]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:40.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:40 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2151128561' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:55:40.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:40 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2751039990' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:55:41.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:41 smithi033 ceph-mon[111306]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:41.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:55:41.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:41 smithi138 ceph-mon[81304]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:55:41.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:55:42.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:42 smithi138 ceph-mon[81304]: pgmap v488: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:42.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:42 smithi033 ceph-mon[111306]: pgmap v488: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:44.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:44 smithi138 ceph-mon[81304]: pgmap v489: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:44.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:44 smithi033 ceph-mon[111306]: pgmap v489: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:46.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:46 smithi138 ceph-mon[81304]: pgmap v490: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:46.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:46 smithi033 ceph-mon[111306]: pgmap v490: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:48.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:48 smithi138 ceph-mon[81304]: pgmap v491: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:48.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:48 smithi033 ceph-mon[111306]: pgmap v491: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:50.293 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:50 smithi138 ceph-mon[81304]: pgmap v492: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:50.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:50 smithi033 ceph-mon[111306]: pgmap v492: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:52.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:52 smithi033 ceph-mon[111306]: pgmap v493: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:52.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:52 smithi138 ceph-mon[81304]: pgmap v493: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:54.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:54 smithi033 ceph-mon[111306]: pgmap v494: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:54.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:54 smithi138 ceph-mon[81304]: pgmap v494: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:56.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:56 smithi033 ceph-mon[111306]: pgmap v495: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:56.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:55:56.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:56 smithi138 ceph-mon[81304]: pgmap v495: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:56.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:55:58.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:55:58 smithi033 ceph-mon[111306]: pgmap v496: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:55:58.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:55:58 smithi138 ceph-mon[81304]: pgmap v496: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:00.345 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:00 smithi138 ceph-mon[81304]: pgmap v497: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:00.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:00 smithi033 ceph-mon[111306]: pgmap v497: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:02.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:02 smithi033 ceph-mon[111306]: pgmap v498: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:02.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:02 smithi138 ceph-mon[81304]: pgmap v498: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:04.320 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:04 smithi033 ceph-mon[111306]: pgmap v499: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:04.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:04 smithi138 ceph-mon[81304]: pgmap v499: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:06.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:06 smithi033 ceph-mon[111306]: pgmap v500: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:06.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:06 smithi138 ceph-mon[81304]: pgmap v500: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:08.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:08 smithi033 ceph-mon[111306]: pgmap v501: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:08.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:08 smithi138 ceph-mon[81304]: pgmap v501: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.136+0000 7f3b91978700 1 -- 172.21.15.33:0/2005778956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 msgr2=0x7f3b8c106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.136+0000 7f3b91978700 1 --2- 172.21.15.33:0/2005778956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 0x7f3b8c106dc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3b80004940 tx=0x7f3b80009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.136+0000 7f3b91978700 1 -- 172.21.15.33:0/2005778956 shutdown_connections 2024-04-25T01:56:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.136+0000 7f3b91978700 1 --2- 172.21.15.33:0/2005778956 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 0x7f3b8c106dc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.136+0000 7f3b91978700 1 --2- 172.21.15.33:0/2005778956 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b8c0fdde0 0x7f3b8c0fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.136+0000 7f3b91978700 1 -- 172.21.15.33:0/2005778956 >> 172.21.15.33:0/2005778956 conn(0x7f3b8c0f7850 msgr2=0x7f3b8c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.136+0000 7f3b91978700 1 -- 172.21.15.33:0/2005778956 shutdown_connections 2024-04-25T01:56:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.136+0000 7f3b91978700 1 -- 172.21.15.33:0/2005778956 wait complete. 2024-04-25T01:56:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.137+0000 7f3b91978700 1 Processor -- start 2024-04-25T01:56:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.137+0000 7f3b91978700 1 -- start start 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.137+0000 7f3b91978700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b8c0fdde0 0x7f3b8c113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.137+0000 7f3b91978700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 0x7f3b8c114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.137+0000 7f3b91978700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c1146e0 con 0x7f3b8c0fe780 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b91978700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b8c10dad0 con 0x7f3b8c0fdde0 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b8a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 0x7f3b8c114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b8a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 0x7f3b8c114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52640/0 (socket says 172.21.15.33:52640) 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b8a7fc700 1 -- 172.21.15.33:0/23966188 learned_addr learned my addr 172.21.15.33:0/23966188 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b8affd700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b8c0fdde0 0x7f3b8c113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b8a7fc700 1 -- 172.21.15.33:0/23966188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b8c0fdde0 msgr2=0x7f3b8c113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b8a7fc700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b8c0fdde0 0x7f3b8c113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b8a7fc700 1 -- 172.21.15.33:0/23966188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b80005040 con 0x7f3b8c0fe780 2024-04-25T01:56:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b8affd700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b8c0fdde0 0x7f3b8c113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:56:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b8a7fc700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 0x7f3b8c114090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009380 tx=0x7f3b80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b737fe700 1 -- 172.21.15.33:0/23966188 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8000e040 con 0x7f3b8c0fe780 2024-04-25T01:56:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.138+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b8c10dd50 con 0x7f3b8c0fe780 2024-04-25T01:56:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.139+0000 7f3b737fe700 1 -- 172.21.15.33:0/23966188 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b80009af0 con 0x7f3b8c0fe780 2024-04-25T01:56:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.139+0000 7f3b737fe700 1 -- 172.21.15.33:0/23966188 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8001c630 con 0x7f3b8c0fe780 2024-04-25T01:56:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.139+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b8c10e290 con 0x7f3b8c0fe780 2024-04-25T01:56:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.140+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8c066750 con 0x7f3b8c0fe780 2024-04-25T01:56:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.140+0000 7f3b737fe700 1 -- 172.21.15.33:0/23966188 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3b8000bbc0 con 0x7f3b8c0fe780 2024-04-25T01:56:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.141+0000 7f3b737fe700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b7406e140 0x7f3b740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.141+0000 7f3b737fe700 1 -- 172.21.15.33:0/23966188 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3b8008edf0 con 0x7f3b8c0fe780 2024-04-25T01:56:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.141+0000 7f3b8affd700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b7406e140 0x7f3b740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:10.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.141+0000 7f3b8affd700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b7406e140 0x7f3b740705f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009fd0 tx=0x7f3b7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:10.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.146+0000 7f3b737fe700 1 -- 172.21.15.33:0/23966188 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3b8005bd20 con 0x7f3b8c0fe780 2024-04-25T01:56:10.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.372+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b8c109e70 con 0x7f3b7406e140 2024-04-25T01:56:10.374 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:10 smithi033 ceph-mon[111306]: pgmap v502: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:10.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.373+0000 7f3b737fe700 1 -- 172.21.15.33:0/23966188 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3b8c109e70 con 0x7f3b7406e140 2024-04-25T01:56:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b7406e140 msgr2=0x7f3b740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b7406e140 0x7f3b740705f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c009fd0 tx=0x7f3b7c00b040 comp rx=0 tx=0).stop 2024-04-25T01:56:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 msgr2=0x7f3b8c114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 0x7f3b8c114090 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009380 tx=0x7f3b80009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 shutdown_connections 2024-04-25T01:56:10.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3b7406e140 0x7f3b740705f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3b8c0fdde0 0x7f3b8c113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 --2- 172.21.15.33:0/23966188 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3b8c0fe780 0x7f3b8c114090 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 >> 172.21.15.33:0/23966188 conn(0x7f3b8c0f7850 msgr2=0x7f3b8c0f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 shutdown_connections 2024-04-25T01:56:10.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.375+0000 7f3b91978700 1 -- 172.21.15.33:0/23966188 wait complete. 2024-04-25T01:56:10.388 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:56:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.480+0000 7f708db3e700 1 -- 172.21.15.33:0/2837434765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7088102f60 msgr2=0x7f7088103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.480+0000 7f708db3e700 1 --2- 172.21.15.33:0/2837434765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7088102f60 0x7f7088103330 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f707c004940 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.480+0000 7f708db3e700 1 -- 172.21.15.33:0/2837434765 shutdown_connections 2024-04-25T01:56:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.480+0000 7f708db3e700 1 --2- 172.21.15.33:0/2837434765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70880fb520 0x7f70880fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.480+0000 7f708db3e700 1 --2- 172.21.15.33:0/2837434765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7088102f60 0x7f7088103330 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.480+0000 7f708db3e700 1 -- 172.21.15.33:0/2837434765 >> 172.21.15.33:0/2837434765 conn(0x7f70880714d0 msgr2=0x7f70880718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.481+0000 7f708db3e700 1 -- 172.21.15.33:0/2837434765 shutdown_connections 2024-04-25T01:56:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.481+0000 7f708db3e700 1 -- 172.21.15.33:0/2837434765 wait complete. 2024-04-25T01:56:10.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.481+0000 7f708db3e700 1 Processor -- start 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.481+0000 7f708db3e700 1 -- start start 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f708db3e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70880fb520 0x7f7088113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f708db3e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7088102f60 0x7f7088113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f708db3e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7088114510 con 0x7f70880fb520 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f708db3e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708810db30 con 0x7f7088102f60 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f70877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70880fb520 0x7f7088113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f70877fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70880fb520 0x7f7088113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52654/0 (socket says 172.21.15.33:52654) 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f70877fe700 1 -- 172.21.15.33:0/508392879 learned_addr learned my addr 172.21.15.33:0/508392879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f70877fe700 1 -- 172.21.15.33:0/508392879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7088102f60 msgr2=0x7f7088113ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T01:56:10.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f70877fe700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7088102f60 0x7f7088113ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f70877fe700 1 -- 172.21.15.33:0/508392879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c005040 con 0x7f70880fb520 2024-04-25T01:56:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f70877fe700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70880fb520 0x7f7088113980 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f707c009180 tx=0x7f707c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f706ffff700 1 -- 172.21.15.33:0/508392879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707c00e040 con 0x7f70880fb520 2024-04-25T01:56:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708810ddb0 con 0x7f70880fb520 2024-04-25T01:56:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f706ffff700 1 -- 172.21.15.33:0/508392879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f707c00bbc0 con 0x7f70880fb520 2024-04-25T01:56:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f706ffff700 1 -- 172.21.15.33:0/508392879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f707c01c630 con 0x7f70880fb520 2024-04-25T01:56:10.486 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.482+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708810e2f0 con 0x7f70880fb520 2024-04-25T01:56:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.483+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7088100250 con 0x7f70880fb520 2024-04-25T01:56:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.484+0000 7f706ffff700 1 -- 172.21.15.33:0/508392879 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f707c00bd30 con 0x7f70880fb520 2024-04-25T01:56:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.484+0000 7f706ffff700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f707006e140 0x7f70700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.485+0000 7f7086ffd700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f707006e140 0x7f70700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.485+0000 7f706ffff700 1 -- 172.21.15.33:0/508392879 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f707c08eed0 con 0x7f70880fb520 2024-04-25T01:56:10.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.485+0000 7f7086ffd700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f707006e140 0x7f70700705f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f708810f220 tx=0x7f7078007d40 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:10.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.489+0000 7f706ffff700 1 -- 172.21.15.33:0/508392879 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f707c0582d0 con 0x7f70880fb520 2024-04-25T01:56:10.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:10 smithi138 ceph-mon[81304]: pgmap v502: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:10.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.713+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f708810efd0 con 0x7f707006e140 2024-04-25T01:56:10.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.715+0000 7f706ffff700 1 -- 172.21.15.33:0/508392879 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f708810efd0 con 0x7f707006e140 2024-04-25T01:56:10.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f707006e140 msgr2=0x7f70700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:10.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f707006e140 0x7f70700705f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f708810f220 tx=0x7f7078007d40 comp rx=0 tx=0).stop 2024-04-25T01:56:10.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70880fb520 msgr2=0x7f7088113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:10.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70880fb520 0x7f7088113980 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f707c009180 tx=0x7f707c009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:10.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 shutdown_connections 2024-04-25T01:56:10.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f707006e140 0x7f70700705f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70880fb520 0x7f7088113980 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 --2- 172.21.15.33:0/508392879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7088102f60 0x7f7088113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 >> 172.21.15.33:0/508392879 conn(0x7f70880714d0 msgr2=0x7f70880fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:10.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 shutdown_connections 2024-04-25T01:56:10.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.718+0000 7f708db3e700 1 -- 172.21.15.33:0/508392879 wait complete. 2024-04-25T01:56:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.821+0000 7f2819e3a700 1 -- 172.21.15.33:0/2232242911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 msgr2=0x7f28140fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.821+0000 7f2819e3a700 1 --2- 172.21.15.33:0/2232242911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 0x7f28140fe1d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2808004940 tx=0x7f2808009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.822+0000 7f2819e3a700 1 -- 172.21.15.33:0/2232242911 shutdown_connections 2024-04-25T01:56:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.822+0000 7f2819e3a700 1 --2- 172.21.15.33:0/2232242911 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140fe7a0 0x7f2814106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.822+0000 7f2819e3a700 1 --2- 172.21.15.33:0/2232242911 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 0x7f28140fe1d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.822+0000 7f2819e3a700 1 -- 172.21.15.33:0/2232242911 >> 172.21.15.33:0/2232242911 conn(0x7f28140f78a0 msgr2=0x7f28140f9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.822+0000 7f2819e3a700 1 -- 172.21.15.33:0/2232242911 shutdown_connections 2024-04-25T01:56:10.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.822+0000 7f2819e3a700 1 -- 172.21.15.33:0/2232242911 wait complete. 2024-04-25T01:56:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.822+0000 7f2819e3a700 1 Processor -- start 2024-04-25T01:56:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.822+0000 7f2819e3a700 1 -- start start 2024-04-25T01:56:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f2819e3a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 0x7f2814115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f2819e3a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140fe7a0 0x7f28141160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f2819e3a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2814116710 con 0x7f28140fde00 2024-04-25T01:56:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f2819e3a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281410fcf0 con 0x7f28140fe7a0 2024-04-25T01:56:10.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f28137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 0x7f2814115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f28137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 0x7f2814115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52676/0 (socket says 172.21.15.33:52676) 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f28137fe700 1 -- 172.21.15.33:0/1443270805 learned_addr learned my addr 172.21.15.33:0/1443270805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f2812ffd700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140fe7a0 0x7f28141160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f28137fe700 1 -- 172.21.15.33:0/1443270805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140fe7a0 msgr2=0x7f28141160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f28137fe700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140fe7a0 0x7f28141160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f28137fe700 1 -- 172.21.15.33:0/1443270805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808005040 con 0x7f28140fde00 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.823+0000 7f28137fe700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 0x7f2814115b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2808009380 tx=0x7f2808009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.824+0000 7f27fbfff700 1 -- 172.21.15.33:0/1443270805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f280800e040 con 0x7f28140fde00 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.824+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281410ff70 con 0x7f28140fde00 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.824+0000 7f27fbfff700 1 -- 172.21.15.33:0/1443270805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2808009af0 con 0x7f28140fde00 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.824+0000 7f27fbfff700 1 -- 172.21.15.33:0/1443270805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f280801c3f0 con 0x7f28140fde00 2024-04-25T01:56:10.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.824+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28141104b0 con 0x7f28140fde00 2024-04-25T01:56:10.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.825+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2814066ab0 con 0x7f28140fde00 2024-04-25T01:56:10.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.825+0000 7f27fbfff700 1 -- 172.21.15.33:0/1443270805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2808013420 con 0x7f28140fde00 2024-04-25T01:56:10.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.826+0000 7f27fbfff700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f27fc06e140 0x7f27fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:10.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.826+0000 7f27fbfff700 1 -- 172.21.15.33:0/1443270805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f280808ee80 con 0x7f28140fde00 2024-04-25T01:56:10.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.826+0000 7f2812ffd700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f27fc06e140 0x7f27fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:10.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.826+0000 7f2812ffd700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f27fc06e140 0x7f27fc0705f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2814111420 tx=0x7f28040043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:10.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:10.830+0000 7f27fbfff700 1 -- 172.21.15.33:0/1443270805 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f280805bd30 con 0x7f28140fde00 2024-04-25T01:56:11.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.047+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f28141111d0 con 0x7f27fc06e140 2024-04-25T01:56:11.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.058+0000 7f27fbfff700 1 -- 172.21.15.33:0/1443270805 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f28141111d0 con 0x7f27fc06e140 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (17m) 10m ago 28m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (14m) 10m ago 28m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (14m) 10m ago 27m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (17m) 10m ago 28m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (10m) 10m ago 23m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (10m) 10m ago 23m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (10m) 10m ago 23m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (23m) 10m ago 23m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:56:11.060 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (18m) 10m ago 29m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (18m) 10m ago 27m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (15m) 10m ago 30m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (15m) 10m ago 27m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (18m) 10m ago 28m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (17m) 10m ago 27m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (14m) 10m ago 26m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (13m) 10m ago 26m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (12m) 10m ago 25m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (12m) 10m ago 25m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (11m) 10m ago 25m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (11m) 10m ago 24m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:56:11.061 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (17m) 10m ago 28m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:56:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f27fc06e140 msgr2=0x7f27fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f27fc06e140 0x7f27fc0705f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2814111420 tx=0x7f28040043c0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 msgr2=0x7f2814115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 0x7f2814115b80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2808009380 tx=0x7f2808009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 shutdown_connections 2024-04-25T01:56:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f27fc06e140 0x7f27fc0705f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28140fde00 0x7f2814115b80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 --2- 172.21.15.33:0/1443270805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28140fe7a0 0x7f28141160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 >> 172.21.15.33:0/1443270805 conn(0x7f28140f78a0 msgr2=0x7f28140f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 shutdown_connections 2024-04-25T01:56:11.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.060+0000 7f2819e3a700 1 -- 172.21.15.33:0/1443270805 wait complete. 2024-04-25T01:56:11.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 -- 172.21.15.33:0/921482083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824103760 msgr2=0x7ff8240fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 --2- 172.21.15.33:0/921482083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824103760 0x7ff8240fcdd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff814004940 tx=0x7ff814009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:11.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 -- 172.21.15.33:0/921482083 shutdown_connections 2024-04-25T01:56:11.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 --2- 172.21.15.33:0/921482083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824103760 0x7ff8240fcdd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 --2- 172.21.15.33:0/921482083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff824102dc0 0x7ff824103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 -- 172.21.15.33:0/921482083 >> 172.21.15.33:0/921482083 conn(0x7ff8240f8930 msgr2=0x7ff8240fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:11.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 -- 172.21.15.33:0/921482083 shutdown_connections 2024-04-25T01:56:11.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 -- 172.21.15.33:0/921482083 wait complete. 2024-04-25T01:56:11.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 Processor -- start 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.166+0000 7ff82c7d3700 1 -- start start 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82c7d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824102dc0 0x7ff824070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82c7d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff824103760 0x7ff8240707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82c7d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff824072240 con 0x7ff824102dc0 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82c7d3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff824070ce0 con 0x7ff824103760 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82a56f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824102dc0 0x7ff824070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82a56f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824102dc0 0x7ff824070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52682/0 (socket says 172.21.15.33:52682) 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82a56f700 1 -- 172.21.15.33:0/3263871720 learned_addr learned my addr 172.21.15.33:0/3263871720 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff829d6e700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff824103760 0x7ff8240707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82a56f700 1 -- 172.21.15.33:0/3263871720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff824103760 msgr2=0x7ff8240707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82a56f700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff824103760 0x7ff8240707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82a56f700 1 -- 172.21.15.33:0/3263871720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814005040 con 0x7ff824102dc0 2024-04-25T01:56:11.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82a56f700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824102dc0 0x7ff824070260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff82000b790 tx=0x7ff820009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:11.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff81affd700 1 -- 172.21.15.33:0/3263871720 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff82000f770 con 0x7ff824102dc0 2024-04-25T01:56:11.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff81affd700 1 -- 172.21.15.33:0/3263871720 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff82000fda0 con 0x7ff824102dc0 2024-04-25T01:56:11.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.167+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff824070fc0 con 0x7ff824102dc0 2024-04-25T01:56:11.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.168+0000 7ff81affd700 1 -- 172.21.15.33:0/3263871720 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff82000d310 con 0x7ff824102dc0 2024-04-25T01:56:11.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.168+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82411a1f0 con 0x7ff824102dc0 2024-04-25T01:56:11.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.168+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8241000d0 con 0x7ff824102dc0 2024-04-25T01:56:11.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.169+0000 7ff81affd700 1 -- 172.21.15.33:0/3263871720 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff820010420 con 0x7ff824102dc0 2024-04-25T01:56:11.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.169+0000 7ff81affd700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff81006e140 0x7ff8100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:11.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.170+0000 7ff81affd700 1 -- 172.21.15.33:0/3263871720 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff82008db50 con 0x7ff824102dc0 2024-04-25T01:56:11.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.170+0000 7ff829d6e700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff81006e140 0x7ff8100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:11.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.170+0000 7ff829d6e700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff81006e140 0x7ff8100705f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff81400b020 tx=0x7ff814009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:11.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.174+0000 7ff81affd700 1 -- 172.21.15.33:0/3263871720 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff820056f50 con 0x7ff824102dc0 2024-04-25T01:56:11.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.473+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff824109f70 con 0x7ff824102dc0 2024-04-25T01:56:11.476 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:56:11.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.473+0000 7ff81affd700 1 -- 172.21.15.33:0/3263871720 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff82005a610 con 0x7ff824102dc0 2024-04-25T01:56:11.477 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:56:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:56:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:56:11.477 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:56:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:56:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:56:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:56:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:56:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:56:11.478 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:56:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:56:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:56:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:56:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:56:11.479 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:56:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:56:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:56:11.480 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:56:11.480 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:56:11.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff81006e140 msgr2=0x7ff8100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff81006e140 0x7ff8100705f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff81400b020 tx=0x7ff814009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824102dc0 msgr2=0x7ff824070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824102dc0 0x7ff824070260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff82000b790 tx=0x7ff820009250 comp rx=0 tx=0).stop 2024-04-25T01:56:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 shutdown_connections 2024-04-25T01:56:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff81006e140 0x7ff8100705f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff824102dc0 0x7ff824070260 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 --2- 172.21.15.33:0/3263871720 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff824103760 0x7ff8240707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 >> 172.21.15.33:0/3263871720 conn(0x7ff8240f8930 msgr2=0x7ff8240fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 shutdown_connections 2024-04-25T01:56:11.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.476+0000 7ff82c7d3700 1 -- 172.21.15.33:0/3263871720 wait complete. 2024-04-25T01:56:11.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:56:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.580+0000 7f3633b17700 1 -- 172.21.15.33:0/1387730842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 msgr2=0x7f362c0fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.580+0000 7f3633b17700 1 --2- 172.21.15.33:0/1387730842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 0x7f362c0fe1d0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3628004940 tx=0x7f3628009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.580+0000 7f3633b17700 1 -- 172.21.15.33:0/1387730842 shutdown_connections 2024-04-25T01:56:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.580+0000 7f3633b17700 1 --2- 172.21.15.33:0/1387730842 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c0fe7a0 0x7f362c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.580+0000 7f3633b17700 1 --2- 172.21.15.33:0/1387730842 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 0x7f362c0fe1d0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.580+0000 7f3633b17700 1 -- 172.21.15.33:0/1387730842 >> 172.21.15.33:0/1387730842 conn(0x7f362c0f7860 msgr2=0x7f362c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.580+0000 7f3633b17700 1 -- 172.21.15.33:0/1387730842 shutdown_connections 2024-04-25T01:56:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.580+0000 7f3633b17700 1 -- 172.21.15.33:0/1387730842 wait complete. 2024-04-25T01:56:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f3633b17700 1 Processor -- start 2024-04-25T01:56:11.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f3633b17700 1 -- start start 2024-04-25T01:56:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f3633b17700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 0x7f362c113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f3633b17700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c0fe7a0 0x7f362c113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f3633b17700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1145e0 con 0x7f362c0fde00 2024-04-25T01:56:11.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f3633b17700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c10dbc0 con 0x7f362c0fe7a0 2024-04-25T01:56:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f36318b3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 0x7f362c113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f36318b3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 0x7f362c113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52698/0 (socket says 172.21.15.33:52698) 2024-04-25T01:56:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f36318b3700 1 -- 172.21.15.33:0/3554761716 learned_addr learned my addr 172.21.15.33:0/3554761716 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.581+0000 7f36310b2700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c0fe7a0 0x7f362c113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.582+0000 7f36318b3700 1 -- 172.21.15.33:0/3554761716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c0fe7a0 msgr2=0x7f362c113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.582+0000 7f36318b3700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c0fe7a0 0x7f362c113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.582+0000 7f36318b3700 1 -- 172.21.15.33:0/3554761716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3628005040 con 0x7f362c0fde00 2024-04-25T01:56:11.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.582+0000 7f36318b3700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 0x7f362c113a50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3628004890 tx=0x7f3628009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:11.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.582+0000 7f36227fc700 1 -- 172.21.15.33:0/3554761716 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f362800e040 con 0x7f362c0fde00 2024-04-25T01:56:11.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.582+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c10de40 con 0x7f362c0fde00 2024-04-25T01:56:11.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.582+0000 7f36227fc700 1 -- 172.21.15.33:0/3554761716 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3628009af0 con 0x7f362c0fde00 2024-04-25T01:56:11.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.582+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c10e380 con 0x7f362c0fde00 2024-04-25T01:56:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.582+0000 7f36227fc700 1 -- 172.21.15.33:0/3554761716 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f362801c3f0 con 0x7f362c0fde00 2024-04-25T01:56:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.583+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c066ad0 con 0x7f362c0fde00 2024-04-25T01:56:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.583+0000 7f36227fc700 1 -- 172.21.15.33:0/3554761716 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f362800baf0 con 0x7f362c0fde00 2024-04-25T01:56:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.584+0000 7f36227fc700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f361806e140 0x7f36180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.584+0000 7f36227fc700 1 -- 172.21.15.33:0/3554761716 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f362808edb0 con 0x7f362c0fde00 2024-04-25T01:56:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.584+0000 7f36310b2700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f361806e140 0x7f36180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:11.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.584+0000 7f36310b2700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f361806e140 0x7f36180705f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f362c10f2f0 tx=0x7f361c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:11.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.589+0000 7f36227fc700 1 -- 172.21.15.33:0/3554761716 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f362805bc60 con 0x7f362c0fde00 2024-04-25T01:56:11.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.840+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f362c109e90 con 0x7f362c0fde00 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.841+0000 7f36227fc700 1 -- 172.21.15.33:0/3554761716 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f362805b7f0 con 0x7f362c0fde00 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:56:11.844 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:56:11.845 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:56:11.846 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:56:11.847 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:56:11.847 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:56:11.847 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f361806e140 msgr2=0x7f36180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f361806e140 0x7f36180705f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f362c10f2f0 tx=0x7f361c009040 comp rx=0 tx=0).stop 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 msgr2=0x7f362c113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 0x7f362c113a50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3628004890 tx=0x7f3628009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 shutdown_connections 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f361806e140 0x7f36180705f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f362c0fde00 0x7f362c113a50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 --2- 172.21.15.33:0/3554761716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f362c0fe7a0 0x7f362c113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 >> 172.21.15.33:0/3554761716 conn(0x7f362c0f7860 msgr2=0x7f362c0f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 shutdown_connections 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.843+0000 7f3633b17700 1 -- 172.21.15.33:0/3554761716 wait complete. 2024-04-25T01:56:11.848 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 -- 172.21.15.33:0/2481786147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 msgr2=0x7fb6e00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 --2- 172.21.15.33:0/2481786147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 0x7fb6e00fc1b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0004940 tx=0x7fb6d0009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 -- 172.21.15.33:0/2481786147 shutdown_connections 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 --2- 172.21.15.33:0/2481786147 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 0x7fb6e00fc1b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 --2- 172.21.15.33:0/2481786147 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e0101d90 0x7fb6e0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 -- 172.21.15.33:0/2481786147 >> 172.21.15.33:0/2481786147 conn(0x7fb6e0071d00 msgr2=0x7fb6e0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 -- 172.21.15.33:0/2481786147 shutdown_connections 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 -- 172.21.15.33:0/2481786147 wait complete. 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 Processor -- start 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.948+0000 7fb6e736b700 1 -- start start 2024-04-25T01:56:11.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e736b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 0x7fb6e010eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e736b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e0101d90 0x7fb6e010f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e736b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e010f6e0 con 0x7fb6e00fbd40 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e736b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e010f820 con 0x7fb6e0101d90 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e5107700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 0x7fb6e010eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e5107700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 0x7fb6e010eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52708/0 (socket says 172.21.15.33:52708) 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e5107700 1 -- 172.21.15.33:0/3836866066 learned_addr learned my addr 172.21.15.33:0/3836866066 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e4906700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e0101d90 0x7fb6e010f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e5107700 1 -- 172.21.15.33:0/3836866066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e0101d90 msgr2=0x7fb6e010f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e5107700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e0101d90 0x7fb6e010f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e5107700 1 -- 172.21.15.33:0/3836866066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0005040 con 0x7fb6e00fbd40 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e5107700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 0x7fb6e010eb50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc002a50 tx=0x7fb6dc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6d5ffb700 1 -- 172.21.15.33:0/3836866066 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6dc007cc0 con 0x7fb6e00fbd40 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6d5ffb700 1 -- 172.21.15.33:0/3836866066 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6dc00f040 con 0x7fb6e00fbd40 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e0117bc0 con 0x7fb6e00fbd40 2024-04-25T01:56:11.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.949+0000 7fb6d5ffb700 1 -- 172.21.15.33:0/3836866066 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6dc009640 con 0x7fb6e00fbd40 2024-04-25T01:56:11.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.950+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e0118120 con 0x7fb6e00fbd40 2024-04-25T01:56:11.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.951+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e00fd430 con 0x7fb6e00fbd40 2024-04-25T01:56:11.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.951+0000 7fb6d5ffb700 1 -- 172.21.15.33:0/3836866066 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb6dc0097a0 con 0x7fb6e00fbd40 2024-04-25T01:56:11.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.951+0000 7fb6d5ffb700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6cc06e140 0x7fb6cc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:11.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.952+0000 7fb6d5ffb700 1 -- 172.21.15.33:0/3836866066 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb6dc08d920 con 0x7fb6e00fbd40 2024-04-25T01:56:11.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.952+0000 7fb6e4906700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6cc06e140 0x7fb6cc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:11.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.952+0000 7fb6e4906700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6cc06e140 0x7fb6cc0705f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0000c00 tx=0x7fb6d0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:11.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:11.956+0000 7fb6d5ffb700 1 -- 172.21.15.33:0/3836866066 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb6dc056ca0 con 0x7fb6e00fbd40 2024-04-25T01:56:12.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.181+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6e0065d90 con 0x7fb6cc06e140 2024-04-25T01:56:12.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:12 smithi033 ceph-mon[111306]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:12.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:12 smithi033 ceph-mon[111306]: pgmap v503: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:12.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:12 smithi033 ceph-mon[111306]: from='client.34964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:12.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:12 smithi033 ceph-mon[111306]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:12.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:12 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3263871720' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:56:12.184 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:12 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3554761716' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.182+0000 7fb6d5ffb700 1 -- 172.21.15.33:0/3836866066 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb6e0065d90 con 0x7fb6cc06e140 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:56:12.185 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:56:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:56:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:56:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:56:12.186 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:56:12.186 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:56:12.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.184+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6cc06e140 msgr2=0x7fb6cc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:12.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.184+0000 7fb6e736b700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6cc06e140 0x7fb6cc0705f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0000c00 tx=0x7fb6d0009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:12.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.184+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 msgr2=0x7fb6e010eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.184+0000 7fb6e736b700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 0x7fb6e010eb50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc002a50 tx=0x7fb6dc0046b0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.185+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 shutdown_connections 2024-04-25T01:56:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.185+0000 7fb6e736b700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb6cc06e140 0x7fb6cc0705f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.185+0000 7fb6e736b700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6e00fbd40 0x7fb6e010eb50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.185+0000 7fb6e736b700 1 --2- 172.21.15.33:0/3836866066 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6e0101d90 0x7fb6e010f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.185+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 >> 172.21.15.33:0/3836866066 conn(0x7fb6e0071d00 msgr2=0x7fb6e00fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:12.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.185+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 shutdown_connections 2024-04-25T01:56:12.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.185+0000 7fb6e736b700 1 -- 172.21.15.33:0/3836866066 wait complete. 2024-04-25T01:56:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.289+0000 7f8f72681700 1 -- 172.21.15.33:0/137438538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 msgr2=0x7f8f6c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.289+0000 7f8f72681700 1 --2- 172.21.15.33:0/137438538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 0x7f8f6c102160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c004940 tx=0x7f8f5c009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:12.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.289+0000 7f8f72681700 1 -- 172.21.15.33:0/137438538 shutdown_connections 2024-04-25T01:56:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.289+0000 7f8f72681700 1 --2- 172.21.15.33:0/137438538 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c0fbd40 0x7f8f6c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.289+0000 7f8f72681700 1 --2- 172.21.15.33:0/137438538 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 0x7f8f6c102160 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.289+0000 7f8f72681700 1 -- 172.21.15.33:0/137438538 >> 172.21.15.33:0/137438538 conn(0x7f8f6c071d00 msgr2=0x7f8f6c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.289+0000 7f8f72681700 1 -- 172.21.15.33:0/137438538 shutdown_connections 2024-04-25T01:56:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.289+0000 7f8f72681700 1 -- 172.21.15.33:0/137438538 wait complete. 2024-04-25T01:56:12.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.289+0000 7f8f72681700 1 Processor -- start 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f72681700 1 -- start start 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f72681700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c0fbd40 0x7f8f6c113b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f72681700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 0x7f8f6c114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f72681700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c1146b0 con 0x7f8f6c101d90 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f72681700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c10daa0 con 0x7f8f6c0fbd40 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 0x7f8f6c114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f6b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 0x7f8f6c114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52726/0 (socket says 172.21.15.33:52726) 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f6b7fe700 1 -- 172.21.15.33:0/2692670455 learned_addr learned my addr 172.21.15.33:0/2692670455 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f6bfff700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c0fbd40 0x7f8f6c113b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:12.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f6b7fe700 1 -- 172.21.15.33:0/2692670455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c0fbd40 msgr2=0x7f8f6c113b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f6b7fe700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c0fbd40 0x7f8f6c113b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.290+0000 7f8f6b7fe700 1 -- 172.21.15.33:0/2692670455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c005040 con 0x7f8f6c101d90 2024-04-25T01:56:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.291+0000 7f8f6b7fe700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 0x7f8f6c114060 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000eb40 tx=0x7f8f6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.291+0000 7f8f68ff9700 1 -- 172.21.15.33:0/2692670455 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f60007cc0 con 0x7f8f6c101d90 2024-04-25T01:56:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.291+0000 7f8f68ff9700 1 -- 172.21.15.33:0/2692670455 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f6000f040 con 0x7f8f6c101d90 2024-04-25T01:56:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.291+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c10dd20 con 0x7f8f6c101d90 2024-04-25T01:56:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.291+0000 7f8f68ff9700 1 -- 172.21.15.33:0/2692670455 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f6000ca10 con 0x7f8f6c101d90 2024-04-25T01:56:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.291+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c10e260 con 0x7f8f6c101d90 2024-04-25T01:56:12.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.292+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c0fd430 con 0x7f8f6c101d90 2024-04-25T01:56:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.292+0000 7f8f68ff9700 1 -- 172.21.15.33:0/2692670455 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8f6000cc80 con 0x7f8f6c101d90 2024-04-25T01:56:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.293+0000 7f8f68ff9700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f5406e140 0x7f8f540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.293+0000 7f8f68ff9700 1 -- 172.21.15.33:0/2692670455 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8f6008d6f0 con 0x7f8f6c101d90 2024-04-25T01:56:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.293+0000 7f8f6bfff700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f5406e140 0x7f8f540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:12.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.293+0000 7f8f6bfff700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f5406e140 0x7f8f540705f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00b0d0 tx=0x7f8f5c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:12.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.297+0000 7f8f68ff9700 1 -- 172.21.15.33:0/2692670455 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8f6005a650 con 0x7f8f6c101d90 2024-04-25T01:56:12.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:12 smithi138 ceph-mon[81304]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:12.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:12 smithi138 ceph-mon[81304]: pgmap v503: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:12.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:12 smithi138 ceph-mon[81304]: from='client.34964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:12.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:12 smithi138 ceph-mon[81304]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:12.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:12 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3263871720' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:56:12.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:12 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3554761716' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:56:12.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.590+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8f6c108df0 con 0x7f8f6c101d90 2024-04-25T01:56:12.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.590+0000 7f8f68ff9700 1 -- 172.21.15.33:0/2692670455 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f8f6005a1e0 con 0x7f8f6c101d90 2024-04-25T01:56:12.593 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:56:12.593 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:56:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:56:12.593 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:56:12.593 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f5406e140 msgr2=0x7f8f540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f5406e140 0x7f8f540705f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c00b0d0 tx=0x7f8f5c009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 msgr2=0x7f8f6c114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 0x7f8f6c114060 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8f6000eb40 tx=0x7f8f6000b040 comp rx=0 tx=0).stop 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 shutdown_connections 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8f5406e140 0x7f8f540705f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8f6c0fbd40 0x7f8f6c113b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 --2- 172.21.15.33:0/2692670455 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8f6c101d90 0x7f8f6c114060 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 >> 172.21.15.33:0/2692670455 conn(0x7f8f6c071d00 msgr2=0x7f8f6c0fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 shutdown_connections 2024-04-25T01:56:12.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:12.593+0000 7f8f72681700 1 -- 172.21.15.33:0/2692670455 wait complete. 2024-04-25T01:56:13.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:13 smithi033 ceph-mon[111306]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:13.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:13 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2692670455' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:56:13.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:13 smithi138 ceph-mon[81304]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:13.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:13 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2692670455' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:56:14.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:14 smithi033 ceph-mon[111306]: pgmap v504: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:14.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:14 smithi138 ceph-mon[81304]: pgmap v504: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:16.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:16 smithi033 ceph-mon[111306]: pgmap v505: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:16.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:16 smithi138 ceph-mon[81304]: pgmap v505: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:18.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:18 smithi033 ceph-mon[111306]: pgmap v506: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:18.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:18 smithi138 ceph-mon[81304]: pgmap v506: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:20.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:20 smithi033 ceph-mon[111306]: pgmap v507: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:20.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:20 smithi138 ceph-mon[81304]: pgmap v507: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:22.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:22 smithi033 ceph-mon[111306]: pgmap v508: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:22.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:22 smithi138 ceph-mon[81304]: pgmap v508: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:24.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:24 smithi033 ceph-mon[111306]: pgmap v509: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:24.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:24 smithi138 ceph-mon[81304]: pgmap v509: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:26.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:26 smithi033 ceph-mon[111306]: pgmap v510: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:26.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:56:26.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:26 smithi138 ceph-mon[81304]: pgmap v510: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:26.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:56:28.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:28 smithi033 ceph-mon[111306]: pgmap v511: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:28.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:28 smithi138 ceph-mon[81304]: pgmap v511: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:30.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:30 smithi033 ceph-mon[111306]: pgmap v512: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:30.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:30 smithi138 ceph-mon[81304]: pgmap v512: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:32.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:32 smithi033 ceph-mon[111306]: pgmap v513: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:32.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:32 smithi138 ceph-mon[81304]: pgmap v513: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:34.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:34 smithi033 ceph-mon[111306]: pgmap v514: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:34.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:34 smithi138 ceph-mon[81304]: pgmap v514: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:36.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:36 smithi033 ceph-mon[111306]: pgmap v515: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:36.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:36 smithi138 ceph-mon[81304]: pgmap v515: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:38.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:38 smithi033 ceph-mon[111306]: pgmap v516: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:38.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:38 smithi138 ceph-mon[81304]: pgmap v516: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:40.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:40 smithi033 ceph-mon[111306]: pgmap v517: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:40.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:40 smithi138 ceph-mon[81304]: pgmap v517: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:41.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:56:41.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:56:42.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:42 smithi033 ceph-mon[111306]: pgmap v518: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:42.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:42 smithi138 ceph-mon[81304]: pgmap v518: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.711+0000 7f6c25076700 1 -- 172.21.15.33:0/3918907421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20065640 msgr2=0x7f6c20065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.711+0000 7f6c25076700 1 --2- 172.21.15.33:0/3918907421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20065640 0x7f6c20065ab0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6c10004940 tx=0x7f6c10009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.712+0000 7f6c25076700 1 -- 172.21.15.33:0/3918907421 shutdown_connections 2024-04-25T01:56:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.712+0000 7f6c25076700 1 --2- 172.21.15.33:0/3918907421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20065640 0x7f6c20065ab0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.712+0000 7f6c25076700 1 --2- 172.21.15.33:0/3918907421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c20102e50 0x7f6c20103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.712+0000 7f6c25076700 1 -- 172.21.15.33:0/3918907421 >> 172.21.15.33:0/3918907421 conn(0x7f6c20071d00 msgr2=0x7f6c20072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.712+0000 7f6c25076700 1 -- 172.21.15.33:0/3918907421 shutdown_connections 2024-04-25T01:56:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.712+0000 7f6c25076700 1 -- 172.21.15.33:0/3918907421 wait complete. 2024-04-25T01:56:42.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.712+0000 7f6c25076700 1 Processor -- start 2024-04-25T01:56:42.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c25076700 1 -- start start 2024-04-25T01:56:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c25076700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c20065640 0x7f6c20113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c25076700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20102e50 0x7f6c201140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c25076700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c20114700 con 0x7f6c20102e50 2024-04-25T01:56:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c25076700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2010daf0 con 0x7f6c20065640 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20102e50 0x7f6c201140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c1e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20102e50 0x7f6c201140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32948/0 (socket says 172.21.15.33:32948) 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c1e59c700 1 -- 172.21.15.33:0/350323073 learned_addr learned my addr 172.21.15.33:0/350323073 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c1ed9d700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c20065640 0x7f6c20113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c1e59c700 1 -- 172.21.15.33:0/350323073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c20065640 msgr2=0x7f6c20113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c1e59c700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c20065640 0x7f6c20113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.713+0000 7f6c1e59c700 1 -- 172.21.15.33:0/350323073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c10005040 con 0x7f6c20102e50 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.714+0000 7f6c1e59c700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20102e50 0x7f6c201140b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009150 tx=0x7f6c10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.714+0000 7f6c177fe700 1 -- 172.21.15.33:0/350323073 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1000e040 con 0x7f6c20102e50 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.714+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2010dd70 con 0x7f6c20102e50 2024-04-25T01:56:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.714+0000 7f6c177fe700 1 -- 172.21.15.33:0/350323073 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c10013420 con 0x7f6c20102e50 2024-04-25T01:56:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.714+0000 7f6c177fe700 1 -- 172.21.15.33:0/350323073 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1001c630 con 0x7f6c20102e50 2024-04-25T01:56:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.714+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2010e330 con 0x7f6c20102e50 2024-04-25T01:56:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.715+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c20100100 con 0x7f6c20102e50 2024-04-25T01:56:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.716+0000 7f6c177fe700 1 -- 172.21.15.33:0/350323073 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6c10013590 con 0x7f6c20102e50 2024-04-25T01:56:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.716+0000 7f6c177fe700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6c0c06e150 0x7f6c0c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.716+0000 7f6c177fe700 1 -- 172.21.15.33:0/350323073 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6c1008eda0 con 0x7f6c20102e50 2024-04-25T01:56:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.716+0000 7f6c1ed9d700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6c0c06e150 0x7f6c0c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.717+0000 7f6c1ed9d700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6c0c06e150 0x7f6c0c070600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c08007fd0 tx=0x7f6c08009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:42.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.721+0000 7f6c177fe700 1 -- 172.21.15.33:0/350323073 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6c1005bcd0 con 0x7f6c20102e50 2024-04-25T01:56:42.952 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.949+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c2010f1c0 con 0x7f6c0c06e150 2024-04-25T01:56:42.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.951+0000 7f6c177fe700 1 -- 172.21.15.33:0/350323073 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6c2010f1c0 con 0x7f6c0c06e150 2024-04-25T01:56:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6c0c06e150 msgr2=0x7f6c0c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6c0c06e150 0x7f6c0c070600 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6c08007fd0 tx=0x7f6c08009040 comp rx=0 tx=0).stop 2024-04-25T01:56:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20102e50 msgr2=0x7f6c201140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20102e50 0x7f6c201140b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009150 tx=0x7f6c10009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 shutdown_connections 2024-04-25T01:56:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6c0c06e150 0x7f6c0c070600 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6c20065640 0x7f6c20113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:42.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 --2- 172.21.15.33:0/350323073 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6c20102e50 0x7f6c201140b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 >> 172.21.15.33:0/350323073 conn(0x7f6c20071d00 msgr2=0x7f6c200fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 shutdown_connections 2024-04-25T01:56:42.957 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:42.953+0000 7f6c25076700 1 -- 172.21.15.33:0/350323073 wait complete. 2024-04-25T01:56:42.966 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:56:43.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 -- 172.21.15.33:0/32782618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fe800 msgr2=0x7f70a8106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 --2- 172.21.15.33:0/32782618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fe800 0x7f70a8106e40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7098004940 tx=0x7f7098009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 -- 172.21.15.33:0/32782618 shutdown_connections 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 --2- 172.21.15.33:0/32782618 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fe800 0x7f70a8106e40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 --2- 172.21.15.33:0/32782618 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80fde60 0x7f70a80fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 -- 172.21.15.33:0/32782618 >> 172.21.15.33:0/32782618 conn(0x7f70a80f7870 msgr2=0x7f70a80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 -- 172.21.15.33:0/32782618 shutdown_connections 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 -- 172.21.15.33:0/32782618 wait complete. 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 Processor -- start 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.058+0000 7f70afb81700 1 -- start start 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70afb81700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fde60 0x7f70a8113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70afb81700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80fe800 0x7f70a8113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70afb81700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a81144b0 con 0x7f70a80fde60 2024-04-25T01:56:43.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70afb81700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a810dab0 con 0x7f70a80fe800 2024-04-25T01:56:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70ad91d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fde60 0x7f70a8113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70ad91d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fde60 0x7f70a8113920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32970/0 (socket says 172.21.15.33:32970) 2024-04-25T01:56:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70ad91d700 1 -- 172.21.15.33:0/3942433800 learned_addr learned my addr 172.21.15.33:0/3942433800 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70ad11c700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80fe800 0x7f70a8113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70ad91d700 1 -- 172.21.15.33:0/3942433800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80fe800 msgr2=0x7f70a8113e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70ad91d700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80fe800 0x7f70a8113e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70ad91d700 1 -- 172.21.15.33:0/3942433800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098005040 con 0x7f70a80fde60 2024-04-25T01:56:43.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70ad91d700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fde60 0x7f70a8113920 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c830 tx=0x7f70a4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f709e7fc700 1 -- 172.21.15.33:0/3942433800 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a40106f0 con 0x7f70a80fde60 2024-04-25T01:56:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f709e7fc700 1 -- 172.21.15.33:0/3942433800 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70a4010d20 con 0x7f70a80fde60 2024-04-25T01:56:43.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a810dd90 con 0x7f70a80fde60 2024-04-25T01:56:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.059+0000 7f709e7fc700 1 -- 172.21.15.33:0/3942433800 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a40113a0 con 0x7f70a80fde60 2024-04-25T01:56:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.060+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a810e2d0 con 0x7f70a80fde60 2024-04-25T01:56:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.060+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a8066ad0 con 0x7f70a80fde60 2024-04-25T01:56:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.061+0000 7f709e7fc700 1 -- 172.21.15.33:0/3942433800 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f70a4012400 con 0x7f70a80fde60 2024-04-25T01:56:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.061+0000 7f709e7fc700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f709406e140 0x7f70940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.062+0000 7f709e7fc700 1 -- 172.21.15.33:0/3942433800 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f70a408da70 con 0x7f70a80fde60 2024-04-25T01:56:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.062+0000 7f70ad11c700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f709406e140 0x7f70940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:43.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.062+0000 7f70ad11c700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f709406e140 0x7f70940705f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f70a810ec70 tx=0x7f7098009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:43.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.066+0000 7f709e7fc700 1 -- 172.21.15.33:0/3942433800 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f70a405a9a0 con 0x7f70a80fde60 2024-04-25T01:56:43.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.293+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70a8109e20 con 0x7f709406e140 2024-04-25T01:56:43.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.294+0000 7f709e7fc700 1 -- 172.21.15.33:0/3942433800 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f70a8109e20 con 0x7f709406e140 2024-04-25T01:56:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.296+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f709406e140 msgr2=0x7f70940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.296+0000 7f70afb81700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f709406e140 0x7f70940705f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f70a810ec70 tx=0x7f7098009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.296+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fde60 msgr2=0x7f70a8113920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.296+0000 7f70afb81700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fde60 0x7f70a8113920 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f70a400c830 tx=0x7f70a4004470 comp rx=0 tx=0).stop 2024-04-25T01:56:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.296+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 shutdown_connections 2024-04-25T01:56:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.296+0000 7f70afb81700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f709406e140 0x7f70940705f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.296+0000 7f70afb81700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70a80fde60 0x7f70a8113920 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.296+0000 7f70afb81700 1 --2- 172.21.15.33:0/3942433800 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70a80fe800 0x7f70a8113e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.296+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 >> 172.21.15.33:0/3942433800 conn(0x7f70a80f7870 msgr2=0x7f70a80f8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.297+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 shutdown_connections 2024-04-25T01:56:43.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.297+0000 7f70afb81700 1 -- 172.21.15.33:0/3942433800 wait complete. 2024-04-25T01:56:43.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.400+0000 7eff8eb68700 1 -- 172.21.15.33:0/2421727702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 msgr2=0x7eff880fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.400+0000 7eff8eb68700 1 --2- 172.21.15.33:0/2421727702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 0x7eff880fbd20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7eff74004940 tx=0x7eff74009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.400+0000 7eff8eb68700 1 -- 172.21.15.33:0/2421727702 shutdown_connections 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.400+0000 7eff8eb68700 1 --2- 172.21.15.33:0/2421727702 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 0x7eff880fbd20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.400+0000 7eff8eb68700 1 --2- 172.21.15.33:0/2421727702 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff88101d10 0x7eff881020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.400+0000 7eff8eb68700 1 -- 172.21.15.33:0/2421727702 >> 172.21.15.33:0/2421727702 conn(0x7eff880f7860 msgr2=0x7eff880f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.400+0000 7eff8eb68700 1 -- 172.21.15.33:0/2421727702 shutdown_connections 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.400+0000 7eff8eb68700 1 -- 172.21.15.33:0/2421727702 wait complete. 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff8eb68700 1 Processor -- start 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff8eb68700 1 -- start start 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff8eb68700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff88101d10 0x7eff8810eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff8eb68700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 0x7eff8810f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff8eb68700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8810f570 con 0x7eff881026b0 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff8eb68700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8810f6b0 con 0x7eff88101d10 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 0x7eff8810f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 0x7eff8810f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32986/0 (socket says 172.21.15.33:32986) 2024-04-25T01:56:43.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff87fff700 1 -- 172.21.15.33:0/3929230080 learned_addr learned my addr 172.21.15.33:0/3929230080 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.401+0000 7eff8c904700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff88101d10 0x7eff8810eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.402+0000 7eff87fff700 1 -- 172.21.15.33:0/3929230080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff88101d10 msgr2=0x7eff8810eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.402+0000 7eff87fff700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff88101d10 0x7eff8810eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.402+0000 7eff87fff700 1 -- 172.21.15.33:0/3929230080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff74005040 con 0x7eff881026b0 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.402+0000 7eff87fff700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 0x7eff8810f030 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7eff74009100 tx=0x7eff74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.402+0000 7eff857fa700 1 -- 172.21.15.33:0/3929230080 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7400e040 con 0x7eff881026b0 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.402+0000 7eff857fa700 1 -- 172.21.15.33:0/3929230080 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff74004290 con 0x7eff881026b0 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.402+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8810f930 con 0x7eff881026b0 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.402+0000 7eff857fa700 1 -- 172.21.15.33:0/3929230080 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7401c3f0 con 0x7eff881026b0 2024-04-25T01:56:43.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.402+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff88117f00 con 0x7eff881026b0 2024-04-25T01:56:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.403+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff880fd460 con 0x7eff881026b0 2024-04-25T01:56:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.404+0000 7eff857fa700 1 -- 172.21.15.33:0/3929230080 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7eff74013690 con 0x7eff881026b0 2024-04-25T01:56:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.404+0000 7eff857fa700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff7806e140 0x7eff780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.404+0000 7eff857fa700 1 -- 172.21.15.33:0/3929230080 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7eff7408ec50 con 0x7eff881026b0 2024-04-25T01:56:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.404+0000 7eff8c904700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff7806e140 0x7eff780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:43.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.405+0000 7eff8c904700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff7806e140 0x7eff780705f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7eff7c004f40 tx=0x7eff7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:43.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.409+0000 7eff857fa700 1 -- 172.21.15.33:0/3929230080 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7eff74058050 con 0x7eff881026b0 2024-04-25T01:56:43.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.625+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7eff88108e60 con 0x7eff7806e140 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.635+0000 7eff857fa700 1 -- 172.21.15.33:0/3929230080 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7eff88108e60 con 0x7eff7806e140 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (17m) 10m ago 29m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (15m) 10m ago 29m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (15m) 10m ago 28m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (17m) 10m ago 28m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (11m) 10m ago 23m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (11m) 10m ago 23m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (10m) 10m ago 23m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (23m) 10m ago 23m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:56:43.638 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (19m) 10m ago 30m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (18m) 10m ago 28m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (16m) 10m ago 30m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (15m) 10m ago 28m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (18m) 10m ago 29m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (18m) 10m ago 28m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (14m) 10m ago 27m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (14m) 10m ago 26m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (13m) 10m ago 26m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (12m) 10m ago 26m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (12m) 10m ago 25m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (11m) 10m ago 25m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:56:43.639 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (18m) 10m ago 28m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:56:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.637+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff7806e140 msgr2=0x7eff780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.637+0000 7eff8eb68700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff7806e140 0x7eff780705f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7eff7c004f40 tx=0x7eff7c00b040 comp rx=0 tx=0).stop 2024-04-25T01:56:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.637+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 msgr2=0x7eff8810f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.637+0000 7eff8eb68700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 0x7eff8810f030 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7eff74009100 tx=0x7eff74009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.638+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 shutdown_connections 2024-04-25T01:56:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.638+0000 7eff8eb68700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff7806e140 0x7eff780705f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.638+0000 7eff8eb68700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff88101d10 0x7eff8810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.638+0000 7eff8eb68700 1 --2- 172.21.15.33:0/3929230080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff881026b0 0x7eff8810f030 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.638+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 >> 172.21.15.33:0/3929230080 conn(0x7eff880f7860 msgr2=0x7eff880f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:43.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.638+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 shutdown_connections 2024-04-25T01:56:43.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.638+0000 7eff8eb68700 1 -- 172.21.15.33:0/3929230080 wait complete. 2024-04-25T01:56:43.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.740+0000 7f38973c1700 1 -- 172.21.15.33:0/977954977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 msgr2=0x7f38900fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.740+0000 7f38973c1700 1 --2- 172.21.15.33:0/977954977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 0x7f38900fcdf0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3880004940 tx=0x7f3880009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:43.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.740+0000 7f38973c1700 1 -- 172.21.15.33:0/977954977 shutdown_connections 2024-04-25T01:56:43.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.740+0000 7f38973c1700 1 --2- 172.21.15.33:0/977954977 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 0x7f38900fcdf0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.740+0000 7f38973c1700 1 --2- 172.21.15.33:0/977954977 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3890102de0 0x7f38901031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.740+0000 7f38973c1700 1 -- 172.21.15.33:0/977954977 >> 172.21.15.33:0/977954977 conn(0x7f38900f8990 msgr2=0x7f38900fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:43.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.740+0000 7f38973c1700 1 -- 172.21.15.33:0/977954977 shutdown_connections 2024-04-25T01:56:43.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.740+0000 7f38973c1700 1 -- 172.21.15.33:0/977954977 wait complete. 2024-04-25T01:56:43.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.741+0000 7f38973c1700 1 Processor -- start 2024-04-25T01:56:43.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.741+0000 7f38973c1700 1 -- start start 2024-04-25T01:56:43.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.741+0000 7f38973c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3890102de0 0x7f3890070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:43.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.741+0000 7f38973c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 0x7f38900707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:43.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.741+0000 7f38973c1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38900721b0 con 0x7f3890103780 2024-04-25T01:56:43.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.741+0000 7f38973c1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3890070ce0 con 0x7f3890102de0 2024-04-25T01:56:43.745 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.741+0000 7f389495c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 0x7f38900707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.741+0000 7f389495c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 0x7f38900707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33002/0 (socket says 172.21.15.33:33002) 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.741+0000 7f389495c700 1 -- 172.21.15.33:0/4101364780 learned_addr learned my addr 172.21.15.33:0/4101364780 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f389515d700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3890102de0 0x7f3890070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f389495c700 1 -- 172.21.15.33:0/4101364780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3890102de0 msgr2=0x7f3890070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f389495c700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3890102de0 0x7f3890070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f389495c700 1 -- 172.21.15.33:0/4101364780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880005040 con 0x7f3890103780 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f389495c700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 0x7f38900707a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3880009100 tx=0x7f3880009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f3885ffb700 1 -- 172.21.15.33:0/4101364780 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388000e040 con 0x7f3890103780 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3890070f60 con 0x7f3890103780 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f3885ffb700 1 -- 172.21.15.33:0/4101364780 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3880013420 con 0x7f3890103780 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f3885ffb700 1 -- 172.21.15.33:0/4101364780 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f388001c610 con 0x7f3890103780 2024-04-25T01:56:43.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.742+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389011a0e0 con 0x7f3890103780 2024-04-25T01:56:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.743+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38900fe530 con 0x7f3890103780 2024-04-25T01:56:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.744+0000 7f3885ffb700 1 -- 172.21.15.33:0/4101364780 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3880013590 con 0x7f3890103780 2024-04-25T01:56:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.744+0000 7f3885ffb700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f387c06e140 0x7f387c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:43.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.744+0000 7f3885ffb700 1 -- 172.21.15.33:0/4101364780 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f388008eda0 con 0x7f3890103780 2024-04-25T01:56:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.744+0000 7f389515d700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f387c06e140 0x7f387c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:43.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.745+0000 7f389515d700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f387c06e140 0x7f387c0705f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fd0 tx=0x7f388c009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:43.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:43.749+0000 7f3885ffb700 1 -- 172.21.15.33:0/4101364780 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f38800581a0 con 0x7f3890103780 2024-04-25T01:56:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.049+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f389011a2d0 con 0x7f3890103780 2024-04-25T01:56:44.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.049+0000 7f3885ffb700 1 -- 172.21.15.33:0/4101364780 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f388005b860 con 0x7f3890103780 2024-04-25T01:56:44.052 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:56:44.052 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:56:44.052 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:56:44.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:56:44.052 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:56:44.052 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:56:44.052 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:56:44.053 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:56:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f387c06e140 msgr2=0x7f387c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f387c06e140 0x7f387c0705f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fd0 tx=0x7f388c009e50 comp rx=0 tx=0).stop 2024-04-25T01:56:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 msgr2=0x7f38900707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 0x7f38900707a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3880009100 tx=0x7f3880009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 shutdown_connections 2024-04-25T01:56:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f387c06e140 0x7f387c0705f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3890102de0 0x7f3890070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 --2- 172.21.15.33:0/4101364780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3890103780 0x7f38900707a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 >> 172.21.15.33:0/4101364780 conn(0x7f38900f8990 msgr2=0x7f38900f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 shutdown_connections 2024-04-25T01:56:44.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.052+0000 7f38973c1700 1 -- 172.21.15.33:0/4101364780 wait complete. 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.157+0000 7f0ccf79d700 1 -- 172.21.15.33:0/350487918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc8101d90 msgr2=0x7f0cc8102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.157+0000 7f0ccf79d700 1 --2- 172.21.15.33:0/350487918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc8101d90 0x7f0cc8102160 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0009ce0 tx=0x7f0cc0004d10 comp rx=0 tx=0).stop 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.157+0000 7f0ccf79d700 1 -- 172.21.15.33:0/350487918 shutdown_connections 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.157+0000 7f0ccf79d700 1 --2- 172.21.15.33:0/350487918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc80fbd40 0x7f0cc80fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.157+0000 7f0ccf79d700 1 --2- 172.21.15.33:0/350487918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc8101d90 0x7f0cc8102160 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.157+0000 7f0ccf79d700 1 -- 172.21.15.33:0/350487918 >> 172.21.15.33:0/350487918 conn(0x7f0cc8071d00 msgr2=0x7f0cc8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.157+0000 7f0ccf79d700 1 -- 172.21.15.33:0/350487918 shutdown_connections 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.157+0000 7f0ccf79d700 1 -- 172.21.15.33:0/350487918 wait complete. 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccf79d700 1 Processor -- start 2024-04-25T01:56:44.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccf79d700 1 -- start start 2024-04-25T01:56:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccf79d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc80fbd40 0x7f0cc810eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccf79d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc8101d90 0x7f0cc810f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccf79d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc810f920 con 0x7f0cc80fbd40 2024-04-25T01:56:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccf79d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc810fa60 con 0x7f0cc8101d90 2024-04-25T01:56:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccd539700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc80fbd40 0x7f0cc810eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccd539700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc80fbd40 0x7f0cc810eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33016/0 (socket says 172.21.15.33:33016) 2024-04-25T01:56:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccd539700 1 -- 172.21.15.33:0/3066241098 learned_addr learned my addr 172.21.15.33:0/3066241098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:44.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0cccd38700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc8101d90 0x7f0cc810f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccd539700 1 -- 172.21.15.33:0/3066241098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc8101d90 msgr2=0x7f0cc810f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccd539700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc8101d90 0x7f0cc810f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccd539700 1 -- 172.21.15.33:0/3066241098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc0009940 con 0x7f0cc80fbd40 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.158+0000 7f0ccd539700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc80fbd40 0x7f0cc810eea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0005010 tx=0x7f0cc0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.159+0000 7f0cc5ffb700 1 -- 172.21.15.33:0/3066241098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc000e040 con 0x7f0cc80fbd40 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.159+0000 7f0cc5ffb700 1 -- 172.21.15.33:0/3066241098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0cc0008e20 con 0x7f0cc80fbd40 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.159+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cc8117db0 con 0x7f0cc80fbd40 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.159+0000 7f0cc5ffb700 1 -- 172.21.15.33:0/3066241098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cc001c780 con 0x7f0cc80fbd40 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.159+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc8118340 con 0x7f0cc80fbd40 2024-04-25T01:56:44.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.160+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc80fd430 con 0x7f0cc80fbd40 2024-04-25T01:56:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.160+0000 7f0cc5ffb700 1 -- 172.21.15.33:0/3066241098 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0cc0012b70 con 0x7f0cc80fbd40 2024-04-25T01:56:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.161+0000 7f0cc5ffb700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0cb006e150 0x7f0cb0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.161+0000 7f0cc5ffb700 1 -- 172.21.15.33:0/3066241098 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0cc008f690 con 0x7f0cc80fbd40 2024-04-25T01:56:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.161+0000 7f0cccd38700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0cb006e150 0x7f0cb0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:44.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.161+0000 7f0cccd38700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0cb006e150 0x7f0cb0070600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc006fd0 tx=0x7f0cbc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:44.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.165+0000 7f0cc5ffb700 1 -- 172.21.15.33:0/3066241098 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0cc0058a90 con 0x7f0cc80fbd40 2024-04-25T01:56:44.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.421+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0cc805f5c0 con 0x7f0cc80fbd40 2024-04-25T01:56:44.424 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:44 smithi033 ceph-mon[111306]: pgmap v519: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:44.424 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:44 smithi033 ceph-mon[111306]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:44.425 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:44 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4101364780' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:56:44.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.422+0000 7f0cc5ffb700 1 -- 172.21.15.33:0/3066241098 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f0cc005c150 con 0x7f0cc80fbd40 2024-04-25T01:56:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:56:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:56:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:56:44.425 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:56:44.425 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:56:44.426 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:56:44.427 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:56:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:56:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:56:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:56:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:56:44.428 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:56:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:56:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:56:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:56:44.428 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:56:44.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0cb006e150 msgr2=0x7f0cb0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0cb006e150 0x7f0cb0070600 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc006fd0 tx=0x7f0cbc008040 comp rx=0 tx=0).stop 2024-04-25T01:56:44.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc80fbd40 msgr2=0x7f0cc810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc80fbd40 0x7f0cc810eea0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0cc0005010 tx=0x7f0cc0004d10 comp rx=0 tx=0).stop 2024-04-25T01:56:44.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 shutdown_connections 2024-04-25T01:56:44.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0cb006e150 0x7f0cb0070600 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0cc80fbd40 0x7f0cc810eea0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 --2- 172.21.15.33:0/3066241098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0cc8101d90 0x7f0cc810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 >> 172.21.15.33:0/3066241098 conn(0x7f0cc8071d00 msgr2=0x7f0cc80fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:44.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 shutdown_connections 2024-04-25T01:56:44.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.425+0000 7f0ccf79d700 1 -- 172.21.15.33:0/3066241098 wait complete. 2024-04-25T01:56:44.430 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:56:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.532+0000 7f2a475a0700 1 -- 172.21.15.33:0/1901367656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 msgr2=0x7f2a40103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.532+0000 7f2a475a0700 1 --2- 172.21.15.33:0/1901367656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 0x7f2a40103190 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c004940 tx=0x7f2a3c009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.532+0000 7f2a475a0700 1 -- 172.21.15.33:0/1901367656 shutdown_connections 2024-04-25T01:56:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.532+0000 7f2a475a0700 1 --2- 172.21.15.33:0/1901367656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a40103760 0x7f2a400fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.532+0000 7f2a475a0700 1 --2- 172.21.15.33:0/1901367656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 0x7f2a40103190 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.532+0000 7f2a475a0700 1 -- 172.21.15.33:0/1901367656 >> 172.21.15.33:0/1901367656 conn(0x7f2a400f8930 msgr2=0x7f2a400fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.532+0000 7f2a475a0700 1 -- 172.21.15.33:0/1901367656 shutdown_connections 2024-04-25T01:56:44.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a475a0700 1 -- 172.21.15.33:0/1901367656 wait complete. 2024-04-25T01:56:44.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a475a0700 1 Processor -- start 2024-04-25T01:56:44.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a475a0700 1 -- start start 2024-04-25T01:56:44.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a475a0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 0x7f2a40070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:44.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a475a0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a40103760 0x7f2a400707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:44.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a475a0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40072240 con 0x7f2a40102dc0 2024-04-25T01:56:44.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a475a0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40070ce0 con 0x7f2a40103760 2024-04-25T01:56:44.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a4533c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 0x7f2a40070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a4533c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 0x7f2a40070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33032/0 (socket says 172.21.15.33:33032) 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.533+0000 7f2a4533c700 1 -- 172.21.15.33:0/3842842895 learned_addr learned my addr 172.21.15.33:0/3842842895 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a4533c700 1 -- 172.21.15.33:0/3842842895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a40103760 msgr2=0x7f2a400707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a44b3b700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a40103760 0x7f2a400707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a4533c700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a40103760 0x7f2a400707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a4533c700 1 -- 172.21.15.33:0/3842842895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a3c005040 con 0x7f2a40102dc0 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a44b3b700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a40103760 0x7f2a400707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a4533c700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 0x7f2a40070260 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00b3b0 tx=0x7f2a3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a35ffb700 1 -- 172.21.15.33:0/3842842895 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c00e040 con 0x7f2a40102dc0 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a40070f60 con 0x7f2a40102dc0 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a35ffb700 1 -- 172.21.15.33:0/3842842895 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a3c00bbc0 con 0x7f2a40102dc0 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a35ffb700 1 -- 172.21.15.33:0/3842842895 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c01c3f0 con 0x7f2a40102dc0 2024-04-25T01:56:44.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.534+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4011a170 con 0x7f2a40102dc0 2024-04-25T01:56:44.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.535+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a401000d0 con 0x7f2a40102dc0 2024-04-25T01:56:44.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.536+0000 7f2a35ffb700 1 -- 172.21.15.33:0/3842842895 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2a3c00bd30 con 0x7f2a40102dc0 2024-04-25T01:56:44.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.536+0000 7f2a35ffb700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a2c06e140 0x7f2a2c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:44.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.536+0000 7f2a35ffb700 1 -- 172.21.15.33:0/3842842895 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2a3c08ed70 con 0x7f2a40102dc0 2024-04-25T01:56:44.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.536+0000 7f2a44b3b700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a2c06e140 0x7f2a2c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:44.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.537+0000 7f2a44b3b700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a2c06e140 0x7f2a2c0705f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2a30006fd0 tx=0x7f2a30008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:44.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:44 smithi138 ceph-mon[81304]: pgmap v519: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:44.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:44 smithi138 ceph-mon[81304]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:44.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:44 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4101364780' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:56:44.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.541+0000 7f2a35ffb700 1 -- 172.21.15.33:0/3842842895 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2a3c0580f0 con 0x7f2a40102dc0 2024-04-25T01:56:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.770+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a40071740 con 0x7f2a2c06e140 2024-04-25T01:56:44.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.771+0000 7f2a35ffb700 1 -- 172.21.15.33:0/3842842895 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2a40071740 con 0x7f2a2c06e140 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:56:44.774 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:56:44.775 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:56:44.775 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:56:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.773+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a2c06e140 msgr2=0x7f2a2c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a2c06e140 0x7f2a2c0705f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2a30006fd0 tx=0x7f2a30008040 comp rx=0 tx=0).stop 2024-04-25T01:56:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 msgr2=0x7f2a40070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 0x7f2a40070260 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00b3b0 tx=0x7f2a3c009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 shutdown_connections 2024-04-25T01:56:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a2c06e140 0x7f2a2c0705f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a40102dc0 0x7f2a40070260 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 --2- 172.21.15.33:0/3842842895 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a40103760 0x7f2a400707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 >> 172.21.15.33:0/3842842895 conn(0x7f2a400f8930 msgr2=0x7f2a40107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 shutdown_connections 2024-04-25T01:56:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.774+0000 7f2a475a0700 1 -- 172.21.15.33:0/3842842895 wait complete. 2024-04-25T01:56:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.877+0000 7fbfaa729700 1 -- 172.21.15.33:0/1732672488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fde10 msgr2=0x7fbfa40fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.878+0000 7fbfaa729700 1 --2- 172.21.15.33:0/1732672488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fde10 0x7fbfa40fe1e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbf94004940 tx=0x7fbf94009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.878+0000 7fbfaa729700 1 -- 172.21.15.33:0/1732672488 shutdown_connections 2024-04-25T01:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.878+0000 7fbfaa729700 1 --2- 172.21.15.33:0/1732672488 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfa40fe7b0 0x7fbfa4106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.878+0000 7fbfaa729700 1 --2- 172.21.15.33:0/1732672488 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fde10 0x7fbfa40fe1e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.878+0000 7fbfaa729700 1 -- 172.21.15.33:0/1732672488 >> 172.21.15.33:0/1732672488 conn(0x7fbfa40f7820 msgr2=0x7fbfa40f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.878+0000 7fbfaa729700 1 -- 172.21.15.33:0/1732672488 shutdown_connections 2024-04-25T01:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.878+0000 7fbfaa729700 1 -- 172.21.15.33:0/1732672488 wait complete. 2024-04-25T01:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.878+0000 7fbfaa729700 1 Processor -- start 2024-04-25T01:56:44.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.878+0000 7fbfaa729700 1 -- start start 2024-04-25T01:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfaa729700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfa40fde10 0x7fbfa4113c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfaa729700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fe7b0 0x7fbfa4114150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfaa729700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa41147a0 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfaa729700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa410ddc0 con 0x7fbfa40fde10 2024-04-25T01:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fe7b0 0x7fbfa4114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fe7b0 0x7fbfa4114150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33060/0 (socket says 172.21.15.33:33060) 2024-04-25T01:56:44.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfa37fe700 1 -- 172.21.15.33:0/43116996 learned_addr learned my addr 172.21.15.33:0/43116996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfa3fff700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfa40fde10 0x7fbfa4113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfa37fe700 1 -- 172.21.15.33:0/43116996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfa40fde10 msgr2=0x7fbfa4113c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfa37fe700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfa40fde10 0x7fbfa4113c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfa37fe700 1 -- 172.21.15.33:0/43116996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94005040 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.879+0000 7fbfa37fe700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fe7b0 0x7fbfa4114150 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf98002a50 tx=0x7fbf980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.880+0000 7fbfa0ff9700 1 -- 172.21.15.33:0/43116996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf98007cc0 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.880+0000 7fbfa0ff9700 1 -- 172.21.15.33:0/43116996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf9800f040 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.880+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa410e0a0 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.880+0000 7fbfa0ff9700 1 -- 172.21.15.33:0/43116996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf9800da30 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.880+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa410e5e0 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.881+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa4066b30 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.881+0000 7fbfa0ff9700 1 -- 172.21.15.33:0/43116996 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbf9800db90 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.882+0000 7fbfa0ff9700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbf8c06e140 0x7fbf8c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:56:44.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.882+0000 7fbfa0ff9700 1 -- 172.21.15.33:0/43116996 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbf9808d8c0 con 0x7fbfa40fe7b0 2024-04-25T01:56:44.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.882+0000 7fbfa3fff700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbf8c06e140 0x7fbf8c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:56:44.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.882+0000 7fbfa3fff700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbf8c06e140 0x7fbf8c0705f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009fd0 tx=0x7fbf94009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:56:44.889 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:44.886+0000 7fbfa0ff9700 1 -- 172.21.15.33:0/43116996 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbf98056c40 con 0x7fbfa40fe7b0 2024-04-25T01:56:45.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.179+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbfa4109ea0 con 0x7fbfa40fe7b0 2024-04-25T01:56:45.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.180+0000 7fbfa0ff9700 1 -- 172.21.15.33:0/43116996 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fbf9805a300 con 0x7fbfa40fe7b0 2024-04-25T01:56:45.182 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:56:45.183 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:56:45.183 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:56:45.183 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:56:45.183 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:56:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbf8c06e140 msgr2=0x7fbf8c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbf8c06e140 0x7fbf8c0705f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009fd0 tx=0x7fbf94009e30 comp rx=0 tx=0).stop 2024-04-25T01:56:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fe7b0 msgr2=0x7fbfa4114150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:56:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fe7b0 0x7fbfa4114150 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fbf98002a50 tx=0x7fbf980046b0 comp rx=0 tx=0).stop 2024-04-25T01:56:45.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 shutdown_connections 2024-04-25T01:56:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbf8c06e140 0x7fbf8c0705f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfa40fde10 0x7fbfa4113c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 --2- 172.21.15.33:0/43116996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfa40fe7b0 0x7fbfa4114150 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:56:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 >> 172.21.15.33:0/43116996 conn(0x7fbfa40f7820 msgr2=0x7fbfa40f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:56:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 shutdown_connections 2024-04-25T01:56:45.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:56:45.182+0000 7fbfaa729700 1 -- 172.21.15.33:0/43116996 wait complete. 2024-04-25T01:56:45.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:45 smithi033 ceph-mon[111306]: from='client.34992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:45.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:45 smithi033 ceph-mon[111306]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:45.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:45 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3066241098' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:56:45.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:45 smithi138 ceph-mon[81304]: from='client.34992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:45.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:45 smithi138 ceph-mon[81304]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:45.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:45 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3066241098' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:56:46.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:46 smithi033 ceph-mon[111306]: pgmap v520: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:46.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:46 smithi033 ceph-mon[111306]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:46.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:46 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/43116996' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:56:46.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:46 smithi138 ceph-mon[81304]: pgmap v520: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:46.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:46 smithi138 ceph-mon[81304]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:56:46.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:46 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/43116996' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:56:48.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:48 smithi033 ceph-mon[111306]: pgmap v521: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:48.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:48 smithi138 ceph-mon[81304]: pgmap v521: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:50.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:50 smithi033 ceph-mon[111306]: pgmap v522: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:50.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:50 smithi138 ceph-mon[81304]: pgmap v522: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:52.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:52 smithi033 ceph-mon[111306]: pgmap v523: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:52.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:52 smithi138 ceph-mon[81304]: pgmap v523: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:54.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:54 smithi033 ceph-mon[111306]: pgmap v524: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:54.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:54 smithi138 ceph-mon[81304]: pgmap v524: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:56.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:56 smithi033 ceph-mon[111306]: pgmap v525: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:56.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:56:56.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:56 smithi138 ceph-mon[81304]: pgmap v525: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:56.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:56:58.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:56:58 smithi033 ceph-mon[111306]: pgmap v526: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:56:58.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:56:58 smithi138 ceph-mon[81304]: pgmap v526: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:00.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:00 smithi033 ceph-mon[111306]: pgmap v527: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:00.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:00 smithi138 ceph-mon[81304]: pgmap v527: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:02.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:02 smithi033 ceph-mon[111306]: pgmap v528: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:02.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:02 smithi138 ceph-mon[81304]: pgmap v528: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:04.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:04 smithi033 ceph-mon[111306]: pgmap v529: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:04.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:04 smithi138 ceph-mon[81304]: pgmap v529: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:06.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:06 smithi033 ceph-mon[111306]: pgmap v530: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:06.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:06 smithi138 ceph-mon[81304]: pgmap v530: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:08.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:08 smithi033 ceph-mon[111306]: pgmap v531: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:08.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:08 smithi138 ceph-mon[81304]: pgmap v531: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:10.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:10 smithi033 ceph-mon[111306]: pgmap v532: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:10.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:10 smithi138 ceph-mon[81304]: pgmap v532: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:11.503 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:57:11.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:57:12.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:12 smithi033 ceph-mon[111306]: pgmap v533: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:12.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:12 smithi138 ceph-mon[81304]: pgmap v533: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:14.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:14 smithi033 ceph-mon[111306]: pgmap v534: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:14.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:14 smithi138 ceph-mon[81304]: pgmap v534: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:15.305 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 -- 172.21.15.33:0/2883009415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c103780 msgr2=0x7f342c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 --2- 172.21.15.33:0/2883009415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c103780 0x7f342c0fcdf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3428004940 tx=0x7f3428009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 -- 172.21.15.33:0/2883009415 shutdown_connections 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 --2- 172.21.15.33:0/2883009415 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c103780 0x7f342c0fcdf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 --2- 172.21.15.33:0/2883009415 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f342c102de0 0x7f342c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 -- 172.21.15.33:0/2883009415 >> 172.21.15.33:0/2883009415 conn(0x7f342c0f8990 msgr2=0x7f342c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 -- 172.21.15.33:0/2883009415 shutdown_connections 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 -- 172.21.15.33:0/2883009415 wait complete. 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 Processor -- start 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.301+0000 7f3433cf4700 1 -- start start 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3433cf4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c102de0 0x7f342c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3433cf4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f342c103780 0x7f342c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:15.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3433cf4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c0721b0 con 0x7f342c102de0 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3433cf4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342c070ce0 con 0x7f342c103780 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3431a90700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c102de0 0x7f342c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3431a90700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c102de0 0x7f342c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60366/0 (socket says 172.21.15.33:60366) 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3431a90700 1 -- 172.21.15.33:0/1241910051 learned_addr learned my addr 172.21.15.33:0/1241910051 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f343128f700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f342c103780 0x7f342c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3431a90700 1 -- 172.21.15.33:0/1241910051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f342c103780 msgr2=0x7f342c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3431a90700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f342c103780 0x7f342c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3431a90700 1 -- 172.21.15.33:0/1241910051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3428005040 con 0x7f342c102de0 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f343128f700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f342c103780 0x7f342c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.302+0000 7f3431a90700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c102de0 0x7f342c070260 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f341c002a50 tx=0x7f341c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.303+0000 7f34227fc700 1 -- 172.21.15.33:0/1241910051 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f341c007cc0 con 0x7f342c102de0 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.303+0000 7f34227fc700 1 -- 172.21.15.33:0/1241910051 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f341c00f040 con 0x7f342c102de0 2024-04-25T01:57:15.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.303+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f342c070fc0 con 0x7f342c102de0 2024-04-25T01:57:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.303+0000 7f34227fc700 1 -- 172.21.15.33:0/1241910051 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f341c009640 con 0x7f342c102de0 2024-04-25T01:57:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.303+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f342c11a100 con 0x7f342c102de0 2024-04-25T01:57:15.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.304+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342c0fe530 con 0x7f342c102de0 2024-04-25T01:57:15.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.304+0000 7f34227fc700 1 -- 172.21.15.33:0/1241910051 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f341c0097a0 con 0x7f342c102de0 2024-04-25T01:57:15.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.305+0000 7f34227fc700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f341806e140 0x7f34180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:15.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.305+0000 7f34227fc700 1 -- 172.21.15.33:0/1241910051 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f341c08d680 con 0x7f342c102de0 2024-04-25T01:57:15.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.305+0000 7f343128f700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f341806e140 0x7f34180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:15.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.305+0000 7f343128f700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f341806e140 0x7f34180705f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3428000c00 tx=0x7f3428009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:15.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.310+0000 7f34227fc700 1 -- 172.21.15.33:0/1241910051 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f341c056a00 con 0x7f342c102de0 2024-04-25T01:57:15.540 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.537+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f342c0fa0a0 con 0x7f341806e140 2024-04-25T01:57:15.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.539+0000 7f34227fc700 1 -- 172.21.15.33:0/1241910051 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f342c0fa0a0 con 0x7f341806e140 2024-04-25T01:57:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f341806e140 msgr2=0x7f34180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f341806e140 0x7f34180705f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3428000c00 tx=0x7f3428009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:15.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c102de0 msgr2=0x7f342c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c102de0 0x7f342c070260 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f341c002a50 tx=0x7f341c0046b0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 shutdown_connections 2024-04-25T01:57:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f341806e140 0x7f34180705f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f342c102de0 0x7f342c070260 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 --2- 172.21.15.33:0/1241910051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f342c103780 0x7f342c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 >> 172.21.15.33:0/1241910051 conn(0x7f342c0f8990 msgr2=0x7f342c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 shutdown_connections 2024-04-25T01:57:15.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.541+0000 7f3433cf4700 1 -- 172.21.15.33:0/1241910051 wait complete. 2024-04-25T01:57:15.554 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.646+0000 7f6918ad3700 1 -- 172.21.15.33:0/3196290442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69141026a0 msgr2=0x7f69140fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.646+0000 7f6918ad3700 1 --2- 172.21.15.33:0/3196290442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69141026a0 0x7f69140fbd10 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f68fc004940 tx=0x7f68fc009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.646+0000 7f6918ad3700 1 -- 172.21.15.33:0/3196290442 shutdown_connections 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.646+0000 7f6918ad3700 1 --2- 172.21.15.33:0/3196290442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f69141026a0 0x7f69140fbd10 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.646+0000 7f6918ad3700 1 --2- 172.21.15.33:0/3196290442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6914101d00 0x7f69141020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.646+0000 7f6918ad3700 1 -- 172.21.15.33:0/3196290442 >> 172.21.15.33:0/3196290442 conn(0x7f69140f7870 msgr2=0x7f69140f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.646+0000 7f6918ad3700 1 -- 172.21.15.33:0/3196290442 shutdown_connections 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.646+0000 7f6918ad3700 1 -- 172.21.15.33:0/3196290442 wait complete. 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f6918ad3700 1 Processor -- start 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f6918ad3700 1 -- start start 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f6918ad3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6914101d00 0x7f69140fd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f6918ad3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69141026a0 0x7f69140fdec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f6918ad3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140ff8d0 con 0x7f6914101d00 2024-04-25T01:57:15.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f6918ad3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69140fe400 con 0x7f69141026a0 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f691259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6914101d00 0x7f69140fd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f691259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6914101d00 0x7f69140fd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55670/0 (socket says 172.21.15.33:55670) 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f691259c700 1 -- 172.21.15.33:0/4176118301 learned_addr learned my addr 172.21.15.33:0/4176118301 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f6911d9b700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69141026a0 0x7f69140fdec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f691259c700 1 -- 172.21.15.33:0/4176118301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69141026a0 msgr2=0x7f69140fdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f691259c700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69141026a0 0x7f69140fdec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f691259c700 1 -- 172.21.15.33:0/4176118301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc005040 con 0x7f6914101d00 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.647+0000 7f6911d9b700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69141026a0 0x7f69140fdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.648+0000 7f691259c700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6914101d00 0x7f69140fd980 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f690400cb10 tx=0x7f6904007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.648+0000 7f690affd700 1 -- 172.21.15.33:0/4176118301 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6904004200 con 0x7f6914101d00 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.648+0000 7f690affd700 1 -- 172.21.15.33:0/4176118301 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f690400acc0 con 0x7f6914101d00 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.648+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69140fe6e0 con 0x7f6914101d00 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.648+0000 7f690affd700 1 -- 172.21.15.33:0/4176118301 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6904003d90 con 0x7f6914101d00 2024-04-25T01:57:15.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.648+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691411a150 con 0x7f6914101d00 2024-04-25T01:57:15.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.649+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69140fd450 con 0x7f6914101d00 2024-04-25T01:57:15.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.649+0000 7f690affd700 1 -- 172.21.15.33:0/4176118301 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6904003430 con 0x7f6914101d00 2024-04-25T01:57:15.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.650+0000 7f690affd700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f690006e140 0x7f69000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:15.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.650+0000 7f690affd700 1 -- 172.21.15.33:0/4176118301 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f690408e0d0 con 0x7f6914101d00 2024-04-25T01:57:15.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.650+0000 7f6911d9b700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f690006e140 0x7f69000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:15.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.650+0000 7f6911d9b700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f690006e140 0x7f69000705f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009fd0 tx=0x7f68fc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:15.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.654+0000 7f690affd700 1 -- 172.21.15.33:0/4176118301 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6904057450 con 0x7f6914101d00 2024-04-25T01:57:15.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.878+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f691404c7b0 con 0x7f690006e140 2024-04-25T01:57:15.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.880+0000 7f690affd700 1 -- 172.21.15.33:0/4176118301 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f691404c7b0 con 0x7f690006e140 2024-04-25T01:57:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.881+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f690006e140 msgr2=0x7f69000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.881+0000 7f6918ad3700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f690006e140 0x7f69000705f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009fd0 tx=0x7f68fc009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.882+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6914101d00 msgr2=0x7f69140fd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.882+0000 7f6918ad3700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6914101d00 0x7f69140fd980 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f690400cb10 tx=0x7f6904007f70 comp rx=0 tx=0).stop 2024-04-25T01:57:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.882+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 shutdown_connections 2024-04-25T01:57:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.882+0000 7f6918ad3700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f690006e140 0x7f69000705f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.882+0000 7f6918ad3700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6914101d00 0x7f69140fd980 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.882+0000 7f6918ad3700 1 --2- 172.21.15.33:0/4176118301 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f69141026a0 0x7f69140fdec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.882+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 >> 172.21.15.33:0/4176118301 conn(0x7f69140f7870 msgr2=0x7f69140f8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.882+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 shutdown_connections 2024-04-25T01:57:15.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.882+0000 7f6918ad3700 1 -- 172.21.15.33:0/4176118301 wait complete. 2024-04-25T01:57:15.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.985+0000 7f9457de0700 1 -- 172.21.15.33:0/2951250624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 msgr2=0x7f9450102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.985+0000 7f9457de0700 1 --2- 172.21.15.33:0/2951250624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 0x7f9450102160 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f9440004940 tx=0x7f9440009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.985+0000 7f9457de0700 1 -- 172.21.15.33:0/2951250624 shutdown_connections 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.985+0000 7f9457de0700 1 --2- 172.21.15.33:0/2951250624 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94500fbd40 0x7f94500fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.985+0000 7f9457de0700 1 --2- 172.21.15.33:0/2951250624 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 0x7f9450102160 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.985+0000 7f9457de0700 1 -- 172.21.15.33:0/2951250624 >> 172.21.15.33:0/2951250624 conn(0x7f9450071d00 msgr2=0x7f9450072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.985+0000 7f9457de0700 1 -- 172.21.15.33:0/2951250624 shutdown_connections 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.985+0000 7f9457de0700 1 -- 172.21.15.33:0/2951250624 wait complete. 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f9457de0700 1 Processor -- start 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f9457de0700 1 -- start start 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f9457de0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94500fbd40 0x7f945010eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f9457de0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 0x7f945010f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f9457de0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945010f6d0 con 0x7f9450101d90 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f9457de0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945010f810 con 0x7f94500fbd40 2024-04-25T01:57:15.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f945537b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 0x7f945010f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f945537b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 0x7f945010f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55676/0 (socket says 172.21.15.33:55676) 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f945537b700 1 -- 172.21.15.33:0/3570872604 learned_addr learned my addr 172.21.15.33:0/3570872604 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f9455b7c700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94500fbd40 0x7f945010eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f945537b700 1 -- 172.21.15.33:0/3570872604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94500fbd40 msgr2=0x7f945010eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f945537b700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94500fbd40 0x7f945010eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.986+0000 7f945537b700 1 -- 172.21.15.33:0/3570872604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440005040 con 0x7f9450101d90 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.987+0000 7f945537b700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 0x7f945010f080 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c830 tx=0x7f944c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.987+0000 7f94467fc700 1 -- 172.21.15.33:0/3570872604 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f944c00a760 con 0x7f9450101d90 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.987+0000 7f94467fc700 1 -- 172.21.15.33:0/3570872604 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f944c00ad90 con 0x7f9450101d90 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.987+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9450117bb0 con 0x7f9450101d90 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.987+0000 7f94467fc700 1 -- 172.21.15.33:0/3570872604 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f944c0113a0 con 0x7f9450101d90 2024-04-25T01:57:15.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.987+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9450118110 con 0x7f9450101d90 2024-04-25T01:57:15.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.988+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94500fd430 con 0x7f9450101d90 2024-04-25T01:57:15.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.988+0000 7f94467fc700 1 -- 172.21.15.33:0/3570872604 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f944c012400 con 0x7f9450101d90 2024-04-25T01:57:15.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.989+0000 7f94467fc700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f943c06e140 0x7f943c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:15.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.989+0000 7f94467fc700 1 -- 172.21.15.33:0/3570872604 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f944c08db40 con 0x7f9450101d90 2024-04-25T01:57:15.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.989+0000 7f9455b7c700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f943c06e140 0x7f943c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:15.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.989+0000 7f9455b7c700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f943c06e140 0x7f943c0705f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f944000b3b0 tx=0x7f9440009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:15.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:15.994+0000 7f94467fc700 1 -- 172.21.15.33:0/3570872604 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f944c056f40 con 0x7f9450101d90 2024-04-25T01:57:16.212 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.210+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f945004bed0 con 0x7f943c06e140 2024-04-25T01:57:16.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.220+0000 7f94467fc700 1 -- 172.21.15.33:0/3570872604 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f945004bed0 con 0x7f943c06e140 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (18m) 11m ago 29m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (15m) 11m ago 29m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (15m) 11m ago 28m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (18m) 11m ago 29m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (11m) 11m ago 24m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (11m) 11m ago 24m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (11m) 11m ago 24m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (24m) 11m ago 24m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (19m) 11m ago 30m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (19m) 11m ago 28m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:57:16.223 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (16m) 11m ago 31m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (16m) 11m ago 28m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (19m) 11m ago 29m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (19m) 11m ago 28m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (15m) 11m ago 27m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (14m) 11m ago 27m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (13m) 11m ago 26m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (13m) 11m ago 26m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (12m) 11m ago 26m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (12m) 11m ago 25m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:57:16.224 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (18m) 11m ago 29m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:57:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f943c06e140 msgr2=0x7f943c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f943c06e140 0x7f943c0705f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f944000b3b0 tx=0x7f9440009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 msgr2=0x7f945010f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 0x7f945010f080 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f944c00c830 tx=0x7f944c004470 comp rx=0 tx=0).stop 2024-04-25T01:57:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 shutdown_connections 2024-04-25T01:57:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f943c06e140 0x7f943c0705f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94500fbd40 0x7f945010eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 --2- 172.21.15.33:0/3570872604 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9450101d90 0x7f945010f080 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 >> 172.21.15.33:0/3570872604 conn(0x7f9450071d00 msgr2=0x7f94500fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:16.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 shutdown_connections 2024-04-25T01:57:16.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.222+0000 7f9457de0700 1 -- 172.21.15.33:0/3570872604 wait complete. 2024-04-25T01:57:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.325+0000 7f20dfcce700 1 -- 172.21.15.33:0/796297943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 msgr2=0x7f20d80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.325+0000 7f20dfcce700 1 --2- 172.21.15.33:0/796297943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 0x7f20d80fc1b0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f20cc004940 tx=0x7f20cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.325+0000 7f20dfcce700 1 -- 172.21.15.33:0/796297943 shutdown_connections 2024-04-25T01:57:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.325+0000 7f20dfcce700 1 --2- 172.21.15.33:0/796297943 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 0x7f20d80fc1b0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.325+0000 7f20dfcce700 1 --2- 172.21.15.33:0/796297943 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d8101d90 0x7f20d8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.325+0000 7f20dfcce700 1 -- 172.21.15.33:0/796297943 >> 172.21.15.33:0/796297943 conn(0x7f20d8071d00 msgr2=0x7f20d8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.325+0000 7f20dfcce700 1 -- 172.21.15.33:0/796297943 shutdown_connections 2024-04-25T01:57:16.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.325+0000 7f20dfcce700 1 -- 172.21.15.33:0/796297943 wait complete. 2024-04-25T01:57:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dfcce700 1 Processor -- start 2024-04-25T01:57:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dfcce700 1 -- start start 2024-04-25T01:57:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dfcce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 0x7f20d8102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dfcce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d8101d90 0x7f20d8103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dfcce700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d81035f0 con 0x7f20d80fbd40 2024-04-25T01:57:16.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dfcce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d8103730 con 0x7f20d8101d90 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dda6a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 0x7f20d8102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dda6a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 0x7f20d8102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55690/0 (socket says 172.21.15.33:55690) 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dda6a700 1 -- 172.21.15.33:0/1978757189 learned_addr learned my addr 172.21.15.33:0/1978757189 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.326+0000 7f20dd269700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d8101d90 0x7f20d8103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.327+0000 7f20dda6a700 1 -- 172.21.15.33:0/1978757189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d8101d90 msgr2=0x7f20d8103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.327+0000 7f20dda6a700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d8101d90 0x7f20d8103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.327+0000 7f20dda6a700 1 -- 172.21.15.33:0/1978757189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20cc005040 con 0x7f20d80fbd40 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.327+0000 7f20dda6a700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 0x7f20d8102b20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f20d0002a50 tx=0x7f20d00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.327+0000 7f20d67fc700 1 -- 172.21.15.33:0/1978757189 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d0007cc0 con 0x7f20d80fbd40 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.327+0000 7f20d67fc700 1 -- 172.21.15.33:0/1978757189 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20d000f040 con 0x7f20d80fbd40 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.327+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d8119eb0 con 0x7f20d80fbd40 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.327+0000 7f20d67fc700 1 -- 172.21.15.33:0/1978757189 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d000da30 con 0x7f20d80fbd40 2024-04-25T01:57:16.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.327+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d811a330 con 0x7f20d80fbd40 2024-04-25T01:57:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.328+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d80fd430 con 0x7f20d80fbd40 2024-04-25T01:57:16.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.329+0000 7f20d67fc700 1 -- 172.21.15.33:0/1978757189 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f20d000db90 con 0x7f20d80fbd40 2024-04-25T01:57:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.329+0000 7f20d67fc700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20c006e140 0x7f20c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.329+0000 7f20d67fc700 1 -- 172.21.15.33:0/1978757189 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f20d008d8d0 con 0x7f20d80fbd40 2024-04-25T01:57:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.329+0000 7f20dd269700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20c006e140 0x7f20c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:16.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.330+0000 7f20dd269700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20c006e140 0x7f20c00705f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20d8104460 tx=0x7f20cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:16.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.334+0000 7f20d67fc700 1 -- 172.21.15.33:0/1978757189 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f20d0056c50 con 0x7f20d80fbd40 2024-04-25T01:57:16.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:16 smithi033 ceph-mon[111306]: pgmap v535: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:16.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:16 smithi138 ceph-mon[81304]: pgmap v535: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.634+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f20d805f5c0 con 0x7f20d80fbd40 2024-04-25T01:57:16.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.635+0000 7f20d67fc700 1 -- 172.21.15.33:0/1978757189 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f20d005a310 con 0x7f20d80fbd40 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:57:16.638 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:57:16.639 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:57:16.639 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:57:16.639 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:57:16.639 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:57:16.639 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:57:16.639 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20c006e140 msgr2=0x7f20c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20c006e140 0x7f20c00705f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f20d8104460 tx=0x7f20cc009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 msgr2=0x7f20d8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 0x7f20d8102b20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f20d0002a50 tx=0x7f20d00046b0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 shutdown_connections 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20c006e140 0x7f20c00705f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20d80fbd40 0x7f20d8102b20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 --2- 172.21.15.33:0/1978757189 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f20d8101d90 0x7f20d8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 >> 172.21.15.33:0/1978757189 conn(0x7f20d8071d00 msgr2=0x7f20d80655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:16.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 shutdown_connections 2024-04-25T01:57:16.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.637+0000 7f20dfcce700 1 -- 172.21.15.33:0/1978757189 wait complete. 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/99626932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 msgr2=0x7fc5f00fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 --2- 172.21.15.33:0/99626932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 0x7fc5f00fb990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc004940 tx=0x7fc5dc009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/99626932 shutdown_connections 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 --2- 172.21.15.33:0/99626932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 0x7fc5f00fb990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 --2- 172.21.15.33:0/99626932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f0102f60 0x7fc5f0103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/99626932 >> 172.21.15.33:0/99626932 conn(0x7fc5f00714d0 msgr2=0x7fc5f00718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/99626932 shutdown_connections 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/99626932 wait complete. 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 Processor -- start 2024-04-25T01:57:16.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.741+0000 7fc5f6e1e700 1 -- start start 2024-04-25T01:57:16.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f6e1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 0x7fc5f0113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:16.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f6e1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f0102f60 0x7fc5f0113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:16.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f6e1e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f0114510 con 0x7fc5f00fb520 2024-04-25T01:57:16.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f6e1e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f010db30 con 0x7fc5f0102f60 2024-04-25T01:57:16.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f4bba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 0x7fc5f0113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f4bba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 0x7fc5f0113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55710/0 (socket says 172.21.15.33:55710) 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f4bba700 1 -- 172.21.15.33:0/3263167403 learned_addr learned my addr 172.21.15.33:0/3263167403 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5effff700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f0102f60 0x7fc5f0113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f4bba700 1 -- 172.21.15.33:0/3263167403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f0102f60 msgr2=0x7fc5f0113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f4bba700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f0102f60 0x7fc5f0113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f4bba700 1 -- 172.21.15.33:0/3263167403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5dc005040 con 0x7fc5f00fb520 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5effff700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f0102f60 0x7fc5f0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f4bba700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 0x7fc5f0113980 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c8e0 tx=0x7fc5e40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/3263167403 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e400a940 con 0x7fc5f00fb520 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/3263167403 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5e400ce80 con 0x7fc5f00fb520 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.742+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f010de10 con 0x7fc5f00fb520 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.743+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/3263167403 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5e40113a0 con 0x7fc5f00fb520 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.743+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f010e350 con 0x7fc5f00fb520 2024-04-25T01:57:16.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.744+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0100250 con 0x7fc5f00fb520 2024-04-25T01:57:16.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.744+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/3263167403 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc5e4012400 con 0x7fc5f00fb520 2024-04-25T01:57:16.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.744+0000 7fc5ed7fa700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc5e006e140 0x7fc5e00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:16.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.745+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/3263167403 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc5e408dc70 con 0x7fc5f00fb520 2024-04-25T01:57:16.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.745+0000 7fc5effff700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc5e006e140 0x7fc5e00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:16.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.745+0000 7fc5effff700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc5e006e140 0x7fc5e00705f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009fd0 tx=0x7fc5dc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:16.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:16.749+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/3263167403 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc5e4094080 con 0x7fc5f00fb520 2024-04-25T01:57:17.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.002+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc5f010a000 con 0x7fc5f00fb520 2024-04-25T01:57:17.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.003+0000 7fc5ed7fa700 1 -- 172.21.15.33:0/3263167403 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7fc5e4056fd0 con 0x7fc5f00fb520 2024-04-25T01:57:17.006 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:57:17.006 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:57:17.006 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:57:17.006 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:57:17.006 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:57:17.006 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:57:17.006 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:57:17.006 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:57:17.007 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:57:17.008 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:57:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:57:17.009 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:57:17.009 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:57:17.009 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:57:17.009 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:57:17.009 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:57:17.009 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc5e006e140 msgr2=0x7fc5e00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc5e006e140 0x7fc5e00705f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc5dc009fd0 tx=0x7fc5dc009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 msgr2=0x7fc5f0113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 0x7fc5f0113980 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fc5e400c8e0 tx=0x7fc5e40044c0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 shutdown_connections 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc5e006e140 0x7fc5e00705f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc5f00fb520 0x7fc5f0113980 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 --2- 172.21.15.33:0/3263167403 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc5f0102f60 0x7fc5f0113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 >> 172.21.15.33:0/3263167403 conn(0x7fc5f00714d0 msgr2=0x7fc5f00fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:17.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 shutdown_connections 2024-04-25T01:57:17.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.005+0000 7fc5f6e1e700 1 -- 172.21.15.33:0/3263167403 wait complete. 2024-04-25T01:57:17.011 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:57:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.110+0000 7f3096da6700 1 -- 172.21.15.33:0/809539552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 msgr2=0x7f30900fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.110+0000 7f3096da6700 1 --2- 172.21.15.33:0/809539552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 0x7f30900fe210 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3088009ce0 tx=0x7f3088004d10 comp rx=0 tx=0).stop 2024-04-25T01:57:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.110+0000 7f3096da6700 1 -- 172.21.15.33:0/809539552 shutdown_connections 2024-04-25T01:57:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.110+0000 7f3096da6700 1 --2- 172.21.15.33:0/809539552 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30900fe7e0 0x7f3090106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.110+0000 7f3096da6700 1 --2- 172.21.15.33:0/809539552 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 0x7f30900fe210 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.110+0000 7f3096da6700 1 -- 172.21.15.33:0/809539552 >> 172.21.15.33:0/809539552 conn(0x7f30900f7850 msgr2=0x7f30900f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.110+0000 7f3096da6700 1 -- 172.21.15.33:0/809539552 shutdown_connections 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.110+0000 7f3096da6700 1 -- 172.21.15.33:0/809539552 wait complete. 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3096da6700 1 Processor -- start 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3096da6700 1 -- start start 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3096da6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 0x7f30900fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3096da6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30900fe7e0 0x7f30900fc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3096da6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30900fc7d0 con 0x7f30900fde40 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3096da6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30900fc910 con 0x7f30900fe7e0 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3094b42700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 0x7f30900fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3094b42700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 0x7f30900fbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55726/0 (socket says 172.21.15.33:55726) 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3094b42700 1 -- 172.21.15.33:0/993274829 learned_addr learned my addr 172.21.15.33:0/993274829 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f308ffff700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30900fe7e0 0x7f30900fc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3094b42700 1 -- 172.21.15.33:0/993274829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30900fe7e0 msgr2=0x7f30900fc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3094b42700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30900fe7e0 0x7f30900fc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.111+0000 7f3094b42700 1 -- 172.21.15.33:0/993274829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3088009940 con 0x7f30900fde40 2024-04-25T01:57:17.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.112+0000 7f3094b42700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 0x7f30900fbc70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3088007c00 tx=0x7f3088004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.112+0000 7f308d7fa700 1 -- 172.21.15.33:0/993274829 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f308800e050 con 0x7f30900fde40 2024-04-25T01:57:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.112+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3090119e40 con 0x7f30900fde40 2024-04-25T01:57:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.112+0000 7f308d7fa700 1 -- 172.21.15.33:0/993274829 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f308801d910 con 0x7f30900fde40 2024-04-25T01:57:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.112+0000 7f308d7fa700 1 -- 172.21.15.33:0/993274829 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30880128c0 con 0x7f30900fde40 2024-04-25T01:57:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.112+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309011a2a0 con 0x7f30900fde40 2024-04-25T01:57:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.113+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30900666f0 con 0x7f30900fde40 2024-04-25T01:57:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.113+0000 7f308d7fa700 1 -- 172.21.15.33:0/993274829 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3088019040 con 0x7f30900fde40 2024-04-25T01:57:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.114+0000 7f308d7fa700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f307c06e150 0x7f307c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:17.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.114+0000 7f308d7fa700 1 -- 172.21.15.33:0/993274829 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f308808f6c0 con 0x7f30900fde40 2024-04-25T01:57:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.114+0000 7f308ffff700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f307c06e150 0x7f307c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:17.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.114+0000 7f308ffff700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f307c06e150 0x7f307c070600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f30900fd5b0 tx=0x7f30800091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:17.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.119+0000 7f308d7fa700 1 -- 172.21.15.33:0/993274829 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3088058a40 con 0x7f30900fde40 2024-04-25T01:57:17.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:17 smithi033 ceph-mon[111306]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:17.268 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:17 smithi033 ceph-mon[111306]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:17.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:17 smithi033 ceph-mon[111306]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:17.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:17 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1978757189' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:57:17.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:17 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3263167403' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:57:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.349+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3090109ef0 con 0x7f307c06e150 2024-04-25T01:57:17.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.350+0000 7f308d7fa700 1 -- 172.21.15.33:0/993274829 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3090109ef0 con 0x7f307c06e150 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:57:17.353 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:57:17.354 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:57:17.354 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:57:17.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f307c06e150 msgr2=0x7f307c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f307c06e150 0x7f307c070600 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f30900fd5b0 tx=0x7f30800091a0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 msgr2=0x7f30900fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 0x7f30900fbc70 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3088007c00 tx=0x7f3088004d10 comp rx=0 tx=0).stop 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 shutdown_connections 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f307c06e150 0x7f307c070600 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f30900fde40 0x7f30900fbc70 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 --2- 172.21.15.33:0/993274829 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30900fe7e0 0x7f30900fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 >> 172.21.15.33:0/993274829 conn(0x7f30900f7850 msgr2=0x7f30900f8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 shutdown_connections 2024-04-25T01:57:17.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.352+0000 7f3096da6700 1 -- 172.21.15.33:0/993274829 wait complete. 2024-04-25T01:57:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.456+0000 7fbee92c4700 1 -- 172.21.15.33:0/1526290471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4103780 msgr2=0x7fbee40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.456+0000 7fbee92c4700 1 --2- 172.21.15.33:0/1526290471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4103780 0x7fbee40fcdf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbecc004940 tx=0x7fbecc009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.456+0000 7fbee92c4700 1 -- 172.21.15.33:0/1526290471 shutdown_connections 2024-04-25T01:57:17.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.456+0000 7fbee92c4700 1 --2- 172.21.15.33:0/1526290471 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4103780 0x7fbee40fcdf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.456+0000 7fbee92c4700 1 --2- 172.21.15.33:0/1526290471 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee4102de0 0x7fbee41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.456+0000 7fbee92c4700 1 -- 172.21.15.33:0/1526290471 >> 172.21.15.33:0/1526290471 conn(0x7fbee40f8950 msgr2=0x7fbee40fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.456+0000 7fbee92c4700 1 -- 172.21.15.33:0/1526290471 shutdown_connections 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.456+0000 7fbee92c4700 1 -- 172.21.15.33:0/1526290471 wait complete. 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.456+0000 7fbee92c4700 1 Processor -- start 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee92c4700 1 -- start start 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee92c4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4102de0 0x7fbee4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee92c4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee4103780 0x7fbee40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee92c4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee4070d30 con 0x7fbee4102de0 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee92c4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee4070e70 con 0x7fbee4103780 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4102de0 0x7fbee4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4102de0 0x7fbee4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55752/0 (socket says 172.21.15.33:55752) 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee2ffd700 1 -- 172.21.15.33:0/1174911490 learned_addr learned my addr 172.21.15.33:0/1174911490 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:17.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee27fc700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee4103780 0x7fbee40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee2ffd700 1 -- 172.21.15.33:0/1174911490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee4103780 msgr2=0x7fbee40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee2ffd700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee4103780 0x7fbee40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee2ffd700 1 -- 172.21.15.33:0/1174911490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc005040 con 0x7fbee4102de0 2024-04-25T01:57:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbee2ffd700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4102de0 0x7fbee4070260 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00cb10 tx=0x7fbedc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.457+0000 7fbedb7fe700 1 -- 172.21.15.33:0/1174911490 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbedc004200 con 0x7fbee4102de0 2024-04-25T01:57:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.458+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee4119ea0 con 0x7fbee4102de0 2024-04-25T01:57:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.458+0000 7fbedb7fe700 1 -- 172.21.15.33:0/1174911490 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbedc00acc0 con 0x7fbee4102de0 2024-04-25T01:57:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.458+0000 7fbedb7fe700 1 -- 172.21.15.33:0/1174911490 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbedc003d90 con 0x7fbee4102de0 2024-04-25T01:57:17.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.458+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee411a320 con 0x7fbee4102de0 2024-04-25T01:57:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.459+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee40fe530 con 0x7fbee4102de0 2024-04-25T01:57:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.459+0000 7fbedb7fe700 1 -- 172.21.15.33:0/1174911490 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbedc003430 con 0x7fbee4102de0 2024-04-25T01:57:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.460+0000 7fbedb7fe700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbed006e150 0x7fbed0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.460+0000 7fbedb7fe700 1 -- 172.21.15.33:0/1174911490 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbedc08e0b0 con 0x7fbee4102de0 2024-04-25T01:57:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.460+0000 7fbee27fc700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbed006e150 0x7fbed0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:17.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.460+0000 7fbee27fc700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbed006e150 0x7fbed0070600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbee4071ba0 tx=0x7fbecc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:17.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.464+0000 7fbedb7fe700 1 -- 172.21.15.33:0/1174911490 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbedc057430 con 0x7fbee4102de0 2024-04-25T01:57:17.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:17 smithi138 ceph-mon[81304]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:17.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:17 smithi138 ceph-mon[81304]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:17.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:17 smithi138 ceph-mon[81304]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:17.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:17 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1978757189' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:57:17.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:17 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3263167403' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:57:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.756+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbee405f5c0 con 0x7fbee4102de0 2024-04-25T01:57:17.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.756+0000 7fbedb7fe700 1 -- 172.21.15.33:0/1174911490 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fbedc05aaf0 con 0x7fbee4102de0 2024-04-25T01:57:17.759 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:57:17.759 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:57:17.759 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:57:17.759 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:57:17.759 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:57:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.758+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbed006e150 msgr2=0x7fbed0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.758+0000 7fbee92c4700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbed006e150 0x7fbed0070600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fbee4071ba0 tx=0x7fbecc009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.758+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4102de0 msgr2=0x7fbee4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.758+0000 7fbee92c4700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4102de0 0x7fbee4070260 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00cb10 tx=0x7fbedc007f90 comp rx=0 tx=0).stop 2024-04-25T01:57:17.761 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.758+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 shutdown_connections 2024-04-25T01:57:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.758+0000 7fbee92c4700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbed006e150 0x7fbed0070600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.758+0000 7fbee92c4700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbee4102de0 0x7fbee4070260 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.758+0000 7fbee92c4700 1 --2- 172.21.15.33:0/1174911490 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbee4103780 0x7fbee40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.758+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 >> 172.21.15.33:0/1174911490 conn(0x7fbee40f8950 msgr2=0x7fbee40f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.759+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 shutdown_connections 2024-04-25T01:57:17.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:17.759+0000 7fbee92c4700 1 -- 172.21.15.33:0/1174911490 wait complete. 2024-04-25T01:57:18.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:18 smithi033 ceph-mon[111306]: pgmap v536: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:18.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:18 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1174911490' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:57:18.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:18 smithi138 ceph-mon[81304]: pgmap v536: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:18.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:18 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1174911490' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:57:19.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:19 smithi033 ceph-mon[111306]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:19.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:19 smithi138 ceph-mon[81304]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:20.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:20 smithi033 ceph-mon[111306]: pgmap v537: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:20.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:20 smithi138 ceph-mon[81304]: pgmap v537: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:22.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:22 smithi033 ceph-mon[111306]: pgmap v538: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:22.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:22 smithi138 ceph-mon[81304]: pgmap v538: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:24.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:24 smithi033 ceph-mon[111306]: pgmap v539: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:24.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:24 smithi138 ceph-mon[81304]: pgmap v539: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:26.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:26 smithi033 ceph-mon[111306]: pgmap v540: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:26.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:57:26.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:26 smithi138 ceph-mon[81304]: pgmap v540: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:26.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:57:28.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:28 smithi033 ceph-mon[111306]: pgmap v541: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:28.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:28 smithi138 ceph-mon[81304]: pgmap v541: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:30.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:30 smithi033 ceph-mon[111306]: pgmap v542: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:30.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:30 smithi138 ceph-mon[81304]: pgmap v542: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:32.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:32 smithi033 ceph-mon[111306]: pgmap v543: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:32.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:32 smithi138 ceph-mon[81304]: pgmap v543: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:34.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:34 smithi033 ceph-mon[111306]: pgmap v544: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:34.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:34 smithi138 ceph-mon[81304]: pgmap v544: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:36.518 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:36 smithi033 ceph-mon[111306]: pgmap v545: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:36.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:36 smithi138 ceph-mon[81304]: pgmap v545: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:38.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:38 smithi033 ceph-mon[111306]: pgmap v546: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:38.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:38 smithi138 ceph-mon[81304]: pgmap v546: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:40.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:40 smithi033 ceph-mon[111306]: pgmap v547: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:40.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:40 smithi138 ceph-mon[81304]: pgmap v547: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:41.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:57:41.543 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:57:42.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:42 smithi033 ceph-mon[111306]: pgmap v548: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:42.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:42 smithi138 ceph-mon[81304]: pgmap v548: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:44.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:44 smithi033 ceph-mon[111306]: pgmap v549: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:44.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:44 smithi138 ceph-mon[81304]: pgmap v549: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:46.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:46 smithi033 ceph-mon[111306]: pgmap v550: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:46.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:46 smithi138 ceph-mon[81304]: pgmap v550: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:47.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.876+0000 7f4858630700 1 -- 172.21.15.33:0/321752089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 msgr2=0x7f4850103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:47.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.876+0000 7f4858630700 1 --2- 172.21.15.33:0/321752089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 0x7f4850103190 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f484c004940 tx=0x7f484c009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:47.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.876+0000 7f4858630700 1 -- 172.21.15.33:0/321752089 shutdown_connections 2024-04-25T01:57:47.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.876+0000 7f4858630700 1 --2- 172.21.15.33:0/321752089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4850103760 0x7f48500fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:47.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.876+0000 7f4858630700 1 --2- 172.21.15.33:0/321752089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 0x7f4850103190 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.876+0000 7f4858630700 1 -- 172.21.15.33:0/321752089 >> 172.21.15.33:0/321752089 conn(0x7f48500f8930 msgr2=0x7f48500fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.877+0000 7f4858630700 1 -- 172.21.15.33:0/321752089 shutdown_connections 2024-04-25T01:57:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.877+0000 7f4858630700 1 -- 172.21.15.33:0/321752089 wait complete. 2024-04-25T01:57:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.877+0000 7f4858630700 1 Processor -- start 2024-04-25T01:57:47.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.877+0000 7f4858630700 1 -- start start 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f4858630700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 0x7f4850070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f4858630700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4850103760 0x7f48500707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f4858630700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500721b0 con 0x7f4850102dc0 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f4858630700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4850070ce0 con 0x7f4850103760 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f48563cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 0x7f4850070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f48563cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 0x7f4850070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46048/0 (socket says 172.21.15.33:46048) 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f48563cc700 1 -- 172.21.15.33:0/277848052 learned_addr learned my addr 172.21.15.33:0/277848052 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f4855bcb700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4850103760 0x7f48500707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f48563cc700 1 -- 172.21.15.33:0/277848052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4850103760 msgr2=0x7f48500707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f48563cc700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4850103760 0x7f48500707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f48563cc700 1 -- 172.21.15.33:0/277848052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c005040 con 0x7f4850102dc0 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f48563cc700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 0x7f4850070260 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b3b0 tx=0x7f484c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f4846ffd700 1 -- 172.21.15.33:0/277848052 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484c00e040 con 0x7f4850102dc0 2024-04-25T01:57:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4850070f60 con 0x7f4850102dc0 2024-04-25T01:57:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f4846ffd700 1 -- 172.21.15.33:0/277848052 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f484c00bbc0 con 0x7f4850102dc0 2024-04-25T01:57:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.878+0000 7f4846ffd700 1 -- 172.21.15.33:0/277848052 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f484c01c3f0 con 0x7f4850102dc0 2024-04-25T01:57:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.879+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485011a0b0 con 0x7f4850102dc0 2024-04-25T01:57:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.880+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48500fe510 con 0x7f4850102dc0 2024-04-25T01:57:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.880+0000 7f4846ffd700 1 -- 172.21.15.33:0/277848052 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f484c00bd30 con 0x7f4850102dc0 2024-04-25T01:57:47.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.881+0000 7f4846ffd700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f483c06e140 0x7f483c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:47.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.881+0000 7f4846ffd700 1 -- 172.21.15.33:0/277848052 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f484c08ed70 con 0x7f4850102dc0 2024-04-25T01:57:47.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.881+0000 7f4855bcb700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f483c06e140 0x7f483c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:47.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.881+0000 7f4855bcb700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f483c06e140 0x7f483c0705f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4850071990 tx=0x7f4840009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:47.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:47.886+0000 7f4846ffd700 1 -- 172.21.15.33:0/277848052 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f484c0580f0 con 0x7f4850102dc0 2024-04-25T01:57:48.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.113+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f48500f9fd0 con 0x7f483c06e140 2024-04-25T01:57:48.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.114+0000 7f4846ffd700 1 -- 172.21.15.33:0/277848052 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f48500f9fd0 con 0x7f483c06e140 2024-04-25T01:57:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f483c06e140 msgr2=0x7f483c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f483c06e140 0x7f483c0705f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4850071990 tx=0x7f4840009ee0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 msgr2=0x7f4850070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 0x7f4850070260 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f484c00b3b0 tx=0x7f484c009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 shutdown_connections 2024-04-25T01:57:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f483c06e140 0x7f483c0705f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4850102dc0 0x7f4850070260 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 --2- 172.21.15.33:0/277848052 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4850103760 0x7f48500707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 >> 172.21.15.33:0/277848052 conn(0x7f48500f8930 msgr2=0x7f48500f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:48.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 shutdown_connections 2024-04-25T01:57:48.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.116+0000 7f4858630700 1 -- 172.21.15.33:0/277848052 wait complete. 2024-04-25T01:57:48.129 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:57:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.222+0000 7faa9299c700 1 -- 172.21.15.33:0/1787705664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c065640 msgr2=0x7faa8c065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.222+0000 7faa9299c700 1 --2- 172.21.15.33:0/1787705664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c065640 0x7faa8c065ab0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7faa80004940 tx=0x7faa80009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:48.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.222+0000 7faa9299c700 1 -- 172.21.15.33:0/1787705664 shutdown_connections 2024-04-25T01:57:48.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.222+0000 7faa9299c700 1 --2- 172.21.15.33:0/1787705664 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c065640 0x7faa8c065ab0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.222+0000 7faa9299c700 1 --2- 172.21.15.33:0/1787705664 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c102e50 0x7faa8c103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.222+0000 7faa9299c700 1 -- 172.21.15.33:0/1787705664 >> 172.21.15.33:0/1787705664 conn(0x7faa8c071d00 msgr2=0x7faa8c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:48.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.222+0000 7faa9299c700 1 -- 172.21.15.33:0/1787705664 shutdown_connections 2024-04-25T01:57:48.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.222+0000 7faa9299c700 1 -- 172.21.15.33:0/1787705664 wait complete. 2024-04-25T01:57:48.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.222+0000 7faa9299c700 1 Processor -- start 2024-04-25T01:57:48.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa9299c700 1 -- start start 2024-04-25T01:57:48.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa9299c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c065640 0x7faa8c113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa9299c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c102e50 0x7faa8c1140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa9299c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c114780 con 0x7faa8c102e50 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa9299c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c10dae0 con 0x7faa8c065640 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c102e50 0x7faa8c1140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c102e50 0x7faa8c1140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46066/0 (socket says 172.21.15.33:46066) 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa8b7fe700 1 -- 172.21.15.33:0/1461475566 learned_addr learned my addr 172.21.15.33:0/1461475566 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa8bfff700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c065640 0x7faa8c113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa8b7fe700 1 -- 172.21.15.33:0/1461475566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c065640 msgr2=0x7faa8c113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa8b7fe700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c065640 0x7faa8c113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.223+0000 7faa8b7fe700 1 -- 172.21.15.33:0/1461475566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa80005040 con 0x7faa8c102e50 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.224+0000 7faa8b7fe700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c102e50 0x7faa8c1140a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faa8000b3b0 tx=0x7faa80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.224+0000 7faa88ff9700 1 -- 172.21.15.33:0/1461475566 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa8000e040 con 0x7faa8c102e50 2024-04-25T01:57:48.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.224+0000 7faa88ff9700 1 -- 172.21.15.33:0/1461475566 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa80013420 con 0x7faa8c102e50 2024-04-25T01:57:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.224+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c10dd60 con 0x7faa8c102e50 2024-04-25T01:57:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.224+0000 7faa88ff9700 1 -- 172.21.15.33:0/1461475566 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa8001c630 con 0x7faa8c102e50 2024-04-25T01:57:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.224+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c10e320 con 0x7faa8c102e50 2024-04-25T01:57:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.225+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c0fe540 con 0x7faa8c102e50 2024-04-25T01:57:48.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.225+0000 7faa88ff9700 1 -- 172.21.15.33:0/1461475566 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faa80013590 con 0x7faa8c102e50 2024-04-25T01:57:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.226+0000 7faa88ff9700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa7406e140 0x7faa740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.226+0000 7faa88ff9700 1 -- 172.21.15.33:0/1461475566 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7faa8008ec30 con 0x7faa8c102e50 2024-04-25T01:57:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.226+0000 7faa8bfff700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa7406e140 0x7faa740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:48.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.226+0000 7faa8bfff700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa7406e140 0x7faa740705f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009bf0 tx=0x7faa7c0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:48.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.231+0000 7faa88ff9700 1 -- 172.21.15.33:0/1461475566 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faa8005bc00 con 0x7faa8c102e50 2024-04-25T01:57:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.461+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa8c109f70 con 0x7faa7406e140 2024-04-25T01:57:48.465 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:48 smithi033 ceph-mon[111306]: pgmap v551: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:48.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.463+0000 7faa88ff9700 1 -- 172.21.15.33:0/1461475566 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faa8c109f70 con 0x7faa7406e140 2024-04-25T01:57:48.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa7406e140 msgr2=0x7faa740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa7406e140 0x7faa740705f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7faa7c009bf0 tx=0x7faa7c0091a0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c102e50 msgr2=0x7faa8c1140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c102e50 0x7faa8c1140a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7faa8000b3b0 tx=0x7faa80009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 shutdown_connections 2024-04-25T01:57:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa7406e140 0x7faa740705f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa8c065640 0x7faa8c113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 --2- 172.21.15.33:0/1461475566 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa8c102e50 0x7faa8c1140a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 >> 172.21.15.33:0/1461475566 conn(0x7faa8c071d00 msgr2=0x7faa8c0f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 shutdown_connections 2024-04-25T01:57:48.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.465+0000 7faa9299c700 1 -- 172.21.15.33:0/1461475566 wait complete. 2024-04-25T01:57:48.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:48 smithi138 ceph-mon[81304]: pgmap v551: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.569+0000 7f6d5aa77700 1 -- 172.21.15.33:0/417231402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 msgr2=0x7f6d540fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.569+0000 7f6d5aa77700 1 --2- 172.21.15.33:0/417231402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 0x7f6d540fbd20 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6d40004940 tx=0x7f6d40009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:48.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.569+0000 7f6d5aa77700 1 -- 172.21.15.33:0/417231402 shutdown_connections 2024-04-25T01:57:48.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.569+0000 7f6d5aa77700 1 --2- 172.21.15.33:0/417231402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 0x7f6d540fbd20 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.569+0000 7f6d5aa77700 1 --2- 172.21.15.33:0/417231402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d54101d10 0x7f6d541020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.569+0000 7f6d5aa77700 1 -- 172.21.15.33:0/417231402 >> 172.21.15.33:0/417231402 conn(0x7f6d540f7860 msgr2=0x7f6d540f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:48.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.569+0000 7f6d5aa77700 1 -- 172.21.15.33:0/417231402 shutdown_connections 2024-04-25T01:57:48.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.569+0000 7f6d5aa77700 1 -- 172.21.15.33:0/417231402 wait complete. 2024-04-25T01:57:48.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.570+0000 7f6d5aa77700 1 Processor -- start 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.570+0000 7f6d5aa77700 1 -- start start 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.570+0000 7f6d5aa77700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d54101d10 0x7f6d541128a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.570+0000 7f6d5aa77700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 0x7f6d54112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.570+0000 7f6d5aa77700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d54113430 con 0x7f6d541026b0 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.570+0000 7f6d5aa77700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5410ca30 con 0x7f6d54101d10 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d53fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 0x7f6d54112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d53fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 0x7f6d54112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46084/0 (socket says 172.21.15.33:46084) 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d53fff700 1 -- 172.21.15.33:0/3064211748 learned_addr learned my addr 172.21.15.33:0/3064211748 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d58813700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d54101d10 0x7f6d541128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d53fff700 1 -- 172.21.15.33:0/3064211748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d54101d10 msgr2=0x7f6d541128a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d53fff700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d54101d10 0x7f6d541128a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d53fff700 1 -- 172.21.15.33:0/3064211748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40005040 con 0x7f6d541026b0 2024-04-25T01:57:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d53fff700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 0x7f6d54112de0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009100 tx=0x7f6d40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d517fa700 1 -- 172.21.15.33:0/3064211748 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d4000e040 con 0x7f6d541026b0 2024-04-25T01:57:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5410ccb0 con 0x7f6d541026b0 2024-04-25T01:57:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d517fa700 1 -- 172.21.15.33:0/3064211748 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d40009af0 con 0x7f6d541026b0 2024-04-25T01:57:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d517fa700 1 -- 172.21.15.33:0/3064211748 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d4001c3f0 con 0x7f6d541026b0 2024-04-25T01:57:48.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.571+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5410d270 con 0x7f6d541026b0 2024-04-25T01:57:48.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.572+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d540fd460 con 0x7f6d541026b0 2024-04-25T01:57:48.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.573+0000 7f6d517fa700 1 -- 172.21.15.33:0/3064211748 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6d4000baf0 con 0x7f6d541026b0 2024-04-25T01:57:48.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.573+0000 7f6d517fa700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6d4406e140 0x7f6d440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:48.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.574+0000 7f6d517fa700 1 -- 172.21.15.33:0/3064211748 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6d4008ed70 con 0x7f6d541026b0 2024-04-25T01:57:48.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.574+0000 7f6d58813700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6d4406e140 0x7f6d440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:48.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.574+0000 7f6d58813700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6d4406e140 0x7f6d440705f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6d48004eb0 tx=0x7f6d4800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:48.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.578+0000 7f6d517fa700 1 -- 172.21.15.33:0/3064211748 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6d4005bca0 con 0x7f6d541026b0 2024-04-25T01:57:48.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.799+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d54108df0 con 0x7f6d4406e140 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.809+0000 7f6d517fa700 1 -- 172.21.15.33:0/3064211748 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6d54108df0 con 0x7f6d4406e140 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (18m) 12m ago 30m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (16m) 12m ago 30m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (16m) 11m ago 29m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (18m) 12m ago 29m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (12m) 12m ago 24m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (12m) 12m ago 24m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (11m) 11m ago 24m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (24m) 11m ago 24m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:57:48.812 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (20m) 12m ago 31m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (19m) 11m ago 29m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (17m) 12m ago 31m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (16m) 11m ago 29m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (19m) 12m ago 30m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (19m) 11m ago 29m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (15m) 12m ago 28m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (15m) 12m ago 27m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (14m) 12m ago 27m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (13m) 11m ago 27m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (13m) 11m ago 26m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (12m) 11m ago 26m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:57:48.813 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (19m) 12m ago 29m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:57:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6d4406e140 msgr2=0x7f6d440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6d4406e140 0x7f6d440705f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6d48004eb0 tx=0x7f6d4800b040 comp rx=0 tx=0).stop 2024-04-25T01:57:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 msgr2=0x7f6d54112de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 0x7f6d54112de0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f6d40009100 tx=0x7f6d40009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 shutdown_connections 2024-04-25T01:57:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6d4406e140 0x7f6d440705f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d54101d10 0x7f6d541128a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 --2- 172.21.15.33:0/3064211748 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d541026b0 0x7f6d54112de0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 >> 172.21.15.33:0/3064211748 conn(0x7f6d540f7860 msgr2=0x7f6d540f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 shutdown_connections 2024-04-25T01:57:48.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.811+0000 7f6d5aa77700 1 -- 172.21.15.33:0/3064211748 wait complete. 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.914+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1300796232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc065640 msgr2=0x7ff1bc065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.914+0000 7ff1c09c1700 1 --2- 172.21.15.33:0/1300796232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc065640 0x7ff1bc065ab0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4004940 tx=0x7ff1a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.914+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1300796232 shutdown_connections 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.914+0000 7ff1c09c1700 1 --2- 172.21.15.33:0/1300796232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc065640 0x7ff1bc065ab0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.914+0000 7ff1c09c1700 1 --2- 172.21.15.33:0/1300796232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1bc102e50 0x7ff1bc103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.914+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1300796232 >> 172.21.15.33:0/1300796232 conn(0x7ff1bc071d00 msgr2=0x7ff1bc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.914+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1300796232 shutdown_connections 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.914+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1300796232 wait complete. 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1c09c1700 1 Processor -- start 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1c09c1700 1 -- start start 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1c09c1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1bc065640 0x7ff1bc113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:48.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1c09c1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc102e50 0x7ff1bc1140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1c09c1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc114700 con 0x7ff1bc102e50 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1c09c1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc10daf0 con 0x7ff1bc065640 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1b9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc102e50 0x7ff1bc1140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1b9d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc102e50 0x7ff1bc1140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46090/0 (socket says 172.21.15.33:46090) 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1b9d9b700 1 -- 172.21.15.33:0/1457683238 learned_addr learned my addr 172.21.15.33:0/1457683238 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1ba59c700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1bc065640 0x7ff1bc113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1b9d9b700 1 -- 172.21.15.33:0/1457683238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1bc065640 msgr2=0x7ff1bc113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1b9d9b700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1bc065640 0x7ff1bc113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1b9d9b700 1 -- 172.21.15.33:0/1457683238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a4005040 con 0x7ff1bc102e50 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.915+0000 7ff1b9d9b700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc102e50 0x7ff1bc1140b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400b3b0 tx=0x7ff1a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:48.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.916+0000 7ff1b2ffd700 1 -- 172.21.15.33:0/1457683238 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a400e040 con 0x7ff1bc102e50 2024-04-25T01:57:48.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.916+0000 7ff1b2ffd700 1 -- 172.21.15.33:0/1457683238 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1a4013420 con 0x7ff1bc102e50 2024-04-25T01:57:48.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.916+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc10dd70 con 0x7ff1bc102e50 2024-04-25T01:57:48.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.916+0000 7ff1b2ffd700 1 -- 172.21.15.33:0/1457683238 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1a401c630 con 0x7ff1bc102e50 2024-04-25T01:57:48.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.916+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc10e330 con 0x7ff1bc102e50 2024-04-25T01:57:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.917+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc100100 con 0x7ff1bc102e50 2024-04-25T01:57:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.917+0000 7ff1b2ffd700 1 -- 172.21.15.33:0/1457683238 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff1a4013590 con 0x7ff1bc102e50 2024-04-25T01:57:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.918+0000 7ff1b2ffd700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff1a806e140 0x7ff1a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.918+0000 7ff1b2ffd700 1 -- 172.21.15.33:0/1457683238 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff1a408ec30 con 0x7ff1bc102e50 2024-04-25T01:57:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.918+0000 7ff1ba59c700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff1a806e140 0x7ff1a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:48.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.918+0000 7ff1ba59c700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff1a806e140 0x7ff1a80705f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac004fd0 tx=0x7ff1ac009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:48.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:48.922+0000 7ff1b2ffd700 1 -- 172.21.15.33:0/1457683238 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff1a405bc00 con 0x7ff1bc102e50 2024-04-25T01:57:49.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.219+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff1bc10f1c0 con 0x7ff1bc102e50 2024-04-25T01:57:49.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.220+0000 7ff1b2ffd700 1 -- 172.21.15.33:0/1457683238 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff1a405b790 con 0x7ff1bc102e50 2024-04-25T01:57:49.223 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:57:49.223 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:57:49.223 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:57:49.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:57:49.223 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:57:49.223 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:57:49.223 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:57:49.224 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:57:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff1a806e140 msgr2=0x7ff1a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff1a806e140 0x7ff1a80705f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac004fd0 tx=0x7ff1ac009040 comp rx=0 tx=0).stop 2024-04-25T01:57:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc102e50 msgr2=0x7ff1bc1140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc102e50 0x7ff1bc1140b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400b3b0 tx=0x7ff1a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 shutdown_connections 2024-04-25T01:57:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff1a806e140 0x7ff1a80705f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff1bc065640 0x7ff1bc113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 --2- 172.21.15.33:0/1457683238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff1bc102e50 0x7ff1bc1140b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 >> 172.21.15.33:0/1457683238 conn(0x7ff1bc071d00 msgr2=0x7ff1bc0fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 shutdown_connections 2024-04-25T01:57:49.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.222+0000 7ff1c09c1700 1 -- 172.21.15.33:0/1457683238 wait complete. 2024-04-25T01:57:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.327+0000 7f741c852700 1 -- 172.21.15.33:0/2300015524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 msgr2=0x7f74180fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.327+0000 7f741c852700 1 --2- 172.21.15.33:0/2300015524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 0x7f74180fc1b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f740c004940 tx=0x7f740c009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.327+0000 7f741c852700 1 -- 172.21.15.33:0/2300015524 shutdown_connections 2024-04-25T01:57:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.327+0000 7f741c852700 1 --2- 172.21.15.33:0/2300015524 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 0x7f74180fc1b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.327+0000 7f741c852700 1 --2- 172.21.15.33:0/2300015524 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7418101d90 0x7f7418102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.327+0000 7f741c852700 1 -- 172.21.15.33:0/2300015524 >> 172.21.15.33:0/2300015524 conn(0x7f7418071d00 msgr2=0x7f7418072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.327+0000 7f741c852700 1 -- 172.21.15.33:0/2300015524 shutdown_connections 2024-04-25T01:57:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.327+0000 7f741c852700 1 -- 172.21.15.33:0/2300015524 wait complete. 2024-04-25T01:57:49.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.328+0000 7f741c852700 1 Processor -- start 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.328+0000 7f741c852700 1 -- start start 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.328+0000 7f741c852700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 0x7f7418112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.328+0000 7f741c852700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7418101d90 0x7f7418112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.328+0000 7f741c852700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7418113640 con 0x7f74180fbd40 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.328+0000 7f741c852700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741810ca30 con 0x7f7418101d90 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.328+0000 7f741659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 0x7f7418112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.328+0000 7f741659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 0x7f7418112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46114/0 (socket says 172.21.15.33:46114) 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.328+0000 7f741659c700 1 -- 172.21.15.33:0/3289872461 learned_addr learned my addr 172.21.15.33:0/3289872461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f7415d9b700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7418101d90 0x7f7418112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f741659c700 1 -- 172.21.15.33:0/3289872461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7418101d90 msgr2=0x7f7418112ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f741659c700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7418101d90 0x7f7418112ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f741659c700 1 -- 172.21.15.33:0/3289872461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740c005040 con 0x7f74180fbd40 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f741659c700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 0x7f7418112ab0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7408002a50 tx=0x7f74080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:49.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f7406ffd700 1 -- 172.21.15.33:0/3289872461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7408007cc0 con 0x7f74180fbd40 2024-04-25T01:57:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741810cd10 con 0x7f74180fbd40 2024-04-25T01:57:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f7406ffd700 1 -- 172.21.15.33:0/3289872461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f740800f040 con 0x7f74180fbd40 2024-04-25T01:57:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f7406ffd700 1 -- 172.21.15.33:0/3289872461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740800da70 con 0x7f74180fbd40 2024-04-25T01:57:49.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.329+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741810d2d0 con 0x7f74180fbd40 2024-04-25T01:57:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.330+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74180fd430 con 0x7f74180fbd40 2024-04-25T01:57:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.331+0000 7f7406ffd700 1 -- 172.21.15.33:0/3289872461 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f740800dc10 con 0x7f74180fbd40 2024-04-25T01:57:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.331+0000 7f7406ffd700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f740006e150 0x7f7400070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.331+0000 7f7406ffd700 1 -- 172.21.15.33:0/3289872461 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f740808d180 con 0x7f74180fbd40 2024-04-25T01:57:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.331+0000 7f7415d9b700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f740006e150 0x7f7400070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:49.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.332+0000 7f7415d9b700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f740006e150 0x7f7400070600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f741810dbf0 tx=0x7f740c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:49.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.336+0000 7f7406ffd700 1 -- 172.21.15.33:0/3289872461 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f740805a030 con 0x7f74180fbd40 2024-04-25T01:57:49.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:49 smithi033 ceph-mon[111306]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:49.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:49 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1457683238' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:57:49.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:49 smithi138 ceph-mon[81304]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:49.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:49 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1457683238' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:57:49.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.590+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7418108e10 con 0x7f74180fbd40 2024-04-25T01:57:49.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.591+0000 7f7406ffd700 1 -- 172.21.15.33:0/3289872461 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f7408059bc0 con 0x7f74180fbd40 2024-04-25T01:57:49.594 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:57:49.594 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:57:49.594 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:57:49.594 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:57:49.594 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:57:49.594 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:57:49.594 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:57:49.595 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:57:49.596 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:57:49.597 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:57:49.597 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:57:49.597 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:57:49.597 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:57:49.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f740006e150 msgr2=0x7f7400070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f740006e150 0x7f7400070600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f741810dbf0 tx=0x7f740c009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 msgr2=0x7f7418112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 0x7f7418112ab0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7408002a50 tx=0x7f74080046b0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 shutdown_connections 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f740006e150 0x7f7400070600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74180fbd40 0x7f7418112ab0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 --2- 172.21.15.33:0/3289872461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7418101d90 0x7f7418112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 >> 172.21.15.33:0/3289872461 conn(0x7f7418071d00 msgr2=0x7f7418104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 shutdown_connections 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.593+0000 7f741c852700 1 -- 172.21.15.33:0/3289872461 wait complete. 2024-04-25T01:57:49.598 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:57:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.698+0000 7faa3616d700 1 -- 172.21.15.33:0/1376064239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fe810 msgr2=0x7faa30106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.698+0000 7faa3616d700 1 --2- 172.21.15.33:0/1376064239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fe810 0x7faa30106e50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faa24004940 tx=0x7faa24009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.698+0000 7faa3616d700 1 -- 172.21.15.33:0/1376064239 shutdown_connections 2024-04-25T01:57:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.698+0000 7faa3616d700 1 --2- 172.21.15.33:0/1376064239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fe810 0x7faa30106e50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.698+0000 7faa3616d700 1 --2- 172.21.15.33:0/1376064239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa300fde70 0x7faa300fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.698+0000 7faa3616d700 1 -- 172.21.15.33:0/1376064239 >> 172.21.15.33:0/1376064239 conn(0x7faa300f7860 msgr2=0x7faa300f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.698+0000 7faa3616d700 1 -- 172.21.15.33:0/1376064239 shutdown_connections 2024-04-25T01:57:49.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.698+0000 7faa3616d700 1 -- 172.21.15.33:0/1376064239 wait complete. 2024-04-25T01:57:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa3616d700 1 Processor -- start 2024-04-25T01:57:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa3616d700 1 -- start start 2024-04-25T01:57:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa3616d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fde70 0x7faa30113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:49.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa3616d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa300fe810 0x7faa30113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa3616d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa301144e0 con 0x7faa300fde70 2024-04-25T01:57:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa3616d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3010dae0 con 0x7faa300fe810 2024-04-25T01:57:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa2f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fde70 0x7faa30113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa2f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fde70 0x7faa30113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46126/0 (socket says 172.21.15.33:46126) 2024-04-25T01:57:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa2f7fe700 1 -- 172.21.15.33:0/2918429567 learned_addr learned my addr 172.21.15.33:0/2918429567 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:49.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa2effd700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa300fe810 0x7faa30113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:49.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa2f7fe700 1 -- 172.21.15.33:0/2918429567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa300fe810 msgr2=0x7faa30113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa2f7fe700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa300fe810 0x7faa30113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.699+0000 7faa2f7fe700 1 -- 172.21.15.33:0/2918429567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24005040 con 0x7faa300fde70 2024-04-25T01:57:49.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.700+0000 7faa2f7fe700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fde70 0x7faa30113950 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faa2000c8c0 tx=0x7faa200044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:49.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.700+0000 7faa17fff700 1 -- 172.21.15.33:0/2918429567 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa2000a980 con 0x7faa300fde70 2024-04-25T01:57:49.707 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.700+0000 7faa17fff700 1 -- 172.21.15.33:0/2918429567 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa2000ce60 con 0x7faa300fde70 2024-04-25T01:57:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.700+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3010ddc0 con 0x7faa300fde70 2024-04-25T01:57:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.700+0000 7faa17fff700 1 -- 172.21.15.33:0/2918429567 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa200115e0 con 0x7faa300fde70 2024-04-25T01:57:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.700+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa3010e300 con 0x7faa300fde70 2024-04-25T01:57:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.701+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa30066af0 con 0x7faa300fde70 2024-04-25T01:57:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.701+0000 7faa17fff700 1 -- 172.21.15.33:0/2918429567 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faa20012400 con 0x7faa300fde70 2024-04-25T01:57:49.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.702+0000 7faa17fff700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa1806e140 0x7faa180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:49.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.702+0000 7faa17fff700 1 -- 172.21.15.33:0/2918429567 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7faa2008db00 con 0x7faa300fde70 2024-04-25T01:57:49.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.702+0000 7faa2effd700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa1806e140 0x7faa180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:49.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.702+0000 7faa2effd700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa1806e140 0x7faa180705f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa3010eca0 tx=0x7faa24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:49.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.707+0000 7faa17fff700 1 -- 172.21.15.33:0/2918429567 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faa2005aad0 con 0x7faa300fde70 2024-04-25T01:57:49.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.933+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa30109e30 con 0x7faa1806e140 2024-04-25T01:57:49.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.934+0000 7faa17fff700 1 -- 172.21.15.33:0/2918429567 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faa30109e30 con 0x7faa1806e140 2024-04-25T01:57:49.937 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:57:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:57:49.937 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:57:49.938 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:57:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa1806e140 msgr2=0x7faa180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa1806e140 0x7faa180705f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7faa3010eca0 tx=0x7faa24009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fde70 msgr2=0x7faa30113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fde70 0x7faa30113950 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faa2000c8c0 tx=0x7faa200044c0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 shutdown_connections 2024-04-25T01:57:49.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faa1806e140 0x7faa180705f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faa300fde70 0x7faa30113950 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 --2- 172.21.15.33:0/2918429567 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faa300fe810 0x7faa30113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 >> 172.21.15.33:0/2918429567 conn(0x7faa300f7860 msgr2=0x7faa300f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 shutdown_connections 2024-04-25T01:57:49.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:49.936+0000 7faa3616d700 1 -- 172.21.15.33:0/2918429567 wait complete. 2024-04-25T01:57:50.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.041+0000 7f39e0f46700 1 -- 172.21.15.33:0/2714902780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc103790 msgr2=0x7f39dc0fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.041+0000 7f39e0f46700 1 --2- 172.21.15.33:0/2714902780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc103790 0x7f39dc0fce00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f39c4004940 tx=0x7f39c4009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.041+0000 7f39e0f46700 1 -- 172.21.15.33:0/2714902780 shutdown_connections 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.041+0000 7f39e0f46700 1 --2- 172.21.15.33:0/2714902780 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc103790 0x7f39dc0fce00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.041+0000 7f39e0f46700 1 --2- 172.21.15.33:0/2714902780 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39dc102df0 0x7f39dc1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.041+0000 7f39e0f46700 1 -- 172.21.15.33:0/2714902780 >> 172.21.15.33:0/2714902780 conn(0x7f39dc0f8980 msgr2=0x7f39dc0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39e0f46700 1 -- 172.21.15.33:0/2714902780 shutdown_connections 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39e0f46700 1 -- 172.21.15.33:0/2714902780 wait complete. 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39e0f46700 1 Processor -- start 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39e0f46700 1 -- start start 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39e0f46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc102df0 0x7f39dc115d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39e0f46700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39dc103790 0x7f39dc1162d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:50.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39e0f46700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc116920 con 0x7f39dc102df0 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39e0f46700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc10fcf0 con 0x7f39dc103790 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc102df0 0x7f39dc115d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39da59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc102df0 0x7f39dc115d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46152/0 (socket says 172.21.15.33:46152) 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.042+0000 7f39da59c700 1 -- 172.21.15.33:0/2693026509 learned_addr learned my addr 172.21.15.33:0/2693026509 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39d9d9b700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39dc103790 0x7f39dc1162d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39da59c700 1 -- 172.21.15.33:0/2693026509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39dc103790 msgr2=0x7f39dc1162d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39da59c700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39dc103790 0x7f39dc1162d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39da59c700 1 -- 172.21.15.33:0/2693026509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c4005040 con 0x7f39dc102df0 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39d9d9b700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39dc103790 0x7f39dc1162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39da59c700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc102df0 0x7f39dc115d90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c8c0 tx=0x7f39cc0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39d2ffd700 1 -- 172.21.15.33:0/2693026509 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39cc00a980 con 0x7f39dc102df0 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc10ffd0 con 0x7f39dc102df0 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39d2ffd700 1 -- 172.21.15.33:0/2693026509 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39cc00ce60 con 0x7f39dc102df0 2024-04-25T01:57:50.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39d2ffd700 1 -- 172.21.15.33:0/2693026509 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39cc0115e0 con 0x7f39dc102df0 2024-04-25T01:57:50.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.043+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc110590 con 0x7f39dc102df0 2024-04-25T01:57:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.044+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39dc0fe540 con 0x7f39dc102df0 2024-04-25T01:57:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.045+0000 7f39d2ffd700 1 -- 172.21.15.33:0/2693026509 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f39cc012400 con 0x7f39dc102df0 2024-04-25T01:57:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.045+0000 7f39d2ffd700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39c806e140 0x7f39c80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:57:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.045+0000 7f39d2ffd700 1 -- 172.21.15.33:0/2693026509 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f39cc08dbf0 con 0x7f39dc102df0 2024-04-25T01:57:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.045+0000 7f39d9d9b700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39c806e140 0x7f39c80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:57:50.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.046+0000 7f39d9d9b700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39c806e140 0x7f39c80705f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f39c400b3b0 tx=0x7f39c4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:57:50.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.050+0000 7f39d2ffd700 1 -- 172.21.15.33:0/2693026509 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f39cc05ab20 con 0x7f39dc102df0 2024-04-25T01:57:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.347+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f39dc05f5c0 con 0x7f39dc102df0 2024-04-25T01:57:50.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.347+0000 7f39d2ffd700 1 -- 172.21.15.33:0/2693026509 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f39cc0126b0 con 0x7f39dc102df0 2024-04-25T01:57:50.351 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:57:50.351 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:57:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:57:50.351 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:57:50.351 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:57:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39c806e140 msgr2=0x7f39c80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39c806e140 0x7f39c80705f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f39c400b3b0 tx=0x7f39c4009e30 comp rx=0 tx=0).stop 2024-04-25T01:57:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc102df0 msgr2=0x7f39dc115d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:57:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc102df0 0x7f39dc115d90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00c8c0 tx=0x7f39cc0044c0 comp rx=0 tx=0).stop 2024-04-25T01:57:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 shutdown_connections 2024-04-25T01:57:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f39c806e140 0x7f39c80705f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39dc102df0 0x7f39dc115d90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:50.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 --2- 172.21.15.33:0/2693026509 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f39dc103790 0x7f39dc1162d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:57:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 >> 172.21.15.33:0/2693026509 conn(0x7f39dc0f8980 msgr2=0x7f39dc0f9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:57:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 shutdown_connections 2024-04-25T01:57:50.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:57:50.350+0000 7f39e0f46700 1 -- 172.21.15.33:0/2693026509 wait complete. 2024-04-25T01:57:50.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:50 smithi033 ceph-mon[111306]: pgmap v552: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:50.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:50 smithi033 ceph-mon[111306]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:50.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:50 smithi033 ceph-mon[111306]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:50.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:50 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3289872461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:57:50.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:50 smithi138 ceph-mon[81304]: pgmap v552: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:50.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:50 smithi138 ceph-mon[81304]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:50.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:50 smithi138 ceph-mon[81304]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:50.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:50 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3289872461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:57:51.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:51 smithi033 ceph-mon[111306]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:51.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:51 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2693026509' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:57:51.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:51 smithi138 ceph-mon[81304]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:57:51.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:51 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2693026509' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:57:52.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:52 smithi033 ceph-mon[111306]: pgmap v553: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:52.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:52 smithi138 ceph-mon[81304]: pgmap v553: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:54.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:54 smithi033 ceph-mon[111306]: pgmap v554: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:54.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:54 smithi138 ceph-mon[81304]: pgmap v554: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:56.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:56 smithi138 ceph-mon[81304]: pgmap v555: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:56.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:57:56.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:56 smithi033 ceph-mon[111306]: pgmap v555: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:56.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:57:58.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:57:58 smithi138 ceph-mon[81304]: pgmap v556: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:57:58.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:57:58 smithi033 ceph-mon[111306]: pgmap v556: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:00.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:00 smithi138 ceph-mon[81304]: pgmap v557: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:00.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:00 smithi033 ceph-mon[111306]: pgmap v557: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:01.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:01 smithi138 ceph-mon[81304]: pgmap v558: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:01.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:01 smithi033 ceph-mon[111306]: pgmap v558: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:03.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:03 smithi033 ceph-mon[111306]: pgmap v559: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:03.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:03 smithi138 ceph-mon[81304]: pgmap v559: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:05.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:05 smithi138 ceph-mon[81304]: pgmap v560: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:06.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:05 smithi033 ceph-mon[111306]: pgmap v560: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:08.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:07 smithi138 ceph-mon[81304]: pgmap v561: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:08.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:07 smithi033 ceph-mon[111306]: pgmap v561: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:10.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:09 smithi138 ceph-mon[81304]: pgmap v562: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:10.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:09 smithi033 ceph-mon[111306]: pgmap v562: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:11.044 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:58:11.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:58:12.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:11 smithi033 ceph-mon[111306]: pgmap v563: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:12.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:11 smithi138 ceph-mon[81304]: pgmap v563: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:14.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:13 smithi033 ceph-mon[111306]: pgmap v564: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:14.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:13 smithi138 ceph-mon[81304]: pgmap v564: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:16.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:15 smithi033 ceph-mon[111306]: pgmap v565: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:16.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:15 smithi138 ceph-mon[81304]: pgmap v565: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:18.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:17 smithi033 ceph-mon[111306]: pgmap v566: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:18.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:17 smithi138 ceph-mon[81304]: pgmap v566: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:20.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:19 smithi033 ceph-mon[111306]: pgmap v567: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:20.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:19 smithi138 ceph-mon[81304]: pgmap v567: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.470+0000 7fe61b094700 1 -- 172.21.15.33:0/2562463564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 msgr2=0x7fe6140fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.470+0000 7fe61b094700 1 --2- 172.21.15.33:0/2562463564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 0x7fe6140fcdf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe604004940 tx=0x7fe604009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.470+0000 7fe61b094700 1 -- 172.21.15.33:0/2562463564 shutdown_connections 2024-04-25T01:58:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.470+0000 7fe61b094700 1 --2- 172.21.15.33:0/2562463564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 0x7fe6140fcdf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.470+0000 7fe61b094700 1 --2- 172.21.15.33:0/2562463564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe614102de0 0x7fe6141031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.470+0000 7fe61b094700 1 -- 172.21.15.33:0/2562463564 >> 172.21.15.33:0/2562463564 conn(0x7fe6140f8990 msgr2=0x7fe6140fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.470+0000 7fe61b094700 1 -- 172.21.15.33:0/2562463564 shutdown_connections 2024-04-25T01:58:20.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.470+0000 7fe61b094700 1 -- 172.21.15.33:0/2562463564 wait complete. 2024-04-25T01:58:20.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.471+0000 7fe61b094700 1 Processor -- start 2024-04-25T01:58:20.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.471+0000 7fe61b094700 1 -- start start 2024-04-25T01:58:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.471+0000 7fe61b094700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe614102de0 0x7fe614113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.471+0000 7fe61b094700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 0x7fe614113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.471+0000 7fe61b094700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6141144e0 con 0x7fe614103780 2024-04-25T01:58:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.471+0000 7fe61b094700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61410dae0 con 0x7fe614102de0 2024-04-25T01:58:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.471+0000 7fe613fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 0x7fe614113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.471+0000 7fe613fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 0x7fe614113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37030/0 (socket says 172.21.15.33:37030) 2024-04-25T01:58:20.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.471+0000 7fe613fff700 1 -- 172.21.15.33:0/2911174394 learned_addr learned my addr 172.21.15.33:0/2911174394 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe618e30700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe614102de0 0x7fe614113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe613fff700 1 -- 172.21.15.33:0/2911174394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe614102de0 msgr2=0x7fe614113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe613fff700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe614102de0 0x7fe614113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe613fff700 1 -- 172.21.15.33:0/2911174394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe604005040 con 0x7fe614103780 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe613fff700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 0x7fe614113e90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe604009180 tx=0x7fe604009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe6117fa700 1 -- 172.21.15.33:0/2911174394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60400e040 con 0x7fe614103780 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe6117fa700 1 -- 172.21.15.33:0/2911174394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe60400bbc0 con 0x7fe614103780 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe61410dd60 con 0x7fe614103780 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe6117fa700 1 -- 172.21.15.33:0/2911174394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe60401c3f0 con 0x7fe614103780 2024-04-25T01:58:20.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.472+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61410e320 con 0x7fe614103780 2024-04-25T01:58:20.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.473+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6140fe530 con 0x7fe614103780 2024-04-25T01:58:20.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.474+0000 7fe6117fa700 1 -- 172.21.15.33:0/2911174394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe60400bd30 con 0x7fe614103780 2024-04-25T01:58:20.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.474+0000 7fe6117fa700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe60006e140 0x7fe6000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:20.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.475+0000 7fe6117fa700 1 -- 172.21.15.33:0/2911174394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe60408eb70 con 0x7fe614103780 2024-04-25T01:58:20.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.475+0000 7fe618e30700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe60006e140 0x7fe6000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:20.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.475+0000 7fe618e30700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe60006e140 0x7fe6000705f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc007fd0 tx=0x7fe5fc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:20.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.479+0000 7fe6117fa700 1 -- 172.21.15.33:0/2911174394 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe60405bb40 con 0x7fe614103780 2024-04-25T01:58:20.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.708+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe614109ec0 con 0x7fe60006e140 2024-04-25T01:58:20.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.709+0000 7fe6117fa700 1 -- 172.21.15.33:0/2911174394 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe614109ec0 con 0x7fe60006e140 2024-04-25T01:58:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe60006e140 msgr2=0x7fe6000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe60006e140 0x7fe6000705f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc007fd0 tx=0x7fe5fc009040 comp rx=0 tx=0).stop 2024-04-25T01:58:20.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 msgr2=0x7fe614113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 0x7fe614113e90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe604009180 tx=0x7fe604009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 shutdown_connections 2024-04-25T01:58:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe60006e140 0x7fe6000705f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe614102de0 0x7fe614113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 --2- 172.21.15.33:0/2911174394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe614103780 0x7fe614113e90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 >> 172.21.15.33:0/2911174394 conn(0x7fe6140f8990 msgr2=0x7fe6140f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 shutdown_connections 2024-04-25T01:58:20.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.711+0000 7fe61b094700 1 -- 172.21.15.33:0/2911174394 wait complete. 2024-04-25T01:58:20.724 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.817+0000 7f7e23357700 1 -- 172.21.15.33:0/4053525827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c065640 msgr2=0x7f7e1c065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.817+0000 7f7e23357700 1 --2- 172.21.15.33:0/4053525827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c065640 0x7f7e1c065ab0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7e10004940 tx=0x7f7e10009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.817+0000 7f7e23357700 1 -- 172.21.15.33:0/4053525827 shutdown_connections 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.817+0000 7f7e23357700 1 --2- 172.21.15.33:0/4053525827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c065640 0x7f7e1c065ab0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.817+0000 7f7e23357700 1 --2- 172.21.15.33:0/4053525827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1c102e50 0x7f7e1c103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.817+0000 7f7e23357700 1 -- 172.21.15.33:0/4053525827 >> 172.21.15.33:0/4053525827 conn(0x7f7e1c071d00 msgr2=0x7f7e1c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.817+0000 7f7e23357700 1 -- 172.21.15.33:0/4053525827 shutdown_connections 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.817+0000 7f7e23357700 1 -- 172.21.15.33:0/4053525827 wait complete. 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e23357700 1 Processor -- start 2024-04-25T01:58:20.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e23357700 1 -- start start 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e23357700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1c065640 0x7f7e1c0fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e23357700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c102e50 0x7f7e1c0fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e23357700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c0fc880 con 0x7f7e1c102e50 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e23357700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1c0fc9c0 con 0x7f7e1c065640 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e208f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c102e50 0x7f7e1c0fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e208f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c102e50 0x7f7e1c0fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37038/0 (socket says 172.21.15.33:37038) 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e208f2700 1 -- 172.21.15.33:0/530398964 learned_addr learned my addr 172.21.15.33:0/530398964 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e210f3700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1c065640 0x7f7e1c0fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e208f2700 1 -- 172.21.15.33:0/530398964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1c065640 msgr2=0x7f7e1c0fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e208f2700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1c065640 0x7f7e1c0fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:20.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.818+0000 7f7e208f2700 1 -- 172.21.15.33:0/530398964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e10005040 con 0x7f7e1c102e50 2024-04-25T01:58:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.819+0000 7f7e208f2700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c102e50 0x7f7e1c0fc2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009150 tx=0x7f7e10009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.819+0000 7f7e19ffb700 1 -- 172.21.15.33:0/530398964 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1000e040 con 0x7f7e1c102e50 2024-04-25T01:58:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.819+0000 7f7e19ffb700 1 -- 172.21.15.33:0/530398964 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e10013420 con 0x7f7e1c102e50 2024-04-25T01:58:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.819+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1c0fcc10 con 0x7f7e1c102e50 2024-04-25T01:58:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.819+0000 7f7e19ffb700 1 -- 172.21.15.33:0/530398964 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e1001c610 con 0x7f7e1c102e50 2024-04-25T01:58:20.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.819+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e1c11a2a0 con 0x7f7e1c102e50 2024-04-25T01:58:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.820+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e1c100100 con 0x7f7e1c102e50 2024-04-25T01:58:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.821+0000 7f7e19ffb700 1 -- 172.21.15.33:0/530398964 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7e1000baf0 con 0x7f7e1c102e50 2024-04-25T01:58:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.821+0000 7f7e19ffb700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7e0406e150 0x7f7e04070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.821+0000 7f7e19ffb700 1 -- 172.21.15.33:0/530398964 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7e1008ed10 con 0x7f7e1c102e50 2024-04-25T01:58:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.821+0000 7f7e210f3700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7e0406e150 0x7f7e04070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:20.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.822+0000 7f7e210f3700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7e0406e150 0x7f7e04070600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c007a40 tx=0x7f7e0c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:20.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:20.826+0000 7f7e19ffb700 1 -- 172.21.15.33:0/530398964 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7e10058110 con 0x7f7e1c102e50 2024-04-25T01:58:21.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.052+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7e1c109f00 con 0x7f7e0406e150 2024-04-25T01:58:21.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.053+0000 7f7e19ffb700 1 -- 172.21.15.33:0/530398964 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7e1c109f00 con 0x7f7e0406e150 2024-04-25T01:58:21.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7e0406e150 msgr2=0x7f7e04070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7e0406e150 0x7f7e04070600 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c007a40 tx=0x7f7e0c009da0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c102e50 msgr2=0x7f7e1c0fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c102e50 0x7f7e1c0fc2f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009150 tx=0x7f7e10009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 shutdown_connections 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7e0406e150 0x7f7e04070600 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7e1c065640 0x7f7e1c0fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 --2- 172.21.15.33:0/530398964 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7e1c102e50 0x7f7e1c0fc2f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 >> 172.21.15.33:0/530398964 conn(0x7f7e1c071d00 msgr2=0x7f7e1c0fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 shutdown_connections 2024-04-25T01:58:21.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.056+0000 7f7e23357700 1 -- 172.21.15.33:0/530398964 wait complete. 2024-04-25T01:58:21.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.160+0000 7fde07a1e700 1 -- 172.21.15.33:0/2885008174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00102df0 msgr2=0x7fde001031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.160+0000 7fde07a1e700 1 --2- 172.21.15.33:0/2885008174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00102df0 0x7fde001031c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fddfc004940 tx=0x7fddfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:21.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.160+0000 7fde07a1e700 1 -- 172.21.15.33:0/2885008174 shutdown_connections 2024-04-25T01:58:21.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.160+0000 7fde07a1e700 1 --2- 172.21.15.33:0/2885008174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde00103790 0x7fde000fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.160+0000 7fde07a1e700 1 --2- 172.21.15.33:0/2885008174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00102df0 0x7fde001031c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.160+0000 7fde07a1e700 1 -- 172.21.15.33:0/2885008174 >> 172.21.15.33:0/2885008174 conn(0x7fde000f8980 msgr2=0x7fde000fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.160+0000 7fde07a1e700 1 -- 172.21.15.33:0/2885008174 shutdown_connections 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.160+0000 7fde07a1e700 1 -- 172.21.15.33:0/2885008174 wait complete. 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde07a1e700 1 Processor -- start 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde07a1e700 1 -- start start 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde07a1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde00102df0 0x7fde00113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde07a1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00103790 0x7fde001140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde07a1e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde00114730 con 0x7fde00103790 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde07a1e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde0010db40 con 0x7fde00102df0 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde04fb9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00103790 0x7fde001140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde04fb9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00103790 0x7fde001140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37060/0 (socket says 172.21.15.33:37060) 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde04fb9700 1 -- 172.21.15.33:0/3304796828 learned_addr learned my addr 172.21.15.33:0/3304796828 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde057ba700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde00102df0 0x7fde00113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde04fb9700 1 -- 172.21.15.33:0/3304796828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde00102df0 msgr2=0x7fde00113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.161+0000 7fde04fb9700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde00102df0 0x7fde00113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.162+0000 7fde04fb9700 1 -- 172.21.15.33:0/3304796828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc005040 con 0x7fde00103790 2024-04-25T01:58:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.162+0000 7fde04fb9700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00103790 0x7fde001140e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fddf000c890 tx=0x7fddf00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.162+0000 7fddf5ffb700 1 -- 172.21.15.33:0/3304796828 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf000a7d0 con 0x7fde00103790 2024-04-25T01:58:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.162+0000 7fddf5ffb700 1 -- 172.21.15.33:0/3304796828 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddf000ae00 con 0x7fde00103790 2024-04-25T01:58:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.162+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde0010de20 con 0x7fde00103790 2024-04-25T01:58:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.162+0000 7fddf5ffb700 1 -- 172.21.15.33:0/3304796828 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddf00113a0 con 0x7fde00103790 2024-04-25T01:58:21.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.162+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde0010e3e0 con 0x7fde00103790 2024-04-25T01:58:21.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.163+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde00100100 con 0x7fde00103790 2024-04-25T01:58:21.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.164+0000 7fddf5ffb700 1 -- 172.21.15.33:0/3304796828 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fddf0012400 con 0x7fde00103790 2024-04-25T01:58:21.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.164+0000 7fddf5ffb700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fddec06e140 0x7fddec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:21.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.164+0000 7fddf5ffb700 1 -- 172.21.15.33:0/3304796828 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fddf008dba0 con 0x7fde00103790 2024-04-25T01:58:21.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.164+0000 7fde057ba700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fddec06e140 0x7fddec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:21.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.164+0000 7fde057ba700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fddec06e140 0x7fddec0705f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fddfc004f90 tx=0x7fddfc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:21.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.169+0000 7fddf5ffb700 1 -- 172.21.15.33:0/3304796828 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fddf0056fa0 con 0x7fde00103790 2024-04-25T01:58:21.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.389+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde0010f1f0 con 0x7fddec06e140 2024-04-25T01:58:21.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.400+0000 7fddf5ffb700 1 -- 172.21.15.33:0/3304796828 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fde0010f1f0 con 0x7fddec06e140 2024-04-25T01:58:21.402 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:58:21.402 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (19m) 12m ago 30m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:58:21.402 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (16m) 12m ago 30m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:58:21.402 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (16m) 12m ago 29m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:58:21.402 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (19m) 12m ago 30m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:58:21.402 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (12m) 12m ago 25m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (13m) 12m ago 25m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (12m) 12m ago 25m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (25m) 12m ago 25m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (20m) 12m ago 31m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (20m) 12m ago 29m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (17m) 12m ago 32m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (17m) 12m ago 29m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (20m) 12m ago 30m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (20m) 12m ago 29m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (16m) 12m ago 28m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (15m) 12m ago 28m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (15m) 12m ago 28m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (14m) 12m ago 27m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:58:21.403 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (13m) 12m ago 27m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:58:21.404 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (13m) 12m ago 26m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:58:21.404 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (19m) 12m ago 30m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:58:21.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fddec06e140 msgr2=0x7fddec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fddec06e140 0x7fddec0705f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fddfc004f90 tx=0x7fddfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:21.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00103790 msgr2=0x7fde001140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00103790 0x7fde001140e0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fddf000c890 tx=0x7fddf00044c0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 shutdown_connections 2024-04-25T01:58:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fddec06e140 0x7fddec0705f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde00102df0 0x7fde00113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 --2- 172.21.15.33:0/3304796828 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde00103790 0x7fde001140e0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 >> 172.21.15.33:0/3304796828 conn(0x7fde000f8980 msgr2=0x7fde001073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 shutdown_connections 2024-04-25T01:58:21.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.402+0000 7fde07a1e700 1 -- 172.21.15.33:0/3304796828 wait complete. 2024-04-25T01:58:21.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.504+0000 7f465f9a8700 1 -- 172.21.15.33:0/2706434959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 msgr2=0x7f46580fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.504+0000 7f465f9a8700 1 --2- 172.21.15.33:0/2706434959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 0x7f46580fce10 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f464c004940 tx=0x7f464c009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:21.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.504+0000 7f465f9a8700 1 -- 172.21.15.33:0/2706434959 shutdown_connections 2024-04-25T01:58:21.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.504+0000 7f465f9a8700 1 --2- 172.21.15.33:0/2706434959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 0x7f46580fce10 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.504+0000 7f465f9a8700 1 --2- 172.21.15.33:0/2706434959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4658102e00 0x7f46581031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.504+0000 7f465f9a8700 1 -- 172.21.15.33:0/2706434959 >> 172.21.15.33:0/2706434959 conn(0x7f46580f89b0 msgr2=0x7f46580fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:21.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.504+0000 7f465f9a8700 1 -- 172.21.15.33:0/2706434959 shutdown_connections 2024-04-25T01:58:21.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.504+0000 7f465f9a8700 1 -- 172.21.15.33:0/2706434959 wait complete. 2024-04-25T01:58:21.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465f9a8700 1 Processor -- start 2024-04-25T01:58:21.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465f9a8700 1 -- start start 2024-04-25T01:58:21.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465f9a8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4658102e00 0x7f4658113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:21.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465f9a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 0x7f4658114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:21.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465f9a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4658114670 con 0x7f46581037a0 2024-04-25T01:58:21.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465f9a8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465810dab0 con 0x7f4658102e00 2024-04-25T01:58:21.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465cf43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 0x7f4658114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:21.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465cf43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 0x7f4658114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37072/0 (socket says 172.21.15.33:37072) 2024-04-25T01:58:21.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465cf43700 1 -- 172.21.15.33:0/2518300941 learned_addr learned my addr 172.21.15.33:0/2518300941 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465d744700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4658102e00 0x7f4658113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.505+0000 7f465cf43700 1 -- 172.21.15.33:0/2518300941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4658102e00 msgr2=0x7f4658113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.506+0000 7f465cf43700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4658102e00 0x7f4658113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.506+0000 7f465cf43700 1 -- 172.21.15.33:0/2518300941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f464c005040 con 0x7f46581037a0 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.506+0000 7f465cf43700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 0x7f4658114020 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f464c000c00 tx=0x7f464c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.506+0000 7f4655ffb700 1 -- 172.21.15.33:0/2518300941 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f464c00e070 con 0x7f46581037a0 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.506+0000 7f4655ffb700 1 -- 172.21.15.33:0/2518300941 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f464c013c00 con 0x7f46581037a0 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.506+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f465810dd30 con 0x7f46581037a0 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.506+0000 7f4655ffb700 1 -- 172.21.15.33:0/2518300941 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f464c00b650 con 0x7f46581037a0 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.506+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f465810e270 con 0x7f46581037a0 2024-04-25T01:58:21.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.507+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46581052d0 con 0x7f46581037a0 2024-04-25T01:58:21.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.508+0000 7f4655ffb700 1 -- 172.21.15.33:0/2518300941 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f464c01d9c0 con 0x7f46581037a0 2024-04-25T01:58:21.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.508+0000 7f4655ffb700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f464006e150 0x7f4640070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:21.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.508+0000 7f4655ffb700 1 -- 172.21.15.33:0/2518300941 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f464c08ed90 con 0x7f46581037a0 2024-04-25T01:58:21.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.508+0000 7f465d744700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f464006e150 0x7f4640070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:21.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.509+0000 7f465d744700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f464006e150 0x7f4640070600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4648001470 tx=0x7f464800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:21.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.513+0000 7f4655ffb700 1 -- 172.21.15.33:0/2518300941 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f464c05bd70 con 0x7f46581037a0 2024-04-25T01:58:21.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.811+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f465810f130 con 0x7f46581037a0 2024-04-25T01:58:21.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.811+0000 7f4655ffb700 1 -- 172.21.15.33:0/2518300941 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f464c022360 con 0x7f46581037a0 2024-04-25T01:58:21.813 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:58:21.813 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:58:21.814 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:58:21.815 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:58:21.815 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:58:21.815 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:58:21.815 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:58:21.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f464006e150 msgr2=0x7f4640070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f464006e150 0x7f4640070600 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4648001470 tx=0x7f464800b040 comp rx=0 tx=0).stop 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 msgr2=0x7f4658114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 0x7f4658114020 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f464c000c00 tx=0x7f464c009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 shutdown_connections 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f464006e150 0x7f4640070600 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4658102e00 0x7f4658113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 --2- 172.21.15.33:0/2518300941 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46581037a0 0x7f4658114020 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 >> 172.21.15.33:0/2518300941 conn(0x7f46580f89b0 msgr2=0x7f46580fa760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 shutdown_connections 2024-04-25T01:58:21.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.813+0000 7f465f9a8700 1 -- 172.21.15.33:0/2518300941 wait complete. 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.917+0000 7f200b31c700 1 -- 172.21.15.33:0/1141909084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 msgr2=0x7f20040fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.917+0000 7f200b31c700 1 --2- 172.21.15.33:0/1141909084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 0x7f20040fbd10 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2000004940 tx=0x7f2000009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.917+0000 7f200b31c700 1 -- 172.21.15.33:0/1141909084 shutdown_connections 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.917+0000 7f200b31c700 1 --2- 172.21.15.33:0/1141909084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 0x7f20040fbd10 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.917+0000 7f200b31c700 1 --2- 172.21.15.33:0/1141909084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2004101d00 0x7f20041020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.917+0000 7f200b31c700 1 -- 172.21.15.33:0/1141909084 >> 172.21.15.33:0/1141909084 conn(0x7f20040f7870 msgr2=0x7f20040f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.917+0000 7f200b31c700 1 -- 172.21.15.33:0/1141909084 shutdown_connections 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.917+0000 7f200b31c700 1 -- 172.21.15.33:0/1141909084 wait complete. 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f200b31c700 1 Processor -- start 2024-04-25T01:58:21.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f200b31c700 1 -- start start 2024-04-25T01:58:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f200b31c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2004101d00 0x7f2004113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f200b31c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 0x7f2004113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f200b31c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2004114520 con 0x7f20041026a0 2024-04-25T01:58:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f200b31c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200410db20 con 0x7f2004101d00 2024-04-25T01:58:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f20088b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 0x7f2004113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f20088b7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 0x7f2004113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37092/0 (socket says 172.21.15.33:37092) 2024-04-25T01:58:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f20088b7700 1 -- 172.21.15.33:0/1717178037 learned_addr learned my addr 172.21.15.33:0/1717178037 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:21.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f20090b8700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2004101d00 0x7f2004113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f20088b7700 1 -- 172.21.15.33:0/1717178037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2004101d00 msgr2=0x7f2004113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f20088b7700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2004101d00 0x7f2004113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.918+0000 7f20088b7700 1 -- 172.21.15.33:0/1717178037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000005040 con 0x7f20041026a0 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.919+0000 7f20088b7700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 0x7f2004113ed0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2000009100 tx=0x7f2000009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.919+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/1717178037 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f200000e040 con 0x7f20041026a0 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.919+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/1717178037 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2000009af0 con 0x7f20041026a0 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.919+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200410dda0 con 0x7f20041026a0 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.919+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/1717178037 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f200001c3f0 con 0x7f20041026a0 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.919+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200410e360 con 0x7f20041026a0 2024-04-25T01:58:21.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.920+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20040fd450 con 0x7f20041026a0 2024-04-25T01:58:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.921+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/1717178037 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f200000baf0 con 0x7f20041026a0 2024-04-25T01:58:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.921+0000 7f1ff9ffb700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1ff006e140 0x7f1ff00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.921+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/1717178037 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f200008ec90 con 0x7f20041026a0 2024-04-25T01:58:21.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.921+0000 7f20090b8700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1ff006e140 0x7f1ff00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:21.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.921+0000 7f20090b8700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1ff006e140 0x7f1ff00705f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4004eb0 tx=0x7f1ff400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:21.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:21.926+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/1717178037 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f200005bc60 con 0x7f20041026a0 2024-04-25T01:58:22.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.181+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f20040fd450 con 0x7f20041026a0 2024-04-25T01:58:22.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:21 smithi033 ceph-mon[111306]: pgmap v568: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:22.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:21 smithi033 ceph-mon[111306]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:22.185 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:21 smithi033 ceph-mon[111306]: from='client.35076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:22.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.182+0000 7f1ff9ffb700 1 -- 172.21.15.33:0/1717178037 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f200005b7f0 con 0x7f20041026a0 2024-04-25T01:58:22.186 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:58:22.186 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:58:22.186 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:58:22.187 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:58:22.187 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:58:22.187 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:58:22.187 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:58:22.187 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:58:22.187 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:58:22.187 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:58:22.188 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:58:22.188 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:58:22.188 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:58:22.188 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:58:22.188 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:58:22.188 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:58:22.189 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:58:22.189 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:58:22.189 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:58:22.189 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:58:22.189 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:58:22.189 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:58:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:58:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:58:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:58:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:58:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:58:22.190 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:58:22.191 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:58:22.191 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:58:22.191 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:58:22.191 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:58:22.191 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:58:22.191 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:58:22.192 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:58:22.192 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:58:22.192 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:58:22.192 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:58:22.192 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:58:22.192 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:58:22.193 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:58:22.193 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:58:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.184+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1ff006e140 msgr2=0x7f1ff00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.184+0000 7f200b31c700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1ff006e140 0x7f1ff00705f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4004eb0 tx=0x7f1ff400b040 comp rx=0 tx=0).stop 2024-04-25T01:58:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.184+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 msgr2=0x7f2004113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.184+0000 7f200b31c700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 0x7f2004113ed0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2000009100 tx=0x7f2000009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.184+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 shutdown_connections 2024-04-25T01:58:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.184+0000 7f200b31c700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1ff006e140 0x7f1ff00705f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.184+0000 7f200b31c700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2004101d00 0x7f2004113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.184+0000 7f200b31c700 1 --2- 172.21.15.33:0/1717178037 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f20041026a0 0x7f2004113ed0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.184+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 >> 172.21.15.33:0/1717178037 conn(0x7f20040f7870 msgr2=0x7f2004066250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:22.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.185+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 shutdown_connections 2024-04-25T01:58:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.185+0000 7f200b31c700 1 -- 172.21.15.33:0/1717178037 wait complete. 2024-04-25T01:58:22.196 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:58:22.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.289+0000 7fd493d4c700 1 -- 172.21.15.33:0/832917822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1026b0 msgr2=0x7fd48c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.289+0000 7fd493d4c700 1 --2- 172.21.15.33:0/832917822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1026b0 0x7fd48c0fbd20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fd47c004940 tx=0x7fd47c009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.289+0000 7fd493d4c700 1 -- 172.21.15.33:0/832917822 shutdown_connections 2024-04-25T01:58:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.289+0000 7fd493d4c700 1 --2- 172.21.15.33:0/832917822 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c1026b0 0x7fd48c0fbd20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.289+0000 7fd493d4c700 1 --2- 172.21.15.33:0/832917822 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c101d10 0x7fd48c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.289+0000 7fd493d4c700 1 -- 172.21.15.33:0/832917822 >> 172.21.15.33:0/832917822 conn(0x7fd48c0f7860 msgr2=0x7fd48c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd493d4c700 1 -- 172.21.15.33:0/832917822 shutdown_connections 2024-04-25T01:58:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd493d4c700 1 -- 172.21.15.33:0/832917822 wait complete. 2024-04-25T01:58:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd493d4c700 1 Processor -- start 2024-04-25T01:58:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd493d4c700 1 -- start start 2024-04-25T01:58:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd493d4c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c101d10 0x7fd48c113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:22.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd493d4c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c1026b0 0x7fd48c113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd493d4c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c114650 con 0x7fd48c101d10 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd493d4c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48c10dba0 con 0x7fd48c1026b0 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd491ae8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c101d10 0x7fd48c113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd491ae8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c101d10 0x7fd48c113a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37114/0 (socket says 172.21.15.33:37114) 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.290+0000 7fd491ae8700 1 -- 172.21.15.33:0/2024297099 learned_addr learned my addr 172.21.15.33:0/2024297099 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd4912e7700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c1026b0 0x7fd48c113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd491ae8700 1 -- 172.21.15.33:0/2024297099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c1026b0 msgr2=0x7fd48c113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd491ae8700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c1026b0 0x7fd48c113f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd491ae8700 1 -- 172.21.15.33:0/2024297099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c005040 con 0x7fd48c101d10 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd4912e7700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c1026b0 0x7fd48c113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:58:22.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd491ae8700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c101d10 0x7fd48c113a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd48800b790 tx=0x7fd488009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd4827fc700 1 -- 172.21.15.33:0/2024297099 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd48800f770 con 0x7fd48c101d10 2024-04-25T01:58:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48c10de20 con 0x7fd48c101d10 2024-04-25T01:58:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd4827fc700 1 -- 172.21.15.33:0/2024297099 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd48800fda0 con 0x7fd48c101d10 2024-04-25T01:58:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd4827fc700 1 -- 172.21.15.33:0/2024297099 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd48800d310 con 0x7fd48c101d10 2024-04-25T01:58:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.291+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd48c10e360 con 0x7fd48c101d10 2024-04-25T01:58:22.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.292+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48c0fd460 con 0x7fd48c101d10 2024-04-25T01:58:22.298 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:21 smithi138 ceph-mon[81304]: pgmap v568: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:22.299 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:21 smithi138 ceph-mon[81304]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:22.299 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:21 smithi138 ceph-mon[81304]: from='client.35076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.293+0000 7fd4827fc700 1 -- 172.21.15.33:0/2024297099 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd488010420 con 0x7fd48c101d10 2024-04-25T01:58:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.293+0000 7fd4827fc700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd47806e140 0x7fd4780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.293+0000 7fd4827fc700 1 -- 172.21.15.33:0/2024297099 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd48808db40 con 0x7fd48c101d10 2024-04-25T01:58:22.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.293+0000 7fd4912e7700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd47806e140 0x7fd4780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:22.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.294+0000 7fd4912e7700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd47806e140 0x7fd4780705f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009fd0 tx=0x7fd47c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:22.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.298+0000 7fd4827fc700 1 -- 172.21.15.33:0/2024297099 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd48805aa70 con 0x7fd48c101d10 2024-04-25T01:58:22.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.528+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd48c10e5e0 con 0x7fd47806e140 2024-04-25T01:58:22.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.529+0000 7fd4827fc700 1 -- 172.21.15.33:0/2024297099 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd48c10e5e0 con 0x7fd47806e140 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:58:22.532 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:58:22.533 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:58:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd47806e140 msgr2=0x7fd4780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd47806e140 0x7fd4780705f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009fd0 tx=0x7fd47c009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c101d10 msgr2=0x7fd48c113a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c101d10 0x7fd48c113a30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fd48800b790 tx=0x7fd488009250 comp rx=0 tx=0).stop 2024-04-25T01:58:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 shutdown_connections 2024-04-25T01:58:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd47806e140 0x7fd4780705f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd48c101d10 0x7fd48c113a30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 --2- 172.21.15.33:0/2024297099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd48c1026b0 0x7fd48c113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 >> 172.21.15.33:0/2024297099 conn(0x7fd48c0f7860 msgr2=0x7fd48c0f8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:22.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 shutdown_connections 2024-04-25T01:58:22.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.532+0000 7fd493d4c700 1 -- 172.21.15.33:0/2024297099 wait complete. 2024-04-25T01:58:22.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.636+0000 7fed6c43c700 1 -- 172.21.15.33:0/616488675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 msgr2=0x7fed64065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.636+0000 7fed6c43c700 1 --2- 172.21.15.33:0/616488675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 0x7fed64065ab0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fed58004940 tx=0x7fed58009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:22.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.636+0000 7fed6c43c700 1 -- 172.21.15.33:0/616488675 shutdown_connections 2024-04-25T01:58:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.636+0000 7fed6c43c700 1 --2- 172.21.15.33:0/616488675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 0x7fed64065ab0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.636+0000 7fed6c43c700 1 --2- 172.21.15.33:0/616488675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed64102e50 0x7fed64103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.636+0000 7fed6c43c700 1 -- 172.21.15.33:0/616488675 >> 172.21.15.33:0/616488675 conn(0x7fed64071d00 msgr2=0x7fed64072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.636+0000 7fed6c43c700 1 -- 172.21.15.33:0/616488675 shutdown_connections 2024-04-25T01:58:22.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.636+0000 7fed6c43c700 1 -- 172.21.15.33:0/616488675 wait complete. 2024-04-25T01:58:22.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6c43c700 1 Processor -- start 2024-04-25T01:58:22.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6c43c700 1 -- start start 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6c43c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 0x7fed64115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6c43c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed64102e50 0x7fed641162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6c43c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed641168f0 con 0x7fed64065640 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6c43c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6410fcc0 con 0x7fed64102e50 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6a1d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 0x7fed64115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6a1d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 0x7fed64115d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37128/0 (socket says 172.21.15.33:37128) 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6a1d8700 1 -- 172.21.15.33:0/744985498 learned_addr learned my addr 172.21.15.33:0/744985498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed699d7700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed64102e50 0x7fed641162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6a1d8700 1 -- 172.21.15.33:0/744985498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed64102e50 msgr2=0x7fed641162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6a1d8700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed64102e50 0x7fed641162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6a1d8700 1 -- 172.21.15.33:0/744985498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed58005040 con 0x7fed64065640 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed699d7700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed64102e50 0x7fed641162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:58:22.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.637+0000 7fed6a1d8700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 0x7fed64115d60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fed5c001d80 tx=0x7fed5c009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:22.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.638+0000 7fed62ffd700 1 -- 172.21.15.33:0/744985498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed5c007d10 con 0x7fed64065640 2024-04-25T01:58:22.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.638+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6410ffa0 con 0x7fed64065640 2024-04-25T01:58:22.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.638+0000 7fed62ffd700 1 -- 172.21.15.33:0/744985498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed5c007e70 con 0x7fed64065640 2024-04-25T01:58:22.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.638+0000 7fed62ffd700 1 -- 172.21.15.33:0/744985498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed5c014640 con 0x7fed64065640 2024-04-25T01:58:22.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.638+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed64110560 con 0x7fed64065640 2024-04-25T01:58:22.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.639+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed64106960 con 0x7fed64065640 2024-04-25T01:58:22.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.639+0000 7fed62ffd700 1 -- 172.21.15.33:0/744985498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fed5c00e960 con 0x7fed64065640 2024-04-25T01:58:22.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.639+0000 7fed62ffd700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed4c06e150 0x7fed4c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:22.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.640+0000 7fed62ffd700 1 -- 172.21.15.33:0/744985498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fed5c08d910 con 0x7fed64065640 2024-04-25T01:58:22.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.640+0000 7fed699d7700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed4c06e150 0x7fed4c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:22.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.640+0000 7fed699d7700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed4c06e150 0x7fed4c070600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fed58004f90 tx=0x7fed58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:22.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.644+0000 7fed62ffd700 1 -- 172.21.15.33:0/744985498 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fed5c05a870 con 0x7fed64065640 2024-04-25T01:58:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.941+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fed640fe540 con 0x7fed64065640 2024-04-25T01:58:22.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.942+0000 7fed62ffd700 1 -- 172.21.15.33:0/744985498 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fed5c05a400 con 0x7fed64065640 2024-04-25T01:58:22.944 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:58:22.944 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:58:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:58:22.944 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:58:22.944 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.943+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed4c06e150 msgr2=0x7fed4c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.943+0000 7fed6c43c700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed4c06e150 0x7fed4c070600 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fed58004f90 tx=0x7fed58009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.943+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 msgr2=0x7fed64115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.943+0000 7fed6c43c700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 0x7fed64115d60 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fed5c001d80 tx=0x7fed5c009ec0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.944+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 shutdown_connections 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.944+0000 7fed6c43c700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fed4c06e150 0x7fed4c070600 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.944+0000 7fed6c43c700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed64065640 0x7fed64115d60 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.944+0000 7fed6c43c700 1 --2- 172.21.15.33:0/744985498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed64102e50 0x7fed641162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.944+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 >> 172.21.15.33:0/744985498 conn(0x7fed64071d00 msgr2=0x7fed640fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.944+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 shutdown_connections 2024-04-25T01:58:22.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:22.944+0000 7fed6c43c700 1 -- 172.21.15.33:0/744985498 wait complete. 2024-04-25T01:58:23.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:22 smithi033 ceph-mon[111306]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:23.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:22 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2518300941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:58:23.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:22 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1717178037' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:58:23.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:22 smithi138 ceph-mon[81304]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:23.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:22 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2518300941' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:58:23.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:22 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1717178037' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:58:24.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:23 smithi033 ceph-mon[111306]: pgmap v569: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:24.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:23 smithi033 ceph-mon[111306]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:24.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:23 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/744985498' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:58:24.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:23 smithi138 ceph-mon[81304]: pgmap v569: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:24.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:23 smithi138 ceph-mon[81304]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:24.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:23 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/744985498' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:58:26.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:25 smithi033 ceph-mon[111306]: pgmap v570: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:26.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:58:26.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:25 smithi138 ceph-mon[81304]: pgmap v570: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:26.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:58:28.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:27 smithi033 ceph-mon[111306]: pgmap v571: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:28.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:27 smithi138 ceph-mon[81304]: pgmap v571: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:30.096 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:29 smithi138 ceph-mon[81304]: pgmap v572: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:30.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:29 smithi033 ceph-mon[111306]: pgmap v572: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:32.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:31 smithi033 ceph-mon[111306]: pgmap v573: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:32.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:31 smithi138 ceph-mon[81304]: pgmap v573: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:34.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:33 smithi033 ceph-mon[111306]: pgmap v574: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:34.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:33 smithi138 ceph-mon[81304]: pgmap v574: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:36.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:35 smithi033 ceph-mon[111306]: pgmap v575: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:36.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:35 smithi138 ceph-mon[81304]: pgmap v575: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:38.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:37 smithi033 ceph-mon[111306]: pgmap v576: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:38.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:37 smithi138 ceph-mon[81304]: pgmap v576: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:40.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:39 smithi033 ceph-mon[111306]: pgmap v577: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:40.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:39 smithi138 ceph-mon[81304]: pgmap v577: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:41.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:58:41.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:58:42.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:41 smithi033 ceph-mon[111306]: pgmap v578: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:42.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:41 smithi138 ceph-mon[81304]: pgmap v578: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:44.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:43 smithi033 ceph-mon[111306]: pgmap v579: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:44.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:43 smithi138 ceph-mon[81304]: pgmap v579: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:46.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:45 smithi033 ceph-mon[111306]: pgmap v580: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:46.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:45 smithi138 ceph-mon[81304]: pgmap v580: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:48.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:47 smithi033 ceph-mon[111306]: pgmap v581: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:48.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:47 smithi138 ceph-mon[81304]: pgmap v581: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:50.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:49 smithi033 ceph-mon[111306]: pgmap v582: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:50.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:49 smithi138 ceph-mon[81304]: pgmap v582: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:52.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:51 smithi033 ceph-mon[111306]: pgmap v583: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:52.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:51 smithi138 ceph-mon[81304]: pgmap v583: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 -- 172.21.15.33:0/3455667825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 msgr2=0x7f26b4106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 --2- 172.21.15.33:0/3455667825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 0x7f26b4106dd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f26a4004940 tx=0x7f26a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 -- 172.21.15.33:0/3455667825 shutdown_connections 2024-04-25T01:58:53.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 --2- 172.21.15.33:0/3455667825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 0x7f26b4106dd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 --2- 172.21.15.33:0/3455667825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b40fddf0 0x7f26b40fe1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 -- 172.21.15.33:0/3455667825 >> 172.21.15.33:0/3455667825 conn(0x7f26b40f7860 msgr2=0x7f26b40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 -- 172.21.15.33:0/3455667825 shutdown_connections 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 -- 172.21.15.33:0/3455667825 wait complete. 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 Processor -- start 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.061+0000 7f26bc2ce700 1 -- start start 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26bc2ce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b40fddf0 0x7f26b41138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26bc2ce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 0x7f26b4113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26bc2ce700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4114460 con 0x7f26b40fe790 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26bc2ce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b410da60 con 0x7f26b40fddf0 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26b9869700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 0x7f26b4113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26b9869700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 0x7f26b4113e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43398/0 (socket says 172.21.15.33:43398) 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26b9869700 1 -- 172.21.15.33:0/2139916256 learned_addr learned my addr 172.21.15.33:0/2139916256 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:53.067 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26ba06a700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b40fddf0 0x7f26b41138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26b9869700 1 -- 172.21.15.33:0/2139916256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b40fddf0 msgr2=0x7f26b41138d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26b9869700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b40fddf0 0x7f26b41138d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26b9869700 1 -- 172.21.15.33:0/2139916256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a4005040 con 0x7f26b40fe790 2024-04-25T01:58:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26b9869700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 0x7f26b4113e10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f26a4004890 tx=0x7f26a4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.062+0000 7f26aaffd700 1 -- 172.21.15.33:0/2139916256 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26a400e040 con 0x7f26b40fe790 2024-04-25T01:58:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.063+0000 7f26aaffd700 1 -- 172.21.15.33:0/2139916256 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f26a4009af0 con 0x7f26b40fe790 2024-04-25T01:58:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.063+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b410dce0 con 0x7f26b40fe790 2024-04-25T01:58:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.063+0000 7f26aaffd700 1 -- 172.21.15.33:0/2139916256 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26a401c3f0 con 0x7f26b40fe790 2024-04-25T01:58:53.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.063+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b410e220 con 0x7f26b40fe790 2024-04-25T01:58:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.064+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b4066bb0 con 0x7f26b40fe790 2024-04-25T01:58:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.064+0000 7f26aaffd700 1 -- 172.21.15.33:0/2139916256 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f26a400baf0 con 0x7f26b40fe790 2024-04-25T01:58:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.065+0000 7f26aaffd700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f26a006e140 0x7f26a00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.065+0000 7f26aaffd700 1 -- 172.21.15.33:0/2139916256 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f26a408edd0 con 0x7f26b40fe790 2024-04-25T01:58:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.065+0000 7f26ba06a700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f26a006e140 0x7f26a00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:53.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.065+0000 7f26ba06a700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f26a006e140 0x7f26a00705f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009fd0 tx=0x7f26b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:53.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.070+0000 7f26aaffd700 1 -- 172.21.15.33:0/2139916256 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f26a405bc80 con 0x7f26b40fe790 2024-04-25T01:58:53.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.296+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26b4109e80 con 0x7f26a006e140 2024-04-25T01:58:53.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.297+0000 7f26aaffd700 1 -- 172.21.15.33:0/2139916256 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f26b4109e80 con 0x7f26a006e140 2024-04-25T01:58:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f26a006e140 msgr2=0x7f26a00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f26a006e140 0x7f26a00705f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009fd0 tx=0x7f26b000b040 comp rx=0 tx=0).stop 2024-04-25T01:58:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 msgr2=0x7f26b4113e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 0x7f26b4113e10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f26a4004890 tx=0x7f26a4009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 shutdown_connections 2024-04-25T01:58:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f26a006e140 0x7f26a00705f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f26b40fddf0 0x7f26b41138d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 --2- 172.21.15.33:0/2139916256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f26b40fe790 0x7f26b4113e10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 >> 172.21.15.33:0/2139916256 conn(0x7f26b40f7860 msgr2=0x7f26b40f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 shutdown_connections 2024-04-25T01:58:53.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.299+0000 7f26bc2ce700 1 -- 172.21.15.33:0/2139916256 wait complete. 2024-04-25T01:58:53.313 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:58:53.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.404+0000 7f419d8e6700 1 -- 172.21.15.33:0/877248631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 msgr2=0x7f41980fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.404+0000 7f419d8e6700 1 --2- 172.21.15.33:0/877248631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 0x7f41980fcdd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f418c004940 tx=0x7f418c009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.404+0000 7f419d8e6700 1 -- 172.21.15.33:0/877248631 shutdown_connections 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.404+0000 7f419d8e6700 1 --2- 172.21.15.33:0/877248631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 0x7f41980fcdd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.404+0000 7f419d8e6700 1 --2- 172.21.15.33:0/877248631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198102dc0 0x7f4198103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.404+0000 7f419d8e6700 1 -- 172.21.15.33:0/877248631 >> 172.21.15.33:0/877248631 conn(0x7f41980f8930 msgr2=0x7f41980fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.404+0000 7f419d8e6700 1 -- 172.21.15.33:0/877248631 shutdown_connections 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.405+0000 7f419d8e6700 1 -- 172.21.15.33:0/877248631 wait complete. 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.405+0000 7f419d8e6700 1 Processor -- start 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.405+0000 7f419d8e6700 1 -- start start 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.405+0000 7f419d8e6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198102dc0 0x7f4198070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:53.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.405+0000 7f419d8e6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 0x7f41980707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:53.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.405+0000 7f419d8e6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4198072240 con 0x7f4198103760 2024-04-25T01:58:53.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.405+0000 7f419d8e6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4198070ce0 con 0x7f4198102dc0 2024-04-25T01:58:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f41967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 0x7f41980707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f41967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 0x7f41980707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43406/0 (socket says 172.21.15.33:43406) 2024-04-25T01:58:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f41967fc700 1 -- 172.21.15.33:0/1966780613 learned_addr learned my addr 172.21.15.33:0/1966780613 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f4196ffd700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198102dc0 0x7f4198070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f41967fc700 1 -- 172.21.15.33:0/1966780613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198102dc0 msgr2=0x7f4198070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f41967fc700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198102dc0 0x7f4198070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f41967fc700 1 -- 172.21.15.33:0/1966780613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f418c005040 con 0x7f4198103760 2024-04-25T01:58:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f41967fc700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 0x7f41980707a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f418c009100 tx=0x7f418c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f417f7fe700 1 -- 172.21.15.33:0/1966780613 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f418c00e040 con 0x7f4198103760 2024-04-25T01:58:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4198070f60 con 0x7f4198103760 2024-04-25T01:58:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f417f7fe700 1 -- 172.21.15.33:0/1966780613 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f418c004290 con 0x7f4198103760 2024-04-25T01:58:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.406+0000 7f417f7fe700 1 -- 172.21.15.33:0/1966780613 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f418c01c3f0 con 0x7f4198103760 2024-04-25T01:58:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.407+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419811a190 con 0x7f4198103760 2024-04-25T01:58:53.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.408+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41981000d0 con 0x7f4198103760 2024-04-25T01:58:53.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.408+0000 7f417f7fe700 1 -- 172.21.15.33:0/1966780613 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f418c013690 con 0x7f4198103760 2024-04-25T01:58:53.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.409+0000 7f417f7fe700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f418006e140 0x7f41800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:53.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.409+0000 7f417f7fe700 1 -- 172.21.15.33:0/1966780613 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f418c08ec40 con 0x7f4198103760 2024-04-25T01:58:53.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.409+0000 7f4196ffd700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f418006e140 0x7f41800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:53.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.409+0000 7f4196ffd700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f418006e140 0x7f41800705f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4188009bf0 tx=0x7f41880091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:53.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.413+0000 7f417f7fe700 1 -- 172.21.15.33:0/1966780613 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f418c058040 con 0x7f4198103760 2024-04-25T01:58:53.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.644+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4198109f10 con 0x7f418006e140 2024-04-25T01:58:53.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.645+0000 7f417f7fe700 1 -- 172.21.15.33:0/1966780613 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4198109f10 con 0x7f418006e140 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f418006e140 msgr2=0x7f41800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f418006e140 0x7f41800705f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4188009bf0 tx=0x7f41880091a0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 msgr2=0x7f41980707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 0x7f41980707a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f418c009100 tx=0x7f418c009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 shutdown_connections 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f418006e140 0x7f41800705f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4198102dc0 0x7f4198070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 --2- 172.21.15.33:0/1966780613 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4198103760 0x7f41980707a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 >> 172.21.15.33:0/1966780613 conn(0x7f41980f8930 msgr2=0x7f41980fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 shutdown_connections 2024-04-25T01:58:53.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.647+0000 7f419d8e6700 1 -- 172.21.15.33:0/1966780613 wait complete. 2024-04-25T01:58:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.750+0000 7f9f4a622700 1 -- 172.21.15.33:0/2011219721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 msgr2=0x7f9f440fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.751+0000 7f9f4a622700 1 --2- 172.21.15.33:0/2011219721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 0x7f9f440fcdf0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c004940 tx=0x7f9f2c009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.751+0000 7f9f4a622700 1 -- 172.21.15.33:0/2011219721 shutdown_connections 2024-04-25T01:58:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.751+0000 7f9f4a622700 1 --2- 172.21.15.33:0/2011219721 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 0x7f9f440fcdf0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.751+0000 7f9f4a622700 1 --2- 172.21.15.33:0/2011219721 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f44102de0 0x7f9f441031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.751+0000 7f9f4a622700 1 -- 172.21.15.33:0/2011219721 >> 172.21.15.33:0/2011219721 conn(0x7f9f440f8990 msgr2=0x7f9f440fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.751+0000 7f9f4a622700 1 -- 172.21.15.33:0/2011219721 shutdown_connections 2024-04-25T01:58:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.751+0000 7f9f4a622700 1 -- 172.21.15.33:0/2011219721 wait complete. 2024-04-25T01:58:53.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.751+0000 7f9f4a622700 1 Processor -- start 2024-04-25T01:58:53.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f4a622700 1 -- start start 2024-04-25T01:58:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f4a622700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f44102de0 0x7f9f44070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f4a622700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 0x7f9f440707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f4a622700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440721b0 con 0x7f9f44103780 2024-04-25T01:58:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f4a622700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44070ce0 con 0x7f9f44102de0 2024-04-25T01:58:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 0x7f9f440707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 0x7f9f440707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43432/0 (socket says 172.21.15.33:43432) 2024-04-25T01:58:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f437fe700 1 -- 172.21.15.33:0/3822519168 learned_addr learned my addr 172.21.15.33:0/3822519168 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:53.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f43fff700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f44102de0 0x7f9f44070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:53.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f437fe700 1 -- 172.21.15.33:0/3822519168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f44102de0 msgr2=0x7f9f44070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f437fe700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f44102de0 0x7f9f44070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f437fe700 1 -- 172.21.15.33:0/3822519168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2c005040 con 0x7f9f44103780 2024-04-25T01:58:53.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.752+0000 7f9f437fe700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 0x7f9f440707a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009100 tx=0x7f9f2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:53.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.753+0000 7f9f40ff9700 1 -- 172.21.15.33:0/3822519168 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f2c00e040 con 0x7f9f44103780 2024-04-25T01:58:53.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.753+0000 7f9f40ff9700 1 -- 172.21.15.33:0/3822519168 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f2c013420 con 0x7f9f44103780 2024-04-25T01:58:53.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.753+0000 7f9f40ff9700 1 -- 172.21.15.33:0/3822519168 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f2c01c630 con 0x7f9f44103780 2024-04-25T01:58:53.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.753+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f44070f60 con 0x7f9f44103780 2024-04-25T01:58:53.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.753+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4411a0e0 con 0x7f9f44103780 2024-04-25T01:58:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.754+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f440fe530 con 0x7f9f44103780 2024-04-25T01:58:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.754+0000 7f9f40ff9700 1 -- 172.21.15.33:0/3822519168 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9f2c013a00 con 0x7f9f44103780 2024-04-25T01:58:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.755+0000 7f9f40ff9700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f3006e140 0x7f9f300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:53.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.755+0000 7f9f40ff9700 1 -- 172.21.15.33:0/3822519168 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9f2c08eb10 con 0x7f9f44103780 2024-04-25T01:58:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.755+0000 7f9f43fff700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f3006e140 0x7f9f300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:53.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.755+0000 7f9f43fff700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f3006e140 0x7f9f300705f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f34007fd0 tx=0x7f9f34009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:53.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.760+0000 7f9f40ff9700 1 -- 172.21.15.33:0/3822519168 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9f2c057fb0 con 0x7f9f44103780 2024-04-25T01:58:53.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.977+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f44109ec0 con 0x7f9f3006e140 2024-04-25T01:58:53.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.987+0000 7f9f40ff9700 1 -- 172.21.15.33:0/3822519168 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9f44109ec0 con 0x7f9f3006e140 2024-04-25T01:58:53.991 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:58:53.991 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (20m) 13m ago 31m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:58:53.991 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (17m) 13m ago 31m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:58:53.991 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (17m) 12m ago 30m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:58:53.991 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (19m) 13m ago 30m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:58:53.991 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (13m) 13m ago 25m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:58:53.991 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (13m) 13m ago 25m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (12m) 12m ago 25m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (25m) 12m ago 25m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (21m) 13m ago 32m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (20m) 12m ago 30m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (18m) 13m ago 32m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (17m) 12m ago 30m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (20m) 13m ago 31m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (20m) 12m ago 30m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (16m) 13m ago 29m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (16m) 13m ago 29m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (15m) 13m ago 28m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (15m) 12m ago 28m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (14m) 12m ago 27m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (13m) 12m ago 27m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:58:53.992 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (20m) 13m ago 30m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:58:53.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.989+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f3006e140 msgr2=0x7f9f300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.989+0000 7f9f4a622700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f3006e140 0x7f9f300705f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9f34007fd0 tx=0x7f9f34009040 comp rx=0 tx=0).stop 2024-04-25T01:58:53.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.989+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 msgr2=0x7f9f440707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:53.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.989+0000 7f9f4a622700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 0x7f9f440707a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c009100 tx=0x7f9f2c009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:53.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.989+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 shutdown_connections 2024-04-25T01:58:53.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.989+0000 7f9f4a622700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9f3006e140 0x7f9f300705f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.989+0000 7f9f4a622700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9f44102de0 0x7f9f44070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.989+0000 7f9f4a622700 1 --2- 172.21.15.33:0/3822519168 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9f44103780 0x7f9f440707a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:53.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.989+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 >> 172.21.15.33:0/3822519168 conn(0x7f9f440f8990 msgr2=0x7f9f440f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:53.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.990+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 shutdown_connections 2024-04-25T01:58:53.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:53.990+0000 7f9f4a622700 1 -- 172.21.15.33:0/3822519168 wait complete. 2024-04-25T01:58:54.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:53 smithi033 ceph-mon[111306]: pgmap v584: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 -- 172.21.15.33:0/118266171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 msgr2=0x7fcd000fe130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 --2- 172.21.15.33:0/118266171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 0x7fcd000fe130 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fccfc004940 tx=0x7fccfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 -- 172.21.15.33:0/118266171 shutdown_connections 2024-04-25T01:58:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 --2- 172.21.15.33:0/118266171 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd000fb520 0x7fcd000fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 --2- 172.21.15.33:0/118266171 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 0x7fcd000fe130 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 -- 172.21.15.33:0/118266171 >> 172.21.15.33:0/118266171 conn(0x7fcd000714d0 msgr2=0x7fcd000718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 -- 172.21.15.33:0/118266171 shutdown_connections 2024-04-25T01:58:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 -- 172.21.15.33:0/118266171 wait complete. 2024-04-25T01:58:54.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 Processor -- start 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.093+0000 7fcd0799c700 1 -- start start 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd0799c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd000fb520 0x7fcd00115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd0799c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 0x7fcd001160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd0799c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd00116700 con 0x7fcd000fdd60 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd0799c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0010fce0 con 0x7fcd000fb520 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd04f37700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 0x7fcd001160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd04f37700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 0x7fcd001160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43452/0 (socket says 172.21.15.33:43452) 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd04f37700 1 -- 172.21.15.33:0/999012156 learned_addr learned my addr 172.21.15.33:0/999012156 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd05738700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd000fb520 0x7fcd00115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd04f37700 1 -- 172.21.15.33:0/999012156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd000fb520 msgr2=0x7fcd00115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd04f37700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd000fb520 0x7fcd00115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd04f37700 1 -- 172.21.15.33:0/999012156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccfc005040 con 0x7fcd000fdd60 2024-04-25T01:58:54.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fcd04f37700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 0x7fcd001160b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fccf4002a50 tx=0x7fccf40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.094+0000 7fccf1ffb700 1 -- 172.21.15.33:0/999012156 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf4007cc0 con 0x7fcd000fdd60 2024-04-25T01:58:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.095+0000 7fccf1ffb700 1 -- 172.21.15.33:0/999012156 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccf400f040 con 0x7fcd000fdd60 2024-04-25T01:58:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.095+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0010ffc0 con 0x7fcd000fdd60 2024-04-25T01:58:54.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.095+0000 7fccf1ffb700 1 -- 172.21.15.33:0/999012156 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf400da30 con 0x7fcd000fdd60 2024-04-25T01:58:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.095+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd00110500 con 0x7fcd000fdd60 2024-04-25T01:58:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.096+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd001073d0 con 0x7fcd000fdd60 2024-04-25T01:58:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.096+0000 7fccf1ffb700 1 -- 172.21.15.33:0/999012156 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fccf400db90 con 0x7fcd000fdd60 2024-04-25T01:58:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.097+0000 7fccf1ffb700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccec06e140 0x7fccec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.097+0000 7fccf1ffb700 1 -- 172.21.15.33:0/999012156 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fccf408d8c0 con 0x7fcd000fdd60 2024-04-25T01:58:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.097+0000 7fcd05738700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccec06e140 0x7fccec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:54.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.097+0000 7fcd05738700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccec06e140 0x7fccec0705f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00b3b0 tx=0x7fccfc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:54.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.101+0000 7fccf1ffb700 1 -- 172.21.15.33:0/999012156 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fccf405a770 con 0x7fcd000fdd60 2024-04-25T01:58:54.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:53 smithi138 ceph-mon[81304]: pgmap v584: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.398+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcd001111c0 con 0x7fcd000fdd60 2024-04-25T01:58:54.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.399+0000 7fccf1ffb700 1 -- 172.21.15.33:0/999012156 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fccf405a300 con 0x7fcd000fdd60 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:58:54.403 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:58:54.404 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:58:54.404 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:58:54.404 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:58:54.404 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:58:54.404 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:58:54.404 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:58:54.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccec06e140 msgr2=0x7fccec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccec06e140 0x7fccec0705f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00b3b0 tx=0x7fccfc009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 msgr2=0x7fcd001160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 0x7fcd001160b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fccf4002a50 tx=0x7fccf40046b0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 shutdown_connections 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fccec06e140 0x7fccec0705f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcd000fb520 0x7fcd00115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 --2- 172.21.15.33:0/999012156 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcd000fdd60 0x7fcd001160b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 >> 172.21.15.33:0/999012156 conn(0x7fcd000714d0 msgr2=0x7fcd000f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 shutdown_connections 2024-04-25T01:58:54.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.401+0000 7fcd0799c700 1 -- 172.21.15.33:0/999012156 wait complete. 2024-04-25T01:58:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.506+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/2021863714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24102e50 msgr2=0x7f9d24103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.506+0000 7f9d2a1ce700 1 --2- 172.21.15.33:0/2021863714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24102e50 0x7f9d24103220 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f9d14004940 tx=0x7f9d14009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.506+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/2021863714 shutdown_connections 2024-04-25T01:58:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.506+0000 7f9d2a1ce700 1 --2- 172.21.15.33:0/2021863714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d24065640 0x7f9d24065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.506+0000 7f9d2a1ce700 1 --2- 172.21.15.33:0/2021863714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24102e50 0x7f9d24103220 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.506+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/2021863714 >> 172.21.15.33:0/2021863714 conn(0x7f9d24071d00 msgr2=0x7f9d24072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.506+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/2021863714 shutdown_connections 2024-04-25T01:58:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.506+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/2021863714 wait complete. 2024-04-25T01:58:54.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d2a1ce700 1 Processor -- start 2024-04-25T01:58:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d2a1ce700 1 -- start start 2024-04-25T01:58:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d2a1ce700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24065640 0x7f9d24113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d2a1ce700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d24102e50 0x7f9d24114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d2a1ce700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24114670 con 0x7f9d24065640 2024-04-25T01:58:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d2a1ce700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2410da60 con 0x7f9d24102e50 2024-04-25T01:58:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24065640 0x7f9d24113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24065640 0x7f9d24113ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43468/0 (socket says 172.21.15.33:43468) 2024-04-25T01:58:54.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d237fe700 1 -- 172.21.15.33:0/1540468796 learned_addr learned my addr 172.21.15.33:0/1540468796 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d22ffd700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d24102e50 0x7f9d24114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d237fe700 1 -- 172.21.15.33:0/1540468796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d24102e50 msgr2=0x7f9d24114020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d237fe700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d24102e50 0x7f9d24114020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.507+0000 7f9d237fe700 1 -- 172.21.15.33:0/1540468796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14005040 con 0x7f9d24065640 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.508+0000 7f9d237fe700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24065640 0x7f9d24113ae0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009380 tx=0x7f9d14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.508+0000 7f9d0bfff700 1 -- 172.21.15.33:0/1540468796 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d14021070 con 0x7f9d24065640 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.508+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2410dce0 con 0x7f9d24065640 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.508+0000 7f9d0bfff700 1 -- 172.21.15.33:0/1540468796 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d14013d50 con 0x7f9d24065640 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.508+0000 7f9d0bfff700 1 -- 172.21.15.33:0/1540468796 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d1400e070 con 0x7f9d24065640 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.508+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2410e1c0 con 0x7f9d24065640 2024-04-25T01:58:54.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.509+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d24100100 con 0x7f9d24065640 2024-04-25T01:58:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.509+0000 7f9d0bfff700 1 -- 172.21.15.33:0/1540468796 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9d1401cbd0 con 0x7f9d24065640 2024-04-25T01:58:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.510+0000 7f9d0bfff700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9d0c06e140 0x7f9d0c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.510+0000 7f9d0bfff700 1 -- 172.21.15.33:0/1540468796 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9d1408f020 con 0x7f9d24065640 2024-04-25T01:58:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.510+0000 7f9d22ffd700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9d0c06e140 0x7f9d0c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:54.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.510+0000 7f9d22ffd700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9d0c06e140 0x7f9d0c0705f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9d18004600 tx=0x7f9d1800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:54.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.515+0000 7f9d0bfff700 1 -- 172.21.15.33:0/1540468796 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9d1405bf50 con 0x7f9d24065640 2024-04-25T01:58:54.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.766+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9d2410f130 con 0x7f9d24065640 2024-04-25T01:58:54.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.766+0000 7f9d0bfff700 1 -- 172.21.15.33:0/1540468796 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f9d1405bae0 con 0x7f9d24065640 2024-04-25T01:58:54.770 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:58:54.770 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:58:54.770 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:58:54.770 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:58:54.770 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:58:54.770 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:58:54.770 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:58:54.771 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:58:54.772 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:58:54.773 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:58:54.773 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:58:54.773 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:58:54.773 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:58:54.773 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:58:54.773 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:58:54.773 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:58:54.773 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:58:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.768+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9d0c06e140 msgr2=0x7f9d0c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.768+0000 7f9d2a1ce700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9d0c06e140 0x7f9d0c0705f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9d18004600 tx=0x7f9d1800b040 comp rx=0 tx=0).stop 2024-04-25T01:58:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.768+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24065640 msgr2=0x7f9d24113ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.768+0000 7f9d2a1ce700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24065640 0x7f9d24113ae0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009380 tx=0x7f9d14009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.768+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 shutdown_connections 2024-04-25T01:58:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.769+0000 7f9d2a1ce700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9d0c06e140 0x7f9d0c0705f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.769+0000 7f9d2a1ce700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9d24065640 0x7f9d24113ae0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.769+0000 7f9d2a1ce700 1 --2- 172.21.15.33:0/1540468796 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9d24102e50 0x7f9d24114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.769+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 >> 172.21.15.33:0/1540468796 conn(0x7f9d24071d00 msgr2=0x7f9d240faf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.769+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 shutdown_connections 2024-04-25T01:58:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.769+0000 7f9d2a1ce700 1 -- 172.21.15.33:0/1540468796 wait complete. 2024-04-25T01:58:54.775 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:58:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.873+0000 7f244a1b8700 1 -- 172.21.15.33:0/3341868447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 msgr2=0x7f24441031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.873+0000 7f244a1b8700 1 --2- 172.21.15.33:0/3341868447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 0x7f24441031b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2434009ce0 tx=0x7f2434004d10 comp rx=0 tx=0).stop 2024-04-25T01:58:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.873+0000 7f244a1b8700 1 -- 172.21.15.33:0/3341868447 shutdown_connections 2024-04-25T01:58:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.873+0000 7f244a1b8700 1 --2- 172.21.15.33:0/3341868447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2444103780 0x7f24440fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.873+0000 7f244a1b8700 1 --2- 172.21.15.33:0/3341868447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 0x7f24441031b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.873+0000 7f244a1b8700 1 -- 172.21.15.33:0/3341868447 >> 172.21.15.33:0/3341868447 conn(0x7f24440f8990 msgr2=0x7f24440fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.873+0000 7f244a1b8700 1 -- 172.21.15.33:0/3341868447 shutdown_connections 2024-04-25T01:58:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.873+0000 7f244a1b8700 1 -- 172.21.15.33:0/3341868447 wait complete. 2024-04-25T01:58:54.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.874+0000 7f244a1b8700 1 Processor -- start 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.874+0000 7f244a1b8700 1 -- start start 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.874+0000 7f244a1b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 0x7f2444115c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.874+0000 7f244a1b8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2444103780 0x7f2444116160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.874+0000 7f244a1b8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24441167b0 con 0x7f2444102de0 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.874+0000 7f244a1b8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244410fde0 con 0x7f2444103780 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.874+0000 7f24437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 0x7f2444115c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.874+0000 7f24437fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 0x7f2444115c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43490/0 (socket says 172.21.15.33:43490) 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.874+0000 7f24437fe700 1 -- 172.21.15.33:0/681164162 learned_addr learned my addr 172.21.15.33:0/681164162 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f2442ffd700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2444103780 0x7f2444116160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:54.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f24437fe700 1 -- 172.21.15.33:0/681164162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2444103780 msgr2=0x7f2444116160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f24437fe700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2444103780 0x7f2444116160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f24437fe700 1 -- 172.21.15.33:0/681164162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2434009940 con 0x7f2444102de0 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f2442ffd700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2444103780 0x7f2444116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f24437fe700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 0x7f2444115c20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2434003400 tx=0x7f2434004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f242bfff700 1 -- 172.21.15.33:0/681164162 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f243400e040 con 0x7f2444102de0 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2444110060 con 0x7f2444102de0 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f242bfff700 1 -- 172.21.15.33:0/681164162 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2434004610 con 0x7f2444102de0 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f242bfff700 1 -- 172.21.15.33:0/681164162 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f243401c6d0 con 0x7f2444102de0 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.875+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2444110620 con 0x7f2444102de0 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.876+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2444106940 con 0x7f2444102de0 2024-04-25T01:58:54.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.877+0000 7f242bfff700 1 -- 172.21.15.33:0/681164162 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2434012b00 con 0x7f2444102de0 2024-04-25T01:58:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.877+0000 7f242bfff700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f242c06e140 0x7f242c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.877+0000 7f242bfff700 1 -- 172.21.15.33:0/681164162 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f243408f5a0 con 0x7f2444102de0 2024-04-25T01:58:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.877+0000 7f2442ffd700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f242c06e140 0x7f242c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:54.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.878+0000 7f2442ffd700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f242c06e140 0x7f242c0705f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2438009b80 tx=0x7f243800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:54.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:54.882+0000 7f242bfff700 1 -- 172.21.15.33:0/681164162 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f243405c450 con 0x7f2444102de0 2024-04-25T01:58:55.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.108+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2444109f50 con 0x7f242c06e140 2024-04-25T01:58:55.112 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:54 smithi033 ceph-mon[111306]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:55.112 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:54 smithi033 ceph-mon[111306]: from='client.35104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:55.112 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:54 smithi033 ceph-mon[111306]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:55.112 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:54 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/999012156' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:58:55.113 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:54 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1540468796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:58:55.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.110+0000 7f242bfff700 1 -- 172.21.15.33:0/681164162 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2444109f50 con 0x7f242c06e140 2024-04-25T01:58:55.113 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:58:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:58:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:58:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:58:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:58:55.113 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:58:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:58:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:58:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:58:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:58:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:58:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:58:55.114 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:58:55.114 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:58:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.111+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f242c06e140 msgr2=0x7f242c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.111+0000 7f244a1b8700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f242c06e140 0x7f242c0705f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2438009b80 tx=0x7f243800b040 comp rx=0 tx=0).stop 2024-04-25T01:58:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.111+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 msgr2=0x7f2444115c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.111+0000 7f244a1b8700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 0x7f2444115c20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2434003400 tx=0x7f2434004d10 comp rx=0 tx=0).stop 2024-04-25T01:58:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.112+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 shutdown_connections 2024-04-25T01:58:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.112+0000 7f244a1b8700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f242c06e140 0x7f242c0705f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.112+0000 7f244a1b8700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2444102de0 0x7f2444115c20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.112+0000 7f244a1b8700 1 --2- 172.21.15.33:0/681164162 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2444103780 0x7f2444116160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:55.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.112+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 >> 172.21.15.33:0/681164162 conn(0x7f24440f8990 msgr2=0x7f24440fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:55.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.112+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 shutdown_connections 2024-04-25T01:58:55.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.112+0000 7f244a1b8700 1 -- 172.21.15.33:0/681164162 wait complete. 2024-04-25T01:58:55.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.218+0000 7f1ac26ea700 1 -- 172.21.15.33:0/563801130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 msgr2=0x7f1abc070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:55.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.218+0000 7f1ac26ea700 1 --2- 172.21.15.33:0/563801130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 0x7f1abc070650 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1aac004940 tx=0x7f1aac009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:55.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.218+0000 7f1ac26ea700 1 -- 172.21.15.33:0/563801130 shutdown_connections 2024-04-25T01:58:55.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.218+0000 7f1ac26ea700 1 --2- 172.21.15.33:0/563801130 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1abc070b90 0x7f1abc109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:55.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.218+0000 7f1ac26ea700 1 --2- 172.21.15.33:0/563801130 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 0x7f1abc070650 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:55.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.218+0000 7f1ac26ea700 1 -- 172.21.15.33:0/563801130 >> 172.21.15.33:0/563801130 conn(0x7f1abc0f89d0 msgr2=0x7f1abc0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:55.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.218+0000 7f1ac26ea700 1 -- 172.21.15.33:0/563801130 shutdown_connections 2024-04-25T01:58:55.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.218+0000 7f1ac26ea700 1 -- 172.21.15.33:0/563801130 wait complete. 2024-04-25T01:58:55.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1ac26ea700 1 Processor -- start 2024-04-25T01:58:55.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1ac26ea700 1 -- start start 2024-04-25T01:58:55.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1ac26ea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 0x7f1abc115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:55.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1ac26ea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1abc070b90 0x7f1abc1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:55.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1ac26ea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc116720 con 0x7f1abc070280 2024-04-25T01:58:55.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1ac26ea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc10fd00 con 0x7f1abc070b90 2024-04-25T01:58:55.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 0x7f1abc115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:55.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1abbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 0x7f1abc115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43516/0 (socket says 172.21.15.33:43516) 2024-04-25T01:58:55.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1abbfff700 1 -- 172.21.15.33:0/1952275893 learned_addr learned my addr 172.21.15.33:0/1952275893 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:58:55.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1abb7fe700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1abc070b90 0x7f1abc1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.219+0000 7f1abbfff700 1 -- 172.21.15.33:0/1952275893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1abc070b90 msgr2=0x7f1abc1160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.220+0000 7f1abbfff700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1abc070b90 0x7f1abc1160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.220+0000 7f1abbfff700 1 -- 172.21.15.33:0/1952275893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac005040 con 0x7f1abc070280 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.220+0000 7f1abbfff700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 0x7f1abc115b90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00b3b0 tx=0x7f1aac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.220+0000 7f1ab8ff9700 1 -- 172.21.15.33:0/1952275893 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aac00e040 con 0x7f1abc070280 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.220+0000 7f1ab8ff9700 1 -- 172.21.15.33:0/1952275893 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1aac013420 con 0x7f1abc070280 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.220+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc10ff80 con 0x7f1abc070280 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.220+0000 7f1ab8ff9700 1 -- 172.21.15.33:0/1952275893 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aac01c630 con 0x7f1abc070280 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.220+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc1104c0 con 0x7f1abc070280 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.221+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc0719c0 con 0x7f1abc070280 2024-04-25T01:58:55.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.222+0000 7f1ab8ff9700 1 -- 172.21.15.33:0/1952275893 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1aac013590 con 0x7f1abc070280 2024-04-25T01:58:55.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.222+0000 7f1ab8ff9700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1aa406e140 0x7f1aa40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:58:55.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.222+0000 7f1ab8ff9700 1 -- 172.21.15.33:0/1952275893 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1aac08ec30 con 0x7f1abc070280 2024-04-25T01:58:55.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.222+0000 7f1abb7fe700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1aa406e140 0x7f1aa40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:58:55.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.222+0000 7f1abb7fe700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1aa406e140 0x7f1aa40705f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1abc111430 tx=0x7f1ab000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:58:55.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.227+0000 7f1ab8ff9700 1 -- 172.21.15.33:0/1952275893 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1aac05bc00 con 0x7f1abc070280 2024-04-25T01:58:55.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:54 smithi138 ceph-mon[81304]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:55.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:54 smithi138 ceph-mon[81304]: from='client.35104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:55.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:54 smithi138 ceph-mon[81304]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:55.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:54 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/999012156' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:58:55.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:54 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1540468796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:58:55.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.520+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1abc1111e0 con 0x7f1abc070280 2024-04-25T01:58:55.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.521+0000 7f1ab8ff9700 1 -- 172.21.15.33:0/1952275893 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f1aac05b790 con 0x7f1abc070280 2024-04-25T01:58:55.524 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:58:55.525 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:58:55.525 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:58:55.525 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:58:55.525 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:58:55.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1aa406e140 msgr2=0x7f1aa40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:55.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1aa406e140 0x7f1aa40705f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1abc111430 tx=0x7f1ab000b040 comp rx=0 tx=0).stop 2024-04-25T01:58:55.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 msgr2=0x7f1abc115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:58:55.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 0x7f1abc115b90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00b3b0 tx=0x7f1aac009e30 comp rx=0 tx=0).stop 2024-04-25T01:58:55.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 shutdown_connections 2024-04-25T01:58:55.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1aa406e140 0x7f1aa40705f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:55.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1abc070280 0x7f1abc115b90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:55.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 --2- 172.21.15.33:0/1952275893 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1abc070b90 0x7f1abc1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:58:55.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 >> 172.21.15.33:0/1952275893 conn(0x7f1abc0f89d0 msgr2=0x7f1abc0f9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:58:55.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 shutdown_connections 2024-04-25T01:58:55.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:58:55.523+0000 7f1ac26ea700 1 -- 172.21.15.33:0/1952275893 wait complete. 2024-04-25T01:58:56.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:55 smithi033 ceph-mon[111306]: pgmap v585: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:56.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:55 smithi033 ceph-mon[111306]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:56.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:58:56.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:55 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1952275893' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:58:56.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:55 smithi138 ceph-mon[81304]: pgmap v585: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:56.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:55 smithi138 ceph-mon[81304]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:58:56.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:58:56.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:55 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1952275893' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:58:58.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:57 smithi033 ceph-mon[111306]: pgmap v586: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:58:58.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:57 smithi138 ceph-mon[81304]: pgmap v586: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:00.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:58:59 smithi033 ceph-mon[111306]: pgmap v587: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:00.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:58:59 smithi138 ceph-mon[81304]: pgmap v587: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:02.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:01 smithi033 ceph-mon[111306]: pgmap v588: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:02.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:01 smithi138 ceph-mon[81304]: pgmap v588: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:04.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:03 smithi033 ceph-mon[111306]: pgmap v589: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:04.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:03 smithi138 ceph-mon[81304]: pgmap v589: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:06.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:05 smithi033 ceph-mon[111306]: pgmap v590: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:06.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:05 smithi138 ceph-mon[81304]: pgmap v590: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:08.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:07 smithi033 ceph-mon[111306]: pgmap v591: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:08.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:07 smithi138 ceph-mon[81304]: pgmap v591: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:10.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:09 smithi033 ceph-mon[111306]: pgmap v592: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:10.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:09 smithi138 ceph-mon[81304]: pgmap v592: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:11.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:59:11.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:59:12.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:11 smithi033 ceph-mon[111306]: pgmap v593: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:12.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:11 smithi138 ceph-mon[81304]: pgmap v593: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:14.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:13 smithi033 ceph-mon[111306]: pgmap v594: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:14.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:13 smithi138 ceph-mon[81304]: pgmap v594: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:16.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:15 smithi033 ceph-mon[111306]: pgmap v595: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:16.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:15 smithi138 ceph-mon[81304]: pgmap v595: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:18.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:17 smithi033 ceph-mon[111306]: pgmap v596: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:18.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:17 smithi138 ceph-mon[81304]: pgmap v596: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:20.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:19 smithi033 ceph-mon[111306]: pgmap v597: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:20.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:19 smithi138 ceph-mon[81304]: pgmap v597: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:22.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:21 smithi033 ceph-mon[111306]: pgmap v598: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:22.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:21 smithi138 ceph-mon[81304]: pgmap v598: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:24.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:23 smithi033 ceph-mon[111306]: pgmap v599: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:24.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:23 smithi138 ceph-mon[81304]: pgmap v599: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.642+0000 7fa476ee6700 1 -- 172.21.15.33:0/2408203472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 msgr2=0x7fa4700fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.642+0000 7fa476ee6700 1 --2- 172.21.15.33:0/2408203472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 0x7fa4700fb3c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa45c004940 tx=0x7fa45c009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.642+0000 7fa476ee6700 1 -- 172.21.15.33:0/2408203472 shutdown_connections 2024-04-25T01:59:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.642+0000 7fa476ee6700 1 --2- 172.21.15.33:0/2408203472 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 0x7fa4700fb3c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.642+0000 7fa476ee6700 1 --2- 172.21.15.33:0/2408203472 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa470102e20 0x7fa4701031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.642+0000 7fa476ee6700 1 -- 172.21.15.33:0/2408203472 >> 172.21.15.33:0/2408203472 conn(0x7fa470072380 msgr2=0x7fa470072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:25.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.642+0000 7fa476ee6700 1 -- 172.21.15.33:0/2408203472 shutdown_connections 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.642+0000 7fa476ee6700 1 -- 172.21.15.33:0/2408203472 wait complete. 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.642+0000 7fa476ee6700 1 Processor -- start 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.643+0000 7fa476ee6700 1 -- start start 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.643+0000 7fa476ee6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa470102e20 0x7fa470113a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.643+0000 7fa476ee6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 0x7fa470113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.643+0000 7fa476ee6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4701145c0 con 0x7fa4701037c0 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.643+0000 7fa476ee6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa47010dbe0 con 0x7fa470102e20 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.643+0000 7fa46ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 0x7fa470113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.643+0000 7fa46ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 0x7fa470113f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44708/0 (socket says 172.21.15.33:44708) 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.643+0000 7fa46ffff700 1 -- 172.21.15.33:0/2372844269 learned_addr learned my addr 172.21.15.33:0/2372844269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.643+0000 7fa474c82700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa470102e20 0x7fa470113a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.644+0000 7fa46ffff700 1 -- 172.21.15.33:0/2372844269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa470102e20 msgr2=0x7fa470113a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.644+0000 7fa46ffff700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa470102e20 0x7fa470113a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:25.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.644+0000 7fa46ffff700 1 -- 172.21.15.33:0/2372844269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45c005040 con 0x7fa4701037c0 2024-04-25T01:59:25.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.644+0000 7fa46ffff700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 0x7fa470113f70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009180 tx=0x7fa45c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:25.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.644+0000 7fa46d7fa700 1 -- 172.21.15.33:0/2372844269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa45c00e040 con 0x7fa4701037c0 2024-04-25T01:59:25.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.644+0000 7fa46d7fa700 1 -- 172.21.15.33:0/2372844269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa45c013420 con 0x7fa4701037c0 2024-04-25T01:59:25.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.644+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47010de60 con 0x7fa4701037c0 2024-04-25T01:59:25.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.644+0000 7fa46d7fa700 1 -- 172.21.15.33:0/2372844269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa45c01c610 con 0x7fa4701037c0 2024-04-25T01:59:25.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.644+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa47010e3a0 con 0x7fa4701037c0 2024-04-25T01:59:25.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.645+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa470100160 con 0x7fa4701037c0 2024-04-25T01:59:25.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.646+0000 7fa46d7fa700 1 -- 172.21.15.33:0/2372844269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa45c013a00 con 0x7fa4701037c0 2024-04-25T01:59:25.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.646+0000 7fa46d7fa700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa46006e140 0x7fa4600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:25.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.646+0000 7fa46d7fa700 1 -- 172.21.15.33:0/2372844269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa45c08ec10 con 0x7fa4701037c0 2024-04-25T01:59:25.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.646+0000 7fa474c82700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa46006e140 0x7fa4600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:25.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.647+0000 7fa474c82700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa46006e140 0x7fa4600705f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa470065e90 tx=0x7fa464009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:25.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.651+0000 7fa46d7fa700 1 -- 172.21.15.33:0/2372844269 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa45c0580c0 con 0x7fa4701037c0 2024-04-25T01:59:25.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.878+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa47004c7b0 con 0x7fa46006e140 2024-04-25T01:59:25.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.879+0000 7fa46d7fa700 1 -- 172.21.15.33:0/2372844269 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa47004c7b0 con 0x7fa46006e140 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa46006e140 msgr2=0x7fa4600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa46006e140 0x7fa4600705f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fa470065e90 tx=0x7fa464009040 comp rx=0 tx=0).stop 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 msgr2=0x7fa470113f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 0x7fa470113f70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009180 tx=0x7fa45c009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 shutdown_connections 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa46006e140 0x7fa4600705f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa470102e20 0x7fa470113a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 --2- 172.21.15.33:0/2372844269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa4701037c0 0x7fa470113f70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 >> 172.21.15.33:0/2372844269 conn(0x7fa470072380 msgr2=0x7fa4700fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 shutdown_connections 2024-04-25T01:59:25.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.881+0000 7fa476ee6700 1 -- 172.21.15.33:0/2372844269 wait complete. 2024-04-25T01:59:25.895 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:59:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.988+0000 7f8b8e46c700 1 -- 172.21.15.33:0/1540895001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 msgr2=0x7f8b880fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.988+0000 7f8b8e46c700 1 --2- 172.21.15.33:0/1540895001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 0x7f8b880fc1b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8b70004940 tx=0x7f8b70009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.988+0000 7f8b8e46c700 1 -- 172.21.15.33:0/1540895001 shutdown_connections 2024-04-25T01:59:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.988+0000 7f8b8e46c700 1 --2- 172.21.15.33:0/1540895001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 0x7f8b880fc1b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.988+0000 7f8b8e46c700 1 --2- 172.21.15.33:0/1540895001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b88101d90 0x7f8b88102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:25.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.988+0000 7f8b8e46c700 1 -- 172.21.15.33:0/1540895001 >> 172.21.15.33:0/1540895001 conn(0x7f8b88071d00 msgr2=0x7f8b88072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.988+0000 7f8b8e46c700 1 -- 172.21.15.33:0/1540895001 shutdown_connections 2024-04-25T01:59:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.988+0000 7f8b8e46c700 1 -- 172.21.15.33:0/1540895001 wait complete. 2024-04-25T01:59:25.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b8e46c700 1 Processor -- start 2024-04-25T01:59:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b8e46c700 1 -- start start 2024-04-25T01:59:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b8e46c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 0x7f8b8810eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b8e46c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b88101d90 0x7f8b8810f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b8e46c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8810f6b0 con 0x7f8b880fbd40 2024-04-25T01:59:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b8e46c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8810f7f0 con 0x7f8b88101d90 2024-04-25T01:59:25.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 0x7f8b8810eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b87fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 0x7f8b8810eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38352/0 (socket says 172.21.15.33:38352) 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b87fff700 1 -- 172.21.15.33:0/2503298324 learned_addr learned my addr 172.21.15.33:0/2503298324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.989+0000 7f8b877fe700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b88101d90 0x7f8b8810f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.990+0000 7f8b87fff700 1 -- 172.21.15.33:0/2503298324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b88101d90 msgr2=0x7f8b8810f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.990+0000 7f8b87fff700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b88101d90 0x7f8b8810f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.990+0000 7f8b87fff700 1 -- 172.21.15.33:0/2503298324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70005040 con 0x7f8b880fbd40 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.990+0000 7f8b87fff700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 0x7f8b8810eb20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800eb40 tx=0x7f8b7800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.990+0000 7f8b84ff9700 1 -- 172.21.15.33:0/2503298324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b78007cc0 con 0x7f8b880fbd40 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.990+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b88117b90 con 0x7f8b880fbd40 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.990+0000 7f8b84ff9700 1 -- 172.21.15.33:0/2503298324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b7800f040 con 0x7f8b880fbd40 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.990+0000 7f8b84ff9700 1 -- 172.21.15.33:0/2503298324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b7800ca10 con 0x7f8b880fbd40 2024-04-25T01:59:25.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.990+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b88117f90 con 0x7f8b880fbd40 2024-04-25T01:59:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.991+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b880fd430 con 0x7f8b880fbd40 2024-04-25T01:59:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.991+0000 7f8b84ff9700 1 -- 172.21.15.33:0/2503298324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8b7800cc80 con 0x7f8b880fbd40 2024-04-25T01:59:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.992+0000 7f8b84ff9700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8b7406e140 0x7f8b740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.992+0000 7f8b84ff9700 1 -- 172.21.15.33:0/2503298324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8b7808d700 con 0x7f8b880fbd40 2024-04-25T01:59:25.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.992+0000 7f8b877fe700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8b7406e140 0x7f8b740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:25.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.992+0000 7f8b877fe700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8b7406e140 0x7f8b740705f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009b30 tx=0x7f8b70009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:26.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:25.997+0000 7f8b84ff9700 1 -- 172.21.15.33:0/2503298324 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8b78056b30 con 0x7f8b880fbd40 2024-04-25T01:59:26.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.224+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b88065de0 con 0x7f8b7406e140 2024-04-25T01:59:26.228 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:25 smithi033 ceph-mon[111306]: pgmap v600: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:26.228 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:59:26.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.225+0000 7f8b84ff9700 1 -- 172.21.15.33:0/2503298324 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8b88065de0 con 0x7f8b7406e140 2024-04-25T01:59:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.227+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8b7406e140 msgr2=0x7f8b740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.227+0000 7f8b8e46c700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8b7406e140 0x7f8b740705f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009b30 tx=0x7f8b70009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:26.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.227+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 msgr2=0x7f8b8810eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.227+0000 7f8b8e46c700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 0x7f8b8810eb20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8b7800eb40 tx=0x7f8b7800b040 comp rx=0 tx=0).stop 2024-04-25T01:59:26.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.227+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 shutdown_connections 2024-04-25T01:59:26.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.227+0000 7f8b8e46c700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8b7406e140 0x7f8b740705f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.227+0000 7f8b8e46c700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8b880fbd40 0x7f8b8810eb20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.227+0000 7f8b8e46c700 1 --2- 172.21.15.33:0/2503298324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8b88101d90 0x7f8b8810f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.227+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 >> 172.21.15.33:0/2503298324 conn(0x7f8b88071d00 msgr2=0x7f8b880f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:26.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.228+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 shutdown_connections 2024-04-25T01:59:26.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.228+0000 7f8b8e46c700 1 -- 172.21.15.33:0/2503298324 wait complete. 2024-04-25T01:59:26.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:25 smithi138 ceph-mon[81304]: pgmap v600: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:26.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.331+0000 7f2bf416f700 1 -- 172.21.15.33:0/1699637143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec1026d0 msgr2=0x7f2bec0fc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.331+0000 7f2bf416f700 1 --2- 172.21.15.33:0/1699637143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec1026d0 0x7f2bec0fc480 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2be0004940 tx=0x7f2be0009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.331+0000 7f2bf416f700 1 -- 172.21.15.33:0/1699637143 shutdown_connections 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.331+0000 7f2bf416f700 1 --2- 172.21.15.33:0/1699637143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec1026d0 0x7f2bec0fc480 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.331+0000 7f2bf416f700 1 --2- 172.21.15.33:0/1699637143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bec101d30 0x7f2bec102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.331+0000 7f2bf416f700 1 -- 172.21.15.33:0/1699637143 >> 172.21.15.33:0/1699637143 conn(0x7f2bec073350 msgr2=0x7f2bec073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.331+0000 7f2bf416f700 1 -- 172.21.15.33:0/1699637143 shutdown_connections 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf416f700 1 -- 172.21.15.33:0/1699637143 wait complete. 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf416f700 1 Processor -- start 2024-04-25T01:59:26.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf416f700 1 -- start start 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf416f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec101d30 0x7f2bec113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf416f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bec1026d0 0x7f2bec113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf416f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1145b0 con 0x7f2bec101d30 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf416f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec10db90 con 0x7f2bec1026d0 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf1f0b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec101d30 0x7f2bec113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf1f0b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec101d30 0x7f2bec113a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38364/0 (socket says 172.21.15.33:38364) 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.332+0000 7f2bf1f0b700 1 -- 172.21.15.33:0/617859932 learned_addr learned my addr 172.21.15.33:0/617859932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.333+0000 7f2bf170a700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bec1026d0 0x7f2bec113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.333+0000 7f2bf1f0b700 1 -- 172.21.15.33:0/617859932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bec1026d0 msgr2=0x7f2bec113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.333+0000 7f2bf1f0b700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bec1026d0 0x7f2bec113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.333+0000 7f2bf1f0b700 1 -- 172.21.15.33:0/617859932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be0005040 con 0x7f2bec101d30 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.333+0000 7f2bf1f0b700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec101d30 0x7f2bec113a20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2be800c8c0 tx=0x7f2be80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.333+0000 7f2bde7fc700 1 -- 172.21.15.33:0/617859932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2be800a980 con 0x7f2bec101d30 2024-04-25T01:59:26.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.333+0000 7f2bde7fc700 1 -- 172.21.15.33:0/617859932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2be800ce60 con 0x7f2bec101d30 2024-04-25T01:59:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.333+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec10de70 con 0x7f2bec101d30 2024-04-25T01:59:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.333+0000 7f2bde7fc700 1 -- 172.21.15.33:0/617859932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2be80115e0 con 0x7f2bec101d30 2024-04-25T01:59:26.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.334+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec10e3b0 con 0x7f2bec101d30 2024-04-25T01:59:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.334+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec0fd490 con 0x7f2bec101d30 2024-04-25T01:59:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.335+0000 7f2bde7fc700 1 -- 172.21.15.33:0/617859932 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2be8012400 con 0x7f2bec101d30 2024-04-25T01:59:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.335+0000 7f2bde7fc700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2bd806e140 0x7f2bd80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.335+0000 7f2bde7fc700 1 -- 172.21.15.33:0/617859932 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2be808db00 con 0x7f2bec101d30 2024-04-25T01:59:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.335+0000 7f2bf170a700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2bd806e140 0x7f2bd80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:26.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.336+0000 7f2bf170a700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2bd806e140 0x7f2bd80705f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2be000b020 tx=0x7f2be0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:26.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.340+0000 7f2bde7fc700 1 -- 172.21.15.33:0/617859932 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2be805aa30 con 0x7f2bec101d30 2024-04-25T01:59:26.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.558+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bec10f070 con 0x7f2bd806e140 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.568+0000 7f2bde7fc700 1 -- 172.21.15.33:0/617859932 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2bec10f070 con 0x7f2bd806e140 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (20m) 13m ago 31m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (17m) 13m ago 31m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (17m) 13m ago 31m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (20m) 13m ago 31m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (13m) 13m ago 26m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (14m) 13m ago 26m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (13m) 13m ago 26m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (26m) 13m ago 26m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:59:26.572 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (22m) 13m ago 33m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (21m) 13m ago 30m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (18m) 13m ago 33m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (18m) 13m ago 30m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (21m) 13m ago 31m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (21m) 13m ago 30m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (17m) 13m ago 30m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (16m) 13m ago 29m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (16m) 13m ago 29m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (15m) 13m ago 28m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (14m) 13m ago 28m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (14m) 13m ago 27m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:59:26.573 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (20m) 13m ago 31m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:59:26.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2bd806e140 msgr2=0x7f2bd80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2bd806e140 0x7f2bd80705f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2be000b020 tx=0x7f2be0009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec101d30 msgr2=0x7f2bec113a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec101d30 0x7f2bec113a20 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f2be800c8c0 tx=0x7f2be80044c0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 shutdown_connections 2024-04-25T01:59:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2bd806e140 0x7f2bd80705f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2bec101d30 0x7f2bec113a20 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 --2- 172.21.15.33:0/617859932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2bec1026d0 0x7f2bec113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 >> 172.21.15.33:0/617859932 conn(0x7f2bec073350 msgr2=0x7f2bec105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 shutdown_connections 2024-04-25T01:59:26.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.570+0000 7f2bf416f700 1 -- 172.21.15.33:0/617859932 wait complete. 2024-04-25T01:59:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.673+0000 7f387687e700 1 -- 172.21.15.33:0/2595198749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 msgr2=0x7f38701020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.673+0000 7f387687e700 1 --2- 172.21.15.33:0/2595198749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 0x7f38701020e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3860004940 tx=0x7f3860009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.673+0000 7f387687e700 1 -- 172.21.15.33:0/2595198749 shutdown_connections 2024-04-25T01:59:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.673+0000 7f387687e700 1 --2- 172.21.15.33:0/2595198749 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38701026b0 0x7f38700fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.673+0000 7f387687e700 1 --2- 172.21.15.33:0/2595198749 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 0x7f38701020e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.673+0000 7f387687e700 1 -- 172.21.15.33:0/2595198749 >> 172.21.15.33:0/2595198749 conn(0x7f38700f7860 msgr2=0x7f38700f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.673+0000 7f387687e700 1 -- 172.21.15.33:0/2595198749 shutdown_connections 2024-04-25T01:59:26.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.673+0000 7f387687e700 1 -- 172.21.15.33:0/2595198749 wait complete. 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.674+0000 7f387687e700 1 Processor -- start 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.674+0000 7f387687e700 1 -- start start 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.674+0000 7f387687e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 0x7f387010eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.674+0000 7f387687e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38701026b0 0x7f387010f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.674+0000 7f387687e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387010f610 con 0x7f3870101d10 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.674+0000 7f387687e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387010f780 con 0x7f38701026b0 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.674+0000 7f386ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 0x7f387010eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.674+0000 7f386ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 0x7f387010eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38380/0 (socket says 172.21.15.33:38380) 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.674+0000 7f386ffff700 1 -- 172.21.15.33:0/686803198 learned_addr learned my addr 172.21.15.33:0/686803198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f386f7fe700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38701026b0 0x7f387010f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:26.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f386ffff700 1 -- 172.21.15.33:0/686803198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38701026b0 msgr2=0x7f387010f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f386ffff700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38701026b0 0x7f387010f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f386ffff700 1 -- 172.21.15.33:0/686803198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3860005040 con 0x7f3870101d10 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f386f7fe700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38701026b0 0x7f387010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f386ffff700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 0x7f387010eb90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f386000b8b0 tx=0x7f3860009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f386cff9700 1 -- 172.21.15.33:0/686803198 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f386000e040 con 0x7f3870101d10 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f386cff9700 1 -- 172.21.15.33:0/686803198 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3860004290 con 0x7f3870101d10 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387010fa00 con 0x7f3870101d10 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f386cff9700 1 -- 172.21.15.33:0/686803198 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f386001c3f0 con 0x7f3870101d10 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.675+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3870117f00 con 0x7f3870101d10 2024-04-25T01:59:26.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.676+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38700fd460 con 0x7f3870101d10 2024-04-25T01:59:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.677+0000 7f386cff9700 1 -- 172.21.15.33:0/686803198 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f386000baf0 con 0x7f3870101d10 2024-04-25T01:59:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.677+0000 7f386cff9700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f385806e140 0x7f38580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.677+0000 7f386cff9700 1 -- 172.21.15.33:0/686803198 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f386008ebd0 con 0x7f3870101d10 2024-04-25T01:59:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.677+0000 7f386f7fe700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f385806e140 0x7f38580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:26.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.678+0000 7f386f7fe700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f385806e140 0x7f38580705f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3864009fd0 tx=0x7f386400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:26.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.682+0000 7f386cff9700 1 -- 172.21.15.33:0/686803198 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3860058070 con 0x7f3870101d10 2024-04-25T01:59:26.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.985+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f387010f9c0 con 0x7f3870101d10 2024-04-25T01:59:26.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.985+0000 7f386cff9700 1 -- 172.21.15.33:0/686803198 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f386005b730 con 0x7f3870101d10 2024-04-25T01:59:26.989 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:59:26.989 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:59:26.990 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:59:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:59:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:59:26.991 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:59:26.991 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.987+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f385806e140 msgr2=0x7f38580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.987+0000 7f387687e700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f385806e140 0x7f38580705f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3864009fd0 tx=0x7f386400b040 comp rx=0 tx=0).stop 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.987+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 msgr2=0x7f387010eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.987+0000 7f387687e700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 0x7f387010eb90 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f386000b8b0 tx=0x7f3860009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.987+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 shutdown_connections 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.987+0000 7f387687e700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f385806e140 0x7f38580705f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.988+0000 7f387687e700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3870101d10 0x7f387010eb90 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.988+0000 7f387687e700 1 --2- 172.21.15.33:0/686803198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f38701026b0 0x7f387010f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.988+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 >> 172.21.15.33:0/686803198 conn(0x7f38700f7860 msgr2=0x7f38700f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.988+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 shutdown_connections 2024-04-25T01:59:26.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:26.988+0000 7f387687e700 1 -- 172.21.15.33:0/686803198 wait complete. 2024-04-25T01:59:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 -- 172.21.15.33:0/1049043463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074102de0 msgr2=0x7f30741031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 --2- 172.21.15.33:0/1049043463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074102de0 0x7f30741031b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3068004940 tx=0x7f3068009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 -- 172.21.15.33:0/1049043463 shutdown_connections 2024-04-25T01:59:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 --2- 172.21.15.33:0/1049043463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3074103780 0x7f30740fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 --2- 172.21.15.33:0/1049043463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074102de0 0x7f30741031b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 -- 172.21.15.33:0/1049043463 >> 172.21.15.33:0/1049043463 conn(0x7f30740f8990 msgr2=0x7f30740fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:27.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 -- 172.21.15.33:0/1049043463 shutdown_connections 2024-04-25T01:59:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 -- 172.21.15.33:0/1049043463 wait complete. 2024-04-25T01:59:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 Processor -- start 2024-04-25T01:59:27.097 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.092+0000 7f307ad78700 1 -- start start 2024-04-25T01:59:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f307ad78700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3074102de0 0x7f3074070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f307ad78700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074103780 0x7f30740707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f307ad78700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3074072160 con 0x7f3074103780 2024-04-25T01:59:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f307ad78700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3074070ce0 con 0x7f3074102de0 2024-04-25T01:59:27.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f3073fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074103780 0x7f30740707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f3073fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074103780 0x7f30740707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38388/0 (socket says 172.21.15.33:38388) 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f3073fff700 1 -- 172.21.15.33:0/1170485698 learned_addr learned my addr 172.21.15.33:0/1170485698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f3078b14700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3074102de0 0x7f3074070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f3073fff700 1 -- 172.21.15.33:0/1170485698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3074102de0 msgr2=0x7f3074070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f3073fff700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3074102de0 0x7f3074070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f3073fff700 1 -- 172.21.15.33:0/1170485698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068005040 con 0x7f3074103780 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f3078b14700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3074102de0 0x7f3074070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.093+0000 7f3073fff700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074103780 0x7f30740707a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3060002a50 tx=0x7f3060004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.094+0000 7f30717fa700 1 -- 172.21.15.33:0/1170485698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3060007cc0 con 0x7f3074103780 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.094+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3074070fc0 con 0x7f3074103780 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.094+0000 7f30717fa700 1 -- 172.21.15.33:0/1170485698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f306000f040 con 0x7f3074103780 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.094+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307411a160 con 0x7f3074103780 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.094+0000 7f30717fa700 1 -- 172.21.15.33:0/1170485698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f306000da90 con 0x7f3074103780 2024-04-25T01:59:27.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.095+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30741000f0 con 0x7f3074103780 2024-04-25T01:59:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.095+0000 7f30717fa700 1 -- 172.21.15.33:0/1170485698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3060009810 con 0x7f3074103780 2024-04-25T01:59:27.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.096+0000 7f30717fa700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f306406e140 0x7f30640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.096+0000 7f30717fa700 1 -- 172.21.15.33:0/1170485698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f306008d8c0 con 0x7f3074103780 2024-04-25T01:59:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.096+0000 7f3078b14700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f306406e140 0x7f30640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:27.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.096+0000 7f3078b14700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f306406e140 0x7f30640705f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3068004560 tx=0x7f3068009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:27.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.101+0000 7f30717fa700 1 -- 172.21.15.33:0/1170485698 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3060056c40 con 0x7f3074103780 2024-04-25T01:59:27.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:26 smithi033 ceph-mon[111306]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:59:27.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:26 smithi033 ceph-mon[111306]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:59:27.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:26 smithi138 ceph-mon[81304]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:59:27.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:26 smithi138 ceph-mon[81304]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:59:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.357+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3074109ee0 con 0x7f3074103780 2024-04-25T01:59:27.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.358+0000 7f30717fa700 1 -- 172.21.15.33:0/1170485698 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f306005a300 con 0x7f3074103780 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:59:27.362 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:59:27.363 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:59:27.364 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:59:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f306406e140 msgr2=0x7f30640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f306406e140 0x7f30640705f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3068004560 tx=0x7f3068009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074103780 msgr2=0x7f30740707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074103780 0x7f30740707a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3060002a50 tx=0x7f3060004600 comp rx=0 tx=0).stop 2024-04-25T01:59:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 shutdown_connections 2024-04-25T01:59:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f306406e140 0x7f30640705f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3074102de0 0x7f3074070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 --2- 172.21.15.33:0/1170485698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3074103780 0x7f30740707a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 >> 172.21.15.33:0/1170485698 conn(0x7f30740f8990 msgr2=0x7f3074107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 shutdown_connections 2024-04-25T01:59:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.360+0000 7f307ad78700 1 -- 172.21.15.33:0/1170485698 wait complete. 2024-04-25T01:59:27.366 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T01:59:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.464+0000 7f2860037700 1 -- 172.21.15.33:0/1442978918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 msgr2=0x7f28580fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.464+0000 7f2860037700 1 --2- 172.21.15.33:0/1442978918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 0x7f28580fe1d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2854004940 tx=0x7f2854009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:27.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.464+0000 7f2860037700 1 -- 172.21.15.33:0/1442978918 shutdown_connections 2024-04-25T01:59:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.464+0000 7f2860037700 1 --2- 172.21.15.33:0/1442978918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28580fe7a0 0x7f2858106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.464+0000 7f2860037700 1 --2- 172.21.15.33:0/1442978918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 0x7f28580fe1d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.464+0000 7f2860037700 1 -- 172.21.15.33:0/1442978918 >> 172.21.15.33:0/1442978918 conn(0x7f28580f7860 msgr2=0x7f28580f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.465+0000 7f2860037700 1 -- 172.21.15.33:0/1442978918 shutdown_connections 2024-04-25T01:59:27.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.465+0000 7f2860037700 1 -- 172.21.15.33:0/1442978918 wait complete. 2024-04-25T01:59:27.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.465+0000 7f2860037700 1 Processor -- start 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.465+0000 7f2860037700 1 -- start start 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.465+0000 7f2860037700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 0x7f2858113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.465+0000 7f2860037700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28580fe7a0 0x7f2858113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.465+0000 7f2860037700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28581144f0 con 0x7f28580fde00 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.465+0000 7f2860037700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285810daf0 con 0x7f28580fe7a0 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f285ddd3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 0x7f2858113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f285ddd3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 0x7f2858113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38402/0 (socket says 172.21.15.33:38402) 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f285ddd3700 1 -- 172.21.15.33:0/3602523912 learned_addr learned my addr 172.21.15.33:0/3602523912 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f285d5d2700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28580fe7a0 0x7f2858113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f285ddd3700 1 -- 172.21.15.33:0/3602523912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28580fe7a0 msgr2=0x7f2858113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f285ddd3700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28580fe7a0 0x7f2858113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f285ddd3700 1 -- 172.21.15.33:0/3602523912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854005040 con 0x7f28580fde00 2024-04-25T01:59:27.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f285ddd3700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 0x7f2858113960 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2854004890 tx=0x7f2854009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f284e7fc700 1 -- 172.21.15.33:0/3602523912 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f285400e040 con 0x7f28580fde00 2024-04-25T01:59:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f284e7fc700 1 -- 172.21.15.33:0/3602523912 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2854009af0 con 0x7f28580fde00 2024-04-25T01:59:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285810dd70 con 0x7f28580fde00 2024-04-25T01:59:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f284e7fc700 1 -- 172.21.15.33:0/3602523912 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f285401c630 con 0x7f28580fde00 2024-04-25T01:59:27.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.466+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285810e2b0 con 0x7f28580fde00 2024-04-25T01:59:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.467+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2858066ad0 con 0x7f28580fde00 2024-04-25T01:59:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.468+0000 7f284e7fc700 1 -- 172.21.15.33:0/3602523912 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f285400bbc0 con 0x7f28580fde00 2024-04-25T01:59:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.468+0000 7f284e7fc700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f284406e140 0x7f28440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.468+0000 7f284e7fc700 1 -- 172.21.15.33:0/3602523912 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f285408ee10 con 0x7f28580fde00 2024-04-25T01:59:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.468+0000 7f285d5d2700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f284406e140 0x7f28440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:27.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.469+0000 7f285d5d2700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f284406e140 0x7f28440705f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2848009fd0 tx=0x7f2848009f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:27.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.473+0000 7f284e7fc700 1 -- 172.21.15.33:0/3602523912 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f285405bd40 con 0x7f28580fde00 2024-04-25T01:59:27.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.696+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2858109e90 con 0x7f284406e140 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.697+0000 7f284e7fc700 1 -- 172.21.15.33:0/3602523912 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2858109e90 con 0x7f284406e140 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T01:59:27.701 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T01:59:27.702 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T01:59:27.702 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f284406e140 msgr2=0x7f28440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f284406e140 0x7f28440705f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2848009fd0 tx=0x7f2848009f60 comp rx=0 tx=0).stop 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 msgr2=0x7f2858113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 0x7f2858113960 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2854004890 tx=0x7f2854009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 shutdown_connections 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f284406e140 0x7f28440705f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28580fde00 0x7f2858113960 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 --2- 172.21.15.33:0/3602523912 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28580fe7a0 0x7f2858113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 >> 172.21.15.33:0/3602523912 conn(0x7f28580f7860 msgr2=0x7f28580f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 shutdown_connections 2024-04-25T01:59:27.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.700+0000 7f2860037700 1 -- 172.21.15.33:0/3602523912 wait complete. 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.804+0000 7fea008f8700 1 -- 172.21.15.33:0/216030041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 msgr2=0x7fe9fc0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.804+0000 7fea008f8700 1 --2- 172.21.15.33:0/216030041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 0x7fe9fc0fb990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0004940 tx=0x7fe9f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.804+0000 7fea008f8700 1 -- 172.21.15.33:0/216030041 shutdown_connections 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.804+0000 7fea008f8700 1 --2- 172.21.15.33:0/216030041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 0x7fe9fc0fb990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.804+0000 7fea008f8700 1 --2- 172.21.15.33:0/216030041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9fc102f60 0x7fe9fc103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.804+0000 7fea008f8700 1 -- 172.21.15.33:0/216030041 >> 172.21.15.33:0/216030041 conn(0x7fe9fc0714d0 msgr2=0x7fe9fc0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.804+0000 7fea008f8700 1 -- 172.21.15.33:0/216030041 shutdown_connections 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.804+0000 7fea008f8700 1 -- 172.21.15.33:0/216030041 wait complete. 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.804+0000 7fea008f8700 1 Processor -- start 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fea008f8700 1 -- start start 2024-04-25T01:59:27.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fea008f8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 0x7fe9fc115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fea008f8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9fc102f60 0x7fe9fc1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fea008f8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc116720 con 0x7fe9fc0fb520 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fea008f8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9fc10fd00 con 0x7fe9fc102f60 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fe9fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 0x7fe9fc115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fe9fa59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 0x7fe9fc115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38418/0 (socket says 172.21.15.33:38418) 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fe9fa59c700 1 -- 172.21.15.33:0/2100251717 learned_addr learned my addr 172.21.15.33:0/2100251717 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fe9f9d9b700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9fc102f60 0x7fe9fc1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fe9fa59c700 1 -- 172.21.15.33:0/2100251717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9fc102f60 msgr2=0x7fe9fc1160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fe9fa59c700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9fc102f60 0x7fe9fc1160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fe9fa59c700 1 -- 172.21.15.33:0/2100251717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f0005040 con 0x7fe9fc0fb520 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fe9f9d9b700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9fc102f60 0x7fe9fc1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T01:59:27.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.805+0000 7fe9fa59c700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 0x7fe9fc115b90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec002a80 tx=0x7fe9ec0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.806+0000 7fe9eaffd700 1 -- 172.21.15.33:0/2100251717 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9ec007cc0 con 0x7fe9fc0fb520 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.806+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9fc10ffe0 con 0x7fe9fc0fb520 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.806+0000 7fe9eaffd700 1 -- 172.21.15.33:0/2100251717 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9ec00f040 con 0x7fe9fc0fb520 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.806+0000 7fe9eaffd700 1 -- 172.21.15.33:0/2100251717 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9ec0096f0 con 0x7fe9fc0fb520 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.806+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9fc110520 con 0x7fe9fc0fb520 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.807+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9fc106b00 con 0x7fe9fc0fb520 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.807+0000 7fe9eaffd700 1 -- 172.21.15.33:0/2100251717 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe9ec00d690 con 0x7fe9fc0fb520 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.808+0000 7fe9eaffd700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe9e406e140 0x7fe9e40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.808+0000 7fe9eaffd700 1 -- 172.21.15.33:0/2100251717 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe9ec08d2d0 con 0x7fe9fc0fb520 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.808+0000 7fe9f9d9b700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe9e406e140 0x7fe9e40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:27.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.808+0000 7fe9f9d9b700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe9e406e140 0x7fe9e40705f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0004f90 tx=0x7fe9f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:27.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:27.812+0000 7fe9eaffd700 1 -- 172.21.15.33:0/2100251717 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe9ec05a180 con 0x7fe9fc0fb520 2024-04-25T01:59:28.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:27 smithi033 ceph-mon[111306]: pgmap v601: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:28.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:27 smithi033 ceph-mon[111306]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:59:28.019 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:27 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/686803198' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:59:28.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:27 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1170485698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:59:28.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.108+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe9fc109fe0 con 0x7fe9fc0fb520 2024-04-25T01:59:28.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.108+0000 7fe9eaffd700 1 -- 172.21.15.33:0/2100251717 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fe9ec059d10 con 0x7fe9fc0fb520 2024-04-25T01:59:28.112 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T01:59:28.112 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T01:59:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T01:59:28.112 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T01:59:28.112 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T01:59:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.110+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe9e406e140 msgr2=0x7fe9e40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.110+0000 7fea008f8700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe9e406e140 0x7fe9e40705f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0004f90 tx=0x7fe9f0009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:28.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.110+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 msgr2=0x7fe9fc115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.110+0000 7fea008f8700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 0x7fe9fc115b90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec002a80 tx=0x7fe9ec0046b0 comp rx=0 tx=0).stop 2024-04-25T01:59:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.111+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 shutdown_connections 2024-04-25T01:59:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.111+0000 7fea008f8700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe9e406e140 0x7fe9e40705f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.111+0000 7fea008f8700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe9fc0fb520 0x7fe9fc115b90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.111+0000 7fea008f8700 1 --2- 172.21.15.33:0/2100251717 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe9fc102f60 0x7fe9fc1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.111+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 >> 172.21.15.33:0/2100251717 conn(0x7fe9fc0714d0 msgr2=0x7fe9fc0fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.111+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 shutdown_connections 2024-04-25T01:59:28.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:28.111+0000 7fea008f8700 1 -- 172.21.15.33:0/2100251717 wait complete. 2024-04-25T01:59:28.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:27 smithi138 ceph-mon[81304]: pgmap v601: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:28.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:27 smithi138 ceph-mon[81304]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:59:28.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:27 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/686803198' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T01:59:28.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:27 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1170485698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T01:59:29.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:28 smithi033 ceph-mon[111306]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:59:29.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:28 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2100251717' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:59:29.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:28 smithi138 ceph-mon[81304]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T01:59:29.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:28 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2100251717' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T01:59:30.196 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:29 smithi138 ceph-mon[81304]: pgmap v602: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:30.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:29 smithi033 ceph-mon[111306]: pgmap v602: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:32.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:31 smithi033 ceph-mon[111306]: pgmap v603: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:32.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:31 smithi138 ceph-mon[81304]: pgmap v603: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:34.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:33 smithi033 ceph-mon[111306]: pgmap v604: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:34.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:33 smithi138 ceph-mon[81304]: pgmap v604: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:36.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:35 smithi033 ceph-mon[111306]: pgmap v605: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:36.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:35 smithi138 ceph-mon[81304]: pgmap v605: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:38.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:37 smithi033 ceph-mon[111306]: pgmap v606: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:38.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:37 smithi138 ceph-mon[81304]: pgmap v606: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:40.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:39 smithi033 ceph-mon[111306]: pgmap v607: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:40.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:39 smithi138 ceph-mon[81304]: pgmap v607: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:41.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:59:41.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:59:42.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:41 smithi033 ceph-mon[111306]: pgmap v608: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:42.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:41 smithi138 ceph-mon[81304]: pgmap v608: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:44.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:43 smithi033 ceph-mon[111306]: pgmap v609: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:44.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:43 smithi138 ceph-mon[81304]: pgmap v609: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:46.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:45 smithi033 ceph-mon[111306]: pgmap v610: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:46.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:45 smithi138 ceph-mon[81304]: pgmap v610: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:48.269 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:47 smithi033 ceph-mon[111306]: pgmap v611: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:48.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:47 smithi138 ceph-mon[81304]: pgmap v611: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:50.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:49 smithi033 ceph-mon[111306]: pgmap v612: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:50.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:49 smithi138 ceph-mon[81304]: pgmap v612: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:52.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:51 smithi033 ceph-mon[111306]: pgmap v613: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:52.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:51 smithi138 ceph-mon[81304]: pgmap v613: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:54.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:53 smithi033 ceph-mon[111306]: pgmap v614: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:54.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:53 smithi138 ceph-mon[81304]: pgmap v614: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:56.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:55 smithi033 ceph-mon[111306]: pgmap v615: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:56.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:59:56.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:55 smithi138 ceph-mon[81304]: pgmap v615: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:56.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T01:59:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.229+0000 7f7fdcffd700 1 -- 172.21.15.33:0/4231857934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8102de0 msgr2=0x7f7fd81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.229+0000 7f7fdcffd700 1 --2- 172.21.15.33:0/4231857934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8102de0 0x7f7fd81031b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8004940 tx=0x7f7fc8009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.229+0000 7f7fdcffd700 1 -- 172.21.15.33:0/4231857934 shutdown_connections 2024-04-25T01:59:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.229+0000 7f7fdcffd700 1 --2- 172.21.15.33:0/4231857934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd8103780 0x7f7fd80fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.229+0000 7f7fdcffd700 1 --2- 172.21.15.33:0/4231857934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8102de0 0x7f7fd81031b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.229+0000 7f7fdcffd700 1 -- 172.21.15.33:0/4231857934 >> 172.21.15.33:0/4231857934 conn(0x7f7fd80f8990 msgr2=0x7f7fd80fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.229+0000 7f7fdcffd700 1 -- 172.21.15.33:0/4231857934 shutdown_connections 2024-04-25T01:59:58.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.229+0000 7f7fdcffd700 1 -- 172.21.15.33:0/4231857934 wait complete. 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.230+0000 7f7fdcffd700 1 Processor -- start 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.230+0000 7f7fdcffd700 1 -- start start 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.230+0000 7f7fdcffd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd8102de0 0x7f7fd8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.230+0000 7f7fdcffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8103780 0x7f7fd80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.230+0000 7f7fdcffd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8072160 con 0x7f7fd8103780 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.230+0000 7f7fdcffd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd8070ce0 con 0x7f7fd8102de0 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.230+0000 7f7fcdd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8103780 0x7f7fd80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.230+0000 7f7fcdd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8103780 0x7f7fd80707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32856/0 (socket says 172.21.15.33:32856) 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.230+0000 7f7fcdd9b700 1 -- 172.21.15.33:0/1462986918 learned_addr learned my addr 172.21.15.33:0/1462986918 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fd659c700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd8102de0 0x7f7fd8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fcdd9b700 1 -- 172.21.15.33:0/1462986918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd8102de0 msgr2=0x7f7fd8070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fcdd9b700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd8102de0 0x7f7fd8070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fcdd9b700 1 -- 172.21.15.33:0/1462986918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc8005040 con 0x7f7fd8103780 2024-04-25T01:59:58.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fcdd9b700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8103780 0x7f7fd80707a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0002a50 tx=0x7f7fc0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:58.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fcf7fe700 1 -- 172.21.15.33:0/1462986918 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc0007cc0 con 0x7f7fd8103780 2024-04-25T01:59:58.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fcf7fe700 1 -- 172.21.15.33:0/1462986918 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fc000f040 con 0x7f7fd8103780 2024-04-25T01:59:58.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd8070fc0 con 0x7f7fd8103780 2024-04-25T01:59:58.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fcf7fe700 1 -- 172.21.15.33:0/1462986918 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fc000da40 con 0x7f7fd8103780 2024-04-25T01:59:58.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.231+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd811a160 con 0x7f7fd8103780 2024-04-25T01:59:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.232+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd81000f0 con 0x7f7fd8103780 2024-04-25T01:59:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.233+0000 7f7fcf7fe700 1 -- 172.21.15.33:0/1462986918 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7fc00097c0 con 0x7f7fd8103780 2024-04-25T01:59:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.233+0000 7f7fcf7fe700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7fc406e140 0x7f7fc40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.233+0000 7f7fcf7fe700 1 -- 172.21.15.33:0/1462986918 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7fc008d950 con 0x7f7fd8103780 2024-04-25T01:59:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.233+0000 7f7fd659c700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7fc406e140 0x7f7fc40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:58.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.234+0000 7f7fd659c700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7fc406e140 0x7f7fc40705f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800b3b0 tx=0x7f7fc8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:58.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.238+0000 7f7fcf7fe700 1 -- 172.21.15.33:0/1462986918 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7fc0056cd0 con 0x7f7fd8103780 2024-04-25T01:59:58.242 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:57 smithi033 ceph-mon[111306]: pgmap v616: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:58.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:57 smithi138 ceph-mon[81304]: pgmap v616: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T01:59:58.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.466+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fd8109ee0 con 0x7f7fc406e140 2024-04-25T01:59:58.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.467+0000 7f7fcf7fe700 1 -- 172.21.15.33:0/1462986918 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7fd8109ee0 con 0x7f7fc406e140 2024-04-25T01:59:58.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7fc406e140 msgr2=0x7f7fc40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7fc406e140 0x7f7fc40705f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800b3b0 tx=0x7f7fc8009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:58.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8103780 msgr2=0x7f7fd80707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8103780 0x7f7fd80707a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0002a50 tx=0x7f7fc0004600 comp rx=0 tx=0).stop 2024-04-25T01:59:58.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 shutdown_connections 2024-04-25T01:59:58.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7fc406e140 0x7f7fc40705f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7fd8102de0 0x7f7fd8070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 --2- 172.21.15.33:0/1462986918 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7fd8103780 0x7f7fd80707a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 >> 172.21.15.33:0/1462986918 conn(0x7f7fd80f8990 msgr2=0x7f7fd8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:58.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 shutdown_connections 2024-04-25T01:59:58.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.469+0000 7f7fdcffd700 1 -- 172.21.15.33:0/1462986918 wait complete. 2024-04-25T01:59:58.483 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T01:59:58.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.574+0000 7f9e51a7a700 1 -- 172.21.15.33:0/3202128239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 msgr2=0x7f9e4c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.574+0000 7f9e51a7a700 1 --2- 172.21.15.33:0/3202128239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 0x7f9e4c0fbd20 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9e40004940 tx=0x7f9e40009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.574+0000 7f9e51a7a700 1 -- 172.21.15.33:0/3202128239 shutdown_connections 2024-04-25T01:59:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.574+0000 7f9e51a7a700 1 --2- 172.21.15.33:0/3202128239 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 0x7f9e4c0fbd20 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.574+0000 7f9e51a7a700 1 --2- 172.21.15.33:0/3202128239 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c101d10 0x7f9e4c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.574+0000 7f9e51a7a700 1 -- 172.21.15.33:0/3202128239 >> 172.21.15.33:0/3202128239 conn(0x7f9e4c0f7860 msgr2=0x7f9e4c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.574+0000 7f9e51a7a700 1 -- 172.21.15.33:0/3202128239 shutdown_connections 2024-04-25T01:59:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.574+0000 7f9e51a7a700 1 -- 172.21.15.33:0/3202128239 wait complete. 2024-04-25T01:59:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e51a7a700 1 Processor -- start 2024-04-25T01:59:58.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e51a7a700 1 -- start start 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e51a7a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c101d10 0x7f9e4c10eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e51a7a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 0x7f9e4c10f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e51a7a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c10f5d0 con 0x7f9e4c1026b0 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e51a7a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c10f710 con 0x7f9e4c101d10 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e4affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 0x7f9e4c10f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e4affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 0x7f9e4c10f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32866/0 (socket says 172.21.15.33:32866) 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e4affd700 1 -- 172.21.15.33:0/393754316 learned_addr learned my addr 172.21.15.33:0/393754316 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.575+0000 7f9e4b7fe700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c101d10 0x7f9e4c10eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.576+0000 7f9e4affd700 1 -- 172.21.15.33:0/393754316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c101d10 msgr2=0x7f9e4c10eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.576+0000 7f9e4affd700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c101d10 0x7f9e4c10eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.576+0000 7f9e4affd700 1 -- 172.21.15.33:0/393754316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e40005040 con 0x7f9e4c1026b0 2024-04-25T01:59:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.576+0000 7f9e4affd700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 0x7f9e4c10f090 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009180 tx=0x7f9e40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.576+0000 7f9e33fff700 1 -- 172.21.15.33:0/393754316 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e4000e040 con 0x7f9e4c1026b0 2024-04-25T01:59:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.576+0000 7f9e33fff700 1 -- 172.21.15.33:0/393754316 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e4000bbc0 con 0x7f9e4c1026b0 2024-04-25T01:59:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.576+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e4c10f990 con 0x7f9e4c1026b0 2024-04-25T01:59:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.576+0000 7f9e33fff700 1 -- 172.21.15.33:0/393754316 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e4001c3f0 con 0x7f9e4c1026b0 2024-04-25T01:59:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.576+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c117ec0 con 0x7f9e4c1026b0 2024-04-25T01:59:58.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.577+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e4c0fd460 con 0x7f9e4c1026b0 2024-04-25T01:59:58.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.577+0000 7f9e33fff700 1 -- 172.21.15.33:0/393754316 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9e4000bd30 con 0x7f9e4c1026b0 2024-04-25T01:59:58.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.578+0000 7f9e33fff700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9e3406e140 0x7f9e340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:58.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.578+0000 7f9e33fff700 1 -- 172.21.15.33:0/393754316 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9e4008ee50 con 0x7f9e4c1026b0 2024-04-25T01:59:58.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.578+0000 7f9e4b7fe700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9e3406e140 0x7f9e340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:58.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.578+0000 7f9e4b7fe700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9e3406e140 0x7f9e340705f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c007fd0 tx=0x7f9e3c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:58.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.583+0000 7f9e33fff700 1 -- 172.21.15.33:0/393754316 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9e40058250 con 0x7f9e4c1026b0 2024-04-25T01:59:58.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.812+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e4c108df0 con 0x7f9e3406e140 2024-04-25T01:59:58.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.813+0000 7f9e33fff700 1 -- 172.21.15.33:0/393754316 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9e4c108df0 con 0x7f9e3406e140 2024-04-25T01:59:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9e3406e140 msgr2=0x7f9e340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9e3406e140 0x7f9e340705f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c007fd0 tx=0x7f9e3c009040 comp rx=0 tx=0).stop 2024-04-25T01:59:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 msgr2=0x7f9e4c10f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 0x7f9e4c10f090 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009180 tx=0x7f9e40009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:58.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 shutdown_connections 2024-04-25T01:59:58.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9e3406e140 0x7f9e340705f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9e4c101d10 0x7f9e4c10eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 --2- 172.21.15.33:0/393754316 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9e4c1026b0 0x7f9e4c10f090 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 >> 172.21.15.33:0/393754316 conn(0x7f9e4c0f7860 msgr2=0x7f9e4c0f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:58.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 shutdown_connections 2024-04-25T01:59:58.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.815+0000 7f9e51a7a700 1 -- 172.21.15.33:0/393754316 wait complete. 2024-04-25T01:59:58.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.919+0000 7f6268fbd700 1 -- 172.21.15.33:0/1611270983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62640fbd40 msgr2=0x7f62640fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.919+0000 7f6268fbd700 1 --2- 172.21.15.33:0/1611270983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62640fbd40 0x7f62640fc1b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f6254004940 tx=0x7f6254009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.919+0000 7f6268fbd700 1 -- 172.21.15.33:0/1611270983 shutdown_connections 2024-04-25T01:59:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.919+0000 7f6268fbd700 1 --2- 172.21.15.33:0/1611270983 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62640fbd40 0x7f62640fc1b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.919+0000 7f6268fbd700 1 --2- 172.21.15.33:0/1611270983 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6264101d90 0x7f6264102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.919+0000 7f6268fbd700 1 -- 172.21.15.33:0/1611270983 >> 172.21.15.33:0/1611270983 conn(0x7f6264071d00 msgr2=0x7f6264072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.919+0000 7f6268fbd700 1 -- 172.21.15.33:0/1611270983 shutdown_connections 2024-04-25T01:59:58.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.919+0000 7f6268fbd700 1 -- 172.21.15.33:0/1611270983 wait complete. 2024-04-25T01:59:58.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.920+0000 7f6268fbd700 1 Processor -- start 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.920+0000 7f6268fbd700 1 -- start start 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.920+0000 7f6268fbd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62640fbd40 0x7f6264112a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.920+0000 7f6268fbd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264101d90 0x7f6264112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.920+0000 7f6268fbd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62641136a0 con 0x7f6264101d90 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.920+0000 7f6268fbd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626410ca00 con 0x7f62640fbd40 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.920+0000 7f6261d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264101d90 0x7f6264112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.920+0000 7f6261d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264101d90 0x7f6264112fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32886/0 (socket says 172.21.15.33:32886) 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.920+0000 7f6261d9b700 1 -- 172.21.15.33:0/4217627075 learned_addr learned my addr 172.21.15.33:0/4217627075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f626259c700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62640fbd40 0x7f6264112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f6261d9b700 1 -- 172.21.15.33:0/4217627075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62640fbd40 msgr2=0x7f6264112a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f6261d9b700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62640fbd40 0x7f6264112a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:58.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f6261d9b700 1 -- 172.21.15.33:0/4217627075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254005040 con 0x7f6264101d90 2024-04-25T01:59:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f6261d9b700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264101d90 0x7f6264112fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6254009180 tx=0x7f6254009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f625affd700 1 -- 172.21.15.33:0/4217627075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f625400e040 con 0x7f6264101d90 2024-04-25T01:59:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626410cc80 con 0x7f6264101d90 2024-04-25T01:59:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f625affd700 1 -- 172.21.15.33:0/4217627075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f625400bbc0 con 0x7f6264101d90 2024-04-25T01:59:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f625affd700 1 -- 172.21.15.33:0/4217627075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f625401c3f0 con 0x7f6264101d90 2024-04-25T01:59:58.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.921+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626410d240 con 0x7f6264101d90 2024-04-25T01:59:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.922+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62640fd430 con 0x7f6264101d90 2024-04-25T01:59:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.923+0000 7f625affd700 1 -- 172.21.15.33:0/4217627075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f625400bd30 con 0x7f6264101d90 2024-04-25T01:59:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.923+0000 7f625affd700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f625006e150 0x7f6250070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.923+0000 7f625affd700 1 -- 172.21.15.33:0/4217627075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f625408ec60 con 0x7f6264101d90 2024-04-25T01:59:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.923+0000 7f626259c700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f625006e150 0x7f6250070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:58.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.924+0000 7f626259c700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f625006e150 0x7f6250070600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f624c009bf0 tx=0x7f624c0091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:58.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:58.928+0000 7f625affd700 1 -- 172.21.15.33:0/4217627075 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f625405bc30 con 0x7f6264101d90 2024-04-25T01:59:59.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.142+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6264108ec0 con 0x7f625006e150 2024-04-25T01:59:59.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.152+0000 7f625affd700 1 -- 172.21.15.33:0/4217627075 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6264108ec0 con 0x7f625006e150 2024-04-25T01:59:59.156 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T01:59:59.156 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (21m) 14m ago 32m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T01:59:59.156 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (18m) 14m ago 32m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (18m) 14m ago 31m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (20m) 14m ago 32m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (14m) 14m ago 26m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (14m) 14m ago 26m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (14m) 14m ago 26m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (26m) 14m ago 26m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (22m) 14m ago 33m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (22m) 14m ago 31m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (19m) 14m ago 33m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (19m) 14m ago 31m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (21m) 14m ago 32m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (21m) 14m ago 31m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (17m) 14m ago 30m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (17m) 14m ago 30m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T01:59:59.157 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (16m) 14m ago 29m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T01:59:59.158 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (16m) 14m ago 29m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T01:59:59.158 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (15m) 14m ago 28m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T01:59:59.158 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (14m) 14m ago 28m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T01:59:59.158 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (21m) 14m ago 31m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T01:59:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f625006e150 msgr2=0x7f6250070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f625006e150 0x7f6250070600 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f624c009bf0 tx=0x7f624c0091a0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264101d90 msgr2=0x7f6264112fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264101d90 0x7f6264112fc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6254009180 tx=0x7f6254009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 shutdown_connections 2024-04-25T01:59:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f625006e150 0x7f6250070600 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62640fbd40 0x7f6264112a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 --2- 172.21.15.33:0/4217627075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6264101d90 0x7f6264112fc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 >> 172.21.15.33:0/4217627075 conn(0x7f6264071d00 msgr2=0x7f62640fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 shutdown_connections 2024-04-25T01:59:59.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.155+0000 7f6268fbd700 1 -- 172.21.15.33:0/4217627075 wait complete. 2024-04-25T01:59:59.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.257+0000 7fa331fb7700 1 -- 172.21.15.33:0/728454858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 msgr2=0x7fa32c1031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.257+0000 7fa331fb7700 1 --2- 172.21.15.33:0/728454858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 0x7fa32c1031f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa31c004940 tx=0x7fa31c009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:59.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.258+0000 7fa331fb7700 1 -- 172.21.15.33:0/728454858 shutdown_connections 2024-04-25T01:59:59.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.258+0000 7fa331fb7700 1 --2- 172.21.15.33:0/728454858 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa32c1037c0 0x7fa32c0fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.258+0000 7fa331fb7700 1 --2- 172.21.15.33:0/728454858 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 0x7fa32c1031f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.258+0000 7fa331fb7700 1 -- 172.21.15.33:0/728454858 >> 172.21.15.33:0/728454858 conn(0x7fa32c072380 msgr2=0x7fa32c072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:59.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.258+0000 7fa331fb7700 1 -- 172.21.15.33:0/728454858 shutdown_connections 2024-04-25T01:59:59.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.258+0000 7fa331fb7700 1 -- 172.21.15.33:0/728454858 wait complete. 2024-04-25T01:59:59.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.258+0000 7fa331fb7700 1 Processor -- start 2024-04-25T01:59:59.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.258+0000 7fa331fb7700 1 -- start start 2024-04-25T01:59:59.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa331fb7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 0x7fa32c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:59.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa331fb7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa32c1037c0 0x7fa32c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:59.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa331fb7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c114500 con 0x7fa32c102e20 2024-04-25T01:59:59.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa331fb7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32c10db20 con 0x7fa32c1037c0 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa32b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 0x7fa32c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa32b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 0x7fa32c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32906/0 (socket says 172.21.15.33:32906) 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa32b7fe700 1 -- 172.21.15.33:0/4044096708 learned_addr learned my addr 172.21.15.33:0/4044096708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa32affd700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa32c1037c0 0x7fa32c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa32b7fe700 1 -- 172.21.15.33:0/4044096708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa32c1037c0 msgr2=0x7fa32c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa32b7fe700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa32c1037c0 0x7fa32c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa32b7fe700 1 -- 172.21.15.33:0/4044096708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c005040 con 0x7fa32c102e20 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.259+0000 7fa32b7fe700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 0x7fa32c113970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009100 tx=0x7fa31c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.260+0000 7fa313fff700 1 -- 172.21.15.33:0/4044096708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31c00e040 con 0x7fa32c102e20 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.260+0000 7fa313fff700 1 -- 172.21.15.33:0/4044096708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa31c013420 con 0x7fa32c102e20 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.260+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa32c10dda0 con 0x7fa32c102e20 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.260+0000 7fa313fff700 1 -- 172.21.15.33:0/4044096708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31c01c610 con 0x7fa32c102e20 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.260+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32c10e2e0 con 0x7fa32c102e20 2024-04-25T01:59:59.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.261+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32c100160 con 0x7fa32c102e20 2024-04-25T01:59:59.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.262+0000 7fa313fff700 1 -- 172.21.15.33:0/4044096708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa31c013590 con 0x7fa32c102e20 2024-04-25T01:59:59.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.262+0000 7fa313fff700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa31406e140 0x7fa3140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:59.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.262+0000 7fa313fff700 1 -- 172.21.15.33:0/4044096708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa31c08edc0 con 0x7fa32c102e20 2024-04-25T01:59:59.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.262+0000 7fa32affd700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa31406e140 0x7fa3140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:59.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.263+0000 7fa32affd700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa31406e140 0x7fa3140705f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa32c10f210 tx=0x7fa320009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:59.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.267+0000 7fa313fff700 1 -- 172.21.15.33:0/4044096708 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa31c0581c0 con 0x7fa32c102e20 2024-04-25T01:59:59.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.561+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa32c109f50 con 0x7fa32c102e20 2024-04-25T01:59:59.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.562+0000 7fa313fff700 1 -- 172.21.15.33:0/4044096708 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7fa31c05b880 con 0x7fa32c102e20 2024-04-25T01:59:59.566 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T01:59:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T01:59:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:59:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:59:59.566 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T01:59:59.567 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T01:59:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa31406e140 msgr2=0x7fa3140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa31406e140 0x7fa3140705f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa32c10f210 tx=0x7fa320009040 comp rx=0 tx=0).stop 2024-04-25T01:59:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 msgr2=0x7fa32c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 0x7fa32c113970 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009100 tx=0x7fa31c009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 shutdown_connections 2024-04-25T01:59:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa31406e140 0x7fa3140705f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa32c102e20 0x7fa32c113970 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 --2- 172.21.15.33:0/4044096708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa32c1037c0 0x7fa32c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 >> 172.21.15.33:0/4044096708 conn(0x7fa32c072380 msgr2=0x7fa32c0face0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 shutdown_connections 2024-04-25T01:59:59.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.564+0000 7fa331fb7700 1 -- 172.21.15.33:0/4044096708 wait complete. 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 -- 172.21.15.33:0/2396855051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0102df0 msgr2=0x7f77a01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 --2- 172.21.15.33:0/2396855051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0102df0 0x7f77a01031c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f779c004940 tx=0x7f779c009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 -- 172.21.15.33:0/2396855051 shutdown_connections 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 --2- 172.21.15.33:0/2396855051 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a0103790 0x7f77a00fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 --2- 172.21.15.33:0/2396855051 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0102df0 0x7f77a01031c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 -- 172.21.15.33:0/2396855051 >> 172.21.15.33:0/2396855051 conn(0x7f77a00f8980 msgr2=0x7f77a00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 -- 172.21.15.33:0/2396855051 shutdown_connections 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 -- 172.21.15.33:0/2396855051 wait complete. 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 Processor -- start 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.669+0000 7f77a7526700 1 -- start start 2024-04-25T01:59:59.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a7526700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a0102df0 0x7f77a0115d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a7526700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0103790 0x7f77a01162b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a7526700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a0116900 con 0x7f77a0103790 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a7526700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a010fcd0 con 0x7f77a0102df0 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a4ac1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0103790 0x7f77a01162b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a4ac1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0103790 0x7f77a01162b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32924/0 (socket says 172.21.15.33:32924) 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a4ac1700 1 -- 172.21.15.33:0/1046196674 learned_addr learned my addr 172.21.15.33:0/1046196674 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a52c2700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a0102df0 0x7f77a0115d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a4ac1700 1 -- 172.21.15.33:0/1046196674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a0102df0 msgr2=0x7f77a0115d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a4ac1700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a0102df0 0x7f77a0115d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a4ac1700 1 -- 172.21.15.33:0/1046196674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f779c005040 con 0x7f77a0103790 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a52c2700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a0102df0 0x7f77a0115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a4ac1700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0103790 0x7f77a01162b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7790002a50 tx=0x7f7790004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:59.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f7795ffb700 1 -- 172.21.15.33:0/1046196674 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7790007cc0 con 0x7f77a0103790 2024-04-25T01:59:59.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f7795ffb700 1 -- 172.21.15.33:0/1046196674 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f779000f040 con 0x7f77a0103790 2024-04-25T01:59:59.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f7795ffb700 1 -- 172.21.15.33:0/1046196674 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f779000daf0 con 0x7f77a0103790 2024-04-25T01:59:59.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.670+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a010ffb0 con 0x7f77a0103790 2024-04-25T01:59:59.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.671+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a0110570 con 0x7f77a0103790 2024-04-25T01:59:59.677 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.672+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a01072f0 con 0x7f77a0103790 2024-04-25T01:59:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.672+0000 7f7795ffb700 1 -- 172.21.15.33:0/1046196674 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7790009810 con 0x7f77a0103790 2024-04-25T01:59:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.672+0000 7f7795ffb700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f778c06e140 0x7f778c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T01:59:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.673+0000 7f7795ffb700 1 -- 172.21.15.33:0/1046196674 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f779008d810 con 0x7f77a0103790 2024-04-25T01:59:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.673+0000 7f77a52c2700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f778c06e140 0x7f778c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T01:59:59.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.673+0000 7f77a52c2700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f778c06e140 0x7f778c0705f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f779c004560 tx=0x7f779c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T01:59:59.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.677+0000 7f7795ffb700 1 -- 172.21.15.33:0/1046196674 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f779005a740 con 0x7f77a0103790 2024-04-25T01:59:59.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.927+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f77a01113c0 con 0x7f77a0103790 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.928+0000 7f7795ffb700 1 -- 172.21.15.33:0/1046196674 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f779005a2d0 con 0x7f77a0103790 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T01:59:59.932 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T01:59:59.933 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:59:59.934 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T01:59:59.935 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T01:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f778c06e140 msgr2=0x7f778c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f778c06e140 0x7f778c0705f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f779c004560 tx=0x7f779c009e30 comp rx=0 tx=0).stop 2024-04-25T01:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0103790 msgr2=0x7f77a01162b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T01:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0103790 0x7f77a01162b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f7790002a50 tx=0x7f7790004600 comp rx=0 tx=0).stop 2024-04-25T01:59:59.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 shutdown_connections 2024-04-25T01:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f778c06e140 0x7f778c0705f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f77a0102df0 0x7f77a0115d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 --2- 172.21.15.33:0/1046196674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f77a0103790 0x7f77a01162b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T01:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 >> 172.21.15.33:0/1046196674 conn(0x7f77a00f8980 msgr2=0x7f77a00fa600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T01:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 shutdown_connections 2024-04-25T01:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T01:59:59.931+0000 7f77a7526700 1 -- 172.21.15.33:0/1046196674 wait complete. 2024-04-25T01:59:59.936 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T02:00:00.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.037+0000 7f665ee10700 1 -- 172.21.15.33:0/618095373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 msgr2=0x7f6658102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:00.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.037+0000 7f665ee10700 1 --2- 172.21.15.33:0/618095373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 0x7f6658102160 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6650009ce0 tx=0x7f6650004d10 comp rx=0 tx=0).stop 2024-04-25T02:00:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.037+0000 7f665ee10700 1 -- 172.21.15.33:0/618095373 shutdown_connections 2024-04-25T02:00:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.037+0000 7f665ee10700 1 --2- 172.21.15.33:0/618095373 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66580fbd40 0x7f66580fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.037+0000 7f665ee10700 1 --2- 172.21.15.33:0/618095373 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 0x7f6658102160 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.037+0000 7f665ee10700 1 -- 172.21.15.33:0/618095373 >> 172.21.15.33:0/618095373 conn(0x7f6658071d00 msgr2=0x7f6658072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.037+0000 7f665ee10700 1 -- 172.21.15.33:0/618095373 shutdown_connections 2024-04-25T02:00:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.037+0000 7f665ee10700 1 -- 172.21.15.33:0/618095373 wait complete. 2024-04-25T02:00:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.037+0000 7f665ee10700 1 Processor -- start 2024-04-25T02:00:00.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f665ee10700 1 -- start start 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f665ee10700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66580fbd40 0x7f665810ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f665ee10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 0x7f665810f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f665ee10700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665810f8a0 con 0x7f6658101d90 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f665ee10700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665810f9e0 con 0x7f66580fbd40 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f6657fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 0x7f665810f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f6657fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 0x7f665810f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32930/0 (socket says 172.21.15.33:32930) 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f6657fff700 1 -- 172.21.15.33:0/5974123 learned_addr learned my addr 172.21.15.33:0/5974123 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f665cbac700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66580fbd40 0x7f665810ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f6657fff700 1 -- 172.21.15.33:0/5974123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66580fbd40 msgr2=0x7f665810ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f6657fff700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66580fbd40 0x7f665810ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.038+0000 7f6657fff700 1 -- 172.21.15.33:0/5974123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650009940 con 0x7f6658101d90 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.039+0000 7f6657fff700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 0x7f665810f360 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f664c002a50 tx=0x7f664c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.039+0000 7f66557fa700 1 -- 172.21.15.33:0/5974123 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f664c007cc0 con 0x7f6658101d90 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.039+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6658117cd0 con 0x7f6658101d90 2024-04-25T02:00:00.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.039+0000 7f66557fa700 1 -- 172.21.15.33:0/5974123 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f664c00f040 con 0x7f6658101d90 2024-04-25T02:00:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.039+0000 7f66557fa700 1 -- 172.21.15.33:0/5974123 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f664c00da30 con 0x7f6658101d90 2024-04-25T02:00:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.039+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6658118210 con 0x7f6658101d90 2024-04-25T02:00:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.040+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66580fd430 con 0x7f6658101d90 2024-04-25T02:00:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.040+0000 7f66557fa700 1 -- 172.21.15.33:0/5974123 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f664c00dc10 con 0x7f6658101d90 2024-04-25T02:00:00.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.041+0000 7f66557fa700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664006e140 0x7f66400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.041+0000 7f66557fa700 1 -- 172.21.15.33:0/5974123 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f664c08d8b0 con 0x7f6658101d90 2024-04-25T02:00:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.041+0000 7f665cbac700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664006e140 0x7f66400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:00.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.041+0000 7f665cbac700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664006e140 0x7f66400705f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f665000b740 tx=0x7f6650004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:00.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.046+0000 7f66557fa700 1 -- 172.21.15.33:0/5974123 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f664c056c30 con 0x7f6658101d90 2024-04-25T02:00:00.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:59 smithi033 ceph-mon[111306]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:00.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:59 smithi033 ceph-mon[111306]: pgmap v617: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:00.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:59 smithi033 ceph-mon[111306]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:00.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:59 smithi033 ceph-mon[111306]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:00.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:59 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4044096708' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:00.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 01:59:59 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1046196674' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:00:00.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T01:59:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:00:00.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.273+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f665804bed0 con 0x7f664006e140 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.274+0000 7f66557fa700 1 -- 172.21.15.33:0/5974123 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f665804bed0 con 0x7f664006e140 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T02:00:00.278 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:00:00.279 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T02:00:00.279 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:00:00.279 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:00:00.279 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:00:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664006e140 msgr2=0x7f66400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664006e140 0x7f66400705f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f665000b740 tx=0x7f6650004d10 comp rx=0 tx=0).stop 2024-04-25T02:00:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 msgr2=0x7f665810f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 0x7f665810f360 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f664c002a50 tx=0x7f664c0046b0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 shutdown_connections 2024-04-25T02:00:00.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664006e140 0x7f66400705f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66580fbd40 0x7f665810ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 --2- 172.21.15.33:0/5974123 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6658101d90 0x7f665810f360 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 >> 172.21.15.33:0/5974123 conn(0x7f6658071d00 msgr2=0x7f6658104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 shutdown_connections 2024-04-25T02:00:00.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.276+0000 7f665ee10700 1 -- 172.21.15.33:0/5974123 wait complete. 2024-04-25T02:00:00.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:59 smithi138 ceph-mon[81304]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:00.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:59 smithi138 ceph-mon[81304]: pgmap v617: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:00.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:59 smithi138 ceph-mon[81304]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:00.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:59 smithi138 ceph-mon[81304]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:00.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:59 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4044096708' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:00.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 01:59:59 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1046196674' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:00:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.381+0000 7f448a9e3700 1 -- 172.21.15.33:0/2196935115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484101d10 msgr2=0x7f44841020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.381+0000 7f448a9e3700 1 --2- 172.21.15.33:0/2196935115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484101d10 0x7f44841020e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4474004940 tx=0x7f4474009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.381+0000 7f448a9e3700 1 -- 172.21.15.33:0/2196935115 shutdown_connections 2024-04-25T02:00:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.381+0000 7f448a9e3700 1 --2- 172.21.15.33:0/2196935115 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44841026b0 0x7f44840fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.381+0000 7f448a9e3700 1 --2- 172.21.15.33:0/2196935115 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4484101d10 0x7f44841020e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.381+0000 7f448a9e3700 1 -- 172.21.15.33:0/2196935115 >> 172.21.15.33:0/2196935115 conn(0x7f44840f7860 msgr2=0x7f44840f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.381+0000 7f448a9e3700 1 -- 172.21.15.33:0/2196935115 shutdown_connections 2024-04-25T02:00:00.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.381+0000 7f448a9e3700 1 -- 172.21.15.33:0/2196935115 wait complete. 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.382+0000 7f448a9e3700 1 Processor -- start 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.382+0000 7f448a9e3700 1 -- start start 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.382+0000 7f448a9e3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484101d10 0x7f448410c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.382+0000 7f448a9e3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44841026b0 0x7f448410cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.382+0000 7f448a9e3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4484067100 con 0x7f44841026b0 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.382+0000 7f448a9e3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4484067240 con 0x7f4484101d10 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f44837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44841026b0 0x7f448410cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f44837fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44841026b0 0x7f448410cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:32948/0 (socket says 172.21.15.33:32948) 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f44837fe700 1 -- 172.21.15.33:0/4080104546 learned_addr learned my addr 172.21.15.33:0/4080104546 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f4483fff700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484101d10 0x7f448410c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f44837fe700 1 -- 172.21.15.33:0/4080104546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484101d10 msgr2=0x7f448410c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f44837fe700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484101d10 0x7f448410c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f44837fe700 1 -- 172.21.15.33:0/4080104546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4474005040 con 0x7f44841026b0 2024-04-25T02:00:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f44837fe700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44841026b0 0x7f448410cec0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f447800c800 tx=0x7f4478004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f4480ff9700 1 -- 172.21.15.33:0/4080104546 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44780106f0 con 0x7f44841026b0 2024-04-25T02:00:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f4480ff9700 1 -- 172.21.15.33:0/4080104546 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4478010d20 con 0x7f44841026b0 2024-04-25T02:00:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448410d550 con 0x7f44841026b0 2024-04-25T02:00:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f4480ff9700 1 -- 172.21.15.33:0/4080104546 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44780113a0 con 0x7f44841026b0 2024-04-25T02:00:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.383+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448411a1b0 con 0x7f44841026b0 2024-04-25T02:00:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.384+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44840fd460 con 0x7f44841026b0 2024-04-25T02:00:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.385+0000 7f4480ff9700 1 -- 172.21.15.33:0/4080104546 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4478012400 con 0x7f44841026b0 2024-04-25T02:00:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.385+0000 7f4480ff9700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f446c06e140 0x7f446c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.385+0000 7f4480ff9700 1 -- 172.21.15.33:0/4080104546 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f447808da60 con 0x7f44841026b0 2024-04-25T02:00:00.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.385+0000 7f4483fff700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f446c06e140 0x7f446c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:00.390 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.386+0000 7f4483fff700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f446c06e140 0x7f446c0705f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f447400b3b0 tx=0x7f4474009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:00.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.390+0000 7f4480ff9700 1 -- 172.21.15.33:0/4080104546 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4478056e60 con 0x7f44841026b0 2024-04-25T02:00:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.685+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f448404c710 con 0x7f44841026b0 2024-04-25T02:00:00.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.685+0000 7f4480ff9700 1 -- 172.21.15.33:0/4080104546 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f447805a520 con 0x7f44841026b0 2024-04-25T02:00:00.689 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:00:00.689 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:00:00.689 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:00:00.690 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:00:00.690 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.687+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f446c06e140 msgr2=0x7f446c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.687+0000 7f448a9e3700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f446c06e140 0x7f446c0705f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f447400b3b0 tx=0x7f4474009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.687+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44841026b0 msgr2=0x7f448410cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.687+0000 7f448a9e3700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44841026b0 0x7f448410cec0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f447800c800 tx=0x7f4478004470 comp rx=0 tx=0).stop 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.687+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 shutdown_connections 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.687+0000 7f448a9e3700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f446c06e140 0x7f446c0705f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.687+0000 7f448a9e3700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4484101d10 0x7f448410c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.687+0000 7f448a9e3700 1 --2- 172.21.15.33:0/4080104546 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44841026b0 0x7f448410cec0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.687+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 >> 172.21.15.33:0/4080104546 conn(0x7f44840f7860 msgr2=0x7f44840f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.688+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 shutdown_connections 2024-04-25T02:00:00.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:00.688+0000 7f448a9e3700 1 -- 172.21.15.33:0/4080104546 wait complete. 2024-04-25T02:00:01.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:00 smithi033 ceph-mon[111306]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:00:01.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:00 smithi033 ceph-mon[111306]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:01.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:00 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4080104546' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:00:01.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:00 smithi138 ceph-mon[81304]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:00:01.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:00 smithi138 ceph-mon[81304]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:01.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:00 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4080104546' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:00:02.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:01 smithi033 ceph-mon[111306]: pgmap v618: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:02.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:01 smithi138 ceph-mon[81304]: pgmap v618: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:04.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:03 smithi033 ceph-mon[111306]: pgmap v619: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:04.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:03 smithi138 ceph-mon[81304]: pgmap v619: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:06.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:06 smithi033 ceph-mon[111306]: pgmap v620: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:06.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:06 smithi138 ceph-mon[81304]: pgmap v620: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:08.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:08 smithi033 ceph-mon[111306]: pgmap v621: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:08.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:08 smithi138 ceph-mon[81304]: pgmap v621: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:10.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:10 smithi033 ceph-mon[111306]: pgmap v622: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:10.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:10 smithi138 ceph-mon[81304]: pgmap v622: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:11.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:00:11.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:00:12.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:12 smithi138 ceph-mon[81304]: pgmap v623: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:12.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:12 smithi033 ceph-mon[111306]: pgmap v623: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:14.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:14 smithi138 ceph-mon[81304]: pgmap v624: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:14.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:14 smithi033 ceph-mon[111306]: pgmap v624: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:16.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:16 smithi138 ceph-mon[81304]: pgmap v625: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:16.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:16 smithi033 ceph-mon[111306]: pgmap v625: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:18.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:18 smithi138 ceph-mon[81304]: pgmap v626: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:18.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:18 smithi033 ceph-mon[111306]: pgmap v626: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:20.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:20 smithi138 ceph-mon[81304]: pgmap v627: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:20.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:20 smithi033 ceph-mon[111306]: pgmap v627: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:22.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:22 smithi138 ceph-mon[81304]: pgmap v628: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:22.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:22 smithi033 ceph-mon[111306]: pgmap v628: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:24.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:24 smithi138 ceph-mon[81304]: pgmap v629: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:24.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:24 smithi033 ceph-mon[111306]: pgmap v629: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:26.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:26 smithi138 ceph-mon[81304]: pgmap v630: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:26.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:00:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:26 smithi033 ceph-mon[111306]: pgmap v630: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:00:28.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:28 smithi033 ceph-mon[111306]: pgmap v631: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:28.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:28 smithi138 ceph-mon[81304]: pgmap v631: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:30.347 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:30 smithi138 ceph-mon[81304]: pgmap v632: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:30.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:30 smithi033 ceph-mon[111306]: pgmap v632: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.806+0000 7f95e9e0c700 1 -- 172.21.15.33:0/2792468645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 msgr2=0x7f95e4109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.806+0000 7f95e9e0c700 1 --2- 172.21.15.33:0/2792468645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 0x7f95e4109180 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f95cc004940 tx=0x7f95cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.806+0000 7f95e9e0c700 1 -- 172.21.15.33:0/2792468645 shutdown_connections 2024-04-25T02:00:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.806+0000 7f95e9e0c700 1 --2- 172.21.15.33:0/2792468645 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 0x7f95e4109180 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.806+0000 7f95e9e0c700 1 --2- 172.21.15.33:0/2792468645 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95e4070280 0x7f95e4070650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.806+0000 7f95e9e0c700 1 -- 172.21.15.33:0/2792468645 >> 172.21.15.33:0/2792468645 conn(0x7f95e40f8980 msgr2=0x7f95e40fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.806+0000 7f95e9e0c700 1 -- 172.21.15.33:0/2792468645 shutdown_connections 2024-04-25T02:00:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.806+0000 7f95e9e0c700 1 -- 172.21.15.33:0/2792468645 wait complete. 2024-04-25T02:00:30.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.807+0000 7f95e9e0c700 1 Processor -- start 2024-04-25T02:00:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.807+0000 7f95e9e0c700 1 -- start start 2024-04-25T02:00:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.807+0000 7f95e9e0c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95e4070280 0x7f95e4115de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.807+0000 7f95e9e0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 0x7f95e4116320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.807+0000 7f95e9e0c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e4116970 con 0x7f95e4070b90 2024-04-25T02:00:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.807+0000 7f95e9e0c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e410fd40 con 0x7f95e4070280 2024-04-25T02:00:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.807+0000 7f95e2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 0x7f95e4116320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.807+0000 7f95e2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 0x7f95e4116320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34584/0 (socket says 172.21.15.33:34584) 2024-04-25T02:00:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.807+0000 7f95e2ffd700 1 -- 172.21.15.33:0/1484170906 learned_addr learned my addr 172.21.15.33:0/1484170906 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:00:30.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95e37fe700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95e4070280 0x7f95e4115de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95e2ffd700 1 -- 172.21.15.33:0/1484170906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95e4070280 msgr2=0x7f95e4115de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95e2ffd700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95e4070280 0x7f95e4115de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95e2ffd700 1 -- 172.21.15.33:0/1484170906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc005040 con 0x7f95e4070b90 2024-04-25T02:00:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95e2ffd700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 0x7f95e4116320 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b3b0 tx=0x7f95cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95dbfff700 1 -- 172.21.15.33:0/1484170906 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95cc00e040 con 0x7f95e4070b90 2024-04-25T02:00:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95dbfff700 1 -- 172.21.15.33:0/1484170906 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95cc013420 con 0x7f95e4070b90 2024-04-25T02:00:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e410ffc0 con 0x7f95e4070b90 2024-04-25T02:00:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95dbfff700 1 -- 172.21.15.33:0/1484170906 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95cc01c630 con 0x7f95e4070b90 2024-04-25T02:00:30.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.808+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e4110500 con 0x7f95e4070b90 2024-04-25T02:00:30.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.809+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e40719c0 con 0x7f95e4070b90 2024-04-25T02:00:30.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.810+0000 7f95dbfff700 1 -- 172.21.15.33:0/1484170906 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f95cc013590 con 0x7f95e4070b90 2024-04-25T02:00:30.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.810+0000 7f95dbfff700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f95d006e140 0x7f95d00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:30.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.810+0000 7f95dbfff700 1 -- 172.21.15.33:0/1484170906 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f95cc08ec30 con 0x7f95e4070b90 2024-04-25T02:00:30.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.810+0000 7f95e37fe700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f95d006e140 0x7f95d00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:30.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.811+0000 7f95e37fe700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f95d006e140 0x7f95d00705f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f95dc007a40 tx=0x7f95dc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:30.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:30.815+0000 7f95dbfff700 1 -- 172.21.15.33:0/1484170906 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f95cc05bc00 con 0x7f95e4070b90 2024-04-25T02:00:31.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.044+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95e410c140 con 0x7f95d006e140 2024-04-25T02:00:31.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.045+0000 7f95dbfff700 1 -- 172.21.15.33:0/1484170906 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f95e410c140 con 0x7f95d006e140 2024-04-25T02:00:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f95d006e140 msgr2=0x7f95d00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f95d006e140 0x7f95d00705f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f95dc007a40 tx=0x7f95dc009da0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 msgr2=0x7f95e4116320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 0x7f95e4116320 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f95cc00b3b0 tx=0x7f95cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 shutdown_connections 2024-04-25T02:00:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f95d006e140 0x7f95d00705f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f95e4070280 0x7f95e4115de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 --2- 172.21.15.33:0/1484170906 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f95e4070b90 0x7f95e4116320 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 >> 172.21.15.33:0/1484170906 conn(0x7f95e40f8980 msgr2=0x7f95e40f9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 shutdown_connections 2024-04-25T02:00:31.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.047+0000 7f95e9e0c700 1 -- 172.21.15.33:0/1484170906 wait complete. 2024-04-25T02:00:31.062 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:00:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.153+0000 7f5c75ab8700 1 -- 172.21.15.33:0/2905474978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70102dc0 msgr2=0x7f5c70103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.153+0000 7f5c75ab8700 1 --2- 172.21.15.33:0/2905474978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70102dc0 0x7f5c70103190 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f5c60004940 tx=0x7f5c60009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.153+0000 7f5c75ab8700 1 -- 172.21.15.33:0/2905474978 shutdown_connections 2024-04-25T02:00:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.153+0000 7f5c75ab8700 1 --2- 172.21.15.33:0/2905474978 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c70103760 0x7f5c700fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.153+0000 7f5c75ab8700 1 --2- 172.21.15.33:0/2905474978 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70102dc0 0x7f5c70103190 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.153+0000 7f5c75ab8700 1 -- 172.21.15.33:0/2905474978 >> 172.21.15.33:0/2905474978 conn(0x7f5c700f8930 msgr2=0x7f5c700fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:31.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.153+0000 7f5c75ab8700 1 -- 172.21.15.33:0/2905474978 shutdown_connections 2024-04-25T02:00:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.153+0000 7f5c75ab8700 1 -- 172.21.15.33:0/2905474978 wait complete. 2024-04-25T02:00:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c75ab8700 1 Processor -- start 2024-04-25T02:00:31.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c75ab8700 1 -- start start 2024-04-25T02:00:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c75ab8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c70102dc0 0x7f5c70113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c75ab8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70103760 0x7f5c70113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c75ab8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c701144f0 con 0x7f5c70103760 2024-04-25T02:00:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c75ab8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7010daf0 con 0x7f5c70102dc0 2024-04-25T02:00:31.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c6effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70103760 0x7f5c70113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c6effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70103760 0x7f5c70113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34608/0 (socket says 172.21.15.33:34608) 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c6effd700 1 -- 172.21.15.33:0/1175140098 learned_addr learned my addr 172.21.15.33:0/1175140098 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.154+0000 7f5c6f7fe700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c70102dc0 0x7f5c70113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.155+0000 7f5c6effd700 1 -- 172.21.15.33:0/1175140098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c70102dc0 msgr2=0x7f5c70113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.155+0000 7f5c6effd700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c70102dc0 0x7f5c70113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.155+0000 7f5c6effd700 1 -- 172.21.15.33:0/1175140098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60005040 con 0x7f5c70103760 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.155+0000 7f5c6effd700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70103760 0x7f5c70113ea0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5c64002a50 tx=0x7f5c64004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.155+0000 7f5c57fff700 1 -- 172.21.15.33:0/1175140098 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c64007cc0 con 0x7f5c70103760 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.155+0000 7f5c57fff700 1 -- 172.21.15.33:0/1175140098 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c6400f040 con 0x7f5c70103760 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.155+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7010ddd0 con 0x7f5c70103760 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.155+0000 7f5c57fff700 1 -- 172.21.15.33:0/1175140098 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c6400d9c0 con 0x7f5c70103760 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.155+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7010e390 con 0x7f5c70103760 2024-04-25T02:00:31.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.156+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c700fe510 con 0x7f5c70103760 2024-04-25T02:00:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.157+0000 7f5c57fff700 1 -- 172.21.15.33:0/1175140098 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5c64009730 con 0x7f5c70103760 2024-04-25T02:00:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.157+0000 7f5c57fff700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5c5806e140 0x7f5c580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.157+0000 7f5c57fff700 1 -- 172.21.15.33:0/1175140098 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5c6408d890 con 0x7f5c70103760 2024-04-25T02:00:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.157+0000 7f5c6f7fe700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5c5806e140 0x7f5c580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:31.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.158+0000 7f5c6f7fe700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5c5806e140 0x7f5c580705f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5c60004f90 tx=0x7f5c60009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:31.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.162+0000 7f5c57fff700 1 -- 172.21.15.33:0/1175140098 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5c6405a740 con 0x7f5c70103760 2024-04-25T02:00:31.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.393+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c70109f60 con 0x7f5c5806e140 2024-04-25T02:00:31.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.394+0000 7f5c57fff700 1 -- 172.21.15.33:0/1175140098 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5c6000b5c0 con 0x7f5c5806e140 2024-04-25T02:00:31.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5c5806e140 msgr2=0x7f5c580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5c5806e140 0x7f5c580705f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5c60004f90 tx=0x7f5c60009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:31.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70103760 msgr2=0x7f5c70113ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70103760 0x7f5c70113ea0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5c64002a50 tx=0x7f5c64004600 comp rx=0 tx=0).stop 2024-04-25T02:00:31.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 shutdown_connections 2024-04-25T02:00:31.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5c5806e140 0x7f5c580705f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5c70102dc0 0x7f5c70113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 --2- 172.21.15.33:0/1175140098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5c70103760 0x7f5c70113ea0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 >> 172.21.15.33:0/1175140098 conn(0x7f5c700f8930 msgr2=0x7f5c700f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:31.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 shutdown_connections 2024-04-25T02:00:31.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.396+0000 7f5c75ab8700 1 -- 172.21.15.33:0/1175140098 wait complete. 2024-04-25T02:00:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.500+0000 7f2f05760700 1 -- 172.21.15.33:0/1762242279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 msgr2=0x7f2f000fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.500+0000 7f2f05760700 1 --2- 172.21.15.33:0/1762242279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 0x7f2f000fbd10 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8004940 tx=0x7f2ee8009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.500+0000 7f2f05760700 1 -- 172.21.15.33:0/1762242279 shutdown_connections 2024-04-25T02:00:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.500+0000 7f2f05760700 1 --2- 172.21.15.33:0/1762242279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 0x7f2f000fbd10 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.500+0000 7f2f05760700 1 --2- 172.21.15.33:0/1762242279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f00101d00 0x7f2f001020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.500+0000 7f2f05760700 1 -- 172.21.15.33:0/1762242279 >> 172.21.15.33:0/1762242279 conn(0x7f2f000f7870 msgr2=0x7f2f000f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.500+0000 7f2f05760700 1 -- 172.21.15.33:0/1762242279 shutdown_connections 2024-04-25T02:00:31.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.500+0000 7f2f05760700 1 -- 172.21.15.33:0/1762242279 wait complete. 2024-04-25T02:00:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.501+0000 7f2f05760700 1 Processor -- start 2024-04-25T02:00:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.501+0000 7f2f05760700 1 -- start start 2024-04-25T02:00:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.501+0000 7f2f05760700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f00101d00 0x7f2f00112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.501+0000 7f2f05760700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 0x7f2f00112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.501+0000 7f2f05760700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f00113410 con 0x7f2f001026a0 2024-04-25T02:00:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.501+0000 7f2f05760700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0010ca10 con 0x7f2f00101d00 2024-04-25T02:00:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2efe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 0x7f2f00112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2efe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 0x7f2f00112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34628/0 (socket says 172.21.15.33:34628) 2024-04-25T02:00:31.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2efe7fc700 1 -- 172.21.15.33:0/657307769 learned_addr learned my addr 172.21.15.33:0/657307769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2efeffd700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f00101d00 0x7f2f00112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2efe7fc700 1 -- 172.21.15.33:0/657307769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f00101d00 msgr2=0x7f2f00112880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2efe7fc700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f00101d00 0x7f2f00112880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2efe7fc700 1 -- 172.21.15.33:0/657307769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee8005040 con 0x7f2f001026a0 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2efeffd700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f00101d00 0x7f2f00112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2efe7fc700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 0x7f2f00112dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8000c00 tx=0x7f2ee8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2ef77fe700 1 -- 172.21.15.33:0/657307769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ee800e070 con 0x7f2f001026a0 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2ef77fe700 1 -- 172.21.15.33:0/657307769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ee801a950 con 0x7f2f001026a0 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0010cc90 con 0x7f2f001026a0 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2ef77fe700 1 -- 172.21.15.33:0/657307769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ee80238f0 con 0x7f2f001026a0 2024-04-25T02:00:31.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.502+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0010d170 con 0x7f2f001026a0 2024-04-25T02:00:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.503+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f000fd450 con 0x7f2f001026a0 2024-04-25T02:00:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.504+0000 7f2ef77fe700 1 -- 172.21.15.33:0/657307769 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2ee801aac0 con 0x7f2f001026a0 2024-04-25T02:00:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.504+0000 7f2ef77fe700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2eec06e140 0x7f2eec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.504+0000 7f2ef77fe700 1 -- 172.21.15.33:0/657307769 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2ee80915c0 con 0x7f2f001026a0 2024-04-25T02:00:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.504+0000 7f2efeffd700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2eec06e140 0x7f2eec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:31.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.505+0000 7f2efeffd700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2eec06e140 0x7f2eec0705f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0004600 tx=0x7f2ef000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:31.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.509+0000 7f2ef77fe700 1 -- 172.21.15.33:0/657307769 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2ee805e4f0 con 0x7f2f001026a0 2024-04-25T02:00:31.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.728+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f00108de0 con 0x7f2eec06e140 2024-04-25T02:00:31.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.739+0000 7f2ef77fe700 1 -- 172.21.15.33:0/657307769 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2f00108de0 con 0x7f2eec06e140 2024-04-25T02:00:31.743 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:00:31.743 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (21m) 14m ago 32m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:00:31.743 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (18m) 14m ago 33m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:00:31.743 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (18m) 14m ago 32m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:00:31.743 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (21m) 14m ago 32m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (14m) 14m ago 27m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (15m) 14m ago 27m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (14m) 14m ago 27m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (27m) 14m ago 27m 10.7M - 17.2.7-902-gc61472c8 0de407c1b3af a61fd5e4c146 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (23m) 14m ago 34m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (22m) 14m ago 31m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (20m) 14m ago 34m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (19m) 14m ago 31m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (22m) 14m ago 32m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (22m) 14m ago 32m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (18m) 14m ago 31m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (17m) 14m ago 30m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (17m) 14m ago 30m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (16m) 14m ago 29m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:00:31.744 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (16m) 14m ago 29m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:00:31.745 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (15m) 14m ago 28m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:00:31.745 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (21m) 14m ago 32m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2eec06e140 msgr2=0x7f2eec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2eec06e140 0x7f2eec0705f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0004600 tx=0x7f2ef000b040 comp rx=0 tx=0).stop 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 msgr2=0x7f2f00112dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 0x7f2f00112dc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8000c00 tx=0x7f2ee8009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 shutdown_connections 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2eec06e140 0x7f2eec0705f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2f00101d00 0x7f2f00112880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 --2- 172.21.15.33:0/657307769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2f001026a0 0x7f2f00112dc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 >> 172.21.15.33:0/657307769 conn(0x7f2f000f7870 msgr2=0x7f2f000ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 shutdown_connections 2024-04-25T02:00:31.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.741+0000 7f2f05760700 1 -- 172.21.15.33:0/657307769 wait complete. 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.844+0000 7ff394c42700 1 -- 172.21.15.33:0/3215221485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390103790 msgr2=0x7ff3900fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.844+0000 7ff394c42700 1 --2- 172.21.15.33:0/3215221485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390103790 0x7ff3900fce00 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff378004940 tx=0x7ff378009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.844+0000 7ff394c42700 1 -- 172.21.15.33:0/3215221485 shutdown_connections 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.844+0000 7ff394c42700 1 --2- 172.21.15.33:0/3215221485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390103790 0x7ff3900fce00 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.844+0000 7ff394c42700 1 --2- 172.21.15.33:0/3215221485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff390102df0 0x7ff3901031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.844+0000 7ff394c42700 1 -- 172.21.15.33:0/3215221485 >> 172.21.15.33:0/3215221485 conn(0x7ff3900f8980 msgr2=0x7ff3900fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff394c42700 1 -- 172.21.15.33:0/3215221485 shutdown_connections 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff394c42700 1 -- 172.21.15.33:0/3215221485 wait complete. 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff394c42700 1 Processor -- start 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff394c42700 1 -- start start 2024-04-25T02:00:31.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff394c42700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390102df0 0x7ff390113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff394c42700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff390103790 0x7ff3901140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff394c42700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3901146f0 con 0x7ff390102df0 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff394c42700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39010dae0 con 0x7ff390103790 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff38e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390102df0 0x7ff390113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.845+0000 7ff38e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390102df0 0x7ff390113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34654/0 (socket says 172.21.15.33:34654) 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff38e59c700 1 -- 172.21.15.33:0/1078748946 learned_addr learned my addr 172.21.15.33:0/1078748946 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff38dd9b700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff390103790 0x7ff3901140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff38e59c700 1 -- 172.21.15.33:0/1078748946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff390103790 msgr2=0x7ff3901140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff38e59c700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff390103790 0x7ff3901140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff38e59c700 1 -- 172.21.15.33:0/1078748946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff378005040 con 0x7ff390102df0 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff38e59c700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390102df0 0x7ff390113b60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff38800ec30 tx=0x7ff38800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff386ffd700 1 -- 172.21.15.33:0/1078748946 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff388007b30 con 0x7ff390102df0 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff386ffd700 1 -- 172.21.15.33:0/1078748946 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff38800f040 con 0x7ff390102df0 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39010ddc0 con 0x7ff390102df0 2024-04-25T02:00:31.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff386ffd700 1 -- 172.21.15.33:0/1078748946 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff38800cd40 con 0x7ff390102df0 2024-04-25T02:00:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.846+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39010e2d0 con 0x7ff390102df0 2024-04-25T02:00:31.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.847+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3901052c0 con 0x7ff390102df0 2024-04-25T02:00:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.848+0000 7ff386ffd700 1 -- 172.21.15.33:0/1078748946 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff388007c90 con 0x7ff390102df0 2024-04-25T02:00:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.848+0000 7ff386ffd700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff37c06e140 0x7ff37c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.848+0000 7ff386ffd700 1 -- 172.21.15.33:0/1078748946 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff38808e080 con 0x7ff390102df0 2024-04-25T02:00:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.848+0000 7ff38dd9b700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff37c06e140 0x7ff37c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:31.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.849+0000 7ff38dd9b700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff37c06e140 0x7ff37c0705f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff378009b30 tx=0x7ff378009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:31.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:31.853+0000 7ff386ffd700 1 -- 172.21.15.33:0/1078748946 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff38805af30 con 0x7ff390102df0 2024-04-25T02:00:32.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.149+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff39010f1b0 con 0x7ff390102df0 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.149+0000 7ff386ffd700 1 -- 172.21.15.33:0/1078748946 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff38805aac0 con 0x7ff390102df0 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:00:32.154 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 3 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 13 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:00:32.155 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.151+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff37c06e140 msgr2=0x7ff37c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.151+0000 7ff394c42700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff37c06e140 0x7ff37c0705f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff378009b30 tx=0x7ff378009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.151+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390102df0 msgr2=0x7ff390113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.151+0000 7ff394c42700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390102df0 0x7ff390113b60 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff38800ec30 tx=0x7ff38800b040 comp rx=0 tx=0).stop 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.152+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 shutdown_connections 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.152+0000 7ff394c42700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff37c06e140 0x7ff37c0705f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.152+0000 7ff394c42700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff390102df0 0x7ff390113b60 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.152+0000 7ff394c42700 1 --2- 172.21.15.33:0/1078748946 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff390103790 0x7ff3901140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.152+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 >> 172.21.15.33:0/1078748946 conn(0x7ff3900f8980 msgr2=0x7ff390102160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.152+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 shutdown_connections 2024-04-25T02:00:32.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.152+0000 7ff394c42700 1 -- 172.21.15.33:0/1078748946 wait complete. 2024-04-25T02:00:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.255+0000 7f3408b99700 1 -- 172.21.15.33:0/3508311175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 msgr2=0x7f3404103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.255+0000 7f3408b99700 1 --2- 172.21.15.33:0/3508311175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 0x7f3404103220 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f33f4004940 tx=0x7f33f4009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.255+0000 7f3408b99700 1 -- 172.21.15.33:0/3508311175 shutdown_connections 2024-04-25T02:00:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.255+0000 7f3408b99700 1 --2- 172.21.15.33:0/3508311175 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3404065640 0x7f3404065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.255+0000 7f3408b99700 1 --2- 172.21.15.33:0/3508311175 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 0x7f3404103220 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.255+0000 7f3408b99700 1 -- 172.21.15.33:0/3508311175 >> 172.21.15.33:0/3508311175 conn(0x7f3404071d00 msgr2=0x7f3404072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.255+0000 7f3408b99700 1 -- 172.21.15.33:0/3508311175 shutdown_connections 2024-04-25T02:00:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.255+0000 7f3408b99700 1 -- 172.21.15.33:0/3508311175 wait complete. 2024-04-25T02:00:32.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3408b99700 1 Processor -- start 2024-04-25T02:00:32.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3408b99700 1 -- start start 2024-04-25T02:00:32.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3408b99700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3404065640 0x7f3404113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:32.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3408b99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 0x7f34041140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:32.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3408b99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404114710 con 0x7f3404102e50 2024-04-25T02:00:32.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3408b99700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340410db00 con 0x7f3404065640 2024-04-25T02:00:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3401d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 0x7f34041140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3401d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 0x7f34041140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34672/0 (socket says 172.21.15.33:34672) 2024-04-25T02:00:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3401d9b700 1 -- 172.21.15.33:0/885258834 learned_addr learned my addr 172.21.15.33:0/885258834 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:00:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f340259c700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3404065640 0x7f3404113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:32.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3401d9b700 1 -- 172.21.15.33:0/885258834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3404065640 msgr2=0x7f3404113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3401d9b700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3404065640 0x7f3404113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3401d9b700 1 -- 172.21.15.33:0/885258834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f4005040 con 0x7f3404102e50 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.256+0000 7f3401d9b700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 0x7f34041140c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f33f8002a50 tx=0x7f33f80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.257+0000 7f33f2ffd700 1 -- 172.21.15.33:0/885258834 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33f8007cc0 con 0x7f3404102e50 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.257+0000 7f33f2ffd700 1 -- 172.21.15.33:0/885258834 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33f800f040 con 0x7f3404102e50 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.257+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340410dde0 con 0x7f3404102e50 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.257+0000 7f33f2ffd700 1 -- 172.21.15.33:0/885258834 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33f800da30 con 0x7f3404102e50 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.257+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340410e3a0 con 0x7f3404102e50 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.258+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e40052b0 con 0x7f3404102e50 2024-04-25T02:00:32.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.258+0000 7f33f2ffd700 1 -- 172.21.15.33:0/885258834 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f33f800db90 con 0x7f3404102e50 2024-04-25T02:00:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.259+0000 7f33f2ffd700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f33ec06e140 0x7f33ec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.259+0000 7f33f2ffd700 1 -- 172.21.15.33:0/885258834 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f33f808d7e0 con 0x7f3404102e50 2024-04-25T02:00:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.259+0000 7f340259c700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f33ec06e140 0x7f33ec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:32.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.259+0000 7f340259c700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f33ec06e140 0x7f33ec0705f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f33f4004f90 tx=0x7f33f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:32.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.263+0000 7f33f2ffd700 1 -- 172.21.15.33:0/885258834 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f33f805a690 con 0x7f3404102e50 2024-04-25T02:00:32.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:32 smithi033 ceph-mon[111306]: pgmap v633: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:32.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:32 smithi033 ceph-mon[111306]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:32.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.535+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f33e4005d70 con 0x7f3404102e50 2024-04-25T02:00:32.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.535+0000 7f33f2ffd700 1 -- 172.21.15.33:0/885258834 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1874 (secure 0 0 0) 0x7f33f808be60 con 0x7f3404102e50 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:e28 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:00:32.540 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:00:32.541 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:00:32.542 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:00:32.543 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:00:32.543 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:00:32.543 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:00:32.543 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44267} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/1198640100,v1:172.21.15.138:6827/1198640100] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T02:00:32.543 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:00:32.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f33ec06e140 msgr2=0x7f33ec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f33ec06e140 0x7f33ec0705f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f33f4004f90 tx=0x7f33f4009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:32.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 msgr2=0x7f34041140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 0x7f34041140c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f33f8002a50 tx=0x7f33f80046b0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 shutdown_connections 2024-04-25T02:00:32.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f33ec06e140 0x7f33ec0705f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3404065640 0x7f3404113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.544 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 --2- 172.21.15.33:0/885258834 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3404102e50 0x7f34041140c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 >> 172.21.15.33:0/885258834 conn(0x7f3404071d00 msgr2=0x7f34040fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:32.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 shutdown_connections 2024-04-25T02:00:32.545 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.537+0000 7f3408b99700 1 -- 172.21.15.33:0/885258834 wait complete. 2024-04-25T02:00:32.545 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T02:00:32.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:32 smithi138 ceph-mon[81304]: pgmap v633: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:32.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:32 smithi138 ceph-mon[81304]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:32.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.642+0000 7fc783f7a700 1 -- 172.21.15.33:0/2524221120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 msgr2=0x7fc77c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.642+0000 7fc783f7a700 1 --2- 172.21.15.33:0/2524221120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 0x7fc77c1031b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc770004940 tx=0x7fc770009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:32.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.642+0000 7fc783f7a700 1 -- 172.21.15.33:0/2524221120 shutdown_connections 2024-04-25T02:00:32.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.642+0000 7fc783f7a700 1 --2- 172.21.15.33:0/2524221120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c103780 0x7fc77c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.642+0000 7fc783f7a700 1 --2- 172.21.15.33:0/2524221120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 0x7fc77c1031b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.642+0000 7fc783f7a700 1 -- 172.21.15.33:0/2524221120 >> 172.21.15.33:0/2524221120 conn(0x7fc77c0f8990 msgr2=0x7fc77c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:32.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.642+0000 7fc783f7a700 1 -- 172.21.15.33:0/2524221120 shutdown_connections 2024-04-25T02:00:32.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.642+0000 7fc783f7a700 1 -- 172.21.15.33:0/2524221120 wait complete. 2024-04-25T02:00:32.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.643+0000 7fc783f7a700 1 Processor -- start 2024-04-25T02:00:32.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.643+0000 7fc783f7a700 1 -- start start 2024-04-25T02:00:32.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.643+0000 7fc783f7a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 0x7fc77c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:32.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.643+0000 7fc783f7a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c103780 0x7fc77c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:32.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.643+0000 7fc783f7a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c114500 con 0x7fc77c102de0 2024-04-25T02:00:32.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.643+0000 7fc783f7a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c10db50 con 0x7fc77c103780 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc781d16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 0x7fc77c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc781d16700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 0x7fc77c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34690/0 (socket says 172.21.15.33:34690) 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc781d16700 1 -- 172.21.15.33:0/2963249077 learned_addr learned my addr 172.21.15.33:0/2963249077 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc781515700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c103780 0x7fc77c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc781d16700 1 -- 172.21.15.33:0/2963249077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c103780 msgr2=0x7fc77c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc781d16700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c103780 0x7fc77c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc781d16700 1 -- 172.21.15.33:0/2963249077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc770005040 con 0x7fc77c102de0 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc781515700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c103780 0x7fc77c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc781d16700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 0x7fc77c113970 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc770009150 tx=0x7fc770009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc76e7fc700 1 -- 172.21.15.33:0/2963249077 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc77000e040 con 0x7fc77c102de0 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc76e7fc700 1 -- 172.21.15.33:0/2963249077 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc770013420 con 0x7fc77c102de0 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c10ddd0 con 0x7fc77c102de0 2024-04-25T02:00:32.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc76e7fc700 1 -- 172.21.15.33:0/2963249077 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc77001c630 con 0x7fc77c102de0 2024-04-25T02:00:32.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.644+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c10e390 con 0x7fc77c102de0 2024-04-25T02:00:32.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.645+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c1000f0 con 0x7fc77c102de0 2024-04-25T02:00:32.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.646+0000 7fc76e7fc700 1 -- 172.21.15.33:0/2963249077 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc770013590 con 0x7fc77c102de0 2024-04-25T02:00:32.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.646+0000 7fc76e7fc700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc76806e140 0x7fc7680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:32.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.646+0000 7fc76e7fc700 1 -- 172.21.15.33:0/2963249077 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc77008ee40 con 0x7fc77c102de0 2024-04-25T02:00:32.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.647+0000 7fc781515700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc76806e140 0x7fc7680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.647+0000 7fc781515700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc76806e140 0x7fc7680705f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc778007fd0 tx=0x7fc778009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:32.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.651+0000 7fc76e7fc700 1 -- 172.21.15.33:0/2963249077 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc77005bcd0 con 0x7fc77c102de0 2024-04-25T02:00:32.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.879+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc77c10f010 con 0x7fc76806e140 2024-04-25T02:00:32.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.881+0000 7fc76e7fc700 1 -- 172.21.15.33:0/2963249077 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc77c10f010 con 0x7fc76806e140 2024-04-25T02:00:32.885 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:00:32.885 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:00:32.885 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:00:32.885 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:00:32.885 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:00:32.885 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:00:32.885 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T02:00:32.885 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:00:32.886 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T02:00:32.886 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:00:32.886 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T02:00:32.886 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:00:32.886 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:00:32.886 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:00:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc76806e140 msgr2=0x7fc7680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc76806e140 0x7fc7680705f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc778007fd0 tx=0x7fc778009040 comp rx=0 tx=0).stop 2024-04-25T02:00:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 msgr2=0x7fc77c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 0x7fc77c113970 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fc770009150 tx=0x7fc770009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 shutdown_connections 2024-04-25T02:00:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc76806e140 0x7fc7680705f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc77c102de0 0x7fc77c113970 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 --2- 172.21.15.33:0/2963249077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc77c103780 0x7fc77c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 >> 172.21.15.33:0/2963249077 conn(0x7fc77c0f8990 msgr2=0x7fc77c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 shutdown_connections 2024-04-25T02:00:32.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.883+0000 7fc783f7a700 1 -- 172.21.15.33:0/2963249077 wait complete. 2024-04-25T02:00:32.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.986+0000 7f76543b5700 1 -- 172.21.15.33:0/332592089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c101d10 msgr2=0x7f764c1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 --2- 172.21.15.33:0/332592089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c101d10 0x7f764c1020e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7648004940 tx=0x7f7648009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 -- 172.21.15.33:0/332592089 shutdown_connections 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 --2- 172.21.15.33:0/332592089 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c1026b0 0x7f764c0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 --2- 172.21.15.33:0/332592089 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c101d10 0x7f764c1020e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 -- 172.21.15.33:0/332592089 >> 172.21.15.33:0/332592089 conn(0x7f764c0f7860 msgr2=0x7f764c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 -- 172.21.15.33:0/332592089 shutdown_connections 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 -- 172.21.15.33:0/332592089 wait complete. 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 Processor -- start 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 -- start start 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c101d10 0x7f764c115ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c1026b0 0x7f764c115ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c116640 con 0x7f764c1026b0 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.987+0000 7f76543b5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c10fc20 con 0x7f764c101d10 2024-04-25T02:00:32.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7651950700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c1026b0 0x7f764c115ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7651950700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c1026b0 0x7f764c115ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:34698/0 (socket says 172.21.15.33:34698) 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7651950700 1 -- 172.21.15.33:0/3817874028 learned_addr learned my addr 172.21.15.33:0/3817874028 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7652151700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c101d10 0x7f764c115ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7651950700 1 -- 172.21.15.33:0/3817874028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c101d10 msgr2=0x7f764c115ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7651950700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c101d10 0x7f764c115ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7651950700 1 -- 172.21.15.33:0/3817874028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7648005040 con 0x7f764c1026b0 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7651950700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c1026b0 0x7f764c115ff0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f763c00eb40 tx=0x7f763c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7642ffd700 1 -- 172.21.15.33:0/3817874028 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763c007cc0 con 0x7f764c1026b0 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7642ffd700 1 -- 172.21.15.33:0/3817874028 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f763c00f040 con 0x7f764c1026b0 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c10fea0 con 0x7f764c1026b0 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f7642ffd700 1 -- 172.21.15.33:0/3817874028 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f763c0095e0 con 0x7f764c1026b0 2024-04-25T02:00:32.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.988+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c1103e0 con 0x7f764c1026b0 2024-04-25T02:00:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.989+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c1058e0 con 0x7f764c1026b0 2024-04-25T02:00:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.990+0000 7f7642ffd700 1 -- 172.21.15.33:0/3817874028 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f763c009740 con 0x7f764c1026b0 2024-04-25T02:00:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.990+0000 7f7642ffd700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f763806e140 0x7f76380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:00:32.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.990+0000 7f7642ffd700 1 -- 172.21.15.33:0/3817874028 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f763c08d510 con 0x7f764c1026b0 2024-04-25T02:00:32.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.990+0000 7f7652151700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f763806e140 0x7f76380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:00:32.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.991+0000 7f7652151700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f763806e140 0x7f76380705f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7648004540 tx=0x7f7648009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:00:32.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:32.995+0000 7f7642ffd700 1 -- 172.21.15.33:0/3817874028 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f763c05a4f0 con 0x7f764c1026b0 2024-04-25T02:00:33.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:33 smithi033 ceph-mon[111306]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:33.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:33 smithi033 ceph-mon[111306]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:33.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:33 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1078748946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:33.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:33 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/885258834' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:00:33.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.290+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f764c0fd460 con 0x7f764c1026b0 2024-04-25T02:00:33.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.290+0000 7f7642ffd700 1 -- 172.21.15.33:0/3817874028 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f763c05a080 con 0x7f764c1026b0 2024-04-25T02:00:33.295 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:00:33.295 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:00:33.295 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:00:33.295 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:00:33.295 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:00:33.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.292+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f763806e140 msgr2=0x7f76380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.292+0000 7f76543b5700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f763806e140 0x7f76380705f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7648004540 tx=0x7f7648009e30 comp rx=0 tx=0).stop 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.292+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c1026b0 msgr2=0x7f764c115ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.292+0000 7f76543b5700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c1026b0 0x7f764c115ff0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f763c00eb40 tx=0x7f763c00b040 comp rx=0 tx=0).stop 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.292+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 shutdown_connections 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.292+0000 7f76543b5700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f763806e140 0x7f76380705f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.292+0000 7f76543b5700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f764c101d10 0x7f764c115ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.292+0000 7f76543b5700 1 --2- 172.21.15.33:0/3817874028 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f764c1026b0 0x7f764c115ff0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.292+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 >> 172.21.15.33:0/3817874028 conn(0x7f764c0f7860 msgr2=0x7f764c1082b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.293+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 shutdown_connections 2024-04-25T02:00:33.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:00:33.293+0000 7f76543b5700 1 -- 172.21.15.33:0/3817874028 wait complete. 2024-04-25T02:00:33.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:33 smithi138 ceph-mon[81304]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:33.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:33 smithi138 ceph-mon[81304]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:33.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:33 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1078748946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:33.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:33 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/885258834' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:00:34.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:34 smithi033 ceph-mon[111306]: pgmap v634: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:34.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:34 smithi033 ceph-mon[111306]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:34.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:34 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3817874028' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:00:34.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:34 smithi138 ceph-mon[81304]: pgmap v634: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:34.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:34 smithi138 ceph-mon[81304]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:00:34.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:34 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3817874028' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:00:36.323 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:36 smithi033 ceph-mon[111306]: pgmap v635: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:36.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:36 smithi138 ceph-mon[81304]: pgmap v635: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:38.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:38 smithi033 ceph-mon[111306]: pgmap v636: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:38.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:38 smithi138 ceph-mon[81304]: pgmap v636: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:40.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:40 smithi033 ceph-mon[111306]: pgmap v637: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:40.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:40 smithi138 ceph-mon[81304]: pgmap v637: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:41.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:00:41.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:00:42.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:42 smithi033 ceph-mon[111306]: pgmap v638: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:42.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:42 smithi138 ceph-mon[81304]: pgmap v638: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:44.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:44 smithi033 ceph-mon[111306]: pgmap v639: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:44.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:44 smithi138 ceph-mon[81304]: pgmap v639: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:46.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:46 smithi033 ceph-mon[111306]: pgmap v640: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:46.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:46 smithi138 ceph-mon[81304]: pgmap v640: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:48.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:48 smithi033 ceph-mon[111306]: pgmap v641: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:48.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:48 smithi138 ceph-mon[81304]: pgmap v641: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:50.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:50 smithi033 ceph-mon[111306]: pgmap v642: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:50.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:50 smithi138 ceph-mon[81304]: pgmap v642: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:52.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:52 smithi033 ceph-mon[111306]: pgmap v643: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:52.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:52 smithi138 ceph-mon[81304]: pgmap v643: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:54.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:54 smithi033 ceph-mon[111306]: pgmap v644: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:54.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:54 smithi138 ceph-mon[81304]: pgmap v644: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:56.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:56 smithi033 ceph-mon[111306]: pgmap v645: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:56.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:00:56.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:56 smithi138 ceph-mon[81304]: pgmap v645: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:56.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:00:57.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:57 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:00:57.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:57 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:00:57.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:57 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:00:57.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:57 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:00:57.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:57 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:00:57.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:57 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:00:58.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:58 smithi033 ceph-mon[111306]: pgmap v646: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:58.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:58 smithi033 ceph-mon[111306]: pgmap v647: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:58.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:58 smithi033 ceph-mon[111306]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-25T02:00:58.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:00:58.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:58.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:58.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:58.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:58 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:58.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:58 smithi138 ceph-mon[81304]: pgmap v646: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:58.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:58 smithi138 ceph-mon[81304]: pgmap v647: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:00:58.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:58 smithi138 ceph-mon[81304]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-04-25T02:00:58.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:00:58.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:58.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:58.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:00:58.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:58 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:01:00.045 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:59 smithi138 ceph-mon[81304]: Upgrade: Updating mds.cephfs.smithi138.rvxnby 2024-04-25T02:01:00.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:59 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:00.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:59 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.rvxnby", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T02:01:00.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:59 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:01:00.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:59 smithi138 ceph-mon[81304]: Deploying daemon mds.cephfs.smithi138.rvxnby on smithi138 2024-04-25T02:01:00.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:00:59 smithi138 ceph-mon[81304]: pgmap v648: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:00.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:59 smithi033 ceph-mon[111306]: Upgrade: Updating mds.cephfs.smithi138.rvxnby 2024-04-25T02:01:00.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:59 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:00.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:59 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi138.rvxnby", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T02:01:00.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:59 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:01:00.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:59 smithi033 ceph-mon[111306]: Deploying daemon mds.cephfs.smithi138.rvxnby on smithi138 2024-04-25T02:01:00.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:00:59 smithi033 ceph-mon[111306]: pgmap v648: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:02.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:02 smithi033 ceph-mon[111306]: osdmap e84: 6 total, 6 up, 6 in 2024-04-25T02:01:02.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:02 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-25T02:01:02.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:02 smithi033 ceph-mon[111306]: pgmap v650: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:02.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:02 smithi138 ceph-mon[81304]: osdmap e84: 6 total, 6 up, 6 in 2024-04-25T02:01:02.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:02 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-25T02:01:02.271 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:02 smithi138 ceph-mon[81304]: pgmap v650: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.411+0000 7f330459d700 1 -- 172.21.15.33:0/3749904372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 msgr2=0x7f32fc0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.411+0000 7f330459d700 1 --2- 172.21.15.33:0/3749904372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 0x7f32fc0fcdf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f32f8004940 tx=0x7f32f8009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.411+0000 7f330459d700 1 -- 172.21.15.33:0/3749904372 shutdown_connections 2024-04-25T02:01:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.411+0000 7f330459d700 1 --2- 172.21.15.33:0/3749904372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 0x7f32fc0fcdf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.411+0000 7f330459d700 1 --2- 172.21.15.33:0/3749904372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32fc102de0 0x7f32fc1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.411+0000 7f330459d700 1 -- 172.21.15.33:0/3749904372 >> 172.21.15.33:0/3749904372 conn(0x7f32fc0f8990 msgr2=0x7f32fc0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.412+0000 7f330459d700 1 -- 172.21.15.33:0/3749904372 shutdown_connections 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.412+0000 7f330459d700 1 -- 172.21.15.33:0/3749904372 wait complete. 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.412+0000 7f330459d700 1 Processor -- start 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.412+0000 7f330459d700 1 -- start start 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.412+0000 7f330459d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32fc102de0 0x7f32fc113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.412+0000 7f330459d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 0x7f32fc113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.412+0000 7f330459d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc114590 con 0x7f32fc103780 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.412+0000 7f330459d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc10db20 con 0x7f32fc102de0 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.412+0000 7f3301b38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 0x7f32fc113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f3301b38700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 0x7f32fc113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39534/0 (socket says 172.21.15.33:39534) 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f3301b38700 1 -- 172.21.15.33:0/2444609209 learned_addr learned my addr 172.21.15.33:0/2444609209 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f3302339700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32fc102de0 0x7f32fc113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f3301b38700 1 -- 172.21.15.33:0/2444609209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32fc102de0 msgr2=0x7f32fc113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:03.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f3301b38700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32fc102de0 0x7f32fc113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f3301b38700 1 -- 172.21.15.33:0/2444609209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f8005040 con 0x7f32fc103780 2024-04-25T02:01:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f3301b38700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 0x7f32fc113eb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009100 tx=0x7f32f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f32eeffd700 1 -- 172.21.15.33:0/2444609209 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f800e040 con 0x7f32fc103780 2024-04-25T02:01:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc10dda0 con 0x7f32fc103780 2024-04-25T02:01:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f32eeffd700 1 -- 172.21.15.33:0/2444609209 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32f8013420 con 0x7f32fc103780 2024-04-25T02:01:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f32eeffd700 1 -- 172.21.15.33:0/2444609209 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f801c630 con 0x7f32fc103780 2024-04-25T02:01:03.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.413+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc10e360 con 0x7f32fc103780 2024-04-25T02:01:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.414+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32fc1000f0 con 0x7f32fc103780 2024-04-25T02:01:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.415+0000 7f32eeffd700 1 -- 172.21.15.33:0/2444609209 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f32f8013a00 con 0x7f32fc103780 2024-04-25T02:01:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.416+0000 7f32eeffd700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32e806e140 0x7f32e80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.416+0000 7f32eeffd700 1 -- 172.21.15.33:0/2444609209 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f32f808ec50 con 0x7f32fc103780 2024-04-25T02:01:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.416+0000 7f3302339700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32e806e140 0x7f32e80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:03.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.416+0000 7f3302339700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32e806e140 0x7f32e80705f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009bf0 tx=0x7f32f00091a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:03.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.421+0000 7f32eeffd700 1 -- 172.21.15.33:0/2444609209 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f32f8058050 con 0x7f32fc103780 2024-04-25T02:01:03.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.650+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32fc10efc0 con 0x7f32e806e140 2024-04-25T02:01:03.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.651+0000 7f32eeffd700 1 -- 172.21.15.33:0/2444609209 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f32fc10efc0 con 0x7f32e806e140 2024-04-25T02:01:03.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32e806e140 msgr2=0x7f32e80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32e806e140 0x7f32e80705f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009bf0 tx=0x7f32f00091a0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 msgr2=0x7f32fc113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 0x7f32fc113eb0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009100 tx=0x7f32f8009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 shutdown_connections 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32e806e140 0x7f32e80705f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f32fc102de0 0x7f32fc113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 --2- 172.21.15.33:0/2444609209 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f32fc103780 0x7f32fc113eb0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 >> 172.21.15.33:0/2444609209 conn(0x7f32fc0f8990 msgr2=0x7f32fc0fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 shutdown_connections 2024-04-25T02:01:03.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.653+0000 7f330459d700 1 -- 172.21.15.33:0/2444609209 wait complete. 2024-04-25T02:01:03.668 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:01:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 -- 172.21.15.33:0/1872969559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 msgr2=0x7f39100fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 --2- 172.21.15.33:0/1872969559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 0x7f39100fbd10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3904004940 tx=0x7f3904009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 -- 172.21.15.33:0/1872969559 shutdown_connections 2024-04-25T02:01:03.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 --2- 172.21.15.33:0/1872969559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 0x7f39100fbd10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 --2- 172.21.15.33:0/1872969559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3910101d00 0x7f39101020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 -- 172.21.15.33:0/1872969559 >> 172.21.15.33:0/1872969559 conn(0x7f39100f7870 msgr2=0x7f39100f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 -- 172.21.15.33:0/1872969559 shutdown_connections 2024-04-25T02:01:03.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 -- 172.21.15.33:0/1872969559 wait complete. 2024-04-25T02:01:03.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 Processor -- start 2024-04-25T02:01:03.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.759+0000 7f3915d2a700 1 -- start start 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f3915d2a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3910101d00 0x7f3910112880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f3915d2a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 0x7f3910112dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f3915d2a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3910113410 con 0x7f39101026a0 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f3915d2a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f391010ca10 con 0x7f3910101d00 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f390effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 0x7f3910112dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f390effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 0x7f3910112dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39550/0 (socket says 172.21.15.33:39550) 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f390effd700 1 -- 172.21.15.33:0/1194113369 learned_addr learned my addr 172.21.15.33:0/1194113369 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f390f7fe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3910101d00 0x7f3910112880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f390f7fe700 1 -- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 msgr2=0x7f3910112dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f390f7fe700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 0x7f3910112dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:03.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f390f7fe700 1 -- 172.21.15.33:0/1194113369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3904005040 con 0x7f3910101d00 2024-04-25T02:01:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f390effd700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 0x7f3910112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T02:01:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.760+0000 7f390f7fe700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3910101d00 0x7f3910112880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3900009fd0 tx=0x7f390000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.761+0000 7f38f7fff700 1 -- 172.21.15.33:0/1194113369 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f390000ed30 con 0x7f3910101d00 2024-04-25T02:01:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.761+0000 7f38f7fff700 1 -- 172.21.15.33:0/1194113369 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f390000ee90 con 0x7f3910101d00 2024-04-25T02:01:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.761+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f391010cc90 con 0x7f3910101d00 2024-04-25T02:01:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.761+0000 7f38f7fff700 1 -- 172.21.15.33:0/1194113369 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3900012670 con 0x7f3910101d00 2024-04-25T02:01:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.761+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f391010d1d0 con 0x7f3910101d00 2024-04-25T02:01:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.762+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39100fd450 con 0x7f3910101d00 2024-04-25T02:01:03.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.762+0000 7f38f7fff700 1 -- 172.21.15.33:0/1194113369 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3900018030 con 0x7f3910101d00 2024-04-25T02:01:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.763+0000 7f38f7fff700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38f806e140 0x7f38f80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.763+0000 7f38f7fff700 1 -- 172.21.15.33:0/1194113369 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3900016070 con 0x7f3910101d00 2024-04-25T02:01:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.763+0000 7f390effd700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38f806e140 0x7f38f80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:03.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.763+0000 7f390effd700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38f806e140 0x7f38f80705f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3904008fe0 tx=0x7f3904009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:03.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.768+0000 7f38f7fff700 1 -- 172.21.15.33:0/1194113369 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f390005cdd0 con 0x7f3910101d00 2024-04-25T02:01:03.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.994+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3910108e50 con 0x7f38f806e140 2024-04-25T02:01:04.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.996+0000 7f38f7fff700 1 -- 172.21.15.33:0/1194113369 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3910108e50 con 0x7f38f806e140 2024-04-25T02:01:04.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38f806e140 msgr2=0x7f38f80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38f806e140 0x7f38f80705f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3904008fe0 tx=0x7f3904009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3910101d00 msgr2=0x7f3910112880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3910101d00 0x7f3910112880 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3900009fd0 tx=0x7f390000b040 comp rx=0 tx=0).stop 2024-04-25T02:01:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 shutdown_connections 2024-04-25T02:01:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f38f806e140 0x7f38f80705f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3910101d00 0x7f3910112880 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 --2- 172.21.15.33:0/1194113369 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f39101026a0 0x7f3910112dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 >> 172.21.15.33:0/1194113369 conn(0x7f39100f7870 msgr2=0x7f39100f87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 shutdown_connections 2024-04-25T02:01:04.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:03.998+0000 7f3915d2a700 1 -- 172.21.15.33:0/1194113369 wait complete. 2024-04-25T02:01:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.106+0000 7fec19484700 1 -- 172.21.15.33:0/3017079379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 msgr2=0x7fec140fba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.106+0000 7fec19484700 1 --2- 172.21.15.33:0/3017079379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 0x7fec140fba70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fec04004940 tx=0x7fec04009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.106+0000 7fec19484700 1 -- 172.21.15.33:0/3017079379 shutdown_connections 2024-04-25T02:01:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.106+0000 7fec19484700 1 --2- 172.21.15.33:0/3017079379 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 0x7fec140fba70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.106+0000 7fec19484700 1 --2- 172.21.15.33:0/3017079379 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec141014d0 0x7fec141018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.106+0000 7fec19484700 1 -- 172.21.15.33:0/3017079379 >> 172.21.15.33:0/3017079379 conn(0x7fec14071460 msgr2=0x7fec14071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.106+0000 7fec19484700 1 -- 172.21.15.33:0/3017079379 shutdown_connections 2024-04-25T02:01:04.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.106+0000 7fec19484700 1 -- 172.21.15.33:0/3017079379 wait complete. 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec19484700 1 Processor -- start 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec19484700 1 -- start start 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec19484700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 0x7fec14115d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec19484700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec141014d0 0x7fec14116240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec19484700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec14116890 con 0x7fec140fb600 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec19484700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1410fc60 con 0x7fec141014d0 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 0x7fec14115d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec12ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 0x7fec14115d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39568/0 (socket says 172.21.15.33:39568) 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec12ffd700 1 -- 172.21.15.33:0/1602184137 learned_addr learned my addr 172.21.15.33:0/1602184137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:04.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.107+0000 7fec127fc700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec141014d0 0x7fec14116240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.108+0000 7fec12ffd700 1 -- 172.21.15.33:0/1602184137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec141014d0 msgr2=0x7fec14116240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.108+0000 7fec12ffd700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec141014d0 0x7fec14116240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.108+0000 7fec12ffd700 1 -- 172.21.15.33:0/1602184137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec04005040 con 0x7fec140fb600 2024-04-25T02:01:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.108+0000 7fec12ffd700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 0x7fec14115d00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7febfc00eb40 tx=0x7febfc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.108+0000 7fec0b7fe700 1 -- 172.21.15.33:0/1602184137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febfc007cc0 con 0x7fec140fb600 2024-04-25T02:01:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.108+0000 7fec0b7fe700 1 -- 172.21.15.33:0/1602184137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febfc00f040 con 0x7fec140fb600 2024-04-25T02:01:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.108+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1410fee0 con 0x7fec140fb600 2024-04-25T02:01:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.108+0000 7fec0b7fe700 1 -- 172.21.15.33:0/1602184137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febfc00ca10 con 0x7fec140fb600 2024-04-25T02:01:04.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.108+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec141103f0 con 0x7fec140fb600 2024-04-25T02:01:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.109+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec140fcbb0 con 0x7fec140fb600 2024-04-25T02:01:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.110+0000 7fec0b7fe700 1 -- 172.21.15.33:0/1602184137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7febfc00cb70 con 0x7fec140fb600 2024-04-25T02:01:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.110+0000 7fec0b7fe700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec0006e140 0x7fec000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.110+0000 7fec0b7fe700 1 -- 172.21.15.33:0/1602184137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7febfc08d620 con 0x7fec140fb600 2024-04-25T02:01:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.110+0000 7fec127fc700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec0006e140 0x7fec000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.111+0000 7fec127fc700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec0006e140 0x7fec000705f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fec04009b30 tx=0x7fec04009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:04.119 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.115+0000 7fec0b7fe700 1 -- 172.21.15.33:0/1602184137 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7febfc05a600 con 0x7fec140fb600 2024-04-25T02:01:04.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:04 smithi033 ceph-mon[111306]: pgmap v651: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:04.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:04 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:04.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:04 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:04.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:04 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:01:04.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.335+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fec1404bed0 con 0x7fec0006e140 2024-04-25T02:01:04.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.345+0000 7fec0b7fe700 1 -- 172.21.15.33:0/1602184137 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fec1404bed0 con 0x7fec0006e140 2024-04-25T02:01:04.350 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (22m) 15m ago 33m 25.6M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (19m) 15m ago 33m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (19m) 15m ago 32m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (21m) 15m ago 33m 78.5M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (15m) 15m ago 27m 13.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (15m) 15m ago 28m 14.0M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (15m) 15m ago 28m 14.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 starting - - - - 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (23m) 15m ago 34m 509M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (23m) 15m ago 32m 451M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (20m) 15m ago 34m 53.5M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:01:04.351 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (20m) 15m ago 32m 43.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:01:04.352 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (22m) 15m ago 33m 16.3M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:01:04.352 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (22m) 15m ago 32m 15.6M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:01:04.352 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (19m) 15m ago 31m 95.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:01:04.352 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (18m) 15m ago 31m 98.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:01:04.352 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (17m) 15m ago 30m 81.7M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:01:04.352 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (17m) 15m ago 30m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:01:04.352 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (16m) 15m ago 29m 101M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:01:04.352 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (16m) 15m ago 29m 93.2M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:01:04.352 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (22m) 15m ago 32m 61.1M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec0006e140 msgr2=0x7fec000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec0006e140 0x7fec000705f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fec04009b30 tx=0x7fec04009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 msgr2=0x7fec14115d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 0x7fec14115d00 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7febfc00eb40 tx=0x7febfc00b040 comp rx=0 tx=0).stop 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 shutdown_connections 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fec0006e140 0x7fec000705f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec140fb600 0x7fec14115d00 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 --2- 172.21.15.33:0/1602184137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec141014d0 0x7fec14116240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 >> 172.21.15.33:0/1602184137 conn(0x7fec14071460 msgr2=0x7fec140fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 shutdown_connections 2024-04-25T02:01:04.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.348+0000 7fec19484700 1 -- 172.21.15.33:0/1602184137 wait complete. 2024-04-25T02:01:04.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.471+0000 7f48829d9700 1 -- 172.21.15.33:0/3487736307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f487c101cd0 msgr2=0x7f487c1020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.471+0000 7f48829d9700 1 --2- 172.21.15.33:0/3487736307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f487c101cd0 0x7f487c1020a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4864004940 tx=0x7f4864009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:04.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.471+0000 7f48829d9700 1 -- 172.21.15.33:0/3487736307 shutdown_connections 2024-04-25T02:01:04.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.471+0000 7f48829d9700 1 --2- 172.21.15.33:0/3487736307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487c102670 0x7f487c0fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.471+0000 7f48829d9700 1 --2- 172.21.15.33:0/3487736307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f487c101cd0 0x7f487c1020a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.471+0000 7f48829d9700 1 -- 172.21.15.33:0/3487736307 >> 172.21.15.33:0/3487736307 conn(0x7f487c0f7860 msgr2=0x7f487c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:04.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.471+0000 7f48829d9700 1 -- 172.21.15.33:0/3487736307 shutdown_connections 2024-04-25T02:01:04.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.471+0000 7f48829d9700 1 -- 172.21.15.33:0/3487736307 wait complete. 2024-04-25T02:01:04.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f48829d9700 1 Processor -- start 2024-04-25T02:01:04.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f48829d9700 1 -- start start 2024-04-25T02:01:04.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f48829d9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487c101cd0 0x7f487c1138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:04.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f48829d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f487c102670 0x7f487c113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:04.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f48829d9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c114440 con 0x7f487c102670 2024-04-25T02:01:04.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f48829d9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c10da40 con 0x7f487c101cd0 2024-04-25T02:01:04.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f487b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f487c102670 0x7f487c113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f487bfff700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487c101cd0 0x7f487c1138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f487b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f487c102670 0x7f487c113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39590/0 (socket says 172.21.15.33:39590) 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.472+0000 7f487b7fe700 1 -- 172.21.15.33:0/1439655381 learned_addr learned my addr 172.21.15.33:0/1439655381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.473+0000 7f487bfff700 1 -- 172.21.15.33:0/1439655381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f487c102670 msgr2=0x7f487c113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.473+0000 7f487bfff700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f487c102670 0x7f487c113df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.473+0000 7f487bfff700 1 -- 172.21.15.33:0/1439655381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c005040 con 0x7f487c101cd0 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.473+0000 7f487bfff700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487c101cd0 0x7f487c1138b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f486400b0d0 tx=0x7f4864009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.473+0000 7f4878ff9700 1 -- 172.21.15.33:0/1439655381 <== mon.1 v2:172.21.15.138:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f486400e070 con 0x7f487c101cd0 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.473+0000 7f4878ff9700 1 -- 172.21.15.33:0/1439655381 <== mon.1 v2:172.21.15.138:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4864013c00 con 0x7f487c101cd0 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.473+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4864005040 con 0x7f487c101cd0 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.473+0000 7f4878ff9700 1 -- 172.21.15.33:0/1439655381 <== mon.1 v2:172.21.15.138:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4864012600 con 0x7f487c101cd0 2024-04-25T02:01:04.479 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.473+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487c10e0f0 con 0x7f487c101cd0 2024-04-25T02:01:04.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.474+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f487c1063d0 con 0x7f487c101cd0 2024-04-25T02:01:04.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.475+0000 7f4878ff9700 1 -- 172.21.15.33:0/1439655381 <== mon.1 v2:172.21.15.138:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f486400b340 con 0x7f487c101cd0 2024-04-25T02:01:04.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.475+0000 7f4878ff9700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f486806e140 0x7f48680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:04.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.475+0000 7f4878ff9700 1 -- 172.21.15.33:0/1439655381 <== mon.1 v2:172.21.15.138:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f486408e540 con 0x7f487c101cd0 2024-04-25T02:01:04.480 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.475+0000 7f487b7fe700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f486806e140 0x7f48680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:04.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.476+0000 7f487b7fe700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f486806e140 0x7f48680705f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f487c10ec00 tx=0x7f486c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:04.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.480+0000 7f4878ff9700 1 -- 172.21.15.33:0/1439655381 <== mon.1 v2:172.21.15.138:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f486405b510 con 0x7f487c101cd0 2024-04-25T02:01:04.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:04 smithi138 ceph-mon[81304]: pgmap v651: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:04.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:04 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:04.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:04 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:04.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:04 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:01:04.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.782+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f487c10ef00 con 0x7f487c101cd0 2024-04-25T02:01:04.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.783+0000 7f4878ff9700 1 -- 172.21.15.33:0/1439655381 <== mon.1 v2:172.21.15.138:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f486400bea0 con 0x7f487c101cd0 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:01:04.788 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:01:04.789 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:01:04.789 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:01:04.789 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:01:04.789 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f486806e140 msgr2=0x7f48680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f486806e140 0x7f48680705f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f487c10ec00 tx=0x7f486c00b040 comp rx=0 tx=0).stop 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487c101cd0 msgr2=0x7f487c1138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487c101cd0 0x7f487c1138b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f486400b0d0 tx=0x7f4864009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 shutdown_connections 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f486806e140 0x7f48680705f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f487c101cd0 0x7f487c1138b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 --2- 172.21.15.33:0/1439655381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f487c102670 0x7f487c113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 >> 172.21.15.33:0/1439655381 conn(0x7f487c0f7860 msgr2=0x7f487c0f7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 shutdown_connections 2024-04-25T02:01:04.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.786+0000 7f48829d9700 1 -- 172.21.15.33:0/1439655381 wait complete. 2024-04-25T02:01:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.893+0000 7f2a38c89700 1 -- 172.21.15.33:0/3316590747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 msgr2=0x7f2a340fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.893+0000 7f2a38c89700 1 --2- 172.21.15.33:0/3316590747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 0x7f2a340fcb80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2a24004940 tx=0x7f2a24009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.893+0000 7f2a38c89700 1 -- 172.21.15.33:0/3316590747 shutdown_connections 2024-04-25T02:01:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.893+0000 7f2a38c89700 1 --2- 172.21.15.33:0/3316590747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 0x7f2a340fcb80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.893+0000 7f2a38c89700 1 --2- 172.21.15.33:0/3316590747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a34101e20 0x7f2a341021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.893+0000 7f2a38c89700 1 -- 172.21.15.33:0/3316590747 >> 172.21.15.33:0/3316590747 conn(0x7f2a34071460 msgr2=0x7f2a34071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.893+0000 7f2a38c89700 1 -- 172.21.15.33:0/3316590747 shutdown_connections 2024-04-25T02:01:04.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.893+0000 7f2a38c89700 1 -- 172.21.15.33:0/3316590747 wait complete. 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a38c89700 1 Processor -- start 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a38c89700 1 -- start start 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a38c89700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 0x7f2a340fd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a38c89700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a34101e20 0x7f2a340fdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a38c89700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340fe440 con 0x7f2a340fc710 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a38c89700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a340fe580 con 0x7f2a34101e20 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 0x7f2a340fd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a3259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 0x7f2a340fd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39618/0 (socket says 172.21.15.33:39618) 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a3259c700 1 -- 172.21.15.33:0/4050832293 learned_addr learned my addr 172.21.15.33:0/4050832293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:04.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a31d9b700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a34101e20 0x7f2a340fdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a3259c700 1 -- 172.21.15.33:0/4050832293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a34101e20 msgr2=0x7f2a340fdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a3259c700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a34101e20 0x7f2a340fdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.894+0000 7f2a3259c700 1 -- 172.21.15.33:0/4050832293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a24005040 con 0x7f2a340fc710 2024-04-25T02:01:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.895+0000 7f2a3259c700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 0x7f2a340fd970 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2a28002a80 tx=0x7f2a280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.895+0000 7f2a22ffd700 1 -- 172.21.15.33:0/4050832293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a28007cc0 con 0x7f2a340fc710 2024-04-25T02:01:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.895+0000 7f2a22ffd700 1 -- 172.21.15.33:0/4050832293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a2800f040 con 0x7f2a340fc710 2024-04-25T02:01:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.895+0000 7f2a22ffd700 1 -- 172.21.15.33:0/4050832293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2800dac0 con 0x7f2a340fc710 2024-04-25T02:01:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.895+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a34119ec0 con 0x7f2a340fc710 2024-04-25T02:01:04.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.895+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a3411a2c0 con 0x7f2a340fc710 2024-04-25T02:01:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.896+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a34105210 con 0x7f2a340fc710 2024-04-25T02:01:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.896+0000 7f2a22ffd700 1 -- 172.21.15.33:0/4050832293 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2a2800dc20 con 0x7f2a340fc710 2024-04-25T02:01:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.897+0000 7f2a22ffd700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a1c06e150 0x7f2a1c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.897+0000 7f2a22ffd700 1 -- 172.21.15.33:0/4050832293 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2a2808d750 con 0x7f2a340fc710 2024-04-25T02:01:04.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.897+0000 7f2a31d9b700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a1c06e150 0x7f2a1c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:04.903 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.897+0000 7f2a31d9b700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a1c06e150 0x7f2a1c070600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2a340ff2b0 tx=0x7f2a24009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:04.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:04.901+0000 7f2a22ffd700 1 -- 172.21.15.33:0/4050832293 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2a28056ab0 con 0x7f2a340fc710 2024-04-25T02:01:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.180+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2a3404bed0 con 0x7f2a340fc710 2024-04-25T02:01:05.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.180+0000 7f2a22ffd700 1 -- 172.21.15.33:0/4050832293 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f2a2805a170 con 0x7f2a340fc710 2024-04-25T02:01:05.185 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:01:05.185 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:01:05.185 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:01:05.185 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:01:05.185 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:01:05.185 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:01:05.186 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:01:05.187 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:01:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:01:05.188 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:01:05.188 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:01:05.188 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:01:05.188 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:01:05.188 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:01:05.188 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.182+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a1c06e150 msgr2=0x7f2a1c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.182+0000 7f2a38c89700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a1c06e150 0x7f2a1c070600 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2a340ff2b0 tx=0x7f2a24009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.182+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 msgr2=0x7f2a340fd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.182+0000 7f2a38c89700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 0x7f2a340fd970 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2a28002a80 tx=0x7f2a280046b0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.183+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 shutdown_connections 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.183+0000 7f2a38c89700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2a1c06e150 0x7f2a1c070600 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.183+0000 7f2a38c89700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2a340fc710 0x7f2a340fd970 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.183+0000 7f2a38c89700 1 --2- 172.21.15.33:0/4050832293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2a34101e20 0x7f2a340fdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.183+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 >> 172.21.15.33:0/4050832293 conn(0x7f2a34071460 msgr2=0x7f2a340fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:05.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.183+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 shutdown_connections 2024-04-25T02:01:05.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.183+0000 7f2a38c89700 1 -- 172.21.15.33:0/4050832293 wait complete. 2024-04-25T02:01:05.190 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:01:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.290+0000 7f360de77700 1 -- 172.21.15.33:0/100803934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fe7e0 msgr2=0x7f3608106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.290+0000 7f360de77700 1 --2- 172.21.15.33:0/100803934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fe7e0 0x7f3608106e20 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f35f0004940 tx=0x7f35f0009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.290+0000 7f360de77700 1 -- 172.21.15.33:0/100803934 shutdown_connections 2024-04-25T02:01:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.290+0000 7f360de77700 1 --2- 172.21.15.33:0/100803934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fe7e0 0x7f3608106e20 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.290+0000 7f360de77700 1 --2- 172.21.15.33:0/100803934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36080fde40 0x7f36080fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.290+0000 7f360de77700 1 -- 172.21.15.33:0/100803934 >> 172.21.15.33:0/100803934 conn(0x7f36080f7850 msgr2=0x7f36080f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.290+0000 7f360de77700 1 -- 172.21.15.33:0/100803934 shutdown_connections 2024-04-25T02:01:05.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.290+0000 7f360de77700 1 -- 172.21.15.33:0/100803934 wait complete. 2024-04-25T02:01:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f360de77700 1 Processor -- start 2024-04-25T02:01:05.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f360de77700 1 -- start start 2024-04-25T02:01:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f360de77700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fde40 0x7f3608113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f360de77700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36080fe7e0 0x7f3608113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f360de77700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36081145e0 con 0x7f36080fde40 2024-04-25T02:01:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f360de77700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360810d9d0 con 0x7f36080fe7e0 2024-04-25T02:01:05.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f36077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fde40 0x7f3608113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f36077fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fde40 0x7f3608113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39628/0 (socket says 172.21.15.33:39628) 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f36077fe700 1 -- 172.21.15.33:0/1111078449 learned_addr learned my addr 172.21.15.33:0/1111078449 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f3606ffd700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36080fe7e0 0x7f3608113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f36077fe700 1 -- 172.21.15.33:0/1111078449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36080fe7e0 msgr2=0x7f3608113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f36077fe700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36080fe7e0 0x7f3608113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.291+0000 7f36077fe700 1 -- 172.21.15.33:0/1111078449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f0005040 con 0x7f36080fde40 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.292+0000 7f36077fe700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fde40 0x7f3608113a50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3600001db0 tx=0x7f3600009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.292+0000 7f35fffff700 1 -- 172.21.15.33:0/1111078449 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f360000ed50 con 0x7f36080fde40 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.292+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f360810dcb0 con 0x7f36080fde40 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.292+0000 7f35fffff700 1 -- 172.21.15.33:0/1111078449 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3600007df0 con 0x7f36080fde40 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.292+0000 7f35fffff700 1 -- 172.21.15.33:0/1111078449 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3600014640 con 0x7f36080fde40 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.292+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f360810e1c0 con 0x7f36080fde40 2024-04-25T02:01:05.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.293+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36080666f0 con 0x7f36080fde40 2024-04-25T02:01:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.293+0000 7f35fffff700 1 -- 172.21.15.33:0/1111078449 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3600007920 con 0x7f36080fde40 2024-04-25T02:01:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.294+0000 7f35fffff700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f35f406e140 0x7f35f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.294+0000 7f35fffff700 1 -- 172.21.15.33:0/1111078449 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f360008d910 con 0x7f36080fde40 2024-04-25T02:01:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.294+0000 7f3606ffd700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f35f406e140 0x7f35f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:05.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.294+0000 7f3606ffd700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f35f406e140 0x7f35f40705f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f360810ecb0 tx=0x7f35f0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:05.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.299+0000 7f35fffff700 1 -- 172.21.15.33:0/1111078449 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f360005a870 con 0x7f36080fde40 2024-04-25T02:01:05.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:05.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: from='client.44843 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:05.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] up:boot 2024-04-25T02:01:05.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T02:01:05.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T02:01:05.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:05.414 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:05.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:05.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:05.415 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:05 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1439655381' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:01:05.466 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:05.466 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: from='client.44843 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:05.466 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] up:boot 2024-04-25T02:01:05.466 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T02:01:05.466 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T02:01:05.466 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:05.466 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:05.466 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:05.467 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:05.467 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:05 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1439655381' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:01:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.526+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3608109e00 con 0x7f35f406e140 2024-04-25T02:01:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.527+0000 7f35fffff700 1 -- 172.21.15.33:0/1111078449 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3608109e00 con 0x7f35f406e140 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "mon", 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "crash" 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "15/21 daemons upgraded", 2024-04-25T02:01:05.532 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:01:05.533 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:01:05.533 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:01:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.529+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f35f406e140 msgr2=0x7f35f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.529+0000 7f360de77700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f35f406e140 0x7f35f40705f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f360810ecb0 tx=0x7f35f0009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.529+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fde40 msgr2=0x7f3608113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.529+0000 7f360de77700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fde40 0x7f3608113a50 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3600001db0 tx=0x7f3600009ee0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.529+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 shutdown_connections 2024-04-25T02:01:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.529+0000 7f360de77700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f35f406e140 0x7f35f40705f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.529+0000 7f360de77700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f36080fde40 0x7f3608113a50 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.529+0000 7f360de77700 1 --2- 172.21.15.33:0/1111078449 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f36080fe7e0 0x7f3608113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.529+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 >> 172.21.15.33:0/1111078449 conn(0x7f36080f7850 msgr2=0x7f36080f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.530+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 shutdown_connections 2024-04-25T02:01:05.535 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.530+0000 7f360de77700 1 -- 172.21.15.33:0/1111078449 wait complete. 2024-04-25T02:01:05.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.635+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/1400402663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8102f60 msgr2=0x7ff8c8103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.635+0000 7ff8cdb7d700 1 --2- 172.21.15.33:0/1400402663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8102f60 0x7ff8c8103330 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8004940 tx=0x7ff8b8009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.635+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/1400402663 shutdown_connections 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.635+0000 7ff8cdb7d700 1 --2- 172.21.15.33:0/1400402663 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c80fb520 0x7ff8c80fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.635+0000 7ff8cdb7d700 1 --2- 172.21.15.33:0/1400402663 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c8102f60 0x7ff8c8103330 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.635+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/1400402663 >> 172.21.15.33:0/1400402663 conn(0x7ff8c80714d0 msgr2=0x7ff8c80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.635+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/1400402663 shutdown_connections 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.635+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/1400402663 wait complete. 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.636+0000 7ff8cdb7d700 1 Processor -- start 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.636+0000 7ff8cdb7d700 1 -- start start 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.636+0000 7ff8cdb7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c80fb520 0x7ff8c81139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.636+0000 7ff8cdb7d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102f60 0x7ff8c8113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.636+0000 7ff8cdb7d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c8114530 con 0x7ff8c80fb520 2024-04-25T02:01:05.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.636+0000 7ff8cdb7d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c810db30 con 0x7ff8c8102f60 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c80fb520 0x7ff8c81139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8c77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c80fb520 0x7ff8c81139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:39640/0 (socket says 172.21.15.33:39640) 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8c77fe700 1 -- 172.21.15.33:0/2340027650 learned_addr learned my addr 172.21.15.33:0/2340027650 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8c6ffd700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102f60 0x7ff8c8113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8c77fe700 1 -- 172.21.15.33:0/2340027650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102f60 msgr2=0x7ff8c8113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8c77fe700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102f60 0x7ff8c8113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8c77fe700 1 -- 172.21.15.33:0/2340027650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8005040 con 0x7ff8c80fb520 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8c77fe700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c80fb520 0x7ff8c81139a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009b30 tx=0x7ff8b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8affff700 1 -- 172.21.15.33:0/2340027650 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b8021070 con 0x7ff8c80fb520 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c810ddb0 con 0x7ff8c80fb520 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8affff700 1 -- 172.21.15.33:0/2340027650 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8b8013c00 con 0x7ff8c80fb520 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8affff700 1 -- 172.21.15.33:0/2340027650 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8b800e070 con 0x7ff8c80fb520 2024-04-25T02:01:05.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.637+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c810e290 con 0x7ff8c80fb520 2024-04-25T02:01:05.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.638+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c8100250 con 0x7ff8c80fb520 2024-04-25T02:01:05.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.639+0000 7ff8affff700 1 -- 172.21.15.33:0/2340027650 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff8b801cad0 con 0x7ff8c80fb520 2024-04-25T02:01:05.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.639+0000 7ff8affff700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8b006e140 0x7ff8b00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:05.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.639+0000 7ff8affff700 1 -- 172.21.15.33:0/2340027650 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff8b808eed0 con 0x7ff8c80fb520 2024-04-25T02:01:05.644 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.639+0000 7ff8c6ffd700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8b006e140 0x7ff8b00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:05.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.640+0000 7ff8c6ffd700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8b006e140 0x7ff8b00705f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff8c810ecf0 tx=0x7ff8bc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:05.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.644+0000 7ff8affff700 1 -- 172.21.15.33:0/2340027650 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff8b805be00 con 0x7ff8c80fb520 2024-04-25T02:01:05.950 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.946+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff8c810eff0 con 0x7ff8c80fb520 2024-04-25T02:01:05.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.947+0000 7ff8affff700 1 -- 172.21.15.33:0/2340027650 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7ff8b8004490 con 0x7ff8c80fb520 2024-04-25T02:01:05.951 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:01:05.951 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:01:05.951 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:01:05.951 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:01:05.951 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:01:05.951 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:01:05.952 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:01:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:01:05.953 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:01:05.953 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:01:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:01:05.953 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:01:05.953 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:01:05.953 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8b006e140 msgr2=0x7ff8b00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8b006e140 0x7ff8b00705f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff8c810ecf0 tx=0x7ff8bc00b040 comp rx=0 tx=0).stop 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c80fb520 msgr2=0x7ff8c81139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c80fb520 0x7ff8c81139a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009b30 tx=0x7ff8b8009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 shutdown_connections 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8b006e140 0x7ff8b00705f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8c80fb520 0x7ff8c81139a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 --2- 172.21.15.33:0/2340027650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8c8102f60 0x7ff8c8113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 >> 172.21.15.33:0/2340027650 conn(0x7ff8c80714d0 msgr2=0x7ff8c80fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 shutdown_connections 2024-04-25T02:01:05.954 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:05.949+0000 7ff8cdb7d700 1 -- 172.21.15.33:0/2340027650 wait complete. 2024-04-25T02:01:06.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:06 smithi033 ceph-mon[111306]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:06.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:06 smithi033 ceph-mon[111306]: pgmap v652: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:06.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:06 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4050832293' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:01:06.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:06 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2340027650' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:01:06.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:06 smithi138 ceph-mon[81304]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:06.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:06 smithi138 ceph-mon[81304]: pgmap v652: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:06.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:06 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4050832293' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:01:06.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:06 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2340027650' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:01:07.411 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:07 smithi033 ceph-mon[111306]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:07.412 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:07.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:07 smithi138 ceph-mon[81304]: from='client.35236 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:07.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:07.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:08.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:08 smithi033 ceph-mon[111306]: pgmap v653: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:08.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:08 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:08.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:08 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:08.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:08 smithi138 ceph-mon[81304]: pgmap v653: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:08.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:08 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:08.413 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:08 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:10.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:10 smithi033 ceph-mon[111306]: pgmap v654: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:10.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:10.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:10.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:10 smithi138 ceph-mon[81304]: pgmap v654: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:10.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:10.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:11.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:11.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:01:11.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:01:11.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:01:12.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:12 smithi033 ceph-mon[111306]: pgmap v655: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:12.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:12 smithi138 ceph-mon[81304]: pgmap v655: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:14.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:14 smithi033 ceph-mon[111306]: pgmap v656: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:14.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:14 smithi138 ceph-mon[81304]: pgmap v656: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:16.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:16 smithi033 ceph-mon[111306]: pgmap v657: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:16.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:16 smithi138 ceph-mon[81304]: pgmap v657: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:18.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:18 smithi033 ceph-mon[111306]: pgmap v658: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:18.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:18 smithi138 ceph-mon[81304]: pgmap v658: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:20.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:20 smithi033 ceph-mon[111306]: pgmap v659: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:20.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:20 smithi138 ceph-mon[81304]: pgmap v659: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:22.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:22 smithi033 ceph-mon[111306]: pgmap v660: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:22.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:22 smithi138 ceph-mon[81304]: pgmap v660: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:24.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:24 smithi033 ceph-mon[111306]: pgmap v661: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:24.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:24 smithi138 ceph-mon[81304]: pgmap v661: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:25.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:25 smithi033 ceph-mon[111306]: pgmap v662: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:25.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:01:25.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:25 smithi138 ceph-mon[81304]: pgmap v662: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:25.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:01:28.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:28 smithi033 ceph-mon[111306]: pgmap v663: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:28.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:28 smithi138 ceph-mon[81304]: pgmap v663: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:30.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:30 smithi033 ceph-mon[111306]: pgmap v664: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:30.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:30 smithi138 ceph-mon[81304]: pgmap v664: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:32.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:32 smithi033 ceph-mon[111306]: pgmap v665: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:32.532 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:32 smithi138 ceph-mon[81304]: pgmap v665: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:34.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:34 smithi033 ceph-mon[111306]: pgmap v666: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:34.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:34 smithi138 ceph-mon[81304]: pgmap v666: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:36.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.070+0000 7f5a91fba700 1 -- 172.21.15.33:0/526692506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c103790 msgr2=0x7f5a8c0fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.070+0000 7f5a91fba700 1 --2- 172.21.15.33:0/526692506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c103790 0x7f5a8c0fce00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5a80004940 tx=0x7f5a80009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.070+0000 7f5a91fba700 1 -- 172.21.15.33:0/526692506 shutdown_connections 2024-04-25T02:01:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.070+0000 7f5a91fba700 1 --2- 172.21.15.33:0/526692506 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c103790 0x7f5a8c0fce00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.070+0000 7f5a91fba700 1 --2- 172.21.15.33:0/526692506 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a8c102df0 0x7f5a8c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.070+0000 7f5a91fba700 1 -- 172.21.15.33:0/526692506 >> 172.21.15.33:0/526692506 conn(0x7f5a8c0f8980 msgr2=0x7f5a8c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.070+0000 7f5a91fba700 1 -- 172.21.15.33:0/526692506 shutdown_connections 2024-04-25T02:01:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.070+0000 7f5a91fba700 1 -- 172.21.15.33:0/526692506 wait complete. 2024-04-25T02:01:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.071+0000 7f5a91fba700 1 Processor -- start 2024-04-25T02:01:36.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.071+0000 7f5a91fba700 1 -- start start 2024-04-25T02:01:36.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.071+0000 7f5a91fba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c102df0 0x7f5a8c113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:36.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.071+0000 7f5a91fba700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a8c103790 0x7f5a8c1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:36.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.071+0000 7f5a91fba700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c114710 con 0x7f5a8c102df0 2024-04-25T02:01:36.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.071+0000 7f5a91fba700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8c10db00 con 0x7f5a8c103790 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c102df0 0x7f5a8c113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a8b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c102df0 0x7f5a8c113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35140/0 (socket says 172.21.15.33:35140) 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a8b7fe700 1 -- 172.21.15.33:0/295893723 learned_addr learned my addr 172.21.15.33:0/295893723 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a8affd700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a8c103790 0x7f5a8c1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a8b7fe700 1 -- 172.21.15.33:0/295893723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a8c103790 msgr2=0x7f5a8c1140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a8b7fe700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a8c103790 0x7f5a8c1140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a8b7fe700 1 -- 172.21.15.33:0/295893723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a80005040 con 0x7f5a8c102df0 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a8affd700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a8c103790 0x7f5a8c1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a8b7fe700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c102df0 0x7f5a8c113b80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00c8c0 tx=0x7f5a7c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a73fff700 1 -- 172.21.15.33:0/295893723 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a7c00a980 con 0x7f5a8c102df0 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8c10dde0 con 0x7f5a8c102df0 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a73fff700 1 -- 172.21.15.33:0/295893723 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a7c00ce60 con 0x7f5a8c102df0 2024-04-25T02:01:36.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a73fff700 1 -- 172.21.15.33:0/295893723 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a7c0115e0 con 0x7f5a8c102df0 2024-04-25T02:01:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.072+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8c10e3a0 con 0x7f5a8c102df0 2024-04-25T02:01:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.073+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8c0fe540 con 0x7f5a8c102df0 2024-04-25T02:01:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.074+0000 7f5a73fff700 1 -- 172.21.15.33:0/295893723 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5a7c012400 con 0x7f5a8c102df0 2024-04-25T02:01:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.074+0000 7f5a73fff700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a7406e140 0x7f5a740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.074+0000 7f5a73fff700 1 -- 172.21.15.33:0/295893723 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5a7c08dc00 con 0x7f5a8c102df0 2024-04-25T02:01:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.075+0000 7f5a8affd700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a7406e140 0x7f5a740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:36.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.075+0000 7f5a8affd700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a7406e140 0x7f5a740705f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c10ecc0 tx=0x7f5a80009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:36.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.079+0000 7f5a73fff700 1 -- 172.21.15.33:0/295893723 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5a7c093020 con 0x7f5a8c102df0 2024-04-25T02:01:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.305+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a8c109ed0 con 0x7f5a7406e140 2024-04-25T02:01:36.310 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.306+0000 7f5a73fff700 1 -- 172.21.15.33:0/295893723 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a8c109ed0 con 0x7f5a7406e140 2024-04-25T02:01:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.308+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a7406e140 msgr2=0x7f5a740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.308+0000 7f5a91fba700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a7406e140 0x7f5a740705f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c10ecc0 tx=0x7f5a80009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.308+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c102df0 msgr2=0x7f5a8c113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.308+0000 7f5a91fba700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c102df0 0x7f5a8c113b80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00c8c0 tx=0x7f5a7c0044c0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.308+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 shutdown_connections 2024-04-25T02:01:36.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.308+0000 7f5a91fba700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a7406e140 0x7f5a740705f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.308+0000 7f5a91fba700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a8c102df0 0x7f5a8c113b80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.308+0000 7f5a91fba700 1 --2- 172.21.15.33:0/295893723 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a8c103790 0x7f5a8c1140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.308+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 >> 172.21.15.33:0/295893723 conn(0x7f5a8c0f8980 msgr2=0x7f5a8c0f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.309+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 shutdown_connections 2024-04-25T02:01:36.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.309+0000 7f5a91fba700 1 -- 172.21.15.33:0/295893723 wait complete. 2024-04-25T02:01:36.324 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:01:36.335 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:36 smithi033 ceph-mon[111306]: pgmap v667: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:36.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/2778300221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48102e50 msgr2=0x7f6e48103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 --2- 172.21.15.33:0/2778300221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48102e50 0x7f6e48103220 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c004940 tx=0x7f6e3c009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:36.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/2778300221 shutdown_connections 2024-04-25T02:01:36.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 --2- 172.21.15.33:0/2778300221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e48065640 0x7f6e48065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 --2- 172.21.15.33:0/2778300221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48102e50 0x7f6e48103220 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/2778300221 >> 172.21.15.33:0/2778300221 conn(0x7f6e48071d00 msgr2=0x7f6e48072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:36.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/2778300221 shutdown_connections 2024-04-25T02:01:36.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/2778300221 wait complete. 2024-04-25T02:01:36.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 Processor -- start 2024-04-25T02:01:36.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.417+0000 7f6e4ea7a700 1 -- start start 2024-04-25T02:01:36.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4ea7a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48065640 0x7f6e48113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:36.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4ea7a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e48102e50 0x7f6e481140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:36.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4ea7a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e48114710 con 0x7f6e48065640 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4ea7a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4810db00 con 0x7f6e48102e50 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4c816700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48065640 0x7f6e48113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4c816700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48065640 0x7f6e48113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35156/0 (socket says 172.21.15.33:35156) 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4c816700 1 -- 172.21.15.33:0/712059360 learned_addr learned my addr 172.21.15.33:0/712059360 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e47fff700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e48102e50 0x7f6e481140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4c816700 1 -- 172.21.15.33:0/712059360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e48102e50 msgr2=0x7f6e481140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4c816700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e48102e50 0x7f6e481140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.418+0000 7f6e4c816700 1 -- 172.21.15.33:0/712059360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3c005040 con 0x7f6e48065640 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.419+0000 7f6e4c816700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48065640 0x7f6e48113b80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00b3b0 tx=0x7f6e3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.419+0000 7f6e457fa700 1 -- 172.21.15.33:0/712059360 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3c00e040 con 0x7f6e48065640 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.419+0000 7f6e457fa700 1 -- 172.21.15.33:0/712059360 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e3c013420 con 0x7f6e48065640 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.419+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4810dd80 con 0x7f6e48065640 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.419+0000 7f6e457fa700 1 -- 172.21.15.33:0/712059360 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e3c01c630 con 0x7f6e48065640 2024-04-25T02:01:36.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.419+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4810e2f0 con 0x7f6e48065640 2024-04-25T02:01:36.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.420+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e48100100 con 0x7f6e48065640 2024-04-25T02:01:36.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.420+0000 7f6e457fa700 1 -- 172.21.15.33:0/712059360 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6e3c013590 con 0x7f6e48065640 2024-04-25T02:01:36.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.421+0000 7f6e457fa700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6e3806e140 0x7f6e380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:36.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.421+0000 7f6e457fa700 1 -- 172.21.15.33:0/712059360 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6e3c08ed20 con 0x7f6e48065640 2024-04-25T02:01:36.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.421+0000 7f6e47fff700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6e3806e140 0x7f6e380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:36.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.421+0000 7f6e47fff700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6e3806e140 0x7f6e380705f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6e4810ecc0 tx=0x7f6e3400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:36.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.426+0000 7f6e457fa700 1 -- 172.21.15.33:0/712059360 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6e3c05bc50 con 0x7f6e48065640 2024-04-25T02:01:36.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:36 smithi138 ceph-mon[81304]: pgmap v667: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:36.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.654+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e4810f1d0 con 0x7f6e3806e140 2024-04-25T02:01:36.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.656+0000 7f6e457fa700 1 -- 172.21.15.33:0/712059360 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6e4810f1d0 con 0x7f6e3806e140 2024-04-25T02:01:36.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6e3806e140 msgr2=0x7f6e380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6e3806e140 0x7f6e380705f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6e4810ecc0 tx=0x7f6e3400b040 comp rx=0 tx=0).stop 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48065640 msgr2=0x7f6e48113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48065640 0x7f6e48113b80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00b3b0 tx=0x7f6e3c009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 shutdown_connections 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6e3806e140 0x7f6e380705f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6e48065640 0x7f6e48113b80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 --2- 172.21.15.33:0/712059360 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6e48102e50 0x7f6e481140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 >> 172.21.15.33:0/712059360 conn(0x7f6e48071d00 msgr2=0x7f6e480fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 shutdown_connections 2024-04-25T02:01:36.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.658+0000 7f6e4ea7a700 1 -- 172.21.15.33:0/712059360 wait complete. 2024-04-25T02:01:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.781+0000 7f0467f03700 1 -- 172.21.15.33:0/1797788015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460102de0 msgr2=0x7f04601031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.781+0000 7f0467f03700 1 --2- 172.21.15.33:0/1797788015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460102de0 0x7f04601031b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f045c004940 tx=0x7f045c009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.781+0000 7f0467f03700 1 -- 172.21.15.33:0/1797788015 shutdown_connections 2024-04-25T02:01:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.781+0000 7f0467f03700 1 --2- 172.21.15.33:0/1797788015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0460103780 0x7f04600fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.781+0000 7f0467f03700 1 --2- 172.21.15.33:0/1797788015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460102de0 0x7f04601031b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.781+0000 7f0467f03700 1 -- 172.21.15.33:0/1797788015 >> 172.21.15.33:0/1797788015 conn(0x7f04600f8990 msgr2=0x7f04600fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.781+0000 7f0467f03700 1 -- 172.21.15.33:0/1797788015 shutdown_connections 2024-04-25T02:01:36.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.781+0000 7f0467f03700 1 -- 172.21.15.33:0/1797788015 wait complete. 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f0467f03700 1 Processor -- start 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f0467f03700 1 -- start start 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f0467f03700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0460102de0 0x7f0460070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f0467f03700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460103780 0x7f04600707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f0467f03700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04600721b0 con 0x7f0460103780 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f0467f03700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0460070ce0 con 0x7f0460102de0 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f046549e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460103780 0x7f04600707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f046549e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460103780 0x7f04600707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35188/0 (socket says 172.21.15.33:35188) 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f046549e700 1 -- 172.21.15.33:0/2163920526 learned_addr learned my addr 172.21.15.33:0/2163920526 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.782+0000 7f0465c9f700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0460102de0 0x7f0460070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.783+0000 7f046549e700 1 -- 172.21.15.33:0/2163920526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0460102de0 msgr2=0x7f0460070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.783+0000 7f046549e700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0460102de0 0x7f0460070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:36.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.783+0000 7f046549e700 1 -- 172.21.15.33:0/2163920526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c005040 con 0x7f0460103780 2024-04-25T02:01:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.783+0000 7f046549e700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460103780 0x7f04600707a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0450002a50 tx=0x7f04500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.783+0000 7f04567fc700 1 -- 172.21.15.33:0/2163920526 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0450007cc0 con 0x7f0460103780 2024-04-25T02:01:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.783+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0460070fc0 con 0x7f0460103780 2024-04-25T02:01:36.789 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.783+0000 7f04567fc700 1 -- 172.21.15.33:0/2163920526 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f045000f040 con 0x7f0460103780 2024-04-25T02:01:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.783+0000 7f04567fc700 1 -- 172.21.15.33:0/2163920526 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f045000db30 con 0x7f0460103780 2024-04-25T02:01:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.783+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f046011a090 con 0x7f0460103780 2024-04-25T02:01:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.784+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04601000f0 con 0x7f0460103780 2024-04-25T02:01:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.785+0000 7f04567fc700 1 -- 172.21.15.33:0/2163920526 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f045000dd30 con 0x7f0460103780 2024-04-25T02:01:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.785+0000 7f04567fc700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f044c06e140 0x7f044c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:36.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.785+0000 7f04567fc700 1 -- 172.21.15.33:0/2163920526 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f045008d800 con 0x7f0460103780 2024-04-25T02:01:36.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.785+0000 7f0465c9f700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f044c06e140 0x7f044c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:36.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.786+0000 7f0465c9f700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f044c06e140 0x7f044c0705f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f045c004f90 tx=0x7f045c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:36.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:36.790+0000 7f04567fc700 1 -- 172.21.15.33:0/2163920526 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0450056ae0 con 0x7f0460103780 2024-04-25T02:01:37.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.006+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0460109ee0 con 0x7f044c06e140 2024-04-25T02:01:37.030 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.025+0000 7f04567fc700 1 -- 172.21.15.33:0/2163920526 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0460109ee0 con 0x7f044c06e140 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (22m) 27s ago 33m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (20m) 27s ago 34m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (19m) 30s ago 33m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (22m) 27s ago 33m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (15m) 27s ago 28m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (16m) 27s ago 28m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (15m) 30s ago 28m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (33s) 30s ago 28m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:01:37.031 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (24m) 27s ago 35m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (23m) 30s ago 33m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (21m) 27s ago 35m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (20m) 30s ago 33m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (23m) 27s ago 34m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (23m) 30s ago 33m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (19m) 27s ago 32m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (18m) 27s ago 31m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (18m) 27s ago 31m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (17m) 30s ago 30m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:01:37.032 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (17m) 30s ago 30m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:01:37.033 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (16m) 30s ago 30m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:01:37.033 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (23m) 27s ago 33m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:01:37.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f044c06e140 msgr2=0x7f044c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f044c06e140 0x7f044c0705f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f045c004f90 tx=0x7f045c009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:37.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460103780 msgr2=0x7f04600707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460103780 0x7f04600707a0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0450002a50 tx=0x7f04500046b0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 shutdown_connections 2024-04-25T02:01:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f044c06e140 0x7f044c0705f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0460102de0 0x7f0460070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 --2- 172.21.15.33:0/2163920526 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0460103780 0x7f04600707a0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 >> 172.21.15.33:0/2163920526 conn(0x7f04600f8990 msgr2=0x7f04600fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 shutdown_connections 2024-04-25T02:01:37.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.028+0000 7f0467f03700 1 -- 172.21.15.33:0/2163920526 wait complete. 2024-04-25T02:01:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.132+0000 7f8562b13700 1 -- 172.21.15.33:0/18252976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 msgr2=0x7f855c0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.132+0000 7f8562b13700 1 --2- 172.21.15.33:0/18252976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 0x7f855c0fc1b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f8548004940 tx=0x7f8548009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.132+0000 7f8562b13700 1 -- 172.21.15.33:0/18252976 shutdown_connections 2024-04-25T02:01:37.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.132+0000 7f8562b13700 1 --2- 172.21.15.33:0/18252976 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 0x7f855c0fc1b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.132+0000 7f8562b13700 1 --2- 172.21.15.33:0/18252976 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c101d90 0x7f855c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.132+0000 7f8562b13700 1 -- 172.21.15.33:0/18252976 >> 172.21.15.33:0/18252976 conn(0x7f855c071d00 msgr2=0x7f855c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.132+0000 7f8562b13700 1 -- 172.21.15.33:0/18252976 shutdown_connections 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.132+0000 7f8562b13700 1 -- 172.21.15.33:0/18252976 wait complete. 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.133+0000 7f8562b13700 1 Processor -- start 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.133+0000 7f8562b13700 1 -- start start 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.133+0000 7f8562b13700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 0x7f855c102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.133+0000 7f8562b13700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c101d90 0x7f855c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.133+0000 7f8562b13700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c1035f0 con 0x7f855c0fbd40 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.133+0000 7f8562b13700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c103730 con 0x7f855c101d90 2024-04-25T02:01:37.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.133+0000 7f85608af700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 0x7f855c102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:37.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.133+0000 7f85608af700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 0x7f855c102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35210/0 (socket says 172.21.15.33:35210) 2024-04-25T02:01:37.144 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.133+0000 7f85608af700 1 -- 172.21.15.33:0/2123064660 learned_addr learned my addr 172.21.15.33:0/2123064660 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f855bfff700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c101d90 0x7f855c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f85608af700 1 -- 172.21.15.33:0/2123064660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c101d90 msgr2=0x7f855c103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f85608af700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c101d90 0x7f855c103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f85608af700 1 -- 172.21.15.33:0/2123064660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8548005040 con 0x7f855c0fbd40 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f855bfff700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c101d90 0x7f855c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f85608af700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 0x7f855c102b20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8550002a50 tx=0x7f85500046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f85597fa700 1 -- 172.21.15.33:0/2123064660 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8550007cc0 con 0x7f855c0fbd40 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c119e80 con 0x7f855c0fbd40 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f85597fa700 1 -- 172.21.15.33:0/2123064660 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f855000f040 con 0x7f855c0fbd40 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f85597fa700 1 -- 172.21.15.33:0/2123064660 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8550009640 con 0x7f855c0fbd40 2024-04-25T02:01:37.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.134+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c11a300 con 0x7f855c0fbd40 2024-04-25T02:01:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.135+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c0fd430 con 0x7f855c0fbd40 2024-04-25T02:01:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.136+0000 7f85597fa700 1 -- 172.21.15.33:0/2123064660 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f85500097a0 con 0x7f855c0fbd40 2024-04-25T02:01:37.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.136+0000 7f85597fa700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f854c06e140 0x7f854c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.136+0000 7f85597fa700 1 -- 172.21.15.33:0/2123064660 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f855008d920 con 0x7f855c0fbd40 2024-04-25T02:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.136+0000 7f855bfff700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f854c06e140 0x7f854c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.137+0000 7f855bfff700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f854c06e140 0x7f854c0705f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8548000c00 tx=0x7f8548009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:37.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.141+0000 7f85597fa700 1 -- 172.21.15.33:0/2123064660 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8550056ca0 con 0x7f855c0fbd40 2024-04-25T02:01:37.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:37 smithi033 ceph-mon[111306]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:37.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.445+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f855c05f610 con 0x7f855c0fbd40 2024-04-25T02:01:37.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.445+0000 7f85597fa700 1 -- 172.21.15.33:0/2123064660 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f855005a360 con 0x7f855c0fbd40 2024-04-25T02:01:37.450 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:01:37.450 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:01:37.450 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:01:37.450 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:01:37.451 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:01:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f854c06e140 msgr2=0x7f854c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f854c06e140 0x7f854c0705f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8548000c00 tx=0x7f8548009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 msgr2=0x7f855c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 0x7f855c102b20 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f8550002a50 tx=0x7f85500046b0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 shutdown_connections 2024-04-25T02:01:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f854c06e140 0x7f854c0705f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f855c0fbd40 0x7f855c102b20 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 --2- 172.21.15.33:0/2123064660 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f855c101d90 0x7f855c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 >> 172.21.15.33:0/2123064660 conn(0x7f855c071d00 msgr2=0x7f855c0fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 shutdown_connections 2024-04-25T02:01:37.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.447+0000 7f8562b13700 1 -- 172.21.15.33:0/2123064660 wait complete. 2024-04-25T02:01:37.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:37 smithi138 ceph-mon[81304]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.556+0000 7fcfb3609700 1 -- 172.21.15.33:0/1703019070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 msgr2=0x7fcfac0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.556+0000 7fcfb3609700 1 --2- 172.21.15.33:0/1703019070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 0x7fcfac0fc1b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8004940 tx=0x7fcfa8009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.556+0000 7fcfb3609700 1 -- 172.21.15.33:0/1703019070 shutdown_connections 2024-04-25T02:01:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.556+0000 7fcfb3609700 1 --2- 172.21.15.33:0/1703019070 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 0x7fcfac0fc1b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.556+0000 7fcfb3609700 1 --2- 172.21.15.33:0/1703019070 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfac101d90 0x7fcfac102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.556+0000 7fcfb3609700 1 -- 172.21.15.33:0/1703019070 >> 172.21.15.33:0/1703019070 conn(0x7fcfac071d00 msgr2=0x7fcfac072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.556+0000 7fcfb3609700 1 -- 172.21.15.33:0/1703019070 shutdown_connections 2024-04-25T02:01:37.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.556+0000 7fcfb3609700 1 -- 172.21.15.33:0/1703019070 wait complete. 2024-04-25T02:01:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb3609700 1 Processor -- start 2024-04-25T02:01:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb3609700 1 -- start start 2024-04-25T02:01:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb3609700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 0x7fcfac10eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb3609700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfac101d90 0x7fcfac10f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:37.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb3609700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac10f6c0 con 0x7fcfac0fbd40 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb3609700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac10f800 con 0x7fcfac101d90 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb13a5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 0x7fcfac10eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb13a5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 0x7fcfac10eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35238/0 (socket says 172.21.15.33:35238) 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb13a5700 1 -- 172.21.15.33:0/3918770859 learned_addr learned my addr 172.21.15.33:0/3918770859 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb0ba4700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfac101d90 0x7fcfac10f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb13a5700 1 -- 172.21.15.33:0/3918770859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfac101d90 msgr2=0x7fcfac10f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb13a5700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfac101d90 0x7fcfac10f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.557+0000 7fcfb13a5700 1 -- 172.21.15.33:0/3918770859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa8005040 con 0x7fcfac0fbd40 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.558+0000 7fcfb13a5700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 0x7fcfac10eb30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c830 tx=0x7fcfa0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.558+0000 7fcf9dffb700 1 -- 172.21.15.33:0/3918770859 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfa000a760 con 0x7fcfac0fbd40 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.558+0000 7fcf9dffb700 1 -- 172.21.15.33:0/3918770859 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfa000ad90 con 0x7fcfac0fbd40 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.558+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac117ba0 con 0x7fcfac0fbd40 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.558+0000 7fcf9dffb700 1 -- 172.21.15.33:0/3918770859 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfa00113a0 con 0x7fcfac0fbd40 2024-04-25T02:01:37.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.558+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac118100 con 0x7fcfac0fbd40 2024-04-25T02:01:37.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.559+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac0fd430 con 0x7fcfac0fbd40 2024-04-25T02:01:37.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.559+0000 7fcf9dffb700 1 -- 172.21.15.33:0/3918770859 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcfa0012400 con 0x7fcfac0fbd40 2024-04-25T02:01:37.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.560+0000 7fcf9dffb700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf9806e140 0x7fcf980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:37.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.560+0000 7fcf9dffb700 1 -- 172.21.15.33:0/3918770859 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcfa008db40 con 0x7fcfac0fbd40 2024-04-25T02:01:37.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.560+0000 7fcfb0ba4700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf9806e140 0x7fcf980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:37.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.560+0000 7fcfb0ba4700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf9806e140 0x7fcf980705f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800b3b0 tx=0x7fcfa8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:37.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.565+0000 7fcf9dffb700 1 -- 172.21.15.33:0/3918770859 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcfa0056f40 con 0x7fcfac0fbd40 2024-04-25T02:01:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.819+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcfac065d90 con 0x7fcfac0fbd40 2024-04-25T02:01:37.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.820+0000 7fcf9dffb700 1 -- 172.21.15.33:0/3918770859 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fcfa005a600 con 0x7fcfac0fbd40 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:01:37.825 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:01:37.826 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:01:37.827 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:01:37.828 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:01:37.828 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf9806e140 msgr2=0x7fcf980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf9806e140 0x7fcf980705f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800b3b0 tx=0x7fcfa8009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 msgr2=0x7fcfac10eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 0x7fcfac10eb30 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcfa000c830 tx=0x7fcfa0004470 comp rx=0 tx=0).stop 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 shutdown_connections 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf9806e140 0x7fcf980705f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcfac0fbd40 0x7fcfac10eb30 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 --2- 172.21.15.33:0/3918770859 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcfac101d90 0x7fcfac10f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 >> 172.21.15.33:0/3918770859 conn(0x7fcfac071d00 msgr2=0x7fcfac104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 shutdown_connections 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.822+0000 7fcfb3609700 1 -- 172.21.15.33:0/3918770859 wait complete. 2024-04-25T02:01:37.829 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:01:37.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.927+0000 7f6662eea700 1 -- 172.21.15.33:0/3386353358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 msgr2=0x7f665c0fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.927+0000 7f6662eea700 1 --2- 172.21.15.33:0/3386353358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 0x7f665c0fbbc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6650004940 tx=0x7f6650009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:37.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.927+0000 7f6662eea700 1 -- 172.21.15.33:0/3386353358 shutdown_connections 2024-04-25T02:01:37.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.927+0000 7f6662eea700 1 --2- 172.21.15.33:0/3386353358 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0fc190 0x7f665c101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.927+0000 7f6662eea700 1 --2- 172.21.15.33:0/3386353358 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 0x7f665c0fbbc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.927+0000 7f6662eea700 1 -- 172.21.15.33:0/3386353358 >> 172.21.15.33:0/3386353358 conn(0x7f665c071460 msgr2=0x7f665c071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.927+0000 7f6662eea700 1 -- 172.21.15.33:0/3386353358 shutdown_connections 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.927+0000 7f6662eea700 1 -- 172.21.15.33:0/3386353358 wait complete. 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.928+0000 7f6662eea700 1 Processor -- start 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.928+0000 7f6662eea700 1 -- start start 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.928+0000 7f6662eea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 0x7f665c113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6662eea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0fc190 0x7f665c113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6662eea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c114590 con 0x7f665c0fb7f0 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6662eea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c10db90 con 0x7f665c0fc190 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6660c86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 0x7f665c113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6660c86700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 0x7f665c113a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35256/0 (socket says 172.21.15.33:35256) 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6660c86700 1 -- 172.21.15.33:0/1870472730 learned_addr learned my addr 172.21.15.33:0/1870472730 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f665bfff700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0fc190 0x7f665c113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:37.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6660c86700 1 -- 172.21.15.33:0/1870472730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0fc190 msgr2=0x7f665c113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6660c86700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0fc190 0x7f665c113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6660c86700 1 -- 172.21.15.33:0/1870472730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6650005040 con 0x7f665c0fb7f0 2024-04-25T02:01:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f665bfff700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0fc190 0x7f665c113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:01:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6660c86700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 0x7f665c113a00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f665000b3b0 tx=0x7f6650009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f66597fa700 1 -- 172.21.15.33:0/1870472730 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665000e040 con 0x7f665c0fb7f0 2024-04-25T02:01:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f66597fa700 1 -- 172.21.15.33:0/1870472730 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f665000bbc0 con 0x7f665c0fb7f0 2024-04-25T02:01:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c10de10 con 0x7f665c0fb7f0 2024-04-25T02:01:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f66597fa700 1 -- 172.21.15.33:0/1870472730 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f665001c3f0 con 0x7f665c0fb7f0 2024-04-25T02:01:37.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.929+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c10e350 con 0x7f665c0fb7f0 2024-04-25T02:01:37.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.930+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c065f30 con 0x7f665c0fb7f0 2024-04-25T02:01:37.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.931+0000 7f66597fa700 1 -- 172.21.15.33:0/1870472730 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f665000bd30 con 0x7f665c0fb7f0 2024-04-25T02:01:37.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.931+0000 7f66597fa700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664c06e140 0x7f664c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:37.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.932+0000 7f66597fa700 1 -- 172.21.15.33:0/1870472730 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f665008edb0 con 0x7f665c0fb7f0 2024-04-25T02:01:37.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.932+0000 7f665bfff700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664c06e140 0x7f664c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:37.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.932+0000 7f665bfff700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664c06e140 0x7f664c0705f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6648007fd0 tx=0x7f6648009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:37.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:37.936+0000 7f66597fa700 1 -- 172.21.15.33:0/1870472730 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f665005bc60 con 0x7f665c0fb7f0 2024-04-25T02:01:38.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.164+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f665c10e4e0 con 0x7f664c06e140 2024-04-25T02:01:38.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.165+0000 7f66597fa700 1 -- 172.21.15.33:0/1870472730 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f665c10e4e0 con 0x7f664c06e140 2024-04-25T02:01:38.170 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:01:38.170 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:01:38.170 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:01:38.170 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:01:38.170 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:01:38.170 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:01:38.171 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:01:38.171 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:01:38.171 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:01:38.171 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:01:38.171 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:01:38.171 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:01:38.171 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:01:38.171 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:01:38.171 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:01:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.167+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664c06e140 msgr2=0x7f664c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.167+0000 7f6662eea700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664c06e140 0x7f664c0705f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6648007fd0 tx=0x7f6648009040 comp rx=0 tx=0).stop 2024-04-25T02:01:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.167+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 msgr2=0x7f665c113a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:38.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.167+0000 7f6662eea700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 0x7f665c113a00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f665000b3b0 tx=0x7f6650009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.168+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 shutdown_connections 2024-04-25T02:01:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.168+0000 7f6662eea700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f664c06e140 0x7f664c0705f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.168+0000 7f6662eea700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f665c0fb7f0 0x7f665c113a00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.168+0000 7f6662eea700 1 --2- 172.21.15.33:0/1870472730 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f665c0fc190 0x7f665c113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.168+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 >> 172.21.15.33:0/1870472730 conn(0x7f665c071460 msgr2=0x7f665c0f93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.168+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 shutdown_connections 2024-04-25T02:01:38.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.168+0000 7f6662eea700 1 -- 172.21.15.33:0/1870472730 wait complete. 2024-04-25T02:01:38.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.271+0000 7fb983fa9700 1 -- 172.21.15.33:0/110087760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 msgr2=0x7fb97c0fe250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:38.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.271+0000 7fb983fa9700 1 --2- 172.21.15.33:0/110087760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 0x7fb97c0fe250 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb978004970 tx=0x7fb978009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:38.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.272+0000 7fb983fa9700 1 -- 172.21.15.33:0/110087760 shutdown_connections 2024-04-25T02:01:38.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.272+0000 7fb983fa9700 1 --2- 172.21.15.33:0/110087760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb97c0fe820 0x7fb97c106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:38.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.272+0000 7fb983fa9700 1 --2- 172.21.15.33:0/110087760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 0x7fb97c0fe250 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:38.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.272+0000 7fb983fa9700 1 -- 172.21.15.33:0/110087760 >> 172.21.15.33:0/110087760 conn(0x7fb97c0f78e0 msgr2=0x7fb97c0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:38.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.272+0000 7fb983fa9700 1 -- 172.21.15.33:0/110087760 shutdown_connections 2024-04-25T02:01:38.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.272+0000 7fb983fa9700 1 -- 172.21.15.33:0/110087760 wait complete. 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.272+0000 7fb983fa9700 1 Processor -- start 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.272+0000 7fb983fa9700 1 -- start start 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb983fa9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 0x7fb97c1138f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb983fa9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb97c0fe820 0x7fb97c113e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb983fa9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c114480 con 0x7fb97c0fde80 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb983fa9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb97c10da80 con 0x7fb97c0fe820 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb981d45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 0x7fb97c1138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb981d45700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 0x7fb97c1138f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:35272/0 (socket says 172.21.15.33:35272) 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb981d45700 1 -- 172.21.15.33:0/3979971927 learned_addr learned my addr 172.21.15.33:0/3979971927 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb981d45700 1 -- 172.21.15.33:0/3979971927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb97c0fe820 msgr2=0x7fb97c113e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb981d45700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb97c0fe820 0x7fb97c113e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:38.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb981d45700 1 -- 172.21.15.33:0/3979971927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb978005040 con 0x7fb97c0fde80 2024-04-25T02:01:38.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb981d45700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 0x7fb97c1138f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb978000c00 tx=0x7fb978009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:38.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb9727fc700 1 -- 172.21.15.33:0/3979971927 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97800e040 con 0x7fb97c0fde80 2024-04-25T02:01:38.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb9727fc700 1 -- 172.21.15.33:0/3979971927 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb97800bc10 con 0x7fb97c0fde80 2024-04-25T02:01:38.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb97c10dd00 con 0x7fb97c0fde80 2024-04-25T02:01:38.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb9727fc700 1 -- 172.21.15.33:0/3979971927 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb97801c3f0 con 0x7fb97c0fde80 2024-04-25T02:01:38.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.273+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb97c10e240 con 0x7fb97c0fde80 2024-04-25T02:01:38.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.274+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb97c066a30 con 0x7fb97c0fde80 2024-04-25T02:01:38.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.275+0000 7fb9727fc700 1 -- 172.21.15.33:0/3979971927 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb97800bd80 con 0x7fb97c0fde80 2024-04-25T02:01:38.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.275+0000 7fb9727fc700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb96806e140 0x7fb9680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:01:38.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.275+0000 7fb9727fc700 1 -- 172.21.15.33:0/3979971927 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb97808eda0 con 0x7fb97c0fde80 2024-04-25T02:01:38.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.275+0000 7fb981544700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb96806e140 0x7fb9680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:01:38.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.276+0000 7fb981544700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb96806e140 0x7fb9680705f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009b80 tx=0x7fb96c009d60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:01:38.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.280+0000 7fb9727fc700 1 -- 172.21.15.33:0/3979971927 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb97805bcd0 con 0x7fb97c0fde80 2024-04-25T02:01:38.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:38 smithi033 ceph-mon[111306]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:38.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:38 smithi033 ceph-mon[111306]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:38.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:38 smithi033 ceph-mon[111306]: pgmap v668: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:38.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:38 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2123064660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:01:38.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:38 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3918770859' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:01:38.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:38 smithi138 ceph-mon[81304]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:38.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:38 smithi138 ceph-mon[81304]: from='client.35252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:38.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:38 smithi138 ceph-mon[81304]: pgmap v668: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:38.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:38 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2123064660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:01:38.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:38 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3918770859' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:01:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.571+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb97c109f70 con 0x7fb97c0fde80 2024-04-25T02:01:38.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.572+0000 7fb9727fc700 1 -- 172.21.15.33:0/3979971927 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fb97805b860 con 0x7fb97c0fde80 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:01:38.577 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:01:38.578 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb96806e140 msgr2=0x7fb9680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb96806e140 0x7fb9680705f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009b80 tx=0x7fb96c009d60 comp rx=0 tx=0).stop 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 msgr2=0x7fb97c1138f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 0x7fb97c1138f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb978000c00 tx=0x7fb978009e30 comp rx=0 tx=0).stop 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 shutdown_connections 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb96806e140 0x7fb9680705f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb97c0fde80 0x7fb97c1138f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 --2- 172.21.15.33:0/3979971927 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb97c0fe820 0x7fb97c113e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 >> 172.21.15.33:0/3979971927 conn(0x7fb97c0f78e0 msgr2=0x7fb97c1036b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:01:38.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 shutdown_connections 2024-04-25T02:01:38.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:01:38.574+0000 7fb983fa9700 1 -- 172.21.15.33:0/3979971927 wait complete. 2024-04-25T02:01:39.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:39 smithi033 ceph-mon[111306]: from='client.35264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:39.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:39 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3979971927' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:01:39.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:39 smithi138 ceph-mon[81304]: from='client.35264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:01:39.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:39 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3979971927' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:01:40.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:40 smithi033 ceph-mon[111306]: pgmap v669: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:40.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:40 smithi138 ceph-mon[81304]: pgmap v669: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:41.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:01:41.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:01:42.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:42 smithi138 ceph-mon[81304]: pgmap v670: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:42.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:42 smithi033 ceph-mon[111306]: pgmap v670: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:44.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:44 smithi033 ceph-mon[111306]: pgmap v671: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:44.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:44 smithi138 ceph-mon[81304]: pgmap v671: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:46.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:46 smithi033 ceph-mon[111306]: pgmap v672: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:46.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:46 smithi138 ceph-mon[81304]: pgmap v672: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:48.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:48 smithi033 ceph-mon[111306]: pgmap v673: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:48.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:48 smithi138 ceph-mon[81304]: pgmap v673: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:50.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:50 smithi033 ceph-mon[111306]: pgmap v674: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:50.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:50 smithi138 ceph-mon[81304]: pgmap v674: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:52.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:52 smithi033 ceph-mon[111306]: pgmap v675: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:52.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:52 smithi138 ceph-mon[81304]: pgmap v675: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:54.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:54 smithi033 ceph-mon[111306]: pgmap v676: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:54.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:54 smithi138 ceph-mon[81304]: pgmap v676: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:56.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:56 smithi033 ceph-mon[111306]: pgmap v677: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:56.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:01:56.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:56 smithi138 ceph-mon[81304]: pgmap v677: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:56.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:01:58.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:01:58 smithi033 ceph-mon[111306]: pgmap v678: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:01:58.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:01:58 smithi138 ceph-mon[81304]: pgmap v678: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:00.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:00 smithi033 ceph-mon[111306]: pgmap v679: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:00.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:00 smithi138 ceph-mon[81304]: pgmap v679: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:02.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:02 smithi033 ceph-mon[111306]: pgmap v680: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:02.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:02 smithi138 ceph-mon[81304]: pgmap v680: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:04.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:04 smithi033 ceph-mon[111306]: pgmap v681: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:04.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:04 smithi138 ceph-mon[81304]: pgmap v681: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:06.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:06 smithi033 ceph-mon[111306]: pgmap v682: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:06.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:06 smithi138 ceph-mon[81304]: pgmap v682: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:08.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:07 smithi033 ceph-mon[111306]: pgmap v683: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:08.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:07 smithi138 ceph-mon[81304]: pgmap v683: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.694+0000 7f87f11d2700 1 -- 172.21.15.33:0/575892751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 msgr2=0x7f87ec0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.694+0000 7f87f11d2700 1 --2- 172.21.15.33:0/575892751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 0x7f87ec0fc1b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f87d4004940 tx=0x7f87d4009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:08.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.694+0000 7f87f11d2700 1 -- 172.21.15.33:0/575892751 shutdown_connections 2024-04-25T02:02:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.694+0000 7f87f11d2700 1 --2- 172.21.15.33:0/575892751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 0x7f87ec0fc1b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.694+0000 7f87f11d2700 1 --2- 172.21.15.33:0/575892751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87ec101d90 0x7f87ec102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.694+0000 7f87f11d2700 1 -- 172.21.15.33:0/575892751 >> 172.21.15.33:0/575892751 conn(0x7f87ec071d00 msgr2=0x7f87ec072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.694+0000 7f87f11d2700 1 -- 172.21.15.33:0/575892751 shutdown_connections 2024-04-25T02:02:08.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.694+0000 7f87f11d2700 1 -- 172.21.15.33:0/575892751 wait complete. 2024-04-25T02:02:08.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.695+0000 7f87f11d2700 1 Processor -- start 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.695+0000 7f87f11d2700 1 -- start start 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.695+0000 7f87f11d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 0x7f87ec115d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.695+0000 7f87f11d2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87ec101d90 0x7f87ec1162a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.695+0000 7f87f11d2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec1168f0 con 0x7f87ec0fbd40 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.695+0000 7f87f11d2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec10fd00 con 0x7f87ec101d90 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.695+0000 7f87ead9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 0x7f87ec115d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.695+0000 7f87ead9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 0x7f87ec115d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43418/0 (socket says 172.21.15.33:43418) 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.695+0000 7f87ead9d700 1 -- 172.21.15.33:0/2342184255 learned_addr learned my addr 172.21.15.33:0/2342184255 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87ea59c700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87ec101d90 0x7f87ec1162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87ead9d700 1 -- 172.21.15.33:0/2342184255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87ec101d90 msgr2=0x7f87ec1162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87ead9d700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87ec101d90 0x7f87ec1162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87ead9d700 1 -- 172.21.15.33:0/2342184255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d4005040 con 0x7f87ec0fbd40 2024-04-25T02:02:08.702 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87ead9d700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 0x7f87ec115d60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f87dc001d80 tx=0x7f87dc009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87e37fe700 1 -- 172.21.15.33:0/2342184255 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87dc007d10 con 0x7f87ec0fbd40 2024-04-25T02:02:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec10ffe0 con 0x7f87ec0fbd40 2024-04-25T02:02:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87e37fe700 1 -- 172.21.15.33:0/2342184255 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87dc001e80 con 0x7f87ec0fbd40 2024-04-25T02:02:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87e37fe700 1 -- 172.21.15.33:0/2342184255 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87dc014640 con 0x7f87ec0fbd40 2024-04-25T02:02:08.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.696+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec1105a0 con 0x7f87ec0fbd40 2024-04-25T02:02:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.697+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec0fd430 con 0x7f87ec0fbd40 2024-04-25T02:02:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.698+0000 7f87e37fe700 1 -- 172.21.15.33:0/2342184255 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f87dc00e960 con 0x7f87ec0fbd40 2024-04-25T02:02:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.698+0000 7f87e37fe700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f87d806e140 0x7f87d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.699+0000 7f87e37fe700 1 -- 172.21.15.33:0/2342184255 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f87dc08d8a0 con 0x7f87ec0fbd40 2024-04-25T02:02:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.699+0000 7f87ea59c700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f87d806e140 0x7f87d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:08.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.699+0000 7f87ea59c700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f87d806e140 0x7f87d80705f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f87ec111640 tx=0x7f87d4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:08.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.703+0000 7f87e37fe700 1 -- 172.21.15.33:0/2342184255 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f87dc056cd0 con 0x7f87ec0fbd40 2024-04-25T02:02:08.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.929+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87ec04bed0 con 0x7f87d806e140 2024-04-25T02:02:08.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.930+0000 7f87e37fe700 1 -- 172.21.15.33:0/2342184255 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f87ec04bed0 con 0x7f87d806e140 2024-04-25T02:02:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f87d806e140 msgr2=0x7f87d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f87d806e140 0x7f87d80705f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f87ec111640 tx=0x7f87d4009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 msgr2=0x7f87ec115d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 0x7f87ec115d60 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f87dc001d80 tx=0x7f87dc009ec0 comp rx=0 tx=0).stop 2024-04-25T02:02:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 shutdown_connections 2024-04-25T02:02:08.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f87d806e140 0x7f87d80705f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87ec0fbd40 0x7f87ec115d60 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 --2- 172.21.15.33:0/2342184255 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87ec101d90 0x7f87ec1162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 >> 172.21.15.33:0/2342184255 conn(0x7f87ec071d00 msgr2=0x7f87ec065600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 shutdown_connections 2024-04-25T02:02:08.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:08.932+0000 7f87f11d2700 1 -- 172.21.15.33:0/2342184255 wait complete. 2024-04-25T02:02:08.947 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:02:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 -- 172.21.15.33:0/2432138962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988102de0 msgr2=0x7ff9881031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 --2- 172.21.15.33:0/2432138962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988102de0 0x7ff9881031b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ff97c004940 tx=0x7ff97c009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 -- 172.21.15.33:0/2432138962 shutdown_connections 2024-04-25T02:02:09.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 --2- 172.21.15.33:0/2432138962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff988103780 0x7ff9880fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 --2- 172.21.15.33:0/2432138962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988102de0 0x7ff9881031b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 -- 172.21.15.33:0/2432138962 >> 172.21.15.33:0/2432138962 conn(0x7ff9880f8990 msgr2=0x7ff9880fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 -- 172.21.15.33:0/2432138962 shutdown_connections 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 -- 172.21.15.33:0/2432138962 wait complete. 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 Processor -- start 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.037+0000 7ff98cc96700 1 -- start start 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff98cc96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff988102de0 0x7ff988113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff98cc96700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988103780 0x7ff988113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff98cc96700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9881145b0 con 0x7ff988103780 2024-04-25T02:02:09.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff98cc96700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98810db20 con 0x7ff988102de0 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff985d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988103780 0x7ff988113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff985d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988103780 0x7ff988113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43444/0 (socket says 172.21.15.33:43444) 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff985d9b700 1 -- 172.21.15.33:0/3636645735 learned_addr learned my addr 172.21.15.33:0/3636645735 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff98659c700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff988102de0 0x7ff988113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff985d9b700 1 -- 172.21.15.33:0/3636645735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff988102de0 msgr2=0x7ff988113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff985d9b700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff988102de0 0x7ff988113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff985d9b700 1 -- 172.21.15.33:0/3636645735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97c005040 con 0x7ff988103780 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff98659c700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff988102de0 0x7ff988113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff985d9b700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988103780 0x7ff988113ed0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b790 tx=0x7ff978009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff976ffd700 1 -- 172.21.15.33:0/3636645735 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff97800f6f0 con 0x7ff988103780 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.038+0000 7ff976ffd700 1 -- 172.21.15.33:0/3636645735 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff97800fd20 con 0x7ff988103780 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.039+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98810de00 con 0x7ff988103780 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.039+0000 7ff976ffd700 1 -- 172.21.15.33:0/3636645735 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9780103a0 con 0x7ff988103780 2024-04-25T02:02:09.045 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.039+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98810e3c0 con 0x7ff988103780 2024-04-25T02:02:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.039+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9881000f0 con 0x7ff988103780 2024-04-25T02:02:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.040+0000 7ff976ffd700 1 -- 172.21.15.33:0/3636645735 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff978011400 con 0x7ff988103780 2024-04-25T02:02:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.040+0000 7ff976ffd700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff97006e140 0x7ff9700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.041+0000 7ff976ffd700 1 -- 172.21.15.33:0/3636645735 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff97808d9d0 con 0x7ff988103780 2024-04-25T02:02:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.041+0000 7ff98659c700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff97006e140 0x7ff9700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:09.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.041+0000 7ff98659c700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff97006e140 0x7ff9700705f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00b3b0 tx=0x7ff97c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:09.050 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.045+0000 7ff976ffd700 1 -- 172.21.15.33:0/3636645735 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff97805a880 con 0x7ff988103780 2024-04-25T02:02:09.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.271+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff988113660 con 0x7ff97006e140 2024-04-25T02:02:09.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.272+0000 7ff976ffd700 1 -- 172.21.15.33:0/3636645735 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff988113660 con 0x7ff97006e140 2024-04-25T02:02:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.274+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff97006e140 msgr2=0x7ff9700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.274+0000 7ff98cc96700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff97006e140 0x7ff9700705f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff97c00b3b0 tx=0x7ff97c009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.274+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988103780 msgr2=0x7ff988113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.274+0000 7ff98cc96700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988103780 0x7ff988113ed0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff97800b790 tx=0x7ff978009220 comp rx=0 tx=0).stop 2024-04-25T02:02:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.275+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 shutdown_connections 2024-04-25T02:02:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.275+0000 7ff98cc96700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff97006e140 0x7ff9700705f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.275+0000 7ff98cc96700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff988102de0 0x7ff988113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.275+0000 7ff98cc96700 1 --2- 172.21.15.33:0/3636645735 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff988103780 0x7ff988113ed0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.275+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 >> 172.21.15.33:0/3636645735 conn(0x7ff9880f8990 msgr2=0x7ff988107330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.275+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 shutdown_connections 2024-04-25T02:02:09.280 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.275+0000 7ff98cc96700 1 -- 172.21.15.33:0/3636645735 wait complete. 2024-04-25T02:02:09.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.379+0000 7eff69da7700 1 -- 172.21.15.33:0/3082726887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 msgr2=0x7eff641020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.379+0000 7eff69da7700 1 --2- 172.21.15.33:0/3082726887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 0x7eff641020e0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7eff54004940 tx=0x7eff54009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:09.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.379+0000 7eff69da7700 1 -- 172.21.15.33:0/3082726887 shutdown_connections 2024-04-25T02:02:09.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.379+0000 7eff69da7700 1 --2- 172.21.15.33:0/3082726887 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff641026b0 0x7eff640fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.379+0000 7eff69da7700 1 --2- 172.21.15.33:0/3082726887 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 0x7eff641020e0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.379+0000 7eff69da7700 1 -- 172.21.15.33:0/3082726887 >> 172.21.15.33:0/3082726887 conn(0x7eff640f7860 msgr2=0x7eff640f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:09.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.379+0000 7eff69da7700 1 -- 172.21.15.33:0/3082726887 shutdown_connections 2024-04-25T02:02:09.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.379+0000 7eff69da7700 1 -- 172.21.15.33:0/3082726887 wait complete. 2024-04-25T02:02:09.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff69da7700 1 Processor -- start 2024-04-25T02:02:09.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff69da7700 1 -- start start 2024-04-25T02:02:09.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff69da7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 0x7eff64106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:09.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff69da7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff641026b0 0x7eff641070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:09.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff69da7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64108ab0 con 0x7eff64101d10 2024-04-25T02:02:09.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff69da7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64107630 con 0x7eff641026b0 2024-04-25T02:02:09.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff637fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 0x7eff64106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff637fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 0x7eff64106bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43460/0 (socket says 172.21.15.33:43460) 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff637fe700 1 -- 172.21.15.33:0/289638779 learned_addr learned my addr 172.21.15.33:0/289638779 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.380+0000 7eff62ffd700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff641026b0 0x7eff641070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff637fe700 1 -- 172.21.15.33:0/289638779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff641026b0 msgr2=0x7eff641070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff637fe700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff641026b0 0x7eff641070f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff637fe700 1 -- 172.21.15.33:0/289638779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff54005040 con 0x7eff64101d10 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff62ffd700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff641026b0 0x7eff641070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff637fe700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 0x7eff64106bb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7eff5400b3b0 tx=0x7eff54009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff4bfff700 1 -- 172.21.15.33:0/289638779 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff5400e040 con 0x7eff64101d10 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff641078b0 con 0x7eff64101d10 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff4bfff700 1 -- 172.21.15.33:0/289638779 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff5400bbc0 con 0x7eff64101d10 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff4bfff700 1 -- 172.21.15.33:0/289638779 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff5401c3f0 con 0x7eff64101d10 2024-04-25T02:02:09.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.381+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff6411a0f0 con 0x7eff64101d10 2024-04-25T02:02:09.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.383+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff640ff020 con 0x7eff64101d10 2024-04-25T02:02:09.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.383+0000 7eff4bfff700 1 -- 172.21.15.33:0/289638779 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7eff5400bd30 con 0x7eff64101d10 2024-04-25T02:02:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.384+0000 7eff4bfff700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff4c06e140 0x7eff4c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.384+0000 7eff4bfff700 1 -- 172.21.15.33:0/289638779 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7eff5408ed70 con 0x7eff64101d10 2024-04-25T02:02:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.384+0000 7eff62ffd700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff4c06e140 0x7eff4c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:09.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.384+0000 7eff62ffd700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff4c06e140 0x7eff4c0705f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7eff58009fd0 tx=0x7eff58009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:09.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.388+0000 7eff4bfff700 1 -- 172.21.15.33:0/289638779 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7eff540580f0 con 0x7eff64101d10 2024-04-25T02:02:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.603+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7eff64108040 con 0x7eff4c06e140 2024-04-25T02:02:09.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.613+0000 7eff4bfff700 1 -- 172.21.15.33:0/289638779 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7eff64108040 con 0x7eff4c06e140 2024-04-25T02:02:09.618 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:02:09.618 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (23m) 60s ago 34m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:02:09.618 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (20m) 60s ago 34m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:02:09.618 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (20m) 62s ago 33m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:02:09.618 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (23m) 60s ago 34m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:02:09.618 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (16m) 60s ago 29m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:02:09.618 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (16m) 60s ago 29m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (16m) 62s ago 29m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (66s) 62s ago 29m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (24m) 60s ago 35m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (24m) 62s ago 33m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (21m) 60s ago 36m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (21m) 62s ago 33m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (23m) 60s ago 34m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (23m) 62s ago 33m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (20m) 60s ago 32m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:02:09.619 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (19m) 60s ago 32m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:02:09.620 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (18m) 60s ago 31m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:02:09.620 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (18m) 62s ago 31m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:02:09.620 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (17m) 62s ago 31m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:02:09.620 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (17m) 62s ago 30m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:02:09.620 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (23m) 60s ago 34m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:02:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff4c06e140 msgr2=0x7eff4c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff4c06e140 0x7eff4c0705f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7eff58009fd0 tx=0x7eff58009ee0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 msgr2=0x7eff64106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 0x7eff64106bb0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7eff5400b3b0 tx=0x7eff54009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 shutdown_connections 2024-04-25T02:02:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7eff4c06e140 0x7eff4c0705f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff64101d10 0x7eff64106bb0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 --2- 172.21.15.33:0/289638779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff641026b0 0x7eff641070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 >> 172.21.15.33:0/289638779 conn(0x7eff640f7860 msgr2=0x7eff64106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 shutdown_connections 2024-04-25T02:02:09.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.615+0000 7eff69da7700 1 -- 172.21.15.33:0/289638779 wait complete. 2024-04-25T02:02:09.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.718+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3204076709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 msgr2=0x7fadc8102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.718+0000 7fadcd3a2700 1 --2- 172.21.15.33:0/3204076709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 0x7fadc8102160 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fadc0009ce0 tx=0x7fadc0004d10 comp rx=0 tx=0).stop 2024-04-25T02:02:09.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.718+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3204076709 shutdown_connections 2024-04-25T02:02:09.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.718+0000 7fadcd3a2700 1 --2- 172.21.15.33:0/3204076709 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc80fbd40 0x7fadc80fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.718+0000 7fadcd3a2700 1 --2- 172.21.15.33:0/3204076709 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 0x7fadc8102160 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.718+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3204076709 >> 172.21.15.33:0/3204076709 conn(0x7fadc8071d00 msgr2=0x7fadc8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:09.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.718+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3204076709 shutdown_connections 2024-04-25T02:02:09.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.718+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3204076709 wait complete. 2024-04-25T02:02:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadcd3a2700 1 Processor -- start 2024-04-25T02:02:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadcd3a2700 1 -- start start 2024-04-25T02:02:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadcd3a2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc80fbd40 0x7fadc8115f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadcd3a2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 0x7fadc81164a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadcd3a2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc8116af0 con 0x7fadc8101d90 2024-04-25T02:02:09.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadcd3a2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc810fec0 con 0x7fadc80fbd40 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadc67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 0x7fadc81164a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadc67fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 0x7fadc81164a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43474/0 (socket says 172.21.15.33:43474) 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadc67fc700 1 -- 172.21.15.33:0/3811205275 learned_addr learned my addr 172.21.15.33:0/3811205275 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadc6ffd700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc80fbd40 0x7fadc8115f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadc67fc700 1 -- 172.21.15.33:0/3811205275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc80fbd40 msgr2=0x7fadc8115f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadc67fc700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc80fbd40 0x7fadc8115f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.719+0000 7fadc67fc700 1 -- 172.21.15.33:0/3811205275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc0009940 con 0x7fadc8101d90 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.720+0000 7fadc67fc700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 0x7fadc81164a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fadb0002a50 tx=0x7fadb00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.720+0000 7fadbf7fe700 1 -- 172.21.15.33:0/3811205275 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadb0007cc0 con 0x7fadc8101d90 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.720+0000 7fadbf7fe700 1 -- 172.21.15.33:0/3811205275 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadb000f040 con 0x7fadc8101d90 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.720+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc81101a0 con 0x7fadc8101d90 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.720+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc81106e0 con 0x7fadc8101d90 2024-04-25T02:02:09.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.720+0000 7fadbf7fe700 1 -- 172.21.15.33:0/3811205275 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadb000da70 con 0x7fadc8101d90 2024-04-25T02:02:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.721+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc80fd430 con 0x7fadc8101d90 2024-04-25T02:02:09.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.722+0000 7fadbf7fe700 1 -- 172.21.15.33:0/3811205275 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fadb000dcb0 con 0x7fadc8101d90 2024-04-25T02:02:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.722+0000 7fadbf7fe700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fadb406e140 0x7fadb40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.722+0000 7fadbf7fe700 1 -- 172.21.15.33:0/3811205275 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fadb008d0c0 con 0x7fadc8101d90 2024-04-25T02:02:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.722+0000 7fadc6ffd700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fadb406e140 0x7fadb40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:09.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.723+0000 7fadc6ffd700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fadb406e140 0x7fadb40705f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fadc000b740 tx=0x7fadc0004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:09.732 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:09.727+0000 7fadbf7fe700 1 -- 172.21.15.33:0/3811205275 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fadb0059ff0 con 0x7fadc8101d90 2024-04-25T02:02:10.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.026+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fadc8108e10 con 0x7fadc8101d90 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.027+0000 7fadbf7fe700 1 -- 172.21.15.33:0/3811205275 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fadb0059b80 con 0x7fadc8101d90 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:02:10.032 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:02:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:02:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:02:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:02:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:02:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:02:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:02:10.033 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:02:10.033 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:02:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fadb406e140 msgr2=0x7fadb40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fadb406e140 0x7fadb40705f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fadc000b740 tx=0x7fadc0004d10 comp rx=0 tx=0).stop 2024-04-25T02:02:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 msgr2=0x7fadc81164a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 0x7fadc81164a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fadb0002a50 tx=0x7fadb00046b0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 shutdown_connections 2024-04-25T02:02:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fadb406e140 0x7fadb40705f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadc80fbd40 0x7fadc8115f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 --2- 172.21.15.33:0/3811205275 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadc8101d90 0x7fadc81164a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 >> 172.21.15.33:0/3811205275 conn(0x7fadc8071d00 msgr2=0x7fadc80655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 shutdown_connections 2024-04-25T02:02:10.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.029+0000 7fadcd3a2700 1 -- 172.21.15.33:0/3811205275 wait complete. 2024-04-25T02:02:10.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.133+0000 7f6545109700 1 -- 172.21.15.33:0/584591850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65401026b0 msgr2=0x7f65400fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.133+0000 7f6545109700 1 --2- 172.21.15.33:0/584591850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65401026b0 0x7f65400fbd20 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6530004940 tx=0x7f6530009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:10.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.133+0000 7f6545109700 1 -- 172.21.15.33:0/584591850 shutdown_connections 2024-04-25T02:02:10.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.133+0000 7f6545109700 1 --2- 172.21.15.33:0/584591850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f65401026b0 0x7f65400fbd20 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.133+0000 7f6545109700 1 --2- 172.21.15.33:0/584591850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6540101d10 0x7f65401020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.133+0000 7f6545109700 1 -- 172.21.15.33:0/584591850 >> 172.21.15.33:0/584591850 conn(0x7f65400f7860 msgr2=0x7f65400f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.133+0000 7f6545109700 1 -- 172.21.15.33:0/584591850 shutdown_connections 2024-04-25T02:02:10.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.133+0000 7f6545109700 1 -- 172.21.15.33:0/584591850 wait complete. 2024-04-25T02:02:10.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f6545109700 1 Processor -- start 2024-04-25T02:02:10.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f6545109700 1 -- start start 2024-04-25T02:02:10.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f6545109700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6540101d10 0x7f65400fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f6545109700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65401026b0 0x7f65400fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f6545109700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65400ff930 con 0x7f6540101d10 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f6545109700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65400fe410 con 0x7f65401026b0 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f653ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6540101d10 0x7f65400fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f653ed9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6540101d10 0x7f65400fd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43488/0 (socket says 172.21.15.33:43488) 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f653ed9d700 1 -- 172.21.15.33:0/3518026198 learned_addr learned my addr 172.21.15.33:0/3518026198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f653e59c700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65401026b0 0x7f65400fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f653ed9d700 1 -- 172.21.15.33:0/3518026198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65401026b0 msgr2=0x7f65400fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.134+0000 7f653ed9d700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65401026b0 0x7f65400fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.135+0000 7f653ed9d700 1 -- 172.21.15.33:0/3518026198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6530005040 con 0x7f6540101d10 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.135+0000 7f653ed9d700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6540101d10 0x7f65400fd990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f653400c8c0 tx=0x7f65340044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.135+0000 7f65277fe700 1 -- 172.21.15.33:0/3518026198 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f653400a760 con 0x7f6540101d10 2024-04-25T02:02:10.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.135+0000 7f65277fe700 1 -- 172.21.15.33:0/3518026198 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f653400ad90 con 0x7f6540101d10 2024-04-25T02:02:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.135+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65400fe6c0 con 0x7f6540101d10 2024-04-25T02:02:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.135+0000 7f65277fe700 1 -- 172.21.15.33:0/3518026198 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65340113a0 con 0x7f6540101d10 2024-04-25T02:02:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.135+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f654011a120 con 0x7f6540101d10 2024-04-25T02:02:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.136+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65400fd460 con 0x7f6540101d10 2024-04-25T02:02:10.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.136+0000 7f65277fe700 1 -- 172.21.15.33:0/3518026198 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6534012400 con 0x7f6540101d10 2024-04-25T02:02:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.137+0000 7f65277fe700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f652806e140 0x7f65280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.137+0000 7f65277fe700 1 -- 172.21.15.33:0/3518026198 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f653408db50 con 0x7f6540101d10 2024-04-25T02:02:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.137+0000 7f653e59c700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f652806e140 0x7f65280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:10.143 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.137+0000 7f653e59c700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f652806e140 0x7f65280705f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6530004890 tx=0x7f6530009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:10.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.142+0000 7f65277fe700 1 -- 172.21.15.33:0/3518026198 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6534056f50 con 0x7f6540101d10 2024-04-25T02:02:10.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.397+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f65400fef40 con 0x7f6540101d10 2024-04-25T02:02:10.403 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:10 smithi033 ceph-mon[111306]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:10.403 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:10 smithi033 ceph-mon[111306]: pgmap v684: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:10.403 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:10 smithi033 ceph-mon[111306]: from='client.35276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:10.403 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:10 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3811205275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:02:10.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.397+0000 7f65277fe700 1 -- 172.21.15.33:0/3518026198 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f653405a610 con 0x7f6540101d10 2024-04-25T02:02:10.404 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:02:10.404 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:02:10.404 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:02:10.404 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:02:10.404 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:02:10.404 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:02:10.404 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:02:10.404 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:02:10.405 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:02:10.405 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:02:10.405 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:02:10.405 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:02:10.405 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:02:10.405 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:02:10.405 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:02:10.405 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:02:10.406 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:02:10.406 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:02:10.406 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:02:10.406 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:02:10.406 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:02:10.406 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:02:10.406 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:02:10.407 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:02:10.407 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:02:10.407 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:02:10.407 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:02:10.407 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:02:10.407 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:02:10.407 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:02:10.408 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:02:10.408 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:02:10.408 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:02:10.408 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:02:10.408 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:02:10.408 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:02:10.408 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:02:10.409 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:02:10.409 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:02:10.409 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:02:10.409 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:02:10.409 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:02:10.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.399+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f652806e140 msgr2=0x7f65280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.399+0000 7f6545109700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f652806e140 0x7f65280705f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6530004890 tx=0x7f6530009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:10.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.399+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6540101d10 msgr2=0x7f65400fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.399+0000 7f6545109700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6540101d10 0x7f65400fd990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f653400c8c0 tx=0x7f65340044c0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.399+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 shutdown_connections 2024-04-25T02:02:10.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.399+0000 7f6545109700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f652806e140 0x7f65280705f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.399+0000 7f6545109700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6540101d10 0x7f65400fd990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.399+0000 7f6545109700 1 --2- 172.21.15.33:0/3518026198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f65401026b0 0x7f65400fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.399+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 >> 172.21.15.33:0/3518026198 conn(0x7f65400f7860 msgr2=0x7f65400662b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:10.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.400+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 shutdown_connections 2024-04-25T02:02:10.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.400+0000 7f6545109700 1 -- 172.21.15.33:0/3518026198 wait complete. 2024-04-25T02:02:10.411 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:02:10.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.504+0000 7fdfec500700 1 -- 172.21.15.33:0/4120145682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 msgr2=0x7fdfe4101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.504+0000 7fdfec500700 1 --2- 172.21.15.33:0/4120145682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 0x7fdfe4101a30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4004940 tx=0x7fdfd4009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 -- 172.21.15.33:0/4120145682 shutdown_connections 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 --2- 172.21.15.33:0/4120145682 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 0x7fdfe4101a30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 --2- 172.21.15.33:0/4120145682 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40fb5a0 0x7fdfe40fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 -- 172.21.15.33:0/4120145682 >> 172.21.15.33:0/4120145682 conn(0x7fdfe4073350 msgr2=0x7fdfe4073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 -- 172.21.15.33:0/4120145682 shutdown_connections 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 -- 172.21.15.33:0/4120145682 wait complete. 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 Processor -- start 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 -- start start 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40fb5a0 0x7fdfe40fc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 0x7fdfe40fca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:10.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40fe480 con 0x7fdfe40fbf40 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.505+0000 7fdfec500700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40fcfb0 con 0x7fdfe40fb5a0 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfe9a9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 0x7fdfe40fca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfe9a9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 0x7fdfe40fca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43498/0 (socket says 172.21.15.33:43498) 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfe9a9b700 1 -- 172.21.15.33:0/1059229860 learned_addr learned my addr 172.21.15.33:0/1059229860 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfea29c700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40fb5a0 0x7fdfe40fc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfe9a9b700 1 -- 172.21.15.33:0/1059229860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40fb5a0 msgr2=0x7fdfe40fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfe9a9b700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40fb5a0 0x7fdfe40fc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfe9a9b700 1 -- 172.21.15.33:0/1059229860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd4005040 con 0x7fdfe40fbf40 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfea29c700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40fb5a0 0x7fdfe40fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfe9a9b700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 0x7fdfe40fca70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009030 tx=0x7fdfd4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfdaffd700 1 -- 172.21.15.33:0/1059229860 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfd400e040 con 0x7fdfe40fbf40 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe40fd230 con 0x7fdfe40fbf40 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfdaffd700 1 -- 172.21.15.33:0/1059229860 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdfd4004290 con 0x7fdfe40fbf40 2024-04-25T02:02:10.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe411a110 con 0x7fdfe40fbf40 2024-04-25T02:02:10.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.506+0000 7fdfdaffd700 1 -- 172.21.15.33:0/1059229860 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdfd401c3f0 con 0x7fdfe40fbf40 2024-04-25T02:02:10.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.507+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe40fe670 con 0x7fdfe40fbf40 2024-04-25T02:02:10.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.508+0000 7fdfdaffd700 1 -- 172.21.15.33:0/1059229860 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdfd4013420 con 0x7fdfe40fbf40 2024-04-25T02:02:10.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.508+0000 7fdfdaffd700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdfd006e140 0x7fdfd00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:10.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.508+0000 7fdfdaffd700 1 -- 172.21.15.33:0/1059229860 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdfd408eca0 con 0x7fdfe40fbf40 2024-04-25T02:02:10.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.508+0000 7fdfea29c700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdfd006e140 0x7fdfd00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:10.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.509+0000 7fdfea29c700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdfd006e140 0x7fdfd00705f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0007fd0 tx=0x7fdfe0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:10.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.513+0000 7fdfdaffd700 1 -- 172.21.15.33:0/1059229860 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdfd4093070 con 0x7fdfe40fbf40 2024-04-25T02:02:10.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:10 smithi138 ceph-mon[81304]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:10.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:10 smithi138 ceph-mon[81304]: pgmap v684: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:10.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:10 smithi138 ceph-mon[81304]: from='client.35276 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:10.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:10 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3811205275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:02:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.741+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdfe4101120 con 0x7fdfd006e140 2024-04-25T02:02:10.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.742+0000 7fdfdaffd700 1 -- 172.21.15.33:0/1059229860 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdfe4101120 con 0x7fdfd006e140 2024-04-25T02:02:10.747 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:02:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:02:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:02:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:02:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:02:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:02:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:02:10.747 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:02:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:02:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:02:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:02:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:02:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:02:10.748 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:02:10.748 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:02:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.744+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdfd006e140 msgr2=0x7fdfd00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.744+0000 7fdfec500700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdfd006e140 0x7fdfd00705f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0007fd0 tx=0x7fdfe0009040 comp rx=0 tx=0).stop 2024-04-25T02:02:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.744+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 msgr2=0x7fdfe40fca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.744+0000 7fdfec500700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 0x7fdfe40fca70 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009030 tx=0x7fdfd4009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.744+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 shutdown_connections 2024-04-25T02:02:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.744+0000 7fdfec500700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdfd006e140 0x7fdfd00705f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.744+0000 7fdfec500700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdfe40fb5a0 0x7fdfe40fc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.744+0000 7fdfec500700 1 --2- 172.21.15.33:0/1059229860 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdfe40fbf40 0x7fdfe40fca70 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.744+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 >> 172.21.15.33:0/1059229860 conn(0x7fdfe4073350 msgr2=0x7fdfe4100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.745+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 shutdown_connections 2024-04-25T02:02:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.745+0000 7fdfec500700 1 -- 172.21.15.33:0/1059229860 wait complete. 2024-04-25T02:02:10.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.848+0000 7f234b762700 1 -- 172.21.15.33:0/2480718811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 msgr2=0x7f23441031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.848+0000 7f234b762700 1 --2- 172.21.15.33:0/2480718811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 0x7f23441031b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f2340004940 tx=0x7f2340009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:10.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.848+0000 7f234b762700 1 -- 172.21.15.33:0/2480718811 shutdown_connections 2024-04-25T02:02:10.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 --2- 172.21.15.33:0/2480718811 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2344103780 0x7f23440fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 --2- 172.21.15.33:0/2480718811 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 0x7f23441031b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 -- 172.21.15.33:0/2480718811 >> 172.21.15.33:0/2480718811 conn(0x7f23440f8990 msgr2=0x7f23440fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:10.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 -- 172.21.15.33:0/2480718811 shutdown_connections 2024-04-25T02:02:10.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 -- 172.21.15.33:0/2480718811 wait complete. 2024-04-25T02:02:10.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 Processor -- start 2024-04-25T02:02:10.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 -- start start 2024-04-25T02:02:10.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 0x7f2344113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:10.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2344103780 0x7f2344113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:10.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.849+0000 7f234b762700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2344114500 con 0x7f2344102de0 2024-04-25T02:02:10.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f234b762700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234410db50 con 0x7f2344103780 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f23494fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 0x7f2344113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f23494fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 0x7f2344113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43520/0 (socket says 172.21.15.33:43520) 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f23494fe700 1 -- 172.21.15.33:0/4042552564 learned_addr learned my addr 172.21.15.33:0/4042552564 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f23494fe700 1 -- 172.21.15.33:0/4042552564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2344103780 msgr2=0x7f2344113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f23494fe700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2344103780 0x7f2344113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f23494fe700 1 -- 172.21.15.33:0/4042552564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340005040 con 0x7f2344102de0 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f23494fe700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 0x7f2344113970 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f234000b3b0 tx=0x7f2340009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f2339ffb700 1 -- 172.21.15.33:0/4042552564 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f234000e040 con 0x7f2344102de0 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234410ddd0 con 0x7f2344102de0 2024-04-25T02:02:10.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f2339ffb700 1 -- 172.21.15.33:0/4042552564 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f234000bbc0 con 0x7f2344102de0 2024-04-25T02:02:10.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f2339ffb700 1 -- 172.21.15.33:0/4042552564 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f234001c3f0 con 0x7f2344102de0 2024-04-25T02:02:10.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.850+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234410e390 con 0x7f2344102de0 2024-04-25T02:02:10.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.851+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23441000f0 con 0x7f2344102de0 2024-04-25T02:02:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.852+0000 7f2339ffb700 1 -- 172.21.15.33:0/4042552564 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f234000bd30 con 0x7f2344102de0 2024-04-25T02:02:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.852+0000 7f2339ffb700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f233006e140 0x7f23300705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.853+0000 7f2339ffb700 1 -- 172.21.15.33:0/4042552564 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f234008edc0 con 0x7f2344102de0 2024-04-25T02:02:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.853+0000 7f2348cfd700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f233006e140 0x7f23300705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:10.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.853+0000 7f2348cfd700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f233006e140 0x7f23300705f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f234410ecc0 tx=0x7f2334009ce0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:10.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:10.858+0000 7f2339ffb700 1 -- 172.21.15.33:0/4042552564 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f234005bc70 con 0x7f2344102de0 2024-04-25T02:02:11.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.150+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f234410eff0 con 0x7f2344102de0 2024-04-25T02:02:11.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.150+0000 7f2339ffb700 1 -- 172.21.15.33:0/4042552564 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f234005b800 con 0x7f2344102de0 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:02:11.156 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:02:11.157 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:02:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.152+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f233006e140 msgr2=0x7f23300705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.152+0000 7f234b762700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f233006e140 0x7f23300705f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f234410ecc0 tx=0x7f2334009ce0 comp rx=0 tx=0).stop 2024-04-25T02:02:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.152+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 msgr2=0x7f2344113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.152+0000 7f234b762700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 0x7f2344113970 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f234000b3b0 tx=0x7f2340009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:11.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.153+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 shutdown_connections 2024-04-25T02:02:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.153+0000 7f234b762700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f233006e140 0x7f23300705f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.153+0000 7f234b762700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2344102de0 0x7f2344113970 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.153+0000 7f234b762700 1 --2- 172.21.15.33:0/4042552564 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2344103780 0x7f2344113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.153+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 >> 172.21.15.33:0/4042552564 conn(0x7f23440f8990 msgr2=0x7f2344107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.153+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 shutdown_connections 2024-04-25T02:02:11.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:11.153+0000 7f234b762700 1 -- 172.21.15.33:0/4042552564 wait complete. 2024-04-25T02:02:11.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:11 smithi033 ceph-mon[111306]: from='client.35280 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:11.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:02:11.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:11 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3518026198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:02:11.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:11 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4042552564' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:02:11.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:11 smithi138 ceph-mon[81304]: from='client.35280 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:11.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:02:11.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:11 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3518026198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:02:11.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:11 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4042552564' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:02:12.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:12 smithi033 ceph-mon[111306]: from='client.35292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:12.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:12 smithi033 ceph-mon[111306]: pgmap v685: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:12.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:12 smithi138 ceph-mon[81304]: from='client.35292 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:12.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:12 smithi138 ceph-mon[81304]: pgmap v685: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:14.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:14 smithi033 ceph-mon[111306]: pgmap v686: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:14.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:14 smithi138 ceph-mon[81304]: pgmap v686: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:16.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:16 smithi033 ceph-mon[111306]: pgmap v687: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:16.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:16 smithi138 ceph-mon[81304]: pgmap v687: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:18.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:18 smithi033 ceph-mon[111306]: pgmap v688: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:18.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:18 smithi138 ceph-mon[81304]: pgmap v688: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:20.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:20 smithi033 ceph-mon[111306]: pgmap v689: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:20.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:20 smithi138 ceph-mon[81304]: pgmap v689: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:22.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:22 smithi033 ceph-mon[111306]: pgmap v690: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:22.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:22 smithi138 ceph-mon[81304]: pgmap v690: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:24.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:24 smithi033 ceph-mon[111306]: pgmap v691: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:24.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:24 smithi138 ceph-mon[81304]: pgmap v691: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:26.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:26 smithi033 ceph-mon[111306]: pgmap v692: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:26.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:02:26.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:26 smithi138 ceph-mon[81304]: pgmap v692: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:26.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:02:28.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:28 smithi033 ceph-mon[111306]: pgmap v693: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:28.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:28 smithi138 ceph-mon[81304]: pgmap v693: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:30.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:30 smithi033 ceph-mon[111306]: pgmap v694: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:30.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:30 smithi138 ceph-mon[81304]: pgmap v694: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:32.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:32 smithi033 ceph-mon[111306]: pgmap v695: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:32.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:32 smithi138 ceph-mon[81304]: pgmap v695: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:34.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:34 smithi033 ceph-mon[111306]: pgmap v696: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:34.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:34 smithi138 ceph-mon[81304]: pgmap v696: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:36.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:36 smithi033 ceph-mon[111306]: pgmap v697: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:36.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:36 smithi138 ceph-mon[81304]: pgmap v697: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:38.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:38 smithi033 ceph-mon[111306]: pgmap v698: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:38.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:38 smithi138 ceph-mon[81304]: pgmap v698: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:40.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:40 smithi033 ceph-mon[111306]: pgmap v699: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:40.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:40 smithi138 ceph-mon[81304]: pgmap v699: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.275+0000 7fe4b47ee700 1 -- 172.21.15.33:0/4282459855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 msgr2=0x7fe4ac0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.275+0000 7fe4b47ee700 1 --2- 172.21.15.33:0/4282459855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 0x7fe4ac0fc1b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fe49c004940 tx=0x7fe49c009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.275+0000 7fe4b47ee700 1 -- 172.21.15.33:0/4282459855 shutdown_connections 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.275+0000 7fe4b47ee700 1 --2- 172.21.15.33:0/4282459855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 0x7fe4ac0fc1b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.275+0000 7fe4b47ee700 1 --2- 172.21.15.33:0/4282459855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4ac101d90 0x7fe4ac102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.275+0000 7fe4b47ee700 1 -- 172.21.15.33:0/4282459855 >> 172.21.15.33:0/4282459855 conn(0x7fe4ac071d00 msgr2=0x7fe4ac072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.275+0000 7fe4b47ee700 1 -- 172.21.15.33:0/4282459855 shutdown_connections 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.275+0000 7fe4b47ee700 1 -- 172.21.15.33:0/4282459855 wait complete. 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.276+0000 7fe4b47ee700 1 Processor -- start 2024-04-25T02:02:41.281 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.276+0000 7fe4b47ee700 1 -- start start 2024-04-25T02:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.276+0000 7fe4b47ee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 0x7fe4ac102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.276+0000 7fe4b47ee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4ac101d90 0x7fe4ac103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.276+0000 7fe4b47ee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac103610 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.276+0000 7fe4b47ee700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac103780 con 0x7fe4ac101d90 2024-04-25T02:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.276+0000 7fe4b258a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 0x7fe4ac102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.276+0000 7fe4b258a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 0x7fe4ac102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56118/0 (socket says 172.21.15.33:56118) 2024-04-25T02:02:41.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.276+0000 7fe4b258a700 1 -- 172.21.15.33:0/2701006818 learned_addr learned my addr 172.21.15.33:0/2701006818 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4b1d89700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4ac101d90 0x7fe4ac103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4b258a700 1 -- 172.21.15.33:0/2701006818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4ac101d90 msgr2=0x7fe4ac103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4b258a700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4ac101d90 0x7fe4ac103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4b258a700 1 -- 172.21.15.33:0/2701006818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c005040 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4b1d89700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4ac101d90 0x7fe4ac103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4b258a700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 0x7fe4ac102b20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800eb40 tx=0x7fe4a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4a2ffd700 1 -- 172.21.15.33:0/2701006818 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a8007cc0 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4a2ffd700 1 -- 172.21.15.33:0/2701006818 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4a800f040 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac119e00 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4a2ffd700 1 -- 172.21.15.33:0/2701006818 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a8009620 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.277+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac11a210 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.278+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac0fd430 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.279+0000 7fe4a2ffd700 1 -- 172.21.15.33:0/2701006818 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe4a8009780 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.279+0000 7fe4a2ffd700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe49806e140 0x7fe4980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.279+0000 7fe4a2ffd700 1 -- 172.21.15.33:0/2701006818 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe4a808d700 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.279+0000 7fe4b1d89700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe49806e140 0x7fe4980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:41.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.280+0000 7fe4b1d89700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe49806e140 0x7fe4980705f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac104460 tx=0x7fe49c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:41.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.284+0000 7fe4a2ffd700 1 -- 172.21.15.33:0/2701006818 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe4a8056bb0 con 0x7fe4ac0fbd40 2024-04-25T02:02:41.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.512+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4ac104210 con 0x7fe49806e140 2024-04-25T02:02:41.517 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:02:41.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.513+0000 7fe4a2ffd700 1 -- 172.21.15.33:0/2701006818 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe4ac104210 con 0x7fe49806e140 2024-04-25T02:02:41.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe49806e140 msgr2=0x7fe4980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe49806e140 0x7fe4980705f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac104460 tx=0x7fe49c009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:41.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 msgr2=0x7fe4ac102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 0x7fe4ac102b20 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe4a800eb40 tx=0x7fe4a800b040 comp rx=0 tx=0).stop 2024-04-25T02:02:41.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 shutdown_connections 2024-04-25T02:02:41.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe49806e140 0x7fe4980705f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe4ac0fbd40 0x7fe4ac102b20 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 --2- 172.21.15.33:0/2701006818 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe4ac101d90 0x7fe4ac103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 >> 172.21.15.33:0/2701006818 conn(0x7fe4ac071d00 msgr2=0x7fe4ac065620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:41.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 shutdown_connections 2024-04-25T02:02:41.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.515+0000 7fe4b47ee700 1 -- 172.21.15.33:0/2701006818 wait complete. 2024-04-25T02:02:41.531 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:02:41.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:02:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.620+0000 7f350f8d2700 1 -- 172.21.15.33:0/226367340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3508101d90 msgr2=0x7f3508102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.620+0000 7f350f8d2700 1 --2- 172.21.15.33:0/226367340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3508101d90 0x7f3508102160 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f3504004940 tx=0x7f3504009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.620+0000 7f350f8d2700 1 -- 172.21.15.33:0/226367340 shutdown_connections 2024-04-25T02:02:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.620+0000 7f350f8d2700 1 --2- 172.21.15.33:0/226367340 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f35080fbd40 0x7f35080fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.620+0000 7f350f8d2700 1 --2- 172.21.15.33:0/226367340 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3508101d90 0x7f3508102160 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.620+0000 7f350f8d2700 1 -- 172.21.15.33:0/226367340 >> 172.21.15.33:0/226367340 conn(0x7f3508071d00 msgr2=0x7f3508072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.620+0000 7f350f8d2700 1 -- 172.21.15.33:0/226367340 shutdown_connections 2024-04-25T02:02:41.626 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350f8d2700 1 -- 172.21.15.33:0/226367340 wait complete. 2024-04-25T02:02:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350f8d2700 1 Processor -- start 2024-04-25T02:02:41.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350f8d2700 1 -- start start 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350f8d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35080fbd40 0x7f3508102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350f8d2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3508101d90 0x7f3508103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350f8d2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35081035f0 con 0x7f35080fbd40 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350f8d2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3508103730 con 0x7f3508101d90 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350d66e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35080fbd40 0x7f3508102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350d66e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35080fbd40 0x7f3508102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56136/0 (socket says 172.21.15.33:56136) 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.621+0000 7f350d66e700 1 -- 172.21.15.33:0/733367173 learned_addr learned my addr 172.21.15.33:0/733367173 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f350ce6d700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3508101d90 0x7f3508103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f350d66e700 1 -- 172.21.15.33:0/733367173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3508101d90 msgr2=0x7f3508103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f350d66e700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3508101d90 0x7f3508103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f350d66e700 1 -- 172.21.15.33:0/733367173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3504005040 con 0x7f35080fbd40 2024-04-25T02:02:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f350d66e700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35080fbd40 0x7f3508102b20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f350400b3b0 tx=0x7f3504009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f34fdffb700 1 -- 172.21.15.33:0/733367173 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f350400e040 con 0x7f35080fbd40 2024-04-25T02:02:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f34fdffb700 1 -- 172.21.15.33:0/733367173 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3504013420 con 0x7f35080fbd40 2024-04-25T02:02:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f34fdffb700 1 -- 172.21.15.33:0/733367173 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f350401c630 con 0x7f35080fbd40 2024-04-25T02:02:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3508103980 con 0x7f35080fbd40 2024-04-25T02:02:41.629 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.622+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350811a2b0 con 0x7f35080fbd40 2024-04-25T02:02:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.623+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35080fd430 con 0x7f35080fbd40 2024-04-25T02:02:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.624+0000 7f34fdffb700 1 -- 172.21.15.33:0/733367173 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3504013590 con 0x7f35080fbd40 2024-04-25T02:02:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.624+0000 7f34fdffb700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f34f406e140 0x7f34f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.624+0000 7f34fdffb700 1 -- 172.21.15.33:0/733367173 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f350408ed80 con 0x7f35080fbd40 2024-04-25T02:02:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.624+0000 7f350ce6d700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f34f406e140 0x7f34f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:41.630 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.625+0000 7f350ce6d700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f34f406e140 0x7f34f40705f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3508104460 tx=0x7f34f800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:41.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.629+0000 7f34fdffb700 1 -- 172.21.15.33:0/733367173 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3504058180 con 0x7f35080fbd40 2024-04-25T02:02:41.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.854+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3508104210 con 0x7f34f406e140 2024-04-25T02:02:41.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.855+0000 7f34fdffb700 1 -- 172.21.15.33:0/733367173 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3508104210 con 0x7f34f406e140 2024-04-25T02:02:41.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f34f406e140 msgr2=0x7f34f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f34f406e140 0x7f34f40705f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3508104460 tx=0x7f34f800b040 comp rx=0 tx=0).stop 2024-04-25T02:02:41.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35080fbd40 msgr2=0x7f3508102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35080fbd40 0x7f3508102b20 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f350400b3b0 tx=0x7f3504009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:41.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 shutdown_connections 2024-04-25T02:02:41.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f34f406e140 0x7f34f40705f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f35080fbd40 0x7f3508102b20 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 --2- 172.21.15.33:0/733367173 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3508101d90 0x7f3508103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 >> 172.21.15.33:0/733367173 conn(0x7f3508071d00 msgr2=0x7f35080fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 shutdown_connections 2024-04-25T02:02:41.864 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.857+0000 7f350f8d2700 1 -- 172.21.15.33:0/733367173 wait complete. 2024-04-25T02:02:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 -- 172.21.15.33:0/717903653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c1026b0 msgr2=0x7fe11c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 --2- 172.21.15.33:0/717903653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c1026b0 0x7fe11c0fbd20 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fe118004940 tx=0x7fe118009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:41.966 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 -- 172.21.15.33:0/717903653 shutdown_connections 2024-04-25T02:02:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 --2- 172.21.15.33:0/717903653 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c1026b0 0x7fe11c0fbd20 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 --2- 172.21.15.33:0/717903653 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe11c101d10 0x7fe11c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 -- 172.21.15.33:0/717903653 >> 172.21.15.33:0/717903653 conn(0x7fe11c0f7860 msgr2=0x7fe11c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 -- 172.21.15.33:0/717903653 shutdown_connections 2024-04-25T02:02:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 -- 172.21.15.33:0/717903653 wait complete. 2024-04-25T02:02:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 Processor -- start 2024-04-25T02:02:41.967 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.961+0000 7fe123df3700 1 -- start start 2024-04-25T02:02:41.968 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe123df3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c101d10 0x7fe11c1128a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe123df3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe11c1026b0 0x7fe11c112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe123df3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c113430 con 0x7fe11c101d10 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe123df3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c10ca30 con 0x7fe11c1026b0 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe121b8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c101d10 0x7fe11c1128a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe121b8f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c101d10 0x7fe11c1128a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56162/0 (socket says 172.21.15.33:56162) 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe121b8f700 1 -- 172.21.15.33:0/1114212640 learned_addr learned my addr 172.21.15.33:0/1114212640 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe12138e700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe11c1026b0 0x7fe11c112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe121b8f700 1 -- 172.21.15.33:0/1114212640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe11c1026b0 msgr2=0x7fe11c112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe121b8f700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe11c1026b0 0x7fe11c112de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe121b8f700 1 -- 172.21.15.33:0/1114212640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe118005040 con 0x7fe11c101d10 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe121b8f700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c101d10 0x7fe11c1128a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe10c002a50 tx=0x7fe10c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe1127fc700 1 -- 172.21.15.33:0/1114212640 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10c007cc0 con 0x7fe11c101d10 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe1127fc700 1 -- 172.21.15.33:0/1114212640 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe10c00f040 con 0x7fe11c101d10 2024-04-25T02:02:41.969 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe11c10cd10 con 0x7fe11c101d10 2024-04-25T02:02:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.962+0000 7fe1127fc700 1 -- 172.21.15.33:0/1114212640 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10c00da30 con 0x7fe11c101d10 2024-04-25T02:02:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.963+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c10d2a0 con 0x7fe11c101d10 2024-04-25T02:02:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.963+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c0fd460 con 0x7fe11c101d10 2024-04-25T02:02:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.964+0000 7fe1127fc700 1 -- 172.21.15.33:0/1114212640 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe10c00dbb0 con 0x7fe11c101d10 2024-04-25T02:02:41.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.964+0000 7fe1127fc700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe10806e140 0x7fe1080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.965+0000 7fe1127fc700 1 -- 172.21.15.33:0/1114212640 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe10c08d7e0 con 0x7fe11c101d10 2024-04-25T02:02:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.965+0000 7fe12138e700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe10806e140 0x7fe1080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:41.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.965+0000 7fe12138e700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe10806e140 0x7fe1080705f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe118004890 tx=0x7fe118009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:41.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:41.969+0000 7fe1127fc700 1 -- 172.21.15.33:0/1114212640 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe10c05a710 con 0x7fe11c101d10 2024-04-25T02:02:42.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.186+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe11c108df0 con 0x7fe10806e140 2024-04-25T02:02:42.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.198+0000 7fe1127fc700 1 -- 172.21.15.33:0/1114212640 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe11c108df0 con 0x7fe10806e140 2024-04-25T02:02:42.203 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (23m) 92s ago 34m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (21m) 92s ago 35m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (20m) 95s ago 34m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (23m) 92s ago 34m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (17m) 92s ago 29m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (17m) 92s ago 29m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (16m) 95s ago 29m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (98s) 95s ago 29m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (25m) 92s ago 36m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (24m) 95s ago 34m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (22m) 92s ago 36m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:02:42.204 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (21m) 95s ago 34m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:02:42.205 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (24m) 92s ago 35m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:02:42.205 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (24m) 95s ago 34m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:02:42.205 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (20m) 92s ago 33m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:02:42.205 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (20m) 92s ago 32m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:02:42.205 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (19m) 92s ago 32m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:02:42.205 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (18m) 95s ago 31m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:02:42.205 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (18m) 95s ago 31m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:02:42.205 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (17m) 95s ago 31m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:02:42.205 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (24m) 92s ago 34m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:02:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe10806e140 msgr2=0x7fe1080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe10806e140 0x7fe1080705f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe118004890 tx=0x7fe118009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c101d10 msgr2=0x7fe11c1128a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c101d10 0x7fe11c1128a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe10c002a50 tx=0x7fe10c0046b0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 shutdown_connections 2024-04-25T02:02:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe10806e140 0x7fe1080705f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe11c101d10 0x7fe11c1128a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 --2- 172.21.15.33:0/1114212640 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe11c1026b0 0x7fe11c112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 >> 172.21.15.33:0/1114212640 conn(0x7fe11c0f7860 msgr2=0x7fe11c0f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 shutdown_connections 2024-04-25T02:02:42.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.200+0000 7fe123df3700 1 -- 172.21.15.33:0/1114212640 wait complete. 2024-04-25T02:02:42.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.303+0000 7f98a87a5700 1 -- 172.21.15.33:0/3646297737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fe7a0 msgr2=0x7f98a0106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.303+0000 7f98a87a5700 1 --2- 172.21.15.33:0/3646297737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fe7a0 0x7f98a0106de0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9890004940 tx=0x7f9890009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:42.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.303+0000 7f98a87a5700 1 -- 172.21.15.33:0/3646297737 shutdown_connections 2024-04-25T02:02:42.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.303+0000 7f98a87a5700 1 --2- 172.21.15.33:0/3646297737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fe7a0 0x7f98a0106de0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.303+0000 7f98a87a5700 1 --2- 172.21.15.33:0/3646297737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00fde00 0x7f98a00fe1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.303+0000 7f98a87a5700 1 -- 172.21.15.33:0/3646297737 >> 172.21.15.33:0/3646297737 conn(0x7f98a00f7870 msgr2=0x7f98a00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:42.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.303+0000 7f98a87a5700 1 -- 172.21.15.33:0/3646297737 shutdown_connections 2024-04-25T02:02:42.309 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.303+0000 7f98a87a5700 1 -- 172.21.15.33:0/3646297737 wait complete. 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.304+0000 7f98a87a5700 1 Processor -- start 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.304+0000 7f98a87a5700 1 -- start start 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.304+0000 7f98a87a5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fde00 0x7f98a0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.304+0000 7f98a87a5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00fe7a0 0x7f98a0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.304+0000 7f98a87a5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0114500 con 0x7f98a00fde00 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.304+0000 7f98a87a5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a010db00 con 0x7f98a00fe7a0 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a6541700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fde00 0x7f98a0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a6541700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fde00 0x7f98a0113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56182/0 (socket says 172.21.15.33:56182) 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a6541700 1 -- 172.21.15.33:0/682130281 learned_addr learned my addr 172.21.15.33:0/682130281 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a5d40700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00fe7a0 0x7f98a0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a6541700 1 -- 172.21.15.33:0/682130281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00fe7a0 msgr2=0x7f98a0113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a6541700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00fe7a0 0x7f98a0113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a6541700 1 -- 172.21.15.33:0/682130281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9890005040 con 0x7f98a00fde00 2024-04-25T02:02:42.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a6541700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fde00 0x7f98a0113970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f989c002a50 tx=0x7f989c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f9896ffd700 1 -- 172.21.15.33:0/682130281 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f989c007cc0 con 0x7f98a00fde00 2024-04-25T02:02:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f9896ffd700 1 -- 172.21.15.33:0/682130281 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f989c00f040 con 0x7f98a00fde00 2024-04-25T02:02:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a010dde0 con 0x7f98a00fde00 2024-04-25T02:02:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f9896ffd700 1 -- 172.21.15.33:0/682130281 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f989c009640 con 0x7f98a00fde00 2024-04-25T02:02:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.305+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a010e320 con 0x7f98a00fde00 2024-04-25T02:02:42.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.306+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a0066b30 con 0x7f98a00fde00 2024-04-25T02:02:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.307+0000 7f9896ffd700 1 -- 172.21.15.33:0/682130281 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f989c0097a0 con 0x7f98a00fde00 2024-04-25T02:02:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.307+0000 7f9896ffd700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f988c06e140 0x7f988c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.308+0000 7f98a5d40700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f988c06e140 0x7f988c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.308+0000 7f9896ffd700 1 -- 172.21.15.33:0/682130281 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f989c08d940 con 0x7f98a00fde00 2024-04-25T02:02:42.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.308+0000 7f98a5d40700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f988c06e140 0x7f988c0705f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f989000b3b0 tx=0x7f9890009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:42.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.312+0000 7f9896ffd700 1 -- 172.21.15.33:0/682130281 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f989c05a7f0 con 0x7f98a00fde00 2024-04-25T02:02:42.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:42 smithi033 ceph-mon[111306]: pgmap v700: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:42.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:42 smithi138 ceph-mon[81304]: pgmap v700: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.604+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f98a0109e90 con 0x7f98a00fde00 2024-04-25T02:02:42.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.605+0000 7f9896ffd700 1 -- 172.21.15.33:0/682130281 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f989c05a380 con 0x7f98a00fde00 2024-04-25T02:02:42.610 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:02:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:02:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:02:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:02:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:02:42.610 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:02:42.611 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:02:42.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f988c06e140 msgr2=0x7f988c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f988c06e140 0x7f988c0705f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f989000b3b0 tx=0x7f9890009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:42.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fde00 msgr2=0x7f98a0113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fde00 0x7f98a0113970 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f989c002a50 tx=0x7f989c0046b0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 shutdown_connections 2024-04-25T02:02:42.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f988c06e140 0x7f988c0705f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98a00fde00 0x7f98a0113970 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 --2- 172.21.15.33:0/682130281 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98a00fe7a0 0x7f98a0113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 >> 172.21.15.33:0/682130281 conn(0x7f98a00f7870 msgr2=0x7f98a00f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:42.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 shutdown_connections 2024-04-25T02:02:42.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.607+0000 7f98a87a5700 1 -- 172.21.15.33:0/682130281 wait complete. 2024-04-25T02:02:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.712+0000 7f7c93b00700 1 -- 172.21.15.33:0/2269447791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c0fbd40 msgr2=0x7f7c8c0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.712+0000 7f7c93b00700 1 --2- 172.21.15.33:0/2269447791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c0fbd40 0x7f7c8c0fc1b0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7c88004940 tx=0x7f7c88009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:42.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.712+0000 7f7c93b00700 1 -- 172.21.15.33:0/2269447791 shutdown_connections 2024-04-25T02:02:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.712+0000 7f7c93b00700 1 --2- 172.21.15.33:0/2269447791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c0fbd40 0x7f7c8c0fc1b0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.712+0000 7f7c93b00700 1 --2- 172.21.15.33:0/2269447791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c8c101d90 0x7f7c8c102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.712+0000 7f7c93b00700 1 -- 172.21.15.33:0/2269447791 >> 172.21.15.33:0/2269447791 conn(0x7f7c8c071d00 msgr2=0x7f7c8c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.712+0000 7f7c93b00700 1 -- 172.21.15.33:0/2269447791 shutdown_connections 2024-04-25T02:02:42.718 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.712+0000 7f7c93b00700 1 -- 172.21.15.33:0/2269447791 wait complete. 2024-04-25T02:02:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c93b00700 1 Processor -- start 2024-04-25T02:02:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c93b00700 1 -- start start 2024-04-25T02:02:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c93b00700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c8c0fbd40 0x7f7c8c102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:42.719 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c93b00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c101d90 0x7f7c8c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c93b00700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c1035f0 con 0x7f7c8c101d90 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c93b00700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8c103730 con 0x7f7c8c0fbd40 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c9109b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c101d90 0x7f7c8c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c9109b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c101d90 0x7f7c8c103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56190/0 (socket says 172.21.15.33:56190) 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c9109b700 1 -- 172.21.15.33:0/2124466063 learned_addr learned my addr 172.21.15.33:0/2124466063 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c9189c700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c8c0fbd40 0x7f7c8c102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c9109b700 1 -- 172.21.15.33:0/2124466063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c8c0fbd40 msgr2=0x7f7c8c102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c9109b700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c8c0fbd40 0x7f7c8c102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.713+0000 7f7c9109b700 1 -- 172.21.15.33:0/2124466063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c88005040 con 0x7f7c8c101d90 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.714+0000 7f7c9109b700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c101d90 0x7f7c8c103060 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800b3b0 tx=0x7f7c88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.714+0000 7f7c827fc700 1 -- 172.21.15.33:0/2124466063 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c8800e040 con 0x7f7c8c101d90 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.714+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8c103980 con 0x7f7c8c101d90 2024-04-25T02:02:42.720 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.714+0000 7f7c827fc700 1 -- 172.21.15.33:0/2124466063 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c88013420 con 0x7f7c8c101d90 2024-04-25T02:02:42.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.714+0000 7f7c827fc700 1 -- 172.21.15.33:0/2124466063 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c8801c630 con 0x7f7c8c101d90 2024-04-25T02:02:42.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.714+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8c11a2e0 con 0x7f7c8c101d90 2024-04-25T02:02:42.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.715+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8c0fd430 con 0x7f7c8c101d90 2024-04-25T02:02:42.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.715+0000 7f7c827fc700 1 -- 172.21.15.33:0/2124466063 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7c88013590 con 0x7f7c8c101d90 2024-04-25T02:02:42.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.716+0000 7f7c827fc700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7c7806e140 0x7f7c780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:42.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.716+0000 7f7c827fc700 1 -- 172.21.15.33:0/2124466063 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7c8808ed80 con 0x7f7c8c101d90 2024-04-25T02:02:42.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.716+0000 7f7c9189c700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7c7806e140 0x7f7c780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:42.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.716+0000 7f7c9189c700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7c7806e140 0x7f7c780705f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c004f40 tx=0x7f7c7c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:42.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.720+0000 7f7c827fc700 1 -- 172.21.15.33:0/2124466063 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7c88058180 con 0x7f7c8c101d90 2024-04-25T02:02:42.979 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.974+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7c8c108e20 con 0x7f7c8c101d90 2024-04-25T02:02:42.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.975+0000 7f7c827fc700 1 -- 172.21.15.33:0/2124466063 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f7c88090770 con 0x7f7c8c101d90 2024-04-25T02:02:42.980 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:02:42.980 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:02:42.980 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:02:42.980 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:02:42.980 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:02:42.981 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:02:42.982 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:02:42.983 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:02:42.983 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:02:42.983 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:02:42.983 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:02:42.983 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:02:42.983 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:02:42.983 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:02:42.983 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7c7806e140 msgr2=0x7f7c780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7c7806e140 0x7f7c780705f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c004f40 tx=0x7f7c7c00b040 comp rx=0 tx=0).stop 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c101d90 msgr2=0x7f7c8c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c101d90 0x7f7c8c103060 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800b3b0 tx=0x7f7c88009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 shutdown_connections 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7c7806e140 0x7f7c780705f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7c8c0fbd40 0x7f7c8c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 --2- 172.21.15.33:0/2124466063 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7c8c101d90 0x7f7c8c103060 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 >> 172.21.15.33:0/2124466063 conn(0x7f7c8c071d00 msgr2=0x7f7c8c0655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:42.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 shutdown_connections 2024-04-25T02:02:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:42.978+0000 7f7c93b00700 1 -- 172.21.15.33:0/2124466063 wait complete. 2024-04-25T02:02:42.985 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.083+0000 7f2b38da6700 1 -- 172.21.15.33:0/306937034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34065640 msgr2=0x7f2b34065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.083+0000 7f2b38da6700 1 --2- 172.21.15.33:0/306937034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34065640 0x7f2b34065ab0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2b24004940 tx=0x7f2b24009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.083+0000 7f2b38da6700 1 -- 172.21.15.33:0/306937034 shutdown_connections 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.083+0000 7f2b38da6700 1 --2- 172.21.15.33:0/306937034 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34065640 0x7f2b34065ab0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.083+0000 7f2b38da6700 1 --2- 172.21.15.33:0/306937034 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34102e50 0x7f2b34103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.083+0000 7f2b38da6700 1 -- 172.21.15.33:0/306937034 >> 172.21.15.33:0/306937034 conn(0x7f2b34071d00 msgr2=0x7f2b34072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.083+0000 7f2b38da6700 1 -- 172.21.15.33:0/306937034 shutdown_connections 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.083+0000 7f2b38da6700 1 -- 172.21.15.33:0/306937034 wait complete. 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b38da6700 1 Processor -- start 2024-04-25T02:02:43.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b38da6700 1 -- start start 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b38da6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34065640 0x7f2b340fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b38da6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34102e50 0x7f2b340fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b38da6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340fc880 con 0x7f2b34102e50 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b38da6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340fc9c0 con 0x7f2b34065640 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b31d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34102e50 0x7f2b340fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b31d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34102e50 0x7f2b340fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56202/0 (socket says 172.21.15.33:56202) 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b31d9b700 1 -- 172.21.15.33:0/1003007218 learned_addr learned my addr 172.21.15.33:0/1003007218 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b3259c700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34065640 0x7f2b340fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b31d9b700 1 -- 172.21.15.33:0/1003007218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34065640 msgr2=0x7f2b340fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b31d9b700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34065640 0x7f2b340fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.084+0000 7f2b31d9b700 1 -- 172.21.15.33:0/1003007218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b24005040 con 0x7f2b34102e50 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.085+0000 7f2b31d9b700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34102e50 0x7f2b340fc2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009150 tx=0x7f2b24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.085+0000 7f2b2affd700 1 -- 172.21.15.33:0/1003007218 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b2400e040 con 0x7f2b34102e50 2024-04-25T02:02:43.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.085+0000 7f2b2affd700 1 -- 172.21.15.33:0/1003007218 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b24013420 con 0x7f2b34102e50 2024-04-25T02:02:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.085+0000 7f2b2affd700 1 -- 172.21.15.33:0/1003007218 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b2401c610 con 0x7f2b34102e50 2024-04-25T02:02:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.085+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b340fcc10 con 0x7f2b34102e50 2024-04-25T02:02:43.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.085+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3411a2a0 con 0x7f2b34102e50 2024-04-25T02:02:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.086+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b34100100 con 0x7f2b34102e50 2024-04-25T02:02:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.086+0000 7f2b2affd700 1 -- 172.21.15.33:0/1003007218 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2b2400baf0 con 0x7f2b34102e50 2024-04-25T02:02:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.087+0000 7f2b2affd700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b2006e150 0x7f2b20070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.087+0000 7f2b2affd700 1 -- 172.21.15.33:0/1003007218 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2b2408ebd0 con 0x7f2b34102e50 2024-04-25T02:02:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.087+0000 7f2b3259c700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b2006e150 0x7f2b20070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:43.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.087+0000 7f2b3259c700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b2006e150 0x7f2b20070600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c004f40 tx=0x7f2b1c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:43.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.092+0000 7f2b2affd700 1 -- 172.21.15.33:0/1003007218 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2b24058070 con 0x7f2b34102e50 2024-04-25T02:02:43.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:43 smithi033 ceph-mon[111306]: from='client.35300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:43.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:43 smithi033 ceph-mon[111306]: from='client.35304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:43.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:43 smithi033 ceph-mon[111306]: from='client.35308 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:43.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:43 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/682130281' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:02:43.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:43 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2124466063' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:02:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.329+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b340fd540 con 0x7f2b2006e150 2024-04-25T02:02:43.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.330+0000 7f2b2affd700 1 -- 172.21.15.33:0/1003007218 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2b340fd540 con 0x7f2b2006e150 2024-04-25T02:02:43.335 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:02:43.335 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:02:43.335 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:02:43.335 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:02:43.335 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:02:43.335 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:02:43.336 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:02:43.336 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:02:43.336 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:02:43.336 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:02:43.336 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:02:43.336 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:02:43.336 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:02:43.336 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:02:43.336 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:02:43.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b2006e150 msgr2=0x7f2b20070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b2006e150 0x7f2b20070600 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c004f40 tx=0x7f2b1c00b040 comp rx=0 tx=0).stop 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34102e50 msgr2=0x7f2b340fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34102e50 0x7f2b340fc2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009150 tx=0x7f2b24009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 shutdown_connections 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2b2006e150 0x7f2b20070600 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2b34065640 0x7f2b340fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 --2- 172.21.15.33:0/1003007218 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2b34102e50 0x7f2b340fc2f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 >> 172.21.15.33:0/1003007218 conn(0x7f2b34071d00 msgr2=0x7f2b340fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 shutdown_connections 2024-04-25T02:02:43.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.332+0000 7f2b38da6700 1 -- 172.21.15.33:0/1003007218 wait complete. 2024-04-25T02:02:43.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.437+0000 7f231fc96700 1 -- 172.21.15.33:0/245284462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318103780 msgr2=0x7f23180fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.437+0000 7f231fc96700 1 --2- 172.21.15.33:0/245284462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318103780 0x7f23180fcdf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f2314004940 tx=0x7f2314009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.437+0000 7f231fc96700 1 -- 172.21.15.33:0/245284462 shutdown_connections 2024-04-25T02:02:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.437+0000 7f231fc96700 1 --2- 172.21.15.33:0/245284462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318103780 0x7f23180fcdf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.437+0000 7f231fc96700 1 --2- 172.21.15.33:0/245284462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2318102de0 0x7f23181031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.437+0000 7f231fc96700 1 -- 172.21.15.33:0/245284462 >> 172.21.15.33:0/245284462 conn(0x7f23180f8950 msgr2=0x7f23180fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.437+0000 7f231fc96700 1 -- 172.21.15.33:0/245284462 shutdown_connections 2024-04-25T02:02:43.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.437+0000 7f231fc96700 1 -- 172.21.15.33:0/245284462 wait complete. 2024-04-25T02:02:43.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.438+0000 7f231fc96700 1 Processor -- start 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.438+0000 7f231fc96700 1 -- start start 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.438+0000 7f231fc96700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318102de0 0x7f2318113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.438+0000 7f231fc96700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2318103780 0x7f23181140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.438+0000 7f231fc96700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2318114730 con 0x7f2318102de0 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.438+0000 7f231fc96700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231810db20 con 0x7f2318103780 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.438+0000 7f231da32700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318102de0 0x7f2318113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.438+0000 7f231da32700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318102de0 0x7f2318113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56218/0 (socket says 172.21.15.33:56218) 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.438+0000 7f231da32700 1 -- 172.21.15.33:0/774105309 learned_addr learned my addr 172.21.15.33:0/774105309 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f231d231700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2318103780 0x7f23181140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f231da32700 1 -- 172.21.15.33:0/774105309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2318103780 msgr2=0x7f23181140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f231da32700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2318103780 0x7f23181140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f231da32700 1 -- 172.21.15.33:0/774105309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314005040 con 0x7f2318102de0 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f231da32700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318102de0 0x7f2318113ba0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2308002a50 tx=0x7f23080046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:43.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f230e7fc700 1 -- 172.21.15.33:0/774105309 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2308007cc0 con 0x7f2318102de0 2024-04-25T02:02:43.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f230e7fc700 1 -- 172.21.15.33:0/774105309 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f230800f040 con 0x7f2318102de0 2024-04-25T02:02:43.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231810de00 con 0x7f2318102de0 2024-04-25T02:02:43.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f230e7fc700 1 -- 172.21.15.33:0/774105309 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f230800da30 con 0x7f2318102de0 2024-04-25T02:02:43.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.439+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231810e390 con 0x7f2318102de0 2024-04-25T02:02:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.440+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23180fe530 con 0x7f2318102de0 2024-04-25T02:02:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.441+0000 7f230e7fc700 1 -- 172.21.15.33:0/774105309 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f230800dbb0 con 0x7f2318102de0 2024-04-25T02:02:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.441+0000 7f230e7fc700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230406e140 0x7f23040705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:02:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.441+0000 7f230e7fc700 1 -- 172.21.15.33:0/774105309 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f230808d7e0 con 0x7f2318102de0 2024-04-25T02:02:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.441+0000 7f231d231700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230406e140 0x7f23040705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:02:43.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.442+0000 7f231d231700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230406e140 0x7f23040705f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2314004890 tx=0x7f2314009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:02:43.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.446+0000 7f230e7fc700 1 -- 172.21.15.33:0/774105309 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f230805a710 con 0x7f2318102de0 2024-04-25T02:02:43.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:43 smithi138 ceph-mon[81304]: from='client.35300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:43.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:43 smithi138 ceph-mon[81304]: from='client.35304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:43.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:43 smithi138 ceph-mon[81304]: from='client.35308 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:43.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:43 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/682130281' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:02:43.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:43 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2124466063' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:02:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.749+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2318109f10 con 0x7f2318102de0 2024-04-25T02:02:43.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.749+0000 7f230e7fc700 1 -- 172.21.15.33:0/774105309 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f230808be60 con 0x7f2318102de0 2024-04-25T02:02:43.754 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:02:43.755 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:02:43.756 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:02:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.751+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230406e140 msgr2=0x7f23040705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.751+0000 7f231fc96700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230406e140 0x7f23040705f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2314004890 tx=0x7f2314009e30 comp rx=0 tx=0).stop 2024-04-25T02:02:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.751+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318102de0 msgr2=0x7f2318113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:02:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.751+0000 7f231fc96700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318102de0 0x7f2318113ba0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f2308002a50 tx=0x7f23080046b0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.751+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 shutdown_connections 2024-04-25T02:02:43.757 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.751+0000 7f231fc96700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230406e140 0x7f23040705f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.751+0000 7f231fc96700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2318102de0 0x7f2318113ba0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.751+0000 7f231fc96700 1 --2- 172.21.15.33:0/774105309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2318103780 0x7f23181140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:02:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.751+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 >> 172.21.15.33:0/774105309 conn(0x7f23180f8950 msgr2=0x7f23180f9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:02:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.752+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 shutdown_connections 2024-04-25T02:02:43.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:02:43.752+0000 7f231fc96700 1 -- 172.21.15.33:0/774105309 wait complete. 2024-04-25T02:02:44.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:44 smithi033 ceph-mon[111306]: pgmap v701: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:44.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:44 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/774105309' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:02:44.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:44 smithi138 ceph-mon[81304]: pgmap v701: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:44.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:44 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/774105309' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:02:45.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:45 smithi033 ceph-mon[111306]: from='client.35320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:45.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:45 smithi138 ceph-mon[81304]: from='client.35320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:02:46.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:46 smithi033 ceph-mon[111306]: pgmap v702: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:46.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:46 smithi138 ceph-mon[81304]: pgmap v702: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:48.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:48 smithi033 ceph-mon[111306]: pgmap v703: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:48.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:48 smithi138 ceph-mon[81304]: pgmap v703: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:50.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:50 smithi033 ceph-mon[111306]: pgmap v704: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:50.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:50 smithi138 ceph-mon[81304]: pgmap v704: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:52.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:52 smithi033 ceph-mon[111306]: pgmap v705: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:52.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:52 smithi138 ceph-mon[81304]: pgmap v705: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:54.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:54 smithi033 ceph-mon[111306]: pgmap v706: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:54.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:54 smithi138 ceph-mon[81304]: pgmap v706: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:56.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:56 smithi033 ceph-mon[111306]: pgmap v707: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:56.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:02:56.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:56 smithi138 ceph-mon[81304]: pgmap v707: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:56.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:02:58.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:02:58 smithi033 ceph-mon[111306]: pgmap v708: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:02:58.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:02:58 smithi138 ceph-mon[81304]: pgmap v708: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:00.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:00 smithi033 ceph-mon[111306]: pgmap v709: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:00.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:00 smithi138 ceph-mon[81304]: pgmap v709: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:02.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:02 smithi033 ceph-mon[111306]: pgmap v710: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:02.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:02 smithi138 ceph-mon[81304]: pgmap v710: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:04.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:04 smithi033 ceph-mon[111306]: pgmap v711: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:04.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:04 smithi138 ceph-mon[81304]: pgmap v711: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:06.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:06 smithi033 ceph-mon[111306]: pgmap v712: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:06.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:06 smithi138 ceph-mon[81304]: pgmap v712: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:08.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:08 smithi033 ceph-mon[111306]: pgmap v713: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:08.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:08 smithi138 ceph-mon[81304]: pgmap v713: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:10.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:10 smithi033 ceph-mon[111306]: pgmap v714: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:10.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:10 smithi138 ceph-mon[81304]: pgmap v714: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:11.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:03:11.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:03:12.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:12 smithi138 ceph-mon[81304]: pgmap v715: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:12.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:12 smithi033 ceph-mon[111306]: pgmap v715: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:13.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 -- 172.21.15.33:0/319431550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fde10 msgr2=0x7f8bcc0fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:13.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 --2- 172.21.15.33:0/319431550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fe1e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009ce0 tx=0x7f8bbc004d10 comp rx=0 tx=0).stop 2024-04-25T02:03:13.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 -- 172.21.15.33:0/319431550 shutdown_connections 2024-04-25T02:03:13.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 --2- 172.21.15.33:0/319431550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:13.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 --2- 172.21.15.33:0/319431550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fe1e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:13.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 -- 172.21.15.33:0/319431550 >> 172.21.15.33:0/319431550 conn(0x7f8bcc0f7860 msgr2=0x7f8bcc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:13.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 -- 172.21.15.33:0/319431550 shutdown_connections 2024-04-25T02:03:13.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 -- 172.21.15.33:0/319431550 wait complete. 2024-04-25T02:03:13.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 Processor -- start 2024-04-25T02:03:13.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.871+0000 7f8bd1163700 1 -- start start 2024-04-25T02:03:13.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bd1163700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:13.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bd1163700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:13.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bd1163700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0fdb90 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bd1163700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bcc0fc6c0 con 0x7f8bcc0fde10 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49604/0 (socket says 172.21.15.33:49604) 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bca59c700 1 -- 172.21.15.33:0/2126423061 learned_addr learned my addr 172.21.15.33:0/2126423061 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bcad9d700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bca59c700 1 -- 172.21.15.33:0/2126423061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 msgr2=0x7f8bcc0fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bca59c700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bca59c700 1 -- 172.21.15.33:0/2126423061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc009940 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.872+0000 7f8bca59c700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0002a50 tx=0x7f8bc00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.873+0000 7f8bb37fe700 1 -- 172.21.15.33:0/2126423061 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc0007cc0 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.873+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bcc0fc9a0 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.873+0000 7f8bb37fe700 1 -- 172.21.15.33:0/2126423061 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bc000f040 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.873+0000 7f8bb37fe700 1 -- 172.21.15.33:0/2126423061 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc000da30 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.873+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bcc11a0d0 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.874+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bcc066b50 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.874+0000 7f8bb37fe700 1 -- 172.21.15.33:0/2126423061 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8bc000db90 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.875+0000 7f8bb37fe700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e140 0x7f8bb40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:13.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.875+0000 7f8bb37fe700 1 -- 172.21.15.33:0/2126423061 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8bc008d8d0 con 0x7f8bcc0fe7b0 2024-04-25T02:03:13.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.875+0000 7f8bcad9d700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e140 0x7f8bb40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:13.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.875+0000 7f8bcad9d700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e140 0x7f8bb40705f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00b740 tx=0x7f8bbc004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:13.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:13.881+0000 7f8bb37fe700 1 -- 172.21.15.33:0/2126423061 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8bc0056c50 con 0x7f8bcc0fe7b0 2024-04-25T02:03:14.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.109+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bcc0f94a0 con 0x7f8bb406e140 2024-04-25T02:03:14.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.110+0000 7f8bb37fe700 1 -- 172.21.15.33:0/2126423061 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8bcc0f94a0 con 0x7f8bb406e140 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e140 msgr2=0x7f8bb40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e140 0x7f8bb40705f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00b740 tx=0x7f8bbc004d10 comp rx=0 tx=0).stop 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 msgr2=0x7f8bcc0fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0002a50 tx=0x7f8bc00046b0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 shutdown_connections 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8bb406e140 0x7f8bb40705f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8bcc0fde10 0x7f8bcc0fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 --2- 172.21.15.33:0/2126423061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8bcc0fe7b0 0x7f8bcc0fc180 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 >> 172.21.15.33:0/2126423061 conn(0x7f8bcc0f7860 msgr2=0x7f8bcc0f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 shutdown_connections 2024-04-25T02:03:14.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.112+0000 7f8bd1163700 1 -- 172.21.15.33:0/2126423061 wait complete. 2024-04-25T02:03:14.128 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:03:14.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.217+0000 7f5157d64700 1 -- 172.21.15.33:0/1683673666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51501026b0 msgr2=0x7f51500fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.217+0000 7f5157d64700 1 --2- 172.21.15.33:0/1683673666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51501026b0 0x7f51500fbd20 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f514c004940 tx=0x7f514c009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:14.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.217+0000 7f5157d64700 1 -- 172.21.15.33:0/1683673666 shutdown_connections 2024-04-25T02:03:14.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.217+0000 7f5157d64700 1 --2- 172.21.15.33:0/1683673666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f51501026b0 0x7f51500fbd20 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.217+0000 7f5157d64700 1 --2- 172.21.15.33:0/1683673666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5150101d10 0x7f51501020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.217+0000 7f5157d64700 1 -- 172.21.15.33:0/1683673666 >> 172.21.15.33:0/1683673666 conn(0x7f51500f7860 msgr2=0x7f51500f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:14.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.217+0000 7f5157d64700 1 -- 172.21.15.33:0/1683673666 shutdown_connections 2024-04-25T02:03:14.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.217+0000 7f5157d64700 1 -- 172.21.15.33:0/1683673666 wait complete. 2024-04-25T02:03:14.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.218+0000 7f5157d64700 1 Processor -- start 2024-04-25T02:03:14.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.218+0000 7f5157d64700 1 -- start start 2024-04-25T02:03:14.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.218+0000 7f5157d64700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5150101d10 0x7f5150113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:14.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.218+0000 7f5157d64700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51501026b0 0x7f5150113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:14.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.218+0000 7f5157d64700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51501144d0 con 0x7f5150101d10 2024-04-25T02:03:14.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.218+0000 7f5157d64700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515010dad0 con 0x7f51501026b0 2024-04-25T02:03:14.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.218+0000 7f5155b00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5150101d10 0x7f5150113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.218+0000 7f5155b00700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5150101d10 0x7f5150113940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49620/0 (socket says 172.21.15.33:49620) 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.218+0000 7f5155b00700 1 -- 172.21.15.33:0/2771860774 learned_addr learned my addr 172.21.15.33:0/2771860774 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f51552ff700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51501026b0 0x7f5150113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f5155b00700 1 -- 172.21.15.33:0/2771860774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51501026b0 msgr2=0x7f5150113e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f5155b00700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51501026b0 0x7f5150113e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f5155b00700 1 -- 172.21.15.33:0/2771860774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c005040 con 0x7f5150101d10 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f5155b00700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5150101d10 0x7f5150113940 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f514000eb40 tx=0x7f514000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f51467fc700 1 -- 172.21.15.33:0/2771860774 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5140007cc0 con 0x7f5150101d10 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f51467fc700 1 -- 172.21.15.33:0/2771860774 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f514000f040 con 0x7f5150101d10 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f515010dd50 con 0x7f5150101d10 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f51467fc700 1 -- 172.21.15.33:0/2771860774 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f514000ca10 con 0x7f5150101d10 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.219+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f515010e260 con 0x7f5150101d10 2024-04-25T02:03:14.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.220+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51500fd460 con 0x7f5150101d10 2024-04-25T02:03:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.221+0000 7f51467fc700 1 -- 172.21.15.33:0/2771860774 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f514000cc80 con 0x7f5150101d10 2024-04-25T02:03:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.221+0000 7f51467fc700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f513c06e140 0x7f513c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.221+0000 7f51467fc700 1 -- 172.21.15.33:0/2771860774 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f514008d6f0 con 0x7f5150101d10 2024-04-25T02:03:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.221+0000 7f51552ff700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f513c06e140 0x7f513c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:14.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.222+0000 7f51552ff700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f513c06e140 0x7f513c0705f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f514c004890 tx=0x7f514c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:14.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.227+0000 7f51467fc700 1 -- 172.21.15.33:0/2771860774 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f514005a650 con 0x7f5150101d10 2024-04-25T02:03:14.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.452+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5150108e60 con 0x7f513c06e140 2024-04-25T02:03:14.457 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:14 smithi033 ceph-mon[111306]: pgmap v716: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:14.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.453+0000 7f51467fc700 1 -- 172.21.15.33:0/2771860774 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5150108e60 con 0x7f513c06e140 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.455+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f513c06e140 msgr2=0x7f513c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.455+0000 7f5157d64700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f513c06e140 0x7f513c0705f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f514c004890 tx=0x7f514c009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.455+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5150101d10 msgr2=0x7f5150113940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.455+0000 7f5157d64700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5150101d10 0x7f5150113940 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f514000eb40 tx=0x7f514000b040 comp rx=0 tx=0).stop 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.455+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 shutdown_connections 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.455+0000 7f5157d64700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f513c06e140 0x7f513c0705f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.455+0000 7f5157d64700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5150101d10 0x7f5150113940 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.455+0000 7f5157d64700 1 --2- 172.21.15.33:0/2771860774 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f51501026b0 0x7f5150113e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.456+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 >> 172.21.15.33:0/2771860774 conn(0x7f51500f7860 msgr2=0x7f51500f8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:14.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.456+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 shutdown_connections 2024-04-25T02:03:14.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.456+0000 7f5157d64700 1 -- 172.21.15.33:0/2771860774 wait complete. 2024-04-25T02:03:14.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:14 smithi138 ceph-mon[81304]: pgmap v716: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.559+0000 7f46f8bc8700 1 -- 172.21.15.33:0/2330606061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070280 msgr2=0x7f46f4070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.559+0000 7f46f8bc8700 1 --2- 172.21.15.33:0/2330606061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070280 0x7f46f4070650 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f46e4004940 tx=0x7f46e4009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 -- 172.21.15.33:0/2330606061 shutdown_connections 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 --2- 172.21.15.33:0/2330606061 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4070b90 0x7f46f4109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 --2- 172.21.15.33:0/2330606061 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070280 0x7f46f4070650 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 -- 172.21.15.33:0/2330606061 >> 172.21.15.33:0/2330606061 conn(0x7f46f40f8990 msgr2=0x7f46f40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 -- 172.21.15.33:0/2330606061 shutdown_connections 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 -- 172.21.15.33:0/2330606061 wait complete. 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 Processor -- start 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 -- start start 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4070280 0x7f46f4115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070b90 0x7f46f41160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f41167a0 con 0x7f46f4070b90 2024-04-25T02:03:14.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.560+0000 7f46f8bc8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46f410fcf0 con 0x7f46f4070280 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070b90 0x7f46f41160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070b90 0x7f46f41160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49644/0 (socket says 172.21.15.33:49644) 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f1d9b700 1 -- 172.21.15.33:0/162240341 learned_addr learned my addr 172.21.15.33:0/162240341 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f259c700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4070280 0x7f46f4115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f1d9b700 1 -- 172.21.15.33:0/162240341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4070280 msgr2=0x7f46f4115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f1d9b700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4070280 0x7f46f4115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f1d9b700 1 -- 172.21.15.33:0/162240341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e4005040 con 0x7f46f4070b90 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f1d9b700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070b90 0x7f46f41160c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f46e800b790 tx=0x7f46e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46e2ffd700 1 -- 172.21.15.33:0/162240341 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46e800f770 con 0x7f46f4070b90 2024-04-25T02:03:14.568 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46f410ff70 con 0x7f46f4070b90 2024-04-25T02:03:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46e2ffd700 1 -- 172.21.15.33:0/162240341 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46e800fda0 con 0x7f46f4070b90 2024-04-25T02:03:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46e2ffd700 1 -- 172.21.15.33:0/162240341 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46e800d310 con 0x7f46f4070b90 2024-04-25T02:03:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.561+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46f41104b0 con 0x7f46f4070b90 2024-04-25T02:03:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.562+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46f40719c0 con 0x7f46f4070b90 2024-04-25T02:03:14.569 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.563+0000 7f46e2ffd700 1 -- 172.21.15.33:0/162240341 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f46e8010420 con 0x7f46f4070b90 2024-04-25T02:03:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.563+0000 7f46e2ffd700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46dc06e140 0x7f46dc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.563+0000 7f46e2ffd700 1 -- 172.21.15.33:0/162240341 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f46e808db40 con 0x7f46f4070b90 2024-04-25T02:03:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.563+0000 7f46f259c700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46dc06e140 0x7f46dc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:14.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.564+0000 7f46f259c700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46dc06e140 0x7f46dc0705f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f46e400b020 tx=0x7f46e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:14.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.569+0000 7f46e2ffd700 1 -- 172.21.15.33:0/162240341 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f46e805aa70 con 0x7f46f4070b90 2024-04-25T02:03:14.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.785+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f46f4110730 con 0x7f46dc06e140 2024-04-25T02:03:14.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.796+0000 7f46e2ffd700 1 -- 172.21.15.33:0/162240341 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f46f4110730 con 0x7f46dc06e140 2024-04-25T02:03:14.802 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:03:14.803 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (24m) 2m ago 35m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:03:14.803 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (21m) 2m ago 35m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:03:14.803 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (21m) 2m ago 34m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:03:14.803 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (24m) 2m ago 35m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:03:14.803 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (17m) 2m ago 30m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:03:14.803 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (17m) 2m ago 30m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:03:14.804 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (17m) 2m ago 30m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:03:14.804 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (2m) 2m ago 30m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:03:14.804 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (25m) 2m ago 36m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:03:14.804 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (25m) 2m ago 34m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:03:14.804 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (22m) 2m ago 37m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:03:14.804 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (22m) 2m ago 34m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:03:14.805 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (25m) 2m ago 35m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:03:14.805 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (25m) 2m ago 34m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:03:14.805 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (21m) 2m ago 33m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:03:14.805 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (20m) 2m ago 33m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:03:14.805 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (19m) 2m ago 32m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:03:14.805 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (19m) 2m ago 32m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:03:14.806 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (18m) 2m ago 32m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:03:14.806 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (18m) 2m ago 31m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:03:14.806 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (24m) 2m ago 35m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:03:14.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46dc06e140 msgr2=0x7f46dc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46dc06e140 0x7f46dc0705f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f46e400b020 tx=0x7f46e4009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070b90 msgr2=0x7f46f41160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070b90 0x7f46f41160c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f46e800b790 tx=0x7f46e8009250 comp rx=0 tx=0).stop 2024-04-25T02:03:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 shutdown_connections 2024-04-25T02:03:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46dc06e140 0x7f46dc0705f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46f4070280 0x7f46f4115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 --2- 172.21.15.33:0/162240341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46f4070b90 0x7f46f41160c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 >> 172.21.15.33:0/162240341 conn(0x7f46f40f8990 msgr2=0x7f46f40f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:14.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 shutdown_connections 2024-04-25T02:03:14.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.798+0000 7f46f8bc8700 1 -- 172.21.15.33:0/162240341 wait complete. 2024-04-25T02:03:14.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 -- 172.21.15.33:0/2933737300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 msgr2=0x7f0e580fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 --2- 172.21.15.33:0/2933737300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 0x7f0e580fcdf0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c004940 tx=0x7f0e4c009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:14.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 -- 172.21.15.33:0/2933737300 shutdown_connections 2024-04-25T02:03:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 --2- 172.21.15.33:0/2933737300 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 0x7f0e580fcdf0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 --2- 172.21.15.33:0/2933737300 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e58102de0 0x7f0e581031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 -- 172.21.15.33:0/2933737300 >> 172.21.15.33:0/2933737300 conn(0x7f0e580f8990 msgr2=0x7f0e580fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 -- 172.21.15.33:0/2933737300 shutdown_connections 2024-04-25T02:03:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 -- 172.21.15.33:0/2933737300 wait complete. 2024-04-25T02:03:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 Processor -- start 2024-04-25T02:03:14.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.901+0000 7f0e5e224700 1 -- start start 2024-04-25T02:03:14.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e5e224700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e58102de0 0x7f0e58113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e5e224700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 0x7f0e58113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e5e224700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e58114520 con 0x7f0e58103780 2024-04-25T02:03:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e5e224700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e5810db20 con 0x7f0e58102de0 2024-04-25T02:03:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 0x7f0e58113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 0x7f0e58113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49668/0 (socket says 172.21.15.33:49668) 2024-04-25T02:03:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e56ffd700 1 -- 172.21.15.33:0/1149485485 learned_addr learned my addr 172.21.15.33:0/1149485485 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e56ffd700 1 -- 172.21.15.33:0/1149485485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e58102de0 msgr2=0x7f0e58113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e577fe700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e58102de0 0x7f0e58113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:14.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e56ffd700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e58102de0 0x7f0e58113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e56ffd700 1 -- 172.21.15.33:0/1149485485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c005040 con 0x7f0e58103780 2024-04-25T02:03:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e56ffd700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 0x7f0e58113ed0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009100 tx=0x7f0e4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e3ffff700 1 -- 172.21.15.33:0/1149485485 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e4c00e040 con 0x7f0e58103780 2024-04-25T02:03:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e5810dda0 con 0x7f0e58103780 2024-04-25T02:03:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e3ffff700 1 -- 172.21.15.33:0/1149485485 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e4c013420 con 0x7f0e58103780 2024-04-25T02:03:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.902+0000 7f0e3ffff700 1 -- 172.21.15.33:0/1149485485 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e4c01c630 con 0x7f0e58103780 2024-04-25T02:03:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.903+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e5810e360 con 0x7f0e58103780 2024-04-25T02:03:14.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.903+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e580fe530 con 0x7f0e58103780 2024-04-25T02:03:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.905+0000 7f0e3ffff700 1 -- 172.21.15.33:0/1149485485 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0e4c013590 con 0x7f0e58103780 2024-04-25T02:03:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.905+0000 7f0e3ffff700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0e4006e140 0x7f0e400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.906+0000 7f0e3ffff700 1 -- 172.21.15.33:0/1149485485 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0e4c08ed60 con 0x7f0e58103780 2024-04-25T02:03:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.906+0000 7f0e577fe700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0e4006e140 0x7f0e400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:14.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.906+0000 7f0e577fe700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0e4006e140 0x7f0e400705f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0e48004d60 tx=0x7f0e4800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:14.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:14.909+0000 7f0e3ffff700 1 -- 172.21.15.33:0/1149485485 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0e4c05bc90 con 0x7f0e58103780 2024-04-25T02:03:15.216 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.212+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0e58109ec0 con 0x7f0e58103780 2024-04-25T02:03:15.217 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.213+0000 7f0e3ffff700 1 -- 172.21.15.33:0/1149485485 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f0e4c05b820 con 0x7f0e58103780 2024-04-25T02:03:15.217 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:03:15.217 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:03:15.217 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:03:15.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:03:15.217 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:03:15.217 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:03:15.217 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:03:15.218 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:03:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0e4006e140 msgr2=0x7f0e400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0e4006e140 0x7f0e400705f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0e48004d60 tx=0x7f0e4800b040 comp rx=0 tx=0).stop 2024-04-25T02:03:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 msgr2=0x7f0e58113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 0x7f0e58113ed0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c009100 tx=0x7f0e4c009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 shutdown_connections 2024-04-25T02:03:15.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0e4006e140 0x7f0e400705f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0e58102de0 0x7f0e58113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 --2- 172.21.15.33:0/1149485485 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0e58103780 0x7f0e58113ed0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 >> 172.21.15.33:0/1149485485 conn(0x7f0e580f8990 msgr2=0x7f0e580f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 shutdown_connections 2024-04-25T02:03:15.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.216+0000 7f0e5e224700 1 -- 172.21.15.33:0/1149485485 wait complete. 2024-04-25T02:03:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.321+0000 7f7a96df2700 1 -- 172.21.15.33:0/3832196598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a901026a0 msgr2=0x7f7a900fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.321+0000 7f7a96df2700 1 --2- 172.21.15.33:0/3832196598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a901026a0 0x7f7a900fbd10 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7a80004940 tx=0x7f7a80009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.321+0000 7f7a96df2700 1 -- 172.21.15.33:0/3832196598 shutdown_connections 2024-04-25T02:03:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.321+0000 7f7a96df2700 1 --2- 172.21.15.33:0/3832196598 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a901026a0 0x7f7a900fbd10 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.321+0000 7f7a96df2700 1 --2- 172.21.15.33:0/3832196598 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a90101d00 0x7f7a901020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.321+0000 7f7a96df2700 1 -- 172.21.15.33:0/3832196598 >> 172.21.15.33:0/3832196598 conn(0x7f7a900f7870 msgr2=0x7f7a900f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:15.327 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.321+0000 7f7a96df2700 1 -- 172.21.15.33:0/3832196598 shutdown_connections 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.321+0000 7f7a96df2700 1 -- 172.21.15.33:0/3832196598 wait complete. 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.322+0000 7f7a96df2700 1 Processor -- start 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.322+0000 7f7a96df2700 1 -- start start 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.322+0000 7f7a96df2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a90101d00 0x7f7a90115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.322+0000 7f7a96df2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a901026a0 0x7f7a90116100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.322+0000 7f7a96df2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a90116750 con 0x7f7a90101d00 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.322+0000 7f7a96df2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9010fd30 con 0x7f7a901026a0 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a94b8e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a90101d00 0x7f7a90115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a94b8e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a90101d00 0x7f7a90115bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:49690/0 (socket says 172.21.15.33:49690) 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a94b8e700 1 -- 172.21.15.33:0/122009441 learned_addr learned my addr 172.21.15.33:0/122009441 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:15.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a8ffff700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a901026a0 0x7f7a90116100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:15.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a94b8e700 1 -- 172.21.15.33:0/122009441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a901026a0 msgr2=0x7f7a90116100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a94b8e700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a901026a0 0x7f7a90116100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a94b8e700 1 -- 172.21.15.33:0/122009441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80005040 con 0x7f7a90101d00 2024-04-25T02:03:15.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a94b8e700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a90101d00 0x7f7a90115bc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7a78002a50 tx=0x7f7a780046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:15.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a8d7fa700 1 -- 172.21.15.33:0/122009441 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a78007cc0 con 0x7f7a90101d00 2024-04-25T02:03:15.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a90110010 con 0x7f7a90101d00 2024-04-25T02:03:15.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a8d7fa700 1 -- 172.21.15.33:0/122009441 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a7800f040 con 0x7f7a90101d00 2024-04-25T02:03:15.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a901105d0 con 0x7f7a90101d00 2024-04-25T02:03:15.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.323+0000 7f7a8d7fa700 1 -- 172.21.15.33:0/122009441 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a78009640 con 0x7f7a90101d00 2024-04-25T02:03:15.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.324+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a700052b0 con 0x7f7a90101d00 2024-04-25T02:03:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.325+0000 7f7a8d7fa700 1 -- 172.21.15.33:0/122009441 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7a7801c400 con 0x7f7a90101d00 2024-04-25T02:03:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.325+0000 7f7a8d7fa700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7a7c06e140 0x7f7a7c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.325+0000 7f7a8d7fa700 1 -- 172.21.15.33:0/122009441 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7a7808d8e0 con 0x7f7a90101d00 2024-04-25T02:03:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.325+0000 7f7a8ffff700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7a7c06e140 0x7f7a7c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:15.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.326+0000 7f7a8ffff700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7a7c06e140 0x7f7a7c0705f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000b3b0 tx=0x7f7a80009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:15.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.330+0000 7f7a8d7fa700 1 -- 172.21.15.33:0/122009441 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7a7805a790 con 0x7f7a90101d00 2024-04-25T02:03:15.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:15 smithi033 ceph-mon[111306]: from='client.35328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:15.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:15 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1149485485' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:03:15.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:15 smithi138 ceph-mon[81304]: from='client.35328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:15.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:15 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1149485485' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:03:15.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.582+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7a70005d70 con 0x7f7a90101d00 2024-04-25T02:03:15.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.582+0000 7f7a8d7fa700 1 -- 172.21.15.33:0/122009441 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f7a7805a320 con 0x7f7a90101d00 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:03:15.587 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:03:15.588 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:03:15.589 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:03:15.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:03:15.590 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:03:15.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7a7c06e140 msgr2=0x7f7a7c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7a7c06e140 0x7f7a7c0705f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7a8000b3b0 tx=0x7f7a80009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:15.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a90101d00 msgr2=0x7f7a90115bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a90101d00 0x7f7a90115bc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f7a78002a50 tx=0x7f7a780046b0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 shutdown_connections 2024-04-25T02:03:15.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7a7c06e140 0x7f7a7c0705f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a90101d00 0x7f7a90115bc0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 --2- 172.21.15.33:0/122009441 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a901026a0 0x7f7a90116100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 >> 172.21.15.33:0/122009441 conn(0x7f7a900f7870 msgr2=0x7f7a900f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:15.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 shutdown_connections 2024-04-25T02:03:15.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.584+0000 7f7a96df2700 1 -- 172.21.15.33:0/122009441 wait complete. 2024-04-25T02:03:15.592 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:03:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.689+0000 7f403e296700 1 -- 172.21.15.33:0/2802093502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 msgr2=0x7f40381031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.689+0000 7f403e296700 1 --2- 172.21.15.33:0/2802093502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 0x7f40381031b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4028004940 tx=0x7f4028009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.689+0000 7f403e296700 1 -- 172.21.15.33:0/2802093502 shutdown_connections 2024-04-25T02:03:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.689+0000 7f403e296700 1 --2- 172.21.15.33:0/2802093502 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4038103780 0x7f40380fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.689+0000 7f403e296700 1 --2- 172.21.15.33:0/2802093502 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 0x7f40381031b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.689+0000 7f403e296700 1 -- 172.21.15.33:0/2802093502 >> 172.21.15.33:0/2802093502 conn(0x7f40380f8990 msgr2=0x7f40380fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.689+0000 7f403e296700 1 -- 172.21.15.33:0/2802093502 shutdown_connections 2024-04-25T02:03:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.689+0000 7f403e296700 1 -- 172.21.15.33:0/2802093502 wait complete. 2024-04-25T02:03:15.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f403e296700 1 Processor -- start 2024-04-25T02:03:15.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f403e296700 1 -- start start 2024-04-25T02:03:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f403e296700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 0x7f4038113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f403e296700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4038103780 0x7f4038113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f403e296700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038114500 con 0x7f4038102de0 2024-04-25T02:03:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f403e296700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403810db50 con 0x7f4038103780 2024-04-25T02:03:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f4037fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 0x7f4038113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f4037fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 0x7f4038113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56124/0 (socket says 172.21.15.33:56124) 2024-04-25T02:03:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f4037fff700 1 -- 172.21.15.33:0/2232571481 learned_addr learned my addr 172.21.15.33:0/2232571481 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:15.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f40377fe700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4038103780 0x7f4038113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f4037fff700 1 -- 172.21.15.33:0/2232571481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4038103780 msgr2=0x7f4038113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f4037fff700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4038103780 0x7f4038113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.690+0000 7f4037fff700 1 -- 172.21.15.33:0/2232571481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4028005040 con 0x7f4038102de0 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.691+0000 7f4037fff700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 0x7f4038113970 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f402800b3b0 tx=0x7f4028009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.691+0000 7f4034ff9700 1 -- 172.21.15.33:0/2232571481 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f402800e040 con 0x7f4038102de0 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.691+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403810ddd0 con 0x7f4038102de0 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.691+0000 7f4034ff9700 1 -- 172.21.15.33:0/2232571481 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f402800bbc0 con 0x7f4038102de0 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.691+0000 7f4034ff9700 1 -- 172.21.15.33:0/2232571481 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f402801c3f0 con 0x7f4038102de0 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.691+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403810e390 con 0x7f4038102de0 2024-04-25T02:03:15.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.692+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40381000f0 con 0x7f4038102de0 2024-04-25T02:03:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.692+0000 7f4034ff9700 1 -- 172.21.15.33:0/2232571481 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f402800bd30 con 0x7f4038102de0 2024-04-25T02:03:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.693+0000 7f4034ff9700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f402406e140 0x7f40240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.693+0000 7f4034ff9700 1 -- 172.21.15.33:0/2232571481 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f402808edc0 con 0x7f4038102de0 2024-04-25T02:03:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.693+0000 7f40377fe700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f402406e140 0x7f40240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:15.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.693+0000 7f40377fe700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f402406e140 0x7f40240705f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4020007fd0 tx=0x7f4020009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:15.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.697+0000 7f4034ff9700 1 -- 172.21.15.33:0/2232571481 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f402805bc70 con 0x7f4038102de0 2024-04-25T02:03:15.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.925+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f403804c7b0 con 0x7f402406e140 2024-04-25T02:03:15.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.926+0000 7f4034ff9700 1 -- 172.21.15.33:0/2232571481 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f403804c7b0 con 0x7f402406e140 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:03:15.931 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:03:15.932 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:03:15.932 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.928+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f402406e140 msgr2=0x7f40240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.928+0000 7f403e296700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f402406e140 0x7f40240705f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4020007fd0 tx=0x7f4020009040 comp rx=0 tx=0).stop 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.928+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 msgr2=0x7f4038113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.928+0000 7f403e296700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 0x7f4038113970 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f402800b3b0 tx=0x7f4028009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.928+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 shutdown_connections 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.928+0000 7f403e296700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f402406e140 0x7f40240705f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.928+0000 7f403e296700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4038102de0 0x7f4038113970 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.928+0000 7f403e296700 1 --2- 172.21.15.33:0/2232571481 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4038103780 0x7f4038113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.929+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 >> 172.21.15.33:0/2232571481 conn(0x7f40380f8990 msgr2=0x7f4038107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:15.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.929+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 shutdown_connections 2024-04-25T02:03:15.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:15.929+0000 7f403e296700 1 -- 172.21.15.33:0/2232571481 wait complete. 2024-04-25T02:03:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.033+0000 7fb019c8c700 1 -- 172.21.15.33:0/1134223094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 msgr2=0x7fb014106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.033+0000 7fb019c8c700 1 --2- 172.21.15.33:0/1134223094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 0x7fb014106e50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb008004940 tx=0x7fb008009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:16.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.033+0000 7fb019c8c700 1 -- 172.21.15.33:0/1134223094 shutdown_connections 2024-04-25T02:03:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.033+0000 7fb019c8c700 1 --2- 172.21.15.33:0/1134223094 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 0x7fb014106e50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.033+0000 7fb019c8c700 1 --2- 172.21.15.33:0/1134223094 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0140fde70 0x7fb0140fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.033+0000 7fb019c8c700 1 -- 172.21.15.33:0/1134223094 >> 172.21.15.33:0/1134223094 conn(0x7fb0140f7860 msgr2=0x7fb0140f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.034+0000 7fb019c8c700 1 -- 172.21.15.33:0/1134223094 shutdown_connections 2024-04-25T02:03:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.034+0000 7fb019c8c700 1 -- 172.21.15.33:0/1134223094 wait complete. 2024-04-25T02:03:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.034+0000 7fb019c8c700 1 Processor -- start 2024-04-25T02:03:16.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.034+0000 7fb019c8c700 1 -- start start 2024-04-25T02:03:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.034+0000 7fb019c8c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0140fde70 0x7fb014113940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:16.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.034+0000 7fb019c8c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 0x7fb014113e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb019c8c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb014114560 con 0x7fb0140fe810 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb019c8c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01410dad0 con 0x7fb0140fde70 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb012ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 0x7fb014113e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb012ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 0x7fb014113e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56142/0 (socket says 172.21.15.33:56142) 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb012ffd700 1 -- 172.21.15.33:0/104878015 learned_addr learned my addr 172.21.15.33:0/104878015 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb0137fe700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0140fde70 0x7fb014113940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb012ffd700 1 -- 172.21.15.33:0/104878015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0140fde70 msgr2=0x7fb014113940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb012ffd700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0140fde70 0x7fb014113940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb012ffd700 1 -- 172.21.15.33:0/104878015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb008005040 con 0x7fb0140fe810 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb012ffd700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 0x7fb014113e80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb008009180 tx=0x7fb008009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7faffbfff700 1 -- 172.21.15.33:0/104878015 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb00800e040 con 0x7fb0140fe810 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7faffbfff700 1 -- 172.21.15.33:0/104878015 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb008013420 con 0x7fb0140fe810 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01410ddb0 con 0x7fb0140fe810 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7faffbfff700 1 -- 172.21.15.33:0/104878015 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb00801c630 con 0x7fb0140fe810 2024-04-25T02:03:16.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.035+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01410e2f0 con 0x7fb0140fe810 2024-04-25T02:03:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.036+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb014066af0 con 0x7fb0140fe810 2024-04-25T02:03:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.037+0000 7faffbfff700 1 -- 172.21.15.33:0/104878015 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb008013590 con 0x7fb0140fe810 2024-04-25T02:03:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.037+0000 7faffbfff700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faffc06e140 0x7faffc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:16.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.038+0000 7faffbfff700 1 -- 172.21.15.33:0/104878015 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb00808edc0 con 0x7fb0140fe810 2024-04-25T02:03:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.038+0000 7fb0137fe700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faffc06e140 0x7faffc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:16.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.038+0000 7fb0137fe700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faffc06e140 0x7faffc0705f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7fb004008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:16.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.042+0000 7faffbfff700 1 -- 172.21.15.33:0/104878015 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb00805bd90 con 0x7fb0140fe810 2024-04-25T02:03:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.348+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb01404c710 con 0x7fb0140fe810 2024-04-25T02:03:16.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.348+0000 7faffbfff700 1 -- 172.21.15.33:0/104878015 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fb00805b920 con 0x7fb0140fe810 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:03:16.353 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:03:16.354 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:03:16.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faffc06e140 msgr2=0x7faffc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:16.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faffc06e140 0x7faffc0705f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb004006fd0 tx=0x7fb004008040 comp rx=0 tx=0).stop 2024-04-25T02:03:16.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 msgr2=0x7fb014113e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:16.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 0x7fb014113e80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb008009180 tx=0x7fb008009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:16.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 shutdown_connections 2024-04-25T02:03:16.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faffc06e140 0x7faffc0705f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:16.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0140fde70 0x7fb014113940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:16.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 --2- 172.21.15.33:0/104878015 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0140fe810 0x7fb014113e80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:16.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 >> 172.21.15.33:0/104878015 conn(0x7fb0140f7860 msgr2=0x7fb0140f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:16.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 shutdown_connections 2024-04-25T02:03:16.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:16.350+0000 7fb019c8c700 1 -- 172.21.15.33:0/104878015 wait complete. 2024-04-25T02:03:16.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:16 smithi138 ceph-mon[81304]: from='client.35332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:16.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:16 smithi138 ceph-mon[81304]: from='client.35336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:16.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:16 smithi138 ceph-mon[81304]: pgmap v717: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:16.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:16 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/122009441' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:03:16.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:16 smithi033 ceph-mon[111306]: from='client.35332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:16.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:16 smithi033 ceph-mon[111306]: from='client.35336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:16.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:16 smithi033 ceph-mon[111306]: pgmap v717: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:16.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:16 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/122009441' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:03:17.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:17 smithi138 ceph-mon[81304]: from='client.35348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:17.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:17 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/104878015' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:03:17.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:17 smithi033 ceph-mon[111306]: from='client.35348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:17.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:17 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/104878015' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:03:18.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:18 smithi138 ceph-mon[81304]: pgmap v718: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:18.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:18 smithi033 ceph-mon[111306]: pgmap v718: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:20.397 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:20 smithi138 ceph-mon[81304]: pgmap v719: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:20.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:20 smithi033 ceph-mon[111306]: pgmap v719: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:22.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:22 smithi033 ceph-mon[111306]: pgmap v720: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:22.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:22 smithi138 ceph-mon[81304]: pgmap v720: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:24.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:24 smithi033 ceph-mon[111306]: pgmap v721: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:24.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:24 smithi138 ceph-mon[81304]: pgmap v721: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:25.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:03:25.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:03:26.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:26 smithi033 ceph-mon[111306]: pgmap v722: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:26.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:26 smithi138 ceph-mon[81304]: pgmap v722: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:28.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:28 smithi033 ceph-mon[111306]: pgmap v723: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:28.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:28 smithi138 ceph-mon[81304]: pgmap v723: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:30.599 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:30 smithi138 ceph-mon[81304]: pgmap v724: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:30.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:30 smithi033 ceph-mon[111306]: pgmap v724: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:32.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:32 smithi033 ceph-mon[111306]: pgmap v725: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:32.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:32 smithi138 ceph-mon[81304]: pgmap v725: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:34.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:34 smithi033 ceph-mon[111306]: pgmap v726: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:34.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:34 smithi138 ceph-mon[81304]: pgmap v726: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:36.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:36 smithi033 ceph-mon[111306]: pgmap v727: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:36.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:36 smithi138 ceph-mon[81304]: pgmap v727: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:38.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:38 smithi033 ceph-mon[111306]: pgmap v728: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:38.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:38 smithi138 ceph-mon[81304]: pgmap v728: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:40.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:40 smithi033 ceph-mon[111306]: pgmap v729: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:40.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:03:40.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:40 smithi138 ceph-mon[81304]: pgmap v729: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:40.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:03:41.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:41 smithi033 ceph-mon[111306]: pgmap v730: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:41.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:41 smithi138 ceph-mon[81304]: pgmap v730: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:44.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:44 smithi033 ceph-mon[111306]: pgmap v731: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:44.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:44 smithi138 ceph-mon[81304]: pgmap v731: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:46.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.467+0000 7ff5708b2700 1 -- 172.21.15.33:0/1724947169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c101ce0 msgr2=0x7ff56c1020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:46.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.467+0000 7ff5708b2700 1 --2- 172.21.15.33:0/1724947169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c101ce0 0x7ff56c1020b0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff560004940 tx=0x7ff560009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:46.474 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:46 smithi033 ceph-mon[111306]: pgmap v732: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:46.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 -- 172.21.15.33:0/1724947169 shutdown_connections 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 --2- 172.21.15.33:0/1724947169 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c102680 0x7ff56c0fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 --2- 172.21.15.33:0/1724947169 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c101ce0 0x7ff56c1020b0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 -- 172.21.15.33:0/1724947169 >> 172.21.15.33:0/1724947169 conn(0x7ff56c0f7850 msgr2=0x7ff56c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 -- 172.21.15.33:0/1724947169 shutdown_connections 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 -- 172.21.15.33:0/1724947169 wait complete. 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 Processor -- start 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 -- start start 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c101ce0 0x7ff56c102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c102680 0x7ff56c103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c103630 con 0x7ff56c102680 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.468+0000 7ff5708b2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c1037a0 con 0x7ff56c101ce0 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff569d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c102680 0x7ff56c103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff569d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c102680 0x7ff56c103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41684/0 (socket says 172.21.15.33:41684) 2024-04-25T02:03:46.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff569d9b700 1 -- 172.21.15.33:0/1813630924 learned_addr learned my addr 172.21.15.33:0/1813630924 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff569d9b700 1 -- 172.21.15.33:0/1813630924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c101ce0 msgr2=0x7ff56c102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff56a59c700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c101ce0 0x7ff56c102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff569d9b700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c101ce0 0x7ff56c102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff569d9b700 1 -- 172.21.15.33:0/1813630924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff560005040 con 0x7ff56c102680 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff56a59c700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c101ce0 0x7ff56c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff569d9b700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c102680 0x7ff56c103080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff55c002a50 tx=0x7ff55c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff55affd700 1 -- 172.21.15.33:0/1813630924 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff55c007cc0 con 0x7ff56c102680 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56c119ea0 con 0x7ff56c102680 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff55affd700 1 -- 172.21.15.33:0/1813630924 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff55c00f040 con 0x7ff56c102680 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff55affd700 1 -- 172.21.15.33:0/1813630924 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff55c009630 con 0x7ff56c102680 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.469+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56c11a300 con 0x7ff56c102680 2024-04-25T02:03:46.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.470+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56c0fd430 con 0x7ff56c102680 2024-04-25T02:03:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.471+0000 7ff55affd700 1 -- 172.21.15.33:0/1813630924 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff55c009790 con 0x7ff56c102680 2024-04-25T02:03:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.471+0000 7ff55affd700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff55406e140 0x7ff5540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.471+0000 7ff55affd700 1 -- 172.21.15.33:0/1813630924 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff55c08d7c0 con 0x7ff56c102680 2024-04-25T02:03:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.471+0000 7ff56a59c700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff55406e140 0x7ff5540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:46.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.472+0000 7ff56a59c700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff55406e140 0x7ff5540705f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff560004560 tx=0x7ff560009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:46.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.476+0000 7ff55affd700 1 -- 172.21.15.33:0/1813630924 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff55c056bf0 con 0x7ff56c102680 2024-04-25T02:03:46.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:46 smithi138 ceph-mon[81304]: pgmap v732: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:46.712 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.707+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff56c04bed0 con 0x7ff55406e140 2024-04-25T02:03:46.713 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.708+0000 7ff55affd700 1 -- 172.21.15.33:0/1813630924 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff56c04bed0 con 0x7ff55406e140 2024-04-25T02:03:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff55406e140 msgr2=0x7ff5540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff55406e140 0x7ff5540705f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff560004560 tx=0x7ff560009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c102680 msgr2=0x7ff56c103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c102680 0x7ff56c103080 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff55c002a50 tx=0x7ff55c0046b0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 shutdown_connections 2024-04-25T02:03:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff55406e140 0x7ff5540705f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff56c101ce0 0x7ff56c102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 --2- 172.21.15.33:0/1813630924 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff56c102680 0x7ff56c103080 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.716 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 >> 172.21.15.33:0/1813630924 conn(0x7ff56c0f7850 msgr2=0x7ff56c0f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 shutdown_connections 2024-04-25T02:03:46.717 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.710+0000 7ff5708b2700 1 -- 172.21.15.33:0/1813630924 wait complete. 2024-04-25T02:03:46.726 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:03:46.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.816+0000 7fadc3b57700 1 -- 172.21.15.33:0/3484471880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 msgr2=0x7fadbc0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:46.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.816+0000 7fadc3b57700 1 --2- 172.21.15.33:0/3484471880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 0x7fadbc0fbd20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fadb8004940 tx=0x7fadb8009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:46.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.816+0000 7fadc3b57700 1 -- 172.21.15.33:0/3484471880 shutdown_connections 2024-04-25T02:03:46.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.816+0000 7fadc3b57700 1 --2- 172.21.15.33:0/3484471880 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 0x7fadbc0fbd20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.816+0000 7fadc3b57700 1 --2- 172.21.15.33:0/3484471880 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadbc101d10 0x7fadbc1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.816+0000 7fadc3b57700 1 -- 172.21.15.33:0/3484471880 >> 172.21.15.33:0/3484471880 conn(0x7fadbc0f7860 msgr2=0x7fadbc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.817+0000 7fadc3b57700 1 -- 172.21.15.33:0/3484471880 shutdown_connections 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.817+0000 7fadc3b57700 1 -- 172.21.15.33:0/3484471880 wait complete. 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.817+0000 7fadc3b57700 1 Processor -- start 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.817+0000 7fadc3b57700 1 -- start start 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.817+0000 7fadc3b57700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadbc101d10 0x7fadbc1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.817+0000 7fadc3b57700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 0x7fadbc113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.817+0000 7fadc3b57700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc1145c0 con 0x7fadbc1026b0 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.817+0000 7fadc3b57700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc10db50 con 0x7fadbc101d10 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc10f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 0x7fadbc113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc10f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 0x7fadbc113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41698/0 (socket says 172.21.15.33:41698) 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc10f2700 1 -- 172.21.15.33:0/71513845 learned_addr learned my addr 172.21.15.33:0/71513845 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc18f3700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadbc101d10 0x7fadbc1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc10f2700 1 -- 172.21.15.33:0/71513845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadbc101d10 msgr2=0x7fadbc1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:46.824 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc10f2700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadbc101d10 0x7fadbc1139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:46.825 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc10f2700 1 -- 172.21.15.33:0/71513845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb8005040 con 0x7fadbc1026b0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc10f2700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 0x7fadbc113ee0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009180 tx=0x7fadb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadb27fc700 1 -- 172.21.15.33:0/71513845 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadb800e040 con 0x7fadbc1026b0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc10ddd0 con 0x7fadbc1026b0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadb27fc700 1 -- 172.21.15.33:0/71513845 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fadb800bbc0 con 0x7fadbc1026b0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadb27fc700 1 -- 172.21.15.33:0/71513845 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadb801c3f0 con 0x7fadbc1026b0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.818+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc10e310 con 0x7fadbc1026b0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.819+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc0fd460 con 0x7fadbc1026b0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.820+0000 7fadb27fc700 1 -- 172.21.15.33:0/71513845 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fadb800bd30 con 0x7fadbc1026b0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.820+0000 7fadb27fc700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fada806e140 0x7fada80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.820+0000 7fadb27fc700 1 -- 172.21.15.33:0/71513845 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fadb808ee60 con 0x7fadbc1026b0 2024-04-25T02:03:46.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.820+0000 7fadc18f3700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fada806e140 0x7fada80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:46.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.821+0000 7fadc18f3700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fada806e140 0x7fada80705f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fadac006fd0 tx=0x7fadac008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:46.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:46.825+0000 7fadb27fc700 1 -- 172.21.15.33:0/71513845 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fadb8058260 con 0x7fadbc1026b0 2024-04-25T02:03:47.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.051+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadbc108e60 con 0x7fada806e140 2024-04-25T02:03:47.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.052+0000 7fadb27fc700 1 -- 172.21.15.33:0/71513845 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fadbc108e60 con 0x7fada806e140 2024-04-25T02:03:47.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fada806e140 msgr2=0x7fada80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fada806e140 0x7fada80705f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fadac006fd0 tx=0x7fadac008040 comp rx=0 tx=0).stop 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 msgr2=0x7fadbc113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 0x7fadbc113ee0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009180 tx=0x7fadb8009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 shutdown_connections 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fada806e140 0x7fada80705f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fadbc101d10 0x7fadbc1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 --2- 172.21.15.33:0/71513845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fadbc1026b0 0x7fadbc113ee0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 >> 172.21.15.33:0/71513845 conn(0x7fadbc0f7860 msgr2=0x7fadbc066270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 shutdown_connections 2024-04-25T02:03:47.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.054+0000 7fadc3b57700 1 -- 172.21.15.33:0/71513845 wait complete. 2024-04-25T02:03:47.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.159+0000 7f24b5f82700 1 -- 172.21.15.33:0/1943041193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 msgr2=0x7f24b0103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.159+0000 7f24b5f82700 1 --2- 172.21.15.33:0/1943041193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 0x7f24b0103220 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f24a0004940 tx=0x7f24a0009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:47.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.159+0000 7f24b5f82700 1 -- 172.21.15.33:0/1943041193 shutdown_connections 2024-04-25T02:03:47.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.159+0000 7f24b5f82700 1 --2- 172.21.15.33:0/1943041193 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0065640 0x7f24b0065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.159+0000 7f24b5f82700 1 --2- 172.21.15.33:0/1943041193 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 0x7f24b0103220 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.159+0000 7f24b5f82700 1 -- 172.21.15.33:0/1943041193 >> 172.21.15.33:0/1943041193 conn(0x7f24b0071d00 msgr2=0x7f24b0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:47.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.159+0000 7f24b5f82700 1 -- 172.21.15.33:0/1943041193 shutdown_connections 2024-04-25T02:03:47.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.159+0000 7f24b5f82700 1 -- 172.21.15.33:0/1943041193 wait complete. 2024-04-25T02:03:47.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.159+0000 7f24b5f82700 1 Processor -- start 2024-04-25T02:03:47.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24b5f82700 1 -- start start 2024-04-25T02:03:47.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24b5f82700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0065640 0x7f24b0113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:47.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24b5f82700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 0x7f24b0114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:47.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24b5f82700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b01146a0 con 0x7f24b0102e50 2024-04-25T02:03:47.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24b5f82700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b010da90 con 0x7f24b0065640 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 0x7f24b0114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24aeffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 0x7f24b0114050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41720/0 (socket says 172.21.15.33:41720) 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24aeffd700 1 -- 172.21.15.33:0/1810940434 learned_addr learned my addr 172.21.15.33:0/1810940434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24af7fe700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0065640 0x7f24b0113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24aeffd700 1 -- 172.21.15.33:0/1810940434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0065640 msgr2=0x7f24b0113b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24aeffd700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0065640 0x7f24b0113b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24aeffd700 1 -- 172.21.15.33:0/1810940434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a0005040 con 0x7f24b0102e50 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.160+0000 7f24aeffd700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 0x7f24b0114050 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f24a400eb70 tx=0x7f24a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.161+0000 7f2497fff700 1 -- 172.21.15.33:0/1810940434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24a4007cc0 con 0x7f24b0102e50 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.161+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b010dd10 con 0x7f24b0102e50 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.161+0000 7f2497fff700 1 -- 172.21.15.33:0/1810940434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24a400f040 con 0x7f24b0102e50 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.161+0000 7f2497fff700 1 -- 172.21.15.33:0/1810940434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24a400ca90 con 0x7f24b0102e50 2024-04-25T02:03:47.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.161+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b010e250 con 0x7f24b0102e50 2024-04-25T02:03:47.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.162+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b0100100 con 0x7f24b0102e50 2024-04-25T02:03:47.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.162+0000 7f2497fff700 1 -- 172.21.15.33:0/1810940434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f24a400cbf0 con 0x7f24b0102e50 2024-04-25T02:03:47.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.163+0000 7f2497fff700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f249806e140 0x7f24980705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:47.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.163+0000 7f2497fff700 1 -- 172.21.15.33:0/1810940434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f24a408d660 con 0x7f24b0102e50 2024-04-25T02:03:47.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.163+0000 7f24af7fe700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f249806e140 0x7f24980705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:47.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.163+0000 7f24af7fe700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f249806e140 0x7f24980705f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009b30 tx=0x7f24a0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:47.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.168+0000 7f2497fff700 1 -- 172.21.15.33:0/1810940434 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f24a405a510 con 0x7f24b0102e50 2024-04-25T02:03:47.391 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.386+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f24b01137e0 con 0x7f249806e140 2024-04-25T02:03:47.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.397+0000 7f2497fff700 1 -- 172.21.15.33:0/1810940434 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f24b01137e0 con 0x7f249806e140 2024-04-25T02:03:47.402 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:03:47.402 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (24m) 2m ago 36m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (22m) 2m ago 36m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (22m) 2m ago 35m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (24m) 2m ago 35m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (18m) 2m ago 30m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (18m) 2m ago 30m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (17m) 2m ago 30m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (2m) 2m ago 30m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (26m) 2m ago 37m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (25m) 2m ago 35m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (23m) 2m ago 37m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:03:47.403 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (22m) 2m ago 35m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:03:47.404 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (25m) 2m ago 36m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:03:47.404 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (25m) 2m ago 35m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:03:47.404 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (21m) 2m ago 34m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:03:47.404 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (21m) 2m ago 33m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:03:47.404 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (20m) 2m ago 33m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:03:47.404 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (19m) 2m ago 33m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:03:47.404 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (19m) 2m ago 32m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:03:47.404 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (18m) 2m ago 32m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:03:47.404 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (25m) 2m ago 35m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f249806e140 msgr2=0x7f24980705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f249806e140 0x7f24980705f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f24a0009b30 tx=0x7f24a0009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 msgr2=0x7f24b0114050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 0x7f24b0114050 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f24a400eb70 tx=0x7f24a400b040 comp rx=0 tx=0).stop 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 shutdown_connections 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f249806e140 0x7f24980705f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f24b0065640 0x7f24b0113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 --2- 172.21.15.33:0/1810940434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f24b0102e50 0x7f24b0114050 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 >> 172.21.15.33:0/1810940434 conn(0x7f24b0071d00 msgr2=0x7f24b00fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 shutdown_connections 2024-04-25T02:03:47.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.399+0000 7f24b5f82700 1 -- 172.21.15.33:0/1810940434 wait complete. 2024-04-25T02:03:47.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.502+0000 7f4c33483700 1 -- 172.21.15.33:0/3799623929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c065640 msgr2=0x7f4c2c065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.502+0000 7f4c33483700 1 --2- 172.21.15.33:0/3799623929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c065640 0x7f4c2c065ab0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f4c20004940 tx=0x7f4c20009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:47.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.502+0000 7f4c33483700 1 -- 172.21.15.33:0/3799623929 shutdown_connections 2024-04-25T02:03:47.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.502+0000 7f4c33483700 1 --2- 172.21.15.33:0/3799623929 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c065640 0x7f4c2c065ab0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.502+0000 7f4c33483700 1 --2- 172.21.15.33:0/3799623929 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2c102e50 0x7f4c2c103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.502+0000 7f4c33483700 1 -- 172.21.15.33:0/3799623929 >> 172.21.15.33:0/3799623929 conn(0x7f4c2c071d00 msgr2=0x7f4c2c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:47.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.502+0000 7f4c33483700 1 -- 172.21.15.33:0/3799623929 shutdown_connections 2024-04-25T02:03:47.510 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.502+0000 7f4c33483700 1 -- 172.21.15.33:0/3799623929 wait complete. 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.502+0000 7f4c33483700 1 Processor -- start 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c33483700 1 -- start start 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c33483700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2c065640 0x7f4c2c0fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c33483700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c102e50 0x7f4c2c0fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c33483700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0fc880 con 0x7f4c2c102e50 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c33483700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c0fc9c0 con 0x7f4c2c065640 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c30a1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c102e50 0x7f4c2c0fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c30a1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c102e50 0x7f4c2c0fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41728/0 (socket says 172.21.15.33:41728) 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c30a1e700 1 -- 172.21.15.33:0/1247242206 learned_addr learned my addr 172.21.15.33:0/1247242206 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c3121f700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2c065640 0x7f4c2c0fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c30a1e700 1 -- 172.21.15.33:0/1247242206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2c065640 msgr2=0x7f4c2c0fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c30a1e700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2c065640 0x7f4c2c0fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c30a1e700 1 -- 172.21.15.33:0/1247242206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c20005040 con 0x7f4c2c102e50 2024-04-25T02:03:47.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c3121f700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2c065640 0x7f4c2c0fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:03:47.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.503+0000 7f4c30a1e700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c102e50 0x7f4c2c0fc2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000b3b0 tx=0x7f4c20009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:47.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.504+0000 7f4c29ffb700 1 -- 172.21.15.33:0/1247242206 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2000e040 con 0x7f4c2c102e50 2024-04-25T02:03:47.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.504+0000 7f4c29ffb700 1 -- 172.21.15.33:0/1247242206 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c2000bbc0 con 0x7f4c2c102e50 2024-04-25T02:03:47.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.504+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c0fcc10 con 0x7f4c2c102e50 2024-04-25T02:03:47.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.504+0000 7f4c29ffb700 1 -- 172.21.15.33:0/1247242206 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c2001c3f0 con 0x7f4c2c102e50 2024-04-25T02:03:47.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.504+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c11a2c0 con 0x7f4c2c102e50 2024-04-25T02:03:47.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.505+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c100100 con 0x7f4c2c102e50 2024-04-25T02:03:47.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.505+0000 7f4c29ffb700 1 -- 172.21.15.33:0/1247242206 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4c2000bd30 con 0x7f4c2c102e50 2024-04-25T02:03:47.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.506+0000 7f4c29ffb700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c1406e140 0x7f4c140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:47.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.506+0000 7f4c29ffb700 1 -- 172.21.15.33:0/1247242206 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4c2008eda0 con 0x7f4c2c102e50 2024-04-25T02:03:47.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.506+0000 7f4c3121f700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c1406e140 0x7f4c140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:47.514 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.506+0000 7f4c3121f700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c1406e140 0x7f4c140705f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009fd0 tx=0x7f4c1c009d70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:47.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.511+0000 7f4c29ffb700 1 -- 172.21.15.33:0/1247242206 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4c20058120 con 0x7f4c2c102e50 2024-04-25T02:03:47.815 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.809+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4c2c0fd4a0 con 0x7f4c2c102e50 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.810+0000 7f4c29ffb700 1 -- 172.21.15.33:0/1247242206 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4c2005b7e0 con 0x7f4c2c102e50 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:03:47.816 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:03:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:03:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:03:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:03:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:03:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:03:47.817 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:03:47.817 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c1406e140 msgr2=0x7f4c140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c1406e140 0x7f4c140705f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009fd0 tx=0x7f4c1c009d70 comp rx=0 tx=0).stop 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c102e50 msgr2=0x7f4c2c0fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c102e50 0x7f4c2c0fc2f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f4c2000b3b0 tx=0x7f4c20009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 shutdown_connections 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c1406e140 0x7f4c140705f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c2c065640 0x7f4c2c0fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 --2- 172.21.15.33:0/1247242206 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c2c102e50 0x7f4c2c0fc2f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 >> 172.21.15.33:0/1247242206 conn(0x7f4c2c071d00 msgr2=0x7f4c2c0faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:47.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 shutdown_connections 2024-04-25T02:03:47.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.812+0000 7f4c33483700 1 -- 172.21.15.33:0/1247242206 wait complete. 2024-04-25T02:03:47.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.916+0000 7f170825e700 1 -- 172.21.15.33:0/226024637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700102de0 msgr2=0x7f17001031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.916+0000 7f170825e700 1 --2- 172.21.15.33:0/226024637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700102de0 0x7f17001031b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f16fc004940 tx=0x7f16fc009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:47.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.916+0000 7f170825e700 1 -- 172.21.15.33:0/226024637 shutdown_connections 2024-04-25T02:03:47.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.916+0000 7f170825e700 1 --2- 172.21.15.33:0/226024637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1700103780 0x7f17000fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.916+0000 7f170825e700 1 --2- 172.21.15.33:0/226024637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700102de0 0x7f17001031b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.916+0000 7f170825e700 1 -- 172.21.15.33:0/226024637 >> 172.21.15.33:0/226024637 conn(0x7f17000f8990 msgr2=0x7f17000fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:47.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.916+0000 7f170825e700 1 -- 172.21.15.33:0/226024637 shutdown_connections 2024-04-25T02:03:47.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.916+0000 7f170825e700 1 -- 172.21.15.33:0/226024637 wait complete. 2024-04-25T02:03:47.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f170825e700 1 Processor -- start 2024-04-25T02:03:47.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f170825e700 1 -- start start 2024-04-25T02:03:47.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f170825e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1700102de0 0x7f1700113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:47.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f170825e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700103780 0x7f1700113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:47.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f170825e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1700114500 con 0x7f1700103780 2024-04-25T02:03:47.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f170825e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170010db50 con 0x7f1700102de0 2024-04-25T02:03:47.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f17057f9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700103780 0x7f1700113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:47.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f17057f9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700103780 0x7f1700113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41734/0 (socket says 172.21.15.33:41734) 2024-04-25T02:03:47.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f17057f9700 1 -- 172.21.15.33:0/1903473293 learned_addr learned my addr 172.21.15.33:0/1903473293 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:47.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f1705ffa700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1700102de0 0x7f1700113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:47.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f17057f9700 1 -- 172.21.15.33:0/1903473293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1700102de0 msgr2=0x7f1700113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:47.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f17057f9700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1700102de0 0x7f1700113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:47.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.917+0000 7f17057f9700 1 -- 172.21.15.33:0/1903473293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc005040 con 0x7f1700103780 2024-04-25T02:03:47.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.918+0000 7f17057f9700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700103780 0x7f1700113eb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f16f000c800 tx=0x7f16f0004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:47.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.918+0000 7f16f67fc700 1 -- 172.21.15.33:0/1903473293 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16f00106f0 con 0x7f1700103780 2024-04-25T02:03:47.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.918+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170010de30 con 0x7f1700103780 2024-04-25T02:03:47.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.918+0000 7f16f67fc700 1 -- 172.21.15.33:0/1903473293 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16f0010d20 con 0x7f1700103780 2024-04-25T02:03:47.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.918+0000 7f16f67fc700 1 -- 172.21.15.33:0/1903473293 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16f00113a0 con 0x7f1700103780 2024-04-25T02:03:47.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.918+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170010e3f0 con 0x7f1700103780 2024-04-25T02:03:47.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.919+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17001000f0 con 0x7f1700103780 2024-04-25T02:03:47.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.919+0000 7f16f67fc700 1 -- 172.21.15.33:0/1903473293 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f16f0012400 con 0x7f1700103780 2024-04-25T02:03:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.920+0000 7f16f67fc700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16ec06e140 0x7f16ec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.920+0000 7f1705ffa700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16ec06e140 0x7f16ec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.920+0000 7f16f67fc700 1 -- 172.21.15.33:0/1903473293 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f16f008da60 con 0x7f1700103780 2024-04-25T02:03:47.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.920+0000 7f1705ffa700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16ec06e140 0x7f16ec0705f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f16fc004f90 tx=0x7f16fc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:47.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:47.925+0000 7f16f67fc700 1 -- 172.21.15.33:0/1903473293 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f16f005a990 con 0x7f1700103780 2024-04-25T02:03:48.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.177+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f170010efc0 con 0x7f1700103780 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.178+0000 7f16f67fc700 1 -- 172.21.15.33:0/1903473293 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f16f005a520 con 0x7f1700103780 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:03:48.184 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:03:48.185 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:03:48.186 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:03:48.187 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:03:48.187 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16ec06e140 msgr2=0x7f16ec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16ec06e140 0x7f16ec0705f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f16fc004f90 tx=0x7f16fc009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700103780 msgr2=0x7f1700113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700103780 0x7f1700113eb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f16f000c800 tx=0x7f16f0004470 comp rx=0 tx=0).stop 2024-04-25T02:03:48.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 shutdown_connections 2024-04-25T02:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f16ec06e140 0x7f16ec0705f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1700102de0 0x7f1700113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 --2- 172.21.15.33:0/1903473293 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1700103780 0x7f1700113eb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 >> 172.21.15.33:0/1903473293 conn(0x7f17000f8990 msgr2=0x7f1700107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 shutdown_connections 2024-04-25T02:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.180+0000 7f170825e700 1 -- 172.21.15.33:0/1903473293 wait complete. 2024-04-25T02:03:48.190 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:03:48.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 -- 172.21.15.33:0/4096826099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 msgr2=0x7f5a3c0fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 --2- 172.21.15.33:0/4096826099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 0x7f5a3c0fe1d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5a30004940 tx=0x7f5a30009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:48.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 -- 172.21.15.33:0/4096826099 shutdown_connections 2024-04-25T02:03:48.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 --2- 172.21.15.33:0/4096826099 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a3c0fe7a0 0x7f5a3c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 --2- 172.21.15.33:0/4096826099 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 0x7f5a3c0fe1d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 -- 172.21.15.33:0/4096826099 >> 172.21.15.33:0/4096826099 conn(0x7f5a3c0f7860 msgr2=0x7f5a3c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 -- 172.21.15.33:0/4096826099 shutdown_connections 2024-04-25T02:03:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 -- 172.21.15.33:0/4096826099 wait complete. 2024-04-25T02:03:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 Processor -- start 2024-04-25T02:03:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.285+0000 7f5a42e99700 1 -- start start 2024-04-25T02:03:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a42e99700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 0x7f5a3c0fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a42e99700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a3c0fe7a0 0x7f5a3c0fc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a42e99700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0fdc10 con 0x7f5a3c0fde00 2024-04-25T02:03:48.292 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a42e99700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0fc740 con 0x7f5a3c0fe7a0 2024-04-25T02:03:48.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a40c35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 0x7f5a3c0fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:48.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a40c35700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 0x7f5a3c0fbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41754/0 (socket says 172.21.15.33:41754) 2024-04-25T02:03:48.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a40c35700 1 -- 172.21.15.33:0/526348515 learned_addr learned my addr 172.21.15.33:0/526348515 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:48.293 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a3bfff700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a3c0fe7a0 0x7f5a3c0fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:48.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a40c35700 1 -- 172.21.15.33:0/526348515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a3c0fe7a0 msgr2=0x7f5a3c0fc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a40c35700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a3c0fe7a0 0x7f5a3c0fc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a40c35700 1 -- 172.21.15.33:0/526348515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30005040 con 0x7f5a3c0fde00 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a40c35700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 0x7f5a3c0fbcc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009180 tx=0x7f5a30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a397fa700 1 -- 172.21.15.33:0/526348515 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3000e040 con 0x7f5a3c0fde00 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c0fc9c0 con 0x7f5a3c0fde00 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.286+0000 7f5a397fa700 1 -- 172.21.15.33:0/526348515 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a30004290 con 0x7f5a3c0fde00 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.287+0000 7f5a397fa700 1 -- 172.21.15.33:0/526348515 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3001c3f0 con 0x7f5a3c0fde00 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.287+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c11a0c0 con 0x7f5a3c0fde00 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.288+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c066ad0 con 0x7f5a3c0fde00 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.288+0000 7f5a397fa700 1 -- 172.21.15.33:0/526348515 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5a30013420 con 0x7f5a3c0fde00 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.289+0000 7f5a397fa700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a2c06e140 0x7f5a2c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.289+0000 7f5a397fa700 1 -- 172.21.15.33:0/526348515 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5a3008ed40 con 0x7f5a3c0fde00 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.289+0000 7f5a3bfff700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a2c06e140 0x7f5a2c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:48.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.289+0000 7f5a3bfff700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a2c06e140 0x7f5a2c0705f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0fd3f0 tx=0x7f5a2800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:48.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.293+0000 7f5a397fa700 1 -- 172.21.15.33:0/526348515 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5a30058140 con 0x7f5a3c0fde00 2024-04-25T02:03:48.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:48 smithi033 ceph-mon[111306]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:48.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:48 smithi033 ceph-mon[111306]: from='client.35360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:48.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:48 smithi033 ceph-mon[111306]: pgmap v733: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:48.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:48 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1247242206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:03:48.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:48 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1903473293' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:03:48.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.520+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a3c0f9460 con 0x7f5a2c06e140 2024-04-25T02:03:48.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.521+0000 7f5a397fa700 1 -- 172.21.15.33:0/526348515 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a3c0f9460 con 0x7f5a2c06e140 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:03:48.527 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:03:48.528 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:03:48.528 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a2c06e140 msgr2=0x7f5a2c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a2c06e140 0x7f5a2c0705f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0fd3f0 tx=0x7f5a2800b040 comp rx=0 tx=0).stop 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 msgr2=0x7f5a3c0fbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 0x7f5a3c0fbcc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009180 tx=0x7f5a30009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 shutdown_connections 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5a2c06e140 0x7f5a2c0705f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5a3c0fde00 0x7f5a3c0fbcc0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 --2- 172.21.15.33:0/526348515 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5a3c0fe7a0 0x7f5a3c0fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 >> 172.21.15.33:0/526348515 conn(0x7f5a3c0f7860 msgr2=0x7f5a3c0f8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 shutdown_connections 2024-04-25T02:03:48.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.523+0000 7f5a42e99700 1 -- 172.21.15.33:0/526348515 wait complete. 2024-04-25T02:03:48.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:48 smithi138 ceph-mon[81304]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:48.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:48 smithi138 ceph-mon[81304]: from='client.35360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:48.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:48 smithi138 ceph-mon[81304]: pgmap v733: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:48.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:48 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1247242206' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:03:48.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:48 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1903473293' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:03:48.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.627+0000 7f1f22dea700 1 -- 172.21.15.33:0/3507232884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 msgr2=0x7f1f1c0fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.627+0000 7f1f22dea700 1 --2- 172.21.15.33:0/3507232884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 0x7f1f1c0fb3c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f1f08004940 tx=0x7f1f08009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:48.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.627+0000 7f1f22dea700 1 -- 172.21.15.33:0/3507232884 shutdown_connections 2024-04-25T02:03:48.633 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.627+0000 7f1f22dea700 1 --2- 172.21.15.33:0/3507232884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 0x7f1f1c0fb3c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.627+0000 7f1f22dea700 1 --2- 172.21.15.33:0/3507232884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1c102e20 0x7f1f1c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.627+0000 7f1f22dea700 1 -- 172.21.15.33:0/3507232884 >> 172.21.15.33:0/3507232884 conn(0x7f1f1c072380 msgr2=0x7f1f1c072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:48.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.627+0000 7f1f22dea700 1 -- 172.21.15.33:0/3507232884 shutdown_connections 2024-04-25T02:03:48.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.627+0000 7f1f22dea700 1 -- 172.21.15.33:0/3507232884 wait complete. 2024-04-25T02:03:48.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.628+0000 7f1f22dea700 1 Processor -- start 2024-04-25T02:03:48.634 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.628+0000 7f1f22dea700 1 -- start start 2024-04-25T02:03:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.628+0000 7f1f22dea700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1c102e20 0x7f1f1c115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.628+0000 7f1f22dea700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 0x7f1f1c1160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.628+0000 7f1f22dea700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c116700 con 0x7f1f1c1037c0 2024-04-25T02:03:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.628+0000 7f1f22dea700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1c10fd20 con 0x7f1f1c102e20 2024-04-25T02:03:48.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.628+0000 7f1f1bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 0x7f1f1c1160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.628+0000 7f1f1bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 0x7f1f1c1160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:41778/0 (socket says 172.21.15.33:41778) 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.628+0000 7f1f1bfff700 1 -- 172.21.15.33:0/1804797423 learned_addr learned my addr 172.21.15.33:0/1804797423 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f20b86700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1c102e20 0x7f1f1c115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f1bfff700 1 -- 172.21.15.33:0/1804797423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1c102e20 msgr2=0x7f1f1c115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f1bfff700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1c102e20 0x7f1f1c115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f1bfff700 1 -- 172.21.15.33:0/1804797423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f08005040 con 0x7f1f1c1037c0 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f1bfff700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 0x7f1f1c1160b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b3b0 tx=0x7f1f08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f197fa700 1 -- 172.21.15.33:0/1804797423 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f0800e040 con 0x7f1f1c1037c0 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f197fa700 1 -- 172.21.15.33:0/1804797423 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f08009b10 con 0x7f1f1c1037c0 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1c10ffa0 con 0x7f1f1c1037c0 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f197fa700 1 -- 172.21.15.33:0/1804797423 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f08013420 con 0x7f1f1c1037c0 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.629+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f1c1104e0 con 0x7f1f1c1037c0 2024-04-25T02:03:48.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.630+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1c107300 con 0x7f1f1c1037c0 2024-04-25T02:03:48.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.631+0000 7f1f197fa700 1 -- 172.21.15.33:0/1804797423 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1f0801c7c0 con 0x7f1f1c1037c0 2024-04-25T02:03:48.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.631+0000 7f1f197fa700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1f0c06e140 0x7f1f0c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:03:48.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.631+0000 7f1f197fa700 1 -- 172.21.15.33:0/1804797423 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1f0808e660 con 0x7f1f1c1037c0 2024-04-25T02:03:48.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.631+0000 7f1f20b86700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1f0c06e140 0x7f1f0c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:03:48.637 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.632+0000 7f1f20b86700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1f0c06e140 0x7f1f0c0705f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c065e90 tx=0x7f1f1000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:03:48.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.637+0000 7f1f197fa700 1 -- 172.21.15.33:0/1804797423 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1f08057b10 con 0x7f1f1c1037c0 2024-04-25T02:03:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.927+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1f1c111200 con 0x7f1f1c1037c0 2024-04-25T02:03:48.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.927+0000 7f1f197fa700 1 -- 172.21.15.33:0/1804797423 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f1f0805b1d0 con 0x7f1f1c1037c0 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:03:48.933 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:03:48.934 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:03:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1f0c06e140 msgr2=0x7f1f0c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1f0c06e140 0x7f1f0c0705f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c065e90 tx=0x7f1f1000b040 comp rx=0 tx=0).stop 2024-04-25T02:03:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 msgr2=0x7f1f1c1160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:03:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 0x7f1f1c1160b0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f1f0800b3b0 tx=0x7f1f08009e30 comp rx=0 tx=0).stop 2024-04-25T02:03:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 shutdown_connections 2024-04-25T02:03:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1f0c06e140 0x7f1f0c0705f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1f1c102e20 0x7f1f1c115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 --2- 172.21.15.33:0/1804797423 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1f1c1037c0 0x7f1f1c1160b0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:03:48.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 >> 172.21.15.33:0/1804797423 conn(0x7f1f1c072380 msgr2=0x7f1f1c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:03:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 shutdown_connections 2024-04-25T02:03:48.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:03:48.930+0000 7f1f22dea700 1 -- 172.21.15.33:0/1804797423 wait complete. 2024-04-25T02:03:49.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:49 smithi033 ceph-mon[111306]: from='client.35364 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:49.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:49 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1804797423' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:03:49.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:49 smithi138 ceph-mon[81304]: from='client.35364 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:49.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:49 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1804797423' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:03:50.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:50 smithi033 ceph-mon[111306]: from='client.35376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:50.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:50 smithi033 ceph-mon[111306]: pgmap v734: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:50.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:50 smithi138 ceph-mon[81304]: from='client.35376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:03:50.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:50 smithi138 ceph-mon[81304]: pgmap v734: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:52.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:52 smithi033 ceph-mon[111306]: pgmap v735: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:52.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:52 smithi138 ceph-mon[81304]: pgmap v735: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:54.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:54 smithi033 ceph-mon[111306]: pgmap v736: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:54.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:54 smithi138 ceph-mon[81304]: pgmap v736: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:56.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:56 smithi033 ceph-mon[111306]: pgmap v737: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:56.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:03:56.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:56 smithi138 ceph-mon[81304]: pgmap v737: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:56.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:03:58.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:03:58 smithi033 ceph-mon[111306]: pgmap v738: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:03:58.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:03:58 smithi138 ceph-mon[81304]: pgmap v738: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:00.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:00 smithi033 ceph-mon[111306]: pgmap v739: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:00.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:00 smithi138 ceph-mon[81304]: pgmap v739: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:02.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:02 smithi033 ceph-mon[111306]: pgmap v740: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:02.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:02 smithi138 ceph-mon[81304]: pgmap v740: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:04.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:04 smithi033 ceph-mon[111306]: pgmap v741: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:04.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:04 smithi138 ceph-mon[81304]: pgmap v741: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:06.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:06 smithi033 ceph-mon[111306]: pgmap v742: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:06.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:06 smithi138 ceph-mon[81304]: pgmap v742: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:08.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:08 smithi033 ceph-mon[111306]: pgmap v743: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:08.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:08 smithi138 ceph-mon[81304]: pgmap v743: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:10.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:10 smithi033 ceph-mon[111306]: pgmap v744: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:10.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:10 smithi138 ceph-mon[81304]: pgmap v744: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:11.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:04:11.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:04:12.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:12 smithi033 ceph-mon[111306]: pgmap v745: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:12.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:12 smithi138 ceph-mon[81304]: pgmap v745: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:14.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:14 smithi033 ceph-mon[111306]: pgmap v746: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:14.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:14 smithi138 ceph-mon[81304]: pgmap v746: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:16.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:16 smithi033 ceph-mon[111306]: pgmap v747: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:16.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:16 smithi138 ceph-mon[81304]: pgmap v747: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:18.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:18 smithi033 ceph-mon[111306]: pgmap v748: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:18.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:18 smithi138 ceph-mon[81304]: pgmap v748: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.050+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/506966559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc102df0 msgr2=0x7f7dbc1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.050+0000 7f7dc3aa0700 1 --2- 172.21.15.33:0/506966559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc102df0 0x7f7dbc1031c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7db8004940 tx=0x7f7db8009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:19.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.050+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/506966559 shutdown_connections 2024-04-25T02:04:19.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.050+0000 7f7dc3aa0700 1 --2- 172.21.15.33:0/506966559 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dbc103790 0x7f7dbc0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.050+0000 7f7dc3aa0700 1 --2- 172.21.15.33:0/506966559 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc102df0 0x7f7dbc1031c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.050+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/506966559 >> 172.21.15.33:0/506966559 conn(0x7f7dbc0f8980 msgr2=0x7f7dbc0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:19.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.050+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/506966559 shutdown_connections 2024-04-25T02:04:19.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.050+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/506966559 wait complete. 2024-04-25T02:04:19.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.051+0000 7f7dc3aa0700 1 Processor -- start 2024-04-25T02:04:19.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.051+0000 7f7dc3aa0700 1 -- start start 2024-04-25T02:04:19.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.051+0000 7f7dc3aa0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dbc102df0 0x7f7dbc070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:19.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.051+0000 7f7dc3aa0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc103790 0x7f7dbc0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:19.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.051+0000 7f7dc3aa0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc070d30 con 0x7f7dbc103790 2024-04-25T02:04:19.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.051+0000 7f7dc3aa0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dbc070e70 con 0x7f7dbc102df0 2024-04-25T02:04:19.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.051+0000 7f7dc103b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc103790 0x7f7dbc0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:19.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.051+0000 7f7dc103b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc103790 0x7f7dbc0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38460/0 (socket says 172.21.15.33:38460) 2024-04-25T02:04:19.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7dc103b700 1 -- 172.21.15.33:0/1335027003 learned_addr learned my addr 172.21.15.33:0/1335027003 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7dc183c700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dbc102df0 0x7f7dbc070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7dc103b700 1 -- 172.21.15.33:0/1335027003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dbc102df0 msgr2=0x7f7dbc070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7dc103b700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dbc102df0 0x7f7dbc070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7dc103b700 1 -- 172.21.15.33:0/1335027003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db8005040 con 0x7f7dbc103790 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7dc103b700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc103790 0x7f7dbc0707a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7dac002a50 tx=0x7f7dac004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7db27fc700 1 -- 172.21.15.33:0/1335027003 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dac007cc0 con 0x7f7dbc103790 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7db27fc700 1 -- 172.21.15.33:0/1335027003 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dac00f040 con 0x7f7dbc103790 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7db27fc700 1 -- 172.21.15.33:0/1335027003 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dac00da40 con 0x7f7dbc103790 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dbc119e80 con 0x7f7dbc103790 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.052+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dbc11a300 con 0x7f7dbc103790 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.053+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dbc100100 con 0x7f7dbc103790 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.054+0000 7f7db27fc700 1 -- 172.21.15.33:0/1335027003 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7dac0097c0 con 0x7f7dbc103790 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.054+0000 7f7db27fc700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7da806e140 0x7f7da80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.054+0000 7f7db27fc700 1 -- 172.21.15.33:0/1335027003 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7dac08d880 con 0x7f7dbc103790 2024-04-25T02:04:19.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.054+0000 7f7dc183c700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7da806e140 0x7f7da80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:19.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.055+0000 7f7dc183c700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7da806e140 0x7f7da80705f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7db8004f90 tx=0x7f7db8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:19.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.059+0000 7f7db27fc700 1 -- 172.21.15.33:0/1335027003 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7dac056c80 con 0x7f7dbc103790 2024-04-25T02:04:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.291+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dbc109f20 con 0x7f7da806e140 2024-04-25T02:04:19.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.292+0000 7f7db27fc700 1 -- 172.21.15.33:0/1335027003 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7dbc109f20 con 0x7f7da806e140 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7da806e140 msgr2=0x7f7da80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7da806e140 0x7f7da80705f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7db8004f90 tx=0x7f7db8009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc103790 msgr2=0x7f7dbc0707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc103790 0x7f7dbc0707a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7dac002a50 tx=0x7f7dac004600 comp rx=0 tx=0).stop 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 shutdown_connections 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7da806e140 0x7f7da80705f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7dbc102df0 0x7f7dbc070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 --2- 172.21.15.33:0/1335027003 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7dbc103790 0x7f7dbc0707a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 >> 172.21.15.33:0/1335027003 conn(0x7f7dbc0f8980 msgr2=0x7f7dbc107390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 shutdown_connections 2024-04-25T02:04:19.300 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.294+0000 7f7dc3aa0700 1 -- 172.21.15.33:0/1335027003 wait complete. 2024-04-25T02:04:19.310 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:04:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2021567024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0103780 msgr2=0x7f6eb00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 --2- 172.21.15.33:0/2021567024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0103780 0x7f6eb00fcdf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c004940 tx=0x7f6e9c009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2021567024 shutdown_connections 2024-04-25T02:04:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 --2- 172.21.15.33:0/2021567024 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0103780 0x7f6eb00fcdf0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 --2- 172.21.15.33:0/2021567024 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb0102de0 0x7f6eb01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2021567024 >> 172.21.15.33:0/2021567024 conn(0x7f6eb00f8990 msgr2=0x7f6eb00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2021567024 shutdown_connections 2024-04-25T02:04:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2021567024 wait complete. 2024-04-25T02:04:19.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 Processor -- start 2024-04-25T02:04:19.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.399+0000 7f6eb6bd4700 1 -- start start 2024-04-25T02:04:19.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb6bd4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0102de0 0x7f6eb0070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:19.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb6bd4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb0103780 0x7f6eb00707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:19.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb6bd4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb00721b0 con 0x7f6eb0102de0 2024-04-25T02:04:19.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb6bd4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb0070ce0 con 0x7f6eb0103780 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb4970700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0102de0 0x7f6eb0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb4970700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0102de0 0x7f6eb0070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38482/0 (socket says 172.21.15.33:38482) 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb4970700 1 -- 172.21.15.33:0/2141393017 learned_addr learned my addr 172.21.15.33:0/2141393017 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eaffff700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb0103780 0x7f6eb00707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb4970700 1 -- 172.21.15.33:0/2141393017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb0103780 msgr2=0x7f6eb00707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb4970700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb0103780 0x7f6eb00707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb4970700 1 -- 172.21.15.33:0/2141393017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c005040 con 0x7f6eb0102de0 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eaffff700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb0103780 0x7f6eb00707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb4970700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0102de0 0x7f6eb0070260 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4002a50 tx=0x7f6ea40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6ead7fa700 1 -- 172.21.15.33:0/2141393017 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ea4007cc0 con 0x7f6eb0102de0 2024-04-25T02:04:19.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6ead7fa700 1 -- 172.21.15.33:0/2141393017 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ea400f040 con 0x7f6eb0102de0 2024-04-25T02:04:19.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.400+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb0070fc0 con 0x7f6eb0102de0 2024-04-25T02:04:19.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.401+0000 7f6ead7fa700 1 -- 172.21.15.33:0/2141393017 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ea4009640 con 0x7f6eb0102de0 2024-04-25T02:04:19.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.401+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb011a100 con 0x7f6eb0102de0 2024-04-25T02:04:19.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.401+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb00fe530 con 0x7f6eb0102de0 2024-04-25T02:04:19.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.402+0000 7f6ead7fa700 1 -- 172.21.15.33:0/2141393017 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6ea40097a0 con 0x7f6eb0102de0 2024-04-25T02:04:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.402+0000 7f6ead7fa700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6ea006e140 0x7f6ea00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.403+0000 7f6ead7fa700 1 -- 172.21.15.33:0/2141393017 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6ea408d920 con 0x7f6eb0102de0 2024-04-25T02:04:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.403+0000 7f6eaffff700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6ea006e140 0x7f6ea00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:19.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.403+0000 7f6eaffff700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6ea006e140 0x7f6ea00705f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00b3b0 tx=0x7f6e9c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:19.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.407+0000 7f6ead7fa700 1 -- 172.21.15.33:0/2141393017 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6ea4056ca0 con 0x7f6eb0102de0 2024-04-25T02:04:19.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.632+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6eb00fa0a0 con 0x7f6ea006e140 2024-04-25T02:04:19.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.633+0000 7f6ead7fa700 1 -- 172.21.15.33:0/2141393017 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6eb00fa0a0 con 0x7f6ea006e140 2024-04-25T02:04:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6ea006e140 msgr2=0x7f6ea00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6ea006e140 0x7f6ea00705f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c00b3b0 tx=0x7f6e9c009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:19.641 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0102de0 msgr2=0x7f6eb0070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0102de0 0x7f6eb0070260 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4002a50 tx=0x7f6ea40046b0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 shutdown_connections 2024-04-25T02:04:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6ea006e140 0x7f6ea00705f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6eb0102de0 0x7f6eb0070260 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 --2- 172.21.15.33:0/2141393017 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6eb0103780 0x7f6eb00707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 >> 172.21.15.33:0/2141393017 conn(0x7f6eb00f8990 msgr2=0x7f6eb00f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 shutdown_connections 2024-04-25T02:04:19.642 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.636+0000 7f6eb6bd4700 1 -- 172.21.15.33:0/2141393017 wait complete. 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.739+0000 7ff444a71700 1 -- 172.21.15.33:0/2445720779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 msgr2=0x7ff4401031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.739+0000 7ff444a71700 1 --2- 172.21.15.33:0/2445720779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 0x7ff4401031b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff430004940 tx=0x7ff430009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 -- 172.21.15.33:0/2445720779 shutdown_connections 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 --2- 172.21.15.33:0/2445720779 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff440103780 0x7ff4400fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 --2- 172.21.15.33:0/2445720779 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 0x7ff4401031b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 -- 172.21.15.33:0/2445720779 >> 172.21.15.33:0/2445720779 conn(0x7ff4400f8990 msgr2=0x7ff4400fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 -- 172.21.15.33:0/2445720779 shutdown_connections 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 -- 172.21.15.33:0/2445720779 wait complete. 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 Processor -- start 2024-04-25T02:04:19.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 -- start start 2024-04-25T02:04:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 0x7ff440070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff440103780 0x7ff4400707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff440072160 con 0x7ff440102de0 2024-04-25T02:04:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.740+0000 7ff444a71700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff440070ce0 con 0x7ff440103780 2024-04-25T02:04:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff43e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 0x7ff440070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff43e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 0x7ff440070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38514/0 (socket says 172.21.15.33:38514) 2024-04-25T02:04:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff43e59c700 1 -- 172.21.15.33:0/3515233484 learned_addr learned my addr 172.21.15.33:0/3515233484 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff43dd9b700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff440103780 0x7ff4400707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:19.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff43e59c700 1 -- 172.21.15.33:0/3515233484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff440103780 msgr2=0x7ff4400707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff43e59c700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff440103780 0x7ff4400707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff43e59c700 1 -- 172.21.15.33:0/3515233484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff430005040 con 0x7ff440102de0 2024-04-25T02:04:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff43e59c700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 0x7ff440070260 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff430009150 tx=0x7ff430009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff42effd700 1 -- 172.21.15.33:0/3515233484 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff43000e040 con 0x7ff440102de0 2024-04-25T02:04:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff42effd700 1 -- 172.21.15.33:0/3515233484 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff430013420 con 0x7ff440102de0 2024-04-25T02:04:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff440070f60 con 0x7ff440102de0 2024-04-25T02:04:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff42effd700 1 -- 172.21.15.33:0/3515233484 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff43001c610 con 0x7ff440102de0 2024-04-25T02:04:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.741+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff44011a110 con 0x7ff440102de0 2024-04-25T02:04:19.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.742+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4401000f0 con 0x7ff440102de0 2024-04-25T02:04:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.743+0000 7ff42effd700 1 -- 172.21.15.33:0/3515233484 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff43000baf0 con 0x7ff440102de0 2024-04-25T02:04:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.743+0000 7ff42effd700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff42806e140 0x7ff4280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.743+0000 7ff42effd700 1 -- 172.21.15.33:0/3515233484 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff43008ebd0 con 0x7ff440102de0 2024-04-25T02:04:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.743+0000 7ff43dd9b700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff42806e140 0x7ff4280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:19.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.744+0000 7ff43dd9b700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff42806e140 0x7ff4280705f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff440071940 tx=0x7ff43400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:19.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.748+0000 7ff42effd700 1 -- 172.21.15.33:0/3515233484 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff430058070 con 0x7ff440102de0 2024-04-25T02:04:19.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.964+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff440071790 con 0x7ff42806e140 2024-04-25T02:04:19.980 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.974+0000 7ff42effd700 1 -- 172.21.15.33:0/3515233484 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff440071790 con 0x7ff42806e140 2024-04-25T02:04:19.980 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:04:19.980 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (25m) 3m ago 36m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:04:19.980 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (22m) 3m ago 36m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (22m) 3m ago 35m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (25m) 3m ago 36m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (18m) 3m ago 31m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (19m) 3m ago 31m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (18m) 3m ago 31m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (3m) 3m ago 31m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (26m) 3m ago 37m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (26m) 3m ago 35m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (23m) 3m ago 38m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (23m) 3m ago 35m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (26m) 3m ago 36m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (26m) 3m ago 35m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:04:19.981 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (22m) 3m ago 34m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:04:19.982 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (21m) 3m ago 34m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:04:19.982 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (21m) 3m ago 34m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:04:19.982 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (20m) 3m ago 33m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:04:19.982 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (19m) 3m ago 33m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:04:19.982 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (19m) 3m ago 32m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:04:19.982 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (25m) 3m ago 36m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.976+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff42806e140 msgr2=0x7ff4280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.976+0000 7ff444a71700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff42806e140 0x7ff4280705f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff440071940 tx=0x7ff43400b040 comp rx=0 tx=0).stop 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.976+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 msgr2=0x7ff440070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.976+0000 7ff444a71700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 0x7ff440070260 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7ff430009150 tx=0x7ff430009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.976+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 shutdown_connections 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.976+0000 7ff444a71700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff42806e140 0x7ff4280705f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.976+0000 7ff444a71700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff440102de0 0x7ff440070260 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.976+0000 7ff444a71700 1 --2- 172.21.15.33:0/3515233484 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff440103780 0x7ff4400707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.976+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 >> 172.21.15.33:0/3515233484 conn(0x7ff4400f8990 msgr2=0x7ff440107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.977+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 shutdown_connections 2024-04-25T02:04:19.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:19.977+0000 7ff444a71700 1 -- 172.21.15.33:0/3515233484 wait complete. 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.080+0000 7fbb2bc85700 1 -- 172.21.15.33:0/695152888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070c10 msgr2=0x7fbb241090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.080+0000 7fbb2bc85700 1 --2- 172.21.15.33:0/695152888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070c10 0x7fbb241090f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbb14004970 tx=0x7fbb14009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.080+0000 7fbb2bc85700 1 -- 172.21.15.33:0/695152888 shutdown_connections 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.080+0000 7fbb2bc85700 1 --2- 172.21.15.33:0/695152888 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070c10 0x7fbb241090f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.080+0000 7fbb2bc85700 1 --2- 172.21.15.33:0/695152888 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb24070300 0x7fbb240706d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.080+0000 7fbb2bc85700 1 -- 172.21.15.33:0/695152888 >> 172.21.15.33:0/695152888 conn(0x7fbb240f8990 msgr2=0x7fbb240fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.080+0000 7fbb2bc85700 1 -- 172.21.15.33:0/695152888 shutdown_connections 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.080+0000 7fbb2bc85700 1 -- 172.21.15.33:0/695152888 wait complete. 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb2bc85700 1 Processor -- start 2024-04-25T02:04:20.087 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb2bc85700 1 -- start start 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb2bc85700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070300 0x7fbb24115b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb2bc85700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb24070c10 0x7fbb241160a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb2bc85700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb241166f0 con 0x7fbb24070300 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb2bc85700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb2410fcd0 con 0x7fbb24070c10 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb29a21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070300 0x7fbb24115b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb29a21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070300 0x7fbb24115b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38540/0 (socket says 172.21.15.33:38540) 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb29a21700 1 -- 172.21.15.33:0/860791339 learned_addr learned my addr 172.21.15.33:0/860791339 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb29220700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb24070c10 0x7fbb241160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb29a21700 1 -- 172.21.15.33:0/860791339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb24070c10 msgr2=0x7fbb241160a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb29a21700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb24070c10 0x7fbb241160a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.088 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.081+0000 7fbb29a21700 1 -- 172.21.15.33:0/860791339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14005040 con 0x7fbb24070300 2024-04-25T02:04:20.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.082+0000 7fbb29a21700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070300 0x7fbb24115b60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbb20002a50 tx=0x7fbb200046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.082+0000 7fbb1a7fc700 1 -- 172.21.15.33:0/860791339 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb20007cc0 con 0x7fbb24070300 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.082+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2410ffb0 con 0x7fbb24070300 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.082+0000 7fbb1a7fc700 1 -- 172.21.15.33:0/860791339 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb2000f040 con 0x7fbb24070300 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.082+0000 7fbb1a7fc700 1 -- 172.21.15.33:0/860791339 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb2000da30 con 0x7fbb24070300 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.082+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb241104c0 con 0x7fbb24070300 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.083+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb24071a40 con 0x7fbb24070300 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.083+0000 7fbb1a7fc700 1 -- 172.21.15.33:0/860791339 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbb2000db90 con 0x7fbb24070300 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.084+0000 7fbb1a7fc700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbb1006e140 0x7fbb100705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.084+0000 7fbb1a7fc700 1 -- 172.21.15.33:0/860791339 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbb2008d8c0 con 0x7fbb24070300 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.084+0000 7fbb29220700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbb1006e140 0x7fbb100705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:20.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.084+0000 7fbb29220700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbb1006e140 0x7fbb100705f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbb14004f50 tx=0x7fbb14009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:20.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.088+0000 7fbb1a7fc700 1 -- 172.21.15.33:0/860791339 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbb2005a770 con 0x7fbb24070300 2024-04-25T02:04:20.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.387+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbb241111b0 con 0x7fbb24070300 2024-04-25T02:04:20.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.387+0000 7fbb1a7fc700 1 -- 172.21.15.33:0/860791339 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbb2005a300 con 0x7fbb24070300 2024-04-25T02:04:20.394 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:04:20.394 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:04:20.394 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:04:20.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:04:20.394 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:04:20.394 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:04:20.394 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:04:20.395 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:04:20.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbb1006e140 msgr2=0x7fbb100705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbb1006e140 0x7fbb100705f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbb14004f50 tx=0x7fbb14009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:20.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070300 msgr2=0x7fbb24115b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070300 0x7fbb24115b60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbb20002a50 tx=0x7fbb200046b0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 shutdown_connections 2024-04-25T02:04:20.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbb1006e140 0x7fbb100705f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbb24070300 0x7fbb24115b60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 --2- 172.21.15.33:0/860791339 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbb24070c10 0x7fbb241160a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 >> 172.21.15.33:0/860791339 conn(0x7fbb240f8990 msgr2=0x7fbb240f9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:20.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 shutdown_connections 2024-04-25T02:04:20.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.389+0000 7fbb2bc85700 1 -- 172.21.15.33:0/860791339 wait complete. 2024-04-25T02:04:20.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.493+0000 7fc475074700 1 -- 172.21.15.33:0/3417174174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 msgr2=0x7fc4700fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.493+0000 7fc475074700 1 --2- 172.21.15.33:0/3417174174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 0x7fc4700fce00 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fc464004940 tx=0x7fc464009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:20.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.493+0000 7fc475074700 1 -- 172.21.15.33:0/3417174174 shutdown_connections 2024-04-25T02:04:20.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.493+0000 7fc475074700 1 --2- 172.21.15.33:0/3417174174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 0x7fc4700fce00 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.493+0000 7fc475074700 1 --2- 172.21.15.33:0/3417174174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470102df0 0x7fc4701031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.493+0000 7fc475074700 1 -- 172.21.15.33:0/3417174174 >> 172.21.15.33:0/3417174174 conn(0x7fc4700f8980 msgr2=0x7fc4700fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.493+0000 7fc475074700 1 -- 172.21.15.33:0/3417174174 shutdown_connections 2024-04-25T02:04:20.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.493+0000 7fc475074700 1 -- 172.21.15.33:0/3417174174 wait complete. 2024-04-25T02:04:20.500 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:20 smithi033 ceph-mon[111306]: pgmap v749: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:20.500 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:20 smithi033 ceph-mon[111306]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.494+0000 7fc475074700 1 Processor -- start 2024-04-25T02:04:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.494+0000 7fc475074700 1 -- start start 2024-04-25T02:04:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.494+0000 7fc475074700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470102df0 0x7fc470113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.494+0000 7fc475074700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 0x7fc4701140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.494+0000 7fc475074700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc470114700 con 0x7fc470103790 2024-04-25T02:04:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.494+0000 7fc475074700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc47010daf0 con 0x7fc470102df0 2024-04-25T02:04:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.494+0000 7fc46e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 0x7fc4701140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:20.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.494+0000 7fc46e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 0x7fc4701140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38572/0 (socket says 172.21.15.33:38572) 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.494+0000 7fc46e59c700 1 -- 172.21.15.33:0/971906617 learned_addr learned my addr 172.21.15.33:0/971906617 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc46ed9d700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470102df0 0x7fc470113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc46e59c700 1 -- 172.21.15.33:0/971906617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470102df0 msgr2=0x7fc470113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc46e59c700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470102df0 0x7fc470113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc46e59c700 1 -- 172.21.15.33:0/971906617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc464005040 con 0x7fc470103790 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc46e59c700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 0x7fc4701140b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc464009180 tx=0x7fc464009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc4577fe700 1 -- 172.21.15.33:0/971906617 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46400e040 con 0x7fc470103790 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc4577fe700 1 -- 172.21.15.33:0/971906617 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc46400bbc0 con 0x7fc470103790 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc47010dd70 con 0x7fc470103790 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc4577fe700 1 -- 172.21.15.33:0/971906617 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc46401c3f0 con 0x7fc470103790 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.495+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47010e330 con 0x7fc470103790 2024-04-25T02:04:20.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.496+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4700fe540 con 0x7fc470103790 2024-04-25T02:04:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.497+0000 7fc4577fe700 1 -- 172.21.15.33:0/971906617 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc46400bd30 con 0x7fc470103790 2024-04-25T02:04:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.497+0000 7fc4577fe700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc45806e140 0x7fc4580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.497+0000 7fc4577fe700 1 -- 172.21.15.33:0/971906617 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc46408ee60 con 0x7fc470103790 2024-04-25T02:04:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.497+0000 7fc46ed9d700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc45806e140 0x7fc4580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:20.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.498+0000 7fc46ed9d700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc45806e140 0x7fc4580705f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc460007fd0 tx=0x7fc460009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:20.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.502+0000 7fc4577fe700 1 -- 172.21.15.33:0/971906617 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc46405bd90 con 0x7fc470103790 2024-04-25T02:04:20.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:20 smithi138 ceph-mon[81304]: pgmap v749: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:20.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:20 smithi138 ceph-mon[81304]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.757+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc470109ed0 con 0x7fc470103790 2024-04-25T02:04:20.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.758+0000 7fc4577fe700 1 -- 172.21.15.33:0/971906617 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fc46405b920 con 0x7fc470103790 2024-04-25T02:04:20.764 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:04:20.764 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:04:20.764 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:04:20.764 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:04:20.765 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:04:20.766 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:04:20.767 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:04:20.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc45806e140 msgr2=0x7fc4580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc45806e140 0x7fc4580705f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc460007fd0 tx=0x7fc460009040 comp rx=0 tx=0).stop 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 msgr2=0x7fc4701140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 0x7fc4701140b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fc464009180 tx=0x7fc464009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 shutdown_connections 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc45806e140 0x7fc4580705f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc470102df0 0x7fc470113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 --2- 172.21.15.33:0/971906617 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc470103790 0x7fc4701140b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 >> 172.21.15.33:0/971906617 conn(0x7fc4700f8980 msgr2=0x7fc4700f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 shutdown_connections 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.761+0000 7fc475074700 1 -- 172.21.15.33:0/971906617 wait complete. 2024-04-25T02:04:20.769 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:04:20.876 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.866+0000 7ff95d4c8700 1 -- 172.21.15.33:0/2599316075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 msgr2=0x7ff9580fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.866+0000 7ff95d4c8700 1 --2- 172.21.15.33:0/2599316075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 0x7ff9580fcdf0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7ff940004940 tx=0x7ff940009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.866+0000 7ff95d4c8700 1 -- 172.21.15.33:0/2599316075 shutdown_connections 2024-04-25T02:04:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.866+0000 7ff95d4c8700 1 --2- 172.21.15.33:0/2599316075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 0x7ff9580fcdf0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.866+0000 7ff95d4c8700 1 --2- 172.21.15.33:0/2599316075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff958102de0 0x7ff9581031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.866+0000 7ff95d4c8700 1 -- 172.21.15.33:0/2599316075 >> 172.21.15.33:0/2599316075 conn(0x7ff9580f8990 msgr2=0x7ff9580fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.866+0000 7ff95d4c8700 1 -- 172.21.15.33:0/2599316075 shutdown_connections 2024-04-25T02:04:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.866+0000 7ff95d4c8700 1 -- 172.21.15.33:0/2599316075 wait complete. 2024-04-25T02:04:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff95d4c8700 1 Processor -- start 2024-04-25T02:04:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff95d4c8700 1 -- start start 2024-04-25T02:04:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff95d4c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff958102de0 0x7ff958113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:20.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff95d4c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 0x7ff958113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:20.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff95d4c8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9581144e0 con 0x7ff958103780 2024-04-25T02:04:20.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff95d4c8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95810dae0 con 0x7ff958102de0 2024-04-25T02:04:20.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff9567fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 0x7ff958113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:20.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff9567fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 0x7ff958113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38582/0 (socket says 172.21.15.33:38582) 2024-04-25T02:04:20.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff9567fc700 1 -- 172.21.15.33:0/102793385 learned_addr learned my addr 172.21.15.33:0/102793385 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:20.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff956ffd700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff958102de0 0x7ff958113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:20.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff9567fc700 1 -- 172.21.15.33:0/102793385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff958102de0 msgr2=0x7ff958113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:20.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff9567fc700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff958102de0 0x7ff958113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:20.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.867+0000 7ff9567fc700 1 -- 172.21.15.33:0/102793385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940005040 con 0x7ff958103780 2024-04-25T02:04:20.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.868+0000 7ff9567fc700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 0x7ff958113e90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff940009180 tx=0x7ff940009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:20.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.868+0000 7ff94f7fe700 1 -- 172.21.15.33:0/102793385 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94000e040 con 0x7ff958103780 2024-04-25T02:04:20.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.868+0000 7ff94f7fe700 1 -- 172.21.15.33:0/102793385 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff940013420 con 0x7ff958103780 2024-04-25T02:04:20.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.868+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95810dd60 con 0x7ff958103780 2024-04-25T02:04:20.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.868+0000 7ff94f7fe700 1 -- 172.21.15.33:0/102793385 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff94001c610 con 0x7ff958103780 2024-04-25T02:04:20.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.868+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95810e320 con 0x7ff958103780 2024-04-25T02:04:20.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.869+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9580fe530 con 0x7ff958103780 2024-04-25T02:04:20.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.869+0000 7ff94f7fe700 1 -- 172.21.15.33:0/102793385 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff940013a00 con 0x7ff958103780 2024-04-25T02:04:20.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.870+0000 7ff94f7fe700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff94406e140 0x7ff9440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:20.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.870+0000 7ff94f7fe700 1 -- 172.21.15.33:0/102793385 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff94008ebd0 con 0x7ff958103780 2024-04-25T02:04:20.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.870+0000 7ff956ffd700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff94406e140 0x7ff9440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:20.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.870+0000 7ff956ffd700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff94406e140 0x7ff9440705f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff950004fd0 tx=0x7ff950009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:20.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:20.875+0000 7ff94f7fe700 1 -- 172.21.15.33:0/102793385 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff94005bae0 con 0x7ff958103780 2024-04-25T02:04:21.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.104+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff958109ec0 con 0x7ff94406e140 2024-04-25T02:04:21.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.105+0000 7ff94f7fe700 1 -- 172.21.15.33:0/102793385 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff958109ec0 con 0x7ff94406e140 2024-04-25T02:04:21.111 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:04:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:04:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:04:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:04:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:04:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:04:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:04:21.111 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:04:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:04:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:04:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:04:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:04:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:04:21.112 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:04:21.112 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff94406e140 msgr2=0x7ff9440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff94406e140 0x7ff9440705f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff950004fd0 tx=0x7ff950009040 comp rx=0 tx=0).stop 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 msgr2=0x7ff958113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 0x7ff958113e90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7ff940009180 tx=0x7ff940009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 shutdown_connections 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff94406e140 0x7ff9440705f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff958102de0 0x7ff958113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 --2- 172.21.15.33:0/102793385 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff958103780 0x7ff958113e90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 >> 172.21.15.33:0/102793385 conn(0x7ff9580f8990 msgr2=0x7ff9580f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 shutdown_connections 2024-04-25T02:04:21.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.107+0000 7ff95d4c8700 1 -- 172.21.15.33:0/102793385 wait complete. 2024-04-25T02:04:21.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.212+0000 7f496f033700 1 -- 172.21.15.33:0/3290655331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 msgr2=0x7f49681031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:21.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.212+0000 7f496f033700 1 --2- 172.21.15.33:0/3290655331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 0x7f49681031c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f495c004940 tx=0x7f495c009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:21.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.212+0000 7f496f033700 1 -- 172.21.15.33:0/3290655331 shutdown_connections 2024-04-25T02:04:21.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.212+0000 7f496f033700 1 --2- 172.21.15.33:0/3290655331 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4968103790 0x7f49680fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.212+0000 7f496f033700 1 --2- 172.21.15.33:0/3290655331 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 0x7f49681031c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.212+0000 7f496f033700 1 -- 172.21.15.33:0/3290655331 >> 172.21.15.33:0/3290655331 conn(0x7f49680f8980 msgr2=0x7f49680fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.212+0000 7f496f033700 1 -- 172.21.15.33:0/3290655331 shutdown_connections 2024-04-25T02:04:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.212+0000 7f496f033700 1 -- 172.21.15.33:0/3290655331 wait complete. 2024-04-25T02:04:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f496f033700 1 Processor -- start 2024-04-25T02:04:21.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f496f033700 1 -- start start 2024-04-25T02:04:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f496f033700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 0x7f4968070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f496f033700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4968103790 0x7f49680707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:21.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f496f033700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4968070d30 con 0x7f4968102df0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f496f033700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4968070e70 con 0x7f4968103790 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f496cdcf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 0x7f4968070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f496cdcf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 0x7f4968070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:38604/0 (socket says 172.21.15.33:38604) 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f496cdcf700 1 -- 172.21.15.33:0/524326041 learned_addr learned my addr 172.21.15.33:0/524326041 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.213+0000 7f4967fff700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4968103790 0x7f49680707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.214+0000 7f496cdcf700 1 -- 172.21.15.33:0/524326041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4968103790 msgr2=0x7f49680707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.214+0000 7f496cdcf700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4968103790 0x7f49680707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.214+0000 7f496cdcf700 1 -- 172.21.15.33:0/524326041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c005040 con 0x7f4968102df0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.214+0000 7f496cdcf700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 0x7f4968070260 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f495c00b3b0 tx=0x7f495c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.214+0000 7f49657fa700 1 -- 172.21.15.33:0/524326041 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495c00e040 con 0x7f4968102df0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.214+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49680710c0 con 0x7f4968102df0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.214+0000 7f49657fa700 1 -- 172.21.15.33:0/524326041 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f495c013420 con 0x7f4968102df0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.214+0000 7f49657fa700 1 -- 172.21.15.33:0/524326041 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495c01c630 con 0x7f4968102df0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.214+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496811a310 con 0x7f4968102df0 2024-04-25T02:04:21.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.215+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4968100100 con 0x7f4968102df0 2024-04-25T02:04:21.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.215+0000 7f49657fa700 1 -- 172.21.15.33:0/524326041 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f495c013590 con 0x7f4968102df0 2024-04-25T02:04:21.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.216+0000 7f49657fa700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f495806e140 0x7f49580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:21.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.216+0000 7f49657fa700 1 -- 172.21.15.33:0/524326041 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f495c08ecf0 con 0x7f4968102df0 2024-04-25T02:04:21.222 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.216+0000 7f4967fff700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f495806e140 0x7f49580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:21.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.216+0000 7f4967fff700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f495806e140 0x7f49580705f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4968071ba0 tx=0x7f49540043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:21.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.221+0000 7f49657fa700 1 -- 172.21.15.33:0/524326041 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f495c0580f0 con 0x7f4968102df0 2024-04-25T02:04:21.520 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.514+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4968100100 con 0x7f4968102df0 2024-04-25T02:04:21.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:21 smithi033 ceph-mon[111306]: from='client.35388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:21.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:21 smithi033 ceph-mon[111306]: from='client.35392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:21.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:21 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/860791339' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:04:21.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:21 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/971906617' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:04:21.521 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.515+0000 7f49657fa700 1 -- 172.21.15.33:0/524326041 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f495c05b7b0 con 0x7f4968102df0 2024-04-25T02:04:21.521 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:04:21.521 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:04:21.521 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:04:21.521 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:04:21.521 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:04:21.521 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:04:21.522 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:04:21.523 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:04:21.523 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:04:21.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.517+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f495806e140 msgr2=0x7f49580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:21.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.517+0000 7f496f033700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f495806e140 0x7f49580705f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4968071ba0 tx=0x7f49540043c0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.523 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.517+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 msgr2=0x7f4968070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.517+0000 7f496f033700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 0x7f4968070260 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f495c00b3b0 tx=0x7f495c009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.517+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 shutdown_connections 2024-04-25T02:04:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.517+0000 7f496f033700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f495806e140 0x7f49580705f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.517+0000 7f496f033700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4968102df0 0x7f4968070260 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.517+0000 7f496f033700 1 --2- 172.21.15.33:0/524326041 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4968103790 0x7f49680707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.517+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 >> 172.21.15.33:0/524326041 conn(0x7f49680f8980 msgr2=0x7f49681073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.518+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 shutdown_connections 2024-04-25T02:04:21.524 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:21.518+0000 7f496f033700 1 -- 172.21.15.33:0/524326041 wait complete. 2024-04-25T02:04:21.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:21 smithi138 ceph-mon[81304]: from='client.35388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:21.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:21 smithi138 ceph-mon[81304]: from='client.35392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:21.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:21 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/860791339' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:04:21.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:21 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/971906617' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:04:22.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:22 smithi033 ceph-mon[111306]: from='client.35404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:22.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:22 smithi033 ceph-mon[111306]: pgmap v750: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:22.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:22 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/524326041' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:04:22.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:22 smithi138 ceph-mon[81304]: from='client.35404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:22.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:22 smithi138 ceph-mon[81304]: pgmap v750: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:22.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:22 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/524326041' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:04:24.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:24 smithi033 ceph-mon[111306]: pgmap v751: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:24.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:24 smithi138 ceph-mon[81304]: pgmap v751: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:26.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:26 smithi033 ceph-mon[111306]: pgmap v752: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:04:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:26 smithi138 ceph-mon[81304]: pgmap v752: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:04:28.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:28 smithi033 ceph-mon[111306]: pgmap v753: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:28.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:28 smithi138 ceph-mon[81304]: pgmap v753: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:30.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:30 smithi033 ceph-mon[111306]: pgmap v754: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:30.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:30 smithi138 ceph-mon[81304]: pgmap v754: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:32.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:32 smithi033 ceph-mon[111306]: pgmap v755: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:32.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:32 smithi138 ceph-mon[81304]: pgmap v755: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:34.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:34 smithi138 ceph-mon[81304]: pgmap v756: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:34.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:34 smithi033 ceph-mon[111306]: pgmap v756: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:36.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:36 smithi138 ceph-mon[81304]: pgmap v757: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:36.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:36 smithi033 ceph-mon[111306]: pgmap v757: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:38.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:38 smithi138 ceph-mon[81304]: pgmap v758: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:38.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:38 smithi033 ceph-mon[111306]: pgmap v758: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:40.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:40 smithi138 ceph-mon[81304]: pgmap v759: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:40.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:40 smithi033 ceph-mon[111306]: pgmap v759: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:41.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:04:41.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:04:42.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:42 smithi138 ceph-mon[81304]: pgmap v760: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:42.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:42 smithi033 ceph-mon[111306]: pgmap v760: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:44.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:44 smithi138 ceph-mon[81304]: pgmap v761: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:44.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:44 smithi033 ceph-mon[111306]: pgmap v761: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:46.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:46 smithi138 ceph-mon[81304]: pgmap v762: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:46.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:46 smithi033 ceph-mon[111306]: pgmap v762: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:48.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:48 smithi033 ceph-mon[111306]: pgmap v763: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:48.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:48 smithi138 ceph-mon[81304]: pgmap v763: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:50.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:50 smithi033 ceph-mon[111306]: pgmap v764: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:50.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:50 smithi138 ceph-mon[81304]: pgmap v764: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:51.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.637+0000 7fd82f84b700 1 -- 172.21.15.33:0/4055172934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 msgr2=0x7fd828070cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:51.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.637+0000 7fd82f84b700 1 --2- 172.21.15.33:0/4055172934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 0x7fd828070cb0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fd824004970 tx=0x7fd824009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:51.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.637+0000 7fd82f84b700 1 -- 172.21.15.33:0/4055172934 shutdown_connections 2024-04-25T02:04:51.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.637+0000 7fd82f84b700 1 --2- 172.21.15.33:0/4055172934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 0x7fd828070cb0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:51.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.637+0000 7fd82f84b700 1 --2- 172.21.15.33:0/4055172934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd828071f30 0x7fd828070300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:51.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.637+0000 7fd82f84b700 1 -- 172.21.15.33:0/4055172934 >> 172.21.15.33:0/4055172934 conn(0x7fd8280f8990 msgr2=0x7fd8280fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:51.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.637+0000 7fd82f84b700 1 -- 172.21.15.33:0/4055172934 shutdown_connections 2024-04-25T02:04:51.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.637+0000 7fd82f84b700 1 -- 172.21.15.33:0/4055172934 wait complete. 2024-04-25T02:04:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82f84b700 1 Processor -- start 2024-04-25T02:04:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82f84b700 1 -- start start 2024-04-25T02:04:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82f84b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 0x7fd828119f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82f84b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd828071f30 0x7fd82811a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82f84b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82811ab00 con 0x7fd828070840 2024-04-25T02:04:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82f84b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8281140e0 con 0x7fd828071f30 2024-04-25T02:04:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82d5e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 0x7fd828119f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82d5e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 0x7fd828119f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51342/0 (socket says 172.21.15.33:51342) 2024-04-25T02:04:51.645 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82d5e7700 1 -- 172.21.15.33:0/707009447 learned_addr learned my addr 172.21.15.33:0/707009447 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82cde6700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd828071f30 0x7fd82811a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82d5e7700 1 -- 172.21.15.33:0/707009447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd828071f30 msgr2=0x7fd82811a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82d5e7700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd828071f30 0x7fd82811a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82d5e7700 1 -- 172.21.15.33:0/707009447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd824005040 con 0x7fd828070840 2024-04-25T02:04:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.638+0000 7fd82cde6700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd828071f30 0x7fd82811a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:04:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.639+0000 7fd82d5e7700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 0x7fd828119f70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd81800c830 tx=0x7fd818004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.639+0000 7fd81dffb700 1 -- 172.21.15.33:0/707009447 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81800a760 con 0x7fd828070840 2024-04-25T02:04:51.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.639+0000 7fd81dffb700 1 -- 172.21.15.33:0/707009447 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd81800ad90 con 0x7fd828070840 2024-04-25T02:04:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.639+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8281143c0 con 0x7fd828070840 2024-04-25T02:04:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.639+0000 7fd81dffb700 1 -- 172.21.15.33:0/707009447 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8180113a0 con 0x7fd828070840 2024-04-25T02:04:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.639+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8281148d0 con 0x7fd828070840 2024-04-25T02:04:51.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.640+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd828071a40 con 0x7fd828070840 2024-04-25T02:04:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.640+0000 7fd81dffb700 1 -- 172.21.15.33:0/707009447 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd818012400 con 0x7fd828070840 2024-04-25T02:04:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.641+0000 7fd81dffb700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd81406e140 0x7fd8140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.641+0000 7fd81dffb700 1 -- 172.21.15.33:0/707009447 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd81808dad0 con 0x7fd828070840 2024-04-25T02:04:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.641+0000 7fd82cde6700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd81406e140 0x7fd8140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:51.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.641+0000 7fd82cde6700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd81406e140 0x7fd8140705f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd824004f50 tx=0x7fd824009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:51.651 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.646+0000 7fd81dffb700 1 -- 172.21.15.33:0/707009447 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd81805aaa0 con 0x7fd828070840 2024-04-25T02:04:51.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.875+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd828115610 con 0x7fd81406e140 2024-04-25T02:04:51.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.876+0000 7fd81dffb700 1 -- 172.21.15.33:0/707009447 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd828115610 con 0x7fd81406e140 2024-04-25T02:04:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd81406e140 msgr2=0x7fd8140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd81406e140 0x7fd8140705f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd824004f50 tx=0x7fd824009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 msgr2=0x7fd828119f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 0x7fd828119f70 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd81800c830 tx=0x7fd818004470 comp rx=0 tx=0).stop 2024-04-25T02:04:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 shutdown_connections 2024-04-25T02:04:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd81406e140 0x7fd8140705f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:51.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd828070840 0x7fd828119f70 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 --2- 172.21.15.33:0/707009447 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd828071f30 0x7fd82811a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 >> 172.21.15.33:0/707009447 conn(0x7fd8280f8990 msgr2=0x7fd8280f99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 shutdown_connections 2024-04-25T02:04:51.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.878+0000 7fd82f84b700 1 -- 172.21.15.33:0/707009447 wait complete. 2024-04-25T02:04:51.894 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:04:51.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.983+0000 7f355a81c700 1 -- 172.21.15.33:0/3332313531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554102de0 msgr2=0x7f35541031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:51.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.983+0000 7f355a81c700 1 --2- 172.21.15.33:0/3332313531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554102de0 0x7f35541031b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3544004940 tx=0x7f3544009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:51.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.983+0000 7f355a81c700 1 -- 172.21.15.33:0/3332313531 shutdown_connections 2024-04-25T02:04:51.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.983+0000 7f355a81c700 1 --2- 172.21.15.33:0/3332313531 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3554103780 0x7f35540fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:51.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.983+0000 7f355a81c700 1 --2- 172.21.15.33:0/3332313531 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554102de0 0x7f35541031b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:51.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.983+0000 7f355a81c700 1 -- 172.21.15.33:0/3332313531 >> 172.21.15.33:0/3332313531 conn(0x7f35540f8990 msgr2=0x7f35540fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:51.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.983+0000 7f355a81c700 1 -- 172.21.15.33:0/3332313531 shutdown_connections 2024-04-25T02:04:51.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.983+0000 7f355a81c700 1 -- 172.21.15.33:0/3332313531 wait complete. 2024-04-25T02:04:51.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.983+0000 7f355a81c700 1 Processor -- start 2024-04-25T02:04:51.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f355a81c700 1 -- start start 2024-04-25T02:04:51.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f355a81c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3554102de0 0x7f3554113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:51.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f355a81c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554103780 0x7f3554113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:51.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f355a81c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3554114500 con 0x7f3554103780 2024-04-25T02:04:51.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f355a81c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355410db50 con 0x7f3554102de0 2024-04-25T02:04:51.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f35537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554103780 0x7f3554113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:51.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f35537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554103780 0x7f3554113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51358/0 (socket says 172.21.15.33:51358) 2024-04-25T02:04:51.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f35537fe700 1 -- 172.21.15.33:0/3629399026 learned_addr learned my addr 172.21.15.33:0/3629399026 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:51.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f3553fff700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3554102de0 0x7f3554113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:51.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f35537fe700 1 -- 172.21.15.33:0/3629399026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3554102de0 msgr2=0x7f3554113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:51.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f35537fe700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3554102de0 0x7f3554113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:51.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f35537fe700 1 -- 172.21.15.33:0/3629399026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544005040 con 0x7f3554103780 2024-04-25T02:04:51.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.984+0000 7f35537fe700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554103780 0x7f3554113eb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3548002a50 tx=0x7f3548004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:51.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.985+0000 7f3550ff9700 1 -- 172.21.15.33:0/3629399026 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3548007cc0 con 0x7f3554103780 2024-04-25T02:04:51.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.985+0000 7f3550ff9700 1 -- 172.21.15.33:0/3629399026 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f354800f040 con 0x7f3554103780 2024-04-25T02:04:51.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.985+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355410de30 con 0x7f3554103780 2024-04-25T02:04:51.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.985+0000 7f3550ff9700 1 -- 172.21.15.33:0/3629399026 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354800d9c0 con 0x7f3554103780 2024-04-25T02:04:51.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.985+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355410e3f0 con 0x7f3554103780 2024-04-25T02:04:51.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.986+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35541000f0 con 0x7f3554103780 2024-04-25T02:04:51.995 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.986+0000 7f3550ff9700 1 -- 172.21.15.33:0/3629399026 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3548009730 con 0x7f3554103780 2024-04-25T02:04:51.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.987+0000 7f3550ff9700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f353c06e140 0x7f353c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:51.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.987+0000 7f3550ff9700 1 -- 172.21.15.33:0/3629399026 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f354808d890 con 0x7f3554103780 2024-04-25T02:04:51.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.987+0000 7f3553fff700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f353c06e140 0x7f353c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:51.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.987+0000 7f3553fff700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f353c06e140 0x7f353c0705f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f354400b3b0 tx=0x7f3544009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:51.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:51.992+0000 7f3550ff9700 1 -- 172.21.15.33:0/3629399026 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f354805a740 con 0x7f3554103780 2024-04-25T02:04:52.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.221+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f355410f030 con 0x7f353c06e140 2024-04-25T02:04:52.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.222+0000 7f3550ff9700 1 -- 172.21.15.33:0/3629399026 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f355410f030 con 0x7f353c06e140 2024-04-25T02:04:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f353c06e140 msgr2=0x7f353c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f353c06e140 0x7f353c0705f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f354400b3b0 tx=0x7f3544009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554103780 msgr2=0x7f3554113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554103780 0x7f3554113eb0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3548002a50 tx=0x7f3548004600 comp rx=0 tx=0).stop 2024-04-25T02:04:52.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 shutdown_connections 2024-04-25T02:04:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f353c06e140 0x7f353c0705f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3554102de0 0x7f3554113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 --2- 172.21.15.33:0/3629399026 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3554103780 0x7f3554113eb0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 >> 172.21.15.33:0/3629399026 conn(0x7f35540f8990 msgr2=0x7f3554107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 shutdown_connections 2024-04-25T02:04:52.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.224+0000 7f355a81c700 1 -- 172.21.15.33:0/3629399026 wait complete. 2024-04-25T02:04:52.335 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.329+0000 7fe7053d8700 1 -- 172.21.15.33:0/2255445969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fe810 msgr2=0x7fe700106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.329+0000 7fe7053d8700 1 --2- 172.21.15.33:0/2255445969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fe810 0x7fe700106e50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4004940 tx=0x7fe6f4009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:52.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.329+0000 7fe7053d8700 1 -- 172.21.15.33:0/2255445969 shutdown_connections 2024-04-25T02:04:52.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.329+0000 7fe7053d8700 1 --2- 172.21.15.33:0/2255445969 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fe810 0x7fe700106e50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.329+0000 7fe7053d8700 1 --2- 172.21.15.33:0/2255445969 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7000fde70 0x7fe7000fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.329+0000 7fe7053d8700 1 -- 172.21.15.33:0/2255445969 >> 172.21.15.33:0/2255445969 conn(0x7fe7000f78e0 msgr2=0x7fe7000f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:52.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.329+0000 7fe7053d8700 1 -- 172.21.15.33:0/2255445969 shutdown_connections 2024-04-25T02:04:52.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.329+0000 7fe7053d8700 1 -- 172.21.15.33:0/2255445969 wait complete. 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.330+0000 7fe7053d8700 1 Processor -- start 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.330+0000 7fe7053d8700 1 -- start start 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.330+0000 7fe7053d8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fde70 0x7fe700113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.330+0000 7fe7053d8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7000fe810 0x7fe700113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.330+0000 7fe7053d8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7001144c0 con 0x7fe7000fde70 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.330+0000 7fe7053d8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70010dac0 con 0x7fe7000fe810 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.330+0000 7fe6feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fde70 0x7fe700113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6feffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fde70 0x7fe700113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51380/0 (socket says 172.21.15.33:51380) 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6feffd700 1 -- 172.21.15.33:0/3266879716 learned_addr learned my addr 172.21.15.33:0/3266879716 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6fe7fc700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7000fe810 0x7fe700113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6feffd700 1 -- 172.21.15.33:0/3266879716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7000fe810 msgr2=0x7fe700113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6feffd700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7000fe810 0x7fe700113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.338 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6feffd700 1 -- 172.21.15.33:0/3266879716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4005040 con 0x7fe7000fde70 2024-04-25T02:04:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6feffd700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fde70 0x7fe700113930 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0002a50 tx=0x7fe6f00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6e77fe700 1 -- 172.21.15.33:0/3266879716 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f0007cc0 con 0x7fe7000fde70 2024-04-25T02:04:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6e77fe700 1 -- 172.21.15.33:0/3266879716 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6f000f040 con 0x7fe7000fde70 2024-04-25T02:04:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70010dda0 con 0x7fe7000fde70 2024-04-25T02:04:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe6e77fe700 1 -- 172.21.15.33:0/3266879716 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6f0009640 con 0x7fe7000fde70 2024-04-25T02:04:52.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.331+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70010e2e0 con 0x7fe7000fde70 2024-04-25T02:04:52.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.332+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe700066ad0 con 0x7fe7000fde70 2024-04-25T02:04:52.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.333+0000 7fe6e77fe700 1 -- 172.21.15.33:0/3266879716 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe6f00097c0 con 0x7fe7000fde70 2024-04-25T02:04:52.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.333+0000 7fe6e77fe700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6e806e140 0x7fe6e80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:52.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.333+0000 7fe6e77fe700 1 -- 172.21.15.33:0/3266879716 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe6f008d8c0 con 0x7fe7000fde70 2024-04-25T02:04:52.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.333+0000 7fe6fe7fc700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6e806e140 0x7fe6e80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:52.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.334+0000 7fe6fe7fc700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6e806e140 0x7fe6e80705f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe70010ec80 tx=0x7fe6f4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:52.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.338+0000 7fe6e77fe700 1 -- 172.21.15.33:0/3266879716 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe6f005a770 con 0x7fe7000fde70 2024-04-25T02:04:52.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:52 smithi033 ceph-mon[111306]: pgmap v765: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:52.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.555+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe700109e30 con 0x7fe6e806e140 2024-04-25T02:04:52.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.566+0000 7fe6e77fe700 1 -- 172.21.15.33:0/3266879716 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe700109e30 con 0x7fe6e806e140 2024-04-25T02:04:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:04:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (26m) 3m ago 37m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:04:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (23m) 3m ago 37m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:04:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (23m) 3m ago 36m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:04:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (25m) 3m ago 36m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:04:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (19m) 3m ago 31m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:04:52.572 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (19m) 3m ago 31m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (18m) 3m ago 31m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (3m) 3m ago 31m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (27m) 3m ago 38m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (26m) 3m ago 36m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (24m) 3m ago 38m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (23m) 3m ago 36m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (26m) 3m ago 37m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (26m) 3m ago 36m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (22m) 3m ago 35m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (22m) 3m ago 35m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (21m) 3m ago 34m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (20m) 3m ago 34m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (20m) 3m ago 33m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:04:52.573 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (19m) 3m ago 33m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:04:52.574 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (26m) 3m ago 36m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:04:52.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6e806e140 msgr2=0x7fe6e80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6e806e140 0x7fe6e80705f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe70010ec80 tx=0x7fe6f4009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fde70 msgr2=0x7fe700113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fde70 0x7fe700113930 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe6f0002a50 tx=0x7fe6f00046b0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 shutdown_connections 2024-04-25T02:04:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6e806e140 0x7fe6e80705f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7000fde70 0x7fe700113930 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 --2- 172.21.15.33:0/3266879716 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7000fe810 0x7fe700113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 >> 172.21.15.33:0/3266879716 conn(0x7fe7000f78e0 msgr2=0x7fe7000f8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 shutdown_connections 2024-04-25T02:04:52.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.568+0000 7fe7053d8700 1 -- 172.21.15.33:0/3266879716 wait complete. 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.673+0000 7fa099dcf700 1 -- 172.21.15.33:0/3656003112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fd0a0 msgr2=0x7fa094109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.673+0000 7fa099dcf700 1 --2- 172.21.15.33:0/3656003112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fd0a0 0x7fa094109350 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa07c004940 tx=0x7fa07c009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.673+0000 7fa099dcf700 1 -- 172.21.15.33:0/3656003112 shutdown_connections 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.673+0000 7fa099dcf700 1 --2- 172.21.15.33:0/3656003112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fd0a0 0x7fa094109350 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.673+0000 7fa099dcf700 1 --2- 172.21.15.33:0/3656003112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0940fc790 0x7fa0940fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.673+0000 7fa099dcf700 1 -- 172.21.15.33:0/3656003112 >> 172.21.15.33:0/3656003112 conn(0x7fa094071d00 msgr2=0x7fa094072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.673+0000 7fa099dcf700 1 -- 172.21.15.33:0/3656003112 shutdown_connections 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.673+0000 7fa099dcf700 1 -- 172.21.15.33:0/3656003112 wait complete. 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.673+0000 7fa099dcf700 1 Processor -- start 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa099dcf700 1 -- start start 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa099dcf700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fc790 0x7fa094115e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:52.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa099dcf700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0940fd0a0 0x7fa094116370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa099dcf700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0941169c0 con 0x7fa0940fc790 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa099dcf700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09410fd90 con 0x7fa0940fd0a0 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa0937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fc790 0x7fa094115e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa0937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fc790 0x7fa094115e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51396/0 (socket says 172.21.15.33:51396) 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa0937fe700 1 -- 172.21.15.33:0/2886135500 learned_addr learned my addr 172.21.15.33:0/2886135500 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa08affd700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0940fd0a0 0x7fa094116370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa0937fe700 1 -- 172.21.15.33:0/2886135500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0940fd0a0 msgr2=0x7fa094116370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa0937fe700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0940fd0a0 0x7fa094116370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa0937fe700 1 -- 172.21.15.33:0/2886135500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c005040 con 0x7fa0940fc790 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa08affd700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0940fd0a0 0x7fa094116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.674+0000 7fa0937fe700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fc790 0x7fa094115e30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa084002a50 tx=0x7fa0840046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.675+0000 7fa090ff9700 1 -- 172.21.15.33:0/2886135500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa084007cc0 con 0x7fa0940fc790 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.675+0000 7fa090ff9700 1 -- 172.21.15.33:0/2886135500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa08400f040 con 0x7fa0940fc790 2024-04-25T02:04:52.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.675+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa094110070 con 0x7fa0940fc790 2024-04-25T02:04:52.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.675+0000 7fa090ff9700 1 -- 172.21.15.33:0/2886135500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa084009640 con 0x7fa0940fc790 2024-04-25T02:04:52.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.675+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0941105b0 con 0x7fa0940fc790 2024-04-25T02:04:52.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.676+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0940fc330 con 0x7fa0940fc790 2024-04-25T02:04:52.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.676+0000 7fa090ff9700 1 -- 172.21.15.33:0/2886135500 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa0840097c0 con 0x7fa0940fc790 2024-04-25T02:04:52.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.677+0000 7fa090ff9700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa08006e140 0x7fa0800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:52.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.677+0000 7fa08affd700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa08006e140 0x7fa0800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:52.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.677+0000 7fa090ff9700 1 -- 172.21.15.33:0/2886135500 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa08408d8c0 con 0x7fa0940fc790 2024-04-25T02:04:52.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.677+0000 7fa08affd700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa08006e140 0x7fa0800705f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa07c004f90 tx=0x7fa07c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:52.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.681+0000 7fa090ff9700 1 -- 172.21.15.33:0/2886135500 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa08405a770 con 0x7fa0940fc790 2024-04-25T02:04:52.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:52 smithi138 ceph-mon[81304]: pgmap v765: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:52.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.977+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa09410c130 con 0x7fa0940fc790 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.978+0000 7fa090ff9700 1 -- 172.21.15.33:0/2886135500 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa08405a300 con 0x7fa0940fc790 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:04:52.984 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:04:52.985 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:04:52.985 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:04:52.985 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:04:52.985 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:04:52.985 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:04:52.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.980+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa08006e140 msgr2=0x7fa0800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.980+0000 7fa099dcf700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa08006e140 0x7fa0800705f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa07c004f90 tx=0x7fa07c009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.980+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fc790 msgr2=0x7fa094115e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.980+0000 7fa099dcf700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fc790 0x7fa094115e30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa084002a50 tx=0x7fa0840046b0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.980+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 shutdown_connections 2024-04-25T02:04:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.980+0000 7fa099dcf700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa08006e140 0x7fa0800705f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.980+0000 7fa099dcf700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa0940fc790 0x7fa094115e30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.980+0000 7fa099dcf700 1 --2- 172.21.15.33:0/2886135500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa0940fd0a0 0x7fa094116370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.980+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 >> 172.21.15.33:0/2886135500 conn(0x7fa094071d00 msgr2=0x7fa0940f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.981+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 shutdown_connections 2024-04-25T02:04:52.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:52.981+0000 7fa099dcf700 1 -- 172.21.15.33:0/2886135500 wait complete. 2024-04-25T02:04:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.085+0000 7f237e42e700 1 -- 172.21.15.33:0/827383142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 msgr2=0x7f23780fe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.085+0000 7f237e42e700 1 --2- 172.21.15.33:0/827383142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 0x7f23780fe200 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f2368004940 tx=0x7f2368009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.085+0000 7f237e42e700 1 -- 172.21.15.33:0/827383142 shutdown_connections 2024-04-25T02:04:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.085+0000 7f237e42e700 1 --2- 172.21.15.33:0/827383142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23780fe7d0 0x7f2378106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.091 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.085+0000 7f237e42e700 1 --2- 172.21.15.33:0/827383142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 0x7f23780fe200 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.085+0000 7f237e42e700 1 -- 172.21.15.33:0/827383142 >> 172.21.15.33:0/827383142 conn(0x7f23780f7870 msgr2=0x7f23780f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.085+0000 7f237e42e700 1 -- 172.21.15.33:0/827383142 shutdown_connections 2024-04-25T02:04:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.085+0000 7f237e42e700 1 -- 172.21.15.33:0/827383142 wait complete. 2024-04-25T02:04:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.085+0000 7f237e42e700 1 Processor -- start 2024-04-25T02:04:53.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f237e42e700 1 -- start start 2024-04-25T02:04:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f237e42e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 0x7f23780fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:53.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f237e42e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23780fe7d0 0x7f23780fc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f237e42e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23780fdc40 con 0x7f23780fde30 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f237e42e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23780fc770 con 0x7f23780fe7d0 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f2377fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 0x7f23780fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f2377fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 0x7f23780fbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51422/0 (socket says 172.21.15.33:51422) 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f2377fff700 1 -- 172.21.15.33:0/168444938 learned_addr learned my addr 172.21.15.33:0/168444938 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f23777fe700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23780fe7d0 0x7f23780fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f2377fff700 1 -- 172.21.15.33:0/168444938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23780fe7d0 msgr2=0x7f23780fc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f2377fff700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23780fe7d0 0x7f23780fc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f2377fff700 1 -- 172.21.15.33:0/168444938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2368005040 con 0x7f23780fde30 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.086+0000 7f23777fe700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23780fe7d0 0x7f23780fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:04:53.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.087+0000 7f2377fff700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 0x7f23780fbcf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f23680093b0 tx=0x7f2368009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.087+0000 7f2374ff9700 1 -- 172.21.15.33:0/168444938 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236800e040 con 0x7f23780fde30 2024-04-25T02:04:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.087+0000 7f2374ff9700 1 -- 172.21.15.33:0/168444938 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2368004290 con 0x7f23780fde30 2024-04-25T02:04:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.087+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23780fc9f0 con 0x7f23780fde30 2024-04-25T02:04:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.087+0000 7f2374ff9700 1 -- 172.21.15.33:0/168444938 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236801c3f0 con 0x7f23780fde30 2024-04-25T02:04:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.087+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237811a0b0 con 0x7f23780fde30 2024-04-25T02:04:53.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.088+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2378066a70 con 0x7f23780fde30 2024-04-25T02:04:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.088+0000 7f2374ff9700 1 -- 172.21.15.33:0/168444938 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2368013890 con 0x7f23780fde30 2024-04-25T02:04:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.089+0000 7f2374ff9700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f236406e140 0x7f23640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.089+0000 7f2374ff9700 1 -- 172.21.15.33:0/168444938 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f236808ed90 con 0x7f23780fde30 2024-04-25T02:04:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.089+0000 7f23777fe700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f236406e140 0x7f23640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:53.096 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.089+0000 7f23777fe700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f236406e140 0x7f23640705f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2360009fd0 tx=0x7f2360009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:53.099 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.093+0000 7f2374ff9700 1 -- 172.21.15.33:0/168444938 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2368058190 con 0x7f23780fde30 2024-04-25T02:04:53.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.349+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f23780fd200 con 0x7f23780fde30 2024-04-25T02:04:53.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.350+0000 7f2374ff9700 1 -- 172.21.15.33:0/168444938 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f236805b850 con 0x7f23780fde30 2024-04-25T02:04:53.356 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:04:53.356 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:04:53.356 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:04:53.356 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:04:53.356 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:04:53.357 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:04:53.358 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:04:53.359 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:04:53.359 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:04:53.359 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:04:53.359 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:04:53.359 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:04:53.359 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:04:53.359 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f236406e140 msgr2=0x7f23640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f236406e140 0x7f23640705f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2360009fd0 tx=0x7f2360009ee0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 msgr2=0x7f23780fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 0x7f23780fbcf0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f23680093b0 tx=0x7f2368009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 shutdown_connections 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f236406e140 0x7f23640705f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23780fde30 0x7f23780fbcf0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 --2- 172.21.15.33:0/168444938 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23780fe7d0 0x7f23780fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 >> 172.21.15.33:0/168444938 conn(0x7f23780f7870 msgr2=0x7f23780f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 shutdown_connections 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.353+0000 7f237e42e700 1 -- 172.21.15.33:0/168444938 wait complete. 2024-04-25T02:04:53.360 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:04:53.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.457+0000 7f6a522dc700 1 -- 172.21.15.33:0/1154777001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 msgr2=0x7f6a4c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.457+0000 7f6a522dc700 1 --2- 172.21.15.33:0/1154777001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 0x7f6a4c103220 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6a34004940 tx=0x7f6a34009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:53.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.457+0000 7f6a522dc700 1 -- 172.21.15.33:0/1154777001 shutdown_connections 2024-04-25T02:04:53.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.457+0000 7f6a522dc700 1 --2- 172.21.15.33:0/1154777001 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a4c065640 0x7f6a4c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 --2- 172.21.15.33:0/1154777001 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 0x7f6a4c103220 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 -- 172.21.15.33:0/1154777001 >> 172.21.15.33:0/1154777001 conn(0x7f6a4c071d00 msgr2=0x7f6a4c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 -- 172.21.15.33:0/1154777001 shutdown_connections 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 -- 172.21.15.33:0/1154777001 wait complete. 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 Processor -- start 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 -- start start 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a4c065640 0x7f6a4c0fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 0x7f6a4c0fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c0fc910 con 0x7f6a4c102e50 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.458+0000 7f6a522dc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c0fca50 con 0x7f6a4c065640 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a4b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 0x7f6a4c0fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a4b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 0x7f6a4c0fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51442/0 (socket says 172.21.15.33:51442) 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a4b7fe700 1 -- 172.21.15.33:0/747251269 learned_addr learned my addr 172.21.15.33:0/747251269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:53.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a4bfff700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a4c065640 0x7f6a4c0fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a4b7fe700 1 -- 172.21.15.33:0/747251269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a4c065640 msgr2=0x7f6a4c0fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a4b7fe700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a4c065640 0x7f6a4c0fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a4b7fe700 1 -- 172.21.15.33:0/747251269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a34005040 con 0x7f6a4c102e50 2024-04-25T02:04:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a4b7fe700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 0x7f6a4c0fc2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00b790 tx=0x7f6a3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a48ff9700 1 -- 172.21.15.33:0/747251269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3c00f770 con 0x7f6a4c102e50 2024-04-25T02:04:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a48ff9700 1 -- 172.21.15.33:0/747251269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a3c00fda0 con 0x7f6a4c102e50 2024-04-25T02:04:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a4c119e80 con 0x7f6a4c102e50 2024-04-25T02:04:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a48ff9700 1 -- 172.21.15.33:0/747251269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a3c00d310 con 0x7f6a4c102e50 2024-04-25T02:04:53.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.459+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a4c11a360 con 0x7f6a4c102e50 2024-04-25T02:04:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.460+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4c0fe540 con 0x7f6a4c102e50 2024-04-25T02:04:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.461+0000 7f6a48ff9700 1 -- 172.21.15.33:0/747251269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6a3c010420 con 0x7f6a4c102e50 2024-04-25T02:04:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.461+0000 7f6a48ff9700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a3806e140 0x7f6a380705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.462+0000 7f6a48ff9700 1 -- 172.21.15.33:0/747251269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6a3c08dbe0 con 0x7f6a4c102e50 2024-04-25T02:04:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.462+0000 7f6a4bfff700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a3806e140 0x7f6a380705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:53.468 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.462+0000 7f6a4bfff700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a3806e140 0x7f6a380705f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6a34004560 tx=0x7f6a34009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:53.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.466+0000 7f6a48ff9700 1 -- 172.21.15.33:0/747251269 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6a3c056f40 con 0x7f6a4c102e50 2024-04-25T02:04:53.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.691+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a4c04bed0 con 0x7f6a3806e140 2024-04-25T02:04:53.697 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:53 smithi033 ceph-mon[111306]: from='client.35412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:53.697 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:53 smithi033 ceph-mon[111306]: from='client.35416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:53.697 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:53 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2886135500' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:04:53.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.692+0000 7f6a48ff9700 1 -- 172.21.15.33:0/747251269 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6a4c04bed0 con 0x7f6a3806e140 2024-04-25T02:04:53.698 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:04:53.698 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:04:53.699 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:04:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.694+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a3806e140 msgr2=0x7f6a380705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.694+0000 7f6a522dc700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a3806e140 0x7f6a380705f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6a34004560 tx=0x7f6a34009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.694+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 msgr2=0x7f6a4c0fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.694+0000 7f6a522dc700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 0x7f6a4c0fc2f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00b790 tx=0x7f6a3c009250 comp rx=0 tx=0).stop 2024-04-25T02:04:53.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.694+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 shutdown_connections 2024-04-25T02:04:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.694+0000 7f6a522dc700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a3806e140 0x7f6a380705f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.694+0000 7f6a522dc700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a4c065640 0x7f6a4c0fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.694+0000 7f6a522dc700 1 --2- 172.21.15.33:0/747251269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a4c102e50 0x7f6a4c0fc2f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.694+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 >> 172.21.15.33:0/747251269 conn(0x7f6a4c071d00 msgr2=0x7f6a4c0fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.695+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 shutdown_connections 2024-04-25T02:04:53.701 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.695+0000 7f6a522dc700 1 -- 172.21.15.33:0/747251269 wait complete. 2024-04-25T02:04:53.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:53 smithi138 ceph-mon[81304]: from='client.35412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:53.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:53 smithi138 ceph-mon[81304]: from='client.35416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:53.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:53 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2886135500' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.799+0000 7fcae3367700 1 -- 172.21.15.33:0/2025950278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fe770 msgr2=0x7fcadc106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.799+0000 7fcae3367700 1 --2- 172.21.15.33:0/2025950278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fe770 0x7fcadc106e40 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcacc004940 tx=0x7fcacc009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.799+0000 7fcae3367700 1 -- 172.21.15.33:0/2025950278 shutdown_connections 2024-04-25T02:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.799+0000 7fcae3367700 1 --2- 172.21.15.33:0/2025950278 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fe770 0x7fcadc106e40 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.799+0000 7fcae3367700 1 --2- 172.21.15.33:0/2025950278 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcadc0fddd0 0x7fcadc0fe1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.799+0000 7fcae3367700 1 -- 172.21.15.33:0/2025950278 >> 172.21.15.33:0/2025950278 conn(0x7fcadc0f7850 msgr2=0x7fcadc0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.799+0000 7fcae3367700 1 -- 172.21.15.33:0/2025950278 shutdown_connections 2024-04-25T02:04:53.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.799+0000 7fcae3367700 1 -- 172.21.15.33:0/2025950278 wait complete. 2024-04-25T02:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae3367700 1 Processor -- start 2024-04-25T02:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae3367700 1 -- start start 2024-04-25T02:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae3367700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fddd0 0x7fcadc0fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae3367700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcadc0fe770 0x7fcadc0fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae3367700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0fc760 con 0x7fcadc0fddd0 2024-04-25T02:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae3367700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0fc8a0 con 0x7fcadc0fe770 2024-04-25T02:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae1103700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fddd0 0x7fcadc0fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:53.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae1103700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fddd0 0x7fcadc0fbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51452/0 (socket says 172.21.15.33:51452) 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae1103700 1 -- 172.21.15.33:0/103286440 learned_addr learned my addr 172.21.15.33:0/103286440 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae0902700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcadc0fe770 0x7fcadc0fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae1103700 1 -- 172.21.15.33:0/103286440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcadc0fe770 msgr2=0x7fcadc0fc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae1103700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcadc0fe770 0x7fcadc0fc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.800+0000 7fcae1103700 1 -- 172.21.15.33:0/103286440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc005040 con 0x7fcadc0fddd0 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.801+0000 7fcae1103700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fddd0 0x7fcadc0fbc90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcad800c8c0 tx=0x7fcad80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.801+0000 7fcad1ffb700 1 -- 172.21.15.33:0/103286440 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcad800a760 con 0x7fcadc0fddd0 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.801+0000 7fcad1ffb700 1 -- 172.21.15.33:0/103286440 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcad800ad90 con 0x7fcadc0fddd0 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.801+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc119e50 con 0x7fcadc0fddd0 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.801+0000 7fcad1ffb700 1 -- 172.21.15.33:0/103286440 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcad80113a0 con 0x7fcadc0fddd0 2024-04-25T02:04:53.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.801+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc11a250 con 0x7fcadc0fddd0 2024-04-25T02:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.802+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcadc0666d0 con 0x7fcadc0fddd0 2024-04-25T02:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.802+0000 7fcad1ffb700 1 -- 172.21.15.33:0/103286440 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcad8012400 con 0x7fcadc0fddd0 2024-04-25T02:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.803+0000 7fcad1ffb700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcac806e150 0x7fcac8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.803+0000 7fcad1ffb700 1 -- 172.21.15.33:0/103286440 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcad808da70 con 0x7fcadc0fddd0 2024-04-25T02:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.803+0000 7fcae0902700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcac806e150 0x7fcac8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:04:53.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.803+0000 7fcae0902700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcac806e150 0x7fcac8070600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0fd5d0 tx=0x7fcacc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:04:53.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:53.808+0000 7fcad1ffb700 1 -- 172.21.15.33:0/103286440 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcad8056f10 con 0x7fcadc0fddd0 2024-04-25T02:04:54.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.100+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcadc04bf60 con 0x7fcadc0fddd0 2024-04-25T02:04:54.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.100+0000 7fcad1ffb700 1 -- 172.21.15.33:0/103286440 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fcad805a5d0 con 0x7fcadc0fddd0 2024-04-25T02:04:54.106 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:04:54.106 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:04:54.106 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:04:54.106 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:04:54.106 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:04:54.106 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:04:54.107 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:04:54.108 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:04:54.108 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:04:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcac806e150 msgr2=0x7fcac8070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcac806e150 0x7fcac8070600 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0fd5d0 tx=0x7fcacc009e30 comp rx=0 tx=0).stop 2024-04-25T02:04:54.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fddd0 msgr2=0x7fcadc0fbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:04:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fddd0 0x7fcadc0fbc90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcad800c8c0 tx=0x7fcad80044c0 comp rx=0 tx=0).stop 2024-04-25T02:04:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 shutdown_connections 2024-04-25T02:04:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcac806e150 0x7fcac8070600 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcadc0fddd0 0x7fcadc0fbc90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 --2- 172.21.15.33:0/103286440 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcadc0fe770 0x7fcadc0fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:04:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 >> 172.21.15.33:0/103286440 conn(0x7fcadc0f7850 msgr2=0x7fcadc0f8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:04:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 shutdown_connections 2024-04-25T02:04:54.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:04:54.102+0000 7fcae3367700 1 -- 172.21.15.33:0/103286440 wait complete. 2024-04-25T02:04:54.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:54 smithi033 ceph-mon[111306]: from='client.35420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:54.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:54 smithi033 ceph-mon[111306]: pgmap v766: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:54.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:54 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/168444938' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:04:54.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:54 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/103286440' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:04:54.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:54 smithi138 ceph-mon[81304]: from='client.35420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:54.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:54 smithi138 ceph-mon[81304]: pgmap v766: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:54.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:54 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/168444938' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:04:54.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:54 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/103286440' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:04:55.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:55 smithi033 ceph-mon[111306]: from='client.35432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:55.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:04:55.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:55 smithi138 ceph-mon[81304]: from='client.35432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:04:55.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:04:56.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:56 smithi033 ceph-mon[111306]: pgmap v767: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:56.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:56 smithi138 ceph-mon[81304]: pgmap v767: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:58.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:04:58 smithi033 ceph-mon[111306]: pgmap v768: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:04:58.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:04:58 smithi138 ceph-mon[81304]: pgmap v768: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:00.599 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:00 smithi138 ceph-mon[81304]: pgmap v769: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:00.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:00 smithi033 ceph-mon[111306]: pgmap v769: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:02.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:02 smithi033 ceph-mon[111306]: pgmap v770: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:02.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:02 smithi138 ceph-mon[81304]: pgmap v770: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:04.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:04 smithi033 ceph-mon[111306]: pgmap v771: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:04.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:04 smithi138 ceph-mon[81304]: pgmap v771: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:06.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:06 smithi033 ceph-mon[111306]: pgmap v772: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:06.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:06 smithi138 ceph-mon[81304]: pgmap v772: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:08.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:08 smithi033 ceph-mon[111306]: pgmap v773: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:08.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:08 smithi138 ceph-mon[81304]: pgmap v773: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:10.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:10 smithi033 ceph-mon[111306]: pgmap v774: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:10.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:05:10.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:10 smithi138 ceph-mon[81304]: pgmap v774: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:10.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:05:12.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:12 smithi033 ceph-mon[111306]: pgmap v775: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:12.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:12 smithi138 ceph-mon[81304]: pgmap v775: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:14.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:14 smithi033 ceph-mon[111306]: pgmap v776: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:14.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:14 smithi138 ceph-mon[81304]: pgmap v776: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:16.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:16 smithi033 ceph-mon[111306]: pgmap v777: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:16.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:16 smithi138 ceph-mon[81304]: pgmap v777: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:18.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:18 smithi033 ceph-mon[111306]: pgmap v778: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:18.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:18 smithi138 ceph-mon[81304]: pgmap v778: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:20.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:20 smithi033 ceph-mon[111306]: pgmap v779: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:20.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:20 smithi138 ceph-mon[81304]: pgmap v779: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:21.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:21 smithi033 ceph-mon[111306]: pgmap v780: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:21.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:21 smithi138 ceph-mon[81304]: pgmap v780: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:24.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 -- 172.21.15.33:0/4254272703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4103780 msgr2=0x7f5ad40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 --2- 172.21.15.33:0/4254272703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4103780 0x7f5ad40fcdf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4004940 tx=0x7f5ac4009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:24.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 -- 172.21.15.33:0/4254272703 shutdown_connections 2024-04-25T02:05:24.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 --2- 172.21.15.33:0/4254272703 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4103780 0x7f5ad40fcdf0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 --2- 172.21.15.33:0/4254272703 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4102de0 0x7f5ad41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 -- 172.21.15.33:0/4254272703 >> 172.21.15.33:0/4254272703 conn(0x7f5ad40f8990 msgr2=0x7f5ad40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:24.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 -- 172.21.15.33:0/4254272703 shutdown_connections 2024-04-25T02:05:24.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 -- 172.21.15.33:0/4254272703 wait complete. 2024-04-25T02:05:24.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 Processor -- start 2024-04-25T02:05:24.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.221+0000 7f5adc1d0700 1 -- start start 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5adc1d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4102de0 0x7f5ad41138d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5adc1d0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4103780 0x7f5ad4113e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5adc1d0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad4114460 con 0x7f5ad4102de0 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5adc1d0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad410da60 con 0x7f5ad4103780 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5ad9f6c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4102de0 0x7f5ad41138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5ad9f6c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4102de0 0x7f5ad41138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44458/0 (socket says 172.21.15.33:44458) 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5ad9f6c700 1 -- 172.21.15.33:0/2933816461 learned_addr learned my addr 172.21.15.33:0/2933816461 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5ad976b700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4103780 0x7f5ad4113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5ad9f6c700 1 -- 172.21.15.33:0/2933816461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4103780 msgr2=0x7f5ad4113e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5ad9f6c700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4103780 0x7f5ad4113e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5ad9f6c700 1 -- 172.21.15.33:0/2933816461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac4005040 con 0x7f5ad4102de0 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5ad976b700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4103780 0x7f5ad4113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5ad9f6c700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4102de0 0x7f5ad41138d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000eb40 tx=0x7f5ad000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5aca7fc700 1 -- 172.21.15.33:0/2933816461 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad0007cc0 con 0x7f5ad4102de0 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5aca7fc700 1 -- 172.21.15.33:0/2933816461 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ad000f040 con 0x7f5ad4102de0 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.222+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad410dce0 con 0x7f5ad4102de0 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.223+0000 7f5aca7fc700 1 -- 172.21.15.33:0/2933816461 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad0009620 con 0x7f5ad4102de0 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.223+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad410e1f0 con 0x7f5ad4102de0 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.223+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad40fe530 con 0x7f5ad4102de0 2024-04-25T02:05:24.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.224+0000 7f5aca7fc700 1 -- 172.21.15.33:0/2933816461 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5ad0009780 con 0x7f5ad4102de0 2024-04-25T02:05:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.224+0000 7f5aca7fc700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5ac006e140 0x7f5ac00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.225+0000 7f5aca7fc700 1 -- 172.21.15.33:0/2933816461 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5ad008d7d0 con 0x7f5ad4102de0 2024-04-25T02:05:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.225+0000 7f5ad976b700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5ac006e140 0x7f5ac00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:24.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.225+0000 7f5ad976b700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5ac006e140 0x7f5ac00705f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009b30 tx=0x7f5ac4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:24.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.229+0000 7f5aca7fc700 1 -- 172.21.15.33:0/2933816461 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5ad005a710 con 0x7f5ad4102de0 2024-04-25T02:05:24.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.456+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ad4109ec0 con 0x7f5ac006e140 2024-04-25T02:05:24.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.457+0000 7f5aca7fc700 1 -- 172.21.15.33:0/2933816461 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5ad4109ec0 con 0x7f5ac006e140 2024-04-25T02:05:24.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5ac006e140 msgr2=0x7f5ac00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5ac006e140 0x7f5ac00705f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009b30 tx=0x7f5ac4009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4102de0 msgr2=0x7f5ad41138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4102de0 0x7f5ad41138d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000eb40 tx=0x7f5ad000b040 comp rx=0 tx=0).stop 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 shutdown_connections 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5ac006e140 0x7f5ac00705f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5ad4102de0 0x7f5ad41138d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 --2- 172.21.15.33:0/2933816461 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5ad4103780 0x7f5ad4113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 >> 172.21.15.33:0/2933816461 conn(0x7f5ad40f8990 msgr2=0x7f5ad40f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 shutdown_connections 2024-04-25T02:05:24.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.459+0000 7f5adc1d0700 1 -- 172.21.15.33:0/2933816461 wait complete. 2024-04-25T02:05:24.466 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:24 smithi033 ceph-mon[111306]: pgmap v781: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:24.476 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:05:24.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:24 smithi138 ceph-mon[81304]: pgmap v781: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:24.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 -- 172.21.15.33:0/2568665474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb674101d10 msgr2=0x7fb6741020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 --2- 172.21.15.33:0/2568665474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb674101d10 0x7fb6741020e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fb664009ce0 tx=0x7fb664004d10 comp rx=0 tx=0).stop 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 -- 172.21.15.33:0/2568665474 shutdown_connections 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 --2- 172.21.15.33:0/2568665474 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb6741026b0 0x7fb6740fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 --2- 172.21.15.33:0/2568665474 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb674101d10 0x7fb6741020e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 -- 172.21.15.33:0/2568665474 >> 172.21.15.33:0/2568665474 conn(0x7fb6740f78e0 msgr2=0x7fb6740f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 -- 172.21.15.33:0/2568665474 shutdown_connections 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 -- 172.21.15.33:0/2568665474 wait complete. 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 Processor -- start 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.565+0000 7fb678f06700 1 -- start start 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb678f06700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb674101d10 0x7fb674102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb678f06700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6741026b0 0x7fb6741030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:24.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb678f06700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb674104ac0 con 0x7fb6741026b0 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb678f06700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6741035f0 con 0x7fb674101d10 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb671d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6741026b0 0x7fb6741030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb671d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6741026b0 0x7fb6741030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44484/0 (socket says 172.21.15.33:44484) 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb671d9b700 1 -- 172.21.15.33:0/196124985 learned_addr learned my addr 172.21.15.33:0/196124985 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb67259c700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb674101d10 0x7fb674102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb671d9b700 1 -- 172.21.15.33:0/196124985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb674101d10 msgr2=0x7fb674102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb671d9b700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb674101d10 0x7fb674102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb671d9b700 1 -- 172.21.15.33:0/196124985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb664009940 con 0x7fb6741026b0 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb671d9b700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6741026b0 0x7fb6741030b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb65c002a50 tx=0x7fb65c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.566+0000 7fb66affd700 1 -- 172.21.15.33:0/196124985 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb65c007cc0 con 0x7fb6741026b0 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.567+0000 7fb66affd700 1 -- 172.21.15.33:0/196124985 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb65c00f040 con 0x7fb6741026b0 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.567+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6741038d0 con 0x7fb6741026b0 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.567+0000 7fb66affd700 1 -- 172.21.15.33:0/196124985 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb65c00daf0 con 0x7fb6741026b0 2024-04-25T02:05:24.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.567+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67411a250 con 0x7fb6741026b0 2024-04-25T02:05:24.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.568+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6740fd460 con 0x7fb6741026b0 2024-04-25T02:05:24.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.568+0000 7fb66affd700 1 -- 172.21.15.33:0/196124985 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb65c009810 con 0x7fb6741026b0 2024-04-25T02:05:24.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.568+0000 7fb66affd700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb66006e140 0x7fb6600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:24.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.569+0000 7fb66affd700 1 -- 172.21.15.33:0/196124985 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb65c08d7f0 con 0x7fb6741026b0 2024-04-25T02:05:24.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.569+0000 7fb67259c700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb66006e140 0x7fb6600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:24.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.569+0000 7fb67259c700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb66006e140 0x7fb6600705f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb66400b740 tx=0x7fb664004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:24.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.573+0000 7fb66affd700 1 -- 172.21.15.33:0/196124985 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb65c056bf0 con 0x7fb6741026b0 2024-04-25T02:05:24.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.801+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb674066960 con 0x7fb66006e140 2024-04-25T02:05:24.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.803+0000 7fb66affd700 1 -- 172.21.15.33:0/196124985 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb674066960 con 0x7fb66006e140 2024-04-25T02:05:24.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb66006e140 msgr2=0x7fb6600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb66006e140 0x7fb6600705f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb66400b740 tx=0x7fb664004d10 comp rx=0 tx=0).stop 2024-04-25T02:05:24.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6741026b0 msgr2=0x7fb6741030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6741026b0 0x7fb6741030b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb65c002a50 tx=0x7fb65c004600 comp rx=0 tx=0).stop 2024-04-25T02:05:24.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 shutdown_connections 2024-04-25T02:05:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb66006e140 0x7fb6600705f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb674101d10 0x7fb674102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 --2- 172.21.15.33:0/196124985 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb6741026b0 0x7fb6741030b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 >> 172.21.15.33:0/196124985 conn(0x7fb6740f78e0 msgr2=0x7fb674066250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 shutdown_connections 2024-04-25T02:05:24.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.805+0000 7fb678f06700 1 -- 172.21.15.33:0/196124985 wait complete. 2024-04-25T02:05:24.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.908+0000 7f6cca22e700 1 -- 172.21.15.33:0/533605530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4103780 msgr2=0x7f6cc40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.908+0000 7f6cca22e700 1 --2- 172.21.15.33:0/533605530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4103780 0x7f6cc40fcdf0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8004940 tx=0x7f6cb8009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.908+0000 7f6cca22e700 1 -- 172.21.15.33:0/533605530 shutdown_connections 2024-04-25T02:05:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.908+0000 7f6cca22e700 1 --2- 172.21.15.33:0/533605530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4103780 0x7f6cc40fcdf0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.908+0000 7f6cca22e700 1 --2- 172.21.15.33:0/533605530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc4102de0 0x7f6cc41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.908+0000 7f6cca22e700 1 -- 172.21.15.33:0/533605530 >> 172.21.15.33:0/533605530 conn(0x7f6cc40f8990 msgr2=0x7f6cc40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.908+0000 7f6cca22e700 1 -- 172.21.15.33:0/533605530 shutdown_connections 2024-04-25T02:05:24.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.908+0000 7f6cca22e700 1 -- 172.21.15.33:0/533605530 wait complete. 2024-04-25T02:05:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.909+0000 7f6cca22e700 1 Processor -- start 2024-04-25T02:05:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.909+0000 7f6cca22e700 1 -- start start 2024-04-25T02:05:24.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.909+0000 7f6cca22e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102de0 0x7f6cc4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.909+0000 7f6cca22e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc4103780 0x7f6cc40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.909+0000 7f6cca22e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc40721b0 con 0x7f6cc4102de0 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.909+0000 7f6cca22e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc4070ce0 con 0x7f6cc4103780 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cc37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102de0 0x7f6cc4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cc37fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102de0 0x7f6cc4070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44494/0 (socket says 172.21.15.33:44494) 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cc37fe700 1 -- 172.21.15.33:0/1953116686 learned_addr learned my addr 172.21.15.33:0/1953116686 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cc2ffd700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc4103780 0x7f6cc40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cc37fe700 1 -- 172.21.15.33:0/1953116686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc4103780 msgr2=0x7f6cc40707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cc37fe700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc4103780 0x7f6cc40707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cc37fe700 1 -- 172.21.15.33:0/1953116686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb8005040 con 0x7f6cc4102de0 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cc37fe700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102de0 0x7f6cc4070260 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400eb40 tx=0x7f6cb400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cabfff700 1 -- 172.21.15.33:0/1953116686 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb4007cc0 con 0x7f6cc4102de0 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc4070f60 con 0x7f6cc4102de0 2024-04-25T02:05:24.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cabfff700 1 -- 172.21.15.33:0/1953116686 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cb400f040 con 0x7f6cc4102de0 2024-04-25T02:05:24.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cabfff700 1 -- 172.21.15.33:0/1953116686 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cb400ca10 con 0x7f6cc4102de0 2024-04-25T02:05:24.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.910+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc4119fd0 con 0x7f6cc4102de0 2024-04-25T02:05:24.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.911+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc40fe530 con 0x7f6cc4102de0 2024-04-25T02:05:24.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.912+0000 7f6cabfff700 1 -- 172.21.15.33:0/1953116686 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6cb400cc80 con 0x7f6cc4102de0 2024-04-25T02:05:24.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.912+0000 7f6cabfff700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cac06e140 0x7f6cac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:24.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.912+0000 7f6cabfff700 1 -- 172.21.15.33:0/1953116686 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6cb408d790 con 0x7f6cc4102de0 2024-04-25T02:05:24.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.912+0000 7f6cc2ffd700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cac06e140 0x7f6cac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:24.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.913+0000 7f6cc2ffd700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cac06e140 0x7f6cac0705f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4071990 tx=0x7f6cb8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:24.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:24.917+0000 7f6cabfff700 1 -- 172.21.15.33:0/1953116686 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6cb4056b20 con 0x7f6cc4102de0 2024-04-25T02:05:25.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.134+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cc40f9f90 con 0x7f6cac06e140 2024-04-25T02:05:25.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.144+0000 7f6cabfff700 1 -- 172.21.15.33:0/1953116686 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6cc40f9f90 con 0x7f6cac06e140 2024-04-25T02:05:25.150 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:05:25.150 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (26m) 4m ago 37m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (23m) 4m ago 37m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (23m) 4m ago 36m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (26m) 4m ago 37m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (19m) 4m ago 32m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (20m) 4m ago 32m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (19m) 4m ago 32m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (4m) 4m ago 32m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (28m) 4m ago 39m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (27m) 4m ago 36m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (24m) 4m ago 39m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (24m) 4m ago 36m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (27m) 4m ago 37m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (27m) 4m ago 36m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (23m) 4m ago 36m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:05:25.151 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (22m) 4m ago 35m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:05:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (22m) 4m ago 35m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:05:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (21m) 4m ago 34m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:05:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (20m) 4m ago 34m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:05:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (20m) 4m ago 33m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:05:25.152 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (26m) 4m ago 37m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.146+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cac06e140 msgr2=0x7f6cac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.146+0000 7f6cca22e700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cac06e140 0x7f6cac0705f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4071990 tx=0x7f6cb8009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.146+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102de0 msgr2=0x7f6cc4070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.146+0000 7f6cca22e700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102de0 0x7f6cc4070260 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400eb40 tx=0x7f6cb400b040 comp rx=0 tx=0).stop 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.146+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 shutdown_connections 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.146+0000 7f6cca22e700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cac06e140 0x7f6cac0705f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.146+0000 7f6cca22e700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6cc4102de0 0x7f6cc4070260 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.146+0000 7f6cca22e700 1 --2- 172.21.15.33:0/1953116686 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6cc4103780 0x7f6cc40707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.146+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 >> 172.21.15.33:0/1953116686 conn(0x7f6cc40f8990 msgr2=0x7f6cc40f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.147+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 shutdown_connections 2024-04-25T02:05:25.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.147+0000 7f6cca22e700 1 -- 172.21.15.33:0/1953116686 wait complete. 2024-04-25T02:05:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.258+0000 7fd87d918700 1 -- 172.21.15.33:0/3573445151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 msgr2=0x7fd878106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.258+0000 7fd87d918700 1 --2- 172.21.15.33:0/3573445151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 0x7fd878106df0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd860004940 tx=0x7fd860009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.258+0000 7fd87d918700 1 -- 172.21.15.33:0/3573445151 shutdown_connections 2024-04-25T02:05:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.258+0000 7fd87d918700 1 --2- 172.21.15.33:0/3573445151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 0x7fd878106df0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.258+0000 7fd87d918700 1 --2- 172.21.15.33:0/3573445151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8780fde10 0x7fd8780fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.258+0000 7fd87d918700 1 -- 172.21.15.33:0/3573445151 >> 172.21.15.33:0/3573445151 conn(0x7fd8780f7820 msgr2=0x7fd8780f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.258+0000 7fd87d918700 1 -- 172.21.15.33:0/3573445151 shutdown_connections 2024-04-25T02:05:25.265 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.258+0000 7fd87d918700 1 -- 172.21.15.33:0/3573445151 wait complete. 2024-04-25T02:05:25.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd87d918700 1 Processor -- start 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd87d918700 1 -- start start 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd87d918700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8780fde10 0x7fd8780fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd87d918700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 0x7fd8780fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd87d918700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780fdb90 con 0x7fd8780fe7b0 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd87d918700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780fc6c0 con 0x7fd8780fde10 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd8767fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 0x7fd8780fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd8767fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 0x7fd8780fc180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44518/0 (socket says 172.21.15.33:44518) 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd8767fc700 1 -- 172.21.15.33:0/812601178 learned_addr learned my addr 172.21.15.33:0/812601178 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd876ffd700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8780fde10 0x7fd8780fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd8767fc700 1 -- 172.21.15.33:0/812601178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8780fde10 msgr2=0x7fd8780fbc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd8767fc700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8780fde10 0x7fd8780fbc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd8767fc700 1 -- 172.21.15.33:0/812601178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd860005040 con 0x7fd8780fe7b0 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.259+0000 7fd876ffd700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8780fde10 0x7fd8780fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:05:25.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.260+0000 7fd8767fc700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 0x7fd8780fc180 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd860009380 tx=0x7fd860009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.260+0000 7fd86f7fe700 1 -- 172.21.15.33:0/812601178 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd86000e040 con 0x7fd8780fe7b0 2024-04-25T02:05:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.260+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8780fc940 con 0x7fd8780fe7b0 2024-04-25T02:05:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.260+0000 7fd86f7fe700 1 -- 172.21.15.33:0/812601178 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd860009af0 con 0x7fd8780fe7b0 2024-04-25T02:05:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.260+0000 7fd86f7fe700 1 -- 172.21.15.33:0/812601178 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd86001c630 con 0x7fd8780fe7b0 2024-04-25T02:05:25.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.260+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87811a340 con 0x7fd8780fe7b0 2024-04-25T02:05:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.261+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd878066b30 con 0x7fd8780fe7b0 2024-04-25T02:05:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.262+0000 7fd86f7fe700 1 -- 172.21.15.33:0/812601178 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd86000bbc0 con 0x7fd8780fe7b0 2024-04-25T02:05:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.262+0000 7fd86f7fe700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd86406e140 0x7fd8640705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.262+0000 7fd86f7fe700 1 -- 172.21.15.33:0/812601178 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd86008ee60 con 0x7fd8780fe7b0 2024-04-25T02:05:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.262+0000 7fd876ffd700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd86406e140 0x7fd8640705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:25.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.262+0000 7fd876ffd700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd86406e140 0x7fd8640705f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd870007fd0 tx=0x7fd870009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:25.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.267+0000 7fd86f7fe700 1 -- 172.21.15.33:0/812601178 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd860058260 con 0x7fd8780fe7b0 2024-04-25T02:05:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.564+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd878065f30 con 0x7fd8780fe7b0 2024-04-25T02:05:25.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.565+0000 7fd86f7fe700 1 -- 172.21.15.33:0/812601178 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd86005b920 con 0x7fd8780fe7b0 2024-04-25T02:05:25.572 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:05:25.572 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:05:25.572 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:05:25.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:05:25.572 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:05:25.572 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:05:25.572 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:05:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:05:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:05:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:05:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:05:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:05:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:05:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:05:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:05:25.573 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:05:25.574 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:05:25.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.567+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd86406e140 msgr2=0x7fd8640705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.567+0000 7fd87d918700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd86406e140 0x7fd8640705f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd870007fd0 tx=0x7fd870009040 comp rx=0 tx=0).stop 2024-04-25T02:05:25.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.567+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 msgr2=0x7fd8780fc180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.567+0000 7fd87d918700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 0x7fd8780fc180 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd860009380 tx=0x7fd860009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:25.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.567+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 shutdown_connections 2024-04-25T02:05:25.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.567+0000 7fd87d918700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd86406e140 0x7fd8640705f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.567+0000 7fd87d918700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8780fde10 0x7fd8780fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.567+0000 7fd87d918700 1 --2- 172.21.15.33:0/812601178 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8780fe7b0 0x7fd8780fc180 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.567+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 >> 172.21.15.33:0/812601178 conn(0x7fd8780f7820 msgr2=0x7fd8780f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:25.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.568+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 shutdown_connections 2024-04-25T02:05:25.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.568+0000 7fd87d918700 1 -- 172.21.15.33:0/812601178 wait complete. 2024-04-25T02:05:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.671+0000 7f179af1e700 1 -- 172.21.15.33:0/4085107747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1794101d90 msgr2=0x7f1794102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.671+0000 7f179af1e700 1 --2- 172.21.15.33:0/4085107747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1794101d90 0x7f1794102160 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f178c009ce0 tx=0x7f178c004d10 comp rx=0 tx=0).stop 2024-04-25T02:05:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.671+0000 7f179af1e700 1 -- 172.21.15.33:0/4085107747 shutdown_connections 2024-04-25T02:05:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.671+0000 7f179af1e700 1 --2- 172.21.15.33:0/4085107747 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17940fbd40 0x7f17940fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.671+0000 7f179af1e700 1 --2- 172.21.15.33:0/4085107747 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1794101d90 0x7f1794102160 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.671+0000 7f179af1e700 1 -- 172.21.15.33:0/4085107747 >> 172.21.15.33:0/4085107747 conn(0x7f1794071d00 msgr2=0x7f1794072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.671+0000 7f179af1e700 1 -- 172.21.15.33:0/4085107747 shutdown_connections 2024-04-25T02:05:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.671+0000 7f179af1e700 1 -- 172.21.15.33:0/4085107747 wait complete. 2024-04-25T02:05:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f179af1e700 1 Processor -- start 2024-04-25T02:05:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f179af1e700 1 -- start start 2024-04-25T02:05:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f179af1e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17940fbd40 0x7f1794102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:25.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f179af1e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1794101d90 0x7f1794103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f179af1e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17941035f0 con 0x7f17940fbd40 2024-04-25T02:05:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f179af1e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1794103730 con 0x7f1794101d90 2024-04-25T02:05:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f1798cba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17940fbd40 0x7f1794102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f1798cba700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17940fbd40 0x7f1794102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50510/0 (socket says 172.21.15.33:50510) 2024-04-25T02:05:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f1798cba700 1 -- 172.21.15.33:0/3915134939 learned_addr learned my addr 172.21.15.33:0/3915134939 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:25.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f1793fff700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1794101d90 0x7f1794103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f1798cba700 1 -- 172.21.15.33:0/3915134939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1794101d90 msgr2=0x7f1794103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f1798cba700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1794101d90 0x7f1794103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.672+0000 7f1798cba700 1 -- 172.21.15.33:0/3915134939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c009940 con 0x7f17940fbd40 2024-04-25T02:05:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.673+0000 7f1798cba700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17940fbd40 0x7f1794102b20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f178c00be90 tx=0x7f178c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.673+0000 7f17917fa700 1 -- 172.21.15.33:0/3915134939 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f178c00e040 con 0x7f17940fbd40 2024-04-25T02:05:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.673+0000 7f17917fa700 1 -- 172.21.15.33:0/3915134939 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f178c008ea0 con 0x7f17940fbd40 2024-04-25T02:05:25.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.673+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1794103980 con 0x7f17940fbd40 2024-04-25T02:05:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.673+0000 7f17917fa700 1 -- 172.21.15.33:0/3915134939 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f178c01c940 con 0x7f17940fbd40 2024-04-25T02:05:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.673+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179411a3d0 con 0x7f17940fbd40 2024-04-25T02:05:25.684 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.674+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17940fd430 con 0x7f17940fbd40 2024-04-25T02:05:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.674+0000 7f17917fa700 1 -- 172.21.15.33:0/3915134939 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f178c012d00 con 0x7f17940fbd40 2024-04-25T02:05:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.675+0000 7f17917fa700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f177c06e140 0x7f177c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.675+0000 7f17917fa700 1 -- 172.21.15.33:0/3915134939 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f178c08f650 con 0x7f17940fbd40 2024-04-25T02:05:25.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.675+0000 7f1793fff700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f177c06e140 0x7f177c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.676+0000 7f1793fff700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f177c06e140 0x7f177c0705f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1788007fd0 tx=0x7f1788009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:25.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.680+0000 7f17917fa700 1 -- 172.21.15.33:0/3915134939 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f178c058a50 con 0x7f17940fbd40 2024-04-25T02:05:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.934+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f179404bed0 con 0x7f17940fbd40 2024-04-25T02:05:25.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.934+0000 7f17917fa700 1 -- 172.21.15.33:0/3915134939 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f178c004e10 con 0x7f17940fbd40 2024-04-25T02:05:25.940 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:05:25.940 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:05:25.941 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:05:25.942 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:05:25.943 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:05:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f177c06e140 msgr2=0x7f177c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f177c06e140 0x7f177c0705f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1788007fd0 tx=0x7f1788009040 comp rx=0 tx=0).stop 2024-04-25T02:05:25.944 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17940fbd40 msgr2=0x7f1794102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17940fbd40 0x7f1794102b20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f178c00be90 tx=0x7f178c004d10 comp rx=0 tx=0).stop 2024-04-25T02:05:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 shutdown_connections 2024-04-25T02:05:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f177c06e140 0x7f177c0705f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17940fbd40 0x7f1794102b20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 --2- 172.21.15.33:0/3915134939 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1794101d90 0x7f1794103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 >> 172.21.15.33:0/3915134939 conn(0x7f1794071d00 msgr2=0x7f1794065610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 shutdown_connections 2024-04-25T02:05:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:25.936+0000 7f179af1e700 1 -- 172.21.15.33:0/3915134939 wait complete. 2024-04-25T02:05:25.945 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:05:26.048 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 -- 172.21.15.33:0/207413112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 msgr2=0x7fefa8103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 --2- 172.21.15.33:0/207413112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 0x7fefa8103330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fef9c004940 tx=0x7fef9c009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 -- 172.21.15.33:0/207413112 shutdown_connections 2024-04-25T02:05:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 --2- 172.21.15.33:0/207413112 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa80fb520 0x7fefa80fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 --2- 172.21.15.33:0/207413112 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 0x7fefa8103330 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 -- 172.21.15.33:0/207413112 >> 172.21.15.33:0/207413112 conn(0x7fefa80714d0 msgr2=0x7fefa80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 -- 172.21.15.33:0/207413112 shutdown_connections 2024-04-25T02:05:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 -- 172.21.15.33:0/207413112 wait complete. 2024-04-25T02:05:26.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 Processor -- start 2024-04-25T02:05:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.042+0000 7fefae934700 1 -- start start 2024-04-25T02:05:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefae934700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa80fb520 0x7fefa80fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefae934700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 0x7fefa80fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefae934700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa80fe650 con 0x7fefa8102f60 2024-04-25T02:05:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefae934700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa80fd180 con 0x7fefa80fb520 2024-04-25T02:05:26.051 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 0x7fefa80fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 0x7fefa80fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50540/0 (socket says 172.21.15.33:50540) 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa77fe700 1 -- 172.21.15.33:0/2629721560 learned_addr learned my addr 172.21.15.33:0/2629721560 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa7fff700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa80fb520 0x7fefa80fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa77fe700 1 -- 172.21.15.33:0/2629721560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa80fb520 msgr2=0x7fefa80fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa77fe700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa80fb520 0x7fefa80fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa77fe700 1 -- 172.21.15.33:0/2629721560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c005040 con 0x7fefa8102f60 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa77fe700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 0x7fefa80fcc40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fef98002a50 tx=0x7fef980046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa4ff9700 1 -- 172.21.15.33:0/2629721560 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef98007cc0 con 0x7fefa8102f60 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa80fd460 con 0x7fefa8102f60 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa4ff9700 1 -- 172.21.15.33:0/2629721560 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef9800f040 con 0x7fefa8102f60 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.043+0000 7fefa4ff9700 1 -- 172.21.15.33:0/2629721560 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef98009640 con 0x7fefa8102f60 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.044+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa811a0f0 con 0x7fefa8102f60 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.044+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa8100250 con 0x7fefa8102f60 2024-04-25T02:05:26.052 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.045+0000 7fefa4ff9700 1 -- 172.21.15.33:0/2629721560 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fef980097a0 con 0x7fefa8102f60 2024-04-25T02:05:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.045+0000 7fefa4ff9700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef9006e140 0x7fef900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.046+0000 7fefa4ff9700 1 -- 172.21.15.33:0/2629721560 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fef9808d920 con 0x7fefa8102f60 2024-04-25T02:05:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.046+0000 7fefa7fff700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef9006e140 0x7fef900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:26.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.046+0000 7fefa7fff700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef9006e140 0x7fef900705f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fef9c004f90 tx=0x7fef9c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:26.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.050+0000 7fefa4ff9700 1 -- 172.21.15.33:0/2629721560 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fef98056ca0 con 0x7fefa8102f60 2024-04-25T02:05:26.285 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.279+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fefa810a050 con 0x7fef9006e140 2024-04-25T02:05:26.286 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.280+0000 7fefa4ff9700 1 -- 172.21.15.33:0/2629721560 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fefa810a050 con 0x7fef9006e140 2024-04-25T02:05:26.286 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:05:26.286 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:05:26.286 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:05:26.286 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:05:26.286 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:05:26.286 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:05:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:05:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:05:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:05:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:05:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:05:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:05:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:05:26.287 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:05:26.287 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:05:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef9006e140 msgr2=0x7fef900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef9006e140 0x7fef900705f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fef9c004f90 tx=0x7fef9c009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 msgr2=0x7fefa80fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 0x7fefa80fcc40 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fef98002a50 tx=0x7fef980046b0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.288 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 shutdown_connections 2024-04-25T02:05:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef9006e140 0x7fef900705f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fefa80fb520 0x7fefa80fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 --2- 172.21.15.33:0/2629721560 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fefa8102f60 0x7fefa80fcc40 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 >> 172.21.15.33:0/2629721560 conn(0x7fefa80714d0 msgr2=0x7fefa80fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 shutdown_connections 2024-04-25T02:05:26.289 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.282+0000 7fefae934700 1 -- 172.21.15.33:0/2629721560 wait complete. 2024-04-25T02:05:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.386+0000 7f305fc36700 1 -- 172.21.15.33:0/1731494809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 msgr2=0x7f30581020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.386+0000 7f305fc36700 1 --2- 172.21.15.33:0/1731494809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 0x7f30581020d0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3048004940 tx=0x7f3048009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.386+0000 7f305fc36700 1 -- 172.21.15.33:0/1731494809 shutdown_connections 2024-04-25T02:05:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.386+0000 7f305fc36700 1 --2- 172.21.15.33:0/1731494809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30581026a0 0x7f30580fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.386+0000 7f305fc36700 1 --2- 172.21.15.33:0/1731494809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 0x7f30581020d0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.386+0000 7f305fc36700 1 -- 172.21.15.33:0/1731494809 >> 172.21.15.33:0/1731494809 conn(0x7f30580f7870 msgr2=0x7f30580f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.387+0000 7f305fc36700 1 -- 172.21.15.33:0/1731494809 shutdown_connections 2024-04-25T02:05:26.393 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.387+0000 7f305fc36700 1 -- 172.21.15.33:0/1731494809 wait complete. 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.387+0000 7f305fc36700 1 Processor -- start 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.387+0000 7f305fc36700 1 -- start start 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.387+0000 7f305fc36700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 0x7f3058115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.387+0000 7f305fc36700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30581026a0 0x7f30581160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.387+0000 7f305fc36700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3058116710 con 0x7f3058101d00 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.387+0000 7f305fc36700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305810fcf0 con 0x7f30581026a0 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305d9d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 0x7f3058115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305d9d2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 0x7f3058115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:50564/0 (socket says 172.21.15.33:50564) 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305d9d2700 1 -- 172.21.15.33:0/1550083708 learned_addr learned my addr 172.21.15.33:0/1550083708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:26.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305d1d1700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30581026a0 0x7f30581160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305d9d2700 1 -- 172.21.15.33:0/1550083708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30581026a0 msgr2=0x7f30581160c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305d9d2700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30581026a0 0x7f30581160c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305d9d2700 1 -- 172.21.15.33:0/1550083708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048005040 con 0x7f3058101d00 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305d1d1700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30581026a0 0x7f30581160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305d9d2700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 0x7f3058115b80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f304800b3b0 tx=0x7f3048009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f304e7fc700 1 -- 172.21.15.33:0/1550083708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304800e040 con 0x7f3058101d00 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f304e7fc700 1 -- 172.21.15.33:0/1550083708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f304800bbc0 con 0x7f3058101d00 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f304e7fc700 1 -- 172.21.15.33:0/1550083708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f304801c3f0 con 0x7f3058101d00 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305810ff70 con 0x7f3058101d00 2024-04-25T02:05:26.396 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.388+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3058110530 con 0x7f3058101d00 2024-04-25T02:05:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.389+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3058106200 con 0x7f3058101d00 2024-04-25T02:05:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.390+0000 7f304e7fc700 1 -- 172.21.15.33:0/1550083708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f304800bd30 con 0x7f3058101d00 2024-04-25T02:05:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.390+0000 7f304e7fc700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f304406e140 0x7f30440705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.390+0000 7f304e7fc700 1 -- 172.21.15.33:0/1550083708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f304808edc0 con 0x7f3058101d00 2024-04-25T02:05:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.390+0000 7f305d1d1700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f304406e140 0x7f30440705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:26.397 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.391+0000 7f305d1d1700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f304406e140 0x7f30440705f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3054009fd0 tx=0x7f3054009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:26.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.395+0000 7f304e7fc700 1 -- 172.21.15.33:0/1550083708 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f304805bc70 con 0x7f3058101d00 2024-04-25T02:05:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:26 smithi033 ceph-mon[111306]: from='client.35440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:26 smithi033 ceph-mon[111306]: from='client.35444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:26 smithi033 ceph-mon[111306]: from='client.35448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:26 smithi033 ceph-mon[111306]: pgmap v782: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:05:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:26 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/812601178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:05:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:26 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3915134939' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:05:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:26 smithi138 ceph-mon[81304]: from='client.35440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:26 smithi138 ceph-mon[81304]: from='client.35444 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:26 smithi138 ceph-mon[81304]: from='client.35448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:26 smithi138 ceph-mon[81304]: pgmap v782: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:05:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:26 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/812601178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:05:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:26 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3915134939' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:05:26.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.690+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f30580fd450 con 0x7f3058101d00 2024-04-25T02:05:26.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.690+0000 7f304e7fc700 1 -- 172.21.15.33:0/1550083708 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f304805b800 con 0x7f3058101d00 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:05:26.697 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:05:26.698 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:05:26.698 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:05:26.698 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:05:26.698 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:05:26.698 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:05:26.698 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:05:26.698 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:05:26.698 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:05:26.698 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f304406e140 msgr2=0x7f30440705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f304406e140 0x7f30440705f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3054009fd0 tx=0x7f3054009ee0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 msgr2=0x7f3058115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 0x7f3058115b80 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f304800b3b0 tx=0x7f3048009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 shutdown_connections 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f304406e140 0x7f30440705f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3058101d00 0x7f3058115b80 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 --2- 172.21.15.33:0/1550083708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f30581026a0 0x7f30581160c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 >> 172.21.15.33:0/1550083708 conn(0x7f30580f7870 msgr2=0x7f30580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.692+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 shutdown_connections 2024-04-25T02:05:26.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:26.693+0000 7f305fc36700 1 -- 172.21.15.33:0/1550083708 wait complete. 2024-04-25T02:05:27.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:27 smithi033 ceph-mon[111306]: from='client.35460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:27.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:27 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1550083708' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:05:27.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:27 smithi138 ceph-mon[81304]: from='client.35460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:27.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:27 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1550083708' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:05:28.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:28 smithi033 ceph-mon[111306]: pgmap v783: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:28.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:28 smithi138 ceph-mon[81304]: pgmap v783: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:30.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:30 smithi033 ceph-mon[111306]: pgmap v784: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:30.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:30 smithi138 ceph-mon[81304]: pgmap v784: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:32.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:32 smithi033 ceph-mon[111306]: pgmap v785: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:32.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:32 smithi138 ceph-mon[81304]: pgmap v785: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:34.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:34 smithi033 ceph-mon[111306]: pgmap v786: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:34.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:34 smithi138 ceph-mon[81304]: pgmap v786: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:36.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:36 smithi033 ceph-mon[111306]: pgmap v787: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:36.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:36 smithi138 ceph-mon[81304]: pgmap v787: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:38.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:38 smithi033 ceph-mon[111306]: pgmap v788: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:38.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:38 smithi138 ceph-mon[81304]: pgmap v788: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:40.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:40 smithi033 ceph-mon[111306]: pgmap v789: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:40.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:40 smithi138 ceph-mon[81304]: pgmap v789: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:41.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:05:41.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:05:42.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:42 smithi033 ceph-mon[111306]: pgmap v790: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:42.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:42 smithi138 ceph-mon[81304]: pgmap v790: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:44.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:44 smithi033 ceph-mon[111306]: pgmap v791: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:44.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:44 smithi138 ceph-mon[81304]: pgmap v791: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:46.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:46 smithi033 ceph-mon[111306]: pgmap v792: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:46.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:46 smithi138 ceph-mon[81304]: pgmap v792: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:48.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:48 smithi033 ceph-mon[111306]: pgmap v793: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:48.549 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:48 smithi138 ceph-mon[81304]: pgmap v793: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:50.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:50 smithi033 ceph-mon[111306]: pgmap v794: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:50.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:50 smithi138 ceph-mon[81304]: pgmap v794: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:52.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:52 smithi033 ceph-mon[111306]: pgmap v795: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:52.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:52 smithi138 ceph-mon[81304]: pgmap v795: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:54.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:54 smithi033 ceph-mon[111306]: pgmap v796: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:54.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:54 smithi138 ceph-mon[81304]: pgmap v796: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:56.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:56 smithi138 ceph-mon[81304]: pgmap v797: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:56.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:05:56.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:56 smithi033 ceph-mon[111306]: pgmap v797: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:56.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.811+0000 7f197ed73700 1 -- 172.21.15.33:0/1865158370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1978101d10 msgr2=0x7f19781020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.811+0000 7f197ed73700 1 --2- 172.21.15.33:0/1865158370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1978101d10 0x7f19781020e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f196c004940 tx=0x7f196c009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.811+0000 7f197ed73700 1 -- 172.21.15.33:0/1865158370 shutdown_connections 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.811+0000 7f197ed73700 1 --2- 172.21.15.33:0/1865158370 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f19781026b0 0x7f19780fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.811+0000 7f197ed73700 1 --2- 172.21.15.33:0/1865158370 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1978101d10 0x7f19781020e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.811+0000 7f197ed73700 1 -- 172.21.15.33:0/1865158370 >> 172.21.15.33:0/1865158370 conn(0x7f19780f7860 msgr2=0x7f19780f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.811+0000 7f197ed73700 1 -- 172.21.15.33:0/1865158370 shutdown_connections 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.811+0000 7f197ed73700 1 -- 172.21.15.33:0/1865158370 wait complete. 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.811+0000 7f197ed73700 1 Processor -- start 2024-04-25T02:05:56.818 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f197ed73700 1 -- start start 2024-04-25T02:05:56.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f197ed73700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978101d10 0x7f1978106bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:56.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f197ed73700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19781026b0 0x7f19781070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:56.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f197ed73700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1978108b00 con 0x7f19781026b0 2024-04-25T02:05:56.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f197ed73700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1978107630 con 0x7f1978101d10 2024-04-25T02:05:56.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f1977fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19781026b0 0x7f19781070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:56.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f1977fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19781026b0 0x7f19781070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42066/0 (socket says 172.21.15.33:42066) 2024-04-25T02:05:56.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f1977fff700 1 -- 172.21.15.33:0/3955229005 learned_addr learned my addr 172.21.15.33:0/3955229005 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:56.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f197cb0f700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978101d10 0x7f1978106bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:56.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f1977fff700 1 -- 172.21.15.33:0/3955229005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978101d10 msgr2=0x7f1978106bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:56.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f1977fff700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978101d10 0x7f1978106bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:56.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.812+0000 7f1977fff700 1 -- 172.21.15.33:0/3955229005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196c005040 con 0x7f19781026b0 2024-04-25T02:05:56.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.813+0000 7f1977fff700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19781026b0 0x7f19781070f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1964002a80 tx=0x7f1964004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:56.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.813+0000 7f19757fa700 1 -- 172.21.15.33:0/3955229005 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1964004e80 con 0x7f19781026b0 2024-04-25T02:05:56.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.813+0000 7f19757fa700 1 -- 172.21.15.33:0/3955229005 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f196400f040 con 0x7f19781026b0 2024-04-25T02:05:56.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.813+0000 7f19757fa700 1 -- 172.21.15.33:0/3955229005 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f196400daa0 con 0x7f19781026b0 2024-04-25T02:05:56.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.813+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1978107910 con 0x7f19781026b0 2024-04-25T02:05:56.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.813+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197811a140 con 0x7f19781026b0 2024-04-25T02:05:56.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.814+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19780ff020 con 0x7f19781026b0 2024-04-25T02:05:56.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.815+0000 7f19757fa700 1 -- 172.21.15.33:0/3955229005 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1964007cc0 con 0x7f19781026b0 2024-04-25T02:05:56.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.815+0000 7f19757fa700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f196806e140 0x7f19680705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:56.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.815+0000 7f19757fa700 1 -- 172.21.15.33:0/3955229005 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f196408d140 con 0x7f19781026b0 2024-04-25T02:05:56.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.815+0000 7f197cb0f700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f196806e140 0x7f19680705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:56.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.816+0000 7f197cb0f700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f196806e140 0x7f19680705f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f196c009fd0 tx=0x7f196c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:56.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:56.820+0000 7f19757fa700 1 -- 172.21.15.33:0/3955229005 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f19640564c0 con 0x7f19781026b0 2024-04-25T02:05:57.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.049+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1978109f30 con 0x7f196806e140 2024-04-25T02:05:57.056 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.050+0000 7f19757fa700 1 -- 172.21.15.33:0/3955229005 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1978109f30 con 0x7f196806e140 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f196806e140 msgr2=0x7f19680705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f196806e140 0x7f19680705f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f196c009fd0 tx=0x7f196c009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19781026b0 msgr2=0x7f19781070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19781026b0 0x7f19781070f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1964002a80 tx=0x7f1964004630 comp rx=0 tx=0).stop 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 shutdown_connections 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f196806e140 0x7f19680705f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1978101d10 0x7f1978106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 --2- 172.21.15.33:0/3955229005 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f19781026b0 0x7f19781070f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 >> 172.21.15.33:0/3955229005 conn(0x7f19780f7860 msgr2=0x7f1978106300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 shutdown_connections 2024-04-25T02:05:57.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.052+0000 7f197ed73700 1 -- 172.21.15.33:0/3955229005 wait complete. 2024-04-25T02:05:57.069 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.158+0000 7feb6f0dd700 1 -- 172.21.15.33:0/1417095568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 msgr2=0x7feb681020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.158+0000 7feb6f0dd700 1 --2- 172.21.15.33:0/1417095568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 0x7feb681020e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7feb5c004940 tx=0x7feb5c009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.158+0000 7feb6f0dd700 1 -- 172.21.15.33:0/1417095568 shutdown_connections 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.158+0000 7feb6f0dd700 1 --2- 172.21.15.33:0/1417095568 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb681026b0 0x7feb680fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.158+0000 7feb6f0dd700 1 --2- 172.21.15.33:0/1417095568 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 0x7feb681020e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.158+0000 7feb6f0dd700 1 -- 172.21.15.33:0/1417095568 >> 172.21.15.33:0/1417095568 conn(0x7feb680f7860 msgr2=0x7feb680f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.158+0000 7feb6f0dd700 1 -- 172.21.15.33:0/1417095568 shutdown_connections 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.158+0000 7feb6f0dd700 1 -- 172.21.15.33:0/1417095568 wait complete. 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb6f0dd700 1 Processor -- start 2024-04-25T02:05:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb6f0dd700 1 -- start start 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb6f0dd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 0x7feb6810c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb6f0dd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb681026b0 0x7feb6810cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb6f0dd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb68067100 con 0x7feb68101d10 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb6f0dd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6810d3e0 con 0x7feb681026b0 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb6ce79700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 0x7feb6810c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb6ce79700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 0x7feb6810c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42084/0 (socket says 172.21.15.33:42084) 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb6ce79700 1 -- 172.21.15.33:0/3567655629 learned_addr learned my addr 172.21.15.33:0/3567655629 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.159+0000 7feb67fff700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb681026b0 0x7feb6810cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.160+0000 7feb6ce79700 1 -- 172.21.15.33:0/3567655629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb681026b0 msgr2=0x7feb6810cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.160+0000 7feb6ce79700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb681026b0 0x7feb6810cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.160+0000 7feb6ce79700 1 -- 172.21.15.33:0/3567655629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c005040 con 0x7feb68101d10 2024-04-25T02:05:57.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.160+0000 7feb6ce79700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 0x7feb6810c960 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00b3b0 tx=0x7feb5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.160+0000 7feb657fa700 1 -- 172.21.15.33:0/3567655629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb5c00e040 con 0x7feb68101d10 2024-04-25T02:05:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.160+0000 7feb657fa700 1 -- 172.21.15.33:0/3567655629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb5c00bbc0 con 0x7feb68101d10 2024-04-25T02:05:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.160+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6810d660 con 0x7feb68101d10 2024-04-25T02:05:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.160+0000 7feb657fa700 1 -- 172.21.15.33:0/3567655629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb5c01c3f0 con 0x7feb68101d10 2024-04-25T02:05:57.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.160+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6811a190 con 0x7feb68101d10 2024-04-25T02:05:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.161+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb680fd460 con 0x7feb68101d10 2024-04-25T02:05:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.162+0000 7feb657fa700 1 -- 172.21.15.33:0/3567655629 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feb5c00bd30 con 0x7feb68101d10 2024-04-25T02:05:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.162+0000 7feb657fa700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb5806e140 0x7feb580705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.162+0000 7feb67fff700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb5806e140 0x7feb580705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.162+0000 7feb657fa700 1 -- 172.21.15.33:0/3567655629 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7feb5c08ed70 con 0x7feb68101d10 2024-04-25T02:05:57.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.163+0000 7feb67fff700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb5806e140 0x7feb580705f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feb54007fd0 tx=0x7feb54009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:57.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.167+0000 7feb657fa700 1 -- 172.21.15.33:0/3567655629 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feb5c05bc20 con 0x7feb68101d10 2024-04-25T02:05:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.395+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb680f8f50 con 0x7feb5806e140 2024-04-25T02:05:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.396+0000 7feb657fa700 1 -- 172.21.15.33:0/3567655629 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7feb680f8f50 con 0x7feb5806e140 2024-04-25T02:05:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb5806e140 msgr2=0x7feb580705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb5806e140 0x7feb580705f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7feb54007fd0 tx=0x7feb54009040 comp rx=0 tx=0).stop 2024-04-25T02:05:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 msgr2=0x7feb6810c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 0x7feb6810c960 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00b3b0 tx=0x7feb5c009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 shutdown_connections 2024-04-25T02:05:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7feb5806e140 0x7feb580705f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7feb68101d10 0x7feb6810c960 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 --2- 172.21.15.33:0/3567655629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7feb681026b0 0x7feb6810cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 >> 172.21.15.33:0/3567655629 conn(0x7feb680f7860 msgr2=0x7feb680f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 shutdown_connections 2024-04-25T02:05:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.398+0000 7feb6f0dd700 1 -- 172.21.15.33:0/3567655629 wait complete. 2024-04-25T02:05:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.502+0000 7f411e3f1700 1 -- 172.21.15.33:0/3027195936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fc790 msgr2=0x7f41180fcb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.502+0000 7f411e3f1700 1 --2- 172.21.15.33:0/3027195936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fc790 0x7f41180fcb60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f4108004940 tx=0x7f4108009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.502+0000 7f411e3f1700 1 -- 172.21.15.33:0/3027195936 shutdown_connections 2024-04-25T02:05:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.502+0000 7f411e3f1700 1 --2- 172.21.15.33:0/3027195936 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41180fd0a0 0x7f4118109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.502+0000 7f411e3f1700 1 --2- 172.21.15.33:0/3027195936 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fc790 0x7f41180fcb60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.502+0000 7f411e3f1700 1 -- 172.21.15.33:0/3027195936 >> 172.21.15.33:0/3027195936 conn(0x7f4118071d00 msgr2=0x7f4118072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.502+0000 7f411e3f1700 1 -- 172.21.15.33:0/3027195936 shutdown_connections 2024-04-25T02:05:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.502+0000 7f411e3f1700 1 -- 172.21.15.33:0/3027195936 wait complete. 2024-04-25T02:05:57.509 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.502+0000 7f411e3f1700 1 Processor -- start 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f411e3f1700 1 -- start start 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f411e3f1700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41180fc790 0x7f4118115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f411e3f1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fd0a0 0x7f41181162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f411e3f1700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4118116930 con 0x7f41180fd0a0 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f411e3f1700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411810fd40 con 0x7f41180fc790 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f41177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fd0a0 0x7f41181162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f41177fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fd0a0 0x7f41181162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42094/0 (socket says 172.21.15.33:42094) 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f41177fe700 1 -- 172.21.15.33:0/648452498 learned_addr learned my addr 172.21.15.33:0/648452498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f4117fff700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41180fc790 0x7f4118115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f41177fe700 1 -- 172.21.15.33:0/648452498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41180fc790 msgr2=0x7f4118115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f41177fe700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41180fc790 0x7f4118115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f41177fe700 1 -- 172.21.15.33:0/648452498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4108005040 con 0x7f41180fd0a0 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.503+0000 7f41177fe700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fd0a0 0x7f41181162e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f410c002a50 tx=0x7f410c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:57.511 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.504+0000 7f4114ff9700 1 -- 172.21.15.33:0/648452498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f410c007cc0 con 0x7f41180fd0a0 2024-04-25T02:05:57.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.504+0000 7f4114ff9700 1 -- 172.21.15.33:0/648452498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f410c00f040 con 0x7f41180fd0a0 2024-04-25T02:05:57.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.504+0000 7f4114ff9700 1 -- 172.21.15.33:0/648452498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f410c00da30 con 0x7f41180fd0a0 2024-04-25T02:05:57.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.504+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4118110020 con 0x7f41180fd0a0 2024-04-25T02:05:57.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.504+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4118110560 con 0x7f41180fd0a0 2024-04-25T02:05:57.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.505+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41180fc330 con 0x7f41180fd0a0 2024-04-25T02:05:57.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.505+0000 7f4114ff9700 1 -- 172.21.15.33:0/648452498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f410c00db90 con 0x7f41180fd0a0 2024-04-25T02:05:57.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.506+0000 7f4114ff9700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f410006e140 0x7f41000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:57.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.506+0000 7f4114ff9700 1 -- 172.21.15.33:0/648452498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f410c08d960 con 0x7f41180fd0a0 2024-04-25T02:05:57.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.506+0000 7f4117fff700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f410006e140 0x7f41000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:57.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.506+0000 7f4117fff700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f410006e140 0x7f41000705f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4108004f90 tx=0x7f4108009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:57.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.510+0000 7f4114ff9700 1 -- 172.21.15.33:0/648452498 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f410c056c40 con 0x7f41180fd0a0 2024-04-25T02:05:57.736 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.730+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f411810c100 con 0x7f410006e140 2024-04-25T02:05:57.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.740+0000 7f4114ff9700 1 -- 172.21.15.33:0/648452498 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f411810c100 con 0x7f410006e140 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (27m) 4m ago 38m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (24m) 4m ago 38m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (24m) 4m ago 37m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (26m) 4m ago 37m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (20m) 4m ago 32m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (20m) 4m ago 32m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (20m) 4m ago 32m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (4m) 4m ago 32m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:05:57.750 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (28m) 4m ago 39m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (28m) 4m ago 37m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (25m) 4m ago 39m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (25m) 4m ago 37m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (27m) 4m ago 38m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (27m) 4m ago 37m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (23m) 4m ago 36m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (23m) 4m ago 36m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (22m) 4m ago 35m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (22m) 4m ago 35m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (21m) 4m ago 34m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (20m) 4m ago 34m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:05:57.751 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (27m) 4m ago 37m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:05:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f410006e140 msgr2=0x7f41000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f410006e140 0x7f41000705f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4108004f90 tx=0x7f4108009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fd0a0 msgr2=0x7f41181162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fd0a0 0x7f41181162e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f410c002a50 tx=0x7f410c0046b0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 shutdown_connections 2024-04-25T02:05:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f410006e140 0x7f41000705f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f41180fc790 0x7f4118115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 --2- 172.21.15.33:0/648452498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f41180fd0a0 0x7f41181162e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.752 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 >> 172.21.15.33:0/648452498 conn(0x7f4118071d00 msgr2=0x7f41180f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:57.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 shutdown_connections 2024-04-25T02:05:57.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.743+0000 7f411e3f1700 1 -- 172.21.15.33:0/648452498 wait complete. 2024-04-25T02:05:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2457336925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 msgr2=0x7fb5bc1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 --2- 172.21.15.33:0/2457336925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 0x7fb5bc1031d0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8004940 tx=0x7fb5b8009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2457336925 shutdown_connections 2024-04-25T02:05:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 --2- 172.21.15.33:0/2457336925 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5bc1037a0 0x7fb5bc0fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 --2- 172.21.15.33:0/2457336925 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 0x7fb5bc1031d0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2457336925 >> 172.21.15.33:0/2457336925 conn(0x7fb5bc0f89d0 msgr2=0x7fb5bc0fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2457336925 shutdown_connections 2024-04-25T02:05:57.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2457336925 wait complete. 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 Processor -- start 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.846+0000 7fb5c3a4b700 1 -- start start 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c3a4b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 0x7fb5bc113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c3a4b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5bc1037a0 0x7fb5bc113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c3a4b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc114510 con 0x7fb5bc102e00 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c3a4b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5bc10db10 con 0x7fb5bc1037a0 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c17e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 0x7fb5bc113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c17e7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 0x7fb5bc113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42098/0 (socket says 172.21.15.33:42098) 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c17e7700 1 -- 172.21.15.33:0/2959023501 learned_addr learned my addr 172.21.15.33:0/2959023501 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c0fe6700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5bc1037a0 0x7fb5bc113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c17e7700 1 -- 172.21.15.33:0/2959023501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5bc1037a0 msgr2=0x7fb5bc113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c17e7700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5bc1037a0 0x7fb5bc113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c17e7700 1 -- 172.21.15.33:0/2959023501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b8005040 con 0x7fb5bc102e00 2024-04-25T02:05:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5c17e7700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 0x7fb5bc113980 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009150 tx=0x7fb5b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5b1ffb700 1 -- 172.21.15.33:0/2959023501 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b800e040 con 0x7fb5bc102e00 2024-04-25T02:05:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.847+0000 7fb5b1ffb700 1 -- 172.21.15.33:0/2959023501 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5b8013420 con 0x7fb5bc102e00 2024-04-25T02:05:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.848+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5bc10dd90 con 0x7fb5bc102e00 2024-04-25T02:05:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.848+0000 7fb5b1ffb700 1 -- 172.21.15.33:0/2959023501 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5b801c630 con 0x7fb5bc102e00 2024-04-25T02:05:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.848+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5bc10e350 con 0x7fb5bc102e00 2024-04-25T02:05:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.849+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5bc100110 con 0x7fb5bc102e00 2024-04-25T02:05:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.849+0000 7fb5b1ffb700 1 -- 172.21.15.33:0/2959023501 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb5b8013590 con 0x7fb5bc102e00 2024-04-25T02:05:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.850+0000 7fb5b1ffb700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb5a806e140 0x7fb5a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.850+0000 7fb5b1ffb700 1 -- 172.21.15.33:0/2959023501 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb5b808ec90 con 0x7fb5bc102e00 2024-04-25T02:05:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.850+0000 7fb5c0fe6700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb5a806e140 0x7fb5a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:57.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.850+0000 7fb5c0fe6700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb5a806e140 0x7fb5a80705f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007fd0 tx=0x7fb5ac009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:57.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:57.854+0000 7fb5b1ffb700 1 -- 172.21.15.33:0/2959023501 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb5b805bc60 con 0x7fb5bc102e00 2024-04-25T02:05:58.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.150+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb5bc100110 con 0x7fb5bc102e00 2024-04-25T02:05:58.158 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.150+0000 7fb5b1ffb700 1 -- 172.21.15.33:0/2959023501 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb5b805b7f0 con 0x7fb5bc102e00 2024-04-25T02:05:58.158 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:05:58.158 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:05:58.158 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:05:58.158 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:05:58.158 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:05:58.159 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:05:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.152+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb5a806e140 msgr2=0x7fb5a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb5a806e140 0x7fb5a80705f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb5ac007fd0 tx=0x7fb5ac009040 comp rx=0 tx=0).stop 2024-04-25T02:05:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 msgr2=0x7fb5bc113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 0x7fb5bc113980 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009150 tx=0x7fb5b8009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:58.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 shutdown_connections 2024-04-25T02:05:58.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb5a806e140 0x7fb5a80705f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb5bc102e00 0x7fb5bc113980 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 --2- 172.21.15.33:0/2959023501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb5bc1037a0 0x7fb5bc113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 >> 172.21.15.33:0/2959023501 conn(0x7fb5bc0f89d0 msgr2=0x7fb5bc0fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:58.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 shutdown_connections 2024-04-25T02:05:58.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.153+0000 7fb5c3a4b700 1 -- 172.21.15.33:0/2959023501 wait complete. 2024-04-25T02:05:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.259+0000 7f5dd00b8700 1 -- 172.21.15.33:0/3839711731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc81026b0 msgr2=0x7f5dc80fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.259+0000 7f5dd00b8700 1 --2- 172.21.15.33:0/3839711731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc81026b0 0x7f5dc80fbd20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc004940 tx=0x7f5dbc009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.259+0000 7f5dd00b8700 1 -- 172.21.15.33:0/3839711731 shutdown_connections 2024-04-25T02:05:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.259+0000 7f5dd00b8700 1 --2- 172.21.15.33:0/3839711731 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc81026b0 0x7f5dc80fbd20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.259+0000 7f5dd00b8700 1 --2- 172.21.15.33:0/3839711731 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc8101d10 0x7f5dc81020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.259+0000 7f5dd00b8700 1 -- 172.21.15.33:0/3839711731 >> 172.21.15.33:0/3839711731 conn(0x7f5dc80f7860 msgr2=0x7f5dc80f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.260+0000 7f5dd00b8700 1 -- 172.21.15.33:0/3839711731 shutdown_connections 2024-04-25T02:05:58.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.260+0000 7f5dd00b8700 1 -- 172.21.15.33:0/3839711731 wait complete. 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.260+0000 7f5dd00b8700 1 Processor -- start 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.260+0000 7f5dd00b8700 1 -- start start 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.260+0000 7f5dd00b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8101d10 0x7f5dc80fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.260+0000 7f5dd00b8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc81026b0 0x7f5dc80fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.260+0000 7f5dd00b8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc80ff930 con 0x7f5dc8101d10 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.260+0000 7f5dd00b8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc80fe410 con 0x7f5dc81026b0 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.260+0000 7f5dcde54700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8101d10 0x7f5dc80fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dcde54700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8101d10 0x7f5dc80fd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42124/0 (socket says 172.21.15.33:42124) 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dcde54700 1 -- 172.21.15.33:0/1322597520 learned_addr learned my addr 172.21.15.33:0/1322597520 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:58.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dcde54700 1 -- 172.21.15.33:0/1322597520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc81026b0 msgr2=0x7f5dc80fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dcd653700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc81026b0 0x7f5dc80fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:58.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dcde54700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc81026b0 0x7f5dc80fded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dcde54700 1 -- 172.21.15.33:0/1322597520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dbc005040 con 0x7f5dc8101d10 2024-04-25T02:05:58.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dcde54700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8101d10 0x7f5dc80fd990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400c8c0 tx=0x7f5dc40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:58.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dba7fc700 1 -- 172.21.15.33:0/1322597520 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dc400a980 con 0x7f5dc8101d10 2024-04-25T02:05:58.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dba7fc700 1 -- 172.21.15.33:0/1322597520 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5dc400ce60 con 0x7f5dc8101d10 2024-04-25T02:05:58.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dba7fc700 1 -- 172.21.15.33:0/1322597520 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dc40115e0 con 0x7f5dc8101d10 2024-04-25T02:05:58.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc80fe6c0 con 0x7f5dc8101d10 2024-04-25T02:05:58.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.261+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc811a120 con 0x7f5dc8101d10 2024-04-25T02:05:58.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.262+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc80fd460 con 0x7f5dc8101d10 2024-04-25T02:05:58.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.263+0000 7f5dba7fc700 1 -- 172.21.15.33:0/1322597520 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5dc4012400 con 0x7f5dc8101d10 2024-04-25T02:05:58.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.263+0000 7f5dba7fc700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5db406e140 0x7f5db40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:58.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.263+0000 7f5dba7fc700 1 -- 172.21.15.33:0/1322597520 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5dc408db30 con 0x7f5dc8101d10 2024-04-25T02:05:58.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.263+0000 7f5dcd653700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5db406e140 0x7f5db40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:58.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.264+0000 7f5dcd653700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5db406e140 0x7f5db40705f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00b3b0 tx=0x7f5dbc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:58.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.268+0000 7f5dba7fc700 1 -- 172.21.15.33:0/1322597520 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5dc4056fd0 con 0x7f5dc8101d10 2024-04-25T02:05:58.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:58 smithi033 ceph-mon[111306]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:58.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:58 smithi033 ceph-mon[111306]: pgmap v798: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:58.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:58 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2959023501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:05:58.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.521+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5dc811a3c0 con 0x7f5dc8101d10 2024-04-25T02:05:58.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.522+0000 7f5dba7fc700 1 -- 172.21.15.33:0/1322597520 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f5dc40126b0 con 0x7f5dc8101d10 2024-04-25T02:05:58.528 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:05:58.528 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:05:58.528 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:05:58.529 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:05:58.530 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:05:58.531 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5db406e140 msgr2=0x7f5db40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5db406e140 0x7f5db40705f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f5dbc00b3b0 tx=0x7f5dbc009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8101d10 msgr2=0x7f5dc80fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8101d10 0x7f5dc80fd990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5dc400c8c0 tx=0x7f5dc40044c0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 shutdown_connections 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5db406e140 0x7f5db40705f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5dc8101d10 0x7f5dc80fd990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 --2- 172.21.15.33:0/1322597520 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5dc81026b0 0x7f5dc80fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 >> 172.21.15.33:0/1322597520 conn(0x7f5dc80f7860 msgr2=0x7f5dc8066270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:58.532 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 shutdown_connections 2024-04-25T02:05:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.524+0000 7f5dd00b8700 1 -- 172.21.15.33:0/1322597520 wait complete. 2024-04-25T02:05:58.533 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:05:58.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:58 smithi138 ceph-mon[81304]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:58.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:58 smithi138 ceph-mon[81304]: pgmap v798: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:05:58.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:58 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2959023501' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:05:58.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 -- 172.21.15.33:0/1868014468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 msgr2=0x7f4c800fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 --2- 172.21.15.33:0/1868014468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 0x7f4c800fcdf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c004940 tx=0x7f4c6c009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 -- 172.21.15.33:0/1868014468 shutdown_connections 2024-04-25T02:05:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 --2- 172.21.15.33:0/1868014468 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 0x7f4c800fcdf0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 --2- 172.21.15.33:0/1868014468 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c80102de0 0x7f4c801031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 -- 172.21.15.33:0/1868014468 >> 172.21.15.33:0/1868014468 conn(0x7f4c800f8990 msgr2=0x7f4c800fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 -- 172.21.15.33:0/1868014468 shutdown_connections 2024-04-25T02:05:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 -- 172.21.15.33:0/1868014468 wait complete. 2024-04-25T02:05:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 Processor -- start 2024-04-25T02:05:58.636 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.629+0000 7f4c86c76700 1 -- start start 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c86c76700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c80102de0 0x7f4c80113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c86c76700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 0x7f4c80113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c86c76700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c801144e0 con 0x7f4c80103780 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c86c76700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8010dae0 con 0x7f4c80102de0 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 0x7f4c80113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c7ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 0x7f4c80113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42152/0 (socket says 172.21.15.33:42152) 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c7ffff700 1 -- 172.21.15.33:0/3583906107 learned_addr learned my addr 172.21.15.33:0/3583906107 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c84a12700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c80102de0 0x7f4c80113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c7ffff700 1 -- 172.21.15.33:0/3583906107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c80102de0 msgr2=0x7f4c80113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.638 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c7ffff700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c80102de0 0x7f4c80113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c7ffff700 1 -- 172.21.15.33:0/3583906107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c005040 con 0x7f4c80103780 2024-04-25T02:05:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c7ffff700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 0x7f4c80113e90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009180 tx=0x7f4c6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c7d7fa700 1 -- 172.21.15.33:0/3583906107 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c6c00e040 con 0x7f4c80103780 2024-04-25T02:05:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c7d7fa700 1 -- 172.21.15.33:0/3583906107 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c6c00bbc0 con 0x7f4c80103780 2024-04-25T02:05:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.630+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8010dd60 con 0x7f4c80103780 2024-04-25T02:05:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.631+0000 7f4c7d7fa700 1 -- 172.21.15.33:0/3583906107 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c6c01c3f0 con 0x7f4c80103780 2024-04-25T02:05:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.631+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c8010e320 con 0x7f4c80103780 2024-04-25T02:05:58.639 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.631+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c800fe530 con 0x7f4c80103780 2024-04-25T02:05:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.632+0000 7f4c7d7fa700 1 -- 172.21.15.33:0/3583906107 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4c6c00bd30 con 0x7f4c80103780 2024-04-25T02:05:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.633+0000 7f4c7d7fa700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c7006e140 0x7f4c700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.633+0000 7f4c7d7fa700 1 -- 172.21.15.33:0/3583906107 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4c6c08ee60 con 0x7f4c80103780 2024-04-25T02:05:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.633+0000 7f4c84a12700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c7006e140 0x7f4c700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:58.640 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.633+0000 7f4c84a12700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c7006e140 0x7f4c700705f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c74004c40 tx=0x7f4c74009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:58.643 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.637+0000 7f4c7d7fa700 1 -- 172.21.15.33:0/3583906107 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4c6c05bd90 con 0x7f4c80103780 2024-04-25T02:05:58.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.868+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c80109ec0 con 0x7f4c7006e140 2024-04-25T02:05:58.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.869+0000 7f4c7d7fa700 1 -- 172.21.15.33:0/3583906107 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4c80109ec0 con 0x7f4c7006e140 2024-04-25T02:05:58.875 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:05:58.875 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:05:58.875 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:05:58.875 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:05:58.875 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:05:58.875 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:05:58.875 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:05:58.876 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:05:58.876 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:05:58.876 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:05:58.876 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:05:58.876 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:05:58.876 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:05:58.876 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:05:58.876 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:05:58.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c7006e140 msgr2=0x7f4c700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.877 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c7006e140 0x7f4c700705f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4c74004c40 tx=0x7f4c74009040 comp rx=0 tx=0).stop 2024-04-25T02:05:58.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 msgr2=0x7f4c80113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 0x7f4c80113e90 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009180 tx=0x7f4c6c009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:58.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 shutdown_connections 2024-04-25T02:05:58.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4c7006e140 0x7f4c700705f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4c80102de0 0x7f4c80113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 --2- 172.21.15.33:0/3583906107 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4c80103780 0x7f4c80113e90 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 >> 172.21.15.33:0/3583906107 conn(0x7f4c800f8990 msgr2=0x7f4c800f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:58.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 shutdown_connections 2024-04-25T02:05:58.878 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.871+0000 7f4c86c76700 1 -- 172.21.15.33:0/3583906107 wait complete. 2024-04-25T02:05:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.976+0000 7f606a8f3700 1 -- 172.21.15.33:0/4288608832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 msgr2=0x7f60641031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.976+0000 7f606a8f3700 1 --2- 172.21.15.33:0/4288608832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 0x7f60641031b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6054004940 tx=0x7f6054009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.976+0000 7f606a8f3700 1 -- 172.21.15.33:0/4288608832 shutdown_connections 2024-04-25T02:05:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.976+0000 7f606a8f3700 1 --2- 172.21.15.33:0/4288608832 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6064103780 0x7f60640fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.976+0000 7f606a8f3700 1 --2- 172.21.15.33:0/4288608832 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 0x7f60641031b0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.976+0000 7f606a8f3700 1 -- 172.21.15.33:0/4288608832 >> 172.21.15.33:0/4288608832 conn(0x7f60640f8990 msgr2=0x7f60640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.976+0000 7f606a8f3700 1 -- 172.21.15.33:0/4288608832 shutdown_connections 2024-04-25T02:05:58.983 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.976+0000 7f606a8f3700 1 -- 172.21.15.33:0/4288608832 wait complete. 2024-04-25T02:05:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f606a8f3700 1 Processor -- start 2024-04-25T02:05:58.984 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f606a8f3700 1 -- start start 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f606a8f3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 0x7f6064113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f606a8f3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6064103780 0x7f6064113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f606a8f3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6064114520 con 0x7f6064102de0 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f606a8f3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606410db70 con 0x7f6064103780 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f6063fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 0x7f6064113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f6063fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 0x7f6064113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:42172/0 (socket says 172.21.15.33:42172) 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f6063fff700 1 -- 172.21.15.33:0/439291963 learned_addr learned my addr 172.21.15.33:0/439291963 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.977+0000 7f60637fe700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6064103780 0x7f6064113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f6063fff700 1 -- 172.21.15.33:0/439291963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6064103780 msgr2=0x7f6064113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:58.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f6063fff700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6064103780 0x7f6064113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:58.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f6063fff700 1 -- 172.21.15.33:0/439291963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6054005040 con 0x7f6064102de0 2024-04-25T02:05:58.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f60637fe700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6064103780 0x7f6064113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:05:58.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f6063fff700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 0x7f6064113990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f605400b8b0 tx=0x7f6054009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:58.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f6060ff9700 1 -- 172.21.15.33:0/439291963 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605400e040 con 0x7f6064102de0 2024-04-25T02:05:58.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606410ddf0 con 0x7f6064102de0 2024-04-25T02:05:58.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f6060ff9700 1 -- 172.21.15.33:0/439291963 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6054004290 con 0x7f6064102de0 2024-04-25T02:05:58.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f6060ff9700 1 -- 172.21.15.33:0/439291963 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f605401c3f0 con 0x7f6064102de0 2024-04-25T02:05:58.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.978+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606410e3b0 con 0x7f6064102de0 2024-04-25T02:05:58.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.979+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60641000f0 con 0x7f6064102de0 2024-04-25T02:05:58.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.979+0000 7f6060ff9700 1 -- 172.21.15.33:0/439291963 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f60540137b0 con 0x7f6064102de0 2024-04-25T02:05:58.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.980+0000 7f6060ff9700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f604c06e140 0x7f604c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:05:58.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.980+0000 7f6060ff9700 1 -- 172.21.15.33:0/439291963 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f605408ee50 con 0x7f6064102de0 2024-04-25T02:05:58.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.980+0000 7f60637fe700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f604c06e140 0x7f604c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:05:58.987 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.980+0000 7f60637fe700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f604c06e140 0x7f604c0705f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6058009fd0 tx=0x7f6058009f20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:05:58.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:58.985+0000 7f6060ff9700 1 -- 172.21.15.33:0/439291963 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f605405bd80 con 0x7f6064102de0 2024-04-25T02:05:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.281+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f606410efe0 con 0x7f6064102de0 2024-04-25T02:05:59.287 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.281+0000 7f6060ff9700 1 -- 172.21.15.33:0/439291963 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f605405b910 con 0x7f6064102de0 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:05:59.288 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:05:59.289 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:05:59.289 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:05:59.289 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:05:59.289 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:05:59.289 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:05:59.289 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:05:59.289 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:05:59.289 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:05:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f604c06e140 msgr2=0x7f604c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f604c06e140 0x7f604c0705f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f6058009fd0 tx=0x7f6058009f20 comp rx=0 tx=0).stop 2024-04-25T02:05:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 msgr2=0x7f6064113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:05:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 0x7f6064113990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f605400b8b0 tx=0x7f6054009e30 comp rx=0 tx=0).stop 2024-04-25T02:05:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 shutdown_connections 2024-04-25T02:05:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f604c06e140 0x7f604c0705f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6064102de0 0x7f6064113990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 --2- 172.21.15.33:0/439291963 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6064103780 0x7f6064113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:05:59.290 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 >> 172.21.15.33:0/439291963 conn(0x7f60640f8990 msgr2=0x7f6064107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:05:59.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 shutdown_connections 2024-04-25T02:05:59.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:05:59.283+0000 7f606a8f3700 1 -- 172.21.15.33:0/439291963 wait complete. 2024-04-25T02:05:59.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:59 smithi138 ceph-mon[81304]: from='client.35472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:59.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:59 smithi138 ceph-mon[81304]: from='client.35476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:59.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:05:59 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1322597520' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:05:59.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:59 smithi033 ceph-mon[111306]: from='client.35472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:59.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:59 smithi033 ceph-mon[111306]: from='client.35476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:05:59.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:05:59 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1322597520' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:06:00.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:00 smithi138 ceph-mon[81304]: from='client.35488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:00.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:00 smithi138 ceph-mon[81304]: pgmap v799: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:00.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:00 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/439291963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:06:00.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:00 smithi033 ceph-mon[111306]: from='client.35488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:00.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:00 smithi033 ceph-mon[111306]: pgmap v799: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:00.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:00 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/439291963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:06:02.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:02 smithi138 ceph-mon[81304]: pgmap v800: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:02.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:02 smithi033 ceph-mon[111306]: pgmap v800: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:04.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:04 smithi138 ceph-mon[81304]: pgmap v801: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:04.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:04 smithi033 ceph-mon[111306]: pgmap v801: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:06.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:06 smithi138 ceph-mon[81304]: pgmap v802: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:06.574 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:06 smithi033 ceph-mon[111306]: pgmap v802: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:08.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:07 smithi033 ceph-mon[111306]: pgmap v803: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:08.297 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:07 smithi138 ceph-mon[81304]: pgmap v803: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:10.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:10 smithi033 ceph-mon[111306]: pgmap v804: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:10.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:10 smithi138 ceph-mon[81304]: pgmap v804: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:11.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:06:11.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:06:12.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:12 smithi033 ceph-mon[111306]: pgmap v805: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:12.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:12 smithi138 ceph-mon[81304]: pgmap v805: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:14.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:14 smithi033 ceph-mon[111306]: pgmap v806: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:14.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:14 smithi138 ceph-mon[81304]: pgmap v806: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:16.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:16 smithi033 ceph-mon[111306]: pgmap v807: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:16.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:16 smithi138 ceph-mon[81304]: pgmap v807: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:18.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:17 smithi033 ceph-mon[111306]: pgmap v808: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:18.297 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:17 smithi138 ceph-mon[81304]: pgmap v808: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:20.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:20 smithi033 ceph-mon[111306]: pgmap v809: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:20.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:20 smithi138 ceph-mon[81304]: pgmap v809: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:22.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:22 smithi033 ceph-mon[111306]: pgmap v810: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:22.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:22 smithi138 ceph-mon[81304]: pgmap v810: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:24.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:24 smithi033 ceph-mon[111306]: pgmap v811: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:24.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:24 smithi138 ceph-mon[81304]: pgmap v811: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:26 smithi033 ceph-mon[111306]: pgmap v812: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:26.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:06:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:26 smithi138 ceph-mon[81304]: pgmap v812: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:26.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:06:28.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:28 smithi033 ceph-mon[111306]: pgmap v813: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:28.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:28 smithi138 ceph-mon[81304]: pgmap v813: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.402+0000 7f134a258700 1 -- 172.21.15.33:0/3320230581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344102df0 msgr2=0x7f13441031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.402+0000 7f134a258700 1 --2- 172.21.15.33:0/3320230581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344102df0 0x7f13441031c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1338004940 tx=0x7f1338009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.402+0000 7f134a258700 1 -- 172.21.15.33:0/3320230581 shutdown_connections 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.402+0000 7f134a258700 1 --2- 172.21.15.33:0/3320230581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1344103790 0x7f13440fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.402+0000 7f134a258700 1 --2- 172.21.15.33:0/3320230581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344102df0 0x7f13441031c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.402+0000 7f134a258700 1 -- 172.21.15.33:0/3320230581 >> 172.21.15.33:0/3320230581 conn(0x7f13440f8980 msgr2=0x7f13440fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.402+0000 7f134a258700 1 -- 172.21.15.33:0/3320230581 shutdown_connections 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.402+0000 7f134a258700 1 -- 172.21.15.33:0/3320230581 wait complete. 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.402+0000 7f134a258700 1 Processor -- start 2024-04-25T02:06:29.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f134a258700 1 -- start start 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f134a258700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1344102df0 0x7f1344113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f134a258700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344103790 0x7f13441140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f134a258700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1344114730 con 0x7f1344103790 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f134a258700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134410db40 con 0x7f1344102df0 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344103790 0x7f13441140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f1342ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344103790 0x7f13441140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55088/0 (socket says 172.21.15.33:55088) 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f1342ffd700 1 -- 172.21.15.33:0/3975085656 learned_addr learned my addr 172.21.15.33:0/3975085656 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f13437fe700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1344102df0 0x7f1344113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f1342ffd700 1 -- 172.21.15.33:0/3975085656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1344102df0 msgr2=0x7f1344113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f1342ffd700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1344102df0 0x7f1344113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f1342ffd700 1 -- 172.21.15.33:0/3975085656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1338005040 con 0x7f1344103790 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.403+0000 7f1342ffd700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344103790 0x7f13441140e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1334002a50 tx=0x7f1334004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:29.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.404+0000 7f132bfff700 1 -- 172.21.15.33:0/3975085656 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1334007cc0 con 0x7f1344103790 2024-04-25T02:06:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.404+0000 7f132bfff700 1 -- 172.21.15.33:0/3975085656 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f133400f040 con 0x7f1344103790 2024-04-25T02:06:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.404+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134410de20 con 0x7f1344103790 2024-04-25T02:06:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.404+0000 7f132bfff700 1 -- 172.21.15.33:0/3975085656 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f133400da40 con 0x7f1344103790 2024-04-25T02:06:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.404+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134410e3e0 con 0x7f1344103790 2024-04-25T02:06:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.405+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1344100100 con 0x7f1344103790 2024-04-25T02:06:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.406+0000 7f132bfff700 1 -- 172.21.15.33:0/3975085656 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f13340097c0 con 0x7f1344103790 2024-04-25T02:06:29.412 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.406+0000 7f132bfff700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132c06e140 0x7f132c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.406+0000 7f13437fe700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132c06e140 0x7f132c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.406+0000 7f132bfff700 1 -- 172.21.15.33:0/3975085656 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f133408d880 con 0x7f1344103790 2024-04-25T02:06:29.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.407+0000 7f13437fe700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132c06e140 0x7f132c0705f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1338004f90 tx=0x7f1338009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:29.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.411+0000 7f132bfff700 1 -- 172.21.15.33:0/3975085656 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1334056c80 con 0x7f1344103790 2024-04-25T02:06:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.640+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f134410f290 con 0x7f132c06e140 2024-04-25T02:06:29.647 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.641+0000 7f132bfff700 1 -- 172.21.15.33:0/3975085656 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f134410f290 con 0x7f132c06e140 2024-04-25T02:06:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132c06e140 msgr2=0x7f132c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132c06e140 0x7f132c0705f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1338004f90 tx=0x7f1338009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344103790 msgr2=0x7f13441140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344103790 0x7f13441140e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1334002a50 tx=0x7f1334004600 comp rx=0 tx=0).stop 2024-04-25T02:06:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 shutdown_connections 2024-04-25T02:06:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132c06e140 0x7f132c0705f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1344102df0 0x7f1344113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 --2- 172.21.15.33:0/3975085656 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1344103790 0x7f13441140e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 >> 172.21.15.33:0/3975085656 conn(0x7f13440f8980 msgr2=0x7f13440facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 shutdown_connections 2024-04-25T02:06:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.643+0000 7f134a258700 1 -- 172.21.15.33:0/3975085656 wait complete. 2024-04-25T02:06:29.660 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:06:29.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 -- 172.21.15.33:0/4199765122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 msgr2=0x7fdbe81021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 --2- 172.21.15.33:0/4199765122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 0x7fdbe81021e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0004940 tx=0x7fdbd0009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 -- 172.21.15.33:0/4199765122 shutdown_connections 2024-04-25T02:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 --2- 172.21.15.33:0/4199765122 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe80fc630 0x7fdbe80fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 --2- 172.21.15.33:0/4199765122 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 0x7fdbe81021e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 -- 172.21.15.33:0/4199765122 >> 172.21.15.33:0/4199765122 conn(0x7fdbe80714d0 msgr2=0x7fdbe80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 -- 172.21.15.33:0/4199765122 shutdown_connections 2024-04-25T02:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 -- 172.21.15.33:0/4199765122 wait complete. 2024-04-25T02:06:29.756 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 Processor -- start 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.749+0000 7fdbee6bb700 1 -- start start 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbee6bb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe80fc630 0x7fdbe8102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbee6bb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 0x7fdbe8103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbee6bb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe8104b80 con 0x7fdbe8101e10 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbee6bb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe81036b0 con 0x7fdbe80fc630 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 0x7fdbe8103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 0x7fdbe8103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55110/0 (socket says 172.21.15.33:55110) 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe77fe700 1 -- 172.21.15.33:0/278854498 learned_addr learned my addr 172.21.15.33:0/278854498 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe7fff700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe80fc630 0x7fdbe8102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe77fe700 1 -- 172.21.15.33:0/278854498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe80fc630 msgr2=0x7fdbe8102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:29.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe77fe700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe80fc630 0x7fdbe8102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe77fe700 1 -- 172.21.15.33:0/278854498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd0005040 con 0x7fdbe8101e10 2024-04-25T02:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe7fff700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe80fc630 0x7fdbe8102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe77fe700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 0x7fdbe8103170 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8002a50 tx=0x7fdbd80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe4ff9700 1 -- 172.21.15.33:0/278854498 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbd8007cc0 con 0x7fdbe8101e10 2024-04-25T02:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe8103990 con 0x7fdbe8101e10 2024-04-25T02:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.750+0000 7fdbe4ff9700 1 -- 172.21.15.33:0/278854498 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbd800f040 con 0x7fdbe8101e10 2024-04-25T02:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.751+0000 7fdbe4ff9700 1 -- 172.21.15.33:0/278854498 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbd8009640 con 0x7fdbe8101e10 2024-04-25T02:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.751+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe811a0c0 con 0x7fdbe8101e10 2024-04-25T02:06:29.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.751+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbe8100140 con 0x7fdbe8101e10 2024-04-25T02:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.752+0000 7fdbe4ff9700 1 -- 172.21.15.33:0/278854498 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdbd80097a0 con 0x7fdbe8101e10 2024-04-25T02:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.752+0000 7fdbe4ff9700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdbd406e140 0x7fdbd40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.753+0000 7fdbe4ff9700 1 -- 172.21.15.33:0/278854498 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdbd808d840 con 0x7fdbe8101e10 2024-04-25T02:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.753+0000 7fdbe7fff700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdbd406e140 0x7fdbd40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:29.760 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.753+0000 7fdbe7fff700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdbd406e140 0x7fdbd40705f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0004560 tx=0x7fdbd0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:29.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.757+0000 7fdbe4ff9700 1 -- 172.21.15.33:0/278854498 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdbd8056c40 con 0x7fdbe8101e10 2024-04-25T02:06:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.982+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbe8108f00 con 0x7fdbd406e140 2024-04-25T02:06:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.984+0000 7fdbe4ff9700 1 -- 172.21.15.33:0/278854498 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdbe8108f00 con 0x7fdbd406e140 2024-04-25T02:06:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdbd406e140 msgr2=0x7fdbd40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdbd406e140 0x7fdbd40705f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0004560 tx=0x7fdbd0009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 msgr2=0x7fdbe8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 0x7fdbe8103170 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8002a50 tx=0x7fdbd80046b0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 shutdown_connections 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdbd406e140 0x7fdbd40705f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdbe80fc630 0x7fdbe8102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 --2- 172.21.15.33:0/278854498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdbe8101e10 0x7fdbe8103170 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 >> 172.21.15.33:0/278854498 conn(0x7fdbe80714d0 msgr2=0x7fdbe81052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 shutdown_connections 2024-04-25T02:06:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:29.986+0000 7fdbee6bb700 1 -- 172.21.15.33:0/278854498 wait complete. 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.091+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/2031405980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fde00 msgr2=0x7fdf380fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.091+0000 7fdf3cc4f700 1 --2- 172.21.15.33:0/2031405980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fde00 0x7fdf380fe1d0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fdf20004940 tx=0x7fdf20009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.091+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/2031405980 shutdown_connections 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.091+0000 7fdf3cc4f700 1 --2- 172.21.15.33:0/2031405980 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf380fe7a0 0x7fdf38106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.091+0000 7fdf3cc4f700 1 --2- 172.21.15.33:0/2031405980 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fde00 0x7fdf380fe1d0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.091+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/2031405980 >> 172.21.15.33:0/2031405980 conn(0x7fdf380f7850 msgr2=0x7fdf380f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/2031405980 shutdown_connections 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/2031405980 wait complete. 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf3cc4f700 1 Processor -- start 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf3cc4f700 1 -- start start 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf3cc4f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf380fde00 0x7fdf380fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf3cc4f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fe7a0 0x7fdf380fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:30.100 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf3cc4f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380fdb80 con 0x7fdf380fe7a0 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf3cc4f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380fc6b0 con 0x7fdf380fde00 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf35d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fe7a0 0x7fdf380fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf35d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fe7a0 0x7fdf380fc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55138/0 (socket says 172.21.15.33:55138) 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.092+0000 7fdf35d9b700 1 -- 172.21.15.33:0/3649622500 learned_addr learned my addr 172.21.15.33:0/3649622500 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf3659c700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf380fde00 0x7fdf380fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf35d9b700 1 -- 172.21.15.33:0/3649622500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf380fde00 msgr2=0x7fdf380fbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf35d9b700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf380fde00 0x7fdf380fbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf35d9b700 1 -- 172.21.15.33:0/3649622500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf20005040 con 0x7fdf380fe7a0 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf3659c700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf380fde00 0x7fdf380fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf35d9b700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fe7a0 0x7fdf380fc170 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdf28002a50 tx=0x7fdf280046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf2effd700 1 -- 172.21.15.33:0/3649622500 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf28007cc0 con 0x7fdf380fe7a0 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf2effd700 1 -- 172.21.15.33:0/3649622500 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf2800f040 con 0x7fdf380fe7a0 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf380fc990 con 0x7fdf380fe7a0 2024-04-25T02:06:30.101 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf2effd700 1 -- 172.21.15.33:0/3649622500 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf28009640 con 0x7fdf380fe7a0 2024-04-25T02:06:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.093+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3811a090 con 0x7fdf380fe7a0 2024-04-25T02:06:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.094+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf38066b10 con 0x7fdf380fe7a0 2024-04-25T02:06:30.102 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.095+0000 7fdf2effd700 1 -- 172.21.15.33:0/3649622500 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fdf280097a0 con 0x7fdf380fe7a0 2024-04-25T02:06:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.095+0000 7fdf2effd700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2406e140 0x7fdf240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.095+0000 7fdf2effd700 1 -- 172.21.15.33:0/3649622500 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fdf2808df10 con 0x7fdf380fe7a0 2024-04-25T02:06:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.095+0000 7fdf3659c700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2406e140 0x7fdf240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:30.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.096+0000 7fdf3659c700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2406e140 0x7fdf240705f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdf20000c00 tx=0x7fdf20009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:30.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.099+0000 7fdf2effd700 1 -- 172.21.15.33:0/3649622500 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fdf28057310 con 0x7fdf380fe7a0 2024-04-25T02:06:30.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.317+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf380f9540 con 0x7fdf2406e140 2024-04-25T02:06:30.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.327+0000 7fdf2effd700 1 -- 172.21.15.33:0/3649622500 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdf380f9540 con 0x7fdf2406e140 2024-04-25T02:06:30.333 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:06:30.333 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (27m) 5m ago 38m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (24m) 5m ago 39m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (24m) 5m ago 38m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (27m) 5m ago 38m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (20m) 5m ago 33m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (21m) 5m ago 33m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (20m) 5m ago 33m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (5m) 5m ago 33m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (29m) 5m ago 40m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (28m) 5m ago 37m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (25m) 5m ago 40m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (25m) 5m ago 37m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (28m) 5m ago 38m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:06:30.334 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (28m) 5m ago 38m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:06:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (24m) 5m ago 37m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:06:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (23m) 5m ago 36m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:06:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (23m) 5m ago 36m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:06:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (22m) 5m ago 35m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:06:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (22m) 5m ago 35m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:06:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (21m) 5m ago 34m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:06:30.335 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (27m) 5m ago 38m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:06:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.329+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2406e140 msgr2=0x7fdf240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.329+0000 7fdf3cc4f700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2406e140 0x7fdf240705f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fdf20000c00 tx=0x7fdf20009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.329+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fe7a0 msgr2=0x7fdf380fc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.329+0000 7fdf3cc4f700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fe7a0 0x7fdf380fc170 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fdf28002a50 tx=0x7fdf280046b0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.330+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 shutdown_connections 2024-04-25T02:06:30.336 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.330+0000 7fdf3cc4f700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fdf2406e140 0x7fdf240705f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.330+0000 7fdf3cc4f700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fdf380fde00 0x7fdf380fbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.330+0000 7fdf3cc4f700 1 --2- 172.21.15.33:0/3649622500 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fdf380fe7a0 0x7fdf380fc170 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.330+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 >> 172.21.15.33:0/3649622500 conn(0x7fdf380f7850 msgr2=0x7fdf380f8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.330+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 shutdown_connections 2024-04-25T02:06:30.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.330+0000 7fdf3cc4f700 1 -- 172.21.15.33:0/3649622500 wait complete. 2024-04-25T02:06:30.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.433+0000 7f08e9a57700 1 -- 172.21.15.33:0/4168592713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e4101d90 msgr2=0x7f08e4102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.433+0000 7f08e9a57700 1 --2- 172.21.15.33:0/4168592713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e4101d90 0x7f08e4102160 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f08d8004940 tx=0x7f08d8009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:30.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.433+0000 7f08e9a57700 1 -- 172.21.15.33:0/4168592713 shutdown_connections 2024-04-25T02:06:30.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.433+0000 7f08e9a57700 1 --2- 172.21.15.33:0/4168592713 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e40fbd40 0x7f08e40fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.433+0000 7f08e9a57700 1 --2- 172.21.15.33:0/4168592713 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e4101d90 0x7f08e4102160 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.433+0000 7f08e9a57700 1 -- 172.21.15.33:0/4168592713 >> 172.21.15.33:0/4168592713 conn(0x7f08e4071d00 msgr2=0x7f08e4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:30.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.433+0000 7f08e9a57700 1 -- 172.21.15.33:0/4168592713 shutdown_connections 2024-04-25T02:06:30.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.433+0000 7f08e9a57700 1 -- 172.21.15.33:0/4168592713 wait complete. 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e9a57700 1 Processor -- start 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e9a57700 1 -- start start 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e9a57700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e40fbd40 0x7f08e4113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e9a57700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e4101d90 0x7f08e41140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e9a57700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e4114720 con 0x7f08e40fbd40 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e9a57700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e410db10 con 0x7f08e4101d90 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e40fbd40 0x7f08e4113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e40fbd40 0x7f08e4113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55154/0 (socket says 172.21.15.33:55154) 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e2ffd700 1 -- 172.21.15.33:0/1309967249 learned_addr learned my addr 172.21.15.33:0/1309967249 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e27fc700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e4101d90 0x7f08e41140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e2ffd700 1 -- 172.21.15.33:0/1309967249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e4101d90 msgr2=0x7f08e41140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e2ffd700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e4101d90 0x7f08e41140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.434+0000 7f08e2ffd700 1 -- 172.21.15.33:0/1309967249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d8005040 con 0x7f08e40fbd40 2024-04-25T02:06:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.435+0000 7f08e2ffd700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e40fbd40 0x7f08e4113b90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b3b0 tx=0x7f08d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:30.443 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.435+0000 7f08cb7fe700 1 -- 172.21.15.33:0/1309967249 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08d800e040 con 0x7f08e40fbd40 2024-04-25T02:06:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.435+0000 7f08cb7fe700 1 -- 172.21.15.33:0/1309967249 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08d800bbc0 con 0x7f08e40fbd40 2024-04-25T02:06:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.435+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e410dd90 con 0x7f08e40fbd40 2024-04-25T02:06:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.435+0000 7f08cb7fe700 1 -- 172.21.15.33:0/1309967249 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08d801c3f0 con 0x7f08e40fbd40 2024-04-25T02:06:30.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.435+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e410e350 con 0x7f08e40fbd40 2024-04-25T02:06:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.436+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e40fd430 con 0x7f08e40fbd40 2024-04-25T02:06:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.436+0000 7f08cb7fe700 1 -- 172.21.15.33:0/1309967249 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f08d800bd30 con 0x7f08e40fbd40 2024-04-25T02:06:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.437+0000 7f08cb7fe700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f08cc06e140 0x7f08cc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.437+0000 7f08e27fc700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f08cc06e140 0x7f08cc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.437+0000 7f08cb7fe700 1 -- 172.21.15.33:0/1309967249 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f08d808eef0 con 0x7f08e40fbd40 2024-04-25T02:06:30.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.437+0000 7f08e27fc700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f08cc06e140 0x7f08cc0705f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f08e410ecd0 tx=0x7f08d4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:30.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.442+0000 7f08cb7fe700 1 -- 172.21.15.33:0/1309967249 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f08d805bd00 con 0x7f08e40fbd40 2024-04-25T02:06:30.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:30 smithi033 ceph-mon[111306]: pgmap v814: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:30.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:30 smithi138 ceph-mon[81304]: pgmap v814: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:30.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.741+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f08e4108e40 con 0x7f08e40fbd40 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.741+0000 7f08cb7fe700 1 -- 172.21.15.33:0/1309967249 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f08d805b890 con 0x7f08e40fbd40 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:06:30.748 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:06:30.749 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:06:30.749 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:06:30.749 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:06:30.749 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:06:30.749 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:06:30.749 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:06:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f08cc06e140 msgr2=0x7f08cc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f08cc06e140 0x7f08cc0705f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f08e410ecd0 tx=0x7f08d4009ee0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e40fbd40 msgr2=0x7f08e4113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e40fbd40 0x7f08e4113b90 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f08d800b3b0 tx=0x7f08d8009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 shutdown_connections 2024-04-25T02:06:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f08cc06e140 0x7f08cc0705f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f08e40fbd40 0x7f08e4113b90 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 --2- 172.21.15.33:0/1309967249 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f08e4101d90 0x7f08e41140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 >> 172.21.15.33:0/1309967249 conn(0x7f08e4071d00 msgr2=0x7f08e40fa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 shutdown_connections 2024-04-25T02:06:30.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.744+0000 7f08e9a57700 1 -- 172.21.15.33:0/1309967249 wait complete. 2024-04-25T02:06:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.848+0000 7f7731852700 1 -- 172.21.15.33:0/3986245217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10a4f0 msgr2=0x7f772c10a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.848+0000 7f7731852700 1 --2- 172.21.15.33:0/3986245217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10a4f0 0x7f772c10a8c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7714004940 tx=0x7f7714009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.848+0000 7f7731852700 1 -- 172.21.15.33:0/3986245217 shutdown_connections 2024-04-25T02:06:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.848+0000 7f7731852700 1 --2- 172.21.15.33:0/3986245217 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f772c10ae90 0x7f772c06eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.848+0000 7f7731852700 1 --2- 172.21.15.33:0/3986245217 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10a4f0 0x7f772c10a8c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.848+0000 7f7731852700 1 -- 172.21.15.33:0/3986245217 >> 172.21.15.33:0/3986245217 conn(0x7f772c06a830 msgr2=0x7f772c06ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:30.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.848+0000 7f7731852700 1 -- 172.21.15.33:0/3986245217 shutdown_connections 2024-04-25T02:06:30.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.849+0000 7f7731852700 1 -- 172.21.15.33:0/3986245217 wait complete. 2024-04-25T02:06:30.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.849+0000 7f7731852700 1 Processor -- start 2024-04-25T02:06:30.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.849+0000 7f7731852700 1 -- start start 2024-04-25T02:06:30.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.849+0000 7f7731852700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f772c10a4f0 0x7f772c11b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:30.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.849+0000 7f7731852700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10ae90 0x7f772c11b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.849+0000 7f7731852700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c11bc10 con 0x7f772c10ae90 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.849+0000 7f7731852700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772c1151f0 con 0x7f772c10a4f0 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f772a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10ae90 0x7f772c11b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f772a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10ae90 0x7f772c11b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55184/0 (socket says 172.21.15.33:55184) 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f772a7fc700 1 -- 172.21.15.33:0/2685337877 learned_addr learned my addr 172.21.15.33:0/2685337877 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f772affd700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f772c10a4f0 0x7f772c11b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f772a7fc700 1 -- 172.21.15.33:0/2685337877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f772c10a4f0 msgr2=0x7f772c11b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f772a7fc700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f772c10a4f0 0x7f772c11b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f772a7fc700 1 -- 172.21.15.33:0/2685337877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7714005040 con 0x7f772c10ae90 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f772a7fc700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10ae90 0x7f772c11b5c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f771c00c800 tx=0x7f771c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f77237fe700 1 -- 172.21.15.33:0/2685337877 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f771c0106f0 con 0x7f772c10ae90 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f77237fe700 1 -- 172.21.15.33:0/2685337877 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f771c010d20 con 0x7f772c10ae90 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f77237fe700 1 -- 172.21.15.33:0/2685337877 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f771c0113a0 con 0x7f772c10ae90 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f772c1154d0 con 0x7f772c10ae90 2024-04-25T02:06:30.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.850+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f772c115a30 con 0x7f772c10ae90 2024-04-25T02:06:30.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.851+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772c06fd40 con 0x7f772c10ae90 2024-04-25T02:06:30.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.852+0000 7f77237fe700 1 -- 172.21.15.33:0/2685337877 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f771c012400 con 0x7f772c10ae90 2024-04-25T02:06:30.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.852+0000 7f77237fe700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f771806e140 0x7f77180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:30.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.852+0000 7f77237fe700 1 -- 172.21.15.33:0/2685337877 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f771c08da70 con 0x7f772c10ae90 2024-04-25T02:06:30.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.852+0000 7f772affd700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f771806e140 0x7f77180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:30.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.853+0000 7f772affd700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f771806e140 0x7f77180705f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f771400b3b0 tx=0x7f7714009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:30.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:30.857+0000 7f77237fe700 1 -- 172.21.15.33:0/2685337877 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f771c05a9a0 con 0x7f772c10ae90 2024-04-25T02:06:31.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.111+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f772c04c710 con 0x7f772c10ae90 2024-04-25T02:06:31.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.112+0000 7f77237fe700 1 -- 172.21.15.33:0/2685337877 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f771c08de60 con 0x7f772c10ae90 2024-04-25T02:06:31.118 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:06:31.119 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:06:31.120 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:06:31.121 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:06:31.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f771806e140 msgr2=0x7f77180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f771806e140 0x7f77180705f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f771400b3b0 tx=0x7f7714009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:31.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10ae90 msgr2=0x7f772c11b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10ae90 0x7f772c11b5c0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f771c00c800 tx=0x7f771c004470 comp rx=0 tx=0).stop 2024-04-25T02:06:31.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 shutdown_connections 2024-04-25T02:06:31.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f771806e140 0x7f77180705f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.122 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f772c10a4f0 0x7f772c11b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 --2- 172.21.15.33:0/2685337877 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f772c10ae90 0x7f772c11b5c0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 >> 172.21.15.33:0/2685337877 conn(0x7f772c06a830 msgr2=0x7f772c10d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:31.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 shutdown_connections 2024-04-25T02:06:31.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.114+0000 7f7731852700 1 -- 172.21.15.33:0/2685337877 wait complete. 2024-04-25T02:06:31.123 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:06:31.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.219+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2349767534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 msgr2=0x7fe6e00fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.219+0000 7fe6e4d0e700 1 --2- 172.21.15.33:0/2349767534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 0x7fe6e00fe1d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0004940 tx=0x7fe6d0009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:31.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2349767534 shutdown_connections 2024-04-25T02:06:31.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 --2- 172.21.15.33:0/2349767534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e00fe7a0 0x7fe6e0106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 --2- 172.21.15.33:0/2349767534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 0x7fe6e00fe1d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2349767534 >> 172.21.15.33:0/2349767534 conn(0x7fe6e00f7870 msgr2=0x7fe6e00f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2349767534 shutdown_connections 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2349767534 wait complete. 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 Processor -- start 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 -- start start 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 0x7fe6e0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e00fe7a0 0x7fe6e0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0114500 con 0x7fe6e00fde00 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6e4d0e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e010db20 con 0x7fe6e00fe7a0 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.220+0000 7fe6de59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 0x7fe6e0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6de59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 0x7fe6e0113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55196/0 (socket says 172.21.15.33:55196) 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6de59c700 1 -- 172.21.15.33:0/2743236120 learned_addr learned my addr 172.21.15.33:0/2743236120 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6ddd9b700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e00fe7a0 0x7fe6e0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:31.230 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6de59c700 1 -- 172.21.15.33:0/2743236120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e00fe7a0 msgr2=0x7fe6e0113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6de59c700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e00fe7a0 0x7fe6e0113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6de59c700 1 -- 172.21.15.33:0/2743236120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d0005040 con 0x7fe6e00fde00 2024-04-25T02:06:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6de59c700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 0x7fe6e0113970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000b3b0 tx=0x7fe6d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6ceffd700 1 -- 172.21.15.33:0/2743236120 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6d000e040 con 0x7fe6e00fde00 2024-04-25T02:06:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6ceffd700 1 -- 172.21.15.33:0/2743236120 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6d0013420 con 0x7fe6e00fde00 2024-04-25T02:06:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e010dda0 con 0x7fe6e00fde00 2024-04-25T02:06:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6ceffd700 1 -- 172.21.15.33:0/2743236120 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6d001c630 con 0x7fe6e00fde00 2024-04-25T02:06:31.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.221+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e010e2e0 con 0x7fe6e00fde00 2024-04-25T02:06:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.222+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e0066b30 con 0x7fe6e00fde00 2024-04-25T02:06:31.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.223+0000 7fe6ceffd700 1 -- 172.21.15.33:0/2743236120 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe6d000baf0 con 0x7fe6e00fde00 2024-04-25T02:06:31.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.223+0000 7fe6ceffd700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6c806e140 0x7fe6c80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:31.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.223+0000 7fe6ceffd700 1 -- 172.21.15.33:0/2743236120 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe6d008ed30 con 0x7fe6e00fde00 2024-04-25T02:06:31.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.223+0000 7fe6ddd9b700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6c806e140 0x7fe6c80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:31.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.224+0000 7fe6ddd9b700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6c806e140 0x7fe6c80705f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe6e010f210 tx=0x7fe6d4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:31.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.228+0000 7fe6ceffd700 1 -- 172.21.15.33:0/2743236120 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe6d00581d0 con 0x7fe6e00fde00 2024-04-25T02:06:31.461 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.454+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6e010f080 con 0x7fe6c806e140 2024-04-25T02:06:31.461 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:31 smithi033 ceph-mon[111306]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:31.461 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:31 smithi033 ceph-mon[111306]: from='client.35500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:31.461 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:31 smithi033 ceph-mon[111306]: from='client.35504 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:31.461 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:31 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1309967249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:06:31.461 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:31 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2685337877' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:06:31.462 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.456+0000 7fe6ceffd700 1 -- 172.21.15.33:0/2743236120 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe6e010f080 con 0x7fe6c806e140 2024-04-25T02:06:31.462 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:06:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:06:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:06:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:06:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:06:31.462 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:06:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:06:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:06:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:06:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:06:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:06:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:06:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:06:31.463 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:06:31.463 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:06:31.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6c806e140 msgr2=0x7fe6c80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6c806e140 0x7fe6c80705f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe6e010f210 tx=0x7fe6d4009ee0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 msgr2=0x7fe6e0113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 0x7fe6e0113970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe6d000b3b0 tx=0x7fe6d0009e30 comp rx=0 tx=0).stop 2024-04-25T02:06:31.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 shutdown_connections 2024-04-25T02:06:31.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe6c806e140 0x7fe6c80705f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe6e00fde00 0x7fe6e0113970 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 --2- 172.21.15.33:0/2743236120 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe6e00fe7a0 0x7fe6e0113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 >> 172.21.15.33:0/2743236120 conn(0x7fe6e00f7870 msgr2=0x7fe6e00f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.458+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 shutdown_connections 2024-04-25T02:06:31.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.459+0000 7fe6e4d0e700 1 -- 172.21.15.33:0/2743236120 wait complete. 2024-04-25T02:06:31.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:31 smithi138 ceph-mon[81304]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:31.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:31 smithi138 ceph-mon[81304]: from='client.35500 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:31.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:31 smithi138 ceph-mon[81304]: from='client.35504 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:31.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:31 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1309967249' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:06:31.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:31 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2685337877' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:06:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.562+0000 7fb26b07f700 1 -- 172.21.15.33:0/132533650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264102de0 msgr2=0x7fb2641031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.562+0000 7fb26b07f700 1 --2- 172.21.15.33:0/132533650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264102de0 0x7fb2641031b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fb25c009ce0 tx=0x7fb25c004d10 comp rx=0 tx=0).stop 2024-04-25T02:06:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.562+0000 7fb26b07f700 1 -- 172.21.15.33:0/132533650 shutdown_connections 2024-04-25T02:06:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.562+0000 7fb26b07f700 1 --2- 172.21.15.33:0/132533650 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb264103780 0x7fb2640fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.562+0000 7fb26b07f700 1 --2- 172.21.15.33:0/132533650 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264102de0 0x7fb2641031b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.562+0000 7fb26b07f700 1 -- 172.21.15.33:0/132533650 >> 172.21.15.33:0/132533650 conn(0x7fb2640f8990 msgr2=0x7fb2640fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.562+0000 7fb26b07f700 1 -- 172.21.15.33:0/132533650 shutdown_connections 2024-04-25T02:06:31.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.562+0000 7fb26b07f700 1 -- 172.21.15.33:0/132533650 wait complete. 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.563+0000 7fb26b07f700 1 Processor -- start 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.563+0000 7fb26b07f700 1 -- start start 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.563+0000 7fb26b07f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb264102de0 0x7fb264113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.563+0000 7fb26b07f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264103780 0x7fb264114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.563+0000 7fb26b07f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb264114660 con 0x7fb264103780 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.563+0000 7fb26b07f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26410dc90 con 0x7fb264102de0 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb263fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264103780 0x7fb264114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb263fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264103780 0x7fb264114010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55216/0 (socket says 172.21.15.33:55216) 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb263fff700 1 -- 172.21.15.33:0/1618946142 learned_addr learned my addr 172.21.15.33:0/1618946142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb268e1b700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb264102de0 0x7fb264113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb263fff700 1 -- 172.21.15.33:0/1618946142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb264102de0 msgr2=0x7fb264113ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb263fff700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb264102de0 0x7fb264113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb263fff700 1 -- 172.21.15.33:0/1618946142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb25c009940 con 0x7fb264103780 2024-04-25T02:06:31.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb263fff700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264103780 0x7fb264114010 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb258002a50 tx=0x7fb258004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb2617fa700 1 -- 172.21.15.33:0/1618946142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb258007cc0 con 0x7fb264103780 2024-04-25T02:06:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26410df70 con 0x7fb264103780 2024-04-25T02:06:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb2617fa700 1 -- 172.21.15.33:0/1618946142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb25800f040 con 0x7fb264103780 2024-04-25T02:06:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb2617fa700 1 -- 172.21.15.33:0/1618946142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb25800daf0 con 0x7fb264103780 2024-04-25T02:06:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.564+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26410e530 con 0x7fb264103780 2024-04-25T02:06:31.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.565+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2641000f0 con 0x7fb264103780 2024-04-25T02:06:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.566+0000 7fb2617fa700 1 -- 172.21.15.33:0/1618946142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb258009810 con 0x7fb264103780 2024-04-25T02:06:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.566+0000 7fb2617fa700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb24c06e140 0x7fb24c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:06:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.566+0000 7fb2617fa700 1 -- 172.21.15.33:0/1618946142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb25808d990 con 0x7fb264103780 2024-04-25T02:06:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.566+0000 7fb268e1b700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb24c06e140 0x7fb24c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:06:31.575 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.567+0000 7fb268e1b700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb24c06e140 0x7fb24c0705f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb25c015040 tx=0x7fb25c004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:06:31.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.571+0000 7fb2617fa700 1 -- 172.21.15.33:0/1618946142 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb25805a7a0 con 0x7fb264103780 2024-04-25T02:06:31.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.865+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb26410f120 con 0x7fb264103780 2024-04-25T02:06:31.872 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.865+0000 7fb2617fa700 1 -- 172.21.15.33:0/1618946142 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fb25805a330 con 0x7fb264103780 2024-04-25T02:06:31.872 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:06:31.872 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:06:31.872 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:06:31.872 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:06:31.873 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:06:31.874 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:06:31.874 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:06:31.874 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:06:31.874 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:06:31.874 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:06:31.874 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.867+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb24c06e140 msgr2=0x7fb24c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.867+0000 7fb26b07f700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb24c06e140 0x7fb24c0705f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb25c015040 tx=0x7fb25c004d10 comp rx=0 tx=0).stop 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.867+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264103780 msgr2=0x7fb264114010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.867+0000 7fb26b07f700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264103780 0x7fb264114010 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb258002a50 tx=0x7fb258004600 comp rx=0 tx=0).stop 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.868+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 shutdown_connections 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.868+0000 7fb26b07f700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb24c06e140 0x7fb24c0705f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.868+0000 7fb26b07f700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb264102de0 0x7fb264113ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.868+0000 7fb26b07f700 1 --2- 172.21.15.33:0/1618946142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb264103780 0x7fb264114010 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.868+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 >> 172.21.15.33:0/1618946142 conn(0x7fb2640f8990 msgr2=0x7fb264107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.868+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 shutdown_connections 2024-04-25T02:06:31.875 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:06:31.868+0000 7fb26b07f700 1 -- 172.21.15.33:0/1618946142 wait complete. 2024-04-25T02:06:32.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:32 smithi033 ceph-mon[111306]: pgmap v815: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:32.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:32 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1618946142' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:06:32.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:32 smithi138 ceph-mon[81304]: pgmap v815: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:32.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:32 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1618946142' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:06:33.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:33 smithi033 ceph-mon[111306]: from='client.35516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:33.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:33 smithi138 ceph-mon[81304]: from='client.35516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:06:34.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:34 smithi033 ceph-mon[111306]: pgmap v816: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:34.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:34 smithi138 ceph-mon[81304]: pgmap v816: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:36.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:36 smithi033 ceph-mon[111306]: pgmap v817: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:36.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:36 smithi138 ceph-mon[81304]: pgmap v817: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:38.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:38 smithi138 ceph-mon[81304]: pgmap v818: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:38.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:38 smithi033 ceph-mon[111306]: pgmap v818: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:40.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:40 smithi138 ceph-mon[81304]: pgmap v819: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:40.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:40 smithi033 ceph-mon[111306]: pgmap v819: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:41.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:06:41.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:06:42.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:42 smithi138 ceph-mon[81304]: pgmap v820: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:42.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:42 smithi033 ceph-mon[111306]: pgmap v820: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:44.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:44 smithi138 ceph-mon[81304]: pgmap v821: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:44.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:44 smithi033 ceph-mon[111306]: pgmap v821: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:46.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:46 smithi138 ceph-mon[81304]: pgmap v822: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:46.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:46 smithi033 ceph-mon[111306]: pgmap v822: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:48.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:48 smithi138 ceph-mon[81304]: pgmap v823: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:48.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:48 smithi033 ceph-mon[111306]: pgmap v823: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:50.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:50 smithi033 ceph-mon[111306]: pgmap v824: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:50.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:50 smithi138 ceph-mon[81304]: pgmap v824: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:52.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:52 smithi033 ceph-mon[111306]: pgmap v825: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:52.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:52 smithi138 ceph-mon[81304]: pgmap v825: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:54.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:54 smithi033 ceph-mon[111306]: pgmap v826: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:54.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:54 smithi138 ceph-mon[81304]: pgmap v826: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:55.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:06:55.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:06:56.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:56 smithi033 ceph-mon[111306]: pgmap v827: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:56.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:56 smithi138 ceph-mon[81304]: pgmap v827: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:58.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:06:58 smithi033 ceph-mon[111306]: pgmap v828: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:06:58.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:06:58 smithi138 ceph-mon[81304]: pgmap v828: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:00.600 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:00 smithi138 ceph-mon[81304]: pgmap v829: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:00.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:00 smithi033 ceph-mon[111306]: pgmap v829: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:01.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.986+0000 7f96ce304700 1 -- 172.21.15.33:0/2905072555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c81037c0 msgr2=0x7f96c80fb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:01.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.986+0000 7f96ce304700 1 --2- 172.21.15.33:0/2905072555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c81037c0 0x7f96c80fb3c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f96b0004940 tx=0x7f96b0009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.986+0000 7f96ce304700 1 -- 172.21.15.33:0/2905072555 shutdown_connections 2024-04-25T02:07:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.986+0000 7f96ce304700 1 --2- 172.21.15.33:0/2905072555 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c81037c0 0x7f96c80fb3c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.986+0000 7f96ce304700 1 --2- 172.21.15.33:0/2905072555 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c8102e20 0x7f96c81031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.986+0000 7f96ce304700 1 -- 172.21.15.33:0/2905072555 >> 172.21.15.33:0/2905072555 conn(0x7f96c8072380 msgr2=0x7f96c8072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.986+0000 7f96ce304700 1 -- 172.21.15.33:0/2905072555 shutdown_connections 2024-04-25T02:07:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.986+0000 7f96ce304700 1 -- 172.21.15.33:0/2905072555 wait complete. 2024-04-25T02:07:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.987+0000 7f96ce304700 1 Processor -- start 2024-04-25T02:07:01.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.987+0000 7f96ce304700 1 -- start start 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.987+0000 7f96ce304700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c8102e20 0x7f96c80fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.987+0000 7f96ce304700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c81037c0 0x7f96c80fcae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.987+0000 7f96ce304700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80fe540 con 0x7f96c8102e20 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.987+0000 7f96ce304700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c80fd020 con 0x7f96c81037c0 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.987+0000 7f96c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c8102e20 0x7f96c80fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.987+0000 7f96c7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c8102e20 0x7f96c80fc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43784/0 (socket says 172.21.15.33:43784) 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.987+0000 7f96c7fff700 1 -- 172.21.15.33:0/1404255495 learned_addr learned my addr 172.21.15.33:0/1404255495 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96c77fe700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c81037c0 0x7f96c80fcae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96c7fff700 1 -- 172.21.15.33:0/1404255495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c81037c0 msgr2=0x7f96c80fcae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96c7fff700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c81037c0 0x7f96c80fcae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96c7fff700 1 -- 172.21.15.33:0/1404255495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0005040 con 0x7f96c8102e20 2024-04-25T02:07:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96c77fe700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c81037c0 0x7f96c80fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:07:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96c7fff700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c8102e20 0x7f96c80fc5a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f96b8002a80 tx=0x7f96b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96c4ff9700 1 -- 172.21.15.33:0/1404255495 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b8007cc0 con 0x7f96c8102e20 2024-04-25T02:07:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96c4ff9700 1 -- 172.21.15.33:0/1404255495 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96b800f040 con 0x7f96c8102e20 2024-04-25T02:07:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96c4ff9700 1 -- 172.21.15.33:0/1404255495 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b800dad0 con 0x7f96c8102e20 2024-04-25T02:07:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c80fd2d0 con 0x7f96c8102e20 2024-04-25T02:07:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.988+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c811a1f0 con 0x7f96c8102e20 2024-04-25T02:07:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.989+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c8100160 con 0x7f96c8102e20 2024-04-25T02:07:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.990+0000 7f96c4ff9700 1 -- 172.21.15.33:0/1404255495 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f96b800dc30 con 0x7f96c8102e20 2024-04-25T02:07:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.990+0000 7f96c4ff9700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96b406e190 0x7f96b4070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.990+0000 7f96c77fe700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96b406e190 0x7f96b4070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.990+0000 7f96c4ff9700 1 -- 172.21.15.33:0/1404255495 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f96b808d970 con 0x7f96c8102e20 2024-04-25T02:07:01.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.991+0000 7f96c77fe700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96b406e190 0x7f96b4070640 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009fd0 tx=0x7f96b0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:02.002 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:01.995+0000 7f96c4ff9700 1 -- 172.21.15.33:0/1404255495 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f96b8056c50 con 0x7f96c8102e20 2024-04-25T02:07:02.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.222+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96c80fda80 con 0x7f96b406e190 2024-04-25T02:07:02.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.223+0000 7f96c4ff9700 1 -- 172.21.15.33:0/1404255495 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f96c80fda80 con 0x7f96b406e190 2024-04-25T02:07:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.225+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96b406e190 msgr2=0x7f96b4070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.225+0000 7f96ce304700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96b406e190 0x7f96b4070640 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f96b0009fd0 tx=0x7f96b0009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.225+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c8102e20 msgr2=0x7f96c80fc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.225+0000 7f96ce304700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c8102e20 0x7f96c80fc5a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f96b8002a80 tx=0x7f96b8004600 comp rx=0 tx=0).stop 2024-04-25T02:07:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.225+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 shutdown_connections 2024-04-25T02:07:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.225+0000 7f96ce304700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f96b406e190 0x7f96b4070640 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.225+0000 7f96ce304700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f96c8102e20 0x7f96c80fc5a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.225+0000 7f96ce304700 1 --2- 172.21.15.33:0/1404255495 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f96c81037c0 0x7f96c80fcae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.225+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 >> 172.21.15.33:0/1404255495 conn(0x7f96c8072380 msgr2=0x7f96c80fad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.226+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 shutdown_connections 2024-04-25T02:07:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.226+0000 7f96ce304700 1 -- 172.21.15.33:0/1404255495 wait complete. 2024-04-25T02:07:02.242 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:07:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 -- 172.21.15.33:0/477715562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 msgr2=0x7fe3b0106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 --2- 172.21.15.33:0/477715562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 0x7fe3b0106dc0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe398004940 tx=0x7fe398009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 -- 172.21.15.33:0/477715562 shutdown_connections 2024-04-25T02:07:02.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 --2- 172.21.15.33:0/477715562 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 0x7fe3b0106dc0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 --2- 172.21.15.33:0/477715562 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3b00fdde0 0x7fe3b00fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 -- 172.21.15.33:0/477715562 >> 172.21.15.33:0/477715562 conn(0x7fe3b00f7890 msgr2=0x7fe3b00f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 -- 172.21.15.33:0/477715562 shutdown_connections 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 -- 172.21.15.33:0/477715562 wait complete. 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 Processor -- start 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.331+0000 7fe3b5019700 1 -- start start 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3b5019700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3b00fdde0 0x7fe3b00fbc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3b5019700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 0x7fe3b00fc150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3b5019700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00fc770 con 0x7fe3b00fe780 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3b5019700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3b00fc8b0 con 0x7fe3b00fdde0 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 0x7fe3b00fc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3ae59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 0x7fe3b00fc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43804/0 (socket says 172.21.15.33:43804) 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3ae59c700 1 -- 172.21.15.33:0/3455214095 learned_addr learned my addr 172.21.15.33:0/3455214095 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:02.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3aed9d700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3b00fdde0 0x7fe3b00fbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3ae59c700 1 -- 172.21.15.33:0/3455214095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3b00fdde0 msgr2=0x7fe3b00fbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3ae59c700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3b00fdde0 0x7fe3b00fbc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3ae59c700 1 -- 172.21.15.33:0/3455214095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398005040 con 0x7fe3b00fe780 2024-04-25T02:07:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3ae59c700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 0x7fe3b00fc150 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe398009380 tx=0x7fe398009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3a77fe700 1 -- 172.21.15.33:0/3455214095 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe39800e040 con 0x7fe3b00fe780 2024-04-25T02:07:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3a77fe700 1 -- 172.21.15.33:0/3455214095 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe398009af0 con 0x7fe3b00fe780 2024-04-25T02:07:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.332+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3b0119e60 con 0x7fe3b00fe780 2024-04-25T02:07:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.333+0000 7fe3a77fe700 1 -- 172.21.15.33:0/3455214095 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe39801c630 con 0x7fe3b00fe780 2024-04-25T02:07:02.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.333+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b011a2c0 con 0x7fe3b00fe780 2024-04-25T02:07:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.334+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3b0066750 con 0x7fe3b00fe780 2024-04-25T02:07:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.334+0000 7fe3a77fe700 1 -- 172.21.15.33:0/3455214095 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe39800bbc0 con 0x7fe3b00fe780 2024-04-25T02:07:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.334+0000 7fe3a77fe700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe39c06e140 0x7fe39c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.335+0000 7fe3a77fe700 1 -- 172.21.15.33:0/3455214095 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe39808ef00 con 0x7fe3b00fe780 2024-04-25T02:07:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.335+0000 7fe3aed9d700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe39c06e140 0x7fe39c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.335+0000 7fe3aed9d700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe39c06e140 0x7fe39c0705f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0001250 tx=0x7fe3a0009620 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:02.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.339+0000 7fe3a77fe700 1 -- 172.21.15.33:0/3455214095 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe398058260 con 0x7fe3b00fe780 2024-04-25T02:07:02.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.566+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3b004bed0 con 0x7fe39c06e140 2024-04-25T02:07:02.573 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:02 smithi033 ceph-mon[111306]: pgmap v830: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:02.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.567+0000 7fe3a77fe700 1 -- 172.21.15.33:0/3455214095 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe3b004bed0 con 0x7fe39c06e140 2024-04-25T02:07:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe39c06e140 msgr2=0x7fe39c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe39c06e140 0x7fe39c0705f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0001250 tx=0x7fe3a0009620 comp rx=0 tx=0).stop 2024-04-25T02:07:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 msgr2=0x7fe3b00fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 0x7fe3b00fc150 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe398009380 tx=0x7fe398009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 shutdown_connections 2024-04-25T02:07:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe39c06e140 0x7fe39c0705f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe3b00fdde0 0x7fe3b00fbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 --2- 172.21.15.33:0/3455214095 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe3b00fe780 0x7fe3b00fc150 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 >> 172.21.15.33:0/3455214095 conn(0x7fe3b00f7890 msgr2=0x7fe3b00ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:02.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.569+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 shutdown_connections 2024-04-25T02:07:02.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.570+0000 7fe3b5019700 1 -- 172.21.15.33:0/3455214095 wait complete. 2024-04-25T02:07:02.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.674+0000 7f72c75c2700 1 -- 172.21.15.33:0/1064486165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 msgr2=0x7f72c01031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.674+0000 7f72c75c2700 1 --2- 172.21.15.33:0/1064486165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 0x7f72c01031c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f72b8009ce0 tx=0x7f72b8004d10 comp rx=0 tx=0).stop 2024-04-25T02:07:02.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.674+0000 7f72c75c2700 1 -- 172.21.15.33:0/1064486165 shutdown_connections 2024-04-25T02:07:02.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.674+0000 7f72c75c2700 1 --2- 172.21.15.33:0/1064486165 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c0103790 0x7f72c00fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.674+0000 7f72c75c2700 1 --2- 172.21.15.33:0/1064486165 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 0x7f72c01031c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.681 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.674+0000 7f72c75c2700 1 -- 172.21.15.33:0/1064486165 >> 172.21.15.33:0/1064486165 conn(0x7f72c00f8980 msgr2=0x7f72c00fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:02.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.674+0000 7f72c75c2700 1 -- 172.21.15.33:0/1064486165 shutdown_connections 2024-04-25T02:07:02.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.674+0000 7f72c75c2700 1 -- 172.21.15.33:0/1064486165 wait complete. 2024-04-25T02:07:02.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c75c2700 1 Processor -- start 2024-04-25T02:07:02.682 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c75c2700 1 -- start start 2024-04-25T02:07:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c75c2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 0x7f72c0070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c75c2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c0103790 0x7f72c00707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c75c2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0070d30 con 0x7f72c0102df0 2024-04-25T02:07:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c75c2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0070e70 con 0x7f72c0103790 2024-04-25T02:07:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c535e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 0x7f72c0070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c535e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 0x7f72c0070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43824/0 (socket says 172.21.15.33:43824) 2024-04-25T02:07:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c535e700 1 -- 172.21.15.33:0/2257027957 learned_addr learned my addr 172.21.15.33:0/2257027957 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c4b5d700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c0103790 0x7f72c00707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:02.685 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c535e700 1 -- 172.21.15.33:0/2257027957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c0103790 msgr2=0x7f72c00707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c535e700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c0103790 0x7f72c00707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c535e700 1 -- 172.21.15.33:0/2257027957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8009940 con 0x7f72c0102df0 2024-04-25T02:07:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.675+0000 7f72c535e700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 0x7f72c0070260 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f72b8003400 tx=0x7f72b8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.676+0000 7f72bdffb700 1 -- 172.21.15.33:0/2257027957 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b800e040 con 0x7f72c0102df0 2024-04-25T02:07:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.676+0000 7f72bdffb700 1 -- 172.21.15.33:0/2257027957 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72b8004610 con 0x7f72c0102df0 2024-04-25T02:07:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.676+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c00710c0 con 0x7f72c0102df0 2024-04-25T02:07:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.676+0000 7f72bdffb700 1 -- 172.21.15.33:0/2257027957 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b801c640 con 0x7f72c0102df0 2024-04-25T02:07:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.676+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c011a470 con 0x7f72c0102df0 2024-04-25T02:07:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.677+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c0100100 con 0x7f72c0102df0 2024-04-25T02:07:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.677+0000 7f72bdffb700 1 -- 172.21.15.33:0/2257027957 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f72b8012a70 con 0x7f72c0102df0 2024-04-25T02:07:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.678+0000 7f72bdffb700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f72a806e140 0x7f72a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.678+0000 7f72bdffb700 1 -- 172.21.15.33:0/2257027957 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f72b808f550 con 0x7f72c0102df0 2024-04-25T02:07:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.678+0000 7f72c4b5d700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f72a806e140 0x7f72a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.678+0000 7f72c4b5d700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f72a806e140 0x7f72a80705f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f72c0071ba0 tx=0x7f72b400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:02.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.683+0000 7f72bdffb700 1 -- 172.21.15.33:0/2257027957 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f72b80588d0 con 0x7f72c0102df0 2024-04-25T02:07:02.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:02 smithi138 ceph-mon[81304]: pgmap v830: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:02.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.898+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f72c004bed0 con 0x7f72a806e140 2024-04-25T02:07:02.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.911+0000 7f72bdffb700 1 -- 172.21.15.33:0/2257027957 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f72c004bed0 con 0x7f72a806e140 2024-04-25T02:07:02.917 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (28m) 5m ago 39m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (25m) 5m ago 39m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (25m) 5m ago 38m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (27m) 5m ago 39m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (21m) 5m ago 33m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (21m) 5m ago 34m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (21m) 5m ago 33m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (5m) 5m ago 33m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (29m) 5m ago 40m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (29m) 5m ago 38m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (26m) 5m ago 40m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (26m) 5m ago 38m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (28m) 5m ago 39m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:07:02.918 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (28m) 5m ago 38m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:07:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (24m) 5m ago 37m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:07:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (24m) 5m ago 37m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:07:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (23m) 5m ago 36m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:07:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (23m) 5m ago 36m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:07:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (22m) 5m ago 35m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:07:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (22m) 5m ago 35m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:07:02.919 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (28m) 5m ago 38m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:07:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f72a806e140 msgr2=0x7f72a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f72a806e140 0x7f72a80705f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f72c0071ba0 tx=0x7f72b400b040 comp rx=0 tx=0).stop 2024-04-25T02:07:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 msgr2=0x7f72c0070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 0x7f72c0070260 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f72b8003400 tx=0x7f72b8004d10 comp rx=0 tx=0).stop 2024-04-25T02:07:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 shutdown_connections 2024-04-25T02:07:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f72a806e140 0x7f72a80705f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f72c0102df0 0x7f72c0070260 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 --2- 172.21.15.33:0/2257027957 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72c0103790 0x7f72c00707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 >> 172.21.15.33:0/2257027957 conn(0x7f72c00f8980 msgr2=0x7f72c01073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 shutdown_connections 2024-04-25T02:07:02.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:02.913+0000 7f72c75c2700 1 -- 172.21.15.33:0/2257027957 wait complete. 2024-04-25T02:07:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.016+0000 7f6a90add700 1 -- 172.21.15.33:0/4215938802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 msgr2=0x7f6a8c103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 --2- 172.21.15.33:0/4215938802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 0x7f6a8c103220 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c004940 tx=0x7f6a7c009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 -- 172.21.15.33:0/4215938802 shutdown_connections 2024-04-25T02:07:03.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 --2- 172.21.15.33:0/4215938802 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a8c065640 0x7f6a8c065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 --2- 172.21.15.33:0/4215938802 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 0x7f6a8c103220 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 -- 172.21.15.33:0/4215938802 >> 172.21.15.33:0/4215938802 conn(0x7f6a8c071d00 msgr2=0x7f6a8c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:03.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 -- 172.21.15.33:0/4215938802 shutdown_connections 2024-04-25T02:07:03.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 -- 172.21.15.33:0/4215938802 wait complete. 2024-04-25T02:07:03.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 Processor -- start 2024-04-25T02:07:03.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 -- start start 2024-04-25T02:07:03.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.017+0000 7f6a90add700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a8c065640 0x7f6a8c113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:03.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a90add700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 0x7f6a8c1140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:03.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a90add700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c114710 con 0x7f6a8c102e50 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a90add700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c10db00 con 0x7f6a8c065640 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a89d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 0x7f6a8c1140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a89d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 0x7f6a8c1140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43840/0 (socket says 172.21.15.33:43840) 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a89d9b700 1 -- 172.21.15.33:0/812119018 learned_addr learned my addr 172.21.15.33:0/812119018 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a8a59c700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a8c065640 0x7f6a8c113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a89d9b700 1 -- 172.21.15.33:0/812119018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a8c065640 msgr2=0x7f6a8c113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a89d9b700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a8c065640 0x7f6a8c113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a89d9b700 1 -- 172.21.15.33:0/812119018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c005040 con 0x7f6a8c102e50 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a89d9b700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 0x7f6a8c1140c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000c830 tx=0x7f6a80004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a7affd700 1 -- 172.21.15.33:0/812119018 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a8000a760 con 0x7f6a8c102e50 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a7affd700 1 -- 172.21.15.33:0/812119018 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a8000ad90 con 0x7f6a8c102e50 2024-04-25T02:07:03.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c10dde0 con 0x7f6a8c102e50 2024-04-25T02:07:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a7affd700 1 -- 172.21.15.33:0/812119018 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a800113a0 con 0x7f6a8c102e50 2024-04-25T02:07:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.018+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c10e3a0 con 0x7f6a8c102e50 2024-04-25T02:07:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.019+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c100100 con 0x7f6a8c102e50 2024-04-25T02:07:03.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.020+0000 7f6a7affd700 1 -- 172.21.15.33:0/812119018 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6a80012400 con 0x7f6a8c102e50 2024-04-25T02:07:03.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.020+0000 7f6a7affd700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a7406e140 0x7f6a740705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:03.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.020+0000 7f6a7affd700 1 -- 172.21.15.33:0/812119018 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6a8008daa0 con 0x7f6a8c102e50 2024-04-25T02:07:03.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.021+0000 7f6a8a59c700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a7406e140 0x7f6a740705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:03.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.021+0000 7f6a8a59c700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a7406e140 0x7f6a740705f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c004f90 tx=0x7f6a7c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:03.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.025+0000 7f6a7affd700 1 -- 172.21.15.33:0/812119018 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6a8005aa70 con 0x7f6a8c102e50 2024-04-25T02:07:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.322+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6a8c10f1d0 con 0x7f6a8c102e50 2024-04-25T02:07:03.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.323+0000 7f6a7affd700 1 -- 172.21.15.33:0/812119018 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6a8005a600 con 0x7f6a8c102e50 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:07:03.330 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:07:03.331 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:07:03.331 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:07:03.331 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:07:03.331 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:07:03.331 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a7406e140 msgr2=0x7f6a740705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a7406e140 0x7f6a740705f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c004f90 tx=0x7f6a7c009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 msgr2=0x7f6a8c1140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 0x7f6a8c1140c0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6a8000c830 tx=0x7f6a80004470 comp rx=0 tx=0).stop 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 shutdown_connections 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a7406e140 0x7f6a740705f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a8c065640 0x7f6a8c113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 --2- 172.21.15.33:0/812119018 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a8c102e50 0x7f6a8c1140c0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 >> 172.21.15.33:0/812119018 conn(0x7f6a8c071d00 msgr2=0x7f6a8c0fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:03.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 shutdown_connections 2024-04-25T02:07:03.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.325+0000 7f6a90add700 1 -- 172.21.15.33:0/812119018 wait complete. 2024-04-25T02:07:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.429+0000 7fbfeeba9700 1 -- 172.21.15.33:0/2948919195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 msgr2=0x7fbfe80fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.429+0000 7fbfeeba9700 1 --2- 172.21.15.33:0/2948919195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 0x7fbfe80fb990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4004940 tx=0x7fbfd4009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.430+0000 7fbfeeba9700 1 -- 172.21.15.33:0/2948919195 shutdown_connections 2024-04-25T02:07:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.430+0000 7fbfeeba9700 1 --2- 172.21.15.33:0/2948919195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 0x7fbfe80fb990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.430+0000 7fbfeeba9700 1 --2- 172.21.15.33:0/2948919195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe8102f60 0x7fbfe8103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.438 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.430+0000 7fbfeeba9700 1 -- 172.21.15.33:0/2948919195 >> 172.21.15.33:0/2948919195 conn(0x7fbfe80714d0 msgr2=0x7fbfe80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.430+0000 7fbfeeba9700 1 -- 172.21.15.33:0/2948919195 shutdown_connections 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.430+0000 7fbfeeba9700 1 -- 172.21.15.33:0/2948919195 wait complete. 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.430+0000 7fbfeeba9700 1 Processor -- start 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.430+0000 7fbfeeba9700 1 -- start start 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfeeba9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 0x7fbfe8113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfeeba9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe8102f60 0x7fbfe8113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfeeba9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8114510 con 0x7fbfe80fb520 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfeeba9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe810db10 con 0x7fbfe8102f60 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfec945700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 0x7fbfe8113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfec945700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 0x7fbfe8113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43864/0 (socket says 172.21.15.33:43864) 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfec945700 1 -- 172.21.15.33:0/806182252 learned_addr learned my addr 172.21.15.33:0/806182252 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfe7fff700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe8102f60 0x7fbfe8113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:03.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfec945700 1 -- 172.21.15.33:0/806182252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe8102f60 msgr2=0x7fbfe8113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfec945700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe8102f60 0x7fbfe8113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfec945700 1 -- 172.21.15.33:0/806182252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4005040 con 0x7fbfe80fb520 2024-04-25T02:07:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfe7fff700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe8102f60 0x7fbfe8113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:07:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfec945700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 0x7fbfe8113980 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc002a50 tx=0x7fbfdc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfe57fa700 1 -- 172.21.15.33:0/806182252 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfdc007cc0 con 0x7fbfe80fb520 2024-04-25T02:07:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe810ddf0 con 0x7fbfe80fb520 2024-04-25T02:07:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfe57fa700 1 -- 172.21.15.33:0/806182252 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfdc00f040 con 0x7fbfe80fb520 2024-04-25T02:07:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfe57fa700 1 -- 172.21.15.33:0/806182252 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfdc009640 con 0x7fbfe80fb520 2024-04-25T02:07:03.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.431+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe810e330 con 0x7fbfe80fb520 2024-04-25T02:07:03.441 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.432+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe8100250 con 0x7fbfe80fb520 2024-04-25T02:07:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.433+0000 7fbfe57fa700 1 -- 172.21.15.33:0/806182252 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbfdc0097a0 con 0x7fbfe80fb520 2024-04-25T02:07:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.433+0000 7fbfe57fa700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbfd8072550 0x7fbfd8074a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.433+0000 7fbfe57fa700 1 -- 172.21.15.33:0/806182252 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbfdc08d940 con 0x7fbfe80fb520 2024-04-25T02:07:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.433+0000 7fbfe7fff700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbfd8072550 0x7fbfd8074a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:03.442 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.434+0000 7fbfe7fff700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbfd8072550 0x7fbfd8074a00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400b3b0 tx=0x7fbfd4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:03.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.438+0000 7fbfe57fa700 1 -- 172.21.15.33:0/806182252 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbfdc05a7f0 con 0x7fbfe80fb520 2024-04-25T02:07:03.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.692+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbfe8109fe0 con 0x7fbfe80fb520 2024-04-25T02:07:03.699 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:03 smithi033 ceph-mon[111306]: from='client.35524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:03.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.692+0000 7fbfe57fa700 1 -- 172.21.15.33:0/806182252 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fbfdc05a380 con 0x7fbfe80fb520 2024-04-25T02:07:03.699 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:07:03.699 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:07:03.700 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:07:03.701 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:07:03.702 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbfd8072550 msgr2=0x7fbfd8074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbfd8072550 0x7fbfd8074a00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbfd400b3b0 tx=0x7fbfd4009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 msgr2=0x7fbfe8113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 0x7fbfe8113980 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fbfdc002a50 tx=0x7fbfdc0046b0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 shutdown_connections 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbfd8072550 0x7fbfd8074a00 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbfe80fb520 0x7fbfe8113980 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 --2- 172.21.15.33:0/806182252 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbfe8102f60 0x7fbfe8113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 >> 172.21.15.33:0/806182252 conn(0x7fbfe80714d0 msgr2=0x7fbfe80fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:03.703 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 shutdown_connections 2024-04-25T02:07:03.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.695+0000 7fbfeeba9700 1 -- 172.21.15.33:0/806182252 wait complete. 2024-04-25T02:07:03.704 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:07:03.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:03 smithi138 ceph-mon[81304]: from='client.35524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 -- 172.21.15.33:0/3445743106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 msgr2=0x7fca64103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 --2- 172.21.15.33:0/3445743106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 0x7fca64103330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fca54004940 tx=0x7fca54009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 -- 172.21.15.33:0/3445743106 shutdown_connections 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 --2- 172.21.15.33:0/3445743106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640fb520 0x7fca640fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 --2- 172.21.15.33:0/3445743106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 0x7fca64103330 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 -- 172.21.15.33:0/3445743106 >> 172.21.15.33:0/3445743106 conn(0x7fca640714d0 msgr2=0x7fca640718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 -- 172.21.15.33:0/3445743106 shutdown_connections 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 -- 172.21.15.33:0/3445743106 wait complete. 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 Processor -- start 2024-04-25T02:07:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.800+0000 7fca68eee700 1 -- start start 2024-04-25T02:07:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca68eee700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640fb520 0x7fca640fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca68eee700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 0x7fca640fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca68eee700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca640fe650 con 0x7fca64102f60 2024-04-25T02:07:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca68eee700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca640fd180 con 0x7fca640fb520 2024-04-25T02:07:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca61d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 0x7fca640fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca61d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 0x7fca640fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43874/0 (socket says 172.21.15.33:43874) 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca61d9b700 1 -- 172.21.15.33:0/3218618410 learned_addr learned my addr 172.21.15.33:0/3218618410 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca6259c700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640fb520 0x7fca640fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca61d9b700 1 -- 172.21.15.33:0/3218618410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640fb520 msgr2=0x7fca640fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca61d9b700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640fb520 0x7fca640fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca61d9b700 1 -- 172.21.15.33:0/3218618410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca54005040 con 0x7fca64102f60 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca61d9b700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 0x7fca640fcc40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00c830 tx=0x7fca4c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.801+0000 7fca5affd700 1 -- 172.21.15.33:0/3218618410 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca4c00a760 con 0x7fca64102f60 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.802+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca640fd460 con 0x7fca64102f60 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.802+0000 7fca5affd700 1 -- 172.21.15.33:0/3218618410 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca4c00ad90 con 0x7fca64102f60 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.802+0000 7fca5affd700 1 -- 172.21.15.33:0/3218618410 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca4c0113a0 con 0x7fca64102f60 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.802+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6411a110 con 0x7fca64102f60 2024-04-25T02:07:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.802+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca64100250 con 0x7fca64102f60 2024-04-25T02:07:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.803+0000 7fca5affd700 1 -- 172.21.15.33:0/3218618410 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fca4c012400 con 0x7fca64102f60 2024-04-25T02:07:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.803+0000 7fca5affd700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fca5006e140 0x7fca500705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.804+0000 7fca5affd700 1 -- 172.21.15.33:0/3218618410 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fca4c08da90 con 0x7fca64102f60 2024-04-25T02:07:03.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.804+0000 7fca6259c700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fca5006e140 0x7fca500705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:03.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.804+0000 7fca6259c700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fca5006e140 0x7fca500705f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fca5400b020 tx=0x7fca54009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:03.814 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:03.808+0000 7fca5affd700 1 -- 172.21.15.33:0/3218618410 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fca4c056f30 con 0x7fca64102f60 2024-04-25T02:07:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.033+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca6410a000 con 0x7fca5006e140 2024-04-25T02:07:04.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.034+0000 7fca5affd700 1 -- 172.21.15.33:0/3218618410 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fca6410a000 con 0x7fca5006e140 2024-04-25T02:07:04.042 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:07:04.042 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:07:04.042 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:07:04.042 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:07:04.043 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.036+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fca5006e140 msgr2=0x7fca500705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.036+0000 7fca68eee700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fca5006e140 0x7fca500705f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fca5400b020 tx=0x7fca54009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.036+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 msgr2=0x7fca640fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.036+0000 7fca68eee700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 0x7fca640fcc40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00c830 tx=0x7fca4c004470 comp rx=0 tx=0).stop 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.036+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 shutdown_connections 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.036+0000 7fca68eee700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fca5006e140 0x7fca500705f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.036+0000 7fca68eee700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fca640fb520 0x7fca640fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.036+0000 7fca68eee700 1 --2- 172.21.15.33:0/3218618410 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fca64102f60 0x7fca640fcc40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.036+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 >> 172.21.15.33:0/3218618410 conn(0x7fca640714d0 msgr2=0x7fca640fa860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.037+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 shutdown_connections 2024-04-25T02:07:04.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.037+0000 7fca68eee700 1 -- 172.21.15.33:0/3218618410 wait complete. 2024-04-25T02:07:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.140+0000 7ffb333f2700 1 -- 172.21.15.33:0/3860265678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 msgr2=0x7ffb2c1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.140+0000 7ffb333f2700 1 --2- 172.21.15.33:0/3860265678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 0x7ffb2c1031e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7ffb28004940 tx=0x7ffb28009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.140+0000 7ffb333f2700 1 -- 172.21.15.33:0/3860265678 shutdown_connections 2024-04-25T02:07:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.140+0000 7ffb333f2700 1 --2- 172.21.15.33:0/3860265678 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb2c0fb530 0x7ffb2c0fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.140+0000 7ffb333f2700 1 --2- 172.21.15.33:0/3860265678 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 0x7ffb2c1031e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.140+0000 7ffb333f2700 1 -- 172.21.15.33:0/3860265678 >> 172.21.15.33:0/3860265678 conn(0x7ffb2c071380 msgr2=0x7ffb2c071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.140+0000 7ffb333f2700 1 -- 172.21.15.33:0/3860265678 shutdown_connections 2024-04-25T02:07:04.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.140+0000 7ffb333f2700 1 -- 172.21.15.33:0/3860265678 wait complete. 2024-04-25T02:07:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb333f2700 1 Processor -- start 2024-04-25T02:07:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb333f2700 1 -- start start 2024-04-25T02:07:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb333f2700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb2c0fb530 0x7ffb2c113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb333f2700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 0x7ffb2c114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:04.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb333f2700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c1146e0 con 0x7ffb2c102e10 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb333f2700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c10dad0 con 0x7ffb2c0fb530 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb3098d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 0x7ffb2c114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb3098d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 0x7ffb2c114090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43900/0 (socket says 172.21.15.33:43900) 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb3098d700 1 -- 172.21.15.33:0/2923148521 learned_addr learned my addr 172.21.15.33:0/2923148521 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb3118e700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb2c0fb530 0x7ffb2c113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb3098d700 1 -- 172.21.15.33:0/2923148521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb2c0fb530 msgr2=0x7ffb2c113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb3098d700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb2c0fb530 0x7ffb2c113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.141+0000 7ffb3098d700 1 -- 172.21.15.33:0/2923148521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb28005040 con 0x7ffb2c102e10 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.142+0000 7ffb3098d700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 0x7ffb2c114090 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c8c0 tx=0x7ffb1c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.142+0000 7ffb21ffb700 1 -- 172.21.15.33:0/2923148521 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb1c00a760 con 0x7ffb2c102e10 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.142+0000 7ffb21ffb700 1 -- 172.21.15.33:0/2923148521 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb1c00ad90 con 0x7ffb2c102e10 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.142+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c10ddb0 con 0x7ffb2c102e10 2024-04-25T02:07:04.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.142+0000 7ffb21ffb700 1 -- 172.21.15.33:0/2923148521 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb1c0113a0 con 0x7ffb2c102e10 2024-04-25T02:07:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.142+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c10e370 con 0x7ffb2c102e10 2024-04-25T02:07:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.143+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c065dd0 con 0x7ffb2c102e10 2024-04-25T02:07:04.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.144+0000 7ffb21ffb700 1 -- 172.21.15.33:0/2923148521 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffb1c012400 con 0x7ffb2c102e10 2024-04-25T02:07:04.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.144+0000 7ffb21ffb700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffb1806e140 0x7ffb180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:04.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.144+0000 7ffb21ffb700 1 -- 172.21.15.33:0/2923148521 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ffb1c08da70 con 0x7ffb2c102e10 2024-04-25T02:07:04.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.144+0000 7ffb3118e700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffb1806e140 0x7ffb180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:04.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.145+0000 7ffb3118e700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffb1806e140 0x7ffb180705f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800b3b0 tx=0x7ffb28009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:04.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.149+0000 7ffb21ffb700 1 -- 172.21.15.33:0/2923148521 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ffb1c05aa40 con 0x7ffb2c102e10 2024-04-25T02:07:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.442+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffb2c109e90 con 0x7ffb2c102e10 2024-04-25T02:07:04.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.442+0000 7ffb21ffb700 1 -- 172.21.15.33:0/2923148521 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7ffb1c05a5d0 con 0x7ffb2c102e10 2024-04-25T02:07:04.449 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:07:04.449 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:07:04.449 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:07:04.449 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:07:04.449 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:07:04.450 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:07:04.451 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:07:04.451 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:07:04.451 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:07:04.451 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:07:04.451 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:07:04.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffb1806e140 msgr2=0x7ffb180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffb1806e140 0x7ffb180705f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ffb2800b3b0 tx=0x7ffb28009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 msgr2=0x7ffb2c114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 0x7ffb2c114090 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c8c0 tx=0x7ffb1c0044c0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 shutdown_connections 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffb1806e140 0x7ffb180705f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffb2c0fb530 0x7ffb2c113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 --2- 172.21.15.33:0/2923148521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffb2c102e10 0x7ffb2c114090 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 >> 172.21.15.33:0/2923148521 conn(0x7ffb2c071380 msgr2=0x7ffb2c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 shutdown_connections 2024-04-25T02:07:04.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:04.444+0000 7ffb333f2700 1 -- 172.21.15.33:0/2923148521 wait complete. 2024-04-25T02:07:04.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:04 smithi033 ceph-mon[111306]: from='client.35528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:04.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:04 smithi033 ceph-mon[111306]: from='client.35532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:04.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:04 smithi033 ceph-mon[111306]: pgmap v831: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:04.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:04 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/812119018' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:07:04.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:04 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/806182252' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:07:04.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:04 smithi138 ceph-mon[81304]: from='client.35528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:04.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:04 smithi138 ceph-mon[81304]: from='client.35532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:04.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:04 smithi138 ceph-mon[81304]: pgmap v831: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:04.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:04 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/812119018' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:07:04.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:04 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/806182252' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:07:05.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:05 smithi033 ceph-mon[111306]: from='client.35544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:05.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:05 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2923148521' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:07:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:05 smithi138 ceph-mon[81304]: from='client.35544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:05 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2923148521' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:07:06.674 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:06 smithi033 ceph-mon[111306]: pgmap v832: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:06.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:06 smithi138 ceph-mon[81304]: pgmap v832: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:08.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:08 smithi033 ceph-mon[111306]: pgmap v833: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:08.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:08 smithi138 ceph-mon[81304]: pgmap v833: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:10 smithi033 ceph-mon[111306]: pgmap v834: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:07:10.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:10 smithi138 ceph-mon[81304]: pgmap v834: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:07:12.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:12 smithi033 ceph-mon[111306]: pgmap v835: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:12.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:12 smithi138 ceph-mon[81304]: pgmap v835: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:14.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:14 smithi033 ceph-mon[111306]: pgmap v836: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:14 smithi138 ceph-mon[81304]: pgmap v836: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:16.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:16 smithi033 ceph-mon[111306]: pgmap v837: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:16.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:16 smithi138 ceph-mon[81304]: pgmap v837: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:18.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:18 smithi033 ceph-mon[111306]: pgmap v838: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:18.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:18 smithi138 ceph-mon[81304]: pgmap v838: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:20.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:20 smithi033 ceph-mon[111306]: pgmap v839: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:20.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:20 smithi138 ceph-mon[81304]: pgmap v839: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:22.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:22 smithi033 ceph-mon[111306]: pgmap v840: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:22.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:22 smithi138 ceph-mon[81304]: pgmap v840: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:24.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:24 smithi033 ceph-mon[111306]: pgmap v841: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:24.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:24 smithi138 ceph-mon[81304]: pgmap v841: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:25.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:07:25.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:07:26.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:26 smithi033 ceph-mon[111306]: pgmap v842: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:26.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:26 smithi138 ceph-mon[81304]: pgmap v842: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:28.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:28 smithi033 ceph-mon[111306]: pgmap v843: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:28.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:28 smithi138 ceph-mon[81304]: pgmap v843: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:29.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:29 smithi033 ceph-mon[111306]: pgmap v844: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:29.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:29 smithi138 ceph-mon[81304]: pgmap v844: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:32.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:32 smithi033 ceph-mon[111306]: pgmap v845: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:32.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:32 smithi138 ceph-mon[81304]: pgmap v845: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:34.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:34 smithi033 ceph-mon[111306]: pgmap v846: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:34.547 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:34 smithi138 ceph-mon[81304]: pgmap v846: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:34.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.562+0000 7fad2610f700 1 -- 172.21.15.33:0/2603438154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad201026a0 msgr2=0x7fad200fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:34.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.562+0000 7fad2610f700 1 --2- 172.21.15.33:0/2603438154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad201026a0 0x7fad200fbd10 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fad08004940 tx=0x7fad08009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:34.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.562+0000 7fad2610f700 1 -- 172.21.15.33:0/2603438154 shutdown_connections 2024-04-25T02:07:34.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.563+0000 7fad2610f700 1 --2- 172.21.15.33:0/2603438154 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad201026a0 0x7fad200fbd10 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:34.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.563+0000 7fad2610f700 1 --2- 172.21.15.33:0/2603438154 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad20101d00 0x7fad201020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:34.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.563+0000 7fad2610f700 1 -- 172.21.15.33:0/2603438154 >> 172.21.15.33:0/2603438154 conn(0x7fad200f7870 msgr2=0x7fad200f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:34.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.563+0000 7fad2610f700 1 -- 172.21.15.33:0/2603438154 shutdown_connections 2024-04-25T02:07:34.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.563+0000 7fad2610f700 1 -- 172.21.15.33:0/2603438154 wait complete. 2024-04-25T02:07:34.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.563+0000 7fad2610f700 1 Processor -- start 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.563+0000 7fad2610f700 1 -- start start 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad2610f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad20101d00 0x7fad20113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad2610f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad201026a0 0x7fad20113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad2610f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad201144f0 con 0x7fad20101d00 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad2610f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2010daf0 con 0x7fad201026a0 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad1f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad20101d00 0x7fad20113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad1f7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad20101d00 0x7fad20113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52428/0 (socket says 172.21.15.33:52428) 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad1f7fe700 1 -- 172.21.15.33:0/1362321974 learned_addr learned my addr 172.21.15.33:0/1362321974 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad1effd700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad201026a0 0x7fad20113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad1f7fe700 1 -- 172.21.15.33:0/1362321974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad201026a0 msgr2=0x7fad20113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad1f7fe700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad201026a0 0x7fad20113ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad1f7fe700 1 -- 172.21.15.33:0/1362321974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad08005040 con 0x7fad20101d00 2024-04-25T02:07:34.572 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad1f7fe700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad20101d00 0x7fad20113960 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c830 tx=0x7fad10004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:34.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.564+0000 7fad17fff700 1 -- 172.21.15.33:0/1362321974 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad1000a760 con 0x7fad20101d00 2024-04-25T02:07:34.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.565+0000 7fad17fff700 1 -- 172.21.15.33:0/1362321974 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad1000ad90 con 0x7fad20101d00 2024-04-25T02:07:34.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.565+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2010ddd0 con 0x7fad20101d00 2024-04-25T02:07:34.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.565+0000 7fad17fff700 1 -- 172.21.15.33:0/1362321974 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad100113a0 con 0x7fad20101d00 2024-04-25T02:07:34.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.565+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2010e390 con 0x7fad20101d00 2024-04-25T02:07:34.573 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.565+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad200fd450 con 0x7fad20101d00 2024-04-25T02:07:34.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.566+0000 7fad17fff700 1 -- 172.21.15.33:0/1362321974 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fad10012400 con 0x7fad20101d00 2024-04-25T02:07:34.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.567+0000 7fad17fff700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad0c06e140 0x7fad0c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:34.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.567+0000 7fad17fff700 1 -- 172.21.15.33:0/1362321974 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fad1008db90 con 0x7fad20101d00 2024-04-25T02:07:34.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.567+0000 7fad1effd700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad0c06e140 0x7fad0c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:34.574 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.567+0000 7fad1effd700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad0c06e140 0x7fad0c0705f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fad08009fd0 tx=0x7fad08009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:34.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.571+0000 7fad17fff700 1 -- 172.21.15.33:0/1362321974 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fad1005aac0 con 0x7fad20101d00 2024-04-25T02:07:34.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.801+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fad2010e610 con 0x7fad0c06e140 2024-04-25T02:07:34.808 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.802+0000 7fad17fff700 1 -- 172.21.15.33:0/1362321974 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fad2010e610 con 0x7fad0c06e140 2024-04-25T02:07:34.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad0c06e140 msgr2=0x7fad0c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:34.811 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad0c06e140 0x7fad0c0705f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fad08009fd0 tx=0x7fad08009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:34.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad20101d00 msgr2=0x7fad20113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:34.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad20101d00 0x7fad20113960 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c830 tx=0x7fad10004470 comp rx=0 tx=0).stop 2024-04-25T02:07:34.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 shutdown_connections 2024-04-25T02:07:34.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fad0c06e140 0x7fad0c0705f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:34.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fad20101d00 0x7fad20113960 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:34.812 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 --2- 172.21.15.33:0/1362321974 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fad201026a0 0x7fad20113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:34.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 >> 172.21.15.33:0/1362321974 conn(0x7fad200f7870 msgr2=0x7fad20066250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:34.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 shutdown_connections 2024-04-25T02:07:34.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.804+0000 7fad2610f700 1 -- 172.21.15.33:0/1362321974 wait complete. 2024-04-25T02:07:34.821 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.908+0000 7eff06491700 1 -- 172.21.15.33:0/1006016674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 msgr2=0x7eff000fb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.908+0000 7eff06491700 1 --2- 172.21.15.33:0/1006016674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 0x7eff000fb7c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7efef4004940 tx=0x7efef4009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.908+0000 7eff06491700 1 -- 172.21.15.33:0/1006016674 shutdown_connections 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.909+0000 7eff06491700 1 --2- 172.21.15.33:0/1006016674 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 0x7eff000fb7c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.909+0000 7eff06491700 1 --2- 172.21.15.33:0/1006016674 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff000fdc10 0x7eff000fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.909+0000 7eff06491700 1 -- 172.21.15.33:0/1006016674 >> 172.21.15.33:0/1006016674 conn(0x7eff00073350 msgr2=0x7eff00073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.909+0000 7eff06491700 1 -- 172.21.15.33:0/1006016674 shutdown_connections 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.909+0000 7eff06491700 1 -- 172.21.15.33:0/1006016674 wait complete. 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.909+0000 7eff06491700 1 Processor -- start 2024-04-25T02:07:34.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.909+0000 7eff06491700 1 -- start start 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7eff06491700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 0x7eff00115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7eff06491700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff000fdc10 0x7eff001160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7eff06491700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff00116740 con 0x7eff000fb350 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7eff06491700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0010fd20 con 0x7eff000fdc10 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 0x7eff00115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 0x7eff00115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52444/0 (socket says 172.21.15.33:52444) 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefffff700 1 -- 172.21.15.33:0/3297015511 learned_addr learned my addr 172.21.15.33:0/3297015511 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efeff7fe700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff000fdc10 0x7eff001160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefffff700 1 -- 172.21.15.33:0/3297015511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff000fdc10 msgr2=0x7eff001160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefffff700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff000fdc10 0x7eff001160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:34.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefffff700 1 -- 172.21.15.33:0/3297015511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef4005040 con 0x7eff000fb350 2024-04-25T02:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefffff700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 0x7eff00115bb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efef0002a50 tx=0x7efef00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefcff9700 1 -- 172.21.15.33:0/3297015511 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efef0007cc0 con 0x7eff000fb350 2024-04-25T02:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff00110000 con 0x7eff000fb350 2024-04-25T02:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefcff9700 1 -- 172.21.15.33:0/3297015511 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efef000f040 con 0x7eff000fb350 2024-04-25T02:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7efefcff9700 1 -- 172.21.15.33:0/3297015511 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efef0009640 con 0x7eff000fb350 2024-04-25T02:07:34.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.910+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff00110540 con 0x7eff000fb350 2024-04-25T02:07:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.912+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff000fc360 con 0x7eff000fb350 2024-04-25T02:07:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.912+0000 7efefcff9700 1 -- 172.21.15.33:0/3297015511 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efef00097a0 con 0x7eff000fb350 2024-04-25T02:07:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.913+0000 7efefcff9700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efee806e140 0x7efee80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.913+0000 7efeff7fe700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efee806e140 0x7efee80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.913+0000 7efefcff9700 1 -- 172.21.15.33:0/3297015511 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efef008dfe0 con 0x7eff000fb350 2024-04-25T02:07:34.920 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.913+0000 7efeff7fe700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efee806e140 0x7efee80705f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7eff00111450 tx=0x7efef4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:34.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:34.917+0000 7efefcff9700 1 -- 172.21.15.33:0/3297015511 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efef005aed0 con 0x7eff000fb350 2024-04-25T02:07:35.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.143+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff00111270 con 0x7efee806e140 2024-04-25T02:07:35.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.144+0000 7efefcff9700 1 -- 172.21.15.33:0/3297015511 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7eff00111270 con 0x7efee806e140 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efee806e140 msgr2=0x7efee80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efee806e140 0x7efee80705f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7eff00111450 tx=0x7efef4009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 msgr2=0x7eff00115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 0x7eff00115bb0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efef0002a50 tx=0x7efef00046b0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 shutdown_connections 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efee806e140 0x7efee80705f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7eff000fb350 0x7eff00115bb0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 --2- 172.21.15.33:0/3297015511 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7eff000fdc10 0x7eff001160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 >> 172.21.15.33:0/3297015511 conn(0x7eff00073350 msgr2=0x7eff000f8730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 shutdown_connections 2024-04-25T02:07:35.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.147+0000 7eff06491700 1 -- 172.21.15.33:0/3297015511 wait complete. 2024-04-25T02:07:35.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 -- 172.21.15.33:0/193369269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d80fb520 msgr2=0x7fa3d80fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 --2- 172.21.15.33:0/193369269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d80fb520 0x7fa3d80fb990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc004940 tx=0x7fa3cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:35.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 -- 172.21.15.33:0/193369269 shutdown_connections 2024-04-25T02:07:35.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 --2- 172.21.15.33:0/193369269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d80fb520 0x7fa3d80fb990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 --2- 172.21.15.33:0/193369269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d8102f60 0x7fa3d8103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 -- 172.21.15.33:0/193369269 >> 172.21.15.33:0/193369269 conn(0x7fa3d80714d0 msgr2=0x7fa3d80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:35.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 -- 172.21.15.33:0/193369269 shutdown_connections 2024-04-25T02:07:35.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 -- 172.21.15.33:0/193369269 wait complete. 2024-04-25T02:07:35.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 Processor -- start 2024-04-25T02:07:35.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.251+0000 7fa3dcf80700 1 -- start start 2024-04-25T02:07:35.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3dcf80700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d80fb520 0x7fa3d80fc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:35.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3dcf80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8102f60 0x7fa3d80fcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:35.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3dcf80700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d80fe650 con 0x7fa3d8102f60 2024-04-25T02:07:35.262 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3dcf80700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d80fd180 con 0x7fa3d80fb520 2024-04-25T02:07:35.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3d5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8102f60 0x7fa3d80fcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:35.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3d5d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8102f60 0x7fa3d80fcc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52456/0 (socket says 172.21.15.33:52456) 2024-04-25T02:07:35.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3d5d9b700 1 -- 172.21.15.33:0/487239443 learned_addr learned my addr 172.21.15.33:0/487239443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:35.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3d659c700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d80fb520 0x7fa3d80fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:35.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3d5d9b700 1 -- 172.21.15.33:0/487239443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d80fb520 msgr2=0x7fa3d80fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.263 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3d5d9b700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d80fb520 0x7fa3d80fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3d5d9b700 1 -- 172.21.15.33:0/487239443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc005040 con 0x7fa3d8102f60 2024-04-25T02:07:35.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.252+0000 7fa3d5d9b700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8102f60 0x7fa3d80fcc40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00b3b0 tx=0x7fa3cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.254+0000 7fa3c6ffd700 1 -- 172.21.15.33:0/487239443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3cc00e040 con 0x7fa3d8102f60 2024-04-25T02:07:35.266 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.254+0000 7fa3c6ffd700 1 -- 172.21.15.33:0/487239443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3cc013420 con 0x7fa3d8102f60 2024-04-25T02:07:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.254+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d80fd400 con 0x7fa3d8102f60 2024-04-25T02:07:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.254+0000 7fa3c6ffd700 1 -- 172.21.15.33:0/487239443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3cc01c630 con 0x7fa3d8102f60 2024-04-25T02:07:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.254+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d811a050 con 0x7fa3d8102f60 2024-04-25T02:07:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.255+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d8100250 con 0x7fa3d8102f60 2024-04-25T02:07:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.256+0000 7fa3c6ffd700 1 -- 172.21.15.33:0/487239443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa3cc013590 con 0x7fa3d8102f60 2024-04-25T02:07:35.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.256+0000 7fa3c6ffd700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3c006e140 0x7fa3c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.256+0000 7fa3c6ffd700 1 -- 172.21.15.33:0/487239443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa3cc08ecf0 con 0x7fa3d8102f60 2024-04-25T02:07:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.256+0000 7fa3d659c700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3c006e140 0x7fa3c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.257+0000 7fa3d659c700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3c006e140 0x7fa3c00705f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8004f40 tx=0x7fa3c800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:35.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.261+0000 7fa3c6ffd700 1 -- 172.21.15.33:0/487239443 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa3cc0580f0 con 0x7fa3d8102f60 2024-04-25T02:07:35.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.478+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3d8109fe0 con 0x7fa3c006e140 2024-04-25T02:07:35.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.488+0000 7fa3c6ffd700 1 -- 172.21.15.33:0/487239443 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa3d8109fe0 con 0x7fa3c006e140 2024-04-25T02:07:35.495 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (28m) 6m ago 39m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (25m) 6m ago 40m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (25m) 6m ago 39m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (28m) 6m ago 39m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (21m) 6m ago 34m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (22m) 6m ago 34m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (21m) 6m ago 34m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (6m) 6m ago 34m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (30m) 6m ago 41m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (29m) 6m ago 39m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:07:35.496 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (27m) 6m ago 41m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (26m) 6m ago 38m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (29m) 6m ago 40m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (29m) 6m ago 39m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (25m) 6m ago 38m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (24m) 6m ago 37m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (24m) 6m ago 37m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (23m) 6m ago 36m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (23m) 6m ago 36m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (22m) 6m ago 36m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:07:35.497 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (29m) 6m ago 39m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:07:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.490+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3c006e140 msgr2=0x7fa3c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.490+0000 7fa3dcf80700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3c006e140 0x7fa3c00705f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8004f40 tx=0x7fa3c800b040 comp rx=0 tx=0).stop 2024-04-25T02:07:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.490+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8102f60 msgr2=0x7fa3d80fcc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.490+0000 7fa3dcf80700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8102f60 0x7fa3d80fcc40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00b3b0 tx=0x7fa3cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.491+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 shutdown_connections 2024-04-25T02:07:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.491+0000 7fa3dcf80700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3c006e140 0x7fa3c00705f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.491+0000 7fa3dcf80700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3d80fb520 0x7fa3d80fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.491+0000 7fa3dcf80700 1 --2- 172.21.15.33:0/487239443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3d8102f60 0x7fa3d80fcc40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.491+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 >> 172.21.15.33:0/487239443 conn(0x7fa3d80714d0 msgr2=0x7fa3d80fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.491+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 shutdown_connections 2024-04-25T02:07:35.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.491+0000 7fa3dcf80700 1 -- 172.21.15.33:0/487239443 wait complete. 2024-04-25T02:07:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3972854458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 msgr2=0x7fcb240fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 --2- 172.21.15.33:0/3972854458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 0x7fcb240fe1e0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcb14004940 tx=0x7fcb14009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3972854458 shutdown_connections 2024-04-25T02:07:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 --2- 172.21.15.33:0/3972854458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb240fe7b0 0x7fcb24106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 --2- 172.21.15.33:0/3972854458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 0x7fcb240fe1e0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3972854458 >> 172.21.15.33:0/3972854458 conn(0x7fcb240f7820 msgr2=0x7fcb240f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3972854458 shutdown_connections 2024-04-25T02:07:35.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3972854458 wait complete. 2024-04-25T02:07:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 Processor -- start 2024-04-25T02:07:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.594+0000 7fcb28dc7700 1 -- start start 2024-04-25T02:07:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb28dc7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 0x7fcb24113c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb28dc7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb240fe7b0 0x7fcb24114150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb28dc7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb241147a0 con 0x7fcb240fde10 2024-04-25T02:07:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb28dc7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2410ddc0 con 0x7fcb240fe7b0 2024-04-25T02:07:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb2259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 0x7fcb24113c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb2259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 0x7fcb24113c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:52466/0 (socket says 172.21.15.33:52466) 2024-04-25T02:07:35.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb2259c700 1 -- 172.21.15.33:0/3658883155 learned_addr learned my addr 172.21.15.33:0/3658883155 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb21d9b700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb240fe7b0 0x7fcb24114150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb2259c700 1 -- 172.21.15.33:0/3658883155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb240fe7b0 msgr2=0x7fcb24114150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb2259c700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb240fe7b0 0x7fcb24114150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb2259c700 1 -- 172.21.15.33:0/3658883155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb14005040 con 0x7fcb240fde10 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.595+0000 7fcb2259c700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 0x7fcb24113c10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009380 tx=0x7fcb14009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.596+0000 7fcb12ffd700 1 -- 172.21.15.33:0/3658883155 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb1400e040 con 0x7fcb240fde10 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.596+0000 7fcb12ffd700 1 -- 172.21.15.33:0/3658883155 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb14009af0 con 0x7fcb240fde10 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.596+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2410e040 con 0x7fcb240fde10 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.596+0000 7fcb12ffd700 1 -- 172.21.15.33:0/3658883155 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb1401c630 con 0x7fcb240fde10 2024-04-25T02:07:35.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.596+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2410e580 con 0x7fcb240fde10 2024-04-25T02:07:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.597+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb24066b30 con 0x7fcb240fde10 2024-04-25T02:07:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.597+0000 7fcb12ffd700 1 -- 172.21.15.33:0/3658883155 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcb1400bbc0 con 0x7fcb240fde10 2024-04-25T02:07:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.598+0000 7fcb12ffd700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcb0c06e140 0x7fcb0c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.598+0000 7fcb12ffd700 1 -- 172.21.15.33:0/3658883155 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcb1408ee90 con 0x7fcb240fde10 2024-04-25T02:07:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.598+0000 7fcb21d9b700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcb0c06e140 0x7fcb0c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:35.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.598+0000 7fcb21d9b700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcb0c06e140 0x7fcb0c0705f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcb2410f4f0 tx=0x7fcb18009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:35.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.602+0000 7fcb12ffd700 1 -- 172.21.15.33:0/3658883155 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcb140581f0 con 0x7fcb240fde10 2024-04-25T02:07:35.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.898+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb2404c710 con 0x7fcb240fde10 2024-04-25T02:07:35.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.899+0000 7fcb12ffd700 1 -- 172.21.15.33:0/3658883155 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fcb1405b8b0 con 0x7fcb240fde10 2024-04-25T02:07:35.907 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:07:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:07:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:07:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:07:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:07:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:07:35.907 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:07:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:07:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:07:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:07:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:07:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:07:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:07:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:07:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:07:35.908 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:07:35.909 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:07:35.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.901+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcb0c06e140 msgr2=0x7fcb0c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.901+0000 7fcb28dc7700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcb0c06e140 0x7fcb0c0705f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcb2410f4f0 tx=0x7fcb18009040 comp rx=0 tx=0).stop 2024-04-25T02:07:35.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.901+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 msgr2=0x7fcb24113c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:35.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.901+0000 7fcb28dc7700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 0x7fcb24113c10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcb14009380 tx=0x7fcb14009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:35.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.901+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 shutdown_connections 2024-04-25T02:07:35.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.901+0000 7fcb28dc7700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcb0c06e140 0x7fcb0c0705f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.901+0000 7fcb28dc7700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcb240fde10 0x7fcb24113c10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.901+0000 7fcb28dc7700 1 --2- 172.21.15.33:0/3658883155 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcb240fe7b0 0x7fcb24114150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:35.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.901+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 >> 172.21.15.33:0/3658883155 conn(0x7fcb240f7820 msgr2=0x7fcb240f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:35.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.902+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 shutdown_connections 2024-04-25T02:07:35.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:35.902+0000 7fcb28dc7700 1 -- 172.21.15.33:0/3658883155 wait complete. 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.005+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/1401630186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 msgr2=0x7fc4ac109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.005+0000 7fc4b1b1d700 1 --2- 172.21.15.33:0/1401630186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 0x7fc4ac109150 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0004970 tx=0x7fc4a0009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.005+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/1401630186 shutdown_connections 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.005+0000 7fc4b1b1d700 1 --2- 172.21.15.33:0/1401630186 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 0x7fc4ac109150 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.005+0000 7fc4b1b1d700 1 --2- 172.21.15.33:0/1401630186 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac0702a0 0x7fc4ac070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.005+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/1401630186 >> 172.21.15.33:0/1401630186 conn(0x7fc4ac0f8990 msgr2=0x7fc4ac0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.005+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/1401630186 shutdown_connections 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.005+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/1401630186 wait complete. 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4b1b1d700 1 Processor -- start 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4b1b1d700 1 -- start start 2024-04-25T02:07:36.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4b1b1d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac0702a0 0x7fc4ac115aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4b1b1d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 0x7fc4ac115fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4b1b1d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac116630 con 0x7fc4ac070bb0 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4b1b1d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac10fcc0 con 0x7fc4ac0702a0 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 0x7fc4ac115fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4aaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 0x7fc4ac115fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60642/0 (socket says 172.21.15.33:60642) 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4aaffd700 1 -- 172.21.15.33:0/4250204737 learned_addr learned my addr 172.21.15.33:0/4250204737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4ab7fe700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac0702a0 0x7fc4ac115aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4aaffd700 1 -- 172.21.15.33:0/4250204737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac0702a0 msgr2=0x7fc4ac115aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4aaffd700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac0702a0 0x7fc4ac115aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4aaffd700 1 -- 172.21.15.33:0/4250204737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a0005040 con 0x7fc4ac070bb0 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.006+0000 7fc4ab7fe700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac0702a0 0x7fc4ac115aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.007+0000 7fc4aaffd700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 0x7fc4ac115fe0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0000c00 tx=0x7fc4a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.007+0000 7fc493fff700 1 -- 172.21.15.33:0/4250204737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4a000e040 con 0x7fc4ac070bb0 2024-04-25T02:07:36.015 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.007+0000 7fc493fff700 1 -- 172.21.15.33:0/4250204737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4a000bc10 con 0x7fc4ac070bb0 2024-04-25T02:07:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.007+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac10ff40 con 0x7fc4ac070bb0 2024-04-25T02:07:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.007+0000 7fc493fff700 1 -- 172.21.15.33:0/4250204737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4a001c630 con 0x7fc4ac070bb0 2024-04-25T02:07:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.007+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac110480 con 0x7fc4ac070bb0 2024-04-25T02:07:36.016 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.008+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ac0719e0 con 0x7fc4ac070bb0 2024-04-25T02:07:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.008+0000 7fc493fff700 1 -- 172.21.15.33:0/4250204737 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc4a000bd80 con 0x7fc4ac070bb0 2024-04-25T02:07:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.009+0000 7fc493fff700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc49406e140 0x7fc4940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.009+0000 7fc493fff700 1 -- 172.21.15.33:0/4250204737 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc4a008ed90 con 0x7fc4ac070bb0 2024-04-25T02:07:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.009+0000 7fc4ab7fe700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc49406e140 0x7fc4940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:36.017 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.010+0000 7fc4ab7fe700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc49406e140 0x7fc4940705f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009fd0 tx=0x7fc49c009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:36.020 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.014+0000 7fc493fff700 1 -- 172.21.15.33:0/4250204737 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc4a0058190 con 0x7fc4ac070bb0 2024-04-25T02:07:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.265+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc4ac10c0e0 con 0x7fc4ac070bb0 2024-04-25T02:07:36.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.266+0000 7fc493fff700 1 -- 172.21.15.33:0/4250204737 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fc4a0004de0 con 0x7fc4ac070bb0 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:07:36.273 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:07:36.274 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:07:36.275 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:07:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc49406e140 msgr2=0x7fc4940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc49406e140 0x7fc4940705f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009fd0 tx=0x7fc49c009ee0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 msgr2=0x7fc4ac115fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 0x7fc4ac115fe0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0000c00 tx=0x7fc4a0009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 shutdown_connections 2024-04-25T02:07:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc49406e140 0x7fc4940705f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc4ac0702a0 0x7fc4ac115aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 --2- 172.21.15.33:0/4250204737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc4ac070bb0 0x7fc4ac115fe0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 >> 172.21.15.33:0/4250204737 conn(0x7fc4ac0f8990 msgr2=0x7fc4ac0f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 shutdown_connections 2024-04-25T02:07:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.268+0000 7fc4b1b1d700 1 -- 172.21.15.33:0/4250204737 wait complete. 2024-04-25T02:07:36.277 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:07:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 -- 172.21.15.33:0/727323699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 msgr2=0x7ffa640fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 --2- 172.21.15.33:0/727323699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 0x7ffa640fc1b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ffa58004940 tx=0x7ffa58009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 -- 172.21.15.33:0/727323699 shutdown_connections 2024-04-25T02:07:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 --2- 172.21.15.33:0/727323699 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 0x7ffa640fc1b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 --2- 172.21.15.33:0/727323699 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa64101d90 0x7ffa64102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 -- 172.21.15.33:0/727323699 >> 172.21.15.33:0/727323699 conn(0x7ffa64071d00 msgr2=0x7ffa64072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:36.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 -- 172.21.15.33:0/727323699 shutdown_connections 2024-04-25T02:07:36.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 -- 172.21.15.33:0/727323699 wait complete. 2024-04-25T02:07:36.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 Processor -- start 2024-04-25T02:07:36.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.373+0000 7ffa6b073700 1 -- start start 2024-04-25T02:07:36.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa6b073700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 0x7ffa6410ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:36.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa6b073700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa64101d90 0x7ffa6410f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:36.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa6b073700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6410f770 con 0x7ffa640fbd40 2024-04-25T02:07:36.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa6b073700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6410f8b0 con 0x7ffa64101d90 2024-04-25T02:07:36.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa68e0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 0x7ffa6410ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:36.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa68e0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 0x7ffa6410ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60660/0 (socket says 172.21.15.33:60660) 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa68e0f700 1 -- 172.21.15.33:0/739816399 learned_addr learned my addr 172.21.15.33:0/739816399 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa63fff700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa64101d90 0x7ffa6410f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa68e0f700 1 -- 172.21.15.33:0/739816399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa64101d90 msgr2=0x7ffa6410f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa68e0f700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa64101d90 0x7ffa6410f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa68e0f700 1 -- 172.21.15.33:0/739816399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58005040 con 0x7ffa640fbd40 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa63fff700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa64101d90 0x7ffa6410f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.374+0000 7ffa68e0f700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 0x7ffa6410ecf0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00cac0 tx=0x7ffa5c007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.375+0000 7ffa617fa700 1 -- 172.21.15.33:0/739816399 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa5c00aa50 con 0x7ffa640fbd40 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.375+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa64117ba0 con 0x7ffa640fbd40 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.375+0000 7ffa617fa700 1 -- 172.21.15.33:0/739816399 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa5c00abb0 con 0x7ffa640fbd40 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.375+0000 7ffa617fa700 1 -- 172.21.15.33:0/739816399 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa5c0115e0 con 0x7ffa640fbd40 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.375+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa64118130 con 0x7ffa640fbd40 2024-04-25T02:07:36.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.376+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa640fd430 con 0x7ffa640fbd40 2024-04-25T02:07:36.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.377+0000 7ffa617fa700 1 -- 172.21.15.33:0/739816399 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffa5c00ad20 con 0x7ffa640fbd40 2024-04-25T02:07:36.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.377+0000 7ffa617fa700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa4c06e140 0x7ffa4c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:36.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.377+0000 7ffa617fa700 1 -- 172.21.15.33:0/739816399 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ffa5c08e990 con 0x7ffa640fbd40 2024-04-25T02:07:36.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.377+0000 7ffa63fff700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa4c06e140 0x7ffa4c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:36.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.378+0000 7ffa63fff700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa4c06e140 0x7ffa4c0705f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800b3b0 tx=0x7ffa58009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:36.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.381+0000 7ffa617fa700 1 -- 172.21.15.33:0/739816399 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ffa5c057d90 con 0x7ffa640fbd40 2024-04-25T02:07:36.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:36 smithi033 ceph-mon[111306]: from='client.35552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:36.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:36 smithi033 ceph-mon[111306]: from='client.35556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:36.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:36 smithi033 ceph-mon[111306]: pgmap v847: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:36.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:36 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3658883155' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:07:36.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:36 smithi138 ceph-mon[81304]: from='client.35552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:36.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:36 smithi138 ceph-mon[81304]: from='client.35556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:36.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:36 smithi138 ceph-mon[81304]: pgmap v847: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:36.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:36 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3658883155' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:07:36.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.609+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa64065db0 con 0x7ffa4c06e140 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.610+0000 7ffa617fa700 1 -- 172.21.15.33:0/739816399 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ffa64065db0 con 0x7ffa4c06e140 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:07:36.617 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:07:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:07:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:07:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:07:36.618 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:07:36.618 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:07:36.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa4c06e140 msgr2=0x7ffa4c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa4c06e140 0x7ffa4c0705f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ffa5800b3b0 tx=0x7ffa58009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:36.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 msgr2=0x7ffa6410ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 0x7ffa6410ecf0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ffa5c00cac0 tx=0x7ffa5c007ee0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 shutdown_connections 2024-04-25T02:07:36.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa4c06e140 0x7ffa4c0705f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa640fbd40 0x7ffa6410ecf0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 --2- 172.21.15.33:0/739816399 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa64101d90 0x7ffa6410f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 >> 172.21.15.33:0/739816399 conn(0x7ffa64071d00 msgr2=0x7ffa640fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 shutdown_connections 2024-04-25T02:07:36.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.612+0000 7ffa6b073700 1 -- 172.21.15.33:0/739816399 wait complete. 2024-04-25T02:07:36.722 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.715+0000 7fc37282b700 1 -- 172.21.15.33:0/2130697232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 msgr2=0x7fc36c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.715+0000 7fc37282b700 1 --2- 172.21.15.33:0/2130697232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 0x7fc36c0fcdf0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc360004940 tx=0x7fc360009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.715+0000 7fc37282b700 1 -- 172.21.15.33:0/2130697232 shutdown_connections 2024-04-25T02:07:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.715+0000 7fc37282b700 1 --2- 172.21.15.33:0/2130697232 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 0x7fc36c0fcdf0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.715+0000 7fc37282b700 1 --2- 172.21.15.33:0/2130697232 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c102de0 0x7fc36c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.715+0000 7fc37282b700 1 -- 172.21.15.33:0/2130697232 >> 172.21.15.33:0/2130697232 conn(0x7fc36c0f8990 msgr2=0x7fc36c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.715+0000 7fc37282b700 1 -- 172.21.15.33:0/2130697232 shutdown_connections 2024-04-25T02:07:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.716+0000 7fc37282b700 1 -- 172.21.15.33:0/2130697232 wait complete. 2024-04-25T02:07:36.723 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.716+0000 7fc37282b700 1 Processor -- start 2024-04-25T02:07:36.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.716+0000 7fc37282b700 1 -- start start 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.716+0000 7fc37282b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c102de0 0x7fc36c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.716+0000 7fc37282b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 0x7fc36c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.716+0000 7fc37282b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c114520 con 0x7fc36c103780 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.716+0000 7fc37282b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c10db20 con 0x7fc36c102de0 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 0x7fc36c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc36b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 0x7fc36c113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:60674/0 (socket says 172.21.15.33:60674) 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc36b7fe700 1 -- 172.21.15.33:0/4239594038 learned_addr learned my addr 172.21.15.33:0/4239594038 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc36bfff700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c102de0 0x7fc36c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc36b7fe700 1 -- 172.21.15.33:0/4239594038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c102de0 msgr2=0x7fc36c113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:36.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc36b7fe700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c102de0 0x7fc36c113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc36b7fe700 1 -- 172.21.15.33:0/4239594038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360005040 con 0x7fc36c103780 2024-04-25T02:07:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc36bfff700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c102de0 0x7fc36c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:07:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc36b7fe700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 0x7fc36c113ed0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc360009100 tx=0x7fc360009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc368ff9700 1 -- 172.21.15.33:0/4239594038 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc36000e040 con 0x7fc36c103780 2024-04-25T02:07:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc368ff9700 1 -- 172.21.15.33:0/4239594038 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc360004290 con 0x7fc36c103780 2024-04-25T02:07:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c10dda0 con 0x7fc36c103780 2024-04-25T02:07:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc368ff9700 1 -- 172.21.15.33:0/4239594038 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc36001c3f0 con 0x7fc36c103780 2024-04-25T02:07:36.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.717+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c10e360 con 0x7fc36c103780 2024-04-25T02:07:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.718+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c0fe530 con 0x7fc36c103780 2024-04-25T02:07:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.719+0000 7fc368ff9700 1 -- 172.21.15.33:0/4239594038 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc360013790 con 0x7fc36c103780 2024-04-25T02:07:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.719+0000 7fc368ff9700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc35406e140 0x7fc3540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:07:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.720+0000 7fc368ff9700 1 -- 172.21.15.33:0/4239594038 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc36008ee60 con 0x7fc36c103780 2024-04-25T02:07:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.720+0000 7fc36bfff700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc35406e140 0x7fc3540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:07:36.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.720+0000 7fc36bfff700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc35406e140 0x7fc3540705f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009fd0 tx=0x7fc35c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:07:36.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:36.724+0000 7fc368ff9700 1 -- 172.21.15.33:0/4239594038 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc36005bcf0 con 0x7fc36c103780 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.016+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc36c109ec0 con 0x7fc36c103780 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.017+0000 7fc368ff9700 1 -- 172.21.15.33:0/4239594038 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fc360013420 con 0x7fc36c103780 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:07:37.024 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:07:37.025 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:07:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc35406e140 msgr2=0x7fc3540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc35406e140 0x7fc3540705f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009fd0 tx=0x7fc35c00b040 comp rx=0 tx=0).stop 2024-04-25T02:07:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 msgr2=0x7fc36c113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:07:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 0x7fc36c113ed0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc360009100 tx=0x7fc360009e30 comp rx=0 tx=0).stop 2024-04-25T02:07:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 shutdown_connections 2024-04-25T02:07:37.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc35406e140 0x7fc3540705f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc36c102de0 0x7fc36c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 --2- 172.21.15.33:0/4239594038 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc36c103780 0x7fc36c113ed0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:07:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 >> 172.21.15.33:0/4239594038 conn(0x7fc36c0f8990 msgr2=0x7fc36c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:07:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 shutdown_connections 2024-04-25T02:07:37.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:07:37.019+0000 7fc37282b700 1 -- 172.21.15.33:0/4239594038 wait complete. 2024-04-25T02:07:37.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:37 smithi033 ceph-mon[111306]: from='client.35560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:37.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:37 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4250204737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:07:37.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:37 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4239594038' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:07:37.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:37 smithi138 ceph-mon[81304]: from='client.35560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:37.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:37 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4250204737' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:07:37.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:37 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4239594038' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:07:38.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:38 smithi033 ceph-mon[111306]: from='client.35572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:38.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:38 smithi033 ceph-mon[111306]: pgmap v848: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:38.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:38 smithi138 ceph-mon[81304]: from='client.35572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:07:38.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:38 smithi138 ceph-mon[81304]: pgmap v848: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:40.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:40 smithi033 ceph-mon[111306]: pgmap v849: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:40.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:40 smithi138 ceph-mon[81304]: pgmap v849: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:41.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:07:41.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:07:42.550 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:42 smithi138 ceph-mon[81304]: pgmap v850: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:42.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:42 smithi033 ceph-mon[111306]: pgmap v850: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:44.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:44 smithi138 ceph-mon[81304]: pgmap v851: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:44.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:44 smithi033 ceph-mon[111306]: pgmap v851: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:46.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:46 smithi138 ceph-mon[81304]: pgmap v852: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:46.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:46 smithi033 ceph-mon[111306]: pgmap v852: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:48.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:48 smithi138 ceph-mon[81304]: pgmap v853: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:48.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:48 smithi033 ceph-mon[111306]: pgmap v853: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:50.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:50 smithi138 ceph-mon[81304]: pgmap v854: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:50.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:50 smithi033 ceph-mon[111306]: pgmap v854: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:52.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:52 smithi138 ceph-mon[81304]: pgmap v855: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:52.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:52 smithi033 ceph-mon[111306]: pgmap v855: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:54.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:54 smithi033 ceph-mon[111306]: pgmap v856: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:54.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:54 smithi138 ceph-mon[81304]: pgmap v856: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:56.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:56 smithi033 ceph-mon[111306]: pgmap v857: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:56.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:07:56.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:56 smithi138 ceph-mon[81304]: pgmap v857: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:56.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:07:58.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:07:58 smithi033 ceph-mon[111306]: pgmap v858: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:07:58.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:07:58 smithi138 ceph-mon[81304]: pgmap v858: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:00.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:00 smithi033 ceph-mon[111306]: pgmap v859: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:00.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:00 smithi138 ceph-mon[81304]: pgmap v859: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:02.600 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:02 smithi138 ceph-mon[81304]: pgmap v860: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:02.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:02 smithi033 ceph-mon[111306]: pgmap v860: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:04.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:04 smithi033 ceph-mon[111306]: pgmap v861: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:04.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:04 smithi138 ceph-mon[81304]: pgmap v861: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:06.575 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:06 smithi033 ceph-mon[111306]: pgmap v862: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:06.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:06 smithi138 ceph-mon[81304]: pgmap v862: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:07.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.137+0000 7f575a42b700 1 -- 172.21.15.33:0/2943968710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fde30 msgr2=0x7f57540fe200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.137+0000 7f575a42b700 1 --2- 172.21.15.33:0/2943968710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fde30 0x7f57540fe200 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5744004940 tx=0x7f5744009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:07.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.137+0000 7f575a42b700 1 -- 172.21.15.33:0/2943968710 shutdown_connections 2024-04-25T02:08:07.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.137+0000 7f575a42b700 1 --2- 172.21.15.33:0/2943968710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fe7d0 0x7f5754106ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.137+0000 7f575a42b700 1 --2- 172.21.15.33:0/2943968710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fde30 0x7f57540fe200 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.137+0000 7f575a42b700 1 -- 172.21.15.33:0/2943968710 >> 172.21.15.33:0/2943968710 conn(0x7f57540f7870 msgr2=0x7f57540f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:07.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.137+0000 7f575a42b700 1 -- 172.21.15.33:0/2943968710 shutdown_connections 2024-04-25T02:08:07.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.137+0000 7f575a42b700 1 -- 172.21.15.33:0/2943968710 wait complete. 2024-04-25T02:08:07.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.138+0000 7f575a42b700 1 Processor -- start 2024-04-25T02:08:07.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.138+0000 7f575a42b700 1 -- start start 2024-04-25T02:08:07.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.138+0000 7f575a42b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fde30 0x7f57541139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:07.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.138+0000 7f575a42b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fe7d0 0x7f5754113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:07.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.138+0000 7f575a42b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5754114530 con 0x7f57540fe7d0 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.138+0000 7f575a42b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575410db50 con 0x7f57540fde30 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f57537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fe7d0 0x7f5754113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f57537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fe7d0 0x7f5754113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56266/0 (socket says 172.21.15.33:56266) 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f57537fe700 1 -- 172.21.15.33:0/2829452534 learned_addr learned my addr 172.21.15.33:0/2829452534 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f5753fff700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fde30 0x7f57541139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f57537fe700 1 -- 172.21.15.33:0/2829452534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fde30 msgr2=0x7f57541139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f57537fe700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fde30 0x7f57541139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f57537fe700 1 -- 172.21.15.33:0/2829452534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5744005040 con 0x7f57540fe7d0 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f57537fe700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fe7d0 0x7f5754113ee0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5748002a50 tx=0x7f57480046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f5750ff9700 1 -- 172.21.15.33:0/2829452534 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5748007cc0 con 0x7f57540fe7d0 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.139+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575410de30 con 0x7f57540fe7d0 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.140+0000 7f5750ff9700 1 -- 172.21.15.33:0/2829452534 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f574800f040 con 0x7f57540fe7d0 2024-04-25T02:08:07.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.140+0000 7f5750ff9700 1 -- 172.21.15.33:0/2829452534 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f574800da30 con 0x7f57540fe7d0 2024-04-25T02:08:07.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.140+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575410e370 con 0x7f57540fe7d0 2024-04-25T02:08:07.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.141+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5754066a70 con 0x7f57540fe7d0 2024-04-25T02:08:07.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.141+0000 7f5750ff9700 1 -- 172.21.15.33:0/2829452534 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f574800db90 con 0x7f57540fe7d0 2024-04-25T02:08:07.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.142+0000 7f5750ff9700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f573c06e140 0x7f573c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:07.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.142+0000 7f5750ff9700 1 -- 172.21.15.33:0/2829452534 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f574808d8e0 con 0x7f57540fe7d0 2024-04-25T02:08:07.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.142+0000 7f5753fff700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f573c06e140 0x7f573c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:07.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.142+0000 7f5753fff700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f573c06e140 0x7f573c0705f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f574400b3b0 tx=0x7f5744009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:07.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.146+0000 7f5750ff9700 1 -- 172.21.15.33:0/2829452534 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5748056cd0 con 0x7f57540fe7d0 2024-04-25T02:08:07.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.377+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5754109ea0 con 0x7f573c06e140 2024-04-25T02:08:07.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.379+0000 7f5750ff9700 1 -- 172.21.15.33:0/2829452534 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5754109ea0 con 0x7f573c06e140 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f573c06e140 msgr2=0x7f573c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f573c06e140 0x7f573c0705f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f574400b3b0 tx=0x7f5744009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fe7d0 msgr2=0x7f5754113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fe7d0 0x7f5754113ee0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5748002a50 tx=0x7f57480046b0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 shutdown_connections 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f573c06e140 0x7f573c0705f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57540fde30 0x7f57541139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 --2- 172.21.15.33:0/2829452534 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57540fe7d0 0x7f5754113ee0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 >> 172.21.15.33:0/2829452534 conn(0x7f57540f7870 msgr2=0x7f57540f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 shutdown_connections 2024-04-25T02:08:07.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.381+0000 7f575a42b700 1 -- 172.21.15.33:0/2829452534 wait complete. 2024-04-25T02:08:07.398 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:08:07.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.485+0000 7fbaee92b700 1 -- 172.21.15.33:0/2347876225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8103790 msgr2=0x7fbae80fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.485+0000 7fbaee92b700 1 --2- 172.21.15.33:0/2347876225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8103790 0x7fbae80fce00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fbadc004940 tx=0x7fbadc009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.486+0000 7fbaee92b700 1 -- 172.21.15.33:0/2347876225 shutdown_connections 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.486+0000 7fbaee92b700 1 --2- 172.21.15.33:0/2347876225 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8103790 0x7fbae80fce00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.486+0000 7fbaee92b700 1 --2- 172.21.15.33:0/2347876225 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae8102df0 0x7fbae81031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.486+0000 7fbaee92b700 1 -- 172.21.15.33:0/2347876225 >> 172.21.15.33:0/2347876225 conn(0x7fbae80f8980 msgr2=0x7fbae80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.486+0000 7fbaee92b700 1 -- 172.21.15.33:0/2347876225 shutdown_connections 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.486+0000 7fbaee92b700 1 -- 172.21.15.33:0/2347876225 wait complete. 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.486+0000 7fbaee92b700 1 Processor -- start 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.486+0000 7fbaee92b700 1 -- start start 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.486+0000 7fbaee92b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8102df0 0x7fbae8113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbaee92b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae8103790 0x7fbae81140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbaee92b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae8114720 con 0x7fbae8102df0 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbaee92b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae810db10 con 0x7fbae8103790 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8102df0 0x7fbae8113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:07.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8102df0 0x7fbae8113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56280/0 (socket says 172.21.15.33:56280) 2024-04-25T02:08:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae7fff700 1 -- 172.21.15.33:0/1904192698 learned_addr learned my addr 172.21.15.33:0/1904192698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae77fe700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae8103790 0x7fbae81140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae7fff700 1 -- 172.21.15.33:0/1904192698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae8103790 msgr2=0x7fbae81140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae7fff700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae8103790 0x7fbae81140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae7fff700 1 -- 172.21.15.33:0/1904192698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbadc005040 con 0x7fbae8102df0 2024-04-25T02:08:07.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae7fff700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8102df0 0x7fbae8113b90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbad8002a50 tx=0x7fbad80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:07.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae4ff9700 1 -- 172.21.15.33:0/1904192698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbad8007cc0 con 0x7fbae8102df0 2024-04-25T02:08:07.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae810ddf0 con 0x7fbae8102df0 2024-04-25T02:08:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae4ff9700 1 -- 172.21.15.33:0/1904192698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbad800f040 con 0x7fbae8102df0 2024-04-25T02:08:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbae4ff9700 1 -- 172.21.15.33:0/1904192698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbad8009640 con 0x7fbae8102df0 2024-04-25T02:08:07.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.487+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae810e3b0 con 0x7fbae8102df0 2024-04-25T02:08:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.488+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae80fe540 con 0x7fbae8102df0 2024-04-25T02:08:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.489+0000 7fbae4ff9700 1 -- 172.21.15.33:0/1904192698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbad80097a0 con 0x7fbae8102df0 2024-04-25T02:08:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.490+0000 7fbae4ff9700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbad006e140 0x7fbad00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.490+0000 7fbae77fe700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbad006e140 0x7fbad00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.490+0000 7fbae4ff9700 1 -- 172.21.15.33:0/1904192698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbad808d940 con 0x7fbae8102df0 2024-04-25T02:08:07.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.490+0000 7fbae77fe700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbad006e140 0x7fbad00705f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbae810ecd0 tx=0x7fbadc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:07.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.494+0000 7fbae4ff9700 1 -- 172.21.15.33:0/1904192698 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbad805a7f0 con 0x7fbae8102df0 2024-04-25T02:08:07.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.717+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbae8109ed0 con 0x7fbad006e140 2024-04-25T02:08:07.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.719+0000 7fbae4ff9700 1 -- 172.21.15.33:0/1904192698 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbae8109ed0 con 0x7fbad006e140 2024-04-25T02:08:07.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbad006e140 msgr2=0x7fbad00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbad006e140 0x7fbad00705f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbae810ecd0 tx=0x7fbadc009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:07.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8102df0 msgr2=0x7fbae8113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8102df0 0x7fbae8113b90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbad8002a50 tx=0x7fbad80046b0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 shutdown_connections 2024-04-25T02:08:07.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbad006e140 0x7fbad00705f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbae8102df0 0x7fbae8113b90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 --2- 172.21.15.33:0/1904192698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbae8103790 0x7fbae81140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 >> 172.21.15.33:0/1904192698 conn(0x7fbae80f8980 msgr2=0x7fbae80f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:07.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 shutdown_connections 2024-04-25T02:08:07.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.721+0000 7fbaee92b700 1 -- 172.21.15.33:0/1904192698 wait complete. 2024-04-25T02:08:07.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.825+0000 7fb924566700 1 -- 172.21.15.33:0/4140476116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c101d90 msgr2=0x7fb91c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.825+0000 7fb924566700 1 --2- 172.21.15.33:0/4140476116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c101d90 0x7fb91c102160 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb918004940 tx=0x7fb918009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.825+0000 7fb924566700 1 -- 172.21.15.33:0/4140476116 shutdown_connections 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.825+0000 7fb924566700 1 --2- 172.21.15.33:0/4140476116 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91c0fbd40 0x7fb91c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.825+0000 7fb924566700 1 --2- 172.21.15.33:0/4140476116 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c101d90 0x7fb91c102160 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.825+0000 7fb924566700 1 -- 172.21.15.33:0/4140476116 >> 172.21.15.33:0/4140476116 conn(0x7fb91c071d00 msgr2=0x7fb91c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.825+0000 7fb924566700 1 -- 172.21.15.33:0/4140476116 shutdown_connections 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.825+0000 7fb924566700 1 -- 172.21.15.33:0/4140476116 wait complete. 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb924566700 1 Processor -- start 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb924566700 1 -- start start 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb924566700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c0fbd40 0x7fb91c115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb924566700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91c101d90 0x7fb91c1162c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:07.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb924566700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c116910 con 0x7fb91c0fbd40 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb924566700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c10fd20 con 0x7fb91c101d90 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb922302700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c0fbd40 0x7fb91c115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb922302700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c0fbd40 0x7fb91c115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56300/0 (socket says 172.21.15.33:56300) 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb922302700 1 -- 172.21.15.33:0/819871711 learned_addr learned my addr 172.21.15.33:0/819871711 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb921b01700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91c101d90 0x7fb91c1162c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb922302700 1 -- 172.21.15.33:0/819871711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91c101d90 msgr2=0x7fb91c1162c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb922302700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91c101d90 0x7fb91c1162c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb922302700 1 -- 172.21.15.33:0/819871711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb918005040 con 0x7fb91c0fbd40 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb921b01700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91c101d90 0x7fb91c1162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.826+0000 7fb922302700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c0fbd40 0x7fb91c115d80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b3b0 tx=0x7fb918009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.827+0000 7fb912ffd700 1 -- 172.21.15.33:0/819871711 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb91800e040 con 0x7fb91c0fbd40 2024-04-25T02:08:07.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.827+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91c10ffa0 con 0x7fb91c0fbd40 2024-04-25T02:08:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.827+0000 7fb912ffd700 1 -- 172.21.15.33:0/819871711 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb91800bbc0 con 0x7fb91c0fbd40 2024-04-25T02:08:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.827+0000 7fb912ffd700 1 -- 172.21.15.33:0/819871711 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb91801c3f0 con 0x7fb91c0fbd40 2024-04-25T02:08:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.827+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91c110560 con 0x7fb91c0fbd40 2024-04-25T02:08:07.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.828+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91c0fd430 con 0x7fb91c0fbd40 2024-04-25T02:08:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.828+0000 7fb912ffd700 1 -- 172.21.15.33:0/819871711 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb91800bd30 con 0x7fb91c0fbd40 2024-04-25T02:08:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.829+0000 7fb912ffd700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb90806e140 0x7fb9080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.829+0000 7fb912ffd700 1 -- 172.21.15.33:0/819871711 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb91808ee50 con 0x7fb91c0fbd40 2024-04-25T02:08:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.829+0000 7fb921b01700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb90806e140 0x7fb9080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:07.837 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.829+0000 7fb921b01700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb90806e140 0x7fb9080705f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009fd0 tx=0x7fb90c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:07.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:07.833+0000 7fb912ffd700 1 -- 172.21.15.33:0/819871711 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb9180581d0 con 0x7fb91c0fbd40 2024-04-25T02:08:08.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.050+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb91c04bed0 con 0x7fb90806e140 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.061+0000 7fb912ffd700 1 -- 172.21.15.33:0/819871711 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb91c04bed0 con 0x7fb90806e140 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (29m) 6m ago 40m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (26m) 6m ago 40m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (26m) 7m ago 39m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (28m) 6m ago 40m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (22m) 6m ago 35m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (22m) 6m ago 35m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (22m) 7m ago 35m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (7m) 7m ago 34m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (30m) 6m ago 41m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (30m) 7m ago 39m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:08:08.068 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (27m) 6m ago 42m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (27m) 7m ago 39m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (29m) 6m ago 40m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (29m) 7m ago 39m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (26m) 6m ago 38m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (25m) 6m ago 38m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (24m) 6m ago 37m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (24m) 7m ago 37m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (23m) 7m ago 37m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (23m) 7m ago 36m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:08:08.069 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (29m) 6m ago 39m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:08:08.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb90806e140 msgr2=0x7fb9080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb90806e140 0x7fb9080705f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009fd0 tx=0x7fb90c00b040 comp rx=0 tx=0).stop 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c0fbd40 msgr2=0x7fb91c115d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c0fbd40 0x7fb91c115d80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb91800b3b0 tx=0x7fb918009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 shutdown_connections 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb90806e140 0x7fb9080705f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb91c0fbd40 0x7fb91c115d80 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 --2- 172.21.15.33:0/819871711 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb91c101d90 0x7fb91c1162c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 >> 172.21.15.33:0/819871711 conn(0x7fb91c071d00 msgr2=0x7fb91c0655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.063+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 shutdown_connections 2024-04-25T02:08:08.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.064+0000 7fb924566700 1 -- 172.21.15.33:0/819871711 wait complete. 2024-04-25T02:08:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.166+0000 7f3171b7d700 1 -- 172.21.15.33:0/96327580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 msgr2=0x7f316c0fe230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.166+0000 7f3171b7d700 1 --2- 172.21.15.33:0/96327580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 0x7f316c0fe230 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f315c004940 tx=0x7f315c009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.166+0000 7f3171b7d700 1 -- 172.21.15.33:0/96327580 shutdown_connections 2024-04-25T02:08:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.166+0000 7f3171b7d700 1 --2- 172.21.15.33:0/96327580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f316c0fe800 0x7f316c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.166+0000 7f3171b7d700 1 --2- 172.21.15.33:0/96327580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 0x7f316c0fe230 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.166+0000 7f3171b7d700 1 -- 172.21.15.33:0/96327580 >> 172.21.15.33:0/96327580 conn(0x7f316c0f7870 msgr2=0x7f316c0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.166+0000 7f3171b7d700 1 -- 172.21.15.33:0/96327580 shutdown_connections 2024-04-25T02:08:08.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.166+0000 7f3171b7d700 1 -- 172.21.15.33:0/96327580 wait complete. 2024-04-25T02:08:08.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.167+0000 7f3171b7d700 1 Processor -- start 2024-04-25T02:08:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.167+0000 7f3171b7d700 1 -- start start 2024-04-25T02:08:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.167+0000 7f3171b7d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 0x7f316c113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.167+0000 7f3171b7d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f316c0fe800 0x7f316c113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.167+0000 7f3171b7d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c114550 con 0x7f316c0fde60 2024-04-25T02:08:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f3171b7d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10dac0 con 0x7f316c0fe800 2024-04-25T02:08:08.176 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f316b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 0x7f316c113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f316b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 0x7f316c113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56306/0 (socket says 172.21.15.33:56306) 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f316b7fe700 1 -- 172.21.15.33:0/1215721892 learned_addr learned my addr 172.21.15.33:0/1215721892 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f316affd700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f316c0fe800 0x7f316c113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f316b7fe700 1 -- 172.21.15.33:0/1215721892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f316c0fe800 msgr2=0x7f316c113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f316b7fe700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f316c0fe800 0x7f316c113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f316b7fe700 1 -- 172.21.15.33:0/1215721892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c005040 con 0x7f316c0fde60 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f316b7fe700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 0x7f316c113930 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f315c009380 tx=0x7f315c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f3153fff700 1 -- 172.21.15.33:0/1215721892 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f315c00e040 con 0x7f316c0fde60 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c10dda0 con 0x7f316c0fde60 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f3153fff700 1 -- 172.21.15.33:0/1215721892 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f315c009af0 con 0x7f316c0fde60 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f3153fff700 1 -- 172.21.15.33:0/1215721892 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f315c01c630 con 0x7f316c0fde60 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.168+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c10e2e0 con 0x7f316c0fde60 2024-04-25T02:08:08.177 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.169+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c066ad0 con 0x7f316c0fde60 2024-04-25T02:08:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.170+0000 7f3153fff700 1 -- 172.21.15.33:0/1215721892 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f315c00bbc0 con 0x7f316c0fde60 2024-04-25T02:08:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.170+0000 7f3153fff700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f315406e140 0x7f31540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.170+0000 7f3153fff700 1 -- 172.21.15.33:0/1215721892 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f315c08eec0 con 0x7f316c0fde60 2024-04-25T02:08:08.178 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.170+0000 7f316affd700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f315406e140 0x7f31540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:08.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.171+0000 7f316affd700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f315406e140 0x7f31540705f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3160006fd0 tx=0x7f3160008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:08.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.175+0000 7f3153fff700 1 -- 172.21.15.33:0/1215721892 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f315c05bd50 con 0x7f316c0fde60 2024-04-25T02:08:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.475+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f316c109ea0 con 0x7f316c0fde60 2024-04-25T02:08:08.483 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.475+0000 7f3153fff700 1 -- 172.21.15.33:0/1215721892 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f315c05b8e0 con 0x7f316c0fde60 2024-04-25T02:08:08.483 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:08:08.483 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:08:08.484 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:08:08.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:08:08.484 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:08:08.484 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:08:08.484 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:08:08.484 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:08:08.484 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:08:08.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:08:08.485 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:08:08.485 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:08:08.485 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:08:08.485 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:08:08.485 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:08:08.485 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:08:08.486 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:08:08.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.477+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f315406e140 msgr2=0x7f31540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.477+0000 7f3171b7d700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f315406e140 0x7f31540705f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3160006fd0 tx=0x7f3160008040 comp rx=0 tx=0).stop 2024-04-25T02:08:08.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.477+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 msgr2=0x7f316c113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.477+0000 7f3171b7d700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 0x7f316c113930 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f315c009380 tx=0x7f315c009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:08.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.478+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 shutdown_connections 2024-04-25T02:08:08.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.478+0000 7f3171b7d700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f315406e140 0x7f31540705f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.478+0000 7f3171b7d700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f316c0fde60 0x7f316c113930 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.478+0000 7f3171b7d700 1 --2- 172.21.15.33:0/1215721892 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f316c0fe800 0x7f316c113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.478+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 >> 172.21.15.33:0/1215721892 conn(0x7f316c0f7870 msgr2=0x7f316c0f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:08.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.478+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 shutdown_connections 2024-04-25T02:08:08.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.478+0000 7f3171b7d700 1 -- 172.21.15.33:0/1215721892 wait complete. 2024-04-25T02:08:08.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.582+0000 7fd8dea24700 1 -- 172.21.15.33:0/421365311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 msgr2=0x7fd8d80fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.582+0000 7fd8dea24700 1 --2- 172.21.15.33:0/421365311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 0x7fd8d80fe1d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0004940 tx=0x7fd8c0009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:08.589 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:08 smithi033 ceph-mon[111306]: pgmap v863: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:08.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.582+0000 7fd8dea24700 1 -- 172.21.15.33:0/421365311 shutdown_connections 2024-04-25T02:08:08.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.582+0000 7fd8dea24700 1 --2- 172.21.15.33:0/421365311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d80fe7a0 0x7fd8d8106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.582+0000 7fd8dea24700 1 --2- 172.21.15.33:0/421365311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 0x7fd8d80fe1d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.582+0000 7fd8dea24700 1 -- 172.21.15.33:0/421365311 >> 172.21.15.33:0/421365311 conn(0x7fd8d80f7870 msgr2=0x7fd8d80f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:08.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.582+0000 7fd8dea24700 1 -- 172.21.15.33:0/421365311 shutdown_connections 2024-04-25T02:08:08.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.582+0000 7fd8dea24700 1 -- 172.21.15.33:0/421365311 wait complete. 2024-04-25T02:08:08.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.582+0000 7fd8dea24700 1 Processor -- start 2024-04-25T02:08:08.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8dea24700 1 -- start start 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8dea24700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 0x7fd8d80fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8dea24700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d80fe7a0 0x7fd8d80fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8dea24700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80fdb80 con 0x7fd8d80fde00 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8dea24700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d80fc6b0 con 0x7fd8d80fe7a0 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 0x7fd8d80fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8d7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 0x7fd8d80fbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56336/0 (socket says 172.21.15.33:56336) 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8d7fff700 1 -- 172.21.15.33:0/3869634044 learned_addr learned my addr 172.21.15.33:0/3869634044 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8d77fe700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d80fe7a0 0x7fd8d80fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8d7fff700 1 -- 172.21.15.33:0/3869634044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d80fe7a0 msgr2=0x7fd8d80fc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8d7fff700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d80fe7a0 0x7fd8d80fc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.583+0000 7fd8d7fff700 1 -- 172.21.15.33:0/3869634044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c0005040 con 0x7fd8d80fde00 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.584+0000 7fd8d7fff700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 0x7fd8d80fbc30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000b3b0 tx=0x7fd8c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.584+0000 7fd8d4ff9700 1 -- 172.21.15.33:0/3869634044 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8c000e040 con 0x7fd8d80fde00 2024-04-25T02:08:08.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.584+0000 7fd8d4ff9700 1 -- 172.21.15.33:0/3869634044 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8c0013420 con 0x7fd8d80fde00 2024-04-25T02:08:08.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.584+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d80fc930 con 0x7fd8d80fde00 2024-04-25T02:08:08.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.584+0000 7fd8d4ff9700 1 -- 172.21.15.33:0/3869634044 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8c001c630 con 0x7fd8d80fde00 2024-04-25T02:08:08.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.584+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d811a040 con 0x7fd8d80fde00 2024-04-25T02:08:08.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.585+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d8066b30 con 0x7fd8d80fde00 2024-04-25T02:08:08.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.585+0000 7fd8d4ff9700 1 -- 172.21.15.33:0/3869634044 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd8c001c8f0 con 0x7fd8d80fde00 2024-04-25T02:08:08.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.586+0000 7fd8d4ff9700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd8c406e140 0x7fd8c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:08.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.586+0000 7fd8d4ff9700 1 -- 172.21.15.33:0/3869634044 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd8c008ed50 con 0x7fd8d80fde00 2024-04-25T02:08:08.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.586+0000 7fd8d77fe700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd8c406e140 0x7fd8c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:08.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.586+0000 7fd8d77fe700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd8c406e140 0x7fd8c40705f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80fd360 tx=0x7fd8c80043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:08.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.591+0000 7fd8d4ff9700 1 -- 172.21.15.33:0/3869634044 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd8c0058150 con 0x7fd8d80fde00 2024-04-25T02:08:08.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:08 smithi138 ceph-mon[81304]: pgmap v863: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:08.849 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.842+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd8d80fd1a0 con 0x7fd8d80fde00 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.843+0000 7fd8d4ff9700 1 -- 172.21.15.33:0/3869634044 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fd8c005b810 con 0x7fd8d80fde00 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:08:08.850 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:08:08.851 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:08:08.852 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:08:08.853 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:08:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd8c406e140 msgr2=0x7fd8c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd8c406e140 0x7fd8c40705f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd8d80fd360 tx=0x7fd8c80043c0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.853 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 msgr2=0x7fd8d80fbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 0x7fd8d80fbc30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000b3b0 tx=0x7fd8c0009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 shutdown_connections 2024-04-25T02:08:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd8c406e140 0x7fd8c40705f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd8d80fde00 0x7fd8d80fbc30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 --2- 172.21.15.33:0/3869634044 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd8d80fe7a0 0x7fd8d80fc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 >> 172.21.15.33:0/3869634044 conn(0x7fd8d80f7870 msgr2=0x7fd8d80f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 shutdown_connections 2024-04-25T02:08:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.845+0000 7fd8dea24700 1 -- 172.21.15.33:0/3869634044 wait complete. 2024-04-25T02:08:08.854 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.950+0000 7f88b73ab700 1 -- 172.21.15.33:0/2432573119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fbc00 msgr2=0x7f88b00fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.950+0000 7f88b73ab700 1 --2- 172.21.15.33:0/2432573119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fbc00 0x7f88b00fbfd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f88ac009ce0 tx=0x7f88ac004d10 comp rx=0 tx=0).stop 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.950+0000 7f88b73ab700 1 -- 172.21.15.33:0/2432573119 shutdown_connections 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.950+0000 7f88b73ab700 1 --2- 172.21.15.33:0/2432573119 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b00fc5a0 0x7f88b0100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.950+0000 7f88b73ab700 1 --2- 172.21.15.33:0/2432573119 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fbc00 0x7f88b00fbfd0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.950+0000 7f88b73ab700 1 -- 172.21.15.33:0/2432573119 >> 172.21.15.33:0/2432573119 conn(0x7f88b00f7860 msgr2=0x7f88b00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.950+0000 7f88b73ab700 1 -- 172.21.15.33:0/2432573119 shutdown_connections 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.950+0000 7f88b73ab700 1 -- 172.21.15.33:0/2432573119 wait complete. 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b73ab700 1 Processor -- start 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b73ab700 1 -- start start 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b73ab700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b00fbc00 0x7f88b011a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b73ab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fc5a0 0x7f88b011a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:08.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b73ab700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b011ac80 con 0x7f88b00fc5a0 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b73ab700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b0114260 con 0x7f88b00fbc00 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b4946700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fc5a0 0x7f88b011a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b4946700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fc5a0 0x7f88b011a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56352/0 (socket says 172.21.15.33:56352) 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b4946700 1 -- 172.21.15.33:0/2048683609 learned_addr learned my addr 172.21.15.33:0/2048683609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b5147700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b00fbc00 0x7f88b011a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b4946700 1 -- 172.21.15.33:0/2048683609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b00fbc00 msgr2=0x7f88b011a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b4946700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b00fbc00 0x7f88b011a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b4946700 1 -- 172.21.15.33:0/2048683609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac009940 con 0x7f88b00fc5a0 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b5147700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b00fbc00 0x7f88b011a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.951+0000 7f88b4946700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fc5a0 0x7f88b011a630 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f88a0002a80 tx=0x7f88a00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.952+0000 7f88a5ffb700 1 -- 172.21.15.33:0/2048683609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88a0007cc0 con 0x7f88b00fc5a0 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.952+0000 7f88a5ffb700 1 -- 172.21.15.33:0/2048683609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88a000f040 con 0x7f88b00fc5a0 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.952+0000 7f88a5ffb700 1 -- 172.21.15.33:0/2048683609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88a00096f0 con 0x7f88b00fc5a0 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.952+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b0114540 con 0x7f88b00fc5a0 2024-04-25T02:08:08.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.952+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b0114a30 con 0x7f88b00fc5a0 2024-04-25T02:08:08.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.953+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b0066b50 con 0x7f88b00fc5a0 2024-04-25T02:08:08.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.953+0000 7f88a5ffb700 1 -- 172.21.15.33:0/2048683609 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f88a000d690 con 0x7f88b00fc5a0 2024-04-25T02:08:08.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.954+0000 7f88a5ffb700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f889c06e140 0x7f889c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:08.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.954+0000 7f88a5ffb700 1 -- 172.21.15.33:0/2048683609 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f88a008d2d0 con 0x7f88b00fc5a0 2024-04-25T02:08:08.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.954+0000 7f88b5147700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f889c06e140 0x7f889c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:08.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.954+0000 7f88b5147700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f889c06e140 0x7f889c0705f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00b740 tx=0x7f88ac004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:08.965 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:08.959+0000 7f88a5ffb700 1 -- 172.21.15.33:0/2048683609 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f88a005a180 con 0x7f88b00fc5a0 2024-04-25T02:08:09.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.183+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f88b0110580 con 0x7f889c06e140 2024-04-25T02:08:09.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.185+0000 7f88a5ffb700 1 -- 172.21.15.33:0/2048683609 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f88b0110580 con 0x7f889c06e140 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:08:09.192 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:08:09.193 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:08:09.193 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:08:09.193 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:08:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f889c06e140 msgr2=0x7f889c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f889c06e140 0x7f889c0705f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00b740 tx=0x7f88ac004d10 comp rx=0 tx=0).stop 2024-04-25T02:08:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fc5a0 msgr2=0x7f88b011a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fc5a0 0x7f88b011a630 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f88a0002a80 tx=0x7f88a00046b0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 shutdown_connections 2024-04-25T02:08:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f889c06e140 0x7f889c0705f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f88b00fbc00 0x7f88b011a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 --2- 172.21.15.33:0/2048683609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f88b00fc5a0 0x7f88b011a630 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.194 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 >> 172.21.15.33:0/2048683609 conn(0x7f88b00f7860 msgr2=0x7f88b010c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 shutdown_connections 2024-04-25T02:08:09.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.187+0000 7f88b73ab700 1 -- 172.21.15.33:0/2048683609 wait complete. 2024-04-25T02:08:09.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.291+0000 7f019a8de700 1 -- 172.21.15.33:0/3932490421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102df0 msgr2=0x7f01941031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:09.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.291+0000 7f019a8de700 1 --2- 172.21.15.33:0/3932490421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102df0 0x7f01941031c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0184004940 tx=0x7f0184009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:09.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.291+0000 7f019a8de700 1 -- 172.21.15.33:0/3932490421 shutdown_connections 2024-04-25T02:08:09.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.291+0000 7f019a8de700 1 --2- 172.21.15.33:0/3932490421 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194103790 0x7f01940fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.291+0000 7f019a8de700 1 --2- 172.21.15.33:0/3932490421 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194102df0 0x7f01941031c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.291+0000 7f019a8de700 1 -- 172.21.15.33:0/3932490421 >> 172.21.15.33:0/3932490421 conn(0x7f01940f8980 msgr2=0x7f01940fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:09.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.291+0000 7f019a8de700 1 -- 172.21.15.33:0/3932490421 shutdown_connections 2024-04-25T02:08:09.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.291+0000 7f019a8de700 1 -- 172.21.15.33:0/3932490421 wait complete. 2024-04-25T02:08:09.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f019a8de700 1 Processor -- start 2024-04-25T02:08:09.299 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f019a8de700 1 -- start start 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f019a8de700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194102df0 0x7f0194113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f019a8de700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194103790 0x7f01941140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f019a8de700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0194114730 con 0x7f0194103790 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f019a8de700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019410db40 con 0x7f0194102df0 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f01937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194103790 0x7f01941140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f01937fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194103790 0x7f01941140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:56372/0 (socket says 172.21.15.33:56372) 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f01937fe700 1 -- 172.21.15.33:0/2694033337 learned_addr learned my addr 172.21.15.33:0/2694033337 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f0193fff700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194102df0 0x7f0194113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f01937fe700 1 -- 172.21.15.33:0/2694033337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194102df0 msgr2=0x7f0194113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f01937fe700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194102df0 0x7f0194113ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f01937fe700 1 -- 172.21.15.33:0/2694033337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0184005040 con 0x7f0194103790 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.292+0000 7f0193fff700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194102df0 0x7f0194113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:08:09.301 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.293+0000 7f01937fe700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194103790 0x7f01941140e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0188002a50 tx=0x7f0188004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.293+0000 7f0190ff9700 1 -- 172.21.15.33:0/2694033337 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0188007cc0 con 0x7f0194103790 2024-04-25T02:08:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.293+0000 7f0190ff9700 1 -- 172.21.15.33:0/2694033337 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f018800f040 con 0x7f0194103790 2024-04-25T02:08:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.293+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019410de20 con 0x7f0194103790 2024-04-25T02:08:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.293+0000 7f0190ff9700 1 -- 172.21.15.33:0/2694033337 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f018800daf0 con 0x7f0194103790 2024-04-25T02:08:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.293+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019410e3e0 con 0x7f0194103790 2024-04-25T02:08:09.302 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.294+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0194100100 con 0x7f0194103790 2024-04-25T02:08:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.294+0000 7f0190ff9700 1 -- 172.21.15.33:0/2694033337 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0188009810 con 0x7f0194103790 2024-04-25T02:08:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.295+0000 7f0190ff9700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f017c06e140 0x7f017c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.295+0000 7f0190ff9700 1 -- 172.21.15.33:0/2694033337 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f018808d810 con 0x7f0194103790 2024-04-25T02:08:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.295+0000 7f0193fff700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f017c06e140 0x7f017c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:09.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.295+0000 7f0193fff700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f017c06e140 0x7f017c0705f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0184004f90 tx=0x7f0184009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:09.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.301+0000 7f0190ff9700 1 -- 172.21.15.33:0/2694033337 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0188056c10 con 0x7f0194103790 2024-04-25T02:08:09.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:09 smithi033 ceph-mon[111306]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:09.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:09 smithi033 ceph-mon[111306]: from='client.35584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:09.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:09 smithi033 ceph-mon[111306]: from='client.35588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:09.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:09 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1215721892' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:08:09.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:09 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3869634044' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:08:09.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.597+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f019410f1f0 con 0x7f0194103790 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.597+0000 7f0190ff9700 1 -- 172.21.15.33:0/2694033337 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f018805a2d0 con 0x7f0194103790 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:08:09.605 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:08:09.606 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:08:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.599+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f017c06e140 msgr2=0x7f017c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.599+0000 7f019a8de700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f017c06e140 0x7f017c0705f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0184004f90 tx=0x7f0184009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.600+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194103790 msgr2=0x7f01941140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.600+0000 7f019a8de700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194103790 0x7f01941140e0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0188002a50 tx=0x7f0188004600 comp rx=0 tx=0).stop 2024-04-25T02:08:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.600+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 shutdown_connections 2024-04-25T02:08:09.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.600+0000 7f019a8de700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f017c06e140 0x7f017c0705f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.600+0000 7f019a8de700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0194102df0 0x7f0194113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.600+0000 7f019a8de700 1 --2- 172.21.15.33:0/2694033337 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0194103790 0x7f01941140e0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.600+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 >> 172.21.15.33:0/2694033337 conn(0x7f01940f8980 msgr2=0x7f01941073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.600+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 shutdown_connections 2024-04-25T02:08:09.608 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:09.600+0000 7f019a8de700 1 -- 172.21.15.33:0/2694033337 wait complete. 2024-04-25T02:08:09.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:09 smithi138 ceph-mon[81304]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:09.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:09 smithi138 ceph-mon[81304]: from='client.35584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:09.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:09 smithi138 ceph-mon[81304]: from='client.35588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:09.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:09 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1215721892' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:08:09.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:09 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3869634044' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:08:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:10 smithi033 ceph-mon[111306]: from='client.35600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:10 smithi033 ceph-mon[111306]: pgmap v864: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:10 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2694033337' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:08:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:08:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:10 smithi138 ceph-mon[81304]: from='client.35600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:10 smithi138 ceph-mon[81304]: pgmap v864: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:10 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2694033337' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:08:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:08:12.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:12 smithi033 ceph-mon[111306]: pgmap v865: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:12.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:12 smithi138 ceph-mon[81304]: pgmap v865: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:14 smithi033 ceph-mon[111306]: pgmap v866: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:14 smithi138 ceph-mon[81304]: pgmap v866: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:16.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:16 smithi033 ceph-mon[111306]: pgmap v867: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:16.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:16 smithi138 ceph-mon[81304]: pgmap v867: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:18.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:18 smithi033 ceph-mon[111306]: pgmap v868: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:18.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:18 smithi138 ceph-mon[81304]: pgmap v868: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:20.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:20 smithi033 ceph-mon[111306]: pgmap v869: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:20.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:20 smithi138 ceph-mon[81304]: pgmap v869: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:22.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:22 smithi033 ceph-mon[111306]: pgmap v870: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:22.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:22 smithi138 ceph-mon[81304]: pgmap v870: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:24.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:24 smithi033 ceph-mon[111306]: pgmap v871: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:24.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:24 smithi138 ceph-mon[81304]: pgmap v871: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:25.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:08:25.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:08:26.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:26 smithi033 ceph-mon[111306]: pgmap v872: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:26.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:26 smithi138 ceph-mon[81304]: pgmap v872: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:28.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:28 smithi033 ceph-mon[111306]: pgmap v873: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:28.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:28 smithi138 ceph-mon[81304]: pgmap v873: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:30.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:30 smithi033 ceph-mon[111306]: pgmap v874: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:30.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:30 smithi138 ceph-mon[81304]: pgmap v874: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:32.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:32 smithi033 ceph-mon[111306]: pgmap v875: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:32.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:32 smithi138 ceph-mon[81304]: pgmap v875: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:34.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:34 smithi033 ceph-mon[111306]: pgmap v876: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:34.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:34 smithi138 ceph-mon[81304]: pgmap v876: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:36.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:36 smithi033 ceph-mon[111306]: pgmap v877: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:36.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:36 smithi138 ceph-mon[81304]: pgmap v877: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:38.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:38 smithi033 ceph-mon[111306]: pgmap v878: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:38.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:38 smithi138 ceph-mon[81304]: pgmap v878: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.718+0000 7f583f25a700 1 -- 172.21.15.33:0/2305302508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 msgr2=0x7f58380fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.718+0000 7f583f25a700 1 --2- 172.21.15.33:0/2305302508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 0x7f58380fcdf0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f582c004940 tx=0x7f582c009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.718+0000 7f583f25a700 1 -- 172.21.15.33:0/2305302508 shutdown_connections 2024-04-25T02:08:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.718+0000 7f583f25a700 1 --2- 172.21.15.33:0/2305302508 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 0x7f58380fcdf0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.718+0000 7f583f25a700 1 --2- 172.21.15.33:0/2305302508 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5838102de0 0x7f58381031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:39.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.718+0000 7f583f25a700 1 -- 172.21.15.33:0/2305302508 >> 172.21.15.33:0/2305302508 conn(0x7f58380f8950 msgr2=0x7f58380fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:39.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.718+0000 7f583f25a700 1 -- 172.21.15.33:0/2305302508 shutdown_connections 2024-04-25T02:08:39.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.718+0000 7f583f25a700 1 -- 172.21.15.33:0/2305302508 wait complete. 2024-04-25T02:08:39.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.719+0000 7f583f25a700 1 Processor -- start 2024-04-25T02:08:39.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.719+0000 7f583f25a700 1 -- start start 2024-04-25T02:08:39.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.719+0000 7f583f25a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5838102de0 0x7f5838115da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:39.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.719+0000 7f583f25a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 0x7f58381162e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:39.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.719+0000 7f583f25a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5838116930 con 0x7f5838103780 2024-04-25T02:08:39.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.719+0000 7f583f25a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583810fd00 con 0x7f5838102de0 2024-04-25T02:08:39.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f5837fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 0x7f58381162e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f5837fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 0x7f58381162e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44596/0 (socket says 172.21.15.33:44596) 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f5837fff700 1 -- 172.21.15.33:0/3448898434 learned_addr learned my addr 172.21.15.33:0/3448898434 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f583cff6700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5838102de0 0x7f5838115da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f5837fff700 1 -- 172.21.15.33:0/3448898434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5838102de0 msgr2=0x7f5838115da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f5837fff700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5838102de0 0x7f5838115da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f5837fff700 1 -- 172.21.15.33:0/3448898434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c005040 con 0x7f5838103780 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f5837fff700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 0x7f58381162e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f582c009100 tx=0x7f582c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f58357fa700 1 -- 172.21.15.33:0/3448898434 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582c00e040 con 0x7f5838103780 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f58357fa700 1 -- 172.21.15.33:0/3448898434 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f582c009af0 con 0x7f5838103780 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583810ff80 con 0x7f5838103780 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f58357fa700 1 -- 172.21.15.33:0/3448898434 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582c01c3f0 con 0x7f5838103780 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.720+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5838110540 con 0x7f5838103780 2024-04-25T02:08:39.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.721+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58380fe530 con 0x7f5838103780 2024-04-25T02:08:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.722+0000 7f58357fa700 1 -- 172.21.15.33:0/3448898434 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f582c00baf0 con 0x7f5838103780 2024-04-25T02:08:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.722+0000 7f58357fa700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f582006e140 0x7f58200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.723+0000 7f583cff6700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f582006e140 0x7f58200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.723+0000 7f58357fa700 1 -- 172.21.15.33:0/3448898434 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f582c08ec90 con 0x7f5838103780 2024-04-25T02:08:39.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.723+0000 7f583cff6700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f582006e140 0x7f58200705f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5828007fd0 tx=0x7f5828009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:39.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.727+0000 7f58357fa700 1 -- 172.21.15.33:0/3448898434 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f582c05bc60 con 0x7f5838103780 2024-04-25T02:08:39.960 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.953+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58380fe530 con 0x7f582006e140 2024-04-25T02:08:39.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.954+0000 7f58357fa700 1 -- 172.21.15.33:0/3448898434 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f58380fe530 con 0x7f582006e140 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f582006e140 msgr2=0x7f58200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f582006e140 0x7f58200705f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5828007fd0 tx=0x7f5828009040 comp rx=0 tx=0).stop 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 msgr2=0x7f58381162e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 0x7f58381162e0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f582c009100 tx=0x7f582c009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 shutdown_connections 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f582006e140 0x7f58200705f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5838102de0 0x7f5838115da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 --2- 172.21.15.33:0/3448898434 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5838103780 0x7f58381162e0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 >> 172.21.15.33:0/3448898434 conn(0x7f58380f8950 msgr2=0x7f58380f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 shutdown_connections 2024-04-25T02:08:39.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:39.956+0000 7f583f25a700 1 -- 172.21.15.33:0/3448898434 wait complete. 2024-04-25T02:08:39.974 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:08:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.062+0000 7f549759e700 1 -- 172.21.15.33:0/2633321837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fdc70 msgr2=0x7f54980fe040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.062+0000 7f549759e700 1 --2- 172.21.15.33:0/2633321837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fdc70 0x7f54980fe040 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5480004940 tx=0x7f5480009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.062+0000 7f549759e700 1 -- 172.21.15.33:0/2633321837 shutdown_connections 2024-04-25T02:08:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.062+0000 7f549759e700 1 --2- 172.21.15.33:0/2633321837 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54980fe580 0x7f5498106c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.062+0000 7f549759e700 1 --2- 172.21.15.33:0/2633321837 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fdc70 0x7f54980fe040 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.062+0000 7f549759e700 1 -- 172.21.15.33:0/2633321837 >> 172.21.15.33:0/2633321837 conn(0x7f54980f7640 msgr2=0x7f54980f9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:40.070 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.062+0000 7f549759e700 1 -- 172.21.15.33:0/2633321837 shutdown_connections 2024-04-25T02:08:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.062+0000 7f549759e700 1 -- 172.21.15.33:0/2633321837 wait complete. 2024-04-25T02:08:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.063+0000 7f549759e700 1 Processor -- start 2024-04-25T02:08:40.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.063+0000 7f549759e700 1 -- start start 2024-04-25T02:08:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.063+0000 7f549759e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54980fdc70 0x7f54980fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.063+0000 7f549759e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fe580 0x7f54980fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.063+0000 7f549759e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980fc570 con 0x7f54980fe580 2024-04-25T02:08:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.063+0000 7f549759e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54980fc6b0 con 0x7f54980fdc70 2024-04-25T02:08:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.063+0000 7f5495d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fe580 0x7f54980fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.063+0000 7f5495d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fe580 0x7f54980fbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44600/0 (socket says 172.21.15.33:44600) 2024-04-25T02:08:40.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.063+0000 7f5495d9b700 1 -- 172.21.15.33:0/1145786795 learned_addr learned my addr 172.21.15.33:0/1145786795 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f549659c700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54980fdc70 0x7f54980fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f5495d9b700 1 -- 172.21.15.33:0/1145786795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54980fdc70 msgr2=0x7f54980fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f5495d9b700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54980fdc70 0x7f54980fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f5495d9b700 1 -- 172.21.15.33:0/1145786795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480005040 con 0x7f54980fe580 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f5495d9b700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fe580 0x7f54980fbfe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5488002a80 tx=0x7f54880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f548effd700 1 -- 172.21.15.33:0/1145786795 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5488007cc0 con 0x7f54980fe580 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f548effd700 1 -- 172.21.15.33:0/1145786795 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f548800f040 con 0x7f54980fe580 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5498119d20 con 0x7f54980fe580 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f548effd700 1 -- 172.21.15.33:0/1145786795 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548800dac0 con 0x7f54980fe580 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.064+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549811a120 con 0x7f54980fe580 2024-04-25T02:08:40.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.065+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54980668b0 con 0x7f54980fe580 2024-04-25T02:08:40.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.066+0000 7f548effd700 1 -- 172.21.15.33:0/1145786795 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f548800dc20 con 0x7f54980fe580 2024-04-25T02:08:40.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.066+0000 7f548effd700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e140 0x7f54840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:40.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.066+0000 7f548effd700 1 -- 172.21.15.33:0/1145786795 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f548808d750 con 0x7f54980fe580 2024-04-25T02:08:40.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.066+0000 7f549659c700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e140 0x7f54840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:40.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.067+0000 7f549659c700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e140 0x7f54840705f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5480004f90 tx=0x7f5480009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:40.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.071+0000 7f548effd700 1 -- 172.21.15.33:0/1145786795 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5488056ad0 con 0x7f54980fe580 2024-04-25T02:08:40.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.304+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5498109d90 con 0x7f548406e140 2024-04-25T02:08:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.307+0000 7f548effd700 1 -- 172.21.15.33:0/1145786795 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5498109d90 con 0x7f548406e140 2024-04-25T02:08:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e140 msgr2=0x7f54840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e140 0x7f54840705f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5480004f90 tx=0x7f5480009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fe580 msgr2=0x7f54980fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fe580 0x7f54980fbfe0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f5488002a80 tx=0x7f54880046b0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 shutdown_connections 2024-04-25T02:08:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f548406e140 0x7f54840705f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f54980fdc70 0x7f54980fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 --2- 172.21.15.33:0/1145786795 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f54980fe580 0x7f54980fbfe0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 >> 172.21.15.33:0/1145786795 conn(0x7f54980f7640 msgr2=0x7f54980f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 shutdown_connections 2024-04-25T02:08:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.309+0000 7f549759e700 1 -- 172.21.15.33:0/1145786795 wait complete. 2024-04-25T02:08:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.415+0000 7f061570d700 1 -- 172.21.15.33:0/3997672272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610103780 msgr2=0x7f06100fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.415+0000 7f061570d700 1 --2- 172.21.15.33:0/3997672272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610103780 0x7f06100fcdf0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f0604004940 tx=0x7f0604009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.415+0000 7f061570d700 1 -- 172.21.15.33:0/3997672272 shutdown_connections 2024-04-25T02:08:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.415+0000 7f061570d700 1 --2- 172.21.15.33:0/3997672272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610103780 0x7f06100fcdf0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.415+0000 7f061570d700 1 --2- 172.21.15.33:0/3997672272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0610102de0 0x7f06101031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.415+0000 7f061570d700 1 -- 172.21.15.33:0/3997672272 >> 172.21.15.33:0/3997672272 conn(0x7f06100f8990 msgr2=0x7f06100fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.415+0000 7f061570d700 1 -- 172.21.15.33:0/3997672272 shutdown_connections 2024-04-25T02:08:40.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.415+0000 7f061570d700 1 -- 172.21.15.33:0/3997672272 wait complete. 2024-04-25T02:08:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.416+0000 7f061570d700 1 Processor -- start 2024-04-25T02:08:40.424 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.416+0000 7f061570d700 1 -- start start 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.416+0000 7f061570d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610102de0 0x7f0610115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.416+0000 7f061570d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0610103780 0x7f06101160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.416+0000 7f061570d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0610116720 con 0x7f0610102de0 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.416+0000 7f061570d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061010fd00 con 0x7f0610103780 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.416+0000 7f060effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610102de0 0x7f0610115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.416+0000 7f060effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610102de0 0x7f0610115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44624/0 (socket says 172.21.15.33:44624) 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.416+0000 7f060effd700 1 -- 172.21.15.33:0/2201225199 learned_addr learned my addr 172.21.15.33:0/2201225199 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f060e7fc700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0610103780 0x7f06101160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f060effd700 1 -- 172.21.15.33:0/2201225199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0610103780 msgr2=0x7f06101160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f060effd700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0610103780 0x7f06101160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.425 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f060effd700 1 -- 172.21.15.33:0/2201225199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0604005040 con 0x7f0610102de0 2024-04-25T02:08:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f060effd700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610102de0 0x7f0610115b90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0600002a50 tx=0x7f06000046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f05f77fe700 1 -- 172.21.15.33:0/2201225199 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0600007cc0 con 0x7f0610102de0 2024-04-25T02:08:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061010ffe0 con 0x7f0610102de0 2024-04-25T02:08:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f05f77fe700 1 -- 172.21.15.33:0/2201225199 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f060000f040 con 0x7f0610102de0 2024-04-25T02:08:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f05f77fe700 1 -- 172.21.15.33:0/2201225199 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f060000da30 con 0x7f0610102de0 2024-04-25T02:08:40.426 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.417+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0610110570 con 0x7f0610102de0 2024-04-25T02:08:40.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.418+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06100fe530 con 0x7f0610102de0 2024-04-25T02:08:40.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.419+0000 7f05f77fe700 1 -- 172.21.15.33:0/2201225199 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f060000db90 con 0x7f0610102de0 2024-04-25T02:08:40.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.419+0000 7f05f77fe700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f05f806e140 0x7f05f80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:40.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.419+0000 7f05f77fe700 1 -- 172.21.15.33:0/2201225199 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f060008d8c0 con 0x7f0610102de0 2024-04-25T02:08:40.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.419+0000 7f060e7fc700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f05f806e140 0x7f05f80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:40.427 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.420+0000 7f060e7fc700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f05f806e140 0x7f05f80705f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0604004890 tx=0x7f0604009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:40.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.424+0000 7f05f77fe700 1 -- 172.21.15.33:0/2201225199 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f060005a770 con 0x7f0610102de0 2024-04-25T02:08:40.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.639+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0610111250 con 0x7f05f806e140 2024-04-25T02:08:40.646 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:40 smithi033 ceph-mon[111306]: pgmap v879: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:40.646 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:08:40.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.649+0000 7f05f77fe700 1 -- 172.21.15.33:0/2201225199 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0610111250 con 0x7f05f806e140 2024-04-25T02:08:40.656 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:08:40.656 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (29m) 7m ago 40m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (27m) 7m ago 41m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (26m) 7m ago 40m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (29m) 7m ago 40m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (23m) 7m ago 35m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (23m) 7m ago 35m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (22m) 7m ago 35m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (7m) 7m ago 35m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (31m) 7m ago 42m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (30m) 7m ago 40m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (28m) 7m ago 42m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (27m) 7m ago 40m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (30m) 7m ago 41m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (30m) 7m ago 40m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (26m) 7m ago 39m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:08:40.657 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (26m) 7m ago 38m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:08:40.658 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (25m) 7m ago 38m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:08:40.658 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (24m) 7m ago 37m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:08:40.658 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (24m) 7m ago 37m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:08:40.658 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (23m) 7m ago 37m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:08:40.658 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (30m) 7m ago 40m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:08:40.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f05f806e140 msgr2=0x7f05f80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f05f806e140 0x7f05f80705f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0604004890 tx=0x7f0604009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610102de0 msgr2=0x7f0610115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610102de0 0x7f0610115b90 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f0600002a50 tx=0x7f06000046b0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 shutdown_connections 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f05f806e140 0x7f05f80705f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0610102de0 0x7f0610115b90 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 --2- 172.21.15.33:0/2201225199 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0610103780 0x7f06101160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 >> 172.21.15.33:0/2201225199 conn(0x7f06100f8990 msgr2=0x7f0610071c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 shutdown_connections 2024-04-25T02:08:40.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.651+0000 7f061570d700 1 -- 172.21.15.33:0/2201225199 wait complete. 2024-04-25T02:08:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.754+0000 7fb567341700 1 -- 172.21.15.33:0/3730092192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560102de0 msgr2=0x7fb5601031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.754+0000 7fb567341700 1 --2- 172.21.15.33:0/3730092192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560102de0 0x7fb5601031b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb55c004940 tx=0x7fb55c009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.754+0000 7fb567341700 1 -- 172.21.15.33:0/3730092192 shutdown_connections 2024-04-25T02:08:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.754+0000 7fb567341700 1 --2- 172.21.15.33:0/3730092192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560103780 0x7fb5600fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.754+0000 7fb567341700 1 --2- 172.21.15.33:0/3730092192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560102de0 0x7fb5601031b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.754+0000 7fb567341700 1 -- 172.21.15.33:0/3730092192 >> 172.21.15.33:0/3730092192 conn(0x7fb5600f8990 msgr2=0x7fb5600fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.755+0000 7fb567341700 1 -- 172.21.15.33:0/3730092192 shutdown_connections 2024-04-25T02:08:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.755+0000 7fb567341700 1 -- 172.21.15.33:0/3730092192 wait complete. 2024-04-25T02:08:40.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.755+0000 7fb567341700 1 Processor -- start 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.755+0000 7fb567341700 1 -- start start 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.755+0000 7fb567341700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560102de0 0x7fb560113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb567341700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560103780 0x7fb560113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb567341700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560114550 con 0x7fb560103780 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb567341700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56010dac0 con 0x7fb560102de0 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb5648dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560103780 0x7fb560113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb5648dc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560103780 0x7fb560113e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44634/0 (socket says 172.21.15.33:44634) 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb5648dc700 1 -- 172.21.15.33:0/3143170006 learned_addr learned my addr 172.21.15.33:0/3143170006 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb5650dd700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560102de0 0x7fb560113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb5648dc700 1 -- 172.21.15.33:0/3143170006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560102de0 msgr2=0x7fb560113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb5648dc700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560102de0 0x7fb560113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:40.764 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb5648dc700 1 -- 172.21.15.33:0/3143170006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c005040 con 0x7fb560103780 2024-04-25T02:08:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb5648dc700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560103780 0x7fb560113e70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb55000b790 tx=0x7fb550009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb555ffb700 1 -- 172.21.15.33:0/3143170006 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb55000f6f0 con 0x7fb560103780 2024-04-25T02:08:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb555ffb700 1 -- 172.21.15.33:0/3143170006 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb55000fd20 con 0x7fb560103780 2024-04-25T02:08:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb56010dda0 con 0x7fb560103780 2024-04-25T02:08:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.756+0000 7fb555ffb700 1 -- 172.21.15.33:0/3143170006 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5500103a0 con 0x7fb560103780 2024-04-25T02:08:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.757+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb56010e360 con 0x7fb560103780 2024-04-25T02:08:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.758+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5601000f0 con 0x7fb560103780 2024-04-25T02:08:40.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.758+0000 7fb555ffb700 1 -- 172.21.15.33:0/3143170006 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb550011400 con 0x7fb560103780 2024-04-25T02:08:40.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.758+0000 7fb555ffb700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb54c06e140 0x7fb54c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:40.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.758+0000 7fb555ffb700 1 -- 172.21.15.33:0/3143170006 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb55008d8d0 con 0x7fb560103780 2024-04-25T02:08:40.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.759+0000 7fb5650dd700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb54c06e140 0x7fb54c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:40.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.759+0000 7fb5650dd700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb54c06e140 0x7fb54c0705f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb55c004f90 tx=0x7fb55c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:40.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:40.763+0000 7fb555ffb700 1 -- 172.21.15.33:0/3143170006 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb55005a7e0 con 0x7fb560103780 2024-04-25T02:08:40.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:40 smithi138 ceph-mon[81304]: pgmap v879: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:40.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:08:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.064+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb5600fe530 con 0x7fb560103780 2024-04-25T02:08:41.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.065+0000 7fb555ffb700 1 -- 172.21.15.33:0/3143170006 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb55005a370 con 0x7fb560103780 2024-04-25T02:08:41.072 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:08:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:08:41.072 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:08:41.073 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.067+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb54c06e140 msgr2=0x7fb54c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.067+0000 7fb567341700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb54c06e140 0x7fb54c0705f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb55c004f90 tx=0x7fb55c009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.067+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560103780 msgr2=0x7fb560113e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.067+0000 7fb567341700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560103780 0x7fb560113e70 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb55000b790 tx=0x7fb550009220 comp rx=0 tx=0).stop 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.067+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 shutdown_connections 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.067+0000 7fb567341700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb54c06e140 0x7fb54c0705f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.067+0000 7fb567341700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb560102de0 0x7fb560113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.068+0000 7fb567341700 1 --2- 172.21.15.33:0/3143170006 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb560103780 0x7fb560113e70 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.068+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 >> 172.21.15.33:0/3143170006 conn(0x7fb5600f8990 msgr2=0x7fb5600fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.068+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 shutdown_connections 2024-04-25T02:08:41.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.068+0000 7fb567341700 1 -- 172.21.15.33:0/3143170006 wait complete. 2024-04-25T02:08:41.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 -- 172.21.15.33:0/3825587118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 msgr2=0x7f00100fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 --2- 172.21.15.33:0/3825587118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 0x7f00100fe1d0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f0004004940 tx=0x7f0004009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:41.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 -- 172.21.15.33:0/3825587118 shutdown_connections 2024-04-25T02:08:41.179 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 --2- 172.21.15.33:0/3825587118 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100fe7a0 0x7f0010106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 --2- 172.21.15.33:0/3825587118 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 0x7f00100fe1d0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 -- 172.21.15.33:0/3825587118 >> 172.21.15.33:0/3825587118 conn(0x7f00100f7870 msgr2=0x7f00100f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:41.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 -- 172.21.15.33:0/3825587118 shutdown_connections 2024-04-25T02:08:41.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 -- 172.21.15.33:0/3825587118 wait complete. 2024-04-25T02:08:41.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 Processor -- start 2024-04-25T02:08:41.180 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.172+0000 7f0017170700 1 -- start start 2024-04-25T02:08:41.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0017170700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 0x7f00100fbc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:41.181 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0017170700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100fe7a0 0x7f00100fc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0017170700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00100fdb80 con 0x7f00100fde00 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0017170700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00100fc6b0 con 0x7f00100fe7a0 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0014f0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 0x7f00100fbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0014f0c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 0x7f00100fbc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44652/0 (socket says 172.21.15.33:44652) 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0014f0c700 1 -- 172.21.15.33:0/3371334698 learned_addr learned my addr 172.21.15.33:0/3371334698 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f000ffff700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100fe7a0 0x7f00100fc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0014f0c700 1 -- 172.21.15.33:0/3371334698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100fe7a0 msgr2=0x7f00100fc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0014f0c700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100fe7a0 0x7f00100fc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0014f0c700 1 -- 172.21.15.33:0/3371334698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0004005040 con 0x7f00100fde00 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0014f0c700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 0x7f00100fbc30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0004000c00 tx=0x7f0004009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f000d7fa700 1 -- 172.21.15.33:0/3371334698 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0004021070 con 0x7f00100fde00 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00100fc930 con 0x7f00100fde00 2024-04-25T02:08:41.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f000d7fa700 1 -- 172.21.15.33:0/3371334698 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0004013cd0 con 0x7f00100fde00 2024-04-25T02:08:41.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0010119fb0 con 0x7f00100fde00 2024-04-25T02:08:41.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.173+0000 7f000d7fa700 1 -- 172.21.15.33:0/3371334698 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f000400e070 con 0x7f00100fde00 2024-04-25T02:08:41.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.174+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0010066b30 con 0x7f00100fde00 2024-04-25T02:08:41.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.175+0000 7f000d7fa700 1 -- 172.21.15.33:0/3371334698 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f000401c9d0 con 0x7f00100fde00 2024-04-25T02:08:41.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.175+0000 7f000d7fa700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f000006e140 0x7f00000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:41.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.176+0000 7f000d7fa700 1 -- 172.21.15.33:0/3371334698 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f000408ef10 con 0x7f00100fde00 2024-04-25T02:08:41.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.176+0000 7f000ffff700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f000006e140 0x7f00000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:41.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.176+0000 7f000ffff700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f000006e140 0x7f00000705f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efffc004600 tx=0x7efffc00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:41.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.180+0000 7f000d7fa700 1 -- 172.21.15.33:0/3371334698 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f00040583b0 con 0x7f00100fde00 2024-04-25T02:08:41.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.437+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f001011a1f0 con 0x7f00100fde00 2024-04-25T02:08:41.444 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.437+0000 7f000d7fa700 1 -- 172.21.15.33:0/3371334698 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f0004004490 con 0x7f00100fde00 2024-04-25T02:08:41.444 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:08:41.445 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:08:41.446 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:08:41.447 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:08:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f000006e140 msgr2=0x7f00000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f000006e140 0x7f00000705f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efffc004600 tx=0x7efffc00b040 comp rx=0 tx=0).stop 2024-04-25T02:08:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 msgr2=0x7f00100fbc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 0x7f00100fbc30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f0004000c00 tx=0x7f0004009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 shutdown_connections 2024-04-25T02:08:41.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f000006e140 0x7f00000705f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f00100fde00 0x7f00100fbc30 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 --2- 172.21.15.33:0/3371334698 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f00100fe7a0 0x7f00100fc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 >> 172.21.15.33:0/3371334698 conn(0x7f00100f7870 msgr2=0x7f00100f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 shutdown_connections 2024-04-25T02:08:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.439+0000 7f0017170700 1 -- 172.21.15.33:0/3371334698 wait complete. 2024-04-25T02:08:41.449 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:08:41.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.545+0000 7f67047a7700 1 -- 172.21.15.33:0/3188231942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc102e10 msgr2=0x7f66fc1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.545+0000 7f67047a7700 1 --2- 172.21.15.33:0/3188231942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc102e10 0x7f66fc1031e0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f66f8004940 tx=0x7f66f8009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.545+0000 7f67047a7700 1 -- 172.21.15.33:0/3188231942 shutdown_connections 2024-04-25T02:08:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.545+0000 7f67047a7700 1 --2- 172.21.15.33:0/3188231942 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66fc0fb530 0x7f66fc0fb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.545+0000 7f67047a7700 1 --2- 172.21.15.33:0/3188231942 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc102e10 0x7f66fc1031e0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.545+0000 7f67047a7700 1 -- 172.21.15.33:0/3188231942 >> 172.21.15.33:0/3188231942 conn(0x7f66fc071380 msgr2=0x7f66fc071780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.545+0000 7f67047a7700 1 -- 172.21.15.33:0/3188231942 shutdown_connections 2024-04-25T02:08:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.545+0000 7f67047a7700 1 -- 172.21.15.33:0/3188231942 wait complete. 2024-04-25T02:08:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f67047a7700 1 Processor -- start 2024-04-25T02:08:41.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f67047a7700 1 -- start start 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f67047a7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc0fb530 0x7f66fc113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f67047a7700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66fc102e10 0x7f66fc114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f67047a7700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc1146e0 con 0x7f66fc0fb530 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f67047a7700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66fc10dad0 con 0x7f66fc102e10 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f6702543700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc0fb530 0x7f66fc113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f6702543700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc0fb530 0x7f66fc113b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44672/0 (socket says 172.21.15.33:44672) 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f6702543700 1 -- 172.21.15.33:0/2661171763 learned_addr learned my addr 172.21.15.33:0/2661171763 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f6701d42700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66fc102e10 0x7f66fc114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f6702543700 1 -- 172.21.15.33:0/2661171763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66fc102e10 msgr2=0x7f66fc114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f6702543700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66fc102e10 0x7f66fc114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.546+0000 7f6702543700 1 -- 172.21.15.33:0/2661171763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f8005040 con 0x7f66fc0fb530 2024-04-25T02:08:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.547+0000 7f6702543700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc0fb530 0x7f66fc113b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f66f800b3b0 tx=0x7f66f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.547+0000 7f66f2ffd700 1 -- 172.21.15.33:0/2661171763 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66f800e040 con 0x7f66fc0fb530 2024-04-25T02:08:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.547+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66fc10dd50 con 0x7f66fc0fb530 2024-04-25T02:08:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.547+0000 7f66f2ffd700 1 -- 172.21.15.33:0/2661171763 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66f8013420 con 0x7f66fc0fb530 2024-04-25T02:08:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.547+0000 7f66f2ffd700 1 -- 172.21.15.33:0/2661171763 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66f801c630 con 0x7f66fc0fb530 2024-04-25T02:08:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.547+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66fc10e310 con 0x7f66fc0fb530 2024-04-25T02:08:41.556 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.548+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66fc065dd0 con 0x7f66fc0fb530 2024-04-25T02:08:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.548+0000 7f66f2ffd700 1 -- 172.21.15.33:0/2661171763 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f66f8013590 con 0x7f66fc0fb530 2024-04-25T02:08:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.549+0000 7f66f2ffd700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66e806e140 0x7f66e80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.549+0000 7f6701d42700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66e806e140 0x7f66e80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.549+0000 7f66f2ffd700 1 -- 172.21.15.33:0/2661171763 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f66f808ed20 con 0x7f66fc0fb530 2024-04-25T02:08:41.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.549+0000 7f6701d42700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66e806e140 0x7f66e80705f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f66fc10ec90 tx=0x7f66ec0043c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:41.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.554+0000 7f66f2ffd700 1 -- 172.21.15.33:0/2661171763 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f66f805bc50 con 0x7f66fc0fb530 2024-04-25T02:08:41.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:41 smithi033 ceph-mon[111306]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:41.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:41 smithi033 ceph-mon[111306]: from='client.35612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:41.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:41 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3143170006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:08:41.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.779+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66fc109e90 con 0x7f66e806e140 2024-04-25T02:08:41.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.781+0000 7f66f2ffd700 1 -- 172.21.15.33:0/2661171763 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f66fc109e90 con 0x7f66e806e140 2024-04-25T02:08:41.788 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:08:41.788 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:08:41.788 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:08:41.788 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:08:41.788 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:08:41.788 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:08:41.788 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:08:41.789 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:08:41.789 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:08:41.789 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:08:41.789 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:08:41.789 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:08:41.789 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:08:41.789 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:08:41.789 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:08:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66e806e140 msgr2=0x7f66e80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66e806e140 0x7f66e80705f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f66fc10ec90 tx=0x7f66ec0043c0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc0fb530 msgr2=0x7f66fc113b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc0fb530 0x7f66fc113b50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f66f800b3b0 tx=0x7f66f8009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 shutdown_connections 2024-04-25T02:08:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f66e806e140 0x7f66e80705f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f66fc0fb530 0x7f66fc113b50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 --2- 172.21.15.33:0/2661171763 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f66fc102e10 0x7f66fc114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 >> 172.21.15.33:0/2661171763 conn(0x7f66fc071380 msgr2=0x7f66fc0f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 shutdown_connections 2024-04-25T02:08:41.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.783+0000 7f67047a7700 1 -- 172.21.15.33:0/2661171763 wait complete. 2024-04-25T02:08:41.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:41 smithi138 ceph-mon[81304]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:41.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:41 smithi138 ceph-mon[81304]: from='client.35612 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:41.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:41 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3143170006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:08:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.887+0000 7f0342360700 1 -- 172.21.15.33:0/1529997326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c0fc460 msgr2=0x7f033c0fc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.887+0000 7f0342360700 1 --2- 172.21.15.33:0/1529997326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c0fc460 0x7f033c0fc8d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f032c004940 tx=0x7f032c009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.887+0000 7f0342360700 1 -- 172.21.15.33:0/1529997326 shutdown_connections 2024-04-25T02:08:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.887+0000 7f0342360700 1 --2- 172.21.15.33:0/1529997326 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c0fc460 0x7f033c0fc8d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.887+0000 7f0342360700 1 --2- 172.21.15.33:0/1529997326 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f033c101d80 0x7f033c102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.887+0000 7f0342360700 1 -- 172.21.15.33:0/1529997326 >> 172.21.15.33:0/1529997326 conn(0x7f033c073350 msgr2=0x7f033c073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.887+0000 7f0342360700 1 -- 172.21.15.33:0/1529997326 shutdown_connections 2024-04-25T02:08:41.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.887+0000 7f0342360700 1 -- 172.21.15.33:0/1529997326 wait complete. 2024-04-25T02:08:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.887+0000 7f0342360700 1 Processor -- start 2024-04-25T02:08:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f0342360700 1 -- start start 2024-04-25T02:08:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f0342360700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f033c0fc460 0x7f033c10eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f0342360700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c101d80 0x7f033c10f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f0342360700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c10f540 con 0x7f033c101d80 2024-04-25T02:08:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f0342360700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c10f680 con 0x7f033c0fc460 2024-04-25T02:08:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f033b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c101d80 0x7f033c10f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:41.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f033b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c101d80 0x7f033c10f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44688/0 (socket says 172.21.15.33:44688) 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f033b7fe700 1 -- 172.21.15.33:0/2495987487 learned_addr learned my addr 172.21.15.33:0/2495987487 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f033bfff700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f033c0fc460 0x7f033c10eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f033b7fe700 1 -- 172.21.15.33:0/2495987487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f033c0fc460 msgr2=0x7f033c10eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f033b7fe700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f033c0fc460 0x7f033c10eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f033b7fe700 1 -- 172.21.15.33:0/2495987487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c005040 con 0x7f033c101d80 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.888+0000 7f033b7fe700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c101d80 0x7f033c10f000 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b8b0 tx=0x7f032c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.889+0000 7f0338ff9700 1 -- 172.21.15.33:0/2495987487 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f032c00e040 con 0x7f033c101d80 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.889+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033c10f900 con 0x7f033c101d80 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.889+0000 7f0338ff9700 1 -- 172.21.15.33:0/2495987487 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f032c004290 con 0x7f033c101d80 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.889+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c117e30 con 0x7f033c101d80 2024-04-25T02:08:41.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.889+0000 7f0338ff9700 1 -- 172.21.15.33:0/2495987487 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f032c01c3f0 con 0x7f033c101d80 2024-04-25T02:08:41.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.890+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c0ff030 con 0x7f033c101d80 2024-04-25T02:08:41.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.890+0000 7f0338ff9700 1 -- 172.21.15.33:0/2495987487 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f032c01c660 con 0x7f033c101d80 2024-04-25T02:08:41.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.891+0000 7f0338ff9700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f032806e140 0x7f03280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:08:41.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.891+0000 7f0338ff9700 1 -- 172.21.15.33:0/2495987487 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f032c08ed70 con 0x7f033c101d80 2024-04-25T02:08:41.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.891+0000 7f033bfff700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f032806e140 0x7f03280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:08:41.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.891+0000 7f033bfff700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f032806e140 0x7f03280705f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0324007fd0 tx=0x7f0324009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:08:41.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:41.895+0000 7f0338ff9700 1 -- 172.21.15.33:0/2495987487 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f032c0581e0 con 0x7f033c101d80 2024-04-25T02:08:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.189+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f033c0fb8f0 con 0x7f033c101d80 2024-04-25T02:08:42.196 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.189+0000 7f0338ff9700 1 -- 172.21.15.33:0/2495987487 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f032c05b8a0 con 0x7f033c101d80 2024-04-25T02:08:42.196 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:08:42.197 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:08:42.198 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:08:42.198 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:08:42.198 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:08:42.198 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:08:42.198 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:08:42.198 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:08:42.198 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:08:42.198 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:08:42.198 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f032806e140 msgr2=0x7f03280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f032806e140 0x7f03280705f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0324007fd0 tx=0x7f0324009040 comp rx=0 tx=0).stop 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c101d80 msgr2=0x7f033c10f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c101d80 0x7f033c10f000 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f032c00b8b0 tx=0x7f032c009e30 comp rx=0 tx=0).stop 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 shutdown_connections 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f032806e140 0x7f03280705f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f033c0fc460 0x7f033c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 --2- 172.21.15.33:0/2495987487 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f033c101d80 0x7f033c10f000 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 >> 172.21.15.33:0/2495987487 conn(0x7f033c073350 msgr2=0x7f033c105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 shutdown_connections 2024-04-25T02:08:42.199 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:08:42.191+0000 7f0342360700 1 -- 172.21.15.33:0/2495987487 wait complete. 2024-04-25T02:08:42.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:42 smithi033 ceph-mon[111306]: from='client.35616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:42.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:42 smithi033 ceph-mon[111306]: pgmap v880: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:42.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:42 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3371334698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:08:42.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:42 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2495987487' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:08:42.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:42 smithi138 ceph-mon[81304]: from='client.35616 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:42.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:42 smithi138 ceph-mon[81304]: pgmap v880: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:42.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:42 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3371334698' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:08:42.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:42 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2495987487' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:08:43.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:43 smithi033 ceph-mon[111306]: from='client.35628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:43.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:43 smithi138 ceph-mon[81304]: from='client.35628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:08:44.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:44 smithi033 ceph-mon[111306]: pgmap v881: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:44.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:44 smithi138 ceph-mon[81304]: pgmap v881: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:45.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:45 smithi033 ceph-mon[111306]: pgmap v882: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:45.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:45 smithi138 ceph-mon[81304]: pgmap v882: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:48.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:48 smithi138 ceph-mon[81304]: pgmap v883: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:48.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:48 smithi033 ceph-mon[111306]: pgmap v883: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:50.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:50 smithi138 ceph-mon[81304]: pgmap v884: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:50.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:50 smithi033 ceph-mon[111306]: pgmap v884: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:52.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:52 smithi138 ceph-mon[81304]: pgmap v885: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:52.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:52 smithi033 ceph-mon[111306]: pgmap v885: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:54.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:54 smithi138 ceph-mon[81304]: pgmap v886: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:54.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:54 smithi033 ceph-mon[111306]: pgmap v886: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:56.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:56 smithi138 ceph-mon[81304]: pgmap v887: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:56.549 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:08:56.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:56 smithi033 ceph-mon[111306]: pgmap v887: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:56.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:08:58.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:08:58 smithi138 ceph-mon[81304]: pgmap v888: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:08:58.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:08:58 smithi033 ceph-mon[111306]: pgmap v888: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:00.548 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:00 smithi138 ceph-mon[81304]: pgmap v889: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:00.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:00 smithi033 ceph-mon[111306]: pgmap v889: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:02.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:02 smithi033 ceph-mon[111306]: pgmap v890: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:02.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:02 smithi138 ceph-mon[81304]: pgmap v890: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:04.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:04 smithi033 ceph-mon[111306]: pgmap v891: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:04.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:04 smithi138 ceph-mon[81304]: pgmap v891: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:06.575 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:06 smithi033 ceph-mon[111306]: pgmap v892: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:06.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:06 smithi138 ceph-mon[81304]: pgmap v892: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:08.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:08 smithi033 ceph-mon[111306]: pgmap v893: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:08.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:08 smithi138 ceph-mon[81304]: pgmap v893: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:10.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:10 smithi033 ceph-mon[111306]: pgmap v894: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:10.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:09:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:10 smithi138 ceph-mon[81304]: pgmap v894: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:09:12.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.312+0000 7fa3cb93a700 1 -- 172.21.15.33:0/1626229493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 msgr2=0x7fa3c4103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.312+0000 7fa3cb93a700 1 --2- 172.21.15.33:0/1626229493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 0x7fa3c4103220 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0004940 tx=0x7fa3c0009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.312+0000 7fa3cb93a700 1 -- 172.21.15.33:0/1626229493 shutdown_connections 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.312+0000 7fa3cb93a700 1 --2- 172.21.15.33:0/1626229493 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3c4065640 0x7fa3c4065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.312+0000 7fa3cb93a700 1 --2- 172.21.15.33:0/1626229493 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 0x7fa3c4103220 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.312+0000 7fa3cb93a700 1 -- 172.21.15.33:0/1626229493 >> 172.21.15.33:0/1626229493 conn(0x7fa3c4071d00 msgr2=0x7fa3c4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.312+0000 7fa3cb93a700 1 -- 172.21.15.33:0/1626229493 shutdown_connections 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.312+0000 7fa3cb93a700 1 -- 172.21.15.33:0/1626229493 wait complete. 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3cb93a700 1 Processor -- start 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3cb93a700 1 -- start start 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3cb93a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3c4065640 0x7fa3c40fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3cb93a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 0x7fa3c40fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3cb93a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40fc880 con 0x7fa3c4102e50 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3cb93a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40fc9c0 con 0x7fa3c4065640 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3c8ed5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 0x7fa3c40fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:12.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3c8ed5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 0x7fa3c40fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48442/0 (socket says 172.21.15.33:48442) 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3c8ed5700 1 -- 172.21.15.33:0/3242957767 learned_addr learned my addr 172.21.15.33:0/3242957767 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.313+0000 7fa3c96d6700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3c4065640 0x7fa3c40fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.314+0000 7fa3c8ed5700 1 -- 172.21.15.33:0/3242957767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3c4065640 msgr2=0x7fa3c40fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.314+0000 7fa3c8ed5700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3c4065640 0x7fa3c40fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.314+0000 7fa3c8ed5700 1 -- 172.21.15.33:0/3242957767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c0005040 con 0x7fa3c4102e50 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.314+0000 7fa3c8ed5700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 0x7fa3c40fc2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400eb40 tx=0x7fa3b400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.314+0000 7fa3b9ffb700 1 -- 172.21.15.33:0/3242957767 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3b4007cc0 con 0x7fa3c4102e50 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.314+0000 7fa3b9ffb700 1 -- 172.21.15.33:0/3242957767 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3b400f040 con 0x7fa3c4102e50 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.314+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c40fcc10 con 0x7fa3c4102e50 2024-04-25T02:09:12.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.314+0000 7fa3b9ffb700 1 -- 172.21.15.33:0/3242957767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3b400ca10 con 0x7fa3c4102e50 2024-04-25T02:09:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.314+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c411a190 con 0x7fa3c4102e50 2024-04-25T02:09:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.315+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c4100100 con 0x7fa3c4102e50 2024-04-25T02:09:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.316+0000 7fa3b9ffb700 1 -- 172.21.15.33:0/3242957767 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa3b400cb70 con 0x7fa3c4102e50 2024-04-25T02:09:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.316+0000 7fa3b9ffb700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3b006e150 0x7fa3b0070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:12.323 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.316+0000 7fa3b9ffb700 1 -- 172.21.15.33:0/3242957767 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa3b408d620 con 0x7fa3c4102e50 2024-04-25T02:09:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.316+0000 7fa3c96d6700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3b006e150 0x7fa3b0070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:12.324 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.317+0000 7fa3c96d6700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3b006e150 0x7fa3b0070600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0007f50 tx=0x7fa3c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:12.328 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.322+0000 7fa3b9ffb700 1 -- 172.21.15.33:0/3242957767 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa3b4056ad0 con 0x7fa3c4102e50 2024-04-25T02:09:12.557 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.550+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3c40fd510 con 0x7fa3b006e150 2024-04-25T02:09:12.558 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.551+0000 7fa3b9ffb700 1 -- 172.21.15.33:0/3242957767 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa3c40fd510 con 0x7fa3b006e150 2024-04-25T02:09:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3b006e150 msgr2=0x7fa3b0070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3b006e150 0x7fa3b0070600 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0007f50 tx=0x7fa3c0009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 msgr2=0x7fa3c40fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 0x7fa3c40fc2f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400eb40 tx=0x7fa3b400b040 comp rx=0 tx=0).stop 2024-04-25T02:09:12.560 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 shutdown_connections 2024-04-25T02:09:12.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa3b006e150 0x7fa3b0070600 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa3c4065640 0x7fa3c40fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 --2- 172.21.15.33:0/3242957767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa3c4102e50 0x7fa3c40fc2f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 >> 172.21.15.33:0/3242957767 conn(0x7fa3c4071d00 msgr2=0x7fa3c40faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:12.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 shutdown_connections 2024-04-25T02:09:12.561 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.554+0000 7fa3cb93a700 1 -- 172.21.15.33:0/3242957767 wait complete. 2024-04-25T02:09:12.571 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:09:12.571 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:12 smithi033 ceph-mon[111306]: pgmap v895: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.659+0000 7fce10fa0700 1 -- 172.21.15.33:0/1600915955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 msgr2=0x7fce0c0fcc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.659+0000 7fce10fa0700 1 --2- 172.21.15.33:0/1600915955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 0x7fce0c0fcc50 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4004940 tx=0x7fcdf4009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.659+0000 7fce10fa0700 1 -- 172.21.15.33:0/1600915955 shutdown_connections 2024-04-25T02:09:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.659+0000 7fce10fa0700 1 --2- 172.21.15.33:0/1600915955 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 0x7fce0c0fcc50 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.659+0000 7fce10fa0700 1 --2- 172.21.15.33:0/1600915955 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce0c102c40 0x7fce0c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.659+0000 7fce10fa0700 1 -- 172.21.15.33:0/1600915955 >> 172.21.15.33:0/1600915955 conn(0x7fce0c0f8810 msgr2=0x7fce0c0fac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.659+0000 7fce10fa0700 1 -- 172.21.15.33:0/1600915955 shutdown_connections 2024-04-25T02:09:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.659+0000 7fce10fa0700 1 -- 172.21.15.33:0/1600915955 wait complete. 2024-04-25T02:09:12.666 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.659+0000 7fce10fa0700 1 Processor -- start 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce10fa0700 1 -- start start 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce10fa0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce0c102c40 0x7fce0c1139d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce10fa0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 0x7fce0c113f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce10fa0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c114560 con 0x7fce0c1035e0 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce10fa0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c10d950 con 0x7fce0c102c40 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce0affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 0x7fce0c113f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce0affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 0x7fce0c113f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48472/0 (socket says 172.21.15.33:48472) 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce0affd700 1 -- 172.21.15.33:0/2628898769 learned_addr learned my addr 172.21.15.33:0/2628898769 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce0b7fe700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce0c102c40 0x7fce0c1139d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce0affd700 1 -- 172.21.15.33:0/2628898769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce0c102c40 msgr2=0x7fce0c1139d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce0affd700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce0c102c40 0x7fce0c1139d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.660+0000 7fce0affd700 1 -- 172.21.15.33:0/2628898769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4005040 con 0x7fce0c1035e0 2024-04-25T02:09:12.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.661+0000 7fce0affd700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 0x7fce0c113f10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009100 tx=0x7fcdf4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.661+0000 7fce03fff700 1 -- 172.21.15.33:0/2628898769 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf400e040 con 0x7fce0c1035e0 2024-04-25T02:09:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.661+0000 7fce03fff700 1 -- 172.21.15.33:0/2628898769 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdf4009af0 con 0x7fce0c1035e0 2024-04-25T02:09:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.661+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c10dbd0 con 0x7fce0c1035e0 2024-04-25T02:09:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.661+0000 7fce03fff700 1 -- 172.21.15.33:0/2628898769 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdf401c3f0 con 0x7fce0c1035e0 2024-04-25T02:09:12.669 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.661+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c10e190 con 0x7fce0c1035e0 2024-04-25T02:09:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.662+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c0fe390 con 0x7fce0c1035e0 2024-04-25T02:09:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.662+0000 7fce03fff700 1 -- 172.21.15.33:0/2628898769 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcdf400baf0 con 0x7fce0c1035e0 2024-04-25T02:09:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.663+0000 7fce03fff700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcdf806e140 0x7fcdf80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.663+0000 7fce03fff700 1 -- 172.21.15.33:0/2628898769 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcdf408ed60 con 0x7fce0c1035e0 2024-04-25T02:09:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.663+0000 7fce0b7fe700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcdf806e140 0x7fcdf80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:12.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.663+0000 7fce0b7fe700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcdf806e140 0x7fcdf80705f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce04001370 tx=0x7fce0400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:12.674 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.667+0000 7fce03fff700 1 -- 172.21.15.33:0/2628898769 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcdf405bc90 con 0x7fce0c1035e0 2024-04-25T02:09:12.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:12 smithi138 ceph-mon[81304]: pgmap v895: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.895+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce0c109d30 con 0x7fcdf806e140 2024-04-25T02:09:12.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.896+0000 7fce03fff700 1 -- 172.21.15.33:0/2628898769 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fce0c109d30 con 0x7fcdf806e140 2024-04-25T02:09:12.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcdf806e140 msgr2=0x7fcdf80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:12.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcdf806e140 0x7fcdf80705f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fce04001370 tx=0x7fce0400b040 comp rx=0 tx=0).stop 2024-04-25T02:09:12.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 msgr2=0x7fce0c113f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:12.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 0x7fce0c113f10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4009100 tx=0x7fcdf4009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:12.906 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 shutdown_connections 2024-04-25T02:09:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcdf806e140 0x7fcdf80705f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fce0c102c40 0x7fce0c1139d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 --2- 172.21.15.33:0/2628898769 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fce0c1035e0 0x7fce0c113f10 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 >> 172.21.15.33:0/2628898769 conn(0x7fce0c0f8810 msgr2=0x7fce0c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.899+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 shutdown_connections 2024-04-25T02:09:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:12.900+0000 7fce10fa0700 1 -- 172.21.15.33:0/2628898769 wait complete. 2024-04-25T02:09:13.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 -- 172.21.15.33:0/1746730778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 msgr2=0x7ff4000fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 --2- 172.21.15.33:0/1746730778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 0x7ff4000fbfb0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4004940 tx=0x7ff3f4009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:13.010 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 -- 172.21.15.33:0/1746730778 shutdown_connections 2024-04-25T02:09:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 --2- 172.21.15.33:0/1746730778 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4000fc4f0 0x7ff400109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 --2- 172.21.15.33:0/1746730778 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 0x7ff4000fbfb0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 -- 172.21.15.33:0/1746730778 >> 172.21.15.33:0/1746730778 conn(0x7ff4000f7850 msgr2=0x7ff4000f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 -- 172.21.15.33:0/1746730778 shutdown_connections 2024-04-25T02:09:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 -- 172.21.15.33:0/1746730778 wait complete. 2024-04-25T02:09:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 Processor -- start 2024-04-25T02:09:13.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.004+0000 7ff406fe4700 1 -- start start 2024-04-25T02:09:13.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff406fe4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 0x7ff400115ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:13.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff406fe4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4000fc4f0 0x7ff4001160e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:13.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff406fe4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff400116730 con 0x7ff4000fbbe0 2024-04-25T02:09:13.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff406fe4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40010fd60 con 0x7ff4000fc4f0 2024-04-25T02:09:13.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff404d80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 0x7ff400115ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:13.012 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff404d80700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 0x7ff400115ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48492/0 (socket says 172.21.15.33:48492) 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff404d80700 1 -- 172.21.15.33:0/1594759659 learned_addr learned my addr 172.21.15.33:0/1594759659 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff3fffff700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4000fc4f0 0x7ff4001160e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff404d80700 1 -- 172.21.15.33:0/1594759659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4000fc4f0 msgr2=0x7ff4001160e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff404d80700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4000fc4f0 0x7ff4001160e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff404d80700 1 -- 172.21.15.33:0/1594759659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f4005040 con 0x7ff4000fbbe0 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff404d80700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 0x7ff400115ba0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009360 tx=0x7ff3f4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff3fd7fa700 1 -- 172.21.15.33:0/1594759659 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f400e040 con 0x7ff4000fbbe0 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40010ffe0 con 0x7ff4000fbbe0 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff3fd7fa700 1 -- 172.21.15.33:0/1594759659 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff3f4013420 con 0x7ff4000fbbe0 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff3fd7fa700 1 -- 172.21.15.33:0/1594759659 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f401c630 con 0x7ff4000fbbe0 2024-04-25T02:09:13.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.005+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4001105a0 con 0x7ff4000fbbe0 2024-04-25T02:09:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.006+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff400066a90 con 0x7ff4000fbbe0 2024-04-25T02:09:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.007+0000 7ff3fd7fa700 1 -- 172.21.15.33:0/1594759659 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff3f4013a90 con 0x7ff4000fbbe0 2024-04-25T02:09:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.007+0000 7ff3fd7fa700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff3f006e140 0x7ff3f00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.007+0000 7ff3fd7fa700 1 -- 172.21.15.33:0/1594759659 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff3f408ed20 con 0x7ff4000fbbe0 2024-04-25T02:09:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.008+0000 7ff3fffff700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff3f006e140 0x7ff3f00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:13.014 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.008+0000 7ff3fffff700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff3f006e140 0x7ff3f00705f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff400111440 tx=0x7ff3ec00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:13.018 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.012+0000 7ff3fd7fa700 1 -- 172.21.15.33:0/1594759659 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff3f405bc50 con 0x7ff4000fbbe0 2024-04-25T02:09:13.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.231+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4001111f0 con 0x7ff3f006e140 2024-04-25T02:09:13.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.241+0000 7ff3fd7fa700 1 -- 172.21.15.33:0/1594759659 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff4001111f0 con 0x7ff3f006e140 2024-04-25T02:09:13.247 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:09:13.247 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (30m) 8m ago 41m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:09:13.247 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (27m) 8m ago 41m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:09:13.247 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (27m) 8m ago 40m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:09:13.247 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (30m) 8m ago 41m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (23m) 8m ago 36m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (23m) 8m ago 36m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (23m) 8m ago 36m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (8m) 8m ago 36m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (31m) 8m ago 42m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (31m) 8m ago 40m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (28m) 8m ago 43m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (28m) 8m ago 40m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (31m) 8m ago 41m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (30m) 8m ago 40m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (27m) 8m ago 39m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (26m) 8m ago 39m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (25m) 8m ago 38m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (25m) 8m ago 38m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (24m) 8m ago 38m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:09:13.248 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (24m) 8m ago 37m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:09:13.249 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (30m) 8m ago 41m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:09:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff3f006e140 msgr2=0x7ff3f00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff3f006e140 0x7ff3f00705f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff400111440 tx=0x7ff3ec00b040 comp rx=0 tx=0).stop 2024-04-25T02:09:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 msgr2=0x7ff400115ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 0x7ff400115ba0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009360 tx=0x7ff3f4009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:13.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 shutdown_connections 2024-04-25T02:09:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff3f006e140 0x7ff3f00705f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff4000fbbe0 0x7ff400115ba0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 --2- 172.21.15.33:0/1594759659 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff4000fc4f0 0x7ff4001160e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 >> 172.21.15.33:0/1594759659 conn(0x7ff4000f7850 msgr2=0x7ff4000f7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 shutdown_connections 2024-04-25T02:09:13.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.243+0000 7ff406fe4700 1 -- 172.21.15.33:0/1594759659 wait complete. 2024-04-25T02:09:13.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.346+0000 7f553a0e9700 1 -- 172.21.15.33:0/2311993498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534102df0 msgr2=0x7f55341031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.346+0000 7f553a0e9700 1 --2- 172.21.15.33:0/2311993498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534102df0 0x7f55341031c0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f5524009ce0 tx=0x7f5524004d10 comp rx=0 tx=0).stop 2024-04-25T02:09:13.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.346+0000 7f553a0e9700 1 -- 172.21.15.33:0/2311993498 shutdown_connections 2024-04-25T02:09:13.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.346+0000 7f553a0e9700 1 --2- 172.21.15.33:0/2311993498 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5534103790 0x7f55340fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.346+0000 7f553a0e9700 1 --2- 172.21.15.33:0/2311993498 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534102df0 0x7f55341031c0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.346+0000 7f553a0e9700 1 -- 172.21.15.33:0/2311993498 >> 172.21.15.33:0/2311993498 conn(0x7f55340f8980 msgr2=0x7f55340fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:13.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.346+0000 7f553a0e9700 1 -- 172.21.15.33:0/2311993498 shutdown_connections 2024-04-25T02:09:13.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.346+0000 7f553a0e9700 1 -- 172.21.15.33:0/2311993498 wait complete. 2024-04-25T02:09:13.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.347+0000 7f553a0e9700 1 Processor -- start 2024-04-25T02:09:13.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.347+0000 7f553a0e9700 1 -- start start 2024-04-25T02:09:13.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.347+0000 7f553a0e9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5534102df0 0x7f5534113d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:13.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.347+0000 7f553a0e9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534103790 0x7f5534114260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.347+0000 7f553a0e9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55341148b0 con 0x7f5534103790 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.347+0000 7f553a0e9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553410dc80 con 0x7f5534102df0 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.347+0000 7f5532ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534103790 0x7f5534114260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.347+0000 7f5532ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534103790 0x7f5534114260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48510/0 (socket says 172.21.15.33:48510) 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.347+0000 7f5532ffd700 1 -- 172.21.15.33:0/3559354008 learned_addr learned my addr 172.21.15.33:0/3559354008 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f55337fe700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5534102df0 0x7f5534113d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f5532ffd700 1 -- 172.21.15.33:0/3559354008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5534102df0 msgr2=0x7f5534113d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f5532ffd700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5534102df0 0x7f5534113d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f5532ffd700 1 -- 172.21.15.33:0/3559354008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524009940 con 0x7f5534103790 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f5532ffd700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534103790 0x7f5534114260 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f551c002a50 tx=0x7f551c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f552bfff700 1 -- 172.21.15.33:0/3559354008 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f551c007cc0 con 0x7f5534103790 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f552bfff700 1 -- 172.21.15.33:0/3559354008 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f551c00f040 con 0x7f5534103790 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f553410df60 con 0x7f5534103790 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f552bfff700 1 -- 172.21.15.33:0/3559354008 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f551c00d9c0 con 0x7f5534103790 2024-04-25T02:09:13.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.348+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f553410e520 con 0x7f5534103790 2024-04-25T02:09:13.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.349+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5534100100 con 0x7f5534103790 2024-04-25T02:09:13.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.350+0000 7f552bfff700 1 -- 172.21.15.33:0/3559354008 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f551c009730 con 0x7f5534103790 2024-04-25T02:09:13.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.350+0000 7f552bfff700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f552006e140 0x7f55200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.350+0000 7f552bfff700 1 -- 172.21.15.33:0/3559354008 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f551c08d8a0 con 0x7f5534103790 2024-04-25T02:09:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.350+0000 7f55337fe700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f552006e140 0x7f55200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:13.358 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.351+0000 7f55337fe700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f552006e140 0x7f55200705f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f552400b740 tx=0x7f5524004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:13.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.355+0000 7f552bfff700 1 -- 172.21.15.33:0/3559354008 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f551c05a750 con 0x7f5534103790 2024-04-25T02:09:13.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.652+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f553410f370 con 0x7f5534103790 2024-04-25T02:09:13.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.653+0000 7f552bfff700 1 -- 172.21.15.33:0/3559354008 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f551c05a2e0 con 0x7f5534103790 2024-04-25T02:09:13.660 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:09:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:09:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:09:13.660 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:09:13.661 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f552006e140 msgr2=0x7f55200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f552006e140 0x7f55200705f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f552400b740 tx=0x7f5524004d10 comp rx=0 tx=0).stop 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534103790 msgr2=0x7f5534114260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534103790 0x7f5534114260 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f551c002a50 tx=0x7f551c004600 comp rx=0 tx=0).stop 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 shutdown_connections 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f552006e140 0x7f55200705f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5534102df0 0x7f5534113d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 --2- 172.21.15.33:0/3559354008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5534103790 0x7f5534114260 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 >> 172.21.15.33:0/3559354008 conn(0x7f55340f8980 msgr2=0x7f55341073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 shutdown_connections 2024-04-25T02:09:13.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.655+0000 7f553a0e9700 1 -- 172.21.15.33:0/3559354008 wait complete. 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.760+0000 7f453acd6700 1 -- 172.21.15.33:0/3426340417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4534101d90 msgr2=0x7f4534102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.760+0000 7f453acd6700 1 --2- 172.21.15.33:0/3426340417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4534101d90 0x7f4534102160 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f4528004940 tx=0x7f4528009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.760+0000 7f453acd6700 1 -- 172.21.15.33:0/3426340417 shutdown_connections 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.760+0000 7f453acd6700 1 --2- 172.21.15.33:0/3426340417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f45340fbd40 0x7f45340fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.760+0000 7f453acd6700 1 --2- 172.21.15.33:0/3426340417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4534101d90 0x7f4534102160 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.760+0000 7f453acd6700 1 -- 172.21.15.33:0/3426340417 >> 172.21.15.33:0/3426340417 conn(0x7f4534071d00 msgr2=0x7f4534072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.760+0000 7f453acd6700 1 -- 172.21.15.33:0/3426340417 shutdown_connections 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.760+0000 7f453acd6700 1 -- 172.21.15.33:0/3426340417 wait complete. 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f453acd6700 1 Processor -- start 2024-04-25T02:09:13.768 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f453acd6700 1 -- start start 2024-04-25T02:09:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f453acd6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45340fbd40 0x7f4534102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f453acd6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4534101d90 0x7f4534103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f453acd6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45341035f0 con 0x7f45340fbd40 2024-04-25T02:09:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f453acd6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4534103730 con 0x7f4534101d90 2024-04-25T02:09:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f4538a72700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45340fbd40 0x7f4534102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f4538a72700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45340fbd40 0x7f4534102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48532/0 (socket says 172.21.15.33:48532) 2024-04-25T02:09:13.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f4538a72700 1 -- 172.21.15.33:0/961884241 learned_addr learned my addr 172.21.15.33:0/961884241 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f4533fff700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4534101d90 0x7f4534103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f4538a72700 1 -- 172.21.15.33:0/961884241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4534101d90 msgr2=0x7f4534103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f4538a72700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4534101d90 0x7f4534103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f4538a72700 1 -- 172.21.15.33:0/961884241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528005040 con 0x7f45340fbd40 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.761+0000 7f4533fff700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4534101d90 0x7f4534103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.762+0000 7f4538a72700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45340fbd40 0x7f4534102b20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f452800b3b0 tx=0x7f4528009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.762+0000 7f45317fa700 1 -- 172.21.15.33:0/961884241 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f452800e040 con 0x7f45340fbd40 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.762+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4534103980 con 0x7f45340fbd40 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.762+0000 7f45317fa700 1 -- 172.21.15.33:0/961884241 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f452800bbc0 con 0x7f45340fbd40 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.762+0000 7f45317fa700 1 -- 172.21.15.33:0/961884241 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f452801c3f0 con 0x7f45340fbd40 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.762+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453411a2d0 con 0x7f45340fbd40 2024-04-25T02:09:13.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.763+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45340fd430 con 0x7f45340fbd40 2024-04-25T02:09:13.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.763+0000 7f45317fa700 1 -- 172.21.15.33:0/961884241 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f452800bd30 con 0x7f45340fbd40 2024-04-25T02:09:13.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.764+0000 7f45317fa700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f452406e140 0x7f45240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:13.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.764+0000 7f45317fa700 1 -- 172.21.15.33:0/961884241 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f452808ee10 con 0x7f45340fbd40 2024-04-25T02:09:13.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.764+0000 7f4533fff700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f452406e140 0x7f45240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:13.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.764+0000 7f4533fff700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f452406e140 0x7f45240705f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4534104460 tx=0x7f4520009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:13.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:13.769+0000 7f45317fa700 1 -- 172.21.15.33:0/961884241 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4528058190 con 0x7f45340fbd40 2024-04-25T02:09:14.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.022+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4534104210 con 0x7f45340fbd40 2024-04-25T02:09:14.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.022+0000 7f45317fa700 1 -- 172.21.15.33:0/961884241 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f452805b850 con 0x7f45340fbd40 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:09:14.030 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:09:14.031 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:09:14.032 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:09:14.033 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:09:14.033 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:09:14.033 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:09:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f452406e140 msgr2=0x7f45240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f452406e140 0x7f45240705f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4534104460 tx=0x7f4520009040 comp rx=0 tx=0).stop 2024-04-25T02:09:14.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45340fbd40 msgr2=0x7f4534102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45340fbd40 0x7f4534102b20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f452800b3b0 tx=0x7f4528009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:14.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 shutdown_connections 2024-04-25T02:09:14.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f452406e140 0x7f45240705f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f45340fbd40 0x7f4534102b20 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 --2- 172.21.15.33:0/961884241 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4534101d90 0x7f4534103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 >> 172.21.15.33:0/961884241 conn(0x7f4534071d00 msgr2=0x7f45340fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:14.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 shutdown_connections 2024-04-25T02:09:14.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.024+0000 7f453acd6700 1 -- 172.21.15.33:0/961884241 wait complete. 2024-04-25T02:09:14.034 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:09:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.129+0000 7f5fef118700 1 -- 172.21.15.33:0/946118045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 msgr2=0x7f5fe80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.138 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.129+0000 7f5fef118700 1 --2- 172.21.15.33:0/946118045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 0x7f5fe80fc1b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4004940 tx=0x7f5fd4009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.129+0000 7f5fef118700 1 -- 172.21.15.33:0/946118045 shutdown_connections 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.129+0000 7f5fef118700 1 --2- 172.21.15.33:0/946118045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 0x7f5fe80fc1b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.129+0000 7f5fef118700 1 --2- 172.21.15.33:0/946118045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe8101d90 0x7f5fe8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.129+0000 7f5fef118700 1 -- 172.21.15.33:0/946118045 >> 172.21.15.33:0/946118045 conn(0x7f5fe8071d00 msgr2=0x7f5fe8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.129+0000 7f5fef118700 1 -- 172.21.15.33:0/946118045 shutdown_connections 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.129+0000 7f5fef118700 1 -- 172.21.15.33:0/946118045 wait complete. 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5fef118700 1 Processor -- start 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5fef118700 1 -- start start 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5fef118700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 0x7f5fe8112ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5fef118700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe8101d90 0x7f5fe8112ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5fef118700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe8113640 con 0x7f5fe80fbd40 2024-04-25T02:09:14.139 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5fef118700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe810ca30 con 0x7f5fe8101d90 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5feceb4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 0x7f5fe8112ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5feceb4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 0x7f5fe8112ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48554/0 (socket says 172.21.15.33:48554) 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5feceb4700 1 -- 172.21.15.33:0/1488481082 learned_addr learned my addr 172.21.15.33:0/1488481082 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5fe7fff700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe8101d90 0x7f5fe8112ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5feceb4700 1 -- 172.21.15.33:0/1488481082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe8101d90 msgr2=0x7f5fe8112ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.130+0000 7f5feceb4700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe8101d90 0x7f5fe8112ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.131+0000 7f5feceb4700 1 -- 172.21.15.33:0/1488481082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd4005040 con 0x7f5fe80fbd40 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.131+0000 7f5feceb4700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 0x7f5fe8112ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c830 tx=0x7f5fdc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.131+0000 7f5fe57fa700 1 -- 172.21.15.33:0/1488481082 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fdc00a760 con 0x7f5fe80fbd40 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.131+0000 7f5fe57fa700 1 -- 172.21.15.33:0/1488481082 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5fdc00ad90 con 0x7f5fe80fbd40 2024-04-25T02:09:14.140 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.131+0000 7f5fe57fa700 1 -- 172.21.15.33:0/1488481082 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fdc0113a0 con 0x7f5fe80fbd40 2024-04-25T02:09:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.131+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe810cd10 con 0x7f5fe80fbd40 2024-04-25T02:09:14.141 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.131+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe810d2d0 con 0x7f5fe80fbd40 2024-04-25T02:09:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.132+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe80fd430 con 0x7f5fe80fbd40 2024-04-25T02:09:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.133+0000 7f5fe57fa700 1 -- 172.21.15.33:0/1488481082 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5fdc012400 con 0x7f5fe80fbd40 2024-04-25T02:09:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.133+0000 7f5fe57fa700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5fd806e150 0x7f5fd8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.133+0000 7f5fe57fa700 1 -- 172.21.15.33:0/1488481082 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5fdc08daa0 con 0x7f5fe80fbd40 2024-04-25T02:09:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.133+0000 7f5fe7fff700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5fd806e150 0x7f5fd8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:14.142 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.134+0000 7f5fe7fff700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5fd806e150 0x7f5fd8070600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5fe810dbf0 tx=0x7f5fd4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:14.145 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.138+0000 7f5fe57fa700 1 -- 172.21.15.33:0/1488481082 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5fdc05aa70 con 0x7f5fe80fbd40 2024-04-25T02:09:14.377 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.369+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fe8108df0 con 0x7f5fd806e150 2024-04-25T02:09:14.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.371+0000 7f5fe57fa700 1 -- 172.21.15.33:0/1488481082 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5fe8108df0 con 0x7f5fd806e150 2024-04-25T02:09:14.378 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:09:14.378 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:09:14.378 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:09:14.378 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:09:14.378 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:09:14.378 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:09:14.379 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:09:14.379 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:09:14.379 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:09:14.379 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:09:14.379 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:09:14.379 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:09:14.379 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:09:14.379 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:09:14.380 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:09:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5fd806e150 msgr2=0x7f5fd8070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.380 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5fd806e150 0x7f5fd8070600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f5fe810dbf0 tx=0x7f5fd4009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 msgr2=0x7f5fe8112ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 0x7f5fe8112ab0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c830 tx=0x7f5fdc004470 comp rx=0 tx=0).stop 2024-04-25T02:09:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 shutdown_connections 2024-04-25T02:09:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5fd806e150 0x7f5fd8070600 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5fe80fbd40 0x7f5fe8112ab0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 --2- 172.21.15.33:0/1488481082 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5fe8101d90 0x7f5fe8112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 >> 172.21.15.33:0/1488481082 conn(0x7f5fe8071d00 msgr2=0x7f5fe80fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:14.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 shutdown_connections 2024-04-25T02:09:14.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.373+0000 7f5fef118700 1 -- 172.21.15.33:0/1488481082 wait complete. 2024-04-25T02:09:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.477+0000 7f4fe95d9700 1 -- 172.21.15.33:0/1730708110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fe7b0 msgr2=0x7f4fe4106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.477+0000 7f4fe95d9700 1 --2- 172.21.15.33:0/1730708110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fe7b0 0x7f4fe4106df0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4fd8004940 tx=0x7f4fd8009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.477+0000 7f4fe95d9700 1 -- 172.21.15.33:0/1730708110 shutdown_connections 2024-04-25T02:09:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.477+0000 7f4fe95d9700 1 --2- 172.21.15.33:0/1730708110 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fe7b0 0x7f4fe4106df0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.477+0000 7f4fe95d9700 1 --2- 172.21.15.33:0/1730708110 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe40fde10 0x7f4fe40fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.477+0000 7f4fe95d9700 1 -- 172.21.15.33:0/1730708110 >> 172.21.15.33:0/1730708110 conn(0x7f4fe40f7860 msgr2=0x7f4fe40f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.477+0000 7f4fe95d9700 1 -- 172.21.15.33:0/1730708110 shutdown_connections 2024-04-25T02:09:14.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.477+0000 7f4fe95d9700 1 -- 172.21.15.33:0/1730708110 wait complete. 2024-04-25T02:09:14.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.478+0000 7f4fe95d9700 1 Processor -- start 2024-04-25T02:09:14.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.478+0000 7f4fe95d9700 1 -- start start 2024-04-25T02:09:14.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.478+0000 7f4fe95d9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fde10 0x7f4fe4113a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:14.487 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.478+0000 7f4fe95d9700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe40fe7b0 0x7f4fe4113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.478+0000 7f4fe95d9700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe41145b0 con 0x7f4fe40fde10 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.478+0000 7f4fe95d9700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe410db90 con 0x7f4fe40fe7b0 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.478+0000 7f4fe2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fde10 0x7f4fe4113a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.478+0000 7f4fe2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fde10 0x7f4fe4113a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48586/0 (socket says 172.21.15.33:48586) 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.478+0000 7f4fe2ffd700 1 -- 172.21.15.33:0/3753859788 learned_addr learned my addr 172.21.15.33:0/3753859788 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fe27fc700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe40fe7b0 0x7f4fe4113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fe2ffd700 1 -- 172.21.15.33:0/3753859788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe40fe7b0 msgr2=0x7f4fe4113f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fe2ffd700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe40fe7b0 0x7f4fe4113f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fe2ffd700 1 -- 172.21.15.33:0/3753859788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd8005040 con 0x7f4fe40fde10 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fe27fc700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe40fe7b0 0x7f4fe4113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fe2ffd700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fde10 0x7f4fe4113a20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4002a50 tx=0x7f4fd40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fcb7fe700 1 -- 172.21.15.33:0/3753859788 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fd4007cc0 con 0x7f4fe40fde10 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fcb7fe700 1 -- 172.21.15.33:0/3753859788 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fd400f040 con 0x7f4fe40fde10 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fe410de70 con 0x7f4fe40fde10 2024-04-25T02:09:14.488 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fcb7fe700 1 -- 172.21.15.33:0/3753859788 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fd4009640 con 0x7f4fe40fde10 2024-04-25T02:09:14.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.479+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fe410e3b0 con 0x7f4fe40fde10 2024-04-25T02:09:14.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.480+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fe4066b50 con 0x7f4fe40fde10 2024-04-25T02:09:14.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.481+0000 7f4fcb7fe700 1 -- 172.21.15.33:0/3753859788 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4fd40097a0 con 0x7f4fe40fde10 2024-04-25T02:09:14.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.481+0000 7f4fcb7fe700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4fcc06e140 0x7f4fcc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:14.489 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.481+0000 7f4fcb7fe700 1 -- 172.21.15.33:0/3753859788 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4fd408d850 con 0x7f4fe40fde10 2024-04-25T02:09:14.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.481+0000 7f4fe27fc700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4fcc06e140 0x7f4fcc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:14.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.482+0000 7f4fe27fc700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4fcc06e140 0x7f4fcc0705f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4fe410f2c0 tx=0x7f4fd8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:14.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.486+0000 7f4fcb7fe700 1 -- 172.21.15.33:0/3753859788 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4fd405a780 con 0x7f4fe40fde10 2024-04-25T02:09:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:14 smithi033 ceph-mon[111306]: from='client.35636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:14 smithi033 ceph-mon[111306]: from='client.35640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:14 smithi033 ceph-mon[111306]: pgmap v896: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:14 smithi033 ceph-mon[111306]: from='client.35644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:14 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3559354008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:09:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:14 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/961884241' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:09:14.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.779+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4fe410f070 con 0x7f4fe40fde10 2024-04-25T02:09:14.787 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.780+0000 7f4fcb7fe700 1 -- 172.21.15.33:0/3753859788 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f4fd405a310 con 0x7f4fe40fde10 2024-04-25T02:09:14.787 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:09:14.788 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:09:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:09:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:09:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:09:14.789 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:09:14.789 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:09:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:09:14.789 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:09:14.789 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4fcc06e140 msgr2=0x7f4fcc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4fcc06e140 0x7f4fcc0705f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4fe410f2c0 tx=0x7f4fd8009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fde10 msgr2=0x7f4fe4113a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fde10 0x7f4fe4113a20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4fd4002a50 tx=0x7f4fd40046b0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 shutdown_connections 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4fcc06e140 0x7f4fcc0705f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4fe40fde10 0x7f4fe4113a20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 --2- 172.21.15.33:0/3753859788 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4fe40fe7b0 0x7f4fe4113f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 >> 172.21.15.33:0/3753859788 conn(0x7f4fe40f7860 msgr2=0x7f4fe40f8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:14.790 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 shutdown_connections 2024-04-25T02:09:14.791 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:14.782+0000 7f4fe95d9700 1 -- 172.21.15.33:0/3753859788 wait complete. 2024-04-25T02:09:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:14 smithi138 ceph-mon[81304]: from='client.35636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:14 smithi138 ceph-mon[81304]: from='client.35640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:14 smithi138 ceph-mon[81304]: pgmap v896: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:14 smithi138 ceph-mon[81304]: from='client.35644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:14 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3559354008' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:09:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:14 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/961884241' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:09:15.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:15 smithi033 ceph-mon[111306]: from='client.35656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:15.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:15 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3753859788' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:09:15.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:15 smithi138 ceph-mon[81304]: from='client.35656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:15.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:15 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3753859788' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:09:16.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:16 smithi033 ceph-mon[111306]: pgmap v897: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:16.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:16 smithi138 ceph-mon[81304]: pgmap v897: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:18.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:18 smithi033 ceph-mon[111306]: pgmap v898: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:18.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:18 smithi138 ceph-mon[81304]: pgmap v898: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:20.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:20 smithi033 ceph-mon[111306]: pgmap v899: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:20.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:20 smithi138 ceph-mon[81304]: pgmap v899: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:22.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:22 smithi033 ceph-mon[111306]: pgmap v900: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:22.835 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:22 smithi138 ceph-mon[81304]: pgmap v900: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:24.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:24 smithi033 ceph-mon[111306]: pgmap v901: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:24.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:24 smithi138 ceph-mon[81304]: pgmap v901: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:25.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:09:25.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:09:26.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:26 smithi033 ceph-mon[111306]: pgmap v902: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:26.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:26 smithi138 ceph-mon[81304]: pgmap v902: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:28.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:28 smithi033 ceph-mon[111306]: pgmap v903: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:28.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:28 smithi138 ceph-mon[81304]: pgmap v903: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:30.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:30 smithi033 ceph-mon[111306]: pgmap v904: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:30.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:30 smithi138 ceph-mon[81304]: pgmap v904: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:32.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:32 smithi033 ceph-mon[111306]: pgmap v905: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:32.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:32 smithi138 ceph-mon[81304]: pgmap v905: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:34.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:34 smithi033 ceph-mon[111306]: pgmap v906: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:34.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:34 smithi138 ceph-mon[81304]: pgmap v906: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:36.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:36 smithi033 ceph-mon[111306]: pgmap v907: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:36.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:36 smithi138 ceph-mon[81304]: pgmap v907: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:38.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:38 smithi033 ceph-mon[111306]: pgmap v908: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:38.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:38 smithi138 ceph-mon[81304]: pgmap v908: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:40.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:40 smithi033 ceph-mon[111306]: pgmap v909: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:40.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:09:40.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:40 smithi138 ceph-mon[81304]: pgmap v909: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:40.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:09:42.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:42 smithi033 ceph-mon[111306]: pgmap v910: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:42.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:42 smithi138 ceph-mon[81304]: pgmap v910: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:44.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:44 smithi033 ceph-mon[111306]: pgmap v911: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:44.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:44 smithi138 ceph-mon[81304]: pgmap v911: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.901+0000 7f2312e2b700 1 -- 172.21.15.33:0/1505910825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c103780 msgr2=0x7f230c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.901+0000 7f2312e2b700 1 --2- 172.21.15.33:0/1505910825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c103780 0x7f230c0fcdf0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f22f8004940 tx=0x7f22f8009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.901+0000 7f2312e2b700 1 -- 172.21.15.33:0/1505910825 shutdown_connections 2024-04-25T02:09:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.901+0000 7f2312e2b700 1 --2- 172.21.15.33:0/1505910825 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c103780 0x7f230c0fcdf0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:44.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.901+0000 7f2312e2b700 1 --2- 172.21.15.33:0/1505910825 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f230c102de0 0x7f230c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:44.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.901+0000 7f2312e2b700 1 -- 172.21.15.33:0/1505910825 >> 172.21.15.33:0/1505910825 conn(0x7f230c0f8990 msgr2=0x7f230c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:44.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.901+0000 7f2312e2b700 1 -- 172.21.15.33:0/1505910825 shutdown_connections 2024-04-25T02:09:44.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.901+0000 7f2312e2b700 1 -- 172.21.15.33:0/1505910825 wait complete. 2024-04-25T02:09:44.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.901+0000 7f2312e2b700 1 Processor -- start 2024-04-25T02:09:44.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2312e2b700 1 -- start start 2024-04-25T02:09:44.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2312e2b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c102de0 0x7f230c113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:44.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2312e2b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f230c103780 0x7f230c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:44.910 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2312e2b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c1144e0 con 0x7f230c102de0 2024-04-25T02:09:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2312e2b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c10dae0 con 0x7f230c103780 2024-04-25T02:09:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2310bc7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c102de0 0x7f230c113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2310bc7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c102de0 0x7f230c113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44320/0 (socket says 172.21.15.33:44320) 2024-04-25T02:09:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2310bc7700 1 -- 172.21.15.33:0/1213457287 learned_addr learned my addr 172.21.15.33:0/1213457287 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f230bfff700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f230c103780 0x7f230c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:44.911 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2310bc7700 1 -- 172.21.15.33:0/1213457287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f230c103780 msgr2=0x7f230c113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.902+0000 7f2310bc7700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f230c103780 0x7f230c113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.903+0000 7f2310bc7700 1 -- 172.21.15.33:0/1213457287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8005040 con 0x7f230c102de0 2024-04-25T02:09:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.903+0000 7f2310bc7700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c102de0 0x7f230c113950 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f230000c8c0 tx=0x7f23000044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.903+0000 7f23097fa700 1 -- 172.21.15.33:0/1213457287 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f230000a760 con 0x7f230c102de0 2024-04-25T02:09:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.903+0000 7f23097fa700 1 -- 172.21.15.33:0/1213457287 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f230000ad90 con 0x7f230c102de0 2024-04-25T02:09:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.903+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c10ddc0 con 0x7f230c102de0 2024-04-25T02:09:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.903+0000 7f23097fa700 1 -- 172.21.15.33:0/1213457287 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23000113a0 con 0x7f230c102de0 2024-04-25T02:09:44.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.903+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c10e350 con 0x7f230c102de0 2024-04-25T02:09:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.904+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c0fe530 con 0x7f230c102de0 2024-04-25T02:09:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.905+0000 7f23097fa700 1 -- 172.21.15.33:0/1213457287 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2300012400 con 0x7f230c102de0 2024-04-25T02:09:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.905+0000 7f23097fa700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f22fc06e140 0x7f22fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.905+0000 7f23097fa700 1 -- 172.21.15.33:0/1213457287 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f230008db60 con 0x7f230c102de0 2024-04-25T02:09:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.905+0000 7f230bfff700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f22fc06e140 0x7f22fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:44.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.906+0000 7f230bfff700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f22fc06e140 0x7f22fc0705f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f22f8004890 tx=0x7f22f8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:44.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:44.910+0000 7f23097fa700 1 -- 172.21.15.33:0/1213457287 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f230005aa90 con 0x7f230c102de0 2024-04-25T02:09:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.140+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f230c109ec0 con 0x7f22fc06e140 2024-04-25T02:09:45.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.141+0000 7f23097fa700 1 -- 172.21.15.33:0/1213457287 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f230c109ec0 con 0x7f22fc06e140 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f22fc06e140 msgr2=0x7f22fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f22fc06e140 0x7f22fc0705f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f22f8004890 tx=0x7f22f8009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c102de0 msgr2=0x7f230c113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c102de0 0x7f230c113950 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f230000c8c0 tx=0x7f23000044c0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 shutdown_connections 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f22fc06e140 0x7f22fc0705f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f230c102de0 0x7f230c113950 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 --2- 172.21.15.33:0/1213457287 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f230c103780 0x7f230c113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 >> 172.21.15.33:0/1213457287 conn(0x7f230c0f8990 msgr2=0x7f230c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 shutdown_connections 2024-04-25T02:09:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.143+0000 7f2312e2b700 1 -- 172.21.15.33:0/1213457287 wait complete. 2024-04-25T02:09:45.161 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:09:45.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.248+0000 7f8949815700 1 -- 172.21.15.33:0/2330178354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 msgr2=0x7f8944102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.248+0000 7f8949815700 1 --2- 172.21.15.33:0/2330178354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 0x7f8944102160 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8934004940 tx=0x7f8934009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:45.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.248+0000 7f8949815700 1 -- 172.21.15.33:0/2330178354 shutdown_connections 2024-04-25T02:09:45.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.248+0000 7f8949815700 1 --2- 172.21.15.33:0/2330178354 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440fbd40 0x7f89440fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.248+0000 7f8949815700 1 --2- 172.21.15.33:0/2330178354 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 0x7f8944102160 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.249+0000 7f8949815700 1 -- 172.21.15.33:0/2330178354 >> 172.21.15.33:0/2330178354 conn(0x7f8944071d00 msgr2=0x7f8944072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:45.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.249+0000 7f8949815700 1 -- 172.21.15.33:0/2330178354 shutdown_connections 2024-04-25T02:09:45.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.249+0000 7f8949815700 1 -- 172.21.15.33:0/2330178354 wait complete. 2024-04-25T02:09:45.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.249+0000 7f8949815700 1 Processor -- start 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.249+0000 7f8949815700 1 -- start start 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.249+0000 7f8949815700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440fbd40 0x7f8944113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.249+0000 7f8949815700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 0x7f89441140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.249+0000 7f8949815700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89441147b0 con 0x7f8944101d90 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.249+0000 7f8949815700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894410db10 con 0x7f89440fbd40 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f89427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 0x7f89441140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f89427fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 0x7f89441140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44330/0 (socket says 172.21.15.33:44330) 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f89427fc700 1 -- 172.21.15.33:0/1307484765 learned_addr learned my addr 172.21.15.33:0/1307484765 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f8942ffd700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440fbd40 0x7f8944113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f89427fc700 1 -- 172.21.15.33:0/1307484765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440fbd40 msgr2=0x7f8944113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f89427fc700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440fbd40 0x7f8944113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f89427fc700 1 -- 172.21.15.33:0/1307484765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934005040 con 0x7f8944101d90 2024-04-25T02:09:45.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f89427fc700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 0x7f89441140d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f893800b790 tx=0x7f8938009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:45.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f892b7fe700 1 -- 172.21.15.33:0/1307484765 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f893800f6f0 con 0x7f8944101d90 2024-04-25T02:09:45.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f892b7fe700 1 -- 172.21.15.33:0/1307484765 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f893800fd20 con 0x7f8944101d90 2024-04-25T02:09:45.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894410ddf0 con 0x7f8944101d90 2024-04-25T02:09:45.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f892b7fe700 1 -- 172.21.15.33:0/1307484765 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89380103a0 con 0x7f8944101d90 2024-04-25T02:09:45.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.250+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894410e3b0 con 0x7f8944101d90 2024-04-25T02:09:45.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.251+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89440fd430 con 0x7f8944101d90 2024-04-25T02:09:45.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.252+0000 7f892b7fe700 1 -- 172.21.15.33:0/1307484765 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8938011400 con 0x7f8944101d90 2024-04-25T02:09:45.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.252+0000 7f892b7fe700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f892c06e140 0x7f892c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:45.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.253+0000 7f892b7fe700 1 -- 172.21.15.33:0/1307484765 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f893808d960 con 0x7f8944101d90 2024-04-25T02:09:45.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.253+0000 7f8942ffd700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f892c06e140 0x7f892c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:45.261 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.253+0000 7f8942ffd700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f892c06e140 0x7f892c0705f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8934004560 tx=0x7f8934009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:45.264 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.257+0000 7f892b7fe700 1 -- 172.21.15.33:0/1307484765 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f893805a890 con 0x7f8944101d90 2024-04-25T02:09:45.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.485+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8944108e70 con 0x7f892c06e140 2024-04-25T02:09:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.486+0000 7f892b7fe700 1 -- 172.21.15.33:0/1307484765 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8944108e70 con 0x7f892c06e140 2024-04-25T02:09:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.488+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f892c06e140 msgr2=0x7f892c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.488+0000 7f8949815700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f892c06e140 0x7f892c0705f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8934004560 tx=0x7f8934009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.488+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 msgr2=0x7f89441140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.488+0000 7f8949815700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 0x7f89441140d0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f893800b790 tx=0x7f8938009250 comp rx=0 tx=0).stop 2024-04-25T02:09:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.488+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 shutdown_connections 2024-04-25T02:09:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.488+0000 7f8949815700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f892c06e140 0x7f892c0705f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.488+0000 7f8949815700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f89440fbd40 0x7f8944113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.488+0000 7f8949815700 1 --2- 172.21.15.33:0/1307484765 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8944101d90 0x7f89441140d0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.488+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 >> 172.21.15.33:0/1307484765 conn(0x7f8944071d00 msgr2=0x7f89440fa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.489+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 shutdown_connections 2024-04-25T02:09:45.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.489+0000 7f8949815700 1 -- 172.21.15.33:0/1307484765 wait complete. 2024-04-25T02:09:45.599 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 -- 172.21.15.33:0/1839960151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 msgr2=0x7fe7381031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 --2- 172.21.15.33:0/1839960151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 0x7fe7381031b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe730009ce0 tx=0x7fe730004d10 comp rx=0 tx=0).stop 2024-04-25T02:09:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 -- 172.21.15.33:0/1839960151 shutdown_connections 2024-04-25T02:09:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 --2- 172.21.15.33:0/1839960151 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe738103780 0x7fe7380fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 --2- 172.21.15.33:0/1839960151 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 0x7fe7381031b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 -- 172.21.15.33:0/1839960151 >> 172.21.15.33:0/1839960151 conn(0x7fe7380f8990 msgr2=0x7fe7380fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 -- 172.21.15.33:0/1839960151 shutdown_connections 2024-04-25T02:09:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 -- 172.21.15.33:0/1839960151 wait complete. 2024-04-25T02:09:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 Processor -- start 2024-04-25T02:09:45.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.592+0000 7fe73dabc700 1 -- start start 2024-04-25T02:09:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe73dabc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 0x7fe738113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe73dabc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe738103780 0x7fe738114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe73dabc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe738114660 con 0x7fe738102de0 2024-04-25T02:09:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe73dabc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73810dc90 con 0x7fe738103780 2024-04-25T02:09:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe7377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 0x7fe738113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe7377fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 0x7fe738113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44348/0 (socket says 172.21.15.33:44348) 2024-04-25T02:09:45.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe7377fe700 1 -- 172.21.15.33:0/882011143 learned_addr learned my addr 172.21.15.33:0/882011143 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe736ffd700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe738103780 0x7fe738114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe7377fe700 1 -- 172.21.15.33:0/882011143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe738103780 msgr2=0x7fe738114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe7377fe700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe738103780 0x7fe738114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe7377fe700 1 -- 172.21.15.33:0/882011143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730009940 con 0x7fe738102de0 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe736ffd700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe738103780 0x7fe738114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe7377fe700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 0x7fe738113ad0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe73000be90 tx=0x7fe730004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe72ffff700 1 -- 172.21.15.33:0/882011143 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe73000e040 con 0x7fe738102de0 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe72ffff700 1 -- 172.21.15.33:0/882011143 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe730008ea0 con 0x7fe738102de0 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe72ffff700 1 -- 172.21.15.33:0/882011143 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe73001c810 con 0x7fe738102de0 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73810df10 con 0x7fe738102de0 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.593+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73810e480 con 0x7fe738102de0 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.594+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7381000f0 con 0x7fe738102de0 2024-04-25T02:09:45.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.595+0000 7fe72ffff700 1 -- 172.21.15.33:0/882011143 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe730012c00 con 0x7fe738102de0 2024-04-25T02:09:45.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.595+0000 7fe72ffff700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe72406e140 0x7fe7240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:45.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.595+0000 7fe72ffff700 1 -- 172.21.15.33:0/882011143 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe73008f750 con 0x7fe738102de0 2024-04-25T02:09:45.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.595+0000 7fe736ffd700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe72406e140 0x7fe7240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:45.604 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.596+0000 7fe736ffd700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe72406e140 0x7fe7240705f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe720007fd0 tx=0x7fe720009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:45.607 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.600+0000 7fe72ffff700 1 -- 172.21.15.33:0/882011143 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe73005c600 con 0x7fe738102de0 2024-04-25T02:09:45.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.816+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe738109ee0 con 0x7fe72406e140 2024-04-25T02:09:45.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.826+0000 7fe72ffff700 1 -- 172.21.15.33:0/882011143 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe738109ee0 con 0x7fe72406e140 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (30m) 8m ago 42m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (28m) 8m ago 42m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (28m) 8m ago 41m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (30m) 8m ago 41m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (24m) 8m ago 36m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (24m) 8m ago 36m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (23m) 8m ago 36m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (8m) 8m ago 36m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (32m) 8m ago 43m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (31m) 8m ago 41m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (29m) 8m ago 43m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (28m) 8m ago 41m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:09:45.834 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (31m) 8m ago 42m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:09:45.835 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (31m) 8m ago 41m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:09:45.835 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (27m) 8m ago 40m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:09:45.835 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (27m) 8m ago 39m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:09:45.835 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (26m) 8m ago 39m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:09:45.835 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (25m) 8m ago 39m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:09:45.835 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (25m) 8m ago 38m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:09:45.835 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (24m) 8m ago 38m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:09:45.835 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (31m) 8m ago 41m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe72406e140 msgr2=0x7fe7240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe72406e140 0x7fe7240705f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe720007fd0 tx=0x7fe720009040 comp rx=0 tx=0).stop 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 msgr2=0x7fe738113ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 0x7fe738113ad0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe73000be90 tx=0x7fe730004d10 comp rx=0 tx=0).stop 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 shutdown_connections 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe72406e140 0x7fe7240705f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe738102de0 0x7fe738113ad0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 --2- 172.21.15.33:0/882011143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe738103780 0x7fe738114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 >> 172.21.15.33:0/882011143 conn(0x7fe7380f8990 msgr2=0x7fe738107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 shutdown_connections 2024-04-25T02:09:45.836 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.828+0000 7fe73dabc700 1 -- 172.21.15.33:0/882011143 wait complete. 2024-04-25T02:09:45.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 -- 172.21.15.33:0/4230282714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 msgr2=0x7fb7a40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 --2- 172.21.15.33:0/4230282714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 0x7fb7a40fcdf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fb78c004940 tx=0x7fb78c009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:45.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 -- 172.21.15.33:0/4230282714 shutdown_connections 2024-04-25T02:09:45.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 --2- 172.21.15.33:0/4230282714 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 0x7fb7a40fcdf0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 --2- 172.21.15.33:0/4230282714 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a4102de0 0x7fb7a41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 -- 172.21.15.33:0/4230282714 >> 172.21.15.33:0/4230282714 conn(0x7fb7a40f89b0 msgr2=0x7fb7a40fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:45.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 -- 172.21.15.33:0/4230282714 shutdown_connections 2024-04-25T02:09:45.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 -- 172.21.15.33:0/4230282714 wait complete. 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 Processor -- start 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.931+0000 7fb7a97ab700 1 -- start start 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a97ab700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a4102de0 0x7fb7a4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a97ab700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 0x7fb7a40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a97ab700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4070d30 con 0x7fb7a4103780 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a97ab700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4070e70 con 0x7fb7a4102de0 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 0x7fb7a40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a27fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 0x7fb7a40707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58394/0 (socket says 172.21.15.33:58394) 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a27fc700 1 -- 172.21.15.33:0/2192120667 learned_addr learned my addr 172.21.15.33:0/2192120667 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a2ffd700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a4102de0 0x7fb7a4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a27fc700 1 -- 172.21.15.33:0/2192120667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a4102de0 msgr2=0x7fb7a4070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a27fc700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a4102de0 0x7fb7a4070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:45.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a27fc700 1 -- 172.21.15.33:0/2192120667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c005040 con 0x7fb7a4103780 2024-04-25T02:09:45.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a2ffd700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a4102de0 0x7fb7a4070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:09:45.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb7a27fc700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 0x7fb7a40707a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009180 tx=0x7fb78c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:45.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb79b7fe700 1 -- 172.21.15.33:0/2192120667 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb78c00e040 con 0x7fb7a4103780 2024-04-25T02:09:45.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb79b7fe700 1 -- 172.21.15.33:0/2192120667 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb78c00bbc0 con 0x7fb7a4103780 2024-04-25T02:09:45.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.932+0000 7fb79b7fe700 1 -- 172.21.15.33:0/2192120667 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb78c01c3f0 con 0x7fb7a4103780 2024-04-25T02:09:45.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.933+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a40710c0 con 0x7fb7a4103780 2024-04-25T02:09:45.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.933+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a411a250 con 0x7fb7a4103780 2024-04-25T02:09:45.942 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.933+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a40fe530 con 0x7fb7a4103780 2024-04-25T02:09:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.934+0000 7fb79b7fe700 1 -- 172.21.15.33:0/2192120667 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb78c00bd30 con 0x7fb7a4103780 2024-04-25T02:09:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.934+0000 7fb79b7fe700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb79006e140 0x7fb7900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.935+0000 7fb79b7fe700 1 -- 172.21.15.33:0/2192120667 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb78c08ee50 con 0x7fb7a4103780 2024-04-25T02:09:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.935+0000 7fb7a2ffd700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb79006e140 0x7fb7900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:45.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.935+0000 7fb7a2ffd700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb79006e140 0x7fb7900705f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb79c004ee0 tx=0x7fb79c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:45.946 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:45.939+0000 7fb79b7fe700 1 -- 172.21.15.33:0/2192120667 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb78c058250 con 0x7fb7a4103780 2024-04-25T02:09:46.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.238+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb7a4109ec0 con 0x7fb7a4103780 2024-04-25T02:09:46.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.238+0000 7fb79b7fe700 1 -- 172.21.15.33:0/2192120667 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fb78c05b910 con 0x7fb7a4103780 2024-04-25T02:09:46.246 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:09:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:09:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:09:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:09:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:09:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:09:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:09:46.246 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:09:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:09:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:09:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:09:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:09:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:09:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:09:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:09:46.247 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:09:46.247 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:09:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.240+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb79006e140 msgr2=0x7fb7900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.240+0000 7fb7a97ab700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb79006e140 0x7fb7900705f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb79c004ee0 tx=0x7fb79c009040 comp rx=0 tx=0).stop 2024-04-25T02:09:46.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.240+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 msgr2=0x7fb7a40707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.240+0000 7fb7a97ab700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 0x7fb7a40707a0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009180 tx=0x7fb78c009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:46.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.240+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 shutdown_connections 2024-04-25T02:09:46.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.240+0000 7fb7a97ab700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb79006e140 0x7fb7900705f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.240+0000 7fb7a97ab700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb7a4102de0 0x7fb7a4070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.240+0000 7fb7a97ab700 1 --2- 172.21.15.33:0/2192120667 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb7a4103780 0x7fb7a40707a0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.240+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 >> 172.21.15.33:0/2192120667 conn(0x7fb7a40f89b0 msgr2=0x7fb7a40f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:46.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.241+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 shutdown_connections 2024-04-25T02:09:46.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.241+0000 7fb7a97ab700 1 -- 172.21.15.33:0/2192120667 wait complete. 2024-04-25T02:09:46.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 -- 172.21.15.33:0/1462271047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 msgr2=0x7f2d9c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 --2- 172.21.15.33:0/1462271047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 0x7f2d9c1031b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c004940 tx=0x7f2d8c009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:46.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 -- 172.21.15.33:0/1462271047 shutdown_connections 2024-04-25T02:09:46.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 --2- 172.21.15.33:0/1462271047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c103780 0x7f2d9c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 --2- 172.21.15.33:0/1462271047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 0x7f2d9c1031b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 -- 172.21.15.33:0/1462271047 >> 172.21.15.33:0/1462271047 conn(0x7f2d9c0f8990 msgr2=0x7f2d9c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 -- 172.21.15.33:0/1462271047 shutdown_connections 2024-04-25T02:09:46.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 -- 172.21.15.33:0/1462271047 wait complete. 2024-04-25T02:09:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 Processor -- start 2024-04-25T02:09:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.345+0000 7f2da2338700 1 -- start start 2024-04-25T02:09:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2da2338700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 0x7f2d9c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2da2338700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c103780 0x7f2d9c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2da2338700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c114500 con 0x7f2d9c102de0 2024-04-25T02:09:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2da2338700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c10db50 con 0x7f2d9c103780 2024-04-25T02:09:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 0x7f2d9c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:46.354 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d9bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 0x7f2d9c113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58412/0 (socket says 172.21.15.33:58412) 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d9bfff700 1 -- 172.21.15.33:0/677888269 learned_addr learned my addr 172.21.15.33:0/677888269 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d9b7fe700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c103780 0x7f2d9c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d9bfff700 1 -- 172.21.15.33:0/677888269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c103780 msgr2=0x7f2d9c113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d9bfff700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c103780 0x7f2d9c113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d9bfff700 1 -- 172.21.15.33:0/677888269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c005040 con 0x7f2d9c102de0 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d9bfff700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 0x7f2d9c113970 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009150 tx=0x7f2d8c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d98ff9700 1 -- 172.21.15.33:0/677888269 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d8c00e040 con 0x7f2d9c102de0 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c10ddd0 con 0x7f2d9c102de0 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d98ff9700 1 -- 172.21.15.33:0/677888269 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d8c013420 con 0x7f2d9c102de0 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.346+0000 7f2d98ff9700 1 -- 172.21.15.33:0/677888269 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d8c01c610 con 0x7f2d9c102de0 2024-04-25T02:09:46.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.347+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c10e390 con 0x7f2d9c102de0 2024-04-25T02:09:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.348+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9c1000f0 con 0x7f2d9c102de0 2024-04-25T02:09:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.348+0000 7f2d98ff9700 1 -- 172.21.15.33:0/677888269 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2d8c00baf0 con 0x7f2d9c102de0 2024-04-25T02:09:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.348+0000 7f2d98ff9700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d8406e140 0x7f2d840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.349+0000 7f2d98ff9700 1 -- 172.21.15.33:0/677888269 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2d8c08ed40 con 0x7f2d9c102de0 2024-04-25T02:09:46.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.349+0000 7f2d9b7fe700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d8406e140 0x7f2d840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:46.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.349+0000 7f2d9b7fe700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d8406e140 0x7f2d840705f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c10ecc0 tx=0x7f2d9000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:46.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.353+0000 7f2d98ff9700 1 -- 172.21.15.33:0/677888269 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2d8c05bc70 con 0x7f2d9c102de0 2024-04-25T02:09:46.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:46 smithi033 ceph-mon[111306]: from='client.35664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:46.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:46 smithi033 ceph-mon[111306]: pgmap v912: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:46.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:46 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2192120667' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:09:46.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.610+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2d9c10efc0 con 0x7f2d9c102de0 2024-04-25T02:09:46.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.610+0000 7f2d98ff9700 1 -- 172.21.15.33:0/677888269 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f2d8c05b800 con 0x7f2d9c102de0 2024-04-25T02:09:46.618 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:09:46.618 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:09:46.618 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:09:46.618 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:09:46.618 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:09:46.619 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:09:46.620 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:09:46.621 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:09:46.621 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:09:46.621 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:09:46.621 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:09:46.621 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:09:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.612+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d8406e140 msgr2=0x7f2d840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.612+0000 7f2da2338700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d8406e140 0x7f2d840705f0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2d9c10ecc0 tx=0x7f2d9000b040 comp rx=0 tx=0).stop 2024-04-25T02:09:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.612+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 msgr2=0x7f2d9c113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.612+0000 7f2da2338700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 0x7f2d9c113970 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c009150 tx=0x7f2d8c009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.612+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 shutdown_connections 2024-04-25T02:09:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.612+0000 7f2da2338700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d8406e140 0x7f2d840705f0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.612+0000 7f2da2338700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d9c102de0 0x7f2d9c113970 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.612+0000 7f2da2338700 1 --2- 172.21.15.33:0/677888269 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d9c103780 0x7f2d9c113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.612+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 >> 172.21.15.33:0/677888269 conn(0x7f2d9c0f8990 msgr2=0x7f2d9c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.613+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 shutdown_connections 2024-04-25T02:09:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.613+0000 7f2da2338700 1 -- 172.21.15.33:0/677888269 wait complete. 2024-04-25T02:09:46.623 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 -- 172.21.15.33:0/967972457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 msgr2=0x7f94f40fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 --2- 172.21.15.33:0/967972457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 0x7f94f40fcdf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f94e4004940 tx=0x7f94e4009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 -- 172.21.15.33:0/967972457 shutdown_connections 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 --2- 172.21.15.33:0/967972457 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 0x7f94f40fcdf0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 --2- 172.21.15.33:0/967972457 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f4102de0 0x7f94f41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 -- 172.21.15.33:0/967972457 >> 172.21.15.33:0/967972457 conn(0x7f94f40f8990 msgr2=0x7f94f40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 -- 172.21.15.33:0/967972457 shutdown_connections 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 -- 172.21.15.33:0/967972457 wait complete. 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 Processor -- start 2024-04-25T02:09:46.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.718+0000 7f94fc6a8700 1 -- start start 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94fc6a8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f4102de0 0x7f94f4070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94fc6a8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 0x7f94f40707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94fc6a8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f40721b0 con 0x7f94f4103780 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94fc6a8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f4070ce0 con 0x7f94f4102de0 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94f9c43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 0x7f94f40707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94f9c43700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 0x7f94f40707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58438/0 (socket says 172.21.15.33:58438) 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94f9c43700 1 -- 172.21.15.33:0/3107312307 learned_addr learned my addr 172.21.15.33:0/3107312307 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94fa444700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f4102de0 0x7f94f4070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94f9c43700 1 -- 172.21.15.33:0/3107312307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f4102de0 msgr2=0x7f94f4070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94f9c43700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f4102de0 0x7f94f4070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94f9c43700 1 -- 172.21.15.33:0/3107312307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e4005040 con 0x7f94f4103780 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94f9c43700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 0x7f94f40707a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f94e400b3b0 tx=0x7f94e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94eaffd700 1 -- 172.21.15.33:0/3107312307 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e400e040 con 0x7f94f4103780 2024-04-25T02:09:46.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94eaffd700 1 -- 172.21.15.33:0/3107312307 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94e4009b10 con 0x7f94f4103780 2024-04-25T02:09:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f4070f60 con 0x7f94f4103780 2024-04-25T02:09:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.719+0000 7f94eaffd700 1 -- 172.21.15.33:0/3107312307 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e4013420 con 0x7f94f4103780 2024-04-25T02:09:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.720+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f411a0a0 con 0x7f94f4103780 2024-04-25T02:09:46.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.720+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f40fe530 con 0x7f94f4103780 2024-04-25T02:09:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.721+0000 7f94eaffd700 1 -- 172.21.15.33:0/3107312307 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f94e401c7c0 con 0x7f94f4103780 2024-04-25T02:09:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.721+0000 7f94eaffd700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94e006e140 0x7f94e00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.722+0000 7f94eaffd700 1 -- 172.21.15.33:0/3107312307 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f94e408e540 con 0x7f94f4103780 2024-04-25T02:09:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.722+0000 7f94fa444700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94e006e140 0x7f94e00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:46.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.722+0000 7f94fa444700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94e006e140 0x7f94e00705f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f94f0004f40 tx=0x7f94f000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:46.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.726+0000 7f94eaffd700 1 -- 172.21.15.33:0/3107312307 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f94e4057a90 con 0x7f94f4103780 2024-04-25T02:09:46.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:46 smithi138 ceph-mon[81304]: from='client.35664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:46.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:46 smithi138 ceph-mon[81304]: pgmap v912: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:46.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:46 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2192120667' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:09:46.959 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.952+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94f40fa0a0 con 0x7f94e006e140 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.953+0000 7f94eaffd700 1 -- 172.21.15.33:0/3107312307 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f94f40fa0a0 con 0x7f94e006e140 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:09:46.961 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:09:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:09:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:09:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:09:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:09:46.962 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:09:46.962 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:09:46.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.955+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94e006e140 msgr2=0x7f94e00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.955+0000 7f94fc6a8700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94e006e140 0x7f94e00705f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f94f0004f40 tx=0x7f94f000b040 comp rx=0 tx=0).stop 2024-04-25T02:09:46.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.955+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 msgr2=0x7f94f40707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:46.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.955+0000 7f94fc6a8700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 0x7f94f40707a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f94e400b3b0 tx=0x7f94e4009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:46.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.955+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 shutdown_connections 2024-04-25T02:09:46.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.955+0000 7f94fc6a8700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94e006e140 0x7f94e00705f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.963 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.955+0000 7f94fc6a8700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94f4102de0 0x7f94f4070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.955+0000 7f94fc6a8700 1 --2- 172.21.15.33:0/3107312307 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94f4103780 0x7f94f40707a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:46.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.955+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 >> 172.21.15.33:0/3107312307 conn(0x7f94f40f8990 msgr2=0x7f94f40f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:46.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.956+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 shutdown_connections 2024-04-25T02:09:46.964 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:46.956+0000 7f94fc6a8700 1 -- 172.21.15.33:0/3107312307 wait complete. 2024-04-25T02:09:47.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 -- 172.21.15.33:0/2296633998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 msgr2=0x7f25b01020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:47.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 --2- 172.21.15.33:0/2296633998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 0x7f25b01020e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f25a0004940 tx=0x7f25a0009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:47.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 -- 172.21.15.33:0/2296633998 shutdown_connections 2024-04-25T02:09:47.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 --2- 172.21.15.33:0/2296633998 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b01026b0 0x7f25b00fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:47.068 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 --2- 172.21.15.33:0/2296633998 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 0x7f25b01020e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 -- 172.21.15.33:0/2296633998 >> 172.21.15.33:0/2296633998 conn(0x7f25b00f7860 msgr2=0x7f25b00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 -- 172.21.15.33:0/2296633998 shutdown_connections 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 -- 172.21.15.33:0/2296633998 wait complete. 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 Processor -- start 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.060+0000 7f25b728b700 1 -- start start 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b728b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 0x7f25b0113a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b728b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b01026b0 0x7f25b0113f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b728b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b01145e0 con 0x7f25b0101d10 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b728b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b010dbc0 con 0x7f25b01026b0 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b5027700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 0x7f25b0113a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b5027700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 0x7f25b0113a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58454/0 (socket says 172.21.15.33:58454) 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b5027700 1 -- 172.21.15.33:0/1098752824 learned_addr learned my addr 172.21.15.33:0/1098752824 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:09:47.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b4826700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b01026b0 0x7f25b0113f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:47.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b5027700 1 -- 172.21.15.33:0/1098752824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b01026b0 msgr2=0x7f25b0113f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:47.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b5027700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b01026b0 0x7f25b0113f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:47.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b5027700 1 -- 172.21.15.33:0/1098752824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a0005040 con 0x7f25b0101d10 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b4826700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b01026b0 0x7f25b0113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b5027700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 0x7f25b0113a50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f25a000b3b0 tx=0x7f25a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25a5ffb700 1 -- 172.21.15.33:0/1098752824 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a000e040 con 0x7f25b0101d10 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b010de40 con 0x7f25b0101d10 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25a5ffb700 1 -- 172.21.15.33:0/1098752824 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25a000bbc0 con 0x7f25b0101d10 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.061+0000 7f25a5ffb700 1 -- 172.21.15.33:0/1098752824 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25a001c3f0 con 0x7f25b0101d10 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.062+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b010e380 con 0x7f25b0101d10 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.062+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b00fd460 con 0x7f25b0101d10 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.063+0000 7f25a5ffb700 1 -- 172.21.15.33:0/1098752824 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f25a000bd30 con 0x7f25b0101d10 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.063+0000 7f25a5ffb700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f259c06e140 0x7f259c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.064+0000 7f25a5ffb700 1 -- 172.21.15.33:0/1098752824 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f25a008edb0 con 0x7f25b0101d10 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.064+0000 7f25b4826700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f259c06e140 0x7f259c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:09:47.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.064+0000 7f25b4826700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f259c06e140 0x7f259c0705f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007fd0 tx=0x7f25ac009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:09:47.076 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.068+0000 7f25a5ffb700 1 -- 172.21.15.33:0/1098752824 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f25a005bc60 con 0x7f25b0101d10 2024-04-25T02:09:47.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.361+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f25b010f0a0 con 0x7f25b0101d10 2024-04-25T02:09:47.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.361+0000 7f25a5ffb700 1 -- 172.21.15.33:0/1098752824 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f25a005b7f0 con 0x7f25b0101d10 2024-04-25T02:09:47.369 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:09:47.370 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:09:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:09:47.370 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:09:47.370 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:09:47.370 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:09:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:09:47.370 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:09:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:09:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:09:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:09:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:09:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:09:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:09:47.371 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:09:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:09:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:09:47.372 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:09:47.372 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:09:47.372 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:09:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:09:47.373 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:09:47.373 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:09:47.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f259c06e140 msgr2=0x7f259c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:47.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f259c06e140 0x7f259c0705f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f25ac007fd0 tx=0x7f25ac009040 comp rx=0 tx=0).stop 2024-04-25T02:09:47.374 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 msgr2=0x7f25b0113a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:09:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 0x7f25b0113a50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f25a000b3b0 tx=0x7f25a0009e30 comp rx=0 tx=0).stop 2024-04-25T02:09:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 shutdown_connections 2024-04-25T02:09:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f259c06e140 0x7f259c0705f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f25b0101d10 0x7f25b0113a50 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 --2- 172.21.15.33:0/1098752824 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f25b01026b0 0x7f25b0113f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:09:47.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 >> 172.21.15.33:0/1098752824 conn(0x7f25b00f7860 msgr2=0x7f25b00f8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:09:47.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 shutdown_connections 2024-04-25T02:09:47.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:09:47.363+0000 7f25b728b700 1 -- 172.21.15.33:0/1098752824 wait complete. 2024-04-25T02:09:47.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:47 smithi033 ceph-mon[111306]: from='client.35668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:47.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:47 smithi033 ceph-mon[111306]: from='client.35672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:47.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:47 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/677888269' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:09:47.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:47 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1098752824' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:09:47.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:47 smithi138 ceph-mon[81304]: from='client.35668 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:47.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:47 smithi138 ceph-mon[81304]: from='client.35672 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:47.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:47 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/677888269' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:09:47.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:47 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1098752824' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:09:48.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:48 smithi033 ceph-mon[111306]: from='client.35684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:48.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:48 smithi033 ceph-mon[111306]: pgmap v913: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:48.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:48 smithi138 ceph-mon[81304]: from='client.35684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:09:48.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:48 smithi138 ceph-mon[81304]: pgmap v913: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:50.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:50 smithi033 ceph-mon[111306]: pgmap v914: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:50.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:50 smithi138 ceph-mon[81304]: pgmap v914: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:52.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:52 smithi033 ceph-mon[111306]: pgmap v915: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:52.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:52 smithi138 ceph-mon[81304]: pgmap v915: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:54.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:54 smithi033 ceph-mon[111306]: pgmap v916: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:54.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:54 smithi138 ceph-mon[81304]: pgmap v916: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:55.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:09:55.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:09:56.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:56 smithi033 ceph-mon[111306]: pgmap v917: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:56.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:56 smithi138 ceph-mon[81304]: pgmap v917: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:57.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:09:57 smithi033 ceph-mon[111306]: pgmap v918: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:09:57.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:09:57 smithi138 ceph-mon[81304]: pgmap v918: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:00.274 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:10:00.274 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:10:00.274 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:10:00.274 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-04-25T02:10:00.274 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:00.274 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-04-25T02:10:00.274 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-04-25T02:10:00.275 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:10:00.276 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:09:59.999+0000 7f05e5d17700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: pgmap v919: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: /usr/bin/podman: timeout after 895 seconds 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: Traceback (most recent call last): 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: return _run_code(code, main_globals, None, 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:10:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: exec(code, run_globals) 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: fs cephfs is degraded 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:10:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:00 smithi033 ceph-mon[111306]: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:10:00.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: pgmap v919: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:00.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:10:00.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:10:00.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:10:00.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: /usr/bin/podman: timeout after 895 seconds 2024-04-25T02:10:00.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: Traceback (most recent call last): 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: return _run_code(code, main_globals, None, 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: exec(code, run_globals) 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:10:00.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:10:00.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:10:00.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:10:00.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:00.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:10:00.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: fs cephfs is degraded 2024-04-25T02:10:00.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:10:00.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:00 smithi138 ceph-mon[81304]: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:10:02.601 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:02 smithi138 ceph-mon[81304]: pgmap v920: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:02.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:02 smithi033 ceph-mon[111306]: pgmap v920: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:04.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:04 smithi033 ceph-mon[111306]: pgmap v921: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:04.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:04 smithi138 ceph-mon[81304]: pgmap v921: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:06.576 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:06 smithi033 ceph-mon[111306]: pgmap v922: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:06.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:06 smithi138 ceph-mon[81304]: pgmap v922: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:08.273 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:07 smithi033 ceph-mon[111306]: pgmap v923: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:08.299 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:07 smithi138 ceph-mon[81304]: pgmap v923: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:10 smithi033 ceph-mon[111306]: pgmap v924: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:10 smithi138 ceph-mon[81304]: pgmap v924: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:11.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:10:11.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:10:12.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:12 smithi033 ceph-mon[111306]: pgmap v925: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:12.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:12 smithi138 ceph-mon[81304]: pgmap v925: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:14 smithi033 ceph-mon[111306]: pgmap v926: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:14 smithi138 ceph-mon[81304]: pgmap v926: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:16.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:16 smithi033 ceph-mon[111306]: pgmap v927: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:16.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:16 smithi138 ceph-mon[81304]: pgmap v927: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.482+0000 7f617b51c700 1 -- 172.21.15.33:0/1464731023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 msgr2=0x7f61740fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.482+0000 7f617b51c700 1 --2- 172.21.15.33:0/1464731023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 0x7f61740fce00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f6170004940 tx=0x7f6170009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.482+0000 7f617b51c700 1 -- 172.21.15.33:0/1464731023 shutdown_connections 2024-04-25T02:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.482+0000 7f617b51c700 1 --2- 172.21.15.33:0/1464731023 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 0x7f61740fce00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.482+0000 7f617b51c700 1 --2- 172.21.15.33:0/1464731023 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6174102df0 0x7f61741031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.482+0000 7f617b51c700 1 -- 172.21.15.33:0/1464731023 >> 172.21.15.33:0/1464731023 conn(0x7f61740f8980 msgr2=0x7f61740fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.482+0000 7f617b51c700 1 -- 172.21.15.33:0/1464731023 shutdown_connections 2024-04-25T02:10:17.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.483+0000 7f617b51c700 1 -- 172.21.15.33:0/1464731023 wait complete. 2024-04-25T02:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.483+0000 7f617b51c700 1 Processor -- start 2024-04-25T02:10:17.492 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.483+0000 7f617b51c700 1 -- start start 2024-04-25T02:10:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.483+0000 7f617b51c700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6174102df0 0x7f6174113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f617b51c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 0x7f61741140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f617b51c700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6174114710 con 0x7f6174103790 2024-04-25T02:10:17.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f617b51c700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617410db00 con 0x7f6174102df0 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6178ab7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 0x7f61741140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6178ab7700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 0x7f61741140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44578/0 (socket says 172.21.15.33:44578) 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6178ab7700 1 -- 172.21.15.33:0/2894675087 learned_addr learned my addr 172.21.15.33:0/2894675087 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f61792b8700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6174102df0 0x7f6174113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6178ab7700 1 -- 172.21.15.33:0/2894675087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6174102df0 msgr2=0x7f6174113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6178ab7700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6174102df0 0x7f6174113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6178ab7700 1 -- 172.21.15.33:0/2894675087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6170005040 con 0x7f6174103790 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6178ab7700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 0x7f61741140c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6170009100 tx=0x7f6170009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6169ffb700 1 -- 172.21.15.33:0/2894675087 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f617000e040 con 0x7f6174103790 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6169ffb700 1 -- 172.21.15.33:0/2894675087 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6170013420 con 0x7f6174103790 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f617410dd80 con 0x7f6174103790 2024-04-25T02:10:17.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.484+0000 7f6169ffb700 1 -- 172.21.15.33:0/2894675087 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f617001c610 con 0x7f6174103790 2024-04-25T02:10:17.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.485+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617410e340 con 0x7f6174103790 2024-04-25T02:10:17.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.486+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61740fe540 con 0x7f6174103790 2024-04-25T02:10:17.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.486+0000 7f6169ffb700 1 -- 172.21.15.33:0/2894675087 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6170013590 con 0x7f6174103790 2024-04-25T02:10:17.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.487+0000 7f6169ffb700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f616006e140 0x7f61600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:17.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.487+0000 7f6169ffb700 1 -- 172.21.15.33:0/2894675087 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f617008ee50 con 0x7f6174103790 2024-04-25T02:10:17.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.487+0000 7f61792b8700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f616006e140 0x7f61600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:17.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.487+0000 7f61792b8700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f616006e140 0x7f61600705f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6164009fd0 tx=0x7f6164009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:17.499 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.492+0000 7f6169ffb700 1 -- 172.21.15.33:0/2894675087 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f617005bce0 con 0x7f6174103790 2024-04-25T02:10:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.722+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6174109ed0 con 0x7f616006e140 2024-04-25T02:10:17.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.723+0000 7f6169ffb700 1 -- 172.21.15.33:0/2894675087 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6174109ed0 con 0x7f616006e140 2024-04-25T02:10:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f616006e140 msgr2=0x7f61600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f616006e140 0x7f61600705f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6164009fd0 tx=0x7f6164009e50 comp rx=0 tx=0).stop 2024-04-25T02:10:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 msgr2=0x7f61741140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 0x7f61741140c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6170009100 tx=0x7f6170009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 shutdown_connections 2024-04-25T02:10:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f616006e140 0x7f61600705f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6174102df0 0x7f6174113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 --2- 172.21.15.33:0/2894675087 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6174103790 0x7f61741140c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:17.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 >> 172.21.15.33:0/2894675087 conn(0x7f61740f8980 msgr2=0x7f61740f99e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:17.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 shutdown_connections 2024-04-25T02:10:17.734 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.725+0000 7f617b51c700 1 -- 172.21.15.33:0/2894675087 wait complete. 2024-04-25T02:10:17.743 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:10:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.830+0000 7f18f94fc700 1 -- 172.21.15.33:0/2819017261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 msgr2=0x7f18f40fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.830+0000 7f18f94fc700 1 --2- 172.21.15.33:0/2819017261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 0x7f18f40fb990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f18dc004940 tx=0x7f18dc009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:17.838 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.830+0000 7f18f94fc700 1 -- 172.21.15.33:0/2819017261 shutdown_connections 2024-04-25T02:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.830+0000 7f18f94fc700 1 --2- 172.21.15.33:0/2819017261 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 0x7f18f40fb990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.830+0000 7f18f94fc700 1 --2- 172.21.15.33:0/2819017261 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4102f60 0x7f18f4103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.830+0000 7f18f94fc700 1 -- 172.21.15.33:0/2819017261 >> 172.21.15.33:0/2819017261 conn(0x7f18f40714d0 msgr2=0x7f18f40718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.830+0000 7f18f94fc700 1 -- 172.21.15.33:0/2819017261 shutdown_connections 2024-04-25T02:10:17.839 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.830+0000 7f18f94fc700 1 -- 172.21.15.33:0/2819017261 wait complete. 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.831+0000 7f18f94fc700 1 Processor -- start 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.831+0000 7f18f94fc700 1 -- start start 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.831+0000 7f18f94fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 0x7f18f4113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.831+0000 7f18f94fc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4102f60 0x7f18f4113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.831+0000 7f18f94fc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f4114590 con 0x7f18f40fb520 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.831+0000 7f18f94fc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f410dbb0 con 0x7f18f4102f60 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.831+0000 7f18f2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 0x7f18f4113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.831+0000 7f18f2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 0x7f18f4113a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44592/0 (socket says 172.21.15.33:44592) 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.831+0000 7f18f2ffd700 1 -- 172.21.15.33:0/1035146867 learned_addr learned my addr 172.21.15.33:0/1035146867 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18f27fc700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4102f60 0x7f18f4113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18f2ffd700 1 -- 172.21.15.33:0/1035146867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4102f60 msgr2=0x7f18f4113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:17.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18f2ffd700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4102f60 0x7f18f4113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18f2ffd700 1 -- 172.21.15.33:0/1035146867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc005040 con 0x7f18f40fb520 2024-04-25T02:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18f27fc700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4102f60 0x7f18f4113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18f2ffd700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 0x7f18f4113a00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f18e400c8c0 tx=0x7f18e40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18eb7fe700 1 -- 172.21.15.33:0/1035146867 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e400a980 con 0x7f18f40fb520 2024-04-25T02:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f410de90 con 0x7f18f40fb520 2024-04-25T02:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18eb7fe700 1 -- 172.21.15.33:0/1035146867 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f18e400ce60 con 0x7f18f40fb520 2024-04-25T02:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18eb7fe700 1 -- 172.21.15.33:0/1035146867 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e40115e0 con 0x7f18f40fb520 2024-04-25T02:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.832+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f410e3d0 con 0x7f18f40fb520 2024-04-25T02:10:17.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.833+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f4100250 con 0x7f18f40fb520 2024-04-25T02:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.834+0000 7f18eb7fe700 1 -- 172.21.15.33:0/1035146867 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f18e4012400 con 0x7f18f40fb520 2024-04-25T02:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.834+0000 7f18eb7fe700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f18e006e140 0x7f18e00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.834+0000 7f18eb7fe700 1 -- 172.21.15.33:0/1035146867 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f18e408dc10 con 0x7f18f40fb520 2024-04-25T02:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.834+0000 7f18f27fc700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f18e006e140 0x7f18e00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:17.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.835+0000 7f18f27fc700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f18e006e140 0x7f18e00705f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000c00 tx=0x7f18dc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:17.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:17.839+0000 7f18eb7fe700 1 -- 172.21.15.33:0/1035146867 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f18e4057010 con 0x7f18f40fb520 2024-04-25T02:10:18.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.067+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18f410a050 con 0x7f18e006e140 2024-04-25T02:10:18.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.068+0000 7f18eb7fe700 1 -- 172.21.15.33:0/1035146867 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f18f410a050 con 0x7f18e006e140 2024-04-25T02:10:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f18e006e140 msgr2=0x7f18e00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f18e006e140 0x7f18e00705f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f18dc000c00 tx=0x7f18dc009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 msgr2=0x7f18f4113a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.078 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 0x7f18f4113a00 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f18e400c8c0 tx=0x7f18e40044c0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 shutdown_connections 2024-04-25T02:10:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f18e006e140 0x7f18e00705f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18f40fb520 0x7f18f4113a00 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 --2- 172.21.15.33:0/1035146867 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18f4102f60 0x7f18f4113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 >> 172.21.15.33:0/1035146867 conn(0x7f18f40714d0 msgr2=0x7f18f40fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 shutdown_connections 2024-04-25T02:10:18.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.070+0000 7f18f94fc700 1 -- 172.21.15.33:0/1035146867 wait complete. 2024-04-25T02:10:18.183 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.174+0000 7f479838b700 1 -- 172.21.15.33:0/1483603528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790102de0 msgr2=0x7f47901031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.174+0000 7f479838b700 1 --2- 172.21.15.33:0/1483603528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790102de0 0x7f47901031b0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4788004940 tx=0x7f4788009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.174+0000 7f479838b700 1 -- 172.21.15.33:0/1483603528 shutdown_connections 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.174+0000 7f479838b700 1 --2- 172.21.15.33:0/1483603528 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4790103780 0x7f47900fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.174+0000 7f479838b700 1 --2- 172.21.15.33:0/1483603528 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790102de0 0x7f47901031b0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.174+0000 7f479838b700 1 -- 172.21.15.33:0/1483603528 >> 172.21.15.33:0/1483603528 conn(0x7f47900f8990 msgr2=0x7f47900fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.174+0000 7f479838b700 1 -- 172.21.15.33:0/1483603528 shutdown_connections 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.174+0000 7f479838b700 1 -- 172.21.15.33:0/1483603528 wait complete. 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f479838b700 1 Processor -- start 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f479838b700 1 -- start start 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f479838b700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4790102de0 0x7f4790070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f479838b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790103780 0x7f47900707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f479838b700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4790072240 con 0x7f4790103780 2024-04-25T02:10:18.184 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f479838b700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4790070ce0 con 0x7f4790102de0 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f4795926700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790103780 0x7f47900707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f4795926700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790103780 0x7f47900707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44614/0 (socket says 172.21.15.33:44614) 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f4795926700 1 -- 172.21.15.33:0/3210253442 learned_addr learned my addr 172.21.15.33:0/3210253442 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f4796127700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4790102de0 0x7f4790070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f4795926700 1 -- 172.21.15.33:0/3210253442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4790102de0 msgr2=0x7f4790070260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f4795926700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4790102de0 0x7f4790070260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f4795926700 1 -- 172.21.15.33:0/3210253442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4788005040 con 0x7f4790103780 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.175+0000 7f4796127700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4790102de0 0x7f4790070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.176+0000 7f4795926700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790103780 0x7f47900707a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f478400b790 tx=0x7f4784009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.176+0000 7f478effd700 1 -- 172.21.15.33:0/3210253442 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478400f6f0 con 0x7f4790103780 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.176+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4790070fc0 con 0x7f4790103780 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.176+0000 7f478effd700 1 -- 172.21.15.33:0/3210253442 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f478400fd20 con 0x7f4790103780 2024-04-25T02:10:18.185 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.176+0000 7f478effd700 1 -- 172.21.15.33:0/3210253442 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47840103a0 con 0x7f4790103780 2024-04-25T02:10:18.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.176+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479011a190 con 0x7f4790103780 2024-04-25T02:10:18.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.177+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47901000f0 con 0x7f4790103780 2024-04-25T02:10:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.177+0000 7f478effd700 1 -- 172.21.15.33:0/3210253442 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4784011400 con 0x7f4790103780 2024-04-25T02:10:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.178+0000 7f478effd700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f477806e140 0x7f47780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.178+0000 7f478effd700 1 -- 172.21.15.33:0/3210253442 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f478408da60 con 0x7f4790103780 2024-04-25T02:10:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.178+0000 7f4796127700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f477806e140 0x7f47780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:18.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.178+0000 7f4796127700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f477806e140 0x7f47780705f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4788004560 tx=0x7f4788009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:18.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.183+0000 7f478effd700 1 -- 172.21.15.33:0/3210253442 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4784056d40 con 0x7f4790103780 2024-04-25T02:10:18.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.401+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4790109f50 con 0x7f477806e140 2024-04-25T02:10:18.409 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:18 smithi033 ceph-mon[111306]: pgmap v928: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:18.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.411+0000 7f478effd700 1 -- 172.21.15.33:0/3210253442 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4790109f50 con 0x7f477806e140 2024-04-25T02:10:18.419 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:10:18.419 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (31m) 9m ago 42m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:10:18.419 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (28m) 9m ago 42m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:10:18.419 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (28m) 9m ago 41m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:10:18.419 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (31m) 9m ago 42m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (24m) 9m ago 37m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (24m) 9m ago 37m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (24m) 9m ago 37m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (9m) 9m ago 37m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (32m) 9m ago 43m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (32m) 9m ago 41m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (29m) 9m ago 44m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (29m) 9m ago 41m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (32m) 9m ago 42m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (32m) 9m ago 41m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (28m) 9m ago 40m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (27m) 9m ago 40m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (27m) 9m ago 40m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (26m) 9m ago 39m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:10:18.420 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (25m) 9m ago 39m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:10:18.421 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (25m) 9m ago 38m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:10:18.421 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (31m) 9m ago 42m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:10:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f477806e140 msgr2=0x7f47780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f477806e140 0x7f47780705f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f4788004560 tx=0x7f4788009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790103780 msgr2=0x7f47900707a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790103780 0x7f47900707a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f478400b790 tx=0x7f4784009220 comp rx=0 tx=0).stop 2024-04-25T02:10:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 shutdown_connections 2024-04-25T02:10:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f477806e140 0x7f47780705f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4790102de0 0x7f4790070260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 --2- 172.21.15.33:0/3210253442 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4790103780 0x7f47900707a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 >> 172.21.15.33:0/3210253442 conn(0x7f47900f8990 msgr2=0x7f47900fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 shutdown_connections 2024-04-25T02:10:18.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.414+0000 7f479838b700 1 -- 172.21.15.33:0/3210253442 wait complete. 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.517+0000 7fdaa11d4700 1 -- 172.21.15.33:0/1772355922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c102de0 msgr2=0x7fda9c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.517+0000 7fdaa11d4700 1 --2- 172.21.15.33:0/1772355922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c102de0 0x7fda9c1031b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fda8c004940 tx=0x7fda8c009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.517+0000 7fdaa11d4700 1 -- 172.21.15.33:0/1772355922 shutdown_connections 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.517+0000 7fdaa11d4700 1 --2- 172.21.15.33:0/1772355922 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c103780 0x7fda9c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.517+0000 7fdaa11d4700 1 --2- 172.21.15.33:0/1772355922 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c102de0 0x7fda9c1031b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.517+0000 7fdaa11d4700 1 -- 172.21.15.33:0/1772355922 >> 172.21.15.33:0/1772355922 conn(0x7fda9c0f8990 msgr2=0x7fda9c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.518+0000 7fdaa11d4700 1 -- 172.21.15.33:0/1772355922 shutdown_connections 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.518+0000 7fdaa11d4700 1 -- 172.21.15.33:0/1772355922 wait complete. 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.518+0000 7fdaa11d4700 1 Processor -- start 2024-04-25T02:10:18.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.518+0000 7fdaa11d4700 1 -- start start 2024-04-25T02:10:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.518+0000 7fdaa11d4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c102de0 0x7fda9c113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.518+0000 7fdaa11d4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c103780 0x7fda9c113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.518+0000 7fdaa11d4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c114500 con 0x7fda9c103780 2024-04-25T02:10:18.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.518+0000 7fdaa11d4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c10db50 con 0x7fda9c102de0 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c103780 0x7fda9c113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c103780 0x7fda9c113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44636/0 (socket says 172.21.15.33:44636) 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda9a59c700 1 -- 172.21.15.33:0/473024537 learned_addr learned my addr 172.21.15.33:0/473024537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda9ad9d700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c102de0 0x7fda9c113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda9a59c700 1 -- 172.21.15.33:0/473024537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c102de0 msgr2=0x7fda9c113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda9a59c700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c102de0 0x7fda9c113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda9a59c700 1 -- 172.21.15.33:0/473024537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda8c005040 con 0x7fda9c103780 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda9ad9d700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c102de0 0x7fda9c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda9a59c700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c103780 0x7fda9c113eb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fda90002a50 tx=0x7fda90004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda837fe700 1 -- 172.21.15.33:0/473024537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda90007cc0 con 0x7fda9c103780 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c10de30 con 0x7fda9c103780 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda837fe700 1 -- 172.21.15.33:0/473024537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda9000f040 con 0x7fda9c103780 2024-04-25T02:10:18.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fda837fe700 1 -- 172.21.15.33:0/473024537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9000da40 con 0x7fda9c103780 2024-04-25T02:10:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.519+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c10e3f0 con 0x7fda9c103780 2024-04-25T02:10:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.520+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c1000f0 con 0x7fda9c103780 2024-04-25T02:10:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.521+0000 7fda837fe700 1 -- 172.21.15.33:0/473024537 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fda900097c0 con 0x7fda9c103780 2024-04-25T02:10:18.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.521+0000 7fda837fe700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fda8406e140 0x7fda840705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:18.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.521+0000 7fda837fe700 1 -- 172.21.15.33:0/473024537 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fda9008d950 con 0x7fda9c103780 2024-04-25T02:10:18.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.521+0000 7fda9ad9d700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fda8406e140 0x7fda840705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:18.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.522+0000 7fda9ad9d700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fda8406e140 0x7fda840705f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fda8c004560 tx=0x7fda8c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:18.533 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.526+0000 7fda837fe700 1 -- 172.21.15.33:0/473024537 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fda9005a800 con 0x7fda9c103780 2024-04-25T02:10:18.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:18 smithi138 ceph-mon[81304]: pgmap v928: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.822+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fda9c0fe530 con 0x7fda9c103780 2024-04-25T02:10:18.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.823+0000 7fda837fe700 1 -- 172.21.15.33:0/473024537 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fda9005a390 con 0x7fda9c103780 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:10:18.831 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:10:18.832 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:10:18.832 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:10:18.832 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:10:18.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fda8406e140 msgr2=0x7fda840705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fda8406e140 0x7fda840705f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fda8c004560 tx=0x7fda8c009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:18.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c103780 msgr2=0x7fda9c113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c103780 0x7fda9c113eb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fda90002a50 tx=0x7fda90004600 comp rx=0 tx=0).stop 2024-04-25T02:10:18.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 shutdown_connections 2024-04-25T02:10:18.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fda8406e140 0x7fda840705f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda9c102de0 0x7fda9c113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 --2- 172.21.15.33:0/473024537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda9c103780 0x7fda9c113eb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 >> 172.21.15.33:0/473024537 conn(0x7fda9c0f8990 msgr2=0x7fda9c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:18.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 shutdown_connections 2024-04-25T02:10:18.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.825+0000 7fdaa11d4700 1 -- 172.21.15.33:0/473024537 wait complete. 2024-04-25T02:10:18.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.929+0000 7f57c93ad700 1 -- 172.21.15.33:0/2174766391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 msgr2=0x7f57c40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.929+0000 7f57c93ad700 1 --2- 172.21.15.33:0/2174766391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 0x7f57c40fc1b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f57ac004940 tx=0x7f57ac009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:18.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.929+0000 7f57c93ad700 1 -- 172.21.15.33:0/2174766391 shutdown_connections 2024-04-25T02:10:18.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.929+0000 7f57c93ad700 1 --2- 172.21.15.33:0/2174766391 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 0x7f57c40fc1b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.929+0000 7f57c93ad700 1 --2- 172.21.15.33:0/2174766391 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c4101d90 0x7f57c4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.929+0000 7f57c93ad700 1 -- 172.21.15.33:0/2174766391 >> 172.21.15.33:0/2174766391 conn(0x7f57c4071d00 msgr2=0x7f57c4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:18.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.929+0000 7f57c93ad700 1 -- 172.21.15.33:0/2174766391 shutdown_connections 2024-04-25T02:10:18.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.929+0000 7f57c93ad700 1 -- 172.21.15.33:0/2174766391 wait complete. 2024-04-25T02:10:18.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.929+0000 7f57c93ad700 1 Processor -- start 2024-04-25T02:10:18.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c93ad700 1 -- start start 2024-04-25T02:10:18.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c93ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 0x7f57c4113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:18.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c93ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c4101d90 0x7f57c41140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:18.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c93ad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c41147a0 con 0x7f57c40fbd40 2024-04-25T02:10:18.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c93ad700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c410db00 con 0x7f57c4101d90 2024-04-25T02:10:18.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 0x7f57c4113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:18.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 0x7f57c4113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44656/0 (socket says 172.21.15.33:44656) 2024-04-25T02:10:18.939 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c2ffd700 1 -- 172.21.15.33:0/127289884 learned_addr learned my addr 172.21.15.33:0/127289884 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c27fc700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c4101d90 0x7f57c41140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c2ffd700 1 -- 172.21.15.33:0/127289884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c4101d90 msgr2=0x7f57c41140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c2ffd700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c4101d90 0x7f57c41140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c2ffd700 1 -- 172.21.15.33:0/127289884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac005040 con 0x7f57c40fbd40 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.930+0000 7f57c2ffd700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 0x7f57c4113b80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f57b400bbb0 tx=0x7f57b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.931+0000 7f57bb7fe700 1 -- 172.21.15.33:0/127289884 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57b40092e0 con 0x7f57c40fbd40 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.931+0000 7f57bb7fe700 1 -- 172.21.15.33:0/127289884 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57b400cb20 con 0x7f57c40fbd40 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.931+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c410dde0 con 0x7f57c40fbd40 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.931+0000 7f57bb7fe700 1 -- 172.21.15.33:0/127289884 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57b4015d70 con 0x7f57c40fbd40 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.931+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c410e370 con 0x7f57c40fbd40 2024-04-25T02:10:18.940 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.932+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c40fd430 con 0x7f57c40fbd40 2024-04-25T02:10:18.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.932+0000 7f57bb7fe700 1 -- 172.21.15.33:0/127289884 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f57b4012070 con 0x7f57c40fbd40 2024-04-25T02:10:18.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.933+0000 7f57bb7fe700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f57b006e140 0x7f57b00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:18.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.933+0000 7f57bb7fe700 1 -- 172.21.15.33:0/127289884 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f57b408d880 con 0x7f57c40fbd40 2024-04-25T02:10:18.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.933+0000 7f57c27fc700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f57b006e140 0x7f57b00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:18.941 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.933+0000 7f57c27fc700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f57b006e140 0x7f57b00705f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f57c410f4b0 tx=0x7f57ac009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:18.945 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:18.937+0000 7f57bb7fe700 1 -- 172.21.15.33:0/127289884 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f57b405a730 con 0x7f57c40fbd40 2024-04-25T02:10:19.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.194+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f57c404bed0 con 0x7f57c40fbd40 2024-04-25T02:10:19.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.194+0000 7f57bb7fe700 1 -- 172.21.15.33:0/127289884 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f57b405a2c0 con 0x7f57c40fbd40 2024-04-25T02:10:19.202 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:10:19.202 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:10:19.203 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:10:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:10:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:10:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:10:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:10:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:10:19.205 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:10:19.206 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:10:19.207 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:10:19.207 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:10:19.207 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:10:19.207 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:10:19.207 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:10:19.207 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:10:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f57b006e140 msgr2=0x7f57b00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f57b006e140 0x7f57b00705f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f57c410f4b0 tx=0x7f57ac009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 msgr2=0x7f57c4113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 0x7f57c4113b80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f57b400bbb0 tx=0x7f57b4009840 comp rx=0 tx=0).stop 2024-04-25T02:10:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 shutdown_connections 2024-04-25T02:10:19.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f57b006e140 0x7f57b00705f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f57c40fbd40 0x7f57c4113b80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 --2- 172.21.15.33:0/127289884 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f57c4101d90 0x7f57c41140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 >> 172.21.15.33:0/127289884 conn(0x7f57c4071d00 msgr2=0x7f57c4065620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 shutdown_connections 2024-04-25T02:10:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.197+0000 7f57c93ad700 1 -- 172.21.15.33:0/127289884 wait complete. 2024-04-25T02:10:19.209 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:10:19.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.302+0000 7f2d18e17700 1 -- 172.21.15.33:0/3686861143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fe800 msgr2=0x7f2d14106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.303+0000 7f2d18e17700 1 --2- 172.21.15.33:0/3686861143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fe800 0x7f2d14106e40 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc004940 tx=0x7f2cfc009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:19.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.303+0000 7f2d18e17700 1 -- 172.21.15.33:0/3686861143 shutdown_connections 2024-04-25T02:10:19.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.303+0000 7f2d18e17700 1 --2- 172.21.15.33:0/3686861143 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fe800 0x7f2d14106e40 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.303+0000 7f2d18e17700 1 --2- 172.21.15.33:0/3686861143 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d140fde60 0x7f2d140fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.303+0000 7f2d18e17700 1 -- 172.21.15.33:0/3686861143 >> 172.21.15.33:0/3686861143 conn(0x7f2d140f7870 msgr2=0x7f2d140f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.303+0000 7f2d18e17700 1 -- 172.21.15.33:0/3686861143 shutdown_connections 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.303+0000 7f2d18e17700 1 -- 172.21.15.33:0/3686861143 wait complete. 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.303+0000 7f2d18e17700 1 Processor -- start 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.303+0000 7f2d18e17700 1 -- start start 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d18e17700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fde60 0x7f2d14113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d18e17700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d140fe800 0x7f2d14113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d18e17700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d141145a0 con 0x7f2d140fde60 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d18e17700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1410db10 con 0x7f2d140fe800 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d1259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fde60 0x7f2d14113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d1259c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fde60 0x7f2d14113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44678/0 (socket says 172.21.15.33:44678) 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d1259c700 1 -- 172.21.15.33:0/1724144644 learned_addr learned my addr 172.21.15.33:0/1724144644 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d09d9b700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d140fe800 0x7f2d14113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:19.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d1259c700 1 -- 172.21.15.33:0/1724144644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d140fe800 msgr2=0x7f2d14113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d1259c700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d140fe800 0x7f2d14113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d1259c700 1 -- 172.21.15.33:0/1724144644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc005040 con 0x7f2d140fde60 2024-04-25T02:10:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d1259c700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fde60 0x7f2d14113980 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b790 tx=0x7f2d04009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d0b7fe700 1 -- 172.21.15.33:0/1724144644 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0400f770 con 0x7f2d140fde60 2024-04-25T02:10:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1410ddf0 con 0x7f2d140fde60 2024-04-25T02:10:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d0b7fe700 1 -- 172.21.15.33:0/1724144644 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d0400fda0 con 0x7f2d140fde60 2024-04-25T02:10:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d0b7fe700 1 -- 172.21.15.33:0/1724144644 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0400d310 con 0x7f2d140fde60 2024-04-25T02:10:19.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.304+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1410e330 con 0x7f2d140fde60 2024-04-25T02:10:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.305+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d14066ad0 con 0x7f2d140fde60 2024-04-25T02:10:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.306+0000 7f2d0b7fe700 1 -- 172.21.15.33:0/1724144644 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2d04010420 con 0x7f2d140fde60 2024-04-25T02:10:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.306+0000 7f2d0b7fe700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d0006e140 0x7f2d000705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.307+0000 7f2d0b7fe700 1 -- 172.21.15.33:0/1724144644 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2d0408db40 con 0x7f2d140fde60 2024-04-25T02:10:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.307+0000 7f2d09d9b700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d0006e140 0x7f2d000705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:19.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.307+0000 7f2d09d9b700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d0006e140 0x7f2d000705f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2d1410ecd0 tx=0x7f2cfc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:19.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.311+0000 7f2d0b7fe700 1 -- 172.21.15.33:0/1724144644 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2d0405aa70 con 0x7f2d140fde60 2024-04-25T02:10:19.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:19 smithi033 ceph-mon[111306]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:19.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:19 smithi033 ceph-mon[111306]: from='client.35696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:19.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:19 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/473024537' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:10:19.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:19 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/127289884' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:10:19.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.541+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d14109f10 con 0x7f2d0006e140 2024-04-25T02:10:19.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.542+0000 7f2d0b7fe700 1 -- 172.21.15.33:0/1724144644 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2d14109f10 con 0x7f2d0006e140 2024-04-25T02:10:19.550 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:10:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:10:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:10:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:10:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:10:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:10:19.550 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:10:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:10:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:10:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:10:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:10:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:10:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:10:19.551 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:10:19.551 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:10:19.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d0006e140 msgr2=0x7f2d000705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d0006e140 0x7f2d000705f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2d1410ecd0 tx=0x7f2cfc009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:19.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fde60 msgr2=0x7f2d14113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fde60 0x7f2d14113980 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2d0400b790 tx=0x7f2d04009250 comp rx=0 tx=0).stop 2024-04-25T02:10:19.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 shutdown_connections 2024-04-25T02:10:19.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d0006e140 0x7f2d000705f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d140fde60 0x7f2d14113980 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 --2- 172.21.15.33:0/1724144644 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d140fe800 0x7f2d14113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.553 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 >> 172.21.15.33:0/1724144644 conn(0x7f2d140f7870 msgr2=0x7f2d14102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:19.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 shutdown_connections 2024-04-25T02:10:19.554 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.545+0000 7f2d18e17700 1 -- 172.21.15.33:0/1724144644 wait complete. 2024-04-25T02:10:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.649+0000 7fee9b59e700 1 -- 172.21.15.33:0/3372826862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 msgr2=0x7fee9c0fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.649+0000 7fee9b59e700 1 --2- 172.21.15.33:0/3372826862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 0x7fee9c0fbfd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fee8c004940 tx=0x7fee8c009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.649+0000 7fee9b59e700 1 -- 172.21.15.33:0/3372826862 shutdown_connections 2024-04-25T02:10:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.649+0000 7fee9b59e700 1 --2- 172.21.15.33:0/3372826862 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 0x7fee9c0fbfd0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.649+0000 7fee9b59e700 1 --2- 172.21.15.33:0/3372826862 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c100a90 0x7fee9c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.649+0000 7fee9b59e700 1 -- 172.21.15.33:0/3372826862 >> 172.21.15.33:0/3372826862 conn(0x7fee9c071b40 msgr2=0x7fee9c071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:19.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.649+0000 7fee9b59e700 1 -- 172.21.15.33:0/3372826862 shutdown_connections 2024-04-25T02:10:19.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.649+0000 7fee9b59e700 1 -- 172.21.15.33:0/3372826862 wait complete. 2024-04-25T02:10:19.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9b59e700 1 Processor -- start 2024-04-25T02:10:19.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9b59e700 1 -- start start 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9b59e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 0x7fee9c10e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9b59e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c100a90 0x7fee9c10eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9b59e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c10f530 con 0x7fee9c0fbb60 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9b59e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c10f670 con 0x7fee9c100a90 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 0x7fee9c10e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9a59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 0x7fee9c10e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44694/0 (socket says 172.21.15.33:44694) 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9a59c700 1 -- 172.21.15.33:0/3130847191 learned_addr learned my addr 172.21.15.33:0/3130847191 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee99d9b700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c100a90 0x7fee9c10eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9a59c700 1 -- 172.21.15.33:0/3130847191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c100a90 msgr2=0x7fee9c10eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9a59c700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c100a90 0x7fee9c10eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.650+0000 7fee9a59c700 1 -- 172.21.15.33:0/3130847191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8c005040 con 0x7fee9c0fbb60 2024-04-25T02:10:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.651+0000 7fee9a59c700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 0x7fee9c10e9a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fee90002a50 tx=0x7fee900046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.651+0000 7fee8affd700 1 -- 172.21.15.33:0/3130847191 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee90007cc0 con 0x7fee9c0fbb60 2024-04-25T02:10:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.651+0000 7fee8affd700 1 -- 172.21.15.33:0/3130847191 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fee9000f040 con 0x7fee9c0fbb60 2024-04-25T02:10:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.651+0000 7fee8affd700 1 -- 172.21.15.33:0/3130847191 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee90009640 con 0x7fee9c0fbb60 2024-04-25T02:10:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.651+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c117a10 con 0x7fee9c0fbb60 2024-04-25T02:10:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.651+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c10c810 con 0x7fee9c0fbb60 2024-04-25T02:10:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.652+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9c0fd200 con 0x7fee9c0fbb60 2024-04-25T02:10:19.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.652+0000 7fee8affd700 1 -- 172.21.15.33:0/3130847191 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fee900097a0 con 0x7fee9c0fbb60 2024-04-25T02:10:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.653+0000 7fee8affd700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fee84072550 0x7fee84074a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.653+0000 7fee8affd700 1 -- 172.21.15.33:0/3130847191 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fee9008d940 con 0x7fee9c0fbb60 2024-04-25T02:10:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.653+0000 7fee99d9b700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fee84072550 0x7fee84074a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:19.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.653+0000 7fee99d9b700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fee84072550 0x7fee84074a00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00b3b0 tx=0x7fee8c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:19.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.657+0000 7fee8affd700 1 -- 172.21.15.33:0/3130847191 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fee90056cc0 con 0x7fee9c0fbb60 2024-04-25T02:10:19.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:19 smithi138 ceph-mon[81304]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:19.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:19 smithi138 ceph-mon[81304]: from='client.35696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:19.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:19 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/473024537' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:10:19.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:19 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/127289884' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:10:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.950+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fee9c04bf60 con 0x7fee9c0fbb60 2024-04-25T02:10:19.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.951+0000 7fee8affd700 1 -- 172.21.15.33:0/3130847191 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fee9005a380 con 0x7fee9c0fbb60 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:10:19.959 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:10:19.960 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:10:19.960 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:10:19.960 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:10:19.960 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:10:19.960 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:19.960 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:10:19.960 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:10:19.960 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:10:19.960 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:10:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fee84072550 msgr2=0x7fee84074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fee84072550 0x7fee84074a00 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00b3b0 tx=0x7fee8c009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 msgr2=0x7fee9c10e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 0x7fee9c10e9a0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fee90002a50 tx=0x7fee900046b0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 shutdown_connections 2024-04-25T02:10:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fee84072550 0x7fee84074a00 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fee9c0fbb60 0x7fee9c10e9a0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 --2- 172.21.15.33:0/3130847191 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fee9c100a90 0x7fee9c10eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:19.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 >> 172.21.15.33:0/3130847191 conn(0x7fee9c071b40 msgr2=0x7fee9c0f9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:19.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 shutdown_connections 2024-04-25T02:10:19.962 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:19.953+0000 7fee9b59e700 1 -- 172.21.15.33:0/3130847191 wait complete. 2024-04-25T02:10:20.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:20 smithi033 ceph-mon[111306]: from='client.35700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:20.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:20 smithi033 ceph-mon[111306]: pgmap v929: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:20.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:20 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3130847191' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:10:20.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:20 smithi138 ceph-mon[81304]: from='client.35700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:20.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:20 smithi138 ceph-mon[81304]: pgmap v929: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:20.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:20 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3130847191' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:10:21.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:21 smithi033 ceph-mon[111306]: from='client.35712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:21.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:21 smithi138 ceph-mon[81304]: from='client.35712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:22.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:22 smithi033 ceph-mon[111306]: pgmap v930: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:22.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:22 smithi138 ceph-mon[81304]: pgmap v930: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:24.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:24 smithi033 ceph-mon[111306]: pgmap v931: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:24.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:24 smithi138 ceph-mon[81304]: pgmap v931: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:25.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:10:25.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:10:26.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:26 smithi033 ceph-mon[111306]: pgmap v932: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:26.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:26 smithi138 ceph-mon[81304]: pgmap v932: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:28.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:28 smithi033 ceph-mon[111306]: pgmap v933: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:28.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:28 smithi138 ceph-mon[81304]: pgmap v933: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:30.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:30 smithi033 ceph-mon[111306]: pgmap v934: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:30.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:30 smithi138 ceph-mon[81304]: pgmap v934: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:32.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:32 smithi033 ceph-mon[111306]: pgmap v935: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:32.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:32 smithi138 ceph-mon[81304]: pgmap v935: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:34.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:34 smithi033 ceph-mon[111306]: pgmap v936: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:34.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:34 smithi138 ceph-mon[81304]: pgmap v936: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:36.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:36 smithi033 ceph-mon[111306]: pgmap v937: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:36.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:36 smithi138 ceph-mon[81304]: pgmap v937: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:38.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:38 smithi033 ceph-mon[111306]: pgmap v938: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:38.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:38 smithi138 ceph-mon[81304]: pgmap v938: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:40.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:40 smithi033 ceph-mon[111306]: pgmap v939: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:40.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:10:40.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:40 smithi138 ceph-mon[81304]: pgmap v939: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:40.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:10:42.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:42 smithi033 ceph-mon[111306]: pgmap v940: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:42.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:42 smithi138 ceph-mon[81304]: pgmap v940: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:44.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:44 smithi033 ceph-mon[111306]: pgmap v941: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:44.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:44 smithi138 ceph-mon[81304]: pgmap v941: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:46.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:46 smithi033 ceph-mon[111306]: pgmap v942: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:46.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:46 smithi138 ceph-mon[81304]: pgmap v942: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:48.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:48 smithi033 ceph-mon[111306]: pgmap v943: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:48.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:48 smithi138 ceph-mon[81304]: pgmap v943: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:50.082 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.073+0000 7f7b242be700 1 -- 172.21.15.33:0/192938959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 msgr2=0x7f7b1c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.073+0000 7f7b242be700 1 --2- 172.21.15.33:0/192938959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 0x7f7b1c0fcdf0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f7b18004940 tx=0x7f7b18009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.073+0000 7f7b242be700 1 -- 172.21.15.33:0/192938959 shutdown_connections 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.073+0000 7f7b242be700 1 --2- 172.21.15.33:0/192938959 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 0x7f7b1c0fcdf0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.073+0000 7f7b242be700 1 --2- 172.21.15.33:0/192938959 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b1c102de0 0x7f7b1c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.073+0000 7f7b242be700 1 -- 172.21.15.33:0/192938959 >> 172.21.15.33:0/192938959 conn(0x7f7b1c0f8990 msgr2=0x7f7b1c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.073+0000 7f7b242be700 1 -- 172.21.15.33:0/192938959 shutdown_connections 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.073+0000 7f7b242be700 1 -- 172.21.15.33:0/192938959 wait complete. 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b242be700 1 Processor -- start 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b242be700 1 -- start start 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b242be700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b1c102de0 0x7f7b1c113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b242be700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 0x7f7b1c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:50.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b242be700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c1144e0 con 0x7f7b1c103780 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b242be700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b1c10dae0 con 0x7f7b1c102de0 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b21859700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 0x7f7b1c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b21859700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 0x7f7b1c113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53542/0 (socket says 172.21.15.33:53542) 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b21859700 1 -- 172.21.15.33:0/3246509083 learned_addr learned my addr 172.21.15.33:0/3246509083 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b2205a700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b1c102de0 0x7f7b1c113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b21859700 1 -- 172.21.15.33:0/3246509083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b1c102de0 msgr2=0x7f7b1c113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b21859700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b1c102de0 0x7f7b1c113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.074+0000 7f7b21859700 1 -- 172.21.15.33:0/3246509083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b18005040 con 0x7f7b1c103780 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.075+0000 7f7b21859700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 0x7f7b1c113e90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009100 tx=0x7f7b18009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.075+0000 7f7b12ffd700 1 -- 172.21.15.33:0/3246509083 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1800e040 con 0x7f7b1c103780 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.075+0000 7f7b12ffd700 1 -- 172.21.15.33:0/3246509083 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b18009af0 con 0x7f7b1c103780 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.075+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b1c10dd60 con 0x7f7b1c103780 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.075+0000 7f7b12ffd700 1 -- 172.21.15.33:0/3246509083 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1801c3f0 con 0x7f7b1c103780 2024-04-25T02:10:50.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.075+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b1c10e320 con 0x7f7b1c103780 2024-04-25T02:10:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.076+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b1c0fe530 con 0x7f7b1c103780 2024-04-25T02:10:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.076+0000 7f7b12ffd700 1 -- 172.21.15.33:0/3246509083 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7b1800baf0 con 0x7f7b1c103780 2024-04-25T02:10:50.085 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.077+0000 7f7b12ffd700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7b0806e140 0x7f7b080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.077+0000 7f7b12ffd700 1 -- 172.21.15.33:0/3246509083 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7b1808ed60 con 0x7f7b1c103780 2024-04-25T02:10:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.077+0000 7f7b2205a700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7b0806e140 0x7f7b080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:50.086 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.077+0000 7f7b2205a700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7b0806e140 0x7f7b080705f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c007fd0 tx=0x7f7b0c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:50.089 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.082+0000 7f7b12ffd700 1 -- 172.21.15.33:0/3246509083 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7b1805bc90 con 0x7f7b1c103780 2024-04-25T02:10:50.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.310+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b1c109ec0 con 0x7f7b0806e140 2024-04-25T02:10:50.319 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.312+0000 7f7b12ffd700 1 -- 172.21.15.33:0/3246509083 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7b1c109ec0 con 0x7f7b0806e140 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7b0806e140 msgr2=0x7f7b080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7b0806e140 0x7f7b080705f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c007fd0 tx=0x7f7b0c009040 comp rx=0 tx=0).stop 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 msgr2=0x7f7b1c113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 0x7f7b1c113e90 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009100 tx=0x7f7b18009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 shutdown_connections 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7b0806e140 0x7f7b080705f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7b1c102de0 0x7f7b1c113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 --2- 172.21.15.33:0/3246509083 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7b1c103780 0x7f7b1c113e90 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 >> 172.21.15.33:0/3246509083 conn(0x7f7b1c0f8990 msgr2=0x7f7b1c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 shutdown_connections 2024-04-25T02:10:50.322 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.314+0000 7f7b242be700 1 -- 172.21.15.33:0/3246509083 wait complete. 2024-04-25T02:10:50.332 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:10:50.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.419+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3635768174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd4102e20 msgr2=0x7f3fd41031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.419+0000 7f3fd8e21700 1 --2- 172.21.15.33:0/3635768174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd4102e20 0x7f3fd41031f0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4004940 tx=0x7f3fc4009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:50.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3635768174 shutdown_connections 2024-04-25T02:10:50.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 --2- 172.21.15.33:0/3635768174 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd41037c0 0x7f3fd40fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 --2- 172.21.15.33:0/3635768174 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd4102e20 0x7f3fd41031f0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3635768174 >> 172.21.15.33:0/3635768174 conn(0x7f3fd4072380 msgr2=0x7f3fd4072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:50.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3635768174 shutdown_connections 2024-04-25T02:10:50.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3635768174 wait complete. 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 Processor -- start 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 -- start start 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd4102e20 0x7f3fd41139f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd41037c0 0x7f3fd4113f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd4114580 con 0x7f3fd41037c0 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.420+0000 7f3fd8e21700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd410db80 con 0x7f3fd4102e20 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fd1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd41037c0 0x7f3fd4113f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fd1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd41037c0 0x7f3fd4113f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53564/0 (socket says 172.21.15.33:53564) 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fd1d9b700 1 -- 172.21.15.33:0/3025356014 learned_addr learned my addr 172.21.15.33:0/3025356014 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fd1d9b700 1 -- 172.21.15.33:0/3025356014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd4102e20 msgr2=0x7f3fd41139f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fd1d9b700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd4102e20 0x7f3fd41139f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fd1d9b700 1 -- 172.21.15.33:0/3025356014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc4005040 con 0x7f3fd41037c0 2024-04-25T02:10:50.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fd1d9b700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd41037c0 0x7f3fd4113f30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800c890 tx=0x7f3fc80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:50.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fc2ffd700 1 -- 172.21.15.33:0/3025356014 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fc800a7d0 con 0x7f3fd41037c0 2024-04-25T02:10:50.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fc2ffd700 1 -- 172.21.15.33:0/3025356014 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fc800ae00 con 0x7f3fd41037c0 2024-04-25T02:10:50.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd410de60 con 0x7f3fd41037c0 2024-04-25T02:10:50.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fc2ffd700 1 -- 172.21.15.33:0/3025356014 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fc80113a0 con 0x7f3fd41037c0 2024-04-25T02:10:50.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.421+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd410e3a0 con 0x7f3fd41037c0 2024-04-25T02:10:50.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.422+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd4100160 con 0x7f3fd41037c0 2024-04-25T02:10:50.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.423+0000 7f3fc2ffd700 1 -- 172.21.15.33:0/3025356014 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3fc8012400 con 0x7f3fd41037c0 2024-04-25T02:10:50.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.423+0000 7f3fc2ffd700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fbc06e140 0x7f3fbc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:50.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.423+0000 7f3fc2ffd700 1 -- 172.21.15.33:0/3025356014 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3fc808db20 con 0x7f3fd41037c0 2024-04-25T02:10:50.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.423+0000 7f3fd259c700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fbc06e140 0x7f3fbc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:50.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.424+0000 7f3fd259c700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fbc06e140 0x7f3fbc0705f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4004f90 tx=0x7f3fc4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:50.435 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.428+0000 7f3fc2ffd700 1 -- 172.21.15.33:0/3025356014 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3fc805aaf0 con 0x7f3fd41037c0 2024-04-25T02:10:50.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.652+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fd410efd0 con 0x7f3fbc06e140 2024-04-25T02:10:50.661 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:50 smithi033 ceph-mon[111306]: pgmap v944: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:50.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.654+0000 7f3fc2ffd700 1 -- 172.21.15.33:0/3025356014 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3fd410efd0 con 0x7f3fbc06e140 2024-04-25T02:10:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fbc06e140 msgr2=0x7f3fbc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fbc06e140 0x7f3fbc0705f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4004f90 tx=0x7f3fc4009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd41037c0 msgr2=0x7f3fd4113f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd41037c0 0x7f3fd4113f30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800c890 tx=0x7f3fc80044c0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.664 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 shutdown_connections 2024-04-25T02:10:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fbc06e140 0x7f3fbc0705f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fd4102e20 0x7f3fd41139f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 --2- 172.21.15.33:0/3025356014 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fd41037c0 0x7f3fd4113f30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 >> 172.21.15.33:0/3025356014 conn(0x7f3fd4072380 msgr2=0x7f3fd40facc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 shutdown_connections 2024-04-25T02:10:50.665 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.656+0000 7f3fd8e21700 1 -- 172.21.15.33:0/3025356014 wait complete. 2024-04-25T02:10:50.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.760+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2663044899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 msgr2=0x7f50f0102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.760+0000 7f50f4cfd700 1 --2- 172.21.15.33:0/2663044899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 0x7f50f0102160 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f50e0004940 tx=0x7f50e0009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.760+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2663044899 shutdown_connections 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.760+0000 7f50f4cfd700 1 --2- 172.21.15.33:0/2663044899 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f00fbd40 0x7f50f00fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.760+0000 7f50f4cfd700 1 --2- 172.21.15.33:0/2663044899 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 0x7f50f0102160 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.760+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2663044899 >> 172.21.15.33:0/2663044899 conn(0x7f50f0071d00 msgr2=0x7f50f0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.760+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2663044899 shutdown_connections 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.760+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2663044899 wait complete. 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.760+0000 7f50f4cfd700 1 Processor -- start 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50f4cfd700 1 -- start start 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50f4cfd700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f00fbd40 0x7f50f0113b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50f4cfd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 0x7f50f0114050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50f4cfd700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01146a0 con 0x7f50f0101d90 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50f4cfd700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f010da90 con 0x7f50f00fbd40 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50edd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 0x7f50f0114050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50edd9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 0x7f50f0114050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53588/0 (socket says 172.21.15.33:53588) 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50edd9b700 1 -- 172.21.15.33:0/2328644494 learned_addr learned my addr 172.21.15.33:0/2328644494 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50ee59c700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f00fbd40 0x7f50f0113b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50edd9b700 1 -- 172.21.15.33:0/2328644494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f00fbd40 msgr2=0x7f50f0113b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50edd9b700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f00fbd40 0x7f50f0113b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50edd9b700 1 -- 172.21.15.33:0/2328644494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e0005040 con 0x7f50f0101d90 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.761+0000 7f50edd9b700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 0x7f50f0114050 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f50d8002a80 tx=0x7f50d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.762+0000 7f50e6ffd700 1 -- 172.21.15.33:0/2328644494 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50d8007cc0 con 0x7f50f0101d90 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.762+0000 7f50e6ffd700 1 -- 172.21.15.33:0/2328644494 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50d800f040 con 0x7f50f0101d90 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.762+0000 7f50e6ffd700 1 -- 172.21.15.33:0/2328644494 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50d800dac0 con 0x7f50f0101d90 2024-04-25T02:10:50.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.762+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f010dd70 con 0x7f50f0101d90 2024-04-25T02:10:50.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.762+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f010e330 con 0x7f50f0101d90 2024-04-25T02:10:50.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.763+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f00fd430 con 0x7f50f0101d90 2024-04-25T02:10:50.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.763+0000 7f50e6ffd700 1 -- 172.21.15.33:0/2328644494 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f50d800dc20 con 0x7f50f0101d90 2024-04-25T02:10:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.764+0000 7f50e6ffd700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f50dc06e140 0x7f50dc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.764+0000 7f50e6ffd700 1 -- 172.21.15.33:0/2328644494 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f50d808d670 con 0x7f50f0101d90 2024-04-25T02:10:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.764+0000 7f50ee59c700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f50dc06e140 0x7f50dc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.764+0000 7f50ee59c700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f50dc06e140 0x7f50dc0705f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f50e0004890 tx=0x7f50e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:50.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.768+0000 7f50e6ffd700 1 -- 172.21.15.33:0/2328644494 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f50d805a5a0 con 0x7f50f0101d90 2024-04-25T02:10:50.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:50 smithi138 ceph-mon[81304]: pgmap v944: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:50.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.984+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f50f0108df0 con 0x7f50dc06e140 2024-04-25T02:10:51.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.996+0000 7f50e6ffd700 1 -- 172.21.15.33:0/2328644494 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f50f0108df0 con 0x7f50dc06e140 2024-04-25T02:10:51.004 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:10:51.004 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (31m) 9m ago 43m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:10:51.004 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (29m) 9m ago 43m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:10:51.004 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (29m) 9m ago 42m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:10:51.004 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (31m) 9m ago 42m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:10:51.004 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (25m) 9m ago 37m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:10:51.004 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (25m) 9m ago 37m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (24m) 9m ago 37m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (9m) 9m ago 37m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (33m) 9m ago 44m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (32m) 9m ago 42m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (30m) 9m ago 44m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (29m) 9m ago 42m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (32m) 9m ago 43m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (32m) 9m ago 42m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (28m) 9m ago 41m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (28m) 9m ago 41m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (27m) 9m ago 40m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (26m) 9m ago 40m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (26m) 9m ago 39m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:10:51.005 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (25m) 9m ago 39m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:10:51.006 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (32m) 9m ago 42m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:10:51.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f50dc06e140 msgr2=0x7f50dc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f50dc06e140 0x7f50dc0705f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f50e0004890 tx=0x7f50e0009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:51.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 msgr2=0x7f50f0114050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 0x7f50f0114050 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f50d8002a80 tx=0x7f50d80046b0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 shutdown_connections 2024-04-25T02:10:51.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f50dc06e140 0x7f50dc0705f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f50f00fbd40 0x7f50f0113b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 --2- 172.21.15.33:0/2328644494 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f50f0101d90 0x7f50f0114050 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 >> 172.21.15.33:0/2328644494 conn(0x7f50f0071d00 msgr2=0x7f50f00fa7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:51.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 shutdown_connections 2024-04-25T02:10:51.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:50.998+0000 7f50f4cfd700 1 -- 172.21.15.33:0/2328644494 wait complete. 2024-04-25T02:10:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.100+0000 7fbcf33d6700 1 -- 172.21.15.33:0/723668106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec103780 msgr2=0x7fbcec0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.100+0000 7fbcf33d6700 1 --2- 172.21.15.33:0/723668106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec103780 0x7fbcec0fcdf0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbce0004940 tx=0x7fbce0009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.100+0000 7fbcf33d6700 1 -- 172.21.15.33:0/723668106 shutdown_connections 2024-04-25T02:10:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.100+0000 7fbcf33d6700 1 --2- 172.21.15.33:0/723668106 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec103780 0x7fbcec0fcdf0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.100+0000 7fbcf33d6700 1 --2- 172.21.15.33:0/723668106 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec102de0 0x7fbcec1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.100+0000 7fbcf33d6700 1 -- 172.21.15.33:0/723668106 >> 172.21.15.33:0/723668106 conn(0x7fbcec0f8990 msgr2=0x7fbcec0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf33d6700 1 -- 172.21.15.33:0/723668106 shutdown_connections 2024-04-25T02:10:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf33d6700 1 -- 172.21.15.33:0/723668106 wait complete. 2024-04-25T02:10:51.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf33d6700 1 Processor -- start 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf33d6700 1 -- start start 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf33d6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec102de0 0x7fbcec070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf33d6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec103780 0x7fbcec0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf33d6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec0721b0 con 0x7fbcec102de0 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf33d6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec070ce0 con 0x7fbcec103780 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf1172700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec102de0 0x7fbcec070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf1172700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec102de0 0x7fbcec070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53618/0 (socket says 172.21.15.33:53618) 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.101+0000 7fbcf1172700 1 -- 172.21.15.33:0/1756480451 learned_addr learned my addr 172.21.15.33:0/1756480451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbcf0971700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec103780 0x7fbcec0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbcf1172700 1 -- 172.21.15.33:0/1756480451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec103780 msgr2=0x7fbcec0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbcf1172700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec103780 0x7fbcec0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbcf1172700 1 -- 172.21.15.33:0/1756480451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce0005040 con 0x7fbcec102de0 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbcf0971700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec103780 0x7fbcec0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbcf1172700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec102de0 0x7fbcec070260 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc001d80 tx=0x7fbcdc009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:51.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbce9ffb700 1 -- 172.21.15.33:0/1756480451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcdc007d10 con 0x7fbcec102de0 2024-04-25T02:10:51.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec070fc0 con 0x7fbcec102de0 2024-04-25T02:10:51.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbce9ffb700 1 -- 172.21.15.33:0/1756480451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcdc007e70 con 0x7fbcec102de0 2024-04-25T02:10:51.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbce9ffb700 1 -- 172.21.15.33:0/1756480451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcdc014640 con 0x7fbcec102de0 2024-04-25T02:10:51.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.102+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec11a140 con 0x7fbcec102de0 2024-04-25T02:10:51.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.103+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec0fe530 con 0x7fbcec102de0 2024-04-25T02:10:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.104+0000 7fbce9ffb700 1 -- 172.21.15.33:0/1756480451 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbcdc00e960 con 0x7fbcec102de0 2024-04-25T02:10:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.104+0000 7fbce9ffb700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcd406e140 0x7fbcd40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.104+0000 7fbce9ffb700 1 -- 172.21.15.33:0/1756480451 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fbcdc08d910 con 0x7fbcec102de0 2024-04-25T02:10:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.104+0000 7fbcf0971700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcd406e140 0x7fbcd40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:51.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.105+0000 7fbcf0971700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcd406e140 0x7fbcd40705f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbce000b3b0 tx=0x7fbce0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:51.116 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.109+0000 7fbce9ffb700 1 -- 172.21.15.33:0/1756480451 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbcdc056d40 con 0x7fbcec102de0 2024-04-25T02:10:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.411+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbcec11a390 con 0x7fbcec102de0 2024-04-25T02:10:51.419 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.412+0000 7fbce9ffb700 1 -- 172.21.15.33:0/1756480451 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbcdc05a400 con 0x7fbcec102de0 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:10:51.420 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:10:51.421 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:10:51.421 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:10:51.421 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:10:51.421 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcd406e140 msgr2=0x7fbcd40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcd406e140 0x7fbcd40705f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbce000b3b0 tx=0x7fbce0009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec102de0 msgr2=0x7fbcec070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec102de0 0x7fbcec070260 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc001d80 tx=0x7fbcdc009ec0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 shutdown_connections 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbcd406e140 0x7fbcd40705f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fbcec102de0 0x7fbcec070260 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 --2- 172.21.15.33:0/1756480451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fbcec103780 0x7fbcec0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 >> 172.21.15.33:0/1756480451 conn(0x7fbcec0f8990 msgr2=0x7fbcec0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 shutdown_connections 2024-04-25T02:10:51.422 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.414+0000 7fbcf33d6700 1 -- 172.21.15.33:0/1756480451 wait complete. 2024-04-25T02:10:51.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.518+0000 7f4d41471700 1 -- 172.21.15.33:0/1208701844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 msgr2=0x7f4d3c0fe210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.518+0000 7f4d41471700 1 --2- 172.21.15.33:0/1208701844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 0x7f4d3c0fe210 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c004940 tx=0x7f4d2c009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:51.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.518+0000 7f4d41471700 1 -- 172.21.15.33:0/1208701844 shutdown_connections 2024-04-25T02:10:51.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.518+0000 7f4d41471700 1 --2- 172.21.15.33:0/1208701844 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d3c0fe7e0 0x7f4d3c106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.518+0000 7f4d41471700 1 --2- 172.21.15.33:0/1208701844 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 0x7f4d3c0fe210 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.518+0000 7f4d41471700 1 -- 172.21.15.33:0/1208701844 >> 172.21.15.33:0/1208701844 conn(0x7f4d3c0f78d0 msgr2=0x7f4d3c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:51.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.518+0000 7f4d41471700 1 -- 172.21.15.33:0/1208701844 shutdown_connections 2024-04-25T02:10:51.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.518+0000 7f4d41471700 1 -- 172.21.15.33:0/1208701844 wait complete. 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.518+0000 7f4d41471700 1 Processor -- start 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d41471700 1 -- start start 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d41471700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 0x7f4d3c1138a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d41471700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d3c0fe7e0 0x7f4d3c113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d41471700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c114430 con 0x7f4d3c0fde40 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d41471700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c10da80 con 0x7f4d3c0fe7e0 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 0x7f4d3c1138a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d3affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 0x7f4d3c1138a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53632/0 (socket says 172.21.15.33:53632) 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d3affd700 1 -- 172.21.15.33:0/3736017805 learned_addr learned my addr 172.21.15.33:0/3736017805 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d3a7fc700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d3c0fe7e0 0x7f4d3c113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d3affd700 1 -- 172.21.15.33:0/3736017805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d3c0fe7e0 msgr2=0x7f4d3c113de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d3affd700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d3c0fe7e0 0x7f4d3c113de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d3affd700 1 -- 172.21.15.33:0/3736017805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c005040 con 0x7f4d3c0fde40 2024-04-25T02:10:51.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d3affd700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 0x7f4d3c1138a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c000c00 tx=0x7f4d2c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.519+0000 7f4d237fe700 1 -- 172.21.15.33:0/3736017805 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2c021070 con 0x7f4d3c0fde40 2024-04-25T02:10:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.520+0000 7f4d237fe700 1 -- 172.21.15.33:0/3736017805 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d2c013c00 con 0x7f4d3c0fde40 2024-04-25T02:10:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.520+0000 7f4d237fe700 1 -- 172.21.15.33:0/3736017805 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2c00e070 con 0x7f4d3c0fde40 2024-04-25T02:10:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.520+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c10dd00 con 0x7f4d3c0fde40 2024-04-25T02:10:51.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.520+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c10e240 con 0x7f4d3c0fde40 2024-04-25T02:10:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.521+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c066bd0 con 0x7f4d3c0fde40 2024-04-25T02:10:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.521+0000 7f4d237fe700 1 -- 172.21.15.33:0/3736017805 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4d2c01c9d0 con 0x7f4d3c0fde40 2024-04-25T02:10:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.522+0000 7f4d237fe700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4d2406e140 0x7f4d240705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.522+0000 7f4d237fe700 1 -- 172.21.15.33:0/3736017805 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4d2c08ede0 con 0x7f4d3c0fde40 2024-04-25T02:10:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.522+0000 7f4d3a7fc700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4d2406e140 0x7f4d240705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:51.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.522+0000 7f4d3a7fc700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4d2406e140 0x7f4d240705f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c10ebf0 tx=0x7f4d3000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:51.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.526+0000 7f4d237fe700 1 -- 172.21.15.33:0/3736017805 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4d2c05bdb0 con 0x7f4d3c0fde40 2024-04-25T02:10:51.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:51 smithi033 ceph-mon[111306]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:51.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.780+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d3c10ef50 con 0x7f4d3c0fde40 2024-04-25T02:10:51.788 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.781+0000 7f4d237fe700 1 -- 172.21.15.33:0/3736017805 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f4d2c05b940 con 0x7f4d3c0fde40 2024-04-25T02:10:51.788 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:10:51.789 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:10:51.790 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:10:51.791 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4d2406e140 msgr2=0x7f4d240705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4d2406e140 0x7f4d240705f0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c10ebf0 tx=0x7f4d3000b040 comp rx=0 tx=0).stop 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 msgr2=0x7f4d3c1138a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 0x7f4d3c1138a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c000c00 tx=0x7f4d2c009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 shutdown_connections 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4d2406e140 0x7f4d240705f0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4d3c0fde40 0x7f4d3c1138a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 --2- 172.21.15.33:0/3736017805 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4d3c0fe7e0 0x7f4d3c113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 >> 172.21.15.33:0/3736017805 conn(0x7f4d3c0f78d0 msgr2=0x7f4d3c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:51.792 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 shutdown_connections 2024-04-25T02:10:51.793 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.783+0000 7f4d41471700 1 -- 172.21.15.33:0/3736017805 wait complete. 2024-04-25T02:10:51.793 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:10:51.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:51 smithi138 ceph-mon[81304]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 -- 172.21.15.33:0/474604581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308101ce0 msgr2=0x7f33081020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 --2- 172.21.15.33:0/474604581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308101ce0 0x7f33081020b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f32fc004940 tx=0x7f32fc009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 -- 172.21.15.33:0/474604581 shutdown_connections 2024-04-25T02:10:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 --2- 172.21.15.33:0/474604581 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3308102680 0x7f33080fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 --2- 172.21.15.33:0/474604581 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308101ce0 0x7f33081020b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 -- 172.21.15.33:0/474604581 >> 172.21.15.33:0/474604581 conn(0x7f33080f7890 msgr2=0x7f33080f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 -- 172.21.15.33:0/474604581 shutdown_connections 2024-04-25T02:10:51.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 -- 172.21.15.33:0/474604581 wait complete. 2024-04-25T02:10:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 Processor -- start 2024-04-25T02:10:51.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.888+0000 7f330ed17700 1 -- start start 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f330ed17700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3308101ce0 0x7f3308102b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f330ed17700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308102680 0x7f3308103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f330ed17700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3308103610 con 0x7f3308102680 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f330ed17700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3308103750 con 0x7f3308101ce0 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f3307fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308102680 0x7f3308103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f3307fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308102680 0x7f3308103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53654/0 (socket says 172.21.15.33:53654) 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f3307fff700 1 -- 172.21.15.33:0/3042974934 learned_addr learned my addr 172.21.15.33:0/3042974934 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f330cab3700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3308101ce0 0x7f3308102b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f3307fff700 1 -- 172.21.15.33:0/3042974934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3308101ce0 msgr2=0x7f3308102b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f3307fff700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3308101ce0 0x7f3308102b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f3307fff700 1 -- 172.21.15.33:0/3042974934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc005040 con 0x7f3308102680 2024-04-25T02:10:51.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.889+0000 7f3307fff700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308102680 0x7f3308103080 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f32f4002a50 tx=0x7f32f4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.890+0000 7f33057fa700 1 -- 172.21.15.33:0/3042974934 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f4007cc0 con 0x7f3308102680 2024-04-25T02:10:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.890+0000 7f33057fa700 1 -- 172.21.15.33:0/3042974934 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32f400f040 con 0x7f3308102680 2024-04-25T02:10:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.890+0000 7f33057fa700 1 -- 172.21.15.33:0/3042974934 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32f400d9c0 con 0x7f3308102680 2024-04-25T02:10:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.890+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3308119e60 con 0x7f3308102680 2024-04-25T02:10:51.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.890+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f330811a2e0 con 0x7f3308102680 2024-04-25T02:10:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.891+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33080fd430 con 0x7f3308102680 2024-04-25T02:10:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.891+0000 7f33057fa700 1 -- 172.21.15.33:0/3042974934 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f32f4009730 con 0x7f3308102680 2024-04-25T02:10:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.891+0000 7f33057fa700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32f806e140 0x7f32f80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.892+0000 7f33057fa700 1 -- 172.21.15.33:0/3042974934 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f32f408d870 con 0x7f3308102680 2024-04-25T02:10:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.892+0000 7f330cab3700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32f806e140 0x7f32f80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:51.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.892+0000 7f330cab3700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32f806e140 0x7f32f80705f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f32fc004f90 tx=0x7f32fc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:51.904 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:51.896+0000 7f33057fa700 1 -- 172.21.15.33:0/3042974934 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f32f4056bf0 con 0x7f3308102680 2024-04-25T02:10:52.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.120+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f330804bed0 con 0x7f32f806e140 2024-04-25T02:10:52.130 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.122+0000 7f33057fa700 1 -- 172.21.15.33:0/3042974934 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f330804bed0 con 0x7f32f806e140 2024-04-25T02:10:52.130 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:10:52.130 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:10:52.130 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:10:52.130 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:10:52.130 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:10:52.130 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:10:52.130 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:10:52.130 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:10:52.131 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:10:52.131 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:10:52.131 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:10:52.131 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:10:52.131 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:10:52.131 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:10:52.131 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:10:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32f806e140 msgr2=0x7f32f80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32f806e140 0x7f32f80705f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f32fc004f90 tx=0x7f32fc009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308102680 msgr2=0x7f3308103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308102680 0x7f3308103080 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f32f4002a50 tx=0x7f32f4004600 comp rx=0 tx=0).stop 2024-04-25T02:10:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 shutdown_connections 2024-04-25T02:10:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f32f806e140 0x7f32f80705f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3308101ce0 0x7f3308102b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 --2- 172.21.15.33:0/3042974934 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3308102680 0x7f3308103080 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:52.133 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 >> 172.21.15.33:0/3042974934 conn(0x7f33080f7890 msgr2=0x7f33080f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:52.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 shutdown_connections 2024-04-25T02:10:52.134 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.125+0000 7f330ed17700 1 -- 172.21.15.33:0/3042974934 wait complete. 2024-04-25T02:10:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.230+0000 7fccae366700 1 -- 172.21.15.33:0/2488664047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca80fb520 msgr2=0x7fcca80fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:52.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.230+0000 7fccae366700 1 --2- 172.21.15.33:0/2488664047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca80fb520 0x7fcca80fb990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c004940 tx=0x7fcc9c009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.230+0000 7fccae366700 1 -- 172.21.15.33:0/2488664047 shutdown_connections 2024-04-25T02:10:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.230+0000 7fccae366700 1 --2- 172.21.15.33:0/2488664047 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca80fb520 0x7fcca80fb990 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.230+0000 7fccae366700 1 --2- 172.21.15.33:0/2488664047 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca8102f60 0x7fcca8103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.230+0000 7fccae366700 1 -- 172.21.15.33:0/2488664047 >> 172.21.15.33:0/2488664047 conn(0x7fcca80714d0 msgr2=0x7fcca80718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.231+0000 7fccae366700 1 -- 172.21.15.33:0/2488664047 shutdown_connections 2024-04-25T02:10:52.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.231+0000 7fccae366700 1 -- 172.21.15.33:0/2488664047 wait complete. 2024-04-25T02:10:52.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.231+0000 7fccae366700 1 Processor -- start 2024-04-25T02:10:52.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.231+0000 7fccae366700 1 -- start start 2024-04-25T02:10:52.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.231+0000 7fccae366700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca80fb520 0x7fcca8113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:52.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.231+0000 7fccae366700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca8102f60 0x7fcca8113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.231+0000 7fccae366700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca8114510 con 0x7fcca8102f60 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fccae366700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca810db10 con 0x7fcca80fb520 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca8102f60 0x7fcca8113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca8102f60 0x7fcca8113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53676/0 (socket says 172.21.15.33:53676) 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca77fe700 1 -- 172.21.15.33:0/3344533277 learned_addr learned my addr 172.21.15.33:0/3344533277 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca7fff700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca80fb520 0x7fcca8113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca77fe700 1 -- 172.21.15.33:0/3344533277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca80fb520 msgr2=0x7fcca8113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca77fe700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca80fb520 0x7fcca8113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca77fe700 1 -- 172.21.15.33:0/3344533277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c005040 con 0x7fcca8102f60 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca77fe700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca8102f60 0x7fcca8113ec0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b8b0 tx=0x7fcc9c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca4ff9700 1 -- 172.21.15.33:0/3344533277 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc9c00e040 con 0x7fcca8102f60 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca4ff9700 1 -- 172.21.15.33:0/3344533277 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc9c004290 con 0x7fcca8102f60 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca810dd90 con 0x7fcca8102f60 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fcca4ff9700 1 -- 172.21.15.33:0/3344533277 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc9c01c3f0 con 0x7fcca8102f60 2024-04-25T02:10:52.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.232+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca810e2d0 con 0x7fcca8102f60 2024-04-25T02:10:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.233+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca8100250 con 0x7fcca8102f60 2024-04-25T02:10:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.234+0000 7fcca4ff9700 1 -- 172.21.15.33:0/3344533277 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcc9c013420 con 0x7fcca8102f60 2024-04-25T02:10:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.234+0000 7fcca4ff9700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc9006e140 0x7fcc900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:10:52.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.235+0000 7fcca4ff9700 1 -- 172.21.15.33:0/3344533277 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcc9c08ec70 con 0x7fcca8102f60 2024-04-25T02:10:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.235+0000 7fcca7fff700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc9006e140 0x7fcc900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:10:52.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.235+0000 7fcca7fff700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc9006e140 0x7fcc900705f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcc98004f40 tx=0x7fcc9800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:10:52.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.239+0000 7fcca4ff9700 1 -- 172.21.15.33:0/3344533277 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcc9c05bc40 con 0x7fcca8102f60 2024-04-25T02:10:52.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.531+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcca804c740 con 0x7fcca8102f60 2024-04-25T02:10:52.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.531+0000 7fcca4ff9700 1 -- 172.21.15.33:0/3344533277 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fcc9c05b7d0 con 0x7fcca8102f60 2024-04-25T02:10:52.539 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:10:52.540 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:10:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:10:52.541 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:10:52.541 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:10:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:10:52.541 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:10:52.541 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:10:52.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.533+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc9006e140 msgr2=0x7fcc900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:52.541 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.533+0000 7fccae366700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc9006e140 0x7fcc900705f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fcc98004f40 tx=0x7fcc9800b040 comp rx=0 tx=0).stop 2024-04-25T02:10:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.533+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca8102f60 msgr2=0x7fcca8113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:10:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.533+0000 7fccae366700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca8102f60 0x7fcca8113ec0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c00b8b0 tx=0x7fcc9c009e30 comp rx=0 tx=0).stop 2024-04-25T02:10:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.534+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 shutdown_connections 2024-04-25T02:10:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.534+0000 7fccae366700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc9006e140 0x7fcc900705f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.534+0000 7fccae366700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcca80fb520 0x7fcca8113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.534+0000 7fccae366700 1 --2- 172.21.15.33:0/3344533277 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcca8102f60 0x7fcca8113ec0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:10:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.534+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 >> 172.21.15.33:0/3344533277 conn(0x7fcca80714d0 msgr2=0x7fcca80fa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:10:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.534+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 shutdown_connections 2024-04-25T02:10:52.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:10:52.534+0000 7fccae366700 1 -- 172.21.15.33:0/3344533277 wait complete. 2024-04-25T02:10:52.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:52 smithi033 ceph-mon[111306]: from='client.35724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:52.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:52 smithi033 ceph-mon[111306]: from='client.35728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:52.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:52 smithi033 ceph-mon[111306]: pgmap v945: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:52.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:52 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1756480451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:10:52.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:52 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3736017805' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:10:52.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:52 smithi138 ceph-mon[81304]: from='client.35724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:52.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:52 smithi138 ceph-mon[81304]: from='client.35728 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:52.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:52 smithi138 ceph-mon[81304]: pgmap v945: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:52.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:52 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1756480451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:10:52.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:52 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3736017805' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:10:53.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:53 smithi033 ceph-mon[111306]: from='client.35740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:53.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:53 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3344533277' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:10:53.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:53 smithi138 ceph-mon[81304]: from='client.35740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:10:53.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:53 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3344533277' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:10:54.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:54 smithi033 ceph-mon[111306]: pgmap v946: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:54.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:54 smithi138 ceph-mon[81304]: pgmap v946: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:55.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:10:55.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:10:56.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:56 smithi033 ceph-mon[111306]: pgmap v947: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:56.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:56 smithi138 ceph-mon[81304]: pgmap v947: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:58.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:10:58 smithi033 ceph-mon[111306]: pgmap v948: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:10:58.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:10:58 smithi138 ceph-mon[81304]: pgmap v948: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:00.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:00 smithi033 ceph-mon[111306]: pgmap v949: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:00.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:00 smithi138 ceph-mon[81304]: pgmap v949: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:02.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:02 smithi033 ceph-mon[111306]: pgmap v950: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:02.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:02 smithi138 ceph-mon[81304]: pgmap v950: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:03.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:03 smithi033 ceph-mon[111306]: pgmap v951: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:03.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:03 smithi138 ceph-mon[81304]: pgmap v951: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:06.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:06 smithi033 ceph-mon[111306]: pgmap v952: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:06.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:06 smithi138 ceph-mon[81304]: pgmap v952: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:08.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:08 smithi033 ceph-mon[111306]: pgmap v953: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:08.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:08 smithi138 ceph-mon[81304]: pgmap v953: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:10.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:10 smithi033 ceph-mon[111306]: pgmap v954: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:10.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:11:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:10 smithi138 ceph-mon[81304]: pgmap v954: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:10.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:11:12.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:12 smithi033 ceph-mon[111306]: pgmap v955: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:12.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:12 smithi138 ceph-mon[81304]: pgmap v955: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:14 smithi033 ceph-mon[111306]: pgmap v956: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:14.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:14 smithi138 ceph-mon[81304]: pgmap v956: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:16.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:16 smithi033 ceph-mon[111306]: pgmap v957: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:16.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:16 smithi138 ceph-mon[81304]: pgmap v957: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:18.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:18 smithi033 ceph-mon[111306]: pgmap v958: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:18.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:18 smithi138 ceph-mon[81304]: pgmap v958: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:20.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:20 smithi033 ceph-mon[111306]: pgmap v959: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:20.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:20 smithi138 ceph-mon[81304]: pgmap v959: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.650+0000 7f4498719700 1 -- 172.21.15.33:0/3391121030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fded0 msgr2=0x7f44900fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:22.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.650+0000 7f4498719700 1 --2- 172.21.15.33:0/3391121030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fded0 0x7f44900fe2a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f448c004940 tx=0x7f448c009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:22.659 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:22 smithi033 ceph-mon[111306]: pgmap v960: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.650+0000 7f4498719700 1 -- 172.21.15.33:0/3391121030 shutdown_connections 2024-04-25T02:11:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.650+0000 7f4498719700 1 --2- 172.21.15.33:0/3391121030 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44900fe870 0x7f4490106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:22.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.650+0000 7f4498719700 1 --2- 172.21.15.33:0/3391121030 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fded0 0x7f44900fe2a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.650+0000 7f4498719700 1 -- 172.21.15.33:0/3391121030 >> 172.21.15.33:0/3391121030 conn(0x7f44900f78e0 msgr2=0x7f44900f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.650+0000 7f4498719700 1 -- 172.21.15.33:0/3391121030 shutdown_connections 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.650+0000 7f4498719700 1 -- 172.21.15.33:0/3391121030 wait complete. 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.651+0000 7f4498719700 1 Processor -- start 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.651+0000 7f4498719700 1 -- start start 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.651+0000 7f4498719700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44900fded0 0x7f44900fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.651+0000 7f4498719700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fe870 0x7f44900fc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.651+0000 7f4498719700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900fdc50 con 0x7f44900fe870 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.651+0000 7f4498719700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44900fc780 con 0x7f44900fded0 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.651+0000 7f4495cb4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fe870 0x7f44900fc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.651+0000 7f4495cb4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fe870 0x7f44900fc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46714/0 (socket says 172.21.15.33:46714) 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.651+0000 7f4495cb4700 1 -- 172.21.15.33:0/2022652595 learned_addr learned my addr 172.21.15.33:0/2022652595 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f44964b5700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44900fded0 0x7f44900fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f4495cb4700 1 -- 172.21.15.33:0/2022652595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44900fded0 msgr2=0x7f44900fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:22.661 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f4495cb4700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44900fded0 0x7f44900fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f4495cb4700 1 -- 172.21.15.33:0/2022652595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448c005040 con 0x7f44900fe870 2024-04-25T02:11:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f4495cb4700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fe870 0x7f44900fc240 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4480002a50 tx=0x7f44800046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f4486ffd700 1 -- 172.21.15.33:0/2022652595 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4480007cc0 con 0x7f44900fe870 2024-04-25T02:11:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44900fca60 con 0x7f44900fe870 2024-04-25T02:11:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f4486ffd700 1 -- 172.21.15.33:0/2022652595 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f448000f040 con 0x7f44900fe870 2024-04-25T02:11:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f4486ffd700 1 -- 172.21.15.33:0/2022652595 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f448000da30 con 0x7f44900fe870 2024-04-25T02:11:22.662 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.652+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4490119fe0 con 0x7f44900fe870 2024-04-25T02:11:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.653+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4490066a70 con 0x7f44900fe870 2024-04-25T02:11:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.654+0000 7f4486ffd700 1 -- 172.21.15.33:0/2022652595 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f448000db90 con 0x7f44900fe870 2024-04-25T02:11:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.654+0000 7f4486ffd700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f447c06e140 0x7f447c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.654+0000 7f4486ffd700 1 -- 172.21.15.33:0/2022652595 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f448008d970 con 0x7f44900fe870 2024-04-25T02:11:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.654+0000 7f44964b5700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f447c06e140 0x7f447c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:22.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.655+0000 7f44964b5700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f447c06e140 0x7f447c0705f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f448c004f90 tx=0x7f448c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:22.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.659+0000 7f4486ffd700 1 -- 172.21.15.33:0/2022652595 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4480056c50 con 0x7f44900fe870 2024-04-25T02:11:22.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:22 smithi138 ceph-mon[81304]: pgmap v960: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:22.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.888+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4490102130 con 0x7f447c06e140 2024-04-25T02:11:22.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.889+0000 7f4486ffd700 1 -- 172.21.15.33:0/2022652595 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4490102130 con 0x7f447c06e140 2024-04-25T02:11:22.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.891+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f447c06e140 msgr2=0x7f447c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:22.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.891+0000 7f4498719700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f447c06e140 0x7f447c0705f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f448c004f90 tx=0x7f448c009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:22.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.891+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fe870 msgr2=0x7f44900fc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:22.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.891+0000 7f4498719700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fe870 0x7f44900fc240 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f4480002a50 tx=0x7f44800046b0 comp rx=0 tx=0).stop 2024-04-25T02:11:22.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.891+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 shutdown_connections 2024-04-25T02:11:22.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.891+0000 7f4498719700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f447c06e140 0x7f447c0705f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:22.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.891+0000 7f4498719700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44900fded0 0x7f44900fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:22.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.891+0000 7f4498719700 1 --2- 172.21.15.33:0/2022652595 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f44900fe870 0x7f44900fc240 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:22.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.891+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 >> 172.21.15.33:0/2022652595 conn(0x7f44900f78e0 msgr2=0x7f4490103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:22.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.892+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 shutdown_connections 2024-04-25T02:11:22.900 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.892+0000 7f4498719700 1 -- 172.21.15.33:0/2022652595 wait complete. 2024-04-25T02:11:22.909 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:11:23.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.997+0000 7f0c855db700 1 -- 172.21.15.33:0/3300963309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80102de0 msgr2=0x7f0c801031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.997+0000 7f0c855db700 1 --2- 172.21.15.33:0/3300963309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80102de0 0x7f0c801031b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0c70009ce0 tx=0x7f0c70004d10 comp rx=0 tx=0).stop 2024-04-25T02:11:23.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.997+0000 7f0c855db700 1 -- 172.21.15.33:0/3300963309 shutdown_connections 2024-04-25T02:11:23.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.997+0000 7f0c855db700 1 --2- 172.21.15.33:0/3300963309 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c80103780 0x7f0c800fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.997+0000 7f0c855db700 1 --2- 172.21.15.33:0/3300963309 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80102de0 0x7f0c801031b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.997+0000 7f0c855db700 1 -- 172.21.15.33:0/3300963309 >> 172.21.15.33:0/3300963309 conn(0x7f0c800f89b0 msgr2=0x7f0c800fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:23.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.997+0000 7f0c855db700 1 -- 172.21.15.33:0/3300963309 shutdown_connections 2024-04-25T02:11:23.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.997+0000 7f0c855db700 1 -- 172.21.15.33:0/3300963309 wait complete. 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c855db700 1 Processor -- start 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c855db700 1 -- start start 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c855db700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c80102de0 0x7f0c80115ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c855db700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80103780 0x7f0c80116420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c855db700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c80116a70 con 0x7f0c80103780 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c855db700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c8010fe40 con 0x7f0c80102de0 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c7e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80103780 0x7f0c80116420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c7e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80103780 0x7f0c80116420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46740/0 (socket says 172.21.15.33:46740) 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c7e7fc700 1 -- 172.21.15.33:0/2351225737 learned_addr learned my addr 172.21.15.33:0/2351225737 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c7effd700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c80102de0 0x7f0c80115ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c7e7fc700 1 -- 172.21.15.33:0/2351225737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c80102de0 msgr2=0x7f0c80115ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c7e7fc700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c80102de0 0x7f0c80115ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.998+0000 7f0c7e7fc700 1 -- 172.21.15.33:0/2351225737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70009940 con 0x7f0c80103780 2024-04-25T02:11:23.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.999+0000 7f0c7e7fc700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80103780 0x7f0c80116420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0c68002a50 tx=0x7f0c68004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:23.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.999+0000 7f0c777fe700 1 -- 172.21.15.33:0/2351225737 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c68007cc0 con 0x7f0c80103780 2024-04-25T02:11:23.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.999+0000 7f0c777fe700 1 -- 172.21.15.33:0/2351225737 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c6800f040 con 0x7f0c80103780 2024-04-25T02:11:23.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.999+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c80110120 con 0x7f0c80103780 2024-04-25T02:11:23.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.999+0000 7f0c777fe700 1 -- 172.21.15.33:0/2351225737 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c6800d9c0 con 0x7f0c80103780 2024-04-25T02:11:23.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:22.999+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c801106e0 con 0x7f0c80103780 2024-04-25T02:11:23.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.000+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c801072e0 con 0x7f0c80103780 2024-04-25T02:11:23.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.000+0000 7f0c777fe700 1 -- 172.21.15.33:0/2351225737 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0c68009730 con 0x7f0c80103780 2024-04-25T02:11:23.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.001+0000 7f0c777fe700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c6c06e140 0x7f0c6c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:23.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.001+0000 7f0c7effd700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c6c06e140 0x7f0c6c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:23.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.001+0000 7f0c777fe700 1 -- 172.21.15.33:0/2351225737 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0c6808d760 con 0x7f0c80103780 2024-04-25T02:11:23.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.001+0000 7f0c7effd700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c6c06e140 0x7f0c6c0705f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000b740 tx=0x7f0c70004d10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:23.013 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.006+0000 7f0c777fe700 1 -- 172.21.15.33:0/2351225737 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0c6805a690 con 0x7f0c80103780 2024-04-25T02:11:23.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.232+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c801115a0 con 0x7f0c6c06e140 2024-04-25T02:11:23.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.233+0000 7f0c777fe700 1 -- 172.21.15.33:0/2351225737 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0c801115a0 con 0x7f0c6c06e140 2024-04-25T02:11:23.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c6c06e140 msgr2=0x7f0c6c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c6c06e140 0x7f0c6c0705f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0c7000b740 tx=0x7f0c70004d10 comp rx=0 tx=0).stop 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80103780 msgr2=0x7f0c80116420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80103780 0x7f0c80116420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0c68002a50 tx=0x7f0c68004600 comp rx=0 tx=0).stop 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 shutdown_connections 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c6c06e140 0x7f0c6c0705f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c80102de0 0x7f0c80115ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 --2- 172.21.15.33:0/2351225737 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c80103780 0x7f0c80116420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 >> 172.21.15.33:0/2351225737 conn(0x7f0c800f89b0 msgr2=0x7f0c800fa5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 shutdown_connections 2024-04-25T02:11:23.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.236+0000 7f0c855db700 1 -- 172.21.15.33:0/2351225737 wait complete. 2024-04-25T02:11:23.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.339+0000 7fc015f58700 1 -- 172.21.15.33:0/3346915630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0100fbd40 msgr2=0x7fc0100fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.339+0000 7fc015f58700 1 --2- 172.21.15.33:0/3346915630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0100fbd40 0x7fc0100fc1b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc004004940 tx=0x7fc004009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.340+0000 7fc015f58700 1 -- 172.21.15.33:0/3346915630 shutdown_connections 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.340+0000 7fc015f58700 1 --2- 172.21.15.33:0/3346915630 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0100fbd40 0x7fc0100fc1b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.340+0000 7fc015f58700 1 --2- 172.21.15.33:0/3346915630 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc010101d90 0x7fc010102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.340+0000 7fc015f58700 1 -- 172.21.15.33:0/3346915630 >> 172.21.15.33:0/3346915630 conn(0x7fc010071d00 msgr2=0x7fc010072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.340+0000 7fc015f58700 1 -- 172.21.15.33:0/3346915630 shutdown_connections 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.340+0000 7fc015f58700 1 -- 172.21.15.33:0/3346915630 wait complete. 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.340+0000 7fc015f58700 1 Processor -- start 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.340+0000 7fc015f58700 1 -- start start 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc015f58700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0100fbd40 0x7fc01010ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc015f58700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc010101d90 0x7fc01010f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc015f58700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01010f750 con 0x7fc010101d90 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc015f58700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01010f890 con 0x7fc0100fbd40 2024-04-25T02:11:23.349 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc00effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc010101d90 0x7fc01010f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:23.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc00effd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc010101d90 0x7fc01010f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46758/0 (socket says 172.21.15.33:46758) 2024-04-25T02:11:23.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc00effd700 1 -- 172.21.15.33:0/1929468429 learned_addr learned my addr 172.21.15.33:0/1929468429 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:23.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc00f7fe700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0100fbd40 0x7fc01010ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:23.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc00effd700 1 -- 172.21.15.33:0/1929468429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0100fbd40 msgr2=0x7fc01010ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc00effd700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0100fbd40 0x7fc01010ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc00effd700 1 -- 172.21.15.33:0/1929468429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004005040 con 0x7fc010101d90 2024-04-25T02:11:23.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.341+0000 7fc00effd700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc010101d90 0x7fc01010f210 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b3b0 tx=0x7fc004009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:23.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.343+0000 7fbff7fff700 1 -- 172.21.15.33:0/1929468429 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00400e040 con 0x7fc010101d90 2024-04-25T02:11:23.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.343+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc010117b80 con 0x7fc010101d90 2024-04-25T02:11:23.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.343+0000 7fbff7fff700 1 -- 172.21.15.33:0/1929468429 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc004013420 con 0x7fc010101d90 2024-04-25T02:11:23.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.343+0000 7fbff7fff700 1 -- 172.21.15.33:0/1929468429 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00401c630 con 0x7fc010101d90 2024-04-25T02:11:23.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.343+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc010118110 con 0x7fc010101d90 2024-04-25T02:11:23.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.344+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0100fd430 con 0x7fc010101d90 2024-04-25T02:11:23.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.345+0000 7fbff7fff700 1 -- 172.21.15.33:0/1929468429 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc004013590 con 0x7fc010101d90 2024-04-25T02:11:23.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.345+0000 7fbff7fff700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbff806e140 0x7fbff80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:23.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.345+0000 7fbff7fff700 1 -- 172.21.15.33:0/1929468429 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc00408ec30 con 0x7fc010101d90 2024-04-25T02:11:23.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.346+0000 7fc00f7fe700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbff806e140 0x7fbff80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:23.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.346+0000 7fc00f7fe700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbff806e140 0x7fbff80705f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc000004f40 tx=0x7fc00000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:23.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.350+0000 7fbff7fff700 1 -- 172.21.15.33:0/1929468429 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc0040580d0 con 0x7fc010101d90 2024-04-25T02:11:23.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.568+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc01004bed0 con 0x7fbff806e140 2024-04-25T02:11:23.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.578+0000 7fbff7fff700 1 -- 172.21.15.33:0/1929468429 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc01004bed0 con 0x7fbff806e140 2024-04-25T02:11:23.586 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:11:23.586 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (32m) 10m ago 43m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:11:23.586 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (29m) 10m ago 43m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:11:23.586 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (29m) 10m ago 42m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:11:23.586 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (32m) 10m ago 43m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (25m) 10m ago 38m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (26m) 10m ago 38m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (25m) 10m ago 38m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (10m) 10m ago 38m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (34m) 10m ago 45m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (33m) 10m ago 42m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (30m) 10m ago 45m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (30m) 10m ago 42m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (33m) 10m ago 43m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (33m) 10m ago 42m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (29m) 10m ago 42m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (28m) 10m ago 41m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (28m) 10m ago 41m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (27m) 10m ago 40m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (26m) 10m ago 40m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (26m) 10m ago 39m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:11:23.587 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (32m) 10m ago 43m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:11:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbff806e140 msgr2=0x7fbff80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbff806e140 0x7fbff80705f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc000004f40 tx=0x7fc00000b040 comp rx=0 tx=0).stop 2024-04-25T02:11:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc010101d90 msgr2=0x7fc01010f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc010101d90 0x7fc01010f210 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc00400b3b0 tx=0x7fc004009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:23.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 shutdown_connections 2024-04-25T02:11:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fbff806e140 0x7fbff80705f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0100fbd40 0x7fc01010ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 --2- 172.21.15.33:0/1929468429 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc010101d90 0x7fc01010f210 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 >> 172.21.15.33:0/1929468429 conn(0x7fc010071d00 msgr2=0x7fc010104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 shutdown_connections 2024-04-25T02:11:23.589 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.580+0000 7fc015f58700 1 -- 172.21.15.33:0/1929468429 wait complete. 2024-04-25T02:11:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.684+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/1810369292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c40fbd40 msgr2=0x7ff6c40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.684+0000 7ff6c8c9d700 1 --2- 172.21.15.33:0/1810369292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c40fbd40 0x7ff6c40fc1b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff6b8004940 tx=0x7ff6b8009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.684+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/1810369292 shutdown_connections 2024-04-25T02:11:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.684+0000 7ff6c8c9d700 1 --2- 172.21.15.33:0/1810369292 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c40fbd40 0x7ff6c40fc1b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.684+0000 7ff6c8c9d700 1 --2- 172.21.15.33:0/1810369292 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c4101d90 0x7ff6c4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.684+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/1810369292 >> 172.21.15.33:0/1810369292 conn(0x7ff6c4071d00 msgr2=0x7ff6c4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.684+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/1810369292 shutdown_connections 2024-04-25T02:11:23.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.684+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/1810369292 wait complete. 2024-04-25T02:11:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c8c9d700 1 Processor -- start 2024-04-25T02:11:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c8c9d700 1 -- start start 2024-04-25T02:11:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c8c9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c40fbd40 0x7ff6c4115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c8c9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101d90 0x7ff6c4116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c8c9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c41168c0 con 0x7ff6c4101d90 2024-04-25T02:11:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c8c9d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c410fc90 con 0x7ff6c40fbd40 2024-04-25T02:11:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101d90 0x7ff6c4116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:23.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c1d9b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101d90 0x7ff6c4116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46780/0 (socket says 172.21.15.33:46780) 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c1d9b700 1 -- 172.21.15.33:0/4094903904 learned_addr learned my addr 172.21.15.33:0/4094903904 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c259c700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c40fbd40 0x7ff6c4115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c1d9b700 1 -- 172.21.15.33:0/4094903904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c40fbd40 msgr2=0x7ff6c4115d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c1d9b700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c40fbd40 0x7ff6c4115d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.685+0000 7ff6c1d9b700 1 -- 172.21.15.33:0/4094903904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6b8005040 con 0x7ff6c4101d90 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.686+0000 7ff6c1d9b700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101d90 0x7ff6c4116270 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800b3b0 tx=0x7ff6b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.686+0000 7ff6b2ffd700 1 -- 172.21.15.33:0/4094903904 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b800e040 con 0x7ff6c4101d90 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.686+0000 7ff6b2ffd700 1 -- 172.21.15.33:0/4094903904 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6b800bbc0 con 0x7ff6c4101d90 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.686+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c410ff10 con 0x7ff6c4101d90 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.686+0000 7ff6b2ffd700 1 -- 172.21.15.33:0/4094903904 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6b801c3f0 con 0x7ff6c4101d90 2024-04-25T02:11:23.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.686+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c41104d0 con 0x7ff6c4101d90 2024-04-25T02:11:23.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.687+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c4104160 con 0x7ff6c4101d90 2024-04-25T02:11:23.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.687+0000 7ff6b2ffd700 1 -- 172.21.15.33:0/4094903904 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff6b800bd30 con 0x7ff6c4101d90 2024-04-25T02:11:23.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.688+0000 7ff6b2ffd700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6ac06e140 0x7ff6ac0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:23.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.688+0000 7ff6b2ffd700 1 -- 172.21.15.33:0/4094903904 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff6b808ee00 con 0x7ff6c4101d90 2024-04-25T02:11:23.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.688+0000 7ff6c259c700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6ac06e140 0x7ff6ac0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:23.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.688+0000 7ff6c259c700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6ac06e140 0x7ff6ac0705f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4004990 tx=0x7ff6b4009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:23.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.692+0000 7ff6b2ffd700 1 -- 172.21.15.33:0/4094903904 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff6b805bcb0 con 0x7ff6c4101d90 2024-04-25T02:11:23.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.988+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff6c40fd430 con 0x7ff6c4101d90 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.989+0000 7ff6b2ffd700 1 -- 172.21.15.33:0/4094903904 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff6b805b840 con 0x7ff6c4101d90 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:11:23.997 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:11:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:11:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:11:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:11:23.998 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:11:23.998 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:11:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.991+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6ac06e140 msgr2=0x7ff6ac0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:23.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.991+0000 7ff6c8c9d700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6ac06e140 0x7ff6ac0705f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4004990 tx=0x7ff6b4009ee0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.992+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101d90 msgr2=0x7ff6c4116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.992+0000 7ff6c8c9d700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101d90 0x7ff6c4116270 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff6b800b3b0 tx=0x7ff6b8009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.992+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 shutdown_connections 2024-04-25T02:11:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.992+0000 7ff6c8c9d700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff6ac06e140 0x7ff6ac0705f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.992+0000 7ff6c8c9d700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff6c40fbd40 0x7ff6c4115d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.992+0000 7ff6c8c9d700 1 --2- 172.21.15.33:0/4094903904 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff6c4101d90 0x7ff6c4116270 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.992+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 >> 172.21.15.33:0/4094903904 conn(0x7ff6c4071d00 msgr2=0x7ff6c40fb060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.992+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 shutdown_connections 2024-04-25T02:11:24.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:23.992+0000 7ff6c8c9d700 1 -- 172.21.15.33:0/4094903904 wait complete. 2024-04-25T02:11:24.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.095+0000 7f521944e700 1 -- 172.21.15.33:0/4229145693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 msgr2=0x7f52140fbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.095+0000 7f521944e700 1 --2- 172.21.15.33:0/4229145693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 0x7f52140fbf90 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f5208004940 tx=0x7f5208009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:24.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.095+0000 7f521944e700 1 -- 172.21.15.33:0/4229145693 shutdown_connections 2024-04-25T02:11:24.103 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.095+0000 7f521944e700 1 --2- 172.21.15.33:0/4229145693 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fc4d0 0x7f5214109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.095+0000 7f521944e700 1 --2- 172.21.15.33:0/4229145693 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 0x7f52140fbf90 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.095+0000 7f521944e700 1 -- 172.21.15.33:0/4229145693 >> 172.21.15.33:0/4229145693 conn(0x7f52140f7870 msgr2=0x7f52140f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.096+0000 7f521944e700 1 -- 172.21.15.33:0/4229145693 shutdown_connections 2024-04-25T02:11:24.104 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.096+0000 7f521944e700 1 -- 172.21.15.33:0/4229145693 wait complete. 2024-04-25T02:11:24.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.096+0000 7f521944e700 1 Processor -- start 2024-04-25T02:11:24.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.096+0000 7f521944e700 1 -- start start 2024-04-25T02:11:24.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.096+0000 7f521944e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 0x7f5214115a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:24.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.096+0000 7f521944e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fc4d0 0x7f5214115fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.096+0000 7f521944e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5214116620 con 0x7f52140fbbc0 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.096+0000 7f521944e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521410fc00 con 0x7f52140fc4d0 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f5212ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 0x7f5214115a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f5212ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 0x7f5214115a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46798/0 (socket says 172.21.15.33:46798) 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f5212ffd700 1 -- 172.21.15.33:0/1363485142 learned_addr learned my addr 172.21.15.33:0/1363485142 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f52127fc700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fc4d0 0x7f5214115fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f5212ffd700 1 -- 172.21.15.33:0/1363485142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fc4d0 msgr2=0x7f5214115fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f5212ffd700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fc4d0 0x7f5214115fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f5212ffd700 1 -- 172.21.15.33:0/1363485142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5208005040 con 0x7f52140fbbc0 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f52127fc700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fc4d0 0x7f5214115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f5212ffd700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 0x7f5214115a90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5208009380 tx=0x7f5208009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f51fb7fe700 1 -- 172.21.15.33:0/1363485142 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520800e040 con 0x7f52140fbbc0 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521410fee0 con 0x7f52140fbbc0 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f51fb7fe700 1 -- 172.21.15.33:0/1363485142 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5208009af0 con 0x7f52140fbbc0 2024-04-25T02:11:24.106 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f51fb7fe700 1 -- 172.21.15.33:0/1363485142 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f520801c630 con 0x7f52140fbbc0 2024-04-25T02:11:24.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.097+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52141104a0 con 0x7f52140fbbc0 2024-04-25T02:11:24.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.099+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5214066ad0 con 0x7f52140fbbc0 2024-04-25T02:11:24.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.099+0000 7f51fb7fe700 1 -- 172.21.15.33:0/1363485142 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f520800bbc0 con 0x7f52140fbbc0 2024-04-25T02:11:24.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.100+0000 7f51fb7fe700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f51fc06e140 0x7f51fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:24.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.100+0000 7f51fb7fe700 1 -- 172.21.15.33:0/1363485142 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f520808ee20 con 0x7f52140fbbc0 2024-04-25T02:11:24.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.100+0000 7f52127fc700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f51fc06e140 0x7f51fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:24.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.100+0000 7f52127fc700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f51fc06e140 0x7f51fc0705f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5214111330 tx=0x7f5204009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:24.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.104+0000 7f51fb7fe700 1 -- 172.21.15.33:0/1363485142 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f520805bd50 con 0x7f52140fbbc0 2024-04-25T02:11:24.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.357+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f52141110e0 con 0x7f52140fbbc0 2024-04-25T02:11:24.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.358+0000 7f51fb7fe700 1 -- 172.21.15.33:0/1363485142 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f520805b8e0 con 0x7f52140fbbc0 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:11:24.366 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:11:24.367 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:11:24.368 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:11:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f51fc06e140 msgr2=0x7f51fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f51fc06e140 0x7f51fc0705f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5214111330 tx=0x7f5204009040 comp rx=0 tx=0).stop 2024-04-25T02:11:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 msgr2=0x7f5214115a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 0x7f5214115a90 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f5208009380 tx=0x7f5208009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 shutdown_connections 2024-04-25T02:11:24.369 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f51fc06e140 0x7f51fc0705f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f52140fbbc0 0x7f5214115a90 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 --2- 172.21.15.33:0/1363485142 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f52140fc4d0 0x7f5214115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 >> 172.21.15.33:0/1363485142 conn(0x7f52140f7870 msgr2=0x7f52140f7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 shutdown_connections 2024-04-25T02:11:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.360+0000 7f521944e700 1 -- 172.21.15.33:0/1363485142 wait complete. 2024-04-25T02:11:24.370 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:11:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.466+0000 7fc98cadc700 1 -- 172.21.15.33:0/314773084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988103780 msgr2=0x7fc9880fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.466+0000 7fc98cadc700 1 --2- 172.21.15.33:0/314773084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988103780 0x7fc9880fcdf0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc97c004940 tx=0x7fc97c009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.466+0000 7fc98cadc700 1 -- 172.21.15.33:0/314773084 shutdown_connections 2024-04-25T02:11:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.466+0000 7fc98cadc700 1 --2- 172.21.15.33:0/314773084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988103780 0x7fc9880fcdf0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.466+0000 7fc98cadc700 1 --2- 172.21.15.33:0/314773084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988102de0 0x7fc9881031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.466+0000 7fc98cadc700 1 -- 172.21.15.33:0/314773084 >> 172.21.15.33:0/314773084 conn(0x7fc9880f8990 msgr2=0x7fc9880fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:24.475 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.466+0000 7fc98cadc700 1 -- 172.21.15.33:0/314773084 shutdown_connections 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.466+0000 7fc98cadc700 1 -- 172.21.15.33:0/314773084 wait complete. 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.466+0000 7fc98cadc700 1 Processor -- start 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98cadc700 1 -- start start 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98cadc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988102de0 0x7fc988070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98cadc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988103780 0x7fc9880707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98cadc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9880721b0 con 0x7fc988102de0 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98cadc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc988070ce0 con 0x7fc988103780 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988102de0 0x7fc988070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988102de0 0x7fc988070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46816/0 (socket says 172.21.15.33:46816) 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98659c700 1 -- 172.21.15.33:0/3862332962 learned_addr learned my addr 172.21.15.33:0/3862332962 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc985d9b700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988103780 0x7fc9880707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98659c700 1 -- 172.21.15.33:0/3862332962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988103780 msgr2=0x7fc9880707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98659c700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988103780 0x7fc9880707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98659c700 1 -- 172.21.15.33:0/3862332962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97c005040 con 0x7fc988102de0 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.467+0000 7fc98659c700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988102de0 0x7fc988070260 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc97800c830 tx=0x7fc978004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.468+0000 7fc976ffd700 1 -- 172.21.15.33:0/3862332962 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc97800a760 con 0x7fc988102de0 2024-04-25T02:11:24.476 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.468+0000 7fc976ffd700 1 -- 172.21.15.33:0/3862332962 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc97800ad90 con 0x7fc988102de0 2024-04-25T02:11:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.468+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc988070fc0 con 0x7fc988102de0 2024-04-25T02:11:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.468+0000 7fc976ffd700 1 -- 172.21.15.33:0/3862332962 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9780113a0 con 0x7fc988102de0 2024-04-25T02:11:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.468+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98811a140 con 0x7fc988102de0 2024-04-25T02:11:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.469+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9880fe530 con 0x7fc988102de0 2024-04-25T02:11:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.469+0000 7fc976ffd700 1 -- 172.21.15.33:0/3862332962 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc978012400 con 0x7fc988102de0 2024-04-25T02:11:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.470+0000 7fc976ffd700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc97006e140 0x7fc9700705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:24.477 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.470+0000 7fc976ffd700 1 -- 172.21.15.33:0/3862332962 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc97808da90 con 0x7fc988102de0 2024-04-25T02:11:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.470+0000 7fc985d9b700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc97006e140 0x7fc9700705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:24.478 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.470+0000 7fc985d9b700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc97006e140 0x7fc9700705f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc988071990 tx=0x7fc97c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:24.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.474+0000 7fc976ffd700 1 -- 172.21.15.33:0/3862332962 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc978056f30 con 0x7fc988102de0 2024-04-25T02:11:24.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.702+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9880fa080 con 0x7fc97006e140 2024-04-25T02:11:24.710 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:24 smithi033 ceph-mon[111306]: from='client.35748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:24.710 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:24 smithi033 ceph-mon[111306]: from='client.35752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:24.710 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:24 smithi033 ceph-mon[111306]: pgmap v961: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:24.710 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:24 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4094903904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.703+0000 7fc976ffd700 1 -- 172.21.15.33:0/3862332962 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc9880fa080 con 0x7fc97006e140 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:11:24.711 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:11:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:11:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:11:24.712 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:11:24.712 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:11:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc97006e140 msgr2=0x7fc9700705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc97006e140 0x7fc9700705f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc988071990 tx=0x7fc97c009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988102de0 msgr2=0x7fc988070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988102de0 0x7fc988070260 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc97800c830 tx=0x7fc978004470 comp rx=0 tx=0).stop 2024-04-25T02:11:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 shutdown_connections 2024-04-25T02:11:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc97006e140 0x7fc9700705f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc988102de0 0x7fc988070260 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 --2- 172.21.15.33:0/3862332962 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc988103780 0x7fc9880707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.714 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 >> 172.21.15.33:0/3862332962 conn(0x7fc9880f8990 msgr2=0x7fc9880f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:24.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 shutdown_connections 2024-04-25T02:11:24.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.706+0000 7fc98cadc700 1 -- 172.21.15.33:0/3862332962 wait complete. 2024-04-25T02:11:24.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:24 smithi138 ceph-mon[81304]: from='client.35748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:24.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:24 smithi138 ceph-mon[81304]: from='client.35752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:24.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:24 smithi138 ceph-mon[81304]: pgmap v961: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:24.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:24 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4094903904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.810+0000 7fb13efbc700 1 -- 172.21.15.33:0/1080092786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 msgr2=0x7fb138103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.810+0000 7fb13efbc700 1 --2- 172.21.15.33:0/1080092786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 0x7fb138103220 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb12c004940 tx=0x7fb12c009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.811+0000 7fb13efbc700 1 -- 172.21.15.33:0/1080092786 shutdown_connections 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.811+0000 7fb13efbc700 1 --2- 172.21.15.33:0/1080092786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb138065640 0x7fb138065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.811+0000 7fb13efbc700 1 --2- 172.21.15.33:0/1080092786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 0x7fb138103220 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.811+0000 7fb13efbc700 1 -- 172.21.15.33:0/1080092786 >> 172.21.15.33:0/1080092786 conn(0x7fb138071d00 msgr2=0x7fb138072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.811+0000 7fb13efbc700 1 -- 172.21.15.33:0/1080092786 shutdown_connections 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.811+0000 7fb13efbc700 1 -- 172.21.15.33:0/1080092786 wait complete. 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.811+0000 7fb13efbc700 1 Processor -- start 2024-04-25T02:11:24.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.811+0000 7fb13efbc700 1 -- start start 2024-04-25T02:11:24.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb13efbc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb138065640 0x7fb1380fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb13efbc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 0x7fb1380fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb13efbc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380fc880 con 0x7fb138102e50 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb13efbc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380fc9c0 con 0x7fb138065640 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb137fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 0x7fb1380fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb137fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 0x7fb1380fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:46848/0 (socket says 172.21.15.33:46848) 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb137fff700 1 -- 172.21.15.33:0/1595992427 learned_addr learned my addr 172.21.15.33:0/1595992427 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb13cd58700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb138065640 0x7fb1380fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb137fff700 1 -- 172.21.15.33:0/1595992427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb138065640 msgr2=0x7fb1380fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb137fff700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb138065640 0x7fb1380fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb137fff700 1 -- 172.21.15.33:0/1595992427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c005040 con 0x7fb138102e50 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb13cd58700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb138065640 0x7fb1380fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb137fff700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 0x7fb1380fc2f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb124002a50 tx=0x7fb1240046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.812+0000 7fb1357fa700 1 -- 172.21.15.33:0/1595992427 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb124007cc0 con 0x7fb138102e50 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.813+0000 7fb1357fa700 1 -- 172.21.15.33:0/1595992427 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb12400f040 con 0x7fb138102e50 2024-04-25T02:11:24.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.813+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb138119e80 con 0x7fb138102e50 2024-04-25T02:11:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.813+0000 7fb1357fa700 1 -- 172.21.15.33:0/1595992427 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb124009640 con 0x7fb138102e50 2024-04-25T02:11:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.813+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13811a300 con 0x7fb138102e50 2024-04-25T02:11:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.814+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb138100100 con 0x7fb138102e50 2024-04-25T02:11:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.814+0000 7fb1357fa700 1 -- 172.21.15.33:0/1595992427 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb1240097c0 con 0x7fb138102e50 2024-04-25T02:11:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.815+0000 7fb1357fa700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb12806e140 0x7fb1280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:24.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.815+0000 7fb1357fa700 1 -- 172.21.15.33:0/1595992427 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb12408d8c0 con 0x7fb138102e50 2024-04-25T02:11:24.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.815+0000 7fb13cd58700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb12806e140 0x7fb1280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:24.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.815+0000 7fb13cd58700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb12806e140 0x7fb1280705f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb12c004560 tx=0x7fb12c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:24.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:24.819+0000 7fb1357fa700 1 -- 172.21.15.33:0/1595992427 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb124056c40 con 0x7fb138102e50 2024-04-25T02:11:25.120 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.113+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb138109ef0 con 0x7fb138102e50 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.113+0000 7fb1357fa700 1 -- 172.21.15.33:0/1595992427 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fb12405a300 con 0x7fb138102e50 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:11:25.121 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:11:25.122 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:11:25.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb12806e140 msgr2=0x7fb1280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:25.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb12806e140 0x7fb1280705f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb12c004560 tx=0x7fb12c009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:25.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 msgr2=0x7fb1380fc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:25.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 0x7fb1380fc2f0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb124002a50 tx=0x7fb1240046b0 comp rx=0 tx=0).stop 2024-04-25T02:11:25.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 shutdown_connections 2024-04-25T02:11:25.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb12806e140 0x7fb1280705f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:25.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb138065640 0x7fb1380fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:25.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 --2- 172.21.15.33:0/1595992427 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb138102e50 0x7fb1380fc2f0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:25.123 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 >> 172.21.15.33:0/1595992427 conn(0x7fb138071d00 msgr2=0x7fb1380fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:25.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 shutdown_connections 2024-04-25T02:11:25.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:25.115+0000 7fb13efbc700 1 -- 172.21.15.33:0/1595992427 wait complete. 2024-04-25T02:11:25.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:25 smithi033 ceph-mon[111306]: from='client.35756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:25.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:25 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1363485142' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:11:25.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:25 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1595992427' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:11:25.524 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:11:25.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:25 smithi138 ceph-mon[81304]: from='client.35756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:25.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:25 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1363485142' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:11:25.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:25 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1595992427' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:11:25.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:11:26.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:26 smithi033 ceph-mon[111306]: from='client.35768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:26.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:26 smithi033 ceph-mon[111306]: pgmap v962: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:26.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:26 smithi138 ceph-mon[81304]: from='client.35768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:26.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:26 smithi138 ceph-mon[81304]: pgmap v962: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:28.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:28 smithi033 ceph-mon[111306]: pgmap v963: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:28.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:28 smithi138 ceph-mon[81304]: pgmap v963: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:30.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:30 smithi033 ceph-mon[111306]: pgmap v964: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:30.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:30 smithi138 ceph-mon[81304]: pgmap v964: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:32.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:32 smithi033 ceph-mon[111306]: pgmap v965: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:32.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:32 smithi138 ceph-mon[81304]: pgmap v965: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:34.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:34 smithi033 ceph-mon[111306]: pgmap v966: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:34.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:34 smithi138 ceph-mon[81304]: pgmap v966: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:36.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:36 smithi033 ceph-mon[111306]: pgmap v967: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:36.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:36 smithi138 ceph-mon[81304]: pgmap v967: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:38.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:38 smithi033 ceph-mon[111306]: pgmap v968: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:38.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:38 smithi138 ceph-mon[81304]: pgmap v968: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:40.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:40 smithi033 ceph-mon[111306]: pgmap v969: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:40.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:11:40.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:40 smithi138 ceph-mon[81304]: pgmap v969: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:40.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:11:42.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:42 smithi033 ceph-mon[111306]: pgmap v970: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:42.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:42 smithi138 ceph-mon[81304]: pgmap v970: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:44.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:44 smithi033 ceph-mon[111306]: pgmap v971: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:44.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:44 smithi138 ceph-mon[81304]: pgmap v971: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:46.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:46 smithi033 ceph-mon[111306]: pgmap v972: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:46.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:46 smithi138 ceph-mon[81304]: pgmap v972: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:48.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:48 smithi033 ceph-mon[111306]: pgmap v973: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:48.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:48 smithi138 ceph-mon[81304]: pgmap v973: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:50.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:50 smithi033 ceph-mon[111306]: pgmap v974: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:50.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:50 smithi138 ceph-mon[81304]: pgmap v974: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:52.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:52 smithi033 ceph-mon[111306]: pgmap v975: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:52.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:52 smithi138 ceph-mon[81304]: pgmap v975: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:54.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:54 smithi033 ceph-mon[111306]: pgmap v976: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:54.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:54 smithi138 ceph-mon[81304]: pgmap v976: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.237+0000 7f6d0e18e700 1 -- 172.21.15.33:0/3097765200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 msgr2=0x7f6d08102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.237+0000 7f6d0e18e700 1 --2- 172.21.15.33:0/3097765200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 0x7f6d08102160 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8004940 tx=0x7f6cf8009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.237+0000 7f6d0e18e700 1 -- 172.21.15.33:0/3097765200 shutdown_connections 2024-04-25T02:11:55.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.237+0000 7f6d0e18e700 1 --2- 172.21.15.33:0/3097765200 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d080fbd40 0x7f6d080fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.237+0000 7f6d0e18e700 1 --2- 172.21.15.33:0/3097765200 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 0x7f6d08102160 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.237+0000 7f6d0e18e700 1 -- 172.21.15.33:0/3097765200 >> 172.21.15.33:0/3097765200 conn(0x7f6d08071d00 msgr2=0x7f6d08072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.237+0000 7f6d0e18e700 1 -- 172.21.15.33:0/3097765200 shutdown_connections 2024-04-25T02:11:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.237+0000 7f6d0e18e700 1 -- 172.21.15.33:0/3097765200 wait complete. 2024-04-25T02:11:55.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.238+0000 7f6d0e18e700 1 Processor -- start 2024-04-25T02:11:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.238+0000 7f6d0e18e700 1 -- start start 2024-04-25T02:11:55.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d0e18e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d080fbd40 0x7f6d08102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d0e18e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 0x7f6d08103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d0e18e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d081035f0 con 0x7f6d08101d90 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d0e18e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d08103730 con 0x7f6d080fbd40 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d06ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 0x7f6d08103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d06ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 0x7f6d08103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40546/0 (socket says 172.21.15.33:40546) 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d06ffd700 1 -- 172.21.15.33:0/4168000310 learned_addr learned my addr 172.21.15.33:0/4168000310 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d077fe700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d080fbd40 0x7f6d08102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d06ffd700 1 -- 172.21.15.33:0/4168000310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d080fbd40 msgr2=0x7f6d08102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d06ffd700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d080fbd40 0x7f6d08102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d06ffd700 1 -- 172.21.15.33:0/4168000310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf8005040 con 0x7f6d08101d90 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6d06ffd700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 0x7f6d08103060 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc002a50 tx=0x7f6cfc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:55.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.239+0000 7f6ceffff700 1 -- 172.21.15.33:0/4168000310 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cfc007cc0 con 0x7f6d08101d90 2024-04-25T02:11:55.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.240+0000 7f6ceffff700 1 -- 172.21.15.33:0/4168000310 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cfc00f040 con 0x7f6d08101d90 2024-04-25T02:11:55.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.240+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d08119e80 con 0x7f6d08101d90 2024-04-25T02:11:55.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.240+0000 7f6ceffff700 1 -- 172.21.15.33:0/4168000310 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cfc009640 con 0x7f6d08101d90 2024-04-25T02:11:55.249 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.240+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0811a300 con 0x7f6d08101d90 2024-04-25T02:11:55.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.241+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d080fd430 con 0x7f6d08101d90 2024-04-25T02:11:55.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.241+0000 7f6ceffff700 1 -- 172.21.15.33:0/4168000310 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6cfc0097a0 con 0x7f6d08101d90 2024-04-25T02:11:55.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.242+0000 7f6ceffff700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cf006e140 0x7f6cf00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:55.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.242+0000 7f6ceffff700 1 -- 172.21.15.33:0/4168000310 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6cfc08d9c0 con 0x7f6d08101d90 2024-04-25T02:11:55.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.242+0000 7f6d077fe700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cf006e140 0x7f6cf00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:55.250 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.242+0000 7f6d077fe700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cf006e140 0x7f6cf00705f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800b3b0 tx=0x7f6cf8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:55.254 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.247+0000 7f6ceffff700 1 -- 172.21.15.33:0/4168000310 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6cfc056ca0 con 0x7f6d08101d90 2024-04-25T02:11:55.481 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.473+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d0804bed0 con 0x7f6cf006e140 2024-04-25T02:11:55.482 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.474+0000 7f6ceffff700 1 -- 172.21.15.33:0/4168000310 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6d0804bed0 con 0x7f6cf006e140 2024-04-25T02:11:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.476+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cf006e140 msgr2=0x7f6cf00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.476+0000 7f6d0e18e700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cf006e140 0x7f6cf00705f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6cf800b3b0 tx=0x7f6cf8009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.476+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 msgr2=0x7f6d08103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.476+0000 7f6d0e18e700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 0x7f6d08103060 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc002a50 tx=0x7f6cfc0046b0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.477+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 shutdown_connections 2024-04-25T02:11:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.477+0000 7f6d0e18e700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6cf006e140 0x7f6cf00705f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.484 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.477+0000 7f6d0e18e700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6d080fbd40 0x7f6d08102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.477+0000 7f6d0e18e700 1 --2- 172.21.15.33:0/4168000310 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6d08101d90 0x7f6d08103060 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.477+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 >> 172.21.15.33:0/4168000310 conn(0x7f6d08071d00 msgr2=0x7f6d080fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.477+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 shutdown_connections 2024-04-25T02:11:55.485 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.477+0000 7f6d0e18e700 1 -- 172.21.15.33:0/4168000310 wait complete. 2024-04-25T02:11:55.494 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:11:55.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:11:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.582+0000 7f7d1052d700 1 -- 172.21.15.33:0/2352331637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d081026b0 msgr2=0x7f7d080fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.582+0000 7f7d1052d700 1 --2- 172.21.15.33:0/2352331637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d081026b0 0x7f7d080fbd20 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8004940 tx=0x7f7cf8009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 -- 172.21.15.33:0/2352331637 shutdown_connections 2024-04-25T02:11:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 --2- 172.21.15.33:0/2352331637 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d081026b0 0x7f7d080fbd20 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 --2- 172.21.15.33:0/2352331637 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d08101d10 0x7f7d081020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 -- 172.21.15.33:0/2352331637 >> 172.21.15.33:0/2352331637 conn(0x7f7d080f7860 msgr2=0x7f7d080f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 -- 172.21.15.33:0/2352331637 shutdown_connections 2024-04-25T02:11:55.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 -- 172.21.15.33:0/2352331637 wait complete. 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 Processor -- start 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 -- start start 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08101d10 0x7f7d08112830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d081026b0 0x7f7d08112d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d081133c0 con 0x7f7d08101d10 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.583+0000 7f7d1052d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0810c9c0 con 0x7f7d081026b0 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d0e2c9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08101d10 0x7f7d08112830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d0e2c9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08101d10 0x7f7d08112830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:40568/0 (socket says 172.21.15.33:40568) 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d0e2c9700 1 -- 172.21.15.33:0/4142284917 learned_addr learned my addr 172.21.15.33:0/4142284917 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d0e2c9700 1 -- 172.21.15.33:0/4142284917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d081026b0 msgr2=0x7f7d08112d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d0dac8700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d081026b0 0x7f7d08112d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d0e2c9700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d081026b0 0x7f7d08112d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.593 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d0e2c9700 1 -- 172.21.15.33:0/4142284917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf8005040 con 0x7f7d08101d10 2024-04-25T02:11:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d0dac8700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d081026b0 0x7f7d08112d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:11:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d0e2c9700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08101d10 0x7f7d08112830 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7d04002a80 tx=0x7f7d040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7cfeffd700 1 -- 172.21.15.33:0/4142284917 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d04007cc0 con 0x7f7d08101d10 2024-04-25T02:11:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7cfeffd700 1 -- 172.21.15.33:0/4142284917 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d0400f040 con 0x7f7d08101d10 2024-04-25T02:11:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7cfeffd700 1 -- 172.21.15.33:0/4142284917 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d040096f0 con 0x7f7d08101d10 2024-04-25T02:11:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.584+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0810cca0 con 0x7f7d08101d10 2024-04-25T02:11:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.585+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0810d260 con 0x7f7d08101d10 2024-04-25T02:11:55.594 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.585+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d080fd460 con 0x7f7d08101d10 2024-04-25T02:11:55.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.586+0000 7f7cfeffd700 1 -- 172.21.15.33:0/4142284917 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7d0400d690 con 0x7f7d08101d10 2024-04-25T02:11:55.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.586+0000 7f7cfeffd700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7cf406e140 0x7f7cf40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:55.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.586+0000 7f7cfeffd700 1 -- 172.21.15.33:0/4142284917 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7d0408d1f0 con 0x7f7d08101d10 2024-04-25T02:11:55.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.586+0000 7f7d0dac8700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7cf406e140 0x7f7cf40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:55.595 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.587+0000 7f7d0dac8700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7cf406e140 0x7f7cf40705f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800b3b0 tx=0x7f7cf8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:55.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.591+0000 7f7cfeffd700 1 -- 172.21.15.33:0/4142284917 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7d0405a120 con 0x7f7d08101d10 2024-04-25T02:11:55.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:11:55.826 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.819+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d08108df0 con 0x7f7cf406e140 2024-04-25T02:11:55.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.820+0000 7f7cfeffd700 1 -- 172.21.15.33:0/4142284917 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7d08108df0 con 0x7f7cf406e140 2024-04-25T02:11:55.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7cf406e140 msgr2=0x7f7cf40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7cf406e140 0x7f7cf40705f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800b3b0 tx=0x7f7cf8009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:55.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08101d10 msgr2=0x7f7d08112830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08101d10 0x7f7d08112830 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7d04002a80 tx=0x7f7d040046b0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 shutdown_connections 2024-04-25T02:11:55.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7cf406e140 0x7f7cf40705f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7d08101d10 0x7f7d08112830 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 --2- 172.21.15.33:0/4142284917 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7d081026b0 0x7f7d08112d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 >> 172.21.15.33:0/4142284917 conn(0x7f7d080f7860 msgr2=0x7f7d080f8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:55.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 shutdown_connections 2024-04-25T02:11:55.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.822+0000 7f7d1052d700 1 -- 172.21.15.33:0/4142284917 wait complete. 2024-04-25T02:11:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.925+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/3447254098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 msgr2=0x7fcf68101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.925+0000 7fcf6fbd3700 1 --2- 172.21.15.33:0/3447254098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 0x7fcf68101a30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fcf58004940 tx=0x7fcf58009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:55.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/3447254098 shutdown_connections 2024-04-25T02:11:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 --2- 172.21.15.33:0/3447254098 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 0x7fcf68101a30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 --2- 172.21.15.33:0/3447254098 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf680fb5a0 0x7fcf680fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/3447254098 >> 172.21.15.33:0/3447254098 conn(0x7fcf68073350 msgr2=0x7fcf68073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/3447254098 shutdown_connections 2024-04-25T02:11:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/3447254098 wait complete. 2024-04-25T02:11:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 Processor -- start 2024-04-25T02:11:55.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 -- start start 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf680fb5a0 0x7fcf681139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 0x7fcf68113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.926+0000 7fcf6fbd3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf68114530 con 0x7fcf680fbf40 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6fbd3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6810db50 con 0x7fcf680fb5a0 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6d16e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 0x7fcf68113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6d16e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 0x7fcf68113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48248/0 (socket says 172.21.15.33:48248) 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6d16e700 1 -- 172.21.15.33:0/682370443 learned_addr learned my addr 172.21.15.33:0/682370443 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6d96f700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf680fb5a0 0x7fcf681139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6d16e700 1 -- 172.21.15.33:0/682370443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf680fb5a0 msgr2=0x7fcf681139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:55.936 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6d16e700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf680fb5a0 0x7fcf681139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6d16e700 1 -- 172.21.15.33:0/682370443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58005040 con 0x7fcf680fbf40 2024-04-25T02:11:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6d16e700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 0x7fcf68113ee0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcf58000c00 tx=0x7fcf58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/682370443 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf5800e040 con 0x7fcf680fbf40 2024-04-25T02:11:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/682370443 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf58013420 con 0x7fcf680fbf40 2024-04-25T02:11:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/682370443 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf5801c630 con 0x7fcf680fbf40 2024-04-25T02:11:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.927+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6810ddd0 con 0x7fcf680fbf40 2024-04-25T02:11:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.928+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6810e310 con 0x7fcf680fbf40 2024-04-25T02:11:55.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.928+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf680fe670 con 0x7fcf680fbf40 2024-04-25T02:11:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.929+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/682370443 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcf58013590 con 0x7fcf680fbf40 2024-04-25T02:11:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.929+0000 7fcf5e7fc700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf5406e140 0x7fcf540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.930+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/682370443 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcf5808ec50 con 0x7fcf680fbf40 2024-04-25T02:11:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.930+0000 7fcf6d96f700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf5406e140 0x7fcf540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:55.938 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.930+0000 7fcf6d96f700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf5406e140 0x7fcf540705f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009fd0 tx=0x7fcf64009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:55.943 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:55.936+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/682370443 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcf580580f0 con 0x7fcf680fbf40 2024-04-25T02:11:56.159 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.151+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf6810e580 con 0x7fcf5406e140 2024-04-25T02:11:56.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.161+0000 7fcf5e7fc700 1 -- 172.21.15.33:0/682370443 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcf6810e580 con 0x7fcf5406e140 2024-04-25T02:11:56.169 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:11:56.169 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (33m) 10m ago 44m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:11:56.169 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (30m) 10m ago 44m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:11:56.169 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (30m) 10m ago 43m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:11:56.169 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (32m) 10m ago 43m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:11:56.169 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (26m) 10m ago 38m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (26m) 10m ago 38m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (26m) 10m ago 38m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (10m) 10m ago 38m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (34m) 10m ago 45m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (33m) 10m ago 43m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (31m) 10m ago 45m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (30m) 10m ago 43m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (33m) 10m ago 44m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (33m) 10m ago 43m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (29m) 10m ago 42m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (29m) 10m ago 42m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:11:56.170 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (28m) 10m ago 41m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:11:56.171 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (28m) 10m ago 41m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:11:56.171 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (27m) 10m ago 40m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:11:56.171 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (26m) 10m ago 40m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:11:56.171 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (33m) 10m ago 43m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.163+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf5406e140 msgr2=0x7fcf540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.163+0000 7fcf6fbd3700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf5406e140 0x7fcf540705f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009fd0 tx=0x7fcf64009e50 comp rx=0 tx=0).stop 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.163+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 msgr2=0x7fcf68113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.163+0000 7fcf6fbd3700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 0x7fcf68113ee0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fcf58000c00 tx=0x7fcf58009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.163+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 shutdown_connections 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.163+0000 7fcf6fbd3700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf5406e140 0x7fcf540705f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.163+0000 7fcf6fbd3700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf680fb5a0 0x7fcf681139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.163+0000 7fcf6fbd3700 1 --2- 172.21.15.33:0/682370443 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf680fbf40 0x7fcf68113ee0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.163+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 >> 172.21.15.33:0/682370443 conn(0x7fcf68073350 msgr2=0x7fcf68100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.164+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 shutdown_connections 2024-04-25T02:11:56.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.164+0000 7fcf6fbd3700 1 -- 172.21.15.33:0/682370443 wait complete. 2024-04-25T02:11:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.267+0000 7f83d6135700 1 -- 172.21.15.33:0/1486726378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 msgr2=0x7f83d00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.267+0000 7f83d6135700 1 --2- 172.21.15.33:0/1486726378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 0x7f83d00fcdf0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f83b8004940 tx=0x7f83b8009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.267+0000 7f83d6135700 1 -- 172.21.15.33:0/1486726378 shutdown_connections 2024-04-25T02:11:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.267+0000 7f83d6135700 1 --2- 172.21.15.33:0/1486726378 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 0x7f83d00fcdf0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.267+0000 7f83d6135700 1 --2- 172.21.15.33:0/1486726378 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d0102de0 0x7f83d01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.267+0000 7f83d6135700 1 -- 172.21.15.33:0/1486726378 >> 172.21.15.33:0/1486726378 conn(0x7f83d00f8990 msgr2=0x7f83d00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.267+0000 7f83d6135700 1 -- 172.21.15.33:0/1486726378 shutdown_connections 2024-04-25T02:11:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.267+0000 7f83d6135700 1 -- 172.21.15.33:0/1486726378 wait complete. 2024-04-25T02:11:56.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.267+0000 7f83d6135700 1 Processor -- start 2024-04-25T02:11:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83d6135700 1 -- start start 2024-04-25T02:11:56.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83d6135700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d0102de0 0x7f83d0113900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83d6135700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 0x7f83d0113e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83d6135700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d0114490 con 0x7f83d0103780 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83d6135700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d010da90 con 0x7f83d0102de0 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83ceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 0x7f83d0113e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83ceffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 0x7f83d0113e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48262/0 (socket says 172.21.15.33:48262) 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83ceffd700 1 -- 172.21.15.33:0/614331184 learned_addr learned my addr 172.21.15.33:0/614331184 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83cf7fe700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d0102de0 0x7f83d0113900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83ceffd700 1 -- 172.21.15.33:0/614331184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d0102de0 msgr2=0x7f83d0113900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83ceffd700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d0102de0 0x7f83d0113900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.268+0000 7f83ceffd700 1 -- 172.21.15.33:0/614331184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b8005040 con 0x7f83d0103780 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.269+0000 7f83ceffd700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 0x7f83d0113e40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000c00 tx=0x7f83b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.269+0000 7f83c7fff700 1 -- 172.21.15.33:0/614331184 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b8021070 con 0x7f83d0103780 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.269+0000 7f83c7fff700 1 -- 172.21.15.33:0/614331184 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83b8013c00 con 0x7f83d0103780 2024-04-25T02:11:56.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.269+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d010dd10 con 0x7f83d0103780 2024-04-25T02:11:56.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.269+0000 7f83c7fff700 1 -- 172.21.15.33:0/614331184 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83b800e070 con 0x7f83d0103780 2024-04-25T02:11:56.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.269+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d010e1f0 con 0x7f83d0103780 2024-04-25T02:11:56.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.270+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d00fe530 con 0x7f83d0103780 2024-04-25T02:11:56.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.270+0000 7f83c7fff700 1 -- 172.21.15.33:0/614331184 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f83b801ca60 con 0x7f83d0103780 2024-04-25T02:11:56.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.271+0000 7f83c7fff700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83bc06e140 0x7f83bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:56.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.271+0000 7f83c7fff700 1 -- 172.21.15.33:0/614331184 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f83b808ef30 con 0x7f83d0103780 2024-04-25T02:11:56.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.271+0000 7f83cf7fe700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83bc06e140 0x7f83bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:56.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.271+0000 7f83cf7fe700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83bc06e140 0x7f83bc0705f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f83c0001470 tx=0x7f83c000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:56.283 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.275+0000 7f83c7fff700 1 -- 172.21.15.33:0/614331184 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f83b80c3740 con 0x7f83d0103780 2024-04-25T02:11:56.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.572+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f83d010ef50 con 0x7f83d0103780 2024-04-25T02:11:56.579 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:56 smithi033 ceph-mon[111306]: pgmap v977: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:56.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.572+0000 7f83c7fff700 1 -- 172.21.15.33:0/614331184 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f83b805be60 con 0x7f83d0103780 2024-04-25T02:11:56.580 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:11:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:11:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:11:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:11:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:11:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:11:56.580 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:11:56.581 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:11:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83bc06e140 msgr2=0x7f83bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83bc06e140 0x7f83bc0705f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f83c0001470 tx=0x7f83c000b040 comp rx=0 tx=0).stop 2024-04-25T02:11:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 msgr2=0x7f83d0113e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 0x7f83d0113e40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000c00 tx=0x7f83b8009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 shutdown_connections 2024-04-25T02:11:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83bc06e140 0x7f83bc0705f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f83d0102de0 0x7f83d0113900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 --2- 172.21.15.33:0/614331184 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f83d0103780 0x7f83d0113e40 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 >> 172.21.15.33:0/614331184 conn(0x7f83d00f8990 msgr2=0x7f83d00f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 shutdown_connections 2024-04-25T02:11:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.575+0000 7f83d6135700 1 -- 172.21.15.33:0/614331184 wait complete. 2024-04-25T02:11:56.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.680+0000 7fe997a70700 1 -- 172.21.15.33:0/3157699836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 msgr2=0x7fe990103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.680+0000 7fe997a70700 1 --2- 172.21.15.33:0/3157699836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 0x7fe990103220 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fe98c004940 tx=0x7fe98c009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:56.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.680+0000 7fe997a70700 1 -- 172.21.15.33:0/3157699836 shutdown_connections 2024-04-25T02:11:56.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.680+0000 7fe997a70700 1 --2- 172.21.15.33:0/3157699836 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe990065640 0x7fe990065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.680+0000 7fe997a70700 1 --2- 172.21.15.33:0/3157699836 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 0x7fe990103220 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.680+0000 7fe997a70700 1 -- 172.21.15.33:0/3157699836 >> 172.21.15.33:0/3157699836 conn(0x7fe990071d00 msgr2=0x7fe990072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:56.688 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.681+0000 7fe997a70700 1 -- 172.21.15.33:0/3157699836 shutdown_connections 2024-04-25T02:11:56.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.681+0000 7fe997a70700 1 -- 172.21.15.33:0/3157699836 wait complete. 2024-04-25T02:11:56.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.681+0000 7fe997a70700 1 Processor -- start 2024-04-25T02:11:56.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.681+0000 7fe997a70700 1 -- start start 2024-04-25T02:11:56.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe997a70700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe990065640 0x7fe990113ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:56.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe997a70700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 0x7fe990114020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:56.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe997a70700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe990114670 con 0x7fe990102e50 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe997a70700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99010da60 con 0x7fe990065640 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe99500b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 0x7fe990114020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe99500b700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 0x7fe990114020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48288/0 (socket says 172.21.15.33:48288) 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe99500b700 1 -- 172.21.15.33:0/3670409517 learned_addr learned my addr 172.21.15.33:0/3670409517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe99580c700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe990065640 0x7fe990113ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe99500b700 1 -- 172.21.15.33:0/3670409517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe990065640 msgr2=0x7fe990113ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe99500b700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe990065640 0x7fe990113ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe99500b700 1 -- 172.21.15.33:0/3670409517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c005040 con 0x7fe990102e50 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe99500b700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 0x7fe990114020 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe98000eb40 tx=0x7fe98000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe9867fc700 1 -- 172.21.15.33:0/3670409517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe980007cc0 con 0x7fe990102e50 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe9867fc700 1 -- 172.21.15.33:0/3670409517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe98000f040 con 0x7fe990102e50 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99010dce0 con 0x7fe990102e50 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe9867fc700 1 -- 172.21.15.33:0/3670409517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe980009620 con 0x7fe990102e50 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.682+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99010e220 con 0x7fe990102e50 2024-04-25T02:11:56.691 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.683+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe990100100 con 0x7fe990102e50 2024-04-25T02:11:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.684+0000 7fe9867fc700 1 -- 172.21.15.33:0/3670409517 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe980009780 con 0x7fe990102e50 2024-04-25T02:11:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.684+0000 7fe9867fc700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe97c06e150 0x7fe97c070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.684+0000 7fe9867fc700 1 -- 172.21.15.33:0/3670409517 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe98008d850 con 0x7fe990102e50 2024-04-25T02:11:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.684+0000 7fe99580c700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe97c06e150 0x7fe97c070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:56.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.685+0000 7fe99580c700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe97c06e150 0x7fe97c070600 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe98c007f50 tx=0x7fe98c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:56.696 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.689+0000 7fe9867fc700 1 -- 172.21.15.33:0/3670409517 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe98005a710 con 0x7fe990102e50 2024-04-25T02:11:56.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:56 smithi138 ceph-mon[81304]: pgmap v977: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.944+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe99010f130 con 0x7fe990102e50 2024-04-25T02:11:56.951 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.944+0000 7fe9867fc700 1 -- 172.21.15.33:0/3670409517 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fe98005a2a0 con 0x7fe990102e50 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:11:56.952 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:11:56.953 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:11:56.954 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:11:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe97c06e150 msgr2=0x7fe97c070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe97c06e150 0x7fe97c070600 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe98c007f50 tx=0x7fe98c009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 msgr2=0x7fe990114020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 0x7fe990114020 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fe98000eb40 tx=0x7fe98000b040 comp rx=0 tx=0).stop 2024-04-25T02:11:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 shutdown_connections 2024-04-25T02:11:56.955 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe97c06e150 0x7fe97c070600 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe990065640 0x7fe990113ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 --2- 172.21.15.33:0/3670409517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe990102e50 0x7fe990114020 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 >> 172.21.15.33:0/3670409517 conn(0x7fe990071d00 msgr2=0x7fe9900faf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 shutdown_connections 2024-04-25T02:11:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:56.947+0000 7fe997a70700 1 -- 172.21.15.33:0/3670409517 wait complete. 2024-04-25T02:11:56.956 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:11:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.052+0000 7f46c2912700 1 -- 172.21.15.33:0/2689765256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 msgr2=0x7f46bc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.052+0000 7f46c2912700 1 --2- 172.21.15.33:0/2689765256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 0x7f46bc1031b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f46ac004940 tx=0x7f46ac009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.052+0000 7f46c2912700 1 -- 172.21.15.33:0/2689765256 shutdown_connections 2024-04-25T02:11:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.052+0000 7f46c2912700 1 --2- 172.21.15.33:0/2689765256 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46bc103780 0x7f46bc0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.052+0000 7f46c2912700 1 --2- 172.21.15.33:0/2689765256 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 0x7f46bc1031b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.052+0000 7f46c2912700 1 -- 172.21.15.33:0/2689765256 >> 172.21.15.33:0/2689765256 conn(0x7f46bc0f8990 msgr2=0x7f46bc0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.052+0000 7f46c2912700 1 -- 172.21.15.33:0/2689765256 shutdown_connections 2024-04-25T02:11:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.052+0000 7f46c2912700 1 -- 172.21.15.33:0/2689765256 wait complete. 2024-04-25T02:11:57.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.053+0000 7f46c2912700 1 Processor -- start 2024-04-25T02:11:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.053+0000 7f46c2912700 1 -- start start 2024-04-25T02:11:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.053+0000 7f46c2912700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 0x7f46bc113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.053+0000 7f46c2912700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46bc103780 0x7f46bc113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.053+0000 7f46c2912700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc114500 con 0x7f46bc102de0 2024-04-25T02:11:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.053+0000 7f46c2912700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46bc10db50 con 0x7f46bc103780 2024-04-25T02:11:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.053+0000 7f46bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 0x7f46bc113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:57.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.053+0000 7f46bbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 0x7f46bc113970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48308/0 (socket says 172.21.15.33:48308) 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.053+0000 7f46bbfff700 1 -- 172.21.15.33:0/3387070242 learned_addr learned my addr 172.21.15.33:0/3387070242 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46bb7fe700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46bc103780 0x7f46bc113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46bbfff700 1 -- 172.21.15.33:0/3387070242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46bc103780 msgr2=0x7f46bc113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46bbfff700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46bc103780 0x7f46bc113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46bbfff700 1 -- 172.21.15.33:0/3387070242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac005040 con 0x7f46bc102de0 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46bbfff700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 0x7f46bc113970 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b3b0 tx=0x7f46ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46b8ff9700 1 -- 172.21.15.33:0/3387070242 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46ac00e040 con 0x7f46bc102de0 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46bc10ddd0 con 0x7f46bc102de0 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46b8ff9700 1 -- 172.21.15.33:0/3387070242 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46ac013420 con 0x7f46bc102de0 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46b8ff9700 1 -- 172.21.15.33:0/3387070242 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46ac01c630 con 0x7f46bc102de0 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.054+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46bc10e390 con 0x7f46bc102de0 2024-04-25T02:11:57.063 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.055+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46bc1000f0 con 0x7f46bc102de0 2024-04-25T02:11:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.056+0000 7f46b8ff9700 1 -- 172.21.15.33:0/3387070242 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f46ac013590 con 0x7f46bc102de0 2024-04-25T02:11:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.056+0000 7f46b8ff9700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46a806e140 0x7f46a80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:57.064 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.056+0000 7f46b8ff9700 1 -- 172.21.15.33:0/3387070242 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f46ac08ebf0 con 0x7f46bc102de0 2024-04-25T02:11:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.056+0000 7f46bb7fe700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46a806e140 0x7f46a80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:57.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.057+0000 7f46bb7fe700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46a806e140 0x7f46a80705f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f46bc10ecc0 tx=0x7f46a400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:57.069 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.061+0000 7f46b8ff9700 1 -- 172.21.15.33:0/3387070242 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f46ac05bbc0 con 0x7f46bc102de0 2024-04-25T02:11:57.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.287+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46bc10f030 con 0x7f46a806e140 2024-04-25T02:11:57.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.288+0000 7f46b8ff9700 1 -- 172.21.15.33:0/3387070242 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f46bc10f030 con 0x7f46a806e140 2024-04-25T02:11:57.295 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:11:57.295 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:11:57.295 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:11:57.295 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:11:57.295 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:11:57.295 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:11:57.296 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:11:57.296 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:11:57.296 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:11:57.296 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:11:57.296 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:11:57.296 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:11:57.296 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:11:57.296 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:11:57.296 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:11:57.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46a806e140 msgr2=0x7f46a80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:57.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46a806e140 0x7f46a80705f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f46bc10ecc0 tx=0x7f46a400b040 comp rx=0 tx=0).stop 2024-04-25T02:11:57.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 msgr2=0x7f46bc113970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:57.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 0x7f46bc113970 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b3b0 tx=0x7f46ac009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 shutdown_connections 2024-04-25T02:11:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46a806e140 0x7f46a80705f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46bc102de0 0x7f46bc113970 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 --2- 172.21.15.33:0/3387070242 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46bc103780 0x7f46bc113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 >> 172.21.15.33:0/3387070242 conn(0x7f46bc0f8990 msgr2=0x7f46bc107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 shutdown_connections 2024-04-25T02:11:57.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.290+0000 7f46c2912700 1 -- 172.21.15.33:0/3387070242 wait complete. 2024-04-25T02:11:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.394+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/2329185745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070280 msgr2=0x7f5b24070650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.394+0000 7f5b2a4ac700 1 --2- 172.21.15.33:0/2329185745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070280 0x7f5b24070650 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f5b14004940 tx=0x7f5b14009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.394+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/2329185745 shutdown_connections 2024-04-25T02:11:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.394+0000 7f5b2a4ac700 1 --2- 172.21.15.33:0/2329185745 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24070b90 0x7f5b24109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.394+0000 7f5b2a4ac700 1 --2- 172.21.15.33:0/2329185745 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070280 0x7f5b24070650 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.394+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/2329185745 >> 172.21.15.33:0/2329185745 conn(0x7f5b240f8990 msgr2=0x7f5b240fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.394+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/2329185745 shutdown_connections 2024-04-25T02:11:57.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.394+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/2329185745 wait complete. 2024-04-25T02:11:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.394+0000 7f5b2a4ac700 1 Processor -- start 2024-04-25T02:11:57.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b2a4ac700 1 -- start start 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b2a4ac700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24070280 0x7f5b24115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b2a4ac700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070b90 0x7f5b241160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b2a4ac700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b24116720 con 0x7f5b24070b90 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b2a4ac700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2410fd00 con 0x7f5b24070280 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070b90 0x7f5b241160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b237fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070b90 0x7f5b241160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48318/0 (socket says 172.21.15.33:48318) 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b237fe700 1 -- 172.21.15.33:0/3847416380 learned_addr learned my addr 172.21.15.33:0/3847416380 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b23fff700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24070280 0x7f5b24115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b237fe700 1 -- 172.21.15.33:0/3847416380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24070280 msgr2=0x7f5b24115b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b237fe700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24070280 0x7f5b24115b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b237fe700 1 -- 172.21.15.33:0/3847416380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14005040 con 0x7f5b24070b90 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.395+0000 7f5b237fe700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070b90 0x7f5b241160d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5b18002a50 tx=0x7f5b180046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.396+0000 7f5b20ff9700 1 -- 172.21.15.33:0/3847416380 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b18007cc0 con 0x7f5b24070b90 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.396+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2410ffe0 con 0x7f5b24070b90 2024-04-25T02:11:57.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.396+0000 7f5b20ff9700 1 -- 172.21.15.33:0/3847416380 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b1800f040 con 0x7f5b24070b90 2024-04-25T02:11:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.396+0000 7f5b20ff9700 1 -- 172.21.15.33:0/3847416380 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b1800da30 con 0x7f5b24070b90 2024-04-25T02:11:57.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.396+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b24110520 con 0x7f5b24070b90 2024-04-25T02:11:57.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.397+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b240719c0 con 0x7f5b24070b90 2024-04-25T02:11:57.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.397+0000 7f5b20ff9700 1 -- 172.21.15.33:0/3847416380 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5b1800db90 con 0x7f5b24070b90 2024-04-25T02:11:57.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.398+0000 7f5b20ff9700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5b0c06e140 0x7f5b0c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:11:57.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.398+0000 7f5b20ff9700 1 -- 172.21.15.33:0/3847416380 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5b1808d8c0 con 0x7f5b24070b90 2024-04-25T02:11:57.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.398+0000 7f5b23fff700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5b0c06e140 0x7f5b0c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:11:57.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.398+0000 7f5b23fff700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5b0c06e140 0x7f5b0c0705f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5b14004f90 tx=0x7f5b14009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:11:57.410 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.403+0000 7f5b20ff9700 1 -- 172.21.15.33:0/3847416380 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5b1805a770 con 0x7f5b24070b90 2024-04-25T02:11:57.705 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.698+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5b2404c710 con 0x7f5b24070b90 2024-04-25T02:11:57.705 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:57 smithi033 ceph-mon[111306]: from='client.35776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:57.705 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:57 smithi033 ceph-mon[111306]: from='client.35780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:57.706 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:57 smithi033 ceph-mon[111306]: from='client.35784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:57.706 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:57 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/614331184' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:11:57.706 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:57 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3670409517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:11:57.706 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.698+0000 7f5b20ff9700 1 -- 172.21.15.33:0/3847416380 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f5b1805a300 con 0x7f5b24070b90 2024-04-25T02:11:57.706 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:11:57.706 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:11:57.706 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:11:57.706 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:11:57.707 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:11:57.708 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:11:57.708 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:11:57.708 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:11:57.708 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:11:57.708 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.700+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5b0c06e140 msgr2=0x7f5b0c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.700+0000 7f5b2a4ac700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5b0c06e140 0x7f5b0c0705f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f5b14004f90 tx=0x7f5b14009e30 comp rx=0 tx=0).stop 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.700+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070b90 msgr2=0x7f5b241160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.700+0000 7f5b2a4ac700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070b90 0x7f5b241160d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f5b18002a50 tx=0x7f5b180046b0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.700+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 shutdown_connections 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.700+0000 7f5b2a4ac700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5b0c06e140 0x7f5b0c0705f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.700+0000 7f5b2a4ac700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b24070280 0x7f5b24115b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.700+0000 7f5b2a4ac700 1 --2- 172.21.15.33:0/3847416380 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b24070b90 0x7f5b241160d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.700+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 >> 172.21.15.33:0/3847416380 conn(0x7f5b240f8990 msgr2=0x7f5b24108450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.701+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 shutdown_connections 2024-04-25T02:11:57.709 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:11:57.701+0000 7f5b2a4ac700 1 -- 172.21.15.33:0/3847416380 wait complete. 2024-04-25T02:11:57.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:57 smithi138 ceph-mon[81304]: from='client.35776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:57.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:57 smithi138 ceph-mon[81304]: from='client.35780 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:57.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:57 smithi138 ceph-mon[81304]: from='client.35784 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:57.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:57 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/614331184' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:11:57.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:57 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3670409517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:11:58.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:58 smithi033 ceph-mon[111306]: pgmap v978: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:58.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:58 smithi033 ceph-mon[111306]: from='client.35796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:58.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:11:58 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3847416380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:11:58.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:58 smithi138 ceph-mon[81304]: pgmap v978: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:11:58.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:58 smithi138 ceph-mon[81304]: from='client.35796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:11:58.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:11:58 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3847416380' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:12:00.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:00 smithi033 ceph-mon[111306]: pgmap v979: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:00.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:00 smithi138 ceph-mon[81304]: pgmap v979: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:01.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:01 smithi033 ceph-mon[111306]: pgmap v980: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:01.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:01 smithi138 ceph-mon[81304]: pgmap v980: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:04.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:04 smithi033 ceph-mon[111306]: pgmap v981: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:04.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:04 smithi138 ceph-mon[81304]: pgmap v981: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:06.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:06 smithi033 ceph-mon[111306]: pgmap v982: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:06.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:06 smithi138 ceph-mon[81304]: pgmap v982: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:08.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:08 smithi033 ceph-mon[111306]: pgmap v983: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:08.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:08 smithi138 ceph-mon[81304]: pgmap v983: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:10 smithi033 ceph-mon[111306]: pgmap v984: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:12:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:10 smithi138 ceph-mon[81304]: pgmap v984: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:12:12.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:12 smithi033 ceph-mon[111306]: pgmap v985: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:12.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:12 smithi138 ceph-mon[81304]: pgmap v985: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:14.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:14 smithi033 ceph-mon[111306]: pgmap v986: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:14.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:14 smithi138 ceph-mon[81304]: pgmap v986: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:16.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:16 smithi033 ceph-mon[111306]: pgmap v987: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:16.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:16 smithi138 ceph-mon[81304]: pgmap v987: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:18.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:18 smithi033 ceph-mon[111306]: pgmap v988: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:18.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:18 smithi138 ceph-mon[81304]: pgmap v988: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:20.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:20 smithi033 ceph-mon[111306]: pgmap v989: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:20.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:20 smithi138 ceph-mon[81304]: pgmap v989: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:22.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:22 smithi033 ceph-mon[111306]: pgmap v990: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:22.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:22 smithi138 ceph-mon[81304]: pgmap v990: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:24.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:24 smithi033 ceph-mon[111306]: pgmap v991: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:24.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:24 smithi138 ceph-mon[81304]: pgmap v991: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:25.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:12:25.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:12:26.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:26 smithi033 ceph-mon[111306]: pgmap v992: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:26.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:26 smithi138 ceph-mon[81304]: pgmap v992: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:27.828 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.819+0000 7f7dfe517700 1 -- 172.21.15.33:0/703300542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 msgr2=0x7f7df81031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.819+0000 7f7dfe517700 1 --2- 172.21.15.33:0/703300542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 0x7f7df81031c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7de8004940 tx=0x7f7de8009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.819+0000 7f7dfe517700 1 -- 172.21.15.33:0/703300542 shutdown_connections 2024-04-25T02:12:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.819+0000 7f7dfe517700 1 --2- 172.21.15.33:0/703300542 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7df8103790 0x7f7df80fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.819+0000 7f7dfe517700 1 --2- 172.21.15.33:0/703300542 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 0x7f7df81031c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.819+0000 7f7dfe517700 1 -- 172.21.15.33:0/703300542 >> 172.21.15.33:0/703300542 conn(0x7f7df80f8980 msgr2=0x7f7df80fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:27.829 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.819+0000 7f7dfe517700 1 -- 172.21.15.33:0/703300542 shutdown_connections 2024-04-25T02:12:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.819+0000 7f7dfe517700 1 -- 172.21.15.33:0/703300542 wait complete. 2024-04-25T02:12:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.820+0000 7f7dfe517700 1 Processor -- start 2024-04-25T02:12:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.820+0000 7f7dfe517700 1 -- start start 2024-04-25T02:12:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.820+0000 7f7dfe517700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 0x7f7df8115d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.820+0000 7f7dfe517700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7df8103790 0x7f7df8116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:27.830 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.820+0000 7f7dfe517700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df81168c0 con 0x7f7df8102df0 2024-04-25T02:12:27.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.820+0000 7f7dfe517700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df810fcd0 con 0x7f7df8103790 2024-04-25T02:12:27.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.820+0000 7f7df7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 0x7f7df8115d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:27.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.820+0000 7f7df7fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 0x7f7df8115d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58744/0 (socket says 172.21.15.33:58744) 2024-04-25T02:12:27.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.820+0000 7f7df7fff700 1 -- 172.21.15.33:0/175660463 learned_addr learned my addr 172.21.15.33:0/175660463 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:12:27.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7df77fe700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7df8103790 0x7f7df8116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:27.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7df7fff700 1 -- 172.21.15.33:0/175660463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7df8103790 msgr2=0x7f7df8116270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:27.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7df7fff700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7df8103790 0x7f7df8116270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:27.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7df7fff700 1 -- 172.21.15.33:0/175660463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de8005040 con 0x7f7df8102df0 2024-04-25T02:12:27.831 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7df7fff700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 0x7f7df8115d30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b3b0 tx=0x7f7de8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:27.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7df4ff9700 1 -- 172.21.15.33:0/175660463 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7de800e040 con 0x7f7df8102df0 2024-04-25T02:12:27.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7df4ff9700 1 -- 172.21.15.33:0/175660463 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7de800bbc0 con 0x7f7df8102df0 2024-04-25T02:12:27.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7df810ff50 con 0x7f7df8102df0 2024-04-25T02:12:27.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7df4ff9700 1 -- 172.21.15.33:0/175660463 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7de801c3f0 con 0x7f7df8102df0 2024-04-25T02:12:27.832 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.821+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7df8110510 con 0x7f7df8102df0 2024-04-25T02:12:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.822+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7df81072f0 con 0x7f7df8102df0 2024-04-25T02:12:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.823+0000 7f7df4ff9700 1 -- 172.21.15.33:0/175660463 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7de800bd30 con 0x7f7df8102df0 2024-04-25T02:12:27.833 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.823+0000 7f7df4ff9700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7de006e140 0x7f7de00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:27.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.823+0000 7f7df4ff9700 1 -- 172.21.15.33:0/175660463 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7de808eda0 con 0x7f7df8102df0 2024-04-25T02:12:27.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.824+0000 7f7df77fe700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7de006e140 0x7f7de00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:27.834 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.824+0000 7f7df77fe700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7de006e140 0x7f7de00705f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7df8111610 tx=0x7f7dec009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:27.835 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:27.828+0000 7f7df4ff9700 1 -- 172.21.15.33:0/175660463 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7de8058120 con 0x7f7df8102df0 2024-04-25T02:12:28.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.054+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7df8109f60 con 0x7f7de006e140 2024-04-25T02:12:28.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.056+0000 7f7df4ff9700 1 -- 172.21.15.33:0/175660463 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7df8109f60 con 0x7f7de006e140 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7de006e140 msgr2=0x7f7de00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7de006e140 0x7f7de00705f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7df8111610 tx=0x7f7dec009ee0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 msgr2=0x7f7df8115d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 0x7f7df8115d30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7de800b3b0 tx=0x7f7de8009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 shutdown_connections 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7de006e140 0x7f7de00705f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7df8102df0 0x7f7df8115d30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 --2- 172.21.15.33:0/175660463 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7df8103790 0x7f7df8116270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 >> 172.21.15.33:0/175660463 conn(0x7f7df80f8980 msgr2=0x7f7df80fa690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 shutdown_connections 2024-04-25T02:12:28.065 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.058+0000 7f7dfe517700 1 -- 172.21.15.33:0/175660463 wait complete. 2024-04-25T02:12:28.075 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:12:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.163+0000 7f8a302b8700 1 -- 172.21.15.33:0/3997806420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 msgr2=0x7f8a281031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.163+0000 7f8a302b8700 1 --2- 172.21.15.33:0/3997806420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 0x7f8a281031b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f8a24004940 tx=0x7f8a24009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.163+0000 7f8a302b8700 1 -- 172.21.15.33:0/3997806420 shutdown_connections 2024-04-25T02:12:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.163+0000 7f8a302b8700 1 --2- 172.21.15.33:0/3997806420 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a28103780 0x7f8a280fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.163+0000 7f8a302b8700 1 --2- 172.21.15.33:0/3997806420 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 0x7f8a281031b0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.163+0000 7f8a302b8700 1 -- 172.21.15.33:0/3997806420 >> 172.21.15.33:0/3997806420 conn(0x7f8a280f8950 msgr2=0x7f8a280fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:28.170 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.163+0000 7f8a302b8700 1 -- 172.21.15.33:0/3997806420 shutdown_connections 2024-04-25T02:12:28.171 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.163+0000 7f8a302b8700 1 -- 172.21.15.33:0/3997806420 wait complete. 2024-04-25T02:12:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.164+0000 7f8a302b8700 1 Processor -- start 2024-04-25T02:12:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.164+0000 7f8a302b8700 1 -- start start 2024-04-25T02:12:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.164+0000 7f8a302b8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 0x7f8a28113aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:28.172 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.164+0000 7f8a302b8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a28103780 0x7f8a28113fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.164+0000 7f8a302b8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a28114630 con 0x7f8a28102de0 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.164+0000 7f8a302b8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2810da70 con 0x7f8a28103780 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.164+0000 7f8a2e054700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 0x7f8a28113aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.164+0000 7f8a2e054700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 0x7f8a28113aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58772/0 (socket says 172.21.15.33:58772) 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.164+0000 7f8a2e054700 1 -- 172.21.15.33:0/4225790786 learned_addr learned my addr 172.21.15.33:0/4225790786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a2d853700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a28103780 0x7f8a28113fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a2e054700 1 -- 172.21.15.33:0/4225790786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a28103780 msgr2=0x7f8a28113fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a2e054700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a28103780 0x7f8a28113fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a2e054700 1 -- 172.21.15.33:0/4225790786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a24005040 con 0x7f8a28102de0 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a2e054700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 0x7f8a28113aa0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009b30 tx=0x7f8a24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a1effd700 1 -- 172.21.15.33:0/4225790786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a24021070 con 0x7f8a28102de0 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a1effd700 1 -- 172.21.15.33:0/4225790786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a24013c00 con 0x7f8a28102de0 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2810dcf0 con 0x7f8a28102de0 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a1effd700 1 -- 172.21.15.33:0/4225790786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a2400e070 con 0x7f8a28102de0 2024-04-25T02:12:28.173 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.165+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2810e1d0 con 0x7f8a28102de0 2024-04-25T02:12:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.166+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a281000f0 con 0x7f8a28102de0 2024-04-25T02:12:28.174 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.167+0000 7f8a1effd700 1 -- 172.21.15.33:0/4225790786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8a2401cad0 con 0x7f8a28102de0 2024-04-25T02:12:28.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.168+0000 7f8a1effd700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8a1406e140 0x7f8a140705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:28.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.168+0000 7f8a1effd700 1 -- 172.21.15.33:0/4225790786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f8a2408f5e0 con 0x7f8a28102de0 2024-04-25T02:12:28.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.168+0000 7f8a2d853700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8a1406e140 0x7f8a140705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:28.175 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.169+0000 7f8a2d853700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8a1406e140 0x7f8a140705f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8a2810ebe0 tx=0x7f8a1800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:28.182 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.172+0000 7f8a1effd700 1 -- 172.21.15.33:0/4225790786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8a2405c5b0 con 0x7f8a28102de0 2024-04-25T02:12:28.405 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.398+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a28109ee0 con 0x7f8a1406e140 2024-04-25T02:12:28.406 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.399+0000 7f8a1effd700 1 -- 172.21.15.33:0/4225790786 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8a28109ee0 con 0x7f8a1406e140 2024-04-25T02:12:28.408 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.401+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8a1406e140 msgr2=0x7f8a140705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.401+0000 7f8a302b8700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8a1406e140 0x7f8a140705f0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8a2810ebe0 tx=0x7f8a1800b040 comp rx=0 tx=0).stop 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.401+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 msgr2=0x7f8a28113aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.401+0000 7f8a302b8700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 0x7f8a28113aa0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009b30 tx=0x7f8a24009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.401+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 shutdown_connections 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.401+0000 7f8a302b8700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f8a1406e140 0x7f8a140705f0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.401+0000 7f8a302b8700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8a28102de0 0x7f8a28113aa0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.401+0000 7f8a302b8700 1 --2- 172.21.15.33:0/4225790786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8a28103780 0x7f8a28113fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.401+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 >> 172.21.15.33:0/4225790786 conn(0x7f8a280f8950 msgr2=0x7f8a28107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.402+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 shutdown_connections 2024-04-25T02:12:28.409 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.402+0000 7f8a302b8700 1 -- 172.21.15.33:0/4225790786 wait complete. 2024-04-25T02:12:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.506+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/1621177972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fe7b0 msgr2=0x7f5e44106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.506+0000 7f5e4c0d5700 1 --2- 172.21.15.33:0/1621177972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fe7b0 0x7f5e44106df0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5e38004940 tx=0x7f5e38009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.506+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/1621177972 shutdown_connections 2024-04-25T02:12:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.506+0000 7f5e4c0d5700 1 --2- 172.21.15.33:0/1621177972 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fe7b0 0x7f5e44106df0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.506+0000 7f5e4c0d5700 1 --2- 172.21.15.33:0/1621177972 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e440fde10 0x7f5e440fe1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.513 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.506+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/1621177972 >> 172.21.15.33:0/1621177972 conn(0x7f5e440f7820 msgr2=0x7f5e440f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:28.513 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:28 smithi033 ceph-mon[111306]: pgmap v993: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.506+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/1621177972 shutdown_connections 2024-04-25T02:12:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.506+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/1621177972 wait complete. 2024-04-25T02:12:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.506+0000 7f5e4c0d5700 1 Processor -- start 2024-04-25T02:12:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e4c0d5700 1 -- start start 2024-04-25T02:12:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e4c0d5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fde10 0x7f5e440fbc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e4c0d5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e440fe7b0 0x7f5e440fc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e4c0d5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440fdb90 con 0x7f5e440fde10 2024-04-25T02:12:28.515 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e4c0d5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e440fc6c0 con 0x7f5e440fe7b0 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e49e71700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fde10 0x7f5e440fbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e49e71700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fde10 0x7f5e440fbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58788/0 (socket says 172.21.15.33:58788) 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e49e71700 1 -- 172.21.15.33:0/3343400146 learned_addr learned my addr 172.21.15.33:0/3343400146 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e49670700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e440fe7b0 0x7f5e440fc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e49e71700 1 -- 172.21.15.33:0/3343400146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e440fe7b0 msgr2=0x7f5e440fc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e49e71700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e440fe7b0 0x7f5e440fc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e49e71700 1 -- 172.21.15.33:0/3343400146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38005040 con 0x7f5e440fde10 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.507+0000 7f5e49670700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e440fe7b0 0x7f5e440fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.508+0000 7f5e49e71700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fde10 0x7f5e440fbc40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400cac0 tx=0x7f5e34007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.508+0000 7f5e427fc700 1 -- 172.21.15.33:0/3343400146 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3400aa50 con 0x7f5e440fde10 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.508+0000 7f5e427fc700 1 -- 172.21.15.33:0/3343400146 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e3400abb0 con 0x7f5e440fde10 2024-04-25T02:12:28.516 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.508+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e440fc9a0 con 0x7f5e440fde10 2024-04-25T02:12:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.508+0000 7f5e427fc700 1 -- 172.21.15.33:0/3343400146 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e340115e0 con 0x7f5e440fde10 2024-04-25T02:12:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.508+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4411a3a0 con 0x7f5e440fde10 2024-04-25T02:12:28.517 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.509+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e44066b30 con 0x7f5e440fde10 2024-04-25T02:12:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.510+0000 7f5e427fc700 1 -- 172.21.15.33:0/3343400146 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5e3400ad20 con 0x7f5e440fde10 2024-04-25T02:12:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.510+0000 7f5e427fc700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e2c06e140 0x7f5e2c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.510+0000 7f5e427fc700 1 -- 172.21.15.33:0/3343400146 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5e3408e1b0 con 0x7f5e440fde10 2024-04-25T02:12:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.510+0000 7f5e49670700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e2c06e140 0x7f5e2c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:28.518 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.511+0000 7f5e49670700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e2c06e140 0x7f5e2c0705f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800b3b0 tx=0x7f5e38009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:28.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.515+0000 7f5e427fc700 1 -- 172.21.15.33:0/3343400146 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5e340575b0 con 0x7f5e440fde10 2024-04-25T02:12:28.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.734+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e44065f30 con 0x7f5e2c06e140 2024-04-25T02:12:28.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.744+0000 7f5e427fc700 1 -- 172.21.15.33:0/3343400146 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5e44065f30 con 0x7f5e2c06e140 2024-04-25T02:12:28.751 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:12:28.751 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (33m) 11m ago 44m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:12:28.751 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (30m) 11m ago 45m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:12:28.751 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (30m) 11m ago 44m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:12:28.751 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (33m) 11m ago 44m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:12:28.751 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (26m) 11m ago 39m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:12:28.751 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (27m) 11m ago 39m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (26m) 11m ago 39m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (11m) 11m ago 39m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (35m) 11m ago 46m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (34m) 11m ago 43m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (31m) 11m ago 46m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (31m) 11m ago 43m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (34m) 11m ago 44m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (34m) 11m ago 43m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (30m) 11m ago 43m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (29m) 11m ago 42m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (29m) 11m ago 42m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (28m) 11m ago 41m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (28m) 11m ago 41m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:12:28.752 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (27m) 11m ago 40m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:12:28.753 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (33m) 11m ago 44m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:12:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e2c06e140 msgr2=0x7f5e2c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e2c06e140 0x7f5e2c0705f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5e3800b3b0 tx=0x7f5e38009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fde10 msgr2=0x7f5e440fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fde10 0x7f5e440fbc40 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400cac0 tx=0x7f5e34007ee0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 shutdown_connections 2024-04-25T02:12:28.753 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5e2c06e140 0x7f5e2c0705f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5e440fde10 0x7f5e440fbc40 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 --2- 172.21.15.33:0/3343400146 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5e440fe7b0 0x7f5e440fc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 >> 172.21.15.33:0/3343400146 conn(0x7f5e440f7820 msgr2=0x7f5e440f8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 shutdown_connections 2024-04-25T02:12:28.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.746+0000 7f5e4c0d5700 1 -- 172.21.15.33:0/3343400146 wait complete. 2024-04-25T02:12:28.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:28 smithi138 ceph-mon[81304]: pgmap v993: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:28.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.849+0000 7efdd6d74700 1 -- 172.21.15.33:0/3563935233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 msgr2=0x7efdd01031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.849+0000 7efdd6d74700 1 --2- 172.21.15.33:0/3563935233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 0x7efdd01031b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efdc0009ce0 tx=0x7efdc0004d10 comp rx=0 tx=0).stop 2024-04-25T02:12:28.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.849+0000 7efdd6d74700 1 -- 172.21.15.33:0/3563935233 shutdown_connections 2024-04-25T02:12:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.849+0000 7efdd6d74700 1 --2- 172.21.15.33:0/3563935233 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd0103780 0x7efdd00fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.849+0000 7efdd6d74700 1 --2- 172.21.15.33:0/3563935233 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 0x7efdd01031b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.849+0000 7efdd6d74700 1 -- 172.21.15.33:0/3563935233 >> 172.21.15.33:0/3563935233 conn(0x7efdd00f8990 msgr2=0x7efdd00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.849+0000 7efdd6d74700 1 -- 172.21.15.33:0/3563935233 shutdown_connections 2024-04-25T02:12:28.857 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.849+0000 7efdd6d74700 1 -- 172.21.15.33:0/3563935233 wait complete. 2024-04-25T02:12:28.858 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.850+0000 7efdd6d74700 1 Processor -- start 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.850+0000 7efdd6d74700 1 -- start start 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.850+0000 7efdd6d74700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 0x7efdd0113ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.850+0000 7efdd6d74700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd0103780 0x7efdd0114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.850+0000 7efdd6d74700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd0114660 con 0x7efdd0102de0 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.850+0000 7efdd6d74700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd010dc90 con 0x7efdd0103780 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.850+0000 7efdd4b10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 0x7efdd0113ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdd4b10700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 0x7efdd0113ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58806/0 (socket says 172.21.15.33:58806) 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdd4b10700 1 -- 172.21.15.33:0/4130707696 learned_addr learned my addr 172.21.15.33:0/4130707696 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdcffff700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd0103780 0x7efdd0114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdd4b10700 1 -- 172.21.15.33:0/4130707696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd0103780 msgr2=0x7efdd0114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdd4b10700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd0103780 0x7efdd0114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdd4b10700 1 -- 172.21.15.33:0/4130707696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc0009940 con 0x7efdd0102de0 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdd4b10700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 0x7efdd0113ad0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efdc000be90 tx=0x7efdc0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:28.859 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdcd7fa700 1 -- 172.21.15.33:0/4130707696 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdc000e040 con 0x7efdd0102de0 2024-04-25T02:12:28.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdcd7fa700 1 -- 172.21.15.33:0/4130707696 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efdc0008ea0 con 0x7efdd0102de0 2024-04-25T02:12:28.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd010df10 con 0x7efdd0102de0 2024-04-25T02:12:28.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdcd7fa700 1 -- 172.21.15.33:0/4130707696 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efdc001c810 con 0x7efdd0102de0 2024-04-25T02:12:28.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.851+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd010e4d0 con 0x7efdd0102de0 2024-04-25T02:12:28.860 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.852+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd01000f0 con 0x7efdd0102de0 2024-04-25T02:12:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.853+0000 7efdcd7fa700 1 -- 172.21.15.33:0/4130707696 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efdc0012c00 con 0x7efdd0102de0 2024-04-25T02:12:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.853+0000 7efdcd7fa700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdb806e140 0x7efdb80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.853+0000 7efdcd7fa700 1 -- 172.21.15.33:0/4130707696 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efdc008f3a0 con 0x7efdd0102de0 2024-04-25T02:12:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.853+0000 7efdcffff700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdb806e140 0x7efdb80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:28.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.854+0000 7efdcffff700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdb806e140 0x7efdb80705f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efdc4007fd0 tx=0x7efdc4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:28.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:28.858+0000 7efdcd7fa700 1 -- 172.21.15.33:0/4130707696 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efdc005c250 con 0x7efdd0102de0 2024-04-25T02:12:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.157+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efdd004c770 con 0x7efdd0102de0 2024-04-25T02:12:29.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.157+0000 7efdcd7fa700 1 -- 172.21.15.33:0/4130707696 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efdc0004e10 con 0x7efdd0102de0 2024-04-25T02:12:29.164 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:12:29.165 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:12:29.166 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:12:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.159+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdb806e140 msgr2=0x7efdb80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.159+0000 7efdd6d74700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdb806e140 0x7efdb80705f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efdc4007fd0 tx=0x7efdc4009040 comp rx=0 tx=0).stop 2024-04-25T02:12:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.159+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 msgr2=0x7efdd0113ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.159+0000 7efdd6d74700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 0x7efdd0113ad0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7efdc000be90 tx=0x7efdc0004d10 comp rx=0 tx=0).stop 2024-04-25T02:12:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.160+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 shutdown_connections 2024-04-25T02:12:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.160+0000 7efdd6d74700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efdb806e140 0x7efdb80705f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.160+0000 7efdd6d74700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efdd0102de0 0x7efdd0113ad0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.160+0000 7efdd6d74700 1 --2- 172.21.15.33:0/4130707696 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efdd0103780 0x7efdd0114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.160+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 >> 172.21.15.33:0/4130707696 conn(0x7efdd00f8990 msgr2=0x7efdd0107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.160+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 shutdown_connections 2024-04-25T02:12:29.167 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.160+0000 7efdd6d74700 1 -- 172.21.15.33:0/4130707696 wait complete. 2024-04-25T02:12:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.264+0000 7fa8adcb3700 1 -- 172.21.15.33:0/1545789629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80fbd40 msgr2=0x7fa8a80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.264+0000 7fa8adcb3700 1 --2- 172.21.15.33:0/1545789629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80fbd40 0x7fa8a80fc1b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa89c004940 tx=0x7fa89c009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.264+0000 7fa8adcb3700 1 -- 172.21.15.33:0/1545789629 shutdown_connections 2024-04-25T02:12:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.264+0000 7fa8adcb3700 1 --2- 172.21.15.33:0/1545789629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a80fbd40 0x7fa8a80fc1b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.264+0000 7fa8adcb3700 1 --2- 172.21.15.33:0/1545789629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a8101d90 0x7fa8a8102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.264+0000 7fa8adcb3700 1 -- 172.21.15.33:0/1545789629 >> 172.21.15.33:0/1545789629 conn(0x7fa8a8071d00 msgr2=0x7fa8a8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.264+0000 7fa8adcb3700 1 -- 172.21.15.33:0/1545789629 shutdown_connections 2024-04-25T02:12:29.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.264+0000 7fa8adcb3700 1 -- 172.21.15.33:0/1545789629 wait complete. 2024-04-25T02:12:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8adcb3700 1 Processor -- start 2024-04-25T02:12:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8adcb3700 1 -- start start 2024-04-25T02:12:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8adcb3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a80fbd40 0x7fa8a8113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8adcb3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a8101d90 0x7fa8a81140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:29.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8adcb3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a8114720 con 0x7fa8a8101d90 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8adcb3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a810db30 con 0x7fa8a80fbd40 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8a6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a8101d90 0x7fa8a81140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8a6ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a8101d90 0x7fa8a81140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58824/0 (socket says 172.21.15.33:58824) 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8a6ffd700 1 -- 172.21.15.33:0/2480505181 learned_addr learned my addr 172.21.15.33:0/2480505181 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8a77fe700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a80fbd40 0x7fa8a8113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8a6ffd700 1 -- 172.21.15.33:0/2480505181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a80fbd40 msgr2=0x7fa8a8113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8a6ffd700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a80fbd40 0x7fa8a8113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.265+0000 7fa8a6ffd700 1 -- 172.21.15.33:0/2480505181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c005040 con 0x7fa8a8101d90 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.266+0000 7fa8a6ffd700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a8101d90 0x7fa8a81140d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00b3b0 tx=0x7fa89c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.266+0000 7fa88ffff700 1 -- 172.21.15.33:0/2480505181 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa89c00e040 con 0x7fa8a8101d90 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.266+0000 7fa88ffff700 1 -- 172.21.15.33:0/2480505181 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa89c00bbc0 con 0x7fa8a8101d90 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.266+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a810ddb0 con 0x7fa8a8101d90 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.266+0000 7fa88ffff700 1 -- 172.21.15.33:0/2480505181 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa89c01c3f0 con 0x7fa8a8101d90 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.266+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a810e370 con 0x7fa8a8101d90 2024-04-25T02:12:29.275 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.267+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a80fd430 con 0x7fa8a8101d90 2024-04-25T02:12:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.268+0000 7fa88ffff700 1 -- 172.21.15.33:0/2480505181 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa89c00bd30 con 0x7fa8a8101d90 2024-04-25T02:12:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.268+0000 7fa88ffff700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa89006e140 0x7fa8900705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.268+0000 7fa88ffff700 1 -- 172.21.15.33:0/2480505181 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa89c08ed70 con 0x7fa8a8101d90 2024-04-25T02:12:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.268+0000 7fa8a77fe700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa89006e140 0x7fa8900705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:29.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.269+0000 7fa8a77fe700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa89006e140 0x7fa8900705f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa898007fd0 tx=0x7fa898009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:29.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.273+0000 7fa88ffff700 1 -- 172.21.15.33:0/2480505181 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa89c058170 con 0x7fa8a8101d90 2024-04-25T02:12:29.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:29 smithi033 ceph-mon[111306]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:12:29.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:29 smithi033 ceph-mon[111306]: from='client.35808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:12:29.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:29 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4130707696' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:12:29.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.531+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa8a8108e10 con 0x7fa8a8101d90 2024-04-25T02:12:29.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.531+0000 7fa88ffff700 1 -- 172.21.15.33:0/2480505181 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fa89c05b830 con 0x7fa8a8101d90 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:12:29.539 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:12:29.540 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:12:29.541 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:12:29.542 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:12:29.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa89006e140 msgr2=0x7fa8900705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa89006e140 0x7fa8900705f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fa898007fd0 tx=0x7fa898009040 comp rx=0 tx=0).stop 2024-04-25T02:12:29.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a8101d90 msgr2=0x7fa8a81140d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a8101d90 0x7fa8a81140d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00b3b0 tx=0x7fa89c009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:29.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 shutdown_connections 2024-04-25T02:12:29.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa89006e140 0x7fa8900705f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa8a80fbd40 0x7fa8a8113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 --2- 172.21.15.33:0/2480505181 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa8a8101d90 0x7fa8a81140d0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 >> 172.21.15.33:0/2480505181 conn(0x7fa8a8071d00 msgr2=0x7fa8a80655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:29.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 shutdown_connections 2024-04-25T02:12:29.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.534+0000 7fa8adcb3700 1 -- 172.21.15.33:0/2480505181 wait complete. 2024-04-25T02:12:29.543 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:12:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.638+0000 7fe119ece700 1 -- 172.21.15.33:0/3493546833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114103780 msgr2=0x7fe1140fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.638+0000 7fe119ece700 1 --2- 172.21.15.33:0/3493546833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114103780 0x7fe1140fcdf0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe108004940 tx=0x7fe108009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.638+0000 7fe119ece700 1 -- 172.21.15.33:0/3493546833 shutdown_connections 2024-04-25T02:12:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.638+0000 7fe119ece700 1 --2- 172.21.15.33:0/3493546833 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114103780 0x7fe1140fcdf0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.638+0000 7fe119ece700 1 --2- 172.21.15.33:0/3493546833 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe114102de0 0x7fe1141031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.638+0000 7fe119ece700 1 -- 172.21.15.33:0/3493546833 >> 172.21.15.33:0/3493546833 conn(0x7fe1140f8990 msgr2=0x7fe1140fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.638+0000 7fe119ece700 1 -- 172.21.15.33:0/3493546833 shutdown_connections 2024-04-25T02:12:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.638+0000 7fe119ece700 1 -- 172.21.15.33:0/3493546833 wait complete. 2024-04-25T02:12:29.646 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.639+0000 7fe119ece700 1 Processor -- start 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.639+0000 7fe119ece700 1 -- start start 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe119ece700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114102de0 0x7fe114070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe119ece700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe114103780 0x7fe1140707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe119ece700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1140721b0 con 0x7fe114102de0 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe119ece700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe114070ce0 con 0x7fe114103780 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe1137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114102de0 0x7fe114070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe1137fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114102de0 0x7fe114070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58842/0 (socket says 172.21.15.33:58842) 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe1137fe700 1 -- 172.21.15.33:0/3423595852 learned_addr learned my addr 172.21.15.33:0/3423595852 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe112ffd700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe114103780 0x7fe1140707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe1137fe700 1 -- 172.21.15.33:0/3423595852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe114103780 msgr2=0x7fe1140707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe1137fe700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe114103780 0x7fe1140707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe1137fe700 1 -- 172.21.15.33:0/3423595852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe108005040 con 0x7fe114102de0 2024-04-25T02:12:29.648 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe1137fe700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114102de0 0x7fe114070260 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe104002a50 tx=0x7fe1040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe0fbfff700 1 -- 172.21.15.33:0/3423595852 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe104007cc0 con 0x7fe114102de0 2024-04-25T02:12:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe114070fc0 con 0x7fe114102de0 2024-04-25T02:12:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe0fbfff700 1 -- 172.21.15.33:0/3423595852 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe10400f040 con 0x7fe114102de0 2024-04-25T02:12:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe0fbfff700 1 -- 172.21.15.33:0/3423595852 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe10400da30 con 0x7fe114102de0 2024-04-25T02:12:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.640+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11411a0a0 con 0x7fe114102de0 2024-04-25T02:12:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.641+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1140fe530 con 0x7fe114102de0 2024-04-25T02:12:29.649 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.642+0000 7fe0fbfff700 1 -- 172.21.15.33:0/3423595852 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe10400db90 con 0x7fe114102de0 2024-04-25T02:12:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.643+0000 7fe0fbfff700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0fc06e140 0x7fe0fc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.643+0000 7fe0fbfff700 1 -- 172.21.15.33:0/3423595852 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe10408d8d0 con 0x7fe114102de0 2024-04-25T02:12:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.643+0000 7fe112ffd700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0fc06e140 0x7fe0fc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:29.650 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.643+0000 7fe112ffd700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0fc06e140 0x7fe0fc0705f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe108004890 tx=0x7fe108009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:29.654 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.647+0000 7fe0fbfff700 1 -- 172.21.15.33:0/3423595852 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe104056c50 con 0x7fe114102de0 2024-04-25T02:12:29.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:29 smithi138 ceph-mon[81304]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:12:29.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:29 smithi138 ceph-mon[81304]: from='client.35808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:12:29.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:29 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4130707696' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:12:29.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.874+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1140fa0a0 con 0x7fe0fc06e140 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.875+0000 7fe0fbfff700 1 -- 172.21.15.33:0/3423595852 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe1140fa0a0 con 0x7fe0fc06e140 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:12:29.882 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:12:29.883 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:12:29.883 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:12:29.883 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:12:29.883 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:12:29.883 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:12:29.883 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:12:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0fc06e140 msgr2=0x7fe0fc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0fc06e140 0x7fe0fc0705f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fe108004890 tx=0x7fe108009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114102de0 msgr2=0x7fe114070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114102de0 0x7fe114070260 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe104002a50 tx=0x7fe1040046b0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 shutdown_connections 2024-04-25T02:12:29.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0fc06e140 0x7fe0fc0705f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe114102de0 0x7fe114070260 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 --2- 172.21.15.33:0/3423595852 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe114103780 0x7fe1140707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 >> 172.21.15.33:0/3423595852 conn(0x7fe1140f8990 msgr2=0x7fe1140f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:29.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 shutdown_connections 2024-04-25T02:12:29.886 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.878+0000 7fe119ece700 1 -- 172.21.15.33:0/3423595852 wait complete. 2024-04-25T02:12:29.989 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.982+0000 7f70bc135700 1 -- 172.21.15.33:0/1874008710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b40fc710 msgr2=0x7f70b40fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.982+0000 7f70bc135700 1 --2- 172.21.15.33:0/1874008710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b40fc710 0x7f70b40fcb80 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f70a8004940 tx=0x7f70a8009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 -- 172.21.15.33:0/1874008710 shutdown_connections 2024-04-25T02:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 --2- 172.21.15.33:0/1874008710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b40fc710 0x7f70b40fcb80 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 --2- 172.21.15.33:0/1874008710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b4101e20 0x7f70b41021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 -- 172.21.15.33:0/1874008710 >> 172.21.15.33:0/1874008710 conn(0x7f70b4071460 msgr2=0x7f70b4071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 -- 172.21.15.33:0/1874008710 shutdown_connections 2024-04-25T02:12:29.990 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 -- 172.21.15.33:0/1874008710 wait complete. 2024-04-25T02:12:29.991 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 Processor -- start 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 -- start start 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b40fc710 0x7f70b410eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b4101e20 0x7f70b410f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b410f670 con 0x7f70b4101e20 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.983+0000 7f70bc135700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b410f7b0 con 0x7f70b40fc710 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70b96d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b4101e20 0x7f70b410f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70b96d0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b4101e20 0x7f70b410f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58854/0 (socket says 172.21.15.33:58854) 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70b96d0700 1 -- 172.21.15.33:0/3119441988 learned_addr learned my addr 172.21.15.33:0/3119441988 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70b9ed1700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b40fc710 0x7f70b410eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70b96d0700 1 -- 172.21.15.33:0/3119441988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b40fc710 msgr2=0x7f70b410eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70b96d0700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b40fc710 0x7f70b410eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70b96d0700 1 -- 172.21.15.33:0/3119441988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a8005040 con 0x7f70b4101e20 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70b96d0700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b4101e20 0x7f70b410f020 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009b30 tx=0x7f70a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70a67fc700 1 -- 172.21.15.33:0/3119441988 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a800e070 con 0x7f70b4101e20 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70a67fc700 1 -- 172.21.15.33:0/3119441988 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70a8013c00 con 0x7f70b4101e20 2024-04-25T02:12:29.992 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70a67fc700 1 -- 172.21.15.33:0/3119441988 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70a800b650 con 0x7f70b4101e20 2024-04-25T02:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b4117b50 con 0x7f70b4101e20 2024-04-25T02:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.984+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b4117fb0 con 0x7f70b4101e20 2024-04-25T02:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.985+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b4104200 con 0x7f70b4101e20 2024-04-25T02:12:29.993 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.986+0000 7f70a67fc700 1 -- 172.21.15.33:0/3119441988 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f70a801ca40 con 0x7f70b4101e20 2024-04-25T02:12:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.986+0000 7f70a67fc700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70a006e140 0x7f70a00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:12:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.986+0000 7f70a67fc700 1 -- 172.21.15.33:0/3119441988 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f70a808f2f0 con 0x7f70b4101e20 2024-04-25T02:12:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.987+0000 7f70b9ed1700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70a006e140 0x7f70a00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:12:29.994 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.987+0000 7f70b9ed1700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70a006e140 0x7f70a00705f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009b80 tx=0x7f70b000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:12:29.998 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:29.991+0000 7f70a67fc700 1 -- 172.21.15.33:0/3119441988 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f70a8058790 con 0x7f70b4101e20 2024-04-25T02:12:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.284+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f70b40fbdb0 con 0x7f70b4101e20 2024-04-25T02:12:30.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.285+0000 7f70a67fc700 1 -- 172.21.15.33:0/3119441988 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f70a8004490 con 0x7f70b4101e20 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:12:30.292 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:12:30.293 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:12:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70a006e140 msgr2=0x7f70a00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70a006e140 0x7f70a00705f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009b80 tx=0x7f70b000b040 comp rx=0 tx=0).stop 2024-04-25T02:12:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b4101e20 msgr2=0x7f70b410f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:12:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b4101e20 0x7f70b410f020 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009b30 tx=0x7f70a8009e30 comp rx=0 tx=0).stop 2024-04-25T02:12:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 shutdown_connections 2024-04-25T02:12:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70a006e140 0x7f70a00705f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70b40fc710 0x7f70b410eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 --2- 172.21.15.33:0/3119441988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70b4101e20 0x7f70b410f020 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:12:30.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 >> 172.21.15.33:0/3119441988 conn(0x7f70b4071460 msgr2=0x7f70b40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:12:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 shutdown_connections 2024-04-25T02:12:30.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:12:30.287+0000 7f70bc135700 1 -- 172.21.15.33:0/3119441988 wait complete. 2024-04-25T02:12:30.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:30 smithi033 ceph-mon[111306]: from='client.35812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:12:30.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:30 smithi033 ceph-mon[111306]: pgmap v994: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:30.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:30 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2480505181' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:12:30.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:30 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3119441988' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:12:30.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:30 smithi138 ceph-mon[81304]: from='client.35812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:12:30.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:30 smithi138 ceph-mon[81304]: pgmap v994: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:30.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:30 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2480505181' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:12:30.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:30 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3119441988' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:12:31.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:31 smithi033 ceph-mon[111306]: from='client.35824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:12:31.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:31 smithi138 ceph-mon[81304]: from='client.35824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:12:32.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:32 smithi033 ceph-mon[111306]: pgmap v995: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:32.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:32 smithi138 ceph-mon[81304]: pgmap v995: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:34.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:34 smithi033 ceph-mon[111306]: pgmap v996: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:34.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:34 smithi138 ceph-mon[81304]: pgmap v996: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:36.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:36 smithi033 ceph-mon[111306]: pgmap v997: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:36.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:36 smithi138 ceph-mon[81304]: pgmap v997: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:38.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:38 smithi033 ceph-mon[111306]: pgmap v998: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:38.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:38 smithi138 ceph-mon[81304]: pgmap v998: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:40.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:40 smithi033 ceph-mon[111306]: pgmap v999: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:40.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:12:40.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:40 smithi138 ceph-mon[81304]: pgmap v999: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:40.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:12:42.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:42 smithi033 ceph-mon[111306]: pgmap v1000: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:42.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:42 smithi138 ceph-mon[81304]: pgmap v1000: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:44.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:44 smithi033 ceph-mon[111306]: pgmap v1001: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:44.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:44 smithi138 ceph-mon[81304]: pgmap v1001: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:46.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:46 smithi033 ceph-mon[111306]: pgmap v1002: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:46.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:46 smithi138 ceph-mon[81304]: pgmap v1002: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:48.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:48 smithi033 ceph-mon[111306]: pgmap v1003: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:48.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:48 smithi138 ceph-mon[81304]: pgmap v1003: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:50.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:50 smithi033 ceph-mon[111306]: pgmap v1004: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:50.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:50 smithi138 ceph-mon[81304]: pgmap v1004: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:52.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:52 smithi033 ceph-mon[111306]: pgmap v1005: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:52.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:52 smithi138 ceph-mon[81304]: pgmap v1005: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:54.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:54 smithi033 ceph-mon[111306]: pgmap v1006: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:54.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:54 smithi138 ceph-mon[81304]: pgmap v1006: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:55.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:55 smithi033 ceph-mon[111306]: pgmap v1007: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:55.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:12:55.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:55 smithi138 ceph-mon[81304]: pgmap v1007: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:55.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:12:58.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:12:58 smithi033 ceph-mon[111306]: pgmap v1008: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:12:58.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:12:58 smithi138 ceph-mon[81304]: pgmap v1008: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 -- 172.21.15.33:0/2576517496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fe7c0 msgr2=0x7f2318106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 --2- 172.21.15.33:0/2576517496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fe7c0 0x7f2318106e90 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2304004940 tx=0x7f2304009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 -- 172.21.15.33:0/2576517496 shutdown_connections 2024-04-25T02:13:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 --2- 172.21.15.33:0/2576517496 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fe7c0 0x7f2318106e90 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 --2- 172.21.15.33:0/2576517496 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23180fde20 0x7f23180fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 -- 172.21.15.33:0/2576517496 >> 172.21.15.33:0/2576517496 conn(0x7f23180f7860 msgr2=0x7f23180f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:00.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 -- 172.21.15.33:0/2576517496 shutdown_connections 2024-04-25T02:13:00.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 -- 172.21.15.33:0/2576517496 wait complete. 2024-04-25T02:13:00.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 Processor -- start 2024-04-25T02:13:00.414 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.406+0000 7f231ebd6700 1 -- start start 2024-04-25T02:13:00.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231ebd6700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fde20 0x7f23180fbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:00.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231ebd6700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23180fe7c0 0x7f23180fc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:00.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231ebd6700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23180fdc30 con 0x7f23180fde20 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231ebd6700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23180fc760 con 0x7f23180fe7c0 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231c972700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fde20 0x7f23180fbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231c972700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fde20 0x7f23180fbce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48850/0 (socket says 172.21.15.33:48850) 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231c972700 1 -- 172.21.15.33:0/3042947789 learned_addr learned my addr 172.21.15.33:0/3042947789 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f2317fff700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23180fe7c0 0x7f23180fc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231c972700 1 -- 172.21.15.33:0/3042947789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23180fe7c0 msgr2=0x7f23180fc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231c972700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23180fe7c0 0x7f23180fc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.407+0000 7f231c972700 1 -- 172.21.15.33:0/3042947789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2304005040 con 0x7f23180fde20 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.408+0000 7f231c972700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fde20 0x7f23180fbce0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f230c002a50 tx=0x7f230c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.408+0000 7f23157fa700 1 -- 172.21.15.33:0/3042947789 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f230c007cc0 con 0x7f23180fde20 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.408+0000 7f23157fa700 1 -- 172.21.15.33:0/3042947789 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f230c00f040 con 0x7f23180fde20 2024-04-25T02:13:00.416 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.408+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23180fca40 con 0x7f23180fde20 2024-04-25T02:13:00.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.408+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231811a0b0 con 0x7f23180fde20 2024-04-25T02:13:00.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.408+0000 7f23157fa700 1 -- 172.21.15.33:0/3042947789 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f230c009640 con 0x7f23180fde20 2024-04-25T02:13:00.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.409+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22fc0052b0 con 0x7f23180fde20 2024-04-25T02:13:00.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.409+0000 7f23157fa700 1 -- 172.21.15.33:0/3042947789 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f230c0097a0 con 0x7f23180fde20 2024-04-25T02:13:00.417 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.410+0000 7f23157fa700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230806e140 0x7f23080705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:00.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.410+0000 7f23157fa700 1 -- 172.21.15.33:0/3042947789 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f230c08d880 con 0x7f23180fde20 2024-04-25T02:13:00.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.410+0000 7f2317fff700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230806e140 0x7f23080705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:00.418 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.410+0000 7f2317fff700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230806e140 0x7f23080705f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f23180fd410 tx=0x7f2304009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:00.421 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.415+0000 7f23157fa700 1 -- 172.21.15.33:0/3042947789 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f230c056c00 con 0x7f23180fde20 2024-04-25T02:13:00.652 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.645+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22fc000bc0 con 0x7f230806e140 2024-04-25T02:13:00.652 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:00 smithi033 ceph-mon[111306]: pgmap v1009: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:00.653 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.647+0000 7f23157fa700 1 -- 172.21.15.33:0/3042947789 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f22fc000bc0 con 0x7f230806e140 2024-04-25T02:13:00.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.648+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230806e140 msgr2=0x7f23080705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:00.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.648+0000 7f231ebd6700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230806e140 0x7f23080705f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f23180fd410 tx=0x7f2304009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:00.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.648+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fde20 msgr2=0x7f23180fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:00.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.648+0000 7f231ebd6700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fde20 0x7f23180fbce0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f230c002a50 tx=0x7f230c0046b0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.649+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 shutdown_connections 2024-04-25T02:13:00.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.649+0000 7f231ebd6700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f230806e140 0x7f23080705f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.649+0000 7f231ebd6700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f23180fde20 0x7f23180fbce0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.649+0000 7f231ebd6700 1 --2- 172.21.15.33:0/3042947789 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f23180fe7c0 0x7f23180fc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.649+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 >> 172.21.15.33:0/3042947789 conn(0x7f23180f7860 msgr2=0x7f23180f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:00.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.649+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 shutdown_connections 2024-04-25T02:13:00.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.649+0000 7f231ebd6700 1 -- 172.21.15.33:0/3042947789 wait complete. 2024-04-25T02:13:00.666 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:13:00.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.755+0000 7f4bf6a19700 1 -- 172.21.15.33:0/9526459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 msgr2=0x7f4bf00fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:00.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.755+0000 7f4bf6a19700 1 --2- 172.21.15.33:0/9526459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 0x7f4bf00fcdf0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4be4004940 tx=0x7f4be4009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:00.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.755+0000 7f4bf6a19700 1 -- 172.21.15.33:0/9526459 shutdown_connections 2024-04-25T02:13:00.762 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.755+0000 7f4bf6a19700 1 --2- 172.21.15.33:0/9526459 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 0x7f4bf00fcdf0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.755+0000 7f4bf6a19700 1 --2- 172.21.15.33:0/9526459 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0102de0 0x7f4bf01031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.755+0000 7f4bf6a19700 1 -- 172.21.15.33:0/9526459 >> 172.21.15.33:0/9526459 conn(0x7f4bf00f8990 msgr2=0x7f4bf00fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:00.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.755+0000 7f4bf6a19700 1 -- 172.21.15.33:0/9526459 shutdown_connections 2024-04-25T02:13:00.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.755+0000 7f4bf6a19700 1 -- 172.21.15.33:0/9526459 wait complete. 2024-04-25T02:13:00.763 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.756+0000 7f4bf6a19700 1 Processor -- start 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.756+0000 7f4bf6a19700 1 -- start start 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.756+0000 7f4bf6a19700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0102de0 0x7f4bf0113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.756+0000 7f4bf6a19700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 0x7f4bf0113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.756+0000 7f4bf6a19700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0114590 con 0x7f4bf0103780 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.756+0000 7f4bf6a19700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf010db20 con 0x7f4bf0102de0 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.756+0000 7f4bef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 0x7f4bf0113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4bef7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 0x7f4bf0113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48882/0 (socket says 172.21.15.33:48882) 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4bef7fe700 1 -- 172.21.15.33:0/3363001386 learned_addr learned my addr 172.21.15.33:0/3363001386 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4beffff700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0102de0 0x7f4bf0113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4bef7fe700 1 -- 172.21.15.33:0/3363001386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0102de0 msgr2=0x7f4bf0113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4bef7fe700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0102de0 0x7f4bf0113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4bef7fe700 1 -- 172.21.15.33:0/3363001386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be4005040 con 0x7f4bf0103780 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4bef7fe700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 0x7f4bf0113eb0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009100 tx=0x7f4be4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:00.765 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4becff9700 1 -- 172.21.15.33:0/3363001386 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be400e040 con 0x7f4bf0103780 2024-04-25T02:13:00.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4becff9700 1 -- 172.21.15.33:0/3363001386 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4be4013420 con 0x7f4bf0103780 2024-04-25T02:13:00.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4becff9700 1 -- 172.21.15.33:0/3363001386 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be401c610 con 0x7f4bf0103780 2024-04-25T02:13:00.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf010dda0 con 0x7f4bf0103780 2024-04-25T02:13:00.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.757+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf010e360 con 0x7f4bf0103780 2024-04-25T02:13:00.766 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.758+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf01000f0 con 0x7f4bf0103780 2024-04-25T02:13:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.759+0000 7f4becff9700 1 -- 172.21.15.33:0/3363001386 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4be4013590 con 0x7f4bf0103780 2024-04-25T02:13:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.759+0000 7f4becff9700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bd806e140 0x7f4bd80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.759+0000 7f4becff9700 1 -- 172.21.15.33:0/3363001386 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f4be408ec90 con 0x7f4bf0103780 2024-04-25T02:13:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.759+0000 7f4beffff700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bd806e140 0x7f4bd80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:00.767 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.760+0000 7f4beffff700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bd806e140 0x7f4bd80705f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4be0006fd0 tx=0x7f4be0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:00.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.764+0000 7f4becff9700 1 -- 172.21.15.33:0/3363001386 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4be4058130 con 0x7f4bf0103780 2024-04-25T02:13:00.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:00 smithi138 ceph-mon[81304]: pgmap v1009: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:00.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.990+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bf0109f30 con 0x7f4bd806e140 2024-04-25T02:13:00.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.991+0000 7f4becff9700 1 -- 172.21.15.33:0/3363001386 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4bf0109f30 con 0x7f4bd806e140 2024-04-25T02:13:01.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.993+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bd806e140 msgr2=0x7f4bd80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.993+0000 7f4bf6a19700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bd806e140 0x7f4bd80705f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4be0006fd0 tx=0x7f4be0008040 comp rx=0 tx=0).stop 2024-04-25T02:13:01.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.993+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 msgr2=0x7f4bf0113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.993+0000 7f4bf6a19700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 0x7f4bf0113eb0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009100 tx=0x7f4be4009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:01.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.994+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 shutdown_connections 2024-04-25T02:13:01.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.994+0000 7f4bf6a19700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f4bd806e140 0x7f4bd80705f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.994+0000 7f4bf6a19700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f4bf0102de0 0x7f4bf0113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.994+0000 7f4bf6a19700 1 --2- 172.21.15.33:0/3363001386 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4bf0103780 0x7f4bf0113eb0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.994+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 >> 172.21.15.33:0/3363001386 conn(0x7f4bf00f8990 msgr2=0x7f4bf00fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:01.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.994+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 shutdown_connections 2024-04-25T02:13:01.001 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:00.994+0000 7f4bf6a19700 1 -- 172.21.15.33:0/3363001386 wait complete. 2024-04-25T02:13:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.098+0000 7fd69a861700 1 -- 172.21.15.33:0/1659273786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 msgr2=0x7fd6941031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.098+0000 7fd69a861700 1 --2- 172.21.15.33:0/1659273786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 0x7fd6941031d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd684004940 tx=0x7fd684009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.098+0000 7fd69a861700 1 -- 172.21.15.33:0/1659273786 shutdown_connections 2024-04-25T02:13:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.098+0000 7fd69a861700 1 --2- 172.21.15.33:0/1659273786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6941037a0 0x7fd6940fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.098+0000 7fd69a861700 1 --2- 172.21.15.33:0/1659273786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 0x7fd6941031d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.098+0000 7fd69a861700 1 -- 172.21.15.33:0/1659273786 >> 172.21.15.33:0/1659273786 conn(0x7fd6940f89b0 msgr2=0x7fd6940fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.098+0000 7fd69a861700 1 -- 172.21.15.33:0/1659273786 shutdown_connections 2024-04-25T02:13:01.105 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.098+0000 7fd69a861700 1 -- 172.21.15.33:0/1659273786 wait complete. 2024-04-25T02:13:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.099+0000 7fd69a861700 1 Processor -- start 2024-04-25T02:13:01.107 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.099+0000 7fd69a861700 1 -- start start 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.099+0000 7fd69a861700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 0x7fd694113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.099+0000 7fd69a861700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6941037a0 0x7fd6941140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.099+0000 7fd69a861700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd694114710 con 0x7fd694102e00 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.099+0000 7fd69a861700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69410db70 con 0x7fd6941037a0 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.099+0000 7fd693fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 0x7fd694113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.099+0000 7fd693fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 0x7fd694113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48908/0 (socket says 172.21.15.33:48908) 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.099+0000 7fd693fff700 1 -- 172.21.15.33:0/2414307198 learned_addr learned my addr 172.21.15.33:0/2414307198 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd693fff700 1 -- 172.21.15.33:0/2414307198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6941037a0 msgr2=0x7fd6941140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd6937fe700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6941037a0 0x7fd6941140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd693fff700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6941037a0 0x7fd6941140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd693fff700 1 -- 172.21.15.33:0/2414307198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd684005040 con 0x7fd694102e00 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd6937fe700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6941037a0 0x7fd6941140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:13:01.108 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd693fff700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 0x7fd694113b80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd684009150 tx=0x7fd684009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd690ff9700 1 -- 172.21.15.33:0/2414307198 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd68400e040 con 0x7fd694102e00 2024-04-25T02:13:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd690ff9700 1 -- 172.21.15.33:0/2414307198 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd684013420 con 0x7fd694102e00 2024-04-25T02:13:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69410ddf0 con 0x7fd694102e00 2024-04-25T02:13:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd690ff9700 1 -- 172.21.15.33:0/2414307198 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd68401c630 con 0x7fd694102e00 2024-04-25T02:13:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.100+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69410e3b0 con 0x7fd694102e00 2024-04-25T02:13:01.109 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.101+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd694100110 con 0x7fd694102e00 2024-04-25T02:13:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.102+0000 7fd690ff9700 1 -- 172.21.15.33:0/2414307198 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd684013590 con 0x7fd694102e00 2024-04-25T02:13:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.102+0000 7fd690ff9700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd68006e150 0x7fd680070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.102+0000 7fd690ff9700 1 -- 172.21.15.33:0/2414307198 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd68408eda0 con 0x7fd694102e00 2024-04-25T02:13:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.102+0000 7fd6937fe700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd68006e150 0x7fd680070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:01.110 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.103+0000 7fd6937fe700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd68006e150 0x7fd680070600 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd67c007fd0 tx=0x7fd67c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:01.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.107+0000 7fd690ff9700 1 -- 172.21.15.33:0/2414307198 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd6840581a0 con 0x7fd694102e00 2024-04-25T02:13:01.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.328+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd694109f70 con 0x7fd68006e150 2024-04-25T02:13:01.344 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.338+0000 7fd690ff9700 1 -- 172.21.15.33:0/2414307198 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd694109f70 con 0x7fd68006e150 2024-04-25T02:13:01.344 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:13:01.344 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (34m) 11m ago 45m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (31m) 11m ago 45m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (31m) 11m ago 44m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (33m) 11m ago 45m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (27m) 11m ago 39m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (27m) 11m ago 40m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (27m) 11m ago 39m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (11m) 11m ago 39m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (35m) 11m ago 46m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (35m) 11m ago 44m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (32m) 11m ago 46m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (32m) 11m ago 44m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (34m) 11m ago 45m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:13:01.345 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (34m) 11m ago 44m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:13:01.346 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (30m) 11m ago 43m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:13:01.346 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (30m) 11m ago 43m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:13:01.346 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (29m) 11m ago 42m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:13:01.346 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (29m) 11m ago 42m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:13:01.346 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (28m) 11m ago 41m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:13:01.346 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (28m) 11m ago 41m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:13:01.346 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (34m) 11m ago 44m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd68006e150 msgr2=0x7fd680070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd68006e150 0x7fd680070600 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd67c007fd0 tx=0x7fd67c009040 comp rx=0 tx=0).stop 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 msgr2=0x7fd694113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 0x7fd694113b80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fd684009150 tx=0x7fd684009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 shutdown_connections 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd68006e150 0x7fd680070600 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd694102e00 0x7fd694113b80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 --2- 172.21.15.33:0/2414307198 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd6941037a0 0x7fd6941140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 >> 172.21.15.33:0/2414307198 conn(0x7fd6940f89b0 msgr2=0x7fd6941073c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:01.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 shutdown_connections 2024-04-25T02:13:01.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.340+0000 7fd69a861700 1 -- 172.21.15.33:0/2414307198 wait complete. 2024-04-25T02:13:01.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 -- 172.21.15.33:0/423361068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8298102e20 msgr2=0x7f82981031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 --2- 172.21.15.33:0/423361068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8298102e20 0x7f82981031f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8288004940 tx=0x7f8288009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 -- 172.21.15.33:0/423361068 shutdown_connections 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 --2- 172.21.15.33:0/423361068 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f82981037c0 0x7f82980fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 --2- 172.21.15.33:0/423361068 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8298102e20 0x7f82981031f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 -- 172.21.15.33:0/423361068 >> 172.21.15.33:0/423361068 conn(0x7f8298072380 msgr2=0x7f8298072780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 -- 172.21.15.33:0/423361068 shutdown_connections 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 -- 172.21.15.33:0/423361068 wait complete. 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 Processor -- start 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.444+0000 7f829d53a700 1 -- start start 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f829d53a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298102e20 0x7f8298115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:01.453 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f829d53a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82981037c0 0x7f82981160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f829d53a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8298116740 con 0x7f82981037c0 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f829d53a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829810fd70 con 0x7f8298102e20 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f82967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82981037c0 0x7f82981160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f82967fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82981037c0 0x7f82981160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48928/0 (socket says 172.21.15.33:48928) 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f82967fc700 1 -- 172.21.15.33:0/1984612928 learned_addr learned my addr 172.21.15.33:0/1984612928 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f8296ffd700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298102e20 0x7f8298115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f82967fc700 1 -- 172.21.15.33:0/1984612928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298102e20 msgr2=0x7f8298115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f82967fc700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298102e20 0x7f8298115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f82967fc700 1 -- 172.21.15.33:0/1984612928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288005040 con 0x7f82981037c0 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f82967fc700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82981037c0 0x7f82981160f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f828c002a50 tx=0x7f828c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f827f7fe700 1 -- 172.21.15.33:0/1984612928 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828c007cc0 con 0x7f82981037c0 2024-04-25T02:13:01.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f827f7fe700 1 -- 172.21.15.33:0/1984612928 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f828c00f040 con 0x7f82981037c0 2024-04-25T02:13:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.445+0000 7f829d53a700 1 -- 172.21.15.33:0/1984612928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8298110050 con 0x7f82981037c0 2024-04-25T02:13:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.446+0000 7f827f7fe700 1 -- 172.21.15.33:0/1984612928 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828c00da40 con 0x7f82981037c0 2024-04-25T02:13:01.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.446+0000 7f829d53a700 1 -- 172.21.15.33:0/1984612928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8298110590 con 0x7f82981037c0 2024-04-25T02:13:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.446+0000 7f827d7fa700 1 -- 172.21.15.33:0/1984612928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82780052b0 con 0x7f82981037c0 2024-04-25T02:13:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.447+0000 7f827f7fe700 1 -- 172.21.15.33:0/1984612928 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f828c0097c0 con 0x7f82981037c0 2024-04-25T02:13:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.447+0000 7f827f7fe700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f828006e140 0x7f82800705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.448+0000 7f827f7fe700 1 -- 172.21.15.33:0/1984612928 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f828c08d950 con 0x7f82981037c0 2024-04-25T02:13:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.448+0000 7f8296ffd700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f828006e140 0x7f82800705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:01.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.448+0000 7f8296ffd700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f828006e140 0x7f82800705f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8288000c00 tx=0x7f8288009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:01.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.452+0000 7f827f7fe700 1 -- 172.21.15.33:0/1984612928 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f828c05a800 con 0x7f82981037c0 2024-04-25T02:13:01.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.748+0000 7f827d7fa700 1 -- 172.21.15.33:0/1984612928 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8278005d70 con 0x7f82981037c0 2024-04-25T02:13:01.755 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.749+0000 7f827f7fe700 1 -- 172.21.15.33:0/1984612928 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f828c05a390 con 0x7f82981037c0 2024-04-25T02:13:01.755 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:13:01.755 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:13:01.756 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:13:01.757 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:13:01.757 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:13:01.757 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:13:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 -- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f828006e140 msgr2=0x7f82800705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f828006e140 0x7f82800705f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8288000c00 tx=0x7f8288009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 -- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82981037c0 msgr2=0x7f82981160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82981037c0 0x7f82981160f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f828c002a50 tx=0x7f828c004600 comp rx=0 tx=0).stop 2024-04-25T02:13:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 -- 172.21.15.33:0/1984612928 shutdown_connections 2024-04-25T02:13:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f828006e140 0x7f82800705f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.758 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8298102e20 0x7f8298115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 --2- 172.21.15.33:0/1984612928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f82981037c0 0x7f82981160f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 -- 172.21.15.33:0/1984612928 >> 172.21.15.33:0/1984612928 conn(0x7f8298072380 msgr2=0x7f82980fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 -- 172.21.15.33:0/1984612928 shutdown_connections 2024-04-25T02:13:01.759 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.751+0000 7f827d7fa700 1 -- 172.21.15.33:0/1984612928 wait complete. 2024-04-25T02:13:01.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.856+0000 7f98e1af5700 1 -- 172.21.15.33:0/4226675563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 msgr2=0x7f98dc1020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.856+0000 7f98e1af5700 1 --2- 172.21.15.33:0/4226675563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 0x7f98dc1020e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f98cc004940 tx=0x7f98cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:01.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.856+0000 7f98e1af5700 1 -- 172.21.15.33:0/4226675563 shutdown_connections 2024-04-25T02:13:01.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.856+0000 7f98e1af5700 1 --2- 172.21.15.33:0/4226675563 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98dc1026b0 0x7f98dc0fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.856+0000 7f98e1af5700 1 --2- 172.21.15.33:0/4226675563 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 0x7f98dc1020e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.856+0000 7f98e1af5700 1 -- 172.21.15.33:0/4226675563 >> 172.21.15.33:0/4226675563 conn(0x7f98dc0f7860 msgr2=0x7f98dc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:01.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.856+0000 7f98e1af5700 1 -- 172.21.15.33:0/4226675563 shutdown_connections 2024-04-25T02:13:01.863 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.856+0000 7f98e1af5700 1 -- 172.21.15.33:0/4226675563 wait complete. 2024-04-25T02:13:01.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.857+0000 7f98e1af5700 1 Processor -- start 2024-04-25T02:13:01.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.857+0000 7f98e1af5700 1 -- start start 2024-04-25T02:13:01.865 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.857+0000 7f98e1af5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 0x7f98dc113a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.857+0000 7f98e1af5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98dc1026b0 0x7f98dc113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.857+0000 7f98e1af5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc114500 con 0x7f98dc101d10 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.857+0000 7f98e1af5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc10db50 con 0x7f98dc1026b0 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.857+0000 7f98db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 0x7f98dc113a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.857+0000 7f98db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 0x7f98dc113a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48944/0 (socket says 172.21.15.33:48944) 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.857+0000 7f98db7fe700 1 -- 172.21.15.33:0/1084056855 learned_addr learned my addr 172.21.15.33:0/1084056855 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98db7fe700 1 -- 172.21.15.33:0/1084056855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98dc1026b0 msgr2=0x7f98dc113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98daffd700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98dc1026b0 0x7f98dc113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98db7fe700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98dc1026b0 0x7f98dc113f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98db7fe700 1 -- 172.21.15.33:0/1084056855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc005040 con 0x7f98dc101d10 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98db7fe700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 0x7f98dc113a10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00b3b0 tx=0x7f98cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:01.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98c3fff700 1 -- 172.21.15.33:0/1084056855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98cc00e040 con 0x7f98dc101d10 2024-04-25T02:13:01.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc10ddd0 con 0x7f98dc101d10 2024-04-25T02:13:01.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98c3fff700 1 -- 172.21.15.33:0/1084056855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98cc00bbc0 con 0x7f98dc101d10 2024-04-25T02:13:01.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98c3fff700 1 -- 172.21.15.33:0/1084056855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98cc01c3f0 con 0x7f98dc101d10 2024-04-25T02:13:01.867 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.858+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc10e390 con 0x7f98dc101d10 2024-04-25T02:13:01.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.859+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc0fd460 con 0x7f98dc101d10 2024-04-25T02:13:01.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.860+0000 7f98c3fff700 1 -- 172.21.15.33:0/1084056855 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f98cc00bd30 con 0x7f98dc101d10 2024-04-25T02:13:01.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.860+0000 7f98c3fff700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f98c406e140 0x7f98c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:01.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.860+0000 7f98c3fff700 1 -- 172.21.15.33:0/1084056855 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f98cc08edc0 con 0x7f98dc101d10 2024-04-25T02:13:01.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.860+0000 7f98daffd700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f98c406e140 0x7f98c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:01.868 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.861+0000 7f98daffd700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f98c406e140 0x7f98c40705f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0f90d0 tx=0x7f98d0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:01.871 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:01.865+0000 7f98c3fff700 1 -- 172.21.15.33:0/1084056855 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f98cc058140 con 0x7f98dc101d10 2024-04-25T02:13:02.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.118+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f98dc10eff0 con 0x7f98dc101d10 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.118+0000 7f98c3fff700 1 -- 172.21.15.33:0/1084056855 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f98cc05b800 con 0x7f98dc101d10 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:13:02.125 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:13:02.126 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:13:02.127 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:13:02.128 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:13:02.128 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:13:02.128 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:13:02.128 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:13:02.128 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:13:02.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f98c406e140 msgr2=0x7f98c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f98c406e140 0x7f98c40705f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0f90d0 tx=0x7f98d0009040 comp rx=0 tx=0).stop 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 msgr2=0x7f98dc113a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 0x7f98dc113a10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f98cc00b3b0 tx=0x7f98cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 shutdown_connections 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f98c406e140 0x7f98c40705f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f98dc101d10 0x7f98dc113a10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 --2- 172.21.15.33:0/1084056855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f98dc1026b0 0x7f98dc113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 >> 172.21.15.33:0/1084056855 conn(0x7f98dc0f7860 msgr2=0x7f98dc066270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.120+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 shutdown_connections 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.121+0000 7f98e1af5700 1 -- 172.21.15.33:0/1084056855 wait complete. 2024-04-25T02:13:02.129 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:13:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.225+0000 7f907214d700 1 -- 172.21.15.33:0/3404672766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c103780 msgr2=0x7f906c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.225+0000 7f907214d700 1 --2- 172.21.15.33:0/3404672766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c103780 0x7f906c0fcdf0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9060004940 tx=0x7f9060009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.225+0000 7f907214d700 1 -- 172.21.15.33:0/3404672766 shutdown_connections 2024-04-25T02:13:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.225+0000 7f907214d700 1 --2- 172.21.15.33:0/3404672766 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c103780 0x7f906c0fcdf0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.225+0000 7f907214d700 1 --2- 172.21.15.33:0/3404672766 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c102de0 0x7f906c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.225+0000 7f907214d700 1 -- 172.21.15.33:0/3404672766 >> 172.21.15.33:0/3404672766 conn(0x7f906c0f8990 msgr2=0x7f906c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:02.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.225+0000 7f907214d700 1 -- 172.21.15.33:0/3404672766 shutdown_connections 2024-04-25T02:13:02.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.225+0000 7f907214d700 1 -- 172.21.15.33:0/3404672766 wait complete. 2024-04-25T02:13:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.226+0000 7f907214d700 1 Processor -- start 2024-04-25T02:13:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.226+0000 7f907214d700 1 -- start start 2024-04-25T02:13:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.226+0000 7f907214d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c102de0 0x7f906c113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.226+0000 7f907214d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c103780 0x7f906c113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:02.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.226+0000 7f907214d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c1144e0 con 0x7f906c102de0 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.226+0000 7f907214d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f906c10dae0 con 0x7f906c103780 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c102de0 0x7f906c113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f906b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c102de0 0x7f906c113950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48964/0 (socket says 172.21.15.33:48964) 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f906b7fe700 1 -- 172.21.15.33:0/547224158 learned_addr learned my addr 172.21.15.33:0/547224158 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f906affd700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c103780 0x7f906c113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f906b7fe700 1 -- 172.21.15.33:0/547224158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c103780 msgr2=0x7f906c113e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f906b7fe700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c103780 0x7f906c113e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f906b7fe700 1 -- 172.21.15.33:0/547224158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9060005040 con 0x7f906c102de0 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f906b7fe700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c102de0 0x7f906c113950 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f905c002a50 tx=0x7f905c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f9053fff700 1 -- 172.21.15.33:0/547224158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f905c007cc0 con 0x7f906c102de0 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f906c10ddc0 con 0x7f906c102de0 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f9053fff700 1 -- 172.21.15.33:0/547224158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f905c00f040 con 0x7f906c102de0 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f9053fff700 1 -- 172.21.15.33:0/547224158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f905c00da30 con 0x7f906c102de0 2024-04-25T02:13:02.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.227+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c10e350 con 0x7f906c102de0 2024-04-25T02:13:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.228+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f906c0fe530 con 0x7f906c102de0 2024-04-25T02:13:02.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.229+0000 7f9053fff700 1 -- 172.21.15.33:0/547224158 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f905c00dbb0 con 0x7f906c102de0 2024-04-25T02:13:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.229+0000 7f9053fff700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f905406e140 0x7f90540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.229+0000 7f9053fff700 1 -- 172.21.15.33:0/547224158 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f905c08d8c0 con 0x7f906c102de0 2024-04-25T02:13:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.229+0000 7f906affd700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f905406e140 0x7f90540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:02.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.230+0000 7f906affd700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f905406e140 0x7f90540705f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9060004890 tx=0x7f9060009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:02.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.234+0000 7f9053fff700 1 -- 172.21.15.33:0/547224158 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f905c05a770 con 0x7f906c102de0 2024-04-25T02:13:02.469 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.462+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f906c109ec0 con 0x7f905406e140 2024-04-25T02:13:02.469 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:02 smithi033 ceph-mon[111306]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:02.469 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:02 smithi033 ceph-mon[111306]: from='client.35836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:02.469 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:02 smithi033 ceph-mon[111306]: pgmap v1010: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:02.469 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:02 smithi033 ceph-mon[111306]: from='client.35840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:02.470 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:02 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1984612928' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:13:02.470 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:02 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1084056855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:13:02.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.464+0000 7f9053fff700 1 -- 172.21.15.33:0/547224158 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f906c109ec0 con 0x7f905406e140 2024-04-25T02:13:02.470 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:13:02.470 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:13:02.470 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:13:02.471 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f905406e140 msgr2=0x7f90540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f905406e140 0x7f90540705f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9060004890 tx=0x7f9060009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c102de0 msgr2=0x7f906c113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c102de0 0x7f906c113950 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f905c002a50 tx=0x7f905c0046b0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 shutdown_connections 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f905406e140 0x7f90540705f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f906c102de0 0x7f906c113950 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 --2- 172.21.15.33:0/547224158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f906c103780 0x7f906c113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 >> 172.21.15.33:0/547224158 conn(0x7f906c0f8990 msgr2=0x7f906c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 shutdown_connections 2024-04-25T02:13:02.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.466+0000 7f907214d700 1 -- 172.21.15.33:0/547224158 wait complete. 2024-04-25T02:13:02.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.570+0000 7f7a06b44700 1 -- 172.21.15.33:0/2116244240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 msgr2=0x7f7a000fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.570+0000 7f7a06b44700 1 --2- 172.21.15.33:0/2116244240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 0x7f7a000fbd20 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f79ec004940 tx=0x7f79ec009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.570+0000 7f7a06b44700 1 -- 172.21.15.33:0/2116244240 shutdown_connections 2024-04-25T02:13:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.570+0000 7f7a06b44700 1 --2- 172.21.15.33:0/2116244240 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 0x7f7a000fbd20 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.570+0000 7f7a06b44700 1 --2- 172.21.15.33:0/2116244240 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00101d10 0x7f7a001020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.570+0000 7f7a06b44700 1 -- 172.21.15.33:0/2116244240 >> 172.21.15.33:0/2116244240 conn(0x7f7a000f7860 msgr2=0x7f7a000f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.570+0000 7f7a06b44700 1 -- 172.21.15.33:0/2116244240 shutdown_connections 2024-04-25T02:13:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.570+0000 7f7a06b44700 1 -- 172.21.15.33:0/2116244240 wait complete. 2024-04-25T02:13:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.571+0000 7f7a06b44700 1 Processor -- start 2024-04-25T02:13:02.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.571+0000 7f7a06b44700 1 -- start start 2024-04-25T02:13:02.579 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.571+0000 7f7a06b44700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00101d10 0x7f7a00115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.571+0000 7f7a06b44700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 0x7f7a001160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.571+0000 7f7a06b44700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a00116740 con 0x7f7a001026b0 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.571+0000 7f7a06b44700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0010fd20 con 0x7f7a00101d10 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.571+0000 7f79fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 0x7f7a001160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.571+0000 7f79fffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 0x7f7a001160f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:48984/0 (socket says 172.21.15.33:48984) 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.571+0000 7f79fffff700 1 -- 172.21.15.33:0/3310919975 learned_addr learned my addr 172.21.15.33:0/3310919975 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f79fffff700 1 -- 172.21.15.33:0/3310919975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00101d10 msgr2=0x7f7a00115bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f7a048e0700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00101d10 0x7f7a00115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f79fffff700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00101d10 0x7f7a00115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f79fffff700 1 -- 172.21.15.33:0/3310919975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec005040 con 0x7f7a001026b0 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f79fffff700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 0x7f7a001160f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009100 tx=0x7f79ec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:02.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f79fd7fa700 1 -- 172.21.15.33:0/3310919975 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ec00e040 con 0x7f7a001026b0 2024-04-25T02:13:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0010ffa0 con 0x7f7a001026b0 2024-04-25T02:13:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f79fd7fa700 1 -- 172.21.15.33:0/3310919975 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79ec004290 con 0x7f7a001026b0 2024-04-25T02:13:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f79fd7fa700 1 -- 172.21.15.33:0/3310919975 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ec01c3f0 con 0x7f7a001026b0 2024-04-25T02:13:02.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.572+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a00110560 con 0x7f7a001026b0 2024-04-25T02:13:02.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.573+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a000fd460 con 0x7f7a001026b0 2024-04-25T02:13:02.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.574+0000 7f79fd7fa700 1 -- 172.21.15.33:0/3310919975 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f79ec013790 con 0x7f7a001026b0 2024-04-25T02:13:02.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.574+0000 7f79fd7fa700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f79f006e140 0x7f79f00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:02.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.574+0000 7f79fd7fa700 1 -- 172.21.15.33:0/3310919975 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f79ec08ede0 con 0x7f7a001026b0 2024-04-25T02:13:02.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.574+0000 7f7a048e0700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f79f006e140 0x7f79f00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:02.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.575+0000 7f7a048e0700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f79f006e140 0x7f79f00705f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f79f4004d60 tx=0x7f79f400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:02.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.579+0000 7f79fd7fa700 1 -- 172.21.15.33:0/3310919975 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f79ec05bd10 con 0x7f7a001026b0 2024-04-25T02:13:02.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:02 smithi138 ceph-mon[81304]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:02.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:02 smithi138 ceph-mon[81304]: from='client.35836 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:02.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:02 smithi138 ceph-mon[81304]: pgmap v1010: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:02.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:02 smithi138 ceph-mon[81304]: from='client.35840 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:02.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:02 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1984612928' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:13:02.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:02 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1084056855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:13:02.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.874+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7a0004c710 con 0x7f7a001026b0 2024-04-25T02:13:02.881 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.875+0000 7f79fd7fa700 1 -- 172.21.15.33:0/3310919975 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f79ec013420 con 0x7f7a001026b0 2024-04-25T02:13:02.882 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:13:02.882 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:13:02.882 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:13:02.882 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:13:02.882 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:13:02.882 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:13:02.882 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:13:02.882 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:13:02.882 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:13:02.883 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:13:02.884 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:13:02.884 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:13:02.884 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:13:02.884 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:13:02.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.877+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f79f006e140 msgr2=0x7f79f00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.884 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.877+0000 7f7a06b44700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f79f006e140 0x7f79f00705f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f79f4004d60 tx=0x7f79f400b040 comp rx=0 tx=0).stop 2024-04-25T02:13:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.877+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 msgr2=0x7f7a001160f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.877+0000 7f7a06b44700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 0x7f7a001160f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f79ec009100 tx=0x7f79ec009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.878+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 shutdown_connections 2024-04-25T02:13:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.878+0000 7f7a06b44700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f79f006e140 0x7f79f00705f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.878+0000 7f7a06b44700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7a00101d10 0x7f7a00115bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.878+0000 7f7a06b44700 1 --2- 172.21.15.33:0/3310919975 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7a001026b0 0x7f7a001160f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.878+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 >> 172.21.15.33:0/3310919975 conn(0x7f7a000f7860 msgr2=0x7f7a000f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.878+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 shutdown_connections 2024-04-25T02:13:02.885 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:02.878+0000 7f7a06b44700 1 -- 172.21.15.33:0/3310919975 wait complete. 2024-04-25T02:13:03.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:03 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3310919975' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:13:03.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:03 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3310919975' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:13:04.699 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:04 smithi138 ceph-mon[81304]: from='client.35852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:04.699 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:04 smithi138 ceph-mon[81304]: pgmap v1011: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:04.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:04 smithi033 ceph-mon[111306]: from='client.35852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:04.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:04 smithi033 ceph-mon[111306]: pgmap v1011: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:06.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:06 smithi033 ceph-mon[111306]: pgmap v1012: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:06.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:06 smithi138 ceph-mon[81304]: pgmap v1012: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:08.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:08 smithi033 ceph-mon[111306]: pgmap v1013: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:08.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:08 smithi138 ceph-mon[81304]: pgmap v1013: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:10.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:10 smithi033 ceph-mon[111306]: pgmap v1014: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:10.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:13:10.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:10 smithi138 ceph-mon[81304]: pgmap v1014: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:10.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:13:12.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:12 smithi033 ceph-mon[111306]: pgmap v1015: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:12.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:12 smithi138 ceph-mon[81304]: pgmap v1015: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:14.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:14 smithi033 ceph-mon[111306]: pgmap v1016: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:14.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:14 smithi138 ceph-mon[81304]: pgmap v1016: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:16.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:16 smithi033 ceph-mon[111306]: pgmap v1017: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:16.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:16 smithi138 ceph-mon[81304]: pgmap v1017: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:18.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:18 smithi033 ceph-mon[111306]: pgmap v1018: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:18.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:18 smithi138 ceph-mon[81304]: pgmap v1018: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:20.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:20 smithi033 ceph-mon[111306]: pgmap v1019: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:20.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:20 smithi138 ceph-mon[81304]: pgmap v1019: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:22.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:22 smithi033 ceph-mon[111306]: pgmap v1020: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:22.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:22 smithi138 ceph-mon[81304]: pgmap v1020: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:24.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:24 smithi033 ceph-mon[111306]: pgmap v1021: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:24.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:24 smithi138 ceph-mon[81304]: pgmap v1021: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:25.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:13:25.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:13:26.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:26 smithi033 ceph-mon[111306]: pgmap v1022: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:26.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:26 smithi138 ceph-mon[81304]: pgmap v1022: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:28.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:28 smithi033 ceph-mon[111306]: pgmap v1023: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:28.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:28 smithi138 ceph-mon[81304]: pgmap v1023: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:30.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:30 smithi033 ceph-mon[111306]: pgmap v1024: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:30.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:30 smithi138 ceph-mon[81304]: pgmap v1024: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:32.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:32 smithi033 ceph-mon[111306]: pgmap v1025: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:32.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:32 smithi138 ceph-mon[81304]: pgmap v1025: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.996+0000 7f6a5db94700 1 -- 172.21.15.33:0/1838302675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fde10 msgr2=0x7f6a580fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.996+0000 7f6a5db94700 1 --2- 172.21.15.33:0/1838302675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fde10 0x7f6a580fe1e0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6a48004940 tx=0x7f6a48009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.996+0000 7f6a5db94700 1 -- 172.21.15.33:0/1838302675 shutdown_connections 2024-04-25T02:13:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.996+0000 7f6a5db94700 1 --2- 172.21.15.33:0/1838302675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a580fe7b0 0x7f6a58106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.996+0000 7f6a5db94700 1 --2- 172.21.15.33:0/1838302675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fde10 0x7f6a580fe1e0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.996+0000 7f6a5db94700 1 -- 172.21.15.33:0/1838302675 >> 172.21.15.33:0/1838302675 conn(0x7f6a580f7820 msgr2=0x7f6a580f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.996+0000 7f6a5db94700 1 -- 172.21.15.33:0/1838302675 shutdown_connections 2024-04-25T02:13:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.996+0000 7f6a5db94700 1 -- 172.21.15.33:0/1838302675 wait complete. 2024-04-25T02:13:33.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a5db94700 1 Processor -- start 2024-04-25T02:13:33.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a5db94700 1 -- start start 2024-04-25T02:13:33.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a5db94700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a580fde10 0x7f6a58113c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:33.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a5db94700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fe7b0 0x7f6a581141c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:33.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a5db94700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58114810 con 0x7f6a580fe7b0 2024-04-25T02:13:33.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a5db94700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5810ddf0 con 0x7f6a580fde10 2024-04-25T02:13:33.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fe7b0 0x7f6a581141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:33.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fe7b0 0x7f6a581141c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45100/0 (socket says 172.21.15.33:45100) 2024-04-25T02:13:33.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a56ffd700 1 -- 172.21.15.33:0/2748266104 learned_addr learned my addr 172.21.15.33:0/2748266104 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.997+0000 7f6a577fe700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a580fde10 0x7f6a58113c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.998+0000 7f6a56ffd700 1 -- 172.21.15.33:0/2748266104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a580fde10 msgr2=0x7f6a58113c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.998+0000 7f6a56ffd700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a580fde10 0x7f6a58113c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.998+0000 7f6a56ffd700 1 -- 172.21.15.33:0/2748266104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a48005040 con 0x7f6a580fe7b0 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.998+0000 7f6a56ffd700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fe7b0 0x7f6a581141c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c002a50 tx=0x7f6a4c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.998+0000 7f6a3ffff700 1 -- 172.21.15.33:0/2748266104 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a4c007cc0 con 0x7f6a580fe7b0 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.998+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5810e0d0 con 0x7f6a580fe7b0 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.998+0000 7f6a3ffff700 1 -- 172.21.15.33:0/2748266104 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a4c00f040 con 0x7f6a580fe7b0 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.998+0000 7f6a3ffff700 1 -- 172.21.15.33:0/2748266104 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a4c009640 con 0x7f6a580fe7b0 2024-04-25T02:13:33.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.998+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5810e610 con 0x7f6a580fe7b0 2024-04-25T02:13:33.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:32.999+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a58066b30 con 0x7f6a580fe7b0 2024-04-25T02:13:33.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.000+0000 7f6a3ffff700 1 -- 172.21.15.33:0/2748266104 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6a4c0097a0 con 0x7f6a580fe7b0 2024-04-25T02:13:33.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.000+0000 7f6a3ffff700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a4006e140 0x7f6a400705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:33.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.000+0000 7f6a3ffff700 1 -- 172.21.15.33:0/2748266104 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f6a4c08d9d0 con 0x7f6a580fe7b0 2024-04-25T02:13:33.007 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.000+0000 7f6a577fe700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a4006e140 0x7f6a400705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:33.008 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.001+0000 7f6a577fe700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a4006e140 0x7f6a400705f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a48004f90 tx=0x7f6a48009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:33.011 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.005+0000 7f6a3ffff700 1 -- 172.21.15.33:0/2748266104 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6a4c05a7e0 con 0x7f6a580fe7b0 2024-04-25T02:13:33.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.237+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a58109f10 con 0x7f6a4006e140 2024-04-25T02:13:33.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.238+0000 7f6a3ffff700 1 -- 172.21.15.33:0/2748266104 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6a58109f10 con 0x7f6a4006e140 2024-04-25T02:13:33.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a4006e140 msgr2=0x7f6a400705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a4006e140 0x7f6a400705f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6a48004f90 tx=0x7f6a48009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:33.246 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fe7b0 msgr2=0x7f6a581141c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fe7b0 0x7f6a581141c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c002a50 tx=0x7f6a4c0046b0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 shutdown_connections 2024-04-25T02:13:33.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f6a4006e140 0x7f6a400705f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6a580fde10 0x7f6a58113c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 --2- 172.21.15.33:0/2748266104 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6a580fe7b0 0x7f6a581141c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 >> 172.21.15.33:0/2748266104 conn(0x7f6a580f7820 msgr2=0x7f6a580f8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:33.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 shutdown_connections 2024-04-25T02:13:33.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.240+0000 7f6a5db94700 1 -- 172.21.15.33:0/2748266104 wait complete. 2024-04-25T02:13:33.257 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:13:33.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.345+0000 7f210fe46700 1 -- 172.21.15.33:0/3582375928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101d10 msgr2=0x7f21081020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.345+0000 7f210fe46700 1 --2- 172.21.15.33:0/3582375928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101d10 0x7f21081020e0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2104004940 tx=0x7f2104009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:33.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.345+0000 7f210fe46700 1 -- 172.21.15.33:0/3582375928 shutdown_connections 2024-04-25T02:13:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.345+0000 7f210fe46700 1 --2- 172.21.15.33:0/3582375928 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f21081026b0 0x7f21080fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.345+0000 7f210fe46700 1 --2- 172.21.15.33:0/3582375928 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2108101d10 0x7f21081020e0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.346+0000 7f210fe46700 1 -- 172.21.15.33:0/3582375928 >> 172.21.15.33:0/3582375928 conn(0x7f21080f7860 msgr2=0x7f21080f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.346+0000 7f210fe46700 1 -- 172.21.15.33:0/3582375928 shutdown_connections 2024-04-25T02:13:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.346+0000 7f210fe46700 1 -- 172.21.15.33:0/3582375928 wait complete. 2024-04-25T02:13:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.346+0000 7f210fe46700 1 Processor -- start 2024-04-25T02:13:33.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.346+0000 7f210fe46700 1 -- start start 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210fe46700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108101d10 0x7f2108113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210fe46700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21081026b0 0x7f2108113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210fe46700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2108114500 con 0x7f21081026b0 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210fe46700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210810db00 con 0x7f2108101d10 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210d3e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21081026b0 0x7f2108113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210d3e1700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21081026b0 0x7f2108113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45122/0 (socket says 172.21.15.33:45122) 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210d3e1700 1 -- 172.21.15.33:0/1443058381 learned_addr learned my addr 172.21.15.33:0/1443058381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210dbe2700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108101d10 0x7f2108113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210d3e1700 1 -- 172.21.15.33:0/1443058381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108101d10 msgr2=0x7f2108113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210d3e1700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108101d10 0x7f2108113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210d3e1700 1 -- 172.21.15.33:0/1443058381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2104005040 con 0x7f21081026b0 2024-04-25T02:13:33.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210d3e1700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21081026b0 0x7f2108113eb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f20f800c800 tx=0x7f20f8004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f20fe7fc700 1 -- 172.21.15.33:0/1443058381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20f800a760 con 0x7f21081026b0 2024-04-25T02:13:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f20fe7fc700 1 -- 172.21.15.33:0/1443058381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20f800ad90 con 0x7f21081026b0 2024-04-25T02:13:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210810dde0 con 0x7f21081026b0 2024-04-25T02:13:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f20fe7fc700 1 -- 172.21.15.33:0/1443058381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20f80113a0 con 0x7f21081026b0 2024-04-25T02:13:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.347+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210810e3a0 con 0x7f21081026b0 2024-04-25T02:13:33.356 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.348+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21080ff020 con 0x7f21081026b0 2024-04-25T02:13:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.349+0000 7f20fe7fc700 1 -- 172.21.15.33:0/1443058381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f20f8012400 con 0x7f21081026b0 2024-04-25T02:13:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.349+0000 7f20fe7fc700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20f406e140 0x7f20f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.349+0000 7f20fe7fc700 1 -- 172.21.15.33:0/1443058381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f20f808db40 con 0x7f21081026b0 2024-04-25T02:13:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.349+0000 7f210dbe2700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20f406e140 0x7f20f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:33.357 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.350+0000 7f210dbe2700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20f406e140 0x7f20f40705f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2104004f90 tx=0x7f2104009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:33.360 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.354+0000 7f20fe7fc700 1 -- 172.21.15.33:0/1443058381 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f20f805aa70 con 0x7f21081026b0 2024-04-25T02:13:33.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.581+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f210810f030 con 0x7f20f406e140 2024-04-25T02:13:33.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.582+0000 7f20fe7fc700 1 -- 172.21.15.33:0/1443058381 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f210810f030 con 0x7f20f406e140 2024-04-25T02:13:33.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20f406e140 msgr2=0x7f20f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.590 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20f406e140 0x7f20f40705f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2104004f90 tx=0x7f2104009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21081026b0 msgr2=0x7f2108113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21081026b0 0x7f2108113eb0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f20f800c800 tx=0x7f20f8004470 comp rx=0 tx=0).stop 2024-04-25T02:13:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 shutdown_connections 2024-04-25T02:13:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f20f406e140 0x7f20f40705f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2108101d10 0x7f2108113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 --2- 172.21.15.33:0/1443058381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f21081026b0 0x7f2108113eb0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 >> 172.21.15.33:0/1443058381 conn(0x7f21080f7860 msgr2=0x7f2108106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 shutdown_connections 2024-04-25T02:13:33.591 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.584+0000 7f210fe46700 1 -- 172.21.15.33:0/1443058381 wait complete. 2024-04-25T02:13:33.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.688+0000 7f3ab8121700 1 -- 172.21.15.33:0/2440049227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 msgr2=0x7f3ab01020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.688+0000 7f3ab8121700 1 --2- 172.21.15.33:0/2440049227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 0x7f3ab01020e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4004940 tx=0x7f3aa4009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:33.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.688+0000 7f3ab8121700 1 -- 172.21.15.33:0/2440049227 shutdown_connections 2024-04-25T02:13:33.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.688+0000 7f3ab8121700 1 --2- 172.21.15.33:0/2440049227 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab01026b0 0x7f3ab00fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.688+0000 7f3ab8121700 1 --2- 172.21.15.33:0/2440049227 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 0x7f3ab01020e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.688+0000 7f3ab8121700 1 -- 172.21.15.33:0/2440049227 >> 172.21.15.33:0/2440049227 conn(0x7f3ab00f7860 msgr2=0x7f3ab00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:33.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.689+0000 7f3ab8121700 1 -- 172.21.15.33:0/2440049227 shutdown_connections 2024-04-25T02:13:33.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.689+0000 7f3ab8121700 1 -- 172.21.15.33:0/2440049227 wait complete. 2024-04-25T02:13:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.689+0000 7f3ab8121700 1 Processor -- start 2024-04-25T02:13:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.689+0000 7f3ab8121700 1 -- start start 2024-04-25T02:13:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.689+0000 7f3ab8121700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 0x7f3ab010eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.689+0000 7f3ab8121700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab01026b0 0x7f3ab010f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:33.697 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.689+0000 7f3ab8121700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab010f600 con 0x7f3ab0101d10 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.689+0000 7f3ab8121700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab010f740 con 0x7f3ab01026b0 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab5ebd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 0x7f3ab010eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab5ebd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 0x7f3ab010eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45136/0 (socket says 172.21.15.33:45136) 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab5ebd700 1 -- 172.21.15.33:0/2057395381 learned_addr learned my addr 172.21.15.33:0/2057395381 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab56bc700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab01026b0 0x7f3ab010f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab5ebd700 1 -- 172.21.15.33:0/2057395381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab01026b0 msgr2=0x7f3ab010f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab5ebd700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab01026b0 0x7f3ab010f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab5ebd700 1 -- 172.21.15.33:0/2057395381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa4005040 con 0x7f3ab0101d10 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab5ebd700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 0x7f3ab010eb80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400b8b0 tx=0x7f3aa4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3aa27fc700 1 -- 172.21.15.33:0/2057395381 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aa400e040 con 0x7f3ab0101d10 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3aa27fc700 1 -- 172.21.15.33:0/2057395381 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3aa4004290 con 0x7f3ab0101d10 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab010f9c0 con 0x7f3ab0101d10 2024-04-25T02:13:33.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3aa27fc700 1 -- 172.21.15.33:0/2057395381 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aa401c3f0 con 0x7f3ab0101d10 2024-04-25T02:13:33.699 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.690+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab0117f90 con 0x7f3ab0101d10 2024-04-25T02:13:33.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.691+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab00fd460 con 0x7f3ab0101d10 2024-04-25T02:13:33.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.692+0000 7f3aa27fc700 1 -- 172.21.15.33:0/2057395381 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3aa40137b0 con 0x7f3ab0101d10 2024-04-25T02:13:33.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.693+0000 7f3aa27fc700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3a9c06e140 0x7f3a9c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:33.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.693+0000 7f3aa27fc700 1 -- 172.21.15.33:0/2057395381 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3aa408ede0 con 0x7f3ab0101d10 2024-04-25T02:13:33.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.693+0000 7f3ab56bc700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3a9c06e140 0x7f3a9c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:33.700 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.693+0000 7f3ab56bc700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3a9c06e140 0x7f3a9c0705f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006fd0 tx=0x7f3aac008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:33.704 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.698+0000 7f3aa27fc700 1 -- 172.21.15.33:0/2057395381 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3aa40581e0 con 0x7f3ab0101d10 2024-04-25T02:13:33.921 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.915+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ab00668b0 con 0x7f3a9c06e140 2024-04-25T02:13:33.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.925+0000 7f3aa27fc700 1 -- 172.21.15.33:0/2057395381 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3ab00668b0 con 0x7f3a9c06e140 2024-04-25T02:13:33.931 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:13:33.931 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (34m) 12m ago 45m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:13:33.931 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (31m) 12m ago 46m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:13:33.931 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (31m) 12m ago 45m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:13:33.931 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (34m) 12m ago 45m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (27m) 12m ago 40m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (28m) 12m ago 40m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (27m) 12m ago 40m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (12m) 12m ago 40m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (36m) 12m ago 47m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (35m) 12m ago 45m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (33m) 12m ago 47m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (32m) 12m ago 44m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (35m) 12m ago 46m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (35m) 12m ago 45m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (31m) 12m ago 44m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (30m) 12m ago 43m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (30m) 12m ago 43m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (29m) 12m ago 42m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:13:33.932 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (29m) 12m ago 42m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:13:33.933 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (28m) 12m ago 42m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:13:33.933 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (35m) 12m ago 45m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:13:33.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3a9c06e140 msgr2=0x7f3a9c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3a9c06e140 0x7f3a9c0705f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f3aac006fd0 tx=0x7f3aac008040 comp rx=0 tx=0).stop 2024-04-25T02:13:33.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 msgr2=0x7f3ab010eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:33.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 0x7f3ab010eb80 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3aa400b8b0 tx=0x7f3aa4009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:33.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 shutdown_connections 2024-04-25T02:13:33.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3a9c06e140 0x7f3a9c0705f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3ab0101d10 0x7f3ab010eb80 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 --2- 172.21.15.33:0/2057395381 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3ab01026b0 0x7f3ab010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:33.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 >> 172.21.15.33:0/2057395381 conn(0x7f3ab00f7860 msgr2=0x7f3ab00f87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:33.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 shutdown_connections 2024-04-25T02:13:33.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:33.927+0000 7f3ab8121700 1 -- 172.21.15.33:0/2057395381 wait complete. 2024-04-25T02:13:34.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.029+0000 7ff0e3901700 1 -- 172.21.15.33:0/234534688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fc190 msgr2=0x7ff0dc101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.029+0000 7ff0e3901700 1 --2- 172.21.15.33:0/234534688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fc190 0x7ff0dc101a70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc004940 tx=0x7ff0cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.029+0000 7ff0e3901700 1 -- 172.21.15.33:0/234534688 shutdown_connections 2024-04-25T02:13:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.029+0000 7ff0e3901700 1 --2- 172.21.15.33:0/234534688 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fc190 0x7ff0dc101a70 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.029+0000 7ff0e3901700 1 --2- 172.21.15.33:0/234534688 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0dc0fb7f0 0x7ff0dc0fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.029+0000 7ff0e3901700 1 -- 172.21.15.33:0/234534688 >> 172.21.15.33:0/234534688 conn(0x7ff0dc071460 msgr2=0x7ff0dc071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.029+0000 7ff0e3901700 1 -- 172.21.15.33:0/234534688 shutdown_connections 2024-04-25T02:13:34.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.029+0000 7ff0e3901700 1 -- 172.21.15.33:0/234534688 wait complete. 2024-04-25T02:13:34.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e3901700 1 Processor -- start 2024-04-25T02:13:34.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e3901700 1 -- start start 2024-04-25T02:13:34.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e3901700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fb7f0 0x7ff0dc0fc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:34.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e3901700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0dc0fc190 0x7ff0dc0fccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:34.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e3901700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0fe6d0 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e3901700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0dc0fd200 con 0x7ff0dc0fc190 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e169d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fb7f0 0x7ff0dc0fc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e169d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fb7f0 0x7ff0dc0fc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45146/0 (socket says 172.21.15.33:45146) 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e169d700 1 -- 172.21.15.33:0/1485422376 learned_addr learned my addr 172.21.15.33:0/1485422376 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.030+0000 7ff0e0e9c700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0dc0fc190 0x7ff0dc0fccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.031+0000 7ff0e169d700 1 -- 172.21.15.33:0/1485422376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0dc0fc190 msgr2=0x7ff0dc0fccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.031+0000 7ff0e169d700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0dc0fc190 0x7ff0dc0fccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.031+0000 7ff0e169d700 1 -- 172.21.15.33:0/1485422376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc005040 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.031+0000 7ff0e169d700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fb7f0 0x7ff0dc0fc780 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8002a50 tx=0x7ff0d80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.031+0000 7ff0d1ffb700 1 -- 172.21.15.33:0/1485422376 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0d8007cc0 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.031+0000 7ff0d1ffb700 1 -- 172.21.15.33:0/1485422376 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0d800f040 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.031+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0dc0fd4e0 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.031+0000 7ff0d1ffb700 1 -- 172.21.15.33:0/1485422376 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0d800da30 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.031+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0dc11a0b0 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.032+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0dc065f30 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.033+0000 7ff0d1ffb700 1 -- 172.21.15.33:0/1485422376 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff0d800db90 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.033+0000 7ff0d1ffb700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0c806e140 0x7ff0c80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:34.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.033+0000 7ff0d1ffb700 1 -- 172.21.15.33:0/1485422376 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff0d808d800 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.033+0000 7ff0e0e9c700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0c806e140 0x7ff0c80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:34.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.034+0000 7ff0e0e9c700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0c806e140 0x7ff0c80705f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00b3b0 tx=0x7ff0cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:34.044 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.038+0000 7ff0d1ffb700 1 -- 172.21.15.33:0/1485422376 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff0d8056c00 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.332+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff0dc0fdcf0 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.332+0000 7ff0d1ffb700 1 -- 172.21.15.33:0/1485422376 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff0d805a2c0 con 0x7ff0dc0fb7f0 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:13:34.339 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:13:34.340 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:13:34.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:13:34.340 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:13:34.340 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:13:34.340 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:13:34.340 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:13:34.340 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:13:34.340 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:13:34.340 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:13:34.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0c806e140 msgr2=0x7ff0c80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0c806e140 0x7ff0c80705f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00b3b0 tx=0x7ff0cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:34.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fb7f0 msgr2=0x7ff0dc0fc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fb7f0 0x7ff0dc0fc780 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8002a50 tx=0x7ff0d80046b0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 shutdown_connections 2024-04-25T02:13:34.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff0c806e140 0x7ff0c80705f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.341 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff0dc0fb7f0 0x7ff0dc0fc780 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 --2- 172.21.15.33:0/1485422376 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff0dc0fc190 0x7ff0dc0fccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 >> 172.21.15.33:0/1485422376 conn(0x7ff0dc071460 msgr2=0x7ff0dc100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 shutdown_connections 2024-04-25T02:13:34.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.335+0000 7ff0e3901700 1 -- 172.21.15.33:0/1485422376 wait complete. 2024-04-25T02:13:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.439+0000 7f80f3377700 1 -- 172.21.15.33:0/1209088657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec103780 msgr2=0x7f80ec0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.439+0000 7f80f3377700 1 --2- 172.21.15.33:0/1209088657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec103780 0x7f80ec0fcdf0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f80e0004940 tx=0x7f80e0009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.439+0000 7f80f3377700 1 -- 172.21.15.33:0/1209088657 shutdown_connections 2024-04-25T02:13:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.439+0000 7f80f3377700 1 --2- 172.21.15.33:0/1209088657 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec103780 0x7f80ec0fcdf0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.439+0000 7f80f3377700 1 --2- 172.21.15.33:0/1209088657 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec102de0 0x7f80ec1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.439+0000 7f80f3377700 1 -- 172.21.15.33:0/1209088657 >> 172.21.15.33:0/1209088657 conn(0x7f80ec0f8990 msgr2=0x7f80ec0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.440+0000 7f80f3377700 1 -- 172.21.15.33:0/1209088657 shutdown_connections 2024-04-25T02:13:34.446 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.440+0000 7f80f3377700 1 -- 172.21.15.33:0/1209088657 wait complete. 2024-04-25T02:13:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.440+0000 7f80f3377700 1 Processor -- start 2024-04-25T02:13:34.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.440+0000 7f80f3377700 1 -- start start 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.440+0000 7f80f3377700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec102de0 0x7f80ec113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.440+0000 7f80f3377700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec103780 0x7f80ec113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.440+0000 7f80f3377700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec114520 con 0x7f80ec102de0 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.440+0000 7f80f3377700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec10db20 con 0x7f80ec103780 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80f1113700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec102de0 0x7f80ec113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80f1113700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec102de0 0x7f80ec113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45164/0 (socket says 172.21.15.33:45164) 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80f1113700 1 -- 172.21.15.33:0/1982527855 learned_addr learned my addr 172.21.15.33:0/1982527855 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80f0912700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec103780 0x7f80ec113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80f1113700 1 -- 172.21.15.33:0/1982527855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec103780 msgr2=0x7f80ec113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80f1113700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec103780 0x7f80ec113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80f1113700 1 -- 172.21.15.33:0/1982527855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0005040 con 0x7f80ec102de0 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80f1113700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec102de0 0x7f80ec113990 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f80e400cac0 tx=0x7f80e4007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:34.449 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80e9ffb700 1 -- 172.21.15.33:0/1982527855 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e400aa50 con 0x7f80ec102de0 2024-04-25T02:13:34.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80e9ffb700 1 -- 172.21.15.33:0/1982527855 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f80e400abb0 con 0x7f80ec102de0 2024-04-25T02:13:34.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec10de00 con 0x7f80ec102de0 2024-04-25T02:13:34.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.441+0000 7f80e9ffb700 1 -- 172.21.15.33:0/1982527855 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80e40115e0 con 0x7f80ec102de0 2024-04-25T02:13:34.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.442+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec10e3c0 con 0x7f80ec102de0 2024-04-25T02:13:34.450 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.442+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ec0fe530 con 0x7f80ec102de0 2024-04-25T02:13:34.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.443+0000 7f80e9ffb700 1 -- 172.21.15.33:0/1982527855 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f80e400ad20 con 0x7f80ec102de0 2024-04-25T02:13:34.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.443+0000 7f80e9ffb700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f80d406e140 0x7f80d40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:34.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.443+0000 7f80e9ffb700 1 -- 172.21.15.33:0/1982527855 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f80e408e260 con 0x7f80ec102de0 2024-04-25T02:13:34.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.444+0000 7f80f0912700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f80d406e140 0x7f80d40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:34.451 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.444+0000 7f80f0912700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f80d406e140 0x7f80d40705f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b3b0 tx=0x7f80e0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:34.454 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.448+0000 7f80e9ffb700 1 -- 172.21.15.33:0/1982527855 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f80e405b110 con 0x7f80ec102de0 2024-04-25T02:13:34.710 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.704+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f80ec109ec0 con 0x7f80ec102de0 2024-04-25T02:13:34.710 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:34 smithi033 ceph-mon[111306]: from='client.35860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:34.711 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:34 smithi033 ceph-mon[111306]: pgmap v1026: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:34.711 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:34 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1485422376' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:13:34.711 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.704+0000 7f80e9ffb700 1 -- 172.21.15.33:0/1982527855 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f80e405aca0 con 0x7f80ec102de0 2024-04-25T02:13:34.711 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:13:34.711 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:13:34.711 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:13:34.711 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:13:34.711 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:13:34.712 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:13:34.713 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:13:34.714 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:13:34.714 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:13:34.714 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:13:34.714 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:13:34.714 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:13:34.714 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:13:34.714 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f80d406e140 msgr2=0x7f80d40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f80d406e140 0x7f80d40705f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f80e000b3b0 tx=0x7f80e0009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec102de0 msgr2=0x7f80ec113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec102de0 0x7f80ec113990 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f80e400cac0 tx=0x7f80e4007ee0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 shutdown_connections 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f80d406e140 0x7f80d40705f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f80ec102de0 0x7f80ec113990 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 --2- 172.21.15.33:0/1982527855 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f80ec103780 0x7f80ec113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 >> 172.21.15.33:0/1982527855 conn(0x7f80ec0f8990 msgr2=0x7f80ec0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 shutdown_connections 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.707+0000 7f80f3377700 1 -- 172.21.15.33:0/1982527855 wait complete. 2024-04-25T02:13:34.715 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:13:34.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:34 smithi138 ceph-mon[81304]: from='client.35860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:34.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:34 smithi138 ceph-mon[81304]: pgmap v1026: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:34.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:34 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1485422376' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:13:34.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.812+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/671580324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 msgr2=0x7f0f480fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.812+0000 7f0f4fdbe700 1 --2- 172.21.15.33:0/671580324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 0x7f0f480fcdf0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0f38004940 tx=0x7f0f38009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:34.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.812+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/671580324 shutdown_connections 2024-04-25T02:13:34.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.812+0000 7f0f4fdbe700 1 --2- 172.21.15.33:0/671580324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 0x7f0f480fcdf0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.812+0000 7f0f4fdbe700 1 --2- 172.21.15.33:0/671580324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102de0 0x7f0f481031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.812+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/671580324 >> 172.21.15.33:0/671580324 conn(0x7f0f480f8990 msgr2=0x7f0f480fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:34.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.813+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/671580324 shutdown_connections 2024-04-25T02:13:34.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.813+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/671580324 wait complete. 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.813+0000 7f0f4fdbe700 1 Processor -- start 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.813+0000 7f0f4fdbe700 1 -- start start 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.813+0000 7f0f4fdbe700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102de0 0x7f0f48113970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.813+0000 7f0f4fdbe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 0x7f0f48113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.813+0000 7f0f4fdbe700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f48114590 con 0x7f0f48103780 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.813+0000 7f0f4fdbe700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4810db20 con 0x7f0f48102de0 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4d359700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 0x7f0f48113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4d359700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 0x7f0f48113eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45186/0 (socket says 172.21.15.33:45186) 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4d359700 1 -- 172.21.15.33:0/2830035987 learned_addr learned my addr 172.21.15.33:0/2830035987 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4db5a700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102de0 0x7f0f48113970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4d359700 1 -- 172.21.15.33:0/2830035987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102de0 msgr2=0x7f0f48113970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4d359700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102de0 0x7f0f48113970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:34.821 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4d359700 1 -- 172.21.15.33:0/2830035987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f38005040 con 0x7f0f48103780 2024-04-25T02:13:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4db5a700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102de0 0x7f0f48113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:13:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4d359700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 0x7f0f48113eb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009180 tx=0x7f0f38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f3e7fc700 1 -- 172.21.15.33:0/2830035987 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f3800e040 con 0x7f0f48103780 2024-04-25T02:13:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f3e7fc700 1 -- 172.21.15.33:0/2830035987 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f3800bbc0 con 0x7f0f48103780 2024-04-25T02:13:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4810dda0 con 0x7f0f48103780 2024-04-25T02:13:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f3e7fc700 1 -- 172.21.15.33:0/2830035987 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f3801c3f0 con 0x7f0f48103780 2024-04-25T02:13:34.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.814+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4810e360 con 0x7f0f48103780 2024-04-25T02:13:34.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.815+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f481000f0 con 0x7f0f48103780 2024-04-25T02:13:34.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.816+0000 7f0f3e7fc700 1 -- 172.21.15.33:0/2830035987 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0f3800bd30 con 0x7f0f48103780 2024-04-25T02:13:34.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.816+0000 7f0f3e7fc700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f3406e140 0x7f0f340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:34.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.816+0000 7f0f3e7fc700 1 -- 172.21.15.33:0/2830035987 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0f3808ee60 con 0x7f0f48103780 2024-04-25T02:13:34.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.816+0000 7f0f4db5a700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f3406e140 0x7f0f340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:34.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.817+0000 7f0f4db5a700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f3406e140 0x7f0f340705f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0f44006fd0 tx=0x7f0f44008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:34.827 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:34.821+0000 7f0f3e7fc700 1 -- 172.21.15.33:0/2830035987 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0f38058260 con 0x7f0f48103780 2024-04-25T02:13:35.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.048+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f4810f030 con 0x7f0f3406e140 2024-04-25T02:13:35.055 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.049+0000 7f0f3e7fc700 1 -- 172.21.15.33:0/2830035987 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0f4810f030 con 0x7f0f3406e140 2024-04-25T02:13:35.055 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:13:35.055 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:13:35.055 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:13:35.055 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:13:35.055 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:13:35.056 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f3406e140 msgr2=0x7f0f340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f3406e140 0x7f0f340705f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0f44006fd0 tx=0x7f0f44008040 comp rx=0 tx=0).stop 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 msgr2=0x7f0f48113eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 0x7f0f48113eb0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009180 tx=0x7f0f38009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 shutdown_connections 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0f3406e140 0x7f0f340705f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0f48102de0 0x7f0f48113970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 --2- 172.21.15.33:0/2830035987 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0f48103780 0x7f0f48113eb0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 >> 172.21.15.33:0/2830035987 conn(0x7f0f480f8990 msgr2=0x7f0f480fa750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 shutdown_connections 2024-04-25T02:13:35.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.051+0000 7f0f4fdbe700 1 -- 172.21.15.33:0/2830035987 wait complete. 2024-04-25T02:13:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.154+0000 7f70de671700 1 -- 172.21.15.33:0/1500608417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fde10 msgr2=0x7f70d80fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.154+0000 7f70de671700 1 --2- 172.21.15.33:0/1500608417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fde10 0x7f70d80fe1e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f70cc004940 tx=0x7f70cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:35.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.154+0000 7f70de671700 1 -- 172.21.15.33:0/1500608417 shutdown_connections 2024-04-25T02:13:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.154+0000 7f70de671700 1 --2- 172.21.15.33:0/1500608417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d80fe7b0 0x7f70d8106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.154+0000 7f70de671700 1 --2- 172.21.15.33:0/1500608417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fde10 0x7f70d80fe1e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.154+0000 7f70de671700 1 -- 172.21.15.33:0/1500608417 >> 172.21.15.33:0/1500608417 conn(0x7f70d80f7820 msgr2=0x7f70d80f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.155+0000 7f70de671700 1 -- 172.21.15.33:0/1500608417 shutdown_connections 2024-04-25T02:13:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.155+0000 7f70de671700 1 -- 172.21.15.33:0/1500608417 wait complete. 2024-04-25T02:13:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.155+0000 7f70de671700 1 Processor -- start 2024-04-25T02:13:35.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.155+0000 7f70de671700 1 -- start start 2024-04-25T02:13:35.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.155+0000 7f70de671700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d80fde10 0x7f70d8115e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.155+0000 7f70de671700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fe7b0 0x7f70d8116390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.155+0000 7f70de671700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81169e0 con 0x7f70d80fe7b0 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.155+0000 7f70de671700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d810ffc0 con 0x7f70d80fde10 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fe7b0 0x7f70d8116390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d77fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fe7b0 0x7f70d8116390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45206/0 (socket says 172.21.15.33:45206) 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d77fe700 1 -- 172.21.15.33:0/2323113330 learned_addr learned my addr 172.21.15.33:0/2323113330 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d7fff700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d80fde10 0x7f70d8115e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d77fe700 1 -- 172.21.15.33:0/2323113330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d80fde10 msgr2=0x7f70d8115e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d77fe700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d80fde10 0x7f70d8115e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d77fe700 1 -- 172.21.15.33:0/2323113330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70cc005040 con 0x7f70d80fe7b0 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d77fe700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fe7b0 0x7f70d8116390 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f70c8002a50 tx=0x7f70c80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:35.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d4ff9700 1 -- 172.21.15.33:0/2323113330 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70c8007cc0 con 0x7f70d80fe7b0 2024-04-25T02:13:35.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d81102a0 con 0x7f70d80fe7b0 2024-04-25T02:13:35.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d4ff9700 1 -- 172.21.15.33:0/2323113330 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70c800f040 con 0x7f70d80fe7b0 2024-04-25T02:13:35.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70d4ff9700 1 -- 172.21.15.33:0/2323113330 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70c800da30 con 0x7f70d80fe7b0 2024-04-25T02:13:35.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.156+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d81107e0 con 0x7f70d80fe7b0 2024-04-25T02:13:35.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.158+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d8066b30 con 0x7f70d80fe7b0 2024-04-25T02:13:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.158+0000 7f70d4ff9700 1 -- 172.21.15.33:0/2323113330 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f70c800db90 con 0x7f70d80fe7b0 2024-04-25T02:13:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.158+0000 7f70d4ff9700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70c006e140 0x7f70c00705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:13:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.159+0000 7f70d4ff9700 1 -- 172.21.15.33:0/2323113330 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f70c808d8c0 con 0x7f70d80fe7b0 2024-04-25T02:13:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.159+0000 7f70d7fff700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70c006e140 0x7f70c00705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:13:35.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.159+0000 7f70d7fff700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70c006e140 0x7f70c00705f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f70cc004f90 tx=0x7f70cc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:13:35.169 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.163+0000 7f70d4ff9700 1 -- 172.21.15.33:0/2323113330 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f70c805a770 con 0x7f70d80fe7b0 2024-04-25T02:13:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.456+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f70d804c710 con 0x7f70d80fe7b0 2024-04-25T02:13:35.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.457+0000 7f70d4ff9700 1 -- 172.21.15.33:0/2323113330 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f70c805a300 con 0x7f70d80fe7b0 2024-04-25T02:13:35.463 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:13:35.463 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:13:35.464 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:13:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:13:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:13:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:13:35.465 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:13:35.465 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:13:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:13:35.465 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:13:35.465 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:13:35.465 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70c006e140 msgr2=0x7f70c00705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70c006e140 0x7f70c00705f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f70cc004f90 tx=0x7f70cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fe7b0 msgr2=0x7f70d8116390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fe7b0 0x7f70d8116390 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f70c8002a50 tx=0x7f70c80046b0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 shutdown_connections 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f70c006e140 0x7f70c00705f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f70d80fde10 0x7f70d8115e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 --2- 172.21.15.33:0/2323113330 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f70d80fe7b0 0x7f70d8116390 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 >> 172.21.15.33:0/2323113330 conn(0x7f70d80f7820 msgr2=0x7f70d80f8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 shutdown_connections 2024-04-25T02:13:35.466 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:13:35.459+0000 7f70de671700 1 -- 172.21.15.33:0/2323113330 wait complete. 2024-04-25T02:13:35.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:35 smithi033 ceph-mon[111306]: from='client.35864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:35.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:35 smithi033 ceph-mon[111306]: from='client.35868 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:35.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:35 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1982527855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:13:35.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:35 smithi138 ceph-mon[81304]: from='client.35864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:35.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:35 smithi138 ceph-mon[81304]: from='client.35868 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:35.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:35 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1982527855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:13:36.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:36 smithi033 ceph-mon[111306]: from='client.35880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:36.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:36 smithi033 ceph-mon[111306]: pgmap v1027: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:36.678 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:36 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2323113330' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:13:36.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:36 smithi138 ceph-mon[81304]: from='client.35880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:13:36.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:36 smithi138 ceph-mon[81304]: pgmap v1027: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:36.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:36 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2323113330' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:13:38.272 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:37 smithi033 ceph-mon[111306]: pgmap v1028: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:38.297 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:37 smithi138 ceph-mon[81304]: pgmap v1028: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:40.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:40 smithi033 ceph-mon[111306]: pgmap v1029: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:40.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:13:40.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:40 smithi138 ceph-mon[81304]: pgmap v1029: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:40.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:13:42.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:42 smithi033 ceph-mon[111306]: pgmap v1030: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:42.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:42 smithi138 ceph-mon[81304]: pgmap v1030: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:44.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:44 smithi033 ceph-mon[111306]: pgmap v1031: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:44.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:44 smithi138 ceph-mon[81304]: pgmap v1031: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:46.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:46 smithi033 ceph-mon[111306]: pgmap v1032: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:46.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:46 smithi138 ceph-mon[81304]: pgmap v1032: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:48.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:48 smithi033 ceph-mon[111306]: pgmap v1033: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:48.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:48 smithi138 ceph-mon[81304]: pgmap v1033: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:50.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:50 smithi033 ceph-mon[111306]: pgmap v1034: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:50.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:50 smithi138 ceph-mon[81304]: pgmap v1034: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:52.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:52 smithi033 ceph-mon[111306]: pgmap v1035: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:52.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:52 smithi138 ceph-mon[81304]: pgmap v1035: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:54.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:54 smithi033 ceph-mon[111306]: pgmap v1036: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:54.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:54 smithi138 ceph-mon[81304]: pgmap v1036: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:55.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:13:55.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:13:56.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:56 smithi033 ceph-mon[111306]: pgmap v1037: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:56.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:56 smithi138 ceph-mon[81304]: pgmap v1037: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:58.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:13:58 smithi033 ceph-mon[111306]: pgmap v1038: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:13:58.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:13:58 smithi138 ceph-mon[81304]: pgmap v1038: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:00.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:00 smithi033 ceph-mon[111306]: pgmap v1039: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:00.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:00 smithi138 ceph-mon[81304]: pgmap v1039: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:02.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:02 smithi033 ceph-mon[111306]: pgmap v1040: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:02.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:02 smithi138 ceph-mon[81304]: pgmap v1040: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:04.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:04 smithi033 ceph-mon[111306]: pgmap v1041: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:04.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:04 smithi138 ceph-mon[81304]: pgmap v1041: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.578+0000 7f7884a02700 1 -- 172.21.15.33:0/4064865913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 msgr2=0x7f7880065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:05.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.578+0000 7f7884a02700 1 --2- 172.21.15.33:0/4064865913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 0x7f7880065ab0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7870004940 tx=0x7f7870009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.578+0000 7f7884a02700 1 -- 172.21.15.33:0/4064865913 shutdown_connections 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 --2- 172.21.15.33:0/4064865913 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 0x7f7880065ab0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 --2- 172.21.15.33:0/4064865913 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7880102e50 0x7f7880103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 -- 172.21.15.33:0/4064865913 >> 172.21.15.33:0/4064865913 conn(0x7f7880071d00 msgr2=0x7f7880072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 -- 172.21.15.33:0/4064865913 shutdown_connections 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 -- 172.21.15.33:0/4064865913 wait complete. 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 Processor -- start 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 -- start start 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 0x7f78800fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7880102e50 0x7f78800fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800fc880 con 0x7f7880065640 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.579+0000 7f7884a02700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800fc9c0 con 0x7f7880102e50 2024-04-25T02:14:05.585 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f787e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 0x7f78800fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f787e59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 0x7f78800fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:33996/0 (socket says 172.21.15.33:33996) 2024-04-25T02:14:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f787e59c700 1 -- 172.21.15.33:0/2556653016 learned_addr learned my addr 172.21.15.33:0/2556653016 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:05.586 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f787dd9b700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7880102e50 0x7f78800fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f787e59c700 1 -- 172.21.15.33:0/2556653016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7880102e50 msgr2=0x7f78800fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f787e59c700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7880102e50 0x7f78800fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f787e59c700 1 -- 172.21.15.33:0/2556653016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7870005040 con 0x7f7880065640 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f787e59c700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 0x7f78800fbdb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7868002a50 tx=0x7f78680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f7876ffd700 1 -- 172.21.15.33:0/2556653016 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7868007cc0 con 0x7f7880065640 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f7876ffd700 1 -- 172.21.15.33:0/2556653016 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f786800f040 con 0x7f7880065640 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7880119e90 con 0x7f7880065640 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f7876ffd700 1 -- 172.21.15.33:0/2556653016 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786800da70 con 0x7f7880065640 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.580+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788011a310 con 0x7f7880065640 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.581+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7880100100 con 0x7f7880065640 2024-04-25T02:14:05.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.582+0000 7f7876ffd700 1 -- 172.21.15.33:0/2556653016 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f786800dc10 con 0x7f7880065640 2024-04-25T02:14:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.582+0000 7f7876ffd700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f786c06e140 0x7f786c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.583+0000 7f7876ffd700 1 -- 172.21.15.33:0/2556653016 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f786808d170 con 0x7f7880065640 2024-04-25T02:14:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.583+0000 7f787dd9b700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f786c06e140 0x7f786c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:05.588 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.583+0000 7f787dd9b700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f786c06e140 0x7f786c0705f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7870004f90 tx=0x7f7870009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:05.592 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.587+0000 7f7876ffd700 1 -- 172.21.15.33:0/2556653016 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f78680564f0 con 0x7f7880065640 2024-04-25T02:14:05.819 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.814+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f788004bed0 con 0x7f786c06e140 2024-04-25T02:14:05.820 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.815+0000 7f7876ffd700 1 -- 172.21.15.33:0/2556653016 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f788004bed0 con 0x7f786c06e140 2024-04-25T02:14:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f786c06e140 msgr2=0x7f786c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f786c06e140 0x7f786c0705f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f7870004f90 tx=0x7f7870009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 msgr2=0x7f78800fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 0x7f78800fbdb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7868002a50 tx=0x7f78680046b0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 shutdown_connections 2024-04-25T02:14:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f786c06e140 0x7f786c0705f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.822 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7880065640 0x7f78800fbdb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 --2- 172.21.15.33:0/2556653016 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7880102e50 0x7f78800fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 >> 172.21.15.33:0/2556653016 conn(0x7f7880071d00 msgr2=0x7f78800faf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 shutdown_connections 2024-04-25T02:14:05.823 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.817+0000 7f7884a02700 1 -- 172.21.15.33:0/2556653016 wait complete. 2024-04-25T02:14:05.832 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:14:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.921+0000 7f8411b52700 1 -- 172.21.15.33:0/2120381341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c1026b0 msgr2=0x7f840c0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.921+0000 7f8411b52700 1 --2- 172.21.15.33:0/2120381341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c1026b0 0x7f840c0fbd20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8400004940 tx=0x7f8400009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.921+0000 7f8411b52700 1 -- 172.21.15.33:0/2120381341 shutdown_connections 2024-04-25T02:14:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.921+0000 7f8411b52700 1 --2- 172.21.15.33:0/2120381341 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c1026b0 0x7f840c0fbd20 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.921+0000 7f8411b52700 1 --2- 172.21.15.33:0/2120381341 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f840c101d10 0x7f840c1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.921+0000 7f8411b52700 1 -- 172.21.15.33:0/2120381341 >> 172.21.15.33:0/2120381341 conn(0x7f840c0f7860 msgr2=0x7f840c0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:05.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.922+0000 7f8411b52700 1 -- 172.21.15.33:0/2120381341 shutdown_connections 2024-04-25T02:14:05.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.922+0000 7f8411b52700 1 -- 172.21.15.33:0/2120381341 wait complete. 2024-04-25T02:14:05.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.922+0000 7f8411b52700 1 Processor -- start 2024-04-25T02:14:05.928 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.922+0000 7f8411b52700 1 -- start start 2024-04-25T02:14:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.922+0000 7f8411b52700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c101d10 0x7f840c115bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.922+0000 7f8411b52700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f840c1026b0 0x7f840c1160f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f8411b52700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c116740 con 0x7f840c101d10 2024-04-25T02:14:05.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f8411b52700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c10fd20 con 0x7f840c1026b0 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c101d10 0x7f840c115bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c101d10 0x7f840c115bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45130/0 (socket says 172.21.15.33:45130) 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f840b7fe700 1 -- 172.21.15.33:0/2130949940 learned_addr learned my addr 172.21.15.33:0/2130949940 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f840affd700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f840c1026b0 0x7f840c1160f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f840b7fe700 1 -- 172.21.15.33:0/2130949940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f840c1026b0 msgr2=0x7f840c1160f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f840b7fe700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f840c1026b0 0x7f840c1160f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f840b7fe700 1 -- 172.21.15.33:0/2130949940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400005040 con 0x7f840c101d10 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f840b7fe700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c101d10 0x7f840c115bb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f83fc002a50 tx=0x7f83fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f83f3fff700 1 -- 172.21.15.33:0/2130949940 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc007cc0 con 0x7f840c101d10 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f83f3fff700 1 -- 172.21.15.33:0/2130949940 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83fc00f040 con 0x7f840c101d10 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.923+0000 7f83f3fff700 1 -- 172.21.15.33:0/2130949940 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc00da30 con 0x7f840c101d10 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.924+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c110000 con 0x7f840c101d10 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.924+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c110590 con 0x7f840c101d10 2024-04-25T02:14:05.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.925+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c0fd460 con 0x7f840c101d10 2024-04-25T02:14:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.925+0000 7f83f3fff700 1 -- 172.21.15.33:0/2130949940 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f83fc00db90 con 0x7f840c101d10 2024-04-25T02:14:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.926+0000 7f83f3fff700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83f406e140 0x7f83f40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.926+0000 7f840affd700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83f406e140 0x7f83f40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.926+0000 7f83f3fff700 1 -- 172.21.15.33:0/2130949940 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f83fc08d7f0 con 0x7f840c101d10 2024-04-25T02:14:05.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.926+0000 7f840affd700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83f406e140 0x7f83f40705f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8400004890 tx=0x7f8400009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:05.935 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:05.930+0000 7f83f3fff700 1 -- 172.21.15.33:0/2130949940 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f83fc05a720 con 0x7f840c101d10 2024-04-25T02:14:06.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.157+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f840c111270 con 0x7f83f406e140 2024-04-25T02:14:06.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.158+0000 7f83f3fff700 1 -- 172.21.15.33:0/2130949940 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f840c111270 con 0x7f83f406e140 2024-04-25T02:14:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83f406e140 msgr2=0x7f83f40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83f406e140 0x7f83f40705f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8400004890 tx=0x7f8400009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c101d10 msgr2=0x7f840c115bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c101d10 0x7f840c115bb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f83fc002a50 tx=0x7f83fc0046b0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 shutdown_connections 2024-04-25T02:14:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f83f406e140 0x7f83f40705f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f840c101d10 0x7f840c115bb0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 --2- 172.21.15.33:0/2130949940 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f840c1026b0 0x7f840c1160f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 >> 172.21.15.33:0/2130949940 conn(0x7f840c0f7860 msgr2=0x7f840c0f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:06.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 shutdown_connections 2024-04-25T02:14:06.166 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.160+0000 7f8411b52700 1 -- 172.21.15.33:0/2130949940 wait complete. 2024-04-25T02:14:06.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.264+0000 7f5b1326e700 1 -- 172.21.15.33:0/3456442762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 msgr2=0x7f5b0c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.264+0000 7f5b1326e700 1 --2- 172.21.15.33:0/3456442762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 0x7f5b0c1031c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f5afc004940 tx=0x7f5afc009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:06.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.264+0000 7f5b1326e700 1 -- 172.21.15.33:0/3456442762 shutdown_connections 2024-04-25T02:14:06.269 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.264+0000 7f5b1326e700 1 --2- 172.21.15.33:0/3456442762 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b0c103790 0x7f5b0c0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.264+0000 7f5b1326e700 1 --2- 172.21.15.33:0/3456442762 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 0x7f5b0c1031c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.264+0000 7f5b1326e700 1 -- 172.21.15.33:0/3456442762 >> 172.21.15.33:0/3456442762 conn(0x7f5b0c0f8980 msgr2=0x7f5b0c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:06.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.264+0000 7f5b1326e700 1 -- 172.21.15.33:0/3456442762 shutdown_connections 2024-04-25T02:14:06.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.264+0000 7f5b1326e700 1 -- 172.21.15.33:0/3456442762 wait complete. 2024-04-25T02:14:06.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1326e700 1 Processor -- start 2024-04-25T02:14:06.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1326e700 1 -- start start 2024-04-25T02:14:06.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1326e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 0x7f5b0c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:06.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1326e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b0c103790 0x7f5b0c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1326e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c070dc0 con 0x7f5b0c102df0 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1326e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b0c070f00 con 0x7f5b0c103790 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1100a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 0x7f5b0c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1100a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 0x7f5b0c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45148/0 (socket says 172.21.15.33:45148) 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1100a700 1 -- 172.21.15.33:0/1112486751 learned_addr learned my addr 172.21.15.33:0/1112486751 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b10809700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b0c103790 0x7f5b0c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1100a700 1 -- 172.21.15.33:0/1112486751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b0c103790 msgr2=0x7f5b0c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1100a700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b0c103790 0x7f5b0c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b1100a700 1 -- 172.21.15.33:0/1112486751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc005040 con 0x7f5b0c102df0 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.265+0000 7f5b10809700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b0c103790 0x7f5b0c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.266+0000 7f5b1100a700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 0x7f5b0c070260 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009150 tx=0x7f5afc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.266+0000 7f5b01ffb700 1 -- 172.21.15.33:0/1112486751 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5afc00e040 con 0x7f5b0c102df0 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.266+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b0c119e90 con 0x7f5b0c102df0 2024-04-25T02:14:06.272 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.266+0000 7f5b01ffb700 1 -- 172.21.15.33:0/1112486751 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5afc013420 con 0x7f5b0c102df0 2024-04-25T02:14:06.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.266+0000 7f5b01ffb700 1 -- 172.21.15.33:0/1112486751 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5afc01c630 con 0x7f5b0c102df0 2024-04-25T02:14:06.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.266+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b0c11a370 con 0x7f5b0c102df0 2024-04-25T02:14:06.273 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.267+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0c0fe540 con 0x7f5b0c102df0 2024-04-25T02:14:06.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.268+0000 7f5b01ffb700 1 -- 172.21.15.33:0/1112486751 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5afc013590 con 0x7f5b0c102df0 2024-04-25T02:14:06.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.268+0000 7f5b01ffb700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5af806e150 0x7f5af8070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:06.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.268+0000 7f5b01ffb700 1 -- 172.21.15.33:0/1112486751 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f5afc08ed60 con 0x7f5b0c102df0 2024-04-25T02:14:06.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.268+0000 7f5b10809700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5af806e150 0x7f5af8070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:06.274 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.269+0000 7f5b10809700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5af806e150 0x7f5af8070600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5b08006fd0 tx=0x7f5b08008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:06.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.273+0000 7f5b01ffb700 1 -- 172.21.15.33:0/1112486751 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5afc058160 con 0x7f5b0c102df0 2024-04-25T02:14:06.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.489+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b0c071950 con 0x7f5af806e150 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.499+0000 7f5b01ffb700 1 -- 172.21.15.33:0/1112486751 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5b0c071950 con 0x7f5af806e150 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (35m) 12m ago 46m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (32m) 12m ago 46m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (32m) 12m ago 45m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (34m) 12m ago 46m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (28m) 12m ago 41m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (28m) 12m ago 41m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (28m) 12m ago 41m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:14:06.504 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (13m) 12m ago 40m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (36m) 12m ago 47m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (36m) 12m ago 45m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (33m) 12m ago 48m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (33m) 12m ago 45m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (35m) 12m ago 46m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (35m) 12m ago 45m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (32m) 12m ago 44m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (31m) 12m ago 44m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (30m) 12m ago 43m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (30m) 12m ago 43m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (29m) 12m ago 42m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (29m) 12m ago 42m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:14:06.505 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (35m) 12m ago 45m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:14:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5af806e150 msgr2=0x7f5af8070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5af806e150 0x7f5af8070600 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5b08006fd0 tx=0x7f5b08008040 comp rx=0 tx=0).stop 2024-04-25T02:14:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 msgr2=0x7f5b0c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 0x7f5b0c070260 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009150 tx=0x7f5afc009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:06.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 shutdown_connections 2024-04-25T02:14:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f5af806e150 0x7f5af8070600 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f5b0c102df0 0x7f5b0c070260 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 --2- 172.21.15.33:0/1112486751 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f5b0c103790 0x7f5b0c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 >> 172.21.15.33:0/1112486751 conn(0x7f5b0c0f8980 msgr2=0x7f5b0c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 shutdown_connections 2024-04-25T02:14:06.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.501+0000 7f5b1326e700 1 -- 172.21.15.33:0/1112486751 wait complete. 2024-04-25T02:14:06.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:06 smithi033 ceph-mon[111306]: pgmap v1042: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.604+0000 7f1ed178f700 1 -- 172.21.15.33:0/953986272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 msgr2=0x7f1ecc0fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.604+0000 7f1ed178f700 1 --2- 172.21.15.33:0/953986272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 0x7f1ecc0fe240 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0004940 tx=0x7f1ec0009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.604+0000 7f1ed178f700 1 -- 172.21.15.33:0/953986272 shutdown_connections 2024-04-25T02:14:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.604+0000 7f1ed178f700 1 --2- 172.21.15.33:0/953986272 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc0fe810 0x7f1ecc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.604+0000 7f1ed178f700 1 --2- 172.21.15.33:0/953986272 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 0x7f1ecc0fe240 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.604+0000 7f1ed178f700 1 -- 172.21.15.33:0/953986272 >> 172.21.15.33:0/953986272 conn(0x7f1ecc0f7860 msgr2=0x7f1ecc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.605+0000 7f1ed178f700 1 -- 172.21.15.33:0/953986272 shutdown_connections 2024-04-25T02:14:06.610 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.605+0000 7f1ed178f700 1 -- 172.21.15.33:0/953986272 wait complete. 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.605+0000 7f1ed178f700 1 Processor -- start 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.605+0000 7f1ed178f700 1 -- start start 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ed178f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 0x7f1ecc113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ed178f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc0fe810 0x7f1ecc113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ed178f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc1144c0 con 0x7f1ecc0fde70 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ed178f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc10dac0 con 0x7f1ecc0fe810 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ecaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 0x7f1ecc113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ecaffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 0x7f1ecc113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45160/0 (socket says 172.21.15.33:45160) 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ecaffd700 1 -- 172.21.15.33:0/1912477375 learned_addr learned my addr 172.21.15.33:0/1912477375 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:06.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1eca7fc700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc0fe810 0x7f1ecc113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ecaffd700 1 -- 172.21.15.33:0/1912477375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc0fe810 msgr2=0x7f1ecc113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ecaffd700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc0fe810 0x7f1ecc113e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ecaffd700 1 -- 172.21.15.33:0/1912477375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec0005040 con 0x7f1ecc0fde70 2024-04-25T02:14:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1ecaffd700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 0x7f1ecc113930 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0004890 tx=0x7f1ec0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1eb37fe700 1 -- 172.21.15.33:0/1912477375 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec000e040 con 0x7f1ecc0fde70 2024-04-25T02:14:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.606+0000 7f1eb37fe700 1 -- 172.21.15.33:0/1912477375 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ec0009af0 con 0x7f1ecc0fde70 2024-04-25T02:14:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.607+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ecc10dda0 con 0x7f1ecc0fde70 2024-04-25T02:14:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.607+0000 7f1eb37fe700 1 -- 172.21.15.33:0/1912477375 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec001c630 con 0x7f1ecc0fde70 2024-04-25T02:14:06.613 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.607+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc10e2e0 con 0x7f1ecc0fde70 2024-04-25T02:14:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.607+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ecc066af0 con 0x7f1ecc0fde70 2024-04-25T02:14:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.608+0000 7f1eb37fe700 1 -- 172.21.15.33:0/1912477375 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1ec000bbc0 con 0x7f1ecc0fde70 2024-04-25T02:14:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.608+0000 7f1eb37fe700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb406e140 0x7f1eb40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.609+0000 7f1eb37fe700 1 -- 172.21.15.33:0/1912477375 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1ec008ee10 con 0x7f1ecc0fde70 2024-04-25T02:14:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.609+0000 7f1eca7fc700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb406e140 0x7f1eb40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:06.614 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.609+0000 7f1eca7fc700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb406e140 0x7f1eb40705f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc007fd0 tx=0x7f1ebc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:06.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.613+0000 7f1eb37fe700 1 -- 172.21.15.33:0/1912477375 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1ec005bd40 con 0x7f1ecc0fde70 2024-04-25T02:14:06.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:06 smithi138 ceph-mon[81304]: pgmap v1042: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:06.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.907+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1ecc10df30 con 0x7f1ecc0fde70 2024-04-25T02:14:06.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.908+0000 7f1eb37fe700 1 -- 172.21.15.33:0/1912477375 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1ecc10df30 con 0x7f1ecc0fde70 2024-04-25T02:14:06.913 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:14:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:14:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:14:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:14:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:14:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:14:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:14:06.913 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:14:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:14:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:14:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:14:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:14:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:14:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:14:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:14:06.914 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:14:06.914 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:14:06.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb406e140 msgr2=0x7f1eb40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb406e140 0x7f1eb40705f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc007fd0 tx=0x7f1ebc009040 comp rx=0 tx=0).stop 2024-04-25T02:14:06.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 msgr2=0x7f1ecc113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:06.916 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 0x7f1ecc113930 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0004890 tx=0x7f1ec0009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:06.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 shutdown_connections 2024-04-25T02:14:06.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1eb406e140 0x7f1eb40705f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1ecc0fde70 0x7f1ecc113930 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 --2- 172.21.15.33:0/1912477375 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1ecc0fe810 0x7f1ecc113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:06.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 >> 172.21.15.33:0/1912477375 conn(0x7f1ecc0f7860 msgr2=0x7f1ecc0f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:06.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 shutdown_connections 2024-04-25T02:14:06.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:06.911+0000 7f1ed178f700 1 -- 172.21.15.33:0/1912477375 wait complete. 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 -- 172.21.15.33:0/3689445876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 msgr2=0x7fec000fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 --2- 172.21.15.33:0/3689445876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 0x7fec000fbd20 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7febe8004940 tx=0x7febe8009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 -- 172.21.15.33:0/3689445876 shutdown_connections 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 --2- 172.21.15.33:0/3689445876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 0x7fec000fbd20 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 --2- 172.21.15.33:0/3689445876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec00101d10 0x7fec001020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 -- 172.21.15.33:0/3689445876 >> 172.21.15.33:0/3689445876 conn(0x7fec000f7860 msgr2=0x7fec000f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 -- 172.21.15.33:0/3689445876 shutdown_connections 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 -- 172.21.15.33:0/3689445876 wait complete. 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 Processor -- start 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.016+0000 7fec05384700 1 -- start start 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7fec05384700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec00101d10 0x7fec0010eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:07.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7fec05384700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 0x7fec0010f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7fec05384700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0010f5d0 con 0x7fec001026b0 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7fec05384700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec0010f710 con 0x7fec00101d10 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febfe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 0x7fec0010f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febfe7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 0x7fec0010f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45188/0 (socket says 172.21.15.33:45188) 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febfe7fc700 1 -- 172.21.15.33:0/1121065451 learned_addr learned my addr 172.21.15.33:0/1121065451 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febfeffd700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec00101d10 0x7fec0010eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febfe7fc700 1 -- 172.21.15.33:0/1121065451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec00101d10 msgr2=0x7fec0010eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febfe7fc700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec00101d10 0x7fec0010eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febfe7fc700 1 -- 172.21.15.33:0/1121065451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8005040 con 0x7fec001026b0 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febfe7fc700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 0x7fec0010f090 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7febe8009100 tx=0x7febe8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febf77fe700 1 -- 172.21.15.33:0/1121065451 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe800e040 con 0x7fec001026b0 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.017+0000 7febf77fe700 1 -- 172.21.15.33:0/1121065451 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febe8009af0 con 0x7fec001026b0 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.018+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec0010f990 con 0x7fec001026b0 2024-04-25T02:14:07.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.018+0000 7febf77fe700 1 -- 172.21.15.33:0/1121065451 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febe801c3f0 con 0x7fec001026b0 2024-04-25T02:14:07.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.018+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec00117ec0 con 0x7fec001026b0 2024-04-25T02:14:07.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.018+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec000fd460 con 0x7fec001026b0 2024-04-25T02:14:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.019+0000 7febf77fe700 1 -- 172.21.15.33:0/1121065451 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7febe800baf0 con 0x7fec001026b0 2024-04-25T02:14:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.020+0000 7febf77fe700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7febec06e140 0x7febec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.020+0000 7febf77fe700 1 -- 172.21.15.33:0/1121065451 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7febe808ed60 con 0x7fec001026b0 2024-04-25T02:14:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.020+0000 7febfeffd700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7febec06e140 0x7febec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:07.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.020+0000 7febfeffd700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7febec06e140 0x7febec0705f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7febf8007970 tx=0x7febf8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:07.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.024+0000 7febf77fe700 1 -- 172.21.15.33:0/1121065451 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7febe8058160 con 0x7fec001026b0 2024-04-25T02:14:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.286+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fec00108df0 con 0x7fec001026b0 2024-04-25T02:14:07.291 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.286+0000 7febf77fe700 1 -- 172.21.15.33:0/1121065451 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7febe805b820 con 0x7fec001026b0 2024-04-25T02:14:07.291 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:14:07.292 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:14:07.293 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:14:07.294 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:14:07.295 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:14:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.288+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7febec06e140 msgr2=0x7febec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.288+0000 7fec05384700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7febec06e140 0x7febec0705f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7febf8007970 tx=0x7febf8009da0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.288+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 msgr2=0x7fec0010f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.295 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.288+0000 7fec05384700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 0x7fec0010f090 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7febe8009100 tx=0x7febe8009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.289+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 shutdown_connections 2024-04-25T02:14:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.289+0000 7fec05384700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7febec06e140 0x7febec0705f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.289+0000 7fec05384700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fec00101d10 0x7fec0010eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.289+0000 7fec05384700 1 --2- 172.21.15.33:0/1121065451 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fec001026b0 0x7fec0010f090 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.289+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 >> 172.21.15.33:0/1121065451 conn(0x7fec000f7860 msgr2=0x7fec000f87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.289+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 shutdown_connections 2024-04-25T02:14:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.289+0000 7fec05384700 1 -- 172.21.15.33:0/1121065451 wait complete. 2024-04-25T02:14:07.296 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:14:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.393+0000 7fcf5a599700 1 -- 172.21.15.33:0/267011306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 msgr2=0x7fcf540fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.393+0000 7fcf5a599700 1 --2- 172.21.15.33:0/267011306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 0x7fcf540fbd20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fcf48004940 tx=0x7fcf48009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.394+0000 7fcf5a599700 1 -- 172.21.15.33:0/267011306 shutdown_connections 2024-04-25T02:14:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.394+0000 7fcf5a599700 1 --2- 172.21.15.33:0/267011306 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 0x7fcf540fbd20 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.394+0000 7fcf5a599700 1 --2- 172.21.15.33:0/267011306 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54101d10 0x7fcf541020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.394+0000 7fcf5a599700 1 -- 172.21.15.33:0/267011306 >> 172.21.15.33:0/267011306 conn(0x7fcf540f7860 msgr2=0x7fcf540f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.394+0000 7fcf5a599700 1 -- 172.21.15.33:0/267011306 shutdown_connections 2024-04-25T02:14:07.399 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.394+0000 7fcf5a599700 1 -- 172.21.15.33:0/267011306 wait complete. 2024-04-25T02:14:07.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.394+0000 7fcf5a599700 1 Processor -- start 2024-04-25T02:14:07.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.394+0000 7fcf5a599700 1 -- start start 2024-04-25T02:14:07.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf5a599700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54101d10 0x7fcf54113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:07.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf5a599700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 0x7fcf54113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:07.401 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf5a599700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf54114510 con 0x7fcf541026b0 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf5a599700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5410db30 con 0x7fcf54101d10 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 0x7fcf54113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf537fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 0x7fcf54113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45208/0 (socket says 172.21.15.33:45208) 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf537fe700 1 -- 172.21.15.33:0/261606464 learned_addr learned my addr 172.21.15.33:0/261606464 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf53fff700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54101d10 0x7fcf54113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf537fe700 1 -- 172.21.15.33:0/261606464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54101d10 msgr2=0x7fcf54113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf537fe700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54101d10 0x7fcf54113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf537fe700 1 -- 172.21.15.33:0/261606464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf48005040 con 0x7fcf541026b0 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.395+0000 7fcf537fe700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 0x7fcf54113ec0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009180 tx=0x7fcf48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.396+0000 7fcf50ff9700 1 -- 172.21.15.33:0/261606464 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4800e040 con 0x7fcf541026b0 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.396+0000 7fcf50ff9700 1 -- 172.21.15.33:0/261606464 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf48013420 con 0x7fcf541026b0 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.396+0000 7fcf50ff9700 1 -- 172.21.15.33:0/261606464 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf4801c610 con 0x7fcf541026b0 2024-04-25T02:14:07.402 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.396+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5410ddb0 con 0x7fcf541026b0 2024-04-25T02:14:07.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.396+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5410e320 con 0x7fcf541026b0 2024-04-25T02:14:07.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.397+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf540fd460 con 0x7fcf541026b0 2024-04-25T02:14:07.403 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.397+0000 7fcf50ff9700 1 -- 172.21.15.33:0/261606464 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcf48013a00 con 0x7fcf541026b0 2024-04-25T02:14:07.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.398+0000 7fcf50ff9700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf3c06e140 0x7fcf3c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:07.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.398+0000 7fcf50ff9700 1 -- 172.21.15.33:0/261606464 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcf4808ec10 con 0x7fcf541026b0 2024-04-25T02:14:07.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.398+0000 7fcf53fff700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf3c06e140 0x7fcf3c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:07.404 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.398+0000 7fcf53fff700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf3c06e140 0x7fcf3c0705f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcf44004f40 tx=0x7fcf4400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:07.407 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.402+0000 7fcf50ff9700 1 -- 172.21.15.33:0/261606464 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcf480580c0 con 0x7fcf541026b0 2024-04-25T02:14:07.631 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.626+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf54108df0 con 0x7fcf3c06e140 2024-04-25T02:14:07.631 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:07 smithi033 ceph-mon[111306]: from='client.35888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:07.631 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:07 smithi033 ceph-mon[111306]: from='client.35892 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:07.632 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:07 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1912477375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:14:07.632 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:07 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1121065451' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:14:07.632 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.627+0000 7fcf50ff9700 1 -- 172.21.15.33:0/261606464 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcf54108df0 con 0x7fcf3c06e140 2024-04-25T02:14:07.632 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:14:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:14:07.632 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:14:07.633 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.629+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf3c06e140 msgr2=0x7fcf3c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.629+0000 7fcf5a599700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf3c06e140 0x7fcf3c0705f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fcf44004f40 tx=0x7fcf4400b040 comp rx=0 tx=0).stop 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.629+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 msgr2=0x7fcf54113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.629+0000 7fcf5a599700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 0x7fcf54113ec0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009180 tx=0x7fcf48009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.629+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 shutdown_connections 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.629+0000 7fcf5a599700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcf3c06e140 0x7fcf3c0705f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.629+0000 7fcf5a599700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcf54101d10 0x7fcf54113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.629+0000 7fcf5a599700 1 --2- 172.21.15.33:0/261606464 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcf541026b0 0x7fcf54113ec0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.629+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 >> 172.21.15.33:0/261606464 conn(0x7fcf540f7860 msgr2=0x7fcf540f8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.630+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 shutdown_connections 2024-04-25T02:14:07.635 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.630+0000 7fcf5a599700 1 -- 172.21.15.33:0/261606464 wait complete. 2024-04-25T02:14:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.734+0000 7fc0f21ad700 1 -- 172.21.15.33:0/60168004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 msgr2=0x7fc0ec1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.734+0000 7fc0f21ad700 1 --2- 172.21.15.33:0/60168004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 0x7fc0ec1031b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc009ce0 tx=0x7fc0dc004d10 comp rx=0 tx=0).stop 2024-04-25T02:14:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.734+0000 7fc0f21ad700 1 -- 172.21.15.33:0/60168004 shutdown_connections 2024-04-25T02:14:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.734+0000 7fc0f21ad700 1 --2- 172.21.15.33:0/60168004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0ec103780 0x7fc0ec0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.734+0000 7fc0f21ad700 1 --2- 172.21.15.33:0/60168004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 0x7fc0ec1031b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.734+0000 7fc0f21ad700 1 -- 172.21.15.33:0/60168004 >> 172.21.15.33:0/60168004 conn(0x7fc0ec0f8990 msgr2=0x7fc0ec0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:07.739 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.734+0000 7fc0f21ad700 1 -- 172.21.15.33:0/60168004 shutdown_connections 2024-04-25T02:14:07.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.734+0000 7fc0f21ad700 1 -- 172.21.15.33:0/60168004 wait complete. 2024-04-25T02:14:07.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.734+0000 7fc0f21ad700 1 Processor -- start 2024-04-25T02:14:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0f21ad700 1 -- start start 2024-04-25T02:14:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0f21ad700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 0x7fc0ec070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0f21ad700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0ec103780 0x7fc0ec0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0f21ad700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec072160 con 0x7fc0ec102de0 2024-04-25T02:14:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0f21ad700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ec070ce0 con 0x7fc0ec103780 2024-04-25T02:14:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0eb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 0x7fc0ec070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0eb7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 0x7fc0ec070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45216/0 (socket says 172.21.15.33:45216) 2024-04-25T02:14:07.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0eb7fe700 1 -- 172.21.15.33:0/532155362 learned_addr learned my addr 172.21.15.33:0/532155362 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0eb7fe700 1 -- 172.21.15.33:0/532155362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0ec103780 msgr2=0x7fc0ec0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0eaffd700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0ec103780 0x7fc0ec0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0eb7fe700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0ec103780 0x7fc0ec0707a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0eb7fe700 1 -- 172.21.15.33:0/532155362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc009940 con 0x7fc0ec102de0 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0eb7fe700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 0x7fc0ec070260 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00be90 tx=0x7fc0dc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.735+0000 7fc0e3fff700 1 -- 172.21.15.33:0/532155362 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0dc00e040 con 0x7fc0ec102de0 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.736+0000 7fc0e3fff700 1 -- 172.21.15.33:0/532155362 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc0dc008ea0 con 0x7fc0ec102de0 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.736+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ec070f60 con 0x7fc0ec102de0 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.736+0000 7fc0e3fff700 1 -- 172.21.15.33:0/532155362 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0dc01c810 con 0x7fc0ec102de0 2024-04-25T02:14:07.742 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.736+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ec11a270 con 0x7fc0ec102de0 2024-04-25T02:14:07.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.737+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ec1000f0 con 0x7fc0ec102de0 2024-04-25T02:14:07.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.737+0000 7fc0e3fff700 1 -- 172.21.15.33:0/532155362 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc0dc012c00 con 0x7fc0ec102de0 2024-04-25T02:14:07.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.738+0000 7fc0e3fff700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc0d806e140 0x7fc0d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:07.743 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.738+0000 7fc0e3fff700 1 -- 172.21.15.33:0/532155362 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fc0dc08f740 con 0x7fc0ec102de0 2024-04-25T02:14:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.738+0000 7fc0eaffd700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc0d806e140 0x7fc0d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:07.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.738+0000 7fc0eaffd700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc0d806e140 0x7fc0d80705f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec071940 tx=0x7fc0d4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:07.747 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:07.742+0000 7fc0e3fff700 1 -- 172.21.15.33:0/532155362 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc0dc058ac0 con 0x7fc0ec102de0 2024-04-25T02:14:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:07 smithi138 ceph-mon[81304]: from='client.35888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:07 smithi138 ceph-mon[81304]: from='client.35892 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:07 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1912477375' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:14:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:07 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1121065451' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:14:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.033+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc0ec071740 con 0x7fc0ec102de0 2024-04-25T02:14:08.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.034+0000 7fc0e3fff700 1 -- 172.21.15.33:0/532155362 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fc0dc004e10 con 0x7fc0ec102de0 2024-04-25T02:14:08.039 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:14:08.039 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:14:08.039 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:14:08.039 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:14:08.039 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:14:08.039 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:14:08.039 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:14:08.040 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:14:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc0d806e140 msgr2=0x7fc0d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc0d806e140 0x7fc0d80705f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec071940 tx=0x7fc0d4009040 comp rx=0 tx=0).stop 2024-04-25T02:14:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 msgr2=0x7fc0ec070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 0x7fc0ec070260 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc0dc00be90 tx=0x7fc0dc004d10 comp rx=0 tx=0).stop 2024-04-25T02:14:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 shutdown_connections 2024-04-25T02:14:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fc0d806e140 0x7fc0d80705f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:08.041 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fc0ec102de0 0x7fc0ec070260 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 --2- 172.21.15.33:0/532155362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fc0ec103780 0x7fc0ec0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 >> 172.21.15.33:0/532155362 conn(0x7fc0ec0f8990 msgr2=0x7fc0ec107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 shutdown_connections 2024-04-25T02:14:08.042 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:08.036+0000 7fc0f21ad700 1 -- 172.21.15.33:0/532155362 wait complete. 2024-04-25T02:14:08.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:08 smithi033 ceph-mon[111306]: from='client.35896 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:08.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:08 smithi033 ceph-mon[111306]: pgmap v1043: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:08.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:08 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/532155362' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:14:08.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:08 smithi138 ceph-mon[81304]: from='client.35896 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:08.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:08 smithi138 ceph-mon[81304]: pgmap v1043: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:08.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:08 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/532155362' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:14:09.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:09 smithi033 ceph-mon[111306]: from='client.35908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:09.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:09 smithi138 ceph-mon[81304]: from='client.35908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:10.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:10 smithi033 ceph-mon[111306]: pgmap v1044: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:10.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:14:10.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:10 smithi138 ceph-mon[81304]: pgmap v1044: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:10.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:14:12.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:12 smithi033 ceph-mon[111306]: pgmap v1045: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:12.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:12 smithi138 ceph-mon[81304]: pgmap v1045: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:14.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:14 smithi033 ceph-mon[111306]: pgmap v1046: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:14.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:14 smithi138 ceph-mon[81304]: pgmap v1046: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:16.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:16 smithi033 ceph-mon[111306]: pgmap v1047: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:16.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:16 smithi138 ceph-mon[81304]: pgmap v1047: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:18.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:18 smithi033 ceph-mon[111306]: pgmap v1048: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:18.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:18 smithi138 ceph-mon[81304]: pgmap v1048: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:20.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:20 smithi033 ceph-mon[111306]: pgmap v1049: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:20.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:20 smithi138 ceph-mon[81304]: pgmap v1049: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:22.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:22 smithi033 ceph-mon[111306]: pgmap v1050: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:22.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:22 smithi138 ceph-mon[81304]: pgmap v1050: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:24.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:24 smithi033 ceph-mon[111306]: pgmap v1051: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:24.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:24 smithi138 ceph-mon[81304]: pgmap v1051: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:25.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:14:25.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:14:26.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:26 smithi033 ceph-mon[111306]: pgmap v1052: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:26.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:26 smithi138 ceph-mon[81304]: pgmap v1052: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:27.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:27 smithi033 ceph-mon[111306]: pgmap v1053: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:27.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:27 smithi138 ceph-mon[81304]: pgmap v1053: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:30.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:30 smithi033 ceph-mon[111306]: pgmap v1054: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:30.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:30 smithi138 ceph-mon[81304]: pgmap v1054: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:32.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:32 smithi033 ceph-mon[111306]: pgmap v1055: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:32.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:32 smithi138 ceph-mon[81304]: pgmap v1055: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:34.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:34 smithi033 ceph-mon[111306]: pgmap v1056: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:34.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:34 smithi138 ceph-mon[81304]: pgmap v1056: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:36.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:36 smithi033 ceph-mon[111306]: pgmap v1057: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:36.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:36 smithi138 ceph-mon[81304]: pgmap v1057: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.153+0000 7fb1d0907700 1 -- 172.21.15.33:0/455524349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc1026b0 msgr2=0x7fb1cc0fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.153+0000 7fb1d0907700 1 --2- 172.21.15.33:0/455524349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc1026b0 0x7fb1cc0fbd20 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0004940 tx=0x7fb1c0009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 -- 172.21.15.33:0/455524349 shutdown_connections 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 --2- 172.21.15.33:0/455524349 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc1026b0 0x7fb1cc0fbd20 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 --2- 172.21.15.33:0/455524349 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc101d10 0x7fb1cc1020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 -- 172.21.15.33:0/455524349 >> 172.21.15.33:0/455524349 conn(0x7fb1cc0f7860 msgr2=0x7fb1cc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 -- 172.21.15.33:0/455524349 shutdown_connections 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 -- 172.21.15.33:0/455524349 wait complete. 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 Processor -- start 2024-04-25T02:14:38.160 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 -- start start 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc101d10 0x7fb1cc0fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc1026b0 0x7fb1cc0fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0ff8e0 con 0x7fb1cc101d10 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.154+0000 7fb1d0907700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1cc0fe410 con 0x7fb1cc1026b0 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1ca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc101d10 0x7fb1cc0fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1ca59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc101d10 0x7fb1cc0fd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36934/0 (socket says 172.21.15.33:36934) 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1ca59c700 1 -- 172.21.15.33:0/2013855499 learned_addr learned my addr 172.21.15.33:0/2013855499 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1c9d9b700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc1026b0 0x7fb1cc0fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1ca59c700 1 -- 172.21.15.33:0/2013855499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc1026b0 msgr2=0x7fb1cc0fded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1ca59c700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc1026b0 0x7fb1cc0fded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1ca59c700 1 -- 172.21.15.33:0/2013855499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c0005040 con 0x7fb1cc101d10 2024-04-25T02:14:38.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1ca59c700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc101d10 0x7fb1cc0fd990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc002a50 tx=0x7fb1bc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1baffd700 1 -- 172.21.15.33:0/2013855499 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1bc007cc0 con 0x7fb1cc101d10 2024-04-25T02:14:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1cc0fe6f0 con 0x7fb1cc101d10 2024-04-25T02:14:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1baffd700 1 -- 172.21.15.33:0/2013855499 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1bc00f040 con 0x7fb1cc101d10 2024-04-25T02:14:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1baffd700 1 -- 172.21.15.33:0/2013855499 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1bc00da30 con 0x7fb1cc101d10 2024-04-25T02:14:38.163 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.155+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1cc11a150 con 0x7fb1cc101d10 2024-04-25T02:14:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.157+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1cc0fd460 con 0x7fb1cc101d10 2024-04-25T02:14:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.157+0000 7fb1baffd700 1 -- 172.21.15.33:0/2013855499 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb1bc00db90 con 0x7fb1cc101d10 2024-04-25T02:14:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.158+0000 7fb1baffd700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1b406e140 0x7fb1b40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.158+0000 7fb1baffd700 1 -- 172.21.15.33:0/2013855499 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb1bc08d8d0 con 0x7fb1cc101d10 2024-04-25T02:14:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.158+0000 7fb1c9d9b700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1b406e140 0x7fb1b40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:38.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.158+0000 7fb1c9d9b700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1b406e140 0x7fb1b40705f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0ff0c0 tx=0x7fb1c0009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:38.168 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.163+0000 7fb1baffd700 1 -- 172.21.15.33:0/2013855499 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb1bc056c50 con 0x7fb1cc101d10 2024-04-25T02:14:38.394 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.389+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1cc04c7b0 con 0x7fb1b406e140 2024-04-25T02:14:38.395 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.390+0000 7fb1baffd700 1 -- 172.21.15.33:0/2013855499 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb1cc04c7b0 con 0x7fb1b406e140 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1b406e140 msgr2=0x7fb1b40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1b406e140 0x7fb1b40705f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0ff0c0 tx=0x7fb1c0009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc101d10 msgr2=0x7fb1cc0fd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc101d10 0x7fb1cc0fd990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc002a50 tx=0x7fb1bc0046b0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 shutdown_connections 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1b406e140 0x7fb1b40705f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1cc101d10 0x7fb1cc0fd990 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 --2- 172.21.15.33:0/2013855499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1cc1026b0 0x7fb1cc0fded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 >> 172.21.15.33:0/2013855499 conn(0x7fb1cc0f7860 msgr2=0x7fb1cc0662a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 shutdown_connections 2024-04-25T02:14:38.398 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.392+0000 7fb1d0907700 1 -- 172.21.15.33:0/2013855499 wait complete. 2024-04-25T02:14:38.408 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:14:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.497+0000 7f020a51f700 1 -- 172.21.15.33:0/2687062433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204103790 msgr2=0x7f02040fce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.497+0000 7f020a51f700 1 --2- 172.21.15.33:0/2687062433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204103790 0x7f02040fce00 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f01f8004940 tx=0x7f01f8009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.497+0000 7f020a51f700 1 -- 172.21.15.33:0/2687062433 shutdown_connections 2024-04-25T02:14:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.497+0000 7f020a51f700 1 --2- 172.21.15.33:0/2687062433 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204103790 0x7f02040fce00 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.497+0000 7f020a51f700 1 --2- 172.21.15.33:0/2687062433 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0204102df0 0x7f02041031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.503 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.497+0000 7f020a51f700 1 -- 172.21.15.33:0/2687062433 >> 172.21.15.33:0/2687062433 conn(0x7f02040f8980 msgr2=0x7f02040fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:38.504 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:38 smithi033 ceph-mon[111306]: pgmap v1058: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:38.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.497+0000 7f020a51f700 1 -- 172.21.15.33:0/2687062433 shutdown_connections 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.497+0000 7f020a51f700 1 -- 172.21.15.33:0/2687062433 wait complete. 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.498+0000 7f020a51f700 1 Processor -- start 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.498+0000 7f020a51f700 1 -- start start 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f020a51f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204102df0 0x7f0204113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f020a51f700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0204103790 0x7f02041140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f020a51f700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0204114720 con 0x7f0204102df0 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f020a51f700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020410db10 con 0x7f0204103790 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f0203fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204102df0 0x7f0204113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f0203fff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204102df0 0x7f0204113b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36950/0 (socket says 172.21.15.33:36950) 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f0203fff700 1 -- 172.21.15.33:0/334787621 learned_addr learned my addr 172.21.15.33:0/334787621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f02037fe700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0204103790 0x7f02041140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f0203fff700 1 -- 172.21.15.33:0/334787621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0204103790 msgr2=0x7f02041140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f0203fff700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0204103790 0x7f02041140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.506 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f0203fff700 1 -- 172.21.15.33:0/334787621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f8005040 con 0x7f0204102df0 2024-04-25T02:14:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.499+0000 7f0203fff700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204102df0 0x7f0204113b90 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f01f400c8c0 tx=0x7f01f40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.500+0000 7f0200ff9700 1 -- 172.21.15.33:0/334787621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01f400a760 con 0x7f0204102df0 2024-04-25T02:14:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.500+0000 7f0200ff9700 1 -- 172.21.15.33:0/334787621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01f400ad90 con 0x7f0204102df0 2024-04-25T02:14:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.500+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020410ddf0 con 0x7f0204102df0 2024-04-25T02:14:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.500+0000 7f0200ff9700 1 -- 172.21.15.33:0/334787621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01f40113a0 con 0x7f0204102df0 2024-04-25T02:14:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.500+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020410e380 con 0x7f0204102df0 2024-04-25T02:14:38.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.501+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02040fe540 con 0x7f0204102df0 2024-04-25T02:14:38.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.501+0000 7f0200ff9700 1 -- 172.21.15.33:0/334787621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f01f4012400 con 0x7f0204102df0 2024-04-25T02:14:38.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.502+0000 7f0200ff9700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f01ec06e140 0x7f01ec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:38.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.502+0000 7f0200ff9700 1 -- 172.21.15.33:0/334787621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f01f408daf0 con 0x7f0204102df0 2024-04-25T02:14:38.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.502+0000 7f02037fe700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f01ec06e140 0x7f01ec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:38.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.503+0000 7f02037fe700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f01ec06e140 0x7f01ec0705f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f020410edf0 tx=0x7f01f8009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:38.512 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.507+0000 7f0200ff9700 1 -- 172.21.15.33:0/334787621 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f01f405aa40 con 0x7f0204102df0 2024-04-25T02:14:38.740 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.734+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0204109ed0 con 0x7f01ec06e140 2024-04-25T02:14:38.741 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.736+0000 7f0200ff9700 1 -- 172.21.15.33:0/334787621 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0204109ed0 con 0x7f01ec06e140 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f01ec06e140 msgr2=0x7f01ec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f01ec06e140 0x7f01ec0705f0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f020410edf0 tx=0x7f01f8009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204102df0 msgr2=0x7f0204113b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204102df0 0x7f0204113b90 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f01f400c8c0 tx=0x7f01f40044c0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 shutdown_connections 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f01ec06e140 0x7f01ec0705f0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0204102df0 0x7f0204113b90 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 --2- 172.21.15.33:0/334787621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0204103790 0x7f02041140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 >> 172.21.15.33:0/334787621 conn(0x7f02040f8980 msgr2=0x7f02040f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 shutdown_connections 2024-04-25T02:14:38.744 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.738+0000 7f020a51f700 1 -- 172.21.15.33:0/334787621 wait complete. 2024-04-25T02:14:38.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:38 smithi138 ceph-mon[81304]: pgmap v1058: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:38.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 -- 172.21.15.33:0/3369750850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 msgr2=0x7f93340fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 --2- 172.21.15.33:0/3369750850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 0x7f93340fcdf0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f9324004940 tx=0x7f9324009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 -- 172.21.15.33:0/3369750850 shutdown_connections 2024-04-25T02:14:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 --2- 172.21.15.33:0/3369750850 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 0x7f93340fcdf0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 --2- 172.21.15.33:0/3369750850 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9334102de0 0x7f93341031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 -- 172.21.15.33:0/3369750850 >> 172.21.15.33:0/3369750850 conn(0x7f93340f8990 msgr2=0x7f93340fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 -- 172.21.15.33:0/3369750850 shutdown_connections 2024-04-25T02:14:38.848 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 -- 172.21.15.33:0/3369750850 wait complete. 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 Processor -- start 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.842+0000 7f933be9d700 1 -- start start 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f933be9d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9334102de0 0x7f9334113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f933be9d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 0x7f9334113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f933be9d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9334114520 con 0x7f9334103780 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f933be9d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933410db20 con 0x7f9334102de0 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f9339438700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 0x7f9334113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f9339438700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 0x7f9334113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36972/0 (socket says 172.21.15.33:36972) 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f9339438700 1 -- 172.21.15.33:0/3048521712 learned_addr learned my addr 172.21.15.33:0/3048521712 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f9339c39700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9334102de0 0x7f9334113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f9339438700 1 -- 172.21.15.33:0/3048521712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9334102de0 msgr2=0x7f9334113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:38.850 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f9339438700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9334102de0 0x7f9334113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f9339438700 1 -- 172.21.15.33:0/3048521712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9324005040 con 0x7f9334103780 2024-04-25T02:14:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f9339c39700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9334102de0 0x7f9334113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:14:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f9339438700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 0x7f9334113ed0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9324009100 tx=0x7f9324009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f932a7fc700 1 -- 172.21.15.33:0/3048521712 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f932400e040 con 0x7f9334103780 2024-04-25T02:14:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f932a7fc700 1 -- 172.21.15.33:0/3048521712 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9324013420 con 0x7f9334103780 2024-04-25T02:14:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933410dda0 con 0x7f9334103780 2024-04-25T02:14:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.843+0000 7f932a7fc700 1 -- 172.21.15.33:0/3048521712 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f932401c610 con 0x7f9334103780 2024-04-25T02:14:38.851 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.844+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f933410e360 con 0x7f9334103780 2024-04-25T02:14:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.845+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93340fe530 con 0x7f9334103780 2024-04-25T02:14:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.845+0000 7f932a7fc700 1 -- 172.21.15.33:0/3048521712 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9324013590 con 0x7f9334103780 2024-04-25T02:14:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.845+0000 7f932a7fc700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932006e140 0x7f93200705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.846+0000 7f932a7fc700 1 -- 172.21.15.33:0/3048521712 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f932408edc0 con 0x7f9334103780 2024-04-25T02:14:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.846+0000 7f9339c39700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932006e140 0x7f93200705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:38.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.846+0000 7f9339c39700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932006e140 0x7f93200705f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9330009fd0 tx=0x7f9330009e50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:38.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:38.850+0000 7f932a7fc700 1 -- 172.21.15.33:0/3048521712 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f932405bcf0 con 0x7f9334103780 2024-04-25T02:14:39.071 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.065+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9334109ec0 con 0x7f932006e140 2024-04-25T02:14:39.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.075+0000 7f932a7fc700 1 -- 172.21.15.33:0/3048521712 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9334109ec0 con 0x7f932006e140 2024-04-25T02:14:39.081 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:14:39.081 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (35m) 13m ago 46m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:14:39.081 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (33m) 13m ago 47m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:14:39.081 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (32m) 13m ago 46m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:14:39.081 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (35m) 13m ago 46m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (28m) 13m ago 41m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (29m) 13m ago 41m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (28m) 13m ago 41m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (13m) 13m ago 41m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (37m) 13m ago 48m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (36m) 13m ago 46m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (34m) 13m ago 48m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (33m) 13m ago 46m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (36m) 13m ago 47m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (36m) 13m ago 46m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (32m) 13m ago 45m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (31m) 13m ago 44m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (31m) 13m ago 44m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (30m) 13m ago 43m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:14:39.082 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (30m) 13m ago 43m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:14:39.083 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (29m) 13m ago 43m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:14:39.083 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (36m) 13m ago 46m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:14:39.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.077+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932006e140 msgr2=0x7f93200705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.077+0000 7f933be9d700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932006e140 0x7f93200705f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9330009fd0 tx=0x7f9330009e50 comp rx=0 tx=0).stop 2024-04-25T02:14:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.077+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 msgr2=0x7f9334113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.077+0000 7f933be9d700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 0x7f9334113ed0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f9324009100 tx=0x7f9324009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.077+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 shutdown_connections 2024-04-25T02:14:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.077+0000 7f933be9d700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f932006e140 0x7f93200705f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.077+0000 7f933be9d700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9334102de0 0x7f9334113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.077+0000 7f933be9d700 1 --2- 172.21.15.33:0/3048521712 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9334103780 0x7f9334113ed0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.077+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 >> 172.21.15.33:0/3048521712 conn(0x7f93340f8990 msgr2=0x7f93340f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.078+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 shutdown_connections 2024-04-25T02:14:39.084 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.078+0000 7f933be9d700 1 -- 172.21.15.33:0/3048521712 wait complete. 2024-04-25T02:14:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.180+0000 7fe0d9499700 1 -- 172.21.15.33:0/1764943323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 msgr2=0x7fe0d4070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.186 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.180+0000 7fe0d9499700 1 --2- 172.21.15.33:0/1764943323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 0x7fe0d4070670 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4004970 tx=0x7fe0c4009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.180+0000 7fe0d9499700 1 -- 172.21.15.33:0/1764943323 shutdown_connections 2024-04-25T02:14:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.180+0000 7fe0d9499700 1 --2- 172.21.15.33:0/1764943323 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d4070bb0 0x7fe0d4109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.180+0000 7fe0d9499700 1 --2- 172.21.15.33:0/1764943323 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 0x7fe0d4070670 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.180+0000 7fe0d9499700 1 -- 172.21.15.33:0/1764943323 >> 172.21.15.33:0/1764943323 conn(0x7fe0d40f8990 msgr2=0x7fe0d40fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.180+0000 7fe0d9499700 1 -- 172.21.15.33:0/1764943323 shutdown_connections 2024-04-25T02:14:39.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.180+0000 7fe0d9499700 1 -- 172.21.15.33:0/1764943323 wait complete. 2024-04-25T02:14:39.188 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.181+0000 7fe0d9499700 1 Processor -- start 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.181+0000 7fe0d9499700 1 -- start start 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.181+0000 7fe0d9499700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 0x7fe0d4115b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.181+0000 7fe0d9499700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d4070bb0 0x7fe0d4116050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d9499700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d41166a0 con 0x7fe0d40702a0 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d9499700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d410fc80 con 0x7fe0d4070bb0 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 0x7fe0d4115b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d2ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 0x7fe0d4115b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:36992/0 (socket says 172.21.15.33:36992) 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d2ffd700 1 -- 172.21.15.33:0/1986470786 learned_addr learned my addr 172.21.15.33:0/1986470786 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d27fc700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d4070bb0 0x7fe0d4116050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d2ffd700 1 -- 172.21.15.33:0/1986470786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d4070bb0 msgr2=0x7fe0d4116050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d2ffd700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d4070bb0 0x7fe0d4116050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d2ffd700 1 -- 172.21.15.33:0/1986470786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c4005040 con 0x7fe0d40702a0 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d2ffd700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 0x7fe0d4115b10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4004f50 tx=0x7fe0c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:39.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0bb7fe700 1 -- 172.21.15.33:0/1986470786 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0c400e040 con 0x7fe0d40702a0 2024-04-25T02:14:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0bb7fe700 1 -- 172.21.15.33:0/1986470786 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0c400bc10 con 0x7fe0d40702a0 2024-04-25T02:14:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.182+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d410ff00 con 0x7fe0d40702a0 2024-04-25T02:14:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.183+0000 7fe0bb7fe700 1 -- 172.21.15.33:0/1986470786 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0c401c630 con 0x7fe0d40702a0 2024-04-25T02:14:39.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.183+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d4110440 con 0x7fe0d40702a0 2024-04-25T02:14:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.184+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d40719e0 con 0x7fe0d40702a0 2024-04-25T02:14:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.184+0000 7fe0bb7fe700 1 -- 172.21.15.33:0/1986470786 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe0c401c790 con 0x7fe0d40702a0 2024-04-25T02:14:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.184+0000 7fe0bb7fe700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0bc06e140 0x7fe0bc0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.185+0000 7fe0bb7fe700 1 -- 172.21.15.33:0/1986470786 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe0c408ed70 con 0x7fe0d40702a0 2024-04-25T02:14:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.185+0000 7fe0d27fc700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0bc06e140 0x7fe0bc0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:39.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.185+0000 7fe0d27fc700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0bc06e140 0x7fe0bc0705f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8007fd0 tx=0x7fe0c8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:39.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.189+0000 7fe0bb7fe700 1 -- 172.21.15.33:0/1986470786 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe0c405bd40 con 0x7fe0d40702a0 2024-04-25T02:14:39.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.484+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe0d4111160 con 0x7fe0d40702a0 2024-04-25T02:14:39.491 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:39 smithi033 ceph-mon[111306]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:39.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.485+0000 7fe0bb7fe700 1 -- 172.21.15.33:0/1986470786 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe0c405b8d0 con 0x7fe0d40702a0 2024-04-25T02:14:39.491 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:14:39.491 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:14:39.491 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:14:39.492 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:14:39.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.487+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0bc06e140 msgr2=0x7fe0bc0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.487+0000 7fe0d9499700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0bc06e140 0x7fe0bc0705f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8007fd0 tx=0x7fe0c8009040 comp rx=0 tx=0).stop 2024-04-25T02:14:39.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.487+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 msgr2=0x7fe0d4115b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.487+0000 7fe0d9499700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 0x7fe0d4115b10 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4004f50 tx=0x7fe0c4009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:39.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.487+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 shutdown_connections 2024-04-25T02:14:39.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.487+0000 7fe0d9499700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0bc06e140 0x7fe0bc0705f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.487+0000 7fe0d9499700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0d40702a0 0x7fe0d4115b10 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.487+0000 7fe0d9499700 1 --2- 172.21.15.33:0/1986470786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0d4070bb0 0x7fe0d4116050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.487+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 >> 172.21.15.33:0/1986470786 conn(0x7fe0d40f8990 msgr2=0x7fe0d40f9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.488+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 shutdown_connections 2024-04-25T02:14:39.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.488+0000 7fe0d9499700 1 -- 172.21.15.33:0/1986470786 wait complete. 2024-04-25T02:14:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.591+0000 7f2829398700 1 -- 172.21.15.33:0/2298134876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fe780 msgr2=0x7f2824106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.591+0000 7f2829398700 1 --2- 172.21.15.33:0/2298134876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fe780 0x7f2824106dc0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2818004940 tx=0x7f2818009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.591+0000 7f2829398700 1 -- 172.21.15.33:0/2298134876 shutdown_connections 2024-04-25T02:14:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.591+0000 7f2829398700 1 --2- 172.21.15.33:0/2298134876 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fe780 0x7f2824106dc0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.597 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.591+0000 7f2829398700 1 --2- 172.21.15.33:0/2298134876 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28240fdde0 0x7f28240fe1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.591+0000 7f2829398700 1 -- 172.21.15.33:0/2298134876 >> 172.21.15.33:0/2298134876 conn(0x7f28240f7890 msgr2=0x7f28240f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:39.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.591+0000 7f2829398700 1 -- 172.21.15.33:0/2298134876 shutdown_connections 2024-04-25T02:14:39.598 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.591+0000 7f2829398700 1 -- 172.21.15.33:0/2298134876 wait complete. 2024-04-25T02:14:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.592+0000 7f2829398700 1 Processor -- start 2024-04-25T02:14:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.592+0000 7f2829398700 1 -- start start 2024-04-25T02:14:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.592+0000 7f2829398700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fdde0 0x7f2824113b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.592+0000 7f2829398700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28240fe780 0x7f2824114070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.592+0000 7f2829398700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28241146c0 con 0x7f28240fdde0 2024-04-25T02:14:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.592+0000 7f2829398700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282410dab0 con 0x7f28240fe780 2024-04-25T02:14:39.600 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.592+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fdde0 0x7f2824113b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.592+0000 7f2822ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fdde0 0x7f2824113b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37010/0 (socket says 172.21.15.33:37010) 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.592+0000 7f2822ffd700 1 -- 172.21.15.33:0/4019164923 learned_addr learned my addr 172.21.15.33:0/4019164923 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f28227fc700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28240fe780 0x7f2824114070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f2822ffd700 1 -- 172.21.15.33:0/4019164923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28240fe780 msgr2=0x7f2824114070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f2822ffd700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28240fe780 0x7f2824114070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f2822ffd700 1 -- 172.21.15.33:0/4019164923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2818005040 con 0x7f28240fdde0 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f2822ffd700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fdde0 0x7f2824113b30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2814002a50 tx=0x7f28140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f280b7fe700 1 -- 172.21.15.33:0/4019164923 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2814007cc0 con 0x7f28240fdde0 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f282410dd90 con 0x7f28240fdde0 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f280b7fe700 1 -- 172.21.15.33:0/4019164923 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f281400f040 con 0x7f28240fdde0 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f280b7fe700 1 -- 172.21.15.33:0/4019164923 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2814009640 con 0x7f28240fdde0 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.593+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282410e2d0 con 0x7f28240fdde0 2024-04-25T02:14:39.601 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.594+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2824066750 con 0x7f28240fdde0 2024-04-25T02:14:39.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.594+0000 7f280b7fe700 1 -- 172.21.15.33:0/4019164923 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f28140097a0 con 0x7f28240fdde0 2024-04-25T02:14:39.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.595+0000 7f280b7fe700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f280c06e140 0x7f280c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:39.602 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.595+0000 7f280b7fe700 1 -- 172.21.15.33:0/4019164923 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f281408d940 con 0x7f28240fdde0 2024-04-25T02:14:39.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.595+0000 7f28227fc700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f280c06e140 0x7f280c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:39.603 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.595+0000 7f28227fc700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f280c06e140 0x7f280c0705f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f282410ec70 tx=0x7f2818009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:39.605 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.600+0000 7f280b7fe700 1 -- 172.21.15.33:0/4019164923 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f281405a7f0 con 0x7f28240fdde0 2024-04-25T02:14:39.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:39 smithi138 ceph-mon[81304]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:39.861 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.855+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2824109e70 con 0x7f28240fdde0 2024-04-25T02:14:39.862 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.856+0000 7f280b7fe700 1 -- 172.21.15.33:0/4019164923 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f281405a380 con 0x7f28240fdde0 2024-04-25T02:14:39.862 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:14:39.862 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:14:39.862 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:14:39.862 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:14:39.862 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:14:39.862 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:14:39.862 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:14:39.863 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:14:39.864 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:14:39.865 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:14:39.865 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:14:39.865 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:14:39.865 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:14:39.865 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f280c06e140 msgr2=0x7f280c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f280c06e140 0x7f280c0705f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f282410ec70 tx=0x7f2818009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fdde0 msgr2=0x7f2824113b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fdde0 0x7f2824113b30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2814002a50 tx=0x7f28140046b0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 shutdown_connections 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f280c06e140 0x7f280c0705f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f28240fdde0 0x7f2824113b30 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 --2- 172.21.15.33:0/4019164923 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f28240fe780 0x7f2824114070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 >> 172.21.15.33:0/4019164923 conn(0x7f28240f7890 msgr2=0x7f28240f8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.858+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 shutdown_connections 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.859+0000 7f2829398700 1 -- 172.21.15.33:0/4019164923 wait complete. 2024-04-25T02:14:39.866 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 -- 172.21.15.33:0/1943974501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 msgr2=0x7f832c1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 --2- 172.21.15.33:0/1943974501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 0x7f832c1031b0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f831c004940 tx=0x7f831c009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 -- 172.21.15.33:0/1943974501 shutdown_connections 2024-04-25T02:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 --2- 172.21.15.33:0/1943974501 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832c103780 0x7f832c0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 --2- 172.21.15.33:0/1943974501 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 0x7f832c1031b0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 -- 172.21.15.33:0/1943974501 >> 172.21.15.33:0/1943974501 conn(0x7f832c0f8990 msgr2=0x7f832c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 -- 172.21.15.33:0/1943974501 shutdown_connections 2024-04-25T02:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 -- 172.21.15.33:0/1943974501 wait complete. 2024-04-25T02:14:39.970 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 Processor -- start 2024-04-25T02:14:39.971 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.964+0000 7f8332241700 1 -- start start 2024-04-25T02:14:39.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f8332241700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 0x7f832c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:39.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f8332241700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832c103780 0x7f832c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:39.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f8332241700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c072160 con 0x7f832c102de0 2024-04-25T02:14:39.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f8332241700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c070ce0 con 0x7f832c103780 2024-04-25T02:14:39.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f832b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 0x7f832c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:39.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f832b7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 0x7f832c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37030/0 (socket says 172.21.15.33:37030) 2024-04-25T02:14:39.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f832b7fe700 1 -- 172.21.15.33:0/3555853462 learned_addr learned my addr 172.21.15.33:0/3555853462 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:39.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f8323fff700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832c103780 0x7f832c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f832b7fe700 1 -- 172.21.15.33:0/3555853462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832c103780 msgr2=0x7f832c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f832b7fe700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832c103780 0x7f832c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f832b7fe700 1 -- 172.21.15.33:0/3555853462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c005040 con 0x7f832c102de0 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f832b7fe700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 0x7f832c070260 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f831c00b3b0 tx=0x7f831c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f8328ff9700 1 -- 172.21.15.33:0/3555853462 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f831c00e040 con 0x7f832c102de0 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f8328ff9700 1 -- 172.21.15.33:0/3555853462 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f831c00bbc0 con 0x7f832c102de0 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c070f60 con 0x7f832c102de0 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.965+0000 7f8328ff9700 1 -- 172.21.15.33:0/3555853462 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f831c01c3f0 con 0x7f832c102de0 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.966+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c11a110 con 0x7f832c102de0 2024-04-25T02:14:39.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.966+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f832c1000f0 con 0x7f832c102de0 2024-04-25T02:14:39.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.967+0000 7f8328ff9700 1 -- 172.21.15.33:0/3555853462 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f831c00bd30 con 0x7f832c102de0 2024-04-25T02:14:39.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.967+0000 7f8328ff9700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f831806e140 0x7f83180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:39.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.968+0000 7f8328ff9700 1 -- 172.21.15.33:0/3555853462 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f831c08ed70 con 0x7f832c102de0 2024-04-25T02:14:39.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.968+0000 7f8323fff700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f831806e140 0x7f83180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:39.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.968+0000 7f8323fff700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f831806e140 0x7f83180705f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f832c071940 tx=0x7f8314009ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:39.978 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:39.972+0000 7f8328ff9700 1 -- 172.21.15.33:0/3555853462 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f831c0580f0 con 0x7f832c102de0 2024-04-25T02:14:40.204 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.198+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f832c071760 con 0x7f831806e140 2024-04-25T02:14:40.205 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.199+0000 7f8328ff9700 1 -- 172.21.15.33:0/3555853462 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f832c071760 con 0x7f831806e140 2024-04-25T02:14:40.205 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:14:40.205 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:14:40.205 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:14:40.205 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:14:40.205 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:14:40.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:14:40.205 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:14:40.205 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:14:40.206 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:14:40.206 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:14:40.206 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:14:40.206 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:14:40.206 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:14:40.206 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:14:40.206 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:14:40.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f831806e140 msgr2=0x7f83180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f831806e140 0x7f83180705f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f832c071940 tx=0x7f8314009ee0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 msgr2=0x7f832c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 0x7f832c070260 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f831c00b3b0 tx=0x7f831c009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 shutdown_connections 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f831806e140 0x7f83180705f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f832c102de0 0x7f832c070260 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 --2- 172.21.15.33:0/3555853462 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f832c103780 0x7f832c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 >> 172.21.15.33:0/3555853462 conn(0x7f832c0f8990 msgr2=0x7f832c107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 shutdown_connections 2024-04-25T02:14:40.208 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.202+0000 7f8332241700 1 -- 172.21.15.33:0/3555853462 wait complete. 2024-04-25T02:14:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.307+0000 7f2d725cb700 1 -- 172.21.15.33:0/2663929680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 msgr2=0x7f2d6c1031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.307+0000 7f2d725cb700 1 --2- 172.21.15.33:0/2663929680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 0x7f2d6c1031c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c004940 tx=0x7f2d5c009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.307+0000 7f2d725cb700 1 -- 172.21.15.33:0/2663929680 shutdown_connections 2024-04-25T02:14:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.307+0000 7f2d725cb700 1 --2- 172.21.15.33:0/2663929680 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d6c103790 0x7f2d6c0fce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.307+0000 7f2d725cb700 1 --2- 172.21.15.33:0/2663929680 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 0x7f2d6c1031c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.307+0000 7f2d725cb700 1 -- 172.21.15.33:0/2663929680 >> 172.21.15.33:0/2663929680 conn(0x7f2d6c0f8980 msgr2=0x7f2d6c0fad90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:40.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.307+0000 7f2d725cb700 1 -- 172.21.15.33:0/2663929680 shutdown_connections 2024-04-25T02:14:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.307+0000 7f2d725cb700 1 -- 172.21.15.33:0/2663929680 wait complete. 2024-04-25T02:14:40.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d725cb700 1 Processor -- start 2024-04-25T02:14:40.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d725cb700 1 -- start start 2024-04-25T02:14:40.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d725cb700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 0x7f2d6c113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:40.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d725cb700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d6c103790 0x7f2d6c1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:40.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d725cb700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c114730 con 0x7f2d6c102df0 2024-04-25T02:14:40.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d725cb700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c10db40 con 0x7f2d6c103790 2024-04-25T02:14:40.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 0x7f2d6c113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d6bfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 0x7f2d6c113ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:37034/0 (socket says 172.21.15.33:37034) 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d6bfff700 1 -- 172.21.15.33:0/1776467042 learned_addr learned my addr 172.21.15.33:0/1776467042 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.308+0000 7f2d6b7fe700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d6c103790 0x7f2d6c1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.309+0000 7f2d6bfff700 1 -- 172.21.15.33:0/1776467042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d6c103790 msgr2=0x7f2d6c1140e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.309+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d6c103790 0x7f2d6c1140e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.309+0000 7f2d6bfff700 1 -- 172.21.15.33:0/1776467042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c005040 con 0x7f2d6c102df0 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.309+0000 7f2d6bfff700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 0x7f2d6c113ba0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009150 tx=0x7f2d5c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.309+0000 7f2d68ff9700 1 -- 172.21.15.33:0/1776467042 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d5c00e040 con 0x7f2d6c102df0 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.309+0000 7f2d68ff9700 1 -- 172.21.15.33:0/1776467042 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d5c013420 con 0x7f2d6c102df0 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.309+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c10ddc0 con 0x7f2d6c102df0 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.309+0000 7f2d68ff9700 1 -- 172.21.15.33:0/1776467042 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d5c01c630 con 0x7f2d6c102df0 2024-04-25T02:14:40.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.309+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c10e380 con 0x7f2d6c102df0 2024-04-25T02:14:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.310+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c100100 con 0x7f2d6c102df0 2024-04-25T02:14:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.311+0000 7f2d68ff9700 1 -- 172.21.15.33:0/1776467042 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2d5c00baf0 con 0x7f2d6c102df0 2024-04-25T02:14:40.317 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.311+0000 7f2d68ff9700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d5806e150 0x7f2d58070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:14:40.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.311+0000 7f2d68ff9700 1 -- 172.21.15.33:0/1776467042 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2d5c08ecd0 con 0x7f2d6c102df0 2024-04-25T02:14:40.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.311+0000 7f2d6b7fe700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d5806e150 0x7f2d58070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:14:40.318 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.312+0000 7f2d6b7fe700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d5806e150 0x7f2d58070600 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2d54007fd0 tx=0x7f2d54009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:14:40.321 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.316+0000 7f2d68ff9700 1 -- 172.21.15.33:0/1776467042 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2d5c058170 con 0x7f2d6c102df0 2024-04-25T02:14:40.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:40 smithi033 ceph-mon[111306]: from='client.35920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:40.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:40 smithi033 ceph-mon[111306]: from='client.35924 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:40.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:40 smithi033 ceph-mon[111306]: pgmap v1059: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:40.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:40 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1986470786' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:14:40.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:40 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4019164923' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:14:40.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.619+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2d6c109ef0 con 0x7f2d6c102df0 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.619+0000 7f2d68ff9700 1 -- 172.21.15.33:0/1776467042 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f2d5c05b830 con 0x7f2d6c102df0 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:14:40.625 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:14:40.626 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:14:40.627 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:14:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d5806e150 msgr2=0x7f2d58070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d5806e150 0x7f2d58070600 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2d54007fd0 tx=0x7f2d54009040 comp rx=0 tx=0).stop 2024-04-25T02:14:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 msgr2=0x7f2d6c113ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:14:40.627 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 0x7f2d6c113ba0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c009150 tx=0x7f2d5c009e30 comp rx=0 tx=0).stop 2024-04-25T02:14:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 shutdown_connections 2024-04-25T02:14:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d5806e150 0x7f2d58070600 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d6c102df0 0x7f2d6c113ba0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 --2- 172.21.15.33:0/1776467042 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d6c103790 0x7f2d6c1140e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:14:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 >> 172.21.15.33:0/1776467042 conn(0x7f2d6c0f8980 msgr2=0x7f2d6c1073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:14:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 shutdown_connections 2024-04-25T02:14:40.628 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:14:40.621+0000 7f2d725cb700 1 -- 172.21.15.33:0/1776467042 wait complete. 2024-04-25T02:14:40.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:40 smithi138 ceph-mon[81304]: from='client.35920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:40.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:40 smithi138 ceph-mon[81304]: from='client.35924 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:40.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:40 smithi138 ceph-mon[81304]: pgmap v1059: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:40.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:40 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1986470786' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:14:40.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:40 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4019164923' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:14:40.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:14:41.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:41 smithi033 ceph-mon[111306]: from='client.35936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:41.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:41 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1776467042' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:14:41.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:41 smithi138 ceph-mon[81304]: from='client.35936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:14:41.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:41 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1776467042' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:14:42.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:42 smithi033 ceph-mon[111306]: pgmap v1060: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:42.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:42 smithi138 ceph-mon[81304]: pgmap v1060: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:44.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:44 smithi033 ceph-mon[111306]: pgmap v1061: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:44.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:44 smithi138 ceph-mon[81304]: pgmap v1061: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:46.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:46 smithi033 ceph-mon[111306]: pgmap v1062: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:46.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:46 smithi138 ceph-mon[81304]: pgmap v1062: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:48.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:48 smithi033 ceph-mon[111306]: pgmap v1063: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:48.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:48 smithi138 ceph-mon[81304]: pgmap v1063: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:50.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:50 smithi033 ceph-mon[111306]: pgmap v1064: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:50.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:50 smithi138 ceph-mon[81304]: pgmap v1064: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:52.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:52 smithi033 ceph-mon[111306]: pgmap v1065: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:52.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:52 smithi138 ceph-mon[81304]: pgmap v1065: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:54.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:54 smithi033 ceph-mon[111306]: pgmap v1066: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:54.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:54 smithi138 ceph-mon[81304]: pgmap v1066: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:55.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:14:55.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:14:56.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:56 smithi033 ceph-mon[111306]: pgmap v1067: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:56.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:56 smithi138 ceph-mon[81304]: pgmap v1067: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:58.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:14:58 smithi033 ceph-mon[111306]: pgmap v1068: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:14:58.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:14:58 smithi138 ceph-mon[81304]: pgmap v1068: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:00.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:00 smithi033 ceph-mon[111306]: pgmap v1069: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:00.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:00 smithi138 ceph-mon[81304]: pgmap v1069: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:02.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:02 smithi033 ceph-mon[111306]: pgmap v1070: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:02.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:02 smithi138 ceph-mon[81304]: pgmap v1070: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:03.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:03 smithi033 ceph-mon[111306]: pgmap v1071: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:03.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:03 smithi138 ceph-mon[81304]: pgmap v1071: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:06.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:06 smithi033 ceph-mon[111306]: pgmap v1072: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:06.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:06 smithi138 ceph-mon[81304]: pgmap v1072: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:08.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:08 smithi033 ceph-mon[111306]: pgmap v1073: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:08.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:08 smithi138 ceph-mon[81304]: pgmap v1073: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.740+0000 7efe4352d700 1 -- 172.21.15.33:0/3305607093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c103780 msgr2=0x7efe3c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.740+0000 7efe4352d700 1 --2- 172.21.15.33:0/3305607093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c103780 0x7efe3c0fcdf0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7efe2c004940 tx=0x7efe2c009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.740+0000 7efe4352d700 1 -- 172.21.15.33:0/3305607093 shutdown_connections 2024-04-25T02:15:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.740+0000 7efe4352d700 1 --2- 172.21.15.33:0/3305607093 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c103780 0x7efe3c0fcdf0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.740+0000 7efe4352d700 1 --2- 172.21.15.33:0/3305607093 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c102de0 0x7efe3c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:10.746 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.740+0000 7efe4352d700 1 -- 172.21.15.33:0/3305607093 >> 172.21.15.33:0/3305607093 conn(0x7efe3c0f8990 msgr2=0x7efe3c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:10.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:10 smithi033 ceph-mon[111306]: pgmap v1074: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:10.746 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:15:10.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.740+0000 7efe4352d700 1 -- 172.21.15.33:0/3305607093 shutdown_connections 2024-04-25T02:15:10.748 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.740+0000 7efe4352d700 1 -- 172.21.15.33:0/3305607093 wait complete. 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe4352d700 1 Processor -- start 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe4352d700 1 -- start start 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe4352d700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c102de0 0x7efe3c070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe4352d700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c103780 0x7efe3c0707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe4352d700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0721b0 con 0x7efe3c102de0 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe4352d700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c070ce0 con 0x7efe3c103780 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe412c9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c102de0 0x7efe3c070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe412c9700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c102de0 0x7efe3c070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55226/0 (socket says 172.21.15.33:55226) 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe412c9700 1 -- 172.21.15.33:0/382919708 learned_addr learned my addr 172.21.15.33:0/382919708 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe40ac8700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c103780 0x7efe3c0707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe412c9700 1 -- 172.21.15.33:0/382919708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c103780 msgr2=0x7efe3c0707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe412c9700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c103780 0x7efe3c0707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.741+0000 7efe412c9700 1 -- 172.21.15.33:0/382919708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c005040 con 0x7efe3c102de0 2024-04-25T02:15:10.749 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.742+0000 7efe412c9700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c102de0 0x7efe3c070260 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7efe38002a50 tx=0x7efe380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.742+0000 7efe31ffb700 1 -- 172.21.15.33:0/382919708 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe38007cc0 con 0x7efe3c102de0 2024-04-25T02:15:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.742+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c070fc0 con 0x7efe3c102de0 2024-04-25T02:15:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.742+0000 7efe31ffb700 1 -- 172.21.15.33:0/382919708 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe3800f040 con 0x7efe3c102de0 2024-04-25T02:15:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.742+0000 7efe31ffb700 1 -- 172.21.15.33:0/382919708 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe3800da30 con 0x7efe3c102de0 2024-04-25T02:15:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.742+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c11a0a0 con 0x7efe3c102de0 2024-04-25T02:15:10.750 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.743+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c0fe530 con 0x7efe3c102de0 2024-04-25T02:15:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.743+0000 7efe31ffb700 1 -- 172.21.15.33:0/382919708 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efe3800db90 con 0x7efe3c102de0 2024-04-25T02:15:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.744+0000 7efe31ffb700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe2806e140 0x7efe280705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.744+0000 7efe31ffb700 1 -- 172.21.15.33:0/382919708 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efe3808d8d0 con 0x7efe3c102de0 2024-04-25T02:15:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.744+0000 7efe40ac8700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe2806e140 0x7efe280705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:10.751 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.744+0000 7efe40ac8700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe2806e140 0x7efe280705f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efe2c004890 tx=0x7efe2c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:10.754 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.749+0000 7efe31ffb700 1 -- 172.21.15.33:0/382919708 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efe38056c50 con 0x7efe3c102de0 2024-04-25T02:15:10.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:10 smithi138 ceph-mon[81304]: pgmap v1074: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:10.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:15:10.981 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.976+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efe3c0fa0a0 con 0x7efe2806e140 2024-04-25T02:15:10.982 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.977+0000 7efe31ffb700 1 -- 172.21.15.33:0/382919708 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7efe3c0fa0a0 con 0x7efe2806e140 2024-04-25T02:15:10.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe2806e140 msgr2=0x7efe280705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:10.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe2806e140 0x7efe280705f0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efe2c004890 tx=0x7efe2c009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:10.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c102de0 msgr2=0x7efe3c070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:10.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c102de0 0x7efe3c070260 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7efe38002a50 tx=0x7efe380046b0 comp rx=0 tx=0).stop 2024-04-25T02:15:10.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 shutdown_connections 2024-04-25T02:15:10.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efe2806e140 0x7efe280705f0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:10.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efe3c102de0 0x7efe3c070260 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:10.985 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 --2- 172.21.15.33:0/382919708 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efe3c103780 0x7efe3c0707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 >> 172.21.15.33:0/382919708 conn(0x7efe3c0f8990 msgr2=0x7efe3c0f9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.979+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 shutdown_connections 2024-04-25T02:15:10.986 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:10.980+0000 7efe4352d700 1 -- 172.21.15.33:0/382919708 wait complete. 2024-04-25T02:15:10.995 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.084+0000 7f71e29c8700 1 -- 172.21.15.33:0/1831903153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 msgr2=0x7f71dc106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.084+0000 7f71e29c8700 1 --2- 172.21.15.33:0/1831903153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 0x7f71dc106e40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f71cc004940 tx=0x7f71cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.084+0000 7f71e29c8700 1 -- 172.21.15.33:0/1831903153 shutdown_connections 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.084+0000 7f71e29c8700 1 --2- 172.21.15.33:0/1831903153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 0x7f71dc106e40 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.084+0000 7f71e29c8700 1 --2- 172.21.15.33:0/1831903153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc0fde60 0x7f71dc0fe230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.084+0000 7f71e29c8700 1 -- 172.21.15.33:0/1831903153 >> 172.21.15.33:0/1831903153 conn(0x7f71dc0f7870 msgr2=0x7f71dc0f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.084+0000 7f71e29c8700 1 -- 172.21.15.33:0/1831903153 shutdown_connections 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.084+0000 7f71e29c8700 1 -- 172.21.15.33:0/1831903153 wait complete. 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.084+0000 7f71e29c8700 1 Processor -- start 2024-04-25T02:15:11.090 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71e29c8700 1 -- start start 2024-04-25T02:15:11.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71e29c8700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc0fde60 0x7f71dc0fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:11.092 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71e29c8700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 0x7f71dc0fc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71e29c8700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0fdbe0 con 0x7f71dc0fe800 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71e29c8700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71dc0fc710 con 0x7f71dc0fde60 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 0x7f71dc0fc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71db7fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 0x7f71dc0fc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55252/0 (socket says 172.21.15.33:55252) 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71db7fe700 1 -- 172.21.15.33:0/1757892368 learned_addr learned my addr 172.21.15.33:0/1757892368 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71dbfff700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc0fde60 0x7f71dc0fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71db7fe700 1 -- 172.21.15.33:0/1757892368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc0fde60 msgr2=0x7f71dc0fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71db7fe700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc0fde60 0x7f71dc0fbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.085+0000 7f71db7fe700 1 -- 172.21.15.33:0/1757892368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71cc005040 con 0x7f71dc0fe800 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.086+0000 7f71db7fe700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 0x7f71dc0fc1d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00b3b0 tx=0x7f71cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.086+0000 7f71d8ff9700 1 -- 172.21.15.33:0/1757892368 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71cc00e040 con 0x7f71dc0fe800 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.086+0000 7f71d8ff9700 1 -- 172.21.15.33:0/1757892368 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71cc013420 con 0x7f71dc0fe800 2024-04-25T02:15:11.093 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.086+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71dc0fc9f0 con 0x7f71dc0fe800 2024-04-25T02:15:11.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.086+0000 7f71d8ff9700 1 -- 172.21.15.33:0/1757892368 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71cc01c630 con 0x7f71dc0fe800 2024-04-25T02:15:11.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.086+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71dc11a050 con 0x7f71dc0fe800 2024-04-25T02:15:11.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.087+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71dc066ad0 con 0x7f71dc0fe800 2024-04-25T02:15:11.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.087+0000 7f71d8ff9700 1 -- 172.21.15.33:0/1757892368 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f71cc013590 con 0x7f71dc0fe800 2024-04-25T02:15:11.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.088+0000 7f71d8ff9700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f71c406e140 0x7f71c40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:11.094 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.088+0000 7f71d8ff9700 1 -- 172.21.15.33:0/1757892368 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f71cc08ec40 con 0x7f71dc0fe800 2024-04-25T02:15:11.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.088+0000 7f71dbfff700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f71c406e140 0x7f71c40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:11.095 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.088+0000 7f71dbfff700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f71c406e140 0x7f71c40705f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f71d0004f40 tx=0x7f71d000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:11.098 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.092+0000 7f71d8ff9700 1 -- 172.21.15.33:0/1757892368 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f71cc0580e0 con 0x7f71dc0fe800 2024-04-25T02:15:11.325 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.320+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71dc0f9460 con 0x7f71c406e140 2024-04-25T02:15:11.326 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.321+0000 7f71d8ff9700 1 -- 172.21.15.33:0/1757892368 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f71dc0f9460 con 0x7f71c406e140 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f71c406e140 msgr2=0x7f71c40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f71c406e140 0x7f71c40705f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f71d0004f40 tx=0x7f71d000b040 comp rx=0 tx=0).stop 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 msgr2=0x7f71dc0fc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 0x7f71dc0fc1d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f71cc00b3b0 tx=0x7f71cc009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 shutdown_connections 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f71c406e140 0x7f71c40705f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f71dc0fde60 0x7f71dc0fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 --2- 172.21.15.33:0/1757892368 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f71dc0fe800 0x7f71dc0fc1d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 >> 172.21.15.33:0/1757892368 conn(0x7f71dc0f7870 msgr2=0x7f71dc0f8d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:11.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 shutdown_connections 2024-04-25T02:15:11.330 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.323+0000 7f71e29c8700 1 -- 172.21.15.33:0/1757892368 wait complete. 2024-04-25T02:15:11.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.428+0000 7f3d07d94700 1 -- 172.21.15.33:0/2923619553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00103780 msgr2=0x7f3d000fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.428+0000 7f3d07d94700 1 --2- 172.21.15.33:0/2923619553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00103780 0x7f3d000fcdf0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc004940 tx=0x7f3cfc009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:11.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.428+0000 7f3d07d94700 1 -- 172.21.15.33:0/2923619553 shutdown_connections 2024-04-25T02:15:11.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.428+0000 7f3d07d94700 1 --2- 172.21.15.33:0/2923619553 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00103780 0x7f3d000fcdf0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.428+0000 7f3d07d94700 1 --2- 172.21.15.33:0/2923619553 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d00102de0 0x7f3d001031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.428+0000 7f3d07d94700 1 -- 172.21.15.33:0/2923619553 >> 172.21.15.33:0/2923619553 conn(0x7f3d000f8990 msgr2=0x7f3d000fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:11.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.429+0000 7f3d07d94700 1 -- 172.21.15.33:0/2923619553 shutdown_connections 2024-04-25T02:15:11.522 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.429+0000 7f3d07d94700 1 -- 172.21.15.33:0/2923619553 wait complete. 2024-04-25T02:15:11.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.429+0000 7f3d07d94700 1 Processor -- start 2024-04-25T02:15:11.525 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.429+0000 7f3d07d94700 1 -- start start 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.429+0000 7f3d07d94700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00102de0 0x7f3d00070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.429+0000 7f3d07d94700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d00103780 0x7f3d000707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.429+0000 7f3d07d94700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000721b0 con 0x7f3d00102de0 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.429+0000 7f3d07d94700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d00070ce0 con 0x7f3d00103780 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d05b30700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00102de0 0x7f3d00070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d05b30700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00102de0 0x7f3d00070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55270/0 (socket says 172.21.15.33:55270) 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d05b30700 1 -- 172.21.15.33:0/114778827 learned_addr learned my addr 172.21.15.33:0/114778827 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d0532f700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d00103780 0x7f3d000707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d05b30700 1 -- 172.21.15.33:0/114778827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d00103780 msgr2=0x7f3d000707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d05b30700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d00103780 0x7f3d000707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d05b30700 1 -- 172.21.15.33:0/114778827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc005040 con 0x7f3d00102de0 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d0532f700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d00103780 0x7f3d000707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:15:11.526 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d05b30700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00102de0 0x7f3d00070260 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400c8c0 tx=0x7f3cf40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3cf27fc700 1 -- 172.21.15.33:0/114778827 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf400a980 con 0x7f3d00102de0 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d00070fc0 con 0x7f3d00102de0 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3cf27fc700 1 -- 172.21.15.33:0/114778827 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cf400ce60 con 0x7f3d00102de0 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3cf27fc700 1 -- 172.21.15.33:0/114778827 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf40115e0 con 0x7f3d00102de0 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.430+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0011a140 con 0x7f3d00102de0 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.431+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d000fe530 con 0x7f3d00102de0 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.432+0000 7f3cf27fc700 1 -- 172.21.15.33:0/114778827 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3cf4012400 con 0x7f3d00102de0 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.433+0000 7f3cf27fc700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3cec06e140 0x7f3cec0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.433+0000 7f3cf27fc700 1 -- 172.21.15.33:0/114778827 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3cf408dbf0 con 0x7f3d00102de0 2024-04-25T02:15:11.527 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.433+0000 7f3d0532f700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3cec06e140 0x7f3cec0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:11.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.433+0000 7f3d0532f700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3cec06e140 0x7f3cec0705f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:11.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.438+0000 7f3cf27fc700 1 -- 172.21.15.33:0/114778827 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3cf4056ff0 con 0x7f3d00102de0 2024-04-25T02:15:11.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.655+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d000fa080 con 0x7f3cec06e140 2024-04-25T02:15:11.670 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.665+0000 7f3cf27fc700 1 -- 172.21.15.33:0/114778827 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3d000fa080 con 0x7f3cec06e140 2024-04-25T02:15:11.670 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:15:11.670 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (36m) 14m ago 47m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:15:11.671 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (33m) 14m ago 47m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:15:11.671 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (33m) 14m ago 46m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:15:11.671 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (36m) 14m ago 47m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:15:11.671 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (29m) 14m ago 42m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:15:11.671 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (29m) 14m ago 42m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:15:11.671 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (29m) 14m ago 42m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:15:11.671 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (14m) 14m ago 42m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:15:11.671 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (37m) 14m ago 48m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (37m) 14m ago 46m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (34m) 14m ago 49m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (34m) 14m ago 46m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (37m) 14m ago 47m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (36m) 14m ago 46m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (33m) 14m ago 45m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (32m) 14m ago 45m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (31m) 14m ago 44m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (31m) 14m ago 44m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (30m) 14m ago 44m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (30m) 14m ago 43m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:15:11.672 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (36m) 14m ago 47m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3cec06e140 msgr2=0x7f3cec0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3cec06e140 0x7f3cec0705f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00102de0 msgr2=0x7f3d00070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00102de0 0x7f3d00070260 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f3cf400c8c0 tx=0x7f3cf40044c0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 shutdown_connections 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3cec06e140 0x7f3cec0705f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3d00102de0 0x7f3d00070260 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 --2- 172.21.15.33:0/114778827 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3d00103780 0x7f3d000707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 >> 172.21.15.33:0/114778827 conn(0x7f3d000f8990 msgr2=0x7f3d000f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 shutdown_connections 2024-04-25T02:15:11.673 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.667+0000 7f3d07d94700 1 -- 172.21.15.33:0/114778827 wait complete. 2024-04-25T02:15:11.776 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.771+0000 7f1e5d408700 1 -- 172.21.15.33:0/4093446988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 msgr2=0x7f1e580fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.771+0000 7f1e5d408700 1 --2- 172.21.15.33:0/4093446988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 0x7f1e580fc1b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c004940 tx=0x7f1e4c009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:11.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.771+0000 7f1e5d408700 1 -- 172.21.15.33:0/4093446988 shutdown_connections 2024-04-25T02:15:11.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.771+0000 7f1e5d408700 1 --2- 172.21.15.33:0/4093446988 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 0x7f1e580fc1b0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.771+0000 7f1e5d408700 1 --2- 172.21.15.33:0/4093446988 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e58101d90 0x7f1e58102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.771+0000 7f1e5d408700 1 -- 172.21.15.33:0/4093446988 >> 172.21.15.33:0/4093446988 conn(0x7f1e58071d00 msgr2=0x7f1e58072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:11.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.771+0000 7f1e5d408700 1 -- 172.21.15.33:0/4093446988 shutdown_connections 2024-04-25T02:15:11.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.771+0000 7f1e5d408700 1 -- 172.21.15.33:0/4093446988 wait complete. 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e5d408700 1 Processor -- start 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e5d408700 1 -- start start 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e5d408700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 0x7f1e58102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e5d408700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e58101d90 0x7f1e58103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e5d408700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e581035a0 con 0x7f1e580fbd40 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e5d408700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e581036e0 con 0x7f1e58101d90 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 0x7f1e58102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e56ffd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 0x7f1e58102b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55294/0 (socket says 172.21.15.33:55294) 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e56ffd700 1 -- 172.21.15.33:0/4175502285 learned_addr learned my addr 172.21.15.33:0/4175502285 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e567fc700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e58101d90 0x7f1e58103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e56ffd700 1 -- 172.21.15.33:0/4175502285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e58101d90 msgr2=0x7f1e58103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e56ffd700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e58101d90 0x7f1e58103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:11.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e56ffd700 1 -- 172.21.15.33:0/4175502285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c005040 con 0x7f1e580fbd40 2024-04-25T02:15:11.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.772+0000 7f1e56ffd700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 0x7f1e58102b20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800c8c0 tx=0x7f1e480044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:11.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.773+0000 7f1e3f7fe700 1 -- 172.21.15.33:0/4175502285 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e4800a760 con 0x7f1e580fbd40 2024-04-25T02:15:11.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.773+0000 7f1e3f7fe700 1 -- 172.21.15.33:0/4175502285 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e4800ad90 con 0x7f1e580fbd40 2024-04-25T02:15:11.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.773+0000 7f1e3f7fe700 1 -- 172.21.15.33:0/4175502285 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e480113a0 con 0x7f1e580fbd40 2024-04-25T02:15:11.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.773+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e58103990 con 0x7f1e580fbd40 2024-04-25T02:15:11.780 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.773+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5811a320 con 0x7f1e580fbd40 2024-04-25T02:15:11.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.774+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e580fd430 con 0x7f1e580fbd40 2024-04-25T02:15:11.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.774+0000 7f1e3f7fe700 1 -- 172.21.15.33:0/4175502285 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1e48012400 con 0x7f1e580fbd40 2024-04-25T02:15:11.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.775+0000 7f1e3f7fe700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4006e150 0x7f1e40070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:11.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.775+0000 7f1e3f7fe700 1 -- 172.21.15.33:0/4175502285 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1e4808d9a0 con 0x7f1e580fbd40 2024-04-25T02:15:11.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.775+0000 7f1e567fc700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4006e150 0x7f1e40070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:11.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.775+0000 7f1e567fc700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4006e150 0x7f1e40070600 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00b8b0 tx=0x7f1e4c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:11.785 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:11.780+0000 7f1e3f7fe700 1 -- 172.21.15.33:0/4175502285 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1e48056da0 con 0x7f1e580fbd40 2024-04-25T02:15:12.080 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.075+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1e5804bf00 con 0x7f1e580fbd40 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.075+0000 7f1e3f7fe700 1 -- 172.21.15.33:0/4175502285 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1e4805a460 con 0x7f1e580fbd40 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:15:12.081 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:15:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:15:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:15:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:15:12.082 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:15:12.082 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4006e150 msgr2=0x7f1e40070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4006e150 0x7f1e40070600 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00b8b0 tx=0x7f1e4c009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 msgr2=0x7f1e58102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 0x7f1e58102b20 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1e4800c8c0 tx=0x7f1e480044c0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 shutdown_connections 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4006e150 0x7f1e40070600 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e580fbd40 0x7f1e58102b20 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 --2- 172.21.15.33:0/4175502285 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e58101d90 0x7f1e58103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 >> 172.21.15.33:0/4175502285 conn(0x7f1e58071d00 msgr2=0x7f1e580655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 shutdown_connections 2024-04-25T02:15:12.083 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.077+0000 7f1e5d408700 1 -- 172.21.15.33:0/4175502285 wait complete. 2024-04-25T02:15:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.181+0000 7f47ac9f4700 1 -- 172.21.15.33:0/1598889080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fb5a0 msgr2=0x7f47a80fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.181+0000 7f47ac9f4700 1 --2- 172.21.15.33:0/1598889080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fb5a0 0x7f47a80fb970 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f4798004940 tx=0x7f4798009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.181+0000 7f47ac9f4700 1 -- 172.21.15.33:0/1598889080 shutdown_connections 2024-04-25T02:15:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.181+0000 7f47ac9f4700 1 --2- 172.21.15.33:0/1598889080 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a80fbf40 0x7f47a8101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.181+0000 7f47ac9f4700 1 --2- 172.21.15.33:0/1598889080 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fb5a0 0x7f47a80fb970 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.181+0000 7f47ac9f4700 1 -- 172.21.15.33:0/1598889080 >> 172.21.15.33:0/1598889080 conn(0x7f47a8073350 msgr2=0x7f47a8073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.181+0000 7f47ac9f4700 1 -- 172.21.15.33:0/1598889080 shutdown_connections 2024-04-25T02:15:12.187 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.181+0000 7f47ac9f4700 1 -- 172.21.15.33:0/1598889080 wait complete. 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f47ac9f4700 1 Processor -- start 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f47ac9f4700 1 -- start start 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f47ac9f4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a80fb5a0 0x7f47a8113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f47ac9f4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fbf40 0x7f47a8113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f47ac9f4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a8114510 con 0x7f47a80fbf40 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f47ac9f4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47a810db30 con 0x7f47a80fb5a0 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f479ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fbf40 0x7f47a8113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f479ffff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fbf40 0x7f47a8113ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55314/0 (socket says 172.21.15.33:55314) 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f479ffff700 1 -- 172.21.15.33:0/2439244458 learned_addr learned my addr 172.21.15.33:0/2439244458 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.182+0000 7f47a659c700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a80fb5a0 0x7f47a8113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:12.189 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.183+0000 7f479ffff700 1 -- 172.21.15.33:0/2439244458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a80fb5a0 msgr2=0x7f47a8113980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.183+0000 7f479ffff700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a80fb5a0 0x7f47a8113980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.183+0000 7f479ffff700 1 -- 172.21.15.33:0/2439244458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4798005040 con 0x7f47a80fbf40 2024-04-25T02:15:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.183+0000 7f479ffff700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fbf40 0x7f47a8113ec0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f479000c810 tx=0x7f47900043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.183+0000 7f479effd700 1 -- 172.21.15.33:0/2439244458 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4790012810 con 0x7f47a80fbf40 2024-04-25T02:15:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.183+0000 7f479effd700 1 -- 172.21.15.33:0/2439244458 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4790012e40 con 0x7f47a80fbf40 2024-04-25T02:15:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.183+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47a810de10 con 0x7f47a80fbf40 2024-04-25T02:15:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.183+0000 7f479effd700 1 -- 172.21.15.33:0/2439244458 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47900102e0 con 0x7f47a80fbf40 2024-04-25T02:15:12.190 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.183+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47a810e350 con 0x7f47a80fbf40 2024-04-25T02:15:12.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.184+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47a80fe670 con 0x7f47a80fbf40 2024-04-25T02:15:12.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.184+0000 7f479effd700 1 -- 172.21.15.33:0/2439244458 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4790011380 con 0x7f47a80fbf40 2024-04-25T02:15:12.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.185+0000 7f479effd700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f479406e140 0x7f47940705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:12.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.185+0000 7f479effd700 1 -- 172.21.15.33:0/2439244458 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f479008dd40 con 0x7f47a80fbf40 2024-04-25T02:15:12.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.185+0000 7f47a659c700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f479406e140 0x7f47940705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:12.191 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.185+0000 7f47a659c700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f479406e140 0x7f47940705f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4798004f90 tx=0x7f4798009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:12.195 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.190+0000 7f479effd700 1 -- 172.21.15.33:0/2439244458 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f47900570c0 con 0x7f47a80fbf40 2024-04-25T02:15:12.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.449+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f47a8109f40 con 0x7f47a80fbf40 2024-04-25T02:15:12.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.449+0000 7f479effd700 1 -- 172.21.15.33:0/2439244458 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7f4790016070 con 0x7f47a80fbf40 2024-04-25T02:15:12.455 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:15:12.455 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:15:12.455 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:15:12.455 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:15:12.455 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:15:12.455 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:15:12.456 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:15:12.457 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:15:12.458 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:15:12.458 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:15:12.458 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:15:12.458 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:15:12.458 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:15:12.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.451+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f479406e140 msgr2=0x7f47940705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.451+0000 7f47ac9f4700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f479406e140 0x7f47940705f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4798004f90 tx=0x7f4798009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:12.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.451+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fbf40 msgr2=0x7f47a8113ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.451+0000 7f47ac9f4700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fbf40 0x7f47a8113ec0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f479000c810 tx=0x7f47900043f0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.451+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 shutdown_connections 2024-04-25T02:15:12.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.451+0000 7f47ac9f4700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f479406e140 0x7f47940705f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.451+0000 7f47ac9f4700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f47a80fb5a0 0x7f47a8113980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.451+0000 7f47ac9f4700 1 --2- 172.21.15.33:0/2439244458 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f47a80fbf40 0x7f47a8113ec0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.451+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 >> 172.21.15.33:0/2439244458 conn(0x7f47a8073350 msgr2=0x7f47a8100a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:12.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.452+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 shutdown_connections 2024-04-25T02:15:12.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.452+0000 7f47ac9f4700 1 -- 172.21.15.33:0/2439244458 wait complete. 2024-04-25T02:15:12.460 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.556+0000 7f6847278700 1 -- 172.21.15.33:0/3000450195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68400fc710 msgr2=0x7f68400fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.556+0000 7f6847278700 1 --2- 172.21.15.33:0/3000450195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68400fc710 0x7f68400fcb80 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f683c004940 tx=0x7f683c009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.556+0000 7f6847278700 1 -- 172.21.15.33:0/3000450195 shutdown_connections 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.556+0000 7f6847278700 1 --2- 172.21.15.33:0/3000450195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f68400fc710 0x7f68400fcb80 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.556+0000 7f6847278700 1 --2- 172.21.15.33:0/3000450195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6840101e20 0x7f68401021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.556+0000 7f6847278700 1 -- 172.21.15.33:0/3000450195 >> 172.21.15.33:0/3000450195 conn(0x7f6840071460 msgr2=0x7f6840071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.556+0000 7f6847278700 1 -- 172.21.15.33:0/3000450195 shutdown_connections 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.556+0000 7f6847278700 1 -- 172.21.15.33:0/3000450195 wait complete. 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6847278700 1 Processor -- start 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6847278700 1 -- start start 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6847278700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68400fc710 0x7f684010ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6847278700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840101e20 0x7f684010f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:12.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6847278700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684010f740 con 0x7f6840101e20 2024-04-25T02:15:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6847278700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684010f880 con 0x7f68400fc710 2024-04-25T02:15:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6844813700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840101e20 0x7f684010f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6844813700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840101e20 0x7f684010f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55322/0 (socket says 172.21.15.33:55322) 2024-04-25T02:15:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6844813700 1 -- 172.21.15.33:0/24624621 learned_addr learned my addr 172.21.15.33:0/24624621 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6845014700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68400fc710 0x7f684010ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6844813700 1 -- 172.21.15.33:0/24624621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68400fc710 msgr2=0x7f684010ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6844813700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68400fc710 0x7f684010ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.557+0000 7f6844813700 1 -- 172.21.15.33:0/24624621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683c005040 con 0x7f6840101e20 2024-04-25T02:15:12.564 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.558+0000 7f6844813700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840101e20 0x7f684010f200 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f683c00b3b0 tx=0x7f683c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.558+0000 7f6835ffb700 1 -- 172.21.15.33:0/24624621 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f683c00e040 con 0x7f6840101e20 2024-04-25T02:15:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.558+0000 7f6835ffb700 1 -- 172.21.15.33:0/24624621 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f683c013420 con 0x7f6840101e20 2024-04-25T02:15:12.565 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.558+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6840117b70 con 0x7f6840101e20 2024-04-25T02:15:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.558+0000 7f6835ffb700 1 -- 172.21.15.33:0/24624621 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f683c01c630 con 0x7f6840101e20 2024-04-25T02:15:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.558+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6840118100 con 0x7f6840101e20 2024-04-25T02:15:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.559+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6840104200 con 0x7f6840101e20 2024-04-25T02:15:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.559+0000 7f6835ffb700 1 -- 172.21.15.33:0/24624621 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f683c013590 con 0x7f6840101e20 2024-04-25T02:15:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.560+0000 7f6835ffb700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f682c06e140 0x7f682c0705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.560+0000 7f6835ffb700 1 -- 172.21.15.33:0/24624621 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f683c08ecf0 con 0x7f6840101e20 2024-04-25T02:15:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.560+0000 7f6845014700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f682c06e140 0x7f682c0705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:12.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.561+0000 7f6845014700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f682c06e140 0x7f682c0705f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6830004f40 tx=0x7f683000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:12.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.565+0000 7f6835ffb700 1 -- 172.21.15.33:0/24624621 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f683c0580f0 con 0x7f6840101e20 2024-04-25T02:15:12.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:12 smithi033 ceph-mon[111306]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:12.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:12 smithi033 ceph-mon[111306]: from='client.35948 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:12.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:12 smithi033 ceph-mon[111306]: pgmap v1075: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:12.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:12 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4175502285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:15:12.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:12 smithi138 ceph-mon[81304]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:12.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:12 smithi138 ceph-mon[81304]: from='client.35948 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:12.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:12 smithi138 ceph-mon[81304]: pgmap v1075: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:12.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:12 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4175502285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:15:12.797 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.791+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6840108eb0 con 0x7f682c06e140 2024-04-25T02:15:12.798 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.793+0000 7f6835ffb700 1 -- 172.21.15.33:0/24624621 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6840108eb0 con 0x7f682c06e140 2024-04-25T02:15:12.798 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:15:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:15:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:15:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:15:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:15:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:15:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:15:12.798 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:15:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:15:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:15:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:15:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:15:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:15:12.799 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:15:12.799 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:15:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f682c06e140 msgr2=0x7f682c0705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f682c06e140 0x7f682c0705f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6830004f40 tx=0x7f683000b040 comp rx=0 tx=0).stop 2024-04-25T02:15:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840101e20 msgr2=0x7f684010f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840101e20 0x7f684010f200 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f683c00b3b0 tx=0x7f683c009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 shutdown_connections 2024-04-25T02:15:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f682c06e140 0x7f682c0705f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f68400fc710 0x7f684010ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 --2- 172.21.15.33:0/24624621 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6840101e20 0x7f684010f200 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 >> 172.21.15.33:0/24624621 conn(0x7f6840071460 msgr2=0x7f68400f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 shutdown_connections 2024-04-25T02:15:12.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.795+0000 7f6847278700 1 -- 172.21.15.33:0/24624621 wait complete. 2024-04-25T02:15:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.899+0000 7f3fe29cc700 1 -- 172.21.15.33:0/3476567077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 msgr2=0x7f3fdc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.899+0000 7f3fe29cc700 1 --2- 172.21.15.33:0/3476567077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 0x7f3fdc1031b0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc004940 tx=0x7f3fcc009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.899+0000 7f3fe29cc700 1 -- 172.21.15.33:0/3476567077 shutdown_connections 2024-04-25T02:15:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.899+0000 7f3fe29cc700 1 --2- 172.21.15.33:0/3476567077 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fdc103780 0x7f3fdc0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.899+0000 7f3fe29cc700 1 --2- 172.21.15.33:0/3476567077 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 0x7f3fdc1031b0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.899+0000 7f3fe29cc700 1 -- 172.21.15.33:0/3476567077 >> 172.21.15.33:0/3476567077 conn(0x7f3fdc0f8990 msgr2=0x7f3fdc0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.899+0000 7f3fe29cc700 1 -- 172.21.15.33:0/3476567077 shutdown_connections 2024-04-25T02:15:12.905 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.899+0000 7f3fe29cc700 1 -- 172.21.15.33:0/3476567077 wait complete. 2024-04-25T02:15:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.900+0000 7f3fe29cc700 1 Processor -- start 2024-04-25T02:15:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.900+0000 7f3fe29cc700 1 -- start start 2024-04-25T02:15:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.900+0000 7f3fe29cc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 0x7f3fdc113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.900+0000 7f3fe29cc700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fdc103780 0x7f3fdc113e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.900+0000 7f3fe29cc700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc114550 con 0x7f3fdc102de0 2024-04-25T02:15:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.900+0000 7f3fe29cc700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc10dac0 con 0x7f3fdc103780 2024-04-25T02:15:12.907 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.900+0000 7f3fdbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 0x7f3fdc113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.900+0000 7f3fdbfff700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 0x7f3fdc113930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:55330/0 (socket says 172.21.15.33:55330) 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.900+0000 7f3fdbfff700 1 -- 172.21.15.33:0/1703333932 learned_addr learned my addr 172.21.15.33:0/1703333932 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.901+0000 7f3fdbfff700 1 -- 172.21.15.33:0/1703333932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fdc103780 msgr2=0x7f3fdc113e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.901+0000 7f3fdbfff700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fdc103780 0x7f3fdc113e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.901+0000 7f3fdbfff700 1 -- 172.21.15.33:0/1703333932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc005040 con 0x7f3fdc102de0 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.901+0000 7f3fdbfff700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 0x7f3fdc113930 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009180 tx=0x7f3fcc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.901+0000 7f3fd8ff9700 1 -- 172.21.15.33:0/1703333932 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fcc00e040 con 0x7f3fdc102de0 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.901+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fdc10dd40 con 0x7f3fdc102de0 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.901+0000 7f3fd8ff9700 1 -- 172.21.15.33:0/1703333932 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fcc00bbc0 con 0x7f3fdc102de0 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.901+0000 7f3fd8ff9700 1 -- 172.21.15.33:0/1703333932 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fcc01c630 con 0x7f3fdc102de0 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.901+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fdc10e300 con 0x7f3fdc102de0 2024-04-25T02:15:12.908 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.902+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fdc1000f0 con 0x7f3fdc102de0 2024-04-25T02:15:12.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.902+0000 7f3fd8ff9700 1 -- 172.21.15.33:0/1703333932 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3fcc00bd30 con 0x7f3fdc102de0 2024-04-25T02:15:12.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.903+0000 7f3fd8ff9700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fc806e140 0x7f3fc80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:12.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.903+0000 7f3fd8ff9700 1 -- 172.21.15.33:0/1703333932 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3fcc08ee10 con 0x7f3fdc102de0 2024-04-25T02:15:12.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.903+0000 7f3fdb7fe700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fc806e140 0x7f3fc80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:12.909 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.903+0000 7f3fdb7fe700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fc806e140 0x7f3fc80705f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4006fd0 tx=0x7f3fc4006db0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:12.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:12.908+0000 7f3fd8ff9700 1 -- 172.21.15.33:0/1703333932 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3fcc05bde0 con 0x7f3fdc102de0 2024-04-25T02:15:13.206 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.201+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3fdc109f50 con 0x7f3fdc102de0 2024-04-25T02:15:13.207 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.201+0000 7f3fd8ff9700 1 -- 172.21.15.33:0/1703333932 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f3fcc05b970 con 0x7f3fdc102de0 2024-04-25T02:15:13.207 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:15:13.207 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:15:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:15:13.207 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:15:13.207 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:15:13.207 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:15:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:15:13.207 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:15:13.208 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:15:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.204+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fc806e140 msgr2=0x7f3fc80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.204+0000 7f3fe29cc700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fc806e140 0x7f3fc80705f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4006fd0 tx=0x7f3fc4006db0 comp rx=0 tx=0).stop 2024-04-25T02:15:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.204+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 msgr2=0x7f3fdc113930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.204+0000 7f3fe29cc700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 0x7f3fdc113930 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009180 tx=0x7f3fcc009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.205+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 shutdown_connections 2024-04-25T02:15:13.210 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.205+0000 7f3fe29cc700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3fc806e140 0x7f3fc80705f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.205+0000 7f3fe29cc700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fdc102de0 0x7f3fdc113930 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.205+0000 7f3fe29cc700 1 --2- 172.21.15.33:0/1703333932 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fdc103780 0x7f3fdc113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.205+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 >> 172.21.15.33:0/1703333932 conn(0x7f3fdc0f8990 msgr2=0x7f3fdc0fa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.205+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 shutdown_connections 2024-04-25T02:15:13.211 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:13.205+0000 7f3fe29cc700 1 -- 172.21.15.33:0/1703333932 wait complete. 2024-04-25T02:15:13.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:13 smithi033 ceph-mon[111306]: from='client.35952 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:13.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:13 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2439244458' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:15:13.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:13 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1703333932' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:15:13.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:13 smithi138 ceph-mon[81304]: from='client.35952 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:13.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:13 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2439244458' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:15:13.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:13 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1703333932' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:15:14.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:14 smithi033 ceph-mon[111306]: from='client.35964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:14.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:14 smithi033 ceph-mon[111306]: pgmap v1076: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:14.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:14 smithi138 ceph-mon[81304]: from='client.35964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:14.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:14 smithi138 ceph-mon[81304]: pgmap v1076: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:16.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:16 smithi033 ceph-mon[111306]: pgmap v1077: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:16.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:16 smithi138 ceph-mon[81304]: pgmap v1077: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:18.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:18 smithi033 ceph-mon[111306]: pgmap v1078: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:18.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:18 smithi138 ceph-mon[81304]: pgmap v1078: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:20.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:20 smithi033 ceph-mon[111306]: pgmap v1079: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:20.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:20 smithi138 ceph-mon[81304]: pgmap v1079: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:22.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:22 smithi033 ceph-mon[111306]: pgmap v1080: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:22.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:22 smithi138 ceph-mon[81304]: pgmap v1080: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:24.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:24 smithi033 ceph-mon[111306]: pgmap v1081: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:24.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:24 smithi138 ceph-mon[81304]: pgmap v1081: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:25.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:15:25.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:15:26.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:26 smithi033 ceph-mon[111306]: pgmap v1082: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:26.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:26 smithi138 ceph-mon[81304]: pgmap v1082: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:28.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:28 smithi033 ceph-mon[111306]: pgmap v1083: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:28.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:28 smithi138 ceph-mon[81304]: pgmap v1083: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:30.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:30 smithi033 ceph-mon[111306]: pgmap v1084: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:30.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:30 smithi138 ceph-mon[81304]: pgmap v1084: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:32.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:32 smithi033 ceph-mon[111306]: pgmap v1085: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:32.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:32 smithi138 ceph-mon[81304]: pgmap v1085: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:34.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:34 smithi033 ceph-mon[111306]: pgmap v1086: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:34.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:34 smithi138 ceph-mon[81304]: pgmap v1086: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:36.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:36 smithi033 ceph-mon[111306]: pgmap v1087: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:36.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:36 smithi138 ceph-mon[81304]: pgmap v1087: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:38.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:38 smithi033 ceph-mon[111306]: pgmap v1088: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:38.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:38 smithi138 ceph-mon[81304]: pgmap v1088: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:40.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:40 smithi033 ceph-mon[111306]: pgmap v1089: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:40.832 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:15:40.832 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:40 smithi138 ceph-mon[81304]: pgmap v1089: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:40.833 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:15:41.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:41 smithi033 ceph-mon[111306]: pgmap v1090: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:41.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:41 smithi138 ceph-mon[81304]: pgmap v1090: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.323+0000 7ffa518f5700 1 -- 172.21.15.33:0/2984652008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 msgr2=0x7ffa4c0fb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.323+0000 7ffa518f5700 1 --2- 172.21.15.33:0/2984652008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 0x7ffa4c0fb990 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ffa40004940 tx=0x7ffa40009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.323+0000 7ffa518f5700 1 -- 172.21.15.33:0/2984652008 shutdown_connections 2024-04-25T02:15:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.323+0000 7ffa518f5700 1 --2- 172.21.15.33:0/2984652008 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 0x7ffa4c0fb990 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.323+0000 7ffa518f5700 1 --2- 172.21.15.33:0/2984652008 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa4c102f60 0x7ffa4c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.323+0000 7ffa518f5700 1 -- 172.21.15.33:0/2984652008 >> 172.21.15.33:0/2984652008 conn(0x7ffa4c0714d0 msgr2=0x7ffa4c0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.323+0000 7ffa518f5700 1 -- 172.21.15.33:0/2984652008 shutdown_connections 2024-04-25T02:15:43.329 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.323+0000 7ffa518f5700 1 -- 172.21.15.33:0/2984652008 wait complete. 2024-04-25T02:15:43.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.324+0000 7ffa518f5700 1 Processor -- start 2024-04-25T02:15:43.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.324+0000 7ffa518f5700 1 -- start start 2024-04-25T02:15:43.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.324+0000 7ffa518f5700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 0x7ffa4c113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:43.331 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.324+0000 7ffa518f5700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa4c102f60 0x7ffa4c113ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.324+0000 7ffa518f5700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1144f0 con 0x7ffa4c0fb520 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.324+0000 7ffa518f5700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c10daf0 con 0x7ffa4c102f60 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 0x7ffa4c113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 0x7ffa4c113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53030/0 (socket says 172.21.15.33:53030) 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa4affd700 1 -- 172.21.15.33:0/2597323345 learned_addr learned my addr 172.21.15.33:0/2597323345 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa4a7fc700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa4c102f60 0x7ffa4c113ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa4affd700 1 -- 172.21.15.33:0/2597323345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa4c102f60 msgr2=0x7ffa4c113ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa4affd700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa4c102f60 0x7ffa4c113ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa4affd700 1 -- 172.21.15.33:0/2597323345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa40005040 con 0x7ffa4c0fb520 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa4a7fc700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa4c102f60 0x7ffa4c113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa4affd700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 0x7ffa4c113960 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00cb10 tx=0x7ffa3c00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa337fe700 1 -- 172.21.15.33:0/2597323345 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3c00a980 con 0x7ffa4c0fb520 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa337fe700 1 -- 172.21.15.33:0/2597323345 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa3c012420 con 0x7ffa4c0fb520 2024-04-25T02:15:43.332 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c10ddd0 con 0x7ffa4c0fb520 2024-04-25T02:15:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa337fe700 1 -- 172.21.15.33:0/2597323345 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3c0115c0 con 0x7ffa4c0fb520 2024-04-25T02:15:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.325+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c10e310 con 0x7ffa4c0fb520 2024-04-25T02:15:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.326+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c100250 con 0x7ffa4c0fb520 2024-04-25T02:15:43.333 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.327+0000 7ffa337fe700 1 -- 172.21.15.33:0/2597323345 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffa3c003020 con 0x7ffa4c0fb520 2024-04-25T02:15:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.327+0000 7ffa337fe700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa3406e140 0x7ffa340705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.328+0000 7ffa337fe700 1 -- 172.21.15.33:0/2597323345 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ffa3c08dfe0 con 0x7ffa4c0fb520 2024-04-25T02:15:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.328+0000 7ffa4a7fc700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa3406e140 0x7ffa340705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:43.334 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.328+0000 7ffa4a7fc700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa3406e140 0x7ffa340705f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009fd0 tx=0x7ffa40009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:43.337 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.332+0000 7ffa337fe700 1 -- 172.21.15.33:0/2597323345 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ffa3c05ae30 con 0x7ffa4c0fb520 2024-04-25T02:15:43.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.561+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa4c10f010 con 0x7ffa3406e140 2024-04-25T02:15:43.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.562+0000 7ffa337fe700 1 -- 172.21.15.33:0/2597323345 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ffa4c10f010 con 0x7ffa3406e140 2024-04-25T02:15:43.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.564+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa3406e140 msgr2=0x7ffa340705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:43.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.564+0000 7ffa518f5700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa3406e140 0x7ffa340705f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009fd0 tx=0x7ffa40009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:43.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.564+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 msgr2=0x7ffa4c113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:43.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.564+0000 7ffa518f5700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 0x7ffa4c113960 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00cb10 tx=0x7ffa3c00ced0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.564+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 shutdown_connections 2024-04-25T02:15:43.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.564+0000 7ffa518f5700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ffa3406e140 0x7ffa340705f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.570 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.564+0000 7ffa518f5700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ffa4c0fb520 0x7ffa4c113960 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.564+0000 7ffa518f5700 1 --2- 172.21.15.33:0/2597323345 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ffa4c102f60 0x7ffa4c113ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.564+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 >> 172.21.15.33:0/2597323345 conn(0x7ffa4c0714d0 msgr2=0x7ffa4c0fa7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.565+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 shutdown_connections 2024-04-25T02:15:43.571 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.565+0000 7ffa518f5700 1 -- 172.21.15.33:0/2597323345 wait complete. 2024-04-25T02:15:43.581 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:15:43.675 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.669+0000 7f868f0d4700 1 -- 172.21.15.33:0/3515038279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 msgr2=0x7f86881031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:43.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.669+0000 7f868f0d4700 1 --2- 172.21.15.33:0/3515038279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 0x7f86881031b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f867c004940 tx=0x7f867c009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:43.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.670+0000 7f868f0d4700 1 -- 172.21.15.33:0/3515038279 shutdown_connections 2024-04-25T02:15:43.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.670+0000 7f868f0d4700 1 --2- 172.21.15.33:0/3515038279 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8688103780 0x7f86880fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.670+0000 7f868f0d4700 1 --2- 172.21.15.33:0/3515038279 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 0x7f86881031b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.670+0000 7f868f0d4700 1 -- 172.21.15.33:0/3515038279 >> 172.21.15.33:0/3515038279 conn(0x7f86880f8990 msgr2=0x7f86880fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:43.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.670+0000 7f868f0d4700 1 -- 172.21.15.33:0/3515038279 shutdown_connections 2024-04-25T02:15:43.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.670+0000 7f868f0d4700 1 -- 172.21.15.33:0/3515038279 wait complete. 2024-04-25T02:15:43.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.670+0000 7f868f0d4700 1 Processor -- start 2024-04-25T02:15:43.676 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.670+0000 7f868f0d4700 1 -- start start 2024-04-25T02:15:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868f0d4700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 0x7f86881138b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868f0d4700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8688103780 0x7f8688113df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868f0d4700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8688114440 con 0x7f8688102de0 2024-04-25T02:15:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868f0d4700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868810da90 con 0x7f8688103780 2024-04-25T02:15:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868ce70700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 0x7f86881138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868ce70700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 0x7f86881138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53054/0 (socket says 172.21.15.33:53054) 2024-04-25T02:15:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868ce70700 1 -- 172.21.15.33:0/664106760 learned_addr learned my addr 172.21.15.33:0/664106760 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:43.678 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868ce70700 1 -- 172.21.15.33:0/664106760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8688103780 msgr2=0x7f8688113df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f8687fff700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8688103780 0x7f8688113df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868ce70700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8688103780 0x7f8688113df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868ce70700 1 -- 172.21.15.33:0/664106760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c005040 con 0x7f8688102de0 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.671+0000 7f868ce70700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 0x7f86881138b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f867c009150 tx=0x7f867c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.672+0000 7f86857fa700 1 -- 172.21.15.33:0/664106760 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f867c021070 con 0x7f8688102de0 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.672+0000 7f86857fa700 1 -- 172.21.15.33:0/664106760 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f867c013c00 con 0x7f8688102de0 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.672+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868810dd10 con 0x7f8688102de0 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.672+0000 7f86857fa700 1 -- 172.21.15.33:0/664106760 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f867c00e070 con 0x7f8688102de0 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.672+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868810e1f0 con 0x7f8688102de0 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.672+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86881000f0 con 0x7f8688102de0 2024-04-25T02:15:43.679 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.673+0000 7f86857fa700 1 -- 172.21.15.33:0/664106760 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f867c01c400 con 0x7f8688102de0 2024-04-25T02:15:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.673+0000 7f86857fa700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f867806e140 0x7f86780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.674+0000 7f86857fa700 1 -- 172.21.15.33:0/664106760 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f867c08eff0 con 0x7f8688102de0 2024-04-25T02:15:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.674+0000 7f8687fff700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f867806e140 0x7f86780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:43.680 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.674+0000 7f8687fff700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f867806e140 0x7f86780705f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8674004600 tx=0x7f867400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:43.683 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.678+0000 7f86857fa700 1 -- 172.21.15.33:0/664106760 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f867c0c3740 con 0x7f8688102de0 2024-04-25T02:15:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.907+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f868810ef00 con 0x7f867806e140 2024-04-25T02:15:43.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.908+0000 7f86857fa700 1 -- 172.21.15.33:0/664106760 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f868810ef00 con 0x7f867806e140 2024-04-25T02:15:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.910+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f867806e140 msgr2=0x7f86780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:43.917 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.910+0000 7f868f0d4700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f867806e140 0x7f86780705f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f8674004600 tx=0x7f867400b040 comp rx=0 tx=0).stop 2024-04-25T02:15:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.910+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 msgr2=0x7f86881138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.910+0000 7f868f0d4700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 0x7f86881138b0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f867c009150 tx=0x7f867c009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.911+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 shutdown_connections 2024-04-25T02:15:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.911+0000 7f868f0d4700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f867806e140 0x7f86780705f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.911+0000 7f868f0d4700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8688102de0 0x7f86881138b0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.911+0000 7f868f0d4700 1 --2- 172.21.15.33:0/664106760 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8688103780 0x7f8688113df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.911+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 >> 172.21.15.33:0/664106760 conn(0x7f86880f8990 msgr2=0x7f86880fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.911+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 shutdown_connections 2024-04-25T02:15:43.918 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:43.911+0000 7f868f0d4700 1 -- 172.21.15.33:0/664106760 wait complete. 2024-04-25T02:15:44.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.015+0000 7fde9556a700 1 -- 172.21.15.33:0/2762069589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde900fbd40 msgr2=0x7fde900fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.015+0000 7fde9556a700 1 --2- 172.21.15.33:0/2762069589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde900fbd40 0x7fde900fc1b0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fde84004940 tx=0x7fde84009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:44.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.015+0000 7fde9556a700 1 -- 172.21.15.33:0/2762069589 shutdown_connections 2024-04-25T02:15:44.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.015+0000 7fde9556a700 1 --2- 172.21.15.33:0/2762069589 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde900fbd40 0x7fde900fc1b0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.015+0000 7fde9556a700 1 --2- 172.21.15.33:0/2762069589 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde90101d90 0x7fde90102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.023 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.015+0000 7fde9556a700 1 -- 172.21.15.33:0/2762069589 >> 172.21.15.33:0/2762069589 conn(0x7fde90071d00 msgr2=0x7fde90072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:44.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.015+0000 7fde9556a700 1 -- 172.21.15.33:0/2762069589 shutdown_connections 2024-04-25T02:15:44.024 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.015+0000 7fde9556a700 1 -- 172.21.15.33:0/2762069589 wait complete. 2024-04-25T02:15:44.025 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.015+0000 7fde9556a700 1 Processor -- start 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde9556a700 1 -- start start 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde9556a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde900fbd40 0x7fde90115e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde9556a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde90101d90 0x7fde90116340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde9556a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde90116990 con 0x7fde90101d90 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde9556a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9010fd60 con 0x7fde900fbd40 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde8e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde90101d90 0x7fde90116340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde8e7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde90101d90 0x7fde90116340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53080/0 (socket says 172.21.15.33:53080) 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde8e7fc700 1 -- 172.21.15.33:0/2502666251 learned_addr learned my addr 172.21.15.33:0/2502666251 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde8effd700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde900fbd40 0x7fde90115e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde8e7fc700 1 -- 172.21.15.33:0/2502666251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde900fbd40 msgr2=0x7fde90115e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde8e7fc700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde900fbd40 0x7fde90115e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde8e7fc700 1 -- 172.21.15.33:0/2502666251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84005040 con 0x7fde90101d90 2024-04-25T02:15:44.026 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.016+0000 7fde8e7fc700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde90101d90 0x7fde90116340 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fde8400b3b0 tx=0x7fde84009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.017+0000 7fde777fe700 1 -- 172.21.15.33:0/2502666251 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde8400e040 con 0x7fde90101d90 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.017+0000 7fde777fe700 1 -- 172.21.15.33:0/2502666251 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde84013420 con 0x7fde90101d90 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.017+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9010ffe0 con 0x7fde90101d90 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.017+0000 7fde777fe700 1 -- 172.21.15.33:0/2502666251 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde8401c630 con 0x7fde90101d90 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.017+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde901105a0 con 0x7fde90101d90 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.017+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde900fd430 con 0x7fde90101d90 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.018+0000 7fde777fe700 1 -- 172.21.15.33:0/2502666251 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fde84013590 con 0x7fde90101d90 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.018+0000 7fde777fe700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fde7806e140 0x7fde780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.019+0000 7fde777fe700 1 -- 172.21.15.33:0/2502666251 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fde8408ed80 con 0x7fde90101d90 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.019+0000 7fde8effd700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fde7806e140 0x7fde780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:44.027 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.019+0000 7fde8effd700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fde7806e140 0x7fde780705f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fde80004f60 tx=0x7fde8000b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:44.028 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.023+0000 7fde777fe700 1 -- 172.21.15.33:0/2502666251 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fde8405bcb0 con 0x7fde90101d90 2024-04-25T02:15:44.247 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.241+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fde90108e80 con 0x7fde7806e140 2024-04-25T02:15:44.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.251+0000 7fde777fe700 1 -- 172.21.15.33:0/2502666251 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fde90108e80 con 0x7fde7806e140 2024-04-25T02:15:44.256 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (36m) 14m ago 48m 25.3M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (34m) 14m ago 48m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (34m) 14m ago 47m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (36m) 14m ago 47m 80.8M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (30m) 14m ago 42m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (30m) 14m ago 42m 19.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (29m) 14m ago 42m 20.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (14m) 14m ago 42m 14.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (38m) 14m ago 49m 513M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (37m) 14m ago 47m 457M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (35m) 14m ago 49m 70.2M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (34m) 14m ago 47m 52.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:15:44.257 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (37m) 14m ago 48m 15.4M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:15:44.258 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (37m) 14m ago 47m 15.3M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:15:44.258 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (33m) 14m ago 46m 98.5M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:15:44.258 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (33m) 14m ago 45m 102M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:15:44.258 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (32m) 14m ago 45m 85.3M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:15:44.258 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (31m) 14m ago 45m 105M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:15:44.258 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (31m) 14m ago 44m 104M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:15:44.258 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (30m) 14m ago 44m 96.7M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:15:44.258 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (37m) 14m ago 47m 66.0M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fde7806e140 msgr2=0x7fde780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fde7806e140 0x7fde780705f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fde80004f60 tx=0x7fde8000b040 comp rx=0 tx=0).stop 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde90101d90 msgr2=0x7fde90116340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde90101d90 0x7fde90116340 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fde8400b3b0 tx=0x7fde84009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 shutdown_connections 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fde7806e140 0x7fde780705f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fde900fbd40 0x7fde90115e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 --2- 172.21.15.33:0/2502666251 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fde90101d90 0x7fde90116340 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 >> 172.21.15.33:0/2502666251 conn(0x7fde90071d00 msgr2=0x7fde900655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 shutdown_connections 2024-04-25T02:15:44.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.253+0000 7fde9556a700 1 -- 172.21.15.33:0/2502666251 wait complete. 2024-04-25T02:15:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 -- 172.21.15.33:0/1024874033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 msgr2=0x7f3edc106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 --2- 172.21.15.33:0/1024874033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 0x7f3edc106e50 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0004940 tx=0x7f3ed0009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 -- 172.21.15.33:0/1024874033 shutdown_connections 2024-04-25T02:15:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 --2- 172.21.15.33:0/1024874033 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 0x7f3edc106e50 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 --2- 172.21.15.33:0/1024874033 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc0fde70 0x7f3edc0fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 -- 172.21.15.33:0/1024874033 >> 172.21.15.33:0/1024874033 conn(0x7f3edc0f7860 msgr2=0x7f3edc0f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:44.361 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 -- 172.21.15.33:0/1024874033 shutdown_connections 2024-04-25T02:15:44.362 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 -- 172.21.15.33:0/1024874033 wait complete. 2024-04-25T02:15:44.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 Processor -- start 2024-04-25T02:15:44.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.356+0000 7f3ee1188700 1 -- start start 2024-04-25T02:15:44.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3ee1188700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc0fde70 0x7f3edc113950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:44.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3ee1188700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 0x7f3edc113e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:44.363 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3ee1188700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc1144e0 con 0x7f3edc0fe810 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3ee1188700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc10dae0 con 0x7f3edc0fde70 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3eda59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 0x7f3edc113e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3eda59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 0x7f3edc113e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53096/0 (socket says 172.21.15.33:53096) 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3eda59c700 1 -- 172.21.15.33:0/1342286517 learned_addr learned my addr 172.21.15.33:0/1342286517 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3edad9d700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc0fde70 0x7f3edc113950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3eda59c700 1 -- 172.21.15.33:0/1342286517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc0fde70 msgr2=0x7f3edc113950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3eda59c700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc0fde70 0x7f3edc113950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3eda59c700 1 -- 172.21.15.33:0/1342286517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed0005040 con 0x7f3edc0fe810 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3eda59c700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 0x7f3edc113e90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009180 tx=0x7f3ed0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3ec37fe700 1 -- 172.21.15.33:0/1342286517 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ed000e040 con 0x7f3edc0fe810 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc10ddc0 con 0x7f3edc0fe810 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.357+0000 7f3ec37fe700 1 -- 172.21.15.33:0/1342286517 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ed0013420 con 0x7f3edc0fe810 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.358+0000 7f3ec37fe700 1 -- 172.21.15.33:0/1342286517 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ed001c630 con 0x7f3edc0fe810 2024-04-25T02:15:44.364 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.358+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc10e300 con 0x7f3edc0fe810 2024-04-25T02:15:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.359+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc066af0 con 0x7f3edc0fe810 2024-04-25T02:15:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.359+0000 7f3ec37fe700 1 -- 172.21.15.33:0/1342286517 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3ed0013590 con 0x7f3edc0fe810 2024-04-25T02:15:44.365 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.360+0000 7f3ec37fe700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ec406e140 0x7f3ec40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.360+0000 7f3ec37fe700 1 -- 172.21.15.33:0/1342286517 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3ed008eeb0 con 0x7f3edc0fe810 2024-04-25T02:15:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.360+0000 7f3edad9d700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ec406e140 0x7f3ec40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:44.366 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.360+0000 7f3edad9d700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ec406e140 0x7f3ec40705f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc007fd0 tx=0x7f3ecc009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:44.370 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.364+0000 7f3ec37fe700 1 -- 172.21.15.33:0/1342286517 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3ed005bde0 con 0x7f3edc0fe810 2024-04-25T02:15:44.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:44 smithi033 ceph-mon[111306]: pgmap v1091: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:44.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.663+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3edc10df50 con 0x7f3edc0fe810 2024-04-25T02:15:44.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.663+0000 7f3ec37fe700 1 -- 172.21.15.33:0/1342286517 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3edc10df50 con 0x7f3edc0fe810 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:15:44.669 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:15:44.670 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:15:44.670 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:15:44.670 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:15:44.670 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ec406e140 msgr2=0x7f3ec40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ec406e140 0x7f3ec40705f0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc007fd0 tx=0x7f3ecc009040 comp rx=0 tx=0).stop 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 msgr2=0x7f3edc113e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 0x7f3edc113e90 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0009180 tx=0x7f3ed0009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 shutdown_connections 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3ec406e140 0x7f3ec40705f0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3edc0fde70 0x7f3edc113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 --2- 172.21.15.33:0/1342286517 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3edc0fe810 0x7f3edc113e90 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 >> 172.21.15.33:0/1342286517 conn(0x7f3edc0f7860 msgr2=0x7f3edc1016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 shutdown_connections 2024-04-25T02:15:44.671 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.665+0000 7f3ee1188700 1 -- 172.21.15.33:0/1342286517 wait complete. 2024-04-25T02:15:44.774 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.769+0000 7fab79b78700 1 -- 172.21.15.33:0/69658081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 msgr2=0x7fab741031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.769+0000 7fab79b78700 1 --2- 172.21.15.33:0/69658081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 0x7fab741031b0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fab6c009ce0 tx=0x7fab6c004d10 comp rx=0 tx=0).stop 2024-04-25T02:15:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.769+0000 7fab79b78700 1 -- 172.21.15.33:0/69658081 shutdown_connections 2024-04-25T02:15:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.769+0000 7fab79b78700 1 --2- 172.21.15.33:0/69658081 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab74103780 0x7fab740fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.769+0000 7fab79b78700 1 --2- 172.21.15.33:0/69658081 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 0x7fab741031b0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.769+0000 7fab79b78700 1 -- 172.21.15.33:0/69658081 >> 172.21.15.33:0/69658081 conn(0x7fab740f8950 msgr2=0x7fab740fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.769+0000 7fab79b78700 1 -- 172.21.15.33:0/69658081 shutdown_connections 2024-04-25T02:15:44.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.769+0000 7fab79b78700 1 -- 172.21.15.33:0/69658081 wait complete. 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab79b78700 1 Processor -- start 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab79b78700 1 -- start start 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab79b78700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 0x7fab74070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab79b78700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab74103780 0x7fab740707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab79b78700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab74070d30 con 0x7fab74102de0 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab79b78700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab74070e70 con 0x7fab74103780 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 0x7fab74070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab737fe700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 0x7fab74070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53108/0 (socket says 172.21.15.33:53108) 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab737fe700 1 -- 172.21.15.33:0/1010770639 learned_addr learned my addr 172.21.15.33:0/1010770639 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab72ffd700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab74103780 0x7fab740707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab737fe700 1 -- 172.21.15.33:0/1010770639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab74103780 msgr2=0x7fab740707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:44.777 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab737fe700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab74103780 0x7fab740707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.770+0000 7fab737fe700 1 -- 172.21.15.33:0/1010770639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c009940 con 0x7fab74102de0 2024-04-25T02:15:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.771+0000 7fab737fe700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 0x7fab74070260 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00be90 tx=0x7fab6c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.771+0000 7fab6bfff700 1 -- 172.21.15.33:0/1010770639 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6c00e040 con 0x7fab74102de0 2024-04-25T02:15:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.771+0000 7fab6bfff700 1 -- 172.21.15.33:0/1010770639 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab6c008ea0 con 0x7fab74102de0 2024-04-25T02:15:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.771+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab740710c0 con 0x7fab74102de0 2024-04-25T02:15:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.771+0000 7fab6bfff700 1 -- 172.21.15.33:0/1010770639 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6c01c810 con 0x7fab74102de0 2024-04-25T02:15:44.778 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.771+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7411a3f0 con 0x7fab74102de0 2024-04-25T02:15:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.772+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab741000f0 con 0x7fab74102de0 2024-04-25T02:15:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.772+0000 7fab6bfff700 1 -- 172.21.15.33:0/1010770639 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fab6c012c00 con 0x7fab74102de0 2024-04-25T02:15:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.773+0000 7fab6bfff700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fab6006e140 0x7fab600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.773+0000 7fab6bfff700 1 -- 172.21.15.33:0/1010770639 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fab6c08f7c0 con 0x7fab74102de0 2024-04-25T02:15:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.773+0000 7fab72ffd700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fab6006e140 0x7fab600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:44.779 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.773+0000 7fab72ffd700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fab6006e140 0x7fab600705f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab5c007fd0 tx=0x7fab5c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:44.782 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:44.777+0000 7fab6bfff700 1 -- 172.21.15.33:0/1010770639 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fab6c058aa0 con 0x7fab74102de0 2024-04-25T02:15:44.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:44 smithi138 ceph-mon[81304]: pgmap v1091: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:45.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.032+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab74109ee0 con 0x7fab74102de0 2024-04-25T02:15:45.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.033+0000 7fab6bfff700 1 -- 172.21.15.33:0/1010770639 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1873 (secure 0 0 0) 0x7fab6c004e10 con 0x7fab74102de0 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:e30 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 21 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T01:45:10.323488+0000 2024-04-25T02:15:45.039 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:up {} 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:failed 0,1 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:15:45.040 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 0 members: 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:Standby daemons: 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{-1:34414} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{-1:44327} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:15:45.041 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{-1:44841} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:15:45.046 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fab6006e140 msgr2=0x7fab600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fab6006e140 0x7fab600705f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fab5c007fd0 tx=0x7fab5c009040 comp rx=0 tx=0).stop 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 msgr2=0x7fab74070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 0x7fab74070260 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00be90 tx=0x7fab6c004d10 comp rx=0 tx=0).stop 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 shutdown_connections 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fab6006e140 0x7fab600705f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fab74102de0 0x7fab74070260 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 --2- 172.21.15.33:0/1010770639 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fab74103780 0x7fab740707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 >> 172.21.15.33:0/1010770639 conn(0x7fab740f8950 msgr2=0x7fab741073d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 shutdown_connections 2024-04-25T02:15:45.047 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.035+0000 7fab79b78700 1 -- 172.21.15.33:0/1010770639 wait complete. 2024-04-25T02:15:45.048 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:15:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.140+0000 7f745b582700 1 -- 172.21.15.33:0/1506111550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 msgr2=0x7f7454065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.140+0000 7f745b582700 1 --2- 172.21.15.33:0/1506111550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 0x7f7454065ab0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7450004940 tx=0x7f7450009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.140+0000 7f745b582700 1 -- 172.21.15.33:0/1506111550 shutdown_connections 2024-04-25T02:15:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.140+0000 7f745b582700 1 --2- 172.21.15.33:0/1506111550 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 0x7f7454065ab0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.140+0000 7f745b582700 1 --2- 172.21.15.33:0/1506111550 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7454102e50 0x7f7454103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.140+0000 7f745b582700 1 -- 172.21.15.33:0/1506111550 >> 172.21.15.33:0/1506111550 conn(0x7f7454071d00 msgr2=0x7f7454072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.140+0000 7f745b582700 1 -- 172.21.15.33:0/1506111550 shutdown_connections 2024-04-25T02:15:45.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.140+0000 7f745b582700 1 -- 172.21.15.33:0/1506111550 wait complete. 2024-04-25T02:15:45.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745b582700 1 Processor -- start 2024-04-25T02:15:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745b582700 1 -- start start 2024-04-25T02:15:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745b582700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 0x7f74540fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745b582700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7454102e50 0x7f74540fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745b582700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540fc880 con 0x7f7454065640 2024-04-25T02:15:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745b582700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74540fc9c0 con 0x7f7454102e50 2024-04-25T02:15:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745931e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 0x7f74540fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:45.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745931e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 0x7f74540fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53134/0 (socket says 172.21.15.33:53134) 2024-04-25T02:15:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745931e700 1 -- 172.21.15.33:0/3195885710 learned_addr learned my addr 172.21.15.33:0/3195885710 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f7458b1d700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7454102e50 0x7f74540fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745931e700 1 -- 172.21.15.33:0/3195885710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7454102e50 msgr2=0x7f74540fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745931e700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7454102e50 0x7f74540fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.141+0000 7f745931e700 1 -- 172.21.15.33:0/3195885710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7450005040 con 0x7f7454065640 2024-04-25T02:15:45.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.142+0000 7f745931e700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 0x7f74540fbdb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f744800c8c0 tx=0x7f74480044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.142+0000 7f7445ffb700 1 -- 172.21.15.33:0/3195885710 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f744800a760 con 0x7f7454065640 2024-04-25T02:15:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.142+0000 7f7445ffb700 1 -- 172.21.15.33:0/3195885710 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f744800ad90 con 0x7f7454065640 2024-04-25T02:15:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.142+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7454119e70 con 0x7f7454065640 2024-04-25T02:15:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.142+0000 7f7445ffb700 1 -- 172.21.15.33:0/3195885710 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74480113a0 con 0x7f7454065640 2024-04-25T02:15:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.142+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f745411a2f0 con 0x7f7454065640 2024-04-25T02:15:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.143+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7454100100 con 0x7f7454065640 2024-04-25T02:15:45.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.143+0000 7f7445ffb700 1 -- 172.21.15.33:0/3195885710 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7448012400 con 0x7f7454065640 2024-04-25T02:15:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.144+0000 7f7445ffb700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f744006e150 0x7f7440070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.144+0000 7f7445ffb700 1 -- 172.21.15.33:0/3195885710 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f744808da70 con 0x7f7454065640 2024-04-25T02:15:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.144+0000 7f7458b1d700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f744006e150 0x7f7440070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:45.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.144+0000 7f7458b1d700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f744006e150 0x7f7440070600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f74540fd6f0 tx=0x7f7450009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:45.156 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.148+0000 7f7445ffb700 1 -- 172.21.15.33:0/3195885710 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7448056f10 con 0x7f7454065640 2024-04-25T02:15:45.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.376+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f745404bed0 con 0x7f744006e150 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.377+0000 7f7445ffb700 1 -- 172.21.15.33:0/3195885710 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f745404bed0 con 0x7f744006e150 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901", 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": true, 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [ 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stdout: "mds", 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr", 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stdout: "crash", 2024-04-25T02:15:45.383 INFO:teuthology.orchestra.run.smithi033.stdout: "osd", 2024-04-25T02:15:45.384 INFO:teuthology.orchestra.run.smithi033.stdout: "mon" 2024-04-25T02:15:45.384 INFO:teuthology.orchestra.run.smithi033.stdout: ], 2024-04-25T02:15:45.384 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T02:15:45.384 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T02:15:45.384 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:15:45.384 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:15:45.386 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f744006e150 msgr2=0x7f7440070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f744006e150 0x7f7440070600 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f74540fd6f0 tx=0x7f7450009e30 comp rx=0 tx=0).stop 2024-04-25T02:15:45.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 msgr2=0x7f74540fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.387 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 0x7f74540fbdb0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f744800c8c0 tx=0x7f74480044c0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 shutdown_connections 2024-04-25T02:15:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f744006e150 0x7f7440070600 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7454065640 0x7f74540fbdb0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 --2- 172.21.15.33:0/3195885710 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7454102e50 0x7f74540fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 >> 172.21.15.33:0/3195885710 conn(0x7f7454071d00 msgr2=0x7f74540fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 shutdown_connections 2024-04-25T02:15:45.389 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.379+0000 7f745b582700 1 -- 172.21.15.33:0/3195885710 wait complete. 2024-04-25T02:15:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.484+0000 7f729161e700 1 -- 172.21.15.33:0/2245611417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c101d90 msgr2=0x7f728c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.484+0000 7f729161e700 1 --2- 172.21.15.33:0/2245611417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c101d90 0x7f728c102160 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f7284009ce0 tx=0x7f7284004d10 comp rx=0 tx=0).stop 2024-04-25T02:15:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.484+0000 7f729161e700 1 -- 172.21.15.33:0/2245611417 shutdown_connections 2024-04-25T02:15:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.484+0000 7f729161e700 1 --2- 172.21.15.33:0/2245611417 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c0fbd40 0x7f728c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.484+0000 7f729161e700 1 --2- 172.21.15.33:0/2245611417 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c101d90 0x7f728c102160 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.490 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.484+0000 7f729161e700 1 -- 172.21.15.33:0/2245611417 >> 172.21.15.33:0/2245611417 conn(0x7f728c071d00 msgr2=0x7f728c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.484+0000 7f729161e700 1 -- 172.21.15.33:0/2245611417 shutdown_connections 2024-04-25T02:15:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.484+0000 7f729161e700 1 -- 172.21.15.33:0/2245611417 wait complete. 2024-04-25T02:15:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.484+0000 7f729161e700 1 Processor -- start 2024-04-25T02:15:45.491 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f729161e700 1 -- start start 2024-04-25T02:15:45.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f729161e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c0fbd40 0x7f728c113cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:45.493 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f729161e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c101d90 0x7f728c114200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f729161e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c114850 con 0x7f728c0fbd40 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f729161e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c10dc20 con 0x7f728c101d90 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f728affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c0fbd40 0x7f728c113cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f728affd700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c0fbd40 0x7f728c113cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:53162/0 (socket says 172.21.15.33:53162) 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f728affd700 1 -- 172.21.15.33:0/2003710235 learned_addr learned my addr 172.21.15.33:0/2003710235 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f728a7fc700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c101d90 0x7f728c114200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f728affd700 1 -- 172.21.15.33:0/2003710235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c101d90 msgr2=0x7f728c114200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f728affd700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c101d90 0x7f728c114200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f728affd700 1 -- 172.21.15.33:0/2003710235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7284009940 con 0x7f728c0fbd40 2024-04-25T02:15:45.494 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f728a7fc700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c101d90 0x7f728c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.485+0000 7f728affd700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c0fbd40 0x7f728c113cc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7284003400 tx=0x7f7284004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.486+0000 7f72837fe700 1 -- 172.21.15.33:0/2003710235 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728400e040 con 0x7f728c0fbd40 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.486+0000 7f72837fe700 1 -- 172.21.15.33:0/2003710235 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7284008e20 con 0x7f728c0fbd40 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.486+0000 7f72837fe700 1 -- 172.21.15.33:0/2003710235 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f728401c760 con 0x7f728c0fbd40 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.486+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c10dea0 con 0x7f728c0fbd40 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.486+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c10e460 con 0x7f728c0fbd40 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.487+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c0feff0 con 0x7f728c0fbd40 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.487+0000 7f72837fe700 1 -- 172.21.15.33:0/2003710235 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7284012b70 con 0x7f728c0fbd40 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.488+0000 7f72837fe700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f727806e140 0x7f72780705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.488+0000 7f72837fe700 1 -- 172.21.15.33:0/2003710235 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f728408f580 con 0x7f728c0fbd40 2024-04-25T02:15:45.495 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.488+0000 7f728a7fc700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f727806e140 0x7f72780705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:15:45.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.489+0000 7f728a7fc700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f727806e140 0x7f72780705f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7274009fd0 tx=0x7f727400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:15:45.498 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.493+0000 7f72837fe700 1 -- 172.21.15.33:0/2003710235 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f728405c4b0 con 0x7f728c0fbd40 2024-04-25T02:15:45.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:45 smithi033 ceph-mon[111306]: from='client.35972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:45.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:45 smithi033 ceph-mon[111306]: from='client.35976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:45.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:45 smithi033 ceph-mon[111306]: from='client.35980 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:45.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:45 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1342286517' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:15:45.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:45 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1010770639' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:15:45.795 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.788+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f728c0fd430 con 0x7f728c0fbd40 2024-04-25T02:15:45.796 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.789+0000 7f72837fe700 1 -- 172.21.15.33:0/2003710235 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f728c0fd430 con 0x7f728c0fbd40 2024-04-25T02:15:45.796 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-04-25T02:15:45.796 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-04-25T02:15:45.796 INFO:teuthology.orchestra.run.smithi033.stdout: host smithi138 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi138/config 2024-04-25T02:15:45.797 INFO:teuthology.orchestra.run.smithi033.stdout:/usr/bin/podman: timeout after 895 seconds 2024-04-25T02:15:45.797 INFO:teuthology.orchestra.run.smithi033.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:15:45.797 INFO:teuthology.orchestra.run.smithi033.stdout:Traceback (most recent call last): 2024-04-25T02:15:45.797 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-04-25T02:15:45.797 INFO:teuthology.orchestra.run.smithi033.stdout: return _run_code(code, main_globals, None, 2024-04-25T02:15:45.798 INFO:teuthology.orchestra.run.smithi033.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-04-25T02:15:45.798 INFO:teuthology.orchestra.run.smithi033.stdout: exec(code, run_globals) 2024-04-25T02:15:45.798 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5608, in 2024-04-25T02:15:45.798 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 5596, in main 2024-04-25T02:15:45.798 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 408, in _infer_config 2024-04-25T02:15:45.798 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 323, in _infer_fsid 2024-04-25T02:15:45.799 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 436, in _infer_image 2024-04-25T02:15:45.799 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 310, in _validate_fsid 2024-04-25T02:15:45.799 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/__main__.py", line 3359, in command_ceph_volume 2024-04-25T02:15:45.799 INFO:teuthology.orchestra.run.smithi033.stdout: File "/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/cephadm.e208d129b1b64c3c550e45caa8541e217cdb2262523be914cc4c710e4ee35bb9/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-04-25T02:15:45.799 INFO:teuthology.orchestra.run.smithi033.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 -e NODE_NAME=smithi138 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/run/ceph:z -v /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp41bihv2q:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 inventory --format=json-pretty --filter-for-batch 2024-04-25T02:15:45.799 INFO:teuthology.orchestra.run.smithi033.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T02:15:45.800 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is degraded 2024-04-25T02:15:45.800 INFO:teuthology.orchestra.run.smithi033.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T02:15:45.800 INFO:teuthology.orchestra.run.smithi033.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T02:15:45.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f727806e140 msgr2=0x7f72780705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f727806e140 0x7f72780705f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7274009fd0 tx=0x7f727400b040 comp rx=0 tx=0).stop 2024-04-25T02:15:45.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c0fbd40 msgr2=0x7f728c113cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:15:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c0fbd40 0x7f728c113cc0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7284003400 tx=0x7f7284004d10 comp rx=0 tx=0).stop 2024-04-25T02:15:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 shutdown_connections 2024-04-25T02:15:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f727806e140 0x7f72780705f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f728c0fbd40 0x7f728c113cc0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 --2- 172.21.15.33:0/2003710235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f728c101d90 0x7f728c114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:15:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 >> 172.21.15.33:0/2003710235 conn(0x7f728c071d00 msgr2=0x7f728c0fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:15:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 shutdown_connections 2024-04-25T02:15:45.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:15:45.791+0000 7f729161e700 1 -- 172.21.15.33:0/2003710235 wait complete. 2024-04-25T02:15:45.804 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:45 smithi138 ceph-mon[81304]: from='client.35972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:45.804 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:45 smithi138 ceph-mon[81304]: from='client.35976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:45.804 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:45 smithi138 ceph-mon[81304]: from='client.35980 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:45.804 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:45 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1342286517' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:15:45.805 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:45 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1010770639' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:15:46.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:46 smithi033 ceph-mon[111306]: pgmap v1092: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:46.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:46 smithi033 ceph-mon[111306]: from='client.35992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:46.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:46 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2003710235' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:15:46.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:46 smithi138 ceph-mon[81304]: pgmap v1092: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:46.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:46 smithi138 ceph-mon[81304]: from='client.35992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:15:46.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:46 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2003710235' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:15:48.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:48 smithi033 ceph-mon[111306]: pgmap v1093: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:48.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:48 smithi138 ceph-mon[81304]: pgmap v1093: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:50.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:50 smithi033 ceph-mon[111306]: pgmap v1094: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:50.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:50 smithi138 ceph-mon[81304]: pgmap v1094: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:52.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:52 smithi033 ceph-mon[111306]: pgmap v1095: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:52.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:52 smithi138 ceph-mon[81304]: pgmap v1095: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:54.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:54 smithi033 ceph-mon[111306]: pgmap v1096: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:54.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:54 smithi138 ceph-mon[81304]: pgmap v1096: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:55.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:15:55.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:15:56.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:56 smithi033 ceph-mon[111306]: pgmap v1097: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:56.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:56 smithi138 ceph-mon[81304]: pgmap v1097: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:58.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:15:58 smithi033 ceph-mon[111306]: pgmap v1098: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:15:58.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:15:58 smithi138 ceph-mon[81304]: pgmap v1098: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:00.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:00 smithi033 ceph-mon[111306]: pgmap v1099: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:00.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:00 smithi138 ceph-mon[81304]: pgmap v1099: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:02.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:02 smithi033 ceph-mon[111306]: pgmap v1100: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:02.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:02 smithi138 ceph-mon[81304]: pgmap v1100: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:04.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:04 smithi033 ceph-mon[111306]: pgmap v1101: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:04.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:04 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:16:04.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:04 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:16:04.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:04 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:04.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:04 smithi138 ceph-mon[81304]: pgmap v1101: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:04.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:04 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:16:04.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:04 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:16:04.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:04 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:05.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: pgmap v1102: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:05.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: pgmap v1103: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.hobkqr"}]': finished 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.qqhhcz"}]': finished 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.jwzxlz"}]': finished 2024-04-25T02:16:05.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T02:16:05.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.rvxnby"}]': finished 2024-04-25T02:16:05.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:05 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-25T02:16:05.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: pgmap v1102: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:05.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: pgmap v1103: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail 2024-04-25T02:16:05.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:16:05.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.hobkqr"}]: dispatch 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.hobkqr"}]': finished 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.qqhhcz"}]: dispatch 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi033.qqhhcz"}]': finished 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.jwzxlz"}]: dispatch 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.jwzxlz"}]': finished 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.rvxnby"}]: dispatch 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi138.rvxnby"}]': finished 2024-04-25T02:16:05.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:05 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-25T02:16:06.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:06 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all mds 2024-04-25T02:16:06.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:06 smithi033 ceph-mon[111306]: Upgrade: Setting filesystem cephfs Joinable 2024-04-25T02:16:06.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:06 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-25T02:16:06.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:06 smithi033 ceph-mon[111306]: Standby daemon mds.cephfs.smithi033.qqhhcz assigned to filesystem cephfs as rank 0 2024-04-25T02:16:06.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:06 smithi033 ceph-mon[111306]: Standby daemon mds.cephfs.smithi033.hobkqr assigned to filesystem cephfs as rank 1 2024-04-25T02:16:06.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:06 smithi033 ceph-mon[111306]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T02:16:06.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:06 smithi033 ceph-mon[111306]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T02:16:06.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:06 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-25T02:16:06.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:06 smithi033 ceph-mon[111306]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:replay,1=cephfs.smithi033.hobkqr=up:replay} 2 up:standby 2024-04-25T02:16:06.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:06 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all mds 2024-04-25T02:16:06.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:06 smithi138 ceph-mon[81304]: Upgrade: Setting filesystem cephfs Joinable 2024-04-25T02:16:06.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:06 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-25T02:16:06.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:06 smithi138 ceph-mon[81304]: Standby daemon mds.cephfs.smithi033.qqhhcz assigned to filesystem cephfs as rank 0 2024-04-25T02:16:06.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:06 smithi138 ceph-mon[81304]: Standby daemon mds.cephfs.smithi033.hobkqr assigned to filesystem cephfs as rank 1 2024-04-25T02:16:06.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:06 smithi138 ceph-mon[81304]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T02:16:06.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:06 smithi138 ceph-mon[81304]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-25T02:16:06.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:06 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-04-25T02:16:06.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:06 smithi138 ceph-mon[81304]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:replay,1=cephfs.smithi033.hobkqr=up:replay} 2 up:standby 2024-04-25T02:16:07.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: pgmap v1104: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 0 op/s 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:replay,1=cephfs.smithi033.hobkqr=up:replay} 2 up:standby 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.772 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-25T02:16:07.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-25T02:16:07.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-25T02:16:07.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-25T02:16:07.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-25T02:16:07.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-25T02:16:07.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-25T02:16:07.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-25T02:16:07.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-25T02:16:07.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.776 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.776 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.776 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-25T02:16:07.776 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-25T02:16:07.776 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:16:07.776 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.776 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.776 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.776 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:07 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: pgmap v1104: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 0 op/s 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:replay,1=cephfs.smithi033.hobkqr=up:replay} 2 up:standby 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-25T02:16:07.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-25T02:16:07.799 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-25T02:16:07.800 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:16:07.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:07.801 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:07 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:08.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:08 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all rgw 2024-04-25T02:16:08.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:08 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all rbd-mirror 2024-04-25T02:16:08.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:08 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all ceph-exporter 2024-04-25T02:16:08.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:08 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all iscsi 2024-04-25T02:16:08.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:08 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all nfs 2024-04-25T02:16:08.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:08 smithi033 ceph-mon[111306]: Upgrade: Setting container_image for all nvmeof 2024-04-25T02:16:08.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:08 smithi033 ceph-mon[111306]: Upgrade: Finalizing container_image settings 2024-04-25T02:16:08.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:08 smithi033 ceph-mon[111306]: Upgrade: Complete! 2024-04-25T02:16:08.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:08 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all rgw 2024-04-25T02:16:08.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:08 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all rbd-mirror 2024-04-25T02:16:08.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:08 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all ceph-exporter 2024-04-25T02:16:08.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:08 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all iscsi 2024-04-25T02:16:08.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:08 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all nfs 2024-04-25T02:16:08.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:08 smithi138 ceph-mon[81304]: Upgrade: Setting container_image for all nvmeof 2024-04-25T02:16:08.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:08 smithi138 ceph-mon[81304]: Upgrade: Finalizing container_image settings 2024-04-25T02:16:08.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:08 smithi138 ceph-mon[81304]: Upgrade: Complete! 2024-04-25T02:16:09.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:09 smithi033 ceph-mon[111306]: pgmap v1105: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 3 op/s 2024-04-25T02:16:09.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:09.523 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:09 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:09.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:09 smithi138 ceph-mon[81304]: pgmap v1105: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 3 op/s 2024-04-25T02:16:09.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:09.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:09 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:11.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:11.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:16:11.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:11 smithi033 ceph-mon[111306]: pgmap v1106: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 8 op/s 2024-04-25T02:16:11.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:11.697 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:11.697 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:16:11.698 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:11 smithi138 ceph-mon[81304]: pgmap v1106: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 8 op/s 2024-04-25T02:16:11.698 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:12.546 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:12 smithi138 ceph-mon[81304]: pgmap v1107: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 9 op/s 2024-04-25T02:16:12.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:12 smithi033 ceph-mon[111306]: pgmap v1107: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 26 MiB/s rd, 9 op/s 2024-04-25T02:16:13.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:13 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:13.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:13 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:13.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:13 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:resolve 2024-04-25T02:16:13.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:13 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:resolve 2024-04-25T02:16:13.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:13 smithi138 ceph-mon[81304]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:resolve,1=cephfs.smithi033.hobkqr=up:resolve} 2 up:standby 2024-04-25T02:16:14.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:13 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:14.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:13 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:14.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:13 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:resolve 2024-04-25T02:16:14.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:13 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:resolve 2024-04-25T02:16:14.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:13 smithi033 ceph-mon[111306]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:resolve,1=cephfs.smithi033.hobkqr=up:resolve} 2 up:standby 2024-04-25T02:16:15.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:14 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:15.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:14 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:15.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:14 smithi033 ceph-mon[111306]: pgmap v1108: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-04-25T02:16:15.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:14 smithi033 ceph-mon[111306]: reconnect by client.14506 192.168.0.1:0/899480764 after 0 2024-04-25T02:16:15.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:14 smithi033 ceph-mon[111306]: reconnect by client.24333 192.168.0.1:0/193185086 after 0.000999983 2024-04-25T02:16:15.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:14 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:reconnect 2024-04-25T02:16:15.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:14 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:reconnect 2024-04-25T02:16:15.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:14 smithi033 ceph-mon[111306]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:reconnect,1=cephfs.smithi033.hobkqr=up:reconnect} 2 up:standby 2024-04-25T02:16:15.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:14 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:15.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:14 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:15.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:14 smithi138 ceph-mon[81304]: pgmap v1108: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-04-25T02:16:15.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:14 smithi138 ceph-mon[81304]: reconnect by client.14506 192.168.0.1:0/899480764 after 0 2024-04-25T02:16:15.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:14 smithi138 ceph-mon[81304]: reconnect by client.24333 192.168.0.1:0/193185086 after 0.000999983 2024-04-25T02:16:15.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:14 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:reconnect 2024-04-25T02:16:15.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:14 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:reconnect 2024-04-25T02:16:15.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:14 smithi138 ceph-mon[81304]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:reconnect,1=cephfs.smithi033.hobkqr=up:reconnect} 2 up:standby 2024-04-25T02:16:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.905+0000 7f442eef0700 1 -- 172.21.15.33:0/1654306425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 msgr2=0x7f44281020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.905+0000 7f442eef0700 1 --2- 172.21.15.33:0/1654306425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 0x7f44281020e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f441c004940 tx=0x7f441c009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:15.912 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.905+0000 7f442eef0700 1 -- 172.21.15.33:0/1654306425 shutdown_connections 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.905+0000 7f442eef0700 1 --2- 172.21.15.33:0/1654306425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44281026b0 0x7f44280fbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.905+0000 7f442eef0700 1 --2- 172.21.15.33:0/1654306425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 0x7f44281020e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.905+0000 7f442eef0700 1 -- 172.21.15.33:0/1654306425 >> 172.21.15.33:0/1654306425 conn(0x7f44280f7860 msgr2=0x7f44280f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.905+0000 7f442eef0700 1 -- 172.21.15.33:0/1654306425 shutdown_connections 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.905+0000 7f442eef0700 1 -- 172.21.15.33:0/1654306425 wait complete. 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f442eef0700 1 Processor -- start 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f442eef0700 1 -- start start 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f442eef0700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 0x7f44281139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f442eef0700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44281026b0 0x7f4428113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f442eef0700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428114530 con 0x7f4428101d10 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f442eef0700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f442810db50 con 0x7f44281026b0 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f442cc8c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 0x7f44281139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f442cc8c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 0x7f44281139a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43210/0 (socket says 172.21.15.33:43210) 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f442cc8c700 1 -- 172.21.15.33:0/3897505530 learned_addr learned my addr 172.21.15.33:0/3897505530 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:16:15.913 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.906+0000 7f4427fff700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44281026b0 0x7f4428113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.907+0000 7f442cc8c700 1 -- 172.21.15.33:0/3897505530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44281026b0 msgr2=0x7f4428113ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.907+0000 7f442cc8c700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44281026b0 0x7f4428113ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.907+0000 7f442cc8c700 1 -- 172.21.15.33:0/3897505530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c005040 con 0x7f4428101d10 2024-04-25T02:16:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.907+0000 7f442cc8c700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 0x7f44281139a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f441c00b3b0 tx=0x7f441c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.907+0000 7f44257fa700 1 -- 172.21.15.33:0/3897505530 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f441c00e040 con 0x7f4428101d10 2024-04-25T02:16:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.907+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f442810ddd0 con 0x7f4428101d10 2024-04-25T02:16:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.907+0000 7f44257fa700 1 -- 172.21.15.33:0/3897505530 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f441c013420 con 0x7f4428101d10 2024-04-25T02:16:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.907+0000 7f44257fa700 1 -- 172.21.15.33:0/3897505530 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f441c01c630 con 0x7f4428101d10 2024-04-25T02:16:15.914 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.907+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f442810e390 con 0x7f4428101d10 2024-04-25T02:16:15.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.908+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44280fd460 con 0x7f4428101d10 2024-04-25T02:16:15.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.909+0000 7f44257fa700 1 -- 172.21.15.33:0/3897505530 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f441c013590 con 0x7f4428101d10 2024-04-25T02:16:15.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.909+0000 7f44257fa700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441806e140 0x7f44180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:15.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.909+0000 7f44257fa700 1 -- 172.21.15.33:0/3897505530 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f441c08ed20 con 0x7f4428101d10 2024-04-25T02:16:15.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.909+0000 7f4427fff700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441806e140 0x7f44180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:15.915 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.910+0000 7f4427fff700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441806e140 0x7f44180705f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f442810f240 tx=0x7f4414009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:15.919 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:15.914+0000 7f44257fa700 1 -- 172.21.15.33:0/3897505530 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f441c058120 con 0x7f4428101d10 2024-04-25T02:16:16.046 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:15 smithi033 ceph-mon[111306]: reconnect by client.14506 192.168.0.1:0/899480764 after 0 2024-04-25T02:16:16.046 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:15 smithi033 ceph-mon[111306]: reconnect by client.24333 192.168.0.1:0/193185086 after 0.000999983 2024-04-25T02:16:16.046 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:15 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:rejoin 2024-04-25T02:16:16.046 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:15 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:rejoin 2024-04-25T02:16:16.046 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:15 smithi033 ceph-mon[111306]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:rejoin,1=cephfs.smithi033.hobkqr=up:rejoin} 2 up:standby 2024-04-25T02:16:16.046 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:15 smithi033 ceph-mon[111306]: daemon mds.cephfs.smithi033.hobkqr is now active in filesystem cephfs as rank 1 2024-04-25T02:16:16.046 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:15 smithi033 ceph-mon[111306]: daemon mds.cephfs.smithi033.qqhhcz is now active in filesystem cephfs as rank 0 2024-04-25T02:16:16.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:15 smithi138 ceph-mon[81304]: reconnect by client.14506 192.168.0.1:0/899480764 after 0 2024-04-25T02:16:16.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:15 smithi138 ceph-mon[81304]: reconnect by client.24333 192.168.0.1:0/193185086 after 0.000999983 2024-04-25T02:16:16.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:15 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:rejoin 2024-04-25T02:16:16.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:15 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:rejoin 2024-04-25T02:16:16.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:15 smithi138 ceph-mon[81304]: fsmap cephfs:2/2 {0=cephfs.smithi033.qqhhcz=up:rejoin,1=cephfs.smithi033.hobkqr=up:rejoin} 2 up:standby 2024-04-25T02:16:16.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:15 smithi138 ceph-mon[81304]: daemon mds.cephfs.smithi033.hobkqr is now active in filesystem cephfs as rank 1 2024-04-25T02:16:16.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:15 smithi138 ceph-mon[81304]: daemon mds.cephfs.smithi033.qqhhcz is now active in filesystem cephfs as rank 0 2024-04-25T02:16:16.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.146+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f442810f060 con 0x7f441806e140 2024-04-25T02:16:16.152 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.147+0000 7f44257fa700 1 -- 172.21.15.33:0/3897505530 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f442810f060 con 0x7f441806e140 2024-04-25T02:16:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.149+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441806e140 msgr2=0x7f44180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:16.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.149+0000 7f442eef0700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441806e140 0x7f44180705f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f442810f240 tx=0x7f4414009040 comp rx=0 tx=0).stop 2024-04-25T02:16:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.149+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 msgr2=0x7f44281139a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.149+0000 7f442eef0700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 0x7f44281139a0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f441c00b3b0 tx=0x7f441c009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.150+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 shutdown_connections 2024-04-25T02:16:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.150+0000 7f442eef0700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f441806e140 0x7f44180705f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.150+0000 7f442eef0700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f4428101d10 0x7f44281139a0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.150+0000 7f442eef0700 1 --2- 172.21.15.33:0/3897505530 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f44281026b0 0x7f4428113ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.150+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 >> 172.21.15.33:0/3897505530 conn(0x7f44280f7860 msgr2=0x7f44280f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.150+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 shutdown_connections 2024-04-25T02:16:16.155 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:16.150+0000 7f442eef0700 1 -- 172.21.15.33:0/3897505530 wait complete. 2024-04-25T02:16:17.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:16 smithi138 ceph-mon[81304]: from='client.36000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:16:17.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:16 smithi138 ceph-mon[81304]: pgmap v1109: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T02:16:17.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:16 smithi138 ceph-mon[81304]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T02:16:17.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:16 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:active 2024-04-25T02:16:17.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:16 smithi138 ceph-mon[81304]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:active 2024-04-25T02:16:17.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:16 smithi138 ceph-mon[81304]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi033.hobkqr=up:active} 2 up:standby 2024-04-25T02:16:17.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:16 smithi138 ceph-mon[81304]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi033.hobkqr=up:active} 2 up:standby-replay 2024-04-25T02:16:17.059 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:16 smithi033 ceph-mon[111306]: from='client.36000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:16:17.059 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:16 smithi033 ceph-mon[111306]: pgmap v1109: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T02:16:17.059 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:16 smithi033 ceph-mon[111306]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T02:16:17.060 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:16 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] up:active 2024-04-25T02:16:17.060 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:16 smithi033 ceph-mon[111306]: mds.? [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] up:active 2024-04-25T02:16:17.060 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:16 smithi033 ceph-mon[111306]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi033.hobkqr=up:active} 2 up:standby 2024-04-25T02:16:17.060 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:16 smithi033 ceph-mon[111306]: fsmap cephfs:2 {0=cephfs.smithi033.qqhhcz=up:active,1=cephfs.smithi033.hobkqr=up:active} 2 up:standby-replay 2024-04-25T02:16:18.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:18 smithi033 ceph-mon[111306]: pgmap v1110: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T02:16:18.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:18 smithi138 ceph-mon[81304]: pgmap v1110: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 511 B/s wr, 10 op/s 2024-04-25T02:16:20.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:19 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:20.022 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:19 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:20.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:19 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:20.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:19 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:21.021 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:20 smithi033 ceph-mon[111306]: pgmap v1111: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-04-25T02:16:21.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:20 smithi138 ceph-mon[81304]: pgmap v1111: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 30 MiB/s rd, 4.9 KiB/s wr, 17 op/s 2024-04-25T02:16:22.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:22 smithi033 ceph-mon[111306]: pgmap v1112: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 4.9 KiB/s wr, 16 op/s 2024-04-25T02:16:22.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:22 smithi138 ceph-mon[81304]: pgmap v1112: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 4.9 KiB/s wr, 16 op/s 2024-04-25T02:16:23.179 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch ps' 2024-04-25T02:16:23.491 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:16:23.883 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:23.883 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:23.883 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:16:23.883 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:16:23.883 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: pgmap v1113: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 5.7 KiB/s wr, 18 op/s 2024-04-25T02:16:23.883 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:23.883 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:16:23.884 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:16:23.884 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:16:23.884 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:24.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:24.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:24.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:16:24.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:16:24.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: pgmap v1113: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 5.7 KiB/s wr, 18 op/s 2024-04-25T02:16:24.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:24.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:16:24.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:16:24.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:16:24.047 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:25.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:24 smithi138 ceph-mon[81304]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-25T02:16:25.046 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:24 smithi138 ceph-mon[81304]: Cluster is now healthy 2024-04-25T02:16:25.106 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:24 smithi033 ceph-mon[111306]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-04-25T02:16:25.106 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:24 smithi033 ceph-mon[111306]: Cluster is now healthy 2024-04-25T02:16:25.223 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.204+0000 7f7d0199e700 1 -- 172.21.15.33:0/1336253997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 msgr2=0x7f7cfc103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.204+0000 7f7d0199e700 1 --2- 172.21.15.33:0/1336253997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 0x7f7cfc103330 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f7cec004940 tx=0x7f7cec009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.204+0000 7f7d0199e700 1 -- 172.21.15.33:0/1336253997 shutdown_connections 2024-04-25T02:16:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.204+0000 7f7d0199e700 1 --2- 172.21.15.33:0/1336253997 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0fb520 0x7f7cfc0fb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.204+0000 7f7d0199e700 1 --2- 172.21.15.33:0/1336253997 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 0x7f7cfc103330 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.204+0000 7f7d0199e700 1 -- 172.21.15.33:0/1336253997 >> 172.21.15.33:0/1336253997 conn(0x7f7cfc0714d0 msgr2=0x7f7cfc0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.204+0000 7f7d0199e700 1 -- 172.21.15.33:0/1336253997 shutdown_connections 2024-04-25T02:16:25.224 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.204+0000 7f7d0199e700 1 -- 172.21.15.33:0/1336253997 wait complete. 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7d0199e700 1 Processor -- start 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7d0199e700 1 -- start start 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7d0199e700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0fb520 0x7f7cfc1139a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7d0199e700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 0x7f7cfc113ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7d0199e700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc114530 con 0x7f7cfc102f60 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7d0199e700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc10db30 con 0x7f7cfc0fb520 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7cfa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 0x7f7cfc113ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7cfa7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 0x7f7cfc113ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43238/0 (socket says 172.21.15.33:43238) 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7cfa7fc700 1 -- 172.21.15.33:0/3014968324 learned_addr learned my addr 172.21.15.33:0/3014968324 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7cfaffd700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0fb520 0x7f7cfc1139a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7cfa7fc700 1 -- 172.21.15.33:0/3014968324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0fb520 msgr2=0x7f7cfc1139a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:25.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7cfa7fc700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0fb520 0x7f7cfc1139a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7cfa7fc700 1 -- 172.21.15.33:0/3014968324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec005040 con 0x7f7cfc102f60 2024-04-25T02:16:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.205+0000 7f7cfa7fc700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 0x7f7cfc113ee0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0002a50 tx=0x7f7cf00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.206+0000 7f7ce37fe700 1 -- 172.21.15.33:0/3014968324 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cf0007cc0 con 0x7f7cfc102f60 2024-04-25T02:16:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.206+0000 7f7ce37fe700 1 -- 172.21.15.33:0/3014968324 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cf000f040 con 0x7f7cfc102f60 2024-04-25T02:16:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.206+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc10de10 con 0x7f7cfc102f60 2024-04-25T02:16:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.206+0000 7f7ce37fe700 1 -- 172.21.15.33:0/3014968324 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cf000da30 con 0x7f7cfc102f60 2024-04-25T02:16:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.206+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc10e350 con 0x7f7cfc102f60 2024-04-25T02:16:25.226 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.207+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc100250 con 0x7f7cfc102f60 2024-04-25T02:16:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.207+0000 7f7ce37fe700 1 -- 172.21.15.33:0/3014968324 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7cf000db90 con 0x7f7cfc102f60 2024-04-25T02:16:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.207+0000 7f7ce37fe700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7ce406e140 0x7f7ce40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.208+0000 7f7ce37fe700 1 -- 172.21.15.33:0/3014968324 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7cf008d8b0 con 0x7f7cfc102f60 2024-04-25T02:16:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.208+0000 7f7cfaffd700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7ce406e140 0x7f7ce40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.208+0000 7f7cfaffd700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7ce406e140 0x7f7ce40705f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7cec004f90 tx=0x7f7cec009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:25.227 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.212+0000 7f7ce37fe700 1 -- 172.21.15.33:0/3014968324 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7cf005a760 con 0x7f7cfc102f60 2024-04-25T02:16:25.434 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.429+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cfc113670 con 0x7f7ce406e140 2024-04-25T02:16:25.445 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.440+0000 7f7ce37fe700 1 -- 172.21.15.33:0/3014968324 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7cfc113670 con 0x7f7ce406e140 2024-04-25T02:16:25.445 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:16:25.445 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (37m) 12s ago 48m 25.5M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (34m) 12s ago 48m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (34m) 16s ago 47m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (37m) 12s ago 48m 80.2M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (30m) 12s ago 43m 115M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (31m) 12s ago 43m 108M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (30m) 16s ago 43m 25.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (15m) 16s ago 43m 20.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (39m) 12s ago 50m 517M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (38m) 16s ago 47m 458M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (35m) 12s ago 50m 85.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (35m) 16s ago 47m 58.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (38m) 12s ago 48m 15.5M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:16:25.446 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (38m) 16s ago 47m 15.5M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:16:25.447 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (34m) 12s ago 47m 104M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:16:25.447 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (33m) 12s ago 46m 109M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:16:25.447 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (33m) 12s ago 46m 90.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:16:25.447 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (32m) 16s ago 45m 112M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:16:25.447 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (31m) 16s ago 45m 109M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:16:25.447 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (31m) 16s ago 44m 103M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:16:25.447 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (37m) 12s ago 48m 76.2M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:16:25.447 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.442+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7ce406e140 msgr2=0x7f7ce40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.442+0000 7f7d0199e700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7ce406e140 0x7f7ce40705f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f7cec004f90 tx=0x7f7cec009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.442+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 msgr2=0x7f7cfc113ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.442+0000 7f7d0199e700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 0x7f7cfc113ee0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0002a50 tx=0x7f7cf00046b0 comp rx=0 tx=0).stop 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.442+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 shutdown_connections 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.442+0000 7f7d0199e700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f7ce406e140 0x7f7ce40705f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.442+0000 7f7d0199e700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7cfc0fb520 0x7f7cfc1139a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.442+0000 7f7d0199e700 1 --2- 172.21.15.33:0/3014968324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7cfc102f60 0x7f7cfc113ee0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.442+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 >> 172.21.15.33:0/3014968324 conn(0x7f7cfc0714d0 msgr2=0x7f7cfc0fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.443+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 shutdown_connections 2024-04-25T02:16:25.448 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:25.443+0000 7f7d0199e700 1 -- 172.21.15.33:0/3014968324 wait complete. 2024-04-25T02:16:26.082 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch upgrade status' 2024-04-25T02:16:26.352 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:16:26.639 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:26 smithi033 ceph-mon[111306]: pgmap v1114: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-04-25T02:16:26.639 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:26.639 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:16:26.639 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:26 smithi033 ceph-mon[111306]: from='client.36004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:16:26.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:26 smithi138 ceph-mon[81304]: pgmap v1114: 65 pgs: 65 active+clean; 257 MiB data, 1021 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 5.7 KiB/s wr, 17 op/s 2024-04-25T02:16:26.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:16:26.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:16:26.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:26 smithi138 ceph-mon[81304]: from='client.36004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:16:27.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.888+0000 7f85f7d73700 1 -- 172.21.15.33:0/1596391679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 msgr2=0x7f85f00fb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.888+0000 7f85f7d73700 1 --2- 172.21.15.33:0/1596391679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 0x7f85f00fb7c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f85e4004940 tx=0x7f85e4009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.888+0000 7f85f7d73700 1 -- 172.21.15.33:0/1596391679 shutdown_connections 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.888+0000 7f85f7d73700 1 --2- 172.21.15.33:0/1596391679 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 0x7f85f00fb7c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.888+0000 7f85f7d73700 1 --2- 172.21.15.33:0/1596391679 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85f00fdc10 0x7f85f00fdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.888+0000 7f85f7d73700 1 -- 172.21.15.33:0/1596391679 >> 172.21.15.33:0/1596391679 conn(0x7f85f0073350 msgr2=0x7f85f0073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.888+0000 7f85f7d73700 1 -- 172.21.15.33:0/1596391679 shutdown_connections 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.888+0000 7f85f7d73700 1 -- 172.21.15.33:0/1596391679 wait complete. 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f7d73700 1 Processor -- start 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f7d73700 1 -- start start 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f7d73700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 0x7f85f0115b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f7d73700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85f00fdc10 0x7f85f0116090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f7d73700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f01166e0 con 0x7f85f00fb350 2024-04-25T02:16:27.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f7d73700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f010fd70 con 0x7f85f00fdc10 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f5b0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 0x7f85f0115b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f5b0f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 0x7f85f0115b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43818/0 (socket says 172.21.15.33:43818) 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f5b0f700 1 -- 172.21.15.33:0/916187676 learned_addr learned my addr 172.21.15.33:0/916187676 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f530e700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85f00fdc10 0x7f85f0116090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f5b0f700 1 -- 172.21.15.33:0/916187676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85f00fdc10 msgr2=0x7f85f0116090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f5b0f700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85f00fdc10 0x7f85f0116090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f5b0f700 1 -- 172.21.15.33:0/916187676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4005040 con 0x7f85f00fb350 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f530e700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85f00fdc10 0x7f85f0116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.889+0000 7f85f5b0f700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 0x7f85f0115b50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f85e800cb10 tx=0x7f85e8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:27.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.890+0000 7f85ee7fc700 1 -- 172.21.15.33:0/916187676 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85e8004200 con 0x7f85f00fb350 2024-04-25T02:16:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.890+0000 7f85ee7fc700 1 -- 172.21.15.33:0/916187676 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85e800acc0 con 0x7f85f00fb350 2024-04-25T02:16:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.890+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f0110050 con 0x7f85f00fb350 2024-04-25T02:16:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.890+0000 7f85ee7fc700 1 -- 172.21.15.33:0/916187676 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85e8003d90 con 0x7f85f00fb350 2024-04-25T02:16:27.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.890+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f0110590 con 0x7f85f00fb350 2024-04-25T02:16:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.891+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f00fc360 con 0x7f85f00fb350 2024-04-25T02:16:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.891+0000 7f85ee7fc700 1 -- 172.21.15.33:0/916187676 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f85e8003430 con 0x7f85f00fb350 2024-04-25T02:16:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.892+0000 7f85ee7fc700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f85d806e140 0x7f85d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.892+0000 7f85ee7fc700 1 -- 172.21.15.33:0/916187676 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f85e808e060 con 0x7f85f00fb350 2024-04-25T02:16:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.892+0000 7f85f530e700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f85d806e140 0x7f85d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:27.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.892+0000 7f85f530e700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f85d806e140 0x7f85d80705f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f85e4004f90 tx=0x7f85e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:27.901 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:27.897+0000 7f85ee7fc700 1 -- 172.21.15.33:0/916187676 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f85e8057460 con 0x7f85f00fb350 2024-04-25T02:16:28.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.122+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85f010c120 con 0x7f85d806e140 2024-04-25T02:16:28.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.123+0000 7f85ee7fc700 1 -- 172.21.15.33:0/916187676 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f85f010c120 con 0x7f85d806e140 2024-04-25T02:16:28.128 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:16:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "target_image": null, 2024-04-25T02:16:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "in_progress": false, 2024-04-25T02:16:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "which": "", 2024-04-25T02:16:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "services_complete": [], 2024-04-25T02:16:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "progress": null, 2024-04-25T02:16:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "message": "", 2024-04-25T02:16:28.129 INFO:teuthology.orchestra.run.smithi033.stdout: "is_paused": false 2024-04-25T02:16:28.129 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:16:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.125+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f85d806e140 msgr2=0x7f85d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.125+0000 7f85f7d73700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f85d806e140 0x7f85d80705f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f85e4004f90 tx=0x7f85e4009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.126+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 msgr2=0x7f85f0115b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.126+0000 7f85f7d73700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 0x7f85f0115b50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f85e800cb10 tx=0x7f85e8007f70 comp rx=0 tx=0).stop 2024-04-25T02:16:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.126+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 shutdown_connections 2024-04-25T02:16:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.126+0000 7f85f7d73700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f85d806e140 0x7f85d80705f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.126+0000 7f85f7d73700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f85f00fb350 0x7f85f0115b50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.126+0000 7f85f7d73700 1 --2- 172.21.15.33:0/916187676 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f85f00fdc10 0x7f85f0116090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:28.131 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.126+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 >> 172.21.15.33:0/916187676 conn(0x7f85f0073350 msgr2=0x7f85f00f87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:28.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.126+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 shutdown_connections 2024-04-25T02:16:28.132 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:28.126+0000 7f85f7d73700 1 -- 172.21.15.33:0/916187676 wait complete. 2024-04-25T02:16:28.498 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:28 smithi033 ceph-mon[111306]: pgmap v1115: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 11 KiB/s wr, 21 op/s 2024-04-25T02:16:28.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:28 smithi138 ceph-mon[81304]: pgmap v1115: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 11 KiB/s wr, 21 op/s 2024-04-25T02:16:28.993 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph health detail' 2024-04-25T02:16:29.265 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:16:29.596 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:29 smithi033 ceph-mon[111306]: from='client.36008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:16:29.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:29 smithi138 ceph-mon[81304]: from='client.36008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:16:30.531 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:30 smithi033 ceph-mon[111306]: pgmap v1116: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-25T02:16:30.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:30 smithi138 ceph-mon[81304]: pgmap v1116: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 10 KiB/s wr, 17 op/s 2024-04-25T02:16:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.247+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3606894324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fc190 msgr2=0x7fd4f0101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:31.256 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.247+0000 7fd4f4b32700 1 --2- 172.21.15.33:0/3606894324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fc190 0x7fd4f0101a70 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4004940 tx=0x7fd4e4009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.247+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3606894324 shutdown_connections 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.247+0000 7fd4f4b32700 1 --2- 172.21.15.33:0/3606894324 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fc190 0x7fd4f0101a70 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.247+0000 7fd4f4b32700 1 --2- 172.21.15.33:0/3606894324 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f00fb7f0 0x7fd4f00fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.247+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3606894324 >> 172.21.15.33:0/3606894324 conn(0x7fd4f0071460 msgr2=0x7fd4f0071860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.247+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3606894324 shutdown_connections 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.247+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3606894324 wait complete. 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4f4b32700 1 Processor -- start 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4f4b32700 1 -- start start 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4f4b32700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fb7f0 0x7fd4f0113a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4f4b32700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f00fc190 0x7fd4f0113f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4f4b32700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0114590 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4f4b32700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f010db90 con 0x7fd4f00fc190 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4ee59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fb7f0 0x7fd4f0113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4ee59c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fb7f0 0x7fd4f0113a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43828/0 (socket says 172.21.15.33:43828) 2024-04-25T02:16:31.257 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4ee59c700 1 -- 172.21.15.33:0/3312795480 learned_addr learned my addr 172.21.15.33:0/3312795480 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4edd9b700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f00fc190 0x7fd4f0113f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4ee59c700 1 -- 172.21.15.33:0/3312795480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f00fc190 msgr2=0x7fd4f0113f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4ee59c700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f00fc190 0x7fd4f0113f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4ee59c700 1 -- 172.21.15.33:0/3312795480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e4005040 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.248+0000 7fd4ee59c700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fb7f0 0x7fd4f0113a00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0002a50 tx=0x7fd4e00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.249+0000 7fd4deffd700 1 -- 172.21.15.33:0/3312795480 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e0007cc0 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.249+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f010de70 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.249+0000 7fd4deffd700 1 -- 172.21.15.33:0/3312795480 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4e000f040 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.249+0000 7fd4deffd700 1 -- 172.21.15.33:0/3312795480 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4e000da70 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.249+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f010e380 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.250+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f0065f30 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.250+0000 7fd4deffd700 1 -- 172.21.15.33:0/3312795480 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd4e000dc10 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.251+0000 7fd4deffd700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd4d806e140 0x7fd4d80705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:31.258 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.251+0000 7fd4deffd700 1 -- 172.21.15.33:0/3312795480 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd4e008d180 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.251+0000 7fd4edd9b700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd4d806e140 0x7fd4d80705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:31.259 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.251+0000 7fd4edd9b700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd4d806e140 0x7fd4d80705f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400b3b0 tx=0x7fd4e4009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:31.260 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.255+0000 7fd4deffd700 1 -- 172.21.15.33:0/3312795480 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd4e005a030 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.558+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd4f010efe0 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.563 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.558+0000 7fd4deffd700 1 -- 172.21.15.33:0/3312795480 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fd4e0059bc0 con 0x7fd4f00fb7f0 2024-04-25T02:16:31.564 INFO:teuthology.orchestra.run.smithi033.stdout:HEALTH_OK 2024-04-25T02:16:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd4d806e140 msgr2=0x7fd4d80705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd4d806e140 0x7fd4d80705f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400b3b0 tx=0x7fd4e4009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fb7f0 msgr2=0x7fd4f0113a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fb7f0 0x7fd4f0113a00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0002a50 tx=0x7fd4e00046b0 comp rx=0 tx=0).stop 2024-04-25T02:16:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 shutdown_connections 2024-04-25T02:16:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd4d806e140 0x7fd4d80705f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd4f00fb7f0 0x7fd4f0113a00 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 --2- 172.21.15.33:0/3312795480 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd4f00fc190 0x7fd4f0113f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:31.566 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 >> 172.21.15.33:0/3312795480 conn(0x7fd4f0071460 msgr2=0x7fd4f0100a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 shutdown_connections 2024-04-25T02:16:31.567 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:31.561+0000 7fd4f4b32700 1 -- 172.21.15.33:0/3312795480 wait complete. 2024-04-25T02:16:32.201 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions' 2024-04-25T02:16:32.470 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:16:32.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:32 smithi033 ceph-mon[111306]: pgmap v1117: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 8.4 MiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-04-25T02:16:32.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:32 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3312795480' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:16:32.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:32 smithi138 ceph-mon[81304]: pgmap v1117: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 8.4 MiB/s rd, 4.9 KiB/s wr, 8 op/s 2024-04-25T02:16:32.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:32 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3312795480' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.140+0000 7f167ca4a700 1 -- 172.21.15.33:0/3164919655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 msgr2=0x7f16781031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.140+0000 7f167ca4a700 1 --2- 172.21.15.33:0/3164919655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 0x7f16781031d0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f166c004940 tx=0x7f166c009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.140+0000 7f167ca4a700 1 -- 172.21.15.33:0/3164919655 shutdown_connections 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.140+0000 7f167ca4a700 1 --2- 172.21.15.33:0/3164919655 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16781037a0 0x7f16780fce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.140+0000 7f167ca4a700 1 --2- 172.21.15.33:0/3164919655 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 0x7f16781031d0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.140+0000 7f167ca4a700 1 -- 172.21.15.33:0/3164919655 >> 172.21.15.33:0/3164919655 conn(0x7f16780f89d0 msgr2=0x7f16780fade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.140+0000 7f167ca4a700 1 -- 172.21.15.33:0/3164919655 shutdown_connections 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.140+0000 7f167ca4a700 1 -- 172.21.15.33:0/3164919655 wait complete. 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.140+0000 7f167ca4a700 1 Processor -- start 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167ca4a700 1 -- start start 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167ca4a700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 0x7f1678113980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167ca4a700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16781037a0 0x7f1678113ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:34.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167ca4a700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1678114510 con 0x7f1678102e00 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167ca4a700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167810db10 con 0x7f16781037a0 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 0x7f1678113980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167659c700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 0x7f1678113980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:43846/0 (socket says 172.21.15.33:43846) 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167659c700 1 -- 172.21.15.33:0/223416634 learned_addr learned my addr 172.21.15.33:0/223416634 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f1675d9b700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16781037a0 0x7f1678113ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167659c700 1 -- 172.21.15.33:0/223416634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16781037a0 msgr2=0x7f1678113ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167659c700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16781037a0 0x7f1678113ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167659c700 1 -- 172.21.15.33:0/223416634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f166c005040 con 0x7f1678102e00 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.141+0000 7f167659c700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 0x7f1678113980 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b3b0 tx=0x7f166c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.142+0000 7f1666ffd700 1 -- 172.21.15.33:0/223416634 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166c00e040 con 0x7f1678102e00 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.142+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167810dd90 con 0x7f1678102e00 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.142+0000 7f1666ffd700 1 -- 172.21.15.33:0/223416634 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f166c013420 con 0x7f1678102e00 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.142+0000 7f1666ffd700 1 -- 172.21.15.33:0/223416634 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166c01c630 con 0x7f1678102e00 2024-04-25T02:16:34.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.142+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167810e350 con 0x7f1678102e00 2024-04-25T02:16:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.143+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1678100110 con 0x7f1678102e00 2024-04-25T02:16:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.143+0000 7f1666ffd700 1 -- 172.21.15.33:0/223416634 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f166c013590 con 0x7f1678102e00 2024-04-25T02:16:34.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.144+0000 7f1666ffd700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f166006e140 0x7f16600705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:16:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.144+0000 7f1666ffd700 1 -- 172.21.15.33:0/223416634 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f166c08ed20 con 0x7f1678102e00 2024-04-25T02:16:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.144+0000 7f1675d9b700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f166006e140 0x7f16600705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:16:34.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.144+0000 7f1675d9b700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f166006e140 0x7f16600705f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f167810ecd0 tx=0x7f166800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:16:34.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.148+0000 7f1666ffd700 1 -- 172.21.15.33:0/223416634 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f166c05bc50 con 0x7f1678102e00 2024-04-25T02:16:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.447+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f167810efd0 con 0x7f1678102e00 2024-04-25T02:16:34.452 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.448+0000 7f1666ffd700 1 -- 172.21.15.33:0/223416634 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f166c05b7e0 con 0x7f1678102e00 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:16:34.453 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:16:34.454 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:16:34.454 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:16:34.454 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:16:34.454 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:16:34.454 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:16:34.454 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f166006e140 msgr2=0x7f16600705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f166006e140 0x7f16600705f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f167810ecd0 tx=0x7f166800b040 comp rx=0 tx=0).stop 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 msgr2=0x7f1678113980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 0x7f1678113980 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f166c00b3b0 tx=0x7f166c009e30 comp rx=0 tx=0).stop 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 shutdown_connections 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f166006e140 0x7f16600705f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1678102e00 0x7f1678113980 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 --2- 172.21.15.33:0/223416634 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f16781037a0 0x7f1678113ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 >> 172.21.15.33:0/223416634 conn(0x7f16780f89d0 msgr2=0x7f16780fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 shutdown_connections 2024-04-25T02:16:34.455 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:16:34.450+0000 7f167ca4a700 1 -- 172.21.15.33:0/223416634 wait complete. 2024-04-25T02:16:34.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:34 smithi033 ceph-mon[111306]: pgmap v1118: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-25T02:16:34.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:34 smithi138 ceph-mon[81304]: pgmap v1118: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-25T02:16:35.144 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-25T02:16:35.409 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:16:35.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:35 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/223416634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:35.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:35 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/223416634' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:16:36.556 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:36 smithi033 ceph-mon[111306]: pgmap v1119: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 586 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T02:16:36.697 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:36 smithi138 ceph-mon[81304]: pgmap v1119: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 586 KiB/s rd, 4.2 KiB/s wr, 4 op/s 2024-04-25T02:16:37.014 INFO:teuthology.orchestra.run.smithi033.stdout:wait for servicemap items w/ changing names to refresh 2024-04-25T02:16:37.587 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'sleep 60' 2024-04-25T02:16:37.863 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:16:38.396 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:38 smithi033 ceph-mon[111306]: pgmap v1120: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 246 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-25T02:16:38.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:38 smithi138 ceph-mon[81304]: pgmap v1120: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 246 KiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-25T02:16:40.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:40 smithi033 ceph-mon[111306]: pgmap v1121: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:16:40.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:16:40.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:40 smithi138 ceph-mon[81304]: pgmap v1121: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:16:40.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:16:42.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:42 smithi033 ceph-mon[111306]: pgmap v1122: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:16:42.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:42 smithi138 ceph-mon[81304]: pgmap v1122: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:16:44.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:44 smithi033 ceph-mon[111306]: pgmap v1123: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:16:44.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:44 smithi138 ceph-mon[81304]: pgmap v1123: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:16:46.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:46 smithi033 ceph-mon[111306]: pgmap v1124: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:16:46.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:46 smithi138 ceph-mon[81304]: pgmap v1124: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:16:48.774 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:48 smithi033 ceph-mon[111306]: pgmap v1125: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:16:48.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:48 smithi138 ceph-mon[81304]: pgmap v1125: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:16:50.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:50 smithi033 ceph-mon[111306]: pgmap v1126: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:16:50.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:50 smithi138 ceph-mon[81304]: pgmap v1126: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:16:52.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:52 smithi033 ceph-mon[111306]: pgmap v1127: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:16:52.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:52 smithi138 ceph-mon[81304]: pgmap v1127: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:16:54.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:54 smithi033 ceph-mon[111306]: pgmap v1128: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:16:54.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:54 smithi138 ceph-mon[81304]: pgmap v1128: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:16:55.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:16:55.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:16:56.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:56 smithi033 ceph-mon[111306]: pgmap v1129: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:16:56.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:56 smithi138 ceph-mon[81304]: pgmap v1129: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:16:58.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:16:58 smithi033 ceph-mon[111306]: pgmap v1130: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:16:58.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:16:58 smithi138 ceph-mon[81304]: pgmap v1130: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:00.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:00 smithi033 ceph-mon[111306]: pgmap v1131: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:00.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:00 smithi138 ceph-mon[81304]: pgmap v1131: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:02.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:02 smithi033 ceph-mon[111306]: pgmap v1132: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:02.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:02 smithi138 ceph-mon[81304]: pgmap v1132: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:04.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:04 smithi033 ceph-mon[111306]: pgmap v1133: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T02:17:04.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:04 smithi138 ceph-mon[81304]: pgmap v1133: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T02:17:06.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:06 smithi033 ceph-mon[111306]: pgmap v1134: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-25T02:17:06.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:06 smithi138 ceph-mon[81304]: pgmap v1134: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-25T02:17:08.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:08 smithi033 ceph-mon[111306]: pgmap v1135: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T02:17:08.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:08 smithi138 ceph-mon[81304]: pgmap v1135: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 4 op/s 2024-04-25T02:17:10.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:10 smithi033 ceph-mon[111306]: pgmap v1136: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-25T02:17:10.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:17:10.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:10 smithi138 ceph-mon[81304]: pgmap v1136: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.6 KiB/s rd, 3 op/s 2024-04-25T02:17:10.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:17:12.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:12 smithi033 ceph-mon[111306]: pgmap v1137: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-25T02:17:12.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:12 smithi138 ceph-mon[81304]: pgmap v1137: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-04-25T02:17:14.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:14 smithi033 ceph-mon[111306]: pgmap v1138: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T02:17:14.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:14 smithi138 ceph-mon[81304]: pgmap v1138: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-04-25T02:17:16.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:16 smithi033 ceph-mon[111306]: pgmap v1139: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:16.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:16 smithi138 ceph-mon[81304]: pgmap v1139: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:18.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:18 smithi033 ceph-mon[111306]: pgmap v1140: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:18.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:18 smithi138 ceph-mon[81304]: pgmap v1140: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:20.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:20 smithi033 ceph-mon[111306]: pgmap v1141: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:20.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:20 smithi138 ceph-mon[81304]: pgmap v1141: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:22.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:22 smithi033 ceph-mon[111306]: pgmap v1142: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:22.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:22 smithi138 ceph-mon[81304]: pgmap v1142: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:23.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:23 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:17:23.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:23 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:17:24.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:24 smithi033 ceph-mon[111306]: pgmap v1143: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:17:24.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:17:24.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:17:24.773 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:17:24.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:24 smithi138 ceph-mon[81304]: pgmap v1143: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:17:24.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:17:24.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:17:24.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:17:25.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:17:25.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:17:26.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:26 smithi033 ceph-mon[111306]: pgmap v1144: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:26.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:26 smithi138 ceph-mon[81304]: pgmap v1144: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:28.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:28 smithi033 ceph-mon[111306]: pgmap v1145: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:28.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:28 smithi138 ceph-mon[81304]: pgmap v1145: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:30.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:30 smithi033 ceph-mon[111306]: pgmap v1146: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:30.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:30 smithi138 ceph-mon[81304]: pgmap v1146: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:32.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:32 smithi033 ceph-mon[111306]: pgmap v1147: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:32.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:32 smithi138 ceph-mon[81304]: pgmap v1147: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:34.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:34 smithi033 ceph-mon[111306]: pgmap v1148: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:17:34.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:34 smithi138 ceph-mon[81304]: pgmap v1148: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:17:36.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:36 smithi033 ceph-mon[111306]: pgmap v1149: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:36.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:36 smithi138 ceph-mon[81304]: pgmap v1149: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:38.771 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:38 smithi033 ceph-mon[111306]: pgmap v1150: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:38.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:38 smithi138 ceph-mon[81304]: pgmap v1150: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:40.071 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph orch ps' 2024-04-25T02:17:40.352 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:17:40.759 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:40 smithi033 ceph-mon[111306]: pgmap v1151: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:40.760 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:17:40.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:40 smithi138 ceph-mon[81304]: pgmap v1151: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:40.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:17:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.052+0000 7f0c74098700 1 -- 172.21.15.33:0/3671709311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 msgr2=0x7f0c6c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.052+0000 7f0c74098700 1 --2- 172.21.15.33:0/3671709311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 0x7f0c6c102160 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f0c60004940 tx=0x7f0c60009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.052+0000 7f0c74098700 1 -- 172.21.15.33:0/3671709311 shutdown_connections 2024-04-25T02:17:42.057 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.052+0000 7f0c74098700 1 --2- 172.21.15.33:0/3671709311 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c6c0fbd40 0x7f0c6c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.052+0000 7f0c74098700 1 --2- 172.21.15.33:0/3671709311 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 0x7f0c6c102160 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.052+0000 7f0c74098700 1 -- 172.21.15.33:0/3671709311 >> 172.21.15.33:0/3671709311 conn(0x7f0c6c071d00 msgr2=0x7f0c6c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.052+0000 7f0c74098700 1 -- 172.21.15.33:0/3671709311 shutdown_connections 2024-04-25T02:17:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.052+0000 7f0c74098700 1 -- 172.21.15.33:0/3671709311 wait complete. 2024-04-25T02:17:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c74098700 1 Processor -- start 2024-04-25T02:17:42.058 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c74098700 1 -- start start 2024-04-25T02:17:42.059 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c74098700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c6c0fbd40 0x7f0c6c102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c74098700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 0x7f0c6c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c74098700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c1035f0 con 0x7f0c6c101d90 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c74098700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c103730 con 0x7f0c6c0fbd40 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c71633700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 0x7f0c6c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c71633700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 0x7f0c6c103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58768/0 (socket says 172.21.15.33:58768) 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c71633700 1 -- 172.21.15.33:0/3884373632 learned_addr learned my addr 172.21.15.33:0/3884373632 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.053+0000 7f0c71e34700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c6c0fbd40 0x7f0c6c102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.054+0000 7f0c71633700 1 -- 172.21.15.33:0/3884373632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c6c0fbd40 msgr2=0x7f0c6c102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.054+0000 7f0c71633700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c6c0fbd40 0x7f0c6c102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.054+0000 7f0c71633700 1 -- 172.21.15.33:0/3884373632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60005040 con 0x7f0c6c101d90 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.054+0000 7f0c71633700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 0x7f0c6c103060 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0c68002a50 tx=0x7f0c680046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.054+0000 7f0c5e7fc700 1 -- 172.21.15.33:0/3884373632 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c68007cc0 con 0x7f0c6c101d90 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.054+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6c119ea0 con 0x7f0c6c101d90 2024-04-25T02:17:42.060 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.054+0000 7f0c5e7fc700 1 -- 172.21.15.33:0/3884373632 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c6800f040 con 0x7f0c6c101d90 2024-04-25T02:17:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.054+0000 7f0c5e7fc700 1 -- 172.21.15.33:0/3884373632 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c6800da30 con 0x7f0c6c101d90 2024-04-25T02:17:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.054+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c11a320 con 0x7f0c6c101d90 2024-04-25T02:17:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.055+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6c0fd430 con 0x7f0c6c101d90 2024-04-25T02:17:42.061 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.056+0000 7f0c5e7fc700 1 -- 172.21.15.33:0/3884373632 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0c6800dbb0 con 0x7f0c6c101d90 2024-04-25T02:17:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.056+0000 7f0c5e7fc700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c5806e150 0x7f0c58070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.056+0000 7f0c5e7fc700 1 -- 172.21.15.33:0/3884373632 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0c6808d800 con 0x7f0c6c101d90 2024-04-25T02:17:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.056+0000 7f0c71e34700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c5806e150 0x7f0c58070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:42.062 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.057+0000 7f0c71e34700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c5806e150 0x7f0c58070600 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000b3b0 tx=0x7f0c60009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:42.066 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.061+0000 7f0c5e7fc700 1 -- 172.21.15.33:0/3884373632 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0c68056c00 con 0x7f0c6c101d90 2024-04-25T02:17:42.284 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.280+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 --> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c6c108e60 con 0x7f0c5806e150 2024-04-25T02:17:42.294 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.290+0000 7f0c5e7fc700 1 -- 172.21.15.33:0/3884373632 <== mgr.34104 v2:172.21.15.33:6800/4075941889 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0c6c108e60 con 0x7f0c5806e150 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:alertmanager.smithi033 smithi033 *:9093,9094 running (38m) 89s ago 49m 25.5M - 0.25.0 c8568f914cd2 31fd56fc5c73 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi033 smithi033 running (36m) 89s ago 50m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 6277a3fa561a 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:crash.smithi138 smithi138 running (36m) 92s ago 49m 6660k - 19.0.0-2445-gd2c45d97 f68e37b807ef 5e23acdaf09e 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:grafana.smithi033 smithi033 *:3000 running (38m) 89s ago 49m 80.2M - 9.4.12 2bacad6d85d8 a4f82dc03e4b 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.hobkqr smithi033 running (32m) 89s ago 44m 115M - 19.0.0-2445-gd2c45d97 f68e37b807ef d0da68b540c0 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi033.qqhhcz smithi033 running (32m) 89s ago 44m 108M - 19.0.0-2445-gd2c45d97 f68e37b807ef 18ccd05c7842 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.jwzxlz smithi138 running (31m) 92s ago 44m 25.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef a48fbf288726 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:mds.cephfs.smithi138.rvxnby smithi138 running (16m) 92s ago 44m 20.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 13209a3a3a8d 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi033.asemba smithi033 *:8443,9283,8765 running (40m) 89s ago 51m 517M - 19.0.0-2445-gd2c45d97 f68e37b807ef 7de07b8d71f5 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:mgr.smithi138.mmfhyi smithi138 *:8443,9283,8765 running (39m) 92s ago 49m 458M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29c0bf3f3317 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi033 smithi033 running (37m) 89s ago 51m 85.1M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef f072ec3fc94f 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:mon.smithi138 smithi138 running (36m) 92s ago 49m 58.7M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef e2b197ed2121 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi033 smithi033 *:9100 running (39m) 89s ago 50m 15.5M - 1.5.0 0da6a335fe13 d0b97e8e3c9f 2024-04-25T02:17:42.295 INFO:teuthology.orchestra.run.smithi033.stdout:node-exporter.smithi138 smithi138 *:9100 running (39m) 92s ago 49m 15.5M - 1.5.0 0da6a335fe13 a07317a38f09 2024-04-25T02:17:42.296 INFO:teuthology.orchestra.run.smithi033.stdout:osd.0 smithi033 running (35m) 89s ago 48m 104M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 1da61cf4b50b 2024-04-25T02:17:42.296 INFO:teuthology.orchestra.run.smithi033.stdout:osd.1 smithi033 running (35m) 89s ago 47m 109M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 4ab9ac022ca9 2024-04-25T02:17:42.296 INFO:teuthology.orchestra.run.smithi033.stdout:osd.2 smithi033 running (34m) 89s ago 47m 90.0M 1559M 19.0.0-2445-gd2c45d97 f68e37b807ef 2cb795d14a3f 2024-04-25T02:17:42.296 INFO:teuthology.orchestra.run.smithi033.stdout:osd.3 smithi138 running (33m) 92s ago 46m 112M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef d00965714a8d 2024-04-25T02:17:42.296 INFO:teuthology.orchestra.run.smithi033.stdout:osd.4 smithi138 running (33m) 92s ago 46m 109M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef 5adf4eaefb0b 2024-04-25T02:17:42.296 INFO:teuthology.orchestra.run.smithi033.stdout:osd.5 smithi138 running (32m) 92s ago 46m 103M 2581M 19.0.0-2445-gd2c45d97 f68e37b807ef ab01b5d50ec7 2024-04-25T02:17:42.296 INFO:teuthology.orchestra.run.smithi033.stdout:prometheus.smithi033 smithi033 *:9095 running (39m) 89s ago 49m 76.2M - 2.43.0 a07b618ecd1d 23a89c7575d5 2024-04-25T02:17:42.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.292+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c5806e150 msgr2=0x7f0c58070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:42.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.292+0000 7f0c74098700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c5806e150 0x7f0c58070600 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000b3b0 tx=0x7f0c60009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:42.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.292+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 msgr2=0x7f0c6c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:42.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.292+0000 7f0c74098700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 0x7f0c6c103060 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0c68002a50 tx=0x7f0c680046b0 comp rx=0 tx=0).stop 2024-04-25T02:17:42.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.292+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 shutdown_connections 2024-04-25T02:17:42.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.292+0000 7f0c74098700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c5806e150 0x7f0c58070600 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:42.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.292+0000 7f0c74098700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c6c0fbd40 0x7f0c6c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:42.297 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.292+0000 7f0c74098700 1 --2- 172.21.15.33:0/3884373632 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c6c101d90 0x7f0c6c103060 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:42.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.292+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 >> 172.21.15.33:0/3884373632 conn(0x7f0c6c071d00 msgr2=0x7f0c6c0fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:42.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.293+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 shutdown_connections 2024-04-25T02:17:42.298 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:42.293+0000 7f0c74098700 1 -- 172.21.15.33:0/3884373632 wait complete. 2024-04-25T02:17:42.663 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:42 smithi033 ceph-mon[111306]: pgmap v1152: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:42.798 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:42 smithi138 ceph-mon[81304]: pgmap v1152: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:42.933 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions' 2024-04-25T02:17:43.199 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:17:43.592 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:43 smithi033 ceph-mon[111306]: from='client.36020 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:17:43.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:43 smithi138 ceph-mon[81304]: from='client.36020 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T02:17:44.708 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:44 smithi033 ceph-mon[111306]: pgmap v1153: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:17:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.722+0000 7f1e63896700 1 -- 172.21.15.33:0/4187309510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 msgr2=0x7f1e5c0fe240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.722+0000 7f1e63896700 1 --2- 172.21.15.33:0/4187309510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 0x7f1e5c0fe240 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1e50004940 tx=0x7f1e50009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.722+0000 7f1e63896700 1 -- 172.21.15.33:0/4187309510 shutdown_connections 2024-04-25T02:17:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.722+0000 7f1e63896700 1 --2- 172.21.15.33:0/4187309510 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5c0fe810 0x7f1e5c106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.722+0000 7f1e63896700 1 --2- 172.21.15.33:0/4187309510 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 0x7f1e5c0fe240 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.722+0000 7f1e63896700 1 -- 172.21.15.33:0/4187309510 >> 172.21.15.33:0/4187309510 conn(0x7f1e5c0f78e0 msgr2=0x7f1e5c0f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.722+0000 7f1e63896700 1 -- 172.21.15.33:0/4187309510 shutdown_connections 2024-04-25T02:17:44.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.722+0000 7f1e63896700 1 -- 172.21.15.33:0/4187309510 wait complete. 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e63896700 1 Processor -- start 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e63896700 1 -- start start 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e63896700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 0x7f1e5c0fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e63896700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5c0fe810 0x7f1e5c0fc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e63896700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0fdbd0 con 0x7f1e5c0fde70 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e63896700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e5c0fc720 con 0x7f1e5c0fe810 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e61632700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 0x7f1e5c0fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e61632700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 0x7f1e5c0fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:58784/0 (socket says 172.21.15.33:58784) 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e61632700 1 -- 172.21.15.33:0/2458094514 learned_addr learned my addr 172.21.15.33:0/2458094514 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:44.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e60e31700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5c0fe810 0x7f1e5c0fc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e61632700 1 -- 172.21.15.33:0/2458094514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5c0fe810 msgr2=0x7f1e5c0fc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e61632700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5c0fe810 0x7f1e5c0fc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e61632700 1 -- 172.21.15.33:0/2458094514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e50005040 con 0x7f1e5c0fde70 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.723+0000 7f1e60e31700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5c0fe810 0x7f1e5c0fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.724+0000 7f1e61632700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 0x7f1e5c0fbca0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1e50004890 tx=0x7f1e50009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.724+0000 7f1e4dffb700 1 -- 172.21.15.33:0/2458094514 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e5000e040 con 0x7f1e5c0fde70 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.724+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e5c0fc9a0 con 0x7f1e5c0fde70 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.724+0000 7f1e4dffb700 1 -- 172.21.15.33:0/2458094514 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e50009af0 con 0x7f1e5c0fde70 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.724+0000 7f1e4dffb700 1 -- 172.21.15.33:0/2458094514 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e5001c630 con 0x7f1e5c0fde70 2024-04-25T02:17:44.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.724+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e5c119f60 con 0x7f1e5c0fde70 2024-04-25T02:17:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.725+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e5c066ad0 con 0x7f1e5c0fde70 2024-04-25T02:17:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.726+0000 7f1e4dffb700 1 -- 172.21.15.33:0/2458094514 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1e5000bbc0 con 0x7f1e5c0fde70 2024-04-25T02:17:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.726+0000 7f1e4dffb700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4806e140 0x7f1e480705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.726+0000 7f1e4dffb700 1 -- 172.21.15.33:0/2458094514 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f1e5008ee30 con 0x7f1e5c0fde70 2024-04-25T02:17:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.726+0000 7f1e60e31700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4806e140 0x7f1e480705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:44.731 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.727+0000 7f1e60e31700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4806e140 0x7f1e480705f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009b80 tx=0x7f1e5800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:44.735 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:44.731+0000 7f1e4dffb700 1 -- 172.21.15.33:0/2458094514 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1e50058230 con 0x7f1e5c0fde70 2024-04-25T02:17:44.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:44 smithi138 ceph-mon[81304]: pgmap v1153: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:17:45.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.028+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1e5c11a150 con 0x7f1e5c0fde70 2024-04-25T02:17:45.033 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.029+0000 7f1e4dffb700 1 -- 172.21.15.33:0/2458094514 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f1e5005b8f0 con 0x7f1e5c0fde70 2024-04-25T02:17:45.033 INFO:teuthology.orchestra.run.smithi033.stdout:{ 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "mon": { 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "mgr": { 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 2 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "osd": { 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 6 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "mds": { 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 4 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: }, 2024-04-25T02:17:45.034 INFO:teuthology.orchestra.run.smithi033.stdout: "overall": { 2024-04-25T02:17:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)": 14 2024-04-25T02:17:45.035 INFO:teuthology.orchestra.run.smithi033.stdout: } 2024-04-25T02:17:45.035 INFO:teuthology.orchestra.run.smithi033.stdout:} 2024-04-25T02:17:45.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4806e140 msgr2=0x7f1e480705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:45.035 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4806e140 0x7f1e480705f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1e58009b80 tx=0x7f1e5800b040 comp rx=0 tx=0).stop 2024-04-25T02:17:45.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 msgr2=0x7f1e5c0fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:45.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 0x7f1e5c0fbca0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1e50004890 tx=0x7f1e50009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:45.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 shutdown_connections 2024-04-25T02:17:45.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f1e4806e140 0x7f1e480705f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:45.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1e5c0fde70 0x7f1e5c0fbca0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:45.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 --2- 172.21.15.33:0/2458094514 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1e5c0fe810 0x7f1e5c0fc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:45.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 >> 172.21.15.33:0/2458094514 conn(0x7f1e5c0f78e0 msgr2=0x7f1e5c1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:45.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 shutdown_connections 2024-04-25T02:17:45.036 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:45.031+0000 7f1e63896700 1 -- 172.21.15.33:0/2458094514 wait complete. 2024-04-25T02:17:45.670 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:45 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2458094514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:17:45.699 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-25T02:17:45.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:45 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2458094514' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:17:45.968 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:17:46.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:46 smithi033 ceph-mon[111306]: pgmap v1154: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:46.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:46 smithi138 ceph-mon[81304]: pgmap v1154: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.570+0000 7f8596e66700 1 -- 172.21.15.33:0/3871013338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 msgr2=0x7f8590065ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.570+0000 7f8596e66700 1 --2- 172.21.15.33:0/3871013338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 0x7f8590065ab0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8584004940 tx=0x7f8584009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:47.576 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.570+0000 7f8596e66700 1 -- 172.21.15.33:0/3871013338 shutdown_connections 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.570+0000 7f8596e66700 1 --2- 172.21.15.33:0/3871013338 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 0x7f8590065ab0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.570+0000 7f8596e66700 1 --2- 172.21.15.33:0/3871013338 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8590102e50 0x7f8590103220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.570+0000 7f8596e66700 1 -- 172.21.15.33:0/3871013338 >> 172.21.15.33:0/3871013338 conn(0x7f8590071d00 msgr2=0x7f8590072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.570+0000 7f8596e66700 1 -- 172.21.15.33:0/3871013338 shutdown_connections 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.570+0000 7f8596e66700 1 -- 172.21.15.33:0/3871013338 wait complete. 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.571+0000 7f8596e66700 1 Processor -- start 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.571+0000 7f8596e66700 1 -- start start 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.571+0000 7f8596e66700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 0x7f85900fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.571+0000 7f8596e66700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8590102e50 0x7f85900fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.571+0000 7f8596e66700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85900fc880 con 0x7f8590065640 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.571+0000 7f8596e66700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85900fc9c0 con 0x7f8590102e50 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.571+0000 7f8594c02700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 0x7f85900fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:47.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f8594c02700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 0x7f85900fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44928/0 (socket says 172.21.15.33:44928) 2024-04-25T02:17:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f8594c02700 1 -- 172.21.15.33:0/363144890 learned_addr learned my addr 172.21.15.33:0/363144890 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f858ffff700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8590102e50 0x7f85900fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f8594c02700 1 -- 172.21.15.33:0/363144890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8590102e50 msgr2=0x7f85900fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f8594c02700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8590102e50 0x7f85900fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f8594c02700 1 -- 172.21.15.33:0/363144890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8584005040 con 0x7f8590065640 2024-04-25T02:17:47.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f8594c02700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 0x7f85900fbdb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f858800cb10 tx=0x7f8588007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f858d7fa700 1 -- 172.21.15.33:0/363144890 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8588004200 con 0x7f8590065640 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f858d7fa700 1 -- 172.21.15.33:0/363144890 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f858800ac70 con 0x7f8590065640 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8590119e70 con 0x7f8590065640 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f858d7fa700 1 -- 172.21.15.33:0/363144890 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8588003d30 con 0x7f8590065640 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.572+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859011a2f0 con 0x7f8590065640 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.573+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8590100100 con 0x7f8590065640 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.574+0000 7f858d7fa700 1 -- 172.21.15.33:0/363144890 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f858800a7a0 con 0x7f8590065640 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.574+0000 7f858d7fa700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f857806e150 0x7f8578070600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.574+0000 7f858d7fa700 1 -- 172.21.15.33:0/363144890 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f858808dfc0 con 0x7f8590065640 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.574+0000 7f858ffff700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f857806e150 0x7f8578070600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:47.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.575+0000 7f858ffff700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f857806e150 0x7f8578070600 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f85900fd6f0 tx=0x7f8584009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:47.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.579+0000 7f858d7fa700 1 -- 172.21.15.33:0/363144890 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f85880c2740 con 0x7f8590065640 2024-04-25T02:17:47.879 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.875+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f859004bed0 con 0x7f8590065640 2024-04-25T02:17:47.880 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.875+0000 7f858d7fa700 1 -- 172.21.15.33:0/363144890 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f8588057340 con 0x7f8590065640 2024-04-25T02:17:47.882 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f857806e150 msgr2=0x7f8578070600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f857806e150 0x7f8578070600 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f85900fd6f0 tx=0x7f8584009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 msgr2=0x7f85900fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 0x7f85900fbdb0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f858800cb10 tx=0x7f8588007f90 comp rx=0 tx=0).stop 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 shutdown_connections 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f857806e150 0x7f8578070600 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8590065640 0x7f85900fbdb0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 --2- 172.21.15.33:0/363144890 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8590102e50 0x7f85900fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 >> 172.21.15.33:0/363144890 conn(0x7f8590071d00 msgr2=0x7f85900fb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 shutdown_connections 2024-04-25T02:17:47.883 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:47.878+0000 7f8596e66700 1 -- 172.21.15.33:0/363144890 wait complete. 2024-04-25T02:17:47.894 INFO:teuthology.orchestra.run.smithi033.stdout:true 2024-04-25T02:17:48.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:48 smithi033 ceph-mon[111306]: pgmap v1155: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:48.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:48 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/363144890' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:17:48.549 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -e sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-25T02:17:48.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:48 smithi138 ceph-mon[81304]: pgmap v1155: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:48.796 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:48 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/363144890' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:17:48.821 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:17:49.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:49 smithi138 ceph-mon[81304]: pgmap v1156: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:49.893 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:49 smithi033 ceph-mon[111306]: pgmap v1156: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.460+0000 7f46338d3700 1 -- 172.21.15.33:0/3041074549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c103780 msgr2=0x7f462c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.460+0000 7f46338d3700 1 --2- 172.21.15.33:0/3041074549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c103780 0x7f462c0fcdf0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f461c004940 tx=0x7f461c009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.460+0000 7f46338d3700 1 -- 172.21.15.33:0/3041074549 shutdown_connections 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.460+0000 7f46338d3700 1 --2- 172.21.15.33:0/3041074549 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c103780 0x7f462c0fcdf0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.460+0000 7f46338d3700 1 --2- 172.21.15.33:0/3041074549 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c102de0 0x7f462c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.460+0000 7f46338d3700 1 -- 172.21.15.33:0/3041074549 >> 172.21.15.33:0/3041074549 conn(0x7f462c0f8990 msgr2=0x7f462c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.461+0000 7f46338d3700 1 -- 172.21.15.33:0/3041074549 shutdown_connections 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.461+0000 7f46338d3700 1 -- 172.21.15.33:0/3041074549 wait complete. 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.461+0000 7f46338d3700 1 Processor -- start 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.461+0000 7f46338d3700 1 -- start start 2024-04-25T02:17:50.472 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.461+0000 7f46338d3700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c102de0 0x7f462c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.461+0000 7f46338d3700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c103780 0x7f462c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.461+0000 7f46338d3700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c114520 con 0x7f462c102de0 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.461+0000 7f46338d3700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462c10db20 con 0x7f462c103780 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.461+0000 7f463166f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c102de0 0x7f462c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f463166f700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c102de0 0x7f462c113990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44948/0 (socket says 172.21.15.33:44948) 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f463166f700 1 -- 172.21.15.33:0/224630701 learned_addr learned my addr 172.21.15.33:0/224630701 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f4630e6e700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c103780 0x7f462c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f463166f700 1 -- 172.21.15.33:0/224630701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c103780 msgr2=0x7f462c113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f463166f700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c103780 0x7f462c113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f463166f700 1 -- 172.21.15.33:0/224630701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c005040 con 0x7f462c102de0 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f463166f700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c102de0 0x7f462c113990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f462800c830 tx=0x7f4628004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f4621ffb700 1 -- 172.21.15.33:0/224630701 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46280106f0 con 0x7f462c102de0 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f4621ffb700 1 -- 172.21.15.33:0/224630701 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4628010d20 con 0x7f462c102de0 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462c10de00 con 0x7f462c102de0 2024-04-25T02:17:50.473 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f4621ffb700 1 -- 172.21.15.33:0/224630701 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46280113a0 con 0x7f462c102de0 2024-04-25T02:17:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.462+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c10e390 con 0x7f462c102de0 2024-04-25T02:17:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.463+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f462c0fe530 con 0x7f462c102de0 2024-04-25T02:17:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.464+0000 7f4621ffb700 1 -- 172.21.15.33:0/224630701 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4628012400 con 0x7f462c102de0 2024-04-25T02:17:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.464+0000 7f4621ffb700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f461806e140 0x7f46180705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.464+0000 7f4621ffb700 1 -- 172.21.15.33:0/224630701 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f462808da70 con 0x7f462c102de0 2024-04-25T02:17:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.464+0000 7f4630e6e700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f461806e140 0x7f46180705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.465+0000 7f4630e6e700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f461806e140 0x7f46180705f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f462c10ee00 tx=0x7f461c009e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:50.474 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.469+0000 7f4621ffb700 1 -- 172.21.15.33:0/224630701 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f462805a9a0 con 0x7f462c102de0 2024-04-25T02:17:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.765+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f462c109f10 con 0x7f462c102de0 2024-04-25T02:17:50.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.766+0000 7f4621ffb700 1 -- 172.21.15.33:0/224630701 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f462805a530 con 0x7f462c102de0 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f461806e140 msgr2=0x7f46180705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f461806e140 0x7f46180705f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f462c10ee00 tx=0x7f461c009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c102de0 msgr2=0x7f462c113990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c102de0 0x7f462c113990 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f462800c830 tx=0x7f4628004470 comp rx=0 tx=0).stop 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 shutdown_connections 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f461806e140 0x7f46180705f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f462c102de0 0x7f462c113990 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 --2- 172.21.15.33:0/224630701 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f462c103780 0x7f462c113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 >> 172.21.15.33:0/224630701 conn(0x7f462c0f8990 msgr2=0x7f462c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 shutdown_connections 2024-04-25T02:17:50.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:50.768+0000 7f46338d3700 1 -- 172.21.15.33:0/224630701 wait complete. 2024-04-25T02:17:50.783 INFO:teuthology.orchestra.run.smithi033.stdout: "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)" 2024-04-25T02:17:51.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:50 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/224630701' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:17:51.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:50 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/224630701' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T02:17:51.431 DEBUG:teuthology.parallel:result is None 2024-04-25T02:17:51.432 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T02:17:51.443 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T02:17:51.443 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T02:17:51.715 INFO:teuthology.orchestra.run.smithi033.stderr:Inferring config /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/mon.smithi033/config 2024-04-25T02:17:52.097 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:51 smithi033 ceph-mon[111306]: pgmap v1157: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:52.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:51 smithi138 ceph-mon[81304]: pgmap v1157: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:17:53.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.235+0000 7fef71489700 1 -- 172.21.15.33:0/2627517022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 msgr2=0x7fef6c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.235+0000 7fef71489700 1 --2- 172.21.15.33:0/2627517022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 0x7fef6c0fcdf0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fef60004940 tx=0x7fef60009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.235+0000 7fef71489700 1 -- 172.21.15.33:0/2627517022 shutdown_connections 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.235+0000 7fef71489700 1 --2- 172.21.15.33:0/2627517022 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 0x7fef6c0fcdf0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.235+0000 7fef71489700 1 --2- 172.21.15.33:0/2627517022 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c102de0 0x7fef6c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.235+0000 7fef71489700 1 -- 172.21.15.33:0/2627517022 >> 172.21.15.33:0/2627517022 conn(0x7fef6c0f8990 msgr2=0x7fef6c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.235+0000 7fef71489700 1 -- 172.21.15.33:0/2627517022 shutdown_connections 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.235+0000 7fef71489700 1 -- 172.21.15.33:0/2627517022 wait complete. 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef71489700 1 Processor -- start 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef71489700 1 -- start start 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef71489700 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c102de0 0x7fef6c113990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef71489700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 0x7fef6c113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef71489700 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c114520 con 0x7fef6c103780 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef71489700 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c10db20 con 0x7fef6c102de0 2024-04-25T02:17:53.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef6a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 0x7fef6c113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:53.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef6a7fc700 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 0x7fef6c113ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44968/0 (socket says 172.21.15.33:44968) 2024-04-25T02:17:53.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef6a7fc700 1 -- 172.21.15.33:0/3633557265 learned_addr learned my addr 172.21.15.33:0/3633557265 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:53.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef6affd700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c102de0 0x7fef6c113990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:53.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef6a7fc700 1 -- 172.21.15.33:0/3633557265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c102de0 msgr2=0x7fef6c113990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:53.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef6a7fc700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c102de0 0x7fef6c113990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:53.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef6a7fc700 1 -- 172.21.15.33:0/3633557265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef60005040 con 0x7fef6c103780 2024-04-25T02:17:53.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef6affd700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c102de0 0x7fef6c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:17:53.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.236+0000 7fef6a7fc700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 0x7fef6c113ed0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fef60009100 tx=0x7fef60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:53.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.237+0000 7fef537fe700 1 -- 172.21.15.33:0/3633557265 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6000e040 con 0x7fef6c103780 2024-04-25T02:17:53.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.237+0000 7fef537fe700 1 -- 172.21.15.33:0/3633557265 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef60013420 con 0x7fef6c103780 2024-04-25T02:17:53.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.237+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef6c10dda0 con 0x7fef6c103780 2024-04-25T02:17:53.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.237+0000 7fef537fe700 1 -- 172.21.15.33:0/3633557265 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6001c610 con 0x7fef6c103780 2024-04-25T02:17:53.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.237+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c10e360 con 0x7fef6c103780 2024-04-25T02:17:53.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.238+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c0fe530 con 0x7fef6c103780 2024-04-25T02:17:53.243 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.238+0000 7fef537fe700 1 -- 172.21.15.33:0/3633557265 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fef60013590 con 0x7fef6c103780 2024-04-25T02:17:53.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.239+0000 7fef537fe700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef5406e140 0x7fef540705f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:53.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.239+0000 7fef537fe700 1 -- 172.21.15.33:0/3633557265 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fef6008ee60 con 0x7fef6c103780 2024-04-25T02:17:53.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.239+0000 7fef6affd700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef5406e140 0x7fef540705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:53.244 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.239+0000 7fef6affd700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef5406e140 0x7fef540705f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0041d0 tx=0x7fef5c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:53.248 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.244+0000 7fef537fe700 1 -- 172.21.15.33:0/3633557265 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fef6005bcf0 con 0x7fef6c103780 2024-04-25T02:17:53.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.496+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fef6c109ec0 con 0x7fef6c103780 2024-04-25T02:17:53.501 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.496+0000 7fef537fe700 1 -- 172.21.15.33:0/3633557265 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 38 v38) v1 ==== 76+0+1967 (secure 0 0 0) 0x7fef6005b880 con 0x7fef6c103780 2024-04-25T02:17:53.501 INFO:teuthology.orchestra.run.smithi033.stdout:e38 2024-04-25T02:17:53.501 INFO:teuthology.orchestra.run.smithi033.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T02:17:53.501 INFO:teuthology.orchestra.run.smithi033.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T02:17:53.501 INFO:teuthology.orchestra.run.smithi033.stdout:legacy client fscid: 1 2024-04-25T02:17:53.501 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:53.501 INFO:teuthology.orchestra.run.smithi033.stdout:Filesystem 'cephfs' (1) 2024-04-25T02:17:53.501 INFO:teuthology.orchestra.run.smithi033.stdout:fs_name cephfs 2024-04-25T02:17:53.501 INFO:teuthology.orchestra.run.smithi033.stdout:epoch 38 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:created 2024-04-25T01:32:56.470326+0000 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:modified 2024-04-25T02:16:16.512021+0000 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:tableserver 0 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:root 0 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:session_timeout 60 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:session_autoclose 300 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:max_file_size 1099511627776 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:max_xattr_size 65536 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:required_client_features {} 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure 0 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:last_failure_osd_epoch 80 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T02:17:53.502 INFO:teuthology.orchestra.run.smithi033.stdout:max_mds 2 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:in 0,1 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:up {0=34414,1=34422} 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:failed 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:damaged 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:stopped 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:data_pools [3] 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:metadata_pool 2 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:inline_data disabled 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:balancer 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:bal_rank_mask -1 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:standby_count_wanted 1 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:qdb_cluster leader: 34414 members: 34422,34414 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.qqhhcz{0:34414} state up:active seq 468 join_fscid=1 addr [v2:172.21.15.33:6826/1943122938,v1:172.21.15.33:6827/1943122938] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.jwzxlz{0:44327} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.138:6824/165236762,v1:172.21.15.138:6825/165236762] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi033.hobkqr{1:34422} state up:active seq 463 join_fscid=1 addr [v2:172.21.15.33:6828/2150451637,v1:172.21.15.33:6829/2150451637] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:17:53.503 INFO:teuthology.orchestra.run.smithi033.stdout:[mds.cephfs.smithi138.rvxnby{1:44841} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.138:6826/882099891,v1:172.21.15.138:6827/882099891] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T02:17:53.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:53.504 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:53.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef5406e140 msgr2=0x7fef540705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:53.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef5406e140 0x7fef540705f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0041d0 tx=0x7fef5c009040 comp rx=0 tx=0).stop 2024-04-25T02:17:53.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 msgr2=0x7fef6c113ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 0x7fef6c113ed0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fef60009100 tx=0x7fef60009e30 comp rx=0 tx=0).stop 2024-04-25T02:17:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 shutdown_connections 2024-04-25T02:17:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fef5406e140 0x7fef540705f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fef6c102de0 0x7fef6c113990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 --2- 172.21.15.33:0/3633557265 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fef6c103780 0x7fef6c113ed0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 >> 172.21.15.33:0/3633557265 conn(0x7fef6c0f8990 msgr2=0x7fef6c0f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 shutdown_connections 2024-04-25T02:17:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:53.499+0000 7fef71489700 1 -- 172.21.15.33:0/3633557265 wait complete. 2024-04-25T02:17:53.505 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 38 2024-04-25T02:17:54.157 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:53 smithi033 ceph-mon[111306]: pgmap v1158: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:17:54.157 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:53 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3633557265' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:17:54.186 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-25T02:17:54.197 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T02:17:54.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:53 smithi138 ceph-mon[81304]: pgmap v1158: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:17:54.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:53 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3633557265' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T02:17:54.303 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3103691238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 msgr2=0x7fd738102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 --2- 172.21.15.33:0/3103691238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 0x7fd738102560 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fd7280048d0 tx=0x7fd728009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3103691238 shutdown_connections 2024-04-25T02:17:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 --2- 172.21.15.33:0/3103691238 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd738102b30 0x7fd7380fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 --2- 172.21.15.33:0/3103691238 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 0x7fd738102560 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3103691238 >> 172.21.15.33:0/3103691238 conn(0x7fd7380f7d00 msgr2=0x7fd7380fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3103691238 shutdown_connections 2024-04-25T02:17:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3103691238 wait complete. 2024-04-25T02:17:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 Processor -- start 2024-04-25T02:17:54.304 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.299+0000 7fd73fc2e640 1 -- start start 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73fc2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 0x7fd738112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73fc2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd738102b30 0x7fd738113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73fc2e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd738113810 con 0x7fd738102190 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73fc2e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd73810ce60 con 0x7fd738102b30 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73d9ca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 0x7fd738112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73d9ca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 0x7fd738112cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:44998/0 (socket says 172.21.15.33:44998) 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73d9ca640 1 -- 172.21.15.33:0/3850881425 learned_addr learned my addr 172.21.15.33:0/3850881425 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73d1c9640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd738102b30 0x7fd738113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73d9ca640 1 -- 172.21.15.33:0/3850881425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd738102b30 msgr2=0x7fd738113210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73d9ca640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd738102b30 0x7fd738113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73d9ca640 1 -- 172.21.15.33:0/3850881425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd728005040 con 0x7fd738102190 2024-04-25T02:17:54.306 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73d9ca640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 0x7fd738112cd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd728004930 tx=0x7fd728009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd72e7fc640 1 -- 172.21.15.33:0/3850881425 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7280094c0 con 0x7fd738102190 2024-04-25T02:17:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd72e7fc640 1 -- 172.21.15.33:0/3850881425 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd728032e50 con 0x7fd738102190 2024-04-25T02:17:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd73810d0e0 con 0x7fd738102190 2024-04-25T02:17:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.300+0000 7fd72e7fc640 1 -- 172.21.15.33:0/3850881425 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7280433b0 con 0x7fd738102190 2024-04-25T02:17:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.301+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd73810d5a0 con 0x7fd738102190 2024-04-25T02:17:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.301+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7380fd900 con 0x7fd738102190 2024-04-25T02:17:54.307 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.302+0000 7fd72e7fc640 1 -- 172.21.15.33:0/3850881425 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd72802f070 con 0x7fd738102190 2024-04-25T02:17:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.302+0000 7fd72e7fc640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd72406e0b0 0x7fd724070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.302+0000 7fd73d1c9640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd72406e0b0 0x7fd724070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.302+0000 7fd72e7fc640 1 -- 172.21.15.33:0/3850881425 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd7280af9a0 con 0x7fd738102190 2024-04-25T02:17:54.308 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.303+0000 7fd73d1c9640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd72406e0b0 0x7fd724070560 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd734009af0 tx=0x7fd73400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:54.311 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.307+0000 7fd72e7fc640 1 -- 172.21.15.33:0/3850881425 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd728078d20 con 0x7fd738102190 2024-04-25T02:17:54.496 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.492+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd73804bcc0 con 0x7fd738102190 2024-04-25T02:17:54.497 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.493+0000 7fd72e7fc640 1 -- 172.21.15.33:0/3850881425 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16530 (secure 0 0 0) 0x7fd72807c3e0 con 0x7fd738102190 2024-04-25T02:17:54.498 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:54.498 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","created":"2024-04-25T01:26:05.674330+0000","modified":"2024-04-25T02:01:00.954060+0000","last_up_change":"2024-04-25T01:45:08.141024+0000","last_in_change":"2024-04-25T01:31:20.199327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:30:26.558656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-25T01:32:55.402156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-25T01:32:55.894097+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68af93d4-7aa8-485f-826e-62a6e362907b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6803","nonce":3103417098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6805","nonce":3103417098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6809","nonce":3103417098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6807","nonce":3103417098}]},"public_addr":"172.21.15.33:6803/3103417098","cluster_addr":"172.21.15.33:6805/3103417098","heartbeat_back_addr":"172.21.15.33:6809/3103417098","heartbeat_front_addr":"172.21.15.33:6807/3103417098","state":["exists","up"]},{"osd":1,"uuid":"1fdb7e9b-1138-43db-a11f-3e30e286c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6811","nonce":166114729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6813","nonce":166114729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6817","nonce":166114729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6815","nonce":166114729}]},"public_addr":"172.21.15.33:6811/166114729","cluster_addr":"172.21.15.33:6813/166114729","heartbeat_back_addr":"172.21.15.33:6817/166114729","heartbeat_front_addr":"172.21.15.33:6815/166114729","state":["exists","up"]},{"osd":2,"uuid":"b1239da3-566e-4df0-b5ea-df4c14959837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6819","nonce":4078679342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6821","nonce":4078679342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6825","nonce":4078679342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6823","nonce":4078679342}]},"public_addr":"172.21.15.33:6819/4078679342","cluster_addr":"172.21.15.33:6821/4078679342","heartbeat_back_addr":"172.21.15.33:6825/4078679342","heartbeat_front_addr":"172.21.15.33:6823/4078679342","state":["exists","up"]},{"osd":3,"uuid":"abc58749-77ca-4476-9601-cdf89d3c1e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6801","nonce":1157003839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6803","nonce":1157003839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6807","nonce":1157003839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6805","nonce":1157003839}]},"public_addr":"172.21.15.138:6801/1157003839","cluster_addr":"172.21.15.138:6803/1157003839","heartbeat_back_addr":"172.21.15.138:6807/1157003839","heartbeat_front_addr":"172.21.15.138:6805/1157003839","state":["exists","up"]},{"osd":4,"uuid":"83e421bd-52ba-47ad-8755-e1354f502ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6809","nonce":4024012885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6811","nonce":4024012885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6815","nonce":4024012885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6813","nonce":4024012885}]},"public_addr":"172.21.15.138:6809/4024012885","cluster_addr":"172.21.15.138:6811/4024012885","heartbeat_back_addr":"172.21.15.138:6815/4024012885","heartbeat_front_addr":"172.21.15.138:6813/4024012885","state":["exists","up"]},{"osd":5,"uuid":"266e5984-f087-408e-a84c-04f81f33af43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6817","nonce":2521547597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6819","nonce":2521547597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6823","nonce":2521547597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6821","nonce":2521547597}]},"public_addr":"172.21.15.138:6817/2521547597","cluster_addr":"172.21.15.138:6819/2521547597","heartbeat_back_addr":"172.21.15.138:6823/2521547597","heartbeat_front_addr":"172.21.15.138:6821/2521547597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:27.636323+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:55.618887+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:23.307858+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:47.331937+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:12.159932+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:37.317499+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6825/3887358693":"2024-04-26T01:45:10.323297+0000","172.21.15.33:6800/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:6800/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6800/1387801224":"2024-04-26T01:35:34.311733+0000","172.21.15.33:0/1869074265":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/2062772238":"2024-04-26T01:35:34.311733+0000","172.21.15.33:6801/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.138:0/4174594849":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/3334291936":"2024-04-26T01:35:34.311733+0000","172.21.15.138:6829/1792929713":"2024-04-26T01:37:35.038935+0000","172.21.15.33:6800/1731533099":"2024-04-26T01:40:40.186857+0000","172.21.15.33:6801/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/1731533099":"2024-04-26T01:40:40.186857+0000","172.21.15.33:6800/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/4160574345":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6826/3303942324":"2024-04-26T01:45:10.323282+0000","172.21.15.33:0/2566719862":"2024-04-26T01:28:16.045570+0000","172.21.15.138:0/330154225":"2024-04-26T01:37:35.038935+0000","172.21.15.33:6801/1387801224":"2024-04-26T01:35:34.311733+0000","172.21.15.138:0/219235313":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/454449040":"2024-04-26T01:35:34.311733+0000","172.21.15.33:0/1401659678":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2117172792":"2024-04-26T01:26:40.396561+0000","172.21.15.138:6828/1792929713":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/2112653252":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6801/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1131828704":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/751781866":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/686286320":"2024-04-26T01:26:40.396561+0000","172.21.15.138:0/1137962382":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/746151230":"2024-04-26T01:40:40.186857+0000","172.21.15.33:0/3951183717":"2024-04-26T01:40:40.186857+0000","172.21.15.33:0/1718672295":"2024-04-26T01:40:40.186857+0000","172.21.15.33:0/2952222145":"2024-04-26T01:35:34.311733+0000","172.21.15.33:0/3161221115":"2024-04-26T01:40:40.186857+0000","172.21.15.33:6827/3303942324":"2024-04-26T01:45:10.323282+0000","172.21.15.138:6824/3887358693":"2024-04-26T01:45:10.323297+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd72406e0b0 msgr2=0x7fd724070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd72406e0b0 0x7fd724070560 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd734009af0 tx=0x7fd73400b040 comp rx=0 tx=0).stop 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 msgr2=0x7fd738112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 0x7fd738112cd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fd728004930 tx=0x7fd728009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 shutdown_connections 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd72406e0b0 0x7fd724070560 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd738102b30 0x7fd738113210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 --2- 172.21.15.33:0/3850881425 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd738102190 0x7fd738112cd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 >> 172.21.15.33:0/3850881425 conn(0x7fd7380f7d00 msgr2=0x7fd7380f8d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 shutdown_connections 2024-04-25T02:17:54.500 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.495+0000 7fd73fc2e640 1 -- 172.21.15.33:0/3850881425 wait complete. 2024-04-25T02:17:54.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:30:26.558656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-25T01:32:55.402156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-25T01:32:55.894097+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T02:17:54.512 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 -- 172.21.15.33:0/3452247499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 msgr2=0x7f94b8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 --2- 172.21.15.33:0/3452247499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 0x7f94b8101460 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f94b00048d0 tx=0x7f94b0009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 -- 172.21.15.33:0/3452247499 shutdown_connections 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 --2- 172.21.15.33:0/3452247499 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94b8101a30 0x7f94b80fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 --2- 172.21.15.33:0/3452247499 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 0x7f94b8101460 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 -- 172.21.15.33:0/3452247499 >> 172.21.15.33:0/3452247499 conn(0x7f94b80f6c20 msgr2=0x7f94b80f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 -- 172.21.15.33:0/3452247499 shutdown_connections 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 -- 172.21.15.33:0/3452247499 wait complete. 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 Processor -- start 2024-04-25T02:17:54.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 -- start start 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.609+0000 7f94bce14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 0x7f94b8112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94bce14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94b8101a30 0x7f94b8113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94bce14640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b8113820 con 0x7f94b8101090 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94bce14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94b810ce70 con 0x7f94b8101a30 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94b659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 0x7f94b8112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94b659c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 0x7f94b8112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45022/0 (socket says 172.21.15.33:45022) 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94b659c640 1 -- 172.21.15.33:0/430394675 learned_addr learned my addr 172.21.15.33:0/430394675 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94b5d9b640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94b8101a30 0x7f94b8113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94b659c640 1 -- 172.21.15.33:0/430394675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94b8101a30 msgr2=0x7f94b8113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94b659c640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94b8101a30 0x7f94b8113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94b659c640 1 -- 172.21.15.33:0/430394675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0005040 con 0x7f94b8101090 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94b659c640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 0x7f94b8112ce0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f94b0004930 tx=0x7f94b0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94a6ffd640 1 -- 172.21.15.33:0/430394675 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b00094c0 con 0x7f94b8101090 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94b810d0f0 con 0x7f94b8101090 2024-04-25T02:17:54.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94a6ffd640 1 -- 172.21.15.33:0/430394675 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94b0032e50 con 0x7f94b8101090 2024-04-25T02:17:54.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94a6ffd640 1 -- 172.21.15.33:0/430394675 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b00433b0 con 0x7f94b8101090 2024-04-25T02:17:54.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.610+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94b810d5b0 con 0x7f94b8101090 2024-04-25T02:17:54.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.611+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94b80fc800 con 0x7f94b8101090 2024-04-25T02:17:54.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.612+0000 7f94a6ffd640 1 -- 172.21.15.33:0/430394675 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f94b002f070 con 0x7f94b8101090 2024-04-25T02:17:54.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.612+0000 7f94a6ffd640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94a006e0b0 0x7f94a0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:54.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.612+0000 7f94b5d9b640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94a006e0b0 0x7f94a0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:54.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.612+0000 7f94a6ffd640 1 -- 172.21.15.33:0/430394675 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f94b00afb70 con 0x7f94b8101090 2024-04-25T02:17:54.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.613+0000 7f94b5d9b640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94a006e0b0 0x7f94a0070560 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f94b810e5a0 tx=0x7f94ac009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:54.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.617+0000 7f94a6ffd640 1 -- 172.21.15.33:0/430394675 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f94b0078ef0 con 0x7f94b8101090 2024-04-25T02:17:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.802+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94b810e350 con 0x7f94b8101090 2024-04-25T02:17:54.806 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.802+0000 7f94a6ffd640 1 -- 172.21.15.33:0/430394675 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f94b007c5b0 con 0x7f94b8101090 2024-04-25T02:17:54.807 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T02:17:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94a006e0b0 msgr2=0x7f94a0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94a006e0b0 0x7f94a0070560 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f94b810e5a0 tx=0x7f94ac009040 comp rx=0 tx=0).stop 2024-04-25T02:17:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 msgr2=0x7f94b8112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 0x7f94b8112ce0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f94b0004930 tx=0x7f94b0009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 shutdown_connections 2024-04-25T02:17:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f94a006e0b0 0x7f94a0070560 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f94b8101a30 0x7f94b8113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 --2- 172.21.15.33:0/430394675 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f94b8101090 0x7f94b8112ce0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 >> 172.21.15.33:0/430394675 conn(0x7f94b80f6c20 msgr2=0x7f94b80f7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:54.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 shutdown_connections 2024-04-25T02:17:54.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.804+0000 7f94bce14640 1 -- 172.21.15.33:0/430394675 wait complete. 2024-04-25T02:17:54.819 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.917+0000 7f6218e21640 1 -- 172.21.15.33:0/2366740372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 msgr2=0x7f6214064750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.917+0000 7f6218e21640 1 --2- 172.21.15.33:0/2366740372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 0x7f6214064750 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f62040048d0 tx=0x7f6204009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.917+0000 7f6218e21640 1 -- 172.21.15.33:0/2366740372 shutdown_connections 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.917+0000 7f6218e21640 1 --2- 172.21.15.33:0/2366740372 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6214064c90 0x7f6214108750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.917+0000 7f6218e21640 1 --2- 172.21.15.33:0/2366740372 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 0x7f6214064750 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.917+0000 7f6218e21640 1 -- 172.21.15.33:0/2366740372 >> 172.21.15.33:0/2366740372 conn(0x7f62140710c0 msgr2=0x7f62140714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.917+0000 7f6218e21640 1 -- 172.21.15.33:0/2366740372 shutdown_connections 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.917+0000 7f6218e21640 1 -- 172.21.15.33:0/2366740372 wait complete. 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f6218e21640 1 Processor -- start 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f6218e21640 1 -- start start 2024-04-25T02:17:54.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f6218e21640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 0x7f6214114f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f6218e21640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6214064c90 0x7f6214115440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f6218e21640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6214115ad0 con 0x7f6214064380 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f6218e21640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621410f190 con 0x7f6214064c90 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f621259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 0x7f6214114f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f621259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 0x7f6214114f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45050/0 (socket says 172.21.15.33:45050) 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f621259c640 1 -- 172.21.15.33:0/150832450 learned_addr learned my addr 172.21.15.33:0/150832450 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.918+0000 7f620bfff640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6214064c90 0x7f6214115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.919+0000 7f621259c640 1 -- 172.21.15.33:0/150832450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6214064c90 msgr2=0x7f6214115440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.919+0000 7f621259c640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6214064c90 0x7f6214115440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.919+0000 7f621259c640 1 -- 172.21.15.33:0/150832450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6204005040 con 0x7f6214064380 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.919+0000 7f621259c640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 0x7f6214114f00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6204004930 tx=0x7f6204009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.919+0000 7f620affd640 1 -- 172.21.15.33:0/150832450 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62040094c0 con 0x7f6214064380 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.919+0000 7f620affd640 1 -- 172.21.15.33:0/150832450 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6204032e50 con 0x7f6214064380 2024-04-25T02:17:54.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.919+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621410f410 con 0x7f6214064380 2024-04-25T02:17:54.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.919+0000 7f620affd640 1 -- 172.21.15.33:0/150832450 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62040433b0 con 0x7f6214064380 2024-04-25T02:17:54.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.919+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621410f8d0 con 0x7f6214064380 2024-04-25T02:17:54.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.920+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62140fb700 con 0x7f6214064380 2024-04-25T02:17:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.921+0000 7f620affd640 1 -- 172.21.15.33:0/150832450 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f620402f070 con 0x7f6214064380 2024-04-25T02:17:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.921+0000 7f620affd640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f620006e0b0 0x7f6200070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.921+0000 7f620affd640 1 -- 172.21.15.33:0/150832450 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f62040af9a0 con 0x7f6214064380 2024-04-25T02:17:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.921+0000 7f620bfff640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f620006e0b0 0x7f6200070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:54.927 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.921+0000 7f620bfff640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f620006e0b0 0x7f6200070560 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f61fc006fd0 tx=0x7f61fc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:54.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:54.926+0000 7f620affd640 1 -- 172.21.15.33:0/150832450 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6204078d20 con 0x7f6214064380 2024-04-25T02:17:55.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.110+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f621404bcc0 con 0x7f6214064380 2024-04-25T02:17:55.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.110+0000 7f620affd640 1 -- 172.21.15.33:0/150832450 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f620407c3e0 con 0x7f6214064380 2024-04-25T02:17:55.115 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T02:17:55.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f620006e0b0 msgr2=0x7f6200070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f620006e0b0 0x7f6200070560 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f61fc006fd0 tx=0x7f61fc008040 comp rx=0 tx=0).stop 2024-04-25T02:17:55.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 msgr2=0x7f6214114f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 0x7f6214114f00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6204004930 tx=0x7f6204009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 shutdown_connections 2024-04-25T02:17:55.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f620006e0b0 0x7f6200070560 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6214064c90 0x7f6214115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 --2- 172.21.15.33:0/150832450 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6214064380 0x7f6214114f00 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 >> 172.21.15.33:0/150832450 conn(0x7f62140710c0 msgr2=0x7f62140f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:55.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 shutdown_connections 2024-04-25T02:17:55.118 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.112+0000 7f6218e21640 1 -- 172.21.15.33:0/150832450 wait complete. 2024-04-25T02:17:55.127 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:54 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3850881425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T02:17:55.127 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:54 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/430394675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T02:17:55.128 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-25T02:17:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.225+0000 7f8766d1d640 1 -- 172.21.15.33:0/3712343084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87601012a0 msgr2=0x7f8760101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.225+0000 7f8766d1d640 1 --2- 172.21.15.33:0/3712343084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87601012a0 0x7f8760101670 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f87580048d0 tx=0x7f8758009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.225+0000 7f8766d1d640 1 -- 172.21.15.33:0/3712343084 shutdown_connections 2024-04-25T02:17:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.225+0000 7f8766d1d640 1 --2- 172.21.15.33:0/3712343084 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f8760101c40 0x7f87600fb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.225+0000 7f8766d1d640 1 --2- 172.21.15.33:0/3712343084 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f87601012a0 0x7f8760101670 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.225+0000 7f8766d1d640 1 -- 172.21.15.33:0/3712343084 >> 172.21.15.33:0/3712343084 conn(0x7f87600f6e70 msgr2=0x7f87600f9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.226+0000 7f8766d1d640 1 -- 172.21.15.33:0/3712343084 shutdown_connections 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.226+0000 7f8766d1d640 1 -- 172.21.15.33:0/3712343084 wait complete. 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.226+0000 7f8766d1d640 1 Processor -- start 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.226+0000 7f8766d1d640 1 -- start start 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.226+0000 7f8766d1d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601012a0 0x7f8760112ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.226+0000 7f8766d1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760101c40 0x7f8760113400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.226+0000 7f8766d1d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8760113a00 con 0x7f8760101c40 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.226+0000 7f8766d1d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f876010d050 con 0x7f87601012a0 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760101c40 0x7f8760113400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760101c40 0x7f8760113400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45072/0 (socket says 172.21.15.33:45072) 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875ffff640 1 -- 172.21.15.33:0/84784207 learned_addr learned my addr 172.21.15.33:0/84784207 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f8764ab9640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601012a0 0x7f8760112ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875ffff640 1 -- 172.21.15.33:0/84784207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601012a0 msgr2=0x7f8760112ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875ffff640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601012a0 0x7f8760112ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875ffff640 1 -- 172.21.15.33:0/84784207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8758005040 con 0x7f8760101c40 2024-04-25T02:17:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f8764ab9640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601012a0 0x7f8760112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:17:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875ffff640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760101c40 0x7f8760113400 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f875000cae0 tx=0x7f8750007e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875d7fa640 1 -- 172.21.15.33:0/84784207 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f875000ab00 con 0x7f8760101c40 2024-04-25T02:17:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875d7fa640 1 -- 172.21.15.33:0/84784207 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f875000ac60 con 0x7f8760101c40 2024-04-25T02:17:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f876010d330 con 0x7f8760101c40 2024-04-25T02:17:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f875d7fa640 1 -- 172.21.15.33:0/84784207 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87500116f0 con 0x7f8760101c40 2024-04-25T02:17:55.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.227+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f876010d850 con 0x7f8760101c40 2024-04-25T02:17:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.228+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87600fca10 con 0x7f8760101c40 2024-04-25T02:17:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.229+0000 7f875d7fa640 1 -- 172.21.15.33:0/84784207 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8750012440 con 0x7f8760101c40 2024-04-25T02:17:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.229+0000 7f875d7fa640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f874c06e0b0 0x7f874c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.229+0000 7f875d7fa640 1 -- 172.21.15.33:0/84784207 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f875008e420 con 0x7f8760101c40 2024-04-25T02:17:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.229+0000 7f8764ab9640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f874c06e0b0 0x7f874c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:55.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.230+0000 7f8764ab9640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f874c06e0b0 0x7f874c070560 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8758000c00 tx=0x7f8758009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:55.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.234+0000 7f875d7fa640 1 -- 172.21.15.33:0/84784207 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f87500577a0 con 0x7f8760101c40 2024-04-25T02:17:55.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:54 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3850881425' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T02:17:55.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:54 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/430394675' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T02:17:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.432+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f876004bcc0 con 0x7f8760101c40 2024-04-25T02:17:55.436 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.432+0000 7f875d7fa640 1 -- 172.21.15.33:0/84784207 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f875005ae60 con 0x7f8760101c40 2024-04-25T02:17:55.437 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f874c06e0b0 msgr2=0x7f874c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f874c06e0b0 0x7f874c070560 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8758000c00 tx=0x7f8758009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760101c40 msgr2=0x7f8760113400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760101c40 0x7f8760113400 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f875000cae0 tx=0x7f8750007e50 comp rx=0 tx=0).stop 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 shutdown_connections 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f874c06e0b0 0x7f874c070560 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f8760101c40 0x7f8760113400 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 --2- 172.21.15.33:0/84784207 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f87601012a0 0x7f8760112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 >> 172.21.15.33:0/84784207 conn(0x7f87600f6e70 msgr2=0x7f8760065560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:55.439 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 shutdown_connections 2024-04-25T02:17:55.440 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.434+0000 7f8766d1d640 1 -- 172.21.15.33:0/84784207 wait complete. 2024-04-25T02:17:55.450 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.542+0000 7efc9655d640 1 -- 172.21.15.33:0/539770497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 msgr2=0x7efc90102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.542+0000 7efc9655d640 1 --2- 172.21.15.33:0/539770497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 0x7efc90102560 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7efc840048d0 tx=0x7efc84009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 -- 172.21.15.33:0/539770497 shutdown_connections 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 --2- 172.21.15.33:0/539770497 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc90102b30 0x7efc900fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 --2- 172.21.15.33:0/539770497 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 0x7efc90102560 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 -- 172.21.15.33:0/539770497 >> 172.21.15.33:0/539770497 conn(0x7efc900f7d00 msgr2=0x7efc900fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 -- 172.21.15.33:0/539770497 shutdown_connections 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 -- 172.21.15.33:0/539770497 wait complete. 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 Processor -- start 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 -- start start 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 0x7efc90112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc90102b30 0x7efc90113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:55.549 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc90113910 con 0x7efc90102190 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.543+0000 7efc9655d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9010ced0 con 0x7efc90102b30 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 0x7efc90112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 0x7efc90112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45100/0 (socket says 172.21.15.33:45100) 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8ffff640 1 -- 172.21.15.33:0/1018313192 learned_addr learned my addr 172.21.15.33:0/1018313192 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8ffff640 1 -- 172.21.15.33:0/1018313192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc90102b30 msgr2=0x7efc90113280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8ffff640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc90102b30 0x7efc90113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8ffff640 1 -- 172.21.15.33:0/1018313192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc84005040 con 0x7efc90102190 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8ffff640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 0x7efc90112d40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efc84004930 tx=0x7efc84009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8cff9640 1 -- 172.21.15.33:0/1018313192 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc840094c0 con 0x7efc90102190 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8cff9640 1 -- 172.21.15.33:0/1018313192 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc84032e50 con 0x7efc90102190 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc9010d150 con 0x7efc90102190 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc8cff9640 1 -- 172.21.15.33:0/1018313192 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc840433b0 con 0x7efc90102190 2024-04-25T02:17:55.550 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.544+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9010d610 con 0x7efc90102190 2024-04-25T02:17:55.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.545+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc900fd900 con 0x7efc90102190 2024-04-25T02:17:55.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.546+0000 7efc8cff9640 1 -- 172.21.15.33:0/1018313192 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efc8402f070 con 0x7efc90102190 2024-04-25T02:17:55.551 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.546+0000 7efc8cff9640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efc7806e100 0x7efc780705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:55.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.547+0000 7efc8cff9640 1 -- 172.21.15.33:0/1018313192 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7efc840afba0 con 0x7efc90102190 2024-04-25T02:17:55.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.547+0000 7efc8f7fe640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efc7806e100 0x7efc780705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:55.552 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.547+0000 7efc8f7fe640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efc7806e100 0x7efc780705b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efc9010e600 tx=0x7efc80006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:55.555 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.551+0000 7efc8cff9640 1 -- 172.21.15.33:0/1018313192 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efc84078e80 con 0x7efc90102190 2024-04-25T02:17:55.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.776+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc9004bcc0 con 0x7efc90102190 2024-04-25T02:17:55.781 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.777+0000 7efc8cff9640 1 -- 172.21.15.33:0/1018313192 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+5248 (secure 0 0 0) 0x7efc8407c540 con 0x7efc90102190 2024-04-25T02:17:55.781 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:55.782 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T02:16:16.512021+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":32,"state":"up:active","state_seq":468,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":1,"incarnation":32,"state":"up:active","state_seq":463,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44327":{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44841":{"gid":44841,"name":"cephfs.smithi138.rvxnby","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.138:6827/882099891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":882099891},{"type":"v1","addr":"172.21.15.138:6827","nonce":882099891}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34414,"qdb_cluster":[34422,34414]},"id":1}]} 2024-04-25T02:17:55.782 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 38 2024-04-25T02:17:55.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efc7806e100 msgr2=0x7efc780705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.783 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efc7806e100 0x7efc780705b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efc9010e600 tx=0x7efc80006d20 comp rx=0 tx=0).stop 2024-04-25T02:17:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 msgr2=0x7efc90112d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 0x7efc90112d40 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efc84004930 tx=0x7efc84009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 shutdown_connections 2024-04-25T02:17:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7efc7806e100 0x7efc780705b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7efc90102b30 0x7efc90113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 --2- 172.21.15.33:0/1018313192 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7efc90102190 0x7efc90112d40 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 >> 172.21.15.33:0/1018313192 conn(0x7efc900f7d00 msgr2=0x7efc900f8ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 shutdown_connections 2024-04-25T02:17:55.784 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.779+0000 7efc9655d640 1 -- 172.21.15.33:0/1018313192 wait complete. 2024-04-25T02:17:55.794 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 15, 'max_mds': 2, 'flags': 50} 2024-04-25T02:17:55.795 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-25T02:17:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.889+0000 7f62b982e640 1 -- 172.21.15.33:0/1760475666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 msgr2=0x7f62b4102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.889+0000 7f62b982e640 1 --2- 172.21.15.33:0/1760475666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 0x7f62b4102640 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f62a40048d0 tx=0x7f62a4009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.894 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.889+0000 7f62b982e640 1 -- 172.21.15.33:0/1760475666 shutdown_connections 2024-04-25T02:17:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.889+0000 7f62b982e640 1 --2- 172.21.15.33:0/1760475666 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4064390 0x7f62b4064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.889+0000 7f62b982e640 1 --2- 172.21.15.33:0/1760475666 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 0x7f62b4102640 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.889+0000 7f62b982e640 1 -- 172.21.15.33:0/1760475666 >> 172.21.15.33:0/1760475666 conn(0x7f62b4071fd0 msgr2=0x7f62b40723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.889+0000 7f62b982e640 1 -- 172.21.15.33:0/1760475666 shutdown_connections 2024-04-25T02:17:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.889+0000 7f62b982e640 1 -- 172.21.15.33:0/1760475666 wait complete. 2024-04-25T02:17:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.890+0000 7f62b982e640 1 Processor -- start 2024-04-25T02:17:55.895 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.890+0000 7f62b982e640 1 -- start start 2024-04-25T02:17:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.890+0000 7f62b982e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4064390 0x7f62b4112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:55.896 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.890+0000 7f62b982e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 0x7f62b4113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.890+0000 7f62b982e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b4113770 con 0x7f62b4102270 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.890+0000 7f62b982e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b410cdc0 con 0x7f62b4064390 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.890+0000 7f62b27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 0x7f62b4113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.890+0000 7f62b27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 0x7f62b4113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51366/0 (socket says 172.21.15.33:51366) 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.890+0000 7f62b27fc640 1 -- 172.21.15.33:0/1197500752 learned_addr learned my addr 172.21.15.33:0/1197500752 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f62b2ffd640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4064390 0x7f62b4112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f62b27fc640 1 -- 172.21.15.33:0/1197500752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4064390 msgr2=0x7f62b4112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f62b27fc640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4064390 0x7f62b4112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f62b27fc640 1 -- 172.21.15.33:0/1197500752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a4005040 con 0x7f62b4102270 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f62b27fc640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 0x7f62b4113170 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f62a8002a50 tx=0x7f62a8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f629b7fe640 1 -- 172.21.15.33:0/1197500752 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a8007c30 con 0x7f62b4102270 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b410d0a0 con 0x7f62b4102270 2024-04-25T02:17:55.897 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f629b7fe640 1 -- 172.21.15.33:0/1197500752 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62a800f040 con 0x7f62b4102270 2024-04-25T02:17:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f629b7fe640 1 -- 172.21.15.33:0/1197500752 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62a800db50 con 0x7f62b4102270 2024-04-25T02:17:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.891+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b410d5c0 con 0x7f62b4102270 2024-04-25T02:17:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.892+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b40fd970 con 0x7f62b4102270 2024-04-25T02:17:55.898 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.893+0000 7f629b7fe640 1 -- 172.21.15.33:0/1197500752 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f62a801c400 con 0x7f62b4102270 2024-04-25T02:17:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.893+0000 7f629b7fe640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f629c06e0b0 0x7f629c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.893+0000 7f629b7fe640 1 -- 172.21.15.33:0/1197500752 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f62a808d370 con 0x7f62b4102270 2024-04-25T02:17:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.893+0000 7f62b2ffd640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f629c06e0b0 0x7f629c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:55.899 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.893+0000 7f62b2ffd640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f629c06e0b0 0x7f629c070560 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f62a4003080 tx=0x7f62a4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:55.902 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:55.898+0000 7f629b7fe640 1 -- 172.21.15.33:0/1197500752 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f62a80567a0 con 0x7f62b4102270 2024-04-25T02:17:56.124 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.120+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f62b4109370 con 0x7f62b4102270 2024-04-25T02:17:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.121+0000 7f629b7fe640 1 -- 172.21.15.33:0/1197500752 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v38) v1 ==== 107+0+4893 (secure 0 0 0) 0x7f62a8059e60 con 0x7f62b4102270 2024-04-25T02:17:56.125 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:56.125 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":16,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:35:01.225052+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14446,"mds_1":24279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14446":{"gid":14446,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6827/3303942324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3303942324},{"type":"v1","addr":"172.21.15.33:6827","nonce":3303942324}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14458":{"gid":14458,"name":"cephfs.smithi033.hobkqr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.33:6829/3968098059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3968098059},{"type":"v1","addr":"172.21.15.33:6829","nonce":3968098059}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24279":{"gid":24279,"name":"cephfs.smithi138.jwzxlz","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/3887358693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3887358693},{"type":"v1","addr":"172.21.15.138:6825","nonce":3887358693}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi138.rvxnby","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.138:6827/4019073381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4019073381},{"type":"v1","addr":"172.21.15.138:6827","nonce":4019073381}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:56.125 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 16 2024-04-25T02:17:56.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f629c06e0b0 msgr2=0x7f629c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.127 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f629c06e0b0 0x7f629c070560 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f62a4003080 tx=0x7f62a4009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 msgr2=0x7f62b4113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 0x7f62b4113170 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f62a8002a50 tx=0x7f62a8004600 comp rx=0 tx=0).stop 2024-04-25T02:17:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 shutdown_connections 2024-04-25T02:17:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f629c06e0b0 0x7f629c070560 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f62b4102270 0x7f62b4113170 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 --2- 172.21.15.33:0/1197500752 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f62b4064390 0x7f62b4112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 >> 172.21.15.33:0/1197500752 conn(0x7f62b4071fd0 msgr2=0x7f62b40fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 shutdown_connections 2024-04-25T02:17:56.128 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.123+0000 7f62b982e640 1 -- 172.21.15.33:0/1197500752 wait complete. 2024-04-25T02:17:56.137 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:55 smithi033 ceph-mon[111306]: pgmap v1159: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:56.137 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:55 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/150832450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T02:17:56.137 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:17:56.137 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:55 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/84784207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T02:17:56.137 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:55 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1018313192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T02:17:56.139 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.232+0000 7fe0d119d640 1 -- 172.21.15.33:0/2784152235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 msgr2=0x7fe0cc1061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.232+0000 7fe0d119d640 1 --2- 172.21.15.33:0/2784152235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 0x7fe0cc1061f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4004900 tx=0x7fe0b4009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.232+0000 7fe0d119d640 1 -- 172.21.15.33:0/2784152235 shutdown_connections 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.232+0000 7fe0d119d640 1 --2- 172.21.15.33:0/2784152235 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 0x7fe0cc1061f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.232+0000 7fe0d119d640 1 --2- 172.21.15.33:0/2784152235 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0cc0fd180 0x7fe0cc0fd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.232+0000 7fe0d119d640 1 -- 172.21.15.33:0/2784152235 >> 172.21.15.33:0/2784152235 conn(0x7fe0cc0f6be0 msgr2=0x7fe0cc0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.232+0000 7fe0d119d640 1 -- 172.21.15.33:0/2784152235 shutdown_connections 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.232+0000 7fe0d119d640 1 -- 172.21.15.33:0/2784152235 wait complete. 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0d119d640 1 Processor -- start 2024-04-25T02:17:56.239 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0d119d640 1 -- start start 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0d119d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0cc0fd180 0x7fe0cc112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0d119d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 0x7fe0cc113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0d119d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc113870 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0d119d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0cc10cec0 con 0x7fe0cc0fd180 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0ca59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 0x7fe0cc113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0ca59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 0x7fe0cc113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51380/0 (socket says 172.21.15.33:51380) 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0ca59c640 1 -- 172.21.15.33:0/2528801247 learned_addr learned my addr 172.21.15.33:0/2528801247 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.233+0000 7fe0cad9d640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0cc0fd180 0x7fe0cc112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.234+0000 7fe0ca59c640 1 -- 172.21.15.33:0/2528801247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0cc0fd180 msgr2=0x7fe0cc112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.234+0000 7fe0ca59c640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0cc0fd180 0x7fe0cc112d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.234+0000 7fe0ca59c640 1 -- 172.21.15.33:0/2528801247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b4005040 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.234+0000 7fe0ca59c640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 0x7fe0cc113270 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4000c00 tx=0x7fe0b4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.234+0000 7fe0c37fe640 1 -- 172.21.15.33:0/2528801247 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0b40094c0 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.234+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0cc10d140 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.234+0000 7fe0c37fe640 1 -- 172.21.15.33:0/2528801247 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe0b403d920 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.240 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.234+0000 7fe0c37fe640 1 -- 172.21.15.33:0/2528801247 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0b403b8c0 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.234+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0cc10d660 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.235+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0cc065fe0 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.241 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.236+0000 7fe0c37fe640 1 -- 172.21.15.33:0/2528801247 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe0b403da90 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.236+0000 7fe0c37fe640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0b806e0b0 0x7fe0b8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:56.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.236+0000 7fe0cad9d640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0b806e0b0 0x7fe0b8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:56.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.236+0000 7fe0c37fe640 1 -- 172.21.15.33:0/2528801247 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe0b40b02f0 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.242 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.237+0000 7fe0cad9d640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0b806e0b0 0x7fe0b8070560 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4004fd0 tx=0x7fe0c4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:56.245 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.241+0000 7fe0c37fe640 1 -- 172.21.15.33:0/2528801247 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe0b4079730 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:55 smithi138 ceph-mon[81304]: pgmap v1159: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:17:56.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:55 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/150832450' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T02:17:56.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:17:56.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:55 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/84784207' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T02:17:56.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:55 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1018313192' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T02:17:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.462+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7fe0cc064350 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.467 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.463+0000 7fe0c37fe640 1 -- 172.21.15.33:0/2528801247 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v38) v1 ==== 107+0+4894 (secure 0 0 0) 0x7fe0b407cdf0 con 0x7fe0cc0fdb20 2024-04-25T02:17:56.468 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:56.468 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":17,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:36:43.415990+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14446,"mds_1":24279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14446":{"gid":14446,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6827/3303942324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3303942324},{"type":"v1","addr":"172.21.15.33:6827","nonce":3303942324}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14458":{"gid":14458,"name":"cephfs.smithi033.hobkqr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.33:6829/3968098059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3968098059},{"type":"v1","addr":"172.21.15.33:6829","nonce":3968098059}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24279":{"gid":24279,"name":"cephfs.smithi138.jwzxlz","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/3887358693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3887358693},{"type":"v1","addr":"172.21.15.138:6825","nonce":3887358693}]},"join_fscid":1,"export_targets":[0],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi138.rvxnby","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.138:6827/4019073381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4019073381},{"type":"v1","addr":"172.21.15.138:6827","nonce":4019073381}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:56.468 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 17 2024-04-25T02:17:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0b806e0b0 msgr2=0x7fe0b8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0b806e0b0 0x7fe0b8070560 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4004fd0 tx=0x7fe0c4009040 comp rx=0 tx=0).stop 2024-04-25T02:17:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 msgr2=0x7fe0cc113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 0x7fe0cc113270 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4000c00 tx=0x7fe0b4009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 shutdown_connections 2024-04-25T02:17:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe0b806e0b0 0x7fe0b8070560 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe0cc0fdb20 0x7fe0cc113270 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.470 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 --2- 172.21.15.33:0/2528801247 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe0cc0fd180 0x7fe0cc112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 >> 172.21.15.33:0/2528801247 conn(0x7fe0cc0f6be0 msgr2=0x7fe0cc0f8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 shutdown_connections 2024-04-25T02:17:56.471 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.465+0000 7fe0d119d640 1 -- 172.21.15.33:0/2528801247 wait complete. 2024-04-25T02:17:56.481 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-25T02:17:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.574+0000 7f17b5aae640 1 -- 172.21.15.33:0/2506045405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101090 msgr2=0x7f17b0101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.574+0000 7f17b5aae640 1 --2- 172.21.15.33:0/2506045405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101090 0x7f17b0101460 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f17a00048d0 tx=0x7f17a0009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 -- 172.21.15.33:0/2506045405 shutdown_connections 2024-04-25T02:17:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 --2- 172.21.15.33:0/2506045405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17b0101a30 0x7f17b00fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 --2- 172.21.15.33:0/2506045405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101090 0x7f17b0101460 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 -- 172.21.15.33:0/2506045405 >> 172.21.15.33:0/2506045405 conn(0x7f17b00f6c20 msgr2=0x7f17b00f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 -- 172.21.15.33:0/2506045405 shutdown_connections 2024-04-25T02:17:56.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 -- 172.21.15.33:0/2506045405 wait complete. 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 Processor -- start 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 -- start start 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17b0101090 0x7f17b0112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101a30 0x7f17b0113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0113810 con 0x7f17b0101a30 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.575+0000 7f17b5aae640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b010ce60 con 0x7f17b0101090 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101a30 0x7f17b0113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17aeffd640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101a30 0x7f17b0113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51400/0 (socket says 172.21.15.33:51400) 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17aeffd640 1 -- 172.21.15.33:0/3035664775 learned_addr learned my addr 172.21.15.33:0/3035664775 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17af7fe640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17b0101090 0x7f17b0112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17aeffd640 1 -- 172.21.15.33:0/3035664775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17b0101090 msgr2=0x7f17b0112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17aeffd640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17b0101090 0x7f17b0112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.582 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17aeffd640 1 -- 172.21.15.33:0/3035664775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a0005040 con 0x7f17b0101a30 2024-04-25T02:17:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17aeffd640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101a30 0x7f17b0113210 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f17a4002a50 tx=0x7f17a4004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f1797fff640 1 -- 172.21.15.33:0/3035664775 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a4007c30 con 0x7f17b0101a30 2024-04-25T02:17:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f1797fff640 1 -- 172.21.15.33:0/3035664775 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17a400f040 con 0x7f17b0101a30 2024-04-25T02:17:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b010d140 con 0x7f17b0101a30 2024-04-25T02:17:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f1797fff640 1 -- 172.21.15.33:0/3035664775 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17a400dad0 con 0x7f17b0101a30 2024-04-25T02:17:56.583 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.576+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b010d660 con 0x7f17b0101a30 2024-04-25T02:17:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.577+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b00fc800 con 0x7f17b0101a30 2024-04-25T02:17:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.578+0000 7f1797fff640 1 -- 172.21.15.33:0/3035664775 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f17a400dd40 con 0x7f17b0101a30 2024-04-25T02:17:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.578+0000 7f1797fff640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f179806e0b0 0x7f1798070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.578+0000 7f1797fff640 1 -- 172.21.15.33:0/3035664775 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f17a408d330 con 0x7f17b0101a30 2024-04-25T02:17:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.578+0000 7f17af7fe640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f179806e0b0 0x7f1798070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:56.584 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.579+0000 7f17af7fe640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f179806e0b0 0x7f1798070560 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000c00 tx=0x7f17a0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:56.587 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.583+0000 7f1797fff640 1 -- 172.21.15.33:0/3035664775 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f17a4056760 con 0x7f17b0101a30 2024-04-25T02:17:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.809+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f17b01081f0 con 0x7f17b0101a30 2024-04-25T02:17:56.813 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.809+0000 7f1797fff640 1 -- 172.21.15.33:0/3035664775 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v38) v1 ==== 107+0+4893 (secure 0 0 0) 0x7f17a4059e20 con 0x7f17b0101a30 2024-04-25T02:17:56.814 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:56.814 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":18,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:40:03.675903+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14446,"mds_1":24279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14446":{"gid":14446,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6827/3303942324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3303942324},{"type":"v1","addr":"172.21.15.33:6827","nonce":3303942324}]},"join_fscid":1,"export_targets":[1],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14458":{"gid":14458,"name":"cephfs.smithi033.hobkqr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.33:6829/3968098059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3968098059},{"type":"v1","addr":"172.21.15.33:6829","nonce":3968098059}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24279":{"gid":24279,"name":"cephfs.smithi138.jwzxlz","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/3887358693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3887358693},{"type":"v1","addr":"172.21.15.138:6825","nonce":3887358693}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi138.rvxnby","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.138:6827/4019073381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4019073381},{"type":"v1","addr":"172.21.15.138:6827","nonce":4019073381}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:56.814 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 18 2024-04-25T02:17:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.811+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f179806e0b0 msgr2=0x7f1798070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.811+0000 7f17b5aae640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f179806e0b0 0x7f1798070560 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f17a0000c00 tx=0x7f17a0009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.811+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101a30 msgr2=0x7f17b0113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.811+0000 7f17b5aae640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101a30 0x7f17b0113210 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f17a4002a50 tx=0x7f17a4004600 comp rx=0 tx=0).stop 2024-04-25T02:17:56.816 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.812+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 shutdown_connections 2024-04-25T02:17:56.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.812+0000 7f17b5aae640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f179806e0b0 0x7f1798070560 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.812+0000 7f17b5aae640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f17b0101a30 0x7f17b0113210 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.812+0000 7f17b5aae640 1 --2- 172.21.15.33:0/3035664775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f17b0101090 0x7f17b0112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.812+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 >> 172.21.15.33:0/3035664775 conn(0x7f17b00f6c20 msgr2=0x7f17b00f7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:56.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.812+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 shutdown_connections 2024-04-25T02:17:56.817 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.812+0000 7f17b5aae640 1 -- 172.21.15.33:0/3035664775 wait complete. 2024-04-25T02:17:56.827 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-25T02:17:56.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.924+0000 7f69354ae640 1 -- 172.21.15.33:0/1859424085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 msgr2=0x7f6930102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.924+0000 7f69354ae640 1 --2- 172.21.15.33:0/1859424085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 0x7f6930102640 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6920009c50 tx=0x7f6920004d10 comp rx=0 tx=0).stop 2024-04-25T02:17:56.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.924+0000 7f69354ae640 1 -- 172.21.15.33:0/1859424085 shutdown_connections 2024-04-25T02:17:56.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.924+0000 7f69354ae640 1 --2- 172.21.15.33:0/1859424085 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6930064390 0x7f6930064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.924+0000 7f69354ae640 1 --2- 172.21.15.33:0/1859424085 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 0x7f6930102640 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.924+0000 7f69354ae640 1 -- 172.21.15.33:0/1859424085 >> 172.21.15.33:0/1859424085 conn(0x7f6930071fd0 msgr2=0x7f69300723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:56.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.924+0000 7f69354ae640 1 -- 172.21.15.33:0/1859424085 shutdown_connections 2024-04-25T02:17:56.930 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.925+0000 7f69354ae640 1 -- 172.21.15.33:0/1859424085 wait complete. 2024-04-25T02:17:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.925+0000 7f69354ae640 1 Processor -- start 2024-04-25T02:17:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.925+0000 7f69354ae640 1 -- start start 2024-04-25T02:17:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.925+0000 7f69354ae640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6930064390 0x7f6930112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:56.931 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.925+0000 7f69354ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 0x7f69301131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.925+0000 7f69354ae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69301137a0 con 0x7f6930102270 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.925+0000 7f69354ae640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f693010cdf0 con 0x7f6930064390 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f692e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 0x7f69301131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f692e7fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 0x7f69301131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51420/0 (socket says 172.21.15.33:51420) 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f692e7fc640 1 -- 172.21.15.33:0/569416770 learned_addr learned my addr 172.21.15.33:0/569416770 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f692effd640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6930064390 0x7f6930112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f692e7fc640 1 -- 172.21.15.33:0/569416770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6930064390 msgr2=0x7f6930112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f692e7fc640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6930064390 0x7f6930112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f692e7fc640 1 -- 172.21.15.33:0/569416770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200098b0 con 0x7f6930102270 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f692e7fc640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 0x7f69301131a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6924002a50 tx=0x7f6924004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f69177fe640 1 -- 172.21.15.33:0/569416770 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6924007c30 con 0x7f6930102270 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f69177fe640 1 -- 172.21.15.33:0/569416770 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f692400f040 con 0x7f6930102270 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f693010d0d0 con 0x7f6930102270 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f69177fe640 1 -- 172.21.15.33:0/569416770 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f692400dbe0 con 0x7f6930102270 2024-04-25T02:17:56.932 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.926+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f693010d670 con 0x7f6930102270 2024-04-25T02:17:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.927+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69300fd970 con 0x7f6930102270 2024-04-25T02:17:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.928+0000 7f69177fe640 1 -- 172.21.15.33:0/569416770 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f692400dd40 con 0x7f6930102270 2024-04-25T02:17:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.928+0000 7f69177fe640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f691806e0b0 0x7f6918070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.928+0000 7f69177fe640 1 -- 172.21.15.33:0/569416770 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f692408db20 con 0x7f6930102270 2024-04-25T02:17:56.933 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.928+0000 7f692effd640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f691806e0b0 0x7f6918070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:56.934 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.929+0000 7f692effd640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f691806e0b0 0x7f6918070560 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6920004a30 tx=0x7f692002e040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:56.937 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:56.933+0000 7f69177fe640 1 -- 172.21.15.33:0/569416770 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6924056ea0 con 0x7f6930102270 2024-04-25T02:17:57.161 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.157+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f6930063f00 con 0x7f6930102270 2024-04-25T02:17:57.162 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.157+0000 7f69177fe640 1 -- 172.21.15.33:0/569416770 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v38) v1 ==== 107+0+4892 (secure 0 0 0) 0x7f692405a560 con 0x7f6930102270 2024-04-25T02:17:57.162 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:57.162 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:40:21.849082+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14446,"mds_1":24279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14446":{"gid":14446,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6827/3303942324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3303942324},{"type":"v1","addr":"172.21.15.33:6827","nonce":3303942324}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14458":{"gid":14458,"name":"cephfs.smithi033.hobkqr","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.33:6829/3968098059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":3968098059},{"type":"v1","addr":"172.21.15.33:6829","nonce":3968098059}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24279":{"gid":24279,"name":"cephfs.smithi138.jwzxlz","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/3887358693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3887358693},{"type":"v1","addr":"172.21.15.138:6825","nonce":3887358693}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24289":{"gid":24289,"name":"cephfs.smithi138.rvxnby","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.138:6827/4019073381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":4019073381},{"type":"v1","addr":"172.21.15.138:6827","nonce":4019073381}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:57.163 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 19 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f691806e0b0 msgr2=0x7f6918070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f691806e0b0 0x7f6918070560 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6920004a30 tx=0x7f692002e040 comp rx=0 tx=0).stop 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 msgr2=0x7f69301131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 0x7f69301131a0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f6924002a50 tx=0x7f6924004600 comp rx=0 tx=0).stop 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 shutdown_connections 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f691806e0b0 0x7f6918070560 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f6930102270 0x7f69301131a0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 --2- 172.21.15.33:0/569416770 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f6930064390 0x7f6930112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 >> 172.21.15.33:0/569416770 conn(0x7f6930071fd0 msgr2=0x7f69300fa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:57.164 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 shutdown_connections 2024-04-25T02:17:57.165 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.159+0000 7f69354ae640 1 -- 172.21.15.33:0/569416770 wait complete. 2024-04-25T02:17:57.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:56 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1197500752' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-25T02:17:57.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:56 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2528801247' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-25T02:17:57.174 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:56 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3035664775' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-25T02:17:57.175 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-25T02:17:57.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.269+0000 7f0da5d9b640 1 -- 172.21.15.33:0/3774195767 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9c031b60 con 0x7f0da8102b80 2024-04-25T02:17:57.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 -- 172.21.15.33:0/3774195767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da8102b80 msgr2=0x7f0da80fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.276 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 --2- 172.21.15.33:0/3774195767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da8102b80 0x7f0da80fc210 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c0048d0 tx=0x7f0d9c009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 -- 172.21.15.33:0/3774195767 shutdown_connections 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 --2- 172.21.15.33:0/3774195767 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da8102b80 0x7f0da80fc210 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 --2- 172.21.15.33:0/3774195767 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da81021e0 0x7f0da81025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 -- 172.21.15.33:0/3774195767 >> 172.21.15.33:0/3774195767 conn(0x7f0da80f7db0 msgr2=0x7f0da80fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 -- 172.21.15.33:0/3774195767 shutdown_connections 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 -- 172.21.15.33:0/3774195767 wait complete. 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 Processor -- start 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.270+0000 7f0dad15d640 1 -- start start 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0dad15d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da81021e0 0x7f0da8112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0dad15d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da8102b80 0x7f0da8113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0dad15d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da8113780 con 0x7f0da81021e0 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0dad15d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da810cdd0 con 0x7f0da8102b80 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0da6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da81021e0 0x7f0da8112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0da6d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da81021e0 0x7f0da8112c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51444/0 (socket says 172.21.15.33:51444) 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0da6d9d640 1 -- 172.21.15.33:0/2613671687 learned_addr learned my addr 172.21.15.33:0/2613671687 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:57.277 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0da659c640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da8102b80 0x7f0da8113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0da6d9d640 1 -- 172.21.15.33:0/2613671687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da8102b80 msgr2=0x7f0da8113180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0da6d9d640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da8102b80 0x7f0da8113180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0da6d9d640 1 -- 172.21.15.33:0/2613671687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d9c005040 con 0x7f0da81021e0 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0da659c640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da8102b80 0x7f0da8113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0da6d9d640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da81021e0 0x7f0da8112c40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800c790 tx=0x7f0d980043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0d8f7fe640 1 -- 172.21.15.33:0/2613671687 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9800a820 con 0x7f0da81021e0 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0d8f7fe640 1 -- 172.21.15.33:0/2613671687 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d9800ae50 con 0x7f0da81021e0 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da810d0b0 con 0x7f0da81021e0 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.271+0000 7f0d8f7fe640 1 -- 172.21.15.33:0/2613671687 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d980115e0 con 0x7f0da81021e0 2024-04-25T02:17:57.278 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.272+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da810d5a0 con 0x7f0da81021e0 2024-04-25T02:17:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.272+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da80fd950 con 0x7f0da81021e0 2024-04-25T02:17:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.273+0000 7f0d8f7fe640 1 -- 172.21.15.33:0/2613671687 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0d98012400 con 0x7f0da81021e0 2024-04-25T02:17:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.273+0000 7f0d8f7fe640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0d9006e0b0 0x7f0d90070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.273+0000 7f0d8f7fe640 1 -- 172.21.15.33:0/2613671687 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0d9808dce0 con 0x7f0da81021e0 2024-04-25T02:17:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.273+0000 7f0da659c640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0d9006e0b0 0x7f0d90070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:57.279 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.274+0000 7f0da659c640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0d9006e0b0 0x7f0d90070560 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0da80fb3b0 tx=0x7f0d9c009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:57.282 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.278+0000 7f0d8f7fe640 1 -- 172.21.15.33:0/2613671687 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0d980570e0 con 0x7f0da81021e0 2024-04-25T02:17:57.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:56 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1197500752' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-25T02:17:57.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:56 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2528801247' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-25T02:17:57.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:56 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3035664775' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-25T02:17:57.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.499+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f0da8104620 con 0x7f0da81021e0 2024-04-25T02:17:57.504 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.500+0000 7f0d8f7fe640 1 -- 172.21.15.33:0/2613671687 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v38) v1 ==== 107+0+3333 (secure 0 0 0) 0x7f0d9805a7a0 con 0x7f0da81021e0 2024-04-25T02:17:57.505 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:57.505 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.158366+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14446,"mds_1":24279},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14446":{"gid":14446,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":3,"state":"up:active","state_seq":4,"addr":"172.21.15.33:6827/3303942324","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":3303942324},{"type":"v1","addr":"172.21.15.33:6827","nonce":3303942324}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24279":{"gid":24279,"name":"cephfs.smithi138.jwzxlz","rank":1,"incarnation":6,"state":"up:active","state_seq":4,"addr":"172.21.15.138:6825/3887358693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":3887358693},{"type":"v1","addr":"172.21.15.138:6825","nonce":3887358693}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14446,"qdb_cluster":[14446,24279]},"id":1}]} 2024-04-25T02:17:57.505 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 20 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0d9006e0b0 msgr2=0x7f0d90070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0d9006e0b0 0x7f0d90070560 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0da80fb3b0 tx=0x7f0d9c009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da81021e0 msgr2=0x7f0da8112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da81021e0 0x7f0da8112c40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800c790 tx=0x7f0d980043c0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 shutdown_connections 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0d9006e0b0 0x7f0d90070560 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0da8102b80 0x7f0da8113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 --2- 172.21.15.33:0/2613671687 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0da81021e0 0x7f0da8112c40 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 >> 172.21.15.33:0/2613671687 conn(0x7f0da80f7db0 msgr2=0x7f0da8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:57.507 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 shutdown_connections 2024-04-25T02:17:57.508 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.502+0000 7f0dad15d640 1 -- 172.21.15.33:0/2613671687 wait complete. 2024-04-25T02:17:57.518 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 20 2024-04-25T02:17:57.518 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-25T02:17:57.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.612+0000 7f3c96a6e640 1 -- 172.21.15.33:0/2527243757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102230 msgr2=0x7f3c90102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.612+0000 7f3c96a6e640 1 --2- 172.21.15.33:0/2527243757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102230 0x7f3c90102600 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f3c840048d0 tx=0x7f3c84009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.612+0000 7f3c96a6e640 1 -- 172.21.15.33:0/2527243757 shutdown_connections 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.612+0000 7f3c96a6e640 1 --2- 172.21.15.33:0/2527243757 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c90102bd0 0x7f3c900fa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.612+0000 7f3c96a6e640 1 --2- 172.21.15.33:0/2527243757 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102230 0x7f3c90102600 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.612+0000 7f3c96a6e640 1 -- 172.21.15.33:0/2527243757 >> 172.21.15.33:0/2527243757 conn(0x7f3c900717e0 msgr2=0x7f3c90071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.612+0000 7f3c96a6e640 1 -- 172.21.15.33:0/2527243757 shutdown_connections 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.612+0000 7f3c96a6e640 1 -- 172.21.15.33:0/2527243757 wait complete. 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c96a6e640 1 Processor -- start 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c96a6e640 1 -- start start 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c96a6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c90102230 0x7f3c900fbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c96a6e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102bd0 0x7f3c900fbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c96a6e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900fd980 con 0x7f3c90102bd0 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c96a6e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900fc520 con 0x7f3c90102230 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c8ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102bd0 0x7f3c900fbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:57.619 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c8ffff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102bd0 0x7f3c900fbfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51464/0 (socket says 172.21.15.33:51464) 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c8ffff640 1 -- 172.21.15.33:0/394462907 learned_addr learned my addr 172.21.15.33:0/394462907 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c9480a640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c90102230 0x7f3c900fbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c8ffff640 1 -- 172.21.15.33:0/394462907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c90102230 msgr2=0x7f3c900fbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c8ffff640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c90102230 0x7f3c900fbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c8ffff640 1 -- 172.21.15.33:0/394462907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c84005040 con 0x7f3c90102bd0 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.613+0000 7f3c9480a640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c90102230 0x7f3c900fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.614+0000 7f3c8ffff640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102bd0 0x7f3c900fbfe0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c002840 tx=0x7f3c7c004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.614+0000 7f3c8d7fa640 1 -- 172.21.15.33:0/394462907 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c7c007c30 con 0x7f3c90102bd0 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.614+0000 7f3c8d7fa640 1 -- 172.21.15.33:0/394462907 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c7c00f040 con 0x7f3c90102bd0 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.614+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c900fc800 con 0x7f3c90102bd0 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.614+0000 7f3c8d7fa640 1 -- 172.21.15.33:0/394462907 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c7c009590 con 0x7f3c90102bd0 2024-04-25T02:17:57.620 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.614+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c90119450 con 0x7f3c90102bd0 2024-04-25T02:17:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.615+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c90065530 con 0x7f3c90102bd0 2024-04-25T02:17:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.615+0000 7f3c8d7fa640 1 -- 172.21.15.33:0/394462907 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3c7c00d6b0 con 0x7f3c90102bd0 2024-04-25T02:17:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.616+0000 7f3c8d7fa640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c8006e0b0 0x7f3c80070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.616+0000 7f3c8d7fa640 1 -- 172.21.15.33:0/394462907 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3c7c08d350 con 0x7f3c90102bd0 2024-04-25T02:17:57.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.616+0000 7f3c9480a640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c8006e0b0 0x7f3c80070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:57.622 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.616+0000 7f3c9480a640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c8006e0b0 0x7f3c80070560 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3c84000c00 tx=0x7f3c84009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:57.625 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.621+0000 7f3c8d7fa640 1 -- 172.21.15.33:0/394462907 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3c7c0566d0 con 0x7f3c90102bd0 2024-04-25T02:17:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.847+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f3c90064350 con 0x7f3c90102bd0 2024-04-25T02:17:57.852 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.848+0000 7f3c8d7fa640 1 -- 172.21.15.33:0/394462907 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v38) v1 ==== 107+0+3298 (secure 0 0 0) 0x7f3c7c017d00 con 0x7f3c90102bd0 2024-04-25T02:17:57.853 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:57.853 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34378,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2956426668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2956426668},{"type":"v1","addr":"172.21.15.33:6829","nonce":2956426668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44267,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1198640100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1198640100},{"type":"v1","addr":"172.21.15.138:6827","nonce":1198640100}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:57.853 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 21 2024-04-25T02:17:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c8006e0b0 msgr2=0x7f3c80070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c8006e0b0 0x7f3c80070560 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3c84000c00 tx=0x7f3c84009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102bd0 msgr2=0x7f3c900fbfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102bd0 0x7f3c900fbfe0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c002840 tx=0x7f3c7c004570 comp rx=0 tx=0).stop 2024-04-25T02:17:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 shutdown_connections 2024-04-25T02:17:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c8006e0b0 0x7f3c80070560 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.855 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c90102bd0 0x7f3c900fbfe0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 --2- 172.21.15.33:0/394462907 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c90102230 0x7f3c900fbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 >> 172.21.15.33:0/394462907 conn(0x7f3c900717e0 msgr2=0x7f3c900fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 shutdown_connections 2024-04-25T02:17:57.856 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.850+0000 7f3c96a6e640 1 -- 172.21.15.33:0/394462907 wait complete. 2024-04-25T02:17:57.866 DEBUG:tasks.fs:max_mds not reduced in epoch 21 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:17:57.866 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 21 2024-04-25T02:17:57.866 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.960+0000 7faf53d1d640 1 -- 172.21.15.33:0/4182786401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c064390 msgr2=0x7faf4c064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.960+0000 7faf53d1d640 1 --2- 172.21.15.33:0/4182786401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c064390 0x7faf4c064800 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0048d0 tx=0x7faf3c009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.960+0000 7faf53d1d640 1 -- 172.21.15.33:0/4182786401 shutdown_connections 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.960+0000 7faf53d1d640 1 --2- 172.21.15.33:0/4182786401 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c064390 0x7faf4c064800 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.960+0000 7faf53d1d640 1 --2- 172.21.15.33:0/4182786401 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c102270 0x7faf4c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.960+0000 7faf53d1d640 1 -- 172.21.15.33:0/4182786401 >> 172.21.15.33:0/4182786401 conn(0x7faf4c071fd0 msgr2=0x7faf4c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.960+0000 7faf53d1d640 1 -- 172.21.15.33:0/4182786401 shutdown_connections 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.960+0000 7faf53d1d640 1 -- 172.21.15.33:0/4182786401 wait complete. 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf53d1d640 1 Processor -- start 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf53d1d640 1 -- start start 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf53d1d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c064390 0x7faf4c112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:57.972 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf53d1d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c102270 0x7faf4c1131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf53d1d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c1137a0 con 0x7faf4c102270 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf53d1d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c10cdf0 con 0x7faf4c064390 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf512b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c102270 0x7faf4c1131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf512b8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c102270 0x7faf4c1131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51476/0 (socket says 172.21.15.33:51476) 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf512b8640 1 -- 172.21.15.33:0/2385689629 learned_addr learned my addr 172.21.15.33:0/2385689629 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf512b8640 1 -- 172.21.15.33:0/2385689629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c064390 msgr2=0x7faf4c112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf512b8640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c064390 0x7faf4c112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.961+0000 7faf512b8640 1 -- 172.21.15.33:0/2385689629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c005040 con 0x7faf4c102270 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.962+0000 7faf512b8640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c102270 0x7faf4c1131a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0047f0 tx=0x7faf3c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.962+0000 7faf427fc640 1 -- 172.21.15.33:0/2385689629 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf3c0094c0 con 0x7faf4c102270 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.962+0000 7faf427fc640 1 -- 172.21.15.33:0/2385689629 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf3c032e50 con 0x7faf4c102270 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.962+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c10d070 con 0x7faf4c102270 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.962+0000 7faf427fc640 1 -- 172.21.15.33:0/2385689629 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf3c0433b0 con 0x7faf4c102270 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.962+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c10d530 con 0x7faf4c102270 2024-04-25T02:17:57.973 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.963+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c0fd970 con 0x7faf4c102270 2024-04-25T02:17:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.963+0000 7faf427fc640 1 -- 172.21.15.33:0/2385689629 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faf3c02f070 con 0x7faf4c102270 2024-04-25T02:17:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.964+0000 7faf427fc640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faf3806e0b0 0x7faf38070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.964+0000 7faf427fc640 1 -- 172.21.15.33:0/2385689629 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7faf3c0af900 con 0x7faf4c102270 2024-04-25T02:17:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.964+0000 7faf51ab9640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faf3806e0b0 0x7faf38070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.964+0000 7faf51ab9640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faf3806e0b0 0x7faf38070560 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faf48009af0 tx=0x7faf48009cd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:57.974 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:57.968+0000 7faf427fc640 1 -- 172.21.15.33:0/2385689629 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faf3c078c80 con 0x7faf4c102270 2024-04-25T02:17:58.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.195+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7faf4c063f00 con 0x7faf4c102270 2024-04-25T02:17:58.200 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.195+0000 7faf427fc640 1 -- 172.21.15.33:0/2385689629 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v38) v1 ==== 107+0+4859 (secure 0 0 0) 0x7faf3c07c340 con 0x7faf4c102270 2024-04-25T02:17:58.200 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:58.200 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34378,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2956426668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2956426668},{"type":"v1","addr":"172.21.15.33:6829","nonce":2956426668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":34386,"name":"cephfs.smithi033.qqhhcz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/4046804099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":4046804099},{"type":"v1","addr":"172.21.15.33:6827","nonce":4046804099}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22},{"gid":44267,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1198640100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1198640100},{"type":"v1","addr":"172.21.15.138:6827","nonce":1198640100}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44275,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2355068967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2355068967},{"type":"v1","addr":"172.21.15.138:6825","nonce":2355068967}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:58.201 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 22 2024-04-25T02:17:58.202 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faf3806e0b0 msgr2=0x7faf38070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faf3806e0b0 0x7faf38070560 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7faf48009af0 tx=0x7faf48009cd0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c102270 msgr2=0x7faf4c1131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c102270 0x7faf4c1131a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0047f0 tx=0x7faf3c009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 shutdown_connections 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7faf3806e0b0 0x7faf38070560 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7faf4c102270 0x7faf4c1131a0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 --2- 172.21.15.33:0/2385689629 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7faf4c064390 0x7faf4c112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 >> 172.21.15.33:0/2385689629 conn(0x7faf4c071fd0 msgr2=0x7faf4c0fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 shutdown_connections 2024-04-25T02:17:58.203 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.198+0000 7faf53d1d640 1 -- 172.21.15.33:0/2385689629 wait complete. 2024-04-25T02:17:58.212 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:57 smithi033 ceph-mon[111306]: pgmap v1160: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:58.212 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:57 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/569416770' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-25T02:17:58.212 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:57 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2613671687' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-25T02:17:58.212 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:57 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/394462907' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-25T02:17:58.213 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:17:58.213 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 22 2024-04-25T02:17:58.213 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-25T02:17:58.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:57 smithi138 ceph-mon[81304]: pgmap v1160: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:17:58.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:57 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/569416770' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-25T02:17:58.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:57 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2613671687' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-25T02:17:58.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:57 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/394462907' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-25T02:17:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.307+0000 7fed08d5d640 1 -- 172.21.15.33:0/1140958195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed041021e0 msgr2=0x7fed041025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.313 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.307+0000 7fed08d5d640 1 --2- 172.21.15.33:0/1140958195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed041021e0 0x7fed041025b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0048d0 tx=0x7fecfc009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.307+0000 7fed08d5d640 1 -- 172.21.15.33:0/1140958195 shutdown_connections 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.307+0000 7fed08d5d640 1 --2- 172.21.15.33:0/1140958195 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed04102b80 0x7fed040fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.307+0000 7fed08d5d640 1 --2- 172.21.15.33:0/1140958195 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed041021e0 0x7fed041025b0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.307+0000 7fed08d5d640 1 -- 172.21.15.33:0/1140958195 >> 172.21.15.33:0/1140958195 conn(0x7fed040f7db0 msgr2=0x7fed040fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.307+0000 7fed08d5d640 1 -- 172.21.15.33:0/1140958195 shutdown_connections 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.307+0000 7fed08d5d640 1 -- 172.21.15.33:0/1140958195 wait complete. 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed08d5d640 1 Processor -- start 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed08d5d640 1 -- start start 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed08d5d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed041021e0 0x7fed04112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed08d5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed04102b80 0x7fed04113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed08d5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04113890 con 0x7fed04102b80 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed08d5d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0410ce50 con 0x7fed041021e0 2024-04-25T02:17:58.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed01d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed04102b80 0x7fed04113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed01d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed04102b80 0x7fed04113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51494/0 (socket says 172.21.15.33:51494) 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed01d9b640 1 -- 172.21.15.33:0/70497394 learned_addr learned my addr 172.21.15.33:0/70497394 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.308+0000 7fed0259c640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed041021e0 0x7fed04112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.309+0000 7fed01d9b640 1 -- 172.21.15.33:0/70497394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed041021e0 msgr2=0x7fed04112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.309+0000 7fed01d9b640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed041021e0 0x7fed04112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.309+0000 7fed01d9b640 1 -- 172.21.15.33:0/70497394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc005040 con 0x7fed04102b80 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.309+0000 7fed01d9b640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed04102b80 0x7fed04113200 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fecf800b790 tx=0x7fecf80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.309+0000 7fecf2ffd640 1 -- 172.21.15.33:0/70497394 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf800c700 con 0x7fed04102b80 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.309+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0410d130 con 0x7fed04102b80 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.309+0000 7fecf2ffd640 1 -- 172.21.15.33:0/70497394 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecf800cd30 con 0x7fed04102b80 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.309+0000 7fecf2ffd640 1 -- 172.21.15.33:0/70497394 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf8013370 con 0x7fed04102b80 2024-04-25T02:17:58.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.309+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0410d650 con 0x7fed04102b80 2024-04-25T02:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.310+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed040fd950 con 0x7fed04102b80 2024-04-25T02:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.310+0000 7fecf2ffd640 1 -- 172.21.15.33:0/70497394 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fecf8012070 con 0x7fed04102b80 2024-04-25T02:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.311+0000 7fecf2ffd640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecec06e0b0 0x7fecec070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.311+0000 7fecf2ffd640 1 -- 172.21.15.33:0/70497394 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fecf808d480 con 0x7fed04102b80 2024-04-25T02:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.311+0000 7fed0259c640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecec06e0b0 0x7fecec070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:58.316 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.311+0000 7fed0259c640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecec06e0b0 0x7fecec070560 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000c00 tx=0x7fecfc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:58.320 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.315+0000 7fecf2ffd640 1 -- 172.21.15.33:0/70497394 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fecf8056800 con 0x7fed04102b80 2024-04-25T02:17:58.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.538+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fed04064350 con 0x7fed04102b80 2024-04-25T02:17:58.543 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.539+0000 7fecf2ffd640 1 -- 172.21.15.33:0/70497394 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v38) v1 ==== 107+0+4080 (secure 0 0 0) 0x7fecf8059ec0 con 0x7fed04102b80 2024-04-25T02:17:58.544 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:58.544 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34378,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2956426668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2956426668},{"type":"v1","addr":"172.21.15.33:6829","nonce":2956426668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44267,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1198640100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1198640100},{"type":"v1","addr":"172.21.15.138:6827","nonce":1198640100}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44275,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2355068967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2355068967},{"type":"v1","addr":"172.21.15.138:6825","nonce":2355068967}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:58.544 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 23 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecec06e0b0 msgr2=0x7fecec070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecec06e0b0 0x7fecec070560 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fecfc000c00 tx=0x7fecfc009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed04102b80 msgr2=0x7fed04113200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed04102b80 0x7fed04113200 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fecf800b790 tx=0x7fecf80091d0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 shutdown_connections 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fecec06e0b0 0x7fecec070560 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fed04102b80 0x7fed04113200 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 --2- 172.21.15.33:0/70497394 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fed041021e0 0x7fed04112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 >> 172.21.15.33:0/70497394 conn(0x7fed040f7db0 msgr2=0x7fed040f9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:58.546 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 shutdown_connections 2024-04-25T02:17:58.547 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.541+0000 7fed08d5d640 1 -- 172.21.15.33:0/70497394 wait complete. 2024-04-25T02:17:58.556 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:17:58.556 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 23 2024-04-25T02:17:58.557 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-25T02:17:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.650+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/951121814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 msgr2=0x7f0c38106230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.650+0000 7f0c3ee2e640 1 --2- 172.21.15.33:0/951121814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 0x7f0c38106230 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f0c240048d0 tx=0x7f0c24009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.650+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/951121814 shutdown_connections 2024-04-25T02:17:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.650+0000 7f0c3ee2e640 1 --2- 172.21.15.33:0/951121814 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 0x7f0c38106230 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.650+0000 7f0c3ee2e640 1 --2- 172.21.15.33:0/951121814 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380fd1c0 0x7f0c380fd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.650+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/951121814 >> 172.21.15.33:0/951121814 conn(0x7f0c380f6c00 msgr2=0x7f0c380f9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.650+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/951121814 shutdown_connections 2024-04-25T02:17:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.650+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/951121814 wait complete. 2024-04-25T02:17:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.651+0000 7f0c3ee2e640 1 Processor -- start 2024-04-25T02:17:58.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.651+0000 7f0c3ee2e640 1 -- start start 2024-04-25T02:17:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.651+0000 7f0c3ee2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380fd1c0 0x7f0c38112d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.651+0000 7f0c3ee2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 0x7f0c381132d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.651+0000 7f0c3ee2e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c381138d0 con 0x7f0c380fdb60 2024-04-25T02:17:58.657 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.651+0000 7f0c3ee2e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3810cf20 con 0x7f0c380fd1c0 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.651+0000 7f0c37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 0x7f0c381132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.651+0000 7f0c37fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 0x7f0c381132d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51520/0 (socket says 172.21.15.33:51520) 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.651+0000 7f0c37fff640 1 -- 172.21.15.33:0/1962255809 learned_addr learned my addr 172.21.15.33:0/1962255809 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c3cbca640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380fd1c0 0x7f0c38112d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c37fff640 1 -- 172.21.15.33:0/1962255809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380fd1c0 msgr2=0x7f0c38112d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c37fff640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380fd1c0 0x7f0c38112d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c37fff640 1 -- 172.21.15.33:0/1962255809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24005040 con 0x7f0c380fdb60 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c37fff640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 0x7f0c381132d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0c240047f0 tx=0x7f0c24009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c357fa640 1 -- 172.21.15.33:0/1962255809 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c240094c0 con 0x7f0c380fdb60 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3810d1a0 con 0x7f0c380fdb60 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c357fa640 1 -- 172.21.15.33:0/1962255809 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c240315f0 con 0x7f0c380fdb60 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c357fa640 1 -- 172.21.15.33:0/1962255809 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c240435f0 con 0x7f0c380fdb60 2024-04-25T02:17:58.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.652+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3810d660 con 0x7f0c380fdb60 2024-04-25T02:17:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.653+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c38065fa0 con 0x7f0c380fdb60 2024-04-25T02:17:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.654+0000 7f0c357fa640 1 -- 172.21.15.33:0/1962255809 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0c2402f070 con 0x7f0c380fdb60 2024-04-25T02:17:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.654+0000 7f0c357fa640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c2806e0b0 0x7f0c28070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:58.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.654+0000 7f0c357fa640 1 -- 172.21.15.33:0/1962255809 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f0c240af9f0 con 0x7f0c380fdb60 2024-04-25T02:17:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.654+0000 7f0c3cbca640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c2806e0b0 0x7f0c28070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:58.660 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.655+0000 7f0c3cbca640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c2806e0b0 0x7f0c28070560 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c007fd0 tx=0x7f0c2c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:58.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.659+0000 7f0c357fa640 1 -- 172.21.15.33:0/1962255809 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0c240b1e60 con 0x7f0c380fdb60 2024-04-25T02:17:58.887 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.883+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f0c38109310 con 0x7f0c380fdb60 2024-04-25T02:17:58.888 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.884+0000 7f0c357fa640 1 -- 172.21.15.33:0/1962255809 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v38) v1 ==== 107+0+4927 (secure 0 0 0) 0x7f0c24078d70 con 0x7f0c380fdb60 2024-04-25T02:17:58.888 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:58.888 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34378,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2956426668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2956426668},{"type":"v1","addr":"172.21.15.33:6829","nonce":2956426668}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1198640100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1198640100},{"type":"v1","addr":"172.21.15.138:6827","nonce":1198640100}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44275,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2355068967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2355068967},{"type":"v1","addr":"172.21.15.138:6825","nonce":2355068967}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:58.889 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 24 2024-04-25T02:17:58.890 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c2806e0b0 msgr2=0x7f0c28070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c2806e0b0 0x7f0c28070560 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c007fd0 tx=0x7f0c2c009040 comp rx=0 tx=0).stop 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 msgr2=0x7f0c381132d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 0x7f0c381132d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f0c240047f0 tx=0x7f0c24009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 shutdown_connections 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f0c2806e0b0 0x7f0c28070560 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f0c380fdb60 0x7f0c381132d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 --2- 172.21.15.33:0/1962255809 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f0c380fd1c0 0x7f0c38112d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 >> 172.21.15.33:0/1962255809 conn(0x7f0c380f6c00 msgr2=0x7f0c380f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 shutdown_connections 2024-04-25T02:17:58.891 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.886+0000 7f0c3ee2e640 1 -- 172.21.15.33:0/1962255809 wait complete. 2024-04-25T02:17:58.901 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:17:58.902 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-04-25T02:17:58.902 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.996+0000 7f3c5986e640 1 -- 172.21.15.33:0/464957363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 msgr2=0x7f3c540fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.996+0000 7f3c5986e640 1 --2- 172.21.15.33:0/464957363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 0x7f3c540fa8c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3c480048d0 tx=0x7f3c48009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.996+0000 7f3c5986e640 1 -- 172.21.15.33:0/464957363 shutdown_connections 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.996+0000 7f3c5986e640 1 --2- 172.21.15.33:0/464957363 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 0x7f3c540fa8c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.996+0000 7f3c5986e640 1 --2- 172.21.15.33:0/464957363 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102230 0x7f3c54102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.996+0000 7f3c5986e640 1 -- 172.21.15.33:0/464957363 >> 172.21.15.33:0/464957363 conn(0x7f3c540717e0 msgr2=0x7f3c54071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.996+0000 7f3c5986e640 1 -- 172.21.15.33:0/464957363 shutdown_connections 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.996+0000 7f3c5986e640 1 -- 172.21.15.33:0/464957363 wait complete. 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.997+0000 7f3c5986e640 1 Processor -- start 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.997+0000 7f3c5986e640 1 -- start start 2024-04-25T02:17:59.003 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.997+0000 7f3c5986e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102230 0x7f3c54112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.997+0000 7f3c5986e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 0x7f3c54113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.997+0000 7f3c5986e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54113890 con 0x7f3c54102bd0 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.997+0000 7f3c5986e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5410cee0 con 0x7f3c54102230 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.997+0000 7f3c527fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 0x7f3c54113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.997+0000 7f3c527fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 0x7f3c54113290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51534/0 (socket says 172.21.15.33:51534) 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.997+0000 7f3c527fc640 1 -- 172.21.15.33:0/1214095727 learned_addr learned my addr 172.21.15.33:0/1214095727 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c52ffd640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102230 0x7f3c54112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c527fc640 1 -- 172.21.15.33:0/1214095727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102230 msgr2=0x7f3c54112d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c527fc640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102230 0x7f3c54112d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c527fc640 1 -- 172.21.15.33:0/1214095727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48005040 con 0x7f3c54102bd0 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c52ffd640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102230 0x7f3c54112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c527fc640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 0x7f3c54113290 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f3c4802bda0 tx=0x7f3c48009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:59.004 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c3b7fe640 1 -- 172.21.15.33:0/1214095727 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c480094c0 con 0x7f3c54102bd0 2024-04-25T02:17:59.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c3b7fe640 1 -- 172.21.15.33:0/1214095727 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c48032e50 con 0x7f3c54102bd0 2024-04-25T02:17:59.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5410d160 con 0x7f3c54102bd0 2024-04-25T02:17:59.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c3b7fe640 1 -- 172.21.15.33:0/1214095727 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c4803b400 con 0x7f3c54102bd0 2024-04-25T02:17:59.005 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.998+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5410d680 con 0x7f3c54102bd0 2024-04-25T02:17:59.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.999+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c54065530 con 0x7f3c54102bd0 2024-04-25T02:17:59.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:58.999+0000 7f3c3b7fe640 1 -- 172.21.15.33:0/1214095727 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3c4802f070 con 0x7f3c54102bd0 2024-04-25T02:17:59.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.000+0000 7f3c3b7fe640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c3c06e0b0 0x7f3c3c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:59.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.000+0000 7f3c3b7fe640 1 -- 172.21.15.33:0/1214095727 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3c480aff40 con 0x7f3c54102bd0 2024-04-25T02:17:59.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.000+0000 7f3c52ffd640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c3c06e0b0 0x7f3c3c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:59.006 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.000+0000 7f3c52ffd640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c3c06e0b0 0x7f3c3c070560 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3c44004d80 tx=0x7f3c4400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:59.009 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.005+0000 7f3c3b7fe640 1 -- 172.21.15.33:0/1214095727 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3c480e4740 con 0x7f3c54102bd0 2024-04-25T02:17:59.234 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.230+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f3c541093b0 con 0x7f3c54102bd0 2024-04-25T02:17:59.235 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.231+0000 7f3c3b7fe640 1 -- 172.21.15.33:0/1214095727 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v38) v1 ==== 107+0+4148 (secure 0 0 0) 0x7f3c480792c0 con 0x7f3c54102bd0 2024-04-25T02:17:59.235 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:59.236 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44267,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1198640100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1198640100},{"type":"v1","addr":"172.21.15.138:6827","nonce":1198640100}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44275,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2355068967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2355068967},{"type":"v1","addr":"172.21.15.138:6825","nonce":2355068967}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:59.236 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 25 2024-04-25T02:17:59.237 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c3c06e0b0 msgr2=0x7f3c3c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c3c06e0b0 0x7f3c3c070560 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3c44004d80 tx=0x7f3c4400b040 comp rx=0 tx=0).stop 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 msgr2=0x7f3c54113290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 0x7f3c54113290 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f3c4802bda0 tx=0x7f3c48009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 shutdown_connections 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3c3c06e0b0 0x7f3c3c070560 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3c54102bd0 0x7f3c54113290 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 --2- 172.21.15.33:0/1214095727 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3c54102230 0x7f3c54112d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 >> 172.21.15.33:0/1214095727 conn(0x7f3c540717e0 msgr2=0x7f3c540fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 shutdown_connections 2024-04-25T02:17:59.238 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.233+0000 7f3c5986e640 1 -- 172.21.15.33:0/1214095727 wait complete. 2024-04-25T02:17:59.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:58 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2385689629' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-25T02:17:59.247 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:58 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/70497394' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-25T02:17:59.248 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:17:59.248 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-04-25T02:17:59.249 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-25T02:17:59.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:58 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2385689629' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-25T02:17:59.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:58 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/70497394' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-25T02:17:59.347 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/307978958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc101170 msgr2=0x7fe7cc101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 --2- 172.21.15.33:0/307978958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc101170 0x7fe7cc101540 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80048d0 tx=0x7fe7c8009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/307978958 shutdown_connections 2024-04-25T02:17:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 --2- 172.21.15.33:0/307978958 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7cc0fb150 0x7fe7cc0fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 --2- 172.21.15.33:0/307978958 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc101170 0x7fe7cc101540 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/307978958 >> 172.21.15.33:0/307978958 conn(0x7fe7cc071fd0 msgr2=0x7fe7cc0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/307978958 shutdown_connections 2024-04-25T02:17:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/307978958 wait complete. 2024-04-25T02:17:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 Processor -- start 2024-04-25T02:17:59.348 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.343+0000 7fe7d3e2e640 1 -- start start 2024-04-25T02:17:59.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d3e2e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0fb150 0x7fe7cc10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:59.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d3e2e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7cc101170 0x7fe7cc10e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:59.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d3e2e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc065310 con 0x7fe7cc0fb150 2024-04-25T02:17:59.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d3e2e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc10e8f0 con 0x7fe7cc101170 2024-04-25T02:17:59.350 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d1bca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0fb150 0x7fe7cc10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d1bca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0fb150 0x7fe7cc10de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51546/0 (socket says 172.21.15.33:51546) 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d1bca640 1 -- 172.21.15.33:0/23308791 learned_addr learned my addr 172.21.15.33:0/23308791 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d13c9640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7cc101170 0x7fe7cc10e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d1bca640 1 -- 172.21.15.33:0/23308791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7cc101170 msgr2=0x7fe7cc10e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d1bca640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7cc101170 0x7fe7cc10e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d1bca640 1 -- 172.21.15.33:0/23308791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8005040 con 0x7fe7cc0fb150 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d13c9640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7cc101170 0x7fe7cc10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d1bca640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0fb150 0x7fe7cc10de40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8004930 tx=0x7fe7c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7c27fc640 1 -- 172.21.15.33:0/23308791 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7c80094c0 con 0x7fe7cc0fb150 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc10eb70 con 0x7fe7cc0fb150 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7c27fc640 1 -- 172.21.15.33:0/23308791 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe7c8032e50 con 0x7fe7cc0fb150 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7c27fc640 1 -- 172.21.15.33:0/23308791 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7c80433b0 con 0x7fe7cc0fb150 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.344+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc1170a0 con 0x7fe7cc0fb150 2024-04-25T02:17:59.351 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.345+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc0fc870 con 0x7fe7cc0fb150 2024-04-25T02:17:59.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.346+0000 7fe7c27fc640 1 -- 172.21.15.33:0/23308791 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe7c802f070 con 0x7fe7cc0fb150 2024-04-25T02:17:59.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.346+0000 7fe7c27fc640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe7b806e0b0 0x7fe7b8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:59.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.346+0000 7fe7c27fc640 1 -- 172.21.15.33:0/23308791 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe7c80af9d0 con 0x7fe7cc0fb150 2024-04-25T02:17:59.352 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.346+0000 7fe7d13c9640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe7b806e0b0 0x7fe7b8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:59.353 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.347+0000 7fe7d13c9640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe7b806e0b0 0x7fe7b8070560 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc064b60 tx=0x7fe7bc009eb0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:59.355 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.352+0000 7fe7c27fc640 1 -- 172.21.15.33:0/23308791 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe7c8078d50 con 0x7fe7cc0fb150 2024-04-25T02:17:59.577 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.573+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7fe7cc063f00 con 0x7fe7cc0fb150 2024-04-25T02:17:59.578 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.574+0000 7fe7c27fc640 1 -- 172.21.15.33:0/23308791 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v38) v1 ==== 107+0+4995 (secure 0 0 0) 0x7fe7c807c410 con 0x7fe7cc0fb150 2024-04-25T02:17:59.578 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:59.578 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44267,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1198640100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1198640100},{"type":"v1","addr":"172.21.15.138:6827","nonce":1198640100}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44275,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/2355068967","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":2355068967},{"type":"v1","addr":"172.21.15.138:6825","nonce":2355068967}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:59.579 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 26 2024-04-25T02:17:59.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe7b806e0b0 msgr2=0x7fe7b8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe7b806e0b0 0x7fe7b8070560 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc064b60 tx=0x7fe7bc009eb0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0fb150 msgr2=0x7fe7cc10de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0fb150 0x7fe7cc10de40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8004930 tx=0x7fe7c8009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 shutdown_connections 2024-04-25T02:17:59.580 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe7b806e0b0 0x7fe7b8070560 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe7cc101170 0x7fe7cc10e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 --2- 172.21.15.33:0/23308791 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe7cc0fb150 0x7fe7cc10de40 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 >> 172.21.15.33:0/23308791 conn(0x7fe7cc071fd0 msgr2=0x7fe7cc0fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:59.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 shutdown_connections 2024-04-25T02:17:59.581 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.576+0000 7fe7d3e2e640 1 -- 172.21.15.33:0/23308791 wait complete. 2024-04-25T02:17:59.591 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:17:59.591 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-04-25T02:17:59.591 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-25T02:17:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.685+0000 7f3fa815d640 1 -- 172.21.15.33:0/470004801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 msgr2=0x7f3fa00fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.685+0000 7f3fa815d640 1 --2- 172.21.15.33:0/470004801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 0x7f3fa00fc1e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f3f900048d0 tx=0x7f3f90009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.685+0000 7f3fa815d640 1 -- 172.21.15.33:0/470004801 shutdown_connections 2024-04-25T02:17:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.685+0000 7f3fa815d640 1 --2- 172.21.15.33:0/470004801 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 0x7f3fa00fc1e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.692 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.685+0000 7f3fa815d640 1 --2- 172.21.15.33:0/470004801 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa01021b0 0x7f3fa0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.685+0000 7f3fa815d640 1 -- 172.21.15.33:0/470004801 >> 172.21.15.33:0/470004801 conn(0x7f3fa00f7d80 msgr2=0x7f3fa00fa190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.685+0000 7f3fa815d640 1 -- 172.21.15.33:0/470004801 shutdown_connections 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.685+0000 7f3fa815d640 1 -- 172.21.15.33:0/470004801 wait complete. 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa815d640 1 Processor -- start 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa815d640 1 -- start start 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa815d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa01021b0 0x7f3fa0114ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa815d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 0x7f3fa01153e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa815d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa01159e0 con 0x7f3fa0102b50 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa815d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa010f030 con 0x7f3fa01021b0 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa56f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 0x7f3fa01153e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa56f8640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 0x7f3fa01153e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51572/0 (socket says 172.21.15.33:51572) 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa56f8640 1 -- 172.21.15.33:0/579709537 learned_addr learned my addr 172.21.15.33:0/579709537 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa5ef9640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa01021b0 0x7f3fa0114ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa56f8640 1 -- 172.21.15.33:0/579709537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa01021b0 msgr2=0x7f3fa0114ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.693 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.686+0000 7f3fa56f8640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa01021b0 0x7f3fa0114ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.687+0000 7f3fa56f8640 1 -- 172.21.15.33:0/579709537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f90005040 con 0x7f3fa0102b50 2024-04-25T02:17:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.687+0000 7f3fa5ef9640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa01021b0 0x7f3fa0114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:17:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.687+0000 7f3fa56f8640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 0x7f3fa01153e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3f90004930 tx=0x7f3f90009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.687+0000 7f3f967fc640 1 -- 172.21.15.33:0/579709537 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f900094c0 con 0x7f3fa0102b50 2024-04-25T02:17:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.687+0000 7f3f967fc640 1 -- 172.21.15.33:0/579709537 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f90032e50 con 0x7f3fa0102b50 2024-04-25T02:17:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.687+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa010f2b0 con 0x7f3fa0102b50 2024-04-25T02:17:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.687+0000 7f3f967fc640 1 -- 172.21.15.33:0/579709537 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f900433b0 con 0x7f3fa0102b50 2024-04-25T02:17:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.687+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa010f770 con 0x7f3fa0102b50 2024-04-25T02:17:59.694 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.688+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa00fd920 con 0x7f3fa0102b50 2024-04-25T02:17:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.689+0000 7f3f967fc640 1 -- 172.21.15.33:0/579709537 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3f9002f070 con 0x7f3fa0102b50 2024-04-25T02:17:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.689+0000 7f3f967fc640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f8c06e0b0 0x7f3f8c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:17:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.689+0000 7f3fa5ef9640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f8c06e0b0 0x7f3f8c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:17:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.689+0000 7f3f967fc640 1 -- 172.21.15.33:0/579709537 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3f900af9a0 con 0x7f3fa0102b50 2024-04-25T02:17:59.695 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.690+0000 7f3fa5ef9640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f8c06e0b0 0x7f3f8c070560 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009af0 tx=0x7f3f9c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:17:59.698 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.694+0000 7f3f967fc640 1 -- 172.21.15.33:0/579709537 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3f90078d20 con 0x7f3fa0102b50 2024-04-25T02:17:59.922 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.918+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f3fa0109320 con 0x7f3fa0102b50 2024-04-25T02:17:59.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.918+0000 7f3f967fc640 1 -- 172.21.15.33:0/579709537 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v38) v1 ==== 107+0+4213 (secure 0 0 0) 0x7f3f9007c3e0 con 0x7f3fa0102b50 2024-04-25T02:17:59.923 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:17:59.923 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44267,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1198640100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1198640100},{"type":"v1","addr":"172.21.15.138:6827","nonce":1198640100}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:17:59.923 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 27 2024-04-25T02:17:59.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.920+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f8c06e0b0 msgr2=0x7f3f8c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.920+0000 7f3fa815d640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f8c06e0b0 0x7f3f8c070560 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009af0 tx=0x7f3f9c00b040 comp rx=0 tx=0).stop 2024-04-25T02:17:59.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.920+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 msgr2=0x7f3fa01153e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:17:59.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.920+0000 7f3fa815d640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 0x7f3fa01153e0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f3f90004930 tx=0x7f3f90009da0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.920+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 shutdown_connections 2024-04-25T02:17:59.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.920+0000 7f3fa815d640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f3f8c06e0b0 0x7f3f8c070560 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.920+0000 7f3fa815d640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f3fa0102b50 0x7f3fa01153e0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.920+0000 7f3fa815d640 1 --2- 172.21.15.33:0/579709537 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f3fa01021b0 0x7f3fa0114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:17:59.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.920+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 >> 172.21.15.33:0/579709537 conn(0x7f3fa00f7d80 msgr2=0x7f3fa0071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:17:59.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.921+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 shutdown_connections 2024-04-25T02:17:59.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:17:59.921+0000 7f3fa815d640 1 -- 172.21.15.33:0/579709537 wait complete. 2024-04-25T02:17:59.936 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:17:59.936 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-04-25T02:17:59.936 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-25T02:18:00.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 -- 172.21.15.33:0/2057678362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc0fb9d0 msgr2=0x7f46cc0fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 --2- 172.21.15.33:0/2057678362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc0fb9d0 0x7f46cc0fbe40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0048d0 tx=0x7f46bc009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 -- 172.21.15.33:0/2057678362 shutdown_connections 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 --2- 172.21.15.33:0/2057678362 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc0fb9d0 0x7f46cc0fbe40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 --2- 172.21.15.33:0/2057678362 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46cc100160 0x7f46cc100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 -- 172.21.15.33:0/2057678362 >> 172.21.15.33:0/2057678362 conn(0x7f46cc0717e0 msgr2=0x7f46cc071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 -- 172.21.15.33:0/2057678362 shutdown_connections 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 -- 172.21.15.33:0/2057678362 wait complete. 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 Processor -- start 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 -- start start 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46cc0fb9d0 0x7f46cc112e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc100160 0x7f46cc1133a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc113a30 con 0x7f46cc100160 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d3d14640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46cc10cff0 con 0x7f46cc0fb9d0 2024-04-25T02:18:00.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d12af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc100160 0x7f46cc1133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d12af640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc100160 0x7f46cc1133a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51588/0 (socket says 172.21.15.33:51588) 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.031+0000 7f46d12af640 1 -- 172.21.15.33:0/2392906045 learned_addr learned my addr 172.21.15.33:0/2392906045 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46d1ab0640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46cc0fb9d0 0x7f46cc112e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46d12af640 1 -- 172.21.15.33:0/2392906045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46cc0fb9d0 msgr2=0x7f46cc112e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46d12af640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46cc0fb9d0 0x7f46cc112e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46d12af640 1 -- 172.21.15.33:0/2392906045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc005040 con 0x7f46cc100160 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46d12af640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc100160 0x7f46cc1133a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f46bc004930 tx=0x7f46bc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46c27fc640 1 -- 172.21.15.33:0/2392906045 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc0094c0 con 0x7f46cc100160 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46cc10d270 con 0x7f46cc100160 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46c27fc640 1 -- 172.21.15.33:0/2392906045 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46bc032e50 con 0x7f46cc100160 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46c27fc640 1 -- 172.21.15.33:0/2392906045 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46bc0433b0 con 0x7f46cc100160 2024-04-25T02:18:00.039 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.032+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46cc10d730 con 0x7f46cc100160 2024-04-25T02:18:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.034+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46cc0fc9e0 con 0x7f46cc100160 2024-04-25T02:18:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.035+0000 7f46c27fc640 1 -- 172.21.15.33:0/2392906045 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f46bc02f070 con 0x7f46cc100160 2024-04-25T02:18:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.035+0000 7f46c27fc640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46b806e0b0 0x7f46b8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.035+0000 7f46c27fc640 1 -- 172.21.15.33:0/2392906045 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f46bc0afa80 con 0x7f46cc100160 2024-04-25T02:18:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.035+0000 7f46d1ab0640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46b806e0b0 0x7f46b8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:00.040 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.035+0000 7f46d1ab0640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46b806e0b0 0x7f46b8070560 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f46c8006fd0 tx=0x7f46c8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:00.043 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.040+0000 7f46c27fc640 1 -- 172.21.15.33:0/2392906045 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f46bc078e00 con 0x7f46cc100160 2024-04-25T02:18:00.267 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.264+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f46cc064350 con 0x7f46cc100160 2024-04-25T02:18:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.264+0000 7f46c27fc640 1 -- 172.21.15.33:0/2392906045 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v38) v1 ==== 107+0+5060 (secure 0 0 0) 0x7f46bc07c4c0 con 0x7f46cc100160 2024-04-25T02:18:00.268 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:00.268 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44267,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/1198640100","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":1198640100},{"type":"v1","addr":"172.21.15.138:6827","nonce":1198640100}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21},{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:18:00.268 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 28 2024-04-25T02:18:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46b806e0b0 msgr2=0x7f46b8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46b806e0b0 0x7f46b8070560 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f46c8006fd0 tx=0x7f46c8008040 comp rx=0 tx=0).stop 2024-04-25T02:18:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc100160 msgr2=0x7f46cc1133a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.270 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc100160 0x7f46cc1133a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f46bc004930 tx=0x7f46bc009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 shutdown_connections 2024-04-25T02:18:00.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f46b806e0b0 0x7f46b8070560 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f46cc100160 0x7f46cc1133a0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 --2- 172.21.15.33:0/2392906045 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f46cc0fb9d0 0x7f46cc112e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 >> 172.21.15.33:0/2392906045 conn(0x7f46cc0717e0 msgr2=0x7f46cc0fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:00.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 shutdown_connections 2024-04-25T02:18:00.271 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.266+0000 7f46d3d14640 1 -- 172.21.15.33:0/2392906045 wait complete. 2024-04-25T02:18:00.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:59 smithi033 ceph-mon[111306]: pgmap v1161: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:00.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:59 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1962255809' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-25T02:18:00.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:59 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1214095727' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-25T02:18:00.271 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:17:59 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/23308791' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-25T02:18:00.281 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:18:00.281 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-04-25T02:18:00.281 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-25T02:18:00.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:59 smithi138 ceph-mon[81304]: pgmap v1161: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:00.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:59 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1962255809' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-25T02:18:00.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:59 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1214095727' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-25T02:18:00.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:17:59 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/23308791' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-25T02:18:00.381 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.376+0000 7fac970ae640 1 -- 172.21.15.33:0/1205943845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900fb9d0 msgr2=0x7fac900fbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.376+0000 7fac970ae640 1 --2- 172.21.15.33:0/1205943845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900fb9d0 0x7fac900fbe40 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fac800048d0 tx=0x7fac80009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.376+0000 7fac970ae640 1 -- 172.21.15.33:0/1205943845 shutdown_connections 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.376+0000 7fac970ae640 1 --2- 172.21.15.33:0/1205943845 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac900fb9d0 0x7fac900fbe40 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.376+0000 7fac970ae640 1 --2- 172.21.15.33:0/1205943845 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac90100160 0x7fac90100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.376+0000 7fac970ae640 1 -- 172.21.15.33:0/1205943845 >> 172.21.15.33:0/1205943845 conn(0x7fac900717e0 msgr2=0x7fac90071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.376+0000 7fac970ae640 1 -- 172.21.15.33:0/1205943845 shutdown_connections 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.376+0000 7fac970ae640 1 -- 172.21.15.33:0/1205943845 wait complete. 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac970ae640 1 Processor -- start 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac970ae640 1 -- start start 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac970ae640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac900fb9d0 0x7fac90112de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac970ae640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90100160 0x7fac90113320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:00.382 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac970ae640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac90113920 con 0x7fac90100160 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac970ae640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9010cf70 con 0x7fac900fb9d0 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac87fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90100160 0x7fac90113320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac87fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90100160 0x7fac90113320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51616/0 (socket says 172.21.15.33:51616) 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac87fff640 1 -- 172.21.15.33:0/84034402 learned_addr learned my addr 172.21.15.33:0/84034402 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac94e4a640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac900fb9d0 0x7fac90112de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac87fff640 1 -- 172.21.15.33:0/84034402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac900fb9d0 msgr2=0x7fac90112de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac87fff640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac900fb9d0 0x7fac90112de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.377+0000 7fac87fff640 1 -- 172.21.15.33:0/84034402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac80005040 con 0x7fac90100160 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.378+0000 7fac87fff640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90100160 0x7fac90113320 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fac80034040 tx=0x7fac80009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.378+0000 7fac857fa640 1 -- 172.21.15.33:0/84034402 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac800092b0 con 0x7fac90100160 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.378+0000 7fac857fa640 1 -- 172.21.15.33:0/84034402 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac8003b400 con 0x7fac90100160 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.378+0000 7fac857fa640 1 -- 172.21.15.33:0/84034402 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac800435d0 con 0x7fac90100160 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.378+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9010d1f0 con 0x7fac90100160 2024-04-25T02:18:00.383 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.378+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9010d6b0 con 0x7fac90100160 2024-04-25T02:18:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.379+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac900fc9e0 con 0x7fac90100160 2024-04-25T02:18:00.384 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.379+0000 7fac857fa640 1 -- 172.21.15.33:0/84034402 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fac8002f070 con 0x7fac90100160 2024-04-25T02:18:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.380+0000 7fac857fa640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac7806e0b0 0x7fac78070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.380+0000 7fac857fa640 1 -- 172.21.15.33:0/84034402 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fac800af810 con 0x7fac90100160 2024-04-25T02:18:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.380+0000 7fac94e4a640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac7806e0b0 0x7fac78070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:00.385 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.380+0000 7fac94e4a640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac7806e0b0 0x7fac78070560 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0049b0 tx=0x7fac8c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:00.388 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.384+0000 7fac857fa640 1 -- 172.21.15.33:0/84034402 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fac80078b90 con 0x7fac90100160 2024-04-25T02:18:00.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.605+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7fac901083e0 con 0x7fac90100160 2024-04-25T02:18:00.609 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.606+0000 7fac857fa640 1 -- 172.21.15.33:0/84034402 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v38) v1 ==== 107+0+4278 (secure 0 0 0) 0x7fac8007c250 con 0x7fac90100160 2024-04-25T02:18:00.609 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:00.609 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:18:00.610 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 29 2024-04-25T02:18:00.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac7806e0b0 msgr2=0x7fac78070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac7806e0b0 0x7fac78070560 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0049b0 tx=0x7fac8c00b040 comp rx=0 tx=0).stop 2024-04-25T02:18:00.611 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90100160 msgr2=0x7fac90113320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90100160 0x7fac90113320 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fac80034040 tx=0x7fac80009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 shutdown_connections 2024-04-25T02:18:00.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fac7806e0b0 0x7fac78070560 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fac90100160 0x7fac90113320 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 --2- 172.21.15.33:0/84034402 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fac900fb9d0 0x7fac90112de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 >> 172.21.15.33:0/84034402 conn(0x7fac900717e0 msgr2=0x7fac900fb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:00.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 shutdown_connections 2024-04-25T02:18:00.612 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.608+0000 7fac970ae640 1 -- 172.21.15.33:0/84034402 wait complete. 2024-04-25T02:18:00.622 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:18:00.622 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-04-25T02:18:00.626 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-25T02:18:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.721+0000 7fa79022e640 1 -- 172.21.15.33:0/3446638504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 msgr2=0x7fa788106170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.721+0000 7fa79022e640 1 --2- 172.21.15.33:0/3446638504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 0x7fa788106170 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa7840048d0 tx=0x7fa784009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.721+0000 7fa79022e640 1 -- 172.21.15.33:0/3446638504 shutdown_connections 2024-04-25T02:18:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.721+0000 7fa79022e640 1 --2- 172.21.15.33:0/3446638504 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 0x7fa788106170 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.721+0000 7fa79022e640 1 --2- 172.21.15.33:0/3446638504 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7880fd190 0x7fa7880fd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.725 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.721+0000 7fa79022e640 1 -- 172.21.15.33:0/3446638504 >> 172.21.15.33:0/3446638504 conn(0x7fa7880f6be0 msgr2=0x7fa7880f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.721+0000 7fa79022e640 1 -- 172.21.15.33:0/3446638504 shutdown_connections 2024-04-25T02:18:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.721+0000 7fa79022e640 1 -- 172.21.15.33:0/3446638504 wait complete. 2024-04-25T02:18:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa79022e640 1 Processor -- start 2024-04-25T02:18:00.726 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa79022e640 1 -- start start 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa79022e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7880fd190 0x7fa788112cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa79022e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 0x7fa788113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa79022e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa788113810 con 0x7fa7880fdb30 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa79022e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78810ce60 con 0x7fa7880fd190 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa78d7c9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 0x7fa788113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa78d7c9640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 0x7fa788113210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51632/0 (socket says 172.21.15.33:51632) 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa78d7c9640 1 -- 172.21.15.33:0/2454537718 learned_addr learned my addr 172.21.15.33:0/2454537718 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.722+0000 7fa78dfca640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7880fd190 0x7fa788112cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.723+0000 7fa78d7c9640 1 -- 172.21.15.33:0/2454537718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7880fd190 msgr2=0x7fa788112cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.723+0000 7fa78d7c9640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7880fd190 0x7fa788112cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.723+0000 7fa78d7c9640 1 -- 172.21.15.33:0/2454537718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa784005040 con 0x7fa7880fdb30 2024-04-25T02:18:00.728 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.723+0000 7fa78d7c9640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 0x7fa788113210 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa784004930 tx=0x7fa784009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.723+0000 7fa77a7fc640 1 -- 172.21.15.33:0/2454537718 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7840094c0 con 0x7fa7880fdb30 2024-04-25T02:18:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.723+0000 7fa77a7fc640 1 -- 172.21.15.33:0/2454537718 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa784032e50 con 0x7fa7880fdb30 2024-04-25T02:18:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.723+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78810d0e0 con 0x7fa7880fdb30 2024-04-25T02:18:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.723+0000 7fa77a7fc640 1 -- 172.21.15.33:0/2454537718 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7840433b0 con 0x7fa7880fdb30 2024-04-25T02:18:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.723+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78810d5a0 con 0x7fa7880fdb30 2024-04-25T02:18:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.724+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa788066040 con 0x7fa7880fdb30 2024-04-25T02:18:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.724+0000 7fa77a7fc640 1 -- 172.21.15.33:0/2454537718 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa78402f070 con 0x7fa7880fdb30 2024-04-25T02:18:00.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.725+0000 7fa77a7fc640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa77406e0b0 0x7fa774070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.725+0000 7fa78dfca640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa77406e0b0 0x7fa774070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.725+0000 7fa77a7fc640 1 -- 172.21.15.33:0/2454537718 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa7840af820 con 0x7fa7880fdb30 2024-04-25T02:18:00.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.726+0000 7fa78dfca640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa77406e0b0 0x7fa774070560 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009fd0 tx=0x7fa77c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:00.733 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.730+0000 7fa77a7fc640 1 -- 172.21.15.33:0/2454537718 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa784078c50 con 0x7fa7880fdb30 2024-04-25T02:18:00.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.954+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fa7881092c0 con 0x7fa7880fdb30 2024-04-25T02:18:00.958 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.955+0000 7fa77a7fc640 1 -- 172.21.15.33:0/2454537718 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v38) v1 ==== 107+0+5125 (secure 0 0 0) 0x7fa78407c310 con 0x7fa7880fdb30 2024-04-25T02:18:00.959 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:00.959 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44841,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/882099891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":882099891},{"type":"v1","addr":"172.21.15.138:6827","nonce":882099891}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":21,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T01:45:10.323488+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:18:00.959 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 30 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa77406e0b0 msgr2=0x7fa774070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa77406e0b0 0x7fa774070560 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa77c009fd0 tx=0x7fa77c00b040 comp rx=0 tx=0).stop 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 msgr2=0x7fa788113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 0x7fa788113210 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa784004930 tx=0x7fa784009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 shutdown_connections 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa77406e0b0 0x7fa774070560 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa7880fdb30 0x7fa788113210 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 --2- 172.21.15.33:0/2454537718 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa7880fd190 0x7fa788112cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 >> 172.21.15.33:0/2454537718 conn(0x7fa7880f6be0 msgr2=0x7fa7880f80a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 shutdown_connections 2024-04-25T02:18:00.961 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:00.957+0000 7fa79022e640 1 -- 172.21.15.33:0/2454537718 wait complete. 2024-04-25T02:18:00.971 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-25T02:18:00.972 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-04-25T02:18:00.972 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-25T02:18:01.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.067+0000 7f740ac6e640 1 -- 172.21.15.33:0/1872080483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 msgr2=0x7f74041025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.072 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.067+0000 7f740ac6e640 1 --2- 172.21.15.33:0/1872080483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 0x7f74041025b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f73f80048d0 tx=0x7f73f8009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.067+0000 7f740ac6e640 1 -- 172.21.15.33:0/1872080483 shutdown_connections 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.067+0000 7f740ac6e640 1 --2- 172.21.15.33:0/1872080483 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404102b80 0x7f74040fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.067+0000 7f740ac6e640 1 --2- 172.21.15.33:0/1872080483 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 0x7f74041025b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.067+0000 7f740ac6e640 1 -- 172.21.15.33:0/1872080483 >> 172.21.15.33:0/1872080483 conn(0x7f74040f7db0 msgr2=0x7f74040fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.067+0000 7f740ac6e640 1 -- 172.21.15.33:0/1872080483 shutdown_connections 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.067+0000 7f740ac6e640 1 -- 172.21.15.33:0/1872080483 wait complete. 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f740ac6e640 1 Processor -- start 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f740ac6e640 1 -- start start 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f740ac6e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 0x7f7404112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f740ac6e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404102b80 0x7f7404113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f740ac6e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74041138b0 con 0x7f74041021e0 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f740ac6e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740410ce70 con 0x7f7404102b80 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f7408a0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 0x7f7404112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:01.073 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f7408a0a640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 0x7f7404112ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51646/0 (socket says 172.21.15.33:51646) 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f7408a0a640 1 -- 172.21.15.33:0/1619679020 learned_addr learned my addr 172.21.15.33:0/1619679020 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f7403fff640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404102b80 0x7f7404113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f7408a0a640 1 -- 172.21.15.33:0/1619679020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404102b80 msgr2=0x7f7404113220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f7408a0a640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404102b80 0x7f7404113220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.068+0000 7f7408a0a640 1 -- 172.21.15.33:0/1619679020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f8005040 con 0x7f74041021e0 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.069+0000 7f7408a0a640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 0x7f7404112ce0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f73f8004930 tx=0x7f73f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.069+0000 7f74017fa640 1 -- 172.21.15.33:0/1619679020 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f80094c0 con 0x7f74041021e0 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.069+0000 7f74017fa640 1 -- 172.21.15.33:0/1619679020 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73f8032e50 con 0x7f74041021e0 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.069+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740410d0f0 con 0x7f74041021e0 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.069+0000 7f74017fa640 1 -- 172.21.15.33:0/1619679020 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f80433b0 con 0x7f74041021e0 2024-04-25T02:18:01.074 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.069+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740410d5b0 con 0x7f74041021e0 2024-04-25T02:18:01.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.070+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74040fd950 con 0x7f74041021e0 2024-04-25T02:18:01.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.070+0000 7f74017fa640 1 -- 172.21.15.33:0/1619679020 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f73f802f070 con 0x7f74041021e0 2024-04-25T02:18:01.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.071+0000 7f74017fa640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f73f406e0b0 0x7f73f4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:01.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.071+0000 7f7403fff640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f73f406e0b0 0x7f73f4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:01.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.071+0000 7f74017fa640 1 -- 172.21.15.33:0/1619679020 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f73f80afb00 con 0x7f74041021e0 2024-04-25T02:18:01.075 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.071+0000 7f7403fff640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f73f406e0b0 0x7f73f4070560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f73f0006fd0 tx=0x7f73f0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:01.079 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.076+0000 7f74017fa640 1 -- 172.21.15.33:0/1619679020 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f73f8078e80 con 0x7f74041021e0 2024-04-25T02:18:01.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:00 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/579709537' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-25T02:18:01.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:00 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2392906045' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-25T02:18:01.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:00 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/84034402' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-25T02:18:01.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:00 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/579709537' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-25T02:18:01.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:00 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2392906045' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-25T02:18:01.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:00 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/84034402' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-25T02:18:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.308+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f740410e380 con 0x7f74041021e0 2024-04-25T02:18:01.312 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.309+0000 7f74017fa640 1 -- 172.21.15.33:0/1619679020 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v38) v1 ==== 107+0+5124 (secure 0 0 0) 0x7f73f807c540 con 0x7f74041021e0 2024-04-25T02:18:01.313 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:01.313 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26},{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44841,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/882099891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":882099891},{"type":"v1","addr":"172.21.15.138:6827","nonce":882099891}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T02:16:04.607027+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:18:01.313 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 31 2024-04-25T02:18:01.314 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f73f406e0b0 msgr2=0x7f73f4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f73f406e0b0 0x7f73f4070560 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f73f0006fd0 tx=0x7f73f0008040 comp rx=0 tx=0).stop 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 msgr2=0x7f7404112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 0x7f7404112ce0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f73f8004930 tx=0x7f73f8009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 shutdown_connections 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f73f406e0b0 0x7f73f4070560 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f7404102b80 0x7f7404113220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 --2- 172.21.15.33:0/1619679020 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f74041021e0 0x7f7404112ce0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 >> 172.21.15.33:0/1619679020 conn(0x7f74040f7db0 msgr2=0x7f74040f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 shutdown_connections 2024-04-25T02:18:01.315 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.311+0000 7f740ac6e640 1 -- 172.21.15.33:0/1619679020 wait complete. 2024-04-25T02:18:01.325 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-04-25T02:18:01.326 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.421+0000 7fcc8439d640 1 -- 172.21.15.33:0/2477032456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c1021e0 msgr2=0x7fcc7c1025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.421+0000 7fcc8439d640 1 --2- 172.21.15.33:0/2477032456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c1021e0 0x7fcc7c1025b0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fcc780048d0 tx=0x7fcc78009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.421+0000 7fcc8439d640 1 -- 172.21.15.33:0/2477032456 shutdown_connections 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.421+0000 7fcc8439d640 1 --2- 172.21.15.33:0/2477032456 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c102b80 0x7fcc7c0fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.421+0000 7fcc8439d640 1 --2- 172.21.15.33:0/2477032456 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c1021e0 0x7fcc7c1025b0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.421+0000 7fcc8439d640 1 -- 172.21.15.33:0/2477032456 >> 172.21.15.33:0/2477032456 conn(0x7fcc7c0f7db0 msgr2=0x7fcc7c0fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.421+0000 7fcc8439d640 1 -- 172.21.15.33:0/2477032456 shutdown_connections 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.421+0000 7fcc8439d640 1 -- 172.21.15.33:0/2477032456 wait complete. 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.421+0000 7fcc8439d640 1 Processor -- start 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc8439d640 1 -- start start 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc8439d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c1021e0 0x7fcc7c112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:01.429 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc8439d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c102b80 0x7fcc7c1131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc8439d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c1137e0 con 0x7fcc7c102b80 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc8439d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7c10ce30 con 0x7fcc7c1021e0 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc81938640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c102b80 0x7fcc7c1131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc81938640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c102b80 0x7fcc7c1131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51660/0 (socket says 172.21.15.33:51660) 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc81938640 1 -- 172.21.15.33:0/3633061137 learned_addr learned my addr 172.21.15.33:0/3633061137 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc82139640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c1021e0 0x7fcc7c112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc81938640 1 -- 172.21.15.33:0/3633061137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c1021e0 msgr2=0x7fcc7c112ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc81938640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c1021e0 0x7fcc7c112ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc81938640 1 -- 172.21.15.33:0/3633061137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc78005040 con 0x7fcc7c102b80 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc81938640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c102b80 0x7fcc7c1131e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c002a50 tx=0x7fcc6c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc72ffd640 1 -- 172.21.15.33:0/3633061137 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc6c007c30 con 0x7fcc7c102b80 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc72ffd640 1 -- 172.21.15.33:0/3633061137 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc6c00f040 con 0x7fcc7c102b80 2024-04-25T02:18:01.430 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.422+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7c10d110 con 0x7fcc7c102b80 2024-04-25T02:18:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.423+0000 7fcc72ffd640 1 -- 172.21.15.33:0/3633061137 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc6c00dad0 con 0x7fcc7c102b80 2024-04-25T02:18:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.423+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc7c10d630 con 0x7fcc7c102b80 2024-04-25T02:18:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.424+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc7c0fd950 con 0x7fcc7c102b80 2024-04-25T02:18:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.424+0000 7fcc72ffd640 1 -- 172.21.15.33:0/3633061137 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcc6c00dd40 con 0x7fcc7c102b80 2024-04-25T02:18:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.424+0000 7fcc72ffd640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc6806e0b0 0x7fcc68070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.425+0000 7fcc72ffd640 1 -- 172.21.15.33:0/3633061137 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fcc6c08d330 con 0x7fcc7c102b80 2024-04-25T02:18:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.425+0000 7fcc82139640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc6806e0b0 0x7fcc68070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:01.431 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.425+0000 7fcc82139640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc6806e0b0 0x7fcc68070560 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcc78000c00 tx=0x7fcc78009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:01.432 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.429+0000 7fcc72ffd640 1 -- 172.21.15.33:0/3633061137 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcc6c056760 con 0x7fcc7c102b80 2024-04-25T02:18:01.655 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.652+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fcc7c109340 con 0x7fcc7c102b80 2024-04-25T02:18:01.656 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.652+0000 7fcc72ffd640 1 -- 172.21.15.33:0/3633061137 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v38) v1 ==== 107+0+5215 (secure 0 0 0) 0x7fcc6c059e20 con 0x7fcc7c102b80 2024-04-25T02:18:01.656 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:01.656 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44841,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/882099891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":882099891},{"type":"v1","addr":"172.21.15.138:6827","nonce":882099891}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T02:16:05.607840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":1,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:18:01.656 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 32 2024-04-25T02:18:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.654+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc6806e0b0 msgr2=0x7fcc68070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.658 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.654+0000 7fcc8439d640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc6806e0b0 0x7fcc68070560 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcc78000c00 tx=0x7fcc78009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.654+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c102b80 msgr2=0x7fcc7c1131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.654+0000 7fcc8439d640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c102b80 0x7fcc7c1131e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c002a50 tx=0x7fcc6c004600 comp rx=0 tx=0).stop 2024-04-25T02:18:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.655+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 shutdown_connections 2024-04-25T02:18:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.655+0000 7fcc8439d640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fcc6806e0b0 0x7fcc68070560 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.655+0000 7fcc8439d640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fcc7c102b80 0x7fcc7c1131e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.655+0000 7fcc8439d640 1 --2- 172.21.15.33:0/3633061137 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fcc7c1021e0 0x7fcc7c112ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.655+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 >> 172.21.15.33:0/3633061137 conn(0x7fcc7c0f7db0 msgr2=0x7fcc7c0f9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.655+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 shutdown_connections 2024-04-25T02:18:01.659 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.655+0000 7fcc8439d640 1 -- 172.21.15.33:0/3633061137 wait complete. 2024-04-25T02:18:01.669 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-04-25T02:18:01.669 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-25T02:18:01.769 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.764+0000 7fb0c8914640 1 -- 172.21.15.33:0/2760712532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101a30 msgr2=0x7fb0c40fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.764+0000 7fb0c8914640 1 --2- 172.21.15.33:0/2760712532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101a30 0x7fb0c40fb0c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80048d0 tx=0x7fb0b8009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.764+0000 7fb0c8914640 1 -- 172.21.15.33:0/2760712532 shutdown_connections 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.764+0000 7fb0c8914640 1 --2- 172.21.15.33:0/2760712532 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101a30 0x7fb0c40fb0c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.764+0000 7fb0c8914640 1 --2- 172.21.15.33:0/2760712532 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4101090 0x7fb0c4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.764+0000 7fb0c8914640 1 -- 172.21.15.33:0/2760712532 >> 172.21.15.33:0/2760712532 conn(0x7fb0c40f6c20 msgr2=0x7fb0c40f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.764+0000 7fb0c8914640 1 -- 172.21.15.33:0/2760712532 shutdown_connections 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.764+0000 7fb0c8914640 1 -- 172.21.15.33:0/2760712532 wait complete. 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c8914640 1 Processor -- start 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c8914640 1 -- start start 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c8914640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101090 0x7fb0c410dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c8914640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4101a30 0x7fb0c410e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c8914640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c410e960 con 0x7fb0c4101090 2024-04-25T02:18:01.770 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c8914640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c410ead0 con 0x7fb0c4101a30 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101090 0x7fb0c410dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c259c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101090 0x7fb0c410dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51682/0 (socket says 172.21.15.33:51682) 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c259c640 1 -- 172.21.15.33:0/4215317158 learned_addr learned my addr 172.21.15.33:0/4215317158 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c1d9b640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4101a30 0x7fb0c410e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c259c640 1 -- 172.21.15.33:0/4215317158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4101a30 msgr2=0x7fb0c410e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c259c640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4101a30 0x7fb0c410e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c259c640 1 -- 172.21.15.33:0/4215317158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b8005040 con 0x7fb0c4101090 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0c259c640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101090 0x7fb0c410dee0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400db00 tx=0x7fb0b4004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.765+0000 7fb0b2ffd640 1 -- 172.21.15.33:0/4215317158 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0b4017070 con 0x7fb0c4101090 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.766+0000 7fb0b2ffd640 1 -- 172.21.15.33:0/4215317158 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb0b400f040 con 0x7fb0c4101090 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.766+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c4116f50 con 0x7fb0c4101090 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.766+0000 7fb0b2ffd640 1 -- 172.21.15.33:0/4215317158 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0b4003ae0 con 0x7fb0c4101090 2024-04-25T02:18:01.771 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.766+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c4117280 con 0x7fb0c4101090 2024-04-25T02:18:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.766+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c40fc800 con 0x7fb0c4101090 2024-04-25T02:18:01.772 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.767+0000 7fb0b2ffd640 1 -- 172.21.15.33:0/4215317158 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb0b4003c40 con 0x7fb0c4101090 2024-04-25T02:18:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.767+0000 7fb0b2ffd640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb0ac06e0b0 0x7fb0ac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.768+0000 7fb0b2ffd640 1 -- 172.21.15.33:0/4215317158 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb0b408dbd0 con 0x7fb0c4101090 2024-04-25T02:18:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.768+0000 7fb0c1d9b640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb0ac06e0b0 0x7fb0ac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:01.773 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.768+0000 7fb0c1d9b640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb0ac06e0b0 0x7fb0ac070560 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4065910 tx=0x7fb0b8009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:01.775 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.772+0000 7fb0b2ffd640 1 -- 172.21.15.33:0/4215317158 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb0b4056f50 con 0x7fb0c4101090 2024-04-25T02:18:01.996 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.993+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fb0c41081f0 con 0x7fb0c4101090 2024-04-25T02:18:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.993+0000 7fb0b2ffd640 1 -- 172.21.15.33:0/4215317158 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v38) v1 ==== 107+0+5214 (secure 0 0 0) 0x7fb0b405a610 con 0x7fb0c4101090 2024-04-25T02:18:01.997 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:01.997 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44841,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/882099891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":882099891},{"type":"v1","addr":"172.21.15.138:6827","nonce":882099891}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T02:16:05.613716+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":1,"incarnation":32,"state":"up:replay","state_seq":1,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:18:01.997 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 33 2024-04-25T02:18:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb0ac06e0b0 msgr2=0x7fb0ac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb0ac06e0b0 0x7fb0ac070560 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4065910 tx=0x7fb0b8009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101090 msgr2=0x7fb0c410dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101090 0x7fb0c410dee0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb0b400db00 tx=0x7fb0b4004e60 comp rx=0 tx=0).stop 2024-04-25T02:18:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 shutdown_connections 2024-04-25T02:18:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb0ac06e0b0 0x7fb0ac070560 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:01.999 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb0c4101a30 0x7fb0c410e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 --2- 172.21.15.33:0/4215317158 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb0c4101090 0x7fb0c410dee0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 >> 172.21.15.33:0/4215317158 conn(0x7fb0c40f6c20 msgr2=0x7fb0c40f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:02.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 shutdown_connections 2024-04-25T02:18:02.000 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:01.995+0000 7fb0c8914640 1 -- 172.21.15.33:0/4215317158 wait complete. 2024-04-25T02:18:02.008 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:01 smithi033 ceph-mon[111306]: pgmap v1162: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:02.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:01 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2454537718' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-25T02:18:02.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:01 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1619679020' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-25T02:18:02.009 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:01 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3633061137' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-25T02:18:02.010 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-25T02:18:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.105+0000 7f9b6459d640 1 -- 172.21.15.33:0/1394897223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 msgr2=0x7f9b5c102640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.105+0000 7f9b6459d640 1 --2- 172.21.15.33:0/1394897223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 0x7f9b5c102640 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9b580048d0 tx=0x7f9b58009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.105+0000 7f9b6459d640 1 -- 172.21.15.33:0/1394897223 shutdown_connections 2024-04-25T02:18:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.105+0000 7f9b6459d640 1 --2- 172.21.15.33:0/1394897223 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b5c064390 0x7f9b5c064800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.111 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.105+0000 7f9b6459d640 1 --2- 172.21.15.33:0/1394897223 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 0x7f9b5c102640 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.105+0000 7f9b6459d640 1 -- 172.21.15.33:0/1394897223 >> 172.21.15.33:0/1394897223 conn(0x7f9b5c071fd0 msgr2=0x7f9b5c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b6459d640 1 -- 172.21.15.33:0/1394897223 shutdown_connections 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b6459d640 1 -- 172.21.15.33:0/1394897223 wait complete. 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b6459d640 1 Processor -- start 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b6459d640 1 -- start start 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b6459d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b5c064390 0x7f9b5c112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b6459d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 0x7f9b5c1131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b6459d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c1137a0 con 0x7f9b5c102270 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b6459d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c10cdf0 con 0x7f9b5c064390 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b61b38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 0x7f9b5c1131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b61b38640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 0x7f9b5c1131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51698/0 (socket says 172.21.15.33:51698) 2024-04-25T02:18:02.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b61b38640 1 -- 172.21.15.33:0/2748332996 learned_addr learned my addr 172.21.15.33:0/2748332996 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.106+0000 7f9b62339640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b5c064390 0x7f9b5c112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.107+0000 7f9b61b38640 1 -- 172.21.15.33:0/2748332996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b5c064390 msgr2=0x7f9b5c112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.107+0000 7f9b61b38640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b5c064390 0x7f9b5c112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.107+0000 7f9b61b38640 1 -- 172.21.15.33:0/2748332996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b58005040 con 0x7f9b5c102270 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.107+0000 7f9b61b38640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 0x7f9b5c1131a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c002a50 tx=0x7f9b4c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.107+0000 7f9b52ffd640 1 -- 172.21.15.33:0/2748332996 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b4c007c30 con 0x7f9b5c102270 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.107+0000 7f9b52ffd640 1 -- 172.21.15.33:0/2748332996 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b4c00f040 con 0x7f9b5c102270 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.107+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c10d0d0 con 0x7f9b5c102270 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.107+0000 7f9b52ffd640 1 -- 172.21.15.33:0/2748332996 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b4c00dbe0 con 0x7f9b5c102270 2024-04-25T02:18:02.113 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.107+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c10d5f0 con 0x7f9b5c102270 2024-04-25T02:18:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.108+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b5c0fd970 con 0x7f9b5c102270 2024-04-25T02:18:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.109+0000 7f9b52ffd640 1 -- 172.21.15.33:0/2748332996 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9b4c00dd40 con 0x7f9b5c102270 2024-04-25T02:18:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.109+0000 7f9b52ffd640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9b4806e0b0 0x7f9b48070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.109+0000 7f9b52ffd640 1 -- 172.21.15.33:0/2748332996 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9b4c08da20 con 0x7f9b5c102270 2024-04-25T02:18:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.109+0000 7f9b62339640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9b4806e0b0 0x7f9b48070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:02.114 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.110+0000 7f9b62339640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9b4806e0b0 0x7f9b48070560 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9b58003080 tx=0x7f9b58009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:02.117 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.114+0000 7f9b52ffd640 1 -- 172.21.15.33:0/2748332996 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9b4c056da0 con 0x7f9b5c102270 2024-04-25T02:18:02.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:01 smithi138 ceph-mon[81304]: pgmap v1162: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:02.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:01 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2454537718' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-25T02:18:02.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:01 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1619679020' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-25T02:18:02.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:01 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3633061137' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-25T02:18:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.338+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f9b5c063f00 con 0x7f9b5c102270 2024-04-25T02:18:02.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.339+0000 7f9b52ffd640 1 -- 172.21.15.33:0/2748332996 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v38) v1 ==== 107+0+5220 (secure 0 0 0) 0x7f9b4c05a460 con 0x7f9b5c102270 2024-04-25T02:18:02.343 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:02.343 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44841,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/882099891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":882099891},{"type":"v1","addr":"172.21.15.138:6827","nonce":882099891}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T02:16:12.677191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":32,"state":"up:resolve","state_seq":465,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":1,"incarnation":32,"state":"up:resolve","state_seq":460,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:18:02.343 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 34 2024-04-25T02:18:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9b4806e0b0 msgr2=0x7f9b48070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9b4806e0b0 0x7f9b48070560 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f9b58003080 tx=0x7f9b58009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 msgr2=0x7f9b5c1131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 0x7f9b5c1131a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c002a50 tx=0x7f9b4c004600 comp rx=0 tx=0).stop 2024-04-25T02:18:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 shutdown_connections 2024-04-25T02:18:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9b4806e0b0 0x7f9b48070560 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.345 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9b5c102270 0x7f9b5c1131a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 --2- 172.21.15.33:0/2748332996 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9b5c064390 0x7f9b5c112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 >> 172.21.15.33:0/2748332996 conn(0x7f9b5c071fd0 msgr2=0x7f9b5c0fa2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:02.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 shutdown_connections 2024-04-25T02:18:02.346 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.341+0000 7f9b6459d640 1 -- 172.21.15.33:0/2748332996 wait complete. 2024-04-25T02:18:02.355 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-25T02:18:02.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.451+0000 7f134086e640 1 -- 172.21.15.33:0/2938542076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 msgr2=0x7f133c0fa8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.451+0000 7f134086e640 1 --2- 172.21.15.33:0/2938542076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 0x7f133c0fa8c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f13300048d0 tx=0x7f1330009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.451+0000 7f134086e640 1 -- 172.21.15.33:0/2938542076 shutdown_connections 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.451+0000 7f134086e640 1 --2- 172.21.15.33:0/2938542076 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 0x7f133c0fa8c0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.451+0000 7f134086e640 1 --2- 172.21.15.33:0/2938542076 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c102230 0x7f133c102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.451+0000 7f134086e640 1 -- 172.21.15.33:0/2938542076 >> 172.21.15.33:0/2938542076 conn(0x7f133c0717e0 msgr2=0x7f133c071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.451+0000 7f134086e640 1 -- 172.21.15.33:0/2938542076 shutdown_connections 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.451+0000 7f134086e640 1 -- 172.21.15.33:0/2938542076 wait complete. 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f134086e640 1 Processor -- start 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f134086e640 1 -- start start 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f134086e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c102230 0x7f133c112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f134086e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 0x7f133c113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f134086e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c113870 con 0x7f133c102bd0 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f134086e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c10cec0 con 0x7f133c102230 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f1339d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 0x7f133c113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:02.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f1339d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 0x7f133c113270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51724/0 (socket says 172.21.15.33:51724) 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f1339d9b640 1 -- 172.21.15.33:0/1735130879 learned_addr learned my addr 172.21.15.33:0/1735130879 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f133a59c640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c102230 0x7f133c112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f1339d9b640 1 -- 172.21.15.33:0/1735130879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c102230 msgr2=0x7f133c112d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f1339d9b640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c102230 0x7f133c112d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f1339d9b640 1 -- 172.21.15.33:0/1735130879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1330005040 con 0x7f133c102bd0 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.452+0000 7f133a59c640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c102230 0x7f133c112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.453+0000 7f1339d9b640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 0x7f133c113270 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f133002bda0 tx=0x7f1330009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.453+0000 7f132affd640 1 -- 172.21.15.33:0/1735130879 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13300094c0 con 0x7f133c102bd0 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.453+0000 7f132affd640 1 -- 172.21.15.33:0/1735130879 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1330032e50 con 0x7f133c102bd0 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.453+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c10d140 con 0x7f133c102bd0 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.453+0000 7f132affd640 1 -- 172.21.15.33:0/1735130879 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f133003b400 con 0x7f133c102bd0 2024-04-25T02:18:02.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.453+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c10d660 con 0x7f133c102bd0 2024-04-25T02:18:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.454+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c065530 con 0x7f133c102bd0 2024-04-25T02:18:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.454+0000 7f132affd640 1 -- 172.21.15.33:0/1735130879 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f133002f070 con 0x7f133c102bd0 2024-04-25T02:18:02.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.455+0000 7f132affd640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132406e0b0 0x7f1324070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:02.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.455+0000 7f133a59c640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132406e0b0 0x7f1324070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:02.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.455+0000 7f132affd640 1 -- 172.21.15.33:0/1735130879 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f13300afef0 con 0x7f133c102bd0 2024-04-25T02:18:02.460 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.455+0000 7f133a59c640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132406e0b0 0x7f1324070560 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f132c009af0 tx=0x7f132c00b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:02.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.460+0000 7f132affd640 1 -- 172.21.15.33:0/1735130879 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1330079270 con 0x7f133c102bd0 2024-04-25T02:18:02.686 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.683+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f133c1093c0 con 0x7f133c102bd0 2024-04-25T02:18:02.687 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.683+0000 7f132affd640 1 -- 172.21.15.33:0/1735130879 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v38) v1 ==== 107+0+5224 (secure 0 0 0) 0x7f133002d040 con 0x7f133c102bd0 2024-04-25T02:18:02.687 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:02.687 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44841,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/882099891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":882099891},{"type":"v1","addr":"172.21.15.138:6827","nonce":882099891}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T02:16:13.546812+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":32,"state":"up:reconnect","state_seq":466,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":1,"incarnation":32,"state":"up:reconnect","state_seq":461,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:18:02.688 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 35 2024-04-25T02:18:02.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132406e0b0 msgr2=0x7f1324070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132406e0b0 0x7f1324070560 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f132c009af0 tx=0x7f132c00b040 comp rx=0 tx=0).stop 2024-04-25T02:18:02.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 msgr2=0x7f133c113270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 0x7f133c113270 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f133002bda0 tx=0x7f1330009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 shutdown_connections 2024-04-25T02:18:02.689 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f132406e0b0 0x7f1324070560 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f133c102bd0 0x7f133c113270 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 --2- 172.21.15.33:0/1735130879 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f133c102230 0x7f133c112d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 >> 172.21.15.33:0/1735130879 conn(0x7f133c0717e0 msgr2=0x7f133c0fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 shutdown_connections 2024-04-25T02:18:02.690 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.685+0000 7f134086e640 1 -- 172.21.15.33:0/1735130879 wait complete. 2024-04-25T02:18:02.700 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-04-25T02:18:02.800 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.795+0000 7ff8f9425640 1 -- 172.21.15.33:0/181660503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 msgr2=0x7ff8f40fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.795+0000 7ff8f9425640 1 --2- 172.21.15.33:0/181660503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 0x7ff8f40fc210 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0048d0 tx=0x7ff8dc009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.795+0000 7ff8f9425640 1 -- 172.21.15.33:0/181660503 shutdown_connections 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.795+0000 7ff8f9425640 1 --2- 172.21.15.33:0/181660503 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 0x7ff8f40fc210 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.795+0000 7ff8f9425640 1 --2- 172.21.15.33:0/181660503 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f41021e0 0x7ff8f41025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.795+0000 7ff8f9425640 1 -- 172.21.15.33:0/181660503 >> 172.21.15.33:0/181660503 conn(0x7ff8f40f7db0 msgr2=0x7ff8f40fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.795+0000 7ff8f9425640 1 -- 172.21.15.33:0/181660503 shutdown_connections 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.795+0000 7ff8f9425640 1 -- 172.21.15.33:0/181660503 wait complete. 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f9425640 1 Processor -- start 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f9425640 1 -- start start 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f9425640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f41021e0 0x7ff8f406f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f9425640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 0x7ff8f406fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:02.801 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f9425640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f4071650 con 0x7ff8f4102b80 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f9425640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f40701f0 con 0x7ff8f41021e0 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 0x7ff8f406fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f27fc640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 0x7ff8f406fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51730/0 (socket says 172.21.15.33:51730) 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f27fc640 1 -- 172.21.15.33:0/212555580 learned_addr learned my addr 172.21.15.33:0/212555580 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f2ffd640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f41021e0 0x7ff8f406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f27fc640 1 -- 172.21.15.33:0/212555580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f41021e0 msgr2=0x7ff8f406f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f27fc640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f41021e0 0x7ff8f406f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.796+0000 7ff8f27fc640 1 -- 172.21.15.33:0/212555580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc005040 con 0x7ff8f4102b80 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.797+0000 7ff8f27fc640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 0x7ff8f406fcb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc004930 tx=0x7ff8dc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.797+0000 7ff8eb7fe640 1 -- 172.21.15.33:0/212555580 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8dc0094c0 con 0x7ff8f4102b80 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.797+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f4070470 con 0x7ff8f4102b80 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.797+0000 7ff8eb7fe640 1 -- 172.21.15.33:0/212555580 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8dc032e50 con 0x7ff8f4102b80 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.797+0000 7ff8eb7fe640 1 -- 172.21.15.33:0/212555580 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8dc0433b0 con 0x7ff8f4102b80 2024-04-25T02:18:02.802 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.797+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f4119330 con 0x7ff8f4102b80 2024-04-25T02:18:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.798+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f40fd950 con 0x7ff8f4102b80 2024-04-25T02:18:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.798+0000 7ff8eb7fe640 1 -- 172.21.15.33:0/212555580 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff8dc02f070 con 0x7ff8f4102b80 2024-04-25T02:18:02.803 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.799+0000 7ff8eb7fe640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8e006e0b0 0x7ff8e0070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:02.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.799+0000 7ff8eb7fe640 1 -- 172.21.15.33:0/212555580 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7ff8dc0afb30 con 0x7ff8f4102b80 2024-04-25T02:18:02.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.799+0000 7ff8f2ffd640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8e006e0b0 0x7ff8e0070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:02.804 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.799+0000 7ff8f2ffd640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8e006e0b0 0x7ff8e0070560 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec004c40 tx=0x7ff8ec009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:02.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:02.804+0000 7ff8eb7fe640 1 -- 172.21.15.33:0/212555580 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff8dc078eb0 con 0x7ff8f4102b80 2024-04-25T02:18:03.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:02 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4215317158' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-25T02:18:03.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:02 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2748332996' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-25T02:18:03.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:02 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1735130879' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-25T02:18:03.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.030+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7ff8f4070c50 con 0x7ff8f4102b80 2024-04-25T02:18:03.034 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.031+0000 7ff8eb7fe640 1 -- 172.21.15.33:0/212555580 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v38) v1 ==== 107+0+5218 (secure 0 0 0) 0x7ff8dc07c570 con 0x7ff8f4102b80 2024-04-25T02:18:03.035 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:03.035 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44841,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/882099891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":882099891},{"type":"v1","addr":"172.21.15.138:6827","nonce":882099891}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T02:16:14.507705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":32,"state":"up:rejoin","state_seq":467,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":1,"incarnation":32,"state":"up:rejoin","state_seq":462,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T02:18:03.035 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 36 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8e006e0b0 msgr2=0x7ff8e0070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8e006e0b0 0x7ff8e0070560 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec004c40 tx=0x7ff8ec009040 comp rx=0 tx=0).stop 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 msgr2=0x7ff8f406fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 0x7ff8f406fcb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc004930 tx=0x7ff8dc009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 shutdown_connections 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7ff8e006e0b0 0x7ff8e0070560 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7ff8f4102b80 0x7ff8f406fcb0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 --2- 172.21.15.33:0/212555580 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7ff8f41021e0 0x7ff8f406f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 >> 172.21.15.33:0/212555580 conn(0x7ff8f40f7db0 msgr2=0x7ff8f40f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:03.037 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 shutdown_connections 2024-04-25T02:18:03.038 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.033+0000 7ff8f9425640 1 -- 172.21.15.33:0/212555580 wait complete. 2024-04-25T02:18:03.047 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-04-25T02:18:03.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 -- 172.21.15.33:0/1781015221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 msgr2=0x7fa6181061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 --2- 172.21.15.33:0/1781015221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 0x7fa6181061f0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa600004900 tx=0x7fa600009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 -- 172.21.15.33:0/1781015221 shutdown_connections 2024-04-25T02:18:03.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 --2- 172.21.15.33:0/1781015221 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 0x7fa6181061f0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 --2- 172.21.15.33:0/1781015221 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6180fd180 0x7fa6180fd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 -- 172.21.15.33:0/1781015221 >> 172.21.15.33:0/1781015221 conn(0x7fa6180f6be0 msgr2=0x7fa6180f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:03.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 -- 172.21.15.33:0/1781015221 shutdown_connections 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 -- 172.21.15.33:0/1781015221 wait complete. 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 Processor -- start 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.143+0000 7fa61cf9d640 1 -- start start 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa61cf9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6180fd180 0x7fa618112da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa61cf9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 0x7fa6181132e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa61cf9d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa618113970 con 0x7fa6180fdb20 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa61cf9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61810cf30 con 0x7fa6180fd180 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa615d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 0x7fa6181132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa615d9b640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 0x7fa6181132e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51746/0 (socket says 172.21.15.33:51746) 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa615d9b640 1 -- 172.21.15.33:0/2308747055 learned_addr learned my addr 172.21.15.33:0/2308747055 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa61659c640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6180fd180 0x7fa618112da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa615d9b640 1 -- 172.21.15.33:0/2308747055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6180fd180 msgr2=0x7fa618112da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa615d9b640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6180fd180 0x7fa618112da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa615d9b640 1 -- 172.21.15.33:0/2308747055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa600005040 con 0x7fa6180fdb20 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa615d9b640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 0x7fa6181132e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa600000c00 tx=0x7fa600009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:03.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa60effd640 1 -- 172.21.15.33:0/2308747055 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6000094c0 con 0x7fa6180fdb20 2024-04-25T02:18:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa60effd640 1 -- 172.21.15.33:0/2308747055 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa60003d920 con 0x7fa6180fdb20 2024-04-25T02:18:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61810d1b0 con 0x7fa6180fdb20 2024-04-25T02:18:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa60effd640 1 -- 172.21.15.33:0/2308747055 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa60003b930 con 0x7fa6180fdb20 2024-04-25T02:18:03.150 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.144+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61810d6d0 con 0x7fa6180fdb20 2024-04-25T02:18:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.145+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa618065fe0 con 0x7fa6180fdb20 2024-04-25T02:18:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.146+0000 7fa60effd640 1 -- 172.21.15.33:0/2308747055 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa60003da90 con 0x7fa6180fdb20 2024-04-25T02:18:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.146+0000 7fa60effd640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa60406e0b0 0x7fa604070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.146+0000 7fa60effd640 1 -- 172.21.15.33:0/2308747055 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa6000b04e0 con 0x7fa6180fdb20 2024-04-25T02:18:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.146+0000 7fa61659c640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa60406e0b0 0x7fa604070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:03.151 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.147+0000 7fa61659c640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa60406e0b0 0x7fa604070560 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa60800a900 tx=0x7fa608008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:03.154 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.151+0000 7fa60effd640 1 -- 172.21.15.33:0/2308747055 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa600079920 con 0x7fa6180fdb20 2024-04-25T02:18:03.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:02 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4215317158' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-25T02:18:03.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:02 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2748332996' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-25T02:18:03.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:02 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1735130879' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-25T02:18:03.375 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.372+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7fa618064350 con 0x7fa6180fdb20 2024-04-25T02:18:03.376 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.373+0000 7fa60effd640 1 -- 172.21.15.33:0/2308747055 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v38) v1 ==== 107+0+5233 (secure 0 0 0) 0x7fa60007cfe0 con 0x7fa6180fdb20 2024-04-25T02:18:03.376 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:03.376 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44327,"name":"cephfs.smithi138.jwzxlz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6825/165236762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6824","nonce":165236762},{"type":"v1","addr":"172.21.15.138:6825","nonce":165236762}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44841,"name":"cephfs.smithi138.rvxnby","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.138:6827/882099891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6826","nonce":882099891},{"type":"v1","addr":"172.21.15.138:6827","nonce":882099891}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T01:32:56.470326+0000","modified":"2024-04-25T02:16:16.506368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34414,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34414":{"gid":34414,"name":"cephfs.smithi033.qqhhcz","rank":0,"incarnation":32,"state":"up:active","state_seq":468,"addr":"172.21.15.33:6827/1943122938","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6826","nonce":1943122938},{"type":"v1","addr":"172.21.15.33:6827","nonce":1943122938}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi033.hobkqr","rank":1,"incarnation":32,"state":"up:active","state_seq":463,"addr":"172.21.15.33:6829/2150451637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6828","nonce":2150451637},{"type":"v1","addr":"172.21.15.33:6829","nonce":2150451637}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34414,"qdb_cluster":[34414,34422]},"id":1}]} 2024-04-25T02:18:03.377 INFO:teuthology.orchestra.run.smithi033.stderr:dumped fsmap epoch 37 2024-04-25T02:18:03.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa60406e0b0 msgr2=0x7fa604070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa60406e0b0 0x7fa604070560 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fa60800a900 tx=0x7fa608008040 comp rx=0 tx=0).stop 2024-04-25T02:18:03.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 msgr2=0x7fa6181132e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.378 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 0x7fa6181132e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa600000c00 tx=0x7fa600009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 shutdown_connections 2024-04-25T02:18:03.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa60406e0b0 0x7fa604070560 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa6180fdb20 0x7fa6181132e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 --2- 172.21.15.33:0/2308747055 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa6180fd180 0x7fa618112da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 >> 172.21.15.33:0/2308747055 conn(0x7fa6180f6be0 msgr2=0x7fa6180f8050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:03.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 shutdown_connections 2024-04-25T02:18:03.379 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.375+0000 7fa61cf9d640 1 -- 172.21.15.33:0/2308747055 wait complete. 2024-04-25T02:18:03.389 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T02:18:03.401 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T02:18:03.401 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T02:18:03.402 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T02:18:03.418 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T02:18:03.419 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T02:18:03.475 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T02:18:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/3625402631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fdb70 msgr2=0x7fa9ec1061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 --2- 172.21.15.33:0/3625402631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fdb70 0x7fa9ec1061b0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00048d0 tx=0x7fa9e0009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/3625402631 shutdown_connections 2024-04-25T02:18:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 --2- 172.21.15.33:0/3625402631 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fdb70 0x7fa9ec1061b0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 --2- 172.21.15.33:0/3625402631 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec0fd1d0 0x7fa9ec0fd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/3625402631 >> 172.21.15.33:0/3625402631 conn(0x7fa9ec0f6ca0 msgr2=0x7fa9ec0f90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/3625402631 shutdown_connections 2024-04-25T02:18:03.615 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/3625402631 wait complete. 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 Processor -- start 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.610+0000 7fa9f1d9d640 1 -- start start 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9f1d9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fd1d0 0x7fa9ec112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9f1d9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec0fdb70 0x7fa9ec113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9f1d9d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec113880 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9f1d9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9ec10ced0 con 0x7fa9ec0fdb70 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fd1d0 0x7fa9ec112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9eb7fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fd1d0 0x7fa9ec112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51768/0 (socket says 172.21.15.33:51768) 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9eb7fe640 1 -- 172.21.15.33:0/1753764405 learned_addr learned my addr 172.21.15.33:0/1753764405 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9eaffd640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec0fdb70 0x7fa9ec113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9eb7fe640 1 -- 172.21.15.33:0/1753764405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec0fdb70 msgr2=0x7fa9ec113280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9eb7fe640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec0fdb70 0x7fa9ec113280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9eb7fe640 1 -- 172.21.15.33:0/1753764405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e0005040 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9eaffd640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec0fdb70 0x7fa9ec113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:18:03.616 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9eb7fe640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fd1d0 0x7fa9ec112d40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc002a50 tx=0x7fa9dc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9d3fff640 1 -- 172.21.15.33:0/1753764405 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9dc007c30 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9ec10d1b0 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9d3fff640 1 -- 172.21.15.33:0/1753764405 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9dc00f040 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9d3fff640 1 -- 172.21.15.33:0/1753764405 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9dc00db50 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.617 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.611+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9ec10d6a0 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.612+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9ec065db0 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.613+0000 7fa9d3fff640 1 -- 172.21.15.33:0/1753764405 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa9dc01c400 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.613+0000 7fa9d3fff640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9d406e0b0 0x7fa9d4070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.613+0000 7fa9d3fff640 1 -- 172.21.15.33:0/1753764405 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fa9dc08d420 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.613+0000 7fa9eaffd640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9d406e0b0 0x7fa9d4070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:03.618 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.614+0000 7fa9eaffd640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9d406e0b0 0x7fa9d4070560 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0fe5f0 tx=0x7fa9e0009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:03.621 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.618+0000 7fa9d3fff640 1 -- 172.21.15.33:0/1753764405 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa9dc0567a0 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.805 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.802+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9ec10e3b0 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.807 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.803+0000 7fa9d3fff640 1 -- 172.21.15.33:0/1753764405 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16530 (secure 0 0 0) 0x7fa9dc059e60 con 0x7fa9ec0fd1d0 2024-04-25T02:18:03.807 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:03.807 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","created":"2024-04-25T01:26:05.674330+0000","modified":"2024-04-25T02:01:00.954060+0000","last_up_change":"2024-04-25T01:45:08.141024+0000","last_in_change":"2024-04-25T01:31:20.199327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:30:26.558656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-25T01:32:55.402156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-25T01:32:55.894097+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68af93d4-7aa8-485f-826e-62a6e362907b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6803","nonce":3103417098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6805","nonce":3103417098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6809","nonce":3103417098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6807","nonce":3103417098}]},"public_addr":"172.21.15.33:6803/3103417098","cluster_addr":"172.21.15.33:6805/3103417098","heartbeat_back_addr":"172.21.15.33:6809/3103417098","heartbeat_front_addr":"172.21.15.33:6807/3103417098","state":["exists","up"]},{"osd":1,"uuid":"1fdb7e9b-1138-43db-a11f-3e30e286c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6811","nonce":166114729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6813","nonce":166114729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6817","nonce":166114729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6815","nonce":166114729}]},"public_addr":"172.21.15.33:6811/166114729","cluster_addr":"172.21.15.33:6813/166114729","heartbeat_back_addr":"172.21.15.33:6817/166114729","heartbeat_front_addr":"172.21.15.33:6815/166114729","state":["exists","up"]},{"osd":2,"uuid":"b1239da3-566e-4df0-b5ea-df4c14959837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6819","nonce":4078679342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6821","nonce":4078679342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6825","nonce":4078679342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6823","nonce":4078679342}]},"public_addr":"172.21.15.33:6819/4078679342","cluster_addr":"172.21.15.33:6821/4078679342","heartbeat_back_addr":"172.21.15.33:6825/4078679342","heartbeat_front_addr":"172.21.15.33:6823/4078679342","state":["exists","up"]},{"osd":3,"uuid":"abc58749-77ca-4476-9601-cdf89d3c1e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6801","nonce":1157003839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6803","nonce":1157003839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6807","nonce":1157003839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6805","nonce":1157003839}]},"public_addr":"172.21.15.138:6801/1157003839","cluster_addr":"172.21.15.138:6803/1157003839","heartbeat_back_addr":"172.21.15.138:6807/1157003839","heartbeat_front_addr":"172.21.15.138:6805/1157003839","state":["exists","up"]},{"osd":4,"uuid":"83e421bd-52ba-47ad-8755-e1354f502ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6809","nonce":4024012885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6811","nonce":4024012885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6815","nonce":4024012885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6813","nonce":4024012885}]},"public_addr":"172.21.15.138:6809/4024012885","cluster_addr":"172.21.15.138:6811/4024012885","heartbeat_back_addr":"172.21.15.138:6815/4024012885","heartbeat_front_addr":"172.21.15.138:6813/4024012885","state":["exists","up"]},{"osd":5,"uuid":"266e5984-f087-408e-a84c-04f81f33af43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6817","nonce":2521547597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6819","nonce":2521547597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6823","nonce":2521547597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6821","nonce":2521547597}]},"public_addr":"172.21.15.138:6817/2521547597","cluster_addr":"172.21.15.138:6819/2521547597","heartbeat_back_addr":"172.21.15.138:6823/2521547597","heartbeat_front_addr":"172.21.15.138:6821/2521547597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:27.636323+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:55.618887+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:23.307858+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:47.331937+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:12.159932+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:37.317499+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6825/3887358693":"2024-04-26T01:45:10.323297+0000","172.21.15.33:6800/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:6800/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6800/1387801224":"2024-04-26T01:35:34.311733+0000","172.21.15.33:0/1869074265":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/2062772238":"2024-04-26T01:35:34.311733+0000","172.21.15.33:6801/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.138:0/4174594849":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/3334291936":"2024-04-26T01:35:34.311733+0000","172.21.15.138:6829/1792929713":"2024-04-26T01:37:35.038935+0000","172.21.15.33:6800/1731533099":"2024-04-26T01:40:40.186857+0000","172.21.15.33:6801/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/1731533099":"2024-04-26T01:40:40.186857+0000","172.21.15.33:6800/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/4160574345":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6826/3303942324":"2024-04-26T01:45:10.323282+0000","172.21.15.33:0/2566719862":"2024-04-26T01:28:16.045570+0000","172.21.15.138:0/330154225":"2024-04-26T01:37:35.038935+0000","172.21.15.33:6801/1387801224":"2024-04-26T01:35:34.311733+0000","172.21.15.138:0/219235313":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/454449040":"2024-04-26T01:35:34.311733+0000","172.21.15.33:0/1401659678":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2117172792":"2024-04-26T01:26:40.396561+0000","172.21.15.138:6828/1792929713":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/2112653252":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6801/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1131828704":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/751781866":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/686286320":"2024-04-26T01:26:40.396561+0000","172.21.15.138:0/1137962382":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/746151230":"2024-04-26T01:40:40.186857+0000","172.21.15.33:0/3951183717":"2024-04-26T01:40:40.186857+0000","172.21.15.33:0/1718672295":"2024-04-26T01:40:40.186857+0000","172.21.15.33:0/2952222145":"2024-04-26T01:35:34.311733+0000","172.21.15.33:0/3161221115":"2024-04-26T01:40:40.186857+0000","172.21.15.33:6827/3303942324":"2024-04-26T01:45:10.323282+0000","172.21.15.138:6824/3887358693":"2024-04-26T01:45:10.323297+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T02:18:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9d406e0b0 msgr2=0x7fa9d4070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9d406e0b0 0x7fa9d4070560 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec0fe5f0 tx=0x7fa9e0009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fd1d0 msgr2=0x7fa9ec112d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fd1d0 0x7fa9ec112d40 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc002a50 tx=0x7fa9dc004600 comp rx=0 tx=0).stop 2024-04-25T02:18:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 shutdown_connections 2024-04-25T02:18:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fa9d406e0b0 0x7fa9d4070560 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.809 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fa9ec0fdb70 0x7fa9ec113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 --2- 172.21.15.33:0/1753764405 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fa9ec0fd1d0 0x7fa9ec112d40 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 >> 172.21.15.33:0/1753764405 conn(0x7fa9ec0f6ca0 msgr2=0x7fa9ec0f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.805+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 shutdown_connections 2024-04-25T02:18:03.810 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.806+0000 7fa9f1d9d640 1 -- 172.21.15.33:0/1753764405 wait complete. 2024-04-25T02:18:03.820 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:30:26.558656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-25T01:32:55.402156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-25T01:32:55.894097+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T02:18:03.820 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.917+0000 7fd349d5d640 1 -- 172.21.15.33:0/2699504694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344102210 msgr2=0x7fd3441025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.917+0000 7fd349d5d640 1 --2- 172.21.15.33:0/2699504694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344102210 0x7fd3441025e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd3340048d0 tx=0x7fd334009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 -- 172.21.15.33:0/2699504694 shutdown_connections 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 --2- 172.21.15.33:0/2699504694 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344064380 0x7fd3440647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 --2- 172.21.15.33:0/2699504694 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344102210 0x7fd3441025e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 -- 172.21.15.33:0/2699504694 >> 172.21.15.33:0/2699504694 conn(0x7fd3440710c0 msgr2=0x7fd3440714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 -- 172.21.15.33:0/2699504694 shutdown_connections 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 -- 172.21.15.33:0/2699504694 wait complete. 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 Processor -- start 2024-04-25T02:18:03.923 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 -- start start 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344064380 0x7fd344112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344102210 0x7fd3441132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd344113930 con 0x7fd344064380 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd349d5d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34410cef0 con 0x7fd344102210 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.918+0000 7fd3437fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344064380 0x7fd344112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd3437fe640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344064380 0x7fd344112d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51786/0 (socket says 172.21.15.33:51786) 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd3437fe640 1 -- 172.21.15.33:0/1070306004 learned_addr learned my addr 172.21.15.33:0/1070306004 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd342ffd640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344102210 0x7fd3441132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd3437fe640 1 -- 172.21.15.33:0/1070306004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344102210 msgr2=0x7fd3441132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd3437fe640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344102210 0x7fd3441132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd3437fe640 1 -- 172.21.15.33:0/1070306004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd334005040 con 0x7fd344064380 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd3437fe640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344064380 0x7fd344112d60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd334004930 tx=0x7fd334009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd32bfff640 1 -- 172.21.15.33:0/1070306004 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3340094c0 con 0x7fd344064380 2024-04-25T02:18:03.924 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd32bfff640 1 -- 172.21.15.33:0/1070306004 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd334032e50 con 0x7fd344064380 2024-04-25T02:18:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34410d170 con 0x7fd344064380 2024-04-25T02:18:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd32bfff640 1 -- 172.21.15.33:0/1070306004 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3340433b0 con 0x7fd344064380 2024-04-25T02:18:03.925 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.919+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34410d630 con 0x7fd344064380 2024-04-25T02:18:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.920+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3440fd910 con 0x7fd344064380 2024-04-25T02:18:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.921+0000 7fd32bfff640 1 -- 172.21.15.33:0/1070306004 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd33402f070 con 0x7fd344064380 2024-04-25T02:18:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.921+0000 7fd32bfff640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd32c06e0b0 0x7fd32c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.921+0000 7fd32bfff640 1 -- 172.21.15.33:0/1070306004 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fd3340afb00 con 0x7fd344064380 2024-04-25T02:18:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.921+0000 7fd342ffd640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd32c06e0b0 0x7fd32c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:03.926 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.922+0000 7fd342ffd640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd32c06e0b0 0x7fd32c070560 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd338006fd0 tx=0x7fd338008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:03.929 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:03.926+0000 7fd32bfff640 1 -- 172.21.15.33:0/1070306004 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd334078e80 con 0x7fd344064380 2024-04-25T02:18:04.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.108+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd34410d850 con 0x7fd344064380 2024-04-25T02:18:04.112 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.108+0000 7fd32bfff640 1 -- 172.21.15.33:0/1070306004 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd33407c540 con 0x7fd344064380 2024-04-25T02:18:04.113 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd32c06e0b0 msgr2=0x7fd32c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd32c06e0b0 0x7fd32c070560 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd338006fd0 tx=0x7fd338008040 comp rx=0 tx=0).stop 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344064380 msgr2=0x7fd344112d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344064380 0x7fd344112d60 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd334004930 tx=0x7fd334009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 shutdown_connections 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd32c06e0b0 0x7fd32c070560 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fd344102210 0x7fd3441132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 --2- 172.21.15.33:0/1070306004 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fd344064380 0x7fd344112d60 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 >> 172.21.15.33:0/1070306004 conn(0x7fd3440710c0 msgr2=0x7fd3440f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 shutdown_connections 2024-04-25T02:18:04.115 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.110+0000 7fd349d5d640 1 -- 172.21.15.33:0/1070306004 wait complete. 2024-04-25T02:18:04.124 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:03 smithi033 ceph-mon[111306]: pgmap v1163: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:04.124 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:03 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/212555580' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-25T02:18:04.124 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:03 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2308747055' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-25T02:18:04.124 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:03 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1753764405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T02:18:04.125 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-25T02:18:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.223+0000 7fda1442e640 1 -- 172.21.15.33:0/3982068786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fdb20 msgr2=0x7fda0c1061f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.223+0000 7fda1442e640 1 --2- 172.21.15.33:0/3982068786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fdb20 0x7fda0c1061f0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc004900 tx=0x7fd9fc009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.223+0000 7fda1442e640 1 -- 172.21.15.33:0/3982068786 shutdown_connections 2024-04-25T02:18:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.223+0000 7fda1442e640 1 --2- 172.21.15.33:0/3982068786 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fdb20 0x7fda0c1061f0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.223+0000 7fda1442e640 1 --2- 172.21.15.33:0/3982068786 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda0c0fd180 0x7fda0c0fd550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.228 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.223+0000 7fda1442e640 1 -- 172.21.15.33:0/3982068786 >> 172.21.15.33:0/3982068786 conn(0x7fda0c0f6c20 msgr2=0x7fda0c0f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:04.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.223+0000 7fda1442e640 1 -- 172.21.15.33:0/3982068786 shutdown_connections 2024-04-25T02:18:04.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.223+0000 7fda1442e640 1 -- 172.21.15.33:0/3982068786 wait complete. 2024-04-25T02:18:04.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.224+0000 7fda1442e640 1 Processor -- start 2024-04-25T02:18:04.229 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.224+0000 7fda1442e640 1 -- start start 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.224+0000 7fda1442e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fd180 0x7fda0c112e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.224+0000 7fda1442e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda0c0fdb20 0x7fda0c113390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.224+0000 7fda1442e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c113990 con 0x7fda0c0fd180 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.224+0000 7fda1442e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0c10cfe0 con 0x7fda0c0fdb20 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.224+0000 7fda121ca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fd180 0x7fda0c112e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.224+0000 7fda121ca640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fd180 0x7fda0c112e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51808/0 (socket says 172.21.15.33:51808) 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.224+0000 7fda121ca640 1 -- 172.21.15.33:0/3729440951 learned_addr learned my addr 172.21.15.33:0/3729440951 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda119c9640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda0c0fdb20 0x7fda0c113390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda121ca640 1 -- 172.21.15.33:0/3729440951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda0c0fdb20 msgr2=0x7fda0c113390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda121ca640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda0c0fdb20 0x7fda0c113390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.231 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda121ca640 1 -- 172.21.15.33:0/3729440951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc005040 con 0x7fda0c0fd180 2024-04-25T02:18:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda119c9640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda0c0fdb20 0x7fda0c113390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T02:18:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda121ca640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fd180 0x7fda0c112e50 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fda08002a50 tx=0x7fda08004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda02ffd640 1 -- 172.21.15.33:0/3729440951 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda08007c30 con 0x7fda0c0fd180 2024-04-25T02:18:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda02ffd640 1 -- 172.21.15.33:0/3729440951 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda0800f040 con 0x7fda0c0fd180 2024-04-25T02:18:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0c10d2c0 con 0x7fda0c0fd180 2024-04-25T02:18:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda02ffd640 1 -- 172.21.15.33:0/3729440951 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda0800db50 con 0x7fda0c0fd180 2024-04-25T02:18:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.225+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0c10d7e0 con 0x7fda0c0fd180 2024-04-25T02:18:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.226+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0c065d70 con 0x7fda0c0fd180 2024-04-25T02:18:04.232 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.226+0000 7fda02ffd640 1 -- 172.21.15.33:0/3729440951 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fda0801c400 con 0x7fda0c0fd180 2024-04-25T02:18:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.227+0000 7fda02ffd640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd9f806e0b0 0x7fd9f8070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.227+0000 7fda02ffd640 1 -- 172.21.15.33:0/3729440951 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fda0808d420 con 0x7fda0c0fd180 2024-04-25T02:18:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.227+0000 7fda119c9640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd9f806e0b0 0x7fd9f8070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:04.233 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.227+0000 7fda119c9640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd9f806e0b0 0x7fd9f8070560 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc000c00 tx=0x7fd9fc009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:04.236 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.232+0000 7fda02ffd640 1 -- 172.21.15.33:0/3729440951 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fda080567a0 con 0x7fda0c0fd180 2024-04-25T02:18:04.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:03 smithi138 ceph-mon[81304]: pgmap v1163: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:04.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:03 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/212555580' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-25T02:18:04.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:03 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2308747055' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-25T02:18:04.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:03 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1753764405' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T02:18:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.416+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fda0c10e4c0 con 0x7fda0c0fd180 2024-04-25T02:18:04.420 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.416+0000 7fda02ffd640 1 -- 172.21.15.33:0/3729440951 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fda08059e60 con 0x7fda0c0fd180 2024-04-25T02:18:04.421 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd9f806e0b0 msgr2=0x7fd9f8070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd9f806e0b0 0x7fd9f8070560 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc000c00 tx=0x7fd9fc009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fd180 msgr2=0x7fda0c112e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fd180 0x7fda0c112e50 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fda08002a50 tx=0x7fda08004600 comp rx=0 tx=0).stop 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 shutdown_connections 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fd9f806e0b0 0x7fd9f8070560 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fda0c0fdb20 0x7fda0c113390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 --2- 172.21.15.33:0/3729440951 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fda0c0fd180 0x7fda0c112e50 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 >> 172.21.15.33:0/3729440951 conn(0x7fda0c0f6c20 msgr2=0x7fda0c0f80d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 shutdown_connections 2024-04-25T02:18:04.423 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.418+0000 7fda1442e640 1 -- 172.21.15.33:0/3729440951 wait complete. 2024-04-25T02:18:04.433 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-25T02:18:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.529+0000 7f189ab9d640 1 -- 172.21.15.33:0/3493836901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894064380 msgr2=0x7f18940647d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.529+0000 7f189ab9d640 1 --2- 172.21.15.33:0/3493836901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894064380 0x7f18940647d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f18800048d0 tx=0x7f1880009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 -- 172.21.15.33:0/3493836901 shutdown_connections 2024-04-25T02:18:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 --2- 172.21.15.33:0/3493836901 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f1894064380 0x7f18940647d0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 --2- 172.21.15.33:0/3493836901 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f18941021e0 0x7f18941025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 -- 172.21.15.33:0/3493836901 >> 172.21.15.33:0/3493836901 conn(0x7f18940710c0 msgr2=0x7f18940714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 -- 172.21.15.33:0/3493836901 shutdown_connections 2024-04-25T02:18:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 -- 172.21.15.33:0/3493836901 wait complete. 2024-04-25T02:18:04.536 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 Processor -- start 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 -- start start 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1894064380 0x7f1894112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941021e0 0x7f18941131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18941137e0 con 0x7f18941021e0 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.530+0000 7f189ab9d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189410ce30 con 0x7f1894064380 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f1893fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941021e0 0x7f18941131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f1893fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941021e0 0x7f18941131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51830/0 (socket says 172.21.15.33:51830) 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f1893fff640 1 -- 172.21.15.33:0/1644523153 learned_addr learned my addr 172.21.15.33:0/1644523153 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f1898939640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1894064380 0x7f1894112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f1893fff640 1 -- 172.21.15.33:0/1644523153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1894064380 msgr2=0x7f1894112ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f1893fff640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1894064380 0x7f1894112ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f1893fff640 1 -- 172.21.15.33:0/1644523153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1880005040 con 0x7f18941021e0 2024-04-25T02:18:04.537 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f1893fff640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941021e0 0x7f18941131e0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1880004930 tx=0x7f1880009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f18917fa640 1 -- 172.21.15.33:0/1644523153 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18800094c0 con 0x7f18941021e0 2024-04-25T02:18:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f18917fa640 1 -- 172.21.15.33:0/1644523153 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1880032e50 con 0x7f18941021e0 2024-04-25T02:18:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189410d0b0 con 0x7f18941021e0 2024-04-25T02:18:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f18917fa640 1 -- 172.21.15.33:0/1644523153 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18800433b0 con 0x7f18941021e0 2024-04-25T02:18:04.538 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.531+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189410d570 con 0x7f18941021e0 2024-04-25T02:18:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.532+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18940fd8e0 con 0x7f18941021e0 2024-04-25T02:18:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.533+0000 7f18917fa640 1 -- 172.21.15.33:0/1644523153 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f188002f070 con 0x7f18941021e0 2024-04-25T02:18:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.533+0000 7f18917fa640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f188406e0b0 0x7f1884070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.533+0000 7f18917fa640 1 -- 172.21.15.33:0/1644523153 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f18800afaf0 con 0x7f18941021e0 2024-04-25T02:18:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.533+0000 7f1898939640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f188406e0b0 0x7f1884070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:04.539 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.534+0000 7f1898939640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f188406e0b0 0x7f1884070560 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1888007fd0 tx=0x7f1888009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:04.542 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.538+0000 7f18917fa640 1 -- 172.21.15.33:0/1644523153 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1880078ec0 con 0x7f18941021e0 2024-04-25T02:18:04.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.722+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f189404bcc0 con 0x7f18941021e0 2024-04-25T02:18:04.727 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.722+0000 7f18917fa640 1 -- 172.21.15.33:0/1644523153 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f188007c580 con 0x7f18941021e0 2024-04-25T02:18:04.727 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T02:18:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.724+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f188406e0b0 msgr2=0x7f1884070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.724+0000 7f189ab9d640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f188406e0b0 0x7f1884070560 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1888007fd0 tx=0x7f1888009040 comp rx=0 tx=0).stop 2024-04-25T02:18:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.724+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941021e0 msgr2=0x7f18941131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.724+0000 7f189ab9d640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941021e0 0x7f18941131e0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f1880004930 tx=0x7f1880009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.724+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 shutdown_connections 2024-04-25T02:18:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.724+0000 7f189ab9d640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f188406e0b0 0x7f1884070560 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.724+0000 7f189ab9d640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f18941021e0 0x7f18941131e0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.724+0000 7f189ab9d640 1 --2- 172.21.15.33:0/1644523153 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f1894064380 0x7f1894112ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.729 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.724+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 >> 172.21.15.33:0/1644523153 conn(0x7f18940710c0 msgr2=0x7f18940f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.725+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 shutdown_connections 2024-04-25T02:18:04.730 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.725+0000 7f189ab9d640 1 -- 172.21.15.33:0/1644523153 wait complete. 2024-04-25T02:18:04.740 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 -- 172.21.15.33:0/3911764965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 msgr2=0x7f72440fa8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 --2- 172.21.15.33:0/3911764965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 0x7f72440fa8b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f72300048d0 tx=0x7f7230009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 -- 172.21.15.33:0/3911764965 shutdown_connections 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 --2- 172.21.15.33:0/3911764965 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 0x7f72440fa8b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 --2- 172.21.15.33:0/3911764965 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72441021c0 0x7f7244102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 -- 172.21.15.33:0/3911764965 >> 172.21.15.33:0/3911764965 conn(0x7f7244070890 msgr2=0x7f7244070c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 -- 172.21.15.33:0/3911764965 shutdown_connections 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 -- 172.21.15.33:0/3911764965 wait complete. 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 Processor -- start 2024-04-25T02:18:04.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.835+0000 7f724b06e640 1 -- start start 2024-04-25T02:18:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f724b06e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72441021c0 0x7f72440fba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f724b06e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 0x7f72440fbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f724b06e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440fd970 con 0x7f7244102b60 2024-04-25T02:18:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f724b06e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72440fc510 con 0x7f72441021c0 2024-04-25T02:18:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f7243fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 0x7f72440fbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f7243fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 0x7f72440fbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51838/0 (socket says 172.21.15.33:51838) 2024-04-25T02:18:04.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f7243fff640 1 -- 172.21.15.33:0/2617475986 learned_addr learned my addr 172.21.15.33:0/2617475986 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f7248e0a640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72441021c0 0x7f72440fba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f7243fff640 1 -- 172.21.15.33:0/2617475986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72441021c0 msgr2=0x7f72440fba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f7243fff640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72441021c0 0x7f72440fba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f7243fff640 1 -- 172.21.15.33:0/2617475986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7230005040 con 0x7f7244102b60 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.836+0000 7f7243fff640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 0x7f72440fbfd0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f723002bda0 tx=0x7f7230009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.837+0000 7f72417fa640 1 -- 172.21.15.33:0/2617475986 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72300094c0 con 0x7f7244102b60 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.837+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72440fc790 con 0x7f7244102b60 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.837+0000 7f72417fa640 1 -- 172.21.15.33:0/2617475986 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7230032e50 con 0x7f7244102b60 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.837+0000 7f72417fa640 1 -- 172.21.15.33:0/2617475986 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f723003b400 con 0x7f7244102b60 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.837+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7244119460 con 0x7f7244102b60 2024-04-25T02:18:04.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.838+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72440652a0 con 0x7f7244102b60 2024-04-25T02:18:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.838+0000 7f72417fa640 1 -- 172.21.15.33:0/2617475986 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f723002f070 con 0x7f7244102b60 2024-04-25T02:18:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.839+0000 7f72417fa640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f723406e0b0 0x7f7234070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.839+0000 7f72417fa640 1 -- 172.21.15.33:0/2617475986 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f72300aff10 con 0x7f7244102b60 2024-04-25T02:18:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.839+0000 7f7248e0a640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f723406e0b0 0x7f7234070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:04.844 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.839+0000 7f7248e0a640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f723406e0b0 0x7f7234070560 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7238004ed0 tx=0x7f723800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:04.847 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:04.843+0000 7f72417fa640 1 -- 172.21.15.33:0/2617475986 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f72300791f0 con 0x7f7244102b60 2024-04-25T02:18:05.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:04 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1070306004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T02:18:05.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:04 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3729440951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T02:18:05.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:04 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/1644523153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T02:18:05.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.045+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f724404bcc0 con 0x7f7244102b60 2024-04-25T02:18:05.049 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.045+0000 7f72417fa640 1 -- 172.21.15.33:0/2617475986 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 37 entries v84) v1 ==== 81+0+2194 (secure 0 0 0) 0x7f723002d040 con 0x7f7244102b60 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:6825/3887358693 2024-04-26T01:45:10.323297+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/395184027 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/4148578452 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1387801224 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1869074265 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2062772238 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/4148578452 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:0/4174594849 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3334291936 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:6829/1792929713 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1731533099 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3376891250 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:05.050 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1731533099 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3376891250 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4160574345 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6826/3303942324 2024-04-26T01:45:10.323282+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2566719862 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:0/330154225 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1387801224 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:0/219235313 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/454449040 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1401659678 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2117172792 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:6828/1792929713 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2112653252 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/395184027 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1131828704 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/751781866 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:05.051 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/686286320 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:05.052 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:0/1137962382 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:05.052 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/746151230 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:05.052 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3951183717 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:05.052 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1718672295 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:05.052 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2952222145 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:05.052 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3161221115 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:05.052 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6827/3303942324 2024-04-26T01:45:10.323282+0000 2024-04-25T02:18:05.052 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:6824/3887358693 2024-04-26T01:45:10.323297+0000 2024-04-25T02:18:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:listed 37 entries 2024-04-25T02:18:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f723406e0b0 msgr2=0x7f7234070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f723406e0b0 0x7f7234070560 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f7238004ed0 tx=0x7f723800b040 comp rx=0 tx=0).stop 2024-04-25T02:18:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 msgr2=0x7f72440fbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 0x7f72440fbfd0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f723002bda0 tx=0x7f7230009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 shutdown_connections 2024-04-25T02:18:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f723406e0b0 0x7f7234070560 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f7244102b60 0x7f72440fbfd0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.053 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 --2- 172.21.15.33:0/2617475986 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f72441021c0 0x7f72440fba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 >> 172.21.15.33:0/2617475986 conn(0x7f7244070890 msgr2=0x7f7244104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 shutdown_connections 2024-04-25T02:18:05.054 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.047+0000 7f724b06e640 1 -- 172.21.15.33:0/2617475986 wait complete. 2024-04-25T02:18:05.063 DEBUG:teuthology.orchestra.run.smithi033:> set -ex 2024-04-25T02:18:05.063 DEBUG:teuthology.orchestra.run.smithi033:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T02:18:05.079 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T02:18:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.212+0000 7fe66106e640 1 -- 172.21.15.33:0/372884197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fd250 msgr2=0x7fe65c0fd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.212+0000 7fe66106e640 1 --2- 172.21.15.33:0/372884197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fd250 0x7fe65c0fd620 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fe6540048d0 tx=0x7fe654009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.212+0000 7fe66106e640 1 -- 172.21.15.33:0/372884197 shutdown_connections 2024-04-25T02:18:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.212+0000 7fe66106e640 1 --2- 172.21.15.33:0/372884197 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c0fdbf0 0x7fe65c106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.218 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.212+0000 7fe66106e640 1 --2- 172.21.15.33:0/372884197 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fd250 0x7fe65c0fd620 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.212+0000 7fe66106e640 1 -- 172.21.15.33:0/372884197 >> 172.21.15.33:0/372884197 conn(0x7fe65c0f6cc0 msgr2=0x7fe65c0f90d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.212+0000 7fe66106e640 1 -- 172.21.15.33:0/372884197 shutdown_connections 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.212+0000 7fe66106e640 1 -- 172.21.15.33:0/372884197 wait complete. 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.212+0000 7fe66106e640 1 Processor -- start 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe66106e640 1 -- start start 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe66106e640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c0fd250 0x7fe65c1130e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe66106e640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fdbf0 0x7fe65c113620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe66106e640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c113c20 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe66106e640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c10d270 con 0x7fe65c0fd250 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe65a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fdbf0 0x7fe65c113620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe65a59c640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fdbf0 0x7fe65c113620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51854/0 (socket says 172.21.15.33:51854) 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe65a59c640 1 -- 172.21.15.33:0/605567431 learned_addr learned my addr 172.21.15.33:0/605567431 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe65ad9d640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c0fd250 0x7fe65c1130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe65a59c640 1 -- 172.21.15.33:0/605567431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c0fd250 msgr2=0x7fe65c1130e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.219 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe65a59c640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c0fd250 0x7fe65c1130e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe65a59c640 1 -- 172.21.15.33:0/605567431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe654005040 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.213+0000 7fe65a59c640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fdbf0 0x7fe65c113620 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe650002a50 tx=0x7fe650004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.214+0000 7fe6437fe640 1 -- 172.21.15.33:0/605567431 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe650007c30 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.214+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c10d550 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.214+0000 7fe6437fe640 1 -- 172.21.15.33:0/605567431 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe65000f040 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.214+0000 7fe6437fe640 1 -- 172.21.15.33:0/605567431 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe65000db50 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.220 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.214+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c10da70 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.215+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c065db0 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.215+0000 7fe6437fe640 1 -- 172.21.15.33:0/605567431 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe65001c400 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.216+0000 7fe6437fe640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe64406e0b0 0x7fe644070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.216+0000 7fe6437fe640 1 -- 172.21.15.33:0/605567431 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe65008d370 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.216+0000 7fe65ad9d640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe64406e0b0 0x7fe644070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:05.221 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.216+0000 7fe65ad9d640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe64406e0b0 0x7fe644070560 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe654003080 tx=0x7fe654009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:05.225 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.220+0000 7fe6437fe640 1 -- 172.21.15.33:0/605567431 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe6500567a0 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:04 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1070306004' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T02:18:05.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:04 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3729440951' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T02:18:05.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:04 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/1644523153' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T02:18:05.411 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.407+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe65c1093b0 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.413 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.408+0000 7fe6437fe640 1 -- 172.21.15.33:0/605567431 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16530 (secure 0 0 0) 0x7fe650059e60 con 0x7fe65c0fdbf0 2024-04-25T02:18:05.413 INFO:teuthology.orchestra.run.smithi033.stdout: 2024-04-25T02:18:05.413 INFO:teuthology.orchestra.run.smithi033.stdout:{"epoch":84,"fsid":"b17b7f0e-02a2-11ef-bc93-c7b262605968","created":"2024-04-25T01:26:05.674330+0000","modified":"2024-04-25T02:01:00.954060+0000","last_up_change":"2024-04-25T01:45:08.141024+0000","last_in_change":"2024-04-25T01:31:20.199327+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T01:30:26.558656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-25T01:32:55.402156+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-25T01:32:55.894097+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"68af93d4-7aa8-485f-826e-62a6e362907b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":48,"up_from":52,"up_thru":77,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6802","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6803","nonce":3103417098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6804","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6805","nonce":3103417098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6808","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6809","nonce":3103417098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6806","nonce":3103417098},{"type":"v1","addr":"172.21.15.33:6807","nonce":3103417098}]},"public_addr":"172.21.15.33:6803/3103417098","cluster_addr":"172.21.15.33:6805/3103417098","heartbeat_back_addr":"172.21.15.33:6809/3103417098","heartbeat_front_addr":"172.21.15.33:6807/3103417098","state":["exists","up"]},{"osd":1,"uuid":"1fdb7e9b-1138-43db-a11f-3e30e286c1a9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":53,"up_from":57,"up_thru":77,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6810","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6811","nonce":166114729}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6812","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6813","nonce":166114729}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6816","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6817","nonce":166114729}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6814","nonce":166114729},{"type":"v1","addr":"172.21.15.33:6815","nonce":166114729}]},"public_addr":"172.21.15.33:6811/166114729","cluster_addr":"172.21.15.33:6813/166114729","heartbeat_back_addr":"172.21.15.33:6817/166114729","heartbeat_front_addr":"172.21.15.33:6815/166114729","state":["exists","up"]},{"osd":2,"uuid":"b1239da3-566e-4df0-b5ea-df4c14959837","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":58,"up_from":62,"up_thru":77,"down_at":59,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6818","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6819","nonce":4078679342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6820","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6821","nonce":4078679342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6824","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6825","nonce":4078679342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.33:6822","nonce":4078679342},{"type":"v1","addr":"172.21.15.33:6823","nonce":4078679342}]},"public_addr":"172.21.15.33:6819/4078679342","cluster_addr":"172.21.15.33:6821/4078679342","heartbeat_back_addr":"172.21.15.33:6825/4078679342","heartbeat_front_addr":"172.21.15.33:6823/4078679342","state":["exists","up"]},{"osd":3,"uuid":"abc58749-77ca-4476-9601-cdf89d3c1e78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":63,"up_from":67,"up_thru":77,"down_at":64,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6800","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6801","nonce":1157003839}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6802","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6803","nonce":1157003839}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6806","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6807","nonce":1157003839}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6804","nonce":1157003839},{"type":"v1","addr":"172.21.15.138:6805","nonce":1157003839}]},"public_addr":"172.21.15.138:6801/1157003839","cluster_addr":"172.21.15.138:6803/1157003839","heartbeat_back_addr":"172.21.15.138:6807/1157003839","heartbeat_front_addr":"172.21.15.138:6805/1157003839","state":["exists","up"]},{"osd":4,"uuid":"83e421bd-52ba-47ad-8755-e1354f502ec5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":68,"up_from":72,"up_thru":77,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6808","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6809","nonce":4024012885}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6810","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6811","nonce":4024012885}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6814","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6815","nonce":4024012885}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6812","nonce":4024012885},{"type":"v1","addr":"172.21.15.138:6813","nonce":4024012885}]},"public_addr":"172.21.15.138:6809/4024012885","cluster_addr":"172.21.15.138:6811/4024012885","heartbeat_back_addr":"172.21.15.138:6815/4024012885","heartbeat_front_addr":"172.21.15.138:6813/4024012885","state":["exists","up"]},{"osd":5,"uuid":"266e5984-f087-408e-a84c-04f81f33af43","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":73,"up_from":77,"up_thru":77,"down_at":74,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6816","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6817","nonce":2521547597}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6818","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6819","nonce":2521547597}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6822","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6823","nonce":2521547597}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.138:6820","nonce":2521547597},{"type":"v1","addr":"172.21.15.138:6821","nonce":2521547597}]},"public_addr":"172.21.15.138:6817/2521547597","cluster_addr":"172.21.15.138:6819/2521547597","heartbeat_back_addr":"172.21.15.138:6823/2521547597","heartbeat_front_addr":"172.21.15.138:6821/2521547597","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:27.636323+0000","dead_epoch":48},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:29:55.618887+0000","dead_epoch":53},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:23.307858+0000","dead_epoch":58},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:30:47.331937+0000","dead_epoch":63},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:12.159932+0000","dead_epoch":68},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T01:31:37.317499+0000","dead_epoch":73}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.138:6825/3887358693":"2024-04-26T01:45:10.323297+0000","172.21.15.33:6800/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:6800/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6800/1387801224":"2024-04-26T01:35:34.311733+0000","172.21.15.33:0/1869074265":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/2062772238":"2024-04-26T01:35:34.311733+0000","172.21.15.33:6801/4148578452":"2024-04-26T01:26:40.396561+0000","172.21.15.138:0/4174594849":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/3334291936":"2024-04-26T01:35:34.311733+0000","172.21.15.138:6829/1792929713":"2024-04-26T01:37:35.038935+0000","172.21.15.33:6800/1731533099":"2024-04-26T01:40:40.186857+0000","172.21.15.33:6801/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6801/1731533099":"2024-04-26T01:40:40.186857+0000","172.21.15.33:6800/3376891250":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/4160574345":"2024-04-26T01:27:11.182264+0000","172.21.15.33:6826/3303942324":"2024-04-26T01:45:10.323282+0000","172.21.15.33:0/2566719862":"2024-04-26T01:28:16.045570+0000","172.21.15.138:0/330154225":"2024-04-26T01:37:35.038935+0000","172.21.15.33:6801/1387801224":"2024-04-26T01:35:34.311733+0000","172.21.15.138:0/219235313":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/454449040":"2024-04-26T01:35:34.311733+0000","172.21.15.33:0/1401659678":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/2117172792":"2024-04-26T01:26:40.396561+0000","172.21.15.138:6828/1792929713":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/2112653252":"2024-04-26T01:26:40.396561+0000","172.21.15.33:6801/395184027":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/1131828704":"2024-04-26T01:28:16.045570+0000","172.21.15.33:0/751781866":"2024-04-26T01:27:11.182264+0000","172.21.15.33:0/686286320":"2024-04-26T01:26:40.396561+0000","172.21.15.138:0/1137962382":"2024-04-26T01:37:35.038935+0000","172.21.15.33:0/746151230":"2024-04-26T01:40:40.186857+0000","172.21.15.33:0/3951183717":"2024-04-26T01:40:40.186857+0000","172.21.15.33:0/1718672295":"2024-04-26T01:40:40.186857+0000","172.21.15.33:0/2952222145":"2024-04-26T01:35:34.311733+0000","172.21.15.33:0/3161221115":"2024-04-26T01:40:40.186857+0000","172.21.15.33:6827/3303942324":"2024-04-26T01:45:10.323282+0000","172.21.15.138:6824/3887358693":"2024-04-26T01:45:10.323297+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe64406e0b0 msgr2=0x7fe644070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe64406e0b0 0x7fe644070560 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe654003080 tx=0x7fe654009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fdbf0 msgr2=0x7fe65c113620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fdbf0 0x7fe65c113620 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe650002a50 tx=0x7fe650004600 comp rx=0 tx=0).stop 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 shutdown_connections 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fe64406e0b0 0x7fe644070560 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fe65c0fdbf0 0x7fe65c113620 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 --2- 172.21.15.33:0/605567431 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fe65c0fd250 0x7fe65c1130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 >> 172.21.15.33:0/605567431 conn(0x7fe65c0f6cc0 msgr2=0x7fe65c0fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 shutdown_connections 2024-04-25T02:18:05.415 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.410+0000 7fe66106e640 1 -- 172.21.15.33:0/605567431 wait complete. 2024-04-25T02:18:05.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T01:30:26.558656+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-25T01:32:55.402156+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-25T01:32:55.894097+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T02:18:05.426 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T02:18:05.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 -- 172.21.15.33:0/567354352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 msgr2=0x7f59b4102560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 --2- 172.21.15.33:0/567354352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 0x7f59b4102560 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f59ac0048d0 tx=0x7f59ac009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 -- 172.21.15.33:0/567354352 shutdown_connections 2024-04-25T02:18:05.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 --2- 172.21.15.33:0/567354352 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59b4102b30 0x7f59b40fc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 --2- 172.21.15.33:0/567354352 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 0x7f59b4102560 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 -- 172.21.15.33:0/567354352 >> 172.21.15.33:0/567354352 conn(0x7f59b40f7da0 msgr2=0x7f59b40fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:05.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 -- 172.21.15.33:0/567354352 shutdown_connections 2024-04-25T02:18:05.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 -- 172.21.15.33:0/567354352 wait complete. 2024-04-25T02:18:05.528 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 Processor -- start 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 -- start start 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 0x7f59b406f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59b4102b30 0x7f59b406fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4071650 con 0x7f59b4102190 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.522+0000 7f59bb59d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b40701f0 con 0x7f59b4102b30 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b9339640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 0x7f59b406f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b9339640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 0x7f59b406f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:51884/0 (socket says 172.21.15.33:51884) 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b9339640 1 -- 172.21.15.33:0/3500518075 learned_addr learned my addr 172.21.15.33:0/3500518075 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b8b38640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59b4102b30 0x7f59b406fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b9339640 1 -- 172.21.15.33:0/3500518075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59b4102b30 msgr2=0x7f59b406fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b9339640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59b4102b30 0x7f59b406fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b9339640 1 -- 172.21.15.33:0/3500518075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ac005040 con 0x7f59b4102190 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b9339640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 0x7f59b406f770 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f59ac004930 tx=0x7f59ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b1ffb640 1 -- 172.21.15.33:0/3500518075 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59ac0094c0 con 0x7f59b4102190 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b4070470 con 0x7f59b4102190 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b1ffb640 1 -- 172.21.15.33:0/3500518075 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59ac032e50 con 0x7f59b4102190 2024-04-25T02:18:05.529 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59b1ffb640 1 -- 172.21.15.33:0/3500518075 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59ac0433b0 con 0x7f59b4102190 2024-04-25T02:18:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.523+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b4119390 con 0x7f59b4102190 2024-04-25T02:18:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.525+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b40fd900 con 0x7f59b4102190 2024-04-25T02:18:05.530 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.525+0000 7f59b1ffb640 1 -- 172.21.15.33:0/3500518075 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f59ac02f070 con 0x7f59b4102190 2024-04-25T02:18:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.525+0000 7f59b1ffb640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f599c06e0b0 0x7f599c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.525+0000 7f59b1ffb640 1 -- 172.21.15.33:0/3500518075 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f59ac0af920 con 0x7f59b4102190 2024-04-25T02:18:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.526+0000 7f59b8b38640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f599c06e0b0 0x7f599c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:05.531 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.526+0000 7f59b8b38640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f599c06e0b0 0x7f599c070560 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f59b4070ea0 tx=0x7f59a800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:05.534 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.530+0000 7f59b1ffb640 1 -- 172.21.15.33:0/3500518075 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f59ac078d50 con 0x7f59b4102190 2024-04-25T02:18:05.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.716+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f59b404bcc0 con 0x7f59b4102190 2024-04-25T02:18:05.721 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.717+0000 7f59b1ffb640 1 -- 172.21.15.33:0/3500518075 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f59ac07c410 con 0x7f59b4102190 2024-04-25T02:18:05.721 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 1 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f599c06e0b0 msgr2=0x7f599c070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f599c06e0b0 0x7f599c070560 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f59b4070ea0 tx=0x7f59a800b040 comp rx=0 tx=0).stop 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 msgr2=0x7f59b406f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 0x7f59b406f770 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f59ac004930 tx=0x7f59ac009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 shutdown_connections 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f599c06e0b0 0x7f599c070560 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f59b4102b30 0x7f59b406fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 --2- 172.21.15.33:0/3500518075 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f59b4102190 0x7f59b406f770 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 >> 172.21.15.33:0/3500518075 conn(0x7f59b40f7da0 msgr2=0x7f59b40f8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 shutdown_connections 2024-04-25T02:18:05.724 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.719+0000 7f59bb59d640 1 -- 172.21.15.33:0/3500518075 wait complete. 2024-04-25T02:18:05.735 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.833+0000 7fb1cad5d640 1 -- 172.21.15.33:0/2743721453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 msgr2=0x7fb1c41014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.833+0000 7fb1cad5d640 1 --2- 172.21.15.33:0/2743721453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 0x7fb1c41014e0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40048d0 tx=0x7fb1b4009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.833+0000 7fb1cad5d640 1 -- 172.21.15.33:0/2743721453 shutdown_connections 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.833+0000 7fb1cad5d640 1 --2- 172.21.15.33:0/2743721453 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c40fb0e0 0x7fb1c40fb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.833+0000 7fb1cad5d640 1 --2- 172.21.15.33:0/2743721453 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 0x7fb1c41014e0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.833+0000 7fb1cad5d640 1 -- 172.21.15.33:0/2743721453 >> 172.21.15.33:0/2743721453 conn(0x7fb1c40710c0 msgr2=0x7fb1c40714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.833+0000 7fb1cad5d640 1 -- 172.21.15.33:0/2743721453 shutdown_connections 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.833+0000 7fb1cad5d640 1 -- 172.21.15.33:0/2743721453 wait complete. 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.834+0000 7fb1cad5d640 1 Processor -- start 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.834+0000 7fb1cad5d640 1 -- start start 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.834+0000 7fb1cad5d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c40fb0e0 0x7fb1c410de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:05.840 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.834+0000 7fb1cad5d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 0x7fb1c410e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.834+0000 7fb1cad5d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40652f0 con 0x7fb1c4101110 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.834+0000 7fb1cad5d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c410e8c0 con 0x7fb1c40fb0e0 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.834+0000 7fb1c3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 0x7fb1c410e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c3fff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 0x7fb1c410e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45264/0 (socket says 172.21.15.33:45264) 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c3fff640 1 -- 172.21.15.33:0/4043439586 learned_addr learned my addr 172.21.15.33:0/4043439586 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c8af9640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c40fb0e0 0x7fb1c410de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c3fff640 1 -- 172.21.15.33:0/4043439586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c40fb0e0 msgr2=0x7fb1c410de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c3fff640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c40fb0e0 0x7fb1c410de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c3fff640 1 -- 172.21.15.33:0/4043439586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b4005040 con 0x7fb1c4101110 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c3fff640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 0x7fb1c410e380 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8002a50 tx=0x7fb1b8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c17fa640 1 -- 172.21.15.33:0/4043439586 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b8007c30 con 0x7fb1c4101110 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c17fa640 1 -- 172.21.15.33:0/4043439586 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1b800f040 con 0x7fb1c4101110 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c410eba0 con 0x7fb1c4101110 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1c17fa640 1 -- 172.21.15.33:0/4043439586 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1b800db50 con 0x7fb1c4101110 2024-04-25T02:18:05.841 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.835+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c4117160 con 0x7fb1c4101110 2024-04-25T02:18:05.842 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.836+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c40fc7d0 con 0x7fb1c4101110 2024-04-25T02:18:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.837+0000 7fb1c17fa640 1 -- 172.21.15.33:0/4043439586 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb1b801c400 con 0x7fb1c4101110 2024-04-25T02:18:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.837+0000 7fb1c17fa640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1ac06e0b0 0x7fb1ac070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.837+0000 7fb1c17fa640 1 -- 172.21.15.33:0/4043439586 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fb1b808d370 con 0x7fb1c4101110 2024-04-25T02:18:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.837+0000 7fb1c8af9640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1ac06e0b0 0x7fb1ac070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:05.843 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.838+0000 7fb1c8af9640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1ac06e0b0 0x7fb1ac070560 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4000c00 tx=0x7fb1b4009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:05.846 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:05.842+0000 7fb1c17fa640 1 -- 172.21.15.33:0/4043439586 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb1b80567a0 con 0x7fb1c4101110 2024-04-25T02:18:06.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:05 smithi033 ceph-mon[111306]: pgmap v1164: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:06.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:05 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/2617475986' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T02:18:06.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:05 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/605567431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T02:18:06.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:05 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/3500518075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T02:18:06.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.024+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fb1c4108250 con 0x7fb1c4101110 2024-04-25T02:18:06.029 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.025+0000 7fb1c17fa640 1 -- 172.21.15.33:0/4043439586 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb1b8059e60 con 0x7fb1c4101110 2024-04-25T02:18:06.029 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T02:18:06.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1ac06e0b0 msgr2=0x7fb1ac070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1ac06e0b0 0x7fb1ac070560 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4000c00 tx=0x7fb1b4009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 msgr2=0x7fb1c410e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.031 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 0x7fb1c410e380 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8002a50 tx=0x7fb1b8004600 comp rx=0 tx=0).stop 2024-04-25T02:18:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 shutdown_connections 2024-04-25T02:18:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7fb1ac06e0b0 0x7fb1ac070560 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7fb1c4101110 0x7fb1c410e380 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 --2- 172.21.15.33:0/4043439586 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7fb1c40fb0e0 0x7fb1c410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 >> 172.21.15.33:0/4043439586 conn(0x7fb1c40710c0 msgr2=0x7fb1c40f9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 shutdown_connections 2024-04-25T02:18:06.032 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.027+0000 7fb1cad5d640 1 -- 172.21.15.33:0/4043439586 wait complete. 2024-04-25T02:18:06.042 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-25T02:18:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.140+0000 7f2d5299d640 1 -- 172.21.15.33:0/3482526521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101110 msgr2=0x7f2d4c1014e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.140+0000 7f2d5299d640 1 --2- 172.21.15.33:0/3482526521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101110 0x7f2d4c1014e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f2d400048d0 tx=0x7f2d40009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.140+0000 7f2d5299d640 1 -- 172.21.15.33:0/3482526521 shutdown_connections 2024-04-25T02:18:06.146 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.140+0000 7f2d5299d640 1 --2- 172.21.15.33:0/3482526521 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d4c0fb0e0 0x7f2d4c0fb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.140+0000 7f2d5299d640 1 --2- 172.21.15.33:0/3482526521 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c101110 0x7f2d4c1014e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.140+0000 7f2d5299d640 1 -- 172.21.15.33:0/3482526521 >> 172.21.15.33:0/3482526521 conn(0x7f2d4c0710c0 msgr2=0x7f2d4c0714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.140+0000 7f2d5299d640 1 -- 172.21.15.33:0/3482526521 shutdown_connections 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.140+0000 7f2d5299d640 1 -- 172.21.15.33:0/3482526521 wait complete. 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.140+0000 7f2d5299d640 1 Processor -- start 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d5299d640 1 -- start start 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d5299d640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c0fb0e0 0x7f2d4c10de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d5299d640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d4c101110 0x7f2d4c10e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d5299d640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c0652f0 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d5299d640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4c10e8b0 con 0x7f2d4c101110 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d4bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c0fb0e0 0x7f2d4c10de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d4bfff640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c0fb0e0 0x7f2d4c10de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45294/0 (socket says 172.21.15.33:45294) 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d4bfff640 1 -- 172.21.15.33:0/4145785609 learned_addr learned my addr 172.21.15.33:0/4145785609 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:06.147 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d4b7fe640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d4c101110 0x7f2d4c10e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d4bfff640 1 -- 172.21.15.33:0/4145785609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d4c101110 msgr2=0x7f2d4c10e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d4bfff640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d4c101110 0x7f2d4c10e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d4bfff640 1 -- 172.21.15.33:0/4145785609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d40005040 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d4b7fe640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d4c101110 0x7f2d4c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.141+0000 7f2d4bfff640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c0fb0e0 0x7f2d4c10de30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2d400041d0 tx=0x7f2d40009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.142+0000 7f2d48ff9640 1 -- 172.21.15.33:0/4145785609 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d400092e0 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.142+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c10eb30 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.142+0000 7f2d48ff9640 1 -- 172.21.15.33:0/4145785609 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d40009440 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.142+0000 7f2d48ff9640 1 -- 172.21.15.33:0/4145785609 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d40043640 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.148 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.142+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4c1170f0 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.143+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4c0fc7d0 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.143+0000 7f2d48ff9640 1 -- 172.21.15.33:0/4145785609 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2d4003b980 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.144+0000 7f2d48ff9640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d3406e0b0 0x7f2d34070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:06.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.144+0000 7f2d4b7fe640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d3406e0b0 0x7f2d34070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:06.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.144+0000 7f2d48ff9640 1 -- 172.21.15.33:0/4145785609 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f2d400b08f0 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.149 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.144+0000 7f2d4b7fe640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d3406e0b0 0x7f2d34070560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c064b40 tx=0x7f2d3c009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:06.153 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.149+0000 7f2d48ff9640 1 -- 172.21.15.33:0/4145785609 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2d40079db0 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:05 smithi138 ceph-mon[81304]: pgmap v1164: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:06.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:05 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/2617475986' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T02:18:06.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:05 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/605567431' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T02:18:06.197 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:05 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/3500518075' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T02:18:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.335+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f2d4c04bcc0 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.340 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.336+0000 7f2d48ff9640 1 -- 172.21.15.33:0/4145785609 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v84) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2d4007d470 con 0x7f2d4c0fb0e0 2024-04-25T02:18:06.340 INFO:teuthology.orchestra.run.smithi033.stdout:pg_num: 32 2024-04-25T02:18:06.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d3406e0b0 msgr2=0x7f2d34070560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.342 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d3406e0b0 0x7f2d34070560 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c064b40 tx=0x7f2d3c009040 comp rx=0 tx=0).stop 2024-04-25T02:18:06.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c0fb0e0 msgr2=0x7f2d4c10de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c0fb0e0 0x7f2d4c10de30 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2d400041d0 tx=0x7f2d40009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 shutdown_connections 2024-04-25T02:18:06.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f2d3406e0b0 0x7f2d34070560 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f2d4c101110 0x7f2d4c10e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 --2- 172.21.15.33:0/4145785609 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f2d4c0fb0e0 0x7f2d4c10de30 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 >> 172.21.15.33:0/4145785609 conn(0x7f2d4c0710c0 msgr2=0x7f2d4c0f9510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:06.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 shutdown_connections 2024-04-25T02:18:06.343 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.338+0000 7f2d5299d640 1 -- 172.21.15.33:0/4145785609 wait complete. 2024-04-25T02:18:06.353 DEBUG:teuthology.orchestra.run.smithi033:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T02:18:06.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 -- 172.21.15.33:0/4081182775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 msgr2=0x7f9c440647c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 --2- 172.21.15.33:0/4081182775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 0x7f9c440647c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f9c300048d0 tx=0x7f9c30009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 -- 172.21.15.33:0/4081182775 shutdown_connections 2024-04-25T02:18:06.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 --2- 172.21.15.33:0/4081182775 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 0x7f9c440647c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 --2- 172.21.15.33:0/4081182775 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44102220 0x7f9c441025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.456 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 -- 172.21.15.33:0/4081182775 >> 172.21.15.33:0/4081182775 conn(0x7f9c440710c0 msgr2=0x7f9c440714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 -- 172.21.15.33:0/4081182775 shutdown_connections 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 -- 172.21.15.33:0/4081182775 wait complete. 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 Processor -- start 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.450+0000 7f9c4ab54640 1 -- start start 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c4ab54640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 0x7f9c44114ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c4ab54640 1 --2- >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44102220 0x7f9c44115410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c4ab54640 1 -- --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c44115a10 con 0x7f9c44064390 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c4ab54640 1 -- --> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4410f060 con 0x7f9c44102220 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c488f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 0x7f9c44114ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c488f0640 1 --2- >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 0x7f9c44114ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.33:3300/0 says I am v2:172.21.15.33:45310/0 (socket says 172.21.15.33:45310) 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c488f0640 1 -- 172.21.15.33:0/706599979 learned_addr learned my addr 172.21.15.33:0/706599979 (peer_addr_for_me v2:172.21.15.33:0/0) 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c43fff640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44102220 0x7f9c44115410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c488f0640 1 -- 172.21.15.33:0/706599979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44102220 msgr2=0x7f9c44115410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c488f0640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44102220 0x7f9c44115410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.457 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c488f0640 1 -- 172.21.15.33:0/706599979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c30005040 con 0x7f9c44064390 2024-04-25T02:18:06.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.451+0000 7f9c488f0640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 0x7f9c44114ed0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c810 tx=0x7f9c380043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:06.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.452+0000 7f9c417fa640 1 -- 172.21.15.33:0/706599979 <== mon.0 v2:172.21.15.33:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3800a7a0 con 0x7f9c44064390 2024-04-25T02:18:06.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.452+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4410f340 con 0x7f9c44064390 2024-04-25T02:18:06.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.452+0000 7f9c417fa640 1 -- 172.21.15.33:0/706599979 <== mon.0 v2:172.21.15.33:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c3800add0 con 0x7f9c44064390 2024-04-25T02:18:06.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.452+0000 7f9c417fa640 1 -- 172.21.15.33:0/706599979 <== mon.0 v2:172.21.15.33:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c380115c0 con 0x7f9c44064390 2024-04-25T02:18:06.458 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.452+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4410f830 con 0x7f9c44064390 2024-04-25T02:18:06.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.453+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c440fd920 con 0x7f9c44064390 2024-04-25T02:18:06.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.453+0000 7f9c417fa640 1 -- 172.21.15.33:0/706599979 <== mon.0 v2:172.21.15.33:3300/0 4 ==== mgrmap(e 41) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9c38012420 con 0x7f9c44064390 2024-04-25T02:18:06.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.453+0000 7f9c417fa640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c3406e0c0 0x7f9c34070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T02:18:06.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.454+0000 7f9c417fa640 1 -- 172.21.15.33:0/706599979 <== mon.0 v2:172.21.15.33:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f9c3808e0b0 con 0x7f9c44064390 2024-04-25T02:18:06.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.454+0000 7f9c43fff640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c3406e0c0 0x7f9c34070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T02:18:06.459 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.454+0000 7f9c43fff640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c3406e0c0 0x7f9c34070570 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9c300048a0 tx=0x7f9c30009da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T02:18:06.463 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.458+0000 7f9c417fa640 1 -- 172.21.15.33:0/706599979 <== mon.0 v2:172.21.15.33:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9c38057430 con 0x7f9c44064390 2024-04-25T02:18:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.658+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 --> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9c44109310 con 0x7f9c44064390 2024-04-25T02:18:06.663 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.659+0000 7f9c417fa640 1 -- 172.21.15.33:0/706599979 <== mon.0 v2:172.21.15.33:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 37 entries v84) v1 ==== 81+0+2194 (secure 0 0 0) 0x7f9c3805aaf0 con 0x7f9c44064390 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:6825/3887358693 2024-04-26T01:45:10.323297+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/395184027 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/4148578452 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1387801224 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1869074265 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2062772238 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/4148578452 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:0/4174594849 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3334291936 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:6829/1792929713 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/1731533099 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/3376891250 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:06.664 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1731533099 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6800/3376891250 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/4160574345 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6826/3303942324 2024-04-26T01:45:10.323282+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2566719862 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:0/330154225 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/1387801224 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:0/219235313 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/454449040 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1401659678 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2117172792 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:6828/1792929713 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2112653252 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6801/395184027 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1131828704 2024-04-26T01:28:16.045570+0000 2024-04-25T02:18:06.665 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/751781866 2024-04-26T01:27:11.182264+0000 2024-04-25T02:18:06.666 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/686286320 2024-04-26T01:26:40.396561+0000 2024-04-25T02:18:06.666 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:0/1137962382 2024-04-26T01:37:35.038935+0000 2024-04-25T02:18:06.666 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/746151230 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:06.666 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3951183717 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:06.666 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/1718672295 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:06.666 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/2952222145 2024-04-26T01:35:34.311733+0000 2024-04-25T02:18:06.666 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:0/3161221115 2024-04-26T01:40:40.186857+0000 2024-04-25T02:18:06.666 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.33:6827/3303942324 2024-04-26T01:45:10.323282+0000 2024-04-25T02:18:06.666 INFO:teuthology.orchestra.run.smithi033.stdout:172.21.15.138:6824/3887358693 2024-04-26T01:45:10.323297+0000 2024-04-25T02:18:06.667 INFO:teuthology.orchestra.run.smithi033.stderr:listed 37 entries 2024-04-25T02:18:06.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c3406e0c0 msgr2=0x7f9c34070570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c3406e0c0 0x7f9c34070570 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9c300048a0 tx=0x7f9c30009da0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 msgr2=0x7f9c44114ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T02:18:06.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 0x7f9c44114ed0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f9c3800c810 tx=0x7f9c380043c0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.667 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 shutdown_connections 2024-04-25T02:18:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:6800/4075941889,v1:172.21.15.33:6801/4075941889] conn(0x7f9c3406e0c0 0x7f9c34070570 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.138:3300/0,v1:172.21.15.138:6789/0] conn(0x7f9c44102220 0x7f9c44115410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 --2- 172.21.15.33:0/706599979 >> [v2:172.21.15.33:3300/0,v1:172.21.15.33:6789/0] conn(0x7f9c44064390 0x7f9c44114ed0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T02:18:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 >> 172.21.15.33:0/706599979 conn(0x7f9c440710c0 msgr2=0x7f9c440fc650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T02:18:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 shutdown_connections 2024-04-25T02:18:06.668 INFO:teuthology.orchestra.run.smithi033.stderr:2024-04-25T02:18:06.661+0000 7f9c4ab54640 1 -- 172.21.15.33:0/706599979 wait complete. 2024-04-25T02:18:06.677 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi033.front.sepia.ceph.com... 2024-04-25T02:18:06.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T02:18:06.677 DEBUG:teuthology.orchestra.run.smithi033:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T02:18:06.702 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T02:18:07.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:06 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4043439586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T02:18:07.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:06 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/4145785609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T02:18:07.020 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:06 smithi033 ceph-mon[111306]: from='client.? 172.21.15.33:0/706599979' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T02:18:07.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:06 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4043439586' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T02:18:07.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:06 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/4145785609' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T02:18:07.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:06 smithi138 ceph-mon[81304]: from='client.? 172.21.15.33:0/706599979' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T02:18:08.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:07 smithi033 ceph-mon[111306]: pgmap v1165: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:08.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:07 smithi138 ceph-mon[81304]: pgmap v1165: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:10.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:09 smithi033 ceph-mon[111306]: pgmap v1166: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:10.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:09 smithi138 ceph-mon[81304]: pgmap v1166: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:11.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:18:11.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:18:12.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:12 smithi033 ceph-mon[111306]: pgmap v1167: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:12.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:12 smithi138 ceph-mon[81304]: pgmap v1167: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:14.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:14 smithi033 ceph-mon[111306]: pgmap v1168: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:14.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:14 smithi138 ceph-mon[81304]: pgmap v1168: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:16.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:16 smithi033 ceph-mon[111306]: pgmap v1169: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:16.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:16 smithi138 ceph-mon[81304]: pgmap v1169: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:18.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:18 smithi033 ceph-mon[111306]: pgmap v1170: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:18.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:18 smithi138 ceph-mon[81304]: pgmap v1170: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:20.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:20 smithi033 ceph-mon[111306]: pgmap v1171: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:20.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:20 smithi138 ceph-mon[81304]: pgmap v1171: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:22.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:22 smithi033 ceph-mon[111306]: pgmap v1172: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:22.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:22 smithi138 ceph-mon[81304]: pgmap v1172: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:24.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:24 smithi138 ceph-mon[81304]: pgmap v1173: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:24.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:24 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:18:24.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:24 smithi033 ceph-mon[111306]: pgmap v1173: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:24.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:24 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:18:25.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:18:25.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:18:25.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:18:25.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:18:25.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:18:25.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:18:26.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:26 smithi138 ceph-mon[81304]: pgmap v1174: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:26.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:18:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:26 smithi033 ceph-mon[111306]: pgmap v1174: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:18:28.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:28 smithi138 ceph-mon[81304]: pgmap v1175: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:28.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:28 smithi033 ceph-mon[111306]: pgmap v1175: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:30.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:30 smithi138 ceph-mon[81304]: pgmap v1176: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:30.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:30 smithi033 ceph-mon[111306]: pgmap v1176: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:32.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:32 smithi138 ceph-mon[81304]: pgmap v1177: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:32.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:32 smithi033 ceph-mon[111306]: pgmap v1177: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:34.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:34 smithi138 ceph-mon[81304]: pgmap v1178: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:34.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:34 smithi033 ceph-mon[111306]: pgmap v1178: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:36.296 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:36 smithi138 ceph-mon[81304]: pgmap v1179: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:36.322 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:36 smithi033 ceph-mon[111306]: pgmap v1179: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:38.295 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:38 smithi138 ceph-mon[81304]: pgmap v1180: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:38.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:38 smithi033 ceph-mon[111306]: pgmap v1180: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:40.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:40 smithi033 ceph-mon[111306]: pgmap v1181: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:40.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:40 smithi138 ceph-mon[81304]: pgmap v1181: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:41.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:18:41.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:18:42.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:42 smithi033 ceph-mon[111306]: pgmap v1182: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:42.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:42 smithi138 ceph-mon[81304]: pgmap v1182: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:44.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:44 smithi033 ceph-mon[111306]: pgmap v1183: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:44.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:44 smithi138 ceph-mon[81304]: pgmap v1183: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:46.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:46 smithi033 ceph-mon[111306]: pgmap v1184: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:46.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:46 smithi138 ceph-mon[81304]: pgmap v1184: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:48.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:48 smithi033 ceph-mon[111306]: pgmap v1185: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:48.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:48 smithi138 ceph-mon[81304]: pgmap v1185: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:50.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:50 smithi033 ceph-mon[111306]: pgmap v1186: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:50.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:50 smithi138 ceph-mon[81304]: pgmap v1186: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:52.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:52 smithi033 ceph-mon[111306]: pgmap v1187: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:52.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:52 smithi138 ceph-mon[81304]: pgmap v1187: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:18:54.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:54 smithi033 ceph-mon[111306]: pgmap v1188: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:54.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:54 smithi138 ceph-mon[81304]: pgmap v1188: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:18:56.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:56 smithi033 ceph-mon[111306]: pgmap v1189: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:56.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:18:56.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:56 smithi138 ceph-mon[81304]: pgmap v1189: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:18:56.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:18:58.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:18:58 smithi033 ceph-mon[111306]: pgmap v1190: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:18:58.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:18:58 smithi138 ceph-mon[81304]: pgmap v1190: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:00.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:00 smithi033 ceph-mon[111306]: pgmap v1191: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:00.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:00 smithi138 ceph-mon[81304]: pgmap v1191: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:02.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:02 smithi033 ceph-mon[111306]: pgmap v1192: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:02.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:02 smithi138 ceph-mon[81304]: pgmap v1192: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:04.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:04 smithi033 ceph-mon[111306]: pgmap v1193: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:04.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:04 smithi138 ceph-mon[81304]: pgmap v1193: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:06.347 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:06 smithi138 ceph-mon[81304]: pgmap v1194: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:06.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:06 smithi033 ceph-mon[111306]: pgmap v1194: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:08.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:08 smithi033 ceph-mon[111306]: pgmap v1195: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:08.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:08 smithi138 ceph-mon[81304]: pgmap v1195: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:10.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:10 smithi033 ceph-mon[111306]: pgmap v1196: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:10.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:10 smithi138 ceph-mon[81304]: pgmap v1196: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:11.270 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:19:11.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:19:12.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:12 smithi033 ceph-mon[111306]: pgmap v1197: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:12.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:12 smithi138 ceph-mon[81304]: pgmap v1197: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:14.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:14 smithi033 ceph-mon[111306]: pgmap v1198: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:14.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:14 smithi138 ceph-mon[81304]: pgmap v1198: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:16.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:16 smithi033 ceph-mon[111306]: pgmap v1199: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:16.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:16 smithi138 ceph-mon[81304]: pgmap v1199: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:18.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:18 smithi033 ceph-mon[111306]: pgmap v1200: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:18.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:18 smithi138 ceph-mon[81304]: pgmap v1200: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:20.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:20 smithi033 ceph-mon[111306]: pgmap v1201: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:20.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:20 smithi138 ceph-mon[81304]: pgmap v1201: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:22.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:22 smithi033 ceph-mon[111306]: pgmap v1202: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:22.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:22 smithi138 ceph-mon[81304]: pgmap v1202: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:24.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:24 smithi033 ceph-mon[111306]: pgmap v1203: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:24.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:24 smithi138 ceph-mon[81304]: pgmap v1203: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:25.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:19:25.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:19:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:26 smithi033 ceph-mon[111306]: pgmap v1204: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:19:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:19:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:19:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:19:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:26 smithi138 ceph-mon[81304]: pgmap v1204: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:19:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:19:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:19:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:19:28.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:28 smithi033 ceph-mon[111306]: pgmap v1205: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:28.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:28 smithi138 ceph-mon[81304]: pgmap v1205: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:30.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:30 smithi033 ceph-mon[111306]: pgmap v1206: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:30.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:30 smithi138 ceph-mon[81304]: pgmap v1206: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:32.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:32 smithi033 ceph-mon[111306]: pgmap v1207: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:32.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:32 smithi138 ceph-mon[81304]: pgmap v1207: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:34.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:34 smithi033 ceph-mon[111306]: pgmap v1208: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:34.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:34 smithi138 ceph-mon[81304]: pgmap v1208: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:36.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:36 smithi033 ceph-mon[111306]: pgmap v1209: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:36.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:36 smithi138 ceph-mon[81304]: pgmap v1209: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:38.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:38 smithi033 ceph-mon[111306]: pgmap v1210: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:38.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:38 smithi138 ceph-mon[81304]: pgmap v1210: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:40.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:40 smithi033 ceph-mon[111306]: pgmap v1211: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:40.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:40 smithi138 ceph-mon[81304]: pgmap v1211: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:41.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:19:41.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:19:42.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:42 smithi033 ceph-mon[111306]: pgmap v1212: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:42.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:42 smithi138 ceph-mon[81304]: pgmap v1212: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:44.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:44 smithi033 ceph-mon[111306]: pgmap v1213: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:44.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:44 smithi138 ceph-mon[81304]: pgmap v1213: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:46.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:46 smithi033 ceph-mon[111306]: pgmap v1214: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:46.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:46 smithi138 ceph-mon[81304]: pgmap v1214: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:48.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:48 smithi033 ceph-mon[111306]: pgmap v1215: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:48.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:48 smithi138 ceph-mon[81304]: pgmap v1215: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:50.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:50 smithi033 ceph-mon[111306]: pgmap v1216: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:50.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:50 smithi138 ceph-mon[81304]: pgmap v1216: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:52.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:52 smithi033 ceph-mon[111306]: pgmap v1217: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:52.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:52 smithi138 ceph-mon[81304]: pgmap v1217: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:19:54.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:54 smithi033 ceph-mon[111306]: pgmap v1218: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:54.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:54 smithi138 ceph-mon[81304]: pgmap v1218: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:19:56.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:56 smithi033 ceph-mon[111306]: pgmap v1219: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:56.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:19:56.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:56 smithi138 ceph-mon[81304]: pgmap v1219: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:19:56.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:19:58.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:19:58 smithi033 ceph-mon[111306]: pgmap v1220: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:19:58.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:19:58 smithi138 ceph-mon[81304]: pgmap v1220: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:00.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:00 smithi033 ceph-mon[111306]: pgmap v1221: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:00.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:00 smithi033 ceph-mon[111306]: overall HEALTH_OK 2024-04-25T02:20:00.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:00 smithi138 ceph-mon[81304]: pgmap v1221: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:00.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:00 smithi138 ceph-mon[81304]: overall HEALTH_OK 2024-04-25T02:20:02.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:02 smithi033 ceph-mon[111306]: pgmap v1222: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:02.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:02 smithi138 ceph-mon[81304]: pgmap v1222: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:04.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:04 smithi033 ceph-mon[111306]: pgmap v1223: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:04.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:04 smithi138 ceph-mon[81304]: pgmap v1223: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:06.521 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:06 smithi033 ceph-mon[111306]: pgmap v1224: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:06.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:06 smithi138 ceph-mon[81304]: pgmap v1224: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:08.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:08 smithi033 ceph-mon[111306]: pgmap v1225: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:08.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:08 smithi138 ceph-mon[81304]: pgmap v1225: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:10.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:10 smithi033 ceph-mon[111306]: pgmap v1226: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:10.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:10 smithi138 ceph-mon[81304]: pgmap v1226: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:11.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:20:11.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:20:12.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:12 smithi033 ceph-mon[111306]: pgmap v1227: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:12.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:12 smithi138 ceph-mon[81304]: pgmap v1227: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:14.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:14 smithi033 ceph-mon[111306]: pgmap v1228: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:14.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:14 smithi138 ceph-mon[81304]: pgmap v1228: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:16.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:16 smithi033 ceph-mon[111306]: pgmap v1229: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:16.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:16 smithi138 ceph-mon[81304]: pgmap v1229: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:18.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:18 smithi033 ceph-mon[111306]: pgmap v1230: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:18.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:18 smithi138 ceph-mon[81304]: pgmap v1230: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:20.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:20 smithi033 ceph-mon[111306]: pgmap v1231: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:20.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:20 smithi138 ceph-mon[81304]: pgmap v1231: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:22.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:22 smithi033 ceph-mon[111306]: pgmap v1232: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:22.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:22 smithi138 ceph-mon[81304]: pgmap v1232: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:24.522 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:24 smithi033 ceph-mon[111306]: pgmap v1233: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:24.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:24 smithi138 ceph-mon[81304]: pgmap v1233: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:26 smithi033 ceph-mon[111306]: pgmap v1234: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:20:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:20:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:20:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:20:26.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:20:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:26 smithi138 ceph-mon[81304]: pgmap v1234: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:20:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:20:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:20:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:20:26.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:20:28.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:28 smithi033 ceph-mon[111306]: pgmap v1235: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:28.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:28 smithi138 ceph-mon[81304]: pgmap v1235: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:30.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:30 smithi033 ceph-mon[111306]: pgmap v1236: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:30.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:30 smithi138 ceph-mon[81304]: pgmap v1236: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:32.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:32 smithi033 ceph-mon[111306]: pgmap v1237: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:32.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:32 smithi138 ceph-mon[81304]: pgmap v1237: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:34.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:34 smithi033 ceph-mon[111306]: pgmap v1238: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:34.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:34 smithi138 ceph-mon[81304]: pgmap v1238: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:36.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:36 smithi033 ceph-mon[111306]: pgmap v1239: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:36.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:36 smithi138 ceph-mon[81304]: pgmap v1239: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:38.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:38 smithi033 ceph-mon[111306]: pgmap v1240: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:38.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:38 smithi138 ceph-mon[81304]: pgmap v1240: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:40.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:40 smithi033 ceph-mon[111306]: pgmap v1241: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:40.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:40 smithi138 ceph-mon[81304]: pgmap v1241: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:41.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:41 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:20:41.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:41 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:20:42.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:42 smithi033 ceph-mon[111306]: pgmap v1242: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:42.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:42 smithi138 ceph-mon[81304]: pgmap v1242: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:44.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:44 smithi033 ceph-mon[111306]: pgmap v1243: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:44.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:44 smithi138 ceph-mon[81304]: pgmap v1243: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:46.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:46 smithi033 ceph-mon[111306]: pgmap v1244: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:46.544 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:46 smithi138 ceph-mon[81304]: pgmap v1244: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:48.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:48 smithi033 ceph-mon[111306]: pgmap v1245: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:48.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:48 smithi138 ceph-mon[81304]: pgmap v1245: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:50.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:50 smithi138 ceph-mon[81304]: pgmap v1246: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:50.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:50 smithi033 ceph-mon[111306]: pgmap v1246: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:52.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:52 smithi138 ceph-mon[81304]: pgmap v1247: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:52.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:52 smithi033 ceph-mon[111306]: pgmap v1247: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:20:54.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:54 smithi138 ceph-mon[81304]: pgmap v1248: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:54.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:54 smithi033 ceph-mon[111306]: pgmap v1248: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:20:56.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:56 smithi138 ceph-mon[81304]: pgmap v1249: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:56.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:56 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:20:56.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:56 smithi033 ceph-mon[111306]: pgmap v1249: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:20:56.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:56 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:20:58.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:20:58 smithi138 ceph-mon[81304]: pgmap v1250: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:20:58.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:20:58 smithi033 ceph-mon[111306]: pgmap v1250: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:00.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:00 smithi138 ceph-mon[81304]: pgmap v1251: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:00.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:00 smithi033 ceph-mon[111306]: pgmap v1251: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:02.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:02 smithi138 ceph-mon[81304]: pgmap v1252: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:02.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:02 smithi033 ceph-mon[111306]: pgmap v1252: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:04.545 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:04 smithi138 ceph-mon[81304]: pgmap v1253: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:04.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:04 smithi033 ceph-mon[111306]: pgmap v1253: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:06.572 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:06 smithi033 ceph-mon[111306]: pgmap v1254: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:06.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:06 smithi138 ceph-mon[81304]: pgmap v1254: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:08.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:08 smithi033 ceph-mon[111306]: pgmap v1255: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:08.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:08 smithi138 ceph-mon[81304]: pgmap v1255: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:10.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:10 smithi033 ceph-mon[111306]: pgmap v1256: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:10.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:10 smithi138 ceph-mon[81304]: pgmap v1256: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:11.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:11 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:21:11.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:11 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:21:12.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:12 smithi033 ceph-mon[111306]: pgmap v1257: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:12.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:12 smithi138 ceph-mon[81304]: pgmap v1257: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:14.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:14 smithi033 ceph-mon[111306]: pgmap v1258: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:14.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:14 smithi138 ceph-mon[81304]: pgmap v1258: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:16.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:16 smithi033 ceph-mon[111306]: pgmap v1259: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:16.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:16 smithi138 ceph-mon[81304]: pgmap v1259: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:18.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:18 smithi033 ceph-mon[111306]: pgmap v1260: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:18.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:18 smithi138 ceph-mon[81304]: pgmap v1260: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:20.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:20 smithi033 ceph-mon[111306]: pgmap v1261: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:20.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:20 smithi138 ceph-mon[81304]: pgmap v1261: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:22.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:22 smithi033 ceph-mon[111306]: pgmap v1262: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:22.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:22 smithi138 ceph-mon[81304]: pgmap v1262: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:24.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:24 smithi033 ceph-mon[111306]: pgmap v1263: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:24.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:24 smithi138 ceph-mon[81304]: pgmap v1263: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:25.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:21:25.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:21:26.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:26 smithi033 ceph-mon[111306]: pgmap v1264: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:26.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:26 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:21:26.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:26 smithi138 ceph-mon[81304]: pgmap v1264: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:26.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:26 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:21:27.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:27 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:21:27.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:27 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:21:27.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:27 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:21:27.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:27 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:21:27.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:27 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:21:27.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:27 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:21:28.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:28 smithi033 ceph-mon[111306]: pgmap v1265: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:28.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:28 smithi138 ceph-mon[81304]: pgmap v1265: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:30.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:30 smithi033 ceph-mon[111306]: pgmap v1266: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:30.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:30 smithi138 ceph-mon[81304]: pgmap v1266: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:32.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:32 smithi033 ceph-mon[111306]: pgmap v1267: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:32.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:32 smithi138 ceph-mon[81304]: pgmap v1267: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:34.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:34 smithi033 ceph-mon[111306]: pgmap v1268: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:34.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:34 smithi138 ceph-mon[81304]: pgmap v1268: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:36.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:36 smithi033 ceph-mon[111306]: pgmap v1269: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:36.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:36 smithi138 ceph-mon[81304]: pgmap v1269: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:38.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:38 smithi033 ceph-mon[111306]: pgmap v1270: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:38.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:38 smithi138 ceph-mon[81304]: pgmap v1270: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:40.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:40 smithi033 ceph-mon[111306]: pgmap v1271: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:40.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:21:40.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:40 smithi138 ceph-mon[81304]: pgmap v1271: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:40.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:21:42.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:42 smithi033 ceph-mon[111306]: pgmap v1272: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:42.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:42 smithi138 ceph-mon[81304]: pgmap v1272: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:44.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:44 smithi033 ceph-mon[111306]: pgmap v1273: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:44.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:44 smithi138 ceph-mon[81304]: pgmap v1273: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:46.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:46 smithi033 ceph-mon[111306]: pgmap v1274: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:46.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:46 smithi138 ceph-mon[81304]: pgmap v1274: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:48.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:48 smithi033 ceph-mon[111306]: pgmap v1275: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:48.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:48 smithi138 ceph-mon[81304]: pgmap v1275: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:50.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:50 smithi033 ceph-mon[111306]: pgmap v1276: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:50.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:50 smithi138 ceph-mon[81304]: pgmap v1276: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:52.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:52 smithi033 ceph-mon[111306]: pgmap v1277: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:52.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:52 smithi138 ceph-mon[81304]: pgmap v1277: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:21:54.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:54 smithi033 ceph-mon[111306]: pgmap v1278: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:54.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:54 smithi138 ceph-mon[81304]: pgmap v1278: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:21:55.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:21:55.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:21:56.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:56 smithi033 ceph-mon[111306]: pgmap v1279: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:56.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:56 smithi138 ceph-mon[81304]: pgmap v1279: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:21:58.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:21:58 smithi033 ceph-mon[111306]: pgmap v1280: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:21:58.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:21:58 smithi138 ceph-mon[81304]: pgmap v1280: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:00.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:00 smithi033 ceph-mon[111306]: pgmap v1281: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:00.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:00 smithi138 ceph-mon[81304]: pgmap v1281: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:02.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:02 smithi033 ceph-mon[111306]: pgmap v1282: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:02.797 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:02 smithi138 ceph-mon[81304]: pgmap v1282: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:04.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:04 smithi033 ceph-mon[111306]: pgmap v1283: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:04.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:04 smithi138 ceph-mon[81304]: pgmap v1283: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:06.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:06 smithi033 ceph-mon[111306]: pgmap v1284: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:06.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:06 smithi138 ceph-mon[81304]: pgmap v1284: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:08.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:08 smithi033 ceph-mon[111306]: pgmap v1285: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:08.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:08 smithi138 ceph-mon[81304]: pgmap v1285: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:10.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:10 smithi033 ceph-mon[111306]: pgmap v1286: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:10.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:10 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:22:10.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:10 smithi138 ceph-mon[81304]: pgmap v1286: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:10.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:10 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:22:12.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:12 smithi033 ceph-mon[111306]: pgmap v1287: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:12.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:12 smithi138 ceph-mon[81304]: pgmap v1287: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:14.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:14 smithi033 ceph-mon[111306]: pgmap v1288: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:14.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:14 smithi138 ceph-mon[81304]: pgmap v1288: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:16.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:16 smithi033 ceph-mon[111306]: pgmap v1289: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:16.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:16 smithi138 ceph-mon[81304]: pgmap v1289: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:18.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:18 smithi033 ceph-mon[111306]: pgmap v1290: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:18.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:18 smithi138 ceph-mon[81304]: pgmap v1290: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:20.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:20 smithi033 ceph-mon[111306]: pgmap v1291: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:20.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:20 smithi138 ceph-mon[81304]: pgmap v1291: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:22.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:22 smithi033 ceph-mon[111306]: pgmap v1292: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:22.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:22 smithi138 ceph-mon[81304]: pgmap v1292: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:24.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:24 smithi033 ceph-mon[111306]: pgmap v1293: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:24.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:24 smithi138 ceph-mon[81304]: pgmap v1293: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:25.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:25 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:22:25.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:25 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:22:26.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:26 smithi033 ceph-mon[111306]: pgmap v1294: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:26.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:26 smithi138 ceph-mon[81304]: pgmap v1294: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:27.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:27 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:22:27.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:27 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T02:22:28.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:28 smithi033 ceph-mon[111306]: pgmap v1295: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:28.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:22:28.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:22:28.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:28 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:22:28.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:28 smithi138 ceph-mon[81304]: pgmap v1295: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:28.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T02:22:28.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T02:22:28.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:28 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' 2024-04-25T02:22:30.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:30 smithi033 ceph-mon[111306]: pgmap v1296: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:30.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:30 smithi138 ceph-mon[81304]: pgmap v1296: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:32.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:32 smithi033 ceph-mon[111306]: pgmap v1297: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:32.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:32 smithi138 ceph-mon[81304]: pgmap v1297: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:34.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:34 smithi033 ceph-mon[111306]: pgmap v1298: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:34.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:34 smithi138 ceph-mon[81304]: pgmap v1298: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:36.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:36 smithi033 ceph-mon[111306]: pgmap v1299: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:36.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:36 smithi138 ceph-mon[81304]: pgmap v1299: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:38.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:38 smithi033 ceph-mon[111306]: pgmap v1300: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:38.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:38 smithi138 ceph-mon[81304]: pgmap v1300: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:40.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:40 smithi033 ceph-mon[111306]: pgmap v1301: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:40.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:40 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:22:40.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:40 smithi138 ceph-mon[81304]: pgmap v1301: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:40.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:40 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:22:42.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:42 smithi033 ceph-mon[111306]: pgmap v1302: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:42.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:42 smithi138 ceph-mon[81304]: pgmap v1302: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:44.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:44 smithi033 ceph-mon[111306]: pgmap v1303: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:44.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:44 smithi138 ceph-mon[81304]: pgmap v1303: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:46.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:46 smithi033 ceph-mon[111306]: pgmap v1304: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:46.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:46 smithi138 ceph-mon[81304]: pgmap v1304: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:48.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:48 smithi033 ceph-mon[111306]: pgmap v1305: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:48.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:48 smithi138 ceph-mon[81304]: pgmap v1305: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:50.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:50 smithi033 ceph-mon[111306]: pgmap v1306: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:50.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:50 smithi138 ceph-mon[81304]: pgmap v1306: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:52.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:52 smithi033 ceph-mon[111306]: pgmap v1307: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:52.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:52 smithi138 ceph-mon[81304]: pgmap v1307: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:22:54.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:54 smithi033 ceph-mon[111306]: pgmap v1308: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:54.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:54 smithi138 ceph-mon[81304]: pgmap v1308: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:22:55.770 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:55 smithi033 ceph-mon[111306]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:22:55.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:55 smithi138 ceph-mon[81304]: from='mgr.34104 172.21.15.33:0/2614649996' entity='mgr.smithi033.asemba' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T02:22:56.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:56 smithi033 ceph-mon[111306]: pgmap v1309: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:56.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:56 smithi138 ceph-mon[81304]: pgmap v1309: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:22:58.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:22:58 smithi033 ceph-mon[111306]: pgmap v1310: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:22:58.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:22:58 smithi138 ceph-mon[81304]: pgmap v1310: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T02:23:00.775 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:00 smithi033 ceph-mon[111306]: pgmap v1311: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:23:00.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:23:00 smithi138 ceph-mon[81304]: pgmap v1311: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-04-25T02:23:02.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:02 smithi033 ceph-mon[111306]: pgmap v1312: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:23:02.795 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:23:02 smithi138 ceph-mon[81304]: pgmap v1312: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-04-25T02:23:04.769 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:04 smithi033 ceph-mon[111306]: pgmap v1313: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:23:04.794 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:23:04 smithi138 ceph-mon[81304]: pgmap v1313: 65 pgs: 65 active+clean; 257 MiB data, 1022 MiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-04-25T02:23:05.774 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-25T02:23:05.774 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-25T02:23:05.776 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T02:23:05.790 INFO:tasks.cephadm:Teardown begin 2024-04-25T02:23:05.790 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-25T02:23:05.792 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T02:23:05.825 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T02:23:05.858 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T02:23:05.858 DEBUG:teuthology.orchestra.run.smithi033:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T02:23:05.884 DEBUG:teuthology.orchestra.run.smithi138:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T02:23:05.915 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T02:23:05.915 INFO:tasks.cephadm.mon.smithi033:Stopping mon.smithi033... 2024-04-25T02:23:05.915 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033 2024-04-25T02:23:06.231 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:05 smithi033 systemd[1]: Stopping Ceph mon.smithi033 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T02:23:06.519 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:06 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:23:06.227+0000 7f05e8d1d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi033 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T02:23:06.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:06 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033[111302]: 2024-04-25T02:23:06.227+0000 7f05e8d1d700 -1 mon.smithi033@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T02:23:06.520 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:06 smithi033 podman[166009]: 2024-04-25 02:23:06.350665751 +0000 UTC m=+0.233789773 container died f072ec3fc94f7ebd13ffe37e042436dffd1a5a39f2d8ec4e5030e2c7f1da2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T02:23:06.983 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:06 smithi033 podman[166009]: 2024-04-25 02:23:06.812011677 +0000 UTC m=+0.695135699 container cleanup f072ec3fc94f7ebd13ffe37e042436dffd1a5a39f2d8ec4e5030e2c7f1da2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS) 2024-04-25T02:23:06.983 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:06 smithi033 bash[166009]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033 2024-04-25T02:23:07.237 INFO:journalctl@ceph.mon.smithi033.smithi033.stdout:Apr 25 02:23:06 smithi033 podman[166034]: 2024-04-25 02:23:06.980334007 +0000 UTC m=+0.634111041 container remove f072ec3fc94f7ebd13ffe37e042436dffd1a5a39f2d8ec4e5030e2c7f1da2863 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi033, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T02:23:07.242 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi033.service' 2024-04-25T02:23:07.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T02:23:07.289 INFO:tasks.cephadm.mon.smithi033:Stopped mon.smithi033 2024-04-25T02:23:07.289 INFO:tasks.cephadm.mon.smithi138:Stopping mon.smithi138... 2024-04-25T02:23:07.289 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi138 2024-04-25T02:23:07.604 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:23:07 smithi138 systemd[1]: Stopping Ceph mon.smithi138 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T02:23:07.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:23:07 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138[81299]: 2024-04-25T02:23:07.599+0000 7fa9dc180700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi138 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T02:23:07.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:23:07 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138[81299]: 2024-04-25T02:23:07.599+0000 7fa9dc180700 -1 mon.smithi138@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T02:23:07.887 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:23:07 smithi138 podman[118511]: 2024-04-25 02:23:07.641296752 +0000 UTC m=+0.171777329 container died e2b197ed2121ac88e0a817c814a82aaa1d0129236c0654364be06717b6bd54eb (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T02:23:08.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:23:08 smithi138 podman[118511]: 2024-04-25 02:23:08.092565552 +0000 UTC m=+0.623046161 container cleanup e2b197ed2121ac88e0a817c814a82aaa1d0129236c0654364be06717b6bd54eb (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T02:23:08.294 INFO:journalctl@ceph.mon.smithi138.smithi138.stdout:Apr 25 02:23:08 smithi138 bash[118511]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-mon-smithi138 2024-04-25T02:23:08.491 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@mon.smithi138.service' 2024-04-25T02:23:08.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T02:23:08.538 INFO:tasks.cephadm.mon.smithi138:Stopped mon.smithi138 2024-04-25T02:23:08.538 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T02:23:08.538 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.0 2024-04-25T02:23:08.861 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:08 smithi033 systemd[1]: Stopping Ceph osd.0 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T02:23:09.269 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:08 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[122751]: 2024-04-25T02:23:08.857+0000 7f8769d71700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T02:23:09.269 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:08 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[122751]: 2024-04-25T02:23:08.857+0000 7f8769d71700 -1 osd.0 84 *** Got signal Terminated *** 2024-04-25T02:23:09.269 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:08 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0[122751]: 2024-04-25T02:23:08.857+0000 7f8769d71700 -1 osd.0 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T02:23:14.697 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:14 smithi033 podman[166142]: 2024-04-25 02:23:14.418024564 +0000 UTC m=+5.681714686 container died 1da61cf4b50bba3c83539faf2ef9269d5baeb2d2db1b383c435eef01ad729796 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T02:23:15.019 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:14 smithi033 podman[166142]: 2024-04-25 02:23:14.792327767 +0000 UTC m=+6.056017889 container cleanup 1da61cf4b50bba3c83539faf2ef9269d5baeb2d2db1b383c435eef01ad729796 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T02:23:15.019 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:14 smithi033 bash[166142]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0 2024-04-25T02:23:15.425 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:15 smithi033 podman[166159]: 2024-04-25 02:23:15.094002977 +0000 UTC m=+0.669982836 container remove 1da61cf4b50bba3c83539faf2ef9269d5baeb2d2db1b383c435eef01ad729796 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T02:23:15.675 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:15 smithi033 podman[166253]: 2024-04-25 02:23:15.326012219 +0000 UTC m=+0.018838997 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T02:23:15.675 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:15 smithi033 podman[166253]: 2024-04-25 02:23:15.445191891 +0000 UTC m=+0.138018713 container create d5633e893f7f51c537eb764dc305d8e6cea5fd68b436135e67aa84141e72be87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T02:23:15.675 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:15 smithi033 podman[166253]: 2024-04-25 02:23:15.672828021 +0000 UTC m=+0.365654798 container init d5633e893f7f51c537eb764dc305d8e6cea5fd68b436135e67aa84141e72be87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T02:23:15.985 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:15 smithi033 podman[166253]: 2024-04-25 02:23:15.676238761 +0000 UTC m=+0.369065541 container start d5633e893f7f51c537eb764dc305d8e6cea5fd68b436135e67aa84141e72be87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T02:23:15.985 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:15 smithi033 podman[166253]: 2024-04-25 02:23:15.764574083 +0000 UTC m=+0.457400879 container attach d5633e893f7f51c537eb764dc305d8e6cea5fd68b436135e67aa84141e72be87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T02:23:16.269 INFO:journalctl@ceph.osd.0.smithi033.stdout:Apr 25 02:23:15 smithi033 podman[166253]: 2024-04-25 02:23:15.982441361 +0000 UTC m=+0.675268140 container died d5633e893f7f51c537eb764dc305d8e6cea5fd68b436135e67aa84141e72be87 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T02:23:16.663 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T02:23:16.712 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T02:23:16.712 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T02:23:16.712 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T02:23:16.712 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.1 2024-04-25T02:23:16.963 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:16 smithi033 systemd[1]: Stopping Ceph osd.1 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T02:23:17.269 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:17 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[128505]: 2024-04-25T02:23:17.046+0000 7fa8c710d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T02:23:17.269 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:17 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[128505]: 2024-04-25T02:23:17.046+0000 7fa8c710d700 -1 osd.1 84 *** Got signal Terminated *** 2024-04-25T02:23:17.270 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:17 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1[128505]: 2024-04-25T02:23:17.046+0000 7fa8c710d700 -1 osd.1 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T02:23:22.701 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:22 smithi033 podman[166362]: 2024-04-25 02:23:22.42375755 +0000 UTC m=+5.503525329 container died 4ab9ac022ca9ab34e180f8acf33e80ecbf86827b6dd437b2ab1312150f6ea971 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T02:23:23.019 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:22 smithi033 podman[166362]: 2024-04-25 02:23:22.914417218 +0000 UTC m=+5.994185003 container cleanup 4ab9ac022ca9ab34e180f8acf33e80ecbf86827b6dd437b2ab1312150f6ea971 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T02:23:23.019 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:22 smithi033 bash[166362]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1 2024-04-25T02:23:23.497 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:23 smithi033 podman[166391]: 2024-04-25 02:23:23.157725288 +0000 UTC m=+0.733672639 container remove 4ab9ac022ca9ab34e180f8acf33e80ecbf86827b6dd437b2ab1312150f6ea971 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T02:23:23.769 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:23 smithi033 podman[166484]: 2024-04-25 02:23:23.397689727 +0000 UTC m=+0.019957345 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T02:23:23.769 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:23 smithi033 podman[166484]: 2024-04-25 02:23:23.5923364 +0000 UTC m=+0.214604004 container create ad58c46b137206cd1ed7a8ca61785d1559a35c767a6b441af39f40fbd3a64fd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T02:23:24.178 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:23 smithi033 podman[166484]: 2024-04-25 02:23:23.870399159 +0000 UTC m=+0.492666763 container init ad58c46b137206cd1ed7a8ca61785d1559a35c767a6b441af39f40fbd3a64fd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T02:23:24.178 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:23 smithi033 podman[166484]: 2024-04-25 02:23:23.873806126 +0000 UTC m=+0.496073730 container start ad58c46b137206cd1ed7a8ca61785d1559a35c767a6b441af39f40fbd3a64fd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T02:23:24.178 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:23 smithi033 podman[166484]: 2024-04-25 02:23:23.945238552 +0000 UTC m=+0.567506172 container attach ad58c46b137206cd1ed7a8ca61785d1559a35c767a6b441af39f40fbd3a64fd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T02:23:24.442 INFO:journalctl@ceph.osd.1.smithi033.stdout:Apr 25 02:23:24 smithi033 podman[166484]: 2024-04-25 02:23:24.175730905 +0000 UTC m=+0.797998514 container died ad58c46b137206cd1ed7a8ca61785d1559a35c767a6b441af39f40fbd3a64fd6 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T02:23:24.745 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T02:23:24.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T02:23:24.788 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T02:23:24.788 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T02:23:24.788 DEBUG:teuthology.orchestra.run.smithi033:> sudo systemctl stop ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.2 2024-04-25T02:23:25.019 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:24 smithi033 systemd[1]: Stopping Ceph osd.2 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T02:23:25.519 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:25 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[134349]: 2024-04-25T02:23:25.110+0000 7f577d7eb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T02:23:25.520 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:25 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[134349]: 2024-04-25T02:23:25.110+0000 7f577d7eb700 -1 osd.2 84 *** Got signal Terminated *** 2024-04-25T02:23:25.520 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:25 smithi033 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2[134349]: 2024-04-25T02:23:25.110+0000 7f577d7eb700 -1 osd.2 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T02:23:30.915 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:30 smithi033 podman[166591]: 2024-04-25 02:23:30.62935889 +0000 UTC m=+5.617601302 container died 2cb795d14a3f502d5070b862f000aeade35e180caa54cf4af5f85c7acb886068 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T02:23:31.270 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:31 smithi033 podman[166591]: 2024-04-25 02:23:31.010777384 +0000 UTC m=+5.999019788 container cleanup 2cb795d14a3f502d5070b862f000aeade35e180caa54cf4af5f85c7acb886068 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T02:23:31.271 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:31 smithi033 bash[166591]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2 2024-04-25T02:23:31.632 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:31 smithi033 podman[166612]: 2024-04-25 02:23:31.283319773 +0000 UTC m=+0.645647075 container remove 2cb795d14a3f502d5070b862f000aeade35e180caa54cf4af5f85c7acb886068 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T02:23:31.927 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:31 smithi033 podman[166706]: 2024-04-25 02:23:31.630128479 +0000 UTC m=+0.104786243 container create 9916059ff05f5c0abac7f0305aa72842d534d2616d88e9d2a36a90f0b3c24370 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T02:23:31.927 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:31 smithi033 podman[166706]: 2024-04-25 02:23:31.543873925 +0000 UTC m=+0.018531670 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T02:23:31.927 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:31 smithi033 podman[166706]: 2024-04-25 02:23:31.924719394 +0000 UTC m=+0.399377123 container init 9916059ff05f5c0abac7f0305aa72842d534d2616d88e9d2a36a90f0b3c24370 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T02:23:32.231 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:31 smithi033 podman[166706]: 2024-04-25 02:23:31.928041093 +0000 UTC m=+0.402698812 container start 9916059ff05f5c0abac7f0305aa72842d534d2616d88e9d2a36a90f0b3c24370 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T02:23:32.231 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:31 smithi033 podman[166706]: 2024-04-25 02:23:31.999554865 +0000 UTC m=+0.474212596 container attach 9916059ff05f5c0abac7f0305aa72842d534d2616d88e9d2a36a90f0b3c24370 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T02:23:32.519 INFO:journalctl@ceph.osd.2.smithi033.stdout:Apr 25 02:23:32 smithi033 podman[166706]: 2024-04-25 02:23:32.228647242 +0000 UTC m=+0.703304979 container died 9916059ff05f5c0abac7f0305aa72842d534d2616d88e9d2a36a90f0b3c24370 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-25T02:23:32.836 DEBUG:teuthology.orchestra.run.smithi033:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T02:23:32.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T02:23:32.879 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T02:23:32.879 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T02:23:32.880 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.3 2024-04-25T02:23:33.173 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:32 smithi138 systemd[1]: Stopping Ceph osd.3 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T02:23:33.545 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:33 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[90836]: 2024-04-25T02:23:33.169+0000 7fb50b6fe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T02:23:33.545 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:33 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[90836]: 2024-04-25T02:23:33.169+0000 7fb50b6fe700 -1 osd.3 84 *** Got signal Terminated *** 2024-04-25T02:23:33.545 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:33 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3[90836]: 2024-04-25T02:23:33.169+0000 7fb50b6fe700 -1 osd.3 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T02:23:38.545 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:38 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:38.239+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:38.818 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:38 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:38.549+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:12.201694+0000 front 2024-04-25T02:23:12.201616+0000 (oldest deadline 2024-04-25T02:23:38.101337+0000) 2024-04-25T02:23:38.819 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:38 smithi138 podman[118636]: 2024-04-25 02:23:38.695279343 +0000 UTC m=+5.625257669 container died d00965714a8d750bb5a15f73a0baaac55b3b751a67fe5b4639c3705756f964f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T02:23:39.530 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:39 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:39.199+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:39.530 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:39 smithi138 podman[118636]: 2024-04-25 02:23:39.08732428 +0000 UTC m=+6.017302612 container cleanup d00965714a8d750bb5a15f73a0baaac55b3b751a67fe5b4639c3705756f964f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-25T02:23:39.530 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:39 smithi138 bash[118636]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3 2024-04-25T02:23:39.531 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:39 smithi138 podman[118654]: 2024-04-25 02:23:39.322294142 +0000 UTC m=+0.626355594 container remove d00965714a8d750bb5a15f73a0baaac55b3b751a67fe5b4639c3705756f964f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-04-25T02:23:39.791 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:39 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:39.525+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:12.201694+0000 front 2024-04-25T02:23:12.201616+0000 (oldest deadline 2024-04-25T02:23:38.101337+0000) 2024-04-25T02:23:39.792 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:39 smithi138 podman[118747]: 2024-04-25 02:23:39.571737493 +0000 UTC m=+0.020210391 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T02:23:39.792 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:39 smithi138 podman[118747]: 2024-04-25 02:23:39.681523671 +0000 UTC m=+0.129996562 container create 52866e184f0c0423088e6b9b356481d926df3fe95b770f203b682844ff9a306b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-25T02:23:40.045 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:39 smithi138 podman[118747]: 2024-04-25 02:23:39.918366361 +0000 UTC m=+0.366839252 container init 52866e184f0c0423088e6b9b356481d926df3fe95b770f203b682844ff9a306b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T02:23:40.045 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:39 smithi138 podman[118747]: 2024-04-25 02:23:39.921745843 +0000 UTC m=+0.370218742 container start 52866e184f0c0423088e6b9b356481d926df3fe95b770f203b682844ff9a306b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T02:23:40.046 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:39 smithi138 podman[118747]: 2024-04-25 02:23:39.992431988 +0000 UTC m=+0.440904889 container attach 52866e184f0c0423088e6b9b356481d926df3fe95b770f203b682844ff9a306b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T02:23:40.542 INFO:journalctl@ceph.osd.3.smithi138.stdout:Apr 25 02:23:40 smithi138 podman[118747]: 2024-04-25 02:23:40.214965428 +0000 UTC m=+0.663438323 container died 52866e184f0c0423088e6b9b356481d926df3fe95b770f203b682844ff9a306b (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T02:23:40.542 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:40 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:40.217+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:40.794 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:40 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:40.537+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:12.201694+0000 front 2024-04-25T02:23:12.201616+0000 (oldest deadline 2024-04-25T02:23:38.101337+0000) 2024-04-25T02:23:40.866 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T02:23:40.904 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T02:23:40.904 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T02:23:40.904 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T02:23:40.904 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.4 2024-04-25T02:23:41.210 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:40 smithi138 systemd[1]: Stopping Ceph osd.4 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T02:23:41.534 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:41 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:41.253+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:41.535 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:41 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:41.206+0000 7f73632ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T02:23:41.535 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:41 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:41.206+0000 7f73632ef700 -1 osd.4 84 *** Got signal Terminated *** 2024-04-25T02:23:41.535 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:41 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:41.206+0000 7f73632ef700 -1 osd.4 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T02:23:41.535 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:41 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:41.531+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:12.201694+0000 front 2024-04-25T02:23:12.201616+0000 (oldest deadline 2024-04-25T02:23:38.101337+0000) 2024-04-25T02:23:42.505 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:42 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:42.226+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:42.794 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:42 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:42.501+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:12.201694+0000 front 2024-04-25T02:23:12.201616+0000 (oldest deadline 2024-04-25T02:23:38.101337+0000) 2024-04-25T02:23:43.481 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:43 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:43.202+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:43.794 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:43 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:43.477+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:12.201694+0000 front 2024-04-25T02:23:12.201616+0000 (oldest deadline 2024-04-25T02:23:38.101337+0000) 2024-04-25T02:23:43.795 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:43 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:43.477+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:18.101865+0000 front 2024-04-25T02:23:18.101877+0000 (oldest deadline 2024-04-25T02:23:43.401811+0000) 2024-04-25T02:23:44.470 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:44 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:44.198+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:44.471 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:44 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:44.198+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:44.794 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:44 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:44.467+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:12.201694+0000 front 2024-04-25T02:23:12.201616+0000 (oldest deadline 2024-04-25T02:23:38.101337+0000) 2024-04-25T02:23:44.795 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:44 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:44.467+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:18.101865+0000 front 2024-04-25T02:23:18.101877+0000 (oldest deadline 2024-04-25T02:23:43.401811+0000) 2024-04-25T02:23:45.516 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:45 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:45.240+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:45.517 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:45 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:45.240+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:45.794 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:45 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:45.513+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:12.201694+0000 front 2024-04-25T02:23:12.201616+0000 (oldest deadline 2024-04-25T02:23:38.101337+0000) 2024-04-25T02:23:45.795 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:45 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4[96151]: 2024-04-25T02:23:45.513+0000 7f735c0dc700 -1 osd.4 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:18.101865+0000 front 2024-04-25T02:23:18.101877+0000 (oldest deadline 2024-04-25T02:23:43.401811+0000) 2024-04-25T02:23:46.544 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:46 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:46.256+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:46.545 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:46 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:46.256+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:46.919 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:46 smithi138 podman[118856]: 2024-04-25 02:23:46.65248047 +0000 UTC m=+5.539701855 container died 5adf4eaefb0b8ece675527bac237972a1bb9310baaf2399377c75fdf6b3fc653 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T02:23:47.261 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:47 smithi138 podman[118856]: 2024-04-25 02:23:47.007755424 +0000 UTC m=+5.894976796 container cleanup 5adf4eaefb0b8ece675527bac237972a1bb9310baaf2399377c75fdf6b3fc653 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T02:23:47.261 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:47 smithi138 bash[118856]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4 2024-04-25T02:23:47.544 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:47 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:47.283+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:47.545 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:47 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:47.283+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:47.545 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:47 smithi138 podman[118873]: 2024-04-25 02:23:47.259042205 +0000 UTC m=+0.601660343 container remove 5adf4eaefb0b8ece675527bac237972a1bb9310baaf2399377c75fdf6b3fc653 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T02:23:47.906 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:47 smithi138 podman[118968]: 2024-04-25 02:23:47.486460219 +0000 UTC m=+0.018509353 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T02:23:47.907 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:47 smithi138 podman[118968]: 2024-04-25 02:23:47.585096382 +0000 UTC m=+0.117145543 container create 6a89cc85150164f43d7db81e8416af54a762b18a2c58942294a6e92e4f294ce8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T02:23:47.907 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:47 smithi138 podman[118968]: 2024-04-25 02:23:47.829052437 +0000 UTC m=+0.361101553 container init 6a89cc85150164f43d7db81e8416af54a762b18a2c58942294a6e92e4f294ce8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T02:23:47.907 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:47 smithi138 podman[118968]: 2024-04-25 02:23:47.832152923 +0000 UTC m=+0.364202039 container start 6a89cc85150164f43d7db81e8416af54a762b18a2c58942294a6e92e4f294ce8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T02:23:47.907 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:47 smithi138 podman[118968]: 2024-04-25 02:23:47.904075419 +0000 UTC m=+0.436124542 container attach 6a89cc85150164f43d7db81e8416af54a762b18a2c58942294a6e92e4f294ce8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T02:23:48.294 INFO:journalctl@ceph.osd.4.smithi138.stdout:Apr 25 02:23:48 smithi138 podman[118994]: 2024-04-25 02:23:48.153600914 +0000 UTC m=+0.018996799 container died 6a89cc85150164f43d7db81e8416af54a762b18a2c58942294a6e92e4f294ce8 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T02:23:48.669 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:48 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:48.310+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:48.670 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:48 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:48.310+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:48.692 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T02:23:48.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T02:23:48.732 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T02:23:48.732 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T02:23:48.732 DEBUG:teuthology.orchestra.run.smithi138:> sudo systemctl stop ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.5 2024-04-25T02:23:49.044 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:48 smithi138 systemd[1]: Stopping Ceph osd.5 for b17b7f0e-02a2-11ef-bc93-c7b262605968... 2024-04-25T02:23:49.338 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:49 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:49.051+0000 7f299dc10700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T02:23:49.338 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:49 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:49.051+0000 7f299dc10700 -1 osd.5 84 *** Got signal Terminated *** 2024-04-25T02:23:49.338 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:49 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:49.051+0000 7f299dc10700 -1 osd.5 84 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T02:23:49.794 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:49 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:49.334+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:49.794 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:49 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:49.334+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:50.794 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:50 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:50.337+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:50.795 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:50 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:50.337+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:51.794 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:51 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:51.372+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:51.794 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:51 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:51.372+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:52.794 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:52.404+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:52.795 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:52.404+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:52.795 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:52 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:52.404+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6822 osd.2 since back 2024-04-25T02:23:26.324376+0000 front 2024-04-25T02:23:26.324469+0000 (oldest deadline 2024-04-25T02:23:52.224151+0000) 2024-04-25T02:23:53.794 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:53 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:53.393+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6806 osd.0 since back 2024-04-25T02:23:09.422025+0000 front 2024-04-25T02:23:09.422010+0000 (oldest deadline 2024-04-25T02:23:38.222511+0000) 2024-04-25T02:23:53.794 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:53 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:53.393+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6814 osd.1 since back 2024-04-25T02:23:21.723530+0000 front 2024-04-25T02:23:21.723660+0000 (oldest deadline 2024-04-25T02:23:43.423377+0000) 2024-04-25T02:23:53.795 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:53 smithi138 ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5[101308]: 2024-04-25T02:23:53.393+0000 7f29969fd700 -1 osd.5 84 heartbeat_check: no reply from 172.21.15.33:6822 osd.2 since back 2024-04-25T02:23:26.324376+0000 front 2024-04-25T02:23:26.324469+0000 (oldest deadline 2024-04-25T02:23:52.224151+0000) 2024-04-25T02:23:54.795 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:54 smithi138 podman[119073]: 2024-04-25 02:23:54.502968392 +0000 UTC m=+5.554948065 container died ab01b5d50ec749143885e7882d30df52c7b6b2273f4f1e3f5d91813b32c6a134 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T02:23:55.294 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:54 smithi138 podman[119073]: 2024-04-25 02:23:54.827610948 +0000 UTC m=+5.879590609 container cleanup ab01b5d50ec749143885e7882d30df52c7b6b2273f4f1e3f5d91813b32c6a134 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T02:23:55.295 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:54 smithi138 bash[119073]: ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5 2024-04-25T02:23:55.295 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:55 smithi138 podman[119093]: 2024-04-25 02:23:55.004122172 +0000 UTC m=+0.501571528 container remove ab01b5d50ec749143885e7882d30df52c7b6b2273f4f1e3f5d91813b32c6a134 (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T02:23:55.768 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:55 smithi138 podman[119182]: 2024-04-25 02:23:55.351325775 +0000 UTC m=+0.017877113 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T02:23:55.768 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:55 smithi138 podman[119182]: 2024-04-25 02:23:55.454849328 +0000 UTC m=+0.121400632 container create 107a74d87100dc0476e684d2304ad6fd2da42d0ceb23866e6c1b9613e551cbcd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T02:23:55.768 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:55 smithi138 podman[119182]: 2024-04-25 02:23:55.68246472 +0000 UTC m=+0.349016026 container init 107a74d87100dc0476e684d2304ad6fd2da42d0ceb23866e6c1b9613e551cbcd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T02:23:55.768 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:55 smithi138 podman[119182]: 2024-04-25 02:23:55.685404413 +0000 UTC m=+0.351955729 container start 107a74d87100dc0476e684d2304ad6fd2da42d0ceb23866e6c1b9613e551cbcd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-25T02:23:55.768 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:55 smithi138 podman[119182]: 2024-04-25 02:23:55.765566474 +0000 UTC m=+0.432117788 container attach 107a74d87100dc0476e684d2304ad6fd2da42d0ceb23866e6c1b9613e551cbcd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T02:23:56.045 INFO:journalctl@ceph.osd.5.smithi138.stdout:Apr 25 02:23:55 smithi138 podman[119182]: 2024-04-25 02:23:55.976640342 +0000 UTC m=+0.643191656 container died 107a74d87100dc0476e684d2304ad6fd2da42d0ceb23866e6c1b9613e551cbcd (image=quay.ceph.io/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T02:23:56.570 DEBUG:teuthology.orchestra.run.smithi138:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b17b7f0e-02a2-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T02:23:56.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T02:23:56.609 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T02:23:56.610 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T02:23:59.911 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi033.stderr:ceph-fuse[74113]: fuse finished with error 0 and tester_r 0 2024-04-25T02:24:18.338 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T02:24:33.517 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T02:24:33.550 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T02:24:33.576 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T02:24:33.579 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322/remote/smithi033/crash 2024-04-25T02:24:33.580 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash -- . 2024-04-25T02:24:33.617 INFO:teuthology.orchestra.run.smithi033.stderr:tar: /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T02:24:33.617 INFO:teuthology.orchestra.run.smithi033.stderr:tar: Error is not recoverable: exiting now 2024-04-25T02:24:33.619 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322/remote/smithi138/crash 2024-04-25T02:24:33.620 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash -- . 2024-04-25T02:24:33.644 INFO:teuthology.orchestra.run.smithi138.stderr:tar: /var/lib/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T02:24:33.644 INFO:teuthology.orchestra.run.smithi138.stderr:tar: Error is not recoverable: exiting now 2024-04-25T02:24:33.644 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T02:24:33.645 DEBUG:teuthology.orchestra.run.smithi033:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-25T02:24:33.699 INFO:teuthology.orchestra.run.smithi033.stdout:1714009055.8845022 mon.smithi033 (mon.0) 903 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-04-25T02:24:33.699 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-04-25T02:24:33.699 INFO:tasks.cephadm:Compressing logs... 2024-04-25T02:24:33.700 DEBUG:teuthology.orchestra.run.smithi033:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T02:24:33.742 DEBUG:teuthology.orchestra.run.smithi138:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T02:24:33.769 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mgr.smithi033.asemba.log 2024-04-25T02:24:33.769 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-25T02:24:33.770 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.log 2024-04-25T02:24:33.770 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mgr.smithi033.asemba.log: -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T02:24:33.770 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mon.smithi033.log 2024-04-25T02:24:33.770 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T02:24:33.770 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T02:24:33.770 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mon.smithi033.log: /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T02:24:33.771 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi033.hobkqr.log 2024-04-25T02:24:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T02:24:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi033.hobkqr.log: 89.3% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T02:24:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi033.qqhhcz.log 2024-04-25T02:24:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T02:24:33.779 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T02:24:33.779 INFO:teuthology.orchestra.run.smithi033.stderr: 91.0% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.audit.log.gz/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi033.qqhhcz.log: 2024-04-25T02:24:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi138.rvxnby.log: No such file or directory 2024-04-25T02:24:33.783 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-25T02:24:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.log 2024-04-25T02:24:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi138.rvxnby.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T02:24:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi138.jwzxlz.log 2024-04-25T02:24:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T02:24:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T02:24:33.783 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi138.jwzxlz.log: /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T02:24:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T02:24:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mgr.smithi138.mmfhyi.log 2024-04-25T02:24:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.1% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T02:24:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T02:24:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mgr.smithi138.mmfhyi.log: 89.3% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T02:24:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mon.smithi138.log 2024-04-25T02:24:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T02:24:33.784 INFO:teuthology.orchestra.run.smithi138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-04-25T02:24:33.787 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mon.smithi138.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T02:24:33.787 INFO:teuthology.orchestra.run.smithi033.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-25T02:24:33.794 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/ceph-client.1.log: 91.0% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T02:24:33.800 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T02:24:33.808 INFO:teuthology.orchestra.run.smithi138.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T02:24:33.814 INFO:teuthology.orchestra.run.smithi033.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 91.2% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T02:24:33.851 INFO:teuthology.orchestra.run.smithi138.stderr: 90.2% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mgr.smithi138.mmfhyi.log.gz 2024-04-25T02:24:33.884 INFO:teuthology.orchestra.run.smithi033.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T02:24:34.437 INFO:teuthology.orchestra.run.smithi033.stderr: 89.5% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mgr.smithi033.asemba.log.gz 2024-04-25T02:24:34.635 INFO:teuthology.orchestra.run.smithi138.stderr: 92.5% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mon.smithi138.log.gz 2024-04-25T02:24:35.628 INFO:teuthology.orchestra.run.smithi033.stderr: 89.9% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mon.smithi033.log.gz 2024-04-25T02:24:38.560 INFO:teuthology.orchestra.run.smithi138.stderr: 93.3% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T02:24:38.899 INFO:teuthology.orchestra.run.smithi033.stderr: 93.2% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T02:24:39.772 INFO:teuthology.orchestra.run.smithi138.stderr: 93.5% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T02:24:39.860 INFO:teuthology.orchestra.run.smithi033.stderr: 93.3% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T02:24:41.666 INFO:teuthology.orchestra.run.smithi033.stderr: 93.3% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T02:24:42.268 INFO:teuthology.orchestra.run.smithi138.stderr: 93.4% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T02:24:42.522 INFO:teuthology.orchestra.run.smithi138.stderr: 95.2% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi138.rvxnby.log.gz 2024-04-25T02:24:42.850 INFO:teuthology.orchestra.run.smithi033.stderr: 95.1% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi033.hobkqr.log.gz 2024-04-25T02:24:50.076 INFO:teuthology.orchestra.run.smithi138.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-25T02:24:50.076 INFO:teuthology.orchestra.run.smithi138.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-25T02:24:50.476 INFO:teuthology.orchestra.run.smithi033.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-25T02:25:21.690 INFO:teuthology.orchestra.run.smithi138.stderr: 93.4% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi138.jwzxlz.log.gz 2024-04-25T02:25:21.691 INFO:teuthology.orchestra.run.smithi138.stderr: 2024-04-25T02:25:21.691 INFO:teuthology.orchestra.run.smithi138.stderr:real 0m47.934s 2024-04-25T02:25:21.691 INFO:teuthology.orchestra.run.smithi138.stderr:user 1m30.280s 2024-04-25T02:25:21.691 INFO:teuthology.orchestra.run.smithi138.stderr:sys 0m2.801s 2024-04-25T02:25:23.568 INFO:teuthology.orchestra.run.smithi033.stderr: 93.3% -- replaced with /var/log/ceph/b17b7f0e-02a2-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi033.qqhhcz.log.gz 2024-04-25T02:25:23.570 INFO:teuthology.orchestra.run.smithi033.stderr: 2024-04-25T02:25:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:real 0m49.815s 2024-04-25T02:25:23.570 INFO:teuthology.orchestra.run.smithi033.stderr:user 1m34.267s 2024-04-25T02:25:23.571 INFO:teuthology.orchestra.run.smithi033.stderr:sys 0m2.805s 2024-04-25T02:25:23.571 INFO:tasks.cephadm:Archiving logs... 2024-04-25T02:25:23.571 DEBUG:teuthology.misc:Transferring archived files from smithi033:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322/remote/smithi033/log 2024-04-25T02:25:23.572 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T02:25:36.514 DEBUG:teuthology.misc:Transferring archived files from smithi138:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322/remote/smithi138/log 2024-04-25T02:25:36.718 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T02:25:49.245 INFO:tasks.cephadm:Removing cluster... 2024-04-25T02:25:49.439 DEBUG:teuthology.orchestra.run.smithi033:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 --force 2024-04-25T02:25:50.028 DEBUG:teuthology.orchestra.run.smithi138:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b17b7f0e-02a2-11ef-bc93-c7b262605968 --force 2024-04-25T02:25:50.590 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T02:25:50.591 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T02:25:50.608 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T02:25:50.623 INFO:tasks.cephadm:Teardown complete 2024-04-25T02:25:50.624 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T02:25:50.638 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-25T02:25:50.638 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T02:25:50.639 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T02:25:50.650 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T02:25:50.738 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T02:25:50.749 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T02:25:50.749 DEBUG:teuthology.orchestra.run.smithi033:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T02:25:50.752 DEBUG:teuthology.orchestra.run.smithi138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T02:25:50.765 INFO:teuthology.orchestra.run.smithi033.stderr:bash: line 1: ntpq: command not found 2024-04-25T02:25:50.766 INFO:teuthology.orchestra.run.smithi138.stderr:bash: line 1: ntpq: command not found 2024-04-25T02:25:50.769 INFO:teuthology.orchestra.run.smithi033.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T02:25:50.769 INFO:teuthology.orchestra.run.smithi033.stdout:=============================================================================== 2024-04-25T02:25:50.769 INFO:teuthology.orchestra.run.smithi033.stdout:^- hv01.front.sepia.ceph.com 2 6 377 62 +1765us[+1769us] +/- 574ms 2024-04-25T02:25:50.769 INFO:teuthology.orchestra.run.smithi033.stdout:^* hv02.front.sepia.ceph.com 3 7 377 57 -239us[ -235us] +/- 55ms 2024-04-25T02:25:50.769 INFO:teuthology.orchestra.run.smithi033.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +337us[ +341us] +/- 63ms 2024-04-25T02:25:50.769 INFO:teuthology.orchestra.run.smithi033.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T02:25:50.771 INFO:teuthology.orchestra.run.smithi138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T02:25:50.771 INFO:teuthology.orchestra.run.smithi138.stdout:=============================================================================== 2024-04-25T02:25:50.771 INFO:teuthology.orchestra.run.smithi138.stdout:^- hv01.front.sepia.ceph.com 2 6 377 61 +1768us[+1768us] +/- 574ms 2024-04-25T02:25:50.771 INFO:teuthology.orchestra.run.smithi138.stdout:^* hv02.front.sepia.ceph.com 3 7 377 6 -233us[ -233us] +/- 56ms 2024-04-25T02:25:50.771 INFO:teuthology.orchestra.run.smithi138.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 0 +324us[ +324us] +/- 64ms 2024-04-25T02:25:50.771 INFO:teuthology.orchestra.run.smithi138.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T02:25:50.771 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T02:25:50.782 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T02:25:50.783 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T02:25:50.794 DEBUG:teuthology.orchestra.run.smithi033:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T02:25:50.893 DEBUG:teuthology.orchestra.run.smithi138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T02:25:50.964 DEBUG:teuthology.orchestra.run.smithi033:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T02:25:50.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T02:25:50.993 DEBUG:teuthology.orchestra.run.smithi138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T02:25:51.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T02:25:51.021 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T02:25:51.032 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T02:25:51.045 INFO:teuthology.task.internal:Duration was 3967.221866 seconds 2024-04-25T02:25:51.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T02:25:51.057 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T02:25:51.057 DEBUG:teuthology.orchestra.run.smithi033:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T02:25:51.059 DEBUG:teuthology.orchestra.run.smithi138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T02:25:51.095 INFO:teuthology.orchestra.run.smithi033.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T02:25:51.097 INFO:teuthology.orchestra.run.smithi138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T02:25:51.332 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T02:25:51.332 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi033.front.sepia.ceph.com 2024-04-25T02:25:51.333 DEBUG:teuthology.orchestra.run.smithi033:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T02:25:51.389 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi138.front.sepia.ceph.com 2024-04-25T02:25:51.389 DEBUG:teuthology.orchestra.run.smithi138:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T02:25:51.423 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T02:25:51.423 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T02:25:51.440 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T02:25:51.489 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T02:25:51.489 DEBUG:teuthology.orchestra.run.smithi033:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T02:25:51.510 DEBUG:teuthology.orchestra.run.smithi138:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T02:25:52.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T02:25:53.003 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T02:25:53.003 DEBUG:teuthology.orchestra.run.smithi033:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T02:25:53.031 DEBUG:teuthology.orchestra.run.smithi138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T02:25:53.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T02:25:53.676 DEBUG:teuthology.orchestra.run.smithi033:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T02:25:53.682 DEBUG:teuthology.orchestra.run.smithi138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T02:25:53.707 INFO:teuthology.orchestra.run.smithi033.stdout:kernel.core_pattern = core 2024-04-25T02:25:53.707 INFO:teuthology.orchestra.run.smithi138.stdout:kernel.core_pattern = core 2024-04-25T02:25:53.749 DEBUG:teuthology.orchestra.run.smithi033:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T02:25:53.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T02:25:53.797 DEBUG:teuthology.orchestra.run.smithi138:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T02:25:53.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T02:25:53.814 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T02:25:54.015 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T02:25:54.130 DEBUG:teuthology.misc:Transferring archived files from smithi033:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322/remote/smithi033 2024-04-25T02:25:54.130 DEBUG:teuthology.orchestra.run.smithi033:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T02:25:54.272 DEBUG:teuthology.misc:Transferring archived files from smithi138:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322/remote/smithi138 2024-04-25T02:25:54.273 DEBUG:teuthology.orchestra.run.smithi138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T02:25:54.329 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T02:25:54.330 DEBUG:teuthology.orchestra.run.smithi033:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T02:25:54.333 DEBUG:teuthology.orchestra.run.smithi138:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T02:25:54.369 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T02:25:54.416 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T02:25:54.416 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T02:25:54.456 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T02:25:54.456 DEBUG:teuthology.orchestra.run.smithi033:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T02:25:54.460 DEBUG:teuthology.orchestra.run.smithi138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T02:25:54.475 INFO:teuthology.orchestra.run.smithi033.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 25 02:25 /home/ubuntu/cephtest 2024-04-25T02:25:54.476 INFO:teuthology.orchestra.run.smithi033.stdout: 554396 4 d--------- 2 ubuntu ubuntu 4096 Apr 25 01:33 /home/ubuntu/cephtest/mnt.0 2024-04-25T02:25:54.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T02:25:54.477 INFO:teuthology.orchestra.run.smithi033.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-25T02:25:54.477 INFO:teuthology.orchestra.run.smithi033.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-25T02:25:54.477 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi033 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-25T02:25:54.478 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T02:25:54.495 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T02:25:54.550 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-25T02:25:54.550 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T02:25:54.579 DEBUG:teuthology.nuke:shortname: smithi033 2024-04-25T02:25:54.579 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T02:25:54.602 DEBUG:teuthology.nuke:shortname: smithi138 2024-04-25T02:25:54.603 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T02:25:54.606 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi033.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 01:14:07.659914', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJC/7aXTZiaFnKvcK7sGicQkZMKeEgNtQzYQd5N1D/83fqflhgyAjTM+1tRbtAhwXp7u0OSXWmGqkQiYafjO3bQ='} 2024-04-25T02:25:54.632 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi138.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672322', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 01:14:07.658276', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB95wc00NAREHwua6/lxVbUx/4V0VFU3obIlX4w3qw1e5EHmad1vzipBPxT7hx4hOWFTf8dSnJoPWjLm02mB2kk='} 2024-04-25T02:25:54.663 INFO:teuthology.orchestra.console.smithi033:Power off 2024-04-25T02:25:54.664 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T02:25:54.689 DEBUG:teuthology.orchestra.console.smithi033:power off output: Chassis Power Control: Down/Off 2024-04-25T02:25:54.689 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T02:25:54.705 INFO:teuthology.orchestra.console.smithi138:Power off 2024-04-25T02:25:54.706 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T02:25:54.716 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-25T02:25:54.731 DEBUG:teuthology.orchestra.console.smithi138:power off output: Chassis Power Control: Down/Off 2024-04-25T02:25:54.731 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T02:25:54.755 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-25T02:25:58.717 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T02:25:58.756 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T02:25:58.829 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-25T02:25:58.867 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is on 2024-04-25T02:26:02.834 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T02:26:02.868 DEBUG:teuthology.orchestra.console.smithi138:pexpect command: ipmitool -H smithi138.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T02:26:02.945 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is on 2024-04-25T02:26:02.977 DEBUG:teuthology.orchestra.console.smithi138:check power output: Chassis Power is off 2024-04-25T02:26:03.077 INFO:teuthology.orchestra.console.smithi138:Power off completed 2024-04-25T02:26:06.950 DEBUG:teuthology.orchestra.console.smithi033:pexpect command: ipmitool -H smithi033.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T02:26:07.062 DEBUG:teuthology.orchestra.console.smithi033:check power output: Chassis Power is off 2024-04-25T02:26:07.163 INFO:teuthology.orchestra.console.smithi033:Power off completed 2024-04-25T02:26:07.268 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 3967.221866130829 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-04-25T02:26:07.268 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T02:26:07.342 INFO:teuthology.run:FAIL